用例:
解析失败https://www.banca-romaneasca.ro/en/tools-and-resources/使用lxml。
...
/opt/python-env/ciur/local/lib/python2.7/site-packages/html5lib/html5parser.py:468: in processComment
self.tree.insertComment(token, self.tree.openElements[-1])
/opt/python-env/ciur/local/lib/python2.7/site-packages/html5lib/treebuilders/etree_lxml.py:312: in insertCommentMain
super(TreeBuilder, self).insertComment(data, parent)
/opt/python-env/ciur/local/lib/python2.7/site-packages/html5lib/treebuilders/_base.py:262: in insertComment
parent.appendChild(self.commentClass(token["data"]))
/opt/python-env/ciur/local/lib/python2.7/site-packages/html5lib/treebuilders/etree.py:148: in __init__
self._element = ElementTree.Comment(data)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
- src/lxml/lxml.etree.pyx:3017: ValueError: Comment may not contain '--' or end with '-'
它来自lxml>https://github.com/lxml/lxml/blob/master/src/lxml/lxml.etree.pyx#L3017
它在中发现了差评https://www.banca-romaneasca.ro/en/tools-and-resources/
...
<script type="text/javascript" src="/_res/js/forms.js"></script>
<!-- Google Code for Remarketing Tag -->
<!--------------------------------------------------
Remarketing tags may not be associated with personally identifiable information or placed on pages related to sensitive categories. See more information and instructions on how to setup the tag on: http://google.com/ads/remarketingsetup
--------------------------------------------------->
<script type="text/javascript">
/* <![CDATA[ */
var google_conversion_id = 958631629;
var google_custom_params = window.google_tag_params;
...
要求解决方案,如:
禁用检查(xml上的一些魔术、标志)
if b'--' in text or text.endswith(b'-'): raise ValueError("Comment may not contain '--' or end with '-'")
猴子补丁(更改代码、注入…)
更新1:
我使用html5lib,并想得到标签,如声音,部分,视频。。。在html5中可用。
from lxml.html import html5parser, fromstring
context = fromstring(document.content) # work
context = html5parser.fromstring(document.content) # do not work
context = html5lib.parse( # do not work
document.content,
treebuilder="lxml",
namespaceHTMLElements=document.namespace,
encoding=document.encoding
)
版本:
- html5lib==0.9999999
- lxml==3.5.0(降级lxml也不是解决方案)
更新2::
这似乎是lxml中的改进/问题https://github.com/lxml/lxml/pull/172#issuecomment-169084439.
正在等待lxml开发人员的反馈。
更新3::
收到反馈,似乎是html5lib故障,github的最后一个开发版本已经修复了。
已经找到了基于github的@opottone的解决方案:
我尝试从github安装最新的html5parser
。现在我只得到一个警告,而不是一个错误。
由于这是要解析的HTML数据,请使用lxml.html
而不是lxml.etree
。
为我工作:
>>> import requests
>>> import lxml.html
>>>
>>> data = requests.get("https://www.banca-romaneasca.ro/en/tools-and-resources/").content
>>> tree = lxml.html.fromstring(data)
>>> tree.xpath("//title/text()")
['Tools and resources - Banca Romaneasca']