BeautifulSoup.find_all()方法不适用于命名空间标签 [英] BeautifulSoup.find_all() method not working with namespaced tags
问题描述
今天与BeautifulSoup合作时，我遇到了一个非常奇怪的行为.
I have encountered a very strange behaviour while working with BeautifulSoup today.
让我们看一个非常简单的html代码段:
Let's have a look at a very simple html snippet:
<html><body><ix:nonfraction>lele</ix:nonfraction></body></html>
我正在尝试通过BeautifulSoup获取< ix:nonfraction> 标记的内容.
I am trying to get the content of the <ix:nonfraction> tag with BeautifulSoup.
使用 find 方法时一切正常:
from bs4 import BeautifulSoup
html = "<html><body><ix:nonfraction>lele</ix:nonfraction></body></html>"
soup = BeautifulSoup(html, 'lxml') # The parser used here does not matter
soup.find('ix:nonfraction')
>>> <ix:nonfraction>lele</ix:nonfraction>
但是，当尝试使用 find_all 方法时，我希望有一个返回此单个元素的列表，情况并非如此！
However, when trying to use the find_all method, I expect to have a list with this single element returned, which is not the case !
soup.find_all('ix:nonfraction')
>>> []
实际上，每当我要搜索的标签中出现冒号时， find_all 似乎都会返回一个空列表.
In fact, find_all seems to return an empty list everytime a colon is present in the tag I am searching for.
我已经能够在两台不同的计算机上重现该问题.
I have been able to reproduce the problem on two different computers.
有人有解释吗，更重要的是，有一种解决方法?我只需要使用 find_all 方法就可以了，因为我的实际情况要求我将所有这些标签都放在整个html页面上.
Does anyone have an explanation, and more importantly, a workaround ?
I need to use the find_all method simply because my actual case requires me to get all these tags on a whole html page.
推荐答案
@yosemite_k解决方案起作用的原因是因为在bs4的源代码中，它跳过了导致这种现象的特定条件.实际上，您可以做很多变化来产生相同的结果.例子:
The reason @yosemite_k's solution works is because in the source code of bs4, it's skipping a certain condition which causes this behavior. You can in fact do many variations which will produce this same result. Examples:
soup.find_all({"ix:nonfraction"})
soup.find_all('ix:nonfraction', limit=1)
soup.find_all('ix:nonfraction', text=True)
下面是 beautifulsoup 的源代码中的一段代码.调用 find 或 find_all 时发生.您将看到 find 仅使用 limit = 1 调用 find_all .在 _find_all 中，它检查条件:
Below is a snippet from the source code of beautifulsoup that shows what happens when you call find or find_all. You will see that find just calls find_all with limit=1. In _find_all it checks for a condition:
if text is None and not limit and not attrs and not kwargs:
如果达到该条件，则最终可能会使其降至此条件:
If it hits that condition, then the it might eventually make it down to this condition:
# Optimization to find all tags with a given name.
if name.count(':') == 1:
如果在此放置，则会重新分配 name :
If it makes it there, then it does a reassignment of name:
# This is a name with a prefix.
prefix, name = name.split(':', 1)
这是您行为不同的地方.只要 find_all 不满足任何先前的条件，就可以找到该元素.
This is where you're behavior is different. As long as find_all doesn't meet any of the prior conditions, then you'll find the element.
beautifulsoup4 == 4.6.0
beautifulsoup4==4.6.0
def find(self, name=None, attrs={}, recursive=True, text=None,
**kwargs):
"""Return only the first child of this Tag matching the given
criteria."""
r = None
l = self.find_all(name, attrs, recursive, text, 1, **kwargs)
if l:
r = l[0]
return r
findChild = find
def find_all(self, name=None, attrs={}, recursive=True, text=None,
limit=None, **kwargs):
"""Extracts a list of Tag objects that match the given
criteria. You can specify the name of the Tag and any
attributes you want the Tag to have.
The value of a key-value pair in the 'attrs' map can be a
string, a list of strings, a regular expression object, or a
callable that takes a string and returns whether or not the
string matches for some custom definition of 'matches'. The
same is true of the tag name."""
generator = self.descendants
if not recursive:
generator = self.children
return self._find_all(name, attrs, text, limit, generator, **kwargs)
def _find_all(self, name, attrs, text, limit, generator, **kwargs):
"Iterates over a generator looking for things that match."
if text is None and 'string' in kwargs:
text = kwargs['string']
del kwargs['string']
if isinstance(name, SoupStrainer):
strainer = name
else:
strainer = SoupStrainer(name, attrs, text, **kwargs)
if text is None and not limit and not attrs and not kwargs:
if name is True or name is None:
# Optimization to find all tags.
result = (element for element in generator
if isinstance(element, Tag))
return ResultSet(strainer, result)
elif isinstance(name, str):
# Optimization to find all tags with a given name.
if name.count(':') == 1:
# This is a name with a prefix.
prefix, name = name.split(':', 1)
else:
prefix = None
result = (element for element in generator
if isinstance(element, Tag)
and element.name == name
and (prefix is None or element.prefix == prefix)
)
return ResultSet(strainer, result)
results = ResultSet(strainer)
while True:
try:
i = next(generator)
except StopIteration:
break
if i:
found = strainer.search(i)
if found:
results.append(found)
if limit and len(results) >= limit:
break
return results
这篇关于BeautifulSoup.find_all()方法不适用于命名空间标签的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
