如何使用 BeautifulSoup 成对提取表格数据? [英] How do I extract table data in pairs using BeautifulSoup?
问题描述
我的数据样本:
<table id = "history">
<tr class = "printCol">
<td class="name">Google</td><td class="date">07/11/2001</td><td class="state">
<span>CA</span>
</td>
</tr>
<tr class = "printCol">
<td class="name">Apple</td><td class="date">27/08/2001</td>
</tr>
<tr class = "printCol">
<td class="name">Microsoft</td><td class="date">01/11/1991</td>
</tr>
</table>
Beautifulsoup 代码:
Beautifulsoup code :
table = soup.find("table", id = "history")
rows = table.findAll('tr')
for tr in rows:
cols = tr.findAll('td')
for td in cols:
print td.find(text=True)
MySQL 存储所需的输出(列表):
Desired Output for MySQL storage (list):
['Google|07/11/2001|CA', 'Apple|27/08/2001', 'Microsoft|01/11/1991']
我的输出(很难将正确的日期与正确的公司联系起来):
Output I have (difficult to associate the right date to the right company) :
Google
07/11/2001
Apple
27/08/2001
Microsoft
01/11/1991
我编写了一个函数来从每个 tr 中提取元素,但我认为在原始 for 循环中有更有效的方法来完成所有操作.我想将它们作为数据对存储在列表中.想法?
I wrote a function to extract elements from each tr but I thought there is a much more efficient way of doing it all in the original for loop. I want to store them in a list as data pairs. Thoughts?
推荐答案
列表理解会更容易:
table = soup.find("table", id = "history")
rows = table.findAll('tr')
data = [[td.findChildren(text=True) for td in tr.findAll("td")] for tr in rows]
# data now contains:
[[u'Google', u'07/11/2001'],
[u'Apple', u'27/08/2001'],
[u'Microsoft', u'01/11/1991']]
# If the data may contain extraneous whitespace you can clean it up
# Additional processing could also be done - but once you hit much more
# complex than this later maintainers, yourself included, will thank you
# for using a series of for loops that call clearly named functions to perform
# the work.
data = [[u"".join(d).strip() for d in l] for l in data]
# If you want to store it joined as name | company
# then simply follow that up with:
data = [u"|".join(d) for d in data]
列表推导式基本上是一个带有聚合的反向for
循环:
The list comprehension is basically a reverse for
loop with aggregation:
[[td.findNext(text=True) for td in tr.findAll("td")] for tr in rows]
翻译成*:
final_list = []
intermediate_list = []
for tr in rows:
for td in tr.findAll("td")
intermediate_list.append(td.findNext(text=True))
final_list.append(intermediate_list)
intermediate_list = []
data = final_list
* 粗略地说 - 我们忽略了涉及生成器的很棒不构建中间列表,因为我现在无法添加生成器把例子弄乱了.
* Roughly - we are leaving out the awesomeness involving generators not building intermediate lists, since I can't add generators right now without cluttering the example.
这篇关于如何使用 BeautifulSoup 成对提取表格数据?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!