在Python中使用BeautifulSoup解析数据 [英] Parsing out data using BeautifulSoup in Python
问题描述
我正在尝试使用BeautifulSoup来解析DOM树并提取作者的姓名.以下是HTML的片段，以显示我要抓取的代码的结构.
I am attempting to use BeautifulSoup to parse through a DOM tree and extract the names of authors. Below is a snippet of HTML to show the structure of the code I'm going to scrape.
<html>
<body>
<div class="list-authors">
<span class="descriptor">Authors:</span>
<a href="/find/astro-ph/1/au:+Lin_D/0/1/0/all/0/1">Dacheng Lin</a>,
<a href="/find/astro-ph/1/au:+Remillard_R/0/1/0/all/0/1">Ronald A. Remillard</a>,
<a href="/find/astro-ph/1/au:+Homan_J/0/1/0/all/0/1">Jeroen Homan</a>
</div>
<div class="list-authors">
<span class="descriptor">Authors:</span>
<a href="/find/astro-ph/1/au:+Kosovichev_A/0/1/0/all/0/1">A.G. Kosovichev</a>
</div>
<!--There are many other div tags with this structure-->
</body>
</html>
我的困惑点是，当我做soup.find时，它会找到我要搜索的div标签的第一个匹配项.之后，我搜索所有"a"链接标签.在此阶段，如何从每个链接标记中提取作者姓名并打印出来?有没有办法使用BeautifulSoup做到这一点，或者我需要使用Regex吗?如何继续遍历其他所有div标签并提取作者姓名?
My point of confusion is that when I do soup.find, it finds the first occurrence of the div tag that I'm searching for. After that, I search for all 'a' link tags. At this stage, how do I extract the authors names from each of the link tags and print them out? Is there a way to do it using BeautifulSoup or do I need to use Regex? How do I continue iterating over every other other div tag and extract the authors names?
import re
import urllib2,sys
from BeautifulSoup import BeautifulSoup, NavigableString
html = urllib2.urlopen(address).read()
soup = BeautifulSoup(html)
try:
authordiv = soup.find('div', attrs={'class': 'list-authors'})
links=tds.findAll('a')
for link in links:
print ''.join(link[0].contents)
#Iterate through entire page and print authors
except IOError:
print 'IO error'
推荐答案
只需将findAll用于您对链接所做的divs链接
just use findAll for the divs link you do for the links
对于汤中的authordiv.findAll('div'，attrs = {'class':'list-authors'}):
for authordiv in soup.findAll('div', attrs={'class': 'list-authors'}):
这篇关于在Python中使用BeautifulSoup解析数据的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
