Python 中的 HTML 文件解析 [英] HTML file parsing in Python

问题描述

我有一个很长的 html 文件，看起来完全像这样 - html 文件.我希望能够解析该文件，以便在 tuple 的表单中获取信息.

I have a very long html file that looks exactly like this - html file . I want to be able to parse the file such that I get the information in the form on a tuple .

示例:

<tr>
      <td>Cech</td>
      <td>Chelsea</td>
      <td>30</td>
      <td>£6.4</td>
</tr>

上面的信息看起来像("Cech", "Chelsea", 30, 6.4).但是，如果您仔细查看我发布的 link，我发布的 html 示例属于 <h2>守门员 标签.我也需要这个标签.所以基本上结果元组看起来像 ("Cech", "Chelsea", 30, 6.4, Goalkeepers) .在文件的更下方，一堆球员属于中场、后卫和前锋的 <h2> 标签.

The above information will look like ("Cech", "Chelsea", 30, 6.4). However if you look closely at the link i posted, the html example i posted comes under a <h2>Goalkeepers</h2> tag. i need this tag too. So basically the result tuple will look like ("Cech", "Chelsea", 30, 6.4, Goalkeepers) . Further down the file a bunch of players come under <h2> tags of Midfielders , Defenders and Forwards.

我尝试使用 beautifulsoup 和 ntlk 库但迷路了.所以现在我有以下代码:

I tried using beautifulsoup and ntlk libraries and got lost. So now I have the following code:

import nltk
from urllib import urlopen

url = "http://fantasy.premierleague.com/player-list/"
html = urlopen(url).read()
raw = nltk.clean_html(html)
print raw

它只是去除所有标签的 html 文件并给出如下内容:

which just strips of the html file of all the tags and gives something like this:

          Cech
          Chelsea
          30
          £6.4

虽然我可以写一段糟糕的代码来读取每一行并将其分配给一个元组.我无法想出任何还可以包含玩家位置的解决方案(出现在 <h2> 标签中的字符串).任何解决方案/建议将不胜感激.

Although I can write a bad piece of code that reads every line and can assign it to a tuple. i cannot come up with any solution which can also incorporate the player position ( the string present in the <h2> tags). Any solution / suggestions will be greatly appreciated.

我倾向于使用元组的原因是我可以使用解包并计划用解包的值填充 MySQl 表.

The reason I am inclined towards using tuples i so that I can use unpacking and plan on populating a MySQl table with the unpacked values.

推荐答案

from bs4 import BeautifulSoup
from pprint import pprint

soup = BeautifulSoup(html)
h2s = soup.select("h2") #get all h2 elements
tables = soup.select("table") #get all tables

first = True
title =""
players = []
for i,table in enumerate(tables):
    if first:
         #every h2 element has 2 tables. table size = 8, h2 size = 4
         #so for every 2 tables 1 h2
         title =  h2s[int(i/2)].text
    for tr in table.select("tr"):
        player = (title,) #create a player
        for td in tr.select("td"):
            player = player + (td.text,) #add td info in the player
        if len(player) > 1: 
            #If the tr contains a player and its not only ("Goalkeaper") add it
            players.append(player)
    first = not first
pprint(players)

输出:

[('Goalkeepers', 'Cech', 'Chelsea', '30', '£6.4'),
 ('Goalkeepers', 'Hart', 'Man City', '28', '£6.4'),
 ('Goalkeepers', 'Krul', 'Newcastle', '21', '£5.0'),
 ('Goalkeepers', 'Ruddy', 'Norwich', '25', '£5.0'),
 ('Goalkeepers', 'Vorm', 'Swansea', '19', '£5.0'),
 ('Goalkeepers', 'Stekelenburg', 'Fulham', '6', '£4.9'),
 ('Goalkeepers', 'Pantilimon', 'Man City', '0', '£4.9'),
 ('Goalkeepers', 'Lindegaard', 'Man Utd', '0', '£4.9'),
 ('Goalkeepers', 'Butland', 'Stoke City', '0', '£4.9'),
 ('Goalkeepers', 'Foster', 'West Brom', '13', '£4.9'),
 ('Goalkeepers', 'Viviano', 'Arsenal', '0', '£4.8'),
 ('Goalkeepers', 'Schwarzer', 'Chelsea', '0', '£4.7'),
 ('Goalkeepers', 'Boruc', 'Southampton', '42', '£4.7'),
 ('Goalkeepers', 'Myhill', 'West Brom', '15', '£4.5'),
 ('Goalkeepers', 'Fabianski', 'Arsenal', '0', '£4.4'),
 ('Goalkeepers', 'Gomes', 'Tottenham', '0', '£4.4'),
 ('Goalkeepers', 'Friedel', 'Tottenham', '0', '£4.4'),
 ('Goalkeepers', 'Henderson', 'West Ham', '0', '£4.0'),
 ('Defenders', 'Baines', 'Everton', '43', '£7.7'),
 ('Defenders', 'Vertonghen', 'Tottenham', '34', '£7.0'),
 ('Defenders', 'Taylor', 'Cardiff City', '14', '£4.5'),
 ('Defenders', 'Zverotic', 'Fulham', '0', '£4.5'),
 ('Defenders', 'Davies', 'Hull City', '28', '£4.5'),
 ('Defenders', 'Flanagan', 'Liverpool', '0', '£4.5'),
 ('Defenders', 'Dawson', 'West Brom', '0', '£3.9'),
 ('Defenders', 'Potts', 'West Ham', '0', '£3.9'),
 ('Defenders', 'Spence', 'West Ham', '0', '£3.9'),
 ('Midfielders', 'Özil', 'Arsenal', '24', '£10.6'),
 ('Midfielders', 'Redmond', 'Norwich', '20', '£5.0'),
 ('Midfielders', 'Mavrias', 'Sunderland', '5', '£5.0'),
 ('Midfielders', 'Gera', 'West Brom', '0', '£5.0'),
 ('Midfielders', 'Essien', 'Chelsea', '0', '£4.9'),
 ('Midfielders', 'Brown', 'West Brom', '0', '£4.3'),
 ('Forwards', 'van Persie', 'Man Utd', '24', '£13.9'),
 ('Forwards', 'Cornelius', 'Cardiff City', '1', '£5.4'),
 ('Forwards', 'Elmander', 'Norwich', '7', '£5.4'),
 ('Forwards', 'Murray', 'Crystal Palace', '0', '£5.3'),
 ('Forwards', 'Vydra', 'West Brom', '2', '£5.3'),
 ('Forwards', 'Proschwitz', 'Hull City', '0', '£4.3')]

这篇关于Python 中的 HTML 文件解析的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！