Table Web Scraping问题与Python [英] Table Web Scraping Issues with Python

问题描述

我在从此网站抓取数据时遇到问题: https://fantasy.premierleague.com/player-列表

I am having issues scraping data from this website: https://fantasy.premierleague.com/player-list

我有兴趣从不同的表格访问玩家的姓名和积分.

I am interested in getting access to the player's names and points from the different tables.

我是python的新手，也是Web抓取的新手.这是我到目前为止的内容:

I'm relatively new to python and completely new to web scraping. Here is what I have so far:

from urllib.request import urlopen
from bs4 import BeautifulSoup


url = 'https://fantasy.premierleague.com/player-list'


html = urlopen(url)
soup = BeautifulSoup(html, "lxml")

rows = soup.find_all('tr')
print(rows)

从这里我将继续查找所有"td"信息.

From here I would go on to find all the 'td' information.

但是我没有得到'tr'的结果.我可以将"a"作为参数传递给站点，但可以从表中获取任何数据.我的理解是传递"tr"将找到网站内任何表的所有行

However I get no results for 'tr'. I can pass 'a' in as an argument and get the links for the site fine but haven't been able to get any data from the tables. My understanding is passing 'tr' will find all rows of any tables within the website

有什么想法我要去哪里吗?谢谢您的帮助

Any ideas where I am going wrong? Thanks for your help

推荐答案

您可以用来获取所有表数据 webdriver ， pandas 和 BeautifulSoup .

You can use to get all the table data webdriver, pandas and BeautifulSoup.

from bs4 import BeautifulSoup
import requests
from selenium import webdriver
import pandas as pd
url = "https://fantasy.premierleague.com/player-list"

driver = webdriver.Firefox()
driver.get(url)

html = driver.page_source
soup = BeautifulSoup(html)
table = soup.find_all('table', {'class': 'Table-ziussd-1 fVnGhl'})

df = pd.read_html(str(table))

print(df)

输出将是:

[             Player            Team  Points  Cost
0           Alisson       Liverpool      99  £6.2
1           Ederson        Man City      89  £6.0
2              Kepa         Chelsea      72  £5.4
3        Schmeichel       Leicester     122  £5.4
4            de Gea         Man Utd     105  £5.3
5            Lloris           Spurs      56  £5.3
6         Henderson   Sheffield Utd     135  £5.3
7          Pickford         Everton      93  £5.2
8          Patrício          Wolves     122  £5.2
9          Dubravka       Newcastle     124  £5.1
10             Leno         Arsenal     114  £5.0
11           Guaita  Crystal Palace     122  £5.0
12             Pope         Burnley     129  £4.9
13           Foster         Watford     113  £4.9
14        Fabianski        West Ham      61  £4.9
15        Caballero         Chelsea       7  £4.8
16             Ryan        Brighton     105  £4.7
17            Bravo        Man City      11  £4.7
18            Grant         Man Utd       0  £4.7
19           Romero         Man Utd       0  £4.6
20             Krul         Norwich      94  £4.6
21         Mignolet       Liverpool       0  £4.5
22         McCarthy     Southampton      74  £4.5
23         Ramsdale     Bournemouth      97  £4.5
24         Fahrmann         Norwich       1  £4.4




and so on........................................]

这篇关于Table Web Scraping问题与Python的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！