如何从Python中的Web爬取构建数据框架 [英] How to construct data frame from Web Scraping in Python

问题描述

我可以通过Python中的网页抓取功能从网页中获取数据.我的数据被提取到列表中.但是不知道如何将列表转换为数据框.有什么办法可以通过Web抓取并直接将数据获取到df? 这是我的代码:

I can fetch data from web page thru web scraping in Python. My data is fetched into a list. But don't know how to transform that list into a data frame. Is there any way I could web scrape and fetch data directly to a df? Here is my code:

import pandas as pd
import requests
from bs4 import BeautifulSoup
from tabulate import tabulate
from pandas import DataFrame
import lxml

# GET the response from the web page using requests library
res = requests.get("https://www.worldometers.info/coronavirus/")

# PARSE and fetch content using BeutifulSoup method of bs4 library
soup = BeautifulSoup(res.content,'lxml')
table = soup.find_all('table')[0]
df = pd.read_html(str(table))

# Here dumping the fetched data to have a look
print( tabulate(df[0], headers='keys', tablefmt='psql') )
print(df[0])

推荐答案

import requests
import pandas as pd

r = requests.get("https://www.worldometers.info/coronavirus/")
df = pd.read_html(r.content)[0]

print(type(df))

# <class 'pandas.core.frame.DataFrame'>

df.to_csv("data.csv", index=False)

输出:视图

这篇关于如何从Python中的Web爬取构建数据框架的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！