如何从 Python 中的 Web Scraping 构建数据框 [英] How to construct data frame from Web Scraping in Python

问题描述

我可以通过 Python 中的网页抓取从网页中获取数据.我的数据被提取到一个列表中.但不知道如何将该列表转换为数据框.有什么办法可以直接通过网络抓取和获取数据到 df 吗?这是我的代码:

将pandas导入为pd进口请求从 bs4 导入 BeautifulSoup从表格导入表格从熊猫导入数据帧导入 lxml# 使用请求库从网页获取响应res = requests.get("https://www.worldometers.info/coronavirus/")# 使用 bs4 库的 BeutifulSoup 方法解析和获取内容汤 = BeautifulSoup(res.content,'lxml')table = 汤.find_all('table')[0]df = pd.read_html(str(table))# 这里把获取的数据dump出来看看打印(制表(df[0]，headers='keys'，tablefmt='psql'))打印(df [0])

解决方案

导入请求将熊猫导入为 pdr = requests.get("https://www.worldometers.info/coronavirus/")df = pd.read_html(r.content)[0]打印(类型(df))# df.to_csv("data.csv", index=False)

输出:

I can fetch data from web page thru web scraping in Python. My data is fetched into a list. But don't know how to transform that list into a data frame. Is there any way I could web scrape and fetch data directly to a df? Here is my code:

import pandas as pd
import requests
from bs4 import BeautifulSoup
from tabulate import tabulate
from pandas import DataFrame
import lxml

# GET the response from the web page using requests library
res = requests.get("https://www.worldometers.info/coronavirus/")

# PARSE and fetch content using BeutifulSoup method of bs4 library
soup = BeautifulSoup(res.content,'lxml')
table = soup.find_all('table')[0]
df = pd.read_html(str(table))

# Here dumping the fetched data to have a look
print( tabulate(df[0], headers='keys', tablefmt='psql') )
print(df[0])

解决方案

import requests
import pandas as pd

r = requests.get("https://www.worldometers.info/coronavirus/")
df = pd.read_html(r.content)[0]

print(type(df))

# <class 'pandas.core.frame.DataFrame'>

df.to_csv("data.csv", index=False)

Output: view

这篇关于如何从 Python 中的 Web Scraping 构建数据框的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！