网页抓取Python(BeautifulSoup，请求) [英] Web Scraping Python (BeautifulSoup,Requests)

问题描述

我正在学习使用python进行网页抓取，但无法获得所需的结果.下面是我的代码和输出

I am learning web scraping using python but I can't get the desired result. Below is my code and the output

代码

import bs4,requests
url = "https://twitter.com/24x7chess"
r = requests.get(url)
soup = bs4.BeautifulSoup(r.text,"html.parser")
soup.find_all("span",{"class":"account-group-inner"})
[]

这就是我要刮的东西

https://i.stack.imgur.com/tHo5S.png

我一直在获取一个空数组.请帮忙.

I keep on getting an empty array. Please Help.

推荐答案

尝试一下.它会为您提供您可能需要的物品.带有BeautifulSoup的Selenium易于处理.我是这样写的.在这里.

Try this. It will give you the items you probably look for. Selenium with BeautifulSoup is easy to handle. I've written it that way. Here it is.

from bs4 import BeautifulSoup 
from selenium import webdriver

driver = webdriver.Chrome()

driver.get("https://twitter.com/24x7chess")
soup = BeautifulSoup(driver.page_source,"lxml")
driver.quit()
for title in soup.select("#page-container"): 
    name = title.select(".ProfileHeaderCard-nameLink")[0].text.strip()
    location = title.select(".ProfileHeaderCard-locationText")[0].text.strip()
    tweets = title.select(".ProfileNav-value")[0].text.strip()
    following = title.select(".ProfileNav-value")[1].text.strip()
    followers = title.select(".ProfileNav-value")[2].text.strip()
    likes = title.select(".ProfileNav-value")[3].text.strip()
    print(name,location,tweets,following,followers,likes)

输出:

akul chhillar New Delhi, India 214 44 17 5

这篇关于网页抓取Python(BeautifulSoup，请求)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！