Twitter 不再使用请求库 python [英] Twitter no longer works with requests library python

问题描述

我有一个 python 函数，它使用请求库和 BeautifulSoup 来抓取特定用户的推文.

I have a python function that uses the requests library and BeautifulSoup to scrape a particular user's tweets.

import requests
from bs4 import BeautifulSoup

contents = requests.get("https://twitter.com/user")
soup = BeautifulSoup(contents.text, "html.parser")

requests 库访问 Twitter 时，使用的是旧版 Twitter.但是，由于 Twitter 最近放弃了对其旧版本的支持，请求库不再有效并返回 html 代码，指出此版本的 Twitter 已过时.

When the requests library accesses Twitter, it uses the legacy version of Twitter. However, since Twitter recently dropped support for its legacy version, the requests library no longer works and returns html code saying that this version of Twitter is out of date.

有没有办法让请求库访问较新版本的 Twitter?

Is there a way to make the requests library access the newer version of Twitter?

推荐答案

我也遇到了这个问题.造成这种情况的根本原因是 Twitter 拒绝传统".浏览器，不幸的是它包含 Python 的请求库.

I also encountered this problem. The root cause of this is Twitter rejecting "legacy" browsers, which unfortunately includes Python's requests library.

Twitter 通过查看作为请求的一部分发送的 User-Agent 标头来确定您使用的浏览器.所以我对这个问题的解决办法就是简单地欺骗这个标题.

Twitter figures out what browser you are using by looking at the User-Agent header sent as part of the request. So my solution to the problem was simply to spoof this header.

在您的特定情况下，请尝试执行以下操作；

In your particular case, try doing something like;

import requests
from bs4 import BeautifulSoup

contents = requests.get(
    "https://twitter.com/user",
    headers={"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.150 Safari/537.36"}
)
soup = BeautifulSoup(contents.text, "html.parser")

这篇关于Twitter 不再使用请求库 python的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！