如何 Python 请求像我的浏览器一样跟随 URL [英] How to Python requests to follow URL like my browser

问题描述

我注意到 Python 的请求库不像我的浏览器那样遵循某些 URL 重定向.

I've noticed that the requests library for Python does not follow certain URL redirects like my browser.

例如，当我这样做时:

response = requests.get('http://www.bbmt.org/', verify=False, allow_redirects=True)

最终 URL 是 HTTPS://secure.jbs.elsevierhealth.com/action/consumeSsoCookie?redirectUri=http%3A%2F%2Fwww.bbmt.org%2Faction%2FconsumeSharedSessionAction%3FMAID%3DJ3%252BqsjOKzWZhWAeF2bXl%252FA%253D%253D%26JSESSIONID%3DaaaorUqRLHgAe4WCenKv%26SERVER%3DWZ6myaEXBLEt1UgI9cIkvA%253D%253D%26ORIGIN%3D470200154%26RD%3DRD&acw=&utt=

然而，在我的浏览器中，我最终被重定向回http://www.bbmt.org/.

However, in my browser, I am eventually redirected back to http://www.bbmt.org/.

在这些情况下，有没有办法让请求表现得像我的浏览器?

Is there a way to have requests behave like my browser in these scenarios?

推荐答案

https://secure.jbs.elsevierhealth.com 中的重定向是 javascript 重定向.
您可以在源代码中看到它:

The redirect inside the https://secure.jbs.elsevierhealth.com is a javascript redirect.
You can see it inside the source-code:

window.location.href = "http://www.bbmt.org/action/consumeSharedSessionAction?SERVER=WZ6myaEXBLHj3ZzqSv9HPw%3D%3D&MAID=IBS8Eq6B1iRWhf2ywTW5pg%3D%3D&JSESSIONID=aaa8eY-zM394XcPptT_Kv&ORIGIN=670572791&RD=RD";

您需要运行 javascript 才能执行此重定向(这是请求不执行的操作).

You will need to run javascript in order to do this redirect (and this is something requests do not do).

如果您需要针对此特定重定向的解决方案，您可以解析响应的内容(在 Python 中)并获取该特定 URL 并使用它来创建新请求.

If you need a solution for this specific redirect, you can parse the content of the response (in python) and take that specific URL and use it to create a new request.

如果您需要通用解决方案 - 您将需要使用无头浏览器来执行此操作.您可以在此问题中找到更多信息.

If you need a general solution - you will need to use a headless browser to do that. You can find more information in this question.

这篇关于如何 Python 请求像我的浏览器一样跟随 URL的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！