Python 请求 - 管理 cookie [英] Python Requests - managing cookies

问题描述

我正在尝试使用请求(和 bs4)从站点自动获取一些内容

I'm trying to get some content automatically from a site using requests (and bs4)

我有一个获取 cookie 的脚本:

I have a script that gets a cookie:

def getCookies(self):
    username = 'username'
    password = 'password'
    URL = 'logonURL'
    r = requests.get(URL, auth=('username', 'password'))
    cookies = r.cookies

cookie 的转储看起来像:

dump of the cookies looks like:

<<class 'requests.cookies.RequestsCookieJar'>[<Cookie ASP.NET_SessionId=yqokjr55ezarqbijyrwnov45 for URL.com/>, <Cookie BIGipServerPE_Journals.lww.com_80=1440336906.20480.0000 for URL.com/>, <Cookie JournalsLockCookie=id=a5720750-3f20-4207-a500-93ae4389213c&ip=IP address for URL.com/>]>

但是当我将 cookie 对象传递给下一个 URL 时:

But when I pass the cookie object to the next URL:

 soup = Soup(s.get(URL, cookies = cookies).content)

它不起作用 - 我可以通过倾倒汤看到我没有正确地向网络服务器提供我的凭据

its not working out - I can see by dumping the soup that I'm not giving the webserver my credentials properly

我尝试运行请求会话:

def getCookies(self):
    self.s = requests.session()
    username = 'username'
    password = 'password'
    URL = 'logURL'
    r = self.s.get(URL, auth=('username', 'password'))

我也一样没有快乐.

当我访问第二页时，我通过 FF 中的 liveHttp 查看了标题，并看到了一个非常不同的形式:

I looked at the header via liveHttp in FF when I visit the 2nd page, and see a very different form:

Cookie: WT_FPC=id=264b0aa85e0247eb4f11355304127862:lv=1355317068013:ss=1355314918680; UserInfo=Username=username; BIGipServerPE_Journals.lww.com_80=1423559690.20480.0000; PlatformAuthCookie=true; Institution=ReferrerUrl=http://logonURL.com/?wa=wsignin1.0&wtrealm=urn:adis&wctx=http://URL.com/_layouts/Authenticate.aspx?Source=%252fpecnews%252ftoc%252f2012%252f06440&token=method|ExpireAbsolute; counterSessionGuidId=6e2bd57f-b6da-4dd4-bcb0-742428e08b5e; MyListsRefresh=12/13/2012 12:59:04 AM; ASP.NET_SessionId=40a04p45zppozc45wbadah45; JournalsLockCookie=id=85d1f38f-dcbb-476a-bc2e-92f7ac1ae493&ip=10.204.217.84; FedAuth=77u/PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0idXRmLTgiPz48U2VjdXJpdHlDb250ZXh0VG9rZW4gcDE6SWQ9Il9mMGU5N2M3Zi1jNzQ5LTQ4ZjktYTUxNS1mODNlYjJiNGNlYzUtNEU1MDQzOEY0RTk5QURCNDFBQTA0Mjc0RDE5QzREMEEiIHhtbG5zOnAxPSJodHRwOi8vZG9jcy5vYXNpcy1vcGVuLm9yZy93c3MvMjAwNC8wMS9vYXNpcy0yMDA0MDEtd3NzLXdzc2VjdXJpdHktdXRpbGl0eS0xLjAueHNkIiB4bWxucz0iaHR0cDovL2RvY3Mub2FzaXMtb3Blbi5vcmcvd3Mtc3gvd3Mtc2VjdXJlY29udmVyc2F0aW9uLzIwMDUxMiI+PElkZW50aWZpZXI+dXJuOnV1aWQ6ZjJmNGY5MGItMmE4Yy00OTdlLTkwNzktY2EwYjM3MTBkN2I1PC9JZGVudGlmaWVyPjxJbnN0YW5jZT51cm46dXVpZDo2NzMxN2U5Ny1lMWQ3LTQ2YzUtOTg2OC05ZGJhYjA3NDkzOWY8L0luc3RhbmNlPjwvU2VjdXJpdHlDb250ZXh0VG9rZW4+

出于显而易见的原因，我已从问题中删除了用户名、密码和 URL.

I have redacted the username, password, and URLS from the question for obvious reasons.

我是否遗漏了一些明显的东西?是否有不同/正确的方法来捕获 cookie - 我正在使用的当前方法不起作用.

Am I missing something obvious? is there a different / proper way to capture the cookie - the current method I'm using is not working.

这是会话代码的独立版本:

This is a self standing version of the sessioned code:

s = requests.session()
username = 'username'
password = 'password'
URL = 'logonURL.aspx'
r = s.get(URL, auth=('username', 'password'))
URL = r"URL.aspx"
soup = Soup(s.get(URL).content)

读取汤的转储，我可以在 html 中看到它告诉我我没有访问权限 - 该字符串仅在您未登录时通过浏览器出现.

reading a dump of the soup, I can see in the html that its telling me I don't have access - this string only appears via browser when you're not logged in.

推荐答案

我遇到了类似的问题，并在此问题中找到了帮助.会话 jar 是空的，为了实际获取使用会话所需的 cookie.

I had a similar problem and found help in this question. The session jar was empty and to actually get the cookie I needed to use a session.

session = requests.session()
p = session.post("http://example.com", {'user':user,'password':password})
print 'headers', p.headers
print 'cookies', requests.utils.dict_from_cookiejar(session.cookies)
print 'html',  p.text

这篇关于Python 请求 - 管理 cookie的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！