Python3-请求-BS4-Cloudflare->403禁止不使用本地代理 [英] Python3 - Requests - BS4 - Cloudflare -> 403 Forbidden not use Local Proxy

问题描述

代码不起作用.由于系统使用cloudflare，因此出现403错误当我使用任何人的http代理(burp suite/fiddler等)时，我看到csrfToken.有用.为什么在使用本地代理时可以使用?

Codes aren't working. It has got 403 error because system using cloudflare When i am using anyone http proxy(burp suite/fiddler etc.), I see csrfToken. It works. Why it works when use local proxy?

import requests
from bs4 import BeautifulSoup

headerIstek = {
    "User-Agent" : "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36 Edge/18.19041",
    "Sec-Fetch-Site" : "none",
    "Accept-Language" : "tr-TR,tr;q=0.9,en-US;q=0.8,en;q=0.7"
    }
istekLazim = {"ref":"","display_type":"popup","loc":""}
istekLogin = requests.get("https://www.example.com/join/login-popup/", headers=headerIstek, cookies={"ud_rule_vars":""}, params=istekLazim, verify=False)
soup = BeautifulSoup(istekLogin.text, "html.parser")
print(istekLogin.request.headers)
csrfToken = soup.find("input", {"name":"csrfmiddlewaretoken"})["value"]
print(csrfToken)

推荐答案

Cloudflare在浏览器上执行JavaScript检查，如果检查成功，则返回会话.如果您想运行一次性脚本来从受CloudFlare保护的服务器上下载内容，请从您先前使用浏览器获得的经过验证的会话中添加会话cookie.

Cloudflare performs JavaScript checks on the browser and returns a session if the checks have been successful. If you want to run a one-off script to download stuff off of a CloudFlare protected server, add a session cookie from a previously validated session you obtained using your browser.

会话cookie的名称为 __ cfduid .您可以通过使用浏览器获取资源，然后打开开发人员工具和网络面板来获取它.检查请求后，您可以看到浏览器发送到服务器的cookie.

The session cookie is named __cfduid. You can get it by fetching a resource using your browser and then opening the developer tools and the network panel. Once you inspect the request, you can see the cookies your browser sent to the server.

然后，您可以使用脚本将该Cookie用于请求:

Then you can use that cookie for requests using your script:

cookies = {
    "__cfduid": "xd0c0985ed80ffbc4dd29d1612168766",
}
response = requests.get(image_url, cookies=cookies)
response.raise_for_status()  

这篇关于Python3-请求-BS4-Cloudflare->403禁止不使用本地代理的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！