Selenium Python-获取网络响应正文 [英] Selenium Python - Get Network response body
问题描述
我使用Selenium对来自网站的GET请求后的数据接收做出反应.网站调用的API是不公开的,因此,如果我使用请求的URL检索数据,则会得到 {"message":未认证".}
.
I use Selenium to react to the reception of data following a GET request from a website.
The API called by the website is not public, so if I use the URL of the request to retrieve the data, I get {"message":"Unauthenticated."}
.
到目前为止,我所能做的就是检索响应的标题.
All I've managed to do so far is to retrieve the header of the response.
我在此处找到了使用 driver.execute_cdp_cmd('Network.getResponseBody',{...})
可能是解决我的问题的方法.
I found here that using driver.execute_cdp_cmd('Network.getResponseBody', {...})
might be a solution to my problem.
以下是我的代码示例:
import json
from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
capabilities = DesiredCapabilities.CHROME
capabilities["goog:loggingPrefs"] = {"performance": "ALL"}
driver = webdriver.Chrome(
r"./chromedriver",
desired_capabilities=capabilities,
)
def processLog(log):
log = json.loads(log["message"])["message"]
if ("Network.response" in log["method"] and "params" in log.keys()):
headers = log["params"]["response"]
body = driver.execute_cdp_cmd('Network.getResponseBody', {'requestId': log["params"]["requestId"]})
print(json.dumps(body, indent=4, sort_keys=True))
return log["params"]
logs = driver.get_log('performance')
responses = [processLog(log) for log in logs]
不幸的是, driver.execute_cdp_cmd('Network.getResponseBody',{...})
返回:
unknown error: unhandled inspector error: {"code":-32000,"message":"No resource with given identifier found"}
您知道我缺少什么吗?
您对如何获取响应正文有任何想法吗?
谢谢您的帮助!
推荐答案
为了检索响应正文,您必须专门收听 Network.responseReceived
:
In order to retrieve response body, you have to listen specifically to Network.responseReceived
:
def processLog(log):
log = json.loads(log["message"])["message"]
if ("Network.responseReceived" in log["method"] and "params" in log.keys()):
body = driver.execute_cdp_cmd('Network.getResponseBody', {'requestId': log["params"]["requestId"]})
但是,我结束了使用另一种依赖于 requests
的方法.我只是从浏览器控制台(网络>标头>请求标头>授权)中检索了授权令牌,并用它来获取我想要的数据:
However, I ended using a different approach relying on requests
. I just retrieved the authorization token from the browser console (Network > Headers > Request Headers > Authorization) and used it to get the data I wanted:
import requests
def get_data():
url = "<your_url>"
headers = {
"Authorization": "Bearer <your_access_token>",
"Content-type": "application/json"
}
params = {
key: value,
...
}
r = requests.get(url, headers = headers, params = params)
if r.status_code == 200:
return r.json()
这篇关于Selenium Python-获取网络响应正文的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!