在操纵up中缺少请求标头 [英] Missing request headers in puppeteer

问题描述

我想在用操纵up编写的测试中读取请求cookie.但是我注意到，我检查的大多数请求都只有引荐来源网址和用户代理标头.如果我在Chrome开发人员工具中查看相同的请求，它们会有更多的标头，包括Cookie.要进行检查，请将下面的代码复制粘贴到 https://try-puppeteer.appspot.com/.

I want to read the request cookie during a test written with the puppeteer. But I noticed that most of the requests I inspect have only referrer and user-agent headers. If I look at the same requests in Chrome dev tools, they have a lot more headers, including Cookie. To check it out, copy-paste the code below into https://try-puppeteer.appspot.com/.

const browser = await puppeteer.launch();
const page = await browser.newPage();

page.on('request', function(request) {
  console.log(JSON.stringify(request.headers, null, 2));
});

await page.goto('https://google.com/', {waitUntil: 'networkidle'});

await browser.close();

是否可以访问和不能访问哪些请求标头?这是Chrome本身或伪造者的限制吗?

Is there a restriction which request headers you can and can not access? Is it a limitation of Chrome itself or puppeteer?

感谢您的建议！

推荐答案

当我尝试使用Puppeteer测试某些CORS行为时，我也看到了这一点-我发现某些请求中缺少Origin标头.

I also saw this when I was trying to use Puppeteer to test some CORS behaviour - I found the Origin header was missing from some requests.

看看GitHub问题，我发现了一个问题提到的Puppeteer不监听基础Chrome DevTools协议的 Network.responseReceivedExtraInfo 事件，此事件提供了 Network.responseReceived 事件不可用的额外响应标头.对于请求，也有一个类似的 Network.requestWillBeSentExtraInfo 事件.

Having a look around the GitHub issues I found an issue which mentioned Puppeteer does not listen to the Network.responseReceivedExtraInfo event of the underlying Chrome DevTools Protocol, this event provides extra response headers not available to the Network.responseReceived event. There is also a similar Network.requestWillBeSentExtraInfo event for requests.

参加这些活动似乎使我获得了所需的所有标题.以下是一些示例代码，该示例代码捕获所有这些事件中的数据并将其合并到以请求ID为键的单个对象中:

Hooking up to these events seemed to get me all the headers I needed. Here is some sample code which captures the data from all these events and merges it onto a single object keyed by request ID:

// Setup.
const browser = await puppeteer.launch()
const page = await browser.newPage()
const cdpRequestDataRaw = await setupLoggingOfAllNetworkData(page)

// Make requests.
await page.goto('http://google.com/')

// Log captured request data.
console.log(JSON.stringify(cdpRequestDataRaw, null, 2))

await browser.close()

// Returns map of request ID to raw CDP request data. This will be populated as requests are made.
async function setupLoggingOfAllNetworkData(page) {
    const cdpSession = await page.target().createCDPSession()
    await cdpSession.send('Network.enable')
    const cdpRequestDataRaw = {}
    const addCDPRequestDataListener = (eventName) => {
        cdpSession.on(eventName, request => {
            cdpRequestDataRaw[request.requestId] = cdpRequestDataRaw[request.requestId] || {}
            Object.assign(cdpRequestDataRaw[request.requestId], { [eventName]: request })
        })
    }
    addCDPRequestDataListener('Network.requestWillBeSent')
    addCDPRequestDataListener('Network.requestWillBeSentExtraInfo')
    addCDPRequestDataListener('Network.responseReceived')
    addCDPRequestDataListener('Network.responseReceivedExtraInfo')
    return cdpRequestDataRaw
}

这篇关于在操纵up中缺少请求标头的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！