UrlFetchApp.fetch()错误，似乎未使用标头 [英] UrlFetchApp.fetch() error, doesn't seem to be using headers

问题描述

尝试使用Google Apps脚本从网站获取数据，以将其直接放入电子表格中.提取似乎无效，在Python请求等效的地方效果很好.

Python代码:

  page = requests.get("someurl?as_data_structure"，headers = {'user-agent':'testagent'}) 

GAS代码:

  var页面= UrlFetchApp.fetch("someurl?as_data_structure"，标头= {'user-agent':'testagent'}); 

唯一需要的标头是用户代理，如果我没有包含标头，我从GAS代码中得到的错误通常就是从Python代码中获得的错误.我是JS新手，但据我所知这是正确的方法.

现在将标题放置在正确的位置，但问题仍然存在，与以前完全一样的错误.

  var选项= {标头":{"User-Agent":"testagent"}};var page = UrlFetchApp.fetch("someurl?as_data_structure"，options); 

解决方案

Google并不总是揭示其限制(讨厌吗?).这样的限制之一就是更改用户代理.它固定为

 "User-Agent":"Mozilla/5.0(兼容； Google-Apps-Script)" 

您无法更改.

样本测试:

  function testUrlFetchAppHeaders(){var options = {标头:{'用户代理':'Mozilla/5.0(X11; Linux x86_64)AppleWebKit/537.36(KHTML，例如Gecko)Chrome/51.0.2704.103 Safari/537.36'，}，};var fakeRequest = UrlFetchApp.getRequest('https://www.httpbin.org/headers'，选项);//提供虚假保证var realRequest = UrlFetchApp.fetch('https://www.httpbin.org/headers'，选项);//像破坏球一样Logger.log({假:fakeRequest，实:realRequest})；} 

示例响应:

  {伪造的": {标题":{用户代理":"Mozilla/5.0(X11; Linux x86_64)AppleWebKit/537.36(KHTML，例如Gecko)Chrome/51.0.2704.103 Safari/537.36"}，"method":"get"，有效载荷":"，"followRedirects":是的，"validateHttpsCertificates":是的，"useIntranet":否，"contentType":null，"url":"https://www.httpbin.org/headers"}，真实的": {标题":{"Accept-Encoding":"gzip，deflate，br"，主机":"www.httpbin.org"，"User-Agent":"Mozilla/5.0(兼容； Google-Apps-Script)"}}} 

getRequest(url)

返回调用该操作后将发出的请求.

此方法实际上并不发出请求.

它也不准确地返回将要发出的请求.

Trying to grab data from a website using Google Apps Script to put it directly into a spreadsheet. The fetch does not seem to be working, where the Python requests equivalent works just fine.

Python code:

page = requests.get("someurl?as_data_structure", headers={'user-agent':'testagent'})

GAS code:

var page = UrlFetchApp.fetch("someurl?as_data_structure", headers={'user-agent':'testagent'});

The only required header is the user-agent, and the error I am getting from the GAS code is what I would usually get from the Python code if I hadn't included the header. I am new to js but as far as I know this is the proper way to do it..?

EDIT: Now got the headers in the right place but the issue persists, exactly the same error as before.

var options = {"headers": {"User-Agent": "testagent"}};
var page = UrlFetchApp.fetch("someurl?as_data_structure", options);

解决方案

Google doesn't always reveal it's restrictions(Annoying?). One such restriction is changing the user agent. It's fixed to

"User-Agent": "Mozilla/5.0 (compatible; Google-Apps-Script)"

You can't change it.

Sample Test:

function testUrlFetchAppHeaders() {
  var options = {
    headers: {
      'User-Agent':
        'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36',
    },
  };
  var fakeRequest = UrlFetchApp.getRequest(
    'https://www.httpbin.org/headers',
    options
  );//providing fake assurance
  var realRequest = UrlFetchApp.fetch(
    'https://www.httpbin.org/headers',
    options
  );//like a wrecking ball
  Logger.log({ fake: fakeRequest, real: realRequest });
}

Sample Response:

{
  "fake": {
    "headers": {
      "User-Agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36"
    },
    "method": "get",
    "payload": "",
    "followRedirects": true,
    "validateHttpsCertificates": true,
    "useIntranet": false,
    "contentType": null,
    "url": "https://www.httpbin.org/headers"
  },
  "real": {
    "headers": {
      "Accept-Encoding": "gzip,deflate,br",
      "Host": "www.httpbin.org",
      "User-Agent": "Mozilla/5.0 (compatible; Google-Apps-Script)"
    }
  }
}

getRequest(url)

Returns the request that would be made if the operation was invoked.

This method does not actually issue the request.

Neither does it accurately return the request that would be made.

这篇关于UrlFetchApp.fetch()错误，似乎未使用标头的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！