Googlebot和空的CORS回应 [英] Googlebot and empty CORS responses

问题描述

我们有一个React应用程式可以从其他网域异步载入一些资料。使用 cors 模式中的 isomorphic-fetch 进行请求，并且测试时请求和响应都很好，使用我自己的浏览器。

We have a React app that loads some data asynchronously from another domain. The requests are made using isomorphic-fetch in cors mode and the requests and responses all look fine and work correctly when testing using my own browser.

我们监视回复和记录失败回到我们的应用程序进行分析。

We have monitoring of the responses and log failures back to our application for analysis.

虽然大部分时间都很好（一切似乎正在索引正确，并在谷歌显示得很好），我们仍然看到很多失败，只有对于Googlebot，它无法获取数据正确。调试响应对象我看到 status 是200，但 statusText 是空的。响应没有正文（因此没有 .json 或 .text 方法），没有标题t的情况），模式正确设置为 cors （不是 opaque ，这可能解释一些其他oddities）。

While most of the time all is well (and everything seems to be getting indexed correctly and showing up fine in Google) we still see a lot of failures, only for Googlebot, where it's failing to fetch the data correctly. Debugging the response object I see that the status is 200, but the statusText is empty. The response has no body (and so no .json or .text methods), and no headers (which shouldn't be the case) and the mode is correctly set as cors (not opaque, which might explain some of the other oddities).

从我对CORS的理解，这一切看起来都是在发送和接收的头部，所以为什么Googlebot有这么多的间歇性问题？ Googlebot说它有一个HTTP 200响应（成功，Promise不被拒绝），但它缺少HTTP 200 responose所有的东西 - 它没有主体和没有头暴露。为什么Googlebot无法返回包含标头和正文的响应（如下所述）？

普通预检请求看起来像这样（来自Chome devtools） （添加了 * / \ * 中的额外斜杠以阻止它认为这是一个注释开启者）

A normal preflight request looks like this (from Chome devtools) (extra slash in */\* added to stop SO thinking that it's a comment opener)

Accept:*/\*
Accept-Encoding:gzip, deflate, sdch, br
Accept-Language:en-GB,en-US;q=0.8,en;q=0.6
Access-Control-Request-Headers:content-type, x-apikey
Access-Control-Request-Method:POST
Cache-Control:no-cache
Connection:keep-alive
DNT:1
Host:my.host.net
Origin:http://my.origin.net
Pragma:no-cache
Referer:http://my.origin.net/
User-Agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.100 Safari/537.36

预检回应如下：

Access-Control-Allow-Headers:content-type,x-apikey
Access-Control-Allow-Origin:*
Cache-Control:no-cache
Connection:keep-alive
Content-Length:0
Date:Mon, 05 Dec 2016 00:55:05 GMT
Expires:-1
Pragma:no-cache
Server:Microsoft-IIS/8.5
X-AspNet-Version:4.0.30319
X-Powered-By:ASP.NET

其后是实际的请求，看起来像这样（作为带有JSON主体的POST发送）

Which is then followed up by the actual request which looks like this (sent as a POST with a JSON body)

accept:application/json
Accept-Encoding:gzip, deflate, br
Accept-Language:en-GB,en-US;q=0.8,en;q=0.6
Cache-Control:no-cache
Connection:keep-alive
Content-Length:62
content-type:application/json
DNT:1
Host:someapi.net
Origin:http://my.origin.net
Pragma:no-cache
Referer:http://my.origin.net/
User-Agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like  Gecko) Chrome/54.0.2840.100 Safari/537.36
x-apikey:someapikey

它返回一个这样的响应（带有一个JSON主体）

Which returns a response like this (with a JSON body)

Access-Control-Allow-Origin:*
Cache-Control:no-cache
Connection:keep-alive
Content-Length:33576
Content-Type:application/json; charset=utf-8
Date:Mon, 05 Dec 2016 00:55:05 GMT
Expires:-1
Pragma:no-cache
Server:Microsoft-IIS/8.5
X-AspNet-Version:4.0.30319
X-Powered-By:ASP.NET

推荐答案

检查失败的GoogleBot通话的IP地址

Check the IP address of the failing GoogleBot calls

这可能是一个恶意的演员，

It may be a nefarious actor, pretending to be google

按照此处所述检查IP地址：

Check the IP addresses as described here:

https://support.google.com/webmasters/answer/80553?hl=zh-CN

这篇关于Googlebot和空的CORS回应的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！