X-Robots-Tag标题vs机器人元标记的优先级 [英] Precedence of X-Robots-Tag header vs robots meta tag

问题描述

我在我的虚拟主机配置中放置了以下 Header ：

  Header set X-Robots-Tagnoindex，nofollow
  

这里的目标是禁用搜索引擎将索引我的测试环境。该网站是Wordpress，并且安装了一个插件来管理每个网页的元机器人设置。例如：

 < meta name =robotscontent =index，follow/> 
  

所以我的问题是，哪个指令会优先于另一个指令，因为在每个页面？

解决方案

我不确定是否可以给这个问题一个确定的答案，因为这个行为可能是依赖于实现的不过，我认为有合理的证据表明 X-Robots-Tag 会带走优先于< meta name =robots... 。请参阅： $ b

X-Robots-Tag 和机器人之间的一个显着区别 meta指令是：

• X-Robots-Tag 是 HTTP 协议标头的一部分。
是 HTML 文档标题的一部分。

$ b

因此， X-Robots -Tag 属于HTTP协议层，而< meta name =robots... 属于HTML协议层。








因为它们属于不同的协议层，所以它们不会被（机器人）客户端获取页面同时解析：首先解析HTTP层，然后在后面的步骤中解析HTML。

（另外，应该注意的是， X-Robots-Tag 和< meta name =机器人... 不被所有机器人支持和雅虎/ Bing支持，但根据这只支持< meta name =robots... ，其他人都不支持。）




小结：如果机器人支持 X-Robots-Tag


/ code>将被首先处理;限制条件（noindex，nofollow）适用（并且< meta name =robots... 被忽略）。

• < meta name =robots... 指示适用。
  



I've placed the following Header in my vhost config:

Header set X-Robots-Tag "noindex, nofollow"

The goal here is to just disable search engines from indexing my testing environment. The site is Wordpress and there is a plugin installed to manage per-page the meta robots settings. For example:

<meta name="robots" content="index, follow" />

So my question is, which directive will take precedence over the other since both are being set on every page?

解决方案

I am not sure if a definitive answer can be given to the question, as the behavior may be implementation-dependent (on the robot side).

However, I think there is reasonable evidence that X-Robots-Tag will take precedence over <meta name="robots" .... See :

One significant difference between the X-Robots-Tag and the robots meta directive is:

• X-Robots-Tag is part of the HTTP protocol header.
• <meta name="robots" ... is part of the HTML document header.

Therefore the the X-Robots-Tag belongs to HTTP protocol layer, while <meta name="robots" ... belongs to the HTML protocol layer.

As they belong to a different protocol layer, they will not be parsed simultaneously by the (robot) client getting the page: The HTTP layer will be parsed first, and the HTML in a later step.

(Also, it should be noted that X-Robots-Tag and <meta name="robots" ... are not suppported by all robots. Google and Yahoo/Bing suppport both, but according to this some support only <meta name="robots" ..., others support neither.)

Summary :

• if supported by the robot, X-Robots-Tag will be processed first ; restrictions (noindex, nofollow) apply (and <meta name="robots" ... is ignored).
• else, <meta name="robots" ... directive applies.

这篇关于X-Robots-Tag标题vs机器人元标记的优先级的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！