是否可以在一行中列出多个用户代理? [英] Is it possible to list multiple user-agents in one line?

问题描述

是否可以在 robots.txt 中向多个机器人发出一条指令而不必重复提及?

Is it possible in robots.txt to give one instruction to multiple bots without repeatedly having to mention it?

示例:

User-agent: googlebot yahoobot microsoftbot
Disallow: /boringstuff/

推荐答案

实际上很难给出明确的答案，因为 robots.txt 没有一个非常明确的标准，而且很多那里的文档含糊不清或相互矛盾.

It's actually pretty hard to give a definitive answer to this, as there isn't a very well-defined standard for robots.txt, and a lot of the documentation out there is vague or contradictory.

Google 漫游器可理解的格式说明是相当全面，包括这个稍微有点乱的句子:

The description of the format understood by Google's bots is quite comprehensive, and includes this slightly garbled sentence:

紧接在彼此之后的多条起始组行将跟随在最后一个起始组行之后的组成员记录.

Muiltiple start-of-group lines directly after each other will follow the group-member records following the final start-of-group line.

这似乎是在摸索以下示例中显示的内容:

Which seems to be groping at something shown in the following example:

user-agent: e
user-agent: f
disallow: /g

根据下面的解释，这构成了一个组"，不允许两个不同的用户代理使用相同的 URL.

According to the explanation below it, this constitutes a single "group", disallowing the same URL for two different User Agents.

因此，您想要的正确语法(关于任何以与 Google 相同方式工作的机器人)将是:

So the correct syntax for what you want (with regards to any bot working the same way as Google's) would then be:

User-agent: googlebot
User-agent: yahoobot
User-agent: microsoftbot
Disallow: /boringstuff/

然而，正如 Jim Mischel 指出的那样，robots.txt 文件中没有任何意义，某些机器人会正确解释，但其他机器人可能会窒息，所以最好使用重复块的最低公分母"，可能是通过使用简单的配方"和更新脚本动态生成文件.

However, as Jim Mischel points out, there is no point in a robots.txt file which some bots will interpret correctly, but others may choke on, so it may be best to go with the "lowest common denominator" of repeating the blocks, perhaps by dynamically generating the file with a simple "recipe" and update script.

这篇关于是否可以在一行中列出多个用户代理?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！