robots.txt 中的通配符 [英] Wildcards in robots.txt
问题描述
如果在 WordPress 网站中我按以下顺序分类:
If in WordPress website I have categories in this order:
-Parent
--Child
---Subchild
我将永久链接设置为:%category%/%postname%
I have permalinks set to: %category%/%postname%
让我们举个例子.我创建帖子名称为体育游戏".它的标签是运动游戏.它的完整网址是:domain.com/parent/child/subchild/sport-game我之所以使用这种固定链接，正是为了更方便地屏蔽 robots.txt 中的一些内容.
Let use an example. I create post with post name "Sport game". It's tag is sport-game. It's full url is: domain.com/parent/child/subchild/sport-game Why I use this kind of permalinks is exactly to block some content easier in robots.txt.
现在这是我有疑问的部分.在 robots.txt 中:
And now this is the part I have question for. In robots.txt:
User-agent: Googlebot
Disallow: /parent/*
Disallow: /parent/*/*
Disallow: /parent/*/*/*
Disallow:/parent/* 这条规则的意思是它阻止了 domain.com/parent/child 而不是 domain.com/parent/child/subchild 而不是 domain.com/parent/?
Disallow: /parent/* Is meaning of this rule that it's blocking domain.com/parent/child but not domain.com/parent/child/subchild and not domain.com/parent/?
Disallow:/parent/*/*/* 这意味着它正在阻止 domain.com/parent/child/subchild/，它只阻止子孩子，不是子项，不是父项，也不是子子项下的帖子?
Disallow: /parent/*/*/* Is meaning of this that it's blocking domain.com/parent/child/subchild/, that it's blocking only subchild, not child, not parent, and not posts under subchild?
推荐答案
请注意，Disallow 中的 * 通配符不是原始 robots.txt 规范的一部分.一些解析器支持它，但由于没有规范，他们可能会以不同的方式处理它.
Note that the * wildcard in Disallow is not part of the original robots.txt specification. Some parsers support it, but as there is no specification, they might all handle it differently.
您似乎对 Googlebot 感兴趣，请查看 Google 的机器人.txt 文档.
As you seem to be interested in Googlebot, have a look at Google’s robots.txt documentation.
在示例中很明显 * 意味着
In the examples it becomes clear that * means
任何字符串
任何字符串"当然也可以包含/.
"Any string" may, of course, also contain /.
所以你的第一行 Disallow:/parent/* 应该阻止每个路径以 /parent/ 开头的 URL，包括用斜杠分隔的路径段.
So your first line Disallow: /parent/* should block every URL whose path starts with /parent/, including path segments separated by slashes.
请注意，这与原始 robots.txt 规范中的 Disallow:/parent/ 相同，它还会阻止路径以 /parent/ 开头的任何 URL，例如:
Note that this would be the same as Disallow: /parent/ in the original robots.txt specification, which also blocks any URL whose paths starts with /parent/, for example:
http://example.com/parent/http://example.com/parent/foohttp://example.com/parent/foo.htmlhttp://example.com/parent/foo/barhttp://example.com/parent/foo/bar/http://example.com/parent/foo/bar/foo.html
这篇关于robots.txt 中的通配符的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
