Robots.txt 允许子文件夹但不允许父文件夹 [英] Robots.txt Allow sub folder but not the parent

问题描述

谁能解释一下以下场景的正确 robots.txt 命令.

Can anybody please explain the correct robots.txt command for the following scenario.

我想允许访问:

/directory/subdirectory/..

但我也想限制对 /directory/ 的访问，尽管有上述例外.

But I would also like to restrict access to /directory/ not withstanding the above exception.

推荐答案

请注意，没有真正的官方标准，任何网络爬虫都可能很乐意忽略您的 robots.txt

Be aware that there is no real official standard and that any web crawler may happily ignore your robots.txt

根据Google 群组帖子，以下内容至少适用于 GoogleBot；

According to a Google groups post, the following works at least with GoogleBot;

User-agent: Googlebot 
Disallow: /directory/ 
Allow: /directory/subdirectory/

这篇关于Robots.txt 允许子文件夹但不允许父文件夹的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！