防止googlebot将robots.txt和.htaccess中的文件类型编入索引 [英] prevent googlebot from indexing file types in robots.txt and .htaccess

问题描述

关于如何防止google bot编制索引有很多Stack Overflow问题，例如 txt 文件。是这样的：

There are many Stack Overflow questions on how to prevent google bot from indexing, for instance, txt files. There's this:

robots.txt

User-agent: Googlebot Disallow: /*.txt$

.htaccess

<Files ~ "\.txt$">
     Header set X-Robots-Tag "noindex, nofollow"
</Files>

但是，当试图防止对两种类型的文件建立索引时，这两种语法是什么？ ？就我而言- txt 和 doc 。

However, what is the syntax for both of these when trying to prevent two types of files from being indexed? In my case - txt and doc.

推荐答案

在robots.txt文件中：

In your robots.txt file:

User-agent: Googlebot
Disallow: /*.txt$
Disallow: /*.doc$

更多详细信息，请访问Google网站管理员：创建robots.txt文件

More details at Google Webmasters: Create a robots.txt file

在您的.htaccess文件中：

In your .htaccess file:

<FilesMatch "\.(txt|doc)$">
    Header set X-Robots-Tag "noindex, nofollow"
</FilesMatch>

此处有更多详细信息： http://httpd.apache.org/docs/current/sections.html

More details here: http://httpd.apache.org/docs/current/sections.html

这篇关于防止googlebot将robots.txt和.htaccess中的文件类型编入索引的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！