如何没有索引特定的URL? [英] How to no index specific URLS?

问题描述

我一直在寻找如何不索引特定的URL，但是我没有找到关于以下内容的任何特定信息.

I was searching around on how to no index specific URLs but I havent found any specific info on the following.

通过添加以下内容

<?php if(is_single(X)): ?>
<meta name="robots" content="noindex,nofollow">
<?php endif; ?>

我将无法对(X)进行索引，其中X可能是帖子ID，例如"Hello World"的帖子标题或"hello-world"的帖子.

I would be able to no index the (X) where X could be the post ID, the post title of "Hello World" for example , or a post slug of "hello-world".

是否可以指定 all 以相同的帖子或标题开头的URL，例如下面的示例?

Would if be possible to specify all URLs which start with the same post slug or title for example, as in the example below?

www.test.com/REF-123-mytest.html
www.test.com/REF-123-yourtest.html
www.test.com/REF-123-histest.html

我是否可以遗漏以REF-123开头的所有 URL?

Could I leave out all URLs which start by REF-123 for example?

推荐答案

通过使用robots.txt，您可以禁止爬网.

By using robots.txt, you can disallow crawling.

通过使用meta-robots(或HTTP标头X-Robots-Tag)，您可以禁止索引.

By using meta-robots (or the HTTP header X-Robots-Tag), you can disallow indexing.

如果您打算禁止建立索引，则您不应禁止robots.txt中的网址，否则漫游器将永远不会知道您不希望对这些URL进行索引.

If you intend to forbid indexing, you shouldn’t disallow the URLs in robots.txt, otherwise bots will never know that you don’t want these URLs to be indexed.

如果要禁止爬网，可以使用以下robots.txt:

In case you want to disallow crawling, you could use this robots.txt:

User-agent: *
Disallow: /REF-123

这将应用于所有路径以REF-123开头的URL(区分大小写！).

This would apply to all URLs whose paths start with REF-123 (case-sensitive!).

如果要禁止索引编制，可以将此meta元素添加到所有这些页面

In case you want to disallow indexing, you could add to all these pages this meta element

<meta name="robots" content="noindex">

或发送相应的HTTP标头X-Robots-Tag:

or send the corresponding HTTP header X-Robots-Tag:

X-Robots-Tag: noindex

这篇关于如何没有索引特定的URL?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！