我可以允许(通过搜索引擎)索引受限内容而不将其公开吗? [英] Can I allow indexing (by search engines) of restricted content without making it public?

问题描述

我有一个包含一些受限内容的网站.我希望我的网站出现在搜索结果中，但我不希望它公开.

I have a site with some restricted content. I want my site to appear in search results, but I do not want it to get public.

有什么方法可以让抓取工具抓取我的网站但阻止它们将其公开?

Is there a way by which I can allow crawlers to crawl through my site but prevent them from making it public?

我找到的最接近的解决方案是 GoogleFirst Click Free 但即使是第一次显示内容也需要我.

The closest solution I have found is Google First Click Free but even it requires me to show the content for the first time.

推荐答案

您为什么要允许人们搜索他们单击链接后无法访问的页面?从技术上讲，它可能会变得困难(如果用户代理包含googlebot"，请检查您的身份验证代码，但如果人们希望您的内容足够糟糕，则没有什么可以阻止人们伪造此用户代理)，但基本上毫无意义.

Why do you want to allow people to search for a page that they can't access if they click the link? Its technically possible to make it difficult (check in your authentication code if useragent contains 'googlebot', though there is nothing stopping people from faking this useragent if they want your content bad enough) but largely pointless.

此外，谷歌的官方路线(IIRC，尽管在任何地方都找不到)是，您可能会因故意向 googlebot 显示与人类用户看到的内容不同的内容而受到处罚.

Also google's official line (IIRC, can't find this anywhere though) is that you may be penalized for deliberately trying to show googlebot different content to what human users see.

这篇关于我可以允许(通过搜索引擎)索引受限内容而不将其公开吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！