将 >10 万个页面链接在一起而不会受到 SEO 处罚 [英] Linking together >100K pages without getting SEO penalized

问题描述

我正在创建一个网站，该网站将对互联网上数十万个其他网站的隐私政策进行审查.它的初始内容基于我对 CommonCrawl 50 亿网页转储和 使用脚本分析所有隐私政策，以识别某些特征(例如出售您的个人信息").

I'm making a site which will have reviews of the privacy policies of hundreds of thousands of other sites on the internet. Its initial content is based on my running through the CommonCrawl 5 billion page web dump and analyzing all the privacy policies with a script, to identify certain characteristics (e.g. "Sells your personal info").

根据 SEO MOZ SEO 初学者指南:

搜索引擎往往只会抓取任何给定页面上的大约 100 个链接.这种宽松的限制是必要的，以减少垃圾邮件并保存排名.

Search engines tend to only crawl about 100 links on any given page. This loose restriction is necessary to keep down on spam and conserve rankings.

我想知道什么是创建导航网络的明智方法，它不会留下任何页面孤立，但仍然可以避免他们所说的这种 SEO 惩罚.我有几个想法:

I was wondering what would be a smart way to create a web of navigation that leaves no page orphaned, but would still avoid this SEO penalty they speak of. I have a few ideas:

• 创建按字母顺序排列的页面(或 Google 站点地图 .xml 的)，例如以 Ado* 开头的站点".例如，它会在那里链接Adobe.com".这种或任何其他无意义的页面拆分似乎有点做作，我想知道 Google 是否会不喜欢它.
• 使用元关键字或描述进行分类
• 找到一些方法来应用更有趣的类别，例如基于地理或基于内容的类别.我在这里担心的是，我不确定如何将这些类别全面应用于这么多网站.我想如果需要的话，我可以编写另一个分类器来尝试分析抓取的页面内容.不过，这本身就是一项艰巨的工作.
• 使用 DMOZ 项目帮助对页面进行分类.

Wikipedia 和 StackOverflow 通过允许用户对所有页面进行分类或标记，显然很好地解决了这个问题.就我而言，我没有那么奢侈，但我想找到最好的选择.

Wikipedia and StackOverflow have obviously solved this problem very well by allowing users to categorize or tag all of the pages. In my case I don't have that luxury, but I want to find the best option available.

这个问题的核心是 Google 如何响应不同的导航结构.它会惩罚那些以编程/无意义的方式创建网页的人吗?还是只要一切都通过链接连接它就不在乎?

At the core of this question is how Google responds to different navigation structures. Does it penalize those who create a web of pages in a programmatic/meaningless way? Or does it not care so long as everything is connected via links?

推荐答案

Google PageRank 不会因为页面上的链接超过 100 个而惩罚您.但是每个超过某个阈值的链接在 PageRank 算法中的价值/重要性都会降低.

Google PageRank does not penalize you for having >100 links on a page. But each link above a certain threshold decreases in value/importance in the PageRank algorithm.

引用 SEOMOZ 和 Matt Cutts:

Quoting SEOMOZ and Matt Cutts:

你会受到惩罚吗?

在我们深入挖掘之前，我想说明一下 100-link限制从来都不是惩罚的情况.在 2007 年 8 月的一次采访中，兰德引用马特·卡茨的话说:

Before we dig in too deep, I want to make it clear that the 100-link limit has never been a penalty situation. In an August 2007 interview, Rand quotes Matt Cutts as saying:

将链接数保持在 100 以下"在技​​术上指南部分，而不是质量指南部分.这意味着如果您有 101 或 102 个链接，我们不会删除页面页.将此更多地视为经验法则.

The "keep the number of links to under 100" is in the technical guideline section, not the quality guidelines section. That means we're not going to remove a page if you have 101 or 102 links on the page. Think of this more as a rule of thumb.

在当时，很可能谷歌在某个时间点后开始忽略链接，但最糟糕的是这使那些后 100 链接无法通过 PageRank.页面本身不会被取消索引或惩罚.

At the time, it's likely that Google started ignoring links after a certain point, but at worst this kept those post-100 links from passing PageRank. The page itself wasn't going to be de-indexed or penalized.

所以真正的问题是如何让 Google 认真对待您的所有链接.您可以通过生成供 Google 抓取的 XML 站点地图来完成此操作(您可以拥有静态的 sitemap.xml 文件，也可以动态生成其内容).您需要仔细阅读关于站点地图 Google 网站站长工具帮助文档的部分.

So the question really is how to get Google to take all your links seriously. You accomplish this by generating a XML sitemap for Google to crawl (you can either have a static sitemap.xml file, or its content can be dynamically generated). You will want to read up on the About Sitemaps section of the Google Webmaster Tools help documents.

就像页面上的链接过多是一个问题一样，XML 站点地图文件中的链接过多也是一个问题.您需要做的是对您的 XML 站点地图进行分页.Jeff Atwood 谈到 StackOverflow 如何实现这一点:站点地图.Jeff 还在 StackOverflow 播客 #24 上讨论了同样的问题.

Just like having too many links on a page is an issue,having too many links in a XML sitemap file is also an issue. What you need to do is paginate your XML sitemap. Jeff Atwood talks about how StackOverflow implements this: The Importance of Sitemaps. Jeff also discusses the same issue on StackOverflow podcast #24.

此外，此概念也适用于 Bing.

Also, this concept applies to Bing as well.

这篇关于将 >10 万个页面链接在一起而不会受到 SEO 处罚的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！