我可以使用 robots.txt 阻止任何以数字开头的目录树吗? [英] Can I use robots.txt to block any directory tree that starts with numbers?
问题描述
我什至不确定这是否是处理这个问题的最佳方法，但我在重写时犯了一个暂时的错误，谷歌(可能是其他人)发现了它，现在它已将它们编入索引并不断提出错误.
I'm not even sure if this is the best way to handle this, but I had made a temporary mistake with my rewrites and Google (possibly others) picked up on it, now it has them indexed and keeps coming up with errors.
基本上，我会根据多种因素生成 URL，其中一个是自动生成的文章 ID.这些然后重定向到正确的位置.
Basically, I'm generating URLs based on a variety of factors, one being the id of an article, which is automatically generated. These then redirect to the correct spot.
我第一次不小心设置了这样的东西:
I had first accidentally set up stuff like this:
/2343/news/blahblahblah
/2343/news/blahblahblah
/7645/reviews/blahblahblah
/7645/reviews/blahblahblah
等
这是一个问题，原因有很多，主要是会有重复，而且东西没有指向正确的地方和 yada yada.我现在将它们修复为:
This was a problem for a lot of reasons, the main one being that there would be duplicates and stuff wasn't pointing to the right places and yada yada. And I fixed them to this now:
/news/2343/blahblahblah
/news/2343/blahblahblah
/reviews/7645/blahblahblah
/reviews/7645/blahblahblah
等
这一切都很好.但我想阻止任何属于第一个模式的东西.换句话说，任何看起来像这样的东西:
And that's all good. But I want to block anything that falls into the pattern of the first. In other words, anything that looks like this:
** = 任何数字模式
/**/这里的任何东西
这样 Google(以及任何其他可能编入错误内容的索引)就停止尝试查找这些全都搞砸了甚至不存在的 URL.这可能吗?我是否应该通过 robots.txt 执行此操作?
So that Google (and any others who have maybe indexed the wrong stuff) stops trying to look for these URLs that were all messed up and that don't even exist anymore. Is this possible? Should I even be doing this through robots.txt?
推荐答案
您不需要为此设置 robots.txt，只需为这些网址返回 404 错误，Google 和其他搜索引擎最终会删除它们.
You don't need to setup a robots.txt for that, just return 404 errors for those urls and Google and other search engines will eventually drop them.
Google 还提供网站管理员工具，您可以使用这些工具对网址进行去索引.我很确定其他主机也有类似的东西.
Google also has Webmaster tools which you can use to deindex urls. I'm pretty sure other hosts have similar things.
这篇关于我可以使用 robots.txt 阻止任何以数字开头的目录树吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
