Rails robots.txt 文件夹 [英] Rails robots.txt folders

问题描述

我将要启动一个 Rails 应用程序，作为最后一个任务，我不想设置 robots.txt 文件.我找不到有关如何为 Rails 应用程序正确编写路径的信息.

I'm about to launch a Rails app and as the last task, I wan't to set the robots.txt file. I couldn't find information about how the paths should be written properly for a Rails app.

起始路径是否总是来自 Rails 应用程序或应用程序文件夹的根路径?我将如何禁止例如img文件夹?

Is the starting path always the root path from the Rails app or the app folder? How would I then disallow e.g. the img folder?

我是否必须按照我在应用程序文件夹中看到的路径来编写路径，或者像在线站点上的路径一样，例如http://example.com/admin ?

Do I have to write the paths as I see it in the app folder, or like how the paths look like on the site online, e.g. http://example.com/admin ?

推荐答案

你必须把你的 robots.txt 放在 /public 文件夹中

you have to put your robots.txt in /public folder

它看起来像下面

以下示例/robots.txt"文件指定机器人不应访问任何以"/cyberworld/map/"或"/tmp/"开头的URL, 或 /foo.html:

The following example "/robots.txt" file specifies that no robots should visit any URL starting with "/cyberworld/map/" or "/tmp/", or /foo.html:

# robots.txt for http://www.example.com/

User-agent: *
Disallow: /cyberworld/map/ # This is an infinite virtual URL space
Disallow: /tmp/ # these will soon disappear
Disallow: /foo.html

欲了解更多信息，请点击以下链接

for more information please follow below links

https://developers.google.com/webmasters/control-crawl-index/docs/robots_txt

和

http://www.robotstxt.org/robotstxt.html

这篇关于Rails robots.txt 文件夹的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！