搜索引擎机器人可以抓取需要登录的页面吗? [英] Can Search Engines bots crawl pages requiring login?

问题描述

如果网站首页有用户未登录时的内容和用户登录时的其他内容，搜索引擎机器人是否能够抓取用户的特定内容?

If a homepage on a website has a content if a user is not logged in and another content when the user login, would a search engine bot be able to crawl the user specific content?

如果他们无法抓取，那么我可以从网站的另一部分复制内容，以便在注册时提到他们的需求的用户可以轻松访问.

If they are not able to crawl, then I can duplicate the content from another part of the website to make it easily accessible to users who have mentioned their needs at the registration time.

我的猜测是否定的，但我宁愿在做蠢事之前先确定一下.

My guess is no, but I would rather make sure before I do something stupid.

推荐答案

您不能假设爬虫支持 cookie，但您可以识别爬虫并通过代码让爬虫在您的站点中登录".然而，这会让任何用户假装成为爬虫来获取登录区域中的数据.

You cannot assume that crawler support cookies, but you can identify the crawler and let the crawler be "Logged in" in your site by code. However this will open up for any user to pretend being a crawler to gain the data in the logged in area.

这篇关于搜索引擎机器人可以抓取需要登录的页面吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！