从需要登录的网站爬取? [英] Scraping from a website that requires a login?

问题描述

如果可以的话，可以这样做吗?我想从xbox.com抓取数据，但是我需要抓取的页面仅在成功登录后出现.

Can this be done if so, how? I want to scrape data from xbox.com but the pages I need to scrape only appear after a successful login.

推荐答案

您将必须通过发送带有CURL请求的POST数据来完成所需的登录事务.就是说，从登录名后面抓取数据是一个坏主意-该网站由于某种原因未将这些信息公开，因此您这样做可能会构成侵犯版权的行为，

You will have to go through the required login transaction by sending POST data with your CURL requests. That said, it is a bad idea to scrape data from behind a login - the site didn't put that information in the public for a reason, and for you to do so might constitute copyright infringement,

这篇关于从需要登录的网站爬取?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！