刮Facebook应用程序的数据 [英] Scraping a Facebook App for Data

问题描述

我使用的是一个Facebook应用程序，有一组丰富的，我想获得的离线信息。要做到这一点，我基本上是需要读取网页的infromation到自己的数据库中。很显然，我倒是preFER不要有手动保存网页，让我的应用程序读取网页，并从中拉出相关细节。不幸的是，我是道路封锁的要求进行身份验证，以Facebook的第一位。所以，当我运行此code：

I'm using a Facebook application that has a rich set of information that I'd like to get at offline. To do this, I essentially need to read the infromation from the web pages into my own database. Obviously, I'd prefer not to have to save pages manually and let my application read the pages and pull the relevant details from them. Unfortunately, I am road-blocked by the requirement to authenticate to Facebook first. So when I run this code:

private static string getPage(string pageAddress)
{
    HttpWebRequest req = (HttpWebRequest)WebRequest.Create(new Uri(baseUri, pageAddress));
    HttpWebResponse response = (HttpWebResponse)req.GetResponse();
    StreamReader readStream = new StreamReader(response.GetResponseStream());
    string page = readStream.ReadToEnd();
    readStream.Close();
    response.Close(); // I know, I'm paranoid and this is likely redundant...
    return page;
}

我得到这样的回应：

I get this response:

<script type="text/javascript">
if (parent != self) 
top.location.href = "http://www.facebook.com/login.php?api_key=<obscured>&canvas&v=1.0";
else self.location.href = "http://www.facebook.com/login.php?api_key=<obscured>&canvas&v=1.0";
</script>

任何想法如何告诉应用程序，我真正的（上午）的正宗我？

推荐答案

据我了解，你只需要登录到Facebook的应用的新的吧？使用任何网页抓取/抓取它的框架（它们支持JS，饼干等）。他们只是模仿usuall网页浏览。例如，请尝试以下：

As far as I understood you just need to login to facebook appliction, right? Use any web scraping/crawling framework for it (they support JS, cookies, etc.). They just emulate usuall web browsing. For example, try these:

http://scrapy.org/

http://wwwsearch.sourceforge.net/mechanize/

http://watin.sourceforge.net/

另请参见

<一个href="http://stackoverflow.com/questions/1852725/net-screen-scraping-and-session">http://stackoverflow.com/questions/1852725/net-screen-scraping-and-session

这篇关于刮Facebook应用程序的数据的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！