来自网站的Python Scraping fb评论 [英] Python Scraping fb comments from a website

问题描述

我一直在尝试使用下面网站页面上的Beautiful Soup刮擦Facebook评论.

I have been trying to scrape facebook comments using Beautiful Soup on the below website pages.

import BeautifulSoup
import urllib2
import re

url = 'http://techcrunch.com/2012/05/15/facebook-lightbox/'

fd = urllib2.urlopen(url)

soup = BeautifulSoup.BeautifulSoup(fd)

fb_comment = soup("div", {"class":"postText"}).find(text=True)

print fb_comment

输出为空集.但是，我可以清楚地看到facebook注释在techcrunch站点的inspect元素中的上述标记内(我对Python并不陌生，并且想知道这种方法是否正确以及我要去哪里了?)

The output is a null set. However, I can clearly see the facebook comment is within those above tags in the inspect element of the techcrunch site (I am little new to Python and was wondering if the approach is correct and where I am going wrong?)

推荐答案

就像Christopher和Thiefmaster:都是因为javascript.

Like Christopher and Thiefmaster: it is all because of javascript.

但是，如果您确实需要该信息，则仍然可以通过 http://seleniumhq.org 然后在此输出上使用beautifulsoup.

But, if you really need that information, you can still retrieve it thanks to Selenium on http://seleniumhq.org then use beautifulsoup on this output.

这篇关于来自网站的Python Scraping fb评论的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！