我想使用GAE刮取网站并将结果发布到Google实体中 [英] I want to scrape a site using GAE and post the results into a Google Entity
问题描述
进入每个链接并提取出各个部分信息例如然后将结果发布到Google应用引擎上的实体中。
我想知道最好的方法吗?
Chris
为了使用纯Python库对HTML进行规范化处理, href =http://code.google.com/p/html5lib/ =nofollow noreferrer> html5lib 比BeautifulSoup。
然而,你只是想提取简单的结构化信息,这实际上并不需要对HTML进行标准化。我在Google App Engine上有一些使用我自己的xpath库与原始HTML一起使用。
或者您可以使用正则表达式进行一次关闭作业。
Go into each of the links and extract out various pieces of information e.g. permissions, prims etc then post the results into a Entity on google app engine.
I want to know the best way to go about it?
Chris
For normalizing HTML using a pure Python library I have had better experiences with html5lib than BeautifulSoup.
However you just want to extract simply structured information, which doesn't actually require normalizing the HTML. I have a few scraping apps on Google App Engine which use my own xpath library that works with raw HTML. Or you can use regular expressions for one off jobs.
这篇关于我想使用GAE刮取网站并将结果发布到Google实体中的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!