在python中html解析和网页抓取之间的区别是多少？ [英] How much is the difference between html parsing and web crawling in python

问题描述

我需要从我的django网站的网站上获取一些数据。
现在我很困惑我是否应该使用python解析库或Web爬行库。搜索引擎库也属于同一类别

我想知道两者之间的区别是多少，如果我想在我的网站中使用这些功能，我应该使用

解决方案

如果你可以摆脱背景网页抓取使用 scrapy 。如果需要立即使用 html5lib （更强大）或 lxml （更快）。如果您要稍后执行，请使用令人敬畏的请求库。我会避免使用BeautifulSoup，机械化，urllib2，httplib。

I need to grab some data from websites in my django website. Now i am confused whether i should use python parsing libraries or web crawling libraries. Does search engine libraries also fall in same category

I want to know how much is the difference between the two and if i want to use those functions inside my website which should i use

解决方案

If you can get away with background web crawling use scrapy. If need to immediately grab something use html5lib (more robust) or lxml (faster). If you are going to be doing the later, use the awesome requests library. I would avoid using BeautifulSoup, mechanize, urllib2, httplib.

这篇关于在python中html解析和网页抓取之间的区别是多少？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！