BeautifulSoup 和 Scrapy 爬虫的区别? [英] Difference between BeautifulSoup and Scrapy crawler?

问题描述

我想做一个网站，显示亚马逊和 e-bay 产品价格之间的比较.其中哪些会更好地工作，为什么?我对 BeautifulSoup 有点熟悉，但对 Scrapy crawler 不太熟悉.

I want to make a website that shows the comparison between amazon and e-bay product price. Which of these will work better and why? I am somewhat familiar with BeautifulSoup but not so much with Scrapy crawler.

推荐答案

Scrapy 是一个 Web-spider 或 网络爬虫框架，你给 Scrapy 一个根 URL 来启动抓取，然后您可以指定对要抓取和获取的 URL 数量(数量)等的约束.它是用于网页抓取或抓取的完整框架.

Scrapy is a Web-spider or web scraper framework, You give Scrapy a root URL to start crawling, then you can specify constraints on how many (number of) URLs you want to crawl and fetch,etc. It is a complete framework for web-scraping or crawling.

虽然

BeautifulSoup 是一个解析库，它在从 URL 获取内容方面也做得非常好，并允许您毫无困难地解析其中的某些部分.它只获取您提供的 URL 的内容，然后停止.除非您手动将其放入具有特定条件的无限循环中，否则它不会抓取.

BeautifulSoup is a parsing library which also does a pretty good job of fetching contents from URL and allows you to parse certain parts of them without any hassle. It only fetches the contents of the URL that you give and then stops. It does not crawl unless you manually put it inside an infinite loop with certain criteria.

简而言之，使用 Beautiful Soup，您可以构建类似于 Scrapy 的东西.Beautiful Soup 是一个库，而 Scrapy 是一个完整的框架.

In simple words, with Beautiful Soup you can build something similar to Scrapy. Beautiful Soup is a library while Scrapy is a complete framework.

来源

这篇关于BeautifulSoup 和 Scrapy 爬虫的区别?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！