使用scrapyd有什么好处? [英] what are the advantages use scrapyd?

问题描述

scrapy doc 说:

The scrapy doc says that:

Scrapy 附带一个名为Scrapyd"的内置服务，它允许您使用 JSON 网络服务部署(又名上传)您的项目并控制它们的爬虫.

Scrapy comes with a built-in service, called "Scrapyd", which allows you to deploy (aka. upload) your projects and control their spiders using a JSON web service.

合规使用scrapyd有什么优势吗?

is there some advantages in comformance use scrapyd?

推荐答案

Scrapyd 允许您通过方便的 Web API 在与您使用的机器不同的机器上运行 scrapy，这意味着您只需使用 curl 甚至是 Web 浏览器来上传新的项目版本并运行它们.否则，如果你想在云端的某个地方运行 Scrapy，你必须 scp 复制新的蜘蛛代码，然后使用 ssh 登录并生成你的 scrapy crawl myspider.

Scrapyd allows you to run scrapy on a different machine than the one you are using via a handy web API which means you can just use curl or even a web browser to upload new project versions and run them. Otherwise if you wanted to run Scrapy in the cloud somewhere you would have to scp copy the new spider code and then login with ssh and spawn your scrapy crawl myspider.

如果你想并行运行多个蜘蛛，Scrapyd 也会为你管理进程；但是如果您在本地机器上安装了 Scrapy 并且可以访问命令行或运行蜘蛛的方法，并且只想一次运行一个蜘蛛，那么您最好手动运行蜘蛛.

Scrapyd will also manage processes for you if you want to run many spiders in parallel; but if you have Scrapy on your local machine and have access to the command-line or a way to run spiders and just want to run one spider at a time, then you're better off running the spider manually.

如果您正在开发蜘蛛，那么您肯定不想使用 scrapyd 进行快速编译/测试迭代，因为它只会增加一层复杂性.

If you are developing spiders then for sure you don't want to use scrapyd for quick compile/test iterations as it just adds a layer of complexity.

这篇关于使用scrapyd有什么好处?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！