Google App Engine - 任务队列与 Cron 作业 [英] Google App Engine - Task Queues vs Cron Jobs

问题描述

最新的 Google App Engine 版本支持新的任务队列 API 在 Python 中.我正在比较此 API 的功能与现有的 Cron 服务.对于非用户启动的后台作业，例如抓取 RSS 提要并每天对其进行解析.任务队列 API 可以而且应该用于非用户发起的请求吗?

The latest Google App Engine release supports a new Task Queue API in Python. I was comparing the capabilities of this API vs the already existing Cron service. For background jobs that are not user-initiated, such as grabbing an RSS feed and parsing it on a daily interval. Can and should the Task Queue API be used for non-user initiated requests such as this?

推荐答案

我会说有点".关于任务队列要记住的事情是:

I'd say "sort of". The things to remember about task queues are:

1) 每分钟/小时/天的操作限制与定期重复某事不同.即使将令牌桶大小设置为 1，我也不认为您可以保证这些重复间隔均匀.这取决于他们说队列作为令牌桶实现时的严重程度，以及该语句是否应该是接口的保证部分.这是实验室，尚无任何保证.

1) a limit of operations per minute/hour/day is not the same as repeating something at regular intervals. Even with the token bucket size set to 1, I don't think you're guaranteed that those repetitions will be evenly spaced. It depends how serious they are when they say the queue is implemented as a token bucket, and whether that statement is supposed to be a guaranteed part of the interface. This being labs, nothing is guaranteed yet.

2) 如果任务失败，则重新排队.如果一个 cron 作业失败，那么它会被记录下来并且不会重试，直到它再次到期.因此，cron 作业的行为方式与添加自身副本然后刷新提要的任务或刷新提要然后添加自身副本的任务的行为方式不同.

2) if a task fails then it's requeued. If a cron job fails, then it's logged and not retried until it's due again. So a cron job doesn't behave the same way either as a task which adds a copy of itself and then refreshes your feed, or as a task which refreshes your feed and then adds a copy of itself.

很可能可以使用任务来模拟 cron 作业，但我怀疑这是否值得.如果您正在尝试解决运行时间超过 30 秒(或达到任何其他请求限制)的 cron 作业，那么您可以将工作拆分为多个部分，并使用一个 cron 作业将所有部分添加到任务队列.有一些关于异步 urlfetch 的讨论(在 GAE 博客中?)，这可能是更新 RSS 提要的最佳方式.

It may well be possible to mock up cron jobs using tasks, but I doubt it's worth it. If you're trying to work around a cron job which takes more than 30 seconds to run (or hits any other request limit), then you can split the work up into pieces, and have a cron job which adds all the pieces to a task queue. There was some talk (in the GAE blog?) about asynchronous urlfetch, which might be the ultimate best way of updating RSS feeds.

这篇关于Google App Engine - 任务队列与 Cron 作业的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！