在谷歌应用程序引擎任务队列中重试最多次数后处理失败 [英] handling failure after maximum number of retries in google app engine task queues
问题描述
我使用的是google-api-python-client，我正在使用谷歌应用程序引擎任务队列进行一些异步操作。
对于特定的任务队列，我还设置了任务应该重试的最大次数（在我的情况下重试不太可能成功，所以我想
有没有办法编写一个处理程序来处理即使在指定次数的重试后任务仍然失败的情况？
基本上，如果我的重试限制是5，在5次重试失败后，我想将任务移动到不同的队列，在此可以重试更多次数，重试，这样更有可能成功。
从这里我相信我可以在每次重试时使用 X-AppEngine-TaskExecutionCount 标题，并编写一些自定义逻辑以知道任务将在最后一次执行并且实现这一点，但我试图找出是否有更清洁的方式。
顺便说一下 X-AppEngine-TaskExecutionCount 指定（从文档）， 执行阶段中此任务先前失败的次数。这个数字不包括由于缺乏可用实例而导致的失败。
至少现在不支持自动将任务从一个队列移动到另一个队列。
一种选择是将任务保留在同一个队列中，增加最大重试次数并使用 retry_parameters 来定制重试退避策略（即重试之间的时间增加）：
retry_parameters
可选。配置失败任务的重试次数。此额外
允许您指定重试特定队列中失败
任务的最大次数。您还可以设置重试
次尝试的时间限制，并控制尝试之间的时间间隔。
重试参数可以包含以下子元素：
task_retry_limit
失败的任务。如果通过 task_age_limit 指定，则App Engine将重试该任务，直到达到
两个限制。如果指定 0 ，则任务将不会重试
。
task_age_limit（推送队列）
重试时间限制从任务第一次运行时开始衡量的失败任务。该值是一个数字，后跟一个时间单位，
，其中单位是 s 数秒， m 数分钟， h 为
小时，或 d 几天。例如，值 5d 指定任务第一次执行尝试后五天的
限制。如果使用 task_retry_limit 指定
，App Engine会重试该任务，直到达到
两个限制。
最小秒数在任务失败后重试任务之前等待。
最大秒数在任务失败后重试任务之前等待。
最大次数在增加变得不变之前，失败的任务重试之间的间隔将加倍。
常数为：2 ** max_doublings * min_backoff_seconds **。
但是增加的模式将是渐进式的 - 在每次失败后翻倍，您不能获得重试之间的重大阶跃 - 如增加的时间。不过，它可能是一个足够好的解决方案，不需要额外的编码。就个人而言，我会采取这种方法。
另一种方法是添加该逻辑以确定该执行是否是原始任务的最终重试，如果是，则排队在具有期望的较慢重试策略的不同队列上的新相应任务。我不确定这是否是你在问题中提到并想避免的。
I am using google-api-python-client and I am using google app engine task queues for some async operations.
For the specific task queue, I am also setting max number of times that the task should be retried(In my case retries are less likely to be successful, so I want to limit them).
Is there a way to write a handler which can handle the case where the task is still failing even after the specified number of retries?
Basically if my retry limit is 5, after 5 unsuccessful retries, I want to move the task to a different queue where it can be retried more number of times with a larger interval between the retries, that way it is more likely to succeed.
From here I believe that I can use X-AppEngine-TaskExecutionCount header in each retry and write some custom logic to know when the task is going to execute for the last time and achieve this but I am trying find out if there is any cleaner way.
By the way X-AppEngine-TaskExecutionCount specifies(from the doc), The number of times this task has previously failed during the execution phase. This number does not include failures due to a lack of available instance.
At least presently there is no support for automatically moving a task from one queue to another.
One option is to keep the task on the same queue, increase the max number of retries and use the retry_parameters to customize the retry backoff policy (i.e. the increase of time between retries):
retry_parameters
Optional. Configures retry attempts for failed tasks. This addition allows you to specify the maximum number of times to retry failed tasks in a specific queue. You can also set a time limit for retry attempts and control the interval between attempts.
The retry parameters can contain the following subelements:
task_retry_limit
The maximum number of retry attempts for a failed task. If specified with task_age_limit, App Engine retries the task until both limits are reached. If 0 is specified, the task will not be retried.
task_age_limit (push queues)
The time limit for retrying a failed task, measured from when the task was first run. The value is a number followed by a unit of time, where the unit is s for seconds, m for minutes, h for hours, or d for days. For example, the value 5d specifies a limit of five days after the task's first execution attempt. If specified with task_retry_limit, App Engine retries the task until both limits are reached.
min_backoff_seconds (push queues)
The minimum number of seconds to wait before retrying a task after it fails.
max_backoff_seconds (push queues)
The maximum number of seconds to wait before retrying a task after it fails.
max_doublings (push queues)
The maximum number of times that the interval between failed task retries will be doubled before the increase becomes constant. The constant is:
2**max_doublings * min_backoff_seconds**.
But the pattern of the increase will be gradual - doubling after each failure, you can't get a significant "step"-like increase of the time between retries. Still, it may be a good enough solution for which no additional coding is required. Personally I'd go for this approach.
Another approach is to add that logic to determine if that execution is the final retry of the original task and, if so, enqueue a new corresponding task on a different queue which has the desired "slower" retry policy. I'm unsure if this is what you were referring to in the question and wanted to avoid.
这篇关于在谷歌应用程序引擎任务队列中重试最多次数后处理失败的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
