多个延迟的工作流程开始同一工作 [英] Multiple delayed job processes starting same job

问题描述

我在安排多名工人的环境中使用延迟的工作。出于我的问题，这并不重要，但是假设我有10个工人（目前在开发模式下运行）。

I'm using delayed job in a setup where I run multiple workers. For the sake of my question, it doesn't really matter, but let's say I run 10 workers (doing that in development mode currently).

我遇到的问题

据我所知，Delayed Job正在使用悲观锁定来执行同一工作，就是在我的工作对象上调用perform方法。可以防止这种情况的发生，但似乎有时仍有足够的时间来锁定偷工的工作，直到第一名工人有时间将其锁定之前。

To the best of my understanding Delayed Job is using pessimistic locking to prevent this from happening, but it seems it sometimes still have enough time to lock steal the job before the first worker has time to actually lock it.

我只是问看看是否有人遇到此问题，或者是我的设置出现问题。我正在使用Postrgres，并且这同时发生在我的开发机和托管它的Heroku上。

I'm just asking to see if anyone else has experienced this problem, or if it is my setup that is misbehaving. I'm using Postrgres and this happens both in my dev machine and on Heroku where I host it.

我将尝试在工作中解决该问题，但这是这种情况仍然有点问题。理想情况下，延迟的工作不会在两个流程中同时作用于同一工作。

I will try to work around it within my jobs, but it is still a bit problematic that this happens. Ideally it would never happen that delayed job works on the same job from two processes.

谢谢！

推荐答案

我们已经通过延迟工作与12名工人一起完成了约6,000万个工作，但从未对此进行报告。您的延迟工作人员正在运行的SQL是什么？您是否使用的宝石会改变postgres的锁定行为？

We've run about 60 million jobs through delayed job with 12 workers and never had a report of this. Whats the SQL that your delayed job worker is running? Are you using a gem that is changing the locking behavior of postgres?

以下是DJ sql对我来说的样子：

Here is what the DJ sql looks like for me:

UPDATE "delayed_jobs" SET locked_at = '2014-05-02 21:16:35.419748', locked_by =
'host:whatever.local pid:4729' WHERE id IN (SELECT id FROM "delayed_jobs" 
WHERE ((run_at <= '2014-05-02 21:16:35.415923' 
AND (locked_at IS NULL OR locked_at < '2014-05-02 17:16:35.415947') 
OR locked_by = 'host:whatever.local pid:4729') AND failed_at IS NULL) 
ORDER BY priority ASC, run_at ASC LIMIT 1 FOR UPDATE) RETURNING *

您是否对任何其他代码有锁定问题？您能否尝试运行两个Rails控制台会话并执行以下操作：

Do you have locking problems with any other code? Could you try running two rails console sessions and doing this:

控制台会话1：

User.find(1).with_lock do sleep(10); puts "worker 1 done" end

控制台会议2：

User.find(1).with_lock do sleep(1); puts "worker 2 done" end

同时开始两者，如果2在1之前结束，您遇到了更普遍的锁定问题，即工作延误。

Start both those at the same time and if 2 end before 1, you've got a locking problem more general that delayed job.

这篇关于多个延迟的工作流程开始同一工作的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！