Redis 队列声明过期 [英] Redis queue with claim expire

问题描述

我有一个要在 redis 中实现的队列接口.诀窍是每个工作人员都可以在 N 秒内领取一件物品，然后假设工作人员已经崩溃并且该物品需要再次领取.工作人员有责任在完成后移除物品.你将如何在 redis 中做到这一点?我正在使用 phpredis，但这有点无关紧要.

I have a queue interface I want to implement in redis. The trick is that each worker can claim an item for N seconds after that it's presumed the worker has crashed and the item needs to be claimable again. It's the worker's responsibility to remove the item when finished. How would you do this in redis? I am using phpredis but that's kind of irrelevant.

推荐答案

为了在 redis 中实现一个简单的队列，可以用来重新提交崩溃的作业，我会尝试这样的事情:

To realize a simple queue in redis that can be used to resubmit crashed jobs I'd try something like this:

• 1 个列表up_for_grabs"
• 1 个列表being_worked_on"
• 自动过期锁

试图抢一份工作的工人会做这样的事情:

a worker trying to grab a job would do something like this:

timeout = 3600
#wrap this in a transaction so our cleanup wont kill the task
#Move the job away from the queue so nobody else tries to claim it
job = RPOPLPUSH(up_for_grabs, being_worked_on)
#Set a lock and expire it, the value tells us when that job will time out. This can be arbitrary though
SETEX('lock:' + job, Time.now + timeout, timeout)
#our application logic
do_work(job)

#Remove the finished item from the queue.
LREM being_worked_on -1 job
#Delete the item's lock. If it crashes here, the expire will take care of it
DEL('lock:' + job)

时不时地，我们可以抓取我们的列表并检查其中的所有作业是否确实有锁.如果我们发现任何没有锁定的作业，这意味着它已过期并且我们的工人可能崩溃了.在这种情况下，我们将重新提交.

And every now and then, we could just grab our list and check that all jobs that are in there actually have a lock. If we find any jobs that DON'T have a lock, this means it expired and our worker probably crashed. In this case we would resubmit.

这将是它的伪代码:

loop do
    items = LRANGE(being_worked_on, 0, -1)
    items.each do |job| 
        if !(EXISTS("lock:" + job))
            puts "We found a job that didn't have a lock, resubmitting"
            LREM being_worked_on -1 job
            LPUSH(up_for_grabs, job)
        end
    end
    sleep 60
end

这篇关于Redis 队列声明过期的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！