如何使用纯Redis以原子方式删除与模式匹配的数百万个键? [英] How to atomically delete millions of keys matching a pattern using pure Redis?

问题描述

假设我有数百万个 prefix: 键.

Say I have millions of prefix:<numeric_id> keys.

我想以原子方式清除它们.

I want to purge them all atomically.

如何使用原子删除匹配模式的键Redis 显示了许多选项.有些使用 redis-cli 或 Bash 脚本，但我需要以编程方式使用我的客户端来完成.

How to atomically delete keys matching a pattern using Redis shows many options. Some use redis-cli or Bash script but I need to do it using my client, programmatically.

Lua 脚本方法很有前途，但使用 KEYS 命令的解决方案失败并出现太多元素无法解包"错误.

A Lua script approach is promising, but solutions with KEYS command fail with "too many elements to unpack" error.

如何实现这一目标?

推荐答案

以下 Lua 脚本使用了 SCAN 命令，因此它会在脚本内分块删除 - 避免太多元素无法解包"错误.

The following Lua script uses SCAN command, so it deletes in chunks within the script - avoiding the "too many elements to unpack" error.

local cursor = 0
local calls = 0
local dels = 0
repeat
    local result = redis.call('SCAN', cursor, 'MATCH', ARGV[1])
    calls = calls + 1
    for _,key in ipairs(result[2]) do
        redis.call('DEL', key)
        dels = dels + 1
    end
    cursor = tonumber(result[1])
until cursor == 0
return "Calls " .. calls .. " Dels " .. dels

它返回调用了多少次SCAN以及删除了多少个键.

It returns how many times SCAN was called and how many keys were deleted.

用作:

EVAL "local cursor = 0 local calls = 0 local dels = 0 repeat    local result = redis.call('SCAN', cursor, 'MATCH', ARGV[1])     calls = calls + 1   for _,key in ipairs(result[2]) do       redis.call('DEL', key)      dels = dels + 1     end     cursor = tonumber(result[1]) until cursor == 0 return 'Calls ' .. calls .. ' Dels ' .. dels" 0 prefix:1

请注意，它会在运行时阻塞服务器，因此不建议按原样用于生产.

Note it will block the server while running, so it is not advised for production as is.

对于生产，考虑将 DEL 更改为 UNLINK.您还可以返回光标(而不是在脚本内重复直到它为零)并将 COUNT 参数添加到 SCAN 以进行节流(请参阅 REDIS 中的 SCAN/HSCAN 命令有没有推荐的 COUNT 值?).这样你就可以分块而不是一次性完成，类似于 如何获取redis中的所有集合?

For production, consider changing DEL for UNLINK. You can also return the cursor (instead of repeating inside the script until it is zero) and add COUNT parameter to SCAN to throttle (see Is there any recommended value of COUNT for SCAN / HSCAN command in REDIS?). This way you do it in chunks instead of one go, similar to How can I get all of the sets in redis?

或者您可以使用此答案中所述的方法做一些更复杂的事情:Redis `SCAN`:如何在可能匹配的新键之间保持平衡并确保在合理的时间内最终结果?

Or you can do something more sophisticated using the approach stated in this answer: Redis `SCAN`: how to maintain a balance between newcomming keys that might match and ensure eventual result in a reasonable time?

这篇关于如何使用纯Redis以原子方式删除与模式匹配的数百万个键?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！