拆分一个Redis大ZSET [英] Split a Redis big ZSET

问题描述

我们有一个 Redis 密钥.这是一个名为 test_key 的 ZSET 结构.关键是userId，比如123,456,789.分数是时间戳，比如1474194838, 1474194839.它的长度达到了五千万.我们想拆分它，就像test_key_1、test_key_2、test_key_3一样.

如何拆分才能让CRUD更容易?

我们是 Java 开发人员.最常用的Redis商品是zadd、zrem、zrange、zrangeByscore、zrangeByscoreWithScores、zcard 等.

解决方案

如果必须将数据拆分为多个 Redis 实例，则需要一个代理将请求分派到一个或多个 Redis 实例，并合并来自这些实例.代理可以实现为库或服务(例如 RPC 服务器).

<块引用>

调度请求

假设您将数据拆分为 3 部分，并存储在 3 个 Redis 实例中.为了保证负载均衡，代理可以使用MurmurHash函数为每个userId创建一个hash key，并根据hash将请求分派给一个或多个Redis实例钥匙.以zadd为例:zadd test_key userId score.

1. 计算Redis实例id:id = MurmurHash(userId) mod 3
2. 向对应的Redis发送zadd命令:zadd test_key_id userId score

<块引用>

合并结果

当代理从一个或多个Redis实例获取结果时，它会合并结果并返回给客户端.以zcard为例:zcard test_key.

1. 获取所有实例的结果:zcard1 = zcard test_key1、zcard2 = zcard test key2、zcard3 = zcard test_key3
2. 合并结果:zcard_res = zcard1 + zcard2 + zcard3
3. 将结果返回给客户端，即 zcard_res.

<块引用>

提高性能

为了提高性能，当代理向多个实例分派请求时，应该并行分派请求.

We hava a Redis key. It's a ZSET structure named test_key. The key is userId like 123,456,789.The score is time stamps like 1474194838, 1474194839. Its length reached fifty million. We want to split it, just like test_key_1, test_key_2, test_key_3.

How to split it that can make the CRUD more easy?

We are java developer. The most frequently used Redis commodity is zadd, zrem, zrange, zrangeByscore, zrangeByscoreWithScores, zcard and so on.

解决方案

If you have to split the data into several Redis instances, then you need a proxy to dispatch the request to one or more Redis instances, and merge the results from these instances. The proxy can be implemented as a library or a service (e.g. a RPC server).

Dispatch request

Say you split the data into 3 parts, and stored in 3 Redis instances. In order to ensure load balance, the proxy can use MurmurHash function to create a hash key for each userId, and dispatches the request to one or more Redis instances based on the hash key. Take zadd as an example: zadd test_key userId score.

1. Calculate Redis instance id: id = MurmurHash(userId) mod 3
2. Send zadd command to the corresponding Redis: zadd test_key_id userId score

Merge results

When the proxy gets results from one or more Redis instances, it merges the results and returns to the client. Take zcard as an example: zcard test_key.

1. Get results from all instances: zcard1 = zcard test_key1, zcard2 = zcard test key2, zcard3 = zcard test_key3
2. Merge results: zcard_res = zcard1 + zcard2 + zcard3
3. Return the result, i.e. zcard_res, to client.

Improve performance

In order to improve performance, when the proxy dispatches request to more than one instance, it should dispatch the request in parallel.

这篇关于拆分一个Redis大ZSET的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！