使用Jedis如何写入Redis集群中的特定插槽/节点 [英] Using jedis how to write to a specific slot/node in redis cluster

问题描述

我正在尝试提高将数据写入Redis集群的性能. 我们正计划从redi-sentinel转换为集群模式以实现可伸缩性.

I'm trying to improve the performance of writing data to redis cluster. We are planning to move from redi-sentinel to cluster mode for scalability.

但是，与redis-sentinel相比，写操作的性能要低得多.我们在redis-sentinel中利用了管道，但是集群模式不支持管道.

But, the performance of write operations is very less compared to redis-sentinel. We leveraged pipeline in redis-sentinel but cluster mode doesn't support pipeline.

因此，我正在考虑将所有进入同一节点的密钥归为一组，然后使用管道将批次发送到该特定节点.

So, I was thinking to group all the keys that go to a same node and send the batch to that specific node using pipeline.

因此，我想知道如何(在写入集群之前)将特定密钥写入哪个节点/插槽?

So, I'm wondering how to know/compute (before writing to cluster) to which node/slot a particular key would be written to?

推荐答案

解决方案1:
找到了一种解决方案，用于识别将要插入钥匙的插槽. JedisCluster有一些API可以获取它.

Solution 1:
Found a solution to identify the slot to which keys would go into. JedisCluster has some APIs to get it.

int slotNum = JedisClusterCRC16.getSlot(key); -提供密钥的插槽号.

int slotNum = JedisClusterCRC16.getSlot(key); - Provides the slot number of the key.

Set<HostAndPort> redisClusterNode = new HashSet<HostAndPort>();
redisClusterNode.add(new HostAndPort(hostItem, port));

JedisSlotBasedConnectionHandler connHandler = new 
              JedisSlotBasedConnectionHandler(redisClusterNode, poolConfig, 60);

Jedis jedis = connHandler.getConnectionFromSlot(slotNum);

这为群集中的特定节点提供了jedis对象(内部来自Jedispool).
现在有了上述jedis对象，就可以轻松地为特定节点(在集群中)流水线化所有命令

This provides the jedis object (from Jedispool internally) for the specific node in the cluster.
Now with the above jedis object all commands can be easily pipelined for the specific node (in cluster)

Pipeline pipeline = jedis.pipelined();
pipeline.multi();
for(Entry<String, Map<String, String>> kvf : kvfs.entrySet()) {
   pipeline.hmset(kvf.getKey(), kvf.getValue());
}
pipeline.exec();

尽管这种方法(使用JedisCluster)提供了适当的节点，但密钥无法到达该节点，因此无法提供预期的性能，我认为这是由于了解插槽号和(插槽的)节点所涉及的过程所致.
每次我们尝试获取包含插槽号的实际节点(jedis)时，上述过程似乎都建立了到该节点(群集中)的物理连接.因此，如果我们拥有数百万个键，这会阻碍性能.
因此，使用Lettuce软件包的另一种方法(下文)帮助我克服了这一点.

Despite this approach (with JedisCluster) gave the appropriate node to which the keys go to this didn't provide me the expected performance, I think it's due to the procedure involved in knowing slot number and node (of the slot).
Above procedure seems to establish a physical connection to the node (in cluster) every time we try to get the actual node (jedis) that contains the slot number. So, this hinders the performance in-case we have millions of keys.
So, another approach (below) using Lettuce package helped me to over come this.

解决方案2:
使用过的Lettuce软件包，它支持以群集模式发送批量命令.

Solution 2:
Used Lettuce package that supports sending batch of commands in cluster mode.

     <groupId>biz.paluch.redis</groupId>
     <artifactId>lettuce</artifactId>
     <version>4.4.3.Final</version>

代码段:

RedisClusterClient client = RedisClusterClient.create(RedisURI.create("hostname", "port"));
StatefulRedisClusterConnection<String, String> connection = client.connect();

RedisAdvancedClusterAsyncCommands<String, String> commands = connection.async();
// Disabling auto-flushing
commands.setAutoFlushCommands(false);

List<RedisFuture<?>> futures = new ArrayList<>();
// kvf is of type Map<String, Map<String, String>>
for (Entry<> e : kvf.entrySet())
{
   futures.add(commands.hmset( (String) e.getKey(), (Map<String, String>) e.getValue()));
}
// write all commands to the transport layer
commands.flushCommands();
// synchronization example: Wait until all futures complete
LettuceFutures.awaitAll(10, TimeUnit.SECONDS,
futures.toArray(new RedisFuture[futures.size()]));

引用: https://github.com/lettuce-io/lettuce-core/wiki/管道和命令冲洗

这篇关于使用Jedis如何写入Redis集群中的特定插槽/节点的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！