Redis hash 写入速度很慢 [英] Redis hash very slow writing speed

问题描述

我面临一个非常奇怪的问题:使用 redis 时我的写入速度非常糟糕(在理想情况下，写入速度应该接近 RAM 上的写入速度).

I'm facing a very strange issue : i get really crappy writing speeds when using redis (in a ideal world the writing speed should be approaching the writing speed on RAM).

这是我的基准:

package redisbenchmark;
import redis.clients.jedis.Jedis;

public class RedisBenchmark {

    private static final String REDIS_KEY = "anon_id";

    private Jedis conn;

    private long writeTimeNano=0;

    private RandomString stringGenerator;

    private String[] fields;

    public RedisBenchmark(){
        conn = new Jedis("localhost");
        stringGenerator = new RandomString(32);
    }

    public void run(int nbWrites, int nbReads){     
        writeBenchmark(nbWrites);
    }

    public void writeBenchmark(int amount){
        fields = new String[amount];

        for(int i=0; i< amount; i++){
            fields[i] = stringGenerator.nextString();           
        }

        long start = System.nanoTime();
        for(int i=0; i< amount; i++){
            write(fields[i]);
        }
        writeTimeNano+=System.nanoTime()-start;

        double seconds = (double)writeTimeNano / 1000000000.0;
        System.out.println("[write]nb:"+amount+"|time:"+seconds+"|speed:"+((amount*33)/(seconds*1024*1024))+" MB/s");
    }

    public void write(String anonId){       
        conn.hsetnx(REDIS_KEY, anonId, "1");
    }


    /**
     * @param args the command line arguments
     */
    public static void main(String[] args) {
        // TODO code application logic here
        RedisBenchmark benchmark = new RedisBenchmark();
        benchmark.run(100000, 200);
    }
}

RandomString 是一个生成随机字符串的类(arg 为字符串长度)

RandomString is a class that generates a random string (the arg is the string length)

下面是几个结果:

[write]nb:100000|time:4.408319378|speed:0.713905907055318 MB/s[write]nb:100000|time:4.447246995|speed:0.707656949946542 MB/s

[write]nb:100000|time:4.408319378|speed:0.713905907055318 MB/s [write]nb:100000|time:4.447246995|speed:0.707656949946542 MB/s

我尝试修改配置文件中的save to hdd参数，但没有改进.

I tried to modify the save to hdd parameters in the config file but with no improvement.

我有两个想法:
1.这是一个socket问题，因为客户端和服务器(redis)在同一台机器上
2.连接器实现存在性能问题

I have 2 ideas:
1. Its a socket problem since client and server (redis) are on the same machine
2. The connector implementation has performance issues

UPDATE集合操作的基准结果:

====== 设置 ======
10000 个请求在 0.09 秒内完成
50 个并行客户端
3 字节有效载荷
保持活力:1

====== SET ======
10000 requests completed in 0.09 seconds
50 parallel clients
3 bytes payload
keep alive: 1

99.51% <= 1 毫秒
100.00% <= 1 毫秒
每秒 111111.11 个请求

99.51% <= 1 milliseconds
100.00% <= 1 milliseconds
111111.11 requests per second

系统规格:
- Ubuntu 11.04
- 8GB 内存
- 英特尔 i5 处理器

System specification :
- Ubuntu 11.04
- 8GB RAM
- Intel i5 processor

任何建议将不胜感激.

推荐答案

您需要更多地考虑使用此程序的真正基准测试.我可以告诉你，它不是 Redis，而是你的系统在两个进程之间运行乒乓球游戏的能力(因为你所有的 hsetnx 调用都是同步的).

You need to think a bit more about what you really benchmark using this program. I can tell you it is not Redis, but rather the capability of your system to run a ping pong game between two processes (because all your hsetnx calls are synchronous).

在尝试对 Redis 进行基准测试之前，请阅读此页面，它一定会对您有所帮助.

Please read this page before trying to benchmark Redis, it will definitely help you.

你认为Redis的速度应该接近RAM的写入速度有点幼稚.Redis 是一个远程存储，对于 O(1) 操作，大部分开销是由于通信成本.对于同步流量(如您的示例)，这也是由于操作系统调度程序的成本.

Your assumption that the speed of Redis should approach the writing speed of RAM is somewhat naive. Redis is a remote store, and for O(1) operations, most of the overhead is due to the communication costs. For synchronous traffic (like your example), it is also due to the cost of the OS scheduler.

如果你想依次应用很多命令，你需要使用pipelining.或者如果你不关心顺序，你可以同时使用多个连接(这是redis-benchmark的默认模式).或者您可以尝试发送异步命令.在所有情况下，想法都是分摊到 Redis 服务器的往返成本

If you want to apply of lot of commands in sequence, you need to use pipelining. Or if you do not care about the sequence, you can work concurrently with several connections (this is the default mode for redis-benchmark). Or you can try to send asynchronous commands instead. In all cases, the idea is to amortize the cost of the roundtrips to the Redis server

通过对具有异步流量的多个连接进行流水线处理，您将获得 Redis 在这台机器上可以实现的最大吞吐量.

With pipelining on several connections with asynchronous traffic, you will get the maximum throughput Redis can achieve on this machine.

这篇关于Redis hash 写入速度很慢的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！