Redis如何实现高吞吐量和性能？ [英] How does Redis achieve the high throughput and performance?

问题描述

我知道这是一个非常一般的问题。但是，我想了解什么是允许Redis（或者Memcached，Cassandra等缓存）在惊人的性能限制下工作的主要架构决定。

< 如何维护连接？

1. 连接是TCP还是HTTP？


2. 我知道它是完全用C写的。 ？


3. 用于实现高吞吐量的同步技术有哪些？b）
   
   




基本上，在内存缓存和服务器上可以响应命令的机器和Redis框的纯粹实现之间的区别是什么？我也明白，答案需要非常巨大，应该包括非常复杂的细节完成。但是，我要找的是一些常用的技术，而不是所有的细微差别。

解决方案

在Redis文档中了解它是如何工作的。现在，请具体回答您的问题：





1）如何维护连接？





使用ae事件循环（由Redis作者设计）维护和管理连接。所有网络I / O操作都是非阻塞的。你可以看到ae作为一个简单的实现使用平台的最好的网络I / O多路分解机制（epoll for Linux，kqueue for BSD，etc ...）就像libevent，libev，libuv等...





2）是连接TCP还是HTTP？





连接是使用Redis协议的TCP是一种简单的telnet兼容，面向文本的协议，支持二进制数据。这个协议通常比HTTP更有效。





3）如何管理内存？ b

内存是通过依靠通用内存分配器来管理的。在某些平台上，这实际上是系统内存分配器。在其他平台（包括Linux）上，jemalloc已经被选中，因为它提供了CPU消耗，并发支持，碎片和内存占用之间的良好平衡。 jemalloc源代码是Redis发行版的一部分。





与其他产品（如memcached）相反，Redis中没有实现slab分配器。





许多优化的数据结构已在通用分配器顶部以减少内存占用。





4）什么是同步技术用于实现高吞吐量竞争读/写？ / strong>





Redis是一个单线程事件循环，所以不需要同步，因为所有命令都是序列化的。现在，一些线程也在后台运行以用于内部目的。在极少数情况下，它们访问由主线程管理的数据，使用经典的pthread同步原语（例如互斥体）。但是代表多个客户端连接进行的数据访问的100％不需要任何同步。





您可以在这里找到更多信息：
Redis是单线程的，那么它是如何并发I / O？





在内存缓存和服务器的机器的简单的实现之间有什么区别，可以响应命令和Redis框？





没有区别。 Redis是一个在内存缓存和服务器上可以响应命令的机器的简单的实现。但它是一个正确的实现：






• 使用单线程事件循环模型


• 使用为其相应用例优化的简单和简约的数据结构


• 提供一组仔细选择的命令以平衡简约和实用性


• 原始性能


• 很好地适应现代操作系统机制


• 提供多个持久机制，因为一个大小适合所有方法只是一个梦想。 / li>
  
• 提供HA机制（例如复制系统）的构建块


• 避免堆叠无用的抽象层，如煎饼


• 产生一个干净，易于理解的代码库，任何好的C开发人员都可以轻松使用





I know this is a very generic question. But, I wanted to understand what are the major architectural decision that allow Redis (or caches like MemCached, Cassandra) to work at amazing performance limits.

1. How are connections maintained?
2. Are connections TCP or HTTP?
3. I know that it is completely written in C. How is the memory managed?
4. What are the synchronization techniques used to achieve high throughput inspite of competing read/writes?

Basically, what is the difference between a plain vanilla implementation of a machine with in memory cache and server that can respond to commands and a Redis box? I also understand that the answer needs to be very huge and should include very complex details for completion. But, what I'm looking for are some general techniques used rather than all nuances.

解决方案

There is a wealth of of information in the Redis documentation to understand how it works. Now, to answer specifically your questions:

1) How are connections maintained?

Connections are maintained and managed using the ae event loop (designed by the Redis author). All network I/O operations are non blocking. You can see ae as a minimalistic implementation using the best network I/O demultiplexing mechanism of the platform (epoll for Linux, kqueue for BSD, etc ...) just like libevent, libev, libuv, etc ...

2) Are connections TCP or HTTP?

Connections are TCP using the Redis protocol, which is a simple telnet compatible, text oriented protocol supporting binary data. This protocol is typically more efficient than HTTP.

3) How is the memory managed?

Memory is managed by relying on a general purpose memory allocator. On some platforms, this is actually the system memory allocator. On some other platforms (including Linux), jemalloc has been selected since it offers a good balance between CPU consumption, concurrency support, fragmentation and memory footprint. jemalloc source code is part of the Redis distribution.

Contrary to other products (such as memcached), there is no implementation of a slab allocator in Redis.

A number of optimized data structures have been implemented on top of the general purpose allocator to reduce the memory footprint.

4) What are the synchronization techniques used to achieve high throughput inspite of competing read/writes?

Redis is a single-threaded event loop, so there is no synchronization to be done since all commands are serialized. Now, some threads also run in the background for internal purposes. In the rare cases they access the data managed by the main thread, classical pthread synchronization primitives are used (mutexes for instance). But 100% of the data accesses made on behalf of multiple client connections do not require any synchronization.

You can find more information there: Redis is single-threaded, then how does it do concurrent I/O?

What is the difference between a plain vanilla implementation of a machine with in memory cache and server that can respond to commands and a Redis box?

There is no difference. Redis is a plain vanilla implementation of a machine with in memory cache and server that can respond to commands. But it is an implementation which is done right:

• using the single threaded event loop model
• using simple and minimalistic data structures optimized for their corresponding use cases
• offering a set of commands carefully chosen to balance minimalism and usefulness
• constantly targeting the best raw performance
• well adapted to modern OS mechanisms
• providing multiple persistence mechanisms because the "one size does fit all" approach is only a dream.
• providing the building blocks for HA mechanisms (replication system for instance)
• avoiding stacking up useless abstraction layers like pancakes
• resulting in a clean and understandable code base that any good C developer can be comfortable with

这篇关于Redis如何实现高吞吐量和性能？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！