为什么卡夫卡这么快 [英] Why Kafka so fast

问题描述

如果我具有相同的硬件，请使用Kafka或我们当前的解决方案(ServiceMix/Camel).有什么区别吗? Kafka可以处理比它更大的数据吗?为什么?

If I have same hardware, to use Kafka or our current solution(ServiceMix/Camel). Is there any difference? Can Kafka handle "bigger" data than it? Why?

有一篇文章谈论它有多快?但是我仍然不清楚为什么Kafka与其他解决方案相比如此之快? 基准化Apache Kafka:200万次写入每秒(在三台便宜的机器上)

There is a article to talk about how fast could it be? But I still don't get clearly why Kafka is so fast comparing to other solutions? Benchmarking Apache Kafka: 2 Million Writes Per Second (On Three Cheap Machines)

推荐答案

Kafka快速运行有很多原因.仅举几例.

Kafka is fast for a number of reasons. To name a few.

• 零复制-请参见 https://en.wikipedia.org/wiki/零复制基本上可以 直接调用OS内核而不是在应用程序层移动 数据快速.
• 批量存储数据-Kafka就是批量处理 数据成块.这样可以最大程度地减少跨机器的延迟 随之而来的缓冲/复制.
• 避免随机磁盘访问-由于Kafka是不可变的提交日志，因此它无需回退磁盘并执行许多随机I/O操作，而只需按顺序访问磁盘即可.这样一来，它就可以从 物理磁盘与内存的比较.
• 可以水平缩放- 单个主题具有数千个分区的能力 在数千台机器中意味着Kafka可以处理巨大的负载.

• Zero Copy - See https://en.wikipedia.org/wiki/Zero-copy basically it calls the OS kernal direct rather than at the application layer to move data fast.
• Batch Data in Chunks - Kafka is all about batching the data into chunks. This minimises cross machine latency with all the buffering/copying that accompanies this.
• Avoids Random Disk Access - as Kafka is an immutable commit log it does not need to rewind the disk and do many random I/O operations and can just access the disk in a sequential manner. This enables it to get similar speeds from a physical disk compared with memory.
• Can Scale Horizontally - The ability to have thousands of partitions for a single topic spread among thousands of machines means Kafka can handle huge loads.

这篇关于为什么卡夫卡这么快的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！