有没有办法提高linux管道的性能? [英] Is there a way to improve performance of linux pipes?

问题描述

我正在尝试使用64位 CentOS a> 6.我已经使用dd完成了以下基准测试，发现管道阻碍了我，而不是程序中的算法.我的目标是达到1.5 GB/s左右的速度.

I'm trying to pipe extremely high speed data from one application to another using 64-bit CentOS6. I have done the following benchmarks using dd to discover that the pipes are holding me back and not the algorithm in my program. My goal is to achieve somewhere around 1.5 GB/s.

首先，不带管道:

dd if=/dev/zero of=/dev/null bs=8M count=1000
1000+0 records in
1000+0 records out
8388608000 bytes (8.4 GB) copied, 0.41925 s, 20.0 GB/s

接下来，两个dd进程之间的管道:

Next, a pipe between two dd processes:

dd if=/dev/zero bs=8M count=1000 | dd of=/dev/null bs=8M
1000+0 records in
1000+0 records out
8388608000 bytes (8.4 GB) copied, 9.39205 s, 893 MB/s

我是否可以对内核进行任何调整，或者可以改善通过管道运行数据的性能的其他任何调整?我也尝试过命名管道，并获得了相似的结果.

Are there any tweaks I can make to the kernel or anything else that will improve performance of running data through a pipe? I have tried named pipes as well, and gotten similar results.

推荐答案

您是否尝试过使用较小的块?

Have you tried with smaller blocks?

当我在自己的工作站上尝试时，我注意到在减小块大小时取得了持续的进步. 在我的测试中，它仅占10％的范围，但仍然是一个改进.您正在寻找100％.

When I try on my own workstation I note successive improvement when lowering the block size. It is only in the realm of 10% in my test, but still an improvement. You are looking for 100%.

事实证明，要进行进一步的测试，实际上很小的块大小似乎可以解决问题:

As it turns out testing further, really small block sizes seem to do the trick:

我尝试了

dd if=/dev/zero bs=32k count=256000 | dd of=/dev/null bs=32k
256000+0 records in
256000+0 records out
256000+0 records in
256000+0 records out
8388608000 bytes (8.4 GB) copied8388608000 bytes (8.4 GB) copied, 1.67965 s, 5.0 GB/s
, 1.68052 s, 5.0 GB/s

以及您的原始

dd if=/dev/zero bs=8M count=1000 | dd of=/dev/null bs=8M
1000+0 records in
1000+0 records out
1000+0 records in
1000+0 records out
8388608000 bytes (8.4 GB) copied8388608000 bytes (8.4 GB) copied, 6.25782 s, 1.3 GB/s
, 6.25203 s, 1.3 GB/s

5.0/1.3 = 3.8，所以这是一个相当大的因素.

5.0/1.3 = 3.8 so that is a sizable factor.

这篇关于有没有办法提高linux管道的性能?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！