如何运行与多内核的bash管道输送过程？ [英] How to run processes piped with bash on multiple cores?

问题描述

我有一个简单的bash脚本一个进程到另一个管道输出。即：

I have a simple bash script that pipes output of one process to another. Namely:.

dostuff | filterstuff

碰巧我的Linux系统上（openSUSE的，如果它事项，内核2.6.27），这些过程都在单核上运行。但是，运行不同的过程的在不同内核上是不会发生在这种情况下触发的默认策略。

It happens that on my Linux system (openSUSE if it matters, kernel 2.6.27) these both processes run on a single core. However, running different processes on different cores is a default policy that doesn't happen to trigger in this case.

什么组件系统的负责人认为，我应该怎样做才能利用多核功能？

What component of the system is responsible for that and what should I do to utilize multicore feature?

注意，有没有这样的问题在2.6.30内核中。

Note that there's no such problem on 2.6.30 kernel.

澄清：已经跟随丹尼斯·威廉姆森的建议，我做的确定的顶级项目，即管道输送过程确实的总是的同一个处理器上运行。 Linux调度，通常做了很好的工作，这个时候没有做。

Clarification: Having followed Dennis Williamson's advice, I made sure with top program, that piped processes are indeed always run on the same processor. Linux scheduler, which usually does a really good job, this time doesn't do it.

我想的东西在bash prevents OS从这样做。事情是，我需要的移动的两个多核心和单核心机解决方案。在使用taskset solution提出丹尼斯·威廉姆森不会在单核机工作。目前我使用：

I figure that something in bash prevents OS from doing it. The thing is that I need a portable solution for both multi-core and single-core machines. The taskset solution proposed by Dennis Williamson won't work on single-core machines. Currently I'm using:,

dostuff | taskset -c 0 filterstuff 

不过这似乎是一个肮脏的黑客。谁能提供更好的解决方案？

but this seems like a dirty hack. Could anyone provide a better solution?

推荐答案

假设 dostuff 是一个CPU上运行。其写入数据转换成一个管道，并且该数据将在该CPU上高速缓存。因为 filterstuff 从该管道读取，调度决定在​​同一个CPU上运行，所以它的输入数据已经在缓存中。

Suppose dostuff is running on one CPU. It writes data into a pipe, and that data will be in cache on that CPU. Because filterstuff is reading from that pipe, the scheduler decides to run it on the same CPU, so that its input data is already in cache.

如果你的内核是建立与 CONFIG_SCHED_DEBUG = Y ，

If your kernel is built with CONFIG_SCHED_DEBUG=y,

# echo NO_SYNC_WAKEUPS > /sys/kernel/debug/sched_features

应该禁用此类启发式的。 （请参阅 /usr/src/linux/kernel/sched_features.h 和的/ proc / sys目录/内核/章附表_ * 对于其他调度可调参数。）

should disable this class of heuristics. (See /usr/src/linux/kernel/sched_features.h and /proc/sys/kernel/sched_* for other scheduler tunables.)

如果帮助，这个问题仍然有新的内核发生，和这真的更快地在不同的CPU比一个CPU上运行，请将此问题报告给Linux内核邮件列表，以便他们可以调整他们的启发。

If that helps, and the problem still happens with a newer kernel, and it's really faster to run on separate CPUs than one CPU, please report the problem to the Linux Kernel Mailing List so that they can adjust their heuristics.

这篇关于如何运行与多内核的bash管道输送过程？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！