从一个进程并行读取n个管道 [英] Read from n pipes from one process in parallel

问题描述

当由多个进程同时写入使用 mkfifo 创建的同一个命名管道时，我遇到了一个并发问题，其中有些写入丢失了.由于写入过程的数目是有限的，所以我想从从n个过程写入到1个管道并从1个单独的读取中切换".通过n个进程写入n个管道并从1个单独的进程读取".

I faced a concurrency problem when writing to the same named pipe created with mkfifo by multiple processes at the same time, where some writes got lost. Since the number of writing processes are limited I want to switch from "writing to 1 pipe from n processes and reading from 1 separate" to "writing to n pipes by n processes and reading from 1 separate process".

当前，我正在循环中通过读取行<"$ pipe" 进行读取，直到满足条件为止. read 会在此处阻止，直到读取一行为止.

Currently I'm reading via read line <"$pipe" in a loop until a condition is met. read blocks here until a line was read.

我如何通过一个循环从多个管道($ pipe1，$ pipe2…$ pipeN)中读取直到满足条件，同时在所有管道上都使用新写入的行呢?

How can I read from multiples pipes ($pipe1, $pipe2 … $pipeN) via one loop until a condition is met, while honouring newly written lines on all pipes the same?

推荐答案

一种解决最初描述的多个子写入单个FIFO问题的方法是让进程打开FIFO进行读取，但从不实际读取.除非FIFO已满，否则这将允许写程序进行写操作.我认为没有一个标准程序会一直持续睡眠直到发出信号.我使用自制程序 pause ，这是一个非常小的C程序:

One way to deal with the initially described problem of multiple children writing to a single FIFO is to have a process open the FIFO for reading but never actually read it. This will allow writers to write unless the FIFO is full. I don't think there's a standard program that simply goes to sleep forever until signalled. I use a home-brew program pause, which is a pretty minimal C program:

#include <unistd.h>

int main(void)
{
    pause();
}

它直到发出信号才退出.在您的Shell脚本中，首先启动子级，告诉它们写入 $ FIFO ，然后运行:

It never exits until signalled. In your shell script, first launch the children, telling them to write to $FIFO, then run:

pause <$FIFO &
pid=$!

请注意，直到重定向完成，类似 pause 的命令才会在后台启动，并且直到有写入FIFO的进程时，FIFO的打开才会完成-因此在执行类似 pause 的过程之前，至少需要在后台启动一个孩子.或者编写 pause (我叫我的 sleepon )的变体，它会打开其参数列表中命名的文件-然后命令行类似于 sleepon $ FIFO& ，然后完成后台操作，并阻塞类似 pause 的程序，直到它能够打开FIFO(这将是其中一个子进程打开FIFO进行写入的时间)，然后无限期地睡觉.但是 sleepon 的代码比 pause 的代码复杂得多.

Note that pause-like command will not be launched into the background until the redirection completes, and the open of the FIFO won't complete until there is a process to write to the FIFO — so at least one child needs to be launched in background before the pause-like process is executed. Or write a variant of pause (I call mine sleepon) which opens the files named in its argument list — then the command line is similar to sleepon $FIFO & and the backgrounding operation completes and the pause-like program blocks until it is able to open the FIFO (which will be when one of the children opens the FIFO for writing), and then goes to sleep indefinitely. But the code for sleepon is a lot more complex than the code for pause.

一旦启动子进程和类似 pause 的进程，父进程就可以继续执行主处理循环.

Once the children and the pause-like process are launched, the parent can continue with the main processing loop.

while read line
do
    …
done < $FIFO

主要要注意的是，每当FIFO清空时，父循环就会退出.您需要知道何时终止(如果有的话).在终止点，它应该终止 pause 进程: kill $ pid .您可能需要在true时包装；围绕行读取循环执行…完成循环-但您可能需要比这更聪明的东西.这部分取决于您的直到满足条件"的条件.要求是.

The main thing to be aware of is that the parent loop will exit whenever the FIFO is emptied. You need to know when it should terminate, if ever. At the point where it does terminate, it should kill the pause process: kill $pid. You may need to wrap a while true; do … done loop around the line-reading loop — but you may need something cleverer than that. It depends, in part, on what your "until a condition is met" requirement is.

从多个FIFO读取，所有FIFO可能都间歇性地有数据"的要求并不容易做到.在C语言中并不是特别琐碎；我认为没有标准的(POSIX)Shell命令可以帮助您解决此问题.在C语言中，您最终将使用POSIX select() 或 poll() 或其中之一许多变体-其中一些是特定于平台的.可能会有特定于平台的命令会有所帮助；不过，我有疑问.

Your requirement to 'read from multiple FIFOs, all of which may intermittently have data on them' is not easy to do. It's not particularly trivial in C; I don't think there's a standard (POSIX) shell command to assist with that. In C, you'd end up using POSIX select() or poll() or one of their many variants — some of which are platform-specific. There might be a platform-specific command that will help; I have my doubts, though.

这篇关于从一个进程并行读取n个管道的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！