在C中实现流水线.做到这一点的最佳方法是什么? [英] Implementing pipelining in C. What would be the best way to do that?
问题描述
我想不出任何可行的方法来在c中实现流水线工作.这就是为什么我决定在这里写的原因.我必须说，我了解管道/叉子/mkfifo的工作方式.我已经看到了很多实施2-3条管道的示例.这简单.当我必须实现Shell时，我的问题就开始了，而管道计数是未知的.
I can't think of any way to implement pipelining in c that would actually work. That's why I've decided to write in here. I have to say, that I understand how do pipe/fork/mkfifo work. I've seen plenty examples of implementing 2-3 pipelines. It's easy. My problem starts, when I've got to implement shell, and pipelines count is unknown.
我现在所拥有的: 例如.
What I've got now: eg.
ls -al | tr a-z A-Z | tr A-Z a-z | tr a-z A-Z
我将这样的行转换为类似的内容:
I transform such line into something like that:
array[0] = {"ls", "-al", NULL"}
array[1] = {"tr", "a-z", "A-Z", NULL"}
array[2] = {"tr", "A-Z", "a-z", NULL"}
array[3] = {"tr", "a-z", "A-Z", NULL"}
这样我就可以使用
execvp(array[0],array)
稍后.
现在，我相信一切都很好.问题开始了，当我试图将输入/输出的功能重定向到彼此时.
Untli now, I believe everything is OK. Problem starts, when I'm trying to redirect those functions input/output to eachother.
这就是我的做法:
mkfifo("queue", 0777);
for (i = 0; i<= pipelines_count; i++) // eg. if there's 3 pipelines, there's 4 functions to execvp
{
int b = fork();
if (b == 0) // child
{
int c = fork();
if (c == 0)
// baby (younger than child)
// I use c process, to unblock desc_read and desc_writ for b process only
// nothing executes in here
{
if (i == 0) // 1st pipeline
{
int desc_read = open("queue", O_RDONLY);
// dup2 here, so after closing there's still something that can read from
// from desc_read
dup2(desc_read, 0);
close(desc_read);
}
if (i == pipelines_count) // last pipeline
{
int desc_write = open("queue", O_WRONLY);
dup2(desc_write, 0);
close(desc_write);
}
if (i > 0 && i < pipelines_count) // pipeline somewhere inside
{
int desc_read = open("queue", O_RDONLY);
int desc_write = open("queue", O_WRONLY);
dup2(desc_write, 1);
dup2(desc_read, 0);
close(desc_write);
close(desc_read);
}
exit(0); // closing every connection between process c and pipeline
}
else
// b process here
// in b process, i execvp commands
{
if (i == 0) // 1st pipeline (changing stdout only)
{
int desc_write = open("queue", O_WRONLY);
dup2(desc_write, 1); // changing stdout -> pdesc[1]
close(desc_write);
}
if (i == pipelines_count) // last pipeline (changing stdin only)
{
int desc_read = open("queue", O_RDONLY);
dup2(desc_read, 0); // changing stdin -> pdesc[0]
close(desc_read);
}
if (i > 0 && i < pipelines_count) // pipeline somewhere inside
{
int desc_write = open("queue", O_WRONLY);
dup2(desc_write, 1); // changing stdout -> pdesc[1]
int desc_read = open("queue", O_RDONLY);
dup2(desc_read, 0); // changing stdin -> pdesc[0]
close(desc_write);
close(desc_read);
}
wait(NULL); // it wait's until, process c is death
execvp(array[0],array);
}
}
else // parent (waits for 1 sub command to be finished)
{
wait(NULL);
}
}
谢谢.
推荐答案
帕特里克(Patryk)，为什么要使用fifo，而且在管道的每个阶段都使用相同的fifo?
Patryk, why are you using a fifo, and moreover the same fifo for each stage of the pipeline?
在我看来，每个阶段之间都需要一个管道.因此流程将类似于:
It seems to me that you need a pipe between each stage. So the flow would be something like:
Shell ls tr tr
----- ---- ---- ----
pipe(fds);
fork();
close(fds[0]); close(fds[1]);
dup2(fds[0],0);
pipe(fds);
fork();
close(fds[0]); close(fds[1]);
dup2(fds[1],1); dup2(fds[0],0);
exex(...); pipe(fds);
fork();
close(fds[0]); etc
dup2(fds[1],1);
exex(...);
在每个叉形外壳(close，dup2，pipe等)中运行的序列看起来像一个函数(带有所需进程的名称和参数).请注意，直到每个exec调用之前，shell的分叉副本都在运行.
The sequence that runs in each forked shell (close, dup2, pipe etc) would seem like a function (taking the name and parameters of the desired process). Note that up until the exec call in each, a forked copy of the shell is running.
帕特里克:
Also, is my thinking correct? Shall it work like that? (pseudocode):
start_fork(ls) -> end_fork(ls) -> start_fork(tr) -> end_fork(tr) ->
start_fork(tr) -> end_fork(tr)
我不确定start_fork和end_fork是什么意思.您是否暗示ls在tr开始之前运行完成?这并不是上面的图真正的意思.您的外壳将不会等待ls完成，然后再启动tr.它按顺序启动管道中的所有进程，为每个进程分别设置stdin和stdout，以使这些进程链接在一起，从ls的stdout到tr的stdin； tr中的stdout到下一个tr中的stdin.这就是dup2调用正在执行的操作.
I'm not sure what you mean by start_fork and end_fork. Are you implying that ls runs to completion before tr starts? This isn't really what is meant by the diagram above. Your shell will not wait for ls to complete before starting tr. It starts all of the processes in the pipe in sequence, setting up stdin and stdout for each one so that the processes are linked together, stdout of ls to stdin of tr; stdout of tr to stdin of the next tr. That is what the dup2 calls are doing.
进程的运行顺序由操作系统(调度程序)确定，但是很明显，如果tr运行并从空的stdin中读取，它必须等待(阻塞)，直到前一个进程写入内容为止到管道. ls很有可能在tr甚至从其stdin读取之前运行完成，但同样有可能不会.例如，如果链中的第一个命令是连续运行的并一直沿途产生输出，那么无论第一个命令沿管道发送什么，管道中的第二个命令都会不定期地进行调度以处理.
The order in which the processes run is determined by the operating system (the scheduler), but clearly if tr runs and reads from an empty stdin it has to wait (to block) until the preceding process writes something to the pipe. It is quite possible that ls might run to completion before tr even reads from its stdin, but it is equally possible that it wont. For example if the first command in the chain was something that ran continually and produced output along the way, the second in the pipeline will get scheduled from time to time to prcess whatever the first sends along the pipe.
希望可以使事情澄清一些:-)
Hope that clarifies things a little :-)
这篇关于在C中实现流水线.做到这一点的最佳方法是什么?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
