保持FIFO在不同执行中可读 [英] Maintaining a FIFO readable across different executions

问题描述

我从来没有使用命名管道，最近意识到这只是我需要的。

I've never used a named pipe before and recently realized that is just what I need.

我运行一个程序使用gnu parallel，可以产生吨（GB到1TB，现在很难知道）为mySQL数据库格式化的输出。

I'm running a program using gnu parallel which could produce tons (GB's to 1TB, hard to know right now) of output formatted for a data base on mySQL.

我想出我可以打开两个终端：
终端1获得类似：

I figured out that I can open two terminals: Terminal 1 gets something like:

find . -type f -name "*.h" | parallel --jobs 12 'cprogram {}' > /home/pipe

其中pipe是由 mkfifo 。

在第二个终端上，我运行类似以下命令：

On a second terminal, I run a command similar to this:

mysql DataBaseName -e "LOAD DATA LOCAL INFILE '/home/pipe' INTO TABLE tableName";

它工作...

这是janky ...如果我理解正确，有一个EOF生成时，第一个进程结束导致管道关闭。

But this is janky...If I understand correctly, there's an EOF generated when the first process ends causing the pipe to close.

理想情况下，我想使用不同的参数在一个循环中运行第一个进程。每次迭代可能需要很长时间，我需要进行健全检查，所以我不会松动一个星期，以发现我有错误或错误的逻辑。

Ideally I want to run the first process in a loop with varying parameters. Each iteration could take a long time and I need to make sanity checks so I don't loose a week to find out I've got bugs or faulty logic.

我

推荐答案

如果我理解正确，当第一个
进程结束导致管道关闭时产生一个EOF。

If I understand correctly, there's an EOF generated when the first process ends causing the pipe to close.

排序的。有一点更多比它 - 在技术上不正确的说，管道关闭，只要第一个过程结束。

Sort of. There's a little bit more to it than that - it is technically incorrect to say that the pipe closes as soon as the first process ends.

如果管道中没有更多数据，并且未被任何进程写入，管道和FIFO返回EOF。

Instead, pipes and FIFOs return EOF when there is no more data left in the pipe and it is not opened for writing by any process.

通常，这是通过读取器进程打开用于​​读取和写入的FIFO来解决的，即使它不会写入 - 例如，通过从FIFO读取接受本地客户端的服务器可以打开FIFO进行读写，以便在没有活动客户端时，服务器不必处理EOF的特殊情况。这是处理它的标准方法，如在UNIX环境中的高级编程在IPC机制一章中所述。

Usually, this is solved by having the reader process open the FIFO both for reading and for writing, even though it will never write - for example, a server that accepts local clients by reading from a FIFO could open the FIFO for reading and writing so that when there are no active clients the server doesn't have to deal with the special case of EOF. This is the "standard" way to deal with it, as outlined in Advanced Programming in the UNIX Environment in the chapter about IPC mechanisms.

在你的情况下，这是真的不可能，因为你没有永久的进程持续运行（也就是说，你没有相当于一个服务器进程）。你基本上需要某种持久性写入器，即在不同迭代期间保持管道打开以进行写入的过程。

In your case though, this is really not possible, because you have no permanent process that keeps running (that is, you don't have the equivalent of a server process). You basically need some sort of "persistent writer", i.e., a process that maintains the pipe opened for writing during the different iterations.

我可以想到的一个解决方案是 cat 标准输入到后台的FIFO。这确保了 cat 打开了写入的FIFO，所以总是有一个活动的写入器，但是通过保持它在后台，你实际上不馈送任何输入从不写入FIFO。请注意，只要 cat 尝试从 stdin （在后台进程组中运行的进程通常发送SIGTTIN并在尝试从 stdin 读取时停止，因为它们没有控制终端到前台）。无论如何，只要你不喂它任何输入，你是好的 - 进程处于停止状态，但FIFO仍然打开写作。

One solution I can think of is to cat standard input to the FIFO in the background. This ensures that cat opens the FIFO for writing, so there is always an active writer, but by keeping it in the background, you don't actually feed it any input and it never writes to the FIFO. Just be aware that the job will be stopped (but not terminated) by the shell as soon as cat attempts to read from stdin (processes running in a background process group are usually sent SIGTTIN and stopped when they attempt to read from stdin, because they don't have a controlling terminal until they are brought to the foreground). Anyway, as long as you don't feed it any input, you're good - the process is in a stopped state, but the FIFO is still opened for writing nonetheless. You'll never see an EOF on the pipe as long as the background job is not terminated.

总之，你：

1. 创建FIFO： mkfifo / home / pipe


2. 启动背景作业打开FIFO写入： cat> / home / pipe&


3. 运行你想要的程序，许多迭代你想要的。忽略有关后台作业停止的shell消息。


4. 当你完成后，杀死背景 cat 通过将其带到前台并发送SIGINT（通常为Ctrl + C）或使用 kill PID 。

1. Create the FIFO: mkfifo /home/pipe
2. Start a background job that opens the FIFO for writing: cat >/home/pipe &
3. Run your programs however you want, with how many iterations you want. Ignore the shell message about the background job being stopped. You can just leave it like that, since the pipe is still opened for writing even though the job is stopped.
4. When you're done, kill the background cat by either bringing it to the foreground and sending it SIGINT (usually, Ctrl+C) or with kill PID.

注意，通过这样做，reader进程（在这种情况下为mysql）永远不会知道输入何时结束。它将总是阻塞更多的输入，除非你在杀死mysql之前杀死后台 cat 。

Note that by doing this the reader process (mysql in this case) will never know when the input is over. It will always block for more input, unless you kill the background cat before killing mysql.

这篇关于保持FIFO在不同执行中可读的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！