使用>&gt ;?将几个并行进程的输出管道到一个文件是安全的 [英] Is it safe to pipe the output of several parallel processes to one file using >>?

问题描述

我从网上抓取数据，我有几个进程，我的刮刀并行运行。

I'm scraping data from the web, and I have several processes of my scraper running in parallel.

我想要每个进程的输出结束在同一个文件中。只要文本行保持完整并且不会相互混淆，行的顺序就没有关系。在UNIX中，我可以使用>>运算符将每个进程的输出传递到同一个文件。

I want the output of each of these processes to end up in the same file. As long as lines of text remain intact and don't get mixed up with each other, the order of the lines does not matter. In UNIX, can I just pipe the output of each process to the same file using the >> operator?

推荐答案

不能保证线条保持不变。

No. It is not guaranteed that lines will remain intact. They can become intermingled.

根据liori的答案搜索，我发现 this ：

From searching based on liori's answer I found this:

写入{PIPE_BUF}字节或更少的请求不得与来自其他进程在同一个管道上进行写操作。大于{PIPE_BUF}字节的写入可以在任意边界上与其他进程的写入交错，无论文件状态标志的O_NONBLOCK标志是否置位。

Write requests of {PIPE_BUF} bytes or less shall not be interleaved with data from other processes doing writes on the same pipe. Writes of greater than {PIPE_BUF} bytes may have data interleaved, on arbitrary boundaries, with writes by other processes, whether or not the O_NONBLOCK flag of the file status flags is set.

因此，不能保证比{PIPE_BUF}字节长的行保持不变。

So lines longer than {PIPE_BUF} bytes are not guaranteed to remain intact.

这篇关于使用>&gt ;?将几个并行进程的输出管道到一个文件是安全的的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！