使用os.pipe和os.fork()问题的Python程序 [英] Python program using os.pipe and os.fork() issue

问题描述

我最近需要编写一个脚本，该脚本执行 os.fork()来拆分为两个进程.子进程成为服务器进程，并使用通过 os.pipe()创建的管道将数据传递回父进程.照常，子级关闭管道的'r'端，父级关闭管道的'w'端.我使用 os.fdopen 将pipe()的返回值转换为文件对象.

I've recently needed to write a script that performs an os.fork() to split into two processes. The child process becomes a server process and passes data back to the parent process using a pipe created with os.pipe(). The child closes the 'r' end of the pipe and the parent closes the 'w' end of the pipe, as usual. I convert the returns from pipe() into file objects with os.fdopen.

我遇到的问题是:进程成功分叉，并且子进程成为服务器.一切都很好，孩子会尽职地将数据写入管道的开放'w'端.不幸的是，管道的父端有两个奇怪的事情:
A)它阻塞了管道'r'端上的read()操作.
其次，除非'w'端完全关闭，否则它无法读取管道上放置的任何数据.

The problem I'm having is this: The process successfully forks, and the child becomes a server. Everything works great and the child dutifully writes data to the open 'w' end of the pipe. Unfortunately the parent end of the pipe does two strange things:
A) It blocks on the read() operation on the 'r' end of the pipe.
Secondly, it fails to read any data that was put on the pipe unless the 'w' end is entirely closed.

我立即认为缓冲是问题所在，并添加了 pipe.flush()调用，但这些操作无济于事.

I immediately thought that buffering was the problem and added pipe.flush() calls, but these didn't help.

有人能阐明为什么直到写入端完全关闭才出现数据的原因吗?有没有使read()通话不阻塞的策略?

Can anyone shed some light on why the data doesn't appear until the writing end is fully closed? And is there a strategy to make the read() call non blocking?

这是我第一个使用派生或使用管道的Python程序，如果我犯了一个简单的错误，请原谅我.

This is my first Python program that forked or used pipes, so forgive me if I've made a simple mistake.

推荐答案

您是否使用read()而不指定大小，或将管道视为迭代器(for line in f)?如果是这样，那可能是您问题的根源-read()被定义为在返回之前读取到文件末尾，而不是仅读取可读取的内容.那将意味着它将阻塞，直到子代调用close().

Are you using read() without specifying a size, or treating the pipe as an iterator (for line in f)? If so, that's probably the source of your problem - read() is defined to read until the end of the file before returning, rather than just read what is available for reading. That will mean it will block until the child calls close().

在链接的示例代码中，这没关系-父级以阻塞方式运行，并且仅将子级用于隔离目的.如果要继续，则可以按照发布的代码使用非阻塞IO(但准备处理半完成的数据)，或者分块读取(例如r.read(size)或r.readline() )将仅在读取特定大小/行之前阻塞. (您仍然需要在孩子身上打电话给同花顺)

In the example code linked to, this is OK - the parent is acting in a blocking manner, and just using the child for isolation purposes. If you want to continue, then either use non-blocking IO as in the code you posted (but be prepared to deal with half-complete data), or read in chunks (eg r.read(size) or r.readline()) which will block only until a specific size / line has been read. (you'll still need to call flush on the child)

似乎将管道视为迭代器也正在使用其他缓冲区，因为如果需要立即使用每行，则"for line in r:"可能无法满足您的需求.可能可以禁用此功能，但仅在fdopen中将缓冲区大小指定为0似乎还不够.

It looks like treating the pipe as an iterator is using some further buffer as well, for "for line in r:" may not give you what you want if you need each line to be immediately consumed. It may be possible to disable this, but just specifying 0 for the buffer size in fdopen doesn't seem sufficient.

这里有一些示例代码应该起作用:

Heres some sample code that should work:

import os, sys, time

r,w=os.pipe()
r,w=os.fdopen(r,'r',0), os.fdopen(w,'w',0)

pid = os.fork()
if pid:          # Parent
    w.close()
    while 1:
        data=r.readline()
        if not data: break
        print "parent read: " + data.strip()
else:           # Child
    r.close()
    for i in range(10):
        print >>w, "line %s" % i
        w.flush()
        time.sleep(1)

这篇关于使用os.pipe和os.fork()问题的Python程序的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！