包装子进程的stdout/stderr [英] Wrap subprocess' stdout/stderr

问题描述

我想捕获并显示我通过Python的子进程调用的进程的输出.

我以为我可以像命名参数stdout和stderr一样传递文件状对象

我可以看到它访问了fileno属性-因此它正在处理该对象. 但是，永远不会调用write()方法.是我的方法完全消失了，还是我只是缺少一些东西?

class Process(object):
    class StreamWrapper(object):
        def __init__(self, stream):
            self._stream = stream
            self._buffer = []
        def _print(self, msg):
            print repr(self), msg
        def __getattr__(self, name):
            if not name in ['fileno']:
                self._print("# Redirecting: %s" % name)
            return getattr(self._stream, name)
        def write(self, data):
            print "###########"
            self._buffer.append(data)
            self._stream.write(data)
            self._stream.flush()
        def getBuffer(self):
            return self._buffer[:]
    def __init__(self, *args, **kwargs):
        print ">> Running `%s`" % " ".join(args[0])
        self._stdout = self.StreamWrapper(sys.stdout)
        self._stderr = self.StreamWrapper(sys.stderr)
        kwargs.setdefault('stdout', self._stdout)
        kwargs.setdefault('stderr', self._stderr)
        self._process = subprocess.Popen(*args, **kwargs)
        self._process.communicate()

更新:

我也想工作的是ANSI控制字符，用于移动光标并覆盖以前的输出内容.我不知道这是否是正确的术语，但这是我的意思的一个示例:我正在尝试使一些GIT东西自动化，并且那里的进度会自动更新，而无需每次都写入新行.

更新2

对我来说很重要，子流程的输出应立即显示.我尝试使用subprocess.PIPE捕获输出并手动显示，但是一旦处理完成，我只能使它显示输出.但是，我想实时查看输出.

解决方案

进程的Stdin，stdout和stderr必须是真实的文件描述符. (这实际上不是Python施加的限制，而是管道在OS级别上的工作方式.)因此，您将需要一个不同的解决方案.

如果要实时跟踪stdout和stderr，则需要异步I/O或线程.

• 异步I/O:使用标准的同步(=阻塞)I/O，对其中一个流的读取可能会阻塞，从而不允许实时访问另一个流.如果您使用的是Unix，则可以使用中显示了有关Python中异步I/O的更多信息和一些替代方法.这部影片.

• 线程:处理此问题的另一种常用方法是为要实时读取的每个文件描述符创建一个线程.线程仅处理它们所分配的文件描述符，因此阻止I/O不会造成损害.

I'd like to both capture and display the output of a process that I invoke through Python's subprocess.

I thought I could just pass my file-like object as named parameter stdout and stderr

I can see that it accesses the filenoattribute - so it is doing something with the object. However, the write() method is never invoked. Is my approach completely off or am I just missing something?

class Process(object):
    class StreamWrapper(object):
        def __init__(self, stream):
            self._stream = stream
            self._buffer = []
        def _print(self, msg):
            print repr(self), msg
        def __getattr__(self, name):
            if not name in ['fileno']:
                self._print("# Redirecting: %s" % name)
            return getattr(self._stream, name)
        def write(self, data):
            print "###########"
            self._buffer.append(data)
            self._stream.write(data)
            self._stream.flush()
        def getBuffer(self):
            return self._buffer[:]
    def __init__(self, *args, **kwargs):
        print ">> Running `%s`" % " ".join(args[0])
        self._stdout = self.StreamWrapper(sys.stdout)
        self._stderr = self.StreamWrapper(sys.stderr)
        kwargs.setdefault('stdout', self._stdout)
        kwargs.setdefault('stderr', self._stderr)
        self._process = subprocess.Popen(*args, **kwargs)
        self._process.communicate()

Update:

Something I'd like to work as well, is the ANSI control characters to move the cursor and override previously output stuff. I don't know whether that is the correct term, but here's an example of what I meant: I'm trying to automate some GIT stuff and there they have the progress that updates itself without writing to a new line each time.

Update 2

It is important to me, that the output of the subprocess is displayed immediately. I've tried using subprocess.PIPE to capture the output, and display it manually, but I was only able to get it to display the output, once the process had completed. However, I'd like to see the output in real-time.

解决方案

Stdin, stdout and stderr of a process need to be real file descriptors. (That is actually not a restriction imposed by Python, but rather how pipes work on the OS level.) So you will need a different solution.

If you want to track both stdout an stderr in real time, you will need asynchronous I/O or threads.

• Asynchronous I/O: With the standard synchronous (=blocking) I/O, a read to one of the streams could block, disallowing access to the other one in real time. If you are on Unix, you can use non-blocking I/O as described in this answer. However, on Windows you will be out of luck with this approach. More on asynchronous I/O in Python and some alternatives are shown in this video.

• Threads: Another common way to deal with this problem is to create one thread for each file descriptor you want to read from in real time. The threads only handle the file descriptor they are assinged to, so blocking I/O won't harm.

这篇关于包装子进程的stdout/stderr的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！