dup2 和管道恶作剧与 Python 和 Windows [英] dup2 and pipe shenanigans with Python and Windows

问题描述

向bug.py问好:

import os, sysstdout2 = os.dup(sys.stdout.fileno())(r,w) = os.pipe()os.dup2(w,sys.stdout.fileno())打印(冰雹斯坦")z = os.read(r,1000)os.write(stdout2,z)

如果你在 OSX 上运行它(我想，在 Linux 上)，这很好用.但是，在 Windows 中，我们得到了:

PS Z:\...>蟒蛇--版本蟒蛇 3.9.2PS Z:\...>蟒蛇错误.py回溯(最近一次调用最后一次):文件Z:\...\bug.py"，第 6 行，在 <module> 中.打印(冰雹斯坦")OSError: [WinError 1] 函数不正确异常被忽略:<_io.TextIOWrapper name='<stdout>'mode='w' encoding='utf-8'>OSError: [WinError 1] 函数不正确

我对任何事情都一无所知，但这闻起来有一些深沉的 Python+Windows 神秘主义，而且 PDB 并没有切奶酪，所以我无法调试解决这个问题的方法.任何人都知道如何使这项工作?

稍微大一点的上下文是我试图在我的应用程序中构建一些类似 tee 的功能，但是我发现的所有其他捕获标准输出的方法都不完整.您可以将 popen 和 subprocess 用于各种事情，但是如果您真的想获取整个进程的 stdout 和 stderr，至少在 Unix 上，我发现的唯一方法是通过管道 dup2 文件描述符.

解决方案

我不知道我是否真的到了最底层，但我已经大致弄清楚了这一点.

sys.stdout 是 TextIOWrapper 类型的对象，其 sys.stdout.write() 方法最终调用 os.write() 或 C 等价物，在文件描述符 sys.stdout.fileno() 上，即 1.文件描述符有几种类型:文件、套接字、串行端口、终端、管道等... C 库函数 write()、close() 等...主要适用于任何类型的文件描述符，但某些功能仅适用当文件描述符是合适的类型时.在创建时，作为 sys.stdout 的 TextIOWrapper 检查其文件描述符 (1) 并确定它是一个终端.仅在 Windows 上，sys.stdout.write() 最终会执行一些仅在文件描述符 1 确实是终端时才有效的操作.一旦您 os.dup2() 使 1 成为 os.pipe() 而不是终端，Windows Python 实现在尝试执行特定于终端的操作时会崩溃管道上的操作.

我不清楚有没有办法让 sys.stdout 重新检查它的文件描述符，以便它注意到它不再是终端，并避免 Python 解释器崩溃.>

作为一种解决方法，仅在 Windows 上，我正在执行 stdout.write = lambda z: os.write(stdout.fileno(),z.encode() if hasattr(z,'encode') elsez)

我还没有深入研究 Python 代码库以查看这是否足够，但它似乎确实允许我的程序正确运行，并且 print() 不再导致Python 解释器崩溃.

许多其他人改为使用 sys.stdout = ...，但出于多种原因我不想这样做.一方面，我担心某些脚本或模块可能会存储 sys.stdout 的本地缓存副本.尽管可以直接缓存 sys.stdout.write，但我认为这种可能性较小.此外，它允许我在我的构造函数中进行monkeypatch，例如StreamCapture(sys.stdout)，无需在我的构造函数中直接引用全局变量 sys.stdout(副作用").如果您相信一个函数可以改变它的输入，那么这是没有副作用的.

Say hello to bug.py:

import os, sys

stdout2 = os.dup(sys.stdout.fileno())
(r,w) = os.pipe()
os.dup2(w,sys.stdout.fileno())
print("Hail Stan")
z = os.read(r,1000)
os.write(stdout2,z)

If you run this on OSX (and I imagine, on Linux), this works great. However, in Windows, we get this:

PS Z:\...> python --version
Python 3.9.2
PS Z:\...> python bug.py
Traceback (most recent call last):
  File "Z:\...\bug.py", line 6, in <module>
    print("Hail Stan")
OSError: [WinError 1] Incorrect function
Exception ignored in: <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>
OSError: [WinError 1] Incorrect function

I don't know much about anything but this smells of some deep Python+Windows mysticism and PDB isn't cutting cheese so I can't debug my way out of this. Anyone knows how to make this work?

The slightly bigger context is that I'm trying to build some tee-like functionality into my application, but all other methods I've found of capturing stdout are incomplete. You can use popen and subprocess for various things, but if you truly want to grab your whole process's stdout and stderr, at least on Unix, the only way I found is to dup2 the filedescriptors through pipes.

解决方案

I don't know if I've truly got to the bottomest bottom but I've mostly figured this one out.

sys.stdout is an object of type TextIOWrapper, whose sys.stdout.write() method eventually calls either os.write() or the C equivalent, on the filedescriptor sys.stdout.fileno(), which is 1. There are several types of filedescriptors: files, sockets, serial ports, terminals, pipes, etc... The C library functions write(), close(), etc... work on mostly any type of filedescriptor, but some features only work when the filedescriptor is of a suitable type. At creation time, the TextIOWrapper that is sys.stdout examines its filedescriptor (1) and determines that it's a terminal. On Windows only, sys.stdout.write() ends up doing some operations that are only valid if the filedescriptor 1 is truly a terminal. Once you os.dup2() so that 1 becomes an os.pipe(), and not a terminal, the Windows Python implementation crashes when it attempts to do terminal-specific operations on a pipe.

It's not clear to me that there's a way of causing sys.stdout to re-examine its filedescriptor so that it notices that it's not a terminal anymore, and avoid crashing the Python interpreter.

As a workaround, on Windows only, I'm doing stdout.write = lambda z: os.write(stdout.fileno(),z.encode() if hasattr(z,'encode') else z)

I haven't done a deep dive through the Python code base to see whether this is sufficient, but it does appear to allow my programs to run correctly, and print() no longer causes the Python interpreter to crash.

Many other people instead do sys.stdout = ..., but I did not want to do this for many reasons. For one, I was worried that some script or module might store a local cached copy of sys.stdout. Although it is possible to cache sys.stdout.write directly, I thought that this was less likely. Furthermore, it allows me to monkeypatch in my constructor, e.g. StreamCapture(sys.stdout), without having to refer to the global variable sys.stdout directly in my constructor (a "side-effect"). If you believe that a function is allowed to mutate its inputs, then this is side-effect-free.

这篇关于dup2 和管道恶作剧与 Python 和 Windows的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！