使用io.TextIOWrapper包装开放流 [英] Wrap an open stream with io.TextIOWrapper

问题描述

如何封装一个打开的二进制流 - 一个Python 2 文件，一个Python 3 io.BufferedReader io.BytesIO - 在 io.TextIOWrapper ？

How can I wrap an open binary stream – a Python 2 file, a Python 3 io.BufferedReader, an io.BytesIO – in an io.TextIOWrapper?

我试图编写将不起作用的代码：

I'm trying to write code that will work unchanged:

• 在Python 2上运行

<

• 使用标准库生成的二进制流（即我无法控制它们是什么类型）




• 生成 io.TextIOWrapper

• Running on Python 2.
• Running on Python 3.
• With binary streams generated from the standard library (i.e. I can't control what type they are)
• With binary streams made to be test doubles (i.e. no file handle, can't re-open).
• Producing an io.TextIOWrapper that wraps the specified stream.

需要 io.TextIOWrapper API由标准库的其他部分预期。其他类似文件的类型存在，但不提供正确的API。

The io.TextIOWrapper is needed because its API is expected by other parts of the standard library. Other file-like types exist, but don't provide the right API.

流显示为 subprocess.Popen.stdout 属性：

import subprocess
import io

gnupg_subprocess = subprocess.Popen(
        ["gpg", "--version"], stdout=subprocess.PIPE)
gnupg_stdout = io.TextIOWrapper(gnupg_subprocess.stdout, encoding="utf-8")

<在单元测试中，用 io.BytesIO 实例替换流以控制其内容，而不触及任何子过程或文件系统。

In unit tests, the stream is replaced with an io.BytesIO instance to control its content without touching any subprocesses or filesystems.

gnupg_subprocess.stdout = io.BytesIO("Lorem ipsum".encode("utf-8"))

在Python 3的标准库创建的流上正常工作。同样的代码在Python 2生成的流上失败：

That works fine on the streams created by Python 3's standard library. The same code, though, fails on streams generated by Python 2:

[Python 2]
>>> type(gnupg_subprocess.stdout)
<type 'file'>
>>> gnupg_stdout = io.TextIOWrapper(gnupg_subprocess.stdout, encoding="utf-8")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'file' object has no attribute 'readable'

处理文件

一个明显的响应是在代码中有一个分支， Python 2 文件对象，并处理与 io。* 对象不同。

Not a solution: Special treatment for file

An obvious response is to have a branch in the code which tests whether the stream actually is a Python 2 file object, and handle that differently from io.* objects.

这不是一个经过良好测试的代码的选项，因为它使一个分支单元测试 - 为了尽可能快地运行，不能创建任何真正的文件系统对象 - 不能锻炼。

That's not an option for well-tested code, because it makes a branch that unit tests – which, in order to run as fast as possible, must not create any real filesystem objects – can't exercise.

单元测试将提供测试双精度，而不是真正的文件对象。因此，创建一个不会被这些测试双精度执行的分支会打败测试套件。

The unit tests will be providing test doubles, not real file objects. So creating a branch which won't be exercised by those test doubles is defeating the test suite.

一些受访者建议重新打开（例如使用 io.open ）底层文件句柄：

Some respondents suggest re-opening (e.g. with io.open) the underlying file handle:

gnupg_stdout = io.open(
        gnupg_subprocess.stdout.fileno(), mode='r', encoding="utf-8")

Python 2：

[Python 3]
>>> type(gnupg_subprocess.stdout)
<class '_io.BufferedReader'>
>>> gnupg_stdout = io.open(gnupg_subprocess.stdout.fileno(), mode='r', encoding="utf-8")
>>> type(gnupg_stdout)
<class '_io.TextIOWrapper'>

[Python 2]
>>> type(gnupg_subprocess.stdout)
<type 'file'>
>>> gnupg_stdout = io.open(gnupg_subprocess.stdout.fileno(), mode='r', encoding="utf-8")
>>> type(gnupg_stdout)
<type '_io.TextIOWrapper'>

但是当然依赖于重新打开一个真实文件文件句柄。因此，当测试double是 io.BytesIO 实例时，它在单元测试中失败：

But of course it relies on re-opening a real file from its file handle. So it fails in unit tests when the test double is an io.BytesIO instance:

>>> gnupg_subprocess.stdout = io.BytesIO("Lorem ipsum".encode("utf-8"))
>>> type(gnupg_subprocess.stdout)
<type '_io.BytesIO'>
>>> gnupg_stdout = io.open(gnupg_subprocess.stdout.fileno(), mode='r', encoding="utf-8")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
io.UnsupportedOperation: fileno

不是解决方案： getreader

标准库还提供了 codecs 模块，

import codecs

gnupg_stdout = codecs.getreader("utf-8")(gnupg_subprocess.stdout)

这很好，因为它不会尝试重新打开流。但它未能提供 io.TextIOWrapper API。具体来说，不会继承 io.IOBase 和没有编码 attribute ：

That's good because it doesn't attempt to re-open the stream. But it fails to provide the io.TextIOWrapper API. Specifically, it doesn't inherit io.IOBase and doesn't have the encoding attribute:

>>> type(gnupg_subprocess.stdout)
<type 'file'>
>>> gnupg_stdout = codecs.getreader("utf-8")(gnupg_subprocess.stdout)
>>> type(gnupg_stdout)
<type 'instance'>
>>> isinstance(gnupg_stdout, io.IOBase)
False
>>> gnupg_stdout.encoding
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/codecs.py", line 643, in __getattr__
    return getattr(self.stream, name)
AttributeError: '_io.BytesIO' object has no attribute 'encoding'

所以 codecs 不提供代替 io.TextIOWrapper 的对象。

So codecs doesn't provide objects which substitute for io.TextIOWrapper.

那么如何编写适用于Python 2和Python 3的代码，同时包含测试双精度和真实对象，包装 io.TextIOWrapper

So how can I write code that works for both Python 2 and Python 3, with both the test doubles and the real objects, which wraps an io.TextIOWrapper around the already-open byte stream?

推荐答案

根据各种不同的建议论坛，并尝试标准库以满足标准，我目前的结论是这不能做与库和类型，因为我们目前有他们。

Based on multiple suggestions in various forums, and experimenting with the standard library to meet the criteria, my current conclusion is this can't be done with the library and types as we currently have them.

这篇关于使用io.TextIOWrapper包装开放流的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！