如何从 ZipFile 流式传输?如何“即时"拉上拉链? [英] How to stream from ZipFile? How to zip "on the fly"?

问题描述

我想压缩一个流并输出结果.我正在使用 AWS Lambda 来做这件事，这在可用磁盘空间和其他限制方面很重要.如果重要的话，我将使用压缩流通过 upload_fileobj() 或 put() 编写 AWS S3 对象.

I want to zip a stream and stream out the result. I'm doing it using AWS Lambda which matters in sense of available disk space and other restrictions. I'm going to use the zipped stream to write an AWS S3 object using upload_fileobj() or put(), if it matters.

我可以将存档创建为文件，直到我有小对象:

I can create an archive as a file until I have small objects:

import zipfile
zf = zipfile.ZipFile("/tmp/byte.zip", "w")
zf.writestr(filename, my_stream.read())
zf.close()

对于大量数据，我可以创建一个对象而不是文件:

For large amount of data I can create an object instead of file:

from io import BytesIO
...
byte = BytesIO()
zf = zipfile.ZipFile(byte, "w")
....

但是如何将压缩流传递给输出?如果我使用 zf.close() - 流将被关闭，如果我不使用它 - 存档将不完整.

but how can I pass the zipped stream to the output? If I use zf.close() - the stream will be closed, if I don't use it - the archive will be incomplete.

推荐答案

你可能想试试 zipstream 版本的压缩文件.例如，使用迭代器将 stdin 压缩到 stdout 作为一个 zip 文件，将数据保存为一个名为 TheLogFile 的文件:

You might like to try the zipstream version of zipfile. For example, to compress stdin to stdout as a zip file holding the data as a file named TheLogFile using iterators:

#!/usr/bin/python3
import sys, zipstream
with zipstream.ZipFile(mode='w', compression=zipstream.ZIP_DEFLATED) as z:
    z.write_iter('TheLogFile', sys.stdin.buffer)
    for chunk in z:
        sys.stdout.buffer.write(chunk)

这篇关于如何从 ZipFile 流式传输?如何“即时"拉上拉链?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！