如何从ZipFile流式传输?如何“即时"压缩? [英] How to stream from ZipFile? How to zip "on the fly"?

问题描述

我想压缩一个流并输出结果.我正在使用AWS Lambda做到这一点，这在可用磁盘空间和其他限制方面很重要. 如果重要的话，我将使用压缩流使用upload_fileobj()或put()编写一个AWS S3对象.

I want to zip a stream and stream out the result. I'm doing it using AWS Lambda which matters in sense of available disk space and other restrictions. I'm going to use the zipped stream to write an AWS S3 object using upload_fileobj() or put(), if it matters.

在我有小对象之前，我可以将存档创建为文件:

I can create an archive as a file until I have small objects:

import zipfile
zf = zipfile.ZipFile("/tmp/byte.zip", "w")
zf.writestr(filename, my_stream.read())
zf.close()

对于大量数据，我可以创建一个对象而不是文件:

For large amount of data I can create an object instead of file:

from io import BytesIO
...
byte = BytesIO()
zf = zipfile.ZipFile(byte, "w")
....

但是如何将压缩后的流传递到输出?如果我使用zf.close()-流将被关闭，如果我不使用它-归档将是不完整的.

but how can I pass the zipped stream to the output? If I use zf.close() - the stream will be closed, if I don't use it - the archive will be incomplete.

推荐答案

您可能想尝试 zipstream 版本的zipfile.例如，使用迭代器将stdin压缩为stdout作为zip文件，将数据保存为名为TheLogFile的文件:

You might like to try the zipstream version of zipfile. For example, to compress stdin to stdout as a zip file holding the data as a file named TheLogFile using iterators:

#!/usr/bin/python3
import sys, zipstream
with zipstream.ZipFile(mode='w', compression=zipstream.ZIP_DEFLATED) as z:
    z.write_iter('TheLogFile', sys.stdin.buffer)
    for chunk in z:
        sys.stdout.buffer.write(chunk)

这篇关于如何从ZipFile流式传输?如何“即时"压缩?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！