使用python压缩大文件 [英] Big file compression with python

问题描述

我想用python压缩大文本文件（我说的是> 20Gb文件）。
我不是专家，所以我尝试收集我发现的信息，以下内容似乎起作用：

I want to compress big text files with python (I am talking about >20Gb files). I am not any how an expert so I tried to gather the info I found and the following seems to work :

import bz2

with open('bigInputfile.txt', 'rb') as input:
    with bz2.BZ2File('bigInputfile.txt.bz2', 'wb', compresslevel = 9) as output:
        while True:
            block = input.read(900000)
                if not block:
                    break
                output.write(block)

input.close()
output.close()

我是想知道此语法是否正确以及是否有优化方法？我的印象是我在这里缺少任何内容。

I am wondering if this syntax is correct and if there is a way to optimize it ? I have an impression that I am missing something here.

非常感谢。

推荐答案

您的脚本似乎正确，但可以缩写：

Your script seems correct, but can be abbreviated:

from shutil import copyfileobj

with open('bigInputfile.txt', 'rb') as input:
    with bz2.BZ2File('bigInputfile.txt.bz2', 'wb', compresslevel=9) as output:
        copyfileobj(input, output)

这篇关于使用python压缩大文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！