在python中将文本文件分成更小的块 [英] Break a text file into smaller chunks in python

问题描述

我有一个文本文件(以 MB 为单位)，我想将其分成多个块(以 KB 为单位).我正在模拟网络上的文件传输行为.到目前为止，我能够根据用户输入的行数(由 '\n' 分隔)制作块

I have a text file (in MBs) and I want to break it into chunks(in KBs). I am simulating the file transfer behaviour over a network. So far I was able to make chunks according to the number of lines(seprated by '\n') inputed by user like this

def make_chunks(fname):
    ifile = file(fname,'rb')
    file_iter = iter(ifile)
    args = [file_iter] * 10 # No of lines you want to have in one chunk
    chunks = list(izip_longest(fillvalue = None, *args))

但是块现在大小不同.我如何制作相同大小的块(比如 4KB)

But the chunks are now of different sizes.How would I make chunks of equal size(say 4KB)

推荐答案

您可以按实际字节大小分块:

You can chunk by actual bytesize:

def chunk(fname):
    with open(fname, 'rb') as fin:
        return list(iter(lambda: fin.read(4096), ''))

请注意，您最好yield每个块而不是构建列表，让调用者决定是否要构建列表.

Note that you might as well yield each chunk instead of building a list, and let the caller decide if it wants to build a list instead.

for chunk in iter(lambda: fin.read(4096), ''):
    yield chunk

这篇关于在python中将文本文件分成更小的块的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！