将(大量)零写入二进制文件 [英] Write (large amount of) zeros into a binary file

问题描述

这可能是一个愚蠢的问题，但我找不到正确的答案.我想将零的(2000, 2000, 2000)数组的二进制表示形式存储(不问为什么)到磁盘上，二进制格式.实现这一目标的传统方法是:

This might be a stupid question but I'm unable to find a proper answer to it. I want to store (don't ask why) a binary representation of a (2000, 2000, 2000) array of zeros into disk, binary format. The traditional approach to achieve so would be:

with open('myfile', 'wb') as f:
    f.write('\0' * 4 * 2000 * 2000 * 2000)  # 4 bytes = float32

但这将意味着创建一个非常大的字符串，根本不需要.我知道另外两个选择:

But that would imply creating a very large string which is not necessary at all. I know of two other options:

• 遍历元素并一次存储一个字节(非常慢)

• Iterate over the elements and store one byte at a time (extremely slow)

创建一个numpy数组并将其刷新到磁盘(与上面示例中创建字符串的内存一样昂贵)

Create a numpy array and flush it to disk (as memory expensive as the string creation in the example above)

我希望找到类似write(char, ntimes)(存在于C和其他语言中)的东西，以C速度而不是以Python循环的速度复制到磁盘char ntimes上，而不必创建这样的东西很大的内存空间.

I was hopping to find something like write(char, ntimes) (as it exists in C and other languages) to copy on disk char ntimes at C speed, and not at Python-loops speed, without having to create such a big array on memory.

推荐答案

我不知道您为什么对"Python的循环速度"如此大惊小怪，但写法是

I don't know why you are making such a fuss about "Python's loop speed", but writing in the way of

for i in range(2000 * 2000):
     f.write('\0' * 4 * 2000)  # 4 bytes = float32

将告诉操作系统写入8000个0字节. write返回后，在下一个循环运行中，将再次调用它.

will tell the OS to write 8000 0-bytes. After write returns, in the next loop run it is called again.

循环的执行速度可能比在C语言中执行的速度稍慢，但这绝对不会有什么改变.

It might be that the loop is executed slightly slower than it would be in C, but that definitely won't make a difference.

如果可以使用稀疏文件，则也可以查找到所需文件大小的位置，然后截断该文件.

If it is ok to have a sparse file, you as well can seek to the desired file size's position and then truncate the file.

这篇关于将(大量)零写入二进制文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！