Python:将二进制数据块写入和读取到文件 [英] Python: write and read blocks of binary data to a file

问题描述

我正在编写一个脚本，它将另一个 python 脚本分解为块并使用 pycrypto 来加密块(到目前为止我已经成功完成了所有这些)，现在我将加密的块存储到一个文件中，以便解密器可以读取它并执行每个块.加密的最终结果是一个二进制输出列表(类似于 blocks=[b'\xa1\r\xa594\x92z\xf8\x16\xaa',b'xfbI\xfdqx|\xcd\xdb\x1b\xb3' 等...]).

I am working on a script where it will breakdown another python script into blocks and using pycrypto to encrypt the blocks (all of this i have successfully done so far), now i am storing the encrypted blocks to a file so that the decrypter can read it and execute each block. The final result of the encryption is a list of binary outputs (something like blocks=[b'\xa1\r\xa594\x92z\xf8\x16\xaa',b'xfbI\xfdqx|\xcd\xdb\x1b\xb3',etc...]).

当将输出写入文件时，它们都以一个巨大的行结束，因此在读取文件时，所有字节都回到一个巨大的行中，而不是原始列表中的每个项目.我也尝试将字节转换为字符串，并在每个字节的末尾添加一个 '\n' ，但问题是我仍然需要字节，我想不通如何撤消字符串以获取原始字节.

When writing the output to a file, they all end up into one giant line, so that when reading the file, all the bytes come back in one giant line, instead of each item from the original list. I also tried converting the bytes into a string, and adding a '\n' at the end of each one, but the problem there is that I still need the bytes, and I can't figure out how to undo the string to get the original byte.

总而言之，我希望:将每个二进制项写入文件中的单独一行，以便我可以轻松读取数据并在解密中使用它，或者我可以将数据转换为字符串并在decrpytion 撤消字符串以取回原始二进制数据.

To summarize this, i am looking to either: write each binary item to a separate line in a file so i can easily read the data and use it in the decryption, or i could translate the data to a string and in the decrpytion undo the string to get back the original binary data.

这是写入文件的代码:

    new_file = open('C:/Python34/testfile.txt','wb')
    for byte_item in byte_list:
        # This or for the string i just replaced wb with w and
        # byte_item with ascii(byte_item) + '\n'
        new_file.write(byte_item)
    new_file.close()

和读取文件:

    # Or 'r' instead of 'rb' if using string method
    byte_list = open('C:/Python34/testfile.txt','rb').readlines()

推荐答案

文件是没有任何隐含结构的字节流.如果你想加载一个二进制 blob 列表，那么你应该存储一些额外的元数据来恢复结构，例如，你可以 使用网络字符串格式:

A file is a stream of bytes without any implied structure. If you want to load a list of binary blobs then you should store some additional metadata to restore the structure e.g., you could use the netstring format:

#!/usr/bin/env python
blocks = [b'\xa1\r\xa594\x92z\xf8\x16\xaa', b'xfbI\xfdqx|\xcd\xdb\x1b\xb3']

# save blocks
with open('blocks.netstring', 'wb') as output_file:
    for blob in blocks:
        # [len]":"[string]","
        output_file.write(str(len(blob)).encode())
        output_file.write(b":")
        output_file.write(blob)
        output_file.write(b",")

再读一遍:

#!/usr/bin/env python3
import re
from mmap import ACCESS_READ, mmap

blocks = []
match_size = re.compile(br'(\d+):').match
with open('blocks.netstring', 'rb') as file, \
     mmap(file.fileno(), 0, access=ACCESS_READ) as mm:
    position = 0
    for m in iter(lambda: match_size(mm, position), None):
        i, size = m.end(), int(m.group(1))
        blocks.append(mm[i:i + size])
        position = i + size + 1 # shift to the next netstring
print(blocks)

作为替代方案，您可以考虑为您的数据采用 BSON 格式 或 ascii 盔甲格式.

As an alternative, you could consider BSON format for your data or ascii armor format.

这篇关于Python:将二进制数据块写入和读取到文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！