io.BytesIO流中的numpy.load [英] numpy.load from io.BytesIO stream

问题描述

我将numpy数组保存在Azure Blob存储中，并将它们加载到这样的流中:

I have numpy arrays saved in Azure Blob Storage, and I'm loading them to a stream like this:

stream = io.BytesIO()
store.get_blob_to_stream(container, 'cat.npy', stream)

我从stream.getvalue()知道流包含用于重构数组的元数据.这是前150个字节:

I know from stream.getvalue() that the stream contains the metadata to reconstruct the array. This is the first 150 bytes:

b"\x93NUMPY\x01\x00v\x00{'descr': '|u1', 'fortran_order': False, 'shape': (720, 1280, 3), }                                                  \n\xc1\xb0\x94\xc2\xb1\x95\xc3\xb2\x96\xc4\xb3\x97\xc5\xb4\x98\xc6\xb5\x99\xc7\xb6\x9a\xc7"

是否可以使用numpy.load或通过其他一些简单的方法来加载字节流?

Is it possible to load the bytes stream with numpy.load or by some other simple method?

我可以将阵列保存到磁盘并从磁盘加载，但是出于某些原因，我想避免这种情况……

I could instead save the array to disk and load it from disk, but I'd like to avoid that for several reasons...

仅强调一下，输出必须是一个numpy数组，其形状和dtype在流的前128个字节中指定.

just to emphasize, the output would need to be a numpy array with the shape and dtype specified in the 128 first bytes of the stream.

推荐答案

涉及np.savez时，上述解决方案通常需要工作.

When it comes to np.savez the above solution normally want work.

import io    
import numpy as np    

stream = io.BytesIO()  
arr1 = np.random.rand(20,4)  
arr2 = np.random.rand(20,4)  
np.savez(stream, A=arr1, B=arr2)  
block_blob_service.create_blob_from_bytes(container, 
                                          "my/path.npz", 
                                          stream.getvalue())

从存储下载:

from numpy.lib.npyio import NpzFile 

stream = io.BytesIO()  
block_blob_service.get_blob_to_stream(container, "my/path.npz", stream)  

ret = NpzFile(stream, own_fid=True, allow_pickle=True)  
print(ret.files)  
""" ['A', 'B'] """  
print(ret['A'].shape)  
""" (20, 4) """  

这篇关于io.BytesIO流中的numpy.load的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！