numpy保存/加载损坏数组 [英] numpy save/load corrupting an array

问题描述

我正在尝试保存一个大的numpy数组并重新加载它.使用numpy.save和numpy.load，数组值已损坏/更改.数组的形状和数据类型在保存前和加载后是相同的，但是加载后的数组将零值的绝大多数都设为零. 数组为(22915,22915)，值是float64的值，将3.94 GB作为.npy文件，并且数据条目的平均值约为.1(不是可能合理地转换为零的微小浮点数).我正在使用numpy 1.5.1.

I am trying to save a large numpy array and reload it. Using numpy.save and numpy.load, the array values are corrupted/change. The shape and data type of the array pre-saving, and post-loading, are the same, but the post-loading array has the vast majority of the values zeroed. The array is (22915,22915), values are float64's, takes 3.94 gb's as a .npy file, and the data entries average about .1 (not tiny floats that might reasonably get converted to zeroes). I am using numpy 1.5.1.

对于造成这种腐败的原因的任何帮助将不胜感激，因为我很茫然.以下是提供上述声明证据的一些代码.

Any help on why this corruption is occurring would be greatly appreciated because I am at a loss. Below is some code providing evidence of the claims above.

In [7]: m
Out[7]: 
      array([[ 0.     ,  0.02023,  0.00703, ...,  0.02362,  0.02939,  0.03656],
             [ 0.02023,  0.     ,  0.0135 , ...,  0.04357,  0.04934,  0.05651],
             [ 0.00703,  0.0135 ,  0.     , ...,  0.03037,  0.03614,  0.04331],
             ..., 
             [ 0.02362,  0.04357,  0.03037, ...,  0.     ,  0.01797,  0.02514],
             [ 0.02939,  0.04934,  0.03614, ...,  0.01797,  0.     ,  0.01919],
             [ 0.03656,  0.05651,  0.04331, ...,  0.02514,  0.01919,  0.     ]])
In [8]: m.shape
Out[8]: (22195, 22195)

In [12]: save('/Users/will/Desktop/m.npy',m)

In [14]: lm = load('/Users/will/Desktop/m.npy')

In [15]: lm
Out[15]: 
       array([[ 0.     ,  0.02023,  0.00703, ...,  0.     ,  0.     ,  0.     ],
              [ 0.     ,  0.     ,  0.     , ...,  0.     ,  0.     ,  0.     ],
              [ 0.     ,  0.     ,  0.     , ...,  0.     ,  0.     ,  0.     ],
              ..., 
              [ 0.     ,  0.     ,  0.     , ...,  0.     ,  0.     ,  0.     ],
              [ 0.     ,  0.     ,  0.     , ...,  0.     ,  0.     ,  0.     ],
              [ 0.     ,  0.     ,  0.     , ...,  0.     ,  0.     ,  0.     ]])
In [17]: type(lm[0][0])
Out[17]: numpy.float64

In [18]: type(m[0][0])
Out[18]: numpy.float64

In [19]: lm.shape
Out[19]: (22195, 22195)

推荐答案

这是已知的 issue (请注意，该链接针对numpy 1.4).如果您确实无法升级，我的建议是尝试以其他方式保存(savez，savetxt).如果有getbuffer可用，您可以尝试直接写入字节.如果其他所有方法均失败(并且您无法升级)，则可以轻松编写自己的保存功能.

This is a known issue (note that that links against numpy 1.4). If you really can't upgrade, my advice would be to try to save in a different way (savez, savetxt). If getbuffer is available you can try to write the bytes directly. If all else fails (and you can't upgrade), you can write your own save function pretty easily.

这篇关于numpy保存/加载损坏数组的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！