拆开保存 pandas 数据框的字典会抛出AttributeError：'Dataframe'对象没有属性'_data' [英] Unpickling dictionary that holds pandas dataframes throws AttributeError: 'Dataframe' object has no attribute '_data'
问题描述
我有一类执行分析并将结果（即熊猫数据帧）附加为对象属性。
I have a class that performs analyses and attaches the results, which are pandas dataframes, as object attributes:
>>> print(test.image.locate_DF)
y x mass ... raw_mass ep frame
0 60.177142 59.788709 33.433414 ... 242.080256 NaN 0
1 60.651991 59.773904 33.724308 ... 242.355784 NaN 1
2 60.790437 60.190234 31.117164 ... 236.276671 NaN 2
3 60.771933 60.048123 33.558372 ... 240.981395 NaN 3
4 60.251282 59.775139 31.881009 ... 239.239022 NaN 4
... ... ... ... ... ... ... ...
7212 68.186380 76.477449 18.122817 ... 176.523091 NaN 9410
7213 68.764444 76.574091 17.486454 ... 173.448306 NaN 9415
7214 68.191152 76.473477 17.402975 ... 172.848119 0.868326 9429
7215 67.034103 76.025885 17.010951 ... 170.928067 -0.600854 9431
7216 68.583276 75.309592 17.852992 ... 178.271558 NaN 9432
随后，我保存了字典中所有重要的对象属性，并将其腌制以备后用：
Subsequently, I save all the important object attributes in a dictionary, and pickle it for later use:
def save_parameters(self, filepath):
param_dict = {}
try:
self.image.locate_DF
except AttributeError:
pass
else:
param_dict['optical_locate_DF'] = self.image.locate_DF
with open(filepath, 'wb') as handle:
pickle.dump(param_dict, handle, 5)
当尝试加载该腌制文件时，我一点都没有问题，数据帧已完美加载：
When trying to load that pickled file, I have no problem at all, the dataframe loads perfectly:
>>> test.save_parameters('test.pickle')
>>> with open('test.pickle', 'rb') as handle:
... result = pickle.load(handle)
...
>>> print(result.keys())
dict_keys(['optical_path', 'optical_feature_diameter', 'optical_feature_minmass', 'optical_locate_DF', 'electrical_path', 'electrical_raw_data', 'electrical_processed_data', 'electrical_mean_voltage'])
>>> print(result['optical_locate_DF'])
y x mass ... raw_mass ep frame
0 60.177142 59.788709 33.433414 ... 242.080256 NaN 0
1 60.651991 59.773904 33.724308 ... 242.355784 NaN 1
2 60.790437 60.190234 31.117164 ... 236.276671 NaN 2
3 60.771933 60.048123 33.558372 ... 240.981395 NaN 3
4 60.251282 59.775139 31.881009 ... 239.239022 NaN 4
... ... ... ... ... ... ... ...
7212 68.186380 76.477449 18.122817 ... 176.523091 NaN 9410
7213 68.764444 76.574091 17.486454 ... 173.448306 NaN 9415
7214 68.191152 76.473477 17.402975 ... 172.848119 0.868326 9429
7215 67.034103 76.025885 17.010951 ... 170.928067 -0.600854 9431
7216 68.583276 75.309592 17.852992 ... 178.271558 NaN 9432
[7217 rows x 9 columns]
但是，在对一堆这样的文件在hpc上运行我的分析之后，然后尝试打开相同的腌制文件（现在的名称有所不同）但它与上面显示的文件相同，并且执行了相同的分析），但熊猫却抛出了属性错误。它指出数据框没有 _data属性。字典具有相同的键，并且打印出不是数据帧的键也没有任何问题。
However, after running my analysis on a bunch of these files on a hpc, and then trying to open that same pickled file (it's named differently now but it's the same file as shown above, with the same analysis performed on it), I get thrown an attribute error by pandas. It states that the dataframe has no '_data' attribute. The dictionary has the same keys and the keys that are not a dataframe are printed without any issues:
>>> resultfile = '../results/diam_15_minmass_17_dist_50_mem_5000_tracklength_500/R9_DNA_50mV_001.pickle'
>>> with open(resultfile, 'rb') as handle:
... result = pickle.load(handle)
...
>>> print(result.keys())
dict_keys(['optical_path', 'optical_feature_diameter', 'optical_feature_minmass', 'optical_locate_DF', 'optical_tracking_distance', 'optical_tracking_memory', 'optical_tracking_DF', 'optical_kinetics_DF', 'electrical_path', 'electrical_raw_data', 'electrical_processed_data', 'electrical_mean_voltage'])
>>> print(result['optical_locate_DF'])
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/stevenvanuytsel/miniconda3/envs/simultaneous_measurements/lib/python3.8/site-packages/pandas/core/frame.py", line 680, in __repr__
self.to_string(
File "/Users/stevenvanuytsel/miniconda3/envs/simultaneous_measurements/lib/python3.8/site-packages/pandas/core/frame.py", line 801, in to_string
formatter = fmt.DataFrameFormatter(
File "/Users/stevenvanuytsel/miniconda3/envs/simultaneous_measurements/lib/python3.8/site-packages/pandas/io/formats/format.py", line 593, in __init__
self.max_rows_displayed = min(max_rows or len(self.frame), len(self.frame))
File "/Users/stevenvanuytsel/miniconda3/envs/simultaneous_measurements/lib/python3.8/site-packages/pandas/core/frame.py", line 1041, in __len__
return len(self.index)
File "/Users/stevenvanuytsel/miniconda3/envs/simultaneous_measurements/lib/python3.8/site-packages/pandas/core/generic.py", line 5270, in __getattr__
return object.__getattribute__(self, name)
File "pandas/_libs/properties.pyx", line 63, in pandas._libs.properties.AxisProperty.__get__
File "/Users/stevenvanuytsel/miniconda3/envs/simultaneous_measurements/lib/python3.8/site-packages/pandas/core/generic.py", line 5270, in __getattr__
return object.__getattribute__(self, name)
AttributeError: 'DataFrame' object has no attribute '_data'
我查看了泡菜手册，并通过一堆SO问题，但我似乎无法找出问题所在。有谁知道如何解决此问题，以及我是否仍然可以访问该数据？
I've looked into the pickle manual, and through a bunch of SO questions, but I can't seem to find out what is going wrong here. Does anyone have an idea how to fix this, and also whether I can still access that data?
推荐答案
我遇到了同样的问题。我在使用Pandas 1.1.1的环境中生成了一个Pandas数据框，并将其保存到一个pickle文件中。
I had the same problem. I generated a Pandas dataframe in an environment with Pandas 1.1.1 and saved it to a pickle file.
with open('file.pkl', 'wb') as f:
pickle.dump(data_frame_object, f)
在另一个会话中将其解封后并打印数据框，我得到了相同的错误。在不同环境中的一些测试显示了以下模式：
After unpickling it in another session and printing the dataframe I got the same error. Some testing in different environments showed the following pattern:
- 熊猫环境> = 1.1.0：有效
- 熊猫环境== 1.0.5：错误消息如上所述
- 熊猫环境== 1.0.3：内核崩溃
使用HDF5格式时出现相同的错误，因此似乎与数据框和不同的Pandas版本存在兼容性问题。
I got the same error using the HDF5 format so it seems to be a compatibility issue with the dataframe and different Pandas versions.
在受影响的环境中将Pandas更新到1.1.1为我解决了这个问题。
Updating Pandas to 1.1.1 in the affected environments solved the issue for me.
这篇关于拆开保存 pandas 数据框的字典会抛出AttributeError：'Dataframe'对象没有属性'_data'的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
