如何从内存中删除多个 Pandas (python) 数据帧以节省 RAM? [英] How to delete multiple pandas (python) dataframes from memory to save RAM?

问题描述

作为预处理的一部分，我创建了很多数据帧.由于我有 6GB 的内存限制，我想从内存中删除所有不必要的数据帧，以避免在 scikit-learn 中运行 GRIDSEARCHCV 时内存不足.

1) 是否有一个函数可以只列出当前加载到内存中的所有数据帧?

我尝试过 dir() 但它提供了除数据帧以外的许多其他对象.

2) 我创建了一个要删除的数据框列表

del_df=[Gender_dummies,胶囊传输，上校，concat_df_list，coup_CAPSULE_dummies]

&跑了

 for i in del_df:熟食店)

但它不会删除数据帧.但是单独删除数据帧如下所示是从内存中删除数据帧.

del Gender_dummies德尔科尔

解决方案

del 语句不删除实例，它只是删除一个名称.

当你执行 del i 时，你只是删除了名称 i - 但实例仍然绑定到其他名称，所以它不会是垃圾 -已收集.

如果您想释放内存，您的数据帧必须垃圾收集，即删除对它们的所有引用.

如果您动态地创建日期框来列出，那么删除该列表将触发垃圾收集.

<预><代码>>>>lst = [pd.DataFrame(), pd.DataFrame(), pd.DataFrame()]>>>del lst #内存被释放

如果您创建了一些变量，则必须将它们全部删除.

<预><代码>>>>a, b, c = pd.DataFrame(), pd.DataFrame(), pd.DataFrame()>>>lst = [a, b, c]>>>del a, b, c # dfs 仍在列表中>>>del lst # 现在释放内存

I have lot of dataframes created as part of preprocessing. Since I have limited 6GB ram, I want to delete all the unnecessary dataframes from RAM to avoid running out of memory when running GRIDSEARCHCV in scikit-learn.

1) Is there a function to list only, all the dataframes currently loaded in memory?

I tried dir() but it gives lot of other object other than dataframes.

2) I created a list of dataframes to delete

del_df=[Gender_dummies,
 capsule_trans,
 col,
 concat_df_list,
 coup_CAPSULE_dummies]

& ran

for i in del_df:
    del (i)

But its not deleting the dataframes. But deleting dataframes individially like below is deleting dataframe from memory.

del Gender_dummies
del col

解决方案

del statement does not delete an instance, it merely deletes a name.

When you do del i, you are deleting just the name i - but the instance is still bound to some other name, so it won't be Garbage-Collected.

If you want to release memory, your dataframes has to be Garbage-Collected, i.e. delete all references to them.

If you created your dateframes dynamically to list, then removing that list will trigger Garbage Collection.

>>> lst = [pd.DataFrame(), pd.DataFrame(), pd.DataFrame()]
>>> del lst     # memory is released

If you created some variables, you have to delete them all.

>>> a, b, c = pd.DataFrame(), pd.DataFrame(), pd.DataFrame()
>>> lst = [a, b, c]
>>> del a, b, c # dfs still in list
>>> del lst     # memory release now

这篇关于如何从内存中删除多个 Pandas (python) 数据帧以节省 RAM?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！