pandas :使用Apply将单个列数组拆分为多个列时出现内存错误 [英] Pandas: Memory error when using apply to split single column array into columns

问题描述

我想知道是否有人对在较大数据上执行以下示例中所示的操作时出现的内存错误有快速解决方案?

I am wondering if anybody has a quick fix for a memory error that appears when doing the same thing as in the below example on larger data?

示例:

import pandas as pd
import numpy as np

nRows = 2
nCols = 3

df = pd.DataFrame(index=range(nRows ), columns=range(1))

df2 = df.apply(lambda row: [np.random.rand(nCols)], axis=1)

df3 = pd.concat(df2.apply(pd.DataFrame, columns=range(nCols)).tolist())

这是在创建df3时出现内存错误.

It is when creating df3 I get memory error.

示例中的DF:

df
     0
0  NaN
1  NaN

df2
0    [[0.6704675101784022, 0.41730480236712697, 0.5...
1    [[0.14038693859523377, 0.1981014890848788, 0.8...
dtype: object

df3
          0         1         2
0  0.670468  0.417305  0.558690
0  0.140387  0.198101  0.800745

推荐答案

首先，我认为在熊猫中使用list不是好主意，如果可以的话，可以避免.

First I think working with lists in pandas is not good idea, if possible, you can avoid it.

因此，我相信您可以大大简化代码:

So I believe you can simplify your code a lot:

nRows = 2
nCols = 3

np.random.seed(2019)
df3 = pd.DataFrame(np.random.rand(nRows, nCols))
print (df3)
          0         1         2
0  0.903482  0.393081  0.623970
1  0.637877  0.880499  0.299172

这篇关于 pandas :使用Apply将单个列数组拆分为多个列时出现内存错误的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！