MemoryError:无法分配具有形状和数据类型对象的数组 [英] MemoryError: Unable to allocate array with shape and data type object

问题描述

我想在两个看起来像这样的数据集之间执行内部联接:

I want to perform an inner join between two dataset which looks like:

theme_ids.head()
id  Loan Theme ID   Loan Theme Type Partner ID
0   638631  a1050000000skGl General 151
1   640322  a1050000000skGl General 151
2   641006  a1050000002X1ij Higher Education    160
3   641019  a1050000002X1ij Higher Education    160
4   641594  a1050000002VbsW Subsistence Agriculture 336

和

theme_reg.head()
Partner ID  Field Partner Name  sector  Loan Theme ID   Loan Theme Type country forkiva region  geocode_old ISO ... amount  LocationName    geocode names   geo lat lon mpi_region  mpi_geo rural_pct
0   9   KREDIT Microfinance Institution General Financial Inclusion a1050000000slfi Higher Education    Cambodia    No  Banteay Meanchey    (13.75, 103.0)  KHM ... 450 Banteay Meanchey, Cambodia  [(13.6672596, 102.8975098)] Banteay Meanchey Province; Cambodia (13.6672596, 102.8975098)   13.667259   102.897507  Banteay Mean Chey, Cambodia (13.6672596, 102.8975098)   90
1   9   KREDIT Microfinance Institution General Financial Inclusion a10500000068jPe Vulnerable Populations  Cambodia    No  Battambang Province NaN KHM ... 20275   Battambang Province, Cambodia   [(13.0286971, 102.989615)]  Battambang Province; Cambodia   (13.0286971, 102.989615)    13.028697   102.989616  Banteay Mean Chey, Cambodia (13.6672596, 102.8975098)   90
2   9   KREDIT Microfinance Institution General Financial Inclusion a1050000000slfi Higher Education    Cambodia    No  Battambang Province NaN KHM ... 9150    Battambang Province, Cambodia   [(13.0286971, 102.989615)]  Battambang Province; Cambodia   (13.0286971, 102.989615)    13.028697   102.989616  Banteay Mean Chey, Cambodia (13.6672596, 102.8975098)   90
3   9   KREDIT Microfinance Institution General Financial Inclusion a10500000068jPe Vulnerable Populations  Cambodia    No  Kampong Cham Province   (12.0, 105.5)   KHM ... 604950  Kampong Cham Province, Cambodia [(12.0982918, 105.3131185)] Kampong Cham Province; Cambodia (12.0982918, 105.3131185)   12.098291   105.313118  Kampong Cham, Cambodia  (11.9924294, 105.4645408)   90
4   9   KREDIT Microfinance Institution General Financial Inclusion a1050000002X1Uu Sanitation  Cambodia    No  Kampong Cham Province   (12.0, 105.5)   KHM ... 275 Kampong Cham Province, Cambodia [(12.0982918, 105.3131185)] Kampong Cham Province; Cambodia (12.0982918, 105.3131185)   12.098291   105.313118  Kampong Cham, Cambodia  (11.9924294, 105.4645408)   90

我尝试过:

data = pd.merge(theme_ids, theme_reg, on='Partner ID', how='inner') 

它引发:

MemoryError:无法分配形状为(15，144356281)的数组，并且 数据类型对象

MemoryError: Unable to allocate array with shape (15, 144356281) and data type object

将获得帮助.谢谢！

推荐答案

正如@Iguananaut在

As @Iguananaut said in Unable to allocate array with shape and data type, you can modify the overcommit behavior if you are running over a Linux system. To allow overcommit, you can write 1 on /proc/sys/vm/overcommit_memory as he explained in the answer.

这篇关于MemoryError:无法分配具有形状和数据类型对象的数组的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！