Python列表内存错误 [英] Python list memory error

问题描述

我正在尝试列出2个元素(最多30个元素)，但出现内存错误.为什么这样?它超出了python中列表的最大限制吗?

I am trying to make a list with 2 raised to 30 elements but am getting a memory error. Why so? Is it outside the maximum limit of a list in python?

m=[None]*(2**30)

推荐答案

是的，Python列表可以容纳多少个元素是有限制的，请参见

Yes, there is a limit to how many elements a Python list can hold, see sys.maxsize. You did not hit it however; very few machines have enough memory to hold that many items.

您正在尝试创建包含1073741824引用的列表；每个引用也占用内存.这取决于您的操作系统多少，但是对于32位系统，通常为4个字节，对于64位操作系统，通常为8个字节，其中2 ^ 30个元素将占用4 GB或8GB的内存，只是用于列表引用.

You are trying to create a list with 1073741824 references; each reference takes memory too. It depends on your OS how much, but typically that is going to be 4 bytes for a 32-bit system, 8 bytes for a 64-bit OS, where 2^30 elements would take 4 GB or 8GB of memory, just for the list references.

4GB加上其他元素已经比大多数当前操作系统允许在内存中使用单个进程的数量容易得多.

4GB plus other elements is already easily more than what most current operating systems will permit a single process to use in memory.

在我的Mac OS X计算机(使用64位OS)上，sys.maxsize为2 ^ 63，列表中的Python对象引用占8个字节:

On my Mac OS X machine (using 64-bit OS), sys.maxsize is 2^63, and Python object references in a list take 8 bytes:

>>> import sys
>>> sys.maxsize
9223372036854775807
>>> sys.maxsize.bit_length()
63
>>> sys.getsizeof([])  # empty list overhead
72
>>> sys.getsizeof([None]) - sys.getsizeof([])  # size of one reference
8

因此，要创建包含sys.maxsize个元素的列表，您需要64个 exbibytes 记忆，仅用于参考.这远远超出了64位计算机可以处理的范围(实际最大值约为16 EB). ).

So to create a list with sys.maxsize elements you'd need 64 exbibytes of memory, just for the references. That is more than what a 64-bit computer could address (the practical maximum is about 16 exbibytes).

所有这些都忽略了列表中引用的对象将占用的内存占用. None是单例，因此它只会占用固定数量的内存.但是大概您要在该列表中存储 else ，在这种情况下，您也需要考虑到这一点.

All this is ignoring the memory footprint that the objects you are referencing in the list will take. None is a singleton, so it'll only ever take a fixed amount of memory. But presumably you were going to store something else in that list, in which case you need to take that into account too.

通常来说，您永远不需要 来创建这么大的列表.使用不同的技术；例如，使用字典创建一个稀疏结构，我怀疑您打算直接在算法中解决所有这2 ^ 30个索引.

And generally speaking you should never need to create such a large list. Use different techniques; create a sparse structure using a dictionary for example, I doubt you were planning to address all those 2^30 indexes directly in your algorithm.

这篇关于Python列表内存错误的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！