为什么python线程会消耗这么多内存? [英] Why does python thread consume so much memory?

问题描述

为什么python线程会消耗这么多内存?

Why does python thread consumes so much memory?

我测量到产生一个线程会消耗8兆的内存，几乎与一个新的python进程一样大！

I measured that spawning one thread consumes 8 megs of memory, almost as big as a whole new python process!

操作系统:Ubuntu 10.10

OS: Ubuntu 10.10

由于需求旺盛，我将举一些多余的例子，这里是:

due to popular demand I'll give some extraneous examples, here it is:

from os import getpid
from time import sleep
from threading import Thread

def nap():
    print 'sleeping child'
    sleep(999999999)

print getpid()
child_thread = Thread(target=nap)
sleep(999999999)

在我的盒子上，pmap pid将显示9424K

On my box, pmap pid will give 9424K

现在，让我们运行子线程:

Now, let's run the child thread:

from os import getpid
from time import sleep
from threading import Thread

def nap():
    print 'sleeping child'
    sleep(999999999)

print getpid()
child_thread = Thread(target=nap)
child_thread.start()             # <--- ADDED THIS LINE
sleep(999999999)

现在pmap pid将给出17620K

Now pmap pid will give 17620K

因此，额外线程的成本为17620K-9424K = 8196K

So, the cost for the extra thread is 17620K - 9424K = 8196K

即. 87％的人运行着一个全新的独立流程！

ie. 87% of running a whole new separate process!

现在不只是这样，对吗?

Now isn't that just, wrong?

推荐答案

这不是特定于Python的，并且与操作系统为每个线程分配的单独堆栈有关.您的操作系统上默认的最大堆栈大小恰巧是8MB.

This is not Python-specific, and has to do with the separate stack that gets allocated by the OS for every thread. The default maximum stack size on your OS happens to be 8MB.

请注意，8MB只是预留了一部分地址空间，最初只分配了很少的内存.必要时，额外的内存将提交到堆栈，最多8MB限制.

Note that the 8MB is simply a chunk of address space that gets set aside, with very little memory committed to it initially. Additional memory gets committed to the stack when required, up to the 8MB limit.

可以使用ulimit -s调整限制，但是在这种情况下，我认为没有必要这样做.

The limit can be tweaked using ulimit -s, but in this instance I see no reason to do this.

顺便说一句，pmap显示地址空间的用法.这不是衡量内存使用率的好方法.如果相关的话，这两个概念是截然不同的.

As an aside, pmap shows address space usage. It isn't a good way to gauge memory usage. The two concepts are quite distinct, if related.

这篇关于为什么python线程会消耗这么多内存?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！