可以设置cpython堆大小？ [英] Possible to set cpython heap size?
问题描述
我有一个扫描并处理大量文本文件的应用程序。
我在内存中拉出并保存的内容至少为200MB。
我很想能告诉CPython虚拟机我需要一个
堆，比如前面300MB，而不是根据需要增长。我已经通过comp.lang.python和python
文档的档案扫描了
，但找不到办法。这样可以用PVM配置
吗？
非常感谢，
Andy
-
I have an application that scans and processes a bunch of text files.
The content I''m pulling out and holding in memory is at least 200MB.
I''d love to be able to tell the CPython virtual machine that I need a
heap of, say 300MB up front rather than have it grow as needed. I''ve
had a scan through the archives of comp.lang.python and the python
docs but cannot find a way to do this. Is this possible to configure
the PVM this way?
Much appreciated,
Andy
--
推荐答案
Andy Watson写道：
Andy Watson wrote:
我有一个扫描并处理大量文本文件的应用程序。
我在内存中拉出并保存的内容至少为200MB。
我'我希望能够告诉CPython虚拟机我需要一个
堆，比如前面300MB，而不是根据需要增长。我已经通过comp.lang.python和python
文档的档案扫描了
，但找不到办法。这样可以用PVM配置
吗？
I have an application that scans and processes a bunch of text files.
The content I''m pulling out and holding in memory is at least 200MB.
I''d love to be able to tell the CPython virtual machine that I need a
heap of, say 300MB up front rather than have it grow as needed. I''ve
had a scan through the archives of comp.lang.python and the python
docs but cannot find a way to do this. Is this possible to configure
the PVM this way?
你为什么这么想？不，这是不可能的。说实话：我有
不知道为什么例如JVM允许这样做。
Diez
Why do you want that? And no, it is not possible. And to be honest: I have
no idea why e.g. the JVM allows for this.
Diez
你为什么要这样？不，这是不可能的。说实话：
我有
Why do you want that? And no, it is not possible. And to be honest:
I have
不知道为什么例如JVM允许这样做。
Diez
no idea why e.g. the JVM allows for this.
Diez
原因很简单就是我大致知道我有多少内存我需要付出
，并且cpython似乎需要花费相当多的时间
在我逐渐读取内容时扩展其堆。
Ta，
Andy
-
The reason why is simply that I know roughly how much memory I''m going
to need, and cpython seems to be taking a fair amount of time
extending its heap as I read in content incrementally.
Ta,
Andy
--
Andy Watson写道：
Andy Watson wrote:
你为什么要这样？不，这是不可能的。说实话：
Why do you want that? And no, it is not possible. And to be honest:
我有
I have
>不知道为什么例如JVM允许这样做。
>no idea why e.g. the JVM allows for this.
之所以这么简单，我大致知道我要花多少内存来支付
，而cpython似乎正在服用相当多的时间
在我逐渐读取内容时扩展其堆。
The reason why is simply that I know roughly how much memory I''m going
to need, and cpython seems to be taking a fair amount of time
extending its heap as I read in content incrementally.
我不是python malloc方案的专家，我知道_some_的东西很好地优化了，但是我很好不知道它在一般情况下做了一些巧妙的自我管理堆。无论如何，任何C扩展的存在都会很复杂。
然而，我怀疑你的观察是否正确。一个简单的
python -m timeit -n 1 -r 1"范围（50000000）
1循环，最好是1：2.38秒循环
将创建一个半个演出ram的python进程 - 一瞬间 - 而我
不要考虑2.38秒一个公平堆分配的时间量。
当我使用4倍大的参数时，我的机器开始交换。那么好了b $ b事情变得丑陋了 - 但是我没看到预分配会如何帮助那里......
Diez
I''m not an expert in python malloc schemes, I know that _some_ things are
heavily optimized, but I''m not aware that it does some clever
self-management of heap in the general case. Which would be complicated in
the presence of arbitrary C extensions anyway.
However, I''m having doubts that your observation is correct. A simple
python -m timeit -n 1 -r 1 "range(50000000)"
1 loops, best of 1: 2.38 sec per loop
will create a python-process of half a gig ram - for a split-second - and I
don''t consider 2.38 seconds a fair amount of time for heap allocation.
When I used a 4 times larger argument, my machine began swapping. THEN
things became ugly - but I don''t see how preallocation will help there...
Diez
这篇关于可以设置cpython堆大小？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
