如何减少 Python 脚本内存使用 [英] How To Reduce Python Script Memory Usage
问题描述
我有一个非常大的 python 脚本，200K，我想使用尽可能少的内存.它看起来像:
I have a very large python script, 200K, that I would like to use as little memory as possible. It looks something like:
# a lot of data structures
r = [34, 78, 43, 12, 99]
# a lot of functions that I use all the time
def func1(word):
return len(word) + 2
# a lot of functions that I rarely use
def func1(word):
return len(word) + 2
# my main loop
while 1:
# lots of code
# calls functions
如果我把很少用到的函数放在一个模块中，只在需要的时候动态导入，我就无法访问数据.这就是我得到的.
If I put the functions that I rarely use in a module, and import them dynamically only if necessary, I can't access the data. That's as far as I've gotten.
我是 Python 新手.
I'm new at python.
谁能让我走上正确的道路?我怎样才能分解这个大脚本以减少它使用的内存?是否值得将很少使用的代码放在模块中并仅在需要时调用它们?
Can anyone put me on the right path? How can I break this large script down so that it uses less memory? Is it worth putting rarely used code in modules and only calling them when needed?
推荐答案
组织:
你的python脚本看起来确实很大，也许你应该先考虑重新组织你的代码，把它分成几个模块或包.它可能会使代码分析和优化任务变得更容易.
Your python script seems indeed to be huge, maybe you should consider reorganizing your code first, to split it into several modules or packages. It will probably make easier the code profiling and the optimization tasks.
你可能想看看那里:
还有可能:
优化:
可以做很多事情来优化您的代码......
There is a lot of things that can be done for optimizing your code ...
例如，关于您的数据结构......如果您大量使用列表或列表推导式，您可以尝试找出您真正需要列表的位置，以及它们可能被非可变数据结构替换的位置，例如元组或易失性"对象，懒惰的"容器，如生成器表达式.
For instance, regarding your data structures ... If you make big use of lists or lists comprehensions, you could try to figure out where do you really need lists, and where they might be replaced by non-mutable data structures like tuples or by "volatile" objects, "lazy" containers, like generator expressions.
见:
在这些页面上，您可以找到一些有用的信息和提示:
On these pages, you could find some useful information and tips:
- http://wiki.python.org/moin/PythonSpeed
- http://wiki.python.org/moin/PythonSpeed/PerformanceTips
- http://wiki.python.org/moin/TimeComplexity
- http://scipy.org/PerformancePython
另外，你应该研究你的做事方式，想知道是否有一种方法可以不那么贪婪地做到这一点，一种最好在 Python 中做到的方法(你会在标签 pythonic) ...在 Python 中尤其如此，因为在 Python 中，通常一个明显的"方法(并且只有一种)做比其他人更好的事情(见 Python 之禅)，据说是 pythonic.它与您的代码的形状并没有特别的关系，而且 - 最重要的是 - 与性能有关.与许多语言不同，它提倡应该有多种方法来做任何事情，Python 更喜欢只关注最好的方法.很明显，有很多方法可以做某事，但通常，其中一种确实更好.
Also, you should study your ways of doing things and wonder whether there is a way to do that less greedily, a way that it's better to do it in Python (you will find some tips in the tag pythonic) ... That is especially true in Python, since, in Python, there is often one "obvious" way (and only one) to do things which are better than the others (see The Zen of Python), which is said to be pythonic. It's not especially related to the shape of your code, but also - and above all - to the performances. Unlike many languages, which promote the idea that there should be many ways to do anything, Python prefers to focus on the best way only. So obviously, there are many ways for doing something, but often, one is really better.
现在，您还应该验证您是否使用了最好的做事方法，因为 Pythonicity 不会为您安排算法.
Now, you should also verify whether you are using the best methods for doing things because pythonicality won't arrange your algorithms for you.
但最后，它因您的代码而异，如果没有看到它很难回答.
But at last, it varies depending on your code and it's difficult to answer without having seen it.
And, make sure to take into account the comments made by eumiro and Amr.
这篇关于如何减少 Python 脚本内存使用的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
