如何减少Python脚本内存使用量 [英] How To Reduce Python Script Memory Usage
问题描述
我有一个非常大的python脚本，200K，我想使用尽可能少的内存.看起来像这样:
I have a very large python script, 200K, that I would like to use as little memory as possible. It looks something like:
# a lot of data structures
r = [34, 78, 43, 12, 99]
# a lot of functions that I use all the time
def func1(word):
return len(word) + 2
# a lot of functions that I rarely use
def func1(word):
return len(word) + 2
# my main loop
while 1:
# lots of code
# calls functions
如果我将很少使用的功能放在模块中，并仅在必要时动态导入它们，则无法访问数据.据我所知.
If I put the functions that I rarely use in a module, and import them dynamically only if necessary, I can't access the data. That's as far as I've gotten.
我是python的新手.
I'm new at python.
有人可以让我走上正确的道路吗?如何分解这个大脚本，以便使用更少的内存?是否值得将很少使用的代码放入模块中并仅在需要时调用它们?
Can anyone put me on the right path? How can I break this large script down so that it uses less memory? Is it worth putting rarely used code in modules and only calling them when needed?
推荐答案
整理:
您的python脚本似乎确实很大，也许您应该考虑首先重新组织代码，以分成几个
Your python script seems indeed to be huge, maybe you should consider reorganizing your code first, to split in into several modules or packages. It will probably make easier the code profiling and the optimization tasks.
您可能想在那看看:
- Python Project Howto
- Python Packages
- SO: Organising my Python project
还有可能:
- SO: Python: What is the common header format?
- How do you organize Python modules?
- The Hitchiker's Guide to Packaging
优化:
可以做很多事情来优化您的代码...
There is a lot of things that can be done for optimizing your code ...
例如，关于您的数据结构...如果您大量使用列表或列表理解，则可以尝试找出真正需要列表的地方，以及在哪些地方可以将它们替换为非可变数据结构例如元组或易失"对象，惰性"容器(例如生成器表达式).
For instance, regarding your data structures ... If you make a big use of lists or lists comprehensions, you could try to figure out where do you really need lists, and where they might be replaced by non-mutable data structures like tuples or by "volatile" objects, "lazy" containers, like generator expressions.
请参阅:
- SO: Are tuples more efficient than lists in Python?
- SO: Generator Expressions vs. List Comprehension
- PEP 255 - Simple Generators and PEP 289 - Generator Expressions
在这些页面上，您可以找到一些有用的信息和提示:
On these pages, you could find some useful information and tips:
- http://wiki.python.org/moin/PythonSpeed
- http://wiki.python.org/moin/PythonSpeed/PerformanceTips
- http://wiki.python.org/moin/TimeComplexity
- http://scipy.org/PerformancePython
- http://wiki.python.org/moin/PythonSpeed
- http://wiki.python.org/moin/PythonSpeed/PerformanceTips
- http://wiki.python.org/moin/TimeComplexity
- http://scipy.org/PerformancePython
此外，您应该研究自己的做事方式，并想知道是否有一种方法可以减少贪婪地做事，最好是用Python做到这一点(您可以在pythonic )...在Python中尤其如此，因为在Python中，通常有一种显而易见的"方式 (只有一个)做比其他事情更好的事情(请参阅 Python )，据说是 pythonic .它与代码的形状并没有特别的关系，但最重要的是与性能有关.与许多语言不同，它们提倡应该有很多方法来做任何事情，而Python则只专注于最好的方法.因此，很显然，有很多方法可以完成某项工作，但通常情况下，确实更好.
Also, you should study your ways for doing things and wonder whether there is a way to do that less greedily, a way that it's better to do it in Python (you will find some tips in the tag pythonic) ... That is especially true in Python, since in Python, there is often one "obvious" way (and only one) to do things which is better than the others (see The Zen of Python), which is said to be pythonic. It's not especially related to the shape of your code, but also - and above all - to the performances. Unlike a lot of languages, which promote the idea that there should be many ways for doing anything, Python prefers to focus on the best way only. So obviously, there are many ways for doing something, but often, one is really better.
现在，您还应该验证您是否在使用最佳方法来做事，因为pythonicality不会为您安排算法.
Now, you should also verify whether you are using the best methods for doing things, because pythonicality won't arrange your algorithms for you.
但是最后，这完全取决于您的代码，如果不看就很难回答.
But at last, it's very depending on your code it's difficult to answer without having seen it.
And, make sure to take into account the comments made by eumiro and Amr.
这篇关于如何减少Python脚本内存使用量的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
