Python Pandas全局vs传递的变量 [英] Python Pandas global vs passed variable

问题描述

我正在创建一个实时"过程，该过程从SierraChart正在更新的专有格式OHLCVTBA文件中获取数据.在粘贴框上

I am creating a "real-time" process that takes data from a proprietary formatted OHLCVTBA file being updated by SierraChart. Code that reads the data and creates a dataframe using a generator is posted on pastebin

我已经意识到我的结构(新数据驱动)是错误的，我将对其进行重组. PhE的问题和Wes的回答带我去填补预填充的数据框效果很好.我的问题是:

I have realized that my structure (new data driven) is wrong and I'm about to reorganize it. PhE's question and Wes's response have taken me in the direction of filling a pre-populated dataframe which works well. My questions here are:

将数据框和指针保存为全局变量，或者将它们传递给使用它们的各种函数是否更快?另外，还有其他考虑因素应促使这一选择吗?

Is it faster to hold my dataframe and pointers as global variables or to pass them to and from the various functions that use them? Also, are there other considerations that should drive this choice?

谢谢.

推荐答案

在python中，局部变量比全局变量访问速度快.

在大熊猫的上下文中，这意味着您应该将变量传递到有意义的函数中(这意味着可以在函数内部更快地找到它们).相反，python中的函数调用非常昂贵(如果要多次调用它们的话)，这就是为什么numpy/pandas在可能的情况下使用矢量化函数的原因. 很显然，如果您在函数中执行操作，则必须小心确保所有计算都就位.

In the context of pandas, this means you should be passing variables into functions where this makes sense (it means they can be found quicker inside the function). Conversely, function calls in python are expensive (if you are calling them lots), which is why numpy/pandas use vectorised functions where possible. Obviously, you have to be careful to ensure all your calculations are done inplace if your doing things inside a function.

在担心速度之前，我通常会先以"pythonic"/"pandastic"方式使事情开始工作.然后使用%timeit来查看它是否已经足够快(通常是这样).添加一个单元测试.调整速度，%timeit，％prun和%timeit.如果这是一个大项目，请 vbench .

I would usually get things working first, in "pythonic"/"pandastic" way, before worrying about speed. Then use %timeit and see if it's fast enough already (usually it is). Add a unittest(s). Tweak for speed, %timeit, %prun and %timeit some more. If it's a big project vbench.

这篇关于Python Pandas全局vs传递的变量的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！