Python pandas 中的GroupBy函数，例如SUM(col_1 * col_2)，加权平均值等 [英] GroupBy functions in Python Pandas like SUM(col_1*col_2), weighted average etc

问题描述

是否可以不使用

grouped.apply(lambda x: (x.a*x.b).sum()

使用速度快(不到我机器的一半)

It is much (less than half the time on my machine) faster to use

df['helper'] = df.a*df.b
grouped= df.groupby(something)
grouped['helper'].sum()
df.drop('helper', axis=1)

但是我真的不喜欢这样做. 例如，计算每组的加权平均值很有用.这里的lambda方法将是

But I don't really like having to do this. It is for example useful to compute the weighted average per group. Here the lambda approach would be

grouped.apply(lambda x: (x.a*x.b).sum()/(df.b).sum())

再一次比将帮助程序除以b.sum()慢得多.

and again is much slower than dividing the helper by b.sum().

推荐答案

我最终希望构建一个嵌入式数组表达式评估器(类固醇上的Numexpr)来执行此类操作.现在，我们正在处理Python的局限性-如果您实现了Cython聚合器来执行(x * y).sum()，则它可以与groupby连接，但是理想情况下，您可以将Python表达式编写为函数:

I want to eventually build an embedded array expression evaluator (Numexpr on steroids) to do things like this. Right now we're working with the limitations of Python-- if you implemented a Cython aggregator to do (x * y).sum() then it could be connected with groupby, but ideally you could write the Python expression as a function:

def weight_sum(x, y):
    return (x * y).sum()

，它将被"JIT编译"，并且大约和groupby(...).sum()一样快.我要描述的是一个非常重要的项目(每个月).如果有与BSD兼容的APL实现，我也许可以更快地完成上述操作(只是大声思考).

and that would get "JIT-compiled" and be about as fast as groupby(...).sum(). What I'm describing is a pretty significant (many month) project. If there were a BSD-compatible APL implementation I might be able to do something like the above quite a bit sooner (just thinking out loud).

这篇关于Python pandas 中的GroupBy函数，例如SUM(col_1 * col_2)，加权平均值等的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！