如何在 pandas 上通过group by来应用累积自定义聚合函数 [英] How to apply an accumulative custom aggregation function with a group by on Pandas

问题描述

我有以下DataFrame

I have the following DataFrame

df = pd.DataFrame({'model': ['A0', 'A0', 'A1', 'A1','A0', 'A0', 'A1', 'A1', 'A0', 'A0', 'A1', 'A1'],
                    'y_true': [1, 2, 3, 3, 4, 5, 6, 7, 8, 9, 10, 11],
                    'y_pred': [0, 1, 5, 5, 7, 8, 8, 12, 8, 7, 14, 15],
                    'week': [1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3]},
                  )

   model  y_true  y_pred  week
0   A0       1       0     1
1   A0       2       1     1
2   A1       3       5     1
3   A1       3       5     1
4   A0       4       7     2
5   A0       5       8     2
6   A1       6       8     2
7   A1       7      12     2
8   A0       8       8     3
9   A0       9       7     3
10  A1      10      14     3
11  A1      11      15     3

而且我想用sklearn进行一些指标演算，所以我做了这个功能

And I want to make some metrics calculus with sklearn, so I did this function

from sklearn.metrics import mean_absolute_error, mean_squared_error, explained_variance_score
import numpy as np
def metrics(df):
    y_true=np.asarray(df['y_true'])
    y_pred=np.asarray(df['y_pred'])
    mae=mean_absolute_error(y_true, y_pred)
    mse=mean_squared_error(y_true, y_pred)
    evs=explained_variance_score(y_true, y_pred)
    return mae,mse,evs

我试图以此方式组队

df.groupby(['model', 'week']).apply(metrics)

它返回每周的指标，但是我希望指标从第1周到其他几周是累积的。我的意思是：

It returns me the metrics for every week, but I want the metrics to be accumulative since week 1 to the other weeks. I mean:

1. For the results of week 1 I want the metrics of y_true and y_pred where the column week takes the value 1.
2. For the results of week 2 I want the metrics of y_true and y_pred where the column week takes the values 1 or 2
3. For the results of week 3 I want the metrics of y_true and y_pred where the column week takes the values 1, 2 or 3

这是部分解决方案，但不是我想要的。

A partial solution is this, but is not what I want.

              y_true    y_pred                              y_true_cum  \
model week                                                               
A0    1       [1, 2]    [0, 1]                                  [1, 2]   
      2       [4, 5]    [7, 8]                            [1, 2, 4, 5]   
      3       [8, 9]    [8, 7]                      [1, 2, 4, 5, 8, 9]   
A1    1       [3, 3]    [5, 5]                [1, 2, 4, 5, 8, 9, 3, 3]   
      2       [6, 7]   [8, 12]          [1, 2, 4, 5, 8, 9, 3, 3, 6, 7]   
      3     [10, 11]  [14, 15]  [1, 2, 4, 5, 8, 9, 3, 3, 6, 7, 10, 11]   

我希望每个模型都有自己的累积周数：

I wanted every model to have his own accumulative weeks:

              y_true    y_pred                              y_true_cum  \
model week                                                               
A0    1       [1, 2]    [0, 1]                                  [1, 2]   
      2       [4, 5]    [7, 8]                            [1, 2, 4, 5]   
      3       [8, 9]    [8, 7]                      [1, 2, 4, 5, 8, 9]   
A1    1       [3, 3]    [5, 5]                                  [3, 3]   
      2       [6, 7]   [8, 12]                           [ 3, 3, 6, 7]   
      3     [10, 11]  [14, 15]                   [ 3, 3, 6, 7, 10, 11] 

推荐答案

应该执行以下操作：

import pandas as pd
from sklearn.metrics import mean_absolute_error, mean_squared_error, explained_variance_score

df = pd.DataFrame({
    'model': ['A0', 'A0', 'A1', 'A1','A0', 'A0', 'A1', 'A1', 'A0', 'A0', 'A1', 'A1'],
    'week': [1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3],
    'y_true': [1, 2, 3, 3, 4, 5, 6, 7, 8, 9, 10, 11],
    'y_pred': [0, 1, 5, 5, 7, 8, 8, 12, 8, 7, 14, 15]
})

def metrics(df):
    df['mae'] = mean_absolute_error(df.y_true, df.y_pred)
    df['mse'] = mean_squared_error(df.y_true, df.y_pred)
    df['evs'] = explained_variance_score(df.y_true, df.y_pred)
    return df


# groupby model, week and keep all values of y_true/y_pred as lists
df_group = df.groupby(['model', 'week']).agg(list)

# accumulate values for y_true and y_pred
df_group = df_group.groupby('model')['y_true', 'y_pred'].apply(lambda x: x.cumsum())

# apply metrics to new columns
df_group.apply(metrics, axis=1)

这篇关于如何在 pandas 上通过group by来应用累积自定义聚合函数的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！