pandas -遍历行并计算-更快 [英] pandas - iterate over rows and calculate - faster

问题描述

我已经有一个解决方案-但是速度非常慢(800行需要13分钟).这是数据框的示例:

I already have a solution -but it is very slow (13 minutes for 800 rows). here is an example of the dataframe:

import pandas as pd
d = {'col1': [20,23,40,41,48,49,50,50], 'col2': [39,32,42,50,63,68,68,69]}
df = pd.DataFrame(data=d)
df

在新列中，我要计算col2的先前值(例如三个)有多少大于或等于col1的行值.我也继续第一行.

In a new column, I want to calculate how many of the previous values (for example three)of col2 are greater or equal than row-value of col1. i also continue the first rows.

这是我的慢代码:

start_at_nr = 3 #variable in which row start to calculate
df["overlap_count"] = "" #create new column

for row in range(len(df)):
    if row <= start_at_nr - 1:
       df["overlap_count"].loc[row] = "x"
    else:
       df["overlap_count"].loc[row] = (
           df["col2"].loc[row - start_at_nr:row - 1] >=
           (df["col1"].loc[row])).sum()

df

我获得了更快的解决方案-谢谢您的宝贵时间！

i obtain a faster solution - thank you for your time!

这是我获得的结果:

col1    col2    overlap_count
0   20  39  x
1   23  32  x
2   40  42  x
3   41  50  1
4   48  63  1
5   49  68  2
6   50  68  3
7   50  69  3

推荐答案

IIUC，您可以这样做:

IIUC, you can do:

df['overlap_count'] = 0
for i in range(1,start_at_nr+1):
    df['overlap_count'] += df['col1'].le(df['col2'].shift(i))

# mask the first few rows
df.iloc[:start_at_nr, -1] = np.nan

输出:

   col1  col2  overlap_count
0    20    39            NaN
1    23    32            NaN
2    40    42            NaN
3    41    50            1.0
4    48    63            1.0
5    49    68            2.0
6    50    68            3.0
7    50    69            3.0

在800行和start_at_nr=3上花费大约11ms的时间.

Takes about 11ms on for 800 rows and start_at_nr=3.

这篇关于 pandas -遍历行并计算-更快的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！