使用滚动中值过滤掉 Pandas 数据框中的异常值 [英] Filtering out outliers in Pandas dataframe with rolling median

问题描述

我正在尝试从带有日期的 GPS 高程位移散点图中过滤掉一些异常值

I am trying to filter out some outliers from a scatter plot of GPS elevation displacements with dates

我正在尝试使用 df.rolling 来计算每个窗口的中值和标准差，然后在它大于 3 个标准差时删除该点.

I'm trying to use df.rolling to compute a median and standard deviation for each window and then remove the point if it is greater than 3 standard deviations.

但是，我想不出一种方法来遍历列并比较计算出的滚动中值.

However, I can't figure out a way to loop through the column and compare the the median value rolling calculated.

这是我目前的代码

import pandas as pd
import numpy as np

def median_filter(df, window):
    cnt = 0
    median = df['b'].rolling(window).median()
    std = df['b'].rolling(window).std()
    for row in df.b:
      #compare each value to its median




df = pd.DataFrame(np.random.randint(0,100,size=(100,2)), columns = ['a', 'b'])

median_filter(df, 10)

如何遍历并比较每个点并将其删除?

How can I loop through and compare each point and remove it?

推荐答案

只过滤数据框

df['median']= df['b'].rolling(window).median()
df['std'] = df['b'].rolling(window).std()

#filter setup
df = df[(df.b <= df['median']+3*df['std']) & (df.b >= df['median']-3*df['std'])]

这篇关于使用滚动中值过滤掉 Pandas 数据框中的异常值的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！