如何平滑仅在某些部分具有大噪声的曲线? [英] How to smooth a curve with large noise which is only in certain part?

问题描述

我想平滑如下所示的散点图(点非常密集)，数据为

I'd like to smooth a scatter plot shown below (the points are very dense), and the data is here.

曲线中间有很大的噪声，我想使曲线平滑，并且y值也应该单调增加.

There is large noise in the middle of the curve, and I'd like to smooth the curve, also the y value should monotonically increase.

由于有很多这样的曲线，因此很难知道噪声在曲线中的什么位置.

Since there are lots of curves like this, it is kind of hard to know where the noise is in the curve.

我尝试了scipy.signal.savgol_filter，但是没有用.

I tried scipy.signal.savgol_filter, but it didn't work.

我使用的代码是:

from scipy.signal import savgol_filter
from scipy import interpolate
import numpy as np
import matplotlib.pyplot as plt

s = np.loadtxt('data.csv', delimiter=',')
x = s[:, 0]
y = s[:, 1]
yhat = savgol_filter(y, 551, 3)

plt.plot(x, y, 'r')
plt.plot(x, yhat, 'b')
plt.show()

我们非常感谢您的建议.谢谢！

Suggestions are really appreciated. Thanks!

-------------------更新--------------------- ----

-------------------update-------------------------

按照Colin的方法，我得到了想要的结果.这是代码:

Following Colin's method, I get the results I want. Here is the code:

from scipy.signal import savgol_filter
from scipy import interpolate
import numpy as np
import matplotlib.pyplot as plt

s = np.loadtxt('data.csv', delimiter=',')
x = s[:, 0]
y = s[:, 1]
yhat = savgol_filter(y, 551, 3)

tolerance = 0.2
increased_span = 150
filter_size = 11

first_pass = medfilt(y,filter_size)
diff = (y-first_pass)**2
first = np.argmax(diff>tolerance) - increased_span
last = len(y) - np.argmax(diff[::-1]>tolerance) + increased_span
print (first, last)
#interpolate between increased span
yhat[first:last] = np.interp(x[first:last], [x[first], x[last]],  [y[first], y[last]])


f = interpolate.interp1d(x, yhat, kind='slinear')
x_inter = np.linspace(x[0], x[-1], 1000)
y_inter = f(x_inter)
y_inter = savgol_filter(y_inter, 41, 3)

plt.plot(x, y, 'r')
plt.plot(x, yhat, 'b')
plt.show()

推荐答案

如果我们首先隔离故障区域，则有很多方法可以将其消除.这是一个示例:

If we firstly isolate the trouble area there are many ways to remove it. Here is an example:

tolerance = 0.2
increased_span = 150
filter_size = 11

#find noise
first_pass = medfilt(y,filter_size)
diff = (yhat-first_pass)**2
first = np.argmax(diff>tolerance) - increased_span
last = len(y) - np.argmax(diff[::-1]>tolerance) + increased_span

#interpolate between increased span
yhat[first:last] = np.interp(x[first:last], [x[first], x[last]],  [y[first], y[last]])

这篇关于如何平滑仅在某些部分具有大噪声的曲线?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！