高斯过程scikit学习-异常 [英] Gaussian Process scikit-learn - Exception

问题描述

我想使用高斯过程来解决回归任务.我的数据如下:每个X向量的长度为37，每个Y向量的长度为8.

我在Python中使用sklearn软件包，但是尝试使用高斯过程会导致Exception:

from sklearn import gaussian_process

print "x :", x__
print "y :", y__

gp = gaussian_process.GaussianProcess(theta0=1e-2, thetaL=1e-4, thetaU=1e-1)
gp.fit(x__, y__) 

x:[[136. 137. 137. 132. 130. 130. 132. 133. 134.
135. 135. 134. 134. 1139. 1019. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 70. 24. 55. 0. 9. 0. 0.] [136. 137. 137. 132. 130. 130. 132. 133. 134. 135. 135. 134. 134. 1139. 1019. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 70. 24. 55. 0. 9. 0. 0.] [82. 76. 80. 103. 135. 155. 159. 156. 145. 138. 130. 122. 122. 689. 569. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] [156. 145. 138. 130. 122. 118. 113. 111. 105. 101. 98. 95. 95. 759. 639. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] [112. 111. 111. 114. 114. 113. 114. 114. 112. 111. 109. 109. 109. 1109. 989. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] [133. 130. 125. 124. 124. 123. 103. 87. 96. 121. 122. 123. 123. 399. 279. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] [104. 109. 111. 106. 91. 86. 117. 123. 123. 120. 121. 115. 115. 549. 429. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] [144. 138. 126. 122. 119. 118. 116. 114. 107. 105. 106. 119. 119. 479. 359. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]]]

y:[[7. 9. 13. 30. 34. 37. 36. 41.] [7. 9. 13. 30. 34. 37. 36. 41.] [-4. -9. -17. -21. -27. -28. -28. -20. ] [-1. -1. -4. -5. 20. 28. 31. 23.] [-1. -2. -3. -1. -4. -7. 8. 58.] [-1. -2. -14.33333333 -14. -13.66666667 -32. -26.66666667 -1. ] [1. 3.33333333 0. -0.66666667 3. 6. 22. 54.] [-2. -8. -11. -17. -17. -16. -16. -23. ]]

--------------------------------------------------- ----------------------------异常回溯(最近的呼叫 最后)在() 11 gp = gaussian_process.GaussianProcess(theta0 = 1e-2，thetaL = 1e-4，thetaU = 1e-1) 12 ---> 13 gp.fit(x__，y __)

/usr/local/lib/python2.7/site-packages/sklearn/gaussian_process/gaussian_process.pyc 适合(自己，X，y) 300如果(np.min(np.sum(D，轴= 1))== 0. 301和self.corr！= correlation.pure_nugget): -> 302引发Exception(多个输入要素不能具有相同的要素" 303目标值".) 304

例外:多个输入要素不能具有相同的目标值.

我发现了与scikit-learn问题相关的一些主题，但是我的版本已是最新.

解决方案

众所周知问题，但实际上尚未解决.

发生这种情况是因为，如果您具有相同的点，则矩阵是不可逆的(奇异的).(这意味着您无法计算A ^ -1-这是GP解决方案的一部分).

为了解决该问题，只需在示例中添加一些小的高斯噪声或使用其他 GP 库

您总是可以尝试实现它，实际上并不难. GP中最重要的是您的内核功能，例如高斯内核:

exponential_kernel = lambda x, y, params: params[0] * \
    np.exp( -0.5 * params[1] * np.sum((x - y)**2) )

现在，我们需要构建协方差矩阵，如下所示:

covariance = lambda kernel, x, y, params: \
    np.array([[kernel(xi, yi, params) for xi in x] for yi in y])

因此，当您要预测新点x时，请计算其协方差:

sigma1 = covariance(exponential_kernel, x, x, theta)

并应用以下内容:

def predict(x, data, kernel, params, sigma, t):
    k = [kernel(x, y, params) for y in data]
    Sinv = np.linalg.inv(sigma)
    y_pred = np.dot(k, Sinv).dot(t)
    sigma_new = kernel(x, x, params) - np.dot(k, Sinv).dot(k)
    return y_pred, sigma_new

这是非常幼稚的实现，对于高维度的数据，运行时间会很高.此处最难计算的是Sinv = np.linalg.inv(sigma)，它需要O(N^3).

I want to use Gaussian Processes to solve a regression task. My data is as follow : each X vector has a length of 37, and each Y vector has a length of 8.

I'm using the sklearnpackage in Python but trying to use gaussian processes leads to an Exception:

from sklearn import gaussian_process

print "x :", x__
print "y :", y__

gp = gaussian_process.GaussianProcess(theta0=1e-2, thetaL=1e-4, thetaU=1e-1)
gp.fit(x__, y__) 

x : [[ 136. 137. 137. 132. 130. 130. 132. 133. 134.
135. 135. 134. 134. 1139. 1019. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 70. 24. 55. 0. 9. 0. 0.] [ 136. 137. 137. 132. 130. 130. 132. 133. 134. 135. 135. 134. 134. 1139. 1019. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 70. 24. 55. 0. 9. 0. 0.] [ 82. 76. 80. 103. 135. 155. 159. 156. 145. 138. 130. 122. 122. 689. 569. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] [ 156. 145. 138. 130. 122. 118. 113. 111. 105. 101. 98. 95. 95. 759. 639. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] [ 112. 111. 111. 114. 114. 113. 114. 114. 112. 111. 109. 109. 109. 1109. 989. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] [ 133. 130. 125. 124. 124. 123. 103. 87. 96. 121. 122. 123. 123. 399. 279. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] [ 104. 109. 111. 106. 91. 86. 117. 123. 123. 120. 121. 115. 115. 549. 429. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] [ 144. 138. 126. 122. 119. 118. 116. 114. 107. 105. 106. 119. 119. 479. 359. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]]

y : [[ 7. 9. 13. 30. 34. 37. 36. 41. ] [ 7. 9. 13. 30. 34. 37. 36. 41. ] [ -4. -9. -17. -21. -27. -28. -28. -20. ] [ -1. -1. -4. -5. 20. 28. 31. 23. ] [ -1. -2. -3. -1. -4. -7. 8. 58. ] [ -1. -2. -14.33333333 -14. -13.66666667 -32. -26.66666667 -1. ] [ 1. 3.33333333 0. -0.66666667 3. 6. 22. 54. ] [ -2. -8. -11. -17. -17. -16. -16. -23. ]]

--------------------------------------------------------------------------- Exception Traceback (most recent call last) in () 11 gp = gaussian_process.GaussianProcess(theta0=1e-2, thetaL=1e-4, thetaU=1e-1) 12 ---> 13 gp.fit(x__, y__)

/usr/local/lib/python2.7/site-packages/sklearn/gaussian_process/gaussian_process.pyc in fit(self, X, y) 300 if (np.min(np.sum(D, axis=1)) == 0. 301 and self.corr != correlation.pure_nugget): --> 302 raise Exception("Multiple input features cannot have the same" 303 " target value.") 304

Exception: Multiple input features cannot have the same target value.

I've found some topics related to a scikit-learn issue, but my version is up-to-date.

解决方案

It is known issue and it still has not actually been resolved.

It is happens, because if you have same points , your matrix is not invertible(singular).(meaning you cannot calculate A^-1 - which is part of solution for GP).

In order to solve it, just add some small gaussian noise to your examples or use other GP library.

You can always try to implement it, it is actually not that hard. The most important thing in GP is your kernel function, for example gaussian kernel:

exponential_kernel = lambda x, y, params: params[0] * \
    np.exp( -0.5 * params[1] * np.sum((x - y)**2) )

Now, we need to build covariance matrix, like this:

covariance = lambda kernel, x, y, params: \
    np.array([[kernel(xi, yi, params) for xi in x] for yi in y])

So, when you want to predict new point x calculate its covariance:

sigma1 = covariance(exponential_kernel, x, x, theta)

and apply following:

def predict(x, data, kernel, params, sigma, t):
    k = [kernel(x, y, params) for y in data]
    Sinv = np.linalg.inv(sigma)
    y_pred = np.dot(k, Sinv).dot(t)
    sigma_new = kernel(x, x, params) - np.dot(k, Sinv).dot(k)
    return y_pred, sigma_new

This is very naive implementation and for data with high dimensions, runtime will be high. Hardest thing to calculate here is Sinv = np.linalg.inv(sigma) which takes O(N^3).

这篇关于高斯过程scikit学习-异常的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！