fmin_cg:由于精度损失而不一定实现所需的错误 [英] fmin_cg: Desired error not necessarily achieved due to precision loss

问题描述

我有以下代码可通过其梯度最小化成本函数".

I have the following code to minimize the Cost Function with its gradient.

def trainLinearReg( X, y, lamda ):
    # theta = zeros( shape(X)[1], 1 )
    theta = random.rand( shape(X)[1], 1 ) # random initialization of theta

    result = scipy.optimize.fmin_cg( computeCost, fprime = computeGradient, x0 = theta, 
                                     args = (X, y, lamda), maxiter = 200, disp = True, full_output = True )
    return result[1], result[0]

但是我有这个警告:

Warning: Desired error not necessarily achieved due to precision loss.
         Current function value: 8403387632289934651424768.000000
         Iterations: 0
         Function evaluations: 15
         Gradient evaluations: 3

我的computeCost和computeGradient被定义为

def computeCost( theta, X, y, lamda ):
    theta = theta.reshape( shape(X)[1], 1 )
    m     = shape(y)[0]
    J     = 0
    grad  = zeros( shape(theta) )

    h = X.dot(theta)
    squaredErrors = (h - y).T.dot(h - y)
    # theta[0] = 0.0
    J = (1.0 / (2 * m)) * (squaredErrors) + (lamda / (2 * m)) * (theta.T.dot(theta))

    return J[0]

def computeGradient( theta, X, y, lamda ):
    theta = theta.reshape( shape(X)[1], 1 )
    m     = shape(y)[0]
    J     = 0
    grad  = zeros( shape(theta) )

    h = X.dot(theta)
    squaredErrors = (h - y).T.dot(h - y)
    # theta[0] = 0.0
    J = (1.0 / (2 * m)) * (squaredErrors) + (lamda / (2 * m)) * (theta.T.dot(theta))
    grad = (1.0 / m) * (X.T.dot(h - y)) + (lamda / m) * theta

    return grad.flatten()

我已经审查了以下类似问题:

I have reviewed these similar questions:

scipy.optimize .fmin_bfgs:由于精度损失而不一定实现所需的误差"

scipy.optimize.fmin_cg :由于精度损失，不一定会达到所需的误差."

scipy没有优化并且返回由于精度损失而不一定实现的所需误差"

但是仍然无法解决我的问题.如何让最小化功能过程收敛而不是一开始陷入困境?

But still cannot have the solution to my problem. How to let the minimization function process converge instead of being stuck at first?

答案:

我根据以下@lejlot的评论解决此问题. 他是对的.数据集X太大，因为我没有正确地将正确的标准化值返回给正确的变量.即使这是一个小错误，它的确可以使您想到遇到此类问题时应该去哪里看.成本函数值太大会导致我的数据集出现错误.

I solve this problem based on @lejlot 's comments below. He is right. The data set X is to large since I did not properly return the correct normalized value to the correct variable. Even though this is a small mistake, it indeed can give you the thought where should we look at when encountering such problems. The Cost Function value is too large leads to the possibility that there are some wrong with my data set.

上一个错误的地方:

X_poly            = polyFeatures(X, p)
X_norm, mu, sigma = featureNormalize(X_poly)
X_poly            = c_[ones((m, 1)), X_poly]

正确的一个:

X_poly            = polyFeatures(X, p)
X_poly, mu, sigma = featureNormalize(X_poly)
X_poly            = c_[ones((m, 1)), X_poly]

其中X_poly实际上在以下训练中用作

where X_poly is actually used in the following traing as

cost, theta = trainLinearReg(X_poly, y, lamda)

推荐答案

答案:

我根据以下@lejlot的评论解决此问题. 他是对的.数据集X太大，因为我没有正确地将正确的标准化值返回给正确的变量.即使这是一个小错误，它的确可以使您想到遇到此类问题时应该去哪里看.成本函数值太大会导致我的数据集出现错误.

I solve this problem based on @lejlot 's comments below. He is right. The data set X is to large since I did not properly return the correct normalized value to the correct variable. Even though this is a small mistake, it indeed can give you the thought where should we look at when encountering such problems. The Cost Function value is too large leads to the possibility that there are some wrong with my data set.

上一个错误的地方:

X_poly            = polyFeatures(X, p)
X_norm, mu, sigma = featureNormalize(X_poly)
X_poly            = c_[ones((m, 1)), X_poly]

正确的一个:

X_poly            = polyFeatures(X, p)
X_poly, mu, sigma = featureNormalize(X_poly)
X_poly            = c_[ones((m, 1)), X_poly]

其中X_poly实际上在以下训练中用作

where X_poly is actually used in the following traing as

cost, theta = trainLinearReg(X_poly, y, lamda)

这篇关于fmin_cg:由于精度损失而不一定实现所需的错误的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！