在此用于正则化线性回归的Python代码中有什么问题? [英] What is wrong in this Python code for Regularized Linear Regression?

问题描述

我用numpy(theta，X是numpy数组)编写了代码:

I wrote code with numpy(theta, X is numpy array):

def CostRegFunction(X, y, theta, lambda_):
    m = len(X)
    # add bias unit
    X = np.concatenate((np.ones((m,1)),X),1)

    H = np.dot(X,theta)

    J = (1 / (2 * m)) * (np.sum([(H[i] - y[i][0])**2 for i in range(len(H))])) + (lambda_ / (2 * m)) * np.sum(theta[1:]**2)

    grad_ = list()

    grad_.append((1 / m) * np.sum([(H[j] - y[j][0]) for j in range(len(H))]))
    for i in range(len(theta)-1):
        grad_.append((1 / m) * np.sum([(H[j] - y[j]) * X[j][i+1] for j in range(len(H))]) + (lambda_ / m) * theta[i+1])

    return J, grad_



def TrainLinearReg(X, y, theta, lambda_, alpha, iter):

    JHistory = list()
    for i in range(iter):
        J, grad = CostRegFunction(X, y, theta, Lambda_)
        JHistory.append(J)
        for j in range(len(theta)):
            theta[j] = theta[j] - alpha * grad[j]

    return theta, JHistory

Theta, JH = TrainLinearReg(X, y, th, Lambda_, 0.01, 50)

但是当我尝试学习theta时，这段代码确实给了我极大的theta和J值的增长.例如，第一次迭代grad = [-15.12452，598.435436]-这是正确的.J是303.3255第2次迭代-grad = [10.23566，-3646.2345] J = 7924等等，J的增长越来越快，但是就LR而言，它必须更低.

But when I try learn theta this code gives me a realy huge grow of theta and value of J. For example first iteration grad = [-15.12452, 598.435436] - it is correct. J is 303.3255 2nd iteration - grad = [10.23566,-3646.2345] J = 7924 and so on J grows faster and faster but on idea of LR it must be lower.

但是，如果我使用法线线性方程，则会得到一个好的Theta.

But if I use Normal Linear Equation in gives me a good Theta.

该代码有什么问题?

推荐答案

numpy版本

.10月17日修改

numpy version

. edited on Oct 17th

我使用numpy库重写了部分代码.

I rewrote parts of the code using numpy libraries.

所有向量现在都是列numpy数组.

All vectors are now columns numpy arrays.

import numpy as np
from copy import deepcopy as dc
from matplotlib import pyplot as plt

_norm = np.linalg.norm    

def CostRegFunction(X, y, theta, lambda_):
    m = len(X)
    H = np.dot(X,theta)
    J = (1 / (2 * m)) * _norm(H-y)**2 + (lambda_ / (2 * m)) * _normal(theta[1:])**2
    grad_ = np.array(sum(H-y)/m,ndmin=2).T
    for i in range(theta.shape[0]-1):
        grad_=np.concatenate((grad_,np.array(sum((H-y)*np.array(X[:,1],ndmin=2).T)/m + (lambda_/m) * theta[i+1],ndmin=2).T),0)
    return J, grad_

def TrainLinearReg(X, y, theta, lambda_, alpha, iter):
    JHistory = list()
    # add bias unit -> it's better to do it here, before entering the loop
    X = np.concatenate((np.ones((X.shape[0],1)),X),1)
    for i in range(iter):
        J, grad = CostRegFunction(X, y, theta, lambda_)
        JHistory.append(J)
        theta = theta -  alpha*grad
    return theta, JHistory

然后，我用白噪声生成了一组简单的x-y多项式数据，并使用 TrainLinearReg 函数拟合了多项式方程.

Then I generated a simple set of x-y polynomial data with white noise and fitted the polynom-equation using the TrainLinearRegfunction.

x = np.concatenate((np.array(np.linspace(0,10,100) + np.random.normal(0,0.01,100),ndmin=2).T,\
np.array(np.linspace(0,10,100)**2 + np.random.normal(0,0.01,100),ndmin=2).T),1)
y = 2 + -3*np.array(x[:,0],ndmin=2).T + np.random.normal(0,3,[100,1]) - 2*np.array(x[:,1],ndmin=2).T
th = np.array([1,2,3],ndmin=2).T
alpha = 0.001
lambda_ = 0.1
Theta, JH = TrainLinearReg(x, y, dc(th), lambda_, alpha, 10000)

我得到的是以下内容.

plt.plot(x[:,0],y,'o',label='Original Data',alpha = 0.5)
x2 = np.linspace(0,10,10)
plt.plot(x2,Theta[0]+x2*Theta[1]+x2**2*Theta[2],'-',label='Fitted     Curve',lw=1.5,alpha=0.8,color='black')
plt.gca().set_xlabel('x')
plt.gca().set_ylabel('y')
plt.legend()

Output >> Theta = array([[ 1.29259285],
                         [-2.97763304],
                         [-1.98758321]])

希望我能有所帮助.

最诚挚的问候，加布里埃尔

Best regards, Gabriel

这篇关于在此用于正则化线性回归的Python代码中有什么问题?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！