用于线性回归的mxnet梯度下降，变量类型错误 [英] mxnet gradient descent for linear regression, variable types error

问题描述

我正在尝试为线性回归实现简单的梯度下降.

I'm trying to implement a simple gradient descent for linear regression.

如果我手动(使用解析表达式)计算梯度，则它可以正常工作，但是现在我正尝试通过mxnet模块中的autograd实现它.

It works normally if I compute the gradient manually (by using the analytical expression), but now i was trying to implement it with autograd from the mxnet module.

这是代码

from mxnet import autograd, np, npx
npx.set_np()

def main():
    # learning algorithm parameters
    nr_epochs = 1000
    alpha = 0.01

    # read data, insert column of ones (to include bias with other parameters)
    data = pd.read_csv("dataset.txt", header=0, index_col=None, sep="\s+")
    data.insert(0, "x_0", 1, True)  # insert column of "1"s as x_0
    m = data.shape[0]  # number of samples
    n = data.shape[1] - 1  # number of features
    X = data.iloc[:, 0:n].values  # array with x values
    Y = data.iloc[:, -1].values  # array with y values

    theta = np.zeros(n)  # initial parameters array
    theta.attach_grad()

    theta, J = GradientDescent(X, Y, theta, alpha, nr_epochs)



#-------------------#
#   loss function   #
#-------------------#
def LossFunction(X, Y, theta):
    m = X.shape[0]                  # number of training samples
    loss = 0

    for i in range(X.shape[0]):
        loss = loss + (1 / (2 * m)) * (H(X[i, :], theta) - Y[i]) ** 2
    return loss


#----------------#
#   hypothesis   #
#----------------#
def H(x, theta):
    return np.dot(x, theta)


#----------------------#
#   gradient descent   #
#----------------------#
def GradientDescent(X, Y, theta, alpha, nr_epochs):

    m = X.shape[0]
    n = X.shape[1]
    grad = np.zeros(n)   

    Loss = np.zeros(nr_epochs)          


    for epoch in range(nr_epochs):
        with autograd.record():
            Loss[epoch] = LossFunction(X, Y, theta)

        Loss[epoch].backward()

        for j in range(n):
            theta[j] = theta[j] - alpha * theta.grad[j]
        return theta, Loss


if __name__ == "__main__":
    main()

问题是当代码计算假设时，在X和theta之间的点积中出现错误

The problem is that I get an error when the code is computing the hypothesis, in the dot product between X and theta

return np.dot(x, theta)

错误消息显示: 参数a必须具有NDArray类型，但得到[1. -5.05358]

The error message says: Argument a must have NDArray type, but got [ 1. -5.05358]

因此，我假设x和theta的类型之间一定存在不兼容性. 我检查了一下，然后得到了

So, I'm assuming there must be some incompatibility between the types of x and theta. I checked them, and I got:

X -> <class 'numpy.ndarray'>
theta -> <class 'mxnet.numpy.ndarray'>

theta是用np.zeros创建的，所以它是一个mxnet数组，而X是使用.values方法从数据集转换而来的...这是问题的根源吗? 谢谢！

theta is created with np.zeros, so it is a mxnet array, while X is converted from a dataset with the .values method...is this the source of the problem? Thanks!

推荐答案

MXNet不使用Numpy NDArray，但是自动区分.它也可以在CPU上工作，在CPU上通常比默认(由OpenBLAS支持)的Numpy更快.

MXNet doesn't uses Numpy NDArray, but mxnet NDArray, which has very similar functionality and API but a different backend; mxnet NDArray is written in C++, uses asynchronous execution, is GPU-compatible and supports automatic differentiation. It also works on CPU, where it usually faster than default (OpenBLAS-backed) Numpy.

因此，要解决您的错误，我建议确保在代码中不要使用numpy，而应在所有地方使用mxnet NDArray.实际上，更改非常容易，因为该API与numpy非常相似.并且，如果需要，您可以与numpy进行相互转换，例如:

So to fix your error I recommend to make sure you don't use numpy in your code, but mxnet NDArray everywhere. It is actually very easy to change because the API is super similar to numpy. And if need be, you can convert to and from numpy, for example:

from mxnet import nd

# Assuming A is an numpy ndarray and B an mxnet ndarray

# from numpy to mxnet
mxnet_array = nd.array(A)


# from mxnet to numpy
np_array = B.asnumpy()

关于您对线性回归的特定兴趣，请参见此处2个python的mxnet演示:

Regarding your specific interest in linear regression, see here 2 mxnet demos in python:

• 从零开始在MXNet中进行线性回归
• 使用胶子在MXNet中进行线性回归(胶子是python命令式前端的名称，有点像TF的keras一样)

• Linear regression in MXNet from scratch
• Linear regression in MXNet with gluon (gluon is the name of the python imperative frontend, a bit like what keras is to TF)

使用这些NDArray是MXNet如此之快的原因之一，因为它使您的代码完全异步，并允许引擎查找优化. 那些NDArrays是使MXNet如此出色的原因之一，尝试一下，您就会爱上它们:)

Using those NDArrays is one of the reasons MXNet is so fast, because it makes your code fully asynchronous and lets the engine find optimizations. Those NDArrays is one of the things that make MXNet so awesome, try them and you'll love them :)

这篇关于用于线性回归的mxnet梯度下降，变量类型错误的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！