了解 torch.nn.Parameter [英] Understanding torch.nn.Parameter

问题描述

我是 pytorch 的新手，我很难理解 torch.nn.Parameter() 的工作原理.

I am new to pytorch and I have difficulty in understanding how torch.nn.Parameter() works.

我已经阅读了 https://pytorch.org/docs/stable/nn 中的文档.html 但可能没有什么意义.

I have gone through the documentation in https://pytorch.org/docs/stable/nn.html but could make a very little sense out of it.

有人可以帮忙吗?

我正在处理的代码片段:

The code snippet that I am working on:

def __init__(self, weight):
    super(Net, self).__init__()
    # initializes the weights of the convolutional layer to be the weights of the 4 defined filters
    k_height, k_width = weight.shape[2:]
    # assumes there are 4 grayscale filters
    self.conv = nn.Conv2d(1, 4, kernel_size=(k_height, k_width), bias=False)
    self.conv.weight = torch.nn.Parameter(weight)

推荐答案

我会为你分解.您可能知道，张量是多维矩阵.原始形式的参数是张量，即多维矩阵.它是 Variable 类的子类.

I will break it down for you. Tensors, as you might know, are multi dimensional matrices. Parameter, in its raw form, is a tensor i.e. a multi dimensional matrix. It sub-classes the Variable class.

当与模块关联时，变量和参数之间的区别就出现了.当参数作为模型属性与模块相关联时，它会自动添加到参数列表中，并且可以使用参数"迭代器进行访问.

The difference between a Variable and a Parameter comes in when associated with a module. When a Parameter is associated with a module as a model attribute, it gets added to the parameter list automatically and can be accessed using the 'parameters' iterator.

最初在 Torch 中，变量(例如可以是中间状态)也将在分配时作为模型的参数添加.后来确定了需要缓存变量而不是将它们添加到参数列表的用例.

Initially in Torch, a Variable (which could for example be an intermediate state) would also get added as a parameter of the model upon assignment. Later on there were use cases identified where a need to cache the variables instead of having them added to the parameter list was identified.

文档中提到的一种情况是 RNN，其中您需要保存最后一个隐藏状态，这样您就不必一次又一次地传递它.需要缓存一个变量而不是让它自动注册为模型的参数，这就是为什么我们有一种显式的方式将参数注册到我们的模型，即 nn.Parameter 类.

One such case, as mentioned in the documentation is that of RNN, where in you need to save the last hidden state so you don't have to pass it again and again. The need to cache a Variable instead of having it automatically register as a parameter to the model is why we have an explicit way of registering parameters to our model i.e. nn.Parameter class.

例如，运行以下代码 -

For instance, run the following code -

import torch
import torch.nn as nn
from torch.optim import Adam

class NN_Network(nn.Module):
    def __init__(self,in_dim,hid,out_dim):
        super(NN_Network, self).__init__()
        self.linear1 = nn.Linear(in_dim,hid)
        self.linear2 = nn.Linear(hid,out_dim)
        self.linear1.weight = torch.nn.Parameter(torch.zeros(in_dim,hid))
        self.linear1.bias = torch.nn.Parameter(torch.ones(hid))
        self.linear2.weight = torch.nn.Parameter(torch.zeros(in_dim,hid))
        self.linear2.bias = torch.nn.Parameter(torch.ones(hid))

    def forward(self, input_array):
        h = self.linear1(input_array)
        y_pred = self.linear2(h)
        return y_pred

in_d = 5
hidn = 2
out_d = 3
net = NN_Network(in_d, hidn, out_d)

现在，检查与此模型关联的参数列表-

Now, check the parameter list associated with this model -

for param in net.parameters():
    print(type(param.data), param.size())

""" Output
<class 'torch.FloatTensor'> torch.Size([5, 2])
<class 'torch.FloatTensor'> torch.Size([2])
<class 'torch.FloatTensor'> torch.Size([5, 2])
<class 'torch.FloatTensor'> torch.Size([2])
"""

或者试试，

list(net.parameters())

这可以很容易地提供给您的优化器 -

This can easily be fed to your optimizer -

opt = Adam(net.parameters(), learning_rate=0.001)

另外，注意参数默认设置了require_grad.

Also, note that Parameters have require_grad set by default.

这篇关于了解 torch.nn.Parameter的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！