找不到原位运算:原位运算已修改了梯度计算所需的变量之一 [英] can't find the inplace operation: one of the variables needed for gradient computation has been modified by an inplace operation

问题描述

我正在尝试计算网络的雅可比损失(即执行双重反向传播)，并且出现以下错误: RuntimeError:梯度计算所需的变量之一已通过就地操作进行了修改

I am trying to compute a loss on the jacobian of the network (i.e. to perform double backprop), and I get the following error: RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation

我在代码中找不到inplace操作，所以我不知道要修复哪一行.

I can't find the inplace operation in my code, so I don't know which line to fix.

*该错误发生在最后一行: loss3.backward()

*The error occurs in the last line: loss3.backward()

            inputs_reg = Variable(data, requires_grad=True)
            output_reg = self.model.forward(inputs_reg)

            num_classes = output.size()[1]
            jacobian_list = []
            grad_output = torch.zeros(*output_reg.size())

            if inputs_reg.is_cuda:
                grad_output = grad_output.cuda()
                jacobian_list = jacobian.cuda()

            for i in range(10):

                zero_gradients(inputs_reg)
                grad_output.zero_()
                grad_output[:, i] = 1
                jacobian_list.append(torch.autograd.grad(outputs=output_reg,
                                                  inputs=inputs_reg,
                                                  grad_outputs=grad_output,
                                                  only_inputs=True,
                                                  retain_graph=True,
                                                  create_graph=True)[0])


            jacobian = torch.stack(jacobian_list, dim=0)
            loss3 = jacobian.norm()
            loss3.backward()

推荐答案

grad_output.zero_()是就位的，grad_output[:, i-1] = 0也是如此.就地意味着修改张量而不是返回已应用修改的新张量".不是就地的示例解决方案是 torch.where .将第1列清零的示例

grad_output.zero_() is in-place and so is grad_output[:, i-1] = 0. In-place means "modify a tensor instead of returning a new one, which has the modifications applied". An example solution which is not in-place is torch.where. An example use to zero out the 1st column

import torch
t = torch.randn(3, 3)
ixs = torch.arange(3, dtype=torch.int64)
zeroed = torch.where(ixs[None, :] == 1, torch.tensor(0.), t)

zeroed
tensor([[-0.6616,  0.0000,  0.7329],
        [ 0.8961,  0.0000, -0.1978],
        [ 0.0798,  0.0000, -1.2041]])

t
tensor([[-0.6616, -1.6422,  0.7329],
        [ 0.8961, -0.9623, -0.1978],
        [ 0.0798, -0.7733, -1.2041]])

请注意t如何保留以前的值，zeroed如何保留所需的值.

Notice how t retains the values it had before and zeroed has the values you want.

这篇关于找不到原位运算:原位运算已修改了梯度计算所需的变量之一的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！