梯度在pytorch中不可用 [英] Gradient is none in pytorch when it shouldn't

问题描述

我正在尝试使用pytorch获取/跟踪变量的梯度，在该变量中，我将其传递给第一个函数，该函数查找某个其他变量的某个最小值，然后输入第一个函数的输出到第二个功能，整个过程会重复多次.

I am trying to get/trace the gradient of a variable using pytorch, where I have that variable, pass it to a first function that looks for some minimum value of some other variable, then the output of the first function is inputted to a second function, and the whole thing repeats multiple times.

这是我的代码:

import torch

def myFirstFunction(parameter_current_here):
    optimalValue = 100000000000000
    Optimal = 100000000000000
    for j in range(2, 10):
        i = torch.ones(1, requires_grad=True)*j
        with torch.enable_grad():
            optimalValueNow = i*parameter_current_here.sum()
        if (optimalValueNow < optimalValue):
            optimalValue = optimalValueNow
            Optimal = i
    return optimalValue, Optimal

def mySecondFunction(Current):
    with torch.enable_grad():
        y = (20*Current)/2 + (Current**2)/10
    return y

counter = 0
while counter < 5:
    parameter_current = torch.randn(2, 2, requires_grad=True)

    outputMyFirstFunction = myFirstFunction(parameter_current)
    outputmySecondFunction = mySecondFunction(outputMyFirstFunction[1])
    outputmySecondFunction.backward()

    print("outputMyFirstFunction after backward:",
               outputMyFirstFunction)
    print("outputmySecondFunction after backward:",
               outputmySecondFunction)
    print("parameter_current Gradient after backward:",
               parameter_current.grad)

    counter = counter + 1

当显然不应该将parameter_current.grad设为不适用时，它对所有迭代都不适用.我究竟做错了什么?我该如何解决?

The parameter_current.grad is none for all iterations when it obviously shouldn't be none. What am I doing wrong? And how can I fix it?

在此方面您的帮助将不胜感激.非常感谢！

Your help on this would be highly appreciated. Thanks a lot!

Aly

推荐答案

我对此也有类似的经验.参考: https://pytorch.org/docs/stable/tensors.html

I have a similar experience with this. Reference: https://pytorch.org/docs/stable/tensors.html

• 对于具有require_grad为True的张量，如果它们是由用户创建的，则它们将是叶张量.这意味着它们不是运算的结果，因此grad_fn为None.
• 在调用向后()时，只会填充叶张量的等级.要获取非叶张量的grad，可以使用retain_grad(). 示例:

• For Tensors that have requires_grad which is True, they will be leaf Tensors if they were created by the user. This means that they are not the result of an operation and so grad_fn is None.
• Only leaf Tensors will have their grad populated during a call to backward(). To get grad populated for non-leaf Tensors, you can use retain_grad(). Example:

    >>> a = torch.tensor([[1,1],[2,2]], dtype=torch.float, requires_grad=True)
    >>> a.is_leaf
    True
    >>> b = a * a
    >>> b.is_leaf
    False
    >>> c = b.mean()
    >>> c.backward()
    >>> print(c.grad)
    None
    >>> print(b.grad)
    None
    >>> print(a.grad)
    tensor([[0.5000, 0.5000],
            [1.0000, 1.0000]])
    >>> b = a * a
    >>> c = b.mean()
    >>> b.retain_grad()
    >>> c.retain_grad()
    >>> c.backward()
    >>> print(a.grad)
    tensor([[1., 1.],
            [2., 2.]])
    >>> print(b.grad)
    tensor([[0.2500, 0.2500],
            [0.2500, 0.2500]])
    >>> print(c.grad)
    tensor(1.)

这篇关于梯度在pytorch中不可用的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！