梯度在pytorch中不可用 [英] Gradient is none in pytorch when it shouldn't
问题描述
我正在尝试使用pytorch获取/跟踪变量的梯度,在该变量中,我将其传递给第一个函数,该函数查找某个其他变量的某个最小值,然后输入第一个函数的输出到第二个功能,整个过程会重复多次.
I am trying to get/trace the gradient of a variable using pytorch, where I have that variable, pass it to a first function that looks for some minimum value of some other variable, then the output of the first function is inputted to a second function, and the whole thing repeats multiple times.
这是我的代码:
import torch
def myFirstFunction(parameter_current_here):
optimalValue = 100000000000000
Optimal = 100000000000000
for j in range(2, 10):
i = torch.ones(1, requires_grad=True)*j
with torch.enable_grad():
optimalValueNow = i*parameter_current_here.sum()
if (optimalValueNow < optimalValue):
optimalValue = optimalValueNow
Optimal = i
return optimalValue, Optimal
def mySecondFunction(Current):
with torch.enable_grad():
y = (20*Current)/2 + (Current**2)/10
return y
counter = 0
while counter < 5:
parameter_current = torch.randn(2, 2, requires_grad=True)
outputMyFirstFunction = myFirstFunction(parameter_current)
outputmySecondFunction = mySecondFunction(outputMyFirstFunction[1])
outputmySecondFunction.backward()
print("outputMyFirstFunction after backward:",
outputMyFirstFunction)
print("outputmySecondFunction after backward:",
outputmySecondFunction)
print("parameter_current Gradient after backward:",
parameter_current.grad)
counter = counter + 1
当显然不应该将parameter_current.grad设为不适用时,它对所有迭代都不适用.我究竟做错了什么?我该如何解决?
The parameter_current.grad is none for all iterations when it obviously shouldn't be none. What am I doing wrong? And how can I fix it?
在此方面您的帮助将不胜感激.非常感谢!
Your help on this would be highly appreciated. Thanks a lot!
Aly
推荐答案
我对此也有类似的经验.参考: https://pytorch.org/docs/stable/tensors.html
I have a similar experience with this. Reference: https://pytorch.org/docs/stable/tensors.html
- 对于具有require_grad为True的张量,如果它们是由用户创建的,则它们将是叶张量.这意味着它们不是运算的结果,因此grad_fn为None.
- 在调用向后()时,只会填充叶张量的等级.要获取非叶张量的grad,可以使用retain_grad(). 示例:
- For Tensors that have requires_grad which is True, they will be leaf Tensors if they were created by the user. This means that they are not the result of an operation and so grad_fn is None.
- Only leaf Tensors will have their grad populated during a call to backward(). To get grad populated for non-leaf Tensors, you can use retain_grad(). Example:
>>> a = torch.tensor([[1,1],[2,2]], dtype=torch.float, requires_grad=True)
>>> a.is_leaf
True
>>> b = a * a
>>> b.is_leaf
False
>>> c = b.mean()
>>> c.backward()
>>> print(c.grad)
None
>>> print(b.grad)
None
>>> print(a.grad)
tensor([[0.5000, 0.5000],
[1.0000, 1.0000]])
>>> b = a * a
>>> c = b.mean()
>>> b.retain_grad()
>>> c.retain_grad()
>>> c.backward()
>>> print(a.grad)
tensor([[1., 1.],
[2., 2.]])
>>> print(b.grad)
tensor([[0.2500, 0.2500],
[0.2500, 0.2500]])
>>> print(c.grad)
tensor(1.)
这篇关于梯度在pytorch中不可用的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!