为什么 pytorch 没有为我最小化 x*x? [英] Why pytorch isn't minimizing x*x for me?
问题描述
我希望 x 收敛到 0,这是 x*x 的最小值.但这不会发生.我在这个小示例代码中做错了什么:
I expect x to converge to 0, which is minimum of x*x. But this doesn't happen. What am I doing wrong in this small sample code:
import torch
from torch.autograd import Variable
tns = torch.FloatTensor([3])
x = Variable(tns, requires_grad=True)
z = x*x
opt = torch.optim.Adam([x], lr=.01, betas=(0.5, 0.999))
for i in range(3000):
z.backward(retain_graph=True) # Calculate gradients
opt.step()
print(x)
推荐答案
您遇到的问题是在计算每个循环时没有将梯度归零.相反,通过在循环的每一步设置 retain_graph=True
而不是调用 opt.zero_grad()
,您实际上是将计算的梯度添加到 ALL 之前的梯度计算.因此,您不是在梯度下降中迈出一步,而是在所有累积梯度方面迈出了一步,这当然不是您想要的.
The problem you have is that you don't zero the gradients when you are calculating each loop. Instead, by setting retain_graph=True
and not calling opt.zero_grad()
at each step of the loop you are actually adding the gradients calculated to ALL previous gradients calculated. So instead of taking a step in gradient descent, you are taking a step with respect to all accumulated gradients which is certainly NOT what you want.
您应该确保在循环开始时调用 opt.zero_grad()
,并将 z=x*x
移动到循环内,以便您不必retain_graph
.
You should instead make sure to call opt.zero_grad()
at the beginning of your loop, and move the z=x*x
inside your loop so that you don't have to retain_graph
.
我做了这些细微的修改:
I made these slight modifications:
import torch
from torch.autograd import Variable
tns = torch.FloatTensor([3])
x = Variable(tns, requires_grad=True)
opt = torch.optim.Adam([x], lr=.01, betas=(0.5, 0.999))
for i in range(3000):
opt.zero_grad()
z = x*x
z.backward() # Calculate gradients
opt.step()
print(x)
我最终的 1e-25
.
这篇关于为什么 pytorch 没有为我最小化 x*x?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!