pytorch-loss.backward()和optimizer.step()之间的连接 [英] pytorch - connection between loss.backward() and optimizer.step()

问题描述

optimizer和loss之间的显式连接在哪里?

Where is an explicit connection between the optimizer and the loss?

优化器如何知道在没有调用的情况下从何处获得损耗的梯度?

How does the optimizer know where to get the gradients of the loss without a call liks this optimizer.step(loss)?

-更多上下文-

当我将损失降到最低时，我不必将梯度传递给优化器.

When I minimize the loss, I didn't have to pass the gradients to the optimizer.

loss.backward() # Back Propagation
optimizer.step() # Gardient Descent

推荐答案

在不深入研究pytorch内部的情况下，我可以提供一个简单的答案:

Without delving too deep into the internals of pytorch, I can offer a simplistic answer:

回想一下，初始化optimizer时，您明确地告诉它应该更新模型的哪些参数(张量).渐变由张量本身存储"(它们具有 grad 和 requires_grad 属性)损失时致电backward().在计算完模型中所有张量的梯度后，调用optimizer.step()可使优化器对应该更新的所有参数(张量)进行迭代，并使用其内部存储的grad来更新其值.

Recall that when initializing optimizer you explicitly tell it what parameters (tensors) of the model it should be updating. The gradients are "stored" by the tensors themselves (they have a grad and a requires_grad attributes) once you call backward() on the loss. After computing the gradients for all tensors in the model, calling optimizer.step() makes the optimizer iterate over all parameters (tensors) it is supposed to update and use their internally stored grad to update their values.

这篇关于pytorch-loss.backward()和optimizer.step()之间的连接的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！