为什么 model.forward(input) 和 model(input) 之间有不同的输出 [英] Why there are different output between model.forward(input) and model(input)

问题描述

我正在使用 pytorch 构建一个像 VGG16 这样的简单模型，并且我在我的模型中重载了函数 forward.

I'm using pytorch to build a simple model like VGG16,and I have overloaded the function forward in my model.

我发现每个人都倾向于使用 model(input) 来获取输出而不是 model.forward(input)，我对它们之间的区别很感兴趣.我尝试输入相同的数据，但结果不同.我很困惑.

I found everyone tends to use model(input) to get the output rather than model.forward(input), and I am interested in the difference between them. I try to input the same data, but the result is different. I'm confused.

我在输入数据之前输出了layer_weight，权重没有改变，我知道当我们使用model(input)时，它使用__call__函数，而这个函数将调用 model.forward.

I have output the layer_weight before I input data, the weight not be changed, and I know when we using model(input) it using __call__ function, and this function will call model.forward.

   vgg = VGG()
   vgg.double()
   for layer in vgg.modules():
      if isinstance(layer,torch.nn.Linear):
         print(layer.weight)
   print("   use model.forward(input)     ")
   result = vgg.forward(array)

   for layer in vgg.modules():
     if isinstance(layer,torch.nn.Linear):
       print(layer.weight) 
   print("   use model(input)     ")
   result_2 = vgg(array)
   print(result)
   print(result_2)

输出:

    Variable containing:1.00000e-02 *
    -0.2931  0.6716 -0.3497 -2.0217 -0.0764  1.2162  1.4983 -1.2881
    [torch.DoubleTensor of size 1x8]

    Variable containing:
    1.00000e-02 *
    0.5302  0.4494 -0.6866 -2.1657 -0.9504  1.0211  0.8308 -1.1665
    [torch.DoubleTensor of size 1x8]

推荐答案

model.forward 只是像你提到的那样调用转发操作，但 __call__ 做了一些额外的工作.

model.forward just calls the forward operations as you mention but __call__ does a little extra.

如果您深入了解代码在 nn.Module 类中，您将看到 __call__ 最终调用前向，但在内部处理前向或后向钩子并管理 pytorch 允许的一些状态.当调用像 MLP 这样的简单模型时，它可能不是真正需要的，但更复杂的模型(如光谱归一化层)具有钩子，因此您应该尽可能多地使用 model(.) 签名，除非您明确只想调用 model.forward

If you dig into the code of nn.Module class you will see __call__ ultimately calls forward but internally handles the forward or backward hooks and manages some states that pytorch allows. When calling a simple model like just an MLP, it may not be really needed but more complex model like spectral normalization layers have hooks and therefore you should use model(.) signature as much as possible unless you explicitly just want to call model.forward

另见不使用 .forward() 调用转发函数

然而，在这种情况下，差异可能是由于某些 dropout 层造成的，您应该调用 vgg.eval() 以确保在比较输出之前关闭网络中的所有随机性.

In this case, however, the difference may be due to some dropout layer, you should call vgg.eval() to make sure all the stochasticity in network is turned off before comparing the outputs.

这篇关于为什么 model.forward(input) 和 model(input) 之间有不同的输出的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！