何时在Caffe中使用就地图层? [英] When to use in-place layers in Caffe?

问题描述

通过将底部和顶部blob设置为相同，我们可以告诉Caffe进行就地"计算以节省内存.

By setting the bottom and the top blob to be the same we can tell Caffe to do "in-place" computation to preserve memory consumption.

目前，我知道我可以安全地使用就地"BatchNorm"，"Scale"和"ReLU"图层了(请告诉我我是否错了).尽管其他层似乎存在一些问题(此问题似乎是一个例子).

Currently I know I can safely use in-place "BatchNorm", "Scale" and "ReLU" layers (please let me know if I'm wrong). While it seems to have some issues for other layers (this issue seems to be an example).

何时在Caffe中使用就地图层?
反向传播如何工作?

When to use in-place layers in Caffe?
How does it work with back-propagation?

推荐答案

正如您所指出的，就地图层通常不能开箱即用"地工作.
对于某些层，这是非常琐碎的("ReLU"和其他神经元激活层).
但是，对于其他人，则需要对代码进行特殊处理.例如， "PReLU" 层的实现具有特定的缓存bottom_memory_成员变量，该成员变量存储反向传播所需的信息.
您可以看到针对其他图层的类似代码，这些代码专门测试了if (top[0] == bottom[0])以查看该图层是否在就地"情况下使用.

As you well noted, in-place layers don't usually work "out of the box".
For some layers, it is quite trivial ("ReLU" and other neuron activation layers).
However, for others it requires special handling in code. For example, the implementation of "PReLU" layer has specific cache bottom_memory_ member variable that stores information needed for backprop.
You can see similar code for other layers that specifically test for if (top[0] == bottom[0]) to see if the layer is used in an "in-place" case.

此外，就地层具有输入和输出具有不同形状的意义不大，因此诸如"Convolution"，"InnerProduct"，"Pool"之类的层不被认为是"in"的候选. -place"图层.

Moreover, it makes little sense to have an in-place layer for which the input and output are of different shapes, thus layers such as "Convolution", "InnerProduct", "Pool" are not considered as candidates for "in-place" layers.

这篇关于何时在Caffe中使用就地图层?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！