什么时候在 Caffe 中使用就地图层? [英] When to use in-place layers in Caffe?

问题描述

通过将底部和顶部的 blob 设置为相同，我们可以告诉 Caffe 进行就地"计算以保持内存消耗.

目前我知道我可以安全地使用就地 "BatchNorm"、"Scale" 和 "ReLU" 层(请让我知道我是否错了).虽然其他层似乎有一些问题(这个问题似乎是一个例子).

何时在 Caffe 中使用就地图层?
它如何与反向传播一起工作?

解决方案

正如您所注意到的，就地图层通常不能开箱即用".
对于某些层，它非常简单(ReLU" 和其他神经元激活层).
但是，对于其他人，它需要在代码中进行特殊处理.例如"PReLU" 层具有特定的缓存 bottom_memory_ 成员变量，用于存储反向传播所需的信息.
您可以看到其他层的类似代码，专门测试 if (top[0] == bottom[0]) 以查看该层是否在就地"情况下使用.>

此外，对于输入和输出具有不同形状的就地层来说意义不大，因此诸如Convolution"、InnerProduct"之类的层是没有意义的.code>, "Pool" 不被视为就地"层的候选者.

By setting the bottom and the top blob to be the same we can tell Caffe to do "in-place" computation to preserve memory consumption.

Currently I know I can safely use in-place "BatchNorm", "Scale" and "ReLU" layers (please let me know if I'm wrong). While it seems to have some issues for other layers (this issue seems to be an example).

When to use in-place layers in Caffe?
How does it work with back-propagation?

解决方案

As you well noted, in-place layers don't usually work "out of the box".
For some layers, it is quite trivial ("ReLU" and other neuron activation layers).
However, for others it requires special handling in code. For example, the implementation of "PReLU" layer has specific cache bottom_memory_ member variable that stores information needed for backprop.
You can see similar code for other layers that specifically test for if (top[0] == bottom[0]) to see if the layer is used in an "in-place" case.

Moreover, it makes little sense to have an in-place layer for which the input and output are of different shapes, thus layers such as "Convolution", "InnerProduct", "Pool" are not considered as candidates for "in-place" layers.

这篇关于什么时候在 Caffe 中使用就地图层?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！