如何在 Tensorflow Object Detection API 中初始化卷积层的权重? [英] How to initialize weight for convolution layers in Tensorflow Object Detection API?

问题描述

我按照这个教程来实现 Tensorflow 对象检测 API.

首选方法是使用预训练模型.

但在某些情况下，我们需要从头开始训练.

为此我们只需要在配置文件中注释掉两行

#fine_tune_checkpoint: "object_detection/data/mobilenet_v1_1.0_224/mobilenet_v1_1.0_224.ckpt"#from_detection_checkpoint: 真

如果我想用 Xavier 权重初始化来初始化权重，我该怎么做?

解决方案

正如你在 配置protobuf定义，你可以使用3个初始化器:

• TruncatedNormalInitializer truncated_normal_initializer
• VarianceScalingInitializervariance_scaling_initializer
• RandomNormalInitializer random_normal_initializer

VarianceScalingInitializer 正是您要找的.它是通用初始化程序，您可以通过设置 factor=1.0, mode='FAN_AVG' 基本上变成 Xavier 初始化程序，如 文档.

所以，通过将初始化器设置为

初始化器{方差_缩放_初始化器{系数:1.0制服:真的模式:FAN_AVG}}

在您的配置中，您获得 Xavier 初始值设定项.

而且，即使您需要对新数据进行训练，也可以考虑使用预训练网络作为初始化，而不是随机初始化.如需了解详情，请参阅这篇文章.>

I followed this tutorial for implementing Tensorflow Object Detection API.

The preferred way is using pretrained models.

But for some cases, we need to train from scratch.

For that we just need to comment out two lines in the configuration file as

#fine_tune_checkpoint: "object_detection/data/mobilenet_v1_1.0_224/mobilenet_v1_1.0_224.ckpt"
#from_detection_checkpoint: true 

If I want to initialize weight with Xavier weight initialization, how can I do that?

解决方案

As you can see in the configuration protobuf definition, there are 3 initializers you can use:

• TruncatedNormalInitializer truncated_normal_initializer
• VarianceScalingInitializer variance_scaling_initializer
• RandomNormalInitializer random_normal_initializer

The VarianceScalingInitializer is what you are looking for. It is general initializer which you can basically turn into Xavier initializer by setting factor=1.0, mode='FAN_AVG', as stated in the documentation.

So, by setting the initializers as

initializer {
    variance_scaling_initializer {
        factor: 1.0
        uniform: true
        mode: FAN_AVG
    }
}

in your configuration, you obtain Xavier initializer.

But also, even if you need to train on new data, consider using pretrained network as initialization instead of random initialization. For more details, see this article.

这篇关于如何在 Tensorflow Object Detection API 中初始化卷积层的权重?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！