如何创建类似于Imagenet或Noisy-student的预训练体重模型? [英] How to create a pre-trained weight model similar to Imagenet or Noisy-student?

问题描述

我正在尝试创建一个预训练的体重模型文件，该文件可用于初始化类似于imagenet预训练的体重文件或嘈杂学生的模型.

I am trying to create a pre-trained weight model file that could be used for initialization of a model similar to imagenet pre-trained weight file or that of noisy-student.

我有足够大的数据集，这些数据集非常多样化，但特定于我感兴趣的领域.我希望创建一个针对我感兴趣的领域的通用的预训练权重文件.

I have large enough data set that is very diverse yet specific to a domain of my interest. I hope to create a generalized pre-trained weight file that is specific to a domain of my interest.

我知道我无法训练和保存模型权重，因为生成的预训练权重文件和网络之间的类数不匹配(因此层数不等)，我正在尝试使用该预训练权重文件体重.

I know I just can't train and save the model weights since the number of classes would not match (thus the number of layer) between generated pre-trained weight file and the network I am trying to use that pre-trained weight.

我在互联网上找不到有关创建自定义预训练体重文件的任何信息，因此任何提示或建议都将非常有帮助.

I could not find any information regarding creating custom pre-trained weight file on the internet, so any tips or advice would be very helpful.

推荐答案

定义用于对数据集进行训练的模型时，其头应该是全局池层，然后是一个密集层.密集层将用于根据您的数据集进行分类.模型完成训练后，您可以基于训练后的模型图创建一个不同的模型，但不包括模型的头部.然后，您可以保存可用于转移学习的新模型的权重.下面是我使用Tensorflow的功能性API进行解释时的简化视图.

When you define your model for training on your data set, the head of it should be a global pooling layer followed by one dense layer. The dense layer will be used to classify based on your data set. Once your model finishes training, you can create a different model that is based on the graph of your trained model but not including the head of it. Then, you can save the weights of this new model which can be used for transfer learning. Below is a simplified view of what I am explaining using Tensorflow's functional API.

inputs = Input(shape)
# All hidden layers
x = (...)(inputs)
# Final pooling layer
x = GlobalAveragePooling2D()(x)
# Prediction layer
x = Dense(num_classes)(x)

model = Model(inputs, x)

训练完此模型后，请创建一个除头部以外的所有模型，并保存重量.

Once you train this model, create a different one connecting all but the head and save the weights.

new_model = Model(inputs, model.layers[-2])
new_model.save_weights(file_path)

此新模型包含与先前模型相同的输入，但是输出是 GlobalAveragePooling2D()层之前的层.

This new model contains the same input as the previous model, but the output is the layer right before the GlobalAveragePooling2D() layer.

这篇关于如何创建类似于Imagenet或Noisy-student的预训练体重模型?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！