Tensorflow object_detection API 微调 ckpt 分类 [英] Tensorflow object_detection API fine tune ckpt classification

问题描述

我只是想知道分类"和分类"之间有什么区别?和检测"对象检测 API 中可用检查点的微调类型.他们都有资格培训小说班吗?从此类训练中获得的检查点，可以使用相同的 pipeline.config 进一步训练，还是需要不同的微调类型?

编辑

为了使问题更清楚，可以参考

物体检测结合

1. 对象本地化
2. 图像分类

除了上述两个之外，还有图像分割，我不会深入研究来自 考虑张量流.

<块引用>

目标检测算法结合了图像分类和目标定位.它将图像作为输入并生成一个或多个边界框，每个边界框都带有类标签.这些算法足以处理多类分类和定位以及处理多次出现的对象.ref.

回答您的问题.

<块引用>

他们都有资格培训新颖的课程

微调检查点选项基本上用于迁移学习.假设您有一个来自先前数据集的预训练模型，这些相同的权重和检查点可用于通过 迁移学习.

当然，这意味着在类上训练的预训练模型可以帮助您进一步将学习转移到具有相应类的新图像集.有关训练的模型列表，请查看这里 Model ZooTF2 OD API.

例如.您可以使用在大约 100 多个类上训练的 coco 数据集模型(以上列表中的任何一个)来训练新的类集，例如 iphone 7/8 或 iphone x/xs/11/12不同环境、不同形状和大小的每个类别只需 100-1000 张图像.

<块引用>

从此类训练中获得的检查点，可以使用相同的 pipeline.config 进一步训练，还是需要不同的微调类型?

不，您必须使用适合您的用例的最少选项修改 pipeline.config.首先，请查看 github 页面上对象检测的官方文档.如果您仍然有疑问或被卡住.针对您的问题提出新问题.

更新:如果您正在寻找多个对象"的本地化 + 分类，则微调检查点将设置为 detection在图像中.classification 仅指简单地标记图像(参考上图).

我提到的 ModelZoo 模型可用于 detection 类型检查点.对象检测据说仅将带有 detection 的模型作为一种类型，这是基于我自己的试验和最终失败.如果您只关注分类，请查看此有用资源:用于图像分类的教学机器.withgoogle.com/train

I am just wondering what is the difference between the "classification" and "detection" fine tuning types for the available checkpoints in the object detection API. Are they both eligible to train novel classes? The checkpoints obtained from such training, can be further trained with the very same pipeline.config, or does it need to have a different fine tuning type?

EDIT

to make the question clearer, one can take as reference the ckpts mentioned in https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf2_training_and_evaluation.md#model-parameter-initialization .

Digging in the code one could find for example that some classes of CenterNet models use the CenterNetResnetV1FpnFeatureExtractor class to restore the model from a checkpoint, which seems to accept "classification" types only (https://github.com/tensorflow/models/blob/1b5a4c9ed33242783eaf29e664618331dbb59e1b/research/object_detection/models/center_net_resnet_v1_fpn_feature_extractor.py#L168), while one can find some CenterNet models addressed as "detection" types (one can compare the fine_tune_checkpoint_type that comes with the pipeline.config file in the download of centernet_hg104_512x512_coco17_tpu-8 ckpt from the "detection" cktp list mentioned above).

I find it in general rather confusing, and I am not sure that this does not generate undesired effects like retraining the whole model from scratch ignoring the desired ckpt or similar. Obviously it would be nice not to have to inspect the whole code to find how every each model is restored from a ckpt.

解决方案

Machine Learning on Images basically have three types.

Object Detection which is a combination of

1. Object Localisation
2. Image Classification

Apart from the above two, There is also Image Segmentation which I won't devlve into as far as object detection API from tensorflow is considered.

Object Detection algorithms act as a combination of image classification and object localization. It takes an image as input and produces one or more bounding boxes with the class label attached to each bounding box. These algorithms are capable enough to deal with multi-class classification and localization as well as to deal with the objects with multiple occurrences.ref.

To answer your questions.

Are they both eligible to train novel classes

The fine tuned checkpoint option is basically for transfer learning. Consider you have a pre-trained model from a previous dataset, those same weights and checkpoint can be used to further train your novel dataset via transfer learning.

Ofcourse, this would mean a pre-trained model trained on classes can you help you futher transfer learning to new set of images with their corresponding classes. For a list of models trained have a look here Model Zoo TF2 OD API.

Eg. You could use a coco dataset model (any from the above list infact) trained on around 100+ classes to train on new set of classes like iphone 7/8 or iphone x/xs/11/12 with as few as 100-1000 images for each category in different surroundings and shapes and sizes.

The checkpoints obtained from such training, can be further trained with the very same pipeline.config, or does it need to have a different fine tuning type?

No, you would have to modify the pipeline.config with a bare minimum of options that fit your use cases. In order to start with, have a look at the official documentation for Object Detection at github pages. If you still have doubts or is stuck. Raise a new question with your issue.

UPDATE: The fine tune checkpoint will be set to detection if you are looking for localisation + classification of multiple "objects" in the image. classification only refers to simply labelling an image (refer above image).

The ModelZoo models I referred can be used for detection type checkpoints. Object Detection supposedly mentions models with detection as a type only, and this based on my own trials and ultimate failure. If you are just looking at classification, have a look at this useful resource : teachablemachine.withgoogle.com/train for image classification

这篇关于Tensorflow object_detection API 微调 ckpt 分类的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！