在语义分割中，如何处理未知类的均值交集(mIOU)? [英] How to handle the mean Intersection Over Union (mIOU) for unknown class in semantic segmentation?

问题描述

我实现了FCN网络来进行语义分割.我正在使用Cityscapes作为数据集.如您所知，在Cityscapes中有一些类别在训练期间会被忽略，它被标记为255.我使用加权损失来忽略未知类别的损失(将未知类别的损失设置为零).现在，我想从评估指标中排除未知类别(均值交集(mIOU)).目前我还不清楚如何排除未知类别.

I implemented a FCN network to do semantic segmentation. I am using Cityscapes as my dataset. As you know, there are some classes in Cityscapes that you ignore during the training and it is labeled as 255. I used weighted loss to ignore the loss for the unknown classes(set the loss to zero for unknown class). Now I want to exclude unknown class from my evaluation metric(mean Intersection Over Union (mIOU)).It is not clear for me how to exclude the unknown class at this point.

目前，我正在使用tensorflow方法考虑所有类，包括未知类:

At the moment I am considering all the classes including the unknown class like this using tensorflow method:

 miou, confusion_mat = tf.metrics.mean_iou(labels=annotation, predictions=pred_annotation, num_classes=num_cls)

with tf.control_dependencies([tf.identity(confusion_mat)]):
    miou = tf.identity(miou)

我尝试了此方法，但未绑定的标签(对于未知的标签)给出了错误

I tried this , but it give an error for unbound label(for the unkonwn label)

miou, confusion_mat = tf.metrics.mean_iou(labels=annotation, predictions=pred_annotation, num_classes=(num_cls-1))

推荐答案

如果您有一个要在mIoU计算期间忽略的类，并且可以访问混淆矩阵，则可以这样做:

If you have a class that you want to ignore during the mIoU calculation, and you have access to the confusion matrix then you can do it like this:

1. 忽略由tensorflow计算的miou(因为它考虑了所有类，而这不是您想要的)
2. 从混淆矩阵中删除与您要忽略的类相对应的行和列
3. 使用新的混淆矩阵重新计算miou指标

1. ignore the miou calculated by tensorflow (since it considers all classes and that is not what you want)
2. remove row and column from the confusion matrix that correspond to the class you want to ignore
3. recalculate miou metric with the new confusion matrix

如何从混淆矩阵中重新计算miou指标?

How to recalculate miou metric from the confusion matrix?

头等舱: iou_0 = conf_mat[0,0] / (sum(conf_mat[0,:]) + sum(conf_mat[:,0]) - conf_mat[0,0]) 第二堂课: iou_1 = conf_mat[1,1] / (sum(conf_mat[1,:]) + sum(conf_mat[:,1]) - conf_mat[1,1])
• ...
对于j类，通常为
• :iou_j = conf_matrix[j,j] / (sum(conf_mat[j,:]) + sum(conf_mat[:,j]) - conf_mat[j,j])

• iou for the first class: iou_0 = conf_mat[0,0] / (sum(conf_mat[0,:]) + sum(conf_mat[:,0]) - conf_mat[0,0])
• iou for the second class: iou_1 = conf_mat[1,1] / (sum(conf_mat[1,:]) + sum(conf_mat[:,1]) - conf_mat[1,1])
• ...
• in general for the class j: iou_j = conf_matrix[j,j] / (sum(conf_mat[j,:]) + sum(conf_mat[:,j]) - conf_mat[j,j])

最后，对所有这些 iou求和并求平均值，得到miou.

At the end, sum and average all these per class iou to get miou.

这篇关于在语义分割中，如何处理未知类的均值交集(mIOU)?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！