Tensorflow:如何在语义分割期间忽略特定标签? [英] Tensorflow: How to ignore specific labels during semantic segmentation?

问题描述

我正在使用 tensorflow 进行语义分割.在计算像素损失时，如何告诉 tensorflow 忽略特定标签?

I'm using tensorflow for semantic segmentation. How can I tell tensorflow to ignore a specific label when computing the pixelwise loss?

我已经阅读了这篇博文中的对于图像分类，可以将标签设置为 -1 它将被忽略.如果这是真的，给定标签张量，我如何修改我的标签，以便将某些值更改为 -1?

I've read in this post that for image classification one can set the label to -1 and it will be ignored. If that is true, given the label-tensor, how can I modify my labels such that certain values are changed to -1?

在 Matlab 中它会是这样的:

In Matlab it would be something like:

ignore_label = 255
myLabelTensor(myLabelTensor == ignore_label) = -1

但我不知道如何在 TF 中做到这一点?

But I don't know how to do this in TF?

一些背景信息:
这是标签的加载方式:

Some background info:
This is how the labels are loaded:

label_contents = tf.read_file(input_queue[1])
label = tf.image.decode_png(label_contents, channels=1)

这是当前计算损失的方式:

This is how the loss is currently computed:

raw_output = net.layers['fc1_voc12']
prediction = tf.reshape(raw_output, [-1, n_classes])
label_proc = prepare_label(label_batch, tf.pack(raw_output.get_shape()[1:3]),n_classes)
gt = tf.reshape(label_proc, [-1, n_classes])

# Pixel-wise softmax loss.
loss = tf.nn.softmax_cross_entropy_with_logits(prediction, gt)
reduced_loss = tf.reduce_mean(loss)

与

def prepare_label(input_batch, new_size, n_classes):
    """Resize masks and perform one-hot encoding.

    Args:
      input_batch: input tensor of shape [batch_size H W 1].
      new_size: a tensor with new height and width.

    Returns:
      Outputs a tensor of shape [batch_size h w 21]
      with last dimension comprised of 0's and 1's only.
    """
    with tf.name_scope('label_encode'):
        input_batch = tf.image.resize_nearest_neighbor(input_batch, new_size) # as labels are integer numbers, need to use NN interp.
        input_batch = tf.squeeze(input_batch, squeeze_dims=[3]) # reducing the channel dimension.
        input_batch = tf.one_hot(input_batch, depth=n_classes)
    return input_batch

我正在使用 tensorflow-deeplab-resnet 模型，它传输 Resnet使用 caffe-tensorflow 在 Caffe 中实现的模型到张量流.

I'm using the tensorflow-deeplab-resnet model which transfers the Resnet model implemented in Caffe to tensorflow using caffe-tensorflow.

推荐答案

根据文档，tf.nn.softmax_cross_entropy_with_logits 必须使用 labels 上的有效概率分布调用，否则计算将不正确，并使用 tf.nn.sparse_softmax_cross_entropy_with_logits(可能更多在您的情况下方便)带有负标签将导致错误或返回 NaN 值.我不会依赖它来忽略某些标签.

According to the documentation, tf.nn.softmax_cross_entropy_with_logits must be called with valid probability distributions on labels, or otherwise the computation will be incorrect, and using tf.nn.sparse_softmax_cross_entropy_with_logits (which might be more convenient in your case) with negative labels will either cause an error or return NaN values. I wouldn't rely on it to have some labels ignored.

我要做的是在那些正确的类是被忽略的像素的像素中用无穷大替换被忽略类的对数，所以它们不会对损失产生任何影响:

What I would do is to replace the logits for the ignored class with infinity in those pixels where the correct class is the ignored one, so they will contribute nothing to the loss:

ignore_label = ...
# Make zeros everywhere except for the ignored label
input_batch_ignored = tf.concat(input_batch.ndims - 1,
    [tf.zeros_like(input_batch[:, :, :, :ignore_label]),
     tf.expand_dims(input_batch[:, :, :, ignore_label], -1),
     tf.zeros_like(input_batch[:, :, :, ignore_label + 1:])])
# Make corresponding logits "infinity" (a big enough number)
predictions_fix = tf.select(input_batch_ignored > 0,
    1e30 * tf.ones_like(predictions), predictions)
# Compute loss with fixed logits
loss = tf.nn.softmax_cross_entropy_with_logits(prediction, gt)

唯一的问题是您正在考虑始终正确预测被忽略类的像素，这意味着包含大量这些像素的图像的损失会人为地变小.根据具体情况，这可能显着也可能不显着，但如果您想真正准确，则必须根据未忽略的像素数对每个图像的损失进行加权，而不仅仅是取平均值.

The only problem with this is that you are considering that pixels of the ignored class are always predicted correctly, which means that the loss for images containing a lot of those pixels will be artificially smaller. Depending on the case this may or may not be significant, but if you want to be really accurate, you would have to weight the loss of each image according to the number of not ignored pixels, instead of just taking the mean.

# Count relevant pixels on each image
input_batch_relevant = 1 - input_batch_ignored
input_batch_weight = tf.reduce_sum(input_batch_relevant, [1, 2, 3])
# Compute relative weights
input_batch_weight = input_batch_weight / tf.reduce_sum(input_batch_weight)
# Compute reduced loss according to weights
reduced_loss = tf.reduce_sum(loss * input_batch_weight)

这篇关于Tensorflow:如何在语义分割期间忽略特定标签?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！