keras ImageDataGenerator内插二进制掩码 [英] keras ImageDataGenerator interpolates binary mask

问题描述

我正在训练一个神经网络，以预测鼠标大脑图像上的二进制掩码.为此，我使用来自keras的ImageDataGenerator扩展了数据.

I am training a neural network to predict a binary mask on mouse brain images. For this I am augmenting my data with the ImageDataGenerator from keras.

但是我已经意识到，在应用空间变换时，数据生成器正在对数据进行插值.

But I have realized that the Data Generator is interpolating the data when applying spatial transformations.

这对图像很好，但是我当然不希望我的遮罩包含非二进制值.

This is fine for the image, but I certainly do not want my mask to contain non-binary values.

在应用转换时是否可以选择类似最近邻插值的方法?我在keras文档中没有找到这样的选项.

Is there any way to choose something like a nearest neighbor interpolation when applying the transformations? I have found no such option in the keras documentation.

(左边是原始二进制掩码，右边是增强的内插掩码)

(To the left is the original binary mask, to the right is the augmented, interpolated mask)

图片代码:

data_gen_args = dict(rotation_range=90,
                     width_shift_range=30,
                     height_shift_range=30,
                     shear_range=5,
                     zoom_range=0.3,
                     horizontal_flip=True,
                     vertical_flip=True,
                     fill_mode='nearest')
image_datagen = kp.image.ImageDataGenerator(**data_gen_args)
image_generator = image_datagen.flow(image, seed=1)
plt.figure()
plt.subplot(1,2,1)
plt.imshow(np.squeeze(image))
plt.axis('off')
plt.subplot(1,2,2)
plt.imshow(np.squeeze(image_generator.next()[0]))
plt.axis('off')
plt.savefig('vis/keras_example')

推荐答案

我自己的二进制图像数据有相同的问题.有几种方法可以解决此问题.

I had the same problem with my own binary image data. There are several ways to approach this issue.

简单答案:我通过将ImageDataGenerator的结果手动转换为二进制来解决了这个问题.如果要手动遍历生成器(使用"next()"方法或"for"循环)，则只需使用numpy的"where"方法即可将非二进制值转换为二进制值:

Simple answer: I solved it by manually converting results of ImageDataGenerator to binary. If you are manually iterating over the generator(using 'next()' method or using a 'for' loop), so you can simply use numpy 'where' method to convert non-binary values to binary:

import numpy as np

batch = image_generator.next()
binary_images = np.where(batch>0, 1, 0)  ## or batch>0.5 or any other thresholds

在ImageDataGenerator中使用preprocessing_function自变量

Using the preprocessing_function argument in ImageDataGenerator

另一种更好的方法是在ImageDataGenerator中使用preprocessing_function自变量.如文档所述，可以指定一个自定义预处理功能将在数据扩充程序之后执行，因此您可以在data_gen_args中指定此功能，如下所示:

Another better way is to use preprocessing_function argument in the ImageDataGenerator. As written in the documentation it is possible to specify a custom preprocessing function that will be executed after the data augmentation procedures, so you can specify this function in your data_gen_args as follows:

from keras.preprocessing.image import ImageDataGenerator

data_gen_args = dict(rotation_range=90,
                     width_shift_range=30,
                     height_shift_range=30,
                     shear_range=5,
                     zoom_range=0.3,
                     horizontal_flip=True,
                     vertical_flip=True,
                     fill_mode='nearest',
                     preprocessing_function = lambda x: np.where(x>0, 1, 0).astype(x.dtype))

注意:根据我的经验，preprocessing_function是在rescale之前执行的，也可以在data_gen_args中将其指定为ImageDataGenerator的参数.这不是您的情况，但是如果您需要指定该参数，请记住这一点.

Note: from my experience the preprocessing_function is executed before the rescale, that is possible to specify also as an argument of the ImageDataGenerator in your data_gen_args. This is not your case but if you will need to specify that argument keep this in mind.

创建自定义生成器

另一种解决方案是编写自定义数据生成器并在其中修改ImageDataGenerator的输出.然后，使用此新生成器来填充model.fit().像这样:

Another solution is to write a custom data generator and modify the output of ImageDataGenerator inside it. Then use this new generator to feed model.fit(). Something like this:

batch_size = 64
image_datagen = kp.image.ImageDataGenerator(**data_gen_args)
image_generator = image_datagen.flow(image, batch_size=batch_size, seed=1)
from tensorflow.keras.utils import Sequence
class MyImageDataGenerator(Sequence):
        def __init__(self, data_size, batch_size):
            self.data_size = data_size
            self.batch_size = batch_size
            super(MyImageDataGenerator).__init__()

        def __len__(self):
            return int(np.ceil(self.data_size / float(self.batch_size)))

        def __getitem__(self, idx):    
            augmented_data = image_generator.next()
            binary_images = np.where(augmented_data>0, 1, 0)
            return binary_images

my_image_generator = MyImageDataGenerator(data_size=len(image), batch_size=batch_size)
model.fit(my_image_generator, epochs=50)

上面的数据生成器也是一个简单的数据生成器.如果需要，您可以对其进行自定义并添加标签(例如此)或多模式数据等.

Also above data generator is a simple data generator. If you need, you can customize it and add your lables (like this) or multimodal data, etc.

这篇关于keras ImageDataGenerator内插二进制掩码的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！