u-net中的多层灰度输入 [英] multi-layer grayscale input in u-net

问题描述

我已经使用(256, 256, 1)灰度输入和(256, 256, 1)二进制标签成功地训练了u-net进行细胞分割的特定任务.我在Keras中使用了zhixuhao的unet实现(git rep.此处).我现在想要做的是使用多个灰度层作为输入来训练相同的模型.

I have successfully trained a u-net for the specific task of cell segmentation using (256, 256, 1) grayscale input and (256, 256, 1) binary label. I used zhixuhao's unet implemention in Keras (git rep. here).What I am trying to do now is to train the same model using multiple grayscale layer as input.

为使事情变得简单，假设我要使用2个灰度图像im1和im2，每个图像的大小为(256, 256, 1).标签Y与im1和im2相同.我想向模型输入尺寸为(256, 256, 2)的输入，其中第三个轴的第一个分量为im1，第二个分量为im2.

To make things easier, let's assume I want to use 2 grayscale image im1 and im2, each of size (256, 256, 1). Label Y is the same for im1 and im2. I want to feed the model an input of size (256, 256, 2), where the first component of the 3rd axis is im1 and the second one is im2.

为此，我将火车数据生成器更改为:

To that end, I changed the train data generator to :

def MultipleInputGenerator(train_path, sub_path_1, sub_path_2, image_folder='images', mask_folder='masks', batch_size, aug_dict, images_color_mode='grayscale', masks_color_mode='grayscale',
            flag_multi_class=False, num_class=2, target_size=(256,256), seed=1):

    # Keras generator
    image_datagen = ImageDataGenerator(**aug_dict)
    mask_datagen = ImageDataGenerator(**aug_dict)

    # Multiple input data augmentation
    image_generator_1 = image_datagen.flow_from_directory(
            sub_path_1,
            classes = [image_folder],
            class_mode = None,
            color_mode = images_color_mode,
            target_size = target_size,
            batch_size = batch_size,
            seed = seed)

    image_generator_2 = image_datagen.flow_from_directory(
            sub_path_2,
            classes = [image_folder],
            class_mode = None,
            color_mode = images_color_mode,
            target_size = target_size,
            batch_size = batch_size,
            save_to_dir = save_to_dir,
            save_prefix  = image_save_prefix,
            seed = seed)

    mask_generator = mask_datagen.flow_from_directory(
            train_path,
            classes = [mask_folder],
            class_mode = None,
            color_mode = masks_color_mode,
            target_size = target_size,
            batch_size = batch_size,
            save_to_dir = save_to_dir,
            save_prefix  = mask_save_prefix,
            seed = seed)

    train_generator = zip(image_generator_1, image_generator_2, mask_generator)

    for (img1, img2, mask) in train_generator:
        img1, mask1 = adjustData(img1, mask, flag_multi_class, num_class)
        img2, mask2 = adjustData(img2, mask, flag_multi_class, num_class)
        yield (np.stack((img1, img2), axis=0), mask1)

，adjustData是辅助函数，用于将数组从[0，255]标准化为[0，1]

with adjustData being an auxillary function which normalises the arrays from [0, 255] to [0, 1]

如您所见，我尝试将灰度阵列堆叠在单个输入中. 创建unet模型时，我将输入大小从(256, 256, 1)更改为(256, 256, 2):

As you can see, I've tried to stack grayscale arrays in a single input. When creating the unet model, I changed the input size from (256, 256, 1) to (256, 256, 2) :

train_gen = MultipleInputGenerator(train_folder, sub_path_1, sub_path_2, batch_size, aug_dict=data_gen_args)
model = unet(input_size=(256,256,2))
model.fit_generator(train_gen, steps_per_epoch=train_steps, epochs=epochs)

但是，当启动命令:python3 main.py时，它会开始正确加载数据，但随后却无法训练模型:

Yet, when lauching the command : python3 main.py, it starts loading the data correctly but then fails to train the model :

Found 224 images belonging to 1 classes.
Epoch 1/2
Found 224 images belonging to 1 classes.
Found 224 images belonging to 1 classes.
Traceback (most recent call last):
  File "main.py", line 50, in <module>
    model.fit_generator(train_gen, steps_per_epoch=train_steps, epochs=epochs)
  File "*/virtenv/env1/lib/python3.6/site-packages/keras/legacy/interfaces.py", line 91, in wrapper
    return func(*args, **kwargs)
  File "*/virtenv/env1/lib/python3.6/site-packages/keras/engine/training.py", line 1732, in fit_generator
    initial_epoch=initial_epoch)
  File "*/virtenv/env1/lib/python3.6/site-packages/keras/engine/training_generator.py", line 220, in fit_generator
    reset_metrics=False)
  File "*/virtenv/env1/lib/python3.6/site-packages/keras/engine/training.py", line 1508, in train_on_batch
    class_weight=class_weight)
  File "*/virtenv/env1/lib/python3.6/site-packages/keras/engine/training.py", line 579, in _standardize_user_data
    exception_prefix='input')
  File "*/virtenv/env1/lib/python3.6/site-packages/keras/engine/training_utils.py", line 135, in standardize_input_data
    'with shape ' + str(data_shape))
ValueError: Error when checking input: expected input_1 to have 4 dimensions, but got array with shape (2, 32, 256, 256, 1)

其中32为batch_size.

with 32 being the batch_size.

除了RGB图像之外，是否有人已经设法通过多层输入来训练unet(或任何其他CNN)?还是有人对我如何使事情正常工作有想法?

Has anyone already managed to train a unet (or any other CNN) with multi-layer input other than RGB images ? Or does anyone have an idea of how I could get thing working ?

谢谢.

推荐答案

正如@ bit01所建议的，np.stack比输入数组增加了一个额外的维数！为了使工作正常，我编辑了MultipleInputTrainGenerator函数的最后一行，如下所示:

As suggested by @bit01, np.stack is adding one additional dimension than the input arrays! To get things working I edited the last line of the MultipleInputTrainGenerator function as below :

img = np.squeeze(np.stack((img1, img2), axis=3), axis=4)
yield (img, mask1)

它也应该与np.concatenate一起使用，但是我没有尝试.

It should work to with np.concatenate too but I didn't try it out.

这篇关于u-net中的多层灰度输入的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！