多输入深度学习中的平均层 [英] Average layer in multi input deep learning

问题描述

我正在努力在Keras中为图像分类创建一个多输入卷积神经网络(CNN)模型，该模型将获取两个图像并给出一个输出，即两个图像的类.

I am working to create a multi-input Convolutional Neural Network (CNN) model in Keras for Images Classification that takes two images and gives one output which is the class of the two images.

我有两个数据集:type1和type2，每个数据集包含相同的类.该模型应从Type1数据集中获取一幅图像，从Type2数据集中获取一幅图像，然后将这些图像分类为一类(ClassA或ClassB或------).

I have two datasets: type1 and type2, and each dataset contains the same classes. The model should take one image from Type1 dataset and one image from Type2 dataset and then classify these images to one class (ClassA or ClassB or------).

我想创建一个模型来预测两个图像，然后计算与以下图像相似的预测平均值:

如何创建此模型? 如何在fit_generator中创建生成器?

How can I create this model? How can I create the generator in fit_generator?

推荐答案

选项1-双方都是同一模型，只是使用了不同的输入

假设您有一个称为谓词"的模型，称为predModel.
创建两个输入张量:

Option 1 - Both sides are the same model, just using different inputs

Suppose you have a model that goes up to "predication", called predModel.
Create two input tensors:

input1 = Input(shape)   
input2 = Input(shape)

获取每个输入的输出:

pred1 = predModel(input1)
pred2 = predModel(input2)   

平均输出:

output = Average()([pred1,pred2])

创建最终模型:

model = Model([input1,input2], output)

选项2-双方都是相似的模型，但是使用不同的权重

与上面基本相同，但分别为每一面创建图层.

Option2 - Both sides are similar models, but use different weights

Basically the same as above, but create the layers individually for each side.

def createCommonPart(inputTensor):
    out = ZeroPadding2D(...)(inputTensor)
    out = Conv2D(...)(out)

    ...
    out = Flatten()(out)
    return Dense(...)(out)

进行两个输入:

input1 = Input(shape)   
input2 = Input(shape)

获取两个输出:

pred1 = createCommonPart(input1)
pred2 = createCommonPart(input2)

平均输出:

output = Average()([pred1,pred2])

创建最终模型:

model = Model([input1,input2], output)

发电机

任何产生[xTrain1,xTrain2], y的东西.

您可以这样创建一个:

def generator(files1,files2, batch_size):

    while True: #must be infinite

        for i in range(len(files1)//batch_size)):
            bStart = i*batch_size
            bEnd = bStart+batch_size

            x1 = loadImagesSomehow(files1[bStart:bEnd])
            x2 = loadImagesSomehow(files2[bStart:bEnd])
            y = loadPredictionsSomeHow(forSamples[bStart:bEnd])

            yield [x1,x2], y

您也可以通过类似的方式实现keras.utils.Sequence.

You can also implement a keras.utils.Sequence in a similar way.

class gen(Sequence):
    def __init__(self, files1, files2, batchSize):
        self.files1 = files1
        self.files2 = files2
        self.batchSize = batchSize

    def __len__(self):
        return self.len(files1) // self.batchSize

    def __getitem__(self,i):

        bStart = i*self.batchSize
        bEnd = bStart+self.batchSize 

        x1 = loadImagesSomehow(files1[bStart:bEnd])
        x2 = loadImagesSomehow(files2[bStart:bEnd])
        y = loadPredictionsSomeHow(forSamples[bStart:bEnd])

        return [x1,x2], y

这篇关于多输入深度学习中的平均层的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！