在keras定制层中包括高级计算(类似于scikit) [英] Including advanced computation (scikit-like) in a keras custom layer

问题描述

通常，在将数据输入模型进行分类之前，我会对其进行预处理.

Normally I would preprocess the data before I feed it into my model for classification.

但这是不可能的，因此被卡在了进一步(以某种方式)进一步提高模型性能的过程中，或者直接在模型内部包含了有用的预处理步骤.

This is however not possible and thus am stuck either to enhance the performance of the model further (somehow) or include useful preprocessing steps directly inside the model.

我该怎么做?到目前为止，我找到的最好的解决方案包括使用Keras后端重新实现我想要的功能.这远不是一个好的解决方案，因此我希望有人对如何解决这种情况有一个想法.

How can I do that? The best solution I found thus far, included re-implementing the functionality I want using Keras backend. This is far from a good solution and thus I am hoping someone has an idea, how to salavage the situation.

下面是我发现有用的链接+当前代码.

Below are links I found useful + my current code.

有用的链接:

具有高级计算功能的Keras自定义层

到目前为止，我的代码:

My code thus far:

def freezeBaseModelLayers(baseModel):
    for layer in baseModel.layers:
        layer.trainable = False


def preprocess_input(x):
    # TODO: Not working, but intention should be clear
    numpy_array = tf.unstack(tf.unstack(tf.unstack(x, 224, 0), 224, 0), 1, 0)
    from skimage.feature import hog
    from skimage import data, exposure
    img_adapteq = exposure.equalize_adapthist(numpy_array, orientations=8, pixels_per_cell=(3, 3),
                                              cells_per_block=(1, 1), visualize=True, multichannel=False)
    [x1, x2, x3] = tf.constant(img_adapteq), tf.constant(img_adapteq), tf.constant(img_adapteq)
    img_conc = Concatenate([x1, x2, x3])
    return img_conc


def create(x):
    is_training = tf.get_variable('is_training', (), dtype=tf.bool, trainable=False)
    with tf.name_scope('pretrained'):
        # Add preprocess step here...
        input_layer = Lambda(preprocess_input(x), input_shape=(224, 224, 1), output_shape=(224, 224, 3))

        baseModel = vgg16.VGG16(weights='imagenet', include_top=False, input_shape=(224, 224, 3))    
        freezeBaseModelLayers(baseModel)
        layer = baseModel(input_layer)
        layer = GlobalMaxPooling2D()(layer)
        layer = Dense(1024, activation='relu')(layer)
        layer = Dense(2, activation=None)(layer)
        model = Model(input=input_layer.input, output=layer)
        output = model(x)
        return output




I would like to include prepocessing steps inside my model

The models I am working with are receiving noisy data. In order to enhance the performance of the models, I would like to do some preprocessing steps e.g. equalize_adapthist.

推荐答案

更好的方法是通过自定义keras层.这是一个示例:

A better way to do this is via a custom keras layer. Here is an example:

import tensorflow as tf
from keras.layers import Layer, Input, Conv2D
from keras.models import Model
from keras import backend as K
from skimage.feature import hog
from skimage import data, exposure

def equalize(img):
  img_adapteq = exposure.equalize_adapthist(img)
  return img_adapteq

def preprocess_input(img):
  return tf.py_func(equalize, 
                     [img],
                     'float32',
                     stateful=False,
                     name='custom_image_op')


class CustomLayer(Layer):
  def __init__(self, output_dim, **kwargs):
    self.output_dim = output_dim
    self.trainable = False
    super(CustomLayer, self).__init__(**kwargs)

  def call(self, x):
    res = tf.map_fn(preprocess_input, x)
    res.set_shape([x.shape[0],
                   self.output_dim[1], 
                   self.output_dim[0],
                   x.shape[-1]])
    return res

output_dim = (224,224)
inputs = Input(shape=(224,224,3))
x = CustomLayer(output_dim)(inputs)
x = Conv2D(32, (3,3))(x)
x = Flatten()(x)
x = Dense(1)(x)

model = Model(inputs, x)
model.summary()

# test
sample = np.random.rand(4, 224,224,3).astype(np.float32)
y = np.random.randint(2, size=(4,))

model.compile("sgd", "mse")
model.fit(sample, y)

这篇关于在keras定制层中包括高级计算(类似于scikit)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！