使用Keras和Google Cloud ML的Base64图像 [英] Base64 images with Keras and Google Cloud ML

问题描述

我正在使用Keras预测图像类别.它可以在Google Cloud ML(GCML)中运行，但是为了提高效率，需要对其进行更改以传递base64字符串而不是json数组. 相关文档

I'm predicting image classes using Keras. It works in Google Cloud ML (GCML), but for efficiency need change it to pass base64 strings instead of json array. Related Documentation

我可以轻松地运行python代码以将base64字符串解码为json数组，但是当使用GCML时，我没有机会运行预处理步骤(除非也许在Keras中使用Lambda层，但我不认为这是正确的方法.

I can easily run python code to decode a base64 string into json array, but when using GCML I don't have the opportunity to run a preprocessing step (unless maybe use a Lambda layer in Keras, but I don't think that is the correct approach).

另一个答案建议添加类型为tf.string的tf.placeholder，这很有意义，但是如何将其合并进入Keras模型?

Another answer suggested adding tf.placeholder with type of tf.string, which makes sense, but how to incorporate that into the Keras model?

这里是用于训练模型并将导出的模型保存为GCML的完整代码...

Here is complete code for training the model and saving the exported model for GCML...

import os
import numpy as np
import tensorflow as tf
import keras
from keras import backend as K
from keras.models import Sequential
from keras.layers import Dense, Dropout, Flatten
from keras.layers import Conv2D, MaxPooling2D
from keras.preprocessing import image
from tensorflow.python.platform import gfile

IMAGE_HEIGHT = 138
IMAGE_WIDTH = 106
NUM_CLASSES = 329

def preprocess(filename):
    # decode the image file starting from the filename
    # end up with pixel values that are in the -1, 1 range
    image_contents = tf.read_file(filename)
    image = tf.image.decode_png(image_contents, channels=1)
    image = tf.image.convert_image_dtype(image, dtype=tf.float32) # 0-1
    image = tf.expand_dims(image, 0) # resize_bilinear needs batches
    image = tf.image.resize_bilinear(image, [IMAGE_HEIGHT, IMAGE_WIDTH], align_corners=False)
    image = tf.subtract(image, 0.5)
    image = tf.multiply(image, 2.0) # -1 to 1
    image = tf.squeeze(image,[0])
    return image



filelist = gfile.ListDirectory("images")
sess = tf.Session()
with sess.as_default():
    x = np.array([np.array(     preprocess(os.path.join("images", filename)).eval()      ) for filename in filelist])

input_shape = (IMAGE_HEIGHT, IMAGE_WIDTH, 1)   # 1, because preprocessing made grayscale

# in our case the labels come from part of the filename
y = np.array([int(filename[filename.index('_')+1:-4]) for filename in filelist])
# convert class labels to numbers
y = keras.utils.to_categorical(y, NUM_CLASSES)

########## TODO: something here? ##########
image = K.placeholder(shape=(), dtype=tf.string)
decoded = tf.image.decode_jpeg(image, channels=3)
# scores = build_model(decoded)


model = Sequential()

# model.add(decoded)

model.add(Conv2D(32, kernel_size=(2, 2), activation='relu', input_shape=input_shape))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(64, activation='relu'))
model.add(Dropout(0.25))
model.add(Dense(num_classes, activation='softmax'))

model.compile(loss=keras.losses.categorical_crossentropy,
            optimizer=keras.optimizers.Adadelta(),
            metrics=['accuracy'])

model.fit(
    x,
    y,
    batch_size=64,
    epochs=20,
    verbose=1,
    validation_split=0.2,
    shuffle=False
    )

predict_signature = tf.saved_model.signature_def_utils.build_signature_def(
    inputs={'input_bytes':tf.saved_model.utils.build_tensor_info(model.input)},
    ########## TODO: something here? ##########
    # inputs={'input': image },    # input name must have "_bytes" suffix to use base64.
    outputs={'formId': tf.saved_model.utils.build_tensor_info(model.output)},
    method_name=tf.saved_model.signature_constants.PREDICT_METHOD_NAME
)

builder = tf.saved_model.builder.SavedModelBuilder("exported_model")

builder.add_meta_graph_and_variables(
    sess=K.get_session(),
    tags=[tf.saved_model.tag_constants.SERVING],
    signature_def_map={
        tf.saved_model.signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY: predict_signature
    },
    legacy_init_op=tf.group(tf.tables_initializer(), name='legacy_init_op')
)

builder.save()

这与我的上一个问题有关

更新:

问题的核心是如何将调用解码的占位符合并到Keras模型中.换句话说，在创建将base64字符串解码为张量的占位符之后，如何将其合并到Keras运行的内容中?我认为它必须是一层.

The heart of the question is how to incorporate the placeholder that calls decode into the Keras model. In other words, after creating the placeholder that decodes the base64 string to a tensor, how to incorporate that into what Keras runs? I assume it needs to be a layer.

image = K.placeholder(shape=(), dtype=tf.string)
decoded = tf.image.decode_jpeg(image, channels=3)
model = Sequential()

# Something like this, but this fails because it is a tensor, not a Keras layer.  Possibly this is where a Lambda layer comes in?
model.add(decoded)
model.add(Conv2D(32, kernel_size=(2, 2), activation='relu', input_shape=input_shape))
...

更新2:

尝试使用lambda层来完成此操作...

Trying to use a lambda layer to accomplish this...

import keras
from keras.models import Sequential
from keras.layers import Lambda
from keras import backend as K
import tensorflow as tf

image = K.placeholder(shape=(), dtype=tf.string)
model = Sequential()
model.add(Lambda(lambda image: tf.image.decode_jpeg(image, channels=3), input_shape=() ))

给出错误:TypeError: Input 'contents' of 'DecodeJpeg' Op has type float32 that does not match expected type of string.

推荐答案

首先，我使用tf.keras，但这应该不是什么大问题. 因此，这是一个如何读取base64解码的jpeg的示例:

first of all I use tf.keras but this should not be a big problem. So here is an example of how you can read a base64 decoded jpeg:

def preprocess_and_decode(img_str, new_shape=[299,299]):
    img = tf.io.decode_base64(img_str)
    img = tf.image.decode_jpeg(img, channels=3)
    img = tf.image.resize_images(img, new_shape, method=tf.image.ResizeMethod.BILINEAR, align_corners=False)
    # if you need to squeeze your input range to [0,1] or [-1,1] do it here
    return img
InputLayer = Input(shape = (1,),dtype="string")
OutputLayer = Lambda(lambda img : tf.map_fn(lambda im : preprocess_and_decode(im[0]), img, dtype="float32"))(InputLayer)
base64_model = tf.keras.Model(InputLayer,OutputLayer)   

上面的代码创建了一个模型，该模型采用任何大小的jpeg，将其大小调整为299x299，并以299x299x3张量返回.该模型可以直接导出到saved_model，并用于Cloud ML Engine服务.这有点愚蠢，因为它唯一要做的就是将base64转换为张量.

The code above creates a model that takes a jpeg of any size, resizes it to 299x299 and returns as 299x299x3 tensor. This model can be exported directly to saved_model and used for Cloud ML Engine serving. It is a little bit stupid, since the only thing it does is the convertion of base64 to tensor.

如果您需要将此模型的输出重定向到现有经过训练和编译的模型(例如inception_v3)的输入，则必须执行以下操作:

If you need to redirect the output of this model to the input of an existing trained and compiled model (e.g inception_v3) you have to do the following:

base64_input = base64_model.input
final_output = inception_v3(base64_model.output)
new_model = tf.keras.Model(base64_input,final_output)

可以保存此new_model.它采用base64 jpeg，并返回由inception_v3部分标识的类.

This new_model can be saved. It takes base64 jpeg and returns classes identified by the inception_v3 part.

这篇关于使用Keras和Google Cloud ML的Base64图像的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！