带有 Keras 和 Google Cloud ML 的 Base64 图像 [英] Base64 images with Keras and Google Cloud ML

问题描述

我正在使用 Keras 预测图像类别.它适用于 Google Cloud ML (GCML)，但为了提高效率需要将其更改为传递 base64 字符串而不是 json 数组.相关文档

I'm predicting image classes using Keras. It works in Google Cloud ML (GCML), but for efficiency need change it to pass base64 strings instead of json array. Related Documentation

我可以轻松地运行 python 代码将 base64 字符串解码为 json 数组，但是在使用 GCML 时我没有机会运行预处理步骤(除非可能在 Keras 中使用 Lambda 层，但我不认为这是正确的方法).

I can easily run python code to decode a base64 string into json array, but when using GCML I don't have the opportunity to run a preprocessing step (unless maybe use a Lambda layer in Keras, but I don't think that is the correct approach).

另一个答案建议添加 tf.placeholder 类型为 tf.string，有道理，但如何将其合并到 Keras 模型中?

Another answer suggested adding tf.placeholder with type of tf.string, which makes sense, but how to incorporate that into the Keras model?

这里是训练模型和为 GCML 保存导出模型的完整代码...

Here is complete code for training the model and saving the exported model for GCML...

import os
import numpy as np
import tensorflow as tf
import keras
from keras import backend as K
from keras.models import Sequential
from keras.layers import Dense, Dropout, Flatten
from keras.layers import Conv2D, MaxPooling2D
from keras.preprocessing import image
from tensorflow.python.platform import gfile

IMAGE_HEIGHT = 138
IMAGE_WIDTH = 106
NUM_CLASSES = 329

def preprocess(filename):
    # decode the image file starting from the filename
    # end up with pixel values that are in the -1, 1 range
    image_contents = tf.read_file(filename)
    image = tf.image.decode_png(image_contents, channels=1)
    image = tf.image.convert_image_dtype(image, dtype=tf.float32) # 0-1
    image = tf.expand_dims(image, 0) # resize_bilinear needs batches
    image = tf.image.resize_bilinear(image, [IMAGE_HEIGHT, IMAGE_WIDTH], align_corners=False)
    image = tf.subtract(image, 0.5)
    image = tf.multiply(image, 2.0) # -1 to 1
    image = tf.squeeze(image,[0])
    return image



filelist = gfile.ListDirectory("images")
sess = tf.Session()
with sess.as_default():
    x = np.array([np.array(     preprocess(os.path.join("images", filename)).eval()      ) for filename in filelist])

input_shape = (IMAGE_HEIGHT, IMAGE_WIDTH, 1)   # 1, because preprocessing made grayscale

# in our case the labels come from part of the filename
y = np.array([int(filename[filename.index('_')+1:-4]) for filename in filelist])
# convert class labels to numbers
y = keras.utils.to_categorical(y, NUM_CLASSES)

########## TODO: something here? ##########
image = K.placeholder(shape=(), dtype=tf.string)
decoded = tf.image.decode_jpeg(image, channels=3)
# scores = build_model(decoded)


model = Sequential()

# model.add(decoded)

model.add(Conv2D(32, kernel_size=(2, 2), activation='relu', input_shape=input_shape))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(64, activation='relu'))
model.add(Dropout(0.25))
model.add(Dense(num_classes, activation='softmax'))

model.compile(loss=keras.losses.categorical_crossentropy,
            optimizer=keras.optimizers.Adadelta(),
            metrics=['accuracy'])

model.fit(
    x,
    y,
    batch_size=64,
    epochs=20,
    verbose=1,
    validation_split=0.2,
    shuffle=False
    )

predict_signature = tf.saved_model.signature_def_utils.build_signature_def(
    inputs={'input_bytes':tf.saved_model.utils.build_tensor_info(model.input)},
    ########## TODO: something here? ##########
    # inputs={'input': image },    # input name must have "_bytes" suffix to use base64.
    outputs={'formId': tf.saved_model.utils.build_tensor_info(model.output)},
    method_name=tf.saved_model.signature_constants.PREDICT_METHOD_NAME
)

builder = tf.saved_model.builder.SavedModelBuilder("exported_model")

builder.add_meta_graph_and_variables(
    sess=K.get_session(),
    tags=[tf.saved_model.tag_constants.SERVING],
    signature_def_map={
        tf.saved_model.signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY: predict_signature
    },
    legacy_init_op=tf.group(tf.tables_initializer(), name='legacy_init_op')
)

builder.save()

这与我的上一个问题有关.

更新:

问题的核心是如何将调用 decode 的占位符合并到 Keras 模型中.换句话说，在创建将 base64 字符串解码为张量的占位符之后，如何将其合并到 Keras 运行中?我认为它需要是一个层.

The heart of the question is how to incorporate the placeholder that calls decode into the Keras model. In other words, after creating the placeholder that decodes the base64 string to a tensor, how to incorporate that into what Keras runs? I assume it needs to be a layer.

image = K.placeholder(shape=(), dtype=tf.string)
decoded = tf.image.decode_jpeg(image, channels=3)
model = Sequential()

# Something like this, but this fails because it is a tensor, not a Keras layer.  Possibly this is where a Lambda layer comes in?
model.add(decoded)
model.add(Conv2D(32, kernel_size=(2, 2), activation='relu', input_shape=input_shape))
...

更新 2:

尝试使用 lambda 层来实现这一点...

Trying to use a lambda layer to accomplish this...

import keras
from keras.models import Sequential
from keras.layers import Lambda
from keras import backend as K
import tensorflow as tf

image = K.placeholder(shape=(), dtype=tf.string)
model = Sequential()
model.add(Lambda(lambda image: tf.image.decode_jpeg(image, channels=3), input_shape=() ))

给出错误:TypeError: Input 'contents' of 'DecodeJpeg' Op 的类型 float32 与预期的字符串类型不匹配.

推荐答案

首先我使用 tf.keras 但这应该不是什么大问题.下面是一个如何读取 base64 解码 jpeg 的示例:

first of all I use tf.keras but this should not be a big problem. So here is an example of how you can read a base64 decoded jpeg:

def preprocess_and_decode(img_str, new_shape=[299,299]):
    img = tf.io.decode_base64(img_str)
    img = tf.image.decode_jpeg(img, channels=3)
    img = tf.image.resize_images(img, new_shape, method=tf.image.ResizeMethod.BILINEAR, align_corners=False)
    # if you need to squeeze your input range to [0,1] or [-1,1] do it here
    return img
InputLayer = Input(shape = (1,),dtype="string")
OutputLayer = Lambda(lambda img : tf.map_fn(lambda im : preprocess_and_decode(im[0]), img, dtype="float32"))(InputLayer)
base64_model = tf.keras.Model(InputLayer,OutputLayer)   

上面的代码创建了一个模型，该模型采用任意大小的 jpeg，将其调整为 299x299 并返回为 299x299x3 张量.此模型可以直接导出到 saved_model 并用于 Cloud ML Engine 服务.这有点愚蠢，因为它唯一做的就是将 base64 转换为张量.

The code above creates a model that takes a jpeg of any size, resizes it to 299x299 and returns as 299x299x3 tensor. This model can be exported directly to saved_model and used for Cloud ML Engine serving. It is a little bit stupid, since the only thing it does is the convertion of base64 to tensor.

如果您需要将此模型的输出重定向到现有训练和编译模型(例如 inception_v3)的输入，您必须执行以下操作:

If you need to redirect the output of this model to the input of an existing trained and compiled model (e.g inception_v3) you have to do the following:

base64_input = base64_model.input
final_output = inception_v3(base64_model.output)
new_model = tf.keras.Model(base64_input,final_output)

这个 new_model 可以保存.它采用 base64 jpeg 并返回由 inception_v3 部分标识的类.

This new_model can be saved. It takes base64 jpeg and returns classes identified by the inception_v3 part.

这篇关于带有 Keras 和 Google Cloud ML 的 Base64 图像的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！