在Cloudml上使用已部署的模型时，在base64中发现无效字符 [英] Invalid character found in base64 while using a deployed model on cloudml

问题描述

为了获得更好的环境，我已经在Cloud ml上上传了预先训练的模型.这是一个从keras转换为Tensorflow中可接受格式的InceptionV3模型.

For better context, I have uploaded a pre-trained model on cloud ml. It's an inceptionV3 model converted from keras to acceptable format in tensorflow.

from keras.applications.inception_v3 import InceptionV3
model = InceptionV3(weights='imagenet') 
from keras.models import Model
intermediate_layer_model = Model(inputs=model.input,outputs=model.layers[311].output) 
with tf.Graph().as_default() as g_input:
    input_b64 = tf.placeholder(shape=(1,),
                               dtype=tf.string,
                               name='input')
    input_bytes = tf.decode_base64(input_b64[0])
    image = tf.image.decode_image(input_bytes)
    image_f = tf.image.convert_image_dtype(image, dtype=tf.float32)
    input_image = tf.expand_dims(image_f, 0)
    output = tf.identity(input_image, name='input_image') 
g_input_def = g_input.as_graph_def()
K.set_learning_phase(0)
sess = K.get_session()
from tensorflow.python.framework import graph_util
g_trans = sess.graph
g_trans_def = graph_util.convert_variables_to_constants(sess,
    g_trans.as_graph_def(),
    [intermediate_layer_model.output.name.replace(':0','')])
with tf.Graph().as_default() as g_combined:
    x = tf.placeholder(tf.string, name="input_b64")

    im, = tf.import_graph_def(g_input_def,
        input_map={'input:0': x},
        return_elements=["input_image:0"])

    pred, = tf.import_graph_def(g_trans_def,
             input_map={intermediate_layer_model.input.name: im,
             'batch_normalization_1/keras_learning_phase:0': False},
             return_elements=[intermediate_layer_model.output.name])

    with tf.Session() as sess2:
        inputs = {"inputs": tf.saved_model.utils.build_tensor_info(x)}
        outputs = {"outputs":tf.saved_model.utils.build_tensor_info(pred)}
        signature =tf.saved_model.signature_def_utils.build_signature_def(
                inputs=inputs,
                outputs=outputs,
        method_name=tf.saved_model.signature_constants.PREDICT_METHOD_NAME
            )

      # save as SavedModel
        b = tf.saved_model.builder.SavedModelBuilder('inceptionv4/')
        b.add_meta_graph_and_variables(sess2,
                      [tf.saved_model.tag_constants.SERVING],
                      signature_def_map={'serving_default': signature})
        b.save()

生成的pb文件在本地使用时可以正常工作.但是，当我将其部署到云ml时，会出现以下错误.

The generated pb file works fine when I use it locally. But when I deploy it on cloud ml I get the following error.

RuntimeError: Prediction failed: Error during model execution: AbortionError(code=StatusCode.INVALID_ARGUMENT, details="Invalid character found in base64.
     [[Node: import/DecodeBase64 = DecodeBase64[_output_shapes=[<unknown>], _device="/job:localhost/replica:0/task:0/device:CPU:0"](import/strided_slice)]]")

以下是我用于获取本地预测的代码.

Following is the code I use for getting local predictions.

import base64
import json

with open('MEL_BE_0.jpg', 'rb') as image_file:
    encoded_string = str(base64.urlsafe_b64encode(image_file.read()),'ascii')

import tensorflow as tf

with tf.Session(graph=tf.Graph()) as sess:
    MetaGraphDef=tf.saved_model.loader.load(
       sess,
       [tf.saved_model.tag_constants.SERVING],
       'inceptionv4')
    input_tensor = tf.get_default_graph().get_tensor_by_name('input_b64:0')
    print(input_tensor)
    avg_tensor = tf.get_default_graph().get_tensor_by_name('import_1/avg_pool/Mean:0')
    print(avg_tensor)
    predictions = sess.run(avg_tensor, {input_tensor: [encoded_string]})

最后，下面是我用来将编码后的字符串包装在发送到cloud-ml引擎的请求中的代码段.

And finally following is the code snippet that I use for wrapping the encoded string in the request that is sent to the cloud-ml engine.

request_body= json.dumps({"key":"0", "image_bytes": {"b64": [encoded_string]}})

推荐答案

您似乎正在尝试使用TensorFlow 进行base64解码，并且使用{"b64": ...} JSON格式.您需要做一个或另一个.我们通常推荐后者.

It looks like you are trying to do the base64 decoding in TensorFlow and use the {"b64": ...} JSON format. You need to do one or the other; we typically recommend the latter.

请注意，输入占位符的外部尺寸必须为None.这可能会使某些事情变得棘手，例如，您要么必须将尺寸调整为大小1(这将阻止您在当前状态下使用批量预测服务)，要么必须我们tf.map_fn进行应用对输入批处理"的每个元素的相同转换集.您可以在此方法中找到该技术的示例例子.

As a side note, your input placeholder must have an outer dimension of None. That can make some things tricky, e.g., you'll either have to reshape the dimensions to be size 1 (which will prevent you from using the batch prediction service in its current state) or you'll have to us tf.map_fn to apply the same set of transformations to each element of the input "batch". You can find an example of that technique in this example.

最后，我建议使用tf.saved_model.simple_save.

总的来说，这是一些修改后的代码.请注意，我要内联您的输入函数(而不是将其序列化为图形def并重新导入):

Putting it altogether, here is some modified code. Note that I'm inlining your input function (as opposed to serializing it to a graph def and reimporting):

HEIGHT = 299
WIDTH = 299

# Get Keras Model
from keras.applications.inception_v3 import InceptionV3
model = InceptionV3(weights='imagenet') 
from keras.models import Model
intermediate_layer_model = Model(inputs=model.input,outputs=model.layers[311].output) 
K.set_learning_phase(0)
sess = K.get_session()
from tensorflow.python.framework import graph_util
g_trans = sess.graph
g_trans_def = graph_util.convert_variables_to_constants(sess,
    g_trans.as_graph_def(),
    [intermediate_layer_model.output.name.replace(':0','')])

# Create inputs to model and export
with tf.Graph().as_default() as g_combined:

  def decode_and_resize(image_bytes):
    image = tf.image.decode_image(image_bytes)
    # Note resize expects a batch_size, but tf_map supresses that index,
    # thus we have to expand then squeeze.  Resize returns float32 in the
    # range [0, uint8_max]
    image = tf.expand_dims(image, 0)
    image = tf.image.resize_bilinear(
        image, [HEIGHT, WIDTH], align_corners=False)
    image = tf.squeeze(image, squeeze_dims=[0])
    image = tf.cast(image, dtype=tf.uint8)
    return image

  input_byes = tf.placeholder(shape=(None,),
                             dtype=tf.string,
                             name='input')

  images = tf.map_fn(
      decode_and_resize, input_bytes, back_prop=False, dtype=tf.uint8)
  images = tf.image.convert_image_dtype(images, dtype=tf.float32)

  pred, = tf.import_graph_def(g_trans_def,
         input_map={intermediate_layer_model.input.name: images,
         'batch_normalization_1/keras_learning_phase:0': False},
         return_elements=[intermediate_layer_model.output.name])

  with tf.Session() as sess2:
      tf.saved_model.simple_save(
          sess2,
          model_dir='inceptionv4/'
          inputs={"inputs": input_bytes},
          outputs={"outputs": pred})

注意:我不确定100％地确定intermediate_layer_model和images的形状是否兼容. images的形状将为[None，height，width，num_channels].

Note: I'm not 100% certain that the shapes of intermediate_layer_model and images are compatible. The shape of images will be [None, height, width, num_channels].

还请注意，您的本地预测代码将有所变化.您无需对图像进行base64编码，而是需要发送批处理"/图像列表，而不是单个图像.像这样:

Also note that your local prediction code will change a bit. You don't base64 encode the images and you need to send a "batch"/list of images rather than single images. Something like:

with open('MEL_BE_0.jpg', 'rb') as image_file:
  encoded_string = image_file.read()

input_tensor = tf.get_default_graph().get_tensor_by_name('input:0')
print(input_tensor)
avg_tensor = tf.get_default_graph().get_tensor_by_name('import_1/avg_pool/Mean:0')
print(avg_tensor)
predictions = sess.run(avg_tensor, {input_tensor: [encoded_string]})

您没有指定要进行批量预测还是在线预测，它们的输入格式相似但略有不同.无论哪种情况，您的模型都不会导出关键"字段(您是要这样做吗?这可能对批量预测很有用，但对于在线而言则无济于事).

You didn't specify whether you're doing batch prediction or online prediction, which have similar but slightly different "formats" for the inputs. In either case, your model is not exporting a "key" field (did you mean to? It's probably helpful for batch prediction, but not for online).

对于批量预测，文件格式为JSON行；每行包含一个示例.每一行都可以像这样从Python生成:

For batch prediction, the file format is JSON lines; each line contains one example. Each line can be generated like so from Python:

example = json.dumps({"image_bytes": {"b64": ENCODED_STRING}})

(请注意现在省略"key").由于您只有一个输入，因此有一个简写:

(Note the omission of "key" for now). Since you only have one input, there is a shorthand:

example = json.dumps({"b64": ENCODED_STRING})

如果要进行在线预测，则会注意到，如果使用gcloud发送请求，则实际上使用与批量预测相同的文件格式.

If you want to do online prediction, you'll note that if you are using gcloud to send requests, you actually use the same file format as for batch prediction.

事实上，我们强烈建议在部署到云之前使用gcloud ml-engine local predict --json-instances=FILE --model-dir=...进行调试.

In fact, we highly recommend using gcloud ml-engine local predict --json-instances=FILE --model-dir=... before deploying to the cloud to help debug.

如果您打算使用gcloud之外的其他客户端(例如，在Web应用程序，移动应用程序，前端服务器等中)，那么您将不会发送文件，并且需要自己构造完整的请求.它与上面的文件格式非常相似.基本上，将JSON行文件的每一行放入一个称为实例"的数组中，即

If you intend to use some other client besides gcloud, e.g., in a web app, mobile app, frontend server, etc., then you won't be sending a file and you need to construct the full request yourself. It's very similar to the file format above. Basically, take each line of the JSON lines file and put them in an array calle "instances", i.e.,

request_body= json.dumps({"instances": [{"image_bytes": {"b64": [encoded_string]}}]})

您可以根据需要使用相同的语法糖:

You can use the same syntactic sugar if you'd like:

request_body= json.dumps({"instances": [{"b64": [encoded_string]}]})

我希望这会有所帮助！

这篇关于在Cloudml上使用已部署的模型时，在base64中发现无效字符的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！