部署在Cloud ML Engine中的Retrained inception_v3模型始终输出相同的预测 [英] Retrained inception_v3 model deployed in Cloud ML Engine always outputs the same predictions

问题描述

我遵循codelab TensorFlow For Poets 进行转移学习inception_v3。它生成retrained_graph.pb和retrained_labels.txt文件，这些文件可用于在本地进行预测（运行 label_image.py ）。

然后，我想将此模型部署到Cloud ML Engine，以便我可以进行在线预测。为此，我必须将retrained_graph.pb导出为SavedModel格式。我设法按照来自Google的@ rhaertel80的这个答案和鲜花云ML引擎的nofollow noreferrer>这个python文件教程。这里是我的代码：

 从tensorflow.contrib导入tensorflow作为tf 
导入图层
 
从tensorflow.python.saved_model导入构建器作为saved_model_builder $ tensorflow.python.saved_model中的$ b $导入signature_constants $ tensorflow.python.saved_model中的b $ b从tensorflow.python.saved_model导入signature_def_utils 
导入tag_constants 
 from tensorflow.python.saved_model import utils as saved_model_utils 
 
 
 export_dir ='../tf_files/saved7'
 retrained_graph ='../tf_files/retrained_graph2.pb'
 label_count = 5 
 
 def build_signature（输入，输出）：
 signature_inputs = {key：saved_model_utils.build_tensor_info（张量）inputs.items（）中的键，张量} 
 signature_outputs = {key：saved_model_utils.build_tensor_info（tensor）for key，tensors in outputs.items（）} 
 
 signature_def = signature_def_utils.build_signature_def（
 signature_inputs，
 si gnature_outputs，
 signature_constants.PREDICT_METHOD_NAME 
）
 
返回signature_def 
 
类GraphReferences（object）：
 def __init __（self）：
 self.examples = None 
 self.train = None 
 self.global_step = None 
 self.metric_updates = [] 
 self.metric_values = [] 
 self .keys = None 
 self.predictions = [] 
 self.input_jpeg = None 
 
 class模型（对象）：
 def __init __（self，label_count）：
 self.label_count = label_count 
 
 def build_image_str_tensor（self）：
 image_str_tensor = tf.placeholder（tf.string，shape = [None]）
 
 def decode_and_resize（image_str_tensor）：
 return image_str_tensor 
 
 image = tf.map_fn（
 decode_and_resize，
 image_str_tensor，
 back_prop = False，
 dtype = tf.string 
）
 
返回image_str_t ensor 
 
 def build_prediction_graph（self，g）：
 tensors = GraphReferences（）
 tensors.examples = tf.placeholder（tf.string，name ='input'，shape = （无））
 tensors.input_jpeg = self.build_image_str_tensor（）
 
 keys_placeholder = tf.placeholder（tf.string，shape = [None]）
 inputs = {
'key'：keys_placeholder，
'image_bytes'：tensors.input_jpeg 
} 
 
 keys = tf.identity（keys_placeholder）
 outputs = {
'key'：键，
'prediction'：g.get_tensor_by_name（'final_result：0'）
} 
 
返回输入，输出
 
 def export（self，output_dir）：
 with tf.Session（graph = tf.Graph（））as sess：
 with tf.gfile.GFile（retrained_graph，rb）as f：
 graph_def = tf.GraphDef（）
 graph_def.ParseFromString（f.read（））
 tf.import_graph_def（graph_def，name =）
 
g = tf.get_default_graph（）
输入，输出= self.build_prediction_graph（g）
 
 signature_def = build_signature （输入=输入，输出=输出）
 signature_def_map = {
 signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY：signature_def 
} 
 
 builder = saved_model_builder.SavedModelBuilder（output_dir）
 builder.add_meta_graph_and_variables（
 sess，
 tags = [tag_constants.SERVING]，
 signature_def_map = signature_def_map 
）
 builder.save（）
 
 model = Model（label_count）
 model.export（export_dir）
  

此代码生成saved_model.pb文件，然后我用它来创建Cloud ML引擎模型。我可以使用 gcloud ml-engine predict --model my_model_name --json-instances request.json 来预测此模型，其中request.json的内容为：

  {key：0，image_bytes：{b64：jpeg_image_base64_encoded}} 
  

然而，无论我在请求中编码哪个jpeg，我都会得到完全相同的错误预测：

预测输出

我想问题在于CloudML Prediction API将base64编码图像字节传递给inception_v3（build_image_str_tensor（）方法的输入张量DecodeJpeg / contents：0码）。任何关于如何解决这个问题的线索，并让我的本地重新训练的模型在Cloud ML引擎上提供正确的预测？

（只要说清楚，问题不在于retrained_graph.pb，因为它在本地运行时会做出正确的预测;也不会在request.json中执行，因为遵循上面的Flowers Cloud ML引擎教程时，同样的请求文件没有问题。）

解决方案

首先是一般警告。 TensorFlow for Poets codelab不是以非常适合生产服务的方式编写的（部分表现为您必须实施的解决方法）。您通常会导出不包含所有额外培训操作的预测特定图。因此，尽管我们可以尝试一起破解某些有用的东西，但可能需要额外的工作来生成此图。

代码的方法似乎是导入一个图，添加一些占位符，然后导出结果。这通常很好。但是，在问题中显示的代码中，您将添加输入占位符，而不实际将它们连接到导入图形中的任何内容。你最终得到一个包含多个断开连接的子图的图，就像（粗略的图）：

$ p $ image_str_tensor [input = image_bytes] - > <没有什么>
keys_placeholder [input = key] - > identity [output = key]
inception_subgraph - > final_graph [output = prediction]


通过 inception_subgraph 我的意思是你正在导入的所有操作。

因此 image_bytes 实际上是无操作并且被忽略; 键会通过;和 prediction 包含运行 inception_subgraph 的结果;因为它没有使用你传递的输入，所以它每次都返回相同的结果（尽管我承认我实际上期望在这里出现错误）。

为了解决这个问题，我们会需要将创建的占位符连接到 inception_subgraph 中已存在的占位符，以创建一个或多或少的图形：

  image_str_tensor [input = image_bytes]  - > inception_subgraph  - > final_graph [output = prediction] 
 keys_placeholder [input = key]  - > identity [output = key] 
  

请注意 image_str_tensor 将按照预测服务的要求成为一批图像，但起始图的输入实际上是单个图像。为了简单起见，我们将以一种拙劣的方式来解决这个问题：我们假设我们将逐个发送图像。如果我们每次发送一个以上的图片，我们会收到错误信息。批量预测将无法正常工作。

您需要的主要更改是import语句，它将已添加的占位符连接到图中现有的输入（您还会看到改变输入形状的代码）：

综合起来，我们得到如下的结果：

 从tensorflow.contrib导入tensorflow作为tf 
导入图层
 $ b从tensorflow.python.saved_model导入生成器作为saved_model_builder 
 from tensorflow.python.saved_model从tensorflow.python.saved_model导入signature_constants 
从tensorflow.python.saved_model导入signature_def_utils 
从tensorflow.python.saved_model导入tag_constants 
导入utils as saved_model_utils 
 
 
 export_dir ='../tf_files/saved7'
 retrained_graph ='../tf_files/retrained_graph2.pb'
 label_count = 5 
 
 class模型（对象）：
 def __init __（self，label_count）：
 self.labe l_count = label_count 
 $ b def build_prediction_graph（self，g）：
 inputs = {$ b $ key'：keys_placeholder，
'image_bytes'：tensors.input_jpeg 
'
 
 keys = tf.identity（keys_placeholder）
 outputs = {
'key'：keys，
'prediction'：g.get_tensor_by_name（'final_result： 0'）
} 
 
返回输入，输出
 
 def export（self，output_dir）：
 with tf.Session（graph = tf.Graph （））as sess：
＃这将是我们接受一批输入的输入
 image_bytes = tf.placeholder（tf.string，name ='input'，shape =（None，））
＃强制它是单个输入;如果我们发送一个批次，会引发错误。 
 coerced = tf.squeeze（image_bytes）
＃当我们导入图表时，我们将`coerced`连接到`DecodeJPGInput：0` 
 input_map = {'DecodeJPGInput：0'：coerced } 
 
 with tf.gfile.GFile（retrained_graph，rb）as f：
 graph_def = tf.GraphDef（）
 graph_def.ParseFromString（f.read（）） 
 tf.import_graph_def（graph_def，input_map = input_map，name =）
 
 keys_placeholder = tf.placeholder（tf.string，shape = [None]）
 
输入= {'image_bytes'：image_bytes，'key'：keys_placeholder} 
 
 keys = tf.identity（keys_placeholder）
 outputs = {
'key'：keys，
'prediction'：tf.get_default_graph（）。get_tensor_by_name（'final_result：0'）} 
} 
 
 tf.simple_save（sess，output_dir，inputs，outputs）
 
 model = Model（label_c ount）
 model.export（export_dir）
  

I followed the codelab TensorFlow For Poets for transfer learning using inception_v3. It generates retrained_graph.pb and retrained_labels.txt files, which can used to make predictions locally (running label_image.py).

Then, I wanted to deploy this model to Cloud ML Engine, so that I could make online predictions. For that, I had to export the retrained_graph.pb to SavedModel format. I managed to do it by following the indications in this answer from Google's @rhaertel80 and this python file from the Flowers Cloud ML Engine Tutorial. Here is my code:

import tensorflow as tf
from tensorflow.contrib import layers

from tensorflow.python.saved_model import builder as saved_model_builder
from tensorflow.python.saved_model import signature_constants
from tensorflow.python.saved_model import signature_def_utils
from tensorflow.python.saved_model import tag_constants
from tensorflow.python.saved_model import utils as saved_model_utils


export_dir = '../tf_files/saved7'
retrained_graph = '../tf_files/retrained_graph2.pb'
label_count = 5

def build_signature(inputs, outputs):
    signature_inputs = { key: saved_model_utils.build_tensor_info(tensor) for key, tensor in inputs.items() }
    signature_outputs = { key: saved_model_utils.build_tensor_info(tensor) for key, tensor in outputs.items() }

    signature_def = signature_def_utils.build_signature_def(
        signature_inputs,
        signature_outputs,
        signature_constants.PREDICT_METHOD_NAME
    )

    return signature_def

class GraphReferences(object):
  def __init__(self):
    self.examples = None
    self.train = None
    self.global_step = None
    self.metric_updates = []
    self.metric_values = []
    self.keys = None
    self.predictions = []
    self.input_jpeg = None

class Model(object):
    def __init__(self, label_count):
        self.label_count = label_count

    def build_image_str_tensor(self):
        image_str_tensor = tf.placeholder(tf.string, shape=[None])

        def decode_and_resize(image_str_tensor):
            return image_str_tensor

        image = tf.map_fn(
            decode_and_resize,
            image_str_tensor,
            back_prop=False,
            dtype=tf.string
        )

        return image_str_tensor

    def build_prediction_graph(self, g):
        tensors = GraphReferences()
        tensors.examples = tf.placeholder(tf.string, name='input', shape=(None,))
        tensors.input_jpeg = self.build_image_str_tensor()

        keys_placeholder = tf.placeholder(tf.string, shape=[None])
        inputs = {
            'key': keys_placeholder,
            'image_bytes': tensors.input_jpeg
        }

        keys = tf.identity(keys_placeholder)
        outputs = {
            'key': keys,
            'prediction': g.get_tensor_by_name('final_result:0')
        }

        return inputs, outputs

    def export(self, output_dir):
        with tf.Session(graph=tf.Graph()) as sess:
            with tf.gfile.GFile(retrained_graph, "rb") as f:
                graph_def = tf.GraphDef()
                graph_def.ParseFromString(f.read())
                tf.import_graph_def(graph_def, name="")

            g = tf.get_default_graph()
            inputs, outputs = self.build_prediction_graph(g)

            signature_def = build_signature(inputs=inputs, outputs=outputs)
            signature_def_map = {
                signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY: signature_def
            }

            builder = saved_model_builder.SavedModelBuilder(output_dir)
            builder.add_meta_graph_and_variables(
                sess,
                tags=[tag_constants.SERVING],
                signature_def_map=signature_def_map
            )
            builder.save()

model = Model(label_count)
model.export(export_dir)

This code generates a saved_model.pb file, which I then used to create the Cloud ML Engine model. I can get predictions from this model using gcloud ml-engine predict --model my_model_name --json-instances request.json, where the contents of request.json are:

{ "key": "0", "image_bytes": { "b64": "jpeg_image_base64_encoded" } }

However, no matter which jpeg I encode in the request, I always get the exact same wrong predictions:

Prediction output

I guess the problem is in the way the CloudML Prediction API passes the base64 encoded image bytes to the input tensor "DecodeJpeg/contents:0" of inception_v3 ("build_image_str_tensor()" method in the previous code). Any clue on how can I solve this issue and have my locally retrained model serving correct predictions on Cloud ML Engine?

(Just to make it clear, the problem is not in retrained_graph.pb, as it makes correct predictions when I run it locally; nor is it in request.json, because the same request file worked without problems when following the Flowers Cloud ML Engine Tutorial pointed above.)

解决方案

First, a general warning. The TensorFlow for Poets codelab was not written in a way that is very amenable to production serving (partly manifested by the workarounds you are having to implement). You would normally export a prediction-specific graph that doesn't contain all of the extra training ops. So while we can try and hack something together that works, extra work may be needed to productionize this graph.

The approach of your code appears to be to import one graph, add some placeholders, and then export the result. This is generally fine. However, in the code shown in the question, you are adding input placeholders without actually connecting them to anything in the imported graph. You end up with a graph containing multiple disconnected subgraphs, something like (excuse the crude diagram):

image_str_tensor [input=image_bytes] -> <nothing>
keys_placeholder [input=key]  -> identity [output=key]
inception_subgraph -> final_graph [output=prediction]

By inception_subgraph I mean all of the ops that you are importing.

So image_bytes is effectively a no-op and is ignored; key gets passed through; and prediction contains the result of running the inception_subgraph; since it's not using the input you are passing, it's returning the same result everytime (though I admit I actually expected an error here).

To address this problem, we would need to connect the placeholder you've created to the one that already exists in inception_subgraph to create a graph more or less like this:

image_str_tensor [input=image_bytes] -> inception_subgraph -> final_graph [output=prediction]
keys_placeholder [input=key]  -> identity [output=key]   

Note that image_str_tensor is going to be a batch of images, as required by the prediction service, but the inception graph's input is actually a single image. In the interest of simplicity, we're going to address this in a hacky way: we'll assume we'll be sending images one-by-one. If we ever send more than one image per request, we'll get errors. Also, batch prediction will never work.

The main change you need is the import statement, which connects the placeholder we've added to the existing input in the graph (you'll also see the code for changing the shape of the input):

Putting it all together, we get something like:

import tensorflow as tf
from tensorflow.contrib import layers

from tensorflow.python.saved_model import builder as saved_model_builder
from tensorflow.python.saved_model import signature_constants
from tensorflow.python.saved_model import signature_def_utils
from tensorflow.python.saved_model import tag_constants
from tensorflow.python.saved_model import utils as saved_model_utils


export_dir = '../tf_files/saved7'
retrained_graph = '../tf_files/retrained_graph2.pb'
label_count = 5

class Model(object):
    def __init__(self, label_count):
        self.label_count = label_count

    def build_prediction_graph(self, g):
        inputs = {
            'key': keys_placeholder,
            'image_bytes': tensors.input_jpeg
        }

        keys = tf.identity(keys_placeholder)
        outputs = {
            'key': keys,
            'prediction': g.get_tensor_by_name('final_result:0')
        }

        return inputs, outputs

    def export(self, output_dir):
        with tf.Session(graph=tf.Graph()) as sess:
            # This will be our input that accepts a batch of inputs
            image_bytes = tf.placeholder(tf.string, name='input', shape=(None,))
            # Force it to be a single input; will raise an error if we send a batch.
            coerced = tf.squeeze(image_bytes)
            # When we import the graph, we'll connect `coerced` to `DecodeJPGInput:0`
            input_map = {'DecodeJPGInput:0': coerced}

            with tf.gfile.GFile(retrained_graph, "rb") as f:
                graph_def = tf.GraphDef()
                graph_def.ParseFromString(f.read())
                tf.import_graph_def(graph_def, input_map=input_map, name="")

            keys_placeholder = tf.placeholder(tf.string, shape=[None])

            inputs = {'image_bytes': image_bytes, 'key': keys_placeholder}

            keys = tf.identity(keys_placeholder)
            outputs = {
                'key': keys,
                'prediction': tf.get_default_graph().get_tensor_by_name('final_result:0')}    
            }

            tf.simple_save(sess, output_dir, inputs, outputs)

model = Model(label_count)
model.export(export_dir)

这篇关于部署在Cloud ML Engine中的Retrained inception_v3模型始终输出相同的预测的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！