对TensorFlow服务的RaggedTensor请求失败 [英] RaggedTensor request to TensorFlow serving fails

问题描述

我创建了一个使用RaggedTensors的TensorFlow模型.模型工作正常，当调用model.predict时，我得到了预期的结果.

I've created a TensorFlow model that uses RaggedTensors. Model works fine and when calling model.predict and I get the expected results.

input = tf.ragged.constant([[[-0.9984272718429565, -0.9422321319580078, -0.27657580375671387, -3.185823678970337, -0.6360141634941101, -1.6579184532165527, -1.9000954627990723, -0.49169546365737915, -0.6758883595466614, -0.6677696704864502, -0.532067060470581], 
                                [-0.9984272718429565, -0.9421600103378296, 2.2048349380493164, -1.273996114730835, -0.6360141634941101, -1.5917999744415283, 0.6147914528846741, -0.49169546365737915, -0.6673409938812256, -0.6583622694015503, -0.5273991227149963], 
                                [-0.9984272718429565, -0.942145586013794, 2.48842453956604, -1.6836735010147095, -0.6360141634941101, -1.5785763263702393, -1.900200605392456, -0.49169546365737915, -0.6656315326690674, -0.6583622694015503, -0.5273991227149963], 
]])
model.predict(input)

>> array([[0.5138151 , 0.3277698 , 0.26122513]], dtype=float32)

我已将模型部署到TensorFlow服务服务器上，并使用以下代码来调用:

I've deployed the model to a TensorFlow serving server and using the following code to invoke:

import json
import requests
headers = {"content-type": "application/json"}
data = json.dumps({"instances":[
    [-1.3523329846758267, ... more data ],
    [-1.3523329846758267, ... more data ],
    [-1.3523329846758267, ... more data ],
    [-1.3523329846758267, ... more data ,
    [-1.3523329846758267, ... more data ],
    [-1.3523329846758267, ... more data ],
    [-1.3523329846758267, ... more data ],
    [-1.3523329846758267, ... more data })
json_response = requests.post('http://localhost:8501/v1/models/fashion_model:predict', data=data, headers=headers)
predictions = json.loads(json_response.text)

但是随后出现以下错误:

But then I get the following error:

"instances is a plain list, but expecting list of objects as multiple input tensors required as per tensorinfo_map"

我的模特描述:

MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:

signature_def['__saved_model_init_op']:
  The given SavedModel SignatureDef contains the following input(s):
  The given SavedModel SignatureDef contains the following output(s):
    outputs['__saved_model_init_op'] tensor_info:
        dtype: DT_INVALID
        shape: unknown_rank
        name: NoOp
  Method name is: 

signature_def['serving_default']:
  The given SavedModel SignatureDef contains the following input(s):
    inputs['args_0'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 11)
        name: serving_default_args_0:0
    inputs['args_0_1'] tensor_info:
        dtype: DT_INT64
        shape: (-1)
        name: serving_default_args_0_1:0
  The given SavedModel SignatureDef contains the following output(s):
    outputs['dense_2'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 3)
        name: StatefulPartitionedCall:0
  Method name is: tensorflow/serving/predict
WARNING: Logging before flag parsing goes to stderr.
W0124 09:33:16.365564 140189730998144 deprecation.py:506] From /usr/local/lib/python2.7/dist-packages/tensorflow_core/python/ops/resource_variable_ops.py:1786: calling __init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.

Defined Functions:
  Function Name: '__call__'
    Option #1
      Callable with:
        Argument #1
          DType: RaggedTensorSpec
          Value: RaggedTensorSpec(TensorShape([None, None, 11]), tf.float32, 1, tf.int64)
        Argument #2
          DType: bool
          Value: True
        Argument #3
          DType: NoneType
          Value: None
    Option #2
      Callable with:
        Argument #1
          DType: RaggedTensorSpec
          Value: RaggedTensorSpec(TensorShape([None, None, 11]), tf.float32, 1, tf.int64)
        Argument #2
          DType: bool
          Value: False
        Argument #3
          DType: NoneType
          Value: None

  Function Name: '_default_save_signature'
    Option #1
      Callable with:
        Argument #1
          DType: RaggedTensorSpec
          Value: RaggedTensorSpec(TensorShape([None, None, 11]), tf.float32, 1, tf.int64)

  Function Name: 'call_and_return_all_conditional_losses'
    Option #1
      Callable with:
        Argument #1
          DType: RaggedTensorSpec
          Value: RaggedTensorSpec(TensorShape([None, None, 11]), tf.float32, 1, tf.int64)
        Argument #2
          DType: bool
          Value: True
        Argument #3
          DType: NoneType
          Value: None
    Option #2
      Callable with:
        Argument #1
          DType: RaggedTensorSpec
          Value: RaggedTensorSpec(TensorShape([None, None, 11]), tf.float32, 1, tf.int64)
        Argument #2
          DType: bool
          Value: False
        Argument #3
          DType: NoneType
          Value: None

我想念什么?

更新: 检查saved_model_cli输出后，我怀疑应该将请求作为如下对象发送，但我不确定输入...

Update: After inspecting saved_model_cli output, I suspect I should send the request as an object like below, but I'm not sure about the inputs...

{
  "instances": [
    {
      "args_0": nested-list ?,
      "args_0_1": ???
    }
  ]
}

更新2 使用 Colab 进行测试的情况，用于下载模型的链接为包含在Colab中.

Update2 A Colab to test this scenario, a link to download the model is included in the Colab.

更新3:

正如@Niteya Shah所建议的那样，我使用以下命令调用了API:

As suggested by @Niteya Shah I've called the API with:

data = json.dumps({
 "inputs": {
   "args_0": [[-0.9984272718429565, -0.9422321319580078, -0.27657580375671387, -3.185823678970337, -0.6360141634941101, -1.6579184532165527, -1.9000954627990723, -0.49169546365737915, -0.6758883595466614, -0.6677696704864502, -0.532067060470581], 
              [-0.9984272718429565, -0.9421600103378296, 2.2048349380493164, -1.273996114730835, -0.6360141634941101, -1.5917999744415283, 0.6147914528846741, -0.49169546365737915, -0.6673409938812256, -0.6583622694015503, -0.5273991227149963]],
   "args_0_1": [1, 2]  #Please Check what inputs come here ?
  }
})

得到结果(最后！):

{'outputs': [[0.466771603, 0.455221593, 0.581544757]]}

然后使用相同的数据调用模型，如下所示:

Then called the model with the same data like so:

import numpy as np
input = tf.ragged.constant([[
                            [-0.9984272718429565, -0.9422321319580078, -0.27657580375671387, -3.185823678970337, -0.6360141634941101, -1.6579184532165527, -1.9000954627990723, -0.49169546365737915, -0.6758883595466614, -0.6677696704864502, -0.532067060470581], 
                            [-0.9984272718429565, -0.9421600103378296, 2.2048349380493164, -1.273996114730835, -0.6360141634941101, -1.5917999744415283, 0.6147914528846741, -0.49169546365737915, -0.6673409938812256, -0.6583622694015503, -0.5273991227149963]
]])
model.predict(input)

得到不同的结果:

array([[0.4817084 , 0.3649785 , 0.01603118]], dtype=float32)

我想我仍然不在那儿.

推荐答案

https://www.tensorflow.org/tfx/serving/api_rest#predict_api

我认为您需要使用REST API中推荐的列格式而不是行格式，因为第0个输入的尺寸不匹配. 这意味着您将不得不使用输入而不是实例. 由于您还具有多个输入，因此您还必须将其作为命名输入提及.

I think that you need to use a columnar format as recommended in the REST API instead of the row format because the dimensions of your 0th input do not match. This means that instead of instances you will have to use inputs. Since you also have multiple inputs, you will have to also mention that as a named input.

一个示例数据请求可能看起来像这样

A sample data request could look like this

data = json.dumps({
 "inputs": {
   "args_0": [[-0.9984272718429565, -0.9422321319580078, -0.27657580375671387, -3.185823678970337, -0.6360141634941101, -1.6579184532165527, -1.9000954627990723, -0.49169546365737915, -0.6758883595466614, -0.6677696704864502, -0.532067060470581], 
              [-0.9984272718429565, -0.9421600103378296, 2.2048349380493164, -1.273996114730835, -0.6360141634941101, -1.5917999744415283, 0.6147914528846741, -0.49169546365737915, -0.6673409938812256, -0.6583622694015503, -0.5273991227149963]],
   "args_0_1": [10, 11]  #Substitute this with the correct row partition values. 
  }
})

我从此处了解了参差不齐的张量，看来第二个输入可能是行分区.我在文档中找不到有关通常使用哪种行分区样式的文档，因此我正在使用行长度方法. 幸运的是，参差不齐的TensorFlow提供了方法来为我们完成此任务.使用values和row_splits属性访问它们.那应该起作用.

I read about Ragged tensors from here and it seems that the second input may be the row partitions. I couldn't find it in the documentation about what row partition style is normally used so I am using the row lengths method. Luckily TensorFlow ragged provides methods that do this for us. Use the values and row_splits properties to access them. That should work.

这篇关于对TensorFlow服务的RaggedTensor请求失败的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！