对 TensorFlow 服务的 RaggedTensor 请求失败 [英] RaggedTensor request to TensorFlow serving fails
问题描述
我创建了一个使用 RaggedTensors 的 TensorFlow 模型.模型工作正常，当调用 model.predict 时，我得到了预期的结果.
I've created a TensorFlow model that uses RaggedTensors. Model works fine and when calling model.predict and I get the expected results.
<代码>输入= tf.ragged.constant([[[ - 0.9984272718429565，-0.9422321319580078，-0.27657580375671387，-3.185823678970337，-0.6360141634941101，-1.6579184532165527，-1.9000954627990723，-0.49169546365737915，-0.6758883595466614，-0.6677696704864502，-0.532067060470581],[-0.9984272718429565，-0.9421600103378296，2.2048349380493164，-1.273996114730835，-0.6360141634941101，-1.5917999744415283，0.6147914528846741，-0.49169546365737915，-0.6673409938812256，-0.6583622694015503，-0.5273991227149963][-0.9984272718429565，-0.942145586013794，2.48842453956604，-1.6836735010147095，-0.6360141634941101，-1.5785763263702393，-1.900200605392456，-0.49169546365737915，-0.6656315326690674，-0.6583622694015503，-0.5273991227149963]]])模型.预测(输入)>>数组([[0.5138151, 0.3277698, 0.26122513]], dtype=float32)
我已将模型部署到 TensorFlow 服务服务器并使用以下代码进行调用:
I've deployed the model to a TensorFlow serving server and using the following code to invoke:
import json
import requests
headers = {"content-type": "application/json"}
data = json.dumps({"instances":[
[-1.3523329846758267, ... more data ],
[-1.3523329846758267, ... more data ],
[-1.3523329846758267, ... more data ],
[-1.3523329846758267, ... more data ,
[-1.3523329846758267, ... more data ],
[-1.3523329846758267, ... more data ],
[-1.3523329846758267, ... more data ],
[-1.3523329846758267, ... more data })
json_response = requests.post('http://localhost:8501/v1/models/fashion_model:predict', data=data, headers=headers)
predictions = json.loads(json_response.text)
但是我收到以下错误:
"instances is a plain list, but expecting list of objects as multiple input tensors required as per tensorinfo_map"
我的模型描述:
MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:
signature_def['__saved_model_init_op']:
The given SavedModel SignatureDef contains the following input(s):
The given SavedModel SignatureDef contains the following output(s):
outputs['__saved_model_init_op'] tensor_info:
dtype: DT_INVALID
shape: unknown_rank
name: NoOp
Method name is:
signature_def['serving_default']:
The given SavedModel SignatureDef contains the following input(s):
inputs['args_0'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 11)
name: serving_default_args_0:0
inputs['args_0_1'] tensor_info:
dtype: DT_INT64
shape: (-1)
name: serving_default_args_0_1:0
The given SavedModel SignatureDef contains the following output(s):
outputs['dense_2'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 3)
name: StatefulPartitionedCall:0
Method name is: tensorflow/serving/predict
WARNING: Logging before flag parsing goes to stderr.
W0124 09:33:16.365564 140189730998144 deprecation.py:506] From /usr/local/lib/python2.7/dist-packages/tensorflow_core/python/ops/resource_variable_ops.py:1786: calling __init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
Defined Functions:
Function Name: '__call__'
Option #1
Callable with:
Argument #1
DType: RaggedTensorSpec
Value: RaggedTensorSpec(TensorShape([None, None, 11]), tf.float32, 1, tf.int64)
Argument #2
DType: bool
Value: True
Argument #3
DType: NoneType
Value: None
Option #2
Callable with:
Argument #1
DType: RaggedTensorSpec
Value: RaggedTensorSpec(TensorShape([None, None, 11]), tf.float32, 1, tf.int64)
Argument #2
DType: bool
Value: False
Argument #3
DType: NoneType
Value: None
Function Name: '_default_save_signature'
Option #1
Callable with:
Argument #1
DType: RaggedTensorSpec
Value: RaggedTensorSpec(TensorShape([None, None, 11]), tf.float32, 1, tf.int64)
Function Name: 'call_and_return_all_conditional_losses'
Option #1
Callable with:
Argument #1
DType: RaggedTensorSpec
Value: RaggedTensorSpec(TensorShape([None, None, 11]), tf.float32, 1, tf.int64)
Argument #2
DType: bool
Value: True
Argument #3
DType: NoneType
Value: None
Option #2
Callable with:
Argument #1
DType: RaggedTensorSpec
Value: RaggedTensorSpec(TensorShape([None, None, 11]), tf.float32, 1, tf.int64)
Argument #2
DType: bool
Value: False
Argument #3
DType: NoneType
Value: None
我错过了什么?
更新:检查 saved_model_cli 输出后，我怀疑我应该将请求作为如下对象发送，但我不确定输入...
Update:
After inspecting saved_model_cli output, I suspect I should send the request as an object like below, but I'm not sure about the inputs...
{
"instances": [
{
"args_0": nested-list ?,
"args_0_1": ???
}
]
}
更新 2一个 Colab 来测试这个场景，下载模型的链接是包含在 Colab 中.
Update2 A Colab to test this scenario, a link to download the model is included in the Colab.
更新 3:
正如@Niteya Shah 所建议的那样，我调用了 API:
As suggested by @Niteya Shah I've called the API with:
data = json.dumps({
"inputs": {
"args_0": [[-0.9984272718429565, -0.9422321319580078, -0.27657580375671387, -3.185823678970337, -0.6360141634941101, -1.6579184532165527, -1.9000954627990723, -0.49169546365737915, -0.6758883595466614, -0.6677696704864502, -0.532067060470581],
[-0.9984272718429565, -0.9421600103378296, 2.2048349380493164, -1.273996114730835, -0.6360141634941101, -1.5917999744415283, 0.6147914528846741, -0.49169546365737915, -0.6673409938812256, -0.6583622694015503, -0.5273991227149963]],
"args_0_1": [1, 2] #Please Check what inputs come here ?
}
})
并得到结果(终于！):
And got the results (Finally!):
{'outputs': [[0.466771603, 0.455221593, 0.581544757]]}
然后使用相同的数据调用模型，如下所示:
Then called the model with the same data like so:
import numpy as np
input = tf.ragged.constant([[
[-0.9984272718429565, -0.9422321319580078, -0.27657580375671387, -3.185823678970337, -0.6360141634941101, -1.6579184532165527, -1.9000954627990723, -0.49169546365737915, -0.6758883595466614, -0.6677696704864502, -0.532067060470581],
[-0.9984272718429565, -0.9421600103378296, 2.2048349380493164, -1.273996114730835, -0.6360141634941101, -1.5917999744415283, 0.6147914528846741, -0.49169546365737915, -0.6673409938812256, -0.6583622694015503, -0.5273991227149963]
]])
model.predict(input)
得到了不同的结果:
array([[0.4817084 , 0.3649785 , 0.01603118]], dtype=float32)
我想我还没到.
推荐答案
https://www.tensorflow.org/tfx/serving/api_rest#predict_api
我认为您需要使用 REST API 中推荐的列格式而不是行格式，因为您的第 0 个输入的尺寸不匹配.这意味着您将不得不使用输入而不是实例.由于您还有多个输入，因此您还必须将其作为命名输入提及.
I think that you need to use a columnar format as recommended in the REST API instead of the row format because the dimensions of your 0th input do not match. This means that instead of instances you will have to use inputs. Since you also have multiple inputs, you will have to also mention that as a named input.
示例数据请求可能如下所示
A sample data request could look like this
data = json.dumps({
"inputs": {
"args_0": [[-0.9984272718429565, -0.9422321319580078, -0.27657580375671387, -3.185823678970337, -0.6360141634941101, -1.6579184532165527, -1.9000954627990723, -0.49169546365737915, -0.6758883595466614, -0.6677696704864502, -0.532067060470581],
[-0.9984272718429565, -0.9421600103378296, 2.2048349380493164, -1.273996114730835, -0.6360141634941101, -1.5917999744415283, 0.6147914528846741, -0.49169546365737915, -0.6673409938812256, -0.6583622694015503, -0.5273991227149963]],
"args_0_1": [10, 11] #Substitute this with the correct row partition values.
}
})
我从 here 读到了 Ragged tensors，似乎第二个输入可能是行分区.我在有关通常使用的行分区样式的文档中找不到它，所以我使用的是行长度方法.幸运的是，TensorFlow ragged 为我们提供了方法.使用 values 和 row_splits 属性来访问它们.那应该可行.
I read about Ragged tensors from here and it seems that the second input may be the row partitions. I couldn't find it in the documentation about what row partition style is normally used so I am using the row lengths method.
Luckily TensorFlow ragged provides methods that do this for us. Use the values and row_splits properties to access them. That should work.
这篇关于对 TensorFlow 服务的 RaggedTensor 请求失败的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
