如何使用 base64 编码图像为 TensorFlow Serving REST 接口制作模型? [英] How to a make a model ready for TensorFlow Serving REST interface with a base64 encoded image?
问题描述
我的理解是，我应该能够从 Google 的 AI Hub 获取一个 TensorFlow 模型，将其部署到 TensorFlow Serving 并使用它通过使用 curl 通过 REST 请求发布图像来进行预测.
我目前在 AI Hub 上找不到任何 bbox 预测器，但我在 TensorFlow 模型动物园找到了一个:
但是如何将此 SignatureDefinition 转换为有效的 JSON 请求?
我假设我应该使用 predict API ...
和 Google 的文档说......
<块引用>网址
发布http://host:port/v1/models/${MODEL_NAME}[/versions/${MODEL_VERSION}]:预测
/versions/${MODEL_VERSION} 是可选的.如果省略最新版本使用.
请求格式
预测 API 的请求主体必须是 JSON 对象格式如下:
<代码>{//(可选)要使用的服务签名.//如果使用未指定的默认服务签名."signature_name": <string>,//以行(实例")或列(输入")格式输入张量.//一个请求可以拥有其中一个，但不能同时拥有.实例":<值>|<(嵌套)列表>|<对象列表>输入":<值>|<(嵌套)列表>|<对象>}编码二进制值 JSON 使用 UTF-8 编码.如果你有输入需要二进制的特征或张量值(如图像字节)，您必须对数据进行 Base64 编码并将其封装在 JSON 对象中以 b64 为键，如下所示:
{ "b64": "base64 编码的字符串" }您可以将此对象指定为输入特征或张量的值.同样的格式也用于编码输出响应.
具有图像(二进制数据)和标题特征的分类请求如下图所示:
{ "signature_name": "classify_objects", "examples": [{"图像": { "b64": "aW1hZ2UgYnl0ZXM="},"caption": "海边"},{"图像": { "b64": "YXdlc29tZSBpbWFnZSBieXRlcw==" },"caption": "山"}]}不确定性包括:
- 我应该在我的 JSON 中使用实例"吗
- 我应该对 JPG 或 PNG 或其他东西进行 base64 编码吗?
- 图片是否应该是特定的宽度和高度?
在 使用 TensorFlow-Serving 的 RESTful API 提供基于图像的深度学习模型 建议采用以下格式:
<代码>{实例":[{"b64": "iVBORw"},{b64":pT4rmN"}，{b64":w0KGg2"}]}我使用了这张图片:https://tensorflow.org/images/blogs/serving/cat.jpg
和 base64 编码如下:
# 下载图片dl_request = requests.get(IMAGE_URL, 流=真)dl_request.raise_for_status()# 编写一个 JSON 预测请求(以 base64 格式发送 JPEG 图像).jpeg_bytes = base64.b64encode(dl_request.content).decode('utf-8')predict_request = '{"instances" : [{"b64": "%s"}]}' % jpeg_bytes但是当我使用 curl 发布 base64 编码的图像时:
{"instances" : [{"b64": "/9j/4AAQSkZJRgABAQAASABIAAD/4QBYRXhpZgAATU0AKgAA...KACiiigAooooAKKKKACiiigAooooA//Z"}]}我收到这样的回复:
>./test_local_tfs.shHEADER=|内容类型:应用程序/json；字符集=UTF-8|URL=|http://127.0.0.1:8501/v1/models/saved_model/versions/1:predict|* 正在尝试 127.0.0.1...* TCP_NODELAY 设置* 连接到 127.0.0.1 (127.0.0.1) 端口 8501 (#0)>POST/v1/models/saved_model/versions/1:predict HTTP/1.1>主机:127.0.0.1:8501>用户代理:curl/7.54.0>接受: */*>内容类型:应用程序/json；字符集=UTF-8>内容长度:85033>期望:100-继续><HTTP/1.1 100 继续*我们已经完全上传好了<HTTP/1.1 400 错误请求<内容类型:应用程序/json<日期:2019 年 9 月 17 日，星期二 10:47:18 GMT<内容长度:85175<{ "error": "Failed to process element: 0 of \'instances\' list. Error: Invalid argument: JSON Value: {\n \"b64\": \"/9j/4AAQSkZJRgABAQAAS...ooooA//Z\"\n} 类型:对象不是预期类型:uint8" }我已经尝试将同一文件的本地版本转换为 base64 像这样(确认 dtype 是 uint8)...
img = cv2.imread('cat.jpg')打印('dtype:' + str(img.dtype))_, buf = cv2.imencode('.jpg', img)jpeg_bytes = base64.b64encode(buf).decode('utf-8')predict_request = '{"instances" : [{"b64": "%s"}]}' % jpeg_bytes但是发布这个 JSON 会产生同样的错误.
然而，当 json 被格式化成这样......
{'instances': [[[[112, 71, 48], [104, 63, 40], [107, 70, 20], [108, 72, 21], [109, 77], 0], [106, 75, 0], [92, 66, 0], [106, 80, 0], [101, 80, 0], [98, 77, 0], [100, 75, 0], [104, 80, 0], [114, 88, 17], [94, 68, 0], [85, 54, 0], [103, 72, 11], [93, 62, 0],[120, 89, 25], [131, 101, 37], [125, 95, 31], [119, 91, 27], [121, 93, 29], [133, 105, 40], [119], 91, 27], [119, 96, 56], [120, 97, 57], [119, 96, 53], [102, 78, 36], [132, 103, 44], [117, 88], 28], [125, 89, 4], [128, 93, 8], [133, 94, 0], [126, 87, 0], [110, 74, 0], [123, 87, 2], [120, 92, 30], [124, 95, 33], [114, 90, 32],..., [43, 24, 33], [30, 17, 36], [24, 11, 30], [29, 20, 38], [37, 28, 46]]]]}...它有效.问题是这个 json 文件的大小 > 11 MB.
如何使 json 的 base64 编码版本正常工作?
更新:看来我们必须编辑预训练模型以在输入层接受 base64 图像
本文介绍了如何编辑模型...媒体:提供图像-基于 TensorFlow-Serving 的 RESTful API 的深度学习模型...不幸的是，它假设我们可以访问生成模型的代码.
user260826 的解决方案提供了一种使用估算器的解决方法，但它假定模型是 keras 模型.在这种情况下不正确.
是否有一种通用方法可以使用适用于任何 TensorFlow 模型格式的 base64 编码图像使模型为 TensorFlow Serving REST 接口做好准备?
第一步是以适当的格式导出训练好的模型.像这样使用 export_inference_graph.py
python export_inference_graph \--input_type encoding_image_string_tensor \--pipeline_config_path path/to/ssd_inception_v2.config \--trained_checkpoint_prefix path/to/model.ckpt \--output_directory path/to/exported_model_directory在上面的代码片段中，重要的是要指定
<块引用>--input_type encoding_image_string_tensor
导出模型后，像往常一样使用新导出的模型运行 tensorflow 服务器.
推理代码如下所示:
from __future__ import print_function导入 base64进口请求SERVER_URL = 'http://localhost:8501/v1/models/vedNet:predict'IMAGE_URL = 'test_images/19_inp.jpg'定义主():使用 open(IMAGE_URL, "rb") 作为 image_file:jpeg_bytes = base64.b64encode(image_file.read()).decode('utf-8')predict_request = '{"instances" : [{"b64": "%s"}]}' % jpeg_bytes响应 = requests.post(SERVER_URL，predict_request)response.raise_for_status()预测 = response.json()['预测'][0]如果 __name__ == '__main__':主要的()My understanding is that I should be able to grab a TensorFlow model from Google's AI Hub, deploy it to TensorFlow Serving and use it to make predictions by POSTing images via REST requests using curl.
I could not find any bbox predictors on AI Hub at this time but I did find one on the TensorFlow model zoo:
http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v2_coco_2018_03_29.tar.gz
I have the model deployed to TensorFlow serving, but the documentation is unclear with respect to exactly what should be included in the JSON of the REST request.
My understanding is that
- The SignatureDefinition of the model determines what the JSON should look like
- I should base64 encode the images
I was able to get the signature definition of the model like so:
>python tensorflow/tensorflow/python/tools/saved_model_cli.py show --dir /Users/alexryan/alpine/git/tfserving-tutorial3/model-volume/models/bbox/1/ --all
MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:
signature_def['serving_default']:
The given SavedModel SignatureDef contains the following input(s):
inputs['in'] tensor_info:
dtype: DT_UINT8
shape: (-1, -1, -1, 3)
name: image_tensor:0
The given SavedModel SignatureDef contains the following output(s):
outputs['out'] tensor_info:
dtype: DT_FLOAT
shape: unknown_rank
name: detection_boxes:0
Method name is: tensorflow/serving/predict
I think the shape info here is telling me that the model can handle images of any dimensions?
The input layer looks like this in Tensorboard:
But how do I convert this SignatureDefinition to a valid JSON request?
I'm assuming that I'm supposed to use the predict API ...
and Google's doc says ...
URL
POST http://host:port/v1/models/${MODEL_NAME}[/versions/${MODEL_VERSION}]:predict
/versions/${MODEL_VERSION} is optional. If omitted the latest version is used.
Request format
The request body for predict API must be JSON object formatted as follows:{ // (Optional) Serving signature to use. // If unspecifed default serving signature is used. "signature_name": <string>, // Input Tensors in row ("instances") or columnar ("inputs") format. // A request can have either of them but NOT both. "instances": <value>|<(nested)list>|<list-of-objects> "inputs": <value>|<(nested)list>|<object> }Encoding binary values JSON uses UTF-8 encoding. If you have input feature or tensor values that need to be binary (like image bytes), you must Base64 encode the data and encapsulate it in a JSON object having b64 as the key as follows:
{ "b64": "base64 encoded string" }You can specify this object as a value for an input feature or tensor. The same format is used to encode output response as well.
A classification request with image (binary data) and caption features is shown below:
{ "signature_name": "classify_objects", "examples": [
{
"image": { "b64": "aW1hZ2UgYnl0ZXM=" },
"caption": "seaside"
},
{
"image": { "b64": "YXdlc29tZSBpbWFnZSBieXRlcw==" },
"caption": "mountains"
} ] }
Uncertainties include:
- should I use "instances" in my JSON
- should I base64 encode a JPG or PNG or something else?
- Should the image be of a particular width and height?
In Serving Image-Based Deep Learning Models with TensorFlow-Serving’s RESTful API this format is suggested:
{
"instances": [
{"b64": "iVBORw"},
{"b64": "pT4rmN"},
{"b64": "w0KGg2"}
]
}
I used this image: https://tensorflow.org/images/blogs/serving/cat.jpg
and base64 encoded it like so:
# Download the image
dl_request = requests.get(IMAGE_URL, stream=True)
dl_request.raise_for_status()
# Compose a JSON Predict request (send JPEG image in base64).
jpeg_bytes = base64.b64encode(dl_request.content).decode('utf-8')
predict_request = '{"instances" : [{"b64": "%s"}]}' % jpeg_bytes
But when I use curl to POST the base64 encoded image like so:
{"instances" : [{"b64": "/9j/4AAQSkZJRgABAQAASABIAAD/4QBYRXhpZgAATU0AKgAA
...
KACiiigAooooAKKKKACiiigAooooA//Z"}]}
I get a response like this:
>./test_local_tfs.sh
HEADER=|Content-Type:application/json;charset=UTF-8|
URL=|http://127.0.0.1:8501/v1/models/saved_model/versions/1:predict|
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 8501 (#0)
> POST /v1/models/saved_model/versions/1:predict HTTP/1.1
> Host: 127.0.0.1:8501
> User-Agent: curl/7.54.0
> Accept: */*
> Content-Type:application/json;charset=UTF-8
> Content-Length: 85033
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
< HTTP/1.1 400 Bad Request
< Content-Type: application/json
< Date: Tue, 17 Sep 2019 10:47:18 GMT
< Content-Length: 85175
<
{ "error": "Failed to process element: 0 of \'instances\' list. Error: Invalid argument: JSON Value: {\n \"b64\": \"/9j/4AAQSkZJRgABAQAAS
...
ooooA//Z\"\n} Type: Object is not of expected type: uint8" }
I've tried converting a local version of the same file to base64 like so (confirming that the dtype is uint8) ...
img = cv2.imread('cat.jpg')
print('dtype: ' + str(img.dtype))
_, buf = cv2.imencode('.jpg', img)
jpeg_bytes = base64.b64encode(buf).decode('utf-8')
predict_request = '{"instances" : [{"b64": "%s"}]}' % jpeg_bytes
But posting this JSON generates the same error.
However, when the json is formated like so ...
{'instances': [[[[112, 71, 48], [104, 63, 40], [107, 70, 20], [108, 72, 21], [109, 77, 0], [106, 75, 0], [92, 66, 0], [106, 80, 0], [101, 80, 0], [98, 77, 0], [100, 75, 0], [104, 80, 0], [114, 88, 17], [94, 68, 0], [85, 54, 0], [103, 72, 11], [93, 62, 0], [120, 89, 25], [131, 101, 37], [125, 95, 31], [119, 91, 27], [121, 93, 29], [133, 105, 40], [119, 91, 27], [119, 96, 56], [120, 97, 57], [119, 96, 53], [102, 78, 36], [132, 103, 44], [117, 88, 28], [125, 89, 4], [128, 93, 8], [133, 94, 0], [126, 87, 0], [110, 74, 0], [123, 87, 2], [120, 92, 30], [124, 95, 33], [114, 90, 32],
...
, [43, 24, 33], [30, 17, 36], [24, 11, 30], [29, 20, 38], [37, 28, 46]]]]}
... it works. The problem is this json file is >11 MB in size.
How do I make the base64 encoded version of the json work?
UPDATE: It seems that we have to edit the pretrained model to accept base64 images at the input layer
This article describes how to edit the model ... Medium: Serving Image-Based Deep Learning Models with TensorFlow-Serving’s RESTful API ... unfortunately, it assumes that we have access to the code which generated the model.
user260826's solution provides a work-around using an estimator but it assumes the model is a keras model. Not true in this case.
Is there a generic method to make a model ready for TensorFlow Serving REST interface with a base64 encoded image that works with any of the TensorFlow model formats?
The first step is to export the trained model in the appropriate format. Use export_inference_graph.py like this
python export_inference_graph \
--input_type encoded_image_string_tensor \
--pipeline_config_path path/to/ssd_inception_v2.config \
--trained_checkpoint_prefix path/to/model.ckpt \
--output_directory path/to/exported_model_directory
in the above code snippet, it is important to specify
--input_type encoded_image_string_tensor
after exporting the model, run the tensorflow server as usual with the newly exported model.
The inference code will look like this:
from __future__ import print_function
import base64
import requests
SERVER_URL = 'http://localhost:8501/v1/models/vedNet:predict'
IMAGE_URL = 'test_images/19_inp.jpg'
def main():
with open(IMAGE_URL, "rb") as image_file:
jpeg_bytes = base64.b64encode(image_file.read()).decode('utf-8')
predict_request = '{"instances" : [{"b64": "%s"}]}' % jpeg_bytes
response = requests.post(SERVER_URL, predict_request)
response.raise_for_status()
prediction = response.json()['predictions'][0]
if __name__ == '__main__':
main()
这篇关于如何使用 base64 编码图像为 TensorFlow Serving REST 接口制作模型?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
