使用gcloud ml服务获取大图像 [英] Using gcloud ml serving for large images
问题描述
我有一个受过训练的张量流网络，我希望将其用于gcloud ml-engine进行预测.
I have a trained net in tensorflow that i wish to use in gcloud ml-engine serving for prediction.
预测gcloud ml服务应接受大小为320x240x3的numpy数组float32类型图像，并返回2个微小矩阵作为输出.
Predict gcloud ml serving should accept numpy array float32 type images with size of 320x240x3 and return 2 tiny matrices as an output.
有人知道我应该如何创建将接受这种输入类型的输入层吗?
Does anyone knows how should i create the input layers that would accept this kind of input type?
我尝试了多种方式，例如使用base64编码的json文件，但是将字符串转换为float类型会产生一个错误，其中不支持该错误:
I have tried multiple ways, for example using base64 encoded json files, but casting the string into float type produces an error in which it's not supported:
"error": "Prediction failed: Exception during model execution: LocalError(code=StatusCode.UNIMPLEMENTED, details=\"Cast string to float is not supported\n\t [[Node: ToFloat = Cast[DstT=DT_FLOAT, SrcT=DT_STRING, _output_shapes=[[-1,320,240,3]], _device=\"/job:localhost/replica:0/task:0/cpu:0\"](ParseExample/ParseExample)]]\")"
这是创建json文件的示例(将上面的numpy数组另存为jpeg之后):
This is an example of creating the json file (after saving the numpy array above as jpeg):
python -c 'import base64, sys, json; img = base64.b64encode(open(sys.argv[1], "rb").read()); print json.dumps({"images": {"b64": img}})' example_img.jpg &> request.json
以及试图处理输入的tensorflow命令:
And the tensorflow commands attempting to handle the input:
raw_str_input = tf.placeholder(tf.string, name='source')
feature_configs = {
'image': tf.FixedLenFeature(
shape=[], dtype=tf.string),
}
tf_example = tf.parse_example(raw_str_input, feature_configs)
input = tf.identity(tf.to_float(tf_example['image/encoded']), name='input')
以上是其中一项测试的示例，还尝试了多次尝试使用不同的tensorflow命令来处理输入，但均无效果...
the above is an example of one of the tests done, also tried multiple attempts of different tensorflow commands to handle the input but none of them worked...
推荐答案
我建议不要使用parse_example作为开头.有几种发送图像数据的选项，每个选项都在复杂性和有效负载大小之间进行权衡:
I would recommended not using parse_example to start with. There are several options for sending image data, each with tradeoffs in complexity and payload size:
- 原始张量编码为JSON
- 张量包装为字节字符串
- 压缩图像数据
在每种情况下，请务必注意，输入占位符的形状的外部尺寸必须为无".这是"batch_size"尺寸(即使您打算将图像一张一张地发送到服务，该尺寸也是必需的).
In each case, it is important to note that the input placeholders must have 'None' as the outer-dimension of their shape. This is the "batch_size" dimension (required, even if you intend to send images one-by-one to the service).
原始张量编码为JSON
# Dimensions represent [batch size, height width, channels]
input_images = tf.placeholder(dtype=tf.float32, shape=[None,320,240,3], name='source')
output_tensor = foo(input_images)
# Export the SavedModel
inputs = {'image': input_images}
outputs = {'output': output_tensor}
# ....
您发送给服务的JSON如下所示:已记录(请参见实例JSON字符串").例如，(我建议尽可能多地删除空格；为了便于阅读，在此处打印漂亮的空格):
The JSON you send to the service will look like as documented (see "Instances JSON string"). For example, (I recommend removing as much white space as possible; pretty printed here for readability):
{
"instances": [
{
"image": [
[
[1,1,1], [1,1,1], ... 240 total ... [1,1,1]
],
... 320 total ...
[
[1,1,1], [1,1,1], ... 240 total ... [1,1,1]
]
]
},
{
"image": [ ... repeat if you have more than one image in the request ... ]
]
}
请注意，gcloud从输入文件格式构建请求正文，其中每个输入都位于单独的行上(并且大多数都打包在一行中)，即:
Please note that gcloud builds that request body from an input file format where each input is on a separate line (and most be packed on a single line), i.e.:
{"image": [[[1,1,1], [1,1,1], <240 of these>] ... <320 of these>]}
{"image": [[[2,2,2], [2,2,2], <240 of these>] ... <320 of these>]}
打包为字节字符串的张量
如果要在客户端上进行大小调整等，我的建议是发送一个字节字符串. JSON可能是通过网络发送浮动消息的一种相当低效的方式.即使发送整数数据也会导致膨胀.相反，您可以在客户端上对字节进行编码，然后在TensorFlow中对其进行解码.我的建议是使用uint8数据.
If you're doing resizing, etc. on the client, my recommendation is to send a byte string. JSON can be a fairly inefficient way to send floats over the wire; even sending integer data causes bloat. Instead, you can encode the bytes on the client and decode them in TensorFlow. My recommendation is to use uint8 data.
这是TensorFlow模型代码，用于解码字节字符串:
This is the TensorFlow Model code to decode bytes strings:
raw_byte_strings = tf.placeholder(dtype=tf.string, shape=[None], name='source')
# Decode the images. The shape of raw_byte_strings is [batch size]
# (were batch size is determined by how many images are sent), and
# the shape of `input_images` is [batch size, 320, 240, 3]. It's
# important that all of the images sent have the same dimensions
# or errors will result.
#
# We have to use a map_fn because decode_raw only works on a single
# image, and we need to decode a batch of images.
decode = lambda raw_byte_str: tf.decode_raw(raw_byte_str, tf.uint8)
input_images = tf.map_fn(decode, raw_byte_strings, dtype=tf.uint8)
output_tensor = foo(input_images)
# Export the SavedModel
inputs = {'image_bytes': input_images}
outputs = {'output': output_tensor}
# ....
此处有一个特别说明:正如Jeremy Lewi所指出的那样，此输入别名的名称 必须 以_bytes(image_bytes)结尾.这是因为JSON没有区分文本形式的二进制数据的方法.
One special note here: as pointed out by Jeremy Lewi, the name of this input alias must end in _bytes (image_bytes). This is because JSON doesn't have a way of distinguish text form binary data.
请注意，可以将相同的技巧应用于浮动数据，而不仅仅是uint8数据.
Note that the same trick can be applied to float data, not just uint8 data.
您的客户端将负责创建uint8s的字节字符串.这是您使用numpy在Python中执行此操作的方式.
Your client would be responsible for creating a bytes string of uint8s. Here's how you would do that in Python using numpy.
import base64
import json
import numpy as np
images = []
# In real life, this is obtained via other means, e.g. scipy.misc.imread), for now, an array of all 1s
images.append(np.array([[[2]*3]*240]*320], dtype=np.uint8))
# If we want, we can send more than one image:
images.append(np.array([[[2]*3]*240]*320], dtype=np.uint8))
# Convert each image to byte strings
bytes_strings = (i.tostring() for i in images)
# Base64 encode the data
encoded = (base64.b64encode(b) for b in bytes_strings)
# Create a list of images suitable to send to the service as JSON:
instances = [{'image_bytes': {'b64': e}} for e in encoded]
# Create a JSON request
request = json.dumps({'instances': instances})
# Or if dumping a file for gcloud:
file_data = '\n'.join(json.dumps(instances))
压缩图像数据
发送原始图像并在TensorFlow中进行大小调整和解码通常非常方便. 此示例中对此进行了举例说明我在这里不再重复.客户端只需要发送原始的JPEG字节.关于_bytes后缀的相同注释.
It is often most convenient to send the original images and do the resizing and decoding in TensorFlow. This is exemplified in this sample, which I won't repeat here. The client simply needs to send the raw JPEG bytes. Same note about _bytes suffix applies here.
这篇关于使用gcloud ml服务获取大图像的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
