在Tensorflow-lite中输入具有动态尺寸的图像 [英] Input images with dynamic dimensions in Tensorflow-lite

问题描述

我有一个tensorflow模型，可以拍摄各种大小的输入图像:

I have a tensorflow model that takes input images of varying size:

inputs = layers.Input(shape=(128,None,1), name='x_input')

<tf.Tensor 'x_input:0' shape=(?, 128, ?, 1) dtype=float32>

当我将此模型转换为tensorflow-lite时，它会抱怨:

When I convert this model to tensorflow-lite it complains:

converter = tf.lite.TFLiteConverter.from_frozen_graph(
  graph_def_file, input_arrays, output_arrays)
tflite_model = converter.convert() 

ValueError: None is only supported in the 1st dimension.
Tensor 'x_input_1' has invalid shape '[None, 128, None, 1]'.

我无法将图像缩放到固定大小.我看到的唯一解决方案是将图像填充到最大尺寸，并在图形中使用该尺寸，但这似乎很浪费.还有其他方法可以使tensorflow-lite与动态图像尺寸一起使用吗?对此限制有任何理由吗?谢谢.

I cannot scale my images to a fixed size. The only solution I see is to pad the images to some maximum size and use that one in the graph, but that seems pretty wasteful. Is there any other way to make tensorflow-lite work with dynamic image dimensions? And is there any rationale for this limitation? Thanks.

推荐答案

是的，您可以在TF-Lite中使用动态张量.无法将形状直接设置为[None, 128, None, 1]的原因是，通过这种方式，将来可以轻松支持更多语言.此外，它充分利用了静态内存分配方案.对于打算用于具有低计算能力的小型设备的框架来说，这是一个明智的设计选择. 以下是有关如何动态设置张量大小的步骤:

Yes, you can use dynamic tensors in TF-Lite. The reason why you can't directly set the shape to [None, 128, None, 1] is because this way, you can easily support more languages in the future. Furthermore, it makes the best use of static memory allocation scheme. This is a smart design choice for a framework that is intended to be used on small devices with low computation power. Here are the steps on how to dynamically set the tensor's size:

似乎您正在从冻结的GraphDef(即*.pb文件)进行转换.假设您的冻结模型的输入形状为[None, 128, None, 1].

It seems like you're converting from a frozen GraphDef, i.e. a *.pb file. Suppose your frozen model has input shape [None, 128, None, 1].

在此步骤中，将输入尺寸设置为模型可以接受的任何有效尺寸.例如:

During this step, set the input size to any valid one that can be accepted by your model. For example:

tflite_convert \
  --graph_def_file='model.pb' \
  --output_file='model.tflite' \
  --input_shapes=1,128,80,1 \     # <-- here, you set an
                                  #     arbitrary valid shape
  --input_arrays='input' \         
  --output_arrays='Softmax'

2.推理步骤

诀窍是在推理过程中实时使用TF-Lite API的函数interpreter::resize_tensor_input(...).我将提供它的python实现. Java和C ++实现应相同(因为它们具有相似的API):

2. Inference step

The trick is to use the function interpreter::resize_tensor_input(...) of the TF-Lite API in real time during inference. I will provide a python implementation of it. The Java and C++ implementation should be the same (as they have similar API):

from tensorflow.contrib.lite.python import interpreter

# Load the *.tflite model and get input details
model = Interpreter(model_path='model.tflite')
input_details = model.get_input_details()

# Your network currently has an input shape (1, 128, 80 , 1),
# but suppose you need the input size to be (2, 128, 200, 1).
model.resize_tensor_input(
    input_details[0]['index'], (2, 128, 200, 1))
model.allocate_tensors()

就是这样.现在，只要您的网络体系结构允许输入这样的形状，就可以将该模型用于形状为(2, 128, 200, 1)的图像.请注意，每次进行这样的重塑时，您都必须执行model.allocate_tensors()，这样效率会非常低下. 强烈建议，以避免在程序中过多使用此功能.

That's it. You can now use that model for images with shape (2, 128, 200, 1), as long as your network architecture allows such an input shape. Beware that you will have to do model.allocate_tensors() every time you do such a reshape, so it will be very inefficient. It is strongly recommended to avoid using this function too much in your program.

这篇关于在Tensorflow-lite中输入具有动态尺寸的图像的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！