TensorFlow Lite 无法识别 op VarHandleOp [英] TensorFlow Lite does not recognize op VarHandleOp

问题描述

我正在尝试将 TF 模型转换为 TFLite.该模型以 .pb 格式保存，我已使用以下代码对其进行了转换:

I am attempting to convert a TF model to TFLite. The model was saved in .pb format and I have converted it with the following code:

import os
import tensorflow as tf
from tensorflow.core.protobuf import meta_graph_pb2

export_dir = os.path.join('export_dir', '0')
if not os.path.exists('export_dir'):
    os.mkdir('export_dir')

tf.compat.v1.enable_control_flow_v2()
tf.compat.v1.enable_v2_tensorshape()

# I took this function from a tutorial on the TF website
def wrap_frozen_graph(graph_def, inputs, outputs):
    def _imports_graph_def():
        tf.compat.v1.import_graph_def(graph_def, name="")
    wrapped_import = tf.compat.v1.wrap_function(_imports_graph_def, [])
    import_graph = wrapped_import.graph
    return wrapped_import.prune(
            inputs, outputs)

graph_def = tf.compat.v1.GraphDef()
loaded = graph_def.ParseFromString(open(os.path.join(export_dir, 'saved_model.pb'),'rb').read())

concrete_func = wrap_frozen_graph(
        graph_def, inputs=['extern_data/placeholders/data/data:0', 'extern_data/placeholders/data/data_dim0_size:0'],
    outputs=['output/output_batch_major:0'])
concrete_func.inputs[0].set_shape([None, 50])
concrete_func.inputs[1].set_shape([None])
concrete_func.outputs[0].set_shape([None, 100])

converter = tf.lite.TFLiteConverter.from_concrete_functions([concrete_func])
converter.experimental_new_converter = True
converter.post_training_quantize=True
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS,
                                               tf.lite.OpsSet.SELECT_TF_OPS]
converter.allow_custom_ops=True

tflite_model = converter.convert()

# Save the model.
if not os.path.exists('tflite'):
    os.mkdir('tflite')
output_model = os.path.join('tflite', 'model.tflite')
with open(output_model, 'wb') as f:
     f.write(tflite_model)

但是，当我尝试在此模型中使用 intepretere 时，出现以下错误:

However, when I try to use the intepretere with this model I get the following error:

INFO: TfLiteFlexDelegate delegate: 8 nodes delegated out of 970 nodes with 3 partitions.

INFO: TfLiteFlexDelegate delegate: 0 nodes delegated out of 4 nodes with 0 partitions.

INFO: TfLiteFlexDelegate delegate: 3 nodes delegated out of 946 nodes with 1 partitions.

INFO: TfLiteFlexDelegate delegate: 0 nodes delegated out of 1 nodes with 0 partitions.

INFO: TfLiteFlexDelegate delegate: 3 nodes delegated out of 16 nodes with 2 partitions.

Traceback (most recent call last):
  File "/path/to/tflite_interpreter.py", line 9, in <module>
    interpreter.allocate_tensors()
  File "/path/to/lib/python3.6/site-packages/tensorflow/lite/python/interpreter.py", line 243, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Encountered unresolved custom op: VarHandleOp.Node number 0 (VarHandleOp) failed to prepare.

现在，我在代码中没有找到任何 VarHandleOp 并且我发现它实际上是在 tensorflow (https://www.tensorflow.org/api_docs/python/tf/raw_ops/VarHandleOp).那么，为什么 TFLite 无法识别它?

Now, I don't find any VarHandleOp in the code and I found out that it is actually in tensorflow (https://www.tensorflow.org/api_docs/python/tf/raw_ops/VarHandleOp). So, why isn't TFLite able to recognize it?

推荐答案

正如 SO 指南所建议的那样，在模型转换的情况下提供最小的可重现示例当然很难，但问题会从更好的指针中受益.例如，与其说我从 TF 网站上的教程中获取了这个函数"，不如提供一个教程链接.TF 网站非常庞大.

It's certainly hard to provide a minimal reproducible example in the case of model conversion, as the SO guidelines recommend, but the questions would benefit from better pointers. For example, instead of saying "I took this function from a tutorial on the TF website", it is a much better idea to provide a link to the tutorial. The TF website is vastly huge.

您所指的教程可能来自关于从 TF1 迁移到 TF2 的部分，特别是处理原始图形文件的部分.最重要的一点是

The tutorial that you are referring to is probably from the section on migrating from TF1 to TF2, specifically the part of handling the raw graph files. The crucially important note is

如果您有冻结图"(a tf.Graph 其中变量已转换为常量)

if you have a "Frozen graph" (a tf.Graph where the variables have been turned into constants)

(粗体突出显示的是我的).显然，您的图形包含 VarHandleOp(同样适用于 Variable 和 VariableV2 节点)，并且没有被这个定义冻结".您的一般方法是有道理的，但您需要一个图表，其中包含 Const 节点.您需要在训练时使用变量，但在推理时，应该将其放入图表中.TFLite 作为推理时间框架，不支持变量.

(the bold highlight is mine). Apparently, your graph contains VarHandleOp (the same applies to the Variable and VariableV2 nodes), and is not "frozen" by this definition. Your general approach makes sense, but you need a graph that contains actual trained values for the variables in the form of the Const node. You need variables at the training time, but for inference time, and should be baked into the graph. TFLite, as an inference-time framework, does not support variables.

你的其他想法看起来不错.TFLiteConverter.from_concrete_functions 目前只需要一个 concrete_function，但这就是你从包装图形中得到的.如果运气好，它可能会奏效.

The rest of your idea seems fine. TFLiteConverter.from_concrete_functions currently takes exactly one concrete_function, but this is what you get from wrapping the graph. With enough luck it may work.

有一个实用程序 tensorflow/python/tools/freeze_graph.py 尝试用最新检查点文件中的常量替换 Graph.pb 中的变量.如果您查看它的代码，无论是使用保存的元图 (checkpoint_name.meta) 文件还是将工具指向训练目录都可以消除很多猜测；另外，我认为提供模型目录是获得单个冻结图和分片模型的唯一方法.

There is a utility tensorflow/python/tools/freeze_graph.py that attempts its best to replace variables in a Graph.pb with constants taken from the latest checkpoint file. If you look at its code, either using the saved metagraph (checkpoint_name.meta) file or pointing the tool to the training directory eliminates a lot of guesswork; also, I think that providing the model directory is the only way to get a single frozen graph a sharded model.

我注意到您在示例中仅使用 input 代替了 tf.nest.map_structure(import_graph.as_graph_element,inputs).您可能有其他原因，但如果您这样做是因为 as_graph_element 抱怨数据类型/形状，这可能会通过正确冻结图形来解决.您从冻结图中获得的具体函数将对其输入形状和数据类型有一个很好的了解.一般来说，需要手动设置它们是出乎意料的，而你这样做的事实对我来说似乎很奇怪(但我并不声称对 TF 的这个黑暗角落有广泛的经验).

I noticed that you use just input in place of tf.nest.map_structure(import_graph.as_graph_element, inputs) in the example. You may have other reasons for that, but if you do it because as_graph_element complains about datatype/shape, this is likely to be resolved by freezing the graph properly. The concrete_function that you obtain from the frozen graph will have a good idea about its input shapes and datatypes. Generally, it's unexpected to need to manually set them, and the fact that you do seems odd to me (but I do not claim a broad experience with this dark corner of TF).

map_structure 有一个关键字参数可以跳过检查.

map_structure has a keyword argument to skip the check.

这篇关于TensorFlow Lite 无法识别 op VarHandleOp的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！