如何优化推理一个简单的、保存的 TensorFlow 1.0.1 图? [英] How to optimize for inference a simple, saved TensorFlow 1.0.1 graph?

问题描述

我无法在简单、保存的 TensorFlow 图(Python 2.7；由 pip install tensorflow-gpu==1.0.1 安装的包)上成功运行 optimize_for_inference 模块.

I cannot successfully run the optimize_for_inference module on a simple, saved TensorFlow graph (Python 2.7; package installed by pip install tensorflow-gpu==1.0.1).

这是我的 Python 脚本，用于生成并保存一个简单的图形，以将 5 添加到我的输入 x placeholder 操作.

Here's my Python script to generate and save a simple graph to add 5 to my input x placeholder operation.

import tensorflow as tf

# make and save a simple graph
G = tf.Graph()
with G.as_default():
    x = tf.placeholder(dtype=tf.float32, shape=(), name="x")
    a = tf.Variable(5.0, name="a")
    y = tf.add(a, x, name="y")
    saver = tf.train.Saver()

with tf.Session(graph=G) as sess:
    sess.run(tf.global_variables_initializer())
    out = sess.run(fetches=[y], feed_dict={x: 1.0})
    print(out)
    saver.save(sess=sess, save_path="test_model")

恢复 TensorFlow 图

我有一个简单的恢复脚本，可以重新创建保存的图形并恢复图形参数.两个保存/恢复脚本产生相同的输出.

Restoring TensorFlow Graph

I have a simple restore script that recreates the saved graph and restores graph params. Both the save/restore scripts produce the same output.

import tensorflow as tf

# Restore simple graph and test model output
G = tf.Graph()

with tf.Session(graph=G) as sess:
    # recreate saved graph (structure)
    saver = tf.train.import_meta_graph('./test_model.meta')
    # restore net params
    saver.restore(sess, tf.train.latest_checkpoint('./'))

    x = G.get_operation_by_name("x").outputs[0]
    y = G.get_operation_by_name("y").outputs
    out = sess.run(fetches=[y], feed_dict={x: 1.0})
    print(out[0])

优化尝试

但是，虽然我对优化的期望并不高，但当我尝试优化推理图时，我收到以下错误消息.预期的输出节点似乎不在保存的图中.

Optimization Attempt

But, while I don't expect much in terms of optimization, when I try to optimize the graph for inference, I get the following error message. The expected output node does not appear to be in the saved graph.

$ python -m tensorflow.python.tools.optimize_for_inference --input test_model.data-00000-of-00001 --output opt_model --input_names=x --output_names=y  
Traceback (most recent call last):  
  File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main  
    "__main__", fname, loader, pkg_name)  
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code  
    exec code in run_globals  
  File "/{path}/lib/python2.7/site-packages/tensorflow/python/tools/optimize_for_inference.py", line 141, in <module>  
    app.run(main=main, argv=[sys.argv[0]] + unparsed)  
  File "/{path}/local/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 44, in run  
    _sys.exit(main(_sys.argv[:1] + flags_passthrough))
  File "/{path}/lib/python2.7/site-packages/tensorflow/python/tools/optimize_for_inference.py", line 90, in main  
    FLAGS.output_names.split(","), FLAGS.placeholder_type_enum)  
  File "/{path}/local/lib/python2.7/site-packages/tensorflow/python/tools/optimize_for_inference_lib.py", line 91, in optimize_for_inference  
    placeholder_type_enum)  
  File "/{path}/local/lib/python2.7/site-packages/tensorflow/python/tools/strip_unused_lib.py", line 71, in strip_unused  
    output_node_names)  
  File "/{path}/local/lib/python2.7/site-packages/tensorflow/python/framework/graph_util_impl.py", line 141, in extract_sub_graph  
    assert d in name_to_node_map, "%s is not in graph" % d  
AssertionError: y is not in graph  

进一步的调查让我检查了保存图形的检查点，它只显示了 1 个张量(a，没有 x 和没有 y).

Further investigation led me to inspect the checkpoint of the saved graph, which only shows 1 tensor (a, no x and no y).

(tf-1.0.1) $ python -m tensorflow.python.tools.inspect_checkpoint --file_name ./test_model --all_tensors
tensor_name:  a
5.0

具体问题

1. 为什么我在检查点中看不到 x 和 y?是因为它们是操作而不是张量吗?
2. 由于我需要为 optimize_for_inference 模块提供输入和输出名称，我该如何构建图形以便引用输入和输出节点?

1. Why do I not see x and y in the checkpoint? Is it because they are operations and not tensors?
2. Since I need to provide input and output names to the optimize_for_inference module, how do I build the graph so I can reference the input and output nodes?

推荐答案

这里是关于如何优化推理的详细指南:

optimize_for_inference 模块将 冻结的二进制 GraphDef 文件作为输入，并输出可用于推理的 optimize_for_inference 文件.并且要获得冻结的二进制GraphDef文件，您需要使用模块freeze_graph，该模块采用GraphDef proto，一个SaverDef proto 和一组存储在检查点文件中的变量.实现这一目标的步骤如下:

The optimize_for_inference module takes a frozen binary GraphDef file as input and outputs the optimized Graph Def file which you can use for inference. And to get the frozen binary GraphDef file you need to use the module freeze_graph which takes a GraphDef proto, a SaverDef proto and a set of variables stored in a checkpoint file. The steps to achieve that is given below:

 # make and save a simple graph
 G = tf.Graph()
 with G.as_default():
   x = tf.placeholder(dtype=tf.float32, shape=(), name="x")
   a = tf.Variable(5.0, name="a")
   y = tf.add(a, x, name="y")
   saver = tf.train.Saver()

with tf.Session(graph=G) as sess:
   sess.run(tf.global_variables_initializer())
   out = sess.run(fetches=[y], feed_dict={x: 1.0})

  # Save GraphDef
  tf.train.write_graph(sess.graph_def,'.','graph.pb')
  # Save checkpoint
  saver.save(sess=sess, save_path="test_model")

2.冻结图

python -m tensorflow.python.tools.freeze_graph --input_graph graph.pb --input_checkpoint test_model --output_graph graph_frozen.pb --output_node_names=y

3.优化推理

python -m tensorflow.python.tools.optimize_for_inference --input graph_frozen.pb --output graph_optimized.pb --input_names=x --output_names=y

4.使用优化图

with tf.gfile.GFile('graph_optimized.pb', 'rb') as f:
   graph_def_optimized = tf.GraphDef()
   graph_def_optimized.ParseFromString(f.read())

G = tf.Graph()

with tf.Session(graph=G) as sess:
    y, = tf.import_graph_def(graph_def_optimized, return_elements=['y:0'])
    print('Operations in Optimized Graph:')
    print([op.name for op in G.get_operations()])
    x = G.get_tensor_by_name('import/x:0')
    out = sess.run(y, feed_dict={x: 1.0})
    print(out)

#Output
#Operations in Optimized Graph:
#['import/x', 'import/a', 'import/y']
#6.0

5.对于多个输出名称

如果有多个输出节点，则指定:output_node_names = 'boxes, score, classes' 并导入图，

 boxes,scores,classes, = tf.import_graph_def(graph_def_optimized, return_elements=['boxes:0', 'scores:0', 'classes:0'])

这篇关于如何优化推理一个简单的、保存的 TensorFlow 1.0.1 图?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！