替换 tensorflow v2 的占位符 [英] Replacing placeholder for tensorflow v2

问题描述

对于我的项目，我需要将有向图转换为该图的 tensorflow 实现，就好像它是一个神经网络一样.在 tensorflow 版本 1 中，我可以将所有输入定义为占位符，然后使用图形的广度优先搜索为输出生成数据流图.然后我将使用 feed_dict 输入我的输入.然而，在 TensorFlow v2.0 中，他们决定完全取消占位符.

For my project, I need to convert a directed graph into a tensorflow implementation of the graph as if it was a neural network. In tensorflow version 1, I could just define all of my inputs as placeholders and then just generate the dataflow graph for the outputs using a breadthfirst search of the graph. Then I would just feed in my inputs using a feed_dict. However, in TensorFlow v2.0 they have decided to do away with placeholders entirely.

如何在不使用占位符的情况下为每个接受可变数量输入并返回可变数量输出的图形制作 tf.function?

How would I make a tf.function for each graph that takes in a variable amount of inputs and returns a variable amount of outputs without using a placeholder?

我想生成一个这样的 tf.function，它适用于任意非循环有向图，以便我可以利用 tensorflow GPU 支持在生成图形后连续运行数千次前馈.

I want to generate a tf.function like this that works for an arbitrary acyclic directed graph so that I can take advantage of tensorflow GPU support to run the graph feed forward a few thousand times in a row after I have generated it.

编辑代码示例:

我的图被定义为字典.每个键代表一个节点，并具有另一个字典的对应值，用于指定传入和传出链接的权重.

My graph is defined as a dictionary. Each key represents a node and has a corresponding value of another dictionary specifying incoming and outgoing links with weights.

{
    "A": {
        "incoming": [("B", 2), ("C", -1)],
        "outgoing": [("D", 3)]
    }
}

为简洁起见，我省略了 B、C 和 D 的条目.以下是我将如何在 tensorflow v1.0 中构建我想要的代码，其中输入只是严格输入图形的关键值列表

I have omitted the entries for B,C, and D for brevity. Here is how I would construct the code I want in tensorflow v1.0 where inputs is just a list of key values that are strictly inputs to the graph

def construct_graph(graph_dict, inputs, outputs):
    queue = inputs[:]
    make_dict = {}
    for key, val in graph_dict.items():
        if key in inputs:
            make_dict[key] = tf.placeholder(tf.float32, name=key)
        else:
            make_dict[key] = None
    # Breadth-First search of graph starting from inputs
    while len(queue) != 0:
        cur = graph_dict[queue[0]]
        for outg in cur["outgoing"]:
            if make_dict[outg[0]]: # If discovered node, do add/multiply operation
                make_dict[outg[0]] = tf.add(make_dict[outg[0]], tf.multiply(outg[1], make_dict[queue[0]]))
            else: # If undiscovered node, input is just coming in multiplied and add outgoing to queue
                make_dict[outg[0]] = tf.multiply(make_dict[queue[0]], outg[1])
                for outgo in graph_dict[outg[0]]["outgoing"]:
                    queue.append(outgo[0])
        queue.pop(0)
    # Returns one data graph for each output
    return [make_dict[x] for x in outputs]

然后我将能够多次运行输出，因为它们只是带有占位符的图形，我将为其提供 feed_dict.

I would then be able to run the outputs many times as they are simply graphs with placeholders that I would provide a feed_dict for.

显然，这不是 TensorFlow v2.0 的预期方式，因为它们似乎强烈反对在这个新版本中使用占位符.

Obviously, this is not the intended way in TensorFlow v2.0 as they seem to strongly discourage the use of placeholders in this new version.

关键是我只需要对图进行一次预处理，因为它返回一个独立于 graph_dict 定义的数据图.

The point is that I only have to do this preprocessing for a graph once, as it returns a datagraph which is independent of the graph_dict definition.

推荐答案

让你的代码在 TF 2.0 中工作

以下是可用于 TF 2.0 的示例代码.它依赖于兼容性 API可作为 tensorflow.compat.v1 访问，并且需要 禁用 v2 行为.我不知道它的行为是否符合您的预期.如果没有，请向我们详细说明您尝试实现的目标.

Make your code work with TF 2.0

Below is a sample code which you can use with TF 2.0. It relies on the compatibility API that is accessible as tensorflow.compat.v1, and requires to disable v2 behaviors. I don't know if it behaves as you expected. If not, then provide us more explanation of what you try to achieve.

import tensorflow.compat.v1 as tf
tf.disable_v2_behavior()

@tf.function
def construct_graph(graph_dict, inputs, outputs):
    queue = inputs[:]
    make_dict = {}
    for key, val in graph_dict.items():
        if key in inputs:
            make_dict[key] = tf.placeholder(tf.float32, name=key)
        else:
            make_dict[key] = None
    # Breadth-First search of graph starting from inputs
    while len(queue) != 0:
        cur = graph_dict[queue[0]]
        for outg in cur["outgoing"]:
            if make_dict[outg[0]]: # If discovered node, do add/multiply operation
                make_dict[outg[0]] = tf.add(make_dict[outg[0]], tf.multiply(outg[1], make_dict[queue[0]]))
            else: # If undiscovered node, input is just coming in multiplied and add outgoing to queue
                make_dict[outg[0]] = tf.multiply(make_dict[queue[0]], outg[1])
                for outgo in graph_dict[outg[0]]["outgoing"]:
                    queue.append(outgo[0])
        queue.pop(0)
    # Returns one data graph for each output
    return [make_dict[x] for x in outputs]

def main():
    graph_def = {
        "B": {
            "incoming": [],
            "outgoing": [("A", 1.0)]
        },
        "C": {
            "incoming": [],
            "outgoing": [("A", 1.0)]
        },
        "A": {
            "incoming": [("B", 2.0), ("C", -1.0)],
            "outgoing": [("D", 3.0)]
        },
        "D": {
            "incoming": [("A", 2.0)],
            "outgoing": []
        }
    }
    outputs = construct_graph(graph_def, ["B", "C"], ["A"])
    print(outputs)

if __name__ == "__main__":
    main()

[<tf.Tensor 'PartitionedCall:0' shape=<unknown> dtype=float32>]

将您的代码迁移到 TF 2.0

虽然上述代码段有效，但它仍与 TF 1.0 相关联.要将其迁移到 TF 2.0，您必须稍微重构一下代码.

 Migrate your code to TF 2.0

While the above snippet is valid, it is still tied to TF 1.0. To migrate it to TF 2.0 you have to refactor a little bit your code.

我建议您返回一个张量列表，而不是返回 TF 1.0 可调用的张量列表keras.layers.Model.

Instead of returning a list of tensors, which were callables with TF 1.0, I advise you to return a list of keras.layers.Model.

下面是一个工作示例:

import tensorflow as tf

def construct_graph(graph_dict, inputs, outputs):
    queue = inputs[:]
    make_dict = {}
    for key, val in graph_dict.items():
        if key in inputs:
            # Use keras.Input instead of placeholders
            make_dict[key] = tf.keras.Input(name=key, shape=(), dtype=tf.dtypes.float32)
        else:
            make_dict[key] = None
    # Breadth-First search of graph starting from inputs
    while len(queue) != 0:
        cur = graph_dict[queue[0]]
        for outg in cur["outgoing"]:
            if make_dict[outg[0]] is not None: # If discovered node, do add/multiply operation
                make_dict[outg[0]] = tf.keras.layers.add([
                    make_dict[outg[0]],
                    tf.keras.layers.multiply(
                        [[outg[1]], make_dict[queue[0]]],
                    )],
                )
            else: # If undiscovered node, input is just coming in multiplied and add outgoing to queue
                make_dict[outg[0]] = tf.keras.layers.multiply(
                    [make_dict[queue[0]], [outg[1]]]
                )
                for outgo in graph_dict[outg[0]]["outgoing"]:
                    queue.append(outgo[0])
        queue.pop(0)
    # Returns one data graph for each output
    model_inputs = [make_dict[key] for key in inputs]
    model_outputs = [make_dict[key] for key in outputs]
    return [tf.keras.Model(inputs=model_inputs, outputs=o) for o in model_outputs]

def main():
    graph_def = {
        "B": {
            "incoming": [],
            "outgoing": [("A", 1.0)]
        },
        "C": {
            "incoming": [],
            "outgoing": [("A", 1.0)]
        },
        "A": {
            "incoming": [("B", 2.0), ("C", -1.0)],
            "outgoing": [("D", 3.0)]
        },
        "D": {
            "incoming": [("A", 2.0)],
            "outgoing": []
        }
    }
    outputs = construct_graph(graph_def, ["B", "C"], ["A"])
    print("Builded models:", outputs)
    for o in outputs:
        o.summary(120)
        print("Output:", o((1.0, 1.0)))

if __name__ == "__main__":
    main()

这里要注意什么?

• 从placeholder改为keras.Input，需要设置输入的形状.
• 使用 keras.layers.[add|multiply] 进行计算.这可能不是必需的，但坚持一个界面.但是，它需要将因子包装在列表中(以处理批处理)
• 构建keras.Model返回
• 使用一组值(不再是字典)调用您的模型

• Change from placeholder to keras.Input, requiring to set the shape of the input.
• Use keras.layers.[add|multiply] for computation. This is probably not required, but stick to one interface. However, it requires to wrap factors inside a list (to handle batching)
• Build keras.Model to return
• Call your model with a tuple of values (not a dictionary anymore)

这是代码的输出.

Builded models: [<tensorflow.python.keras.engine.training.Model object at 0x7fa0b49f0f50>]
Model: "model"
________________________________________________________________________________________________________________________
Layer (type)                           Output Shape               Param #       Connected to                            
========================================================================================================================
B (InputLayer)                         [(None,)]                  0                                                     
________________________________________________________________________________________________________________________
C (InputLayer)                         [(None,)]                  0                                                     
________________________________________________________________________________________________________________________
tf_op_layer_mul (TensorFlowOpLayer)    [(None,)]                  0             B[0][0]                                 
________________________________________________________________________________________________________________________
tf_op_layer_mul_1 (TensorFlowOpLayer)  [(None,)]                  0             C[0][0]                                 
________________________________________________________________________________________________________________________
add (Add)                              (None,)                    0             tf_op_layer_mul[0][0]                   
                                                                                tf_op_layer_mul_1[0][0]                 
========================================================================================================================
Total params: 0
Trainable params: 0
Non-trainable params: 0
________________________________________________________________________________________________________________________
Output: tf.Tensor([2.], shape=(1,), dtype=float32)

这篇关于替换 tensorflow v2 的占位符的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！