如何在TensorFlow中实现递归神经网络? [英] How can I implement a recursive neural network in TensorFlow?

问题描述

是否有某种实现递归神经网络的方法，例如>中的方法. 2011] 使用TensorFlow? 请注意，这与递归神经网络不同，后者由TensorFlow很好地支持. 区别在于网络不是复制到线性操作序列中，而是复制到树状结构中.

Is there some way of implementing a recursive neural network like the one in [Socher et al. 2011] using TensorFlow? Note that this is different from recurrent neural networks, which are nicely supported by TensorFlow. The difference is that the network is not replicated into a linear sequence of operations, but into a tree structure.

我想我可以使用While op为我的数据集的每个条目构造诸如树数据结构的广度优先遍历之类的东西.

I imagine that I could use the While op to construct something like a breadth-first traversal of the tree data structure for each entry of my dataset.

也许可以将树遍历作为TensorFlow中的新C ++ op来实现，类似于While(但更通用)?

Maybe it would be possible to implement tree traversal as a new C++ op in TensorFlow, similar to While (but more general)?

推荐答案

当前，由于图结构取决于输入，因此很难在TensorFlow中高效，清晰地实现这些模型.这也使得很难进行迷你批处理.可以使用您提到的while循环之类的东西，但是干净地做起来并不容易.

Currently, these models are very hard to implement efficiently and cleanly in TensorFlow because the graph structure depends on the input. That also makes it very hard to do minibatching. It is possible using things like the while loop you mentioned, but doing it cleanly isn't easy.

您可以为每个示例构建一个新图形，但这将非常烦人.如果对于给定的输入大小，您可以枚举相当数量的可能的图形，则可以在它们之间进行选择并立即构建所有图形，但是对于较大的输入将无法实现.

You can build a new graph for each example, but this will be very annoying. If, for a given input size, you can enumerate a reasonably small number of possible graphs you can select between them and build them all at once, but this won't be possible for larger inputs.

您还可以通过复杂的tf.gather逻辑和掩码在图形中路由示例，但这也可能会非常麻烦.

You can also route examples through your graph with complicated tf.gather logic and masks, but this can also be a huge pain.

最终，为每个示例动态构建图形可能是最简单的，并且将来有可能会有其他替代方法支持更好的即时样式执行.但是从v0.8开始，我希望这会有些烦人，并且会像Yaroslav在他的评论中提到的那样引入一些开销.

Ultimately, building the graph on the fly for each example is probably the easiest and there is a chance that there will be alternatives in the future that support better immediate style execution. But as of v0.8 I would expect this to be a bit annoying and introduce some overhead as Yaroslav mentions in his comment.

自从我回答以来，这是一个使用带while循环的静态图形的示例: https://github.com/bogatyy/cs224d/tree/master/assignment3 我不确定将其与此类模型的自定义C ++代码相比性能如何，尽管原则上可以批量处理.

Since I answered, here is an example using a static graph with while loops: https://github.com/bogatyy/cs224d/tree/master/assignment3 I am not sure how performant it is compared to custom C++ code for models like this, although in principle it could be batched.

这篇关于如何在TensorFlow中实现递归神经网络?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！