如何在 seq2seq 中使用 output_projection(权重、偏差)? [英] how to use output_projection (weights, biases) in seq2seq?

问题描述

我正在尝试确定如何在 中使用 output_projection 参数embedding_rnn_decoder.seq2seq教程代码展示了output_projection的生成这里 -- 我可以假设变量的名称不会改变吗?

I am trying to determine how to use the output_projection parameter in the embedding_rnn_decoder. The seq2seq tutorial code shows the generation of output_projection here -- am I able to assume that the names of the variables do not change?

w_t = tf.get_variable("proj_w", [self.target_vocab_size, size], dtype=dtype)
w = tf.transpose(w_t)
b = tf.get_variable("proj_b", [self.target_vocab_size], dtype=dtype)
output_projection = (w, b)

我不清楚语法，他们目前是否创建"了权重和偏差的变量，或者这些实际上是他们得到的变量?如果他们正在创建它们，那么我的理解是这些变量现在将存储 RNN 最后一层的权重/偏差，并可用于生成 logits

The syntax is unclear to me, are they currently "creating" the variables for the weights and biases or are these actually variables that they are getting? If they are creating them, then my understanding would be that these variables would now store the weights/biases of the last layer of the RNN and could be used to generate the logits

logits = tf.matmul(outputs, w) + b

这种理解是否正确?

推荐答案

我不清楚语法，他们目前是否创建"了权重和偏差的变量，或者这些实际上是他们得到的变量?

The syntax is unclear to me, are they currently "creating" the variables for the weights and biases or are these actually variables that they are getting?

首次初始化图形时，会创建这些变量，并在连续批处理中学习和重用这些参数.

When the graph is first initialized, those variables are created and for the consecutive batch processing those parameters are learned and reused.

使用output_projecttion

它不完全是 logits(它是预计的输出)；输出投影变量用于投影前一个单元格的输出，并将投影输出与当前单元格输入组合.

It is not exactly logits (it's projected output); the output projection variables are used to project the output of the previous cell and the projected output is combined with the current cell inputs.

projected_output = tf.matmul(outputs, w) + b

这篇关于如何在 seq2seq 中使用 output_projection(权重、偏差)?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！