Tensorflow 中的计划采样 [英] scheduled sampling in Tensorflow

问题描述

最新的关于seq2seq模型的Tensorflow api已经包含了定时采样:

The newest Tensorflow api about seq2seq model has included scheduled sampling:

https://www.tensorflow.org/api_docs/python/tf/contrib/seq2seq/ScheduledEmbeddingTrainingHelperhttps://www.tensorflow.org/api_docs/python/tf/contrib/seq2seq/ScheduledOutputTrainingHelper

预定抽样的原始论文可以在这里找到:https://arxiv.org/abs/1506.03099

The original paper of scheduled sampling can be found here: https://arxiv.org/abs/1506.03099

我阅读了论文，但我无法理解ScheduledEmbeddingTrainingHelper 和ScheduledOutputTrainingHelper 之间的区别.文档只说 ScheduledEmbeddingTrainingHelper 是一个添加预定采样的训练助手，而 ScheduledOutputTrainingHelper 是一个直接将预定采样添加到输出的训练助手.

I read the paper but I cannot understand the difference between ScheduledEmbeddingTrainingHelper and ScheduledOutputTrainingHelper. The documentation only says ScheduledEmbeddingTrainingHelper is a training helper that adds scheduled sampling while ScheduledOutputTrainingHelper is a training helper that adds scheduled sampling directly to outputs.

我想知道这两个助手有什么区别?

I wonder what's the difference between these two helpers?

推荐答案

我联系了这背后的工程师，他回复了:

I contacted the engineer behind this, and he responded:

输出采样器在该时间步发出原始 rnn 输出或原始地面实况.嵌入采样器将 rnn 输出视为分布的对数，并发出来自该分类分布的采样 id 的嵌入查找或该时间步的原始地面实况.

The output sampler either emits the raw rnn output or the raw ground truth at that time step. The embedding sampler treats the rnn output as logits of a distribution and either emits the embedding lookup of a sampled id from that categorical distribution or the raw ground truth at that time step.

这篇关于Tensorflow 中的计划采样的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！