Tensorflow中的预定采样 [英] scheduled sampling in Tensorflow

问题描述

有关seq2seq模型的最新Tensorflow api包括计划的采样:

The newest Tensorflow api about seq2seq model has included scheduled sampling:

https://www.tensorflow.org/api_docs/python/tf/contrib/seq2seq/ScheduledEmbeddingTrainingHelper https://www.tensorflow.org/api_docs/python/tf/contrib /seq2seq/ScheduledOutputTrainingHelper

预定抽样的原始文件可以在这里找到: https://arxiv.org/abs/1506.03099

The original paper of scheduled sampling can be found here: https://arxiv.org/abs/1506.03099

我读了这篇论文，但是我不明白ScheduledEmbeddingTrainingHelper和ScheduledOutputTrainingHelper之间的区别.该文档只说ScheduledEmbeddingTrainingHelper是一个训练助手，它添加了计划的采样，而ScheduledOutputTrainingHelper是一个训练助手，它直接将了计划的采样添加到输出.

I read the paper but I cannot understand the difference between ScheduledEmbeddingTrainingHelper and ScheduledOutputTrainingHelper. The documentation only says ScheduledEmbeddingTrainingHelper is a training helper that adds scheduled sampling while ScheduledOutputTrainingHelper is a training helper that adds scheduled sampling directly to outputs.

我想知道这两个助手之间有什么区别吗?

I wonder what's the difference between these two helpers?

推荐答案

我联系了此背后的工程师，他回答:

I contacted the engineer behind this, and he responded:

输出采样器在该时间步发出原始rnn输出或原始地面真相.嵌入采样器将rnn输出视为分布的对数，并在该时间步从该分类分布或原始地面真相发出采样ID的嵌入查找.

The output sampler either emits the raw rnn output or the raw ground truth at that time step. The embedding sampler treats the rnn output as logits of a distribution and either emits the embedding lookup of a sampled id from that categorical distribution or the raw ground truth at that time step.

这篇关于Tensorflow中的预定采样的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！