在TensorFlow CTC中使用自定义波束计分器(语言模型) [英] Using custom beam scorer in TensorFlow CTC (language model)

问题描述

是否可以从Python方面在TensorFlow CTC实现中自定义波束计分器?我在CTCBeamSearchDecoder C ++类构造函数的注释中看到了这种可能性，但想知道如何为Python用户提供此功能?

Is it possible to customize beam scorer in TensorFlow CTC implementation from Python side? I see this possibility in comment for CTCBeamSearchDecoder C++ class constructor but wonder how to provide this functionality for Python users?

我们遇到的特定问题是将语言模型插入基于CTC的语音解码器中.语言模型可能是预训练的TensorFlow子图，能够输出光束分数调整的概率.但是我们需要一种将其注入到波束计分器中的方法.

Specific issue that we have is the plugging of language model into CTC based speech decoder. Language model can possibly be a pre-trained TensorFlow sub-graph, capable of outputting probabilities for beam score adjustment. But we need a way to inject this into beam scorer.

推荐答案

当前没有用于Python的API可以与自定义评分器一起使用语言模型.欢迎做出贡献，但是要在Python API中做到这一点存在一些困难，因为这要求在解码器op内的独立会话中运行TF LM子图，而这些子图不能很好地融合在一起.

There's currently no API for Python to use language model with a custom scorer. Contributions are welcome, but there's some difficulty in making this possible in the Python API as it would require running the TF LM sub-graph in an independent session inside the decoder op, and those wouldn't blend nicely together.

最简单的方法是在C ++中，它需要将BaseBeamScorer类与BeamState一起扩展(类似于在测试中可以看到的)，并在tensorflow图的输出之上进一步运行CTCBeamSearchDecoder :: Decode通常进入ctc_beam_search_decoder操作.

The easiest way of doing this is in C++ and requires extending the BaseBeamScorer class along with a BeamState (similar to what can be seen in tests) and further run CTCBeamSearchDecoder::Decode on top of the outputs from the tensorflow graph that would normally go in the ctc_beam_search_decoder op.

这样做，您的BeamScorer实现可以利用您手头的任何语言模型，并且在将波束从一种状态扩展到另一种状态时，只需返回适当的分数即可.

By doing this, your BeamScorer implementation could make use of any language model you have at hand and simply needs to return the appropriate scores when expanding the beam from one state to another.

这篇关于在TensorFlow CTC中使用自定义波束计分器(语言模型)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！