使用 tensorflow 进行序列标记:同步序列输入和输出 [英] Using tensorflow for sequence tagging : Synced sequence input and output

问题描述

我想使用 Tensorflow 进行序列标记，即词性标记.我尝试使用此处概述的相同模型:

注意:对于像 pos 标记这样的东西，其中任何一方的令牌都有明确的信号，你肯定会从双向网络中受益.

如果你想发疯，也会有一些有趣的字符级变体，你只在标记边界发出标记(理由是 pos 标记受益于字符级特征；例如，像词汇名称之外的东西).这么多变种可以尝试！:D

I would like to use Tensorflow for sequence tagging namely Part of Speech tagging. I tried to use the same model outlined here: http://tensorflow.org/tutorials/seq2seq/index.md (which outlines a model to translate English to French).

Since in tagging, the input sequence and output sequence have exactly the same length, I configured the buckets so that input and output sequences have same length and tried to learn a POS tagger using this model on ConLL 2000.

However it seems that the decoder sometimes outputs a taggedsequence shorter than the input sequence (it seems to feel that the EOS tag appears prematurely)

For example: He reckons the current account deficit will narrow to only # 1.8 billion in September .

The above sentence is tokenized to have 18 tokens which gets padded to 20 (due to bucketing).

When asked to decode the above, the decoder spits out the following:

PRP VBD DT JJ JJ NN MD VB TO VB DT NN IN NN . _EOS . _EOS CD CD

So here it ends the sequence (EOS) after 15 tokens not 18.

How can I force the sequence to learn that the decoded sequence should be the same length as the encoded one in my scenario.

解决方案

If your input and output sequences are the same length you probably want something simpler than a seq2seq model (since handling different sequence lengths is one of it's strengths)

Have you tried just training (word -> tag) ?

note: that for something like pos tagging where there is clear signal from tokens on either side you'll definitely get a benefit from a bidirectional net.

If you want to go all crazy there would be some fun character level variants too where you only emit the tag at the token boundary (the rationale being that pos tagging benefits from character level features; e.g. things like out of vocab names). So many variants to try! :D

这篇关于使用 tensorflow 进行序列标记:同步序列输入和输出的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！