多个观测变量的隐马尔可夫模型 [英] Hidden Markov Model for multiple observed variables

问题描述

我正在尝试使用一个隐马尔可夫模型(HMM)解决一个问题，在该问题中，每个时间点t有M个不同的观察变量(Yti)和一个隐变量(Xt).为了清楚起见，让我们假设所有观察到的变量(Yti)是分类的，其中每个Yti传达不同的信息，因此可能具有不同的基数. 下图给出了一个说明性示例，其中M = 3.

I am trying to use a hidden Markov model (HMM) for a problem where I have M different observed variables (Yti) and a single hidden variable (Xt) at each time point, t. For clarity, let us assume all observed variables (Yti) are categorical, where each Yti conveys different information and as such may have different cardinalities. An illustrative example is given in the figure below, where M=3.

我的目标是使用Baum-Welch算法从观察到的变量序列(Yti)中训练HMM的跃迁，发射和先验概率.假设Xt最初将具有2个隐藏状态.

My goal is to train the transition,emission and prior probabilities of an HMM, using the Baum-Welch algorithm, from my observed variable sequences (Yti). Let's say, Xt will initially have 2 hidden states.

我已经阅读了一些教程(包括著名的Rabiner论文)，并阅读了一些HMM软件包的代码，即" MatLab中的HMM工具箱"和" hmmpytk软件包"中的代码. Python ".总的来说，我进行了广泛的网络搜索，所有资源-我可以找到的-仅涵盖了以下情况:每个时间点只有一个观察到的变量(M = 1).这越来越使我认为HMM不适用于具有多个观察变量的情况.

I have read a few tutorials (including the famous Rabiner paper) and went through the codes of a few HMM software packages, namely 'HMM Toolbox in MatLab' and 'hmmpytk package in Python'. Overall, I did an extensive web search and all the resources -that I could find- only cover the case, where there is only a single observed variable (M=1) at each time point. This increasingly makes me think HMM's are not suitable for situations with multiple observed variables.

• 是否可以将图中描述的问题建模为HMM?
• 如果是，那么如何能够基于多变量观测(发射)概率来修改Baum-Welch算法以适应训练HMM参数的要求?
• 如果不是，您是否知道一种更适合图中所示情况的方法?

谢谢.

修改: 在此纸张中，图中所示的情况是称为动态朴素贝叶斯，就训练和估计算法而言，它需要对单变量HMM的Baum-Welch算法和Viterbi算法稍作扩展.

In this paper, the situation depicted in the figure is described as a Dynamic Naive Bayes, which -in terms of the training and estimation algorithms- requires a slight extension to Baum-Welch and Viterbi algorithms for a single-variable HMM.

推荐答案

做到这一点并使模型保持可生成性的最简单方法是，在给定x_is的情况下使y_is有条件地独立.这导致估计量微不足道，参数相对较少，但是在某些情况下是一个限制性的假设(基本上是朴素贝叶斯分类器的HMM形式).

The simplest way to do this, and have the model remain generative, is to make the y_is conditionally independent given the x_is. This leads to trivial estimators, and relatively few parameters, but is a fairly restrictive assumption in some cases (it's basically the HMM form of the Naive Bayes classifier).

这是什么意思.对于每个时间步长i，您都有一个多元观察值y_i = {y_i1...y_in}.在给定x_i的情况下，您将y_ij视为有条件独立，因此:

what this means. For each timestep i, you have a multivariate observation y_i = {y_i1...y_in}. You treat the y_ij as being conditionally independent given x_i, so that:

p(y_i|x_i) = \prod_j p(y_ij | x_i)

然后，您将有效地学习隐藏变量x的每个可能值的朴素贝叶斯分类器. (在此情况下，条件独立非常重要:ys的无条件分布中存在依赖项).这可以通过HMM的标准EM来学习.

you're then effectively learning a naive Bayes classifier for each possible value of the hidden variable x. (Conditionally independent is important here: there are dependencies in the unconditional distribution of the ys). This can be learned with standard EM for an HMM.

正如一位评论者所说，您也可以将y_ijs的连接视为单个观察值，但是，如果j个变量中任何一个的维数都不是微不足道的，这将导致 lot 参数，那么您将需要更多的训练数据.

You could also, as one commenter said, treat the concatenation of the y_ijs as a single observation, but if the dimensionality of any of the j variables is beyond trivial this will lead to a lot of parameters, and you'll need way more training data.

您是否特别需要生成模型?如果您只想在x_is中进行推断，则最好使用有条件的服务随机字段，通过其特征函数，可以在没有相同的限制性独立假设的情况下，获得更复杂的观测结果.

Do you specifically need the model to be generative? If you're only looking for inference in the x_is, you'd probably be much better served with a conditional random field, which through its feature functions can have far more complex observations without the same restrictive assumptions of independence.

这篇关于多个观测变量的隐马尔可夫模型的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！