给定转移概率矩阵，如何获得马尔可夫链的平稳分布 [英] How can I obtain stationary distribution of a Markov Chain given a transition probability matrix

问题描述

我正在尝试以矢量形式&矩阵形式.有人可以帮我吗?

此外，我正在尝试查找每个状态的平稳分布.

Pi_0 = ?
Pi_1 = ?
Pi_2 = ?
...
Pi_5 = ?

这是我编写的代码:

P <- matrix(c(0, 0, 0, 0.5, 0, 0.5, 0.1, 0.1, 0, 0.4, 0, 0.4, 0, 0.2, 0.2, 0.3, 0, 0.3, 0, 0, 0.3, 0.5, 0, 0.2, 0, 0, 0, 0.4, 0.6, 0, 0, 0, 0, 0, 0.4, 0.6), nrow = 6, ncol = 6, byrow = TRUE)

mpow <- function(P, n) {
 if (n == 0) diag(nrow(P))
 else if (n == 1) P
 else P %*% mpow(P, n - 1)
 }

mpow(P, 18)

解决方案

在您的问题中，矩阵P是转移概率.当前状态为i而下一个状态为j的概率为:

P[i, j] = Pr(k = j | k = i)

mpow(P, n)计算转换矩阵的n次幂.例如，

> mpow(P, 3)
      [,1]  [,2]  [,3]  [,4]  [,5]  [,6]
[1,] 0.000 0.030 0.105 0.250 0.280 0.335
[2,] 0.001 0.025 0.111 0.254 0.260 0.349
[3,] 0.006 0.032 0.113 0.266 0.224 0.359
[4,] 0.006 0.048 0.144 0.289 0.172 0.341
[5,] 0.000 0.024 0.156 0.400 0.248 0.172
[6,] 0.000 0.000 0.048 0.272 0.432 0.248

> mpow(P, 10)
            [,1]       [,2]      [,3]      [,4]      [,5]      [,6]
[1,] 0.002603379 0.02615891 0.1174816 0.3118660 0.2703684 0.2715217
[2,] 0.002591038 0.02612154 0.1175283 0.3121341 0.2705060 0.2711190
[3,] 0.002565915 0.02600925 0.1174628 0.3124644 0.2710401 0.2704575
[4,] 0.002523007 0.02573033 0.1169686 0.3125272 0.2725643 0.2696866
[5,] 0.002560361 0.02545419 0.1150961 0.3094197 0.2749053 0.2725643
[6,] 0.002708774 0.02649409 0.1171436 0.3096530 0.2690952 0.2749053

> mpow(P,50)
            [,1]       [,2]      [,3]      [,4]      [,5]      [,6]
[1,] 0.002590674 0.02590674 0.1165803 0.3108808 0.2720207 0.2720207
[2,] 0.002590674 0.02590674 0.1165803 0.3108808 0.2720207 0.2720207
[3,] 0.002590674 0.02590674 0.1165803 0.3108808 0.2720207 0.2720207
[4,] 0.002590674 0.02590674 0.1165803 0.3108808 0.2720207 0.2720207
[5,] 0.002590674 0.02590674 0.1165803 0.3108808 0.2720207 0.2720207
[6,] 0.002590674 0.02590674 0.1165803 0.3108808 0.2720207 0.2720207

如您所见，当n很大时，您会到达一个固定的分布，其中所有行都是相等的. 换句话说，无论初始状态如何，最终进入某个状态的可能性都是相同的..

一旦达到这种收敛，该矩阵的任何行都是平稳分布.例如，您可以提取第一行:

> mpow(P,50)[1, ]
[1] 0.002590674 0.025906736 0.116580311 0.310880829 0.272020725 0.272020725

I'm trying to write mpow(P, 18) in vector form & matrix form. Can anyone help me with that?

Also, I'm trying to find the stationary distribution of each state.

Pi_0 = ?
Pi_1 = ?
Pi_2 = ?
...
Pi_5 = ?

Here is the code I've written:

P <- matrix(c(0, 0, 0, 0.5, 0, 0.5, 0.1, 0.1, 0, 0.4, 0, 0.4, 0, 0.2, 0.2, 0.3, 0, 0.3, 0, 0, 0.3, 0.5, 0, 0.2, 0, 0, 0, 0.4, 0.6, 0, 0, 0, 0, 0, 0.4, 0.6), nrow = 6, ncol = 6, byrow = TRUE)

mpow <- function(P, n) {
 if (n == 0) diag(nrow(P))
 else if (n == 1) P
 else P %*% mpow(P, n - 1)
 }

mpow(P, 18)

解决方案

In your question, the matrix P is the transition probability. The probability that the current state is i while the next state is j is:

P[i, j] = Pr(k = j | k = i)

mpow(P, n) computes the n-th power of the transition matrix. For example,

> mpow(P, 3)
      [,1]  [,2]  [,3]  [,4]  [,5]  [,6]
[1,] 0.000 0.030 0.105 0.250 0.280 0.335
[2,] 0.001 0.025 0.111 0.254 0.260 0.349
[3,] 0.006 0.032 0.113 0.266 0.224 0.359
[4,] 0.006 0.048 0.144 0.289 0.172 0.341
[5,] 0.000 0.024 0.156 0.400 0.248 0.172
[6,] 0.000 0.000 0.048 0.272 0.432 0.248

> mpow(P, 10)
            [,1]       [,2]      [,3]      [,4]      [,5]      [,6]
[1,] 0.002603379 0.02615891 0.1174816 0.3118660 0.2703684 0.2715217
[2,] 0.002591038 0.02612154 0.1175283 0.3121341 0.2705060 0.2711190
[3,] 0.002565915 0.02600925 0.1174628 0.3124644 0.2710401 0.2704575
[4,] 0.002523007 0.02573033 0.1169686 0.3125272 0.2725643 0.2696866
[5,] 0.002560361 0.02545419 0.1150961 0.3094197 0.2749053 0.2725643
[6,] 0.002708774 0.02649409 0.1171436 0.3096530 0.2690952 0.2749053

> mpow(P,50)
            [,1]       [,2]      [,3]      [,4]      [,5]      [,6]
[1,] 0.002590674 0.02590674 0.1165803 0.3108808 0.2720207 0.2720207
[2,] 0.002590674 0.02590674 0.1165803 0.3108808 0.2720207 0.2720207
[3,] 0.002590674 0.02590674 0.1165803 0.3108808 0.2720207 0.2720207
[4,] 0.002590674 0.02590674 0.1165803 0.3108808 0.2720207 0.2720207
[5,] 0.002590674 0.02590674 0.1165803 0.3108808 0.2720207 0.2720207
[6,] 0.002590674 0.02590674 0.1165803 0.3108808 0.2720207 0.2720207

As you can see, when n is large, you reach a stationary distribution, where all rows are equal. In other words, regardless the initial state, the probability of ending up with a certain state is the same.

Once such convergence is reached, any row of this matrix is the stationary distribution. For example, you can extract the first row:

> mpow(P,50)[1, ]
[1] 0.002590674 0.025906736 0.116580311 0.310880829 0.272020725 0.272020725

这篇关于给定转移概率矩阵，如何获得马尔可夫链的平稳分布的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！