# Solved – Classification of states in Markov Chain

Question

Consider the following transition matrix:

`` P=   0     0     1     0     0     0   0     0     0     0     0     1  0     0     0     0     1     0 1/4   1/4    0    1/2    0     0  1     0     0     0     0     0  0    1/3    0     0     0    2/3 ``

a) Which states are transient?
b) Which states are recurrent?
c) Identify all closed sets of states.
d) Is this chain ergodic?

Dear friends ,

I have thought the states are as follows,
{1,5} {0,2,4} recurrent since they communicate with each other
{3} transient

and since state 3 does not communicate with other states, it is not an ergodic Mc

Am I correct?

Thank you..

Contents

#### Best Answer

Let the state space of the Markov Chain be \$S={1,2,3,4,5,6}\$. Now draw the state transition diagram.

(a). From the figure, we observe that \${4}\$, and \${6}\$ form non-closed communicating classes. State \$2\$ does not communicate even with itself and such a state is called a non-return state. Hence, the states 2, 4 and 6 are transient.

(b)&(c). The class \${1,3,5}\$ is a closed-communicating class. Hence, states 1, 3 and 5 are recurrent states.

(c). There is only one closed-communicating class, \${1,3,5}\$.

(d). As the chain is not an irreducible Markov Chain, it is not an ergodic chain.

Rate this post