Question
Consider the following transition matrix:
P= 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 1 0 1/4 1/4 0 1/2 0 0 1 0 0 0 0 0 0 1/3 0 0 0 2/3 a) Which states are transient?
b) Which states are recurrent?
c) Identify all closed sets of states.
d) Is this chain ergodic?
Dear friends ,
I have thought the states are as follows,
{1,5} {0,2,4} recurrent since they communicate with each other
{3} transient
and since state 3 does not communicate with other states, it is not an ergodic Mc
Am I correct?
Thank you..
Best Answer
Let the state space of the Markov Chain be $S={1,2,3,4,5,6}$. Now draw the state transition diagram.
(a). From the figure, we observe that ${4}$, and ${6}$ form non-closed communicating classes. State $2$ does not communicate even with itself and such a state is called a non-return state. Hence, the states 2, 4 and 6 are transient.
(b)&(c). The class ${1,3,5}$ is a closed-communicating class. Hence, states 1, 3 and 5 are recurrent states.
(c). There is only one closed-communicating class, ${1,3,5}$.
(d). As the chain is not an irreducible Markov Chain, it is not an ergodic chain.