Solved – Steady state probabilities for a continuous-time Markov chain

I have a finite state and time-homogeneous continuous-time Markov chain (CTMC) which is not irreducible. Will steady state probabilities exist for this CTMC? How to prove this?

The short answer is "No."

First, it would be helpful to know if your underlying discrete-time Markov chain is aperiodic, unless you are using the phrase "steady state probabilities" loosely to mean "long-run proportion of the time the CTMC is in the various states" or something else other than "stationary distribution." Aperiodicity in combination with irreducibility is sufficient to guarantee a unique stationary distribution in the case of finite state-spaces, which you are assuming.

Second, the lack of irreducibility means that you will have either some transient states and/or more than 1 closed communicating class. Each class will have its own steady state probabilities (given aperiodicity) and there may be stationary distributions that span multiple communicating classes. Which class you wind up in depends upon what happened during the transient part of the chain's operation, and perhaps upon the initial state.

Consider a two-state discrete-time MC with transition matrix $P(1,1) = P(2,2)=1, P(1,2)=P(2,1) = 0$. Clearly it is aperiodic but not irreducible. Any steady-state distribution $pi$ satisfies $pi = pi P$, is nonnegative, and sums to one. Obviously this is satisfied for any $pi$ that is nonnegative and sums to one. So every distribution on ${1,2}$ is a steady-state distribution for this example. (Hence, clearly, a unique one does not exist.)

Similar Posts:

Rate this post

Leave a Comment