Solved – Essential transient state in a Markov chain

According to Wikipedia,

• A state $i$ is accessible from a state $j$ (written $jto i$) if a system started in state $j$ has a non-zero probability of transitioning into state $i$ at some point.

• A state $i$ is essential if for all $j$ such that $i to j$ it is also true that $j to i$.

• A state $i$ is said to be transient if, given that we start in state $i$, there is a non-zero probability that we will never return to $i$.

A Markov chain with an essential transient state can be constructed from three states $i,j,k$ for which $ito j$, $jto i$, $jto k$, and $k$ never returns to $i$. The transition $jto k$ guarantees $i$ is transient.

The transition matrix is

$$pmatrix{0 & 1 & 0\ 1-rho & 0 & rho \ 0 & 0 & 1}$$

for some number $rho$ with $0lt rho lt 1$. It is the chance of never returning to $i$ when starting at $i$.