# Solved – Steady State Calculation in Markov Chain in R

I am using the package markovchain in R.
My transition matrix looks like this

``> transition_matrix      Arriving Playing.on.Phone Paying.Attention Writing.Notes Listening Kicked.Out [1,]        0              0.5             0.50           0.0         0       0.00 [2,]        0              0.0             0.99           0.0         0       0.01 [3,]        0              0.8             0.00           0.2         0       0.00 [4,]        0              0.0             0.00           0.0         1       0.00 [5,]        0              0.0             0.00           1.0         0       0.00 [6,]        0              0.0             0.00           0.0         0       1.00 ``

Now I am building a markov chain object

mcstates <- new("markovchain", states = colnames(transition_matrix),
transitionMatrix = transition_matrix ,name = "state")

Setting initial value as

``init <- c(1,0,0,0,0,0) ``

After 10 steps

``> init * (mcstates ^ 10)      Arriving Playing.on.Phone Paying.Attention Writing.Notes Listening Kicked.Out [1,]        0        0.1573841        0.1947628     0.3309517 0.2886897 0.02821181 ``

After 100 steps

``> init * (mcstates ^ 100)      Arriving Playing.on.Phone Paying.Attention Writing.Notes Listening Kicked.Out [1,]        0     4.361078e-06     5.396834e-06     0.4807651 0.4759563 0.04326881 ``

After 1000 steps

``> init * (mcstates ^ 1000)      Arriving Playing.on.Phone Paying.Attention Writing.Notes Listening Kicked.Out [1,]        0     1.163927e-51     1.440359e-51     0.4807692 0.4759615 0.04326923 ``

Showing that there is no change in distribution

However when I try to calculate the steadystate

``> steadyStates(mcstates)      Arriving Playing.on.Phone Paying.Attention Writing.Notes Listening  Kicked.Out [1,]        0     8.211848e-16     1.055809e-15     0.5170262 0.5170262 -0.03405231 [2,]        0     0.000000e+00     0.000000e+00     0.0000000 0.0000000  1.00000000 ``

I have two questions

1. How is the steady state different from the stationary distribution I am hitting when I keep on multiplying with the transition matrix

2. Why is there a negative probability in the steady state solution

Any insight on this will be greatly appreciated

Contents