Solved – Roots within the unit circle and non-stationarity

Not restricted to time-series analysis, characteristic equations (CE) are used in many applications or problems, such as differential/difference equation solving, signal processing, control systems etc. And, it is directly related with commonly used transforms, e.g. Z, (Disc./Cont.) Fourier, Laplace transform etc. Using back-shift operator is a type of analysis allowing one to transform the time-series equations into another domain and deduce related properties. Most analyses are not easy to do in time-domain, which is why transforms exist.

And, CE obtained by the backshift operator is a way of analyses. This is a matter of which transform you use. You could get the CE of $y_t+alpha y_{t-1}+beta y_{t-2}=epsilon_t$ as $1+alpha B+beta B^2$ or if you use $mathcal{Z}$ transform, that'd be $1+alpha z^{-1}+beta z^{-2}=0$, which requires your roots, i.e. $z$'s, should be inside the unit circle if you want to be stationary, instead of outside as in $B$, since roots obtained by $B$ and $z$ are reciprocals of each other.

If we go back to what you're accustomed to, i.e. using the back-shift operator, $B$, we can try to find the roots' relation to stationarity. Consider the time series $y_t=2y_{t-1}+epsilon_t$, we use the previous output, double it, and add it up with $epsilon_t$ to obtain current output. The process is clearly exploding. The CE of this is $1-2B=0$, which yields $|B|=1/2<1$. But, if it were $y_t=0.5y_{t-1}+epsilon_t$, the root would be $|B|=2>1$. This is for just an intuition why would you need $|B|>1$ for being stationary.

Now consider the general case, $y_t=alpha y_{t-1}+epsilon_t$. We can obtain the following equation by back-substitution: i.e. $y_t=alpha^2y_{t-2}+alphaepsilon_{t-1}+epsilon_t$, and so… $$y_t=sum_{i=0}^infty{alpha^iepsilon_{t-i}}$$ mean and variance of $y_t$ would be $$E[y_t]=sum_{i=0}^{infty}{alpha^iE[epsilon_{t-i}]}=mu_epsilonsum_{i=0}^inftyalpha^i=frac{mu_epsilon}{1-alpha} text{iff} |alpha|<1 rightarrow |B|>1$$ since CE is $1-Balpha=0rightarrow B=1/alpha$. Similarly for the variance, we have $$sigma_Y^2=var(y_t)=sum_{i=0}^{infty}alpha^{2i}sigma_{epsilon}^2=frac{sigma_{epsilon}^2}{1-alpha^2} text{iff} |alpha^2|<1 rightarrow |alpha=1/B|<1$$

The auto-correlation is $r_Y(k)=E[y_t y_{t-k}]:$ $$E[y_t y_{t-1}]=E[(alpha y_{t-1}+epsilon_t)y_{t-1}]=alphasigma_Y^2+mu_epsilonmu_Y$$ $$E[y_t y_{t-2}]=E[(alpha y_{t-1}+epsilon_t)y_{t-2}]=alpha r_Y(1)+mu_epsilonmu_Y=alpha^2sigma_Y^2+mu_epsilonmu_Y(1+alpha)$$ … $$E[y_t y_{t-k}]=alpha^ksigma_Y^2+mu_epsilonmu_Y(1+alpha+…+alpha^{k-1})$$ which is irrespective of $t$, and conditioned on $sigma_Y^2$ and the means don't depend on $t$.

This is for AR(1), but AR(k) can be reduced down to a series of AR(1)'s: $$(1-alpha B)(1-beta B)y_t=epsilon_t rightarrow x_t = (1-beta B)y_t=x_t & (1-alpha B)x_t=epsilon_t$$ and the analysis can be performed recursively. Here, we're actually referring to weak stationarity but if $epsilon_t$ is Gaussian as usual (i.e. which is the noise process is assumed to be distributed with typically), weak stationarity corresponds to stationarity.