Solved – Why are random effects shrunk towards 0

generally speaking, most "random effects" occur in situations where there is also a "fixed effect" or some other part of the model. The general linear mixed model looks like this:

$$y_i=x_i^Tbeta+z_i^Tu+epsilon_i$$

Where $beta$ is the "fixed effects" and $u$ is the "random effects". Clearly, the distinction can only be at the conceptual level, or in the method of estimation of $u$ and $beta$. For if I define a new "fixed effect" $tilde{x}_i=(x_i^T,z_i^T)^T$ and $tilde{beta}=(beta^T,u^T)^T$ then I have an ordinary linear regression:

$$y_i=tilde{x}_i^Ttilde{beta}+epsilon_i$$

This is often a real practical problem when it comes to fitting mixed models when the underlying conceptual goals are not clear. I think the fact that the random effects $u$ are shrunk toward zero, and that the fixed effects $beta$ are not provides some help here. This means that we will tend to favour the model with only $beta$ included (i.e. $u=0$) when the estimates of $u$ have low precision in the OLS formulation, and tend to favour the full OLS formulation when the estimates $u$ have high precision.