Solved – Why is linear regression overestimating small values and underestimating big values

Short answer is that your variables are generated from the age. When you try to regress the age on the variables, your model violates one of the assumptions of linear regression called exogeneity. In a nutshell this assumption requires that the errors are not correlated with regressors. However, by construction your errors and variables contain the same psudo random sequences in them, therefore they are correlated.

Here's how it happens. Your data generation process (DGP) is $$x_{it}=t+varepsilon_{it}$$ where $x_i$ – a variable and $varepsilon_i$ is its error. We also know that the errors are independent and Gaussian: $var[varepsilon_i,varepsilon_j]=sigma^2delta_{ij}$. So, the only model that is consistent with DGP is $$X_t=tB+E_t,$$ where $X_t,B,E_t$ are vectors.

However, you're trying to invert the problem, so to speak. Let's see why it leads to an issue. Consider a linear combination of your variables: $$sum_ix_{it}beta_i=tsum_ibeta_i+sum_ibeta_ivarepsilon_{it}$$ Re-arrange it as follows to get to a form that starts to look similar to your model: $$t=sum_ifrac{beta_i}{sum_ibeta_i}x_{it}-sum_ifrac{beta_i}{sum_ibeta_i}varepsilon_{it}$$

If we introduce the scaled coefficients: $beta'_i=frac{beta_i}{sum_ibeta_i}$, and substitute them in the equation above, we get the following: $$t=sum_ibeta'_ix_{it}-sum_ibeta'_ivarepsilon_{it}$$

Since the linear combination of Gaussians is Gaussian itself, we can introduce a new stochastic variable:$xi_t=-sum_ibeta'_ivarepsilon_{it}$, which is $xi_tsimmathcal N(0,sum_isigma^2beta'^2_i)$, hence we have what looks like your regression model at a first glance: $$t=sum_ibeta'_ix_{it}+xi_t$$

Despite the similarity there is a big difference. The linear regression model $y=Xbeta+epsilon$ in order for it to possess nice properties such as unbiasedness needs to satisfy certain conditions, called Gauss-Markov conditions, including one called exogeneity: $E[epsilon|X]=0$ This is exactly what you wanted to see, i.e. that the residuals are centered around predictors.

Unfortunately, the dataset that you created violates this condition by design. Intuitively it's easy to see if you consider that the psudo random errors that you used to generate random regressors, are also the only source of randomness in your data set, hence they also form the "errors" of the model. Therefore, it must be that the errors in your model are correllated with the regressors, which violates exogeneity. You are observing it as errors having bias dependent on the level of variables X.