I know that linear regression is based on the assumption that the errors are normally distributed (from both bayesian and classical views). I'm just trying to verify this assumption based on the final model.
Assume I've got 3 normal random variables x1, x2, x3. I can regress (linearly) x1 on x2, x3 and get a linear regression model of the form:
x1 = b0 + b1*x2 + b2*x3 + e.
Reorganizing, e = x1 – b0 – b1*x2 – b2*x3.
Here, if I estimate bi's using least squares method, I can say 'e' is a linear combination of normally distributed variables, so it's normally distributed.
But if I estimate bi's using a Bayesian method and assume bi's also follow normal distribution, then 'e' is no longer a linear combination of normal random variables. Effectively, it's sum of normally (b0) and product-normally (b1*x2, b2*x3) distributed variables, and product-normal distribution is not normal in general. (http://math.stackexchange.com/questions/101062/is-the-product-of-two-gaussian-random-variables-also-a-gaussian)
Is there anything improper in above reasoning? How else can I try to validate the normal assumption for the error term?
Best Answer
When you do the Bayesian analysis are you allowing x1, x2, and x3 to vary/be updated? or are they fixed throughout the analysis?
Most analyses that I have seen, Frequentist or Bayesian, will treat the observed data as fixed and compute the estimates of the coefficients conditional on the observed data. So in that case you can think about the x's as being fixed and not worry about products of normals.
You should also note that while regression models (both Frequentist and Bayesian) assume that the "True" errors are iid Normal, or than the respones is independent normal with mean based on the regression equation, neither assume that the observed residuals are iid Normal (else why would there be standardized, studentized, etc. residuals?). Focusing too much on the distributional properties of the observed residuals may lead away from enlightenment rather than too it.
Similar Posts:
- Solved – Interpret t-values when not assuming normal distribution of the error term
- Solved – Normally distributed errors – Why not use the observed residual histogram
- Solved – distribution of errors in simple linear regression
- Solved – distribution of errors in simple linear regression
- Solved – Why is the correlation between independent variables/regressor and residuals zero for OLS