Consider the following linear regressions on two sets of data x and y (of same length)
- y=ax
- x=by
As you know, the usual optimisation by OLS is usually not giving (at all!) b=1/a
This is because the optimisation algorithm is not symmetric.
My question: is there coded somewhere in R the algorithm that fits symmetrically? (using as error the shortest distance to the line – for those who follow me here..)
Edit: http://en.wikipedia.org/wiki/Total_least_squares
Best Answer
The reason regression is not symmetrical is because it specifically is minimizing the error between the regression line and the response variable, i.e. in the direction of the y-axis for y=ax and in the direction of the x axis for x=by. for a "symmetric" fit, you want to minimize errors orthogonal to the fit line. PCA does this. The fit line you are looking for well be the first principal component. Check out the built in R functions prcomp and princomp.
PCA will give you a vector that passes through your data with the symmetry property you are looking for and therefore you are getting the slope component of a simple linear regression, but it won't give you intercept term (because PCA is not typically used for this purpose). But you should be able to determine the intercept easily with the knowledge that the "regression" line will pass through $(bar{x},bar{y})$.
Similar Posts:
- Solved – symmetry of linear regression
- Solved – symmetry of linear regression
- Solved – How does centering the data get rid of the intercept in regression and PCA
- Solved – Centering in linear regression
- Solved – PCA: How to the first principal component both maximize variance AND define the line that most closely fits the data