Solved – Can least-squares linear regression ever produce no solution at all

Is it ever possible for least-squares linear regression (linear in both features and weights) NOT to produce a solution? That is, after we set each partial derivative to zero, can the resulting system of n linear equations in n weights be "inconsistent"? An example of "inconsistent system of linear equations" is two parallel lines (for 2 weights) or two parallel planes (for 3 weights). If the answer is yes, can you please give a simple numerical example with 2 or 3 data points?

A linear system $Ax=b$ can have no-solution case due to the inconsistencies introduced in its equations. But, linear regression does not aim to satisfy $Ax=b$, it merely tries to minimize $||Ax-b||_2$. This is a simple optimization problem, and you'll have your solution, either unique or infinitely many. Intuitively, one can always draw a line as best fit to a given set of points.

Similar Posts:

Rate this post

Leave a Comment