Solved – Ridge regression coefficients that are larger than OLS coefficients or that change sign depending on $lambda$

When running ridge regression, how do you interpret coefficients that end up larger than their corresponding coefficients under least squares (for certain values of $lambda$)? Isn't ridge regression supposed to monotonically shrink coefficients?

On a related note, how does one interpret a coefficient whose sign changes during ridge regression (i.e., the ridge trace crosses from negative to positive on a ridge trace plot)?

As $lambda$ increases from zero the contribution of various coefficients changes to suit the optimization, allowing both value increases and sign changes. Have a look at Ryan Tibshirani's ridge regression charts (PDF) illustrating both of your questions (charts 17, 19).

Similar Posts:

Rate this post

Leave a Comment