Solved – Adjusted R squared on a holdout set

The formula for adjusted $R^2$ is:
1 – frac{(n-1)}{(n-p-1)}(1-R^2)

where $r^2$ is the coefficient of determination, $n$ is the number of points, and $p$ is the number of parameters the model has.

If I want the adjusted $R^2$ from a holdout set, is $n$ the number of points in the data the model was trained on, or the number of points in the holdout set?

There is no need to make the adjustment on a hold-out sample, or if you did, $p$ would equal 1. For evaluation in a hold-out sample, all coefficients estimated in the training sample must be frozen. But you really should not be using that formula for $R^2$ as it implies you are forcing $R^2$ to be positive. The correct formula allows $R^{2} < 0$ because predictions can be worse than chance. So compute $R^{2} = 1 – frac{SSE}{SST}$ where $SSE$ is sum of squared errors and $SST$ is sum of squares total ($(n-1) times$ the variance of $Y$).

Similar Posts:

Rate this post

Leave a Comment