I have a question concerning the comparison of negative binomial regression models in SPSS. I need to compare two models to check whether the second model significantly increases the fit of the model/explained variance in the dependent. The second model contains 1 extra predictor and I want to know whether this predictor significantly improves the model over the first model.
I have already tried comparing fit indices (I find only very small differences) and calculating pseudo R squares (okay, but do not indicate whether the second model is significantly better).
Is it possible to compare two negative binomial regression models aside from the fit indices? Any help?
You can do a likelihood ratio test. Take the difference of -2Log(Likelihood) and compare to a chi-squared distribution with 1 degree of freedom (1 df because the models differ by 1 parameter). SPSS supplies -2log(likelihood). It calls it the Deviance, and it's in an output box labelled "Goodness of Fit".
Note that this is an asymptotic test, so the accuracy of the p-value that you get from this will depend on sample size, and how well your data actually follow a negative binomial distribution.
This same approach can be used for any nested, generalized linear models. (nested means that all the parameters of the smaller model are found in the larger model).
- So, to recap, fit model 1 and locate the Deviance $D_1$
- Fit model2 and locate its Deviance, $D_2$
- Calculate $D_2-D_1=S$
- If S is larger than 3.84, reject the null hypothesis that the extra predictor is useless at the 5% level.
- For a p-value, use S and a set of chi-squared tables.
This gives you an hypothesis test to answer your question. Another approach would be to assess how well the respective models do in predicting outcomes. If you have a reserve set of data, you can see if the larger model does a better job of predicting outcomes. By "Better", I mean that the larger model has a meaningful impact on whatever it is you are using the model for. Strictly speaking, neither the fit indices nor a significance test actually tell you if the model fits. They just let you compare between models.