One of the primary hypotheses in logistic regression theory is the independence of observations. Once the model is fitted to the data (using likelihood maximization procedure for example), I am wondering, is it possible to test independence of the residuals in logistic regression? (as is common practice in time series area)
In my opinion, the independence hypothesis is material in the theory as the likelihood function is based on that fact. Unfortunately, I wasn't able to find something about this and I was wondering why.
Best Answer
Yes, it is possible. You have to be somewhat careful about how to define the residuals. To be concrete, suppose we have a time series. Let the binary outcome be $y_t$ and the vector of regressors be $x_t$. An obvious thing is to take $hat{epsilon}_t = y_t – frac{e^{x_t hat{beta}} } {1+e^{x_t hat{beta}} }$. However, these residuals are heteroskedastic, which makes their asymptotic distribution under the null slightly more complicated. It's easier to work with $tilde{epsilon}_t = frac{hat{epsilon}_t}{ sqrt{ (frac{e^{x_t hat{beta}} } {1+e^{x_t hat{beta}} })(1 – frac{e^{x_t hat{beta}} } {1+e^{x_t hat{beta}} }) } } $. Hsiao, Pesaran, and Pick (2007) discuss this approach for testing independence in panel data.
Alternatively, you can work with what Gorieroux, Monfort, and Trognon (1985) call generalized residuals, $hat{varepsilon}_t = mathrm{E}[y_t^ast – x_t beta | y_t, x_t] = y_t – frac{1}{1+e^{-x_tbeta}}$. They show that $S_1 = frac{(sum_{t=2}^T hat{varepsilon}_t hat{varepsilon}_{t-1})^2}{sum_{t=2}^T hat{varepsilon}_t^2 hat{varepsilon}_{t-1}^2} overset{d}{rightarrow} chi^2_1$. They also discuss how $S_1$ relates to the usual Durbin-Watson test statistic.