I have binomial outcome data for three groups that I am analyzing with Cochran's Q test. As discussed at this site, Cochran.

However, I have to repeat this seven different times because I am analyzing performance of three diagnostic tests to see if they can find resistance to seven different drugs.

When there are significant differences between the three groups, I perform a series of three paired post-hoc McNemar tests to further analyze the data.

However, my question is when there are not significant differences, do I still perform the three paired post-hoc tests ?

Part of me says why not, these McNemar's are valid tests. However, another part of me figures that if this is the case, why do the Cochran's Q test at all ?

Could some one point me in the correct direction ?

Thanks !

**Contents**hide

#### Best Answer

Cochran's *Q* test is an analog to the repeated measures ANOVA for binary outcomes. As such it is an omnibus test with a null hypothesis that the population proportions behind each of the $k$ samples (three in your case) are all equal—H$_{0}text{: }p_{1} = p_{2} = dots = p_{k}$—with the alternative hypothesis that *at least one* population proportion is different than *at least one* other population proportion. Those who use such one-way omnibus tests typically proceed thus:

- Conduct the omnibus test;
- If you did not reject the omnibus test
**stop**, you are done; otherwise: - If you did reject the omnibus test conduct multiple
*post hoc*pairwise tests to determine which pairs have different population proportions (McNemar's test is the appropriate test to use, although it is exactly equivalent to Cochran's*Q***for two groups**, so you can you either for the*post hoc*tests); and - If you are feeling extra competent, perform either family-wise error rate or false discovery rate multiple comparisons adjustments.

So how would that work out in your case?

- Conduct Cochran's
*Q*test for your three groups. - If you do not reject (1)
**stop**you conclude you have no evidence that none of the population proportions behind your three samples are different from one another; otherwise: - Conduct three pairwise tests (if you have groups A, B and C, these would be tests of A vs. B, A vs. C, and B vs. C). You can conduct these three tests using either McNemar's test or Cochran's
*Q*for two groups; - If you are feeling extra competent (and I just bet you are), you can perform a Bonferroni, Holm-Sidak, Benjamini-Hochberg or some other adjustment for multiple comparisons. At this point you may or may not determine that any numbers of these pairs are significantly different.
- Profit!

Why do people bother with the omnibus tests in the first place? Because laziness is a good thing, and the total number of possible tests, $m$, grows quite big as $k$ increases ($m$ can be as large as $frac{k(k-1)}{2}$). Thus if we do not reject the omnibus test's null hypothesis we do not need to carry on with the arduous, achingly banal tedium of conducting more tests.