Solved – Confidence intervals from the Holm-Bonferroni test

by

I'm a newcomer in the problem of multiple comparisons. I wonder how to calculate confidence intervals for Holm-Bonferroni method?

I know that for Bonferroni method we can just change the confidence level from $1-alpha$ to $1-frac{alpha}{m}$.

Does this method also work for Holm-Bonferroni?

$bf{Edit}:$ It seems that H-B method doesn't provide a procedure to correct the conf. interval. But would you comment on can I use one method for p-value correction and the other method for interval correction?

Contents hide
1 Best Answer
2 Similar Posts:

Best Answer

[This answer is completely rewritten from yesterday.]

First nomenclature. The Holm method is also called the Holm step-down method, or the Holm-Ryan method. Those are all the same. Whichever of those names you use, there are two alternative calculations. The original Holm method is based on Bonferroni. An alternative slightly more powerful method is based on Sidak instead, so is called the Holm-Sidak method.

The Holm method can be used for multiple comparisons in a variety of contexts. Its input is a stack of P values. One use is following ANOVA, comparing pairs of means while correcting for multiple corrections. When this is done, as far as I can see, it is very rare to report confidence intervals (corrected for multiple comparisons, so properly called simultaneous confidence intervals) as well as conclusions about statistical significance and multiplicity adjusted P values.

I've found two papers that explain how to compute such confidence intervals, but they differ.

Serlin, R. (1993). Confidence intervals and the scientific method: A case for Holm on the range. Journal of Experimental Education, 61(4), 350–360.

enter image description here

Ludbrook, J. MULTIPLE INFERENCES USING CONFIDENCE INTERVALS. Clinical and Experimental Pharmacology and Physiology (2000) 27, 212–215

enter image description here

For the comparisons with the smallest P values, the two methods are the same (but one uses C as the # of comparisons and the other uses m) . But for the comparisons with larger P values, the two methods differ. For the comparison with the largest P value, Ludbrook would compute the 95% CI normally, with no correction for multiple comparisons. Serlin would use the same adjustment for all comparisons with an adjusted P value greater than 0.05 (assuming you want 95% intervals), so the intervals for the comparisons with large P values would be wider than the ones that Ludbrook method generates.

Both methods use the Bonferroni approach, but could be easily adjusted to the Sidak approach.

Any thoughts on which method is correct/better?

Similar Posts:

Rate this post

Leave a Comment