Solved – Multiple comparison correction (Bonferroni vs. Holm Bonferroni)

I have four questions:

If i have to run a test on 4 colors; red, blue, green, yellow. I
ran the same test 4 times (one time on each color). (H1~H4) If i
already have a predefined or existing reason to do each of these
tests (not exploratory), do i have to correct for multiple
comparison (no. of H/4)? What If it was exploratory?
If i have only 2 colored flowers; Black flowers and White flowers. I
ran the same test 2 times (one time on each color). (H1~H2) But then
i found something interesting to further investigate. E.g. the
flower stem and leaves. If i were to correct for multiple comparisons for the stem and leaves. Do i divide the p-value by 2? or 4?
Which is better to use for the above Bonferroni or Holm-Bonferroni?
If i am correcting using the Holm-Bonferroni. Is it possible to report adjusted p values? If yes, how do you do that? e.g. in Bonferroni you multiply by the no. of tests run.

Contents hide

It depends on context. Let us say you really want to write an article about how males vs females react to colors. If you are going to publish if you find a significant difference for at least one color then you absolutely need to correct for multiple testing. Usually even one significant result is considered interesting, it is almost always better to correct.
For your original hypotheses, you count only the number original hypotheses. For any data-driven hypotheses you are out of luck. You will not be able to correct for multiple testing after having explored the data. You would have to put the number of all interesting things you might potentially have noticed in, and that is not quantifiable. In general, you should test your new hypotheses in a new experiment with new data.
Holm-Bonferroni is always superior. It has more power and still strictly controls the familywise error rate.
For Bonferroni you multiply (not divide) by the number of p-values to get adjusted p-values. Holm-Bonferroni is slightly more complicated to correct, but most statistical packages have functions for that and you can find the description of the procedure in the article by Wright here: Adjusted P-Values for Simultaneous Inference.