I have recently sent a paper on aflatoxin levels in nuts for publishing. The aim of the study was to analyze a total amount of 106 nuts. The nuts were divided into four categories: walnuts (n=33), hazelnuts (n=19), pistacchio (n=29) and almonds (n=25).
The reviewer seems not to have a broad knowledge on study design and statistical analysis, which can be seen in his critique: "In my opinion, every sample should be equal in size, in order to improve the quality of the research."
I totally disagree with his opinion, mostly because following statistical analysis did not require the same sample size. This was a cohort epidemiological study, not a experimental one.
Can somebody back me up with references or explain in details why do i don't need equal sample sizes to do a high quality research?
Or in other words:
Would the research be better if i compared 4 categories with the same sample size (eg. walnuts=23, hazelnuts=23, almonds=23, pistacchio=23), compared to my sample sizes?
Best Answer
I feel that your observational study design argument is strong. There is no way to trim all the group down to 19; that would be silly.
Even or uneven group assignment is not the main issue, but rather your samples are large enough to detect a meaningful difference in a group as small as 19. This is also known as the statistical "power" of the sample. If you attached a power calculation, and support with an argument that even your smallest group can help detecting a meaningful difference, then there shouldn't be a point to criticize the uneven distribution.
The reviewer is half-correct. This is perhaps due to some rules of thumb gone wrong. Given all criteria being the same, even groups have higher power than uneven groups. If 15-15 have 80% power to detect a difference of, say, 5 mmol/dl of toxin. Then a 10-20 splitting will become underpowered (e.g. lower than 80%), becoming less likely to detect the same difference if such difference does exist.
Similar Posts:
- Solved – need statistical power for AB testing if the results are significant
- Solved – How to choose a sample size for two group repeated measures anova design given power, alpha, and effect size
- Solved – Why does one report statistical power only when results are non significant
- Solved – Effect size calculation for One-way ANOVA and Tukey-HSD
- Solved – Power calculation of study using 2 sample z-test