I'm new to using goodness of fit tests and am encountering what I guess is a beginners misunderstanding of how the tables for critical values and different significance levels have to be read. I have searched for similar questions but did not find an answer that would help me.
In the tables, such as this one, the critical values at a given sample size increase with (1 – alpha). It was my understanding that the higher 1 – alpha (or the lower alpha), the more likely my hypothesis is to be true. If this is so, I don't understand why the test statistics have to be smaller than the critical values in order to not reject the hypothesis. Something seems to be upside down here. Could you please point out my mistake?
Thanks in advance!
Best Answer
Hypothesis testing can be tricky to understand. You wrote "the more likely my hypothesis is to be true", but this isn't how it works.
You assume a model (called the "null model"), and then see if there is evidence to reject it. That is, you ask "how surprised would I be to see the observed data, if the hypothesis really were true".
The "how surprised" you are is measured by the "level" $alpha$, where a smaller $alpha$ means more surprised. The critical value (at which you reject the hypothesised model, due to being so shocked at what you observe) is determined by $alpha$.
To reject at small level $alpha$, you need to observe a more shocking test statistic, which is what the higher critical value says.
Similar Posts:
- Solved – What if the P-Value is less than 0.05, but the test statistic is also less than the critical value
- Solved – Wilcoxon-Mann-Whitney critical values in R
- Solved – Why does the critical value varies inversely with the level of significance in hypothesis testing
- Solved – Why does the critical value varies inversely with the level of significance in hypothesis testing
- Solved – What’s the null hypothesis in a one-sided Kolmogorov-Smirnov test