Let's say that we have 2 data samples $A$ and $B$, containing $N = 10000$ samples each, which are generated by two random variables $X_{A}$ and $X_B$. If I apply the student's T-test and apply a value below 0.01, can I formulate the following statement?
Data samples $A$ and $B$ are significantly different by a threshold of $0.01$.
Otherwise, how can I correctly interpret this with a statement?
Best Answer
It's just a very conventional answer and if anything is unclear please feel free to follow up. Usually, when reporting results of t-test, the statement should include the t-statistics and p-value. For instance (data are made up):
"The mean of $X_A$ is significantly higher/lower than that of $X_B$ (t = 3.36, p = 0.0004)"
Whether the difference is statistically significant or not depends on the p-value and false positive rate (aka type I error rate). For test that have a type I error rate of 5%, we declare the difference is significant if the p-value is lower than 0.05.
Meanwhile, t-statistics is a summary of the ratio of mean difference to the variances (You can roughly conceptualize this as the signal to noise ratio.) For that statistics, the bigger it is, the lower will be the p-value; and a significant difference is more likely.
Similar Posts:
- Solved – “The statistician should set the probability of Type I error as low as possible”
- Solved – “The statistician should set the probability of Type I error as low as possible”
- Solved – How to test the equality of two Pearson correlation coefficients computed from the same sample
- Solved – When do Post-hoc tests require an adjustment for multiple comparisons
- Solved – Does false discovery rate depend on the p-value or only on the alpha level