In one clinical trial comparing one experimental regimen A with control regimen B in order to test whether or not the regimen A is superior than regimen B. The p value for the hazard ratio of progression free survival is reported as 0.009 in the paper. Even though it's statistical significant, but since the median survival time for patients with control regimen is in fact longer than that with experimental regimen, 12 months versus 8 months. In another word, the control regimen has better treatment effect.
What will be the p value for testing experimental regimen versus control? Will it be different from 0.009 if it's one-sided test?
Hi Ben, Thank you for your reply, which is pretty detail and very helpful. However, when I read the WIKIPEDIA at http://en.wikipedia.org/wiki/One-_and_two-tailed_tests#Applications for One- and two-tailed tests, it is said under section APPLICATIONS, at second paragraph, 'For a given test statistic there is a single two-tailed test, and two one-tailed tests, one each for either direction. Given data of a given significance level in a two-tailed test for a test statistic, in the corresponding one-tailed tests for the same test statistic it will be considered either twice as significant (half the p-value), if the data is in the direction specified by the test, or not significant at all (p-value above 0.5), if the data is in the direction opposite that specified by the test.'. I cannot relate 'the p-value above 0.5' to your calculation of one-sided p-value. I am not exactly sure why it's p-value above 0.5. I am able to understand your calculation though.
Best Answer
Yes. The one-tailed p-value, testing whether the experiment worked better than the control, would be something like 0.995.
The two-tailed p-value asks "how likely would it be to observe at least this extreme of an outcome if there were no real effect?" The one-tailed p-value asks "how likely would it be to observe at least this good of an outcome if there were no real effect?"
In this case, the outcome was extreme, but bad. If there were no true effect, you would be unlikely to observe an outcome this extreme, so the two-tailed p-value would be low. But you would be really likely to observe a better outcome, so the one-tailed p-value would be very high.
If the distribution of outcomes under the null hypothesis is symmetrical, then there's a relationship between one-tailed and two-tailed p-values as follows.
If the relationship is in the expected direction, then the one-tailed p-value is half of the two-tailed p-value. That's because, under the null distribution, half of the extreme outcomes you'd observe would be bad extreme outcomes–so you'd be half as likely to observe an outcome that was good and extreme.
If the relationship is in the opposite direction, then the one-tailed p-value is one minus half of the two-tailed p-value, by symmetry.
The diagram below might help. It shows the distribution of expected outcomes of the null hypothesis:
The green area is all the extreme and good outcomes. If you observed the outcome of 2, then the green area is the probability of an outcome better than that, so that would be the one-tailed p-value.
The red area is all the extreme bad outcomes. If you observed the outcome of 2, then outcomes of -2 or worse would be just as extreme, so the two-tailed p-value would be the area of the red and green sections combined. Since the distribution is symmetrical, that's twice the one-tailed p value.
On the other hand, if you observed the outcome of -2, then the probability of getting a better outcome would be the non-red areas–that's green plus tan. Since the area of the red is half the two-tailed p-value, the area of the rest is one minus that.
So if the two-tailed p-value is ~= 0.01, then the one-tailed p-value for the opposite effect is 1-0.005 = 0.995.