Solved – Multiple comparisons – adjusting p values when some variables are correlated

by

I'm working with 400+ variables (mostly 2D gel data) across 2 conditions for which I've calculated p values with a students t-test. Some of the variables are highly correlated. I'm wanting to adjust the p values to allow for multiple comparisons and I've used p.adjust in R with the "BH" method but I understand that it supposed to be for independent variables. I also tried the "BY" method as I thought being "under dependency" meant that it allowed for some variables to be non-independent (ie correlated), and so I assumed the adjusted p values would be lower than with the "BH" method, but they're not. (When I use the p.adjust BY method on my data the adjusted p values are all 1). Is there a method of adjusting p values that allows for some correlated variables? Thanks for any help.

> pvals$adjusted_BY <- p.adjust(pvals$p_value, method = "BY") > pvals$adjusted_BH <- p.adjust(pvals$p_value, method = "BH") > head(pvals)     ID     p_value adjusted_BY adjusted_BH 1 2366 0.001361787           1   0.5828448 2  746 0.008377390           1   0.9516818 3 1635 0.017267163           1   0.9516818 4 1577 0.025868990           1   0.9516818 5 1588 0.029862316           1   0.9516818 6  891 0.036119770           1   0.9516818
Contents hide
1 Best Answer
2 Similar Posts:

Best Answer

You could build a permutation test. The basic idea is find the null distribution of the minimum p-values (equivalently, the maximum statistic of the test) when you do 400 tests, one for variable. This is a multivariate permutation approach, taking into account the correlation structure of your data

First, calculate the unadjusted p-value with the observed response (you already have this)

The process of permutations is the next:

  1. Permute the response variable (the 2 conditions in your case)
  2. Do the 400 tests with the permuted response
  3. Save the minimun p-value of the tests from these 400 p-values
  4. Repeat the process 1000 or 10000 times

Finally, you calculate the adjusted p-value. For each unadjusted p-value, the adjusted p-value is the number of p-values obtained from the permutations smaller than that p-value, divided by the number of permutations.

Similar Posts:

Rate this post

Leave a Comment