I have a repeated measures design. Each subject was tested in 2 conditions (**RE**, **re**) with different cognitive load levels (**low**, **mid**, **high**). I originally analysed this dataset using a repeated measures ANOVA (mu_N1 ~ lang*load + Error (subj/lang * load)). This is an unbalanced design, i.e. the number of observations per each cell is slightly different (e.g. subject "KOS" has fewer observations in "ER" and "low" than, say, subject "GRU").

Now I want to bootstrap F-statistics. The question is how do I resample the data? In the simple case of comparing the means of two samples (unpaired t-test), the bootstrap is straightforward: put all the observations into one pile, then take two random samples of the original size from that pile, compute t-statistic, repeat many times. With my data, there are "nested" levels (i.e. each subject was subjected to **ER** and **re**, and within **RE** and **ER** subjected to **low**, **mid**, **high**. Below is a rough slice of my data (disregard the column **CLred**). Would it be reasonable to leave the the order of columns "subj", "lang", "load" unchanged, but only reshuffle (resample with replacement) the numbers in column mu_N1 at each bootstrap resample?

"subj" "lang" "CLred" "load" "mu_N1" "1" "KOS" "RE" 0.21077 "low" 2.039479 "2" "KOS" "RE" 0.50301 "mid" 7.050688 "3" "KOS" "RE" 0.50301 "mid" 6.321543 "4" "KOS" "RE" 0.30664 "low" -7.347104 "5" "KOS" "RE" 0.5031 "mid" 4.850391 "1412" "KOS" "er" 0.22893 "low" -3.867834 "1413" "KOS" "er" 0.77563 "mid" -2.939469 "1414" "KOS" "er" 1.1121 "mid" 17.26057 "1415" "KOS" "er" 1.669 "mid" -4.96131 "1416" "KOS" "er" 1.9141 "high" 11.5676 "1417" "KOS" "er" 1.3287 "mid" 2.687416 "1418" "KOS" "er" 1.2408 "mid" -9.099418 "6835" "GRU" "er" 4.0121 "mid" -9.006017 "6836" "GRU" "er" 4.4279 "mid" -18.26898 "6837" "GRU" "er" 4.782 "mid" -10.48242 "6838" "GRU" "er" 4.9231 "mid" 17.72892 "6839" "GRU" "er" 5.3071 "high" -7.620954 "6840" "GRU" "er" 5.7038 "high" 0.7890218 "6841" "GRU" "er" 6.4355 "high" 3.29691 "6842" "GRU" "er" 7.0723 "high" -7.693555 "7218" "GRU" "RE" 2.0459 "mid" 2.386349 "7219" "GRU" "RE" 2.3365 "mid" -3.752826 "7220" "GRU" "RE" 2.7053 "high" -0.5608671 "7221" "GRU" "RE" 2.7053 "high" -1.798806 "7222" "GRU" "RE" 1.8141 "mid" -2.58064 "7223" "GRU" "RE" 2.2764 "mid" 2.066321 "7224" "GRU" "RE" 2.2764 "mid" 2.385393 "7225" "GRU" "RE" 1.9051 "mid" 0.9645662 "7226" "GRU" "RE" 1.907 "mid" -3.036294 "7227" "GRU" "RE" 1.907 "mid" 1.307736 "26" "KOS" "RE" 0.42917 "mid" -5.46118 "27" "KOS" "RE" 0.22734 "low" -15.93142 "28" "KOS" "RE" 0.35941 "low" -11.60953 "29" "KOS" "RE" 0.62127 "mid" -16.10649 "30" "KOS" "RE" 0.62127 "mid" -4.435908

**Contents**hide

#### Best Answer

In general, probably not. The proper procedure is going to depend on the assumptions you are comfortable making about what is independent. In this case, I suspect that you would be comfortable stating that individuals are independent of eachother, but that within individuals things are (at least potentially) related to eachother.

If that is the case, then you should resample at the level of individuals rather than at the level of responses. That way, the relationship within individuals is preserved, and the randomness that is a result of the specific sample you got from the population will be approximated by the resampling.

### Similar Posts:

- Solved – the difference between a Within Subject and a Repeated Measure design
- Solved – Does random effects allow me to do repeated measures
- Solved – Multiple measurements per trial in R
- Solved – “Random effects must be less than the number of observations” error in lmer package with categorical predictors
- Solved – Repeated measures: aggregate, fixed effects, or random effects