Solved – How to calculate the confidence interval of an ICC

Here's the output from my the summary of lmer function which I used to calculate the ICC.

   Linear mixed model fit by REML ['lmerMod']    Formula: CareChange ~ 1 + (1 | PROVIDER)       Data: MEA_data_1     REML criterion at convergence: 35.2     Scaled residuals:         Min      1Q  Median      3Q     Max     -0.3093 -0.2829 -0.2711 -0.2599  3.6206      Random effects:     Groups            Name        Variance  Std.Dev.     PROVIDER          (Intercept) 0.0003096 0.01759      Residual                      0.0660667 0.25703     Number of obs: 239, groups:  PROVIDER_CALENDAR, 14     Fixed effects:         Estimate Std. Error t value         (Intercept)  0.07119    0.01742   4.086     ICC = 0.0003096 /(0.0003096 +0.0660667 ) = 0.004664315 

I did not find any literature that shows how to calculate the CI for this ICC value. Any help on this issue is much appreciated.

A straightforward way to calculate a confidence interval would be to create a bootstrapped distribution, then obtain the relevant quantiles from that distribution. This can be done as a parametric or nonparametric bootstrap, depending on what assumptions you're comfortable with. Generally, if you accept a Normal distribution for the random effects, a parametric bootstrap is also appropriate.

Since you're using R, the lme4 package offers a very nice parametric bootstrap function, bootMer() to calculate a bootstrapped distribution of any statistic of interest derived from a random effects model. The parametric method is fully implemented, which simulates the model of interest using new values of the random effects with each iteration. New values are drawn from a Normal distribution using parameters derived from the mixed model. See the documentation in bootMer() for some detail on control over what effects get bootstrapped. Set your own number of iterations, but 1000+ is a good rule of thumb.

To calculate this in R, you need to fit the random effects model and pass a function to bootMer() that calculates the statistic of interest. An example implementation is below:

Example Code using R

#Make some mocked data library(lme4) library(reshape2) set.seed(2024)  #For the Bell Riots id <- factor(seq(1, 15)) <- rnorm(15, 10, 5) mydat <- NULL for (a in 1:length(id)){   score <- rnorm(2,[a], 3) <- data.frame(id=id[a], score1=score[1], score2=score[2])   mydat <- rbind(mydat, } mydat <- melt(mydat, id.vars='id','score')  #Create function to calculate ICC from fitted model calc.icc <- function(y) {   sumy <- summary(y)   (sumy$varcor$id[1]) / (sumy$varcor$id[1] + sumy$sigma^2) }  #Fit the random effects model and calculate the ICC mymod <- lmer(score ~ 1 + (1|id), data=mydat) summary(mymod) calc.icc(mymod)  #Calculate the bootstrap distribution boot.icc <- bootMer(mymod, calc.icc, nsim=1000)  #Draw from the bootstrap distribution the usual 95% upper and lower confidence limits quantile(boot.icc$t, c(0.025, 0.975)) 

Similar Posts:

Rate this post

Leave a Comment