Solved – Meta analysis across studies with multiple response measures

I'm trying to do a meta analysis across a bunch of bird playback studies. Thanks to help on a previous post here I'm fairly certain I need to use inverse variance weighting (maybe with the 'meta' package in R).

These playback studies use different response measures with some overlap. Some studies use 10 response measures while others use one. I have the means and standard errors for each study's response measures. I want to look at if, across birds, these response measures increase along with another variable that I have for each bird.

I am unsure where to go from here. It doesn't seem appropriate to average the means and standard errors within each study. I'm also not incredibly familiar with the 'meta' package and can't find resources online detailing how to conduct a meta analysis with data like this. What should be my next step?

I'll try to address a few different components of your question

You seem to be under the impression that you want to meta-analyze means of some bird-related variable, but after reading…

I want to look at if, across birds, these response measures increase along with another variable that I have for each bird

it instead appears as though what you're really interested in meta-analyzing are the correlations between that bird-related variable, and some other variable that you have identified (potentially looking at differences across birds). Meta-analyzing correlations is certainly more typical than meta-analyzing means, though the latter is certainly possible–you can meta-analyze virtually any statistic with a corresponding variance or standard error. If it is, in fact, the case that you want to meta-analyze correlations between variables, then recording the means and standard errors of the one variable within each study won't help much. Instead, you will want to collect the correlations between the variables you are interested in, as well as the size of the sample for each correlation, as you'll use sample size to calculate the standard error and/or variance of each correlation.

Once you have this information, it appears as though you'll run into another issue: dependency of effect sizes. Meta-analysis, in many ways, is just a fancy weighted regression, and so many of the same assumptions apply–including assuming that all observations are independent. As you have indicated that…

Some studies use 10 response measures while others use one seems likely that you will have some studies contributing multiple correlations, and therefore will be violating this assumption. Though it sounds scary (especially if you haven't don't meta-analysis before), you can use a method of 3-level meta-analysis (using multilevel structural equation models) to account for this dependency; this is easily accomplished using Cheung's metaSEM package for R (it's syntax is very similar to metafor and other meta-analysis packages). In effect, metaSEM allows you to specify a clustering variable that corresponds to how effect sizes (your correlations) are nested, so you could assign each study an ID number, and just use that as your clustering variable.

Bringing it all together, based on your description, it seems as though you would be interested in at least two models:

Model 1: An "intercept-only" model, whereby you estimate the meta-analytic average correlation between your two bird-related variables of interest.

Model 2: A model whereby you test if/how this correlation is moderated by some other variable(s) (e.g., type of bird, type of outcome measure used, etc.,).

The corresponding metaSEM code for each model is below. In the hypothetical example, you'd be using a data frame called "mydata", with columns called "ID", "corrs", "corrs_v", "bird_type", and "measure_type", corresponding to the ID # you assigned each study, the correlation(s) from the study, the variance of each correlation (metaSEM also allows you to specify standard errors instead), the type of bird, and the type of outcome measure respectively.

#Install and call metaSEM package install.packages("metaSEM") library(metaSEM)  #Fit Model 1-Intercept-only model.1=meta3(y = corrs, v = corrs_v, cluster = ID, data = mydata, = "Intercept-only") summary(model.1)  #Fit Model 2a-Moderation by Bird Type model.2a=meta3(y = corrs, v = corrs_v, cluster = ID, data = mydata, x = cbind(bird_type), = "Moderation by Bird Type") summary(model.2a)  #Fit Model 2b-Moderation by Outcome Measure Type model.2a=meta3(y = corrs, v = corrs_v, cluster = ID, data = mydata, x = cbind(measure_type), = "Moderation by Outcome Measure Type") summary(model.2a) 

What's nice about the 3-level meta-analysis approach (and metaSEM) is that you will get the descriptive statistics of effect size variability that you normally would from a random-effects model (e.g., $tau^2$, and $I^2$), except broken down for each level of clustering (i.e., one of each for within-study variability, and one for between study variability). Then, for your moderation models, you also get the benefit of having $R^2$ for each level of clustering, so you have an idea of how much variance in effect sizes your moderator(s) are explaining at each level.

So, to summarize, collect the effect sizes you are actually interested in meta-analyzing (it doesn't sound like you're interested in means, but rather, correlations of some sort) and their corresponding sample sizes (so you can calculate standard error or variance of the correlation; any introductory meta-analysis text will have these formulas). Then, once you have all this information entered, you can code an ID variable corresponding to which correlation(s) came from which study, and use metaSEM to estimate your meta-analytic model while appropriately accounting for the dependency among the effect sizes that come from the same study.

That should be plenty to get you started; if you have remaining questions, or if I misunderstood something, just comment this response and I can edit it as needed.

Similar Posts:

Rate this post

Leave a Comment