I have a question about comparing the means of two variables that have a different meaning but a (possibly) similar ways of measuring them. To give an example:
Let's say I am conducting a study in the social sciences in which I ask my participants about their experiences of "Anger" and "Anxiety" while coping with a difficult situation. Each participant answers to both questions on a 7-point Likert-Scale.
Now my question(s):
- Can I compare the means of the two variables? (e.g., "Do my participants rather experience Anger or Anxiety in the given situation?")
- What statistical procedure is appropriate?
- Is the result actually meaningful?
The answers to the first two question might be "Yes, do a paired-sample t-test or a repeated measures ANOVA." However, the two variables are not equivalent content-wise. And in my opinion the comparison of means between two such variables is not sensible unless it can be assumed that the variable share the same scale.
So while I think that the question is possible to answer from a "strictly statistical point", I wonder if interpreting the results would actually be meaningful. From my point of view "measuring" the two variables by the same Likert-Scale is not sufficient to make the comparison meaningful but I wonder if I am discarding it too readily.
Best Answer
Rank-based approaches (e.g. Wilcoxon paired test) would address "Which feeling occurs more strongly?" even in the presence of difference in scales. But the question could be confounded by the fact that one variable is going to be a little stronger – more willingly shared in a survey. Yet this can no longer be addressed at the analysis stage, once the survey, where participants were responding to a known scale, has been completed.