# Solved – Testing approach Chi Square vs Mann Whitney

I am looking to understand the best approach to determine if students in one school performed better on a quiz than another. In this scenario the quiz has five questions and only whole points are awarded, giving 0, 1, 2, 3, 4, & 5 as the only possible results.

The results:

``| score | school_a | school_b | |-------|----------|----------| | 0     | 150      | 175      | | 1     | 50       | 40       | | 2     | 30       | 30       | | 3     | 20       | 15       | | 4     | 5        | 10       | | 5     | 80       | 90       | ``

Given the non-normality of the distribution it seems like the Mann-Whitney test would be appropriate, but because scores are integers I am concerned that the ties may cause issues. Should I instead treat the scores as categorical variables and perform a chi-square test instead?

Is there another approach I should be considering instead?

Contents

Mann-Whitney (also two-sample Wilcoxon test): Data are

``a = rep(0:5, c(150, 50, 30, 20, 5, 80)) b = rep(0:5, c(175, 40, 30, 15, 10, 90)) summary(a)     Min. 1st Qu.  Median    Mean 3rd Qu.    Max.     0.000   0.000   1.000   1.761   4.000   5.000   summary(b)     Min. 1st Qu.  Median    Mean 3rd Qu.    Max.     0.000   0.000   1.000   1.764   4.250   5.000  ``

The means and medians look very nearly the same for the two schools. The formal test has P-value $$0.6723 > 0.05$$ so the difference is not significant at the 5% level. For samples as large as these, the implementation of the Wilcoxon rank sum test in R, reports no difficulty handling ties.

`` wilcox.test(a,b)          Wilcoxon rank sum test with continuity correction  data:  a and b W = 61350, p-value = 0.6723 alternative hypothesis:     true location shift is not equal to 0 ``

Chisquared test of homogeneity of probabilities: Counts are

``f.a = c(150, 50, 30, 20, 5, 80) f.b = c(175, 40, 30, 15, 10, 90) MAT = rbind(f.a, f.b);  MAT     [,1] [,2] [,3] [,4] [,5] [,6] f.a  150   50   30   20    5   80 f.b  175   40   30   15   10   90 ``

I agree with @Glen_b that the Wilcoxon test is best here, because the issue seems to be which school had the higher scores overall. However, the chi-squared test will test whether the probabilities of getting individual scores 0 through 5 are substantially the same at the two schools. No difference between the two distributions is found.

``chisq.test(MAT)          Pearson's Chi-squared test  data:  MAT X-squared = 5.1107, df = 5, p-value = 0.4025 ``

Kolmogorov-Smirnov test of of differences in CDFs:

This is a test to see if there is a substantial difference between the empirical CDFs (ECDFs) of the two samples. To make an ECDF, sort the data and plot a stairstep function that increases by $$1/n,$$ where $$n$$ is the sample size, at each data value. If there are $$k$$ tied observations at a point, then jump up by $$k/n.$$ Here are the ECDFs of for the two schools, blue for a and orange for b.

``plot(ecdf(a), col="blue", lwd=2) lines(ecdf(b), col="orange", lty="dotted", pch="o") `` The test statistic $$D$$ of the K-S test is the maximum vertical distance between the two ECDFs. Again here, no significant difference is found. For your data, this test may be considered an alternative to the chi-squared test.

``ks.test(a,b)          Two-sample Kolmogorov-Smirnov test  data:  a and b D = 0.03835, p-value = 0.9605 alternative hypothesis: two-sided  Warning message: In ks.test(a, b) :     p-value will be approximate in the presence of ties  ``

Rate this post