# Solved – How to test equivalence of two models using MSE

I want to compare two generalized linear mixed models (GLMM), model A and model B, which differ from their link function. More specifically, I want to test wheter these models are equivalent or model A is better than model B.
The dependant variable is an always positive and continuous variable. Model A is the linear mixed model and model B is the GLMM with the log link function. I want to proove that using the additive model or multiplicative model is equivalent\$^*\$. I have no assumption about whether the model is additive or multiplicative.
I choose the Mean Squared Error (MSE i.e. \$displaystyle frac{1}{N}sum^N_{i=1} left(widehat{y}_i – y_iright)^2\$) as criterion for comparing the two models.

Thus, the hypotheses are \$H_0 : MSE_A = MSE_B \$ and \$H_1 : MSE_A < MSE_B\$

The test statistic is \$T = MSE_A – MSE_B \$

As the T-distribution is unknow, I thought to :

1. create P datasets randomly permuted between the predictions of the two models (i.e. for each \$i\$, \$widehat{y}_{A,i}\$ randomly assigne to the A or B model and \$widehat{y}_{B,i}\$ to the other one)
2. for each permuted dataset \$p\$, compute the \$T^*_p\$ statistic
3. compare the observed \$T\$ value to the \$T^*\$ distribution to conclude on the test

Is it a valid process? If not, any suggestion on how to implement this test?

\$^*\$ My boss want me to use the additive model. I want to proove him (or not) that is equivalent to use the multiplicative model,regarding model performance. Thus, if two models are equivalent, it is more accurate to use the GLMM with log link function because the dependant variable is always positive.

Contents