I'm trying to determine the effect of taking a training course on various aspects of being a good quality worker. I have pre-test and post-test data on 7 different ways job performance can be viewed. I'm currently performing 7 different paired samples t-tests on the data, to see the effect on each individual aspect. Is there a test to calculate the overall effect on job performance (by comparing all 7 aspects at once)? I'm trying to use SPSS but not sure how to proceed.
Best Answer
You need to first spell out what amounts to "overall improvement". If "overall improvement" means improvement in each of the aspects then your test is essentially AND of individual tests. i.e.
Suppose your model is:
$ improvement = beta_0 + beta_1cdot x_1 + … + beta_7cdot x_7 + training*(delta_0 + delta_1cdot x_1 + … + delta_7cdot x_7) $
where $training$ is a binary indicator whether training was received or not. The parameters of this model are $vec{theta} = (beta_0, …, beta_7, delta_0, …, delta_7)^T$. Say $vec{theta}_delta = (delta_0, …, delta_7)^T$
Then as you mentioned you can individually test for:
$H_0: delta_i vs H_a: delta_i <= 0, forall i in (1, 7)$.
However if you wish to test for all of them simultaneously, then you are essentially testing for:
$H_0: vec{theta}_delta > vec{0} vs H_a: vec{theta}_delta <= 0$
Now, you can easily find the estimates $vec{hat{theta}_delta}$ of $vec{theta}_delta$ and $hat{sigma}{vec{theta}_delta}$.
Also,
$frac{vec{hat{theta}_delta} – vec{theta}_delta}{hat{sigma}{vec{theta}_delta}} sim t_{df}$ and $frac{vec{hat{theta}_delta}}{hat{sigma}{vec{theta}_delta}} stackrel{H_0}{sim} t_{df}$
Thus, all you need is a simple one-sided t-test.