Solved – the difference between the anti-image covariance and the anti-image correlation

A excerpt of another answer (about factor analysis), structured.

Let $bf R$ be a correlation or covariance matrix, and $bf D$ be the diagonal matrix comprised of the inverses of diagonal elements of $bf R^{-1}$. Then

• $bf DR^{-1}D$ is known as anti-image covariance matrix of $bf R$. Its off-diagonal entries are the negatives of the partial covariance coefficients (between two variables controlled for all the other variables). The diagonal is equal to the diagonal $bf D$, – these diagonal values are called anti-images in $bf R$.

• $bf (DR^{-1}D)-2D+R$ is called image covariance matrix of $bf R$. Its diagonal entries are called images in $bf R$ (they are equal to the diagonal of $bf R-D)$. An image is $R_i^2 sigma_i^2$, where $R_i^2$ is the squared multiple correlation coefficient of dependency of variable $i$ on the rest variables, and $sigma_i^2$ is the diagonal element in $bf R$, the variance (or $1$, in case of correlation matrix).

• From the above it becomes clear that image + anti-image = $sigma_i^2$, and that the two are the portions of a variable's variation being, respectively, explained and unexpained (residual) by the other variables. Thus, if $bf R$ is correlations then image is the $R_i^2$ and anti-image is $1-R_i^2$; while if $bf R$ is covariances then image is $R_i^2 sigma_i^2$ and anti-image is $sigma_i^2-R_i^2sigma_i^2=sigma_i^2(1-R_i^2)$.

• Terminologic warning: the image and anti-image covariance matrices bear label "covariance" irrespective of whether $bf R$ is covariances or correlations.

• Anti-image correlation matrix of $bf R$ is computed from anti-image covariances the very usual way like we convert usual covariance into usual correlation, $r_{ij}=cov_{ij}/(sigma_i sigma_j)$, – i.e. here the cov and the two sigmas will be the values from an anti-image covariance matrix. Or in matrix notation: $bf D^{-1/2} A D^{-1/2}$, where $bf A$ is the anti-image covariance matrix and $bf D^{-1/2}$ is its diagonal, square-rooted and inversed. Equivalent formula also is $bf D^{1/2} R^{-1} D^{1/2}$, where $bf D^{1/2}$ is the $bf R^{-1}$'s diagonal, square-rooted and inversed. Off-diagonal elements of anti-image correlation matrix are the negatives of the partial correlation coefficients (between two variables controlled for all the other variables). And that is popular way to compute partial correlations.

• One can also convert, analogously, image covariance matrix into image correlation matrix, if needed.

• Anti-image correlation matrix will be the same – be $bf R$ covariances or correlations (while anti-image covariance matrix was different in the two instances).