In this book on matrix factorizations, the author states the following, which I don't find to be true empirically. Is it true and under what conditions?
ADD: Trying to recreate the answer in R, what is the issue?
> D<-matrix(c(7,1,4,5,2,2,4,5,5,2,5,1,7,8,0,7),nrow=4,ncol=2,byrow=TRUE) > D [,1] [,2] [1,] 7 1 [2,] 4 5 [3,] 2 2 [4,] 4 5 > cor(D) [,1] [,2] [1,] 1.0000000 -0.3333333 [2,] -0.3333333 1.0000000 > > D[,1]<-D[,1]-mean(D[,1]) > D[,2]<-D[,2]-mean(D[,2]) > > D[,1]<-D[,1]/norm(as.matrix(D[,1])) > D[,2]<-D[,2]/norm(as.matrix(D[,2])) > D [,1] [,2] [1,] 0.50000000 -0.3214286 [2,] -0.04545455 0.2500000 [3,] -0.40909091 -0.1785714 [4,] -0.04545455 0.2500000 > t(D)%*%D [,1] [,2] [1,] 0.4214876 -0.1103896 [2,] -0.1103896 0.2602041
Best Answer
I think the author is simply handy-wavy there. I believe it is assumed that the columns of $A$ have norm 1 and mean 0.
$A^TA$ is a Gram matrix. Given you are using random variables to construct $A$, $A^TA$ is approximately proportional to the covariance matrix (scale by $n$ the number of variables). The correlation matrix is simply the scaled version of the covariance matrix. Clearly if your random variables in the columns of $A$ are already normalized to unit-norm then $A^TA$ does not need any further normalization and it is immediately a correlation matrix.
For example using MATLAB:
% Set the seed and generate a random matrix rng(123); A = randn(10,3); A'*A % ans = % 4.867589606955550 -3.004809945345502 -1.615373090426428 % -3.004809945345502 5.356131932084330 -0.457643222208441 % -1.615373090426428 -0.457643222208441 5.574303027408192. % Normalized now all the variables to have unit norm and mean zeros A = A - repmat(mean(A),10,1); A(:,1) = A(:,1)./norm(A(:,1)); A(:,2) = A(:,2)./norm(A(:,2)); A(:,3) = A(:,3)./norm(A(:,3)); A'*A % ans = % 1.000000000000000 -0.568373671796724 -0.336996715690262 % -0.568373671796724 1.000000000000000 -0.052272607974969 % -0.336996715690262 -0.052272607974969 1.000000000000000
If we wanted to make the columns of $A$ orthogonal too, we would orthonormalize $A$. In that case we would do a Gram-Schmidt process on $A$; a process relating to a lot of wonderful things eg. Givens rotations, Householder transformation, etc.)
Similar Posts:
- Solved – Matrix multiplication to find correlation matrix
- Solved – Matrix multiplication to find correlation matrix
- Solved – Calculating Cosine Similarity with Matrix Decomposition (matrix multiplication with normalized columns)
- Solved – Bounds on correlation to ensure covariance matrix is positive definite
- Solved – With an R function that expects a covariance matrix, can I give it a correlation matrix