# Solved – PCA: Eigenvectors of opposite sign and not being able to compute eigenvectors with `solve` in R

I'm learning PCA in R language. I met two problems right now that I don't understand.

1. I am performing a PCA analysis in R on a 318×17 dataset using some custom code. I take eigen function in R to find eigenvalues and eigenvectors. But my 1st and 3rd eigenvectors are of the opposite sign to my handbook. My second eigenvectors is almost the same.

2. I know that given a square matrix A, the condition that characterizes an eigenvalue, $$lambda$$, is the existence of a nonzero vector $$x$$ such that $$Ax=lambda x$$; this equation can be rewritten as follows: $$(A – lambda)x=0$$.

Now I calculate covariance of my data and have eigenvalues. I want to solve this linear combination equation to find $$x$$ and compare with initial eigenvectors. When I take solve function in R, my $$x$$ vector is always zero.

Here are my questions: Why the sign is different? How to use solve function in R to find a non-zero vector $$x$$?

Contents

1) The definition of eigenvector \$Ax = lambda x\$ is ambidextrous. If \$x\$ is an eigenvector, so is \$-x\$, for then

\$\$A(-x) = -Ax = -lambda x = lambda (-x)\$\$

So the definition of an eigenbasis is ambiguous of sign.

2) It's hard to know for sure, but I have a strong suspicion of what is happening here. Your equation

\$\$ (A – lambda)x = 0 \$\$

is technically incorrect. The correct equation is

\$\$ (A – lambda I)x\$\$

The first equation is often used as a shorthand for the second. In general, this is unambiguous, because there is no real mathematical way to subtract a vector from a square matrix, but it is abuse of notation. In `R` though, you have broadcasting. So if you do

``> M <- matrix(c(1, 1, 1, 1), nrow=2) > M - .5      [,1] [,2] [1,]  0.5  0.5 [2,]  0.5  0.5 ``

its not really what you want. The proper way would be

``> M - diag(.5, 2)      [,1] [,2] [1,]  0.5  1.0 [2,]  1.0  0.5 ``

The reason you are getting zero solutions is that the matrix you are starting with \$A\$ is invertible. More than likely (almost surely), the matrix you get by subtracting the same number from every entry will also be invertible. For invertible matrices, the only solution to \$Ax = 0\$ is the zero vector.

Rate this post

# Solved – PCA: Eigenvectors of opposite sign and not being able to compute eigenvectors with `solve` in R

I'm learning PCA in R language. I met two problems right now that I don't understand.

1. I am performing a PCA analysis in R on a 318×17 dataset using some custom code. I take eigen function in R to find eigenvalues and eigenvectors. But my 1st and 3rd eigenvectors are of the opposite sign to my handbook. My second eigenvectors is almost the same.

2. I know that given a square matrix A, the condition that characterizes an eigenvalue, $$lambda$$, is the existence of a nonzero vector $$x$$ such that $$Ax=lambda x$$; this equation can be rewritten as follows: $$(A – lambda)x=0$$.

Now I calculate covariance of my data and have eigenvalues. I want to solve this linear combination equation to find $$x$$ and compare with initial eigenvectors. When I take solve function in R, my $$x$$ vector is always zero.

Here are my questions: Why the sign is different? How to use solve function in R to find a non-zero vector $$x$$?

1) The definition of eigenvector \$Ax = lambda x\$ is ambidextrous. If \$x\$ is an eigenvector, so is \$-x\$, for then

\$\$A(-x) = -Ax = -lambda x = lambda (-x)\$\$

So the definition of an eigenbasis is ambiguous of sign.

2) It's hard to know for sure, but I have a strong suspicion of what is happening here. Your equation

\$\$ (A – lambda)x = 0 \$\$

is technically incorrect. The correct equation is

\$\$ (A – lambda I)x\$\$

The first equation is often used as a shorthand for the second. In general, this is unambiguous, because there is no real mathematical way to subtract a vector from a square matrix, but it is abuse of notation. In `R` though, you have broadcasting. So if you do

``> M <- matrix(c(1, 1, 1, 1), nrow=2) > M - .5      [,1] [,2] [1,]  0.5  0.5 [2,]  0.5  0.5 ``

its not really what you want. The proper way would be

``> M - diag(.5, 2)      [,1] [,2] [1,]  0.5  1.0 [2,]  1.0  0.5 ``

The reason you are getting zero solutions is that the matrix you are starting with \$A\$ is invertible. More than likely (almost surely), the matrix you get by subtracting the same number from every entry will also be invertible. For invertible matrices, the only solution to \$Ax = 0\$ is the zero vector.

Rate this post