Solved – Why bother with low rank approximations

If you have a matrix with n rows and m columns, you can use SVD or other methods to calculate a low-rank approximation of the given matrix.

However, the low rank approximation will still have n rows and m columns. How can low-rank-approximations be useful for machine learning and natural language processing, given that you are left with the same number of features?

A low rank approximation $hat{X}$ of $X$ can be decomposed into a matrix square root as $G=U_{r}lambda_{r}^frac{1}{2}$ where the eigen decomposition of $X$ is $Ulambda U^T$, thereby reducing the number of features, which can be represented by $G$ based on the rank-r approximation as $hat{X}=GG^T$. Note that the subscript $r$ represents the number of eigen-vectors and eigen-values used in the approximation. Hence, it does reduce the number of features to represent the data. In some examples low-rank approximations are considered as basis or latent variable (dictionary) based expansions of the original data, under special constraints like orthogonality, non-negativity (non-negative matrix factorization) etc.

Similar Posts:

Rate this post

Leave a Comment