If you have a matrix with n rows and m columns, you can use SVD or other methods to calculate a low-rank approximation of the given matrix.
However, the low rank approximation will still have n rows and m columns. How can low-rank-approximations be useful for machine learning and natural language processing, given that you are left with the same number of features?
Best Answer
A low rank approximation $hat{X}$ of $X$ can be decomposed into a matrix square root as $G=U_{r}lambda_{r}^frac{1}{2}$ where the eigen decomposition of $X$ is $Ulambda U^T$, thereby reducing the number of features, which can be represented by $G$ based on the rank-r approximation as $hat{X}=GG^T$. Note that the subscript $r$ represents the number of eigen-vectors and eigen-values used in the approximation. Hence, it does reduce the number of features to represent the data. In some examples low-rank approximations are considered as basis or latent variable (dictionary) based expansions of the original data, under special constraints like orthogonality, non-negativity (non-negative matrix factorization) etc.
Similar Posts:
- Solved – Full-Rank design matrix from overdetermined linear model
- Solved – Rank of a matrix in Polynomial regression model
- Solved – How do we know $X’X$ is nonsingular in OLS
- Solved – Low-rank SVD reconstruction and linear projection of the data
- Solved – Low-rank SVD reconstruction and linear projection of the data