Whats the difference between LDA and Linear SVM for dimensionality reduction. I am little confuse as LDA also looks for projection that separates the classes of data and SVM we also look for hyperplane …

**Contents**hide

#### Best Answer

None of the algorithms are for dimensionality reduction.(but see last comment)

In terms of classification, LDA finds the hyperplane that "best separates" **all** the data points while linear SVM looks for the hyperplane that "best separates" **only** the points in the frontier between the two classes.

Thus LDA has to do with the mean of each of the two classes (since it deals with all the points and the mean is a "good summary" of all the points). If the two classes are basically spherical then the LDA hyperplane is perpendicular to the line linking the two means of the points.

What is confusing is that most of the explanations of LDA talks about "**projection** of the data that best separates the classes". What is confusing is that the projection is onto a line and not a hyperplane – the LDA projection line (or LDA direction) is the line orthogonal to the separating hyperplane.

Take the image below (I did not create the image – found through Google but the original site no longer exists. I will remove it if it violates any copyright).

The LDA is the pane in the right. The green line is the LDA **direction** . The LDA hyperplane is the line perpendicular to the green that separates the two regions (further confusing is that in a 2D data the separating hyperplane is also a line!!)

The figure in sklearn example of LDA shows the separating hyperplane of the LDA (also notice that since the data are not spherically distributed in each class the LDA separating hyperplane is **not** perpendicular to the line joining the two centers/means)

Finally, LDA can be seen as an **extreme** dimensionality reduction (contrary to what I wrote first). It reduces the data to **one** dimension – if you have to remember only one dimension of the data and you still want to classify it, the best thing you can do is to project the data to the LDA direction!!!