Let's say I have the following perceptron activation line:

$y = 1$ if $sum_{i}^{d} w_ix_i geq theta $ and $y=0$ else. Now, for d = 4, I have derived the following question.

$$ w_4x_4+w_3x_3+w_2x_2+w_1x_1 + w_o$$

I am trying to prove that the distance from the decision boundary to the origin is $ frac{w^Tx}{||w||}$. I have read the following post.

However, I'm not able to understand why we are trying to minimize $x$, which confuses the rest of the reasoning for me. Can someone please elucidate this?

**Contents**hide

#### Best Answer

The key observation is that the boundary is exactly

$ w_4x_4+w_3x_3+w_2x_2+w_1x_1 + w_o = 0$,

and that this is exactly the equation of a hyperplane. So the question is how to find the point on this hyperplane that is closest to the origin.

The answer to which you refer solves this particularly elegantly, but, if you prefer, you can bypass it and do it any other number of ways.

For example, the point on this hyperplane with the shortest distance to the origin, is exactly the point on this hyperplane with the shortest *square* distance to the origin. Using a Lagrange multipliers, therefore, you can optimize

$ sum_i [x_i^2] $

subject to

$ w_4x_4+w_3x_3+w_2x_2+w_1x_1 + w_0 = 0$,

by solving

$ sum_i [x_i^2] + lambda ( w_4x_4+w_3x_3+w_2x_2+w_1x_1 + w_0)$

for the zero derivative.

Differentiating with respect to $x_j$ gives

$2 x_j + lambda w_j = 0 Rightarrow x_j = -frac{lambda}{2} w_j$ (1).

Differentiating with respect to $lambda$ gives

$w_4x_4+w_3x_3+w_2x_2+w_1x_1 + w_0 = 0$ (2).

Inserting (1) into (2) gives

$w_0 = frac{lambda}{2} left[w_1^2 + w_2^2 + w_3^2 + w_4^2right] Rightarrow lambda = frac{2 w_0}{left[w_1^2 + w_2^2 + w_3^2 + w_4^2right]}$.

Inserting back into (1) gives

$x_j = frac{w_j w_0}{left[w_1^2 + w_2^2 + w_3^2 + w_4^2right]}$ (3).

The distance to the origin is $sqrt{x_1^2 + x_2^2 + x_3^2 + x_4^2}$. Inserting (3) into this, gives $frac{left| w_0 right|}{sqrt{w_1^2 + w_2^2 + w_3^2 + w_4^2}}$

### Similar Posts:

- Solved – Why One class SVM seperate from the origin
- Solved – Getting distance of points from decision boundary with linear SVM
- Solved – Understanding linear projection in “The Elements of Statistical Learning”
- Solved – SVM why do we maximize 2/||w||
- Solved – Suppose X and Y are independent Poisson random variables with respective parameters $lambda$ and 2$lambda$. Find $E[Y-2*X|X+Y=10]$