Solved – Finding the decision boundary between two gaussians

by

Assume we are trying to classify between 2 classes, each has a Gaussian conditional probability, with different means but same variance, i.e. $X | y=0 sim N(mu_0, Sigma); X | y=1 sim N(mu_1, Sigma)$.

Our decision rule would be $1 iff P(y = 1 | X) > P(y=0|X)$ (and vice versa for 0).

Using Bayes rule we can invert the conditional probabilities, and get: $iff frac{P(X|y=1)P(y=1)}{P(X)} > frac{P(X|y=0)P(y=0)}{P(X)}$.

We can next eliminate the denominator.

Now, if $P(y=1) = P(y=0)$ we could eliminate that as well, and the decision rule would be simplified to $P(X|y=1) > P(X|y=0)$, which basically is which $mu$ is $X$ closer to. Now intuitively this translates to $1 iff X > c = frac{mu_0 +mu_1}{2}$.

I have 2 questions:

  1. Can we prove this intuition mathematically?

  2. What happens if $P(y=1) neq P(y=0)$ ?

Contents hide
1 Best Answer
2 Similar Posts:

Best Answer

  1. Yes you can. For example, $$ mathbb{P}left(X=x|y=1right)=mathbb{P}left(X=x|y=0right) $$ happens exactly when $x$ is between the means, which can be calculated directly from the definition. Essentially, you are finding the decision boundary for this image, description,

but it's even simpler because you are assuming that the standard deviations are equal. You'd need to show with a simple equation that the probability (area under the curve) on the right is equal to the area on the left if the boundary is in the middle.

  1. I'm not willing to do the math here, but you have to slightly alter the equation above to find what happens to the boundary. I can tell it doesn't change linearly, because, for example, if you believe the left bubble to be twice as likely as the right bubble, you're looking for a boundary so that the area under what's left of the left bubble is half that of what's right of the boundary under the right bubble.

Similar Posts:

Rate this post

Leave a Comment