Solved – How to find and draw the Bayes decision boundary in LDA (2 classes)

Can anyone point me (or even show me if that's allowed here) how to calculate and draw the boundary line in LDA when we have only 2 classes? In the example I'm trying to do (in a paper), the informations given to us are the following:

Prior probabilities
$$π_1 = 0.7$$
$$π_2 = 0.3$$
Mean vectors
$$μ_1 = binom{0}{4}$$
$$μ_2 = binom{2}{2}$$
Covariance matrix
$$Σ = begin{pmatrix}
1 & 0.25\
0.25 & 1\

Main idea:

Let $y in {1,2}$ be the output, and $x in mathbb{R}^2$ be the input. Using Bayes' theorem:

  1. $p(y=1 mid x) = frac{p(x mid y=1)pi_1}{p(x)}$
  2. $p(y=2 mid x) = frac{p(x mid y=2)pi_2}{p(x)}$.

Plugging in a two-dimensional input $x$ into these formulas, you will get the probability of ending up with the conditional probability that $y=1$ or $y=2$. The decision boundary is the line in $mathbb{R}^2$ where these two conditional probabilities are equal. To find this line, solve $p(y=1 mid x) = p(y=2 mid x)$ or $$ p(x mid y=1)pi_1 = p(x mid y=2)pi_2. $$ Taking it a little further: begin{align*} .7 expleft(-.5[x – mu_1]^TSigma^{-1}[x – mu_1] right) – .3 expleft(-.5[x – mu_2]^TSigma^{-1}[x – mu_2] right)&= 0 \ expleft(-.5left{-2x^TSigma^{-1}mu_1 + 2 x^TSigma^{-1}mu_2 + mu_1^T Sigma^{-1}mu_1 – mu_2^T Sigma^{-1}mu_2 right} right) &= frac{3}{7} \ -2x^TSigma^{-1}mu_1 + 2 x^TSigma^{-1}mu_2 + mu_1^T Sigma^{-1}mu_1 – mu_2^T Sigma^{-1}mu_2 &= 1.69\ x^Tleft[2Sigma^{-1}(mu_2 – mu_1)right]+ left{mu_1^T Sigma^{-1}mu_1 – mu_2^T Sigma^{-1}mu_2right} &= 1.69. end{align*}

Assuming both classes have the same $Sigma$ comes in handy a few times. You can see how it starts to look linear (in $x$). Hence the "L" in LDA.

Similar Posts:

Rate this post

Leave a Comment