Solved – How to find and draw the Bayes decision boundary in LDA (2 classes)

Main idea:

Let $y in {1,2}$ be the output, and $x in mathbb{R}^2$ be the input. Using Bayes' theorem:

1. $p(y=1 mid x) = frac{p(x mid y=1)pi_1}{p(x)}$
2. $p(y=2 mid x) = frac{p(x mid y=2)pi_2}{p(x)}$.

Plugging in a two-dimensional input $x$ into these formulas, you will get the probability of ending up with the conditional probability that $y=1$ or $y=2$. The decision boundary is the line in $mathbb{R}^2$ where these two conditional probabilities are equal. To find this line, solve $p(y=1 mid x) = p(y=2 mid x)$ or $$ p(x mid y=1)pi_1 = p(x mid y=2)pi_2. $$ Taking it a little further: begin{align*} .7 expleft(-.5[x – mu_1]^TSigma^{-1}[x – mu_1] right) – .3 expleft(-.5[x – mu_2]^TSigma^{-1}[x – mu_2] right)&= 0 \ expleft(-.5left{-2x^TSigma^{-1}mu_1 + 2 x^TSigma^{-1}mu_2 + mu_1^T Sigma^{-1}mu_1 – mu_2^T Sigma^{-1}mu_2 right} right) &= frac{3}{7} \ -2x^TSigma^{-1}mu_1 + 2 x^TSigma^{-1}mu_2 + mu_1^T Sigma^{-1}mu_1 – mu_2^T Sigma^{-1}mu_2 &= 1.69\ x^Tleft[2Sigma^{-1}(mu_2 – mu_1)right]+ left{mu_1^T Sigma^{-1}mu_1 – mu_2^T Sigma^{-1}mu_2right} &= 1.69. end{align*}

Assuming both classes have the same $Sigma$ comes in handy a few times. You can see how it starts to look linear (in $x$). Hence the "L" in LDA.