Solved – Why do we use the natural exponential in logistic regression

I would like to intuitively understand the benefit of using the natural exponential in the sigmoid function used in logistic regression.

Why should it have to be $e^x$ instead of, for example $2^x$?

Because base $e$ is convenient, and it doesn't matter if you can freely scale your coefficient estimate.

Would using a functional form of $frac{a^mathbf{xcdot b}}{1 + a^mathbf{xcdot b} }$ change your explanatory power? No.


I gave basically the same answer here for the softmax function.

Observe that $ e^ { mathbf{x} cdot mathbf{b} left( ln a right) } = a^ {mathbf{x} cdot mathbf{b}}$. Hence:

$$ frac{a^mathbf{xcdot b}}{1 + a^mathbf{xcdot b} } = frac{e^mathbf{xcdot tilde{b}}}{1 + e^mathbf{xcdot tilde{b}} } $$

Where $tilde{mathbf{b}} = left( ln a right) mathbf{b} $. So using a different base than $e$ in the sigmoid function is the same as scaling your $mathbf{b}$ vector.

Similar Posts:

Rate this post

Leave a Comment