I would like to intuitively understand the benefit of using the natural exponential in the sigmoid function used in logistic regression.
Why should it have to be $e^x$ instead of, for example $2^x$?
Best Answer
Because base $e$ is convenient, and it doesn't matter if you can freely scale your coefficient estimate.
Would using a functional form of $frac{a^mathbf{xcdot b}}{1 + a^mathbf{xcdot b} }$ change your explanatory power? No.
Explanation:
I gave basically the same answer here for the softmax function.
Observe that $ e^ { mathbf{x} cdot mathbf{b} left( ln a right) } = a^ {mathbf{x} cdot mathbf{b}}$. Hence:
$$ frac{a^mathbf{xcdot b}}{1 + a^mathbf{xcdot b} } = frac{e^mathbf{xcdot tilde{b}}}{1 + e^mathbf{xcdot tilde{b}} } $$
Where $tilde{mathbf{b}} = left( ln a right) mathbf{b} $. So using a different base than $e$ in the sigmoid function is the same as scaling your $mathbf{b}$ vector.
Similar Posts:
- Solved – Why do we use the natural exponential in logistic regression
- Solved – Why ‘e’ in softmax
- Solved – Formula for standardized Regression Coefficients(derivation and intuition)
- Solved – Posterior Predictive Distribution as Expectation of Likelihood
- Solved – Predictive Posterior Distribution of Normal Distribution with Unknown Mean and Variance