What is the Likelihood function of a linear probability model?
I know the likelihood function is the joint probability density, but how to construct the likelihood function when we only have the probability $P(Y_i=1|X_i)$ and $P(Y_i=0|X_i)$?
Best Answer
In your example you simply don't have a likelihood function, because you defined just a probability model rather than a statistical one. If you know the probabilities $$ P(Y_i = 1 | X_i) = p_i quad mbox{and} quad P(Y_i =0| X_i) = 1-p_i, $$ then you have a "unique" conditional probability model. First you have to understand the differences between probability and statistical models. Please, see this post.
In order to have a statistical model, the above probabilities must be unknown. Typically, some relation is imposed: $$ P_theta(Y_i = 1 | X_i=x_i) = p_i(theta),$$ where $theta$ is the unknown parameter vector and $p_i(theta) in [0,1]$ for $i=1, ldots, n$. Now, we have a parametric statistical model, since for each $theta$ we have a probability model.
The likelihood function is $$ L(theta) = prod_{i=1}^n P_theta(Y_i = y_i | X_i=x_i) = prod_{i=1}^n p_i(theta)^{y_i}(1-p_i(theta))^{1-y_i}. $$
Usually, the shape of $p_i(theta)$ is commonly specified as:
- $p_i(theta) = frac{exp(eta_i(theta))}{1 + exp(eta_i(theta))},$
- $p_i(theta) = F(eta_i(theta)),$
where $eta_i(theta) = alpha + beta x_i$ and $F$ is a cumulative distribution function. You can find a good shape for $p_i$ by looking at the data behavior (plots, data dispersion and so forth).
Similar Posts:
- Solved – Maximum Likelihood Estimator (MLE) for $2 theta^2 x^{-3}$
- Solved – Some problems understanding the definition of a function in a maximum likelihood method
- Solved – exponential density & bernoulli distribution
- Solved – Conditional Maximum Likelihood – How is marginal probability of inputs are independent of the parameter we are estimating
- Solved – MLE of the mixture parameter in mixing two normal densities