Solved – How to specify logistic regression as transformed linear regression

I am trying to reproduce the following example of logistic regression with a transformed linear regression:

am.glm <- glm(am ~ hp + wt, data=mtcars, family=binomial) newdata <- data.frame(hp=120, wt=2.8) predict(am.glm, newdata, type="response")  ##         1  ## 0.6418125 

The equation for the probability of $Y=1$ is the following:
P(Y=1) = {1 over 1+e^{-(b_0+sum{(b_iX_i)})}}

So I tried something like this:

am.lm <- lm(am ~ 1/(1+exp(-(hp + wt))),data=mtcars) predict(am.lm, newdata) ##       1  ## 0.40625 

So this is obviously wrong! (I also tried transforming the given value but nothing worked so far).

My question
How would I have to set up logistic regression with explicitly specifying the formula for the non-linear transformation of the linear model?

The short answer is "you don't". They don't correspond.

Logistic regression is not a transformed linear regression.

Even though $E(Y)$ ($=P(Y=1)$) may be written as $text{logit}(Xbeta)$, and so seemingly linearized, you can't transform the $y$ values to make a linear regression, nor can you fit a nonlinear least squares model (transforming the x's) that reproduces the logistic regression. [You may be able to fit a function of the logistic form via nonlinear least squares but it won't have the same estimates.] Answers to this question gives some additional details.

The observations enter the model estimation quite differently.

Logistic regression is fitted by maximum likelihood estimation for a binomial model with the natural link function (which is the logit for the binomial). That is, the data are seen as $y_isimtext{binomial}(n_i,1/(1+e^{-X_ibeta}))$

(Where $X_ibeta=beta_0+sum_j beta_j x_{ij}$ is the linear predictor for casec$i$.)

Once you find the MLE for the model, you could find a set of weights and a set of pseudo-observations such that a linear model yields the parameter estimates, but the connection to the original data is quite indirect and in general you can't get to the point of doing so until you've already found the solution.

It's not particularly useful to think of that as a transformation of the data.

Similar Posts:

Rate this post

Leave a Comment