Solved – Can a neural net (MLP) approximate a GLM

I know the activation function (logistic) is linear around 0.

Does this then imply a neural net should be able to approximate a GLM given the right bias and weights?

I am currently playing in R with a simple model y~x1.

The GLM performs well out of sample, yet the nnet with 1 hidden variable does poorly.

What am I missing?

Using the same reasoning, you could say a 2nd degree polynomial function can approximate a linear function. It is true, but the polynomial model is more complex.

This means that you will need more data to fit it correctly and it will have a stronger tendency to overfit. So, if the true model is a line and you fit/evaluate a linear model versus a 2nd degree polynomial model, the linear model will do likely better for a given amount of data.

Similar Posts:

Rate this post

Leave a Comment