I know the activation function (logistic) is linear around 0.

Does this then imply a neural net should be able to approximate a GLM given the right bias and weights?

I am currently playing in R with a simple model `y~x1`

.

The GLM performs well out of sample, yet the nnet with 1 hidden variable does poorly.

What am I missing?

**Contents**hide

#### Best Answer

Using the same reasoning, you could say a 2nd degree polynomial function can approximate a linear function. It is true, but the polynomial model is more complex.

This means that you will need more data to fit it correctly and it will have a stronger tendency to overfit. So, if the true model is a line and you fit/evaluate a linear model versus a 2nd degree polynomial model, the linear model will do likely better for a given amount of data.

### Similar Posts:

- Solved – Logistic Regression – Adding a polynomial basis to the input matrix make sense
- Solved – Polynomial approximations of nonlinearities in neural networks
- Solved – Polynomial approximations of nonlinearities in neural networks
- Solved – Rationale behind shrinking regression coefficients in Ridge or LASSO regression
- Solved – Difference between a non-linear neuron vs non-linear activation function