Why are polynomial functions bad as activations?
Contents
hide
Best Answer
There has been some work which experiments with quadratic activations — see "neural tensor networks" but in general a disadvantage of second order and higher polynomials is that they don't have a bounded derivative, which could lead to exploding gradients.
Similar Posts:
- Solved – Polynomial approximations of nonlinearities in neural networks
- Solved – Polynomial approximations of nonlinearities in neural networks
- Solved – Neural Networks: What activation function should I choose for hidden layers in regression models
- Solved – Neural Networks: What activation function should I choose for hidden layers in regression models
- Solved – To what exactly does the term “activations” refer in neural networks