Solved – the approximation power of Gaussian mixture models

What are the probability density functions that GMM can approximate? a reference in appreciated about this.

Goodfellow et al. 2016, p. 65 states:

A Gaussian mixture model is a universal approximator of densities, in the sense that any smooth density can be approximated with any specific nonzero amount of error by a Gaussian mixture model with enough components.

M. Carreira ascribes this property to kernel density estimation with reference to Scott 1992 and another source that I could not find. Given the connection between KDE and GMMs this is understandable.

The user Xi'an provided an explanation for the above statement in this answer.

While this answers the question, it has to be noted that while it can theoretically approximate any smooth density, it shouldn't be used as a general purpose model.
Fitting a mixture of many components can quickly become more computationally expensive than using a more fitting parametric model. Examples of this could be distributions with very thin peaks, which one needs to approximate with very thin bandwidths, as well as distributions with long tails, which will be difficult to get right with either very wide Gaussians or many small ones. In these and probably many other cases it is preferrable to use a more fitting, if less general, model.

Similar Posts:

Rate this post

Leave a Comment