Solved – What exactly is a hyperparameter

Title says it all.

I have seen both "the hyperparameter of the Dirichlet distribution" and "the parameter of the Dirichlet distribution"

What are the differences?

A hyperparameter is a parameter for the (prior) distribution of some parameter.

So for a simple example, let's say we state that the variance parameter $tau^2$ in some problem has a uniform prior on $(0,theta)$.

(I personally would be unlikely to do such a thing, but it happens; I might in some very particular circumstance)

Then $tau^2$ is a parameter (in the distribution of the data) and $theta$ is a hyperparameter.

If we then in turn specify a (prior) distribution for $theta$ (e.g. that it's Gamma with mean 100 and shape parameter 2), that's a hyperprior – a prior distribution on a parameter of a prior distribution.

Similar Posts:

Rate this post

Leave a Comment