When we estimate the posterior density function, we have the following equation:
$p(x|data) = frac{P(data|x)*p(x)}{p(data)}$
Let us think that our prior is a continuous distribution, say normal. We are trying to get probability of success in a trial that is Binomial, that is p(x|data). Now, What exactly this multiplication means?
$P(data|x)*p(x)$ ?
P(data|x) – is 1 value, say 0.10, or 0.5, etc. On the other hand p(x) is the value of the continuous probability density function (normal here).
How can we multiply the value of pdf with P(data|x)?
P(data|x) is the concrete realization of the binomial trial with some given parameter x. According to Bayes rule p(x) should be the probability of the parameter, but p(x) is density! Without integration it has no sense.
Could someone explain please?
Best Answer
$p$ is a function that returns the density evaluated at its argument; so $p(x)$ is a number — the value of the prior density evaluated at $x$.
Moving to a more typical statistics notation, where parameters are Greek letters and random variables/observations are Roman letters (presently handwaving the distinction between the latter two):
$p(theta mid y) = frac{p(y mid theta) cdot p(theta)}{p(y)}$
For any given value of $theta$, we can find $p(y|theta)$ (the likelihood) and $p(theta)$, and thereby take the product of those two numbers, obtaining the numerator and hence (up to the scaling factor in the denominator) the posterior evaluated at $theta$.
Similar Posts:
- Solved – Different notation for Bayes’ prior and posterior distributions
- Solved – Different notation for Bayes’ prior and posterior distributions
- Solved – When using Jeffrey’s prior for Normal model, what is $p_J(theta, sigma^{2} | y_{1}, …, y_{n})$ supposed to be
- Solved – When re-parametrizing a likelihood function, is it enough just to plug in the transformed variable instead of a change of variables formula
- Solved – Updating priors based on outcome of Bernoulli trials