Solved – Examples of when confidence interval and credible interval coincide

normal distribution:

Take a normal distribution with known variance. We can take this variance to be 1 without losing generality (by simply dividing each observation by the square root of the variance). This has sampling distribution:

$$p(X_{1}…X_{N}|mu)=left(2piright)^{-frac{N}{2}}expleft(-frac{1}{2}sum_{i=1}^{N}(X_{i}-mu)^{2}right)=Aexpleft(-frac{N}{2}(overline{X}-mu)^{2}right)$$

Where $A$ is a constant which depends only on the data. This shows that the sample mean is a sufficient statistic for the population mean. If we use a uniform prior, then the posterior distribution for $mu$ will be:

$$(mu|X_{1}…X_{N})sim Normalleft(overline{X},frac{1}{N}right)implies left(sqrt{N}(mu-overline{X})|X_{1}…X_{N}right)sim Normal(0,1)$$

So a $1-alpha$ credible interval will be of the form:

$$left(overline{X}+frac{1}{sqrt{N}}L_{alpha},overline{X}+frac{1}{sqrt{N}}U_{alpha}right)$$

Where $L_{alpha}$ and $U_{alpha}$ are chosen such that a standard normal random variable $Z$ satisfies:

$$Prleft(L_{alpha}<Z<U_{alpha}right)=1-alpha$$

Now we can start from this "pivotal quantity" for constructing a confidence interval. The sampling distribution of $sqrt{N}(mu-overline{X})$ for fixed $mu$ is a standard normal distribution, so we can substitute this into the above probability:

$$Prleft(L_{alpha}<sqrt{N}(mu-overline{X})<U_{alpha}right)=1-alpha$$

Then re-arrange to solve for $mu$, and the confidence interval will be the same as the credible interval.

Scale parameters:

For scale parameters, the pdfs have the form $p(X_{i}|s)=frac{1}{s}fleft(frac{X_{i}}{s}right)$. We can take the $(X_{i}|s)sim Uniform(0,s)$, which corresponds to $f(t)=1$. The joint sampling distribution is:

$$p(X_{1}…X_{N}|s)=s^{-N};;;;;;;0<X_{1}…X_{N}<s$$

From which we find the sufficient statistic to be equal to $X_{max}$ (the maximum of the observations). We now find its sampling distribution:

$$Pr(X_{max}<y|s)=Pr(X_{1}<y,X_{2}<y…X_{N}<y|s)=left(frac{y}{s}right)^{N}$$

Now we can make this independent of the parameter by taking $y=qs$. This means our "pivotal quantity" is given by $Q=s^{-1}X_{max}$ with $Pr(Q<q)=q^{N}$ which is the $beta(N,1)$ distribution. So, we can choose $L_{alpha},U_{alpha}$ using the beta quantiles such that:

$$Pr(L_{alpha}<Q<U_{alpha})=1-alpha=U_{alpha}^{N}-L_{alpha}^{N}$$

And we substitute the pivotal quantity:

$$Pr(L_{alpha}<s^{-1}X_{max}<U_{alpha})=1-alpha=Pr(X_{max}L_{alpha}^{-1}>s>X_{max}U_{alpha}^{-1})$$

And there is our confidence interval. For the Bayesian solution with jeffreys prior we have:

$$p(s|X_{1}…X_{N})=frac{s^{-N-1}}{int_{X_{max}}^{infty}r^{-N-1}dr}=N (X_{max})^{N}s^{-N-1}$$ $$implies Pr(s>t|X_{1}…X_{N})=N (X_{max})^{N}int_{t}^{infty}s^{-N-1}ds=left(frac{X_{max}}{t}right)^{N}$$

We now plug in the confidence interval, and calculate its credibility

$$Pr(X_{max}L_{alpha}^{-1}>s>X_{max}U_{alpha}^{-1}|X_{1}…X_{N})=left(frac{X_{max}}{X_{max}U_{alpha}^{-1}}right)^{N}-left(frac{X_{max}}{X_{max}L_{alpha}^{-1}}right)^{N}$$

$$=U_{alpha}^{N}-L_{alpha}^{N}=Pr(L_{alpha}<Q<U_{alpha})$$

And presto, we have $1-alpha$ credibility and coverage.