In the wikipedia article on Credible Interval, it says:

For the case of a single parameter and

data that can be summarised in a

single sufficient statistic, it can be

shown that the credible interval and

the confidence interval will coincide

if the unknown parameter is a location

parameter (i.e. the forward

probability function has the form Pr(x

| μ) = f(x − μ) ), with a prior that

is a uniform flat distribution;[5] and

also if the unknown parameter is a

scale parameter (i.e. the forward

probability function has the form Pr(x

| s) = f(x / s) ), with a Jeffreys'

prior [5] — the latter following

because taking the logarithm of such a

scale parameter turns it into a

location parameter with a uniform

distribution. But these are distinctly

special (albeit important) cases; in

general no such equivalence can be

made."

Could people give specific examples of this? When does the 95% CI actually correspond to "95% chance", thus "violating" the general definition of CI?

#### Best Answer

## normal distribution:

Take a normal distribution with known variance. We can take this variance to be 1 without losing generality (by simply dividing each observation by the square root of the variance). This has sampling distribution:

$$p(X_{1}…X_{N}|mu)=left(2piright)^{-frac{N}{2}}expleft(-frac{1}{2}sum_{i=1}^{N}(X_{i}-mu)^{2}right)=Aexpleft(-frac{N}{2}(overline{X}-mu)^{2}right)$$

Where $A$ is a constant which depends only on the data. This shows that the sample mean is a sufficient statistic for the population mean. If we use a uniform prior, then the posterior distribution for $mu$ will be:

$$(mu|X_{1}…X_{N})sim Normalleft(overline{X},frac{1}{N}right)implies left(sqrt{N}(mu-overline{X})|X_{1}…X_{N}right)sim Normal(0,1)$$

So a $1-alpha$ credible interval will be of the form:

$$left(overline{X}+frac{1}{sqrt{N}}L_{alpha},overline{X}+frac{1}{sqrt{N}}U_{alpha}right)$$

Where $L_{alpha}$ and $U_{alpha}$ are chosen such that a standard normal random variable $Z$ satisfies:

$$Prleft(L_{alpha}<Z<U_{alpha}right)=1-alpha$$

Now we can start from this "pivotal quantity" for constructing a confidence interval. The sampling distribution of $sqrt{N}(mu-overline{X})$ for fixed $mu$ is a standard normal distribution, so we can substitute this into the above probability:

$$Prleft(L_{alpha}<sqrt{N}(mu-overline{X})<U_{alpha}right)=1-alpha$$

Then re-arrange to solve for $mu$, and the confidence interval will be the same as the credible interval.

## Scale parameters:

For scale parameters, the pdfs have the form $p(X_{i}|s)=frac{1}{s}fleft(frac{X_{i}}{s}right)$. We can take the $(X_{i}|s)sim Uniform(0,s)$, which corresponds to $f(t)=1$. The joint sampling distribution is:

$$p(X_{1}…X_{N}|s)=s^{-N};;;;;;;0<X_{1}…X_{N}<s$$

From which we find the sufficient statistic to be equal to $X_{max}$ (the maximum of the observations). We now find its sampling distribution:

$$Pr(X_{max}<y|s)=Pr(X_{1}<y,X_{2}<y…X_{N}<y|s)=left(frac{y}{s}right)^{N}$$

Now we can make this independent of the parameter by taking $y=qs$. This means our "pivotal quantity" is given by $Q=s^{-1}X_{max}$ with $Pr(Q<q)=q^{N}$ which is the $beta(N,1)$ distribution. So, we can choose $L_{alpha},U_{alpha}$ using the beta quantiles such that:

$$Pr(L_{alpha}<Q<U_{alpha})=1-alpha=U_{alpha}^{N}-L_{alpha}^{N}$$

And we substitute the pivotal quantity:

$$Pr(L_{alpha}<s^{-1}X_{max}<U_{alpha})=1-alpha=Pr(X_{max}L_{alpha}^{-1}>s>X_{max}U_{alpha}^{-1})$$

And there is our confidence interval. For the Bayesian solution with jeffreys prior we have:

$$p(s|X_{1}…X_{N})=frac{s^{-N-1}}{int_{X_{max}}^{infty}r^{-N-1}dr}=N (X_{max})^{N}s^{-N-1}$$ $$implies Pr(s>t|X_{1}…X_{N})=N (X_{max})^{N}int_{t}^{infty}s^{-N-1}ds=left(frac{X_{max}}{t}right)^{N}$$

We now plug in the confidence interval, and calculate its credibility

$$Pr(X_{max}L_{alpha}^{-1}>s>X_{max}U_{alpha}^{-1}|X_{1}…X_{N})=left(frac{X_{max}}{X_{max}U_{alpha}^{-1}}right)^{N}-left(frac{X_{max}}{X_{max}L_{alpha}^{-1}}right)^{N}$$

$$=U_{alpha}^{N}-L_{alpha}^{N}=Pr(L_{alpha}<Q<U_{alpha})$$

And presto, we have $1-alpha$ credibility and coverage.

### Similar Posts:

- Solved – relation between confidence interval and likelihood function
- Solved – 95% Confidence interval of $lambda$ for $X_1,…,X_n$ IID exponential with rate $lambda$
- Solved – Prediction interval = credible interval
- Solved – the connection between credible regions and Bayesian hypothesis tests
- Solved – Confidence interval using Central Limit Theorem