It is known that for the normal distribution, the mean and the standard deviation are independent; it is also the case for samples as sample size tends to infinity.
However, for small samples, the sample mean and the sample standard deviation are not independent. Indeed, the sample mean is used to compute the sample standard deviation.
As an example, simulating 100,000 samples of size 1000, and computing for each means and standard deviations, I get a correlation of 0.0005 between the 2 statistics. For 1,000,000 samples of size 3, I get a negative correlation of -0.0003.
Is there a way to find the expected correlation as a function of sample size? Its distribution? Or is it zero on average?
Best Answer
While the (usual) formula for sample standard deviation contains the sample mean (so you might say that it depends on the mean algebraically), that doesn't impact their independence in the statistical sense.
The population correlation of the sample mean and the sample standard deviation is zero because the random variables (sample mean, sample standard deviation) are independent.
Indeed $bar{X}$ and $X_i-bar{X}$ are independent and the independence of the sample variance (and similarly standard deviation) from $bar{X}$ follows from that.
Below is a plot of $x_1-bar{x}$ vs $bar{x}$ for samples of size 10 from a normal distribution; it's possible to show these are independent (and indeed, jointly normal).
The sample correlation between the random variables won't be exactly zero — it, too, is a random variable. If you gather $m$ samples of size $n$ and compute the mean and sd in each sample of size $n$ and calculate their correlation over the $m$ samples, that will have some deviation from 0 each time, but the population quantity it estimates is 0. Imagine choosing say 50 points in the plot above for example; the sample correlation of those 50 points won't be exactly 0.
Similar Posts:
- Solved – How to estimate population mean from a single sample
- Solved – How to estimate population mean from a single sample
- Solved – How to estimate population mean from a single sample
- Solved – How to estimate population mean from a single sample
- Solved – Is it possible to estimate the standard deviation of a normal distribution if I only have the mean of the population