This question got me thinking about the meaning of variance: Intuition behind standard deviation.
Variance of a set of data is calculated the same way that the moment of inertia is calculated for a physical body. The moment of inertia is related to the energy required to rotate the body at a given speed. A figure skater will rotate faster with arms pulled in than stretched out. So what would be the analogous result of reducing variance, if any. Perhaps the analogy simply breaks down. Are there any publications that have investigated this analogy?
Best Answer
The direct analogy is pretty clear:
To make it simple we'll assume it's for a continuous random variable on $(a,b)$. Without loss of generality, let $c=b-a$ and consider the corresponding variable on $(0,c)$; call that random variable $X$.
Now imagine a very thin rod of length $c$, whose density (mass per element of length) is variable in the x-direction (along its length) and consider that the rod happens to have the same material-density as a function of $x$ as the random variable has probability density as a function of $x$.
Then then second moment of inertia of the rod is the variance of $X$.
And hence what it 'means' to rotate a distribution is clear enough – it's quite literally rotating the 'rod' whose density represents probability-density. Variance is how 'hard' it would be to rotate the rod (low variance means 'easy to spin', high variance means it takes more push to spin it … and stop it, if you spin it).
Think about what inertia (how hard it is to spin) reflects here, which is simply how close the mass is to the mean. The closer the mass is to the mean the easier it is to spin. If you made a physical object whose physical density represents the probability density and the random variable had low variance, the corresponding object would be easy to spin, because most of the mass would be close to the mean – both inertia and variance are how close the mass is to the mean, in a particular (and directly analogous) sense.
You don't actually 'spin' a probability density and imagine that to be physically difficult, any more than electricity is wet because of the water analogy. To expect that level of correspondence is to miss the point of such analogies (the aspects that correspond, correspond, but not every consequence of the correspondence in one realm carries over with it).
The point of saying the 'rod is hard to spin' is to give a pretty direct sense of what high variance is telling you about density. But to insist that the probability density itself spin is to miss the point.