Lambdia

Five Numbers, Two Answers, and the Divisor Nobody Asks About

The standard deviation of 1, 2, 3, 4, 5 is either 1.4142 or 1.5811, and offering one of them without asking which question you are answering is the only wrong move. The sum of squared deviations is 10 either way, so everything turns on whether you divide it by 5 or by 4. Bessel's correction makes the variance unbiased and leaves the standard deviation biased low by about six percent at this sample size, and a third divisor beats both of them if you optimise for mean squared error instead.

What is the standard deviation of 1, 2, 3, 4, 5? It looks like a warm-up question, the kind you answer while the interviewer is still finishing the sentence. There are two defensible answers, 21.4142\sqrt{2} \approx 1.4142 and 2.51.5811\sqrt{2.5} \approx 1.5811, and the fastest way to fail is to pick one of them without saying why.

The question is not a trick. It is genuinely under-specified, and the missing piece is not a number. It is whether those five values are everything there is, or a handful drawn from something larger that you cannot see.

The part nobody disputes

Both answers share every step but the last one. The mean of an arithmetic progression is its midpoint, so xˉ=3\bar{x} = 3 without any addition. The deviations from that mean are 2,1,0,1,2-2, -1, 0, 1, 2, and squaring them gives 4, 1, 0, 1, 4:

i=15(xixˉ)2  =  4+1+0+1+4  =  10\sum_{i=1}^{5} (x_i - \bar{x})^2 \;=\; 4 + 1 + 0 + 1 + 4 \;=\; 10
(1)
Fig. 1 — Everything up to the sum of squares is forced. The disagreement is entirely about what to divide 10 by.

Now the fork. Divide by 5 and you get a variance of 2, so the deviation is 2\sqrt{2}. Divide by 4 and you get 2.5, so the deviation is 2.5\sqrt{2.5}. The ratio between the two answers is fixed and does not depend on the data at all:

σ=105=2=1.4142,s=104=52=1.5811,sσ=54\sigma = \sqrt{\tfrac{10}{5}} = \sqrt{2} = 1.4142, \qquad s = \sqrt{\tfrac{10}{4}} = \sqrt{\tfrac{5}{2}} = 1.5811, \qquad \frac{s}{\sigma} = \sqrt{\tfrac{5}{4}}
(2)
The two quantities, kept apart

The population variance of a finite list of NN values divides the sum of squared deviations by NN and describes that list, nothing else. The sample variance divides by n1n-1 and is not a description of anything. It is a guess at the variance of a distribution you never see, made from nn draws out of it.

Read that way, the two numbers answer different questions and both are right. The wrong answer is a single number offered with no question attached.

It is worth knowing how large the disagreement actually is, because the ratio in equation (2) is n/(n1)\sqrt{n/(n-1)} whatever the numbers happen to be. That is 11.8 per cent apart at n=5n = 5, 2.6 per cent at n=20n = 20, and half a per cent at n=100n = 100. The divisor is a genuine fork on five values and a rounding detail on a hundred, which is exactly why the question gets asked with five. Small samples are the only place the two conventions can be told apart by eye.

Where the missing divisor goes

The usual explanation for n1n-1 is a slogan about degrees of freedom, and the slogan is true. The deviations are not five independent numbers, because they satisfy one equation whether you like it or not:

i=1n(xixˉ)=0\sum_{i=1}^{n} (x_i - \bar{x}) = 0
(3)

Tell me four of the deviations and I can write down the fifth. So the deviations live in an (n1)(n-1)-dimensional subspace, not an nn-dimensional one, and dividing a sum of squares by 5 when only 4 of the terms could vary is dividing by too much.

The slogan is a picture, though, and the reason is a computation. Suppose the xix_i are independent draws from a distribution with mean μ\mu and variance σ2\sigma^2. Expand the sum of squares around the sample mean and take expectations:

E[i(xixˉ)2]=E[ixi2]nE[xˉ2]=n(σ2+μ2)n(σ2n+μ2)=(n1)σ2\mathbb{E}\Big[\sum_i (x_i - \bar{x})^2\Big] = \mathbb{E}\Big[\sum_i x_i^2\Big] - n\,\mathbb{E}[\bar{x}^2] = n(\sigma^2 + \mu^2) - n\Big(\tfrac{\sigma^2}{n} + \mu^2\Big) = (n-1)\sigma^2
(4)

The μ2\mu^2 terms cancel, which is the whole trick, and what is left is short by exactly one σ2\sigma^2. That shortfall has a cause you can name: the sample mean was fitted from the same five numbers, so it sits closer to them than the true mean does, and deviations measured from it are systematically too small. Dividing by n1n-1 pays that debt back precisely.

Bessel fixes the square, not the root

Here is the part that most treatments leave out, and it matters more than the choice of divisor. Equation (4) says the n1n-1 version is an unbiased estimator of the variance. It says nothing about the standard deviation, and the standard deviation is what you were asked for.

Taking a square root does not commute with taking an expectation. The square root is concave, so Jensen’s inequality runs in one direction only:

E[s]=E[s2]  <  E[s2]  =  σ\mathbb{E}[s] = \mathbb{E}\big[\sqrt{s^2}\,\big] \;<\; \sqrt{\mathbb{E}[s^2]} \;=\; \sigma
(5)

The inequality is strict unless s2s^2 is constant, which for real data it never is. So ss is biased low, always, and no choice of divisor repairs it. For a normal sample the size of the bias is known in closed form. Write E[s]=c4(n)σ\mathbb{E}[s] = c_4(n)\,\sigma with

c4(n)  =  2n1  Γ(n/2)Γ ⁣(n12),c4(5)=12Γ(5/2)Γ(2)=0.939986c_4(n) \;=\; \sqrt{\frac{2}{n-1}}\;\frac{\Gamma(n/2)}{\Gamma\!\left(\frac{n-1}{2}\right)}, \qquad c_4(5) = \sqrt{\tfrac{1}{2}}\,\frac{\Gamma(5/2)}{\Gamma(2)} = 0.939986
(6)

At n=5n = 5 the corrected sample deviation still lands about six percent below the truth on average. A simulation of two hundred thousand normal samples of size five measures the mean of s2s^2 at 0.9987 times σ2\sigma^2, which is the unbiasedness of equation (4) showing up as advertised, and the mean of ss at 0.9391, against the exact 0.9400. The factor climbs slowly: 0.8862 at n=3n = 3, 0.9727 at n=10n = 10, 0.9914 at n=30n = 30.

Fig. 2 — The expected value of the sample deviation, as a fraction of the truth. Bessel's correction removes the bias in the variance and leaves this one behind.

None of this makes 2.5\sqrt{2.5} the wrong answer. It makes it a correctly computed sample standard deviation, which is a different thing from an unbiased estimate of anything.

A cross-check that never picks a divisor

If 2\sqrt{2} only appeared because somebody divided by 5, it would be an artefact. It is not, and there are two ways to see that without touching the sum of squares. The first is the shortcut identity:

Var(X)=E[X2](E[X])2=1+4+9+16+25532=119=2\operatorname{Var}(X) = \mathbb{E}[X^2] - \big(\mathbb{E}[X]\big)^2 = \frac{1+4+9+16+25}{5} - 3^2 = 11 - 9 = 2
(7)

The second treats 1 to 5 as what it is under the population reading, a discrete uniform distribution on five consecutive integers. That family has a closed-form variance:

Var(U{1,,n})=n2112,25112=2\operatorname{Var}\big(\mathcal{U}\{1,\dots,n\}\big) = \frac{n^2 - 1}{12}, \qquad \frac{25 - 1}{12} = 2
(8)

Two routes with no shared arithmetic, both landing on 2. So 2\sqrt{2} is a property of a distribution and would be the answer even if you had never written down a deviation.

Where the question is genuinely ill posed

A list handed to you as 1, 2, 3, 4, 5 with nothing else said is a complete enumeration, and for a complete enumeration n1n-1 is not a refinement, it is a mistake. There is no larger population to estimate, no sampling error to correct, and inflating the variance by 5/45/4 describes a spread the data does not have. So if forced to give one number with no chance to ask, 2\sqrt{2} is the better guess about what the asker meant.

One more wrinkle is worth knowing, because it shows the fork is not really about the words population and sample. Compute the sample variance without Bessel’s correction and you also get 2. The two candidate answers correspond to two divisors, and three readings of the question map onto them, which is why arguing about the vocabulary settles nothing and asking about the divisor settles everything.

And once you accept that the divisor follows from what you are trying to achieve, a third candidate walks in. Suppose you want the estimator of the variance with the smallest mean squared error among all multiples of the sum of squares, rather than the one with no bias. For normal data T=i(xixˉ)2T = \sum_i (x_i - \bar{x})^2 is σ2χn12\sigma^2 \chi^2_{n-1}, so the error of cTcT is

MSE(cT)=[2(n1)c2+(c(n1)1)2]σ4,minimised at c=1n+1\mathrm{MSE}(cT) = \Big[2(n-1)c^2 + \big(c(n-1) - 1\big)^2\Big]\sigma^4, \qquad \text{minimised at } c = \frac{1}{n+1}
(9)

Dividing by n+1n+1 beats dividing by n1n-1 on that criterion, and at n=5n = 5 it returns 10/6=1.66710/6 = 1.667 and a deviation of about 1.2910. Nobody teaches that number, because unbiasedness won the vocabulary war a century ago. It is still the right answer to a reasonable question, which is the cleanest evidence that the standard deviation of five numbers is not a well defined object until somebody says what it is for.

The formulas also disagree about how they die. At n=1n = 1 the population deviation is 0, which is correct and useless, while the sample version reads 0/00/0 and is undefined. That is the honest behaviour: a single observation carries no information at all about spread, and the estimator that admits it is the better one.

Sources and further reading

Every figure here was recomputed in exact rational arithmetic before publication, with the c4c_4 value checked against a two hundred thousand sample simulation that measured 0.9391 where the closed form predicts 0.9400.

Comments · 0

Be the first to comment.