Statistics

The bell curve

Why sums of many small, independent chances end up looking the same, whatever the chances were to begin with.

From coins to curves

Toss a fair coin a hundred times and count the heads. Fifty is the most likely result, but only just: it happens about 8% of the time. Forty-five or fifty-five are nearly as likely, thirty or seventy almost never happen. Plot the chance of every count and you get a symmetrical hump.

In 1733 Abraham de Moivre showed that for a fair coin this hump approaches a particular smooth curve, the one we now call the normal or Gaussian curve. Laplace extended the result in 1812 to coins that are not fair, with any probability p of heads. The statement is usually called the de Moivre–Laplace theorem.

–mean, n·p
–spread, √(n·p·(1−p))
–largest error of the curve
–quality of the fit

Bars are the exact binomial probabilities; the red line is the normal curve with the same mean and spread. “Largest error” compares each bar with the area under the curve from k−½ to k+½. Try a small n with p near 0.05 to see the approximation fail, then raise n.

The central limit theorem

Laplace went further. In a memoir read in 1810 he argued that the same curve appears for the sum, or average, of many independent errors of almost any kind, not just coin tosses. That is the central limit theorem, and it explains why the normal curve shows up for measurement errors, heights, test scores and the balls in a Galton board.

It also justified a practical habit. If an average of many observations is approximately normal, you can say how far it is likely to be from the truth. Laplace used this to combine astronomical measurements and to judge whether a small effect in his data was real.

Gauss and Laplace

Carl Friedrich Gauss arrived at the normal curve from the other direction in 1809, asking which error law would make the average the best estimate. Laplace read Gauss's work within months and saw that his own limit theorem supplied the missing reason to believe in that law. The curve took Gauss's name in most of the world. In France it is still sometimes called la loi de Laplace–Gauss.