Probability

The sunrise problem

How confident should you be that something will happen again, given only how often it has happened before? Laplace's answer is a two-line formula with a long afterlife.

The famous paragraph

In the Essai philosophique sur les probabilités of 1814, Laplace gives an example that has been quoted, and argued over, ever since:

Placing the most ancient epoch of history at five thousand years ago, or at 1826213 days, and the sun having risen constantly in the interval at each revolution of twenty-four hours, it is a bet of 1826214 to one that it will rise again to-morrow.

He immediately adds that this number is “incomparably greater” for anyone who understands the mechanics of the solar system. The example is about reasoning from bare counts when you know nothing else, not about astronomy.

The rule of succession

Suppose an event has some unknown chance p of happening on each trial, and before seeing any data every value of p between 0 and 1 seems equally plausible. After s successes in n trials, the probability that the next trial succeeds is

P(next success) = (s + 1) / (n + 2)

The formula behaves sensibly where the plain frequency s/n does not. With no data at all it gives ½. After one success in one trial it gives ⅔, not certainty. And it never reaches exactly 0 or 1, however long the run.

–chance the next trial succeeds
–as a bet
–95% range for the true chance
–plain frequency s/n

The curve is the posterior distribution of p: a Beta(s+1, n−s+1) distribution. The shaded band holds 95% of it and the dashed line marks its mean, which is the rule of succession. The horizontal axis zooms to wherever the curve is.

Inverse probability

Laplace reached this result in a memoir of 1774 on “the probability of causes from events”. It is the idea now called Bayes' theorem: reason backwards from what you observed to how plausible each explanation is. Thomas Bayes had found a special case a decade earlier, but his essay was little read in France. Laplace developed the method on his own, stated it in general form and, crucially, used it on real data.

Boys and girls in Paris

His favourite real example was the ratio of births. Parish registers in Paris and London showed slightly more boys than girls, year after year. Was that chance, or a law of nature? Using the same reasoning as above on several hundred thousand recorded births, Laplace found that the odds against the true proportion of boys being one half or less were astronomically large. He concluded, correctly, that slightly more boys are born than girls, and then went looking for why the ratio differed a little between the two cities.

He applied the same ideas to estimate the population of France from a sample of communes, to judge the reliability of witnesses and courts, and to measure the errors in astronomical observations. They are the beginning of modern statistics.