One Head Tilts a Flat Prior to 2p, and the Average Bias to Two Thirds

A flat belief about a coin's bias is an input to the calculation, not a conclusion of it, and a single head does not leave it standing. The density tilts to 2p, the cumulative law becomes p squared, the average bias moves to 2/3, and the old answer of one half is demoted to the lower quartile. The general update is the Beta conjugate family, which sends 750 heads in 1000 to Beta(751, 251) with mean 0.749501.

A coin comes off a badly run production line. Its bias, meaning the probability $P$ that it lands heads, is equally likely to be any number between $0$ and $1$ . You toss it once. It lands heads. What do you now believe about $P$ ?

Most people answer that nothing has changed, or that the coin is presumably fair, which is the same answer said twice. One toss carries almost no information, the reasoning goes. The instinct about the quantity of information is right and the conclusion is wrong. After that single head the average value of $P$ is $2/3$ , and the shape of the belief has changed from a flat line into a straight ramp.

Where the answer one half legitimately lives

The reflex has a defensible version, and locating it is worth more than dismissing it. Had the problem stated that the coin was fair, one head would change nothing at all: a fair coin stays fair whatever it does. The problem says something else. It says the bias is unknown and flat across the unit interval, and a flat description of ignorance is an input to the calculation rather than a conclusion of it. Inputs do not survive observations.

There is exactly one place where $1/2$ belongs, and it is the denominator we are about to write down. It is also worth knowing what becomes of that number afterwards. Once the head has been observed, $1/2$ is the lower quartile of the belief about $P$ , so three quarters of the remaining credence sits above it. The trap answer does not simply move. It is demoted to the twenty-fifth percentile.

Bayes in continuous form, done by hand

Write $f$ for the density of $P$ before the toss. Uniformity means $f(p) = 1$ on $[0,1]$ , which integrates to one, so every value really is on an equal footing. The probability of the observation, given a particular bias $p$ , is $p$ itself. That is the entire likelihood. Bayes' rule in its density form then reads:

f(p \mid \text{heads}) \;=\; \frac{p \cdot f(p)}{\displaystyle\int_0^1 q \, f(q)\, \mathrm{d}q}

(1)

The denominator is the unconditional probability of seeing a head before you know anything, and with a flat prior it equals $\int_0^1 q \,\mathrm{d}q = 1/2$ . Dividing by it forces the constant rather than leaving it to taste:

f(p \mid \text{heads}) \;=\; 2p, \qquad 0 \le p \le 1.

(2)

The mechanism is worth saying in words. Each candidate bias is reweighted in proportion to how readily it would have produced the head you actually saw. A bias of $0.9$ delivers a head nine times as often as a bias of $0.1$ , so it ends up nine times as credible. Renormalising that reweighted curve turns the horizontal line into a line of slope two.

The whole law, not only its average

Integrating the density gives the cumulative law, and from there every quantile follows:

F(p) = \int_0^p 2q\,\mathrm{d}q = p^2, \qquad \mathbb{E}[P \mid \text{heads}] = \int_0^1 p \cdot 2p \,\mathrm{d}p = \frac{2}{3}.

(3)

Solving $p^2 = 1/2$ puts the median at $1/\sqrt{2} = 0.7071$ , and the quartiles are exactly $1/2$ and $\sqrt{3}/2 = 0.8660$ . Note that the mean and the median disagree, which they must, because the density is skewed. When the answer to this question is quoted as $2/3$ it is the average bias being quoted, not the typical one. And since $F(1/2) = 1/4$ , the probability that the coin actually favours heads is $3/4$ after a single toss.

Fig. 1 — One head tilts the flat line into the ramp 2p. The mean lands at 2/3, the median further right at 0.7071, and the old answer 1/2 becomes the lower quartile.

From one toss to a thousand

Nothing above depended on the toss count being one. With $k$ heads in $n$ independent tosses the likelihood is $p^{k}(1-p)^{n-k}$ , and reweighting the flat prior by it produces a density proportional to the same expression. That is a Beta law:

f(p \mid k, n) \;=\; \frac{p^{k}(1-p)^{n-k}}{B(k+1,\, n-k+1)}, \qquad \mathbb{E}[P \mid k, n] \;=\; \frac{k+1}{n+2}.

(4)

Set $k = n = 1$ and the mean returns $2/3$ . The single-toss answer is the general answer evaluated at a small argument, not a separate construction, and that is the cleanest reason to trust it.

Conjugate prior — Beta against Bernoulli

A $\mathrm{Beta}(\alpha, \beta)$ belief about a success probability, updated on $k$ successes and $n-k$ failures, returns a $\mathrm{Beta}(\alpha + k,\, \beta + n - k)$ belief with mean $(\alpha + k)/(\alpha + \beta + n)$ . The family is closed under the update, which is what conjugate means. The uniform prior is $\mathrm{Beta}(1,1)$ , so one head sends it to $\mathrm{Beta}(2,1)$ , whose density is $2p$ .

Now run the same machine on real data. Seven hundred and fifty heads in a thousand tosses: equation (4) gives $\mathrm{Beta}(751, 251)$ , with mean $751/1002 = 0.749501$ and standard deviation $0.013682$ . Its mode is $750/1000 = 0.75$ exactly, the observed frequency. The cumulative law climbs from $F(0.70) = 0.00024$ to $F(0.80) = 0.99995$ , so essentially all of the belief lives inside a window one tenth of a unit wide.

Here a wording error is easy to make and worth naming, because it is the kind that survives editing. It is the cumulative law that is nearly a step at $0.75$ . The density is a narrow spike, and a spike is not a step. The two objects live on different vertical scales, the density peaking near $29$ while the cumulative law is capped at one by construction, and calling either one by the other name gives away that the distinction was never noticed.

Fig. 2 — After 750 heads in 1000 tosses. The cumulative law is nearly a step at 0.75; the density is a spike of spread 0.0137. Reading one as the other is the standard slip.

What you should pay for the next toss

The distribution is the answer to the question asked, but the question a trader would ask next is what to bet on toss number two. Conditioning on the bias and averaging over it gives the predictive probability, and the tower property collapses the whole calculation to the posterior mean:

\mathbb{P}(\text{next is heads} \mid k, n) \;=\; \mathbb{E}\!\left[\,\mathbb{P}(\text{heads} \mid P)\,\right] \;=\; \mathbb{E}[P \mid k, n] \;=\; \frac{k+1}{n+2}.

(5)

This is the rule of succession that Laplace wrote down in 1774. After one head it says $2/3$ , the same number as before. After 750 heads in a thousand it says $751/1002 = 0.7495$ rather than $0.75$ . The rule never quite reaches the observed frequency because the prior keeps pulling toward the middle, by an amount of order $1/n$ that dies out but never vanishes.

Where this stops working

Three hypotheses carry the whole derivation, and each fails in a way worth recognising.

The flat prior is a choice and not the absence of one. Jeffreys' prior for a Bernoulli parameter is $\mathrm{Beta}(1/2, 1/2)$ , which is the one invariant under reparameterisation and therefore has a better claim to being uninformative. Update it on one head and you land on $\mathrm{Beta}(3/2, 1/2)$ , whose mean is $3/4$ . Two defensible descriptions of ignorance, two different answers from the same single observation. The disagreement is fatal at $n = 1$ and invisible by $n = 1000$ , where the Jeffreys posterior mean is $750.5/1001 = 0.749750$ against $0.749501$ . Data washes out the prior, and how much data it takes depends on how much the two priors disagree.

Independence given $P$ is doing real work. It is what turns the likelihood into a product and hands you the Beta form. A coin whose surface wears with use, or a tosser who learns, breaks the product and everything downstream of it.

The concentration statement needs the true bias to be interior. Suppose the coin has heads on both faces, so $P = 1$ . After $n$ heads the belief is $\mathrm{Beta}(n+1, 1)$ with mean $(n+1)/(n+2)$ , which crawls toward one from below without arriving, and whose spread shrinks like $1/n$ rather than the usual $1/\sqrt{n}$ . The comfortable picture of a symmetric spike with a two-sigma interval around it belongs to interior truths. At the boundary the spike is one-sided and the interval runs off the end of the parameter space, where it means nothing.

Sources and further reading

Pierre-Simon Laplace, Mémoire sur la probabilité des causes par les événements (1774), where the rule of succession first appears.
Harold Jeffreys, “An Invariant Form for the Prior Probability in Estimation Problems”, Proceedings of the Royal Society A186 (1946), 453–461.
Wikipedia: Bayes' theorem, Beta distribution and Rule of succession.

Every quantity above was checked twice before publication: symbolically, by integrating the densities, and numerically, by a rejection sampler that reproduces the law $2p$ without ever using the closed form.