Una prueba que nunca falla, un positivo, y 6,7 %

A disease affects one person in two hundred. There is a test for it, and the test is flawless in the direction you would worry about first: if you have the disease, it always says so. It has never once returned a clean result for a sick person. Its only weakness runs the other way, calling about $7\%$ of perfectly healthy people sick.

You take the test. It comes back positive. What is the probability that you actually have the disease?

Most people answer somewhere above $90\%$, and a good number say exactly $93\%$. The true figure is

So more than nine positives out of ten belong to people with nothing wrong with them. This is question 4.30 in Timothy Crack's Heard on the Street, and some version of it turns up in quantitative interviews constantly, because it separates candidates who manipulate conditional probabilities from candidates who pattern-match on the number they were handed.

Where the instant answer goes wrong

The $93\%$ answer comes from a single silent substitution. You are told that the test misfires on $7\%$of healthy people, so it feels natural to say the test is “93% accurate” and therefore that a positive result is 93% likely to be right. That reads the conditional backwards.

The missing ingredient is the prevalence, $P(D) = 1/200$. It never appears in the reflex answer, which is exactly why the reflex answer cannot be right: change the disease from common to vanishingly rare, leave the test untouched, and the reflex would return the same 93% every time. A quantity that ignores its own input is not measuring anything.

A thousand people, and the answer you can see

Before any algebra, stop thinking about yourself and think about a crowd. Take a thousand people and run the test on all of them.

Five of them have the disease, since the prevalence is one in two hundred. The test catches every one of those five, because it has no false negatives. That leaves 995 healthy people, and the test flags $7\%$ of them, which is

So about 75 people walk out holding a positive result, and only 5 of them are sick. Your result is one of those 75, and you have no way to tell which kind you are. That gives

The rounded version is easier to carry into an interview: five real cases against seventy false ones is one in fifteen, near enough. The reason the healthy group wins is not that the test is bad. It is that $7\%$ of a very large group is much bigger than $100\%$ of a very small one.

The same result in one line of algebra

The crowd argument is Bayes' theorem with the denominators cleared. Written out, with $D$ for having the disease and $+$ for a positive result,

and substituting the three numbers the question gives,

There is a faster route that is worth knowing, because it survives contact with messier numbers: work in odds instead of probabilities. The prior odds of being sick are $\tfrac{1/200}{199/200} = \tfrac{1}{199}$. The likelihood ratio of a positive result is

and Bayes in odds form is a single multiplication:

Same answer, no denominator to assemble. The likelihood ratio says the evidence multiplies your odds by about fourteen. That sounds decisive until you remember what it multiplies: odds of 1 to 199. Fourteen times almost nothing is still almost nothing.

What happens when the disease gets rarer

Keep the test exactly as it is and make the disease ten times rarer, one person in two thousand. With $p$ for prevalence and $f$ for the false positive rate, and still no false negatives, the general form is

which at $p = 1/2000$ and $f = 0.07$ gives

A positive result now means roughly one chance in a hundred and forty. Nothing about the instrument changed. For small $p$ the expression is close to $p/f$, so the answer falls in proportion to the prevalence: the rarer the thing you are hunting, the more of your alarms are noise. That approximation is the one worth memorising, since it turns the whole question into a division you can do out loud.

Screening a population is not testing a patient

None of this says the test is useless, and the difference is where the practical lesson lives. Everything above assumed you were pulled at random out of the general population, so the only thing known about you before the test was the base rate.

Someone who walks into a clinic with symptoms is not a random draw. Their prior is not 1 in 200, it might be 1 in 5, and the same test on the same day produces

The identical positive result now means something almost entirely different. This is why mass screening for rare conditions generates so many frightening letters and so few diagnoses, and why a second, independent test changes the picture so fast: run one with the same likelihood ratio and the odds multiply again, which is what pushes $100/1393$ up past even money.

In an interview, the move being tested is whether you ask for the prevalence at all. A candidate who answers before asking has told the room they will estimate a conditional probability without checking what it is conditioned on.

Sources and further reading

The problem is question 4.30 in Timothy Falcon Crack, Heard on the Street: Quantitative Questions from Wall Street Job Interviews, which collects problems used in real trading and quant interviews. The underlying result is standard: see the Wikipedia entries for Bayes' theorem, the base rate fallacy, and sensitivity and specificity, which is where the vocabulary of false positives and false negatives is defined precisely.

The framing in terms of a crowd rather than a single person follows Gerd Gigerenzer's work on natural frequencies, which found that doctors given the same problem in counts rather than percentages get it right far more often than when it is posed with conditional probabilities.