Bayesian Thinking
Thomas Bayes' 18th-century theorem for updating beliefs as new evidence arrives — and why it took over a century to be widely appreciated.
Bernstein covers Thomas Bayes' theorem, developed in the mid-1700s and published posthumously, as a fundamentally different way of thinking about probability from the frequency-based approach covered earlier in this course (the law of large numbers, insurance pricing from observed claims frequency). Bayesian probability treats probability as a measure of subjective degree of belief that should be systematically updated as new evidence arrives, using a precise mathematical rule for combining a prior belief with new evidence to produce an updated, "posterior" belief — a formal method for the everyday, intuitive process of revising an opinion as new information comes in.
The book notes that Bayesian reasoning was, for a long time, treated with suspicion by more traditional, purely frequency-based statisticians, precisely because it explicitly incorporates a subjective starting belief (the "prior") rather than relying purely on observed frequency data — an objection that eased as Bayesian methods proved practically valuable across many fields, including finance, medicine, and — as Bernstein notes — reportedly used in classified applications like wartime code-breaking and search-and-rescue probability calculations, precisely in situations where there isn't enough repeatable historical frequency data available and a systematic way to update from limited, evolving evidence is genuinely necessary.
A new belief is formed by combining what was believed before with how strongly the new evidence supports or contradicts it — a formal, mathematical version of the everyday process of updating an opinion as new information arrives.
Bernstein doesn't gloss over the genuine philosophical difficulty at the center of Bayesian reasoning: the theorem tells you how to update a belief given new evidence, but it says nothing about where the starting prior belief itself should come from, which means two equally rational people can start from different priors, observe the identical evidence, and rationally arrive at different conclusions. The book treats this not as a flaw to be embarrassed about but as an honest reflection of how real reasoning under uncertainty actually works — prior experience and existing belief legitimately shape how new evidence should be weighted, and pretending otherwise, as purely frequency-based approaches sometimes implicitly do, is its own kind of false objectivity.
- Bayes' theorem provides a mathematical rule for updating a belief as new evidence arrives, combining a prior belief with the strength of the new evidence.
- This differs fundamentally from frequency-based probability (the law of large numbers) by explicitly incorporating a subjective starting belief.
- Bayesian methods proved especially valuable in situations lacking enough repeatable historical data for a purely frequency-based approach to work.