Bayesian Thinking
Thomas Bayes' 18th-century theorem for updating beliefs as new evidence arrives — and why it took over a century to be widely appreciated.
Bernstein covers Thomas Bayes' theorem, developed in the mid-1700s and published posthumously, as a fundamentally different way of thinking about probability from the frequency-based approach covered earlier in this course (the law of large numbers, insurance pricing from observed claims frequency). Bayesian probability treats probability as a measure of subjective degree of belief that should be systematically updated as new evidence arrives, using a precise mathematical rule for combining a prior belief with new evidence to produce an updated, "posterior" belief — a formal method for the everyday, intuitive process of revising an opinion as new information comes in.
The book notes that Bayesian reasoning was, for a long time, treated with suspicion by more traditional, purely frequency-based statisticians, precisely because it explicitly incorporates a subjective starting belief (the "prior") rather than relying purely on observed frequency data — an objection that eased as Bayesian methods proved practically valuable across many fields, including finance, medicine, and — as Bernstein notes — reportedly used in classified applications like wartime code-breaking and search-and-rescue probability calculations, precisely in situations where there isn't enough repeatable historical frequency data available and a systematic way to update from limited, evolving evidence is genuinely necessary.
A new belief is formed by combining what was believed before with how strongly the new evidence supports or contradicts it — a formal, mathematical version of the everyday process of updating an opinion as new information arrives.
- Bayes' theorem provides a mathematical rule for updating a belief as new evidence arrives, combining a prior belief with the strength of the new evidence.
- This differs fundamentally from frequency-based probability (the law of large numbers) by explicitly incorporating a subjective starting belief.
- Bayesian methods proved especially valuable in situations lacking enough repeatable historical data for a purely frequency-based approach to work.