Home Statistical Dictionary Bayes' Theorem

Bayes' Theorem

A formula for reversing conditional probabilities, computing the probability of a hypothesis given observed evidence from the probability of the evidence given the hypothesis, combined with the hypothesis's prior probability.

ProbabilityIntermediateFree

In Plain English

Bayes' Theorem is the mathematical recipe for updating your beliefs when new evidence comes in. It answers a question people naturally get backwards all the time, given that I've observed some evidence, how likely is a particular explanation, by carefully combining how likely that evidence would be under each possible explanation with how likely each explanation was to begin with.

Definition

For events $A$ and $B$ with $P (B) > 0$ , Bayes' Theorem states $P (A ∣ B) = \frac{P ( B ∣ A ) P ( A )}{P ( B )}$ , where $P (A)$ is the prior probability, $P (B ∣ A)$ is the likelihood, $P (A ∣ B)$ is the posterior probability, and $P (B)$ (typically computed via the Law of Total Probability) is the normalizing constant. It is the mathematical foundation of the entire Bayesian approach to statistics, providing the formal mechanism for updating a prior belief into a posterior belief in light of observed data.

Formula

P (A ∣ B) = \frac{P ( B ∣ A ) P ( A )}{P ( B )}

Notation

P (A)

—The prior probability of A

P (B ∣ A)

—The likelihood of the evidence B, given A

P (A ∣ B)

—The posterior probability of A, given the evidence B

Worked Example

If a disease affects 1% of a population ( $P (D) = 0.01$ ), a test has 95% sensitivity ( $P (+ ∣ D) = 0.95$ ) and 90% specificity ( $P (- ∣ no D) = 0.90$ , so $P (+ ∣ no D) = 0.10$ ), then given a positive result, Bayes' Theorem gives $P (D ∣ +) = \frac{0.95 \times 0.01}{0.95 \times 0.01 + 0.10 \times 0.99} \approx 0.088$ , only about 8.8%, a classic illustration of the base rate fallacy, most positive results on a rare disease are false positives.

Common Misconceptions

Confusing $P (A ∣ B)$ with $P (B ∣ A)$ , treating a test's sensitivity as if it were the post-test probability of disease, is one of the most consequential errors in applied probabilistic reasoning, and is exactly the error Bayes' Theorem is designed to correct for.

Properties

Bayes' Theorem is the mathematical basis for the entire Bayesian school of statistical inference, treating parameters and hypotheses as themselves having probability distributions (priors) that are updated into posteriors in light of observed data via exactly this formula.
The classic 'false positive paradox', where a highly accurate medical test on a rare condition still produces mostly false positives among all positive results, is a direct and famous consequence of Bayes' Theorem correctly weighting the low prior probability against the test's imperfect specificity.
The theorem's denominator, $P (B)$ , is typically not known directly and must be computed via the Law of Total Probability, summing $P (B ∣ A_{i}) P (A_{i})$ across every possible hypothesis $A_{i}$ , tying Bayes' Theorem and the Law of Total Probability tightly together in practice.

At a Glance

FormulaP(A|B) = P(B|A) * P(A) / P(B)

P(A)the prior probability

P(B|A)the likelihood

P(A|B)the posterior probability

Last updated August 9, 2026← Back to the Dictionary