Type II Error

A Type II error occurs when a hypothesis test fails to reject a null hypothesis that is actually false, a false negative. Its probability is denoted $\beta =P(\text{fail to reject }{H}_0\mid H_1\text{ true})$, evaluated at a specific alternative value since β generally depends on how far the true parameter is from the null. Type II error is directly complementary to statistical power, $\text{Power}=1-\beta$, and unlike Type I error (which is fixed by the chosen significance level regardless of sample size), β shrinks as sample size increases, as the effect size grows, or as measurement variability decreases.

• Unlike the significance level α, which is fixed by the researcher's choice regardless of sample size, β is not a single fixed number, it depends on the true (unknown) size of the effect, β is smaller for larger true effects and larger for smaller true effects, which is why power calculations must specify a minimum effect size worth detecting.
• There is an inherent tradeoff between Type I and Type II error for a fixed sample size and test statistic, making the rejection region smaller (lowering α, to reduce false positives) simultaneously makes β larger (increasing the false-negative rate), the only way to reduce both simultaneously is to collect more data or use a more efficient test statistic.
• Underpowered studies, those with a high β for practically meaningful effect sizes, are a well-documented and persistent problem across many fields, a non-significant result from an underpowered study is only weak evidence that no effect exists, since a real effect could easily have gone undetected by chance.