Home Statistical Dictionary Selection Bias
Selection Bias
A systematic distortion in a sample or study group caused by the way units were selected or included, such that the sample is not representative of the population it is meant to describe.
In Plain English
Selection bias is what happens when the process that decided who's in your sample or study is itself tangled up with the thing you're trying to measure, so the people (or units) you end up observing aren't a fair cross-section of everyone you care about. A famous example is surveying gym-goers about exercise habits, the very act of being at the gym already selects for people more inclined toward exercise, so any conclusion about 'typical' exercise habits drawn from that sample will be systematically skewed. Selection bias can creep in through the sampling design itself (convenience sampling), through who chooses to respond (nonresponse bias), or through who happens to survive long enough to be observed (survivorship bias), and unlike random sampling error, it doesn't shrink or average out no matter how large the biased sample gets.
Definition
Selection bias is a systematic error that arises when the mechanism determining which units are observed is correlated with the outcome under study, producing a sample that is not representative of the target population regardless of sample size. Selection bias can arise from the sampling design itself (non-probability methods like convenience sampling), from differential participation (nonresponse bias, where respondents systematically differ from nonrespondents), from a study's eligibility criteria interacting with the outcome (as in survivorship bias, where only units that 'survived' some prior process are observable), or from self-selection into a group being studied; unlike sampling error, which shrinks toward zero as sample size grows, selection bias is a bias in the strict statistical sense, its expected magnitude does not diminish with a larger sample, and can only be addressed through the sampling or study design itself, or through explicit statistical correction (such as inverse-probability weighting) when the selection mechanism is well understood.
Properties
- Selection bias is a bias, not a variance problem, this is the key distinction from sampling error, increasing the sample size makes a selection-biased estimate more precisely wrong, it narrows the confidence interval around an estimate that is still centered on the wrong value, rather than correcting the underlying distortion.
- Survivorship bias is a particularly subtle and common form of selection bias, arising whenever a study can only observe units that survived some earlier filtering process, such as analyzing the investment returns of currently operating mutual funds while ignoring funds that closed or failed, which mechanically inflates the apparent average return.
- Selection bias can sometimes be statistically corrected when the selection mechanism is well understood and measurable, methods like inverse-probability weighting or Heckman-style selection models explicitly model the probability of being included in the observed sample and reweight or adjust accordingly, though such corrections are only as good as the selection model's assumptions.
At a Glance