Sampling Error

Sampling error is the difference between a sample-based estimate $\hat{\theta }$ and the true population parameter $\theta$ that arises solely from observing a sample rather than the entire population, formally captured by the sampling variance of the estimator, $\text{Var}(\hat{\theta })$, whose square root is the standard error reported alongside survey estimates. Sampling error is fully quantifiable for any probability sample, since the design determines the estimator's variance directly, and it decreases with sample size at a rate of roughly $1/\sqrt{n}$ (quadrupling the sample size roughly halves the sampling error), which is why increasing the sample size is the standard lever for tightening a margin of error; sampling error is fundamentally distinct from non-sampling error, which encompasses every other source of inaccuracy (nonresponse, measurement error, coverage gaps) and does not necessarily shrink, or may even grow, as the sample size increases.

• Sampling error is fully quantifiable only for probability samples, where the design assigns every unit a known selection probability, for non-probability samples there is no design-based way to compute a valid sampling error, which is exactly why margin-of-error claims attached to convenience or quota samples are on shaky statistical ground.
• A census, surveying the entire population rather than a sample, has zero sampling error by definition, but is rarely practical; the standard trade-off in survey design is accepting some quantifiable sampling error in exchange for the dramatically lower cost of surveying a sample instead.
• Because sampling error shrinks only slowly, at the rate 1/√n, doubling precision (halving the margin of error) requires quadrupling the sample size, which is why very tight margins of error (well under 1%) require enormous samples and are rarely seen outside of large-scale government surveys or censuses.