Home Statistical Dictionary Measurement Error
Measurement Error
The discrepancy between a survey response or recorded value and the true value it is meant to capture, arising from imprecise instruments, ambiguous questions, respondent recall failure, or social-desirability distortion -- one of the core components of total survey error alongside sampling and nonresponse error.
In Plain English
Measurement error is the gap between what a survey or instrument records and what's actually true, someone misremembers how many times they visited a doctor last year, a poorly worded question gets interpreted two different ways by different respondents, a person shades their answer toward what sounds more socially acceptable, or a scale is simply miscalibrated. It's one of the four classic pillars of total survey error, alongside coverage, sampling, and nonresponse error, and unlike sampling error, it doesn't get better just because you asked more people, it gets better by asking better questions, using better instruments, and designing data collection more carefully.
Definition
Measurement error is the difference between an observed response or recorded value and the true value of the construct being measured, arising from sources including respondent recall failure, ambiguous or leading question wording, social-desirability bias (respondents shading answers toward perceived social norms), interviewer effects, mode effects (differences between phone, web, and in-person data collection), and instrument miscalibration. Measurement error is one of the four classic components of total survey error (alongside coverage error, sampling error, and nonresponse error), and unlike sampling error, it is not reduced by increasing sample size, it is instead addressed through questionnaire design and pretesting (cognitive interviewing, split-ballot experiments), interviewer training, and, when a validation source is available, direct comparison against administrative records or repeated measurement; random measurement error in a variable used purely as an outcome inflates its variance without necessarily biasing point estimates, while error correlated with the true value or with other variables in the analysis (as in classical measurement error in a regression predictor, which produces attenuation bias) introduces genuine bias that a validation study or instrumental-variable approach may be needed to correct.
Properties
- Measurement error is addressed at a fundamentally different stage of survey design than sampling error, questionnaire pretesting (cognitive interviewing, split-ballot wording experiments), interviewer training, and instrument validation are the standard tools, none of which involve changing the sample size at all.
- Social-desirability bias, respondents systematically shading answers toward what they believe is socially acceptable, is a particularly well-documented and persistent form of measurement error, especially prevalent on sensitive topics (substance use, income, voting behavior), and is one motivation for indirect elicitation techniques like the randomized response technique or list experiments.
- This general survey-methodology concept of measurement error is distinct from measurement error bias, the specific econometric consequence that arises when a regressor in a regression model is measured with random error, producing attenuation bias in the estimated coefficient; the general concept here covers any construct measured with error in any context, while the econometric term names one particular, formally derived downstream effect of it.
At a Glance