Linear Regression, Part 4: Diagnosing Linear Regression -- When Your Estimates Are Wrong
Three problems that no amount of data fixes -- with a real dataset where dropping one variable shifts a coefficient by exactly a provable amount, a simulation where a coefficient converges to the wrong number at one million observations, and a market where OLS cannot tell supply from demand
The fourth installment of the Linear Regression series, turning to the second and more serious failure bucket the series is organized around: problems that violate exogeneity itself (A3), so that OLS's coefficient estimates no longer converge to the truth even as the sample grows without bound -- a categorically worse failure than Part 3's, where more data at least helped once the right correction was applied. Covers omitted variable bias with a real, exactly-verifiable example (dropping horsepower from a weight-on-mpg regression shifts the weight coefficient from -3.878 to -5.344, matching the Frisch-Waugh-Lovell identity to thirteen decimal places), classical measurement-error attenuation (injecting realistic noise into a real predictor shrinks its coefficient toward zero and keeps it there through n=320,000), and simultaneity bias (a simulated supply-and-demand market where a naive regression of quantity on price converges cleanly, even at a million observations, to a number that is neither the true demand slope nor the true supply slope). Closes with an explicit, honest statement that neither Part 3's diagnostics nor Part 5's upcoming remedies fix any of these -- that requires instrumental variables and two-stage least squares, which Part 7 develops properly.
This article is part of an Analyze subscription.
Subscribe to read Linear Regression, Part 4: Diagnosing Linear Regression -- When Your Estimates Are Wrong in full, along with every other article in the library.
See plans