Linear Regression, Part 1: Origins and What "Linear" Actually Means
The first installment of a complete reference on linear regression -- where the model actually came from (it is not one story but two), and why a model can bend like a curve on the page and still be, in the only sense that matters to the mathematics, linear
The opening article of a seven-part series covering linear regression end to end. Untangles a history that is really two separate inventions eighty years apart: Legendre (1805) and Gauss (1809, claiming 1795) developed least squares for astronomy with no notion yet of regression, while Galton's 1886 study of height introduced the statistical concept and word, formalized mathematically by Pearson (1896) and extended to multiple predictors by Yule (1897, 1899). Lays out the model equation, the design matrix, and the fixed-versus-stochastic-X distinction that quietly determines what 'unbiased' means and previews exactly where Part 4's endogeneity problems come from. The centerpiece is the article's title claim itself: the precise, commonly conflated difference between linearity in the variables and linearity in the parameters, illustrated with two real, numerically verified datasets -- a quadratic and a log-log fit on the classic cars stopping-distance data (both linear in the parameters despite curving visibly), and the Puromycin enzyme-kinetics dataset, where the genuinely nonlinear Michaelis-Menten model is fit two ways to show that the classic Lineweaver-Burk linearizing shortcut is not a free simplification -- it measurably costs accuracy relative to a direct nonlinear fit on the same data. Closes with the transformation toolkit (log-log, log-lin, Box-Cox) that lets real curved relationships stay inside the linear-in-parameters framework the rest of the series builds on.
This article is part of an Analyze subscription.
Subscribe to read Linear Regression, Part 1: Origins and What "Linear" Actually Means in full, along with every other article in the library.
See plans