Linear Regression, Part 2: Ordinary Least Squares and the Gauss-Markov Theorem

The second installment of the Linear Regression series, turning Part 1's model equation into an actual estimator and the full inference machinery built on top of it. Derives the OLS normal equations and closed-form solution, then catalogues the classical assumptions precisely by what each one buys -- unbiasedness from exogeneity alone, the Gauss-Markov theorem's best-linear-unbiased-estimator guarantee from adding homoskedasticity and no autocorrelation, and exact finite-sample inference from normality on top of that -- explicitly mapping each assumption to the later article that covers its failure. Proves Gauss-Markov in full (the classic C=(X'X)^{-1}X'+D argument) and then, because a proof on paper is not the same as watching it happen, confirms it by Monte Carlo simulation: a genuinely unbiased alternative estimator built from only two of fifty real data points shows almost exactly the 4.68x variance penalty the theorem predicts. Builds the standard error, t-test, F-test, and R-squared machinery, then gives the confidence-interval-versus-prediction-interval distinction its own full treatment with real numbers -- a 9-foot-wide confidence band next to a 62-foot-wide prediction band from the identical fitted line, on the same cars dataset introduced in Part 1.