Lasso Regression

The second installment in the Regularization series, giving the L1 penalty the same full treatment Ridge received: the constrained-optimization and penalized-form derivations, why a diamond-shaped constraint region produces exact zeros (an ellipse tangent to a diamond's vertex, verified geometrically) where Ridge's ball never does, and the soft-thresholding operator derived in closed form for the orthogonal-design special case and cross-checked numerically against scikit-learn. Covers how the lasso is actually solved in general -- LARS's exact piecewise-linear path (Efron, Hastie, Johnstone and Tibshirani, 2004) and coordinate descent (Friedman, Hastie and Tibshirani's glmnet, 2010) -- plus the Zou-Hastie-Tibshirani (2007) degrees-of-freedom result and the lambda.min vs. lambda.1se cross-validation convention. Two real worked examples: the diabetes dataset from the original LARS paper, where cross-validated sparsity recovers a smaller, clinically sensible model at negligible cost; and the same standardized Longley dataset used in the Ridge article, where the lasso path shows a dramatic, dataset-driven swap between two 0.995-correlated predictors (GNP and Year) as lambda changes, and five-fold vs. leave-one-out cross-validation disagree about whether GNP belongs in the model at all -- real motivation for the Elastic Net, the series' final article.