Generalized Linear Model

A generalized linear model (Nelder and Wedderburn, 1972) specifies a response variable $Y_i$ drawn from a distribution in the exponential family with mean ${\mu }_i=E[Y_i]$, related to a linear predictor ${\eta }_i={\mathbf{x}}_i^{\top }\mathbf{\beta }$ through a link function $g$ such that $g({\mu }_i)={\eta }_i$. The framework unifies ordinary linear regression (identity link, Gaussian response), logistic regression (logit link, Bernoulli/binomial response), and Poisson regression (log link, Poisson response) as special cases of the same estimation machinery, fit by maximum likelihood (typically via iteratively reweighted least squares) rather than ordinary least squares.

• GLMs require the response distribution to belong to the exponential family (Gaussian, binomial, Poisson, gamma, inverse Gaussian, and others), which guarantees the iteratively reweighted least squares fitting algorithm converges to the maximum likelihood estimate under standard regularity conditions.
• Unlike ordinary linear regression, a GLM does not require the response's variance to be constant -- most exponential-family distributions have a variance that is itself a function of the mean, e.g. the Poisson's variance equals its mean, and the model's variance function captures this relationship explicitly.
• The framework was later extended in several directions still active in applied statistics: generalized additive models (GAMs) replace the linear predictor with smooth functions, while generalized estimating equations (GEE) and generalized linear mixed models (GLMMs) extend GLMs to correlated or clustered data.