Home Statistical Dictionary Projection Matrix

Projection Matrix

A symmetric, idempotent matrix that maps any vector onto a specific subspace -- the general linear-algebra concept behind the regression hat matrix, which is the specific projection matrix mapping observed responses onto a model's fitted values.

Linear Algebra for StatisticsAdvancedFree

In Plain English

A projection matrix takes any vector and drops it, at a right angle, onto some flat subspace, the matrix version of casting a shadow straight down onto a wall or floor. Ordinary least squares regression is, geometrically, exactly this operation: the hat matrix projects the vector of observed outcomes onto the subspace of all possible fitted values a linear model with these predictors could produce, and the residuals are simply what's left over, the part of the original vector orthogonal to that subspace.

Definition

A projection matrix $P$ is a matrix satisfying $P^{2} = P$ (idempotent) that maps any vector $x$ onto a subspace $S$ , $Px \in S$ for every $x$ . When $P$ is additionally symmetric ( $P = P^{⊤}$ ), it is an orthogonal projection matrix, meaning the residual $x - Px$ is orthogonal to every vector in $S$ , the shortest-distance ('perpendicular drop') projection onto the subspace. In ordinary least squares regression, the hat matrix $H = X (X^{⊤} X)^{- 1} X^{⊤}$ is exactly such an orthogonal projection matrix, projecting the observed response vector $y$ onto the column space of the design matrix $X$ (the subspace of all vectors the model could possibly fit), producing the fitted values $\hat{y} = Hy$ , while $I - H$ is the complementary projection producing the residuals.

Formula

P^{2} = P, P = P^{⊤} (orthogonal case)

Notation

P

—a projection matrix

S

—the subspace P projects onto

Properties

The regression hat matrix is the single most important applied instance of a projection matrix in statistics -- it is precisely the orthogonal projection matrix onto the column space of the design matrix, which is the deeper geometric reason OLS fitted values are the closest possible point, in ordinary Euclidean distance, to the observed responses among every vector the linear model could produce.
A projection matrix's eigenvalues are always exactly 0 or 1 (inherited from idempotency), and its rank equals the dimension of the subspace it projects onto -- for the regression hat matrix, this dimension is the number of estimated model parameters, which is why trace(H) always equals exactly that count.
If $P$ projects orthogonally onto a subspace $S$ , then $I - P$ projects orthogonally onto $S$ 's orthogonal complement -- in regression this decomposition is exactly why total sum of squares splits cleanly into fitted (explained) sum of squares plus residual sum of squares, two orthogonal projections that together reconstruct the original response vector.

At a Glance

Defining propertyidempotent (P^2 = P); orthogonal projection also requires P = P'

General linear-algebra concept behindthe regression hat matrix H

Complementary projectionI - P projects onto the orthogonal complement subspace

Geometric meaningthe shortest-distance ('perpendicular') drop onto a subspace

Last updated September 8, 2026← Back to the Dictionary