Home Statistical Dictionary QR Decomposition

QR Decomposition

The factorization of a matrix into an orthogonal matrix times an upper triangular matrix -- the numerically stable method statistical software actually uses to solve the ordinary least squares equations, avoiding the extra rounding error of forming X'X directly.

Linear Algebra for StatisticsAdvancedFree

In Plain English

QR decomposition splits a matrix into a perfectly length-and-angle-preserving orthogonal piece and a triangular piece, and this combination turns out to be an especially numerically safe way to fit a regression. Rather than computing X'X and inverting it (a formula that's easy to write down but numerically risky, since squaring a matrix can amplify small rounding errors), most real statistical software actually fits ordinary least squares via QR decomposition instead, arriving at the same coefficients through a more stable computational route.

Definition

The QR decomposition of an $m \times n$ matrix $A$ (with $m \geq n$ , full column rank) expresses it as $A = QR$ , where $Q$ is an $m \times n$ matrix with orthonormal columns and $R$ is an $n \times n$ upper triangular matrix. For a regression design matrix $X = QR$ , the least squares normal equations $X^{⊤} X \hat{β} = X^{⊤} y$ simplify, using $Q^{⊤} Q = I$ , to the triangular system $R \hat{β} = Q^{⊤} y$ , solved directly by back substitution without ever forming $X^{⊤} X$ ; because squaring a matrix (as in forming $X^{⊤} X$ ) roughly squares its condition number and correspondingly amplifies floating-point rounding error, the QR-based approach is substantially more numerically stable, and is the actual algorithm used inside most professional-grade statistical software's regression routines.

Formula

A = QR, Q^{⊤} Q = I

Notation

Q

—matrix with orthonormal columns

R

—upper triangular matrix

Properties

Solving ordinary least squares via QR decomposition avoids ever forming $X^{⊤} X$ , which matters because squaring a matrix roughly squares its condition number, an ill-conditioned (near-multicollinear) design matrix becomes dramatically worse-conditioned once squared, so QR-based fitting is meaningfully more numerically stable exactly in the cases (high multicollinearity) where stability matters most.
The QR decomposition connects directly to the Gram-Schmidt orthogonalization process, one standard way to compute it is by sequentially orthogonalizing the columns of $A$ to build $Q$ , while $R$ records the coefficients used in that orthogonalization -- though numerical software typically uses a more stable variant (Householder reflections) rather than classical Gram-Schmidt directly.
Because $Q$ has orthonormal columns, $Q^{⊤} Q = I$ , so no matrix inversion is needed to move $Q^{⊤}$ across an equation -- this is exactly the property that reduces the least squares normal equations to a directly back-substitutable triangular system once the QR factors are known.

At a Glance

FactorizationA = QR, Q orthonormal columns, R upper triangular

Simplifies OLS tothe triangular system R*beta-hat = Q'y

Numerical advantageavoids forming X'X directly, far more stable

Used bymost professional statistical software's regression routines

Last updated September 8, 2026← Back to the Dictionary