Null Space

The null space (or kernel) of an $m\times n$ matrix $\mathbf{A}$ is the set of all vectors $\mathbf{x}\in {\mathbb{R}}^n$ satisfying $\mathbf{Ax}=0$, denoted $\text{Null}(\mathbf{A})$, itself always a vector space (containing at minimum the zero vector). By the rank-nullity theorem, $\text{rank}(\mathbf{A})+\dim (\text{Null}(\mathbf{A}))=n$, the number of columns, so a matrix's columns are linearly independent exactly when the null space contains only the zero vector (the trivial case, dimension 0). In regression, a design matrix $\mathbf{X}$ with a nontrivial null space means multiple, genuinely different coefficient vectors $\mathbf{\beta }$ produce identical fitted values $\mathbf{X}\mathbf{\beta }$, since adding any nonzero vector from the null space to $\mathbf{\beta }$ leaves $\mathbf{X}\mathbf{\beta }$ unchanged, precisely the algebraic description of an over-parameterized (non-identifiable) model.

• The rank-nullity theorem links the null space's dimension directly to the matrix's rank -- a matrix with full column rank has a null space containing only the zero vector, while a rank-deficient matrix necessarily has a null space of positive dimension, whose size exactly measures how much redundancy is present among the columns.
• An over-parameterized ANOVA or regression design matrix, e.g. one using a full set of dummy variables for every level of a categorical factor plus an intercept, has a nontrivial null space, which is the formal reason its individual coefficients are not uniquely identifiable even though the fitted values themselves remain perfectly well defined.
• The null space and the row space of a matrix are orthogonal complements of each other within ${\mathbb{R}}^n$, every vector in the null space is orthogonal to every vector in the row space -- this relationship is part of the broader Fundamental Theorem of Linear Algebra connecting all four fundamental subspaces (column space, row space, null space, and the left null space).