Rank (Matrix)

The rank of a matrix $\mathbf{A}$ is the maximum number of linearly independent rows (equivalently, the maximum number of linearly independent columns -- row rank always equals column rank), equal to the dimension of the column space (and row space) of $\mathbf{A}$. A matrix is said to have full rank if its rank equals the smaller of its row count and column count; for a square $n\times n$ matrix, full rank ($\text{rank}=n$) is equivalent to a nonzero determinant and to invertibility. In regression, a design matrix $\mathbf{X}$ that is not full column rank (some predictor is an exact linear combination of others, perfect multicollinearity) causes ${\mathbf{X}}^{\top }\mathbf{X}$ to be singular, so the ordinary least squares normal equations have no unique solution, forcing the use of a generalized inverse or the removal/combination of redundant predictors.

• Row rank always equals column rank for any matrix, a nontrivial but foundational fact of linear algebra -- this is why 'rank' can be stated as a single number for a matrix rather than needing separate row-rank and column-rank figures.
• A design matrix that is not full column rank, because one predictor column is an exact linear combination of others, produces a singular ${\mathbf{X}}^{\top }\mathbf{X}$ matrix, meaning ordinary least squares cannot identify a unique coefficient vector -- this is the formal linear-algebra description of the practical problem statisticians call perfect (as opposed to merely high) multicollinearity.
• The rank of a matrix equals the number of its nonzero singular values (from the singular value decomposition) and also the number of nonzero eigenvalues of a symmetric matrix -- these equivalent characterizations are why numerical software typically determines rank by counting singular values above a small numerical tolerance rather than by direct inspection of linear dependence.