Home Statistical Dictionary Condition Number (Matrix)
Condition Number (Matrix)
The general linear-algebra ratio of a matrix's largest to smallest singular value, measuring how close any matrix is to being numerically singular -- the underlying concept behind Regression Analysis's condition number, which applies this same ratio specifically to a predictor correlation matrix as a multicollinearity diagnostic.
In Plain English
Every matrix, not just a regression's predictor matrix, has a condition number describing how numerically fragile it is: a small condition number means the matrix behaves predictably under computation, while a huge one means tiny changes to the input can produce wildly different results when the matrix is inverted or used to solve a system of equations. The regression-specific Condition Number entry in this dictionary's Regression Analysis category is exactly this same ratio, applied specifically to a standardized predictor correlation matrix as a multicollinearity check.
Definition
The condition number of a matrix is , the ratio of its largest to smallest singular value (for a symmetric positive definite matrix, this equals the ratio of largest to smallest eigenvalue, under some conventions). A condition number near 1 indicates a well-conditioned matrix, computations involving it (solving linear systems, inverting it) are numerically stable, while a very large condition number indicates the matrix is ill-conditioned, close to singular, and small input errors or floating-point rounding can produce disproportionately large errors in the result. This general concept underlies the Regression Analysis category's Condition Number entry, which applies exactly this ratio to a regression's standardized predictor correlation matrix as an overall multicollinearity diagnostic.
Formula
Notation
Properties
- A condition number is always at least 1, with equality only for a perfectly well-conditioned matrix such as an orthogonal matrix, whose singular values are all equal -- the further above 1 the condition number rises, the more numerically fragile the matrix is under inversion or when used to solve a linear system.
- An ill-conditioned matrix's near-singularity means small errors, whether from measurement noise, rounding in floating-point arithmetic, or minor changes to the underlying data, can produce disproportionately large changes in a computed inverse or solution -- this is the deeper numerical-analysis reason regression coefficients become unstable under severe multicollinearity, not just a statistical inference concern but a genuine computational one.
- This general matrix-theoretic condition number is the same mathematical object as the Regression Analysis category's Condition Number entry, that entry applies this exact ratio to a regression's standardized predictor correlation matrix specifically, using it as a single overall multicollinearity summary, complementary to the per-predictor Variance Inflation Factor.
At a Glance