Positive Definite Matrix

A symmetric matrix $\mathbf{A}$ is positive definite if the quadratic form ${\mathbf{x}}^{\top }\mathbf{Ax}>0$ for every nonzero vector $\mathbf{x}$. Equivalently, a symmetric matrix is positive definite exactly when all of its eigenvalues are strictly positive, a condition that guarantees the matrix is invertible and that its Cholesky decomposition exists. A theoretical covariance matrix is always positive semi-definite, and is positive definite (strictly) whenever no linear combination of the underlying variables is deterministic (a constant), which is the condition needed for the multivariate normal density formula, which requires dividing by the determinant and inverting the covariance matrix, to be well defined.

• Positive definiteness is equivalent to every eigenvalue of the (symmetric) matrix being strictly positive -- this is usually the most practical way to check the condition numerically, rather than testing the quadratic form against every possible vector directly.
• A theoretical covariance matrix is guaranteed positive semi-definite but fails to be strictly positive definite exactly when some linear combination of the variables is perfectly determined by the others (a form of exact multicollinearity), which is also precisely the condition under which the covariance matrix becomes singular and non-invertible.
• Every positive definite matrix admits a Cholesky decomposition, $\mathbf{A}=\mathbf{L}{\mathbf{L}}^{\top }$, which is why the Cholesky decomposition is the standard numerical test for positive definiteness in practice, if the decomposition succeeds without encountering a negative number under a square root, the matrix is confirmed positive definite.