Orthogonal Matrix

A square matrix $\mathbf{Q}$ is orthogonal if its columns (equivalently its rows) form an orthonormal set -- each column has unit length and every pair of distinct columns is orthogonal (dot product zero) -- which is equivalent to the algebraic condition ${\mathbf{Q}}^{\top }\mathbf{Q}=\mathbf{Q}{\mathbf{Q}}^{\top }=\mathbf{I}$, so that ${\mathbf{Q}}^{-1}={\mathbf{Q}}^{\top }$, the inverse is simply the transpose, avoiding any need for the more expensive general matrix-inversion computation. Multiplying a vector by an orthogonal matrix preserves its length exactly, $\Vert \mathbf{Qx}\Vert =\Vert \mathbf{x}\Vert$, and preserves the angle between any two vectors, which is why orthogonal matrices represent pure rotations or reflections and appear as the eigenvector matrix $\mathbf{Q}$ in the spectral decomposition of any symmetric matrix.

• Because an orthogonal matrix's inverse is simply its transpose, no numerically expensive matrix inversion is ever needed to undo an orthogonal transformation -- a computational convenience that makes orthogonal matrices especially attractive building blocks in numerical linear algebra algorithms like QR decomposition.
• Multiplying any vector by an orthogonal matrix preserves its Euclidean length and preserves the angle between any two vectors -- geometrically, an orthogonal matrix can only rotate or reflect, never stretch, shrink, or shear, the space it acts on.
• In a symmetric matrix's spectral decomposition, $\mathbf{A}=\mathbf{Q\Lambda }{\mathbf{Q}}^{\top }$, the eigenvector matrix $\mathbf{Q}$ is always orthogonal -- this is precisely why PCA's principal component directions are mutually perpendicular (uncorrelated) axes, a direct geometric consequence of the underlying orthogonal eigenvector matrix.