Identity Matrix

The $n\times n$ identity matrix ${\mathbf{I}}_n$ has 1's along its main diagonal and 0's in every off-diagonal entry, $I_{ij}=1$ if $i=j$ and $I_{ij}=0$ if $i\ne j$, and satisfies ${\mathbf{I}}_n\mathbf{A}=\mathbf{A}{\mathbf{I}}_n=\mathbf{A}$ for any conformable matrix $\mathbf{A}$, making it the multiplicative identity of matrix algebra. The identity matrix is a special case of a diagonal matrix (all diagonal entries equal to 1) and defines the matrix inverse: ${\mathbf{A}}^{-1}$ is, by definition, the unique matrix satisfying $\mathbf{A}{\mathbf{A}}^{-1}={\mathbf{A}}^{-1}\mathbf{A}={\mathbf{I}}_n$.

• The identity matrix is the matrix-algebra analog of the number 1 under multiplication -- just as multiplying any number by 1 leaves it unchanged, multiplying any conformable matrix or vector by the identity matrix returns that exact same matrix or vector, on either side of the product.
• The identity matrix's defining role in $\mathbf{A}{\mathbf{A}}^{-1}=\mathbf{I}$ makes it the reference point for checking whether a claimed inverse is correct -- multiplying a candidate inverse by the original matrix and confirming the identity matrix results out is the standard verification step, both by hand and numerically.
• Subtracting a projection matrix from the identity, $\mathbf{I}-\mathbf{P}$, produces the complementary projection onto the orthogonal subspace -- in ordinary least squares, $\mathbf{I}-\mathbf{H}$ (where $\mathbf{H}$ is the hat matrix) projects the response vector onto the residual space, giving the residual vector directly as $(\mathbf{I}-\mathbf{H})\mathbf{y}$.