Basis (Linear Algebra)

A basis for a vector space $V$ is a set of vectors $\{{\mathbf{v}}_1,\dots ,{\mathbf{v}}_n\}$ that is both linearly independent and spans $V$ (every vector in $V$ can be written as some linear combination of the basis vectors), and this combination is always unique for a given basis. The number of vectors in any basis for $V$ is the same regardless of which basis is chosen, and this common number is called the dimension of $V$; the standard basis for ${\mathbb{R}}^n$ consists of the $n$ unit vectors each with a single 1 and zeros elsewhere, but any set of $n$ linearly independent vectors in ${\mathbb{R}}^n$ forms a valid alternative basis, and choosing a more statistically meaningful basis, such as the eigenvector basis found by PCA, is a common strategy for re-expressing data in coordinates that reveal its structure more clearly than the original variable axes do.

• Every vector in a space has a unique representation as a combination of a given basis's vectors -- uniqueness follows directly from linear independence, if two different combinations produced the same vector, their difference would be a nontrivial combination equal to zero, contradicting independence.
• A vector space has infinitely many possible bases, but every valid basis for the same space contains exactly the same number of vectors, this shared count is the space's dimension, an intrinsic property of the space itself, not of any particular basis chosen to describe it.
• Principal component analysis can be understood precisely as a change of basis, from the original variable axes to a new basis formed by a covariance matrix's eigenvectors, chosen specifically because it re-expresses the data so that as much variance as possible is captured by as few basis directions as possible.