Home Statistical Dictionary Vector Norm

Vector Norm

A single number measuring a vector's length or magnitude, generalizing the ordinary Euclidean distance formula to any number of dimensions -- the basis of distance-based statistical procedures from k-nearest-neighbors to ridge-regression penalties.

Linear Algebra for StatisticsIntermediateFree

In Plain English

A vector norm answers a simple question: how big is this vector? For a vector in the ordinary plane, that's just the Pythagorean-theorem distance from the origin, and the vector norm extends that same idea to vectors of any dimension. Different norms measure 'size' in different ways, the most familiar (Euclidean) norm penalizes large entries more heavily than the sum-of-absolute-values norm does, which is exactly the mathematical distinction between ridge regression's and lasso's penalty terms.

Definition

The $p$ -norm of a vector $x = (x_{1}, \dots, x_{n})$ is defined as $∥ x ∥_{p} = (\sum_{i = 1}^{n} ∣ x_{i} ∣^{p})^{1/ p}$ for $p \geq 1$ , with the Euclidean norm ( $p = 2$ ), $∥ x ∥_{2} = \sum_{i} x_{i}^{2}$ , by far the most common choice in statistics, and the $L_{1}$ norm ( $p = 1$ ), $∥ x ∥_{1} = \sum_{i} ∣ x_{i} ∣$ , the second most common. Every vector norm satisfies three defining properties -- non-negativity ( $∥ x ∥ \geq 0$ , with equality only for the zero vector), positive homogeneity ( $∥ c x ∥ = ∣ c ∣∥ x ∥$ ), and the triangle inequality ( $∥ x + y ∥ \leq ∥ x ∥ + ∥ y ∥$ ) -- and the Euclidean norm in particular underlies the residual sum of squares in ordinary least squares, $RSS = ∥ y - X \hat{β} ∥_{2}^{2}$ .

Formula

∥ x ∥_{p} = (i = 1 \sum n ∣ x_{i} ∣^{p})^{1/ p}

Notation

∥ x ∥_{p}

—the p-norm of vector x

p

—the norm's order (p=2 Euclidean, p=1 L1/Manhattan)

Worked Example

For $x = (3, 4)$ , the Euclidean norm is $∥ x ∥_{2} = 3^{2} + 4^{2} = 25 = 5$ (the familiar 3-4-5 right triangle), while the L1 norm is $∥ x ∥_{1} = ∣3∣ + ∣4∣ = 7$ -- two different, both valid, measures of the same vector's size.

Properties

The Euclidean norm squared, $∥ x ∥_{2}^{2} = \sum_{i} x_{i}^{2}$ , is exactly the sum of squares that ordinary least squares minimizes when fitting a regression model -- OLS can be described precisely as choosing coefficients that minimize the Euclidean norm of the residual vector.
Ridge regression penalizes the squared L2 norm of the coefficient vector, while lasso penalizes the L1 norm -- the same underlying vector-norm concept, applied with a different value of p, produces two regularization methods with meaningfully different behavior (ridge shrinks coefficients smoothly toward zero, lasso can shrink them to exactly zero).
A norm is always non-negative and equals zero only for the zero vector -- this positive-definiteness property is what makes a norm a legitimate measure of 'distance from the origin,' and underlies its use in convergence proofs, optimization stopping criteria, and distance-based clustering and classification methods.

At a Glance

Euclidean (p=2)sqrt of sum of squared entries -- ordinary straight-line distance

L1 (p=1)sum of absolute values -- underlies the lasso penalty

Must satisfynon-negativity, homogeneity, the triangle inequality

DrivesOLS's squared-error loss, ridge/lasso penalty terms, k-NN distances

Last updated September 8, 2026← Back to the Dictionary