Degrees of Freedom

Degrees of freedom is the number of independent pieces of information available in a sample to estimate a statistical quantity, calculated as the total number of observations minus the number of parameters or constraints already estimated from that same data, the classic illustration is the sample variance, calculated as $\sum (x_i-\bar{x}{)}^2/(n-1)$, using $n-1$ rather than $n$ in the denominator, because one degree of freedom has already been 'used up' estimating the sample mean $\bar{x}$ from the same data, once the mean and $n-1$ of the observations are known, the final observation is mathematically determined, not free to vary independently. In analysis of variance specifically, degrees of freedom are calculated separately for each source of variation being partitioned, the between-groups (treatment) degrees of freedom equal the number of groups minus 1, reflecting that only that many group means can vary independently once the grand mean is fixed, and the within-groups (error) degrees of freedom equal the total sample size minus the number of groups, reflecting the information remaining after estimating each group's own mean, these degrees of freedom values are not a minor bookkeeping detail, they directly determine which specific F-distribution (or t-distribution, or chi-square distribution, depending on the test) is the correct reference distribution for computing a p-value, using the wrong degrees of freedom produces a technically incorrect statistical test even if every other calculation in the analysis is done correctly.

• Degrees of freedom is not an arbitrary correction factor, it reflects a genuine mathematical constraint, once a statistic like the sample mean has been computed from the data, not every observation remains free to vary independently, the final observation is mathematically determined by the others and the already-computed statistic.
• In analysis of variance, degrees of freedom are computed separately and additively for each source of variation in the partition, the between-groups and within-groups degrees of freedom sum to the total degrees of freedom, exactly mirroring how the corresponding sums of squares also partition additively.
• Using the correct degrees of freedom is not a cosmetic detail, it directly determines which reference distribution, and therefore which specific critical value or p-value, is correct for a given hypothesis test, an otherwise perfectly executed calculation using the wrong degrees of freedom produces an invalid statistical conclusion.