Degrees of Freedom

In statistics, the degrees of freedom is a measure of the level of precision required to estimate a parameter (i.e., a quantity representing some aspect of the population). It expresses the number of independent factors on which the parameter estimation is based and is often a function of sample size. In general, the number of degrees of freedom increases with increasing sample size and with decreasing number of estimated parameters. The quantity is commonly abbreviated df or denoted by the lowercase Greek letter nu, v.

For a set of observations, the degrees of freedom is the minimum number of independent values required to resolve the entire data set. It is equal to the number of independent observations being used to determine the estimate (n) minus the number of parameters being estimated in the approximation of the parameter itself, as determined by the statistical procedure under [Page 342]consideration. In other words, a mathematical restraint is used to compensate for estimating one parameter from other estimated parameters. For a single sample, one parameter is estimated. Often the population mean (μ), a frequently unknown value, is based on the sample mean (

), thereby resulting in n − 1 degrees of freedom for estimating population variability. For two samples, two parameters are estimated from two independent samples (n1 and n2), thus producing n1 + n2 − 2 degrees of freedom. In simple linear regression, the relationship between two variables, x and y, is described by the equation y = bx + a, where b is the slope of the line and a is the y-intercept (i.e., where the line crosses the y-axis). In estimating a and b to determine the relationship between the independent variable x and dependent variable y, 2 degrees of freedom are then lost. For multiple sample groups (n1 + …+nk), the number of parameters estimated increases by k, and subsequently, the degrees of freedom is equal to n1 + … + nk – k. The denominator in the analysis of variance (ANOVA) F test statistic, for example, accounts for estimating multiple population means for each group under comparison.

The concept of degrees of freedom is fundamental to understanding the estimation of population parameters (e.g., mean) based on information obtained from a sample. The amount of information used to make a population estimate can vary considerably as a function of sample size. For instance, the standard deviation (a measure of variability) of a population estimated on a sample size of 100 is based on 10 times more information than is a sample size of 10. The use of large amounts of independent information (i.e., a large sample size) to make an estimate of the population usually means that the likelihood that the sample estimates are truly representative of the entire population is greater. This is the meaning behind the number of degrees of freedom. The larger the degrees of freedom, the greater the confidence the researcher can have that the statistics gained from the sample accurately describe the population.

To demonstrate this concept, consider a sample data set of the following observations (n = 5): 1, 2, 3, 4, and 5. The sample mean (the sum of the observations divided by the number of observations) equals 3, and the deviations about the mean are − 2, −1, 0, +1, and +2, respectively. Since the sum of the deviations about the mean is equal to zero, at least four deviations are needed to determine the fifth; hence, one deviation is fixed and cannot vary. The number of values that are free to vary is the degrees of freedom. In this example, the number of degrees of freedom is equal to 4; this is based on five data observations (n) minus one estimated parameter (i.e., using the sample mean to estimate the population mean). Generally stated, the degrees of freedom for a single sample are equal to n − 1 given that if n − 1 observations and the sample mean are known, the remaining ftth observation can be determined.

...