Sampling Variance

Sampling variance is the variance of the sampling distribution for a random variable. It measures the spread or variability of the sample estimate about its expected value in hypothetical repetitions of the sample. Sampling variance is one of the two components of sampling error associated with any sample survey that does not cover the entire population of interest. The other component of sampling error is coverage bias due to systematic nonobservation. The totality of sampling errors in all possible samples of the same size generates the sampling distribution for a given variable. Sampling variance arises because only a sample rather than the entire population is observed. The particular sample selected is one of a large number of possible samples of the same size that could have been selected using the same sample design. To the extent that different samples lead to different estimates for the population statistic of interest, the sample estimates derived from the different samples will differ from each other.

The positive square root of the sampling variance is called the standard error. For example, the square root of the variance of the sample mean is known as the standard error of the mean. The sample estimate and its standard error can be used to make inferences about the underlying population, for example, through constructing confidence intervals and conducting hypothesis testing. It is important to note, however, that sampling variance is measured about the expected value of the statistic under the sample design rather than the true population value. Therefore, inferences based on sampling variance do not reflect sampling biases or any possible nonsampling errors.

Under probability sampling, the sampling variance can be estimated using data collected from the sample. The estimation methodology for the sampling variance should take into account both the sample design and the estimation method. For standard sampling designs and estimators, standard variance estimation formulae are available. In his book Introduction to Variance Estimation, Kirk M. Wolter discussed nine basic sampling designs and their associated variance estimators.

In many samples, however, data are collected from individuals or organizations using complex sample designs that typically involve unequal selection probabilities, sample stratification, clustering, and multistage sampling. For such complex sample designs, although it is possible to produce unbiased point estimates by using proper sample weights, it is generally not possible to estimate appropriate sampling variances using standard estimation methods. In fact, for many complex sample designs and estimators, exact algebraic expressions of the sampling variances are not available, and hence there are no direct analytic methods for producing unbiased variance estimates.

One general approach to approximating the sampling variance of an estimator is to use sample replication methods such as jackknife and balanced repeated replication. An alternative approach is to approximate the estimator analytically using Taylor series expansion and then compute the variance of the linearized estimator. Statistical software packages that specialize in complex variance estimation include SUDAAN, WesVar, and STATA, among others.