Precision

Precision in statistical surveys relates to the variation of a survey estimator for a population parameter that is attributable to having sampled a portion of the full population of interest using a specific probability-based sampling design. It refers to the size of deviations from a survey estimate (i.e. a survey statistic, such as a mean or percentage) that occurs over repeated application of the same sampling procedures using the same sampling frame and sample size. While precision is a measure of the variation among survey estimates, over repeated application of the same sampling procedures, accuracy is a measure of the difference between the survey estimate and the true value of a population parameter. Precision and accuracy measure two dimensions for a survey estimate. A sampling design can result in survey estimates with a high level of precision and accuracy (the ideal). In general, sampling designs are developed to achieve an acceptable balance of accuracy and precision.

The sampling variance is a commonly used measure of precision. Precision can also be represented by the standard error (the square root of the sampling variance for a survey estimate), the relative standard error (the standard error scaled by the survey estimate), the [Page 599]confidence interval width for a survey estimate (using the standard error and the values from an assumed probability density function for the survey estimate), and difference between estimated percentiles for a population parameter (for example, the intraquartile range for the survey estimate).

The sampling variance is conceptually the squared difference between the estimate for a specifie sample and the expected value of the estimate summed over all possible samples selected in the same fashion with the same sample size using the same sampling scheme. The sampling variance is different from the classical “population” variance (a measure of the variation among the observations in the population) in the sense that the population variance is a constant, independent of any sampling issues, while the sampling variance becomes smaller as the sample size increases. The sampling variance is zero when the full population is observed, as in a census.

Because only a single sample is selected and the expected value of an estimate is unknown, sampling theory has provided formulae to estimate the sampling variance from a single sample. To allow for the computation of a sampling variance, the sample scheme must be reproducible, that is, it can be completely replicated. Moreover, to compute an unbiased estimate of the sampling variance, (a) every unit in a sampling frame needs to have a positive chance of being selected (the unit selection probability is greater than zero), and (b) every pair of units must have a positive chance of being in a sample (the joint selection probability for any pair of units is greater than zero).

The sampling variance is a function of the form of the statistic, the underlying population, and the nature of the sampling design, and it is called the “design-based sampling variance.” The design-based variance generally assumes the use of a sampling weight, which is computed from the inverse of the probability of selection of the sample.

...