Inference

Design-Based and Model-Based Inferences

There are two approaches to making inferences from survey data. First, in the design-based approach, inferences are made by looking at how statistics vary as samples are repeatedly drawn using the same sampling procedures as were employed in the actual sampling.

Second, in the model-based approach, inferences are made by looking at how statistics vary as the population, as described by a probability model, is allowed to vary without changing the sample. The model-based approach is also called the prediction approach because the model is used to predict the population units not in the sample. It is called the superpopulation approach as well because the population can be regarded as selected from a still larger population according to the probability model.

Inference procedures (e.g. hypothesis testing or estimating confidence intervals) can be carried out under either the design-based or the model-based approach. The design-based approach is more traditional in survey sampling. The model-based approach, on the other hand, is more consistent with statistical approaches used outside of survey sampling.

Confidence Intervals

Confidence intervals allow one to infer with a high degree of confidence that a quantity being estimated lies within an interval computed by a specified procedure. [Page 334]The precise meaning of “confidence” depends on whether one is adopting the design-based or model-based approach. Clearly a confidence interval is more informative than a numerical estimate of a population quantity (called a point estimate) in that the confidence interval conveys information about how precise the point estimate is.

Hypothesis Testing

The purpose of hypothesis testing is to ascertain whether an observed difference in the sample is statistically significant or whether it can instead be adequately explained by chance alone. Hypothesis tests are designed so that, if there is in fact no difference, the probability of (erroneously) rejecting the hypothesis that there is no difference (i.e. the null hypothesis) is kept to a specified low level; often this probability, called the Type I error, is set to .05. A well-designed hypothesis test will also minimize the other potential error on inference, namely, not rejecting the hypothesis of no difference when a difference actually exists (i.e. Type II error). In survey sampling, it is often the case that two sample averages are independent and approximately normally distributed so the hypothesis that their difference is zero can be tested using properties of the normal distribution (this is called a t-tesi).

...