Bootstrapping

Resampling

First, a statistic's sampling distribution is estimated by treating the sample as the population and conducting a form of Monte Carlo simulation on it. This is done by randomly resampling with replacement a large number of samples of size n from the original sample of size n. Replacement sampling causes the resamples to be similar to, but slightly different from, the original sample, because an individual case in the original sample may appear once, more than once, or not at all in any given resample.

For the resulting estimate of the statistic's sampling distribution to be unbiased, resampling needs to be conducted to mimic the sampling process that generated the original sample. Any stratification, weighting, clustering, stages, and so forth used to draw the original sample need to be used to draw each resample. In this way, the random variation that was introduced into the original sample will be introduced into the resamples in a similar fashion. The ability to make inferences from complex random samples is one of the important advantages of bootstrapping over parametric inference. In addition to mimicking the original sampling procedure, resampling ought to be conducted only on the random component of a statistical model. For example, an analyst would resample the error term of a regression model to make inferences about regression parameters, as needed, unless the data are all drawn from the same source, as in the case of using data from a single survey as both the dependent and independent variables in a model. In such a case, since the independent variables have the same source of randomness—an error as the dependent variable—the proper approach is to resample whole cases of data.

...