Nonparametric Methods

Nonparametric methods are a class of statistical techniques that use minimal assumptions for both testing and estimation. Here, standard statistical assumptions are often replaced with computationally intensive calculations. Nonparametric methods provide a valuable alternative to classical parametric techniques. They are often called weak assumption statistics because the assumptions required for validity are quite general compared with classical parametric techniques. In many cases, even the weak assumptions made can be further relaxed. As such, conclusions based on nonparametric techniques need not be tempered by qualifying statements about the underlying assumptions. Other advantages are that they (a) are often easy to understand and apply, (b) are especially good for small samples, (c) are frequently appropriate for discrete data, and (d) may be robust with incomplete or imprecise data. Nonparametric techniques date back to 1710 when John Arbuthnott introduced the sign test. More widespread development of nonparametric techniques did not occur until the 1940s when Frank Wilcoxon developed rank-based tests. Nonparametric statistical techniques come in many different forms. Below, three different types of nonparametric techniques are highlighted. The first is the bootstrap, a method for replacing distributional assumptions in statistical tests. The second is an example of a classical nonparametric test. The final technique is that of nonparametric regression.

Bootstrap

The bootstrap relies on resampling. While it has many applications, it is often used when errors in a regression model are nonnormal. Assume that one estimates the following simple regression model:

In this model, we assume that the errors, ε, follow a normal distribution. We can relax this assumption using the bootstrap. The bootstrap simulates the sampling distribution for b through sampling from the original data with replacement. In short, we treat the sample as the population and then sample from it. For example, assume that the sample size for this regression model is 100 cases. For the bootstrap, we would take a random sample of size 100 with replacement from the original data. Since we sample with replacement, some data points will appear in this sample more than once. We then reestimate the original model using this new sample and save the new estimate of β. We repeat this process a large number of times. Typically, we would resample and estimate the model from 1,000 to 5,000 times. This results in 1,000 to 5,000 estimates of β. This new set of β estimates serves as an empirical sampling distribution for this parameter. Percentiles of this distribution can serve as confidence intervals for the parameter. With minor adjustments, one can also calculate a p value to test hypotheses about β. The bootstrap is nonparametric since we use an empirical estimate for the sampling distribution instead of assuming that the test statistic is from a t distribution. One strength of the bootstrap stems from the fact that this basic algorithm can be used to provide inferences for a wide class of statistics.