Goodness-of-Fit Tests

Most of the commonly used statistical methods are known to be parametric tests, which impose distributional assumptions on the data. For instance, the t test is popular in comparing the means of two independent samples. This test assumes that the underlying distribution from which each of the samples came is a normal distribution. This assumption is critical, especially in cases where the sample sizes are small. If this distributional assumption is not met, results may be invalid and misleading. Some statistical procedures based on the normal distribution are still approximately valid regardless of the distribution of the data as long as the sample size is large enough. The problem is that it is not clear what “large enough” means. In some cases, a sample size of 30 is large enough, whereas in other cases, a sample size of 30 is not sufficient. On the other hand, one may opt to use non-parametric statistical methods, which do not assume a specific form of the distribution of the samples. These procedures are valid regardless of the sample size. However, it is a well-known fact that nonparametric tests are not as powerful as parametric tests; that is, a nonparametric test requires a larger sample size than its corresponding parametric test to detect a difference, if one truly exists, as long as the distributional assumption of the parametric test is satisfied. Therefore, it is important, especially when sample sizes are small, that the distributional assumption of parametric tests be checked and validated before reporting the results of the statistical analyses are reported. Goodness-of-fit (GOF) tests provide methods to achieve this purpose.

The null and alternative hypotheses of the GOF tests are as follows:

• Null hypothesis (H0): assumed distribution has a good fit
• Alternative hypothesis (Ha): assumed distribution is not a good fit

A GOF test does not try to prove that the underlying distribution is true. Instead, it starts by assuming that the data follow the underlying distribution. It rejects this assumption if there is strong evidence of violation of this assumption, and it does not suggest an alternative distribution to consider. A GOF test does not give any information on how the data deviate from the hypothesized distribution; for this reason, it is highly recommended that GOF tests be accompanied by graphical representation of the data distribution, such as a probability plot, if one exists for the distribution being tested, or a histogram. Moreover, it is possible that GOF tests will not reject a number of distributions, implying that these distributions are a good fit to the data. GOF tests are not designed to choose which among these distributions best fits the data.

Numerous goodness-of-fit tests exist, and they can range from simple to very complex depending on whether the underlying distribution is univariate or multivariate. The most popular and simplest univariate GOF tests are the chi-square goodness-of-fit test and the Kolmogorov-Smirnov test. The chi-square GOF test may be applied whether the underlying distribution is discrete and continuous. The Kolmogorov-Smirnov test applies only when the underlying distribution is continuous. Both of these tests are available in most statistical packages.

...