Newman-Keuls Test

Newman and Keuls proposed a procedure for pairwise testing of means in a one-way analysis of variance. The procedure is routinely applied after a significant overall F test. A series of critical differences (CDs) is used to evaluate the significance of the difference between each pair of means. The pairs are investigated systematically from the largest to the smallest difference. CDs are all based on the Studentized range distribution. The largest CD is applied to the difference between the largest and smallest means and is the same as the CD of Tukey's honestly significant difference procedure.

The second largest CD is applied to two differences: (a) the difference between the largest mean and the next-to-smallest mean and (b) the smallest mean and the next-to-largest mean. If there are k means altogether, there will be k 1 CDs. The second largest CD is identical to the CD for Tukey's honestly significant difference, which would be applied to a set of k 1 means. Testing is continued until all pairs are tested, but with the restriction that no pair can be significantly different if that pair is between two means that are not significantly different. This restriction requires careful ordering of the testing of each pair. However, with equal sample sizes, the process is not too difficult.

Even when applied correctly, the original Newman-Keuls procedure has several problems. First, it can have excessively high Type I error rates. Second, it can have lower power (i.e., higher Type II error rates) that procedures that have good control of Type I errors. Third, when it appears to be more powerful than alternative methods, it may actually be less powerful than an alternative that provides equivalent Type I error control. Fourth, it is not suited for data sets with unequal sample sizes, and some approximation must be considered.

In the case of exactly k = 3 means, Fisher's least significant difference procedure can be more powerful than the original Newman-Keuls, and least significant difference limits the probability of a Type I error rate to the nominal level α of the test. For k ≥ 4, Roy Welsch provided a table of modified values from the Studentized range distribution. Those values produce CDs that limit the probability of one or more Type I errors to the level α of the statistical test. The Welsch CDs remove most of the objections to the original Newman-Keuls procedure.

The Welsch modification of the Newman-Keuls can be slightly modified further to give an additional increase in power. The original Newman-Keuls and the Welsch modified Newman-Keuls use the Studentized range statistic to test the full null hypothesis that all k population means are equal. That is, the overall F test is not required for either of these procedures. However, Juliet Shaffer noted that a step-down procedure such as the Newman-Keuls can be modified to test the full null hypothesis with the overall F test of the analysis of variance. In that case, the largest and smallest means are tested for a significant difference with the CD for k 1 means, provided the overall F test is significant. All other pairs are tested as before. This Shaffer-Welsch version of the Newman-Keuls eliminates all Type I error problems in the equal-sample-size case and is more powerful than most other procedures of equal difficulty.

...