Duncan's Multiple Range Test

Duncan's multiple range test, or Duncan's test, or Duncan's new multiple range test, provides significance levels for the difference between any pair of means, regardless of whether a significant F resulted from an initial analysis of variance. Duncan's test differs from the Newman-Keuls test (which slightly preceded it) in that it does not require an initial significant analysis of variance. It is a more powerful (in the statistical sense) alternative to almost all other post hoc methods.

When introducing the test in a 1955 article in the journal Biometrics, David B. Duncan described the procedures for identifying which pairs of means resulting from a group comparison study with more than two groups are significantly different from each other. Some sample mean values taken from the example presented by Duncan are given. Duncan worked in agronomics, so imagine that the means represent agricultural yields on some metric. The first step in the analysis is to sort the means in order from lowest to highest, as shown.

From tables of values that Duncan developed from the t-test formula, standard critical differentials at the .05 level are identified. These are significant studentized differences, which must be met or surpassed. To maintain the nominal significance level one has chosen, these differentials get slightly higher as the two means that are compared become further apart in terms of their rank ordering. In the example shown, the means for groups A and F have an interval of 2 because they are adjacent to each other. Means A and E have an interval of 7 as there are seven means in the span between them. By multiplying the critical differentials by the standard error of the mean, one can compute the shortest significant ranges for each interval width (in the example, the possible intervals are 2, 3, 4, 5, 6, and 7). With the standard error of the mean of 3.643 (which is supplied by Duncan for this example), the shortest significant ranges are calculated.

For any two means to be significantly different, their distance must be equal to or greater than the associated shortest significant range. For example, the distance between mean F (58.1) and mean B (71.2) is 13.1. Within the rank ordering of the means, the two means form an interval of width 5, with an associated shortest significant range of 11.66. Because 13.1 > 11.66, the two means are significantly different at the .05 level.

Duncan suggested a graphical method of displaying all possible mean comparisons and whether they are significant compared with one another. This method involved underlining those clusters of means that are not statistically different. Following his suggestion, the results for this sample are shown below.

The philosophical approach taken by Duncan is an unusually liberal one. It allows for multiple pairwise comparisons without concern for inflation of the Type I error rate. A researcher may perform dozens of post hoc analyses in the absence of specific hypotheses and treat all tests as if they are conducted at the .05 (or whatever the nominal value chosen) level of significance. The comparisons may be analyzed even in the absence of an overall F test indicating that any differences exist. Not surprisingly, Duncan's multiple range test is not recommended by many statisticians who prefer more conservative approaches that minimize the Type I error rate. Duncan's response to those concerns was to argue that because the null hypothesis is almost always known to be false to begin with, it is more reasonable to be concerned about making Type II errors, missing true population differences, and his method certainly minimizes the true Type II error rate.

...