Post Hoc Comparisons

Post hoc comparisons among sample means from three or more groups are generally performed only after obtaining a significant omnibus F when we use an ANOVA. After we find the various means are not all equal, the second step is using post hoc comparisons to find out exactly which means are significantly different from which other ones. In contrast to a priori comparisons, which are chosen before the data are collected, post hoc comparisons are tested after the researcher has collected the data.

Post hoc comparisons include pairwise comparisons and nonpairwise comparisons. Pairwise comparisons compare two sample means at a time, whereas nonpairwise comparisons compare more than two sample means at a time.

The main post hoc comparison procedures include Scheffé procedure and Tukey HSD (honestly significant difference) procedure. The Scheffé procedure allows for a comparison of all possible paired comparisons and complex comparisons between combined means.

The formula for the Scheffé test is as follows:

where

Σ, the Greek letter sigma, is the summation sign;

Cj is the coefficient used for any group;

X¯j is the mean of the corresponding group;

MSW is the mean square within from the analysis of variance.

The Tukey HSD procedure only allows for a comparison of the possible pairs of means. The formula for the Tukey HSD test is as follows:

where

X¯1 and X¯2are two sample means that are needed for comparison,

MSW is the mean square within from the analysis of variance,

n1 is the number of scores of Group 1,

n2 is the number of scores of Group 2.

For example, the data set in Table 1 consists of 10 cases with two variables, Group and Test Score.

To produce the post hoc comparison, follow these steps:

1. Compute the sample mean for each group.

2. Check the assumption of equal variance.

To test the null hypothesis that groups come from populations with the same variance, the Levene test is produced. The observed significance level is larger than .05. The null hypothesis is not rejected. Four groups come from populations with the same variance.

3. Produce ANOVA to get the omnibus F test. The null hypothesis of the F test is that all the population means are the same.

According to the results of the F test, F(3,36) = 5.382, p = .004, p < .05, so we reject the null hypothesis, and the four population means are not all equal.

4. Use post hoc comparison to determine which means are significantly different from each other.

The results of the Scheffé procedure show that only Group 2 and Group 4 are significantly different from each other, whereas the results of the Tukey HSD procedure show that Group 4 is significantly different from Group 1, Group 2, and Group 3. The reason is that the Scheffé test is more considerate than the Tukey HSD test in holding Type I error low.

The SPSS output is shown in Tables 2 and 3.