Post Hoc Comparisons

Notations

The experimental design is a one-factor ANOVA. The total number of observations is denoted N, with S denoting the number of observations per group. The number of groups is denoted A, a given group is labeled with the letter a, the group means are denoted Ma+, and the grand mean is denoted M++. A contrast is denoted ψa, the contrast coefficients (contrast weights) are denoted Ca. The α-level per comparison is denoted α[PC] and the α-level per family of comparisons is denoted α[PF].

Planned versus Post Hoc Comparisons

Planned (or a priori) comparisons are selected before running the experiment. In general, they correspond to the research hypotheses. Because these comparisons are planned, they are usually few in number. In order to avoid an inflation of the Type I error (i.e., declaring an effect significant when it is not), the pvalues of these comparisons are corrected with the standard Bonferroni or Šidàk approaches.

In contrast, post hoc (or a posteriori) comparisons are decided after the experiment has been run and analyzed. The aim of post hoc comparisons is to make sure that (unexpected) patterns seen in the results are reliable. This implies that the actual family of comparisons consists of all possible comparisons, even if they are not explicitly tested.

Contrast

A contrast is a prediction precise enough to be translated into a set of numbers called contrast coefficients or contrast weights (denoted Ca) that express this prediction. For convenience, contrast coefficients have a mean equal to zero. The correlation between the contrast coefficients and the values of the group means quantifies the similarity between the prediction and the results.

Figure 1(a) Appropriate Context (b) Partial Context for Bransford and Johnson's (1972) Experiment

Source: Reprinted from Journal of Verbal Learning and Verbal Behavior, 11, Bransford, J.D. Contextual prerequisites for understanding: Some investigations of comprehension and recall,” 717726, Copyright 1972, with permission from Elsevier.

All contrasts are evaluated using the same general procedure. A contrast is formalized as a set of contrast coefficients that represents the predicted pattern of experimental results. For example, if we have four groups and we predict that the first group should have better performance than the other three groups, and that these remaining three groups are equivalent, we get the following

...