Orthogonal Comparisons

The use of orthogonal comparisons within an analysis of variance is a common method of multiple comparisons. The general field of multiple comparisons considers how to analyze a multitreatment experiment to answer the specific questions of interest to the experimenter. Orthogonal comparisons are appropriate when the researcher has a clearly defined, independent set of research hypotheses that will constitute the only analyses done on the treatment means. Of the multiple comparison techniques available with comparable control over Type I error (rejecting a true null hypothesis), a set of orthogonal comparisons will provide a more powerful approach to testing differences that may exist between the treatment groups than those that are more exploratory in nature and thus more extensive. The power is gained by restricting the analysis to a small number of preplanned, independent questions rather than exploring for whatever difference may be present in the data.

Basic to an understanding of orthogonal comparisons is the concept of a comparison within a multigroup analysis of variance. Assume a balanced design in which each treatment group has an equal number of subjects. The great majority of comparisons deal with either differences between pairs of means, such as whether Group 1 differs from Group 2, or the differences between combinations of means, such as whether the average of Groups 1 and 2 differs from the average of Groups 3 and 4. This latter type of compound hypothesis, involving combinations of groups, is common in orthogonal comparisons. It is appropriate when groups have common elements that make the combination a meaningful grouping. If a learning experiment involves treatment groups that differ with respect to the type of prompts provided to students, there may be some groups that have in common that the prompts are delivered verbally, although they differ in other respects. Other treatment groups may have the prompts delivered in written form, again differing in other respects. A compound question focuses on the overall difference between the verbally delivered prompts and those where the prompts were delivered in writing.

The comparison is conceptualized as a set of weights, one for each of the k treatment groups, aj = {aj, 1, aj, 2, … aj, k,) whose sum must be zero and whose pattern reflects the question being addressed. For example, the set {1 − 1 0 0} in a four-group experiment is a comparison because Σa = 0. It would be used to compare the first group with the second group, ignoring the third and fourth groups. A different comparison is defined by the weights {1, 1, −1, −1}, which also sum to zero and combine the first two groups and compare that sum with the sum of the last two groups. The important information in a set of weights is the pattern. The set {1/2, 1/2, −1/2, −1/2} has the same pattern and would produce equivalent results to the set {1, 1, −1, −1}.

The null hypothesis for a comparison asserts that the sum of the products of weights multiplied by the population means is zero. Call this sum of products ψ. The null hypothesis is that

...