Solomon Four-Group Design

History of the Design

Richard Solomon first proposed a three-group design that consisted of an Experimental Group that was exposed to the pretest and the treatment; Control Group I that was exposed to the pretest, but not the treatment; and Control Group II that was not exposed to the pretest but received the treatment. The three-group design served the same purpose as the four-group design, but the two designs used slightly different statistical analyses. While Solomon believed that the three-group design was sufficient for laboratory experiments, a four-group design would be better suited for field studies, including those that use sample surveys to gather data. Therefore, he added Control Group III, in which units were not exposed to either the pretest or the treatment. He originally saw the fourth group as a way to estimate effects of threats to validity other than testing, such as maturation or history, but this four-group design has become the preferred method for examining testing effects.

Analysis and Interpretation

Using this design, pretest sensitization is determined by conducting a two-factor analysis of variance (ANOVA), in which treatment and presence of pretest are the factors and posttest scores are the dependent variable. It is not necessary to include the pretest scores in the analysis because random assignment ensures that pretest scores should be the same for each group on expectation. A significant interaction between treatment and pretest suggests that exposure to the pretest may influence the treatment effect. If there is no interaction, a significant main effect for treatment indicates that the treatment had an effect, and a significant main effect for the pretest indicates that pretest may influence posttest.

Improving Validity

The Solomon four-group design provides information relevant to both internal and external validity. Regarding internal validity, for example, many single-group quasi-experiments use both pretests and posttests. If a pretest sensitization occurs, it could be mistaken for a treatment effect, so that scores could change from pretest to posttest even if the treatment had no effect at all. In randomized experiments, pretest sensitization [Page 830]effects do not affect internal validity because they are randomized over conditions, but those effects affect external validity if the size of the effect depends on whether a pretest is administered.

...