Pretest Sensitization

Research

Pretest effects have been of concern for many decades. Glenn H. Bracht and Gene V Glass discussed pretest effects in the context of external validity problems in interpreting research. That is, if a pretest is given in the context of the experiment, but will not normally be employed in practice, what is the potential effect of the pretest on the outcomes, because in effect it has become part of the treatment? Few authors have followed up this line of critical analysis of research in the intervening [Page 1092]years, so this external validity problem is probably understudied and underreported.

Victor L. Willson and Richard R. Putnam conducted a meta-analysis of randomized experiments that directly examined the effects of giving a pretest versus not giving one for various kinds of tests, including cognitive, psychological, and even behavioral/physical measurements. They discussed the classic Solomon four-group design as the original method intended to evaluate the effects of a pretest on posttest performance. In this design, four groups are randomized into treatment or control, and within each subgroup, a pretest was given and withheld. Analysis of variance provides results for the effect of the pretest as well as of pretest-treatment interaction. The latter is particularly of interest if it is significant. Depending on the direction of the effect, its presence will indicate the nature of the pretest effect, either enhancing or suppressing the treatment effect in comparison with control conditions. Willson and Putnam reported that pretest effects on cognitive and psychological functioning were positive, enhancing both on posttest scores in comparison to non-pretested groups. Cognitive measurements evoked greater effects than personality or psychological measurements or reports of behavior, but all can substantially alter the interpretation of experimental results if not considered. Their conclusion was that such effects typically peak at about 2 weeks, and after a month or so diminish to a fraction of a standard deviation, perhaps 0.100.20 over longer periods. These values are consistent with the reports of the Educational Testing Service for its various tests for test-takers who retake a test after a year. The mechanism for this was not discussed, and in the intervening 25 years, little research has focused on the actual cognitive processes that increase performance. Indirect evidence, however, from well-established research on memory, activation theory, or other cognitive theory such as dual coding can provide rationales for such effects.

...