Pretest-Posttest Design

The basic premise behind the pretestposttest design involves obtaining a pretest measure of the outcome of interest prior to administering some treatment, followed by a posttest on the same measure after treatment occurs. Pretest posttest designs are employed in both experimental and quasi-experimental research and can be used with or without control groups. For example, quasi-experimental pretestposttest designs may or may not include control groups, whereas experimental pretestposttest designs must include control groups. Furthermore, despite the versatility of the pretestposttest designs, in general, they still have limitations, including threats to internal validity. Although such threats are of particular concern for quasi-experimental pretestposttest designs, experimental pretestposttest designs also contain threats to internal validity.

Types of PretestPosttest Designs without Control Groups

One-Group PretestPosttest Design

In the simplest pretestposttest design, researchers gather data about some outcome through a single pretest, administer a treatment, and then gather posttest data on the same measure. This design is typically represented as follows:

O1X O2

where O1represents the pretest, X represents some treatment, and O2represents the posttest.

Including a pretest measure is an improvement over the posttest-only research design; however, this design is still relatively weak in terms of internal validity. Although this design allows researchers to examine some outcome of interest prior to some treatment (O1), it does not eliminate the possibility that O2might have occurred regardless of the treatment. For example, threats to internal validity, such as maturation, history, and testing, could be responsible for any observed difference between the pretest and posttest. Also, the longer the time lapse between the pretest and posttest, the [Page 1087]harder it is to rule out alternative explanations for any observed differences.

When the outcome of interest is continuous, data obtained from the one-group pretest-posttest design can be analyzed with the dependent-means ttest (also sometimes called the correlated-means ttest or the paired-difference ttest), which tests H0: μD= 0. When the outcome of interest is categorical and has only two levels, one-group pretest-posttest data can be analyzed with McNemar's chi-square to test the null hypothesis that the distribution of responses is the same across time periods. However, when the outcome of interest is categorical and has more than two levels, McNemar's chi-square cannot be used. Instead, data can be analyzed through Mantel-Haenszel methods.

One-Group Pretest-Posttest Design Using a Double Pretest

The one-group pretest-posttest design can be improved upon by adding a second pretest prior to treatment administration:

where O1and O2represent the two pretests, X represents some treatment, and O3represents the posttest.

Adding a second pretest to the traditional one-group pretest-posttest design can help reduce maturation and regression to the mean threats as plausible explanations for any observed differences. For example, instead of comparing O3only to O1or O2, any observed difference between O2and O3can also be compared to any differences between O1and O2. If the difference between O2and O3is larger than the difference between O1and O2, then the observed change is less likely solely due to maturation.

When the outcome of interest is continuous, data obtained from the one-group pretest-posttest design using a double pretest can be analyzed through a within-subject design analysis of variance (ANOVA) (also sometimes referred to as repeated measures ANOVA) with appropriate contrasts to test the null hypothesis of interest, H0:μpost- μpost2= μpost2- μpost1. When the outcome of interest is categorical, data from this design can be analyzed with conditional logistic regression (also referred to as subject-specific models) to test the null hypothesis that the distribution of responses is the same across time periods.

...