Policy-Relevant Methods in Applied Developmental Science

Random Assignment

Experimental evaluations of social programs frequently involve randomly assigning participants to either receive the program being evaluated or to a comparison group receiving no services or “services as usual.” In doing so, researchers control for potential differences between individuals who receive the treatment and those who do not, and thus increase the internal validity of the study, that is, the confidence with which one can attribute differences between the two groups to the treatment program.

Threats to Validity

However, this investment in internal validity may be associated with considerable loss of external validity (e.g., Sue, 1999). Specifically, outside of randomized clinical research trials, people are not randomly assigned to programs that are provided in communities (e.g., children are not randomly assigned to Head Start). Furthermore, in applied developmental research, participants typically know what treatment group they are in, and the perception and opinion of the participant about the program may substantially influence treatment effectiveness.

This can influence external validity in several ways. First, people willing to be randomly assigned in a demonstration project may differ substantially from those who would choose a particular treatment when it is offered as a community service. This may be true especially among particularly “overstudied” minority populations, who may be reluctant to volunteer for [Page 846]“a research study” but be quite willing to participate in services that are publicly offered in the community (Cauce, Ryan, & Grove, 1998).

Second, the treatment may produce much better outcomes, for example, for those who actively seek out that treatment than for those who are randomly assigned to it. In this case, random assignment can produce misleading results, such as showing that a treatment is ineffective when it is actually quite effective for certain types of participants. Because of these factors, it is important that selection processes and motivational factors leading to positive outcomes be studied directly in social interventions rather than “controlled” through randomization (e.g., Koroloff & Friesen, 1997). For example, consider a design in which half of a sample of divorcing families is randomly assigned to either (a) court determination or (b) mediation, as in a traditional randomized trial, while the remaining parents choose whether they want court determination or mediation. Such a design permits a direct examination of random assignment versus selected participation.

...