Field Experimentation

Field experiments, as distinct from LABORATORY EXPERIMENTS, are randomized interventions that take place in naturalistic settings. The nature of these interventions varies widely. Examples include randomized school voucher programs, voter mobilization campaigns, police raids on crack houses, job training programs, income tax schedules, and health insurance plans. The unit of analysis in field experimentation also varies. Most field interventions target individuals, but many studies randomly assign treatments to groups or institutions.

In contrast to ethnographic or descriptive research, field experiments are principally designed to establish CAUSAL relationships. Well-executed field experiments combine the strengths of randomized designs with the EXTERNAL VALIDITY of field studies. Whereas a laboratory study might examine the effects of commercial advertising by exposing TREATMENT and CONTROL GROUPS to different advertising stimuli within an artificial setting and gauging their purchasing preferences within the context of a simulated marketplace, field experiments randomly manipulate the content and timing of actual advertisement campaigns and attempt to link these varied interventions to observed patterns of consumer demand. Both types of studies use randomization, but the latter has the advantage of linking cause and effect in terms that have direct, real-world applicability.

Field experimentation is also a valuable tool in program evaluation, particularly when used to gauge the effects of a new program or alternative versions of existing programs. Suppose, for example, that one sought to evaluate the effectiveness of school-based drug prevention programs. Some schools might receive drug awareness programs and others might not, the sequence in which schools receive drug awareness programs might be varied randomly, or the entire population of schools might receive alternative drug awareness programs. By comparing outcomes in treatment and control groups that were initially formed by random chance, the evaluator may make an unbiased assessment of the intervention’s effects.

One attractive feature of experiments in natural settings is the transparency of the data analysis. In contrast to nonexperimental data analysis, where results often vary markedly depending on the model the researcher imposes on the data and where researchers often fit a great many models in an effort to find the right one, experimental data analysis tends to be quite robust. Simple comparisons between control and treatment groups often suffice to give an unbiased account of the treatment effect, and additional analysis with COVARIATES merely estimates the causal parameters with greater precision. This is not to say that experimental research is free from data mining, but the transformation of raw data into statistical results involves less discretion and therefore fewer moral hazards.

In principle, field experimentation represents the strongest basis for sound causal inference available to socialscientists, butinpractice, thismethodologyfaces important ethical and pragmatic constraints. Rarely do social scientists have the opportunity to manipulate the variables of most interest to them, such as culture, political systems, economic prosperity, and so on. Even in situations where random interventions are attempted, they are sometimes undone by the people charged with implementing them. The well-intentioned physician who smuggles the sickest patients into the treatment group may cause the experiment to produce misleading results.

The challenge of orchestrating field experiments and maintaining the integrity of the randomization means that the experiments tend to occur in a small number of sites that are chosen for reasons of logistics rather than representativeness. This constraint raises the issue of whether the study’s conclusions apply only to the types of people who actually participate in an experiment. In general, REPLICATION is the appropriate response to concerns about drawing conclusions based on studies of particular times, places, and people.

...