Experimental Design

Experimental design is one of several forms of scientific inquiry employed to identify the cause-and-effect relation between two or more variables and to assess the magnitude of the effect(s) produced. The independent variable is the experiment or treatment applied (e.g. a social policy measure, an educational reform, different incentive amounts and types) and the dependent variable is the condition (e.g. attitude, behavior) presumed to be influenced by the treatment. In the course of the experiment it is necessary to demonstrate the existence of covariation between variables, its nonspuriousness, and to show that the cause occurred before the effect. This sort of inquiry can take the form of an artificial experiment, carried out in a laboratory scenario, or a natural experiment implemented in a real-life context, where the level of control is lower. For both cases, the literature presents several taxonomies, from which four main types are considered: (1) true or classical experimental, (2) pre-experimental, single-subject experimental, and (3) quasi-experimental. In addition, there are a number of variations of the classic experimental design as well as of the quasi-experimental design.

In a true or classic experimental design, there are at least two groups of individuals or units of analysis: the experiment group and the control group. Participants are randomly assigned to both groups. These two groups are identical except that one of them is exposed to the experiment or causal agent, and the other, the control group, is not. In many instances, a pretest and a posttest are administered to all individuals in the two groups; but the pretest is not a necessary aspect of the true experiment. If there is a significant difference between members of the two groups, it is inferred that there is a cause-and-effect link between that treatment and the outcome.

The pre-experimental design does not have a control group to be compared with the experiment group. There is a pretest and a posttest applied to the same participants. In a single-subject experimental design, there is only one participant, or a small number, that is [Page 253]analyzed over a period of time. In quasi-experimental designs, participants come from naturally assembled or pre-determined groups (e.g. a family, a school class, a professional category, or inhabitants of a neighborhood) and are not therefore assigned randomly to the control and treatment groups.

An experimental design has to fulfill several conditions. The variables must be measured with accuracy and precision, and the statistical test must be denned before starting the experiment. If necessary, it must be possible to repeat the experiment in order to confirm that the outcome is statistically significant and that no other factors, other than the independent variable) the researcher manipulates, are responsible for the outcome. In practice, however, not every experiment meets all these conditions. Internal and external validity can be affected by several factors. For example, internal validity (cause-effect relation) can be influenced by the length of the time between the pretest and the posttest, by changes in the measurement instruments, the influence of the pretest on subsequent behavior, and so on, but can be improved with matching procedures and by randomization. External validity can be enhanced by the use of a representative sample and by avoiding artificial experiment settings lacking mundane

...