Experiments, Natural

The importance of natural experiments—analyses of real-world situations that attempt to employ some type of random assignment—is in their contribution to addressing confounding, a pervasive problem in the social sciences. Consider the obstacles to addressing the following hypothesis: Extending property titles to poor land squatters boosts access to credit markets and promotes beliefs in individual political efficacy, thereby fostering socioeconomic development. To test this idea, researchers might compare poor squatters who possess land titles with those who do not. Yet confounding may be a problem because differences in individual attitudes and behaviors could in part be due to factors—such as family background—that also make certain poor squatters more likely to acquire titles to their property. This entry examines both the strengths and potential limitations of natural experiments and how they fit among the various research strategies available to political scientists.

It is useful to contrast natural experiments with both conventional observational studies and true experiments. In the first approach, investigators seek to control for potential confounders in observational (nonexperimental) data. For instance, they may compare titled and untitled squatters within strata defined by measures of family background. At the core of conventional quantitative methods is the hope that such confounders can be identified, measured, and controlled. Yet this is not easy to do. Even within the strata defined by family background and intelligence, there may be other difficult-to-measure confounders—say, determination—that are associated with obtaining titles and that also influence economic and political behaviors.

A second approach would be to use a randomized controlled experiment to estimate the effects of land titling. Subjects could be randomly assigned to receive titles or not; family background, determination, and other possible confounders would then be equivalent, on average and up to random error, across these two groups. Thus, large posttitling differences would then be credible evidence for a causal effect of land titles. However, experimental research in such contexts may be expensive, impractical, or unethical.

Scholars therefore increasingly make use of natural experiments—attempting to identify and analyze real-world situations in which some process of random or as-if random assignment places cases in alternative categories of the key independent variable. In the social sciences, this approach has been used to study the relationship between [Page 876]lottery winnings and political attitudes, the effect of voting costs on turnout, the impact of quotas for women village councilors on public goods provision in India, and many other topics. In the health sciences, a paradigmatic example comes from John Snow's 19th-century tests of the hypothesis that cholera is waterborne.

Natural experiments share one crucial attribute with true experiments and partially share a second attribute. First, outcomes are typically compared across subjects exposed to a treatment and those exposed to a control condition (or a different treatment). Second, in partial contrast with true experiments, subjects are often assigned to the treatment not at random but rather as-if at random (though sometimes true randomization occurs, as in lottery studies). Given that the data come from naturally occurring phenomena that often entail social and political processes, the manipulation of the treatment is not under the control of the analyst; thus, the study is observational.

...