Realist Evaluation

Realist evaluation, at its core, focuses on developing explanations of the consequences of social actions that contribute to a better understanding of why, where, and for whom programs work or fail to work. To this end, realist evaluators place a great deal of emphasis on (a) identifying the mechanisms that produce observable program effects and (b) testing these mechanisms and the other contextual variables or individual characteristics, often referred to as moderators, that may have impacts on the effects that are observed. Realist evaluation stands as an alternative to the two most pervasive schools of thought around which evaluation practice frequently has been organized: logical positivism and social constructivism. Rather than focusing on the contrasting philosophies, however, in this entry we will delve into the issues that everyday evaluators can use to frame their own work from a realist perspective, beginning with an example.

A number of evaluations of early education programs indicate a common pattern of results: high quality programs accelerate increases in children's cognitive skills, but the cognitive skills of program participants and other children appear similar 3 to 5 years after the children have left the programs. However, the children who participated in these preschool programs benefit in terms of their social outcomes, such as lower incidence of retention in grade, higher high school graduation rates, and increased likelihood of employment. Patterns that repeat themselves, such as these, are the building blocks for constructing social programs that successfully improve life chances for individuals, as well as social conditions more generally. Why do these patterns occur? Many hypotheses have been offered as explanations, but one team of evaluation researchers, led by Arthur Reynolds, has shown that the cognitive advantage hypothesis is most consistent with the pattern of long-term social outcomes and that other hypotheses, such as increasing family support for the children's education or improving the children's attitudes about schooling, do not match the observed patterns of results as well. Other evaluations of the largest compensatory early childhood program in the United States, Head Start, show different patterns of long-term outcomes for European American children and African American children. Again, we have a pattern or observed regularity that begs for an explanation. Are the differences in effects attributable to differences in the quality of the Head Start centers that served European American children and those that enrolled African American children? Are the differences associated with family risk factors, such as poverty or lack of education of parents? Does the quality of later schooling explain differences? The race of the child appears to moderate the effects of Head Start, but realist evaluators would not stop with the apparent distinction. They would ask, “Does the [Page 360]observed difference have anything to do with the race of the child, or is race simply correlated with an underlying difference that has not been directly observed or measured?”

Beginning with the two related findings that are presented in this brief example, we can illustrate some of the more salient aspects of realist evaluation. We will present several commonly held beliefs that unify realist evaluators and some about which they differ, including the importance of ongoing social relations onto which social programs are overlaid, the investigation of patterns in the data, methods for uncovering the mechanisms that explain the observed patterns, and the importance of context.

...