Qualitative-Quantitative Debate in Evaluation

Historical Notes

The quantitative-qualitative debate was precipitated primarily by two simultaneous developments: a broad paradigm conflict in the larger social scientific community that occurred throughout the mid to late 20th century but really caught fire in the 1970s and 1980s and an explosion of alternative evaluation theories in the 1970s.

The overall quantitative-qualitative debate that troubled the larger social scientific community was rooted in longstanding arguments within the philosophy of science. A quantitative paradigm, anchored in a positivist philosophy, dominated Western social science for much of the 20th century. Within this paradigm, social scientists aspired to test theories using objective methods that were valid, reliable, and replicable and experimental designs, whenever possible, that drew from successes in the physical and medical sciences. However, qualitative paradigms—rooted in interpretive, phenomenological, and constructivist philosophies and thus invested in contextual, value-laden, and contingent social knowledge—offered [Page 346]serious challenges to positivist dominance by the middle of the 20th century.

These challenges gradually began to make their way into the conversations of everyday researchers and evaluators. Many evaluators had to revisit the meanings of objectivity and subjectivity and learn anew about realist versus constructivist ontologies so that they could better understand where they stood on the underlying issues. As applied social scientists, evaluators were engaged in this debate, some staking out their position on the qualitative side, some on the quantitative side, and others trying to reflect on the criticisms fired from both sides and find a middle ground within which to work.

At the same time, the evaluation community itself was showing signs of frustration, primarily in response to the disappointing incidence of direct instrumental impacts of evaluations on important program and policy decisions. Evaluators who lent their expertise to the 1960s evaluations of the Great Society programs in the United States stretched to adopt the methods they had competently employed in small-scale studies, general population surveys, and controlled experiments to field settings that were frequently national in scope and targeted to special populations. These methods were not yet sufficiently refined to provide compelling evidence in the dynamic and politicized contexts of evaluation. The evaluation community responded to this failure to meet largely unrealistic expectations for immediate policy impacts by launching important research on evaluation utilization, continuing to refine and develop quantitative methods, and generating an explosive proliferation of alternative models for evaluation—notably, models that were responsive, decision oriented, judicial, management focused, and expert driven, as in connoisseurship, participatory evaluation, and evaluation intent on social justice. Qualitative methods and the interpretive philosophical traditions from which they drew were a good fit for many of these diverse and creative evaluation alternatives. Evaluation practice was no longer dominated by quantitative methodologies, and some of the alternative models were expressly tied to qualitative methodologies. The paradigm war was joined within the evaluation community.

...