Meta-Analysis

Meta-analysis is the quantitative cumulation of scientific evidence. There has long been pessimism in the younger social, behavioral, educational, and biomedical sciences in that our progress has been slower and less orderly than we would like, at least when compared to the progress of older, more programmatic sciences, such as physics and chemistry. In other words, the more recent work in physics and chemistry seems to build directly on the older work of those sciences, whereas the more recent work of the social, behavioral, educational, and biomedical sciences seems often to be starting from scratch. Those who have looked closely at the issue of cumulation in the physical sciences have pointed out that these disciplines have ample problems of their own. Nonetheless, in the matter of cumulating evidence, the social, behavioral, educational, and biomedical sciences have much to be modest about.

Limited success in the process of cumulation does not seem to be due to lack of replication or to the failure to recognize the need for replication. Indeed, there are many areas of the social, behavioral, educational, and biomedical sciences for which the results of many studies, all addressing essentially the same question, are available. Our summaries of the results of these sets of studies, however, have not been nearly as informative as they might have been, either with respect to summarized significance levels or, more important, with respect to summarized effect sizes, that is, the magnitudes of the effects examined. Even the best reviews of research by the most sophisticated scholars have been primarily qualitative narratives and have rarely told us much more about each study in a set of studies than the direction of the relationship between the variables investigated, and often not even that, and whether or not a given significance level was attained.

This state of affairs is beginning to change, however. More and more reviews of the literature are moving from the traditional literary approach to quantitative approaches to research synthesis described in an increasing number of textbooks of meta-analysis. The goals of these quantitative approaches of meta-analysis are to help us discover what we have learned from the results of the studies conducted and to help us discover what we have not yet learned.

In what follows, this entry defines the concept of “research results,” briefly examines the history of meta-analysis, defines the concept of “successful replication,” and concludes by comparing a more traditional view of replication success with a newer, probably more useful view.