Meta-Analysis

Cumulative knowledge is one of the cornerstones of science, and yet when numerous studies are undertaken over time, often with different populations, measures, and study designs, it can be difficult to assess what is known. Meta-analysis is a quantitative technique used to synthesize research findings in a given body of literature, providing a clear and objective synthesis of a body of research findings. This entry discusses the limitations of traditional narrative reviews; describes meta-analysis, its history, and why it is superior to narrative reviews; lists the steps undertaken in a meta-analysis; and provides examples of applications of meta-analysis in health communication.

Traditionally, the narrative review was the method used to summarize a body of literature. Such reviews are undertaken by gathering many articles in a field of study and writing a review article about what was known in that particular area. However, such reviews can be biased in at least three ways. The first issue is a selection problem. That is, rather than a systematic and comprehensive search of the literature, narrative reviews often rely on a convenience sample of studies that can be easily located. If studies are selected only on this basis, the conclusions of the review are likely to be biased. The second issue is a rigor problem. Many narrative reviews are not conducted systematically and lack transparent reporting about how the review was done; thus, such reviews often cannot be replicated and concerns about the integrity of the review can result. The third issue is a synthesis problem. Narrative reviewers lack a clear method of how to synthesize the data that are reported across studies. While the typical approach is to categorize studies into groups of “significant” and “nonsignificant” studies, there are several problems with this approach, including the fact that this can be a rather subjective judgment, particularly in cases where individual studies have mixed results.

Meta-analysis is a systematic approach to research synthesis that is focused on the quantitative integration of research findings. A central feature of meta-analysis is a focus on effect size, a term that refers to the magnitude of effect found in a given study. Focusing on effect size is a relatively recent concept in the social and communication sciences, as traditionally the focus was only on the statistical significance of study effects.

For example, a health communication campaign study might report that after the campaign, the intervention city had significantly higher exercise behavior than the control city. This means that on the basis of statistical testing, the campaign was successful. However, this result provides little information about the magnitude of the change; that is, did people change their behavior modestly or much more significantly? A meta-analysis would convert study findings to a precise effect size that represented the magnitude of change (e.g., those in the intervention city were 25 percent more likely to exercise after the campaign compared to those in the control city). This results in more precise research findings and allows for more efficient synthesis across numerous studies.

Meta-analytic techniques were initially developed in the 1970s, largely out of frustration with the status quo narrative review method. Mary L. Smith and Gene V. Glass's work on outcomes of psychotherapy is often described as the first meta-analysis, although other researchers (Robert Rosenthal and Donald Rubin at Harvard University, John Hunter and Frank Schmidt at Michigan State University) were concurrently and independently developing what later became known as meta-analytic procedures. Today, meta-analytic methods can be viewed in terms of three sets of [Page 869]approaches, including those of Larry Hedges and Ingram Olkin, John Hunter and Frank Schmidt, and Robert Rosenthal. All three approaches are quite similar, and they all follow the same steps outlined below. The differences emerge in the statistical details of how effect sizes are calculated and synthesized together. In the literature, examples of all of these approaches can be found, as well as meta-analyses that blend particular approaches together.

...