Meta-Analysis: Estimation of Average Effect

Estimating an average effect in meta-analysis is the process of combining the effect sizes from the set of studies collected for the meta-analysis. Meta-analysis is a study of studies where research synthesis is performed through statistical aggregation of empirical research reports. Meta-analysis allows scholars to clarify inconsistent or discrepant findings in a research literature. Among others, meta-analysis serves two useful functions by allowing for the possibility of (a) estimating a weighted mean effect size from a sample of cases (the subject of the present entry and what many people would call the main function of a meta-analysis) and (b) testing moderating variables that may account for discrepant findings in research literature (which is beyond the scope of this entry). There are many steps to a meta-analysis, including conducting a search of the research literature, creating criteria for including and excluding studies that is objective and can be consistently applied, converting disparate study effects into a common metric, and computing meta-analytic results, which includes estimating the average effect of the communication construct of interest. The average effect of the treatment in comparison to some control uses a weighting process that takes into account the precision of the given effect sizes.

The purpose and utility of estimating an average effect in meta-analysis can best be demonstrated with a few illustrative cases. The first example dates back to the origin of the term meta-analysis. In the [Page 984]early 1950s, a debate about the benefits of psychotherapy began that continued unabated into the mid-1970s. Some researchers claimed it had no positive effects on patients and others vehemently disagreed. Hundreds of studies existed, but some studies found positive results for patients whereas others found negative effects and others found null effects. Narrative reviews failed to resolve the controversy, but finally the results of 375 studies were standardized and an average treatment effect was estimated. The conclusion was that psychotherapy was indeed effective for patients, and the debate was settled and a new era of research synthesis began.

A more contemporary example of using meta-analysis to calculate an average effect from the communication discipline can be found in the meta-analysis of research on inoculation theory. Inoculation theory uses a viral metaphor to understand how resistance to persuasive influence is conferred to individual attitudes. Instead of simply isolating attitudes from attacks, inoculation theory suggests that people should be forewarned their attitudes may be challenged and then introduced to counterattitudinal arguments that are then refuted (akin to being exposed to a weakened virus in medical inoculations). Inoculation research continued for 50 years, with several narrative reviews, before a meta-analysis was conducted. During that time, many studies found support that inoculation treatments were superior to controls, but others found inoculation to be no better and, in some cases, less effective than control messages. The meta-analysis found that inoculation was superior to both no-treatment control messages as well as supportive control messages, and that the average effect size was small, verging on moderate.

This entry discusses issues of precision, data adjustment, the inverse variance weight, as well as fixed- versus random-effects analyses in meta-analysis. Finally, this entry discusses some potential problems that may arise with random assignment, particularly when working with human participants.

...