Meta-Analysis

Meta-analysis is a statistical method that integrates the results of several independent studies considered to be “combinable.” It has become one of the major tools to integrate research findings in social and medical sciences in general and in education and psychology in particular. Although the history of meta-analytic procedures goes all the way back [Page 794]to the early 1900s and the work of Karl Pearson and others, who devised statistical tools to compare studies from different samples, Gene V. Glass coined the term in 1976. Glass, Barry McGaw, and Mary Lee Smith described the essential characteristics of meta-analysis as follows:

• 1.
  It is undeniably quantitative, that is, it uses numbers and statistical methods for organizing and extracting information.
• 2.
  It does not prejudge research findings in terms of research quality (i.e., no a priori arbitrary and nonempirical criteria of research quality are imposed to exclude a large number of studies).
• 3.
  It seeks general conclusions from many separate investigations that address related or identical hypotheses.

Meta-analysis involves developing concise criteria for inclusion (i.e., sampling), searching the literature for relevant studies (i.e., recruitment), coding study variables (i.e., data entry), calculating standardized effect sizes for individual studies, and generating an overall effect size across studies (i.e., data analysis). Unlike primary studies, in which each case in a sample is a unit of analysis, the unit of analysis for meta-analysis is the individual study. The effect sizes calculated from the data in an individual study are analogous to the dependent variable, and the substantive and methodological characteristics affecting the study results are defined as independent variables. Any standardized index that can be used to understand different statistical findings across studies in a common metric can be used as an “effect size.” The effect size metric represents both the magnitude and direction of the relation of interest across different primary studies in a standardized metric. A variety of alternatives are available for use with variables that are either continuous or discrete, such as the accumulation of correlations (effect size r), and standardized differences between mean scores (effect size d), p values, or z scores effect size (ES). The dependent variable in meta-analysis is computed by transforming findings of each reviewed study into a common metric that relies on either r or d as the combined statistic.

Meta-analysis is not limited to descriptive reviews of research results but can also examine how and why such findings occur. With the use of multivariate statistical applications, meta-analysis can address multiple hypotheses. It may examine the relation between several variables and account for consistencies as well as inconsistencies within a sample of study findings. Because of demand for robust research findings and with the advance of statistical procedures, meta-analysis has become one of the major tools for integrating research findings in social and medical science as well as the field of education, where it originated. A recent search of the ERIC database identified more than 618 articles published between 1980 and 2000 that use meta-analysis in their title, as opposed to only 36 written before 1980. In the field of psychology, the gap was 12 versus 1,623, and in the field of medical studies, the difference is even more striking: 7 versus 3,571. Evidence in other fields shows the same trend toward meta-analysis's becoming one of the main tools for evidence-based research.

...