Evidence Synthesis

Types and Sources of Evidence

One of the important factors to be considered in both carrying out and making use of a synthesis is the tremendous diversity in the types and sources of evidence that a synthesis might potentially consider. Most current evidence syntheses restrict themselves to research studies published in peer-reviewed journals. Yet even this restriction can yield an overwhelming amount of evidence, given the numerous electronic biographic databases that can be searched (of which MEDLINE, PubMed, PsycINFO, and EMBASE are only a few examples) and the number of languages in which health research studies are published. Other evidence synthesis methodologies also strive to include “fugitive” literature, that is, the search for evidence is expanded to include unpublished studies through searching conference proceedings and the Internet, by including relevant government reports and unpublished studies conducted by pharmaceutical companies, and by using personal networking to identify other studies that [Page 471]may not have been submitted to or published by peer-reviewed journals. The main advantage of this strategy is that there is still a bias on the part of journals to publish studies that report positive findings. However, an important limitation of including these reports is that they have not undergone (or passed) a peer review process. Plus, attempting to include all published and unpublished studies can become an overwhelming task because of the sheer volume of information.

Still other evidence syntheses draw on professional expertise and opinion. Although professional expertise must surely be considered a form of evidence, good research studies, where they are available, should take precedence in an evidence synthesis. However, professional expertise and expert consensus can be especially useful when there are few studies available, and they are invaluable in helping interpret the meaning of and importance of study findings where they do exist. Issues of practicality and evidence of public acceptance of a particular practice (e.g., a particular prevention or intervention strategy) should also be important considerations, although not all evidence syntheses attend to these issues. However, in using an evidence synthesis to aid in clinical or policy decision making, these are important issues to consider.

...