Clinical Epidemiology & Evidence-Based Medicine: Fundamental Principles of Clinical Reasoning & Research


David L. Katz

  • Citations
  • Add to My List
  • Text Size

  • Chapters
  • Front Matter
  • Back Matter
  • Subject Index
  • Dedication

    I dedicate this book to my wife, Catherine: the reason why.


    View Copyright Page


    Evidence has securely claimed its place among the dominant concepts in modern medical practice. To the extent possible, clinicians are expected to base their decisions (or recommendations) on the best available evidence. Physicians may see this as one possible silver lining in the dark cloud of managed care. Insurers competing for clientele and revenue have increasingly made it a practice to include in benefits packages only those items for which there is convincing proof of benefit. Moreover, these items must provide their benefit at reasonable and acceptable cost. Thus, when applying evidence to the practice of medicine the benefit of the evidence must be measurable and definable, the cost must be measurable, and, perhaps the subtlest challenge of all, evidence itself must be defined and measured.

    Despite current efforts to bridge the gap between medicine and public health through the Medicine-Public Health Initiative,1,2 the philosophical divide between a discipline devoted to the concerns of populations and one devoted to the advocacy of an individual seems impassable. However, the consistent application of evidence to clinical decision making is the bridge between the concerns of clinical practice and the goals of public health.

    Evidence-based practice is population-based practice. Evidence applied clinically is derived from the medical literature, where the standards of evidence, and therefore practice, continuously evolve. But what is reported in the literature is not the experience of an individual patient (other than in case reports, a modest although time-honored and often important source of evidence, or in n-of-1 experiments), and certainly not the experience of our individual patient, but rather the experience of a population of patients. Therefore the practice of evidence-based medicine requires the application of population-based data to the care of an individual patient whose experiences will be different in ways both discernible and not, from the collective experience reported in the literature. All evidence-based decisions made on behalf of (or preferably, with) individual patients are extrapolation or interpolation from the prior experience of other patients. Clinical medicine is evidence-based only if it is population-based.

    This may or may not seem a provocative concept, but consider the alternative. To base clinical decisions for an individual on the individual alone, the outcome of an intervention would need to be known in advance. In other words, medicine would need to borrow from astrology or some other system of predicting future events. The choice of an initial antihypertensive drug for a hypertensive patient cannot be based, before the drug is prescribed, on the response of the patient in question. Nor can the benefits to the patient be known in advance. The drug is chosen based on the published results of antihypertensive therapy in other patients. The particular drug is selected based on how closely the characteristics of our patient match those of others who have benefited from specific therapies. Once the drug is selected, while the therapeutic effect on the surrogate measure (e.g., blood pressure) is detectable, any outcome benefit to our patient (e.g., stroke prevention) remains unknowable. We can never identify the stroke we have prevented in an individual. The strokes we prevent by prescribing antihypertensives, the myocardial infarctions we prevent by prescribing aspirin or statins, are statistical events. We know the rate at which such conditions occur in particular populations, and research demonstrates how these rates can be changed. By applying the intervention to our patient, we expect the risk of the event to decline comparably. But unless an intervention eliminates the risk of a clinical event entirely (few, if any, do), our patient may suffer the event despite intervention. Alternatively, our patient may have appeared to be at risk, but would have never suffered the event even without intervention. We can never know. We never base what we do for an individual on the outcomes particular to that individual. We base what we do on the experience of populations, and the probability that our patient will share that experience. Astute medical care is predicated on the capacity to identify similarities between a single patient and the particular population whose collective experience is most likely to inform and anticipate the single patient's experience. In his Poetics, Aristotle considers this “eye for resemblances,” or “intuitive perception of the similarity in dissimilars,” a mark of genius.3 If so, it is a genius the clinician frequently has cause to invoke.

    The science of applying the principles of population-based (epidemiologic) evidence to the management of individual patients has come to be known as clinical epidemiology. While epidemiology characterizes the impact of health related conditions on populations, clinical epidemiology applies such data to individual patient care. Clinicians are traditionally uncomfortable with the notion of catering to populations rather than individual patients. Clinical epidemiology asserts that the two are effectively the same, or at least inextricably conjoined. Individual patient care is informed by the interpretation of population-based data. When populations are well served by the health care they receive, the individual members of those populations are (generally) well served. When individuals are well served by the clinical care they receive, in the aggregate, the pattern of that care becomes (usually) the sound practice of health care delivery to populations.

    Implicit in the concept of evidence being the derivative of a populations experience is the need to relate that experience back to the individual patient. The inapplicability of some evidence to some patients is self-evident. Studies of prostate cancer are irrelevant to our female patients; studies of cervical cancer are irrelevant to our male patients. Yet beyond the obvious exclusions is a vast sea of gray. If our patient is older than, younger than, sicker than, healthier than, ethnically different from, taller, shorter, simply different from the subjects of a study, do the results pertain? As our individual patient will never be entirely like the subjects in a study (unless they were a subject, and even then their individual experience might or might not reflect the collective experience), can the results of a study ever be truly pertinent? Clinical epidemiology is a sextant, or in more modern but equally nautical terms, the geographical positioning system (GPS), on a vast sea of medical uncertainty. And, to extend the metaphor, the skills of piloting can be acquired and increase the reliability with which a particular destination (i.e., diagnosis, therapeutic outcome) is achieved. Yet each crossing will be unique, often with previously unencountered challenges and hazards. No degree of evidence will fully chart the expanse of idiosyncrasy in human health and disease. Thus, to work skillfully with evidence is to acknowledge its limits. Judgment must be prepared to cross those seas as yet uncharted by evidence.

    It is expected that some of the material in this text, particularly the more statistically involved, will diverge from what we would accept as intuitive. However, comfort can be taken from the fact that we are all de facto clinical epidemiologists. As clinicians, we decide which information pertains to a particular patient every day of practice: who does and does not get prescribed an antibiotic (and if so, which one); who does and does not get treated with insulin, metformin, a sulfonylurea, a thiazolidinedione; who does and does not get advised to be x-rayed, injected, phlebotomized, cannulated, or instrumented. Cognizant or not of the subtleties as they play out, in each such decision we are comparing our patient to others that have come before; others in our own practice, relying as we tend to do (though we tell one another we should not) on the compelling lessons of personal anecdote, or others whose experience has been more formally conveyed, in the tables and graphs of a peer-reviewed article. While the choices we ultimately make in clinical testing and management are a product of our interaction with patients, our shared and disparate values, beliefs, and preferences, the decisions that delineate those choices are largely the product of clinical epidemiology.

    Because we all practice clinical epidemiology, an understanding of this tool (or array of tools) we use is incumbent upon us all. If every clinical decision derives in whole or in part (and it does) from the tacit comparison of our patient to a population of patients, then the skill with which that comparison is made is fundamental to the skill with which medicine is practiced. Integral to that comparison is the capacity to recognize the defining characteristics of both patients and populations as the basis for defining the bounds of similarity and dissimilarity. The physician's capacity to evaluate the context in which “evidence” was gathered is equally important. The ability to evaluate the quality as well as the pertinence of evidence is essential. Of course, finding the best available evidence when one is uncertain about a clinical decision is prerequisite to its interpretation.

    Viewed with the cool glare of reductionism, the practice of evidence-based medicine requires a discrete and modest skill set. One must be able to find the available evidence. One must be able to evaluate the relevance and quality of evidence. And one must be able to interpret evidence presented in terms pertinent to populations so that the same data may inform patient care decisions. These skills, like any others, can be learned and mastered. The various tools of our trade—stethoscopes and sphygmomanometers—were handed to us along with the lessons that made us competent in their use. While the tools of evidence-based practice have become enjoined among the more highly valued items in our proverbial black bags, most of us have had no formal instruction in their use. Consequently, many of us are likely using these tools less effectively than we might.

    While clinical choices (for both testing and treatment) are predicated on, at a minimum, the knowledge, judgment, values, preconceived notions, experiences, preferences and fears of both clinician and patient, clinical decision-making is greatly influenced by three considerations: probability, risk, and alternative. Probability is fundamental to such decisions, as we evaluate and treat patients only for a given condition or conditions it seems they might have. We do not order CT scans of every patient's brain, yet we do order some. The distinction is derived from our estimate of the probability of finding relevant pathology. A clinical decision cannot be reached without a semiquantitative estimate of probability. A patient either seems likely enough, or not likely enough, to need a particular test or treatment, to result in our recommending it. This is a truism for any test applied only to some patients.

    Some low probability diagnoses are pursued because they pose such high risk. Here, too, the natural tendencies of our minds are in alignment with clinical epidemiology. We admit some patients to the hospital to “rule out MI” even though we believe the probability of myocardial infarction (MI) to be low, because the risk associated with undetected MI is high. We have all been taught to do a lumbar puncture (LP) whenever we wonder “should I do an LP?” because of the devastating consequences of missing meningitis.

    Finally, we factor in alternatives: alternative treatments, alternative tests, alternative diagnoses. When chest pain seems atypical for angina, but no alternative explanation is at hand, we are more apt to treat the pain as angina. When pneumonia is present to explain shortness of breath, we will be less inclined to work up pulmonary embolism (PE), despite pleuritic chest pain and tachycardia. When we have excluded the impossible we are apt to focus on what remains, however improbable.4 By a process to which we are, for the most part, comfortably incognizant, we make every decision factoring in considerations of probability, risk, and alternatives.

    But an unconscious process is a process that cannot be optimally regulated. By knowing that our decisions are borne on our musings over probability, risk, and alternative, these parameters should become of sufficient interest to us to warrant conscious monitoring. Each of these parameters is population-based. There is no probability of genuine relevance to an individual: there is the rate of occurrence in populations, and the degree of concordance between individual and population characteristics. There is no true individual risk; for an individual, an event occurs (100% risk) or does not (0% risk). There, is however, the comparability of the patient to groups in whom the event rate in question is higher or lower. The alternatives available for an individual patient are those options and interventions applied under similar circumstances to other patients, with varying degrees of success.

    Similar principles underlie the research that constitutes the evidence base (or its greater portion) for clinical practice. As is detailed later in the text, studies are constructed in an effort to establish the collective experience of a few (the study subjects) as representative of the many. While the clinician looks for correspondence between patient and study participants, the investigator must consider the relevance of the study to the larger population of potential future patients. Just as the probability of outcomes, good and bad, guides clinical management, the probabilities of outcomes, good and bad, false and true, are estimated and factored into the statistical stipulations and design of a study. The appropriateness of a particular methodology depends as much on alternatives as does the appropriateness of a clinical intervention. As is expressed by the conventional safeguards against false-positive and false-negative error (see Chapter 9) and the application of progressively stringent standards of human subject protection,5,6,7,8 thorough consideration of risk is intrinsic to the research process. Even less rigorous means of conveying evidence, such as case reports and case series, depend for their interest and relevance on probability, alternative, and risk. Such reports are meaningful only when the putative association is apparent and convincing; the clinical need nontrivial; the risks of application acceptable; alternative explanations unlikely; and the pertinence to our patients probable. Probability, alternative, and risk influence one another within the disciplines of clinical practice and clinical research, and these disciplines in turn interact. The needs, insights and frustrations of practice are an important source of hypotheses and attendant methods in clinical research. The evidence generated by such studies naturally serves to inform clinical practice. These interactions are displayed in Table 1.

    TABLE 1 The Influence of Probability, Risk, and Alternatives on Clinical Research and Clinical Reasoning, and the Salient Interactions

    Ultimately, then, while judicious practice depends on evidence, the derivation of evidence depends on many of the same principles as judicious practice. The thoughtful and diligent practitioner remains abreast of the literature to apply the best practice an evolving evidence base supports. The diligent and thoughtful investigator exploits evolving methodologies to generate evidence most conducive to advances in science and practice. The highest standards of evidence-based practice are achieved not only when evidence is well applied, but also when that evidence is well produced. Part of the burden for the responsible cultivation of higher standards and better outcomes in medicine falls, naturally, to researchers and those that screen and publish their findings. But application is ultimately the responsibility of the clinician, who is obligated to consider not only the pertinence of particular evidence to his or her practice but the adequacy and reliability of the evidence itself. At every step, from the design of a study to clinical counseling, probability, alternative, and risk must be addressed. For evidence to be well applied the correspondence of this one patient to those that came before must be considered, the compatibility of prior knowledge with current need revisited.

    We cannot, therefore, practice clinical medicine and avoid population-based principles. We cannot practice clinical medicine and avoid the practice of clinical epidemiology. But the discipline of evidence-based practice/clinical epidemiology (the terms might be used interchangeably) is not one in which most of us have had any formal initiation. All of the art and all of the science of medicine depend on how artfully and scientifically we as practitioners reach our decisions. The art of clinical decision-making is judgment, an even more difficult concept to grapple with than evidence. As the quality and scope of evidence to support clinical interventions is, and will likely always remain, limited in comparison to the demands of clinical practice, the practice of evidence-based medicine requires an appreciation for the limits of evidence, and the arbiters of practice at and beyond its perimeters. Judgment fortified by the highest standards of decision-making science is a force to be reckoned with, enabling us each to extract the best possible results from a process to which we are naturally inclined. Ultimately that is the validation of evidence-based practice, or population-based practice, or clinical epidemiology—the outcomes to which such concepts contribute. Rigorous reasoning is the means, desirable outcomes the ends.

    For Whom is This Book Intended?

    This book is about concepts, or rather the methodology of arriving at robust clinical decisions that binds together an array of concepts. This book is not about the facts, or current fund of medical knowledge, on which such decisions rest. The life span of medical facts is short and shortening further all of the time. Fortunately the methods for extracting optimal performance from the prevailing facts of the day are enduring. The intent here is to provide a basic mastery of such methods, that is, the capacity to harness the power of our intrinsic heuristics (decision-making pathways) and apply it to a constantly evolving body of knowledge. The medical literature and clinical vignettes will be referenced as required to demonstrate applications of the methods described. But the message is in the methods rather than their application to any particular study or article or case.

    The intended audience for this text is anyone who makes, or will make, clinical decisions. Worth noting are a number of excellent texts on the subjects of clinical epidemiology and evidence-based medicine already available, many of which I have used liberally as sources (see Text Sources). Compared to most of these, this text is intended to be more clinician-friendly and assumes less prior knowledge. Every effort has been made to present material in a simple and uncluttered manner. Most tables in the text, for example, should be interpretable at a glance.

    One of the important distinctions I have made while writing this text is to endeavor to teach less and clarify more. The contention on which this text is based is that clinicians are intuitive clinical epidemiologists, and therefore don't really need to learn to function as such. This text is designed to help reveal the influence and application of this intuition. By doing so, it should illuminate the processes of converting evidence to decisions, research to practice. The more we understand the ways in which we approach evidence and make decisions, the more reliably we can control these processes, and their attendant outcomes.

    While a fair amount of statistics is included, the use of a calculator in clinical practice is certainly not intended. Rather, as quantitative principles already underlie clinical reasoning, one is well advised to have a basic familiarity with those principles. Fundamentals of practice truly hang in the balance. A positive or negative test result is at times highly reliable, at other times highly unreliable. A bit of number crunching demonstrates how different clinical conclusions can, and should, be under different circumstances. The numbers need not be recalled for the importance of the concepts to be retained.

    The consistent application of the basic principles of clinical epidemiology infuses with the strengths of science the decision making that presupposes all else in clinical practice, including its outcomes. That science and evidence are limited and are dependent upon judgment for their application is implicit in the text everywhere it is not explicit. Also implicit throughout the text is that the medical decisions reached by clinicians serve only to provide patients—the ultimate decision makers—with good information upon which to base their decisions.

    I am grateful to the many accomplished clinicians and clinical epidemiologists whose contributions I have drawn on so heavily, both in the drafting of this text and in my own clinical and research efforts. I acknowledge with appreciation and humility that in drafting this text I have followed where many luminaries have led. That said, if I have wandered off the trails blazed by the leaders of this field, I can blame no one but myself. Any misstep—ambiguity, miscalculation, or distortion—is of course my responsibility. While hoping that none is found, I apologize and offer my sincere regret in advance on the chance that any is.

    With a great reverence for the unique burdens and privileges of clinical practice, I submit the principles of this text in the belief and hope that they will enhance your ability to obtain the best possible outcomes for your patients.

    ReiserSJ. Medicine and public health. Pursuing a common destiny. JAMA. 1999;276:1429–1430.
    ReiserSJ. Topics for our times: The medicine/public health initiative. Am J Public Health. 1997;87:1098–1099.
    BarnesJ, (ed). The Complete Works of Aristotle.2Princeton, NJ: Princeton University Press; 1984:2335.
    Conan DoyleA. The sign of four. In: Conan DoyleA.The Complete Sherlock Holmes. New York: Doubleday; 1930:87–138.
    AmdurRJ. Improving the protection of human research subjects. Acad Med. 2000;75:718–720.
    BragadottirH. Children's rights in clinical research. J Nurs Scholarsh. 2000;32:179–184.
    BeasleyJW. Primary care research and protection for human subjects. JAMA. 1999;281:1697–1698.
    HighDM, DooleMM. Ethical and legal issues in conducting research involving elderly subjects. Behav Sci Law. 1995;13:319–335.


    I am grateful to Dr. Ralph Horwitz, chairman of medicine at the Yale School of Medicine, for setting the standard so many of us strive (without much hope of success) to meet.

    I sincerely appreciate the vision of Dan Ruth, and the guidance and support of C. Deborah Laughton, at Sage. The transition from idea to book is a relay race, in which their laps were very well run indeed.

    I acknowledge with thanks the contributions of my collaborators, Dr. Laura Greci, a senior resident in preventive medicine/internal medicine at Griffin Hospital in Derby, CT, and Dr. Haq Nawaz, associate director of the same preventive medicine residency.

    I am grateful to my parents, Dr. Donald Katz and Susan Katz, for never (well, hardly ever…) discouraging me when, as a child, I incessantly asked, “why?” To my Dad, I also add appreciation for the walks at Horse Heaven; I do my best thinking there.

    I am deeply indebted to Jennifer Ballard, administrator of the Yale Prevention Research Center, who makes me wonder every day how I managed before!

    I am grateful to my children, Rebecca, Corinda, Valerie, Natalia, and Gabriel, for their patience and unconditional love, seemingly unattenuated by the countless times I have turned from them to the computer, and turned down their invitations to play.

    Above all, I am grateful to my wife, Catherine, my best editor as well as my best friend, for the love and the coffee and the kind words as much as for the ruthlessly honest (and always constructive) criticism.

  • Appendices

    Appendix A: Getting at the Evidence

    Appendix A.1: Accessing the Medical Literature: How to Get There from Here

    Online search engines offer the advantages of up-to-date, tailored information access. The disadvantages of online searching include the potential time lost in futile efforts and the need to weigh the quality of evidence from a diverse array of sources.

    Online medical databases are collections of published medical articles from peer-reviewed (and sometimes non peer-reviewed) journals. The National Library of Medicine's (NLM) MEDLINE is the premier example. For the generalist clinician, MEDLINE tends to be the most useful database. It is appropriate for the efficient identification of available answers to most clinical questions. There are numerous searchable databases with more targeted application as well.

    NLM Online Databases and Databanks

    The National Library of Medicine maintains a computerized database bank, collectively called MEDLARS® (MEDical Literature Analysis and Retrieval System). MEDLARS contains several online databases including some 20 million references (and growing). Some of the more important databases maintained by the NLM are listed in Table A1.1. Table A1.2 provides the characteristics of some of the more commonly used NLM databases.

    TABLE A1.1 Some Databases Maintained by the National Library of Medicine
    CancerLit®MeSH Vocabulary®
    Source: National Library of Medicine website (
    TABLE A1.2 Databases Maintained by the NLM and Their Characteristics


    MEDLINE covers many broad areas in the fields of medicine, nursing, dentistry, veterinary medicine, public health, and health care systems, among other clinical and basic sciences. The MEDLINE database consists of citations and abstracts from approximately 4,300 journals as of mid 2000. The majority of journals published in the US are included in MEDLINE along with journals from almost 70 other countries. MEDLINE contains foreign language citations to articles with an English abstract. MEDLINE'S ever-expanding pool of citations reached 11 million in 2000, of which about three-quarters provide abstracts. The database, however, is limited to articles published as of 1966 or after. Citations prior to 1966 are available in a database called OLDMEDLINE, but do not provide abstracts or MeSH (see below) term indexing.

    PreMEDLINE represents a preliminary MEDLINE, providing basic citation information and abstracts before the completed records are prepared and added to MEDLINE. Each record is identified with a Unique Identification (UI) number at this stage. Each article is then indexed according to a predetermined indexing system known as Medical Subject Heading (MeSH) terms, or “mesh terms.” Subsequently, indexing for Publication Type (e.g., Randomized Controlled Trial, Review, etc.) is added to enhance searchability. Once this process is completed, a citation is added to MEDLINE. As the record is transferred to MEDLINE, it is deleted from PreMEDLINE. More than 8,000 of these citations are transferred every week.

    MeSH Terminology

    The NLM indexes each article with a standardized vocabulary system. The MeSH vocabulary contains approximately 17,000 terms. Each MeSH term represents a solitary concept in medicine. New terms are entered as they develop in the medical literature. MeSH is the major means by which a search engine will retrieve articles. Each article published in biomedical journals is indexed with several MeSH terms that represent the overall theme and contents of the article. MeSH terms themselves are grouped under major divisions called trees. For example, atrial fibrillation is a MeSH term, but it also is a subcategory of cardiovascular disease which itself is a branch of disease as illustrated in Figure A1.1. Each MeSH term is linked to other terms in a vocabulary hierarchy.

    Figure A1.1. Illustration of a major tree, MeSH term, and MeSH subheading in descending order.

    Major MeSH terms (MeSH subheadings) are further categorized into MeSH subtopics to further refine the indexing. A searcher can narrow his or her search by choosing more specific terms down the line, or may expand the search to be more inclusive by choosing terms higher up in the MeSH hierarchy. In the above instance, a search with the more general term cardiovascular diseases would yield far more records than a search with the term atrial fibrillation.

    For most clinicians, the term atrial fibrillation is apt to be broader than the goals of a particular search. The addition of qualifiers to the MeSH headings or sub-headings results in a narrower search. Qualifiers further dissect a concept into categories such as etiology, pathology, diagnosis, etc. Available qualifiers for “atrial fibrillation” are listed in Table A1.3.

    TABLE A1.3 Listing of Qualifiers Used for MeSH Term atrial fibrillation
    chemically inducedpathology
    complicationsprevention & control
    drug therapyradiography
    economicsradionuclide imaging
    Source: National Library of Medicine website (

    A listing of all (current) qualifiers included in Medline is provided in Table A1.4. Qualifiers are designated by an intervening forward slash (/) character. For instance, the term atrial fibrillation/etiology indicates a search targeting articles on the etiology of atrial fibrillation. A search with no qualifiers added will include all of the qualifiers that exist for that term. As shown in Table A1.4, two-letter abbreviations may be used for qualifier terms. Table A1.5 lists “Publication Type” terms for MEDLINE.

    TABLE A1.4 Subheadings (Qualifer terms) Used with MeSH Terms in MEDLINE and Their Standard Abbreviations
    Abnormalities ABEmbryology EMPhysiopathology PP
    Administration and Dosage ADEnzymology ENPoisoning PO
    Adverse Effects AEEpidemiology EPPrevention and Control PC
    Agonists AGEthnology EHPsychology PX
    Analogs and Derivatives AAEtiology ETRadiation Effects RE
    Analysis ANGenetics GERadiography RA
    Anatomy and Histology AHGrowth and Development GDRadionuclide Imaging RI
    Antagonists and Inhibitors AlHistory HIRadiotherapy RT
    Biosynthesis BIImmunology IMRehabilitation RH
    Blood Supply BSInjuries INSecondary SC
    Blood BLInnervation IRSecretion SE
    Cerebrospinal Fluid CFInstrumentation ISStandards ST
    Chemical Synthesis CSIsolation and Purification IPStatistics and Numerical Data SN
    Chemically Induced CILegislation and Jurisprudence LJSupply and Distribution SD
    Chemistry CHManpower MASurgery SU
    Classification CLMetabolism METherapeutic Use TU
    Complications COMethods MTTherapy TH
    Congenital CNMicrobiology MlToxicity TO
    Contraindications CTMortality MOTransmission TM
    Cytology CYNursing NUTransplantation TR
    Deficiency DFOrganization andTrends TD
    Diagnosis DlAdministration OGUltrasonography US
    Diagnostic Use DUParasitology PSUltrastructure UL
    Diet Therapy DHPathogenicity PYUrine UR
    Drug Effects DEPathology PAUtilization UT
    Drug Therapy DTPharmacokinetics PKVeterinary VE
    Economics ECPharmacology PDVirology VI
    Education EDPhysiology PH
    Source: National Library of Medicine website (
    TABLE A1.5 Terms Used for “Publication Type” in MEDLINE
    AddressesLegal Cases [includes law review, legal case study]
    BibliographyLetter [includes letters to editor]
    BiographyMeeting Abstract
    Classical Article [for republished seminal articles]Meta-Analysis [quantitative summary combining results of independent studies]
    Clinical Conference [for reports of clinical case conferences only]Multicenter Study
    Clinical Trial [includes all types and phases of clinical trials]News [for medical or scientific news]
    Clinical Trial, Phase INewsletter Article
    Clinical Trial, Phase IIOverall [collection of articles; consider Meeting Report]
    Clinical Trial, Phase IIIPeriodical Index [for cumulated indexes to journals]
    Clinical Trial, Phase IVPractice Guideline [for specific health care guidelines]
    Congresses Controlled Clinical TrialPublished Erratum [consider Corrected and Republished Article]
    Randomized Controlled TrialRetracted Publication [article later retracted by author]
    Comment [for comment on previously published article]Retraction of Publication [author's statement of retraction]
    Consensus Development ConferenceReview [includes all reviews; consider specific types]
    Consensus Development Conference, NIHReview, Academic [comprehensive, critical, or analytical review]
    Corrected and Republished Article [consider Published Erratum]Review, Multicase [review with epidemiological applications]
    Dictionary Directory Duplicate Publication [duplication of material published elsewhere]Review of Reported Cases [review of known cases of a disease]
    Editorial Festschrift [for commemorative articles]Review Literature [general review article; consider other reviews]
    Guideline [for administrative, procedural guidelines in general]Review, Tutorial [broad review for nonspecialist or student]
    Historical Article [for articles about past events]Scientific Integrity Review [U.S. Office of Scientific Integrity reports]
    InterviewTechnical Report
    Journal Article [excludes Letter, Editorial, News, etc.]Twin Study [for studies of twins]
    Source: National Library of Medicine web sites (
    How to Obtain Access to MEDLINE

    MEDLINE is available on CD-ROM and web-based search engines. Some private companies download NLM databases and then market them on CD-ROMs, mostly to colleges and universities (e.g., Silver Platter), where closed hospital computer systems may not allow for Internet access.

    NLM offers two free web-based programs, PubMed and Internet Grateful Med (IGM) to which many websites link. Most MEDLINE vendors charge for full text retrievals and document deliveries but some (e.g., OVID), offer a majority of the articles online as full text.

    PubMed and Internet Grateful Med (IGM)

    PubMed and Internet Grateful Med (IGM) are the two main search engines provided by the NLM. Both are web-based, user-friendly, relatively quick, and are offered free of charge without registration. Both allow MEDLINE searches, and IGM has the added feature of allowing searches of most of the other NLM databases (see Table A1.2) by selecting that database on the first page. Both have an automatic MeSH mapping feature. PubMed also provides links to journal sites on the web (when available) which may allow access to the full text. PubMed's “advanced search mode” allows quick searches (by author, date, page number, journal, volume, etc.).

    Searching MEDLINE

    MEDLINE records can be searched using MeSH terms, title or text words or phrases, author name, journal name, publication type, dates, language, study subjects or any combination of these. As stated above, MeSH terms are a vocabulary of medical and scientific terms assigned to most documents in MEDLINE by the indexers at the NLM. Text terms are strictly word searches for the exact words or phrases found in the title and abstract of a document. Whenever possible, it is advantageous to use MeSH terms over text terms for several reasons:

    • MeSH terms are assigned on the theme subject of the entire document and not just the citation text (title and abstract). Thus, a search using a MeSH term can find relevant documents even when the exact term in question is not found in the citation. This feature is especially important for documents that do not have abstracts, because titles are very short and often omit important terms.
    • Some MeSH terms are assigned as Major Topic terms, meaning that the indexer has determined that these terms represent the major thrust of the document. Using the Major Topic field in MEDLINE searches can help discard documents that are less relevant to your search. These terms are designated in the citation and MEDLINE reports with an asterisk (*).
    • The use of MeSH terms and MeSH subheadings can allow searches that are very narrow and specific, and can help to speed the search process and avoid the retrieval of irrelevant material.

    Both PubMed and IGM have links to the NLM's MeSH Browser through which these vocabulary trees can be visualized, and searches focused or expanded. Automatic MeSH mapping refers to the process where words or queries entered in the search fields are automatically checked against the NLM's MeSH directory and appropriately replaced in the search. This system is not foolproof, however, and matching to a proper MeSH term is sometimes not achieved.

    Searches can be conducted by use of Boolean operators (mathematical set language) or natural language. Boolean search operators are more exact.

    Boolean Search Operators
    ANDis used for combining terms or phrases. Only those records that match with all the specific terms separated by AND are retrieved.
    The above field will result in retrieval of only those records that contain all these words in the title or the abstract. This is the most commonly used search operator being used (Hint: it happens automatically when you enter several words)
    ORSearches for records containing either of the words it separates. Using OR will lead to a higher number of articles retrieved than using AND.
    In the above, the preferred search term is unknown, therefore OR is used to conduct a search based on the alternatives.
    NOTSearches for records containing the query word preceding it without containing the word following it. That is, it discards records containing the terms after the not. For example:
    Would retrieve all articles on congestive cardiomyopathy except those pertaining to hypertrophic cardiomyopathy
    *is a wild card operator to match part of a word or phrase. Examples: micro* matches microorganism, microsome, etc. Similarly, by placing* before a word or part of a word will match all words ending with that word (e.g., *itis will match carditis, arthritis, encephalitis, etc.)

    While there are texts available with useful instruction in accessing medical information online,1,2,3 printed information about accessing the published evidence base on-line will be somewhat dated by the time it is published. The pertinent websites (e.g., links to Pubmed or Internet Grateful Med off of the National Library of Medicine home page, generally provide tutorials and/or guidance, with the advantage of remaining constantly current.

    Appendix A.2 requires internet access, and provides step-by-step guidance through several illustrative searches.

    Appendix A.2: A Walking Tour of MEDLINE

    Use of this appendix requires internet access. Log on to the National Library of Medicine ( home page, or select some alternative means to access PubMed to proceed through the illustrative searches provided below. Conduct the searches as detailed below, and refer to the screen for the content discussed. Note that the exact number of citations retrieved at each step of the searches will likely have changed over time; this will not interfere with the demonstration of search techniques.

    Search 1: Basic PubMed Search

    A 30-year-old Brazilian women presents to the emergency room with a one month history of fatigue, and a three day history of left pleuritic chest pain and dull epigastric pain. Examination reveals stable vital signs and is notable only for a slightly enlarged spleen. Laboratory results show liver enzymes to be elevated (AST 400 IU/L, ALT 350 IU/L, total bilirubin 1.8 mg/dL, alkaline phosphatase 333 IU/L, and Prothrombin time 15s). An abdominal ultrasound confirms splenomegaly and suggests hepatic fibrosis. A nuclear liver-spleen scan demonstrates portal hypertension with colloid shift. The diagnosis of autoimmune hepatitis is strongly suggested by a positive ANA titer (1:2560, homogeneous pattern).

    When this information is conveyed to the patient, she asks what the treatment options are. You conduct a search.

    Step I: Formulate a Question and a Search Plan

    In this case, the question would be

    And, as a corollary

    Initial search terms from these questions might include

    Step II: Identify Appropriate MeSH Terms
    • Log on to the NLM's Entrez-PubMed Start Page (
    • Next, click on the link to the MeSH Browser.
    • Type the search words or phrase into the “search for” field. This is where natural language is transcribed into MeSH terminology. Autoimmune hepatitis is not a MeSH term. However, Hepatitis, Autoimmune is, and it will be automatically replaced in the search. A brief definition of the term and the year it was introduced are also provided. Note also that the MeSH tree is displayed, that the “Detailed Display” button provides all of the qualifiers for the term, and that the “Add” button allows for the insertion of search operators (And, Or, Not).
    • As we are particularly interested in the treatment of autoimmune hepatitis, we select the “detailed display” link to view the qualifers.
    • From the available qualifiers, we select “drug therapy,” “therapy,” and “restrict to Major Topic headings only.” We then select the “Add” button to reformulate the search.

    Notice the shaded area at the top now includes technical information for the appropriate MeSH terms (hepatitis, autoimmune) with the appropriate qualifiers (listed as/drug therapy and/therapy) as the major topic area ([MAJR]). Select “PubMed Search” to return to the original PubMed screen with the newly adopted search terms incorporated.

    While autoimmune hepatitis therapy in natural language pulls up over 400 citations, the use of MeSH terms as detailed above yields approximately 45 articles.

    Step III: Setting Limits

    At this point, the articles retrieved can be reviewed, or the search can be further refined by applying limits. (Note that limits could have been incorporated into the search right from the beginning.) Select the “Limits” button below the search term field.

    Assign the following limits to the search: publication type (review), age (all adult: 19+ years), subject type (human), publication dates (1995–2000). Entrez dates (5 years) is an alternative to limiting on the basis of publication dates. Conduct the search.

    The limited search yields three very relevant articles. Note that the limits applied are listed at the top and that these limits can be applied all at once (as we have done) or one at a time to whittle down the number of articles retrieved.

    If at this point search results were not satisfactory (e.g., if the search retrieved no articles), there are several options. The first is to expand some of the stipulated limits (e.g., remove age limits, or expand the acceptable period of publication). Alternatively, were any articles retrieved, select the “related articles” feature to expand the search.

    Selecting the “related articles” hyperlink next to the first article of the three retrieved in the above search identifies over 100 related papers.

    At this point you can either apply limits to the search as before or continue to retrieve “related articles” to those most pertinent and useful.

    Search 2: PubMed Searching with Search Operators

    A 65-year-old woman is brought to your office by her daughter for increasing memory loss. History and examination suggest early dementia, probably of the Alzheimer type. After you reveal your suspicion to the daughter, she asks whether her mother should use ginkgo biloba. Since the patient's daughter is a nurse, she wants to know if there is any good evidence that ginkgo biloba really works. You tell her that you are unable to answer a question that specific, but promise to look into the issue and get back to her.

    Step I: Formulate a Question and a Search Plan

    Step II: Find Appropriate MeSH Terms

    Search terms: Ginkgo biloba, dementia Other related terms: cognition, memory

    Initial PubMed search of MEDLINE entering just ginkgo biloba yields some 80 articles.

    Step III: Setting Limits

    Dementia, memory and cognition are similar, but separate MeSH terms. Using each of the terms with the search operator “AND” is illustrated in Table A2.1. Conduct these searches with and without publication type limited to “clinical trial.” As shown in Table A2.1, these constraints result in the retrieval of a few highly relevant articles. (Note again that the actual retrieval may have changed by this time.)

    TABLE A2.1 Using Search Operators and Limits to Narrow a MEDLINE Search
    Search Terms UsedLimitsNumber of Articles Retrieved
    ginkgo bilobanone80
    ginkgo biloba AND dementianone15
    ginkgo biloba AND dementiaclinical trial2
    ginkgo biloba AND memorynone9
    ginkgo biloba AND memoryclinical trial1
    ginkgo biloba AND cognitionnone4
    ginkgo biloba AND cognitionclinical trial1
    Search 3: Retrieving a Particular Article

    Often, a citation in a lecture or discussion will be of interest. Equally often, the information provided, particularly in the context of informal discussion or clinical rounds, will be incomplete. For example, one might hear a comment such as, “Well of course that's true given that famous actual causes of death paper…”

    When interested in retrieving an article alluded to in such a way, the most expedient method is to use the “Single Citation Matcher” of PubMed. Select the “Single Citation Matcher” along the left side of the PubMed Services on the PubMed homepage ( Use of this feature requires that some specific information about the article be available, e.g., the year, journal and an author.

    If the necessary identifying information to use the “Single Citation Matcher” is unavailable, we can limit the search to match text terms in the title. Select limits and change the “All Fields” to “Title Text.” Search results based on progressive inclusion of more of the terms from the “allusion” above are shown in Table A2.2.

    TABLE A2.2 Progressive Text Term Search by Title Text
    Search Terms AddedLimitsNumber of Articles Retrieved
    Actualtitle text1504
    Actual causestitle text9
    Actual causes of deathtitle text6
    Actual causes of death in the United Statestitle text6

    *See Figure A2.1.

    Other known terms listed under “All Fields” can also be used to perform focused searching of partial citations (for example, by author or journal name). PubMed offers specific selections on the start page which allow you to “Enter one or more search terms,” “Enter author names,” or “Enter journal titles.”

    Note that every citation in MEDLINE is labeled with a MEDLINE Unique Identifier (UI) number, generally listed at the end of a citation. Some libraries may require this number, or the PubMed identification number (the PMID) to locate the citation. An example of a typical MEDLINE citation providing these identifiers is shown in Figure A2.1.

    Figure A2.1. Use of MEDLINE Ul number and PubMed ID.

    By any of several approaches to the search, the article of interest (McGinnis JM, Foege WH. Actual causes of death in the United States. JAMA. 1993;270:2207–2212) can be retrieved.

    Appendix A.3: Publication Bias: The Limits of Accessible Evidence

    Any search of the literature, however sophisticated, is only as good as the literature it is searching. One of the principal concerns about the quality of the published literature is its susceptibility to publication bias. Publication bias is the preferential publication of studies with significantly positive findings, as compared to those with negative (nonsignificant) findings.4

    There are two interpretations of bias that are germane to this topic. The first, referred to as the use of “bias” in the vernacular, implies preference. It is perhaps understandable that editors and reviewers might prefer studies that demonstrate an association over those that fail to do so. The second interpretation, more related to customary use of “bias” in medicine, is the introduction of a systematic distortion from the truth. The published literature is biased if it distorts reality.

    There are two ways in which the literature could distort reality—by describing associations that do not truly exist, or refuting those that do. The first represents false-positive error, the second false-negative error. To some degree, the susceptibility of the literature to systematic distortions resides in the conventional thresholds of tolerance for type I (false-positive) and type II (false-negative) error.

    Prevailing convention stipulates that alpha, the threshold for type I error, be set at 0.05, and beta, the threshold for type II error, at 0.2. Thus, when these conventions are honored, there is a 1 in 5 chance that any negative study is misleading, and a 1 in 20 chance that any positive study is misleading. As systematic distortion from the truth requires consideration of what constitutes truth, some conjecture about the “unbiased” medical literature is, at least theoretically, prerequisite to the assessment of publication bias. If all studies performed, regardless of outcome, were equally likely to be submitted for publication, one could posit that the unbiased literature should include 4 positive studies for every negative one, so that the probability of a false conclusion were equal between positive and negative reports.

    There are problems with this position, however. Almost certainly, the probability of submission does vary with outcome, so that the journals are less likely to see negative studies.5,6,7 This distorts the denominator of papers reviewed, and alters the ratio of publication needed to achieve unbiased representation. Further, if 4 positive papers were published for every negative paper (so that the probability of a single misleading report in either category were the same), the weight of published evidence would still favor the positive outcome (i.e., there would be a preponderance of positive papers in the literature, even with no bias). There would, as well, be no way to know which positive outcome were true, and which a fluke.

    This problem might be addressed if no paper were published without considering the ratio of positive to negative papers assessing the same outcome or association. One could only be confident of a true positive association when the positive papers outnumbered the negative by more than 4 to 1, and confident of the converse when negative papers occurred more often than 1 for each 4 positive papers. This approach would require systematic submission of all completed studies to a single evaluator who would see “the big picture” and publish the pattern of truth that emerged. However, this approach is not remotely feasible. And, extreme as it is, it is also flawed. Negative studies might be less likely than positive studies to be conducted to completion. Also compounding the situation is actual results that are well beyond the stipulated threshold for tolerable error. For example, p-values much lower than an alpha of 0.05 denote a risk of false-positive much less than 5%.

    While publication bias refers to the preferential publication of positive studies, there is a distortion of the literature when studies purport to be negative but are perhaps not. This occurs because many studies, particularly prior to the 1990's, did not address adequately the issue of power. A negative outcome form an under-powered study is not truly negative; rather, it is as if a study were simply not done. When such findings are published, they suggest a conclusion about lack of association that is not justified.8,9,10,11,12,13

    It is almost certainly true that there are distortions from “the truth” in the published literature. What they are, and how to detect, judge, or measure them is unclear. An appreciation for the limitations of even the most diligent search of the published evidence is useful for, if nothing else, conferring the humility evidence-based practice requires. Even if evidence is available, even if it can be found, even if it can be interpreted, even if it seems conclusive, and even if it pertains to the clinical need at hand, it may not reflect the truth, the whole truth, and nothing but the truth of association. To appreciate and apply evidence is to appreciate and respect its many limitations.

    BemmelJ, Van BemmelJ, MusenMA (eds). Handbook of Medical Informatics. Springer Verlag; 1997.
    SmithRP, EdwardsMJA.The Internet for Physicians (Book with CD-ROM). Springer Verlag; 1999.
    BernerES, BallMJ (eds). Clinical Decision Support Systems: Theory and Practice. Springer Verlag; 1998.
    PetittiDB.Meta-Analysis, Decision Analysis, and Cost-Effectiveness Analysis. Methods for Quantitative Synthesis in Medicine.
    2nd ed
    . New York: Oxford University Press; 2000.
    SuttonAT, DuvalSJ, TweedieRL, AbramsKR, JonesDR.Empirical assessment of effect of publication bias on meta-analyses. BMJ. 2000;320:1574–1577.
    ThorntonA, LeeP.Publication bias in meta-analysis: Its causes and consequences. J Clin Epidemiol. 2000;53:207–216.
    VickersA, GoyalN, HarlandR, ReesR.Do certain countries produce only positive results? A systematic review of controlled trials. Control Clin Trials. 1998;19:159–166.
    OttenbacherKJ, MaasF.How to detect effects: Statistical power and evidence-based practice in occupational therapy research. Am J Occup Ther. 1999;53:181–188.
    MengelMB, DavisAB.The statistical power of family practice research. Fam Pract Res J. 1993;13:105–111.
    RossiJS.Statistical power of psychological research: What have we gained in 20 years?J Consult Clin Psychol. 1990;58:646–656.
    MittendorfR, ArunV, SapugayAM.The problem of the type II statistical error. Obstet Gynecol. 1995;86:857–859.
    FoxN, MathersN.Empowering research: Statistical power in general practice research. Fam Pract. 1997;14:324–329.
    WilliamsTL, HathawayCA, KlosterKL, LayneBH.Low power, type II errors, and other statistical problems in recent cardiovascular research. Am J Physiol. 1997;273(1 Pt2):H487–H493.
    LoweHJ, BarnettGO. Understanding and using the medical subject heading (MeSH) vocabulary to perform literature searches. JAMA. 1994;271:1103–1108. NLM URL: Internet URL:

    Appendix B: The Constraint of Resource Limitations

    Considering Cost in Clinical Practice

    In applying quantitative methods and a population-based perspective to patient care, no factor is more naturally, yet less comfortably, drawn into consideration than cost. Cost is an unavoidable element in both clinical practice and public health,1 and a convenient quantifier of efficiency, and to some degree even quality of care.2,3 Yet any notion of equating pain and suffering, living and dying, with dollars causes the Hippocratic hairs on a clinician's neck to lift in suspicion and disdain.

    One might wish that we were at liberty to ignore cost, that resources were unlimited. If considerations of cost were truly damaging to clinical care, to devotion to quality, then it's elimination from clinical decision making would be a good thing. But is that so?

    Consider a situation in which all health care is free. Not just free to a particular consumer or payer, but actually free, due to limitless resources. In such a context, a patient with a poor clinical response could, and perhaps should, be sent to every relevant expert, receive every relevant therapy. Yet what if the probability of a response from each successive referral, each successive treatment, were diminished relative to the last? With unlimited resources, a declining probability of benefit would not, in and of itself, be cause to withhold any service. Perhaps the only reason to do so would be increasing risk from those very services.

    But if the disease in question were serious, the risk of nonintervention could be so high that the risk of any intervention might be justified. But what of effectiveness?4 In an environment of unlimited resources, effectiveness, the reliability with which a given intervention produces the intended effect under real-world conditions, would be in the eyes of the beholder. Because it is almost impossible to prove conclusively that something does not work, there is always some small chance, even if it has never worked before, that it could. In situations of clinical desperation, one might be inclined to try something—anything—no matter how nominal the chances of benefit. At some point in this approach to care, unsound interventions will be brought into practice. There will be risk imposed without the justifying concurrence of potential benefit. And the ethical principals underlying medical practice will be violated.

    The notion that cost could be the arbiter of medical ethics may seem far-fetched, but some currency, some medium is required so that the relative effectiveness of treatments can be compared.1 Alternatives to dollars could be considered. Cost could be measured in terms of risk or time or inconvenience, but dollars tend to be a surrogate for these. Benefit could be measured in functional status or survival or quality of life or quality-adjusted life years. But both the “costs” of medical interventions, and the attendant benefits, need to be measured in some way so that the relative merits of alternative interventions are comparable. In the real world, where resources are in fact limited, the dollar costs of procedures and therapies become a convenient, and appropriate, basis for such comparisons.2

    Once the notion that medical practice standards require a measure of cost becomes acceptable, the distinction between public and private cost arises. Cost is of profound importance to public health, where the impact of resource limitations is felt acutely.5 In the care of the individual patient, resource limitations are generally an abstraction in the US health care system where money to pay for interventions is the only medium likely to be in short supply. Yet the aggregated costs of individual patient care become the costs of caring for the public, and practice patterns in the aggregate have an enormous impact on expenditures. Ultimately, interventions of great potential benefit, both to individuals and the public, are abandoned due to resource limitations.

    Cost-Benefit Analysis

    The simplest and generally least acceptable means of assessing cost tradeoffs in clinical practice is cost-benefit analysis. The objectionable element in such an approach is the need to measure both cost and benefit in dollars. Thus, if the benefit expected is clinical, such as symptom resolution or extension of life, a translation of the benefit into dollar terms would be required to determine the cost-benefit implications of the intervention. Cost-benefit analysis is used to determine if an intervention will save or cost money.

    There are situations in which a cost-benefit analysis makes sense. In an occupational setting, there is likely to be a strong interest in overall profitability. Health-related interventions for employees will be fine up to a point, but beyond a certain point they would be prohibited by cost and the erosion of profit. But if an intervention, such as stress reduction counseling for employees, translated into fewer days missed and greater productivity, the costs of the intervention might be more than offset by the resultant gains. In such a situation, consideration of cost might lead to interventions that would otherwise never be made available.

    Another situation in which cost considerations can enhance services is prevention. Clinical preventive services all cost something. But in many instances, the cost of not providing them is higher still. An example is provided by a cost-benefit analysis of influenza vaccination in the occupational setting.6 When a clinical benefit is associated with an actual and measurable reduction in costs, a compelling argument is advanced for the provision of new services.

    Cost-Effectiveness Analysis

    Often, dollar savings from an intervention are not expected. A clear example is renal dialysis.7 The intervention is life prolonging, even life saving, but costly. These costs are sustained and accumulated over time, with little to no economic return on the investment. At the same time, the value of such costly clinical interventions is clear in human terms: reduced suffering, increased survival.

    One means of comparing financial cost to clinical benefit is to assess cost-effectiveness. Cost-effectiveness considers the cost in dollars per pertinent outcome, such as life saved or illness prevented. Such an analysis is undertaken when the clinical outcome has been established as a social priority. Cost-effectiveness permits comparisons among alternative strategies to determine which achieves the desired outcome at the lowest cost.8 The greatest generation of pertinent outcomes for each dollar spent, or alternatively the methods leading to the least dollars spent per unit outcome achieved, are considered the most cost-effective. Such an analysis does not require that clinical endpoints be cast in dollar terms, nor that an intervention can produce savings, but merely that its costs be justified by the magnitude of its clinical benefit.

    Of note, cost-effectiveness and quality measures often, although not always, correlate well. Poor clinical outcomes and complications of suboptimal care are costly. The efficient achievement of clinical recovery through optimal use of available resources tends to be both desirable professionally and cost-effective. The lowest cost means of achieving a clinical endpoint is apt to be the most direct, most efficient, most reliable means, and as such clinically and financially preferable to the alternatives.

    Standards of cost-effectiveness are contextually defined and bound. In the US, interventions with costs in the range of $50,000 per year of life extended are widely supported. Costlier interventions are generally considered unattractive.9 Programs are adopted by the World Health Organization (WHO) for developing countries only with evidence of dramatically greater cost-effectiveness, in the range of $10 per year of life saved.10 Obviously the adoption of a global standard of cost-effectiveness in health care delivery would require massive reallocations of resources.

    Cost-Utility Analysis

    A variation on the concept of cost-effectiveness is cost utility. Whereas cost-effectiveness assumes that the clinical outcome is constant and seeks to demonstrate the least costly means of achieving it, a utility measure permits the outcome side of the formula to vary as well. The prevailing utility measure is a composite of both quality and length of life gained, or quality adjusted life years (QALY's).3,11 Typically, a year of perfect health is expressed as 1, with reductions in either time or the degree of intact health yielding portions of a quality adjusted life year (QALY). For example, living 10 years with a 50% reduction in quality of life due to a disability would result in a QALY measure of 5. Living 5 years in perfect health would yield the same measure, as would living 20 years with such severe functional limitations that the quality of life were reduced by 75% (to “0.25” the quality of intact health). Dollars spent per QALY gained is a measure of cost utility, allowing variation in both costs and benefits to be considered without specifying a dollar value for those benefits.

    Marginal Costs and Benefits

    The relevance of cost considerations to clinical decision making is mediated by the influence of marginal cost and benefit, or utility. That some additional benefit might result from some additional expenditure is often the case, but once conventional practices have been applied, both the probability and magnitude of additional benefit from additional spending tend to wane. Thus, the benefit at the margin of usual practice, or the marginal benefit, will decline with the extent of the evaluation or treatment. Often it is in just such situations that the costs per additional intervention rise steeply—as the more extravagant the management plan, the more costly its components. As noted above, failure to consider cost at all can result in an inattention to the imbalance between marginal costs and utilities, so that very expensive interventions of little potential value are applied as routinely as low cost interventions of high potential value. In situations where cost and potential benefit rise together, or fall together, considerations of cost-effectiveness are required to determine how to proceed. In situations where costs rise and marginal utilities fall, the intervention is apt to be of questionable merit. But when marginal utility rises as costs fall, there is a clear indication to proceed.

    The tension between costs and utilities can be captured in a 2 × 2 table used to portray “cost-consequence space,” as shown in Box B.1.

    Other Considerations

    Among other important considerations in cost evaluations are perspective, externalities, discounting, and projections. Perspective refers to the group (or individual) for whom a particular cost (or benefit) is relevant. For example, society (or a given population) might benefit if a pharmaceutical company made a drug or vaccine available free in a developing country, while the company (or its consumers) would bear the cost. The trade-offs between cost and benefit would be quite different depending on whose perspective were considered. Externalities refer to costs or benefits that occur as an indirect result of a particular intervention, often affecting a different individual or group. For example, if the care of a premature baby in the neonatal intensive care unit requires one or both parents to miss work, the lost productivity is an additional, indirect, or external cost. Discounting refers to the erosion of monetary benefits or costs that are postponed until some future time. Projections in economic analysis are an effort to anticipate the financial impact of time trends; Markov models (see Chapter 10) are used for this purpose.

    Economic analysis in medicine and health care is far too expansive a topic to receive comprehensive treatment here. The intent of this appendix is to highlight the importance of cost considerations in evidence-based practice and to provide examples of application. The reader interested in a thorough discussion of the subject should consult other sources.12

    BrownGC, SharmaS, BrownMM, GarrettS. Evidence-based medicine and cost-effectiveness. J Health Care Finance. 1999;26:14–23.
    LarsonEB. Evidence-based medicine: Is translating evidence into practice a solution to the cost-quality challenges facing medicine?Jt Comm J Qual Improv. 1999;25:480–485.
    EarleCC, ChapmanRH, BakerCS, et al. Systematic overview of cost-utility assessments in oncology. J Clin Oncol. 2000;18:3302–3317.
    ShawLJ, MillerDD. Defining quality health care with outcomes assessment while achieving economic value. Top Health Inf Manage. 2000;20:44–54.
    BergerML. The once and future application of cost-effectiveness analysis. Jt Comm J Qual Improv. 1999;25:455–461.
    BurckelE, AshrafT, de Sousa FilhoJP, et al. Economic impact of providing workplace influenza vaccination. A model and case study application at a Brazilian pharma-chemical company. Pharmacoeconomics. 1999;16(5 Pt 2):563–576.
    MannsBJ, TaubKJ, DonaldsonC. Economic evaluation and end-stage renal disease: From basics to bedside. Am J Kidney Dis. 2000;36:12–28.
    McCabeC, DixonS. Testing the validity of cost-effectiveness models. Pharmacoeconomics. 2000;17:501–513.
    NewbyLK, EisensteinEL, CaliffRM, et al. Cost effectiveness of early discharge after uncomplicated acute myocardial infarction. N Engl J Med. 2000;342:749–755.
    HinmanAR. Economic aspects of vaccines and immunizations. C R Acad Sci III. 1999;322:989–994.
    VijanS, HoferTP, HaywardRA. Cost-utility analysis of screening intervals for diabetic retinopathy in patients with type 2 diabetes mellitus. JAMA. 2000;283:889–896.
    PetittiDB. Meta-Analysis, Decision Analysis, and Cost-Effectiveness Analysis. Methods for Quantitative Synthesis in Medicine.
    2nd ed
    . New York: Oxford University Press; 2000.

    Appendix C: Clinically Useful Measures Derived from the 2 × 2 Contingency Table

    a =true-positive cases
    b =false-positive cases
    c =false-negative cases
    d =true-negative cases
    (a + b) =all test-positives
    (a + c) =all disease-positives
    (b + d) =all disease-negatives
    (c + d) =all test-negatives
    (a + b + c + d) =sample size
    (a + c)/(a + b + c + d) =prevalence
    (b + d)/(a + b + c + d)=proportion disease free (1 − prevalence)
    a/(a + c) =sensitivity (the proportion1of those with disease correctly detected by the test)
    d/(b + d) =specificity (the proportion1 of those without disease correctly identified as disease-free by the test)
    c/(a + c) =false-negative error rate (the proportion1 of those with disease with a negative test result; 1 − sensitivity)
    b/(b + d) =false-positive error rate (the proportion1 of those truly disease-free with a positive test result; 1 − specificity)
    a/[a + b) =positive predictive value (PPV) (the proportion1 of those with a positive test who truly have disease)
    b/(a + b) =proportion misleading positives2 (PMP) (the proportion1 of positive test results that are incorrect; 1 − PPV)
    d/(c + d) =negative predictive value (NPV) (the proportion1 of those with a negative test result who are truly disease-free)
    c/(c + d) =proportion misleading negatives2 (PMN) (the proportion1 of negative test results that are incorrect; 1 − NPV)
    b/a =false-positive index2 (FPI) (the ratio of false- to true-positives)
    c/d =false-negative index2 (FNI) (the ratio of false- to true-negatives)
    (a/c)/(b/d) =odds ratio
    [a/[a + b)]/[c/(c + d)] =risk ratio (relative risk)

    Measures influenced by prevalence: positive predictive value; negative predictive value; proportion misleading positives; proportion misleading negatives; false positive index; false negative index

    1 While identified as a proportion, the measure is typically expressed as a percentage.

    2 Newly established term/measure.

    Specific Measures Are More Readily Recalled if the Denominator Is Considered First
    disease positives (a + c)sensitivity false-negative error rate
    disease negatives (b + d)specificity false-positive error rate
    test positives (a + b)positive predictive value proportion misleading positives
    test negatives (c + d)negative predictive value proportion misleading negatives


    accuracythe tendency for a test to approximate, on average, the truth
    ACESactive control equivalence study
    Active Control Equivalence Studya study in which a novel therapy is compared to established therapy with statistical methods adapted to demonstrate lack of difference
    Actuarial methodan approach to survival analysis in which time intervals are fixed in advance (see Kaplan-Meier)
    adherencethe degree to which study subjects comply with the treatments to which they are assigned; compliance
    alpha errorfalse-positive error; also referred to as type I error
    alternative hypothesisin conventional hypothesis testing, the assertion of the association of interest
    analyze as randomizeda description applied to intention-to-treat analysis of data from randomized trials; outcome is assessed in groups based on treatment assignment rather than adherence to the assigned treatment
    associationthe appearance of a meaningful (i.e., cause and effect) relationship between variables
    attributable riskthe absolute risk ascribable to a particular exposure or factor; the difference between the risk in the exposed and the risk in the unexposed
    attributable risk percentthe attributable risk expressed as a percentage; the difference between the risk in the exposed and the risk in the unexposed, divided by the risk in the exposed, multiplied by 100
    benefit-to-risk ratioa measure of the ratio of probable benefit to potential risk useful in deciding whether or not a particular clinical intervention is advisable; useful in establishing the therapeutic threshold, 1/(B: R + 1), where B is benefit and R is risk measured in the same units
    beta errorfalse-negative error; also referred to as type II error
    biassystematic distortion from the true
    binary datadichotomous data; data with only two possible values (e.g., yes/no)
    bivariate analysisstatistical analysis of the relationship between a single independent and single dependent variable
    blindingconcealing treatment status from subject and/or investigator (referred to as double-blinding when concealed from both) in a trial
    case reporta detailed description of a single, unusual case
    case seriesa detailed description of several related cases
    case-control studya study in which groups are assembled on the basis of whether they do (cases) or do not (controls) have the outcome of interest, and are then assessed for differences in exposure
    case-findinguse of clinical tests to uncover occult disease or risk factors in individuals undergoing medical care/evaluation (see screening)
    causalitya cause and effect relationship between independent and dependent variables, the establishment of which requires the satisfaction of Mill's canons or Koch's postulates
    Central limit theoremthe assertion that for relatively large samples (i.e., n > 30), the distribution of the means of many such samples will be normally distributed even if the distribution for an individual sample is non-normal
    cohort studya study in which groups are assembled on the basis of exposure status and then assessed for differences in outcome
    compliancethe degree to which study subjects adhere to assigned treatments; adherence
    confidence interval (95%)a means of conveying not only statistical significance, but the actual range of probable outcome values if a test were to be repeated; the actual outcome measure observed ±1.96 standard errors
    confoundera third variable linked to both putative cause and putative effect that creates the appearance of an association when there is none (positive confounding) or the appearance of no association when there is one (negative confounding)
    confoundingthe creation, by a third variable linked to both putative cause and effect variables, of the appearance of an association when there is none (positive confounding) or the appearance of no association when there is one (negative confounding)
    contingency tablea table that displays the outcome values of one dichotomous or ordinal variable relative to (contingent upon) the outcomes of a second dichotomous or ordinal variable
    continuous datadata with values uninterrupted across a given range; e.g., temperature
    correlationan expression of the degree to which movement in one variable influences/predicts movement in a second variable
    Cox proportional hazards modelinga method of multivariable modeling for survival analysis data
    critical ratiothe ratio of outcome effect (signal) to variance (noise) in hypothesis testing; the ratio is critical in that it determines the statistical significance
    cross-sectional studya study in which, often by survey, measures of outcome and exposure status are obtained at the same time
    cut-off pointthe designated value that distinguishes normal from abnormal results of a test, or the presence from the absence of disease
    data dredgingan unflattering term applied to the use of a large data set to test for multiple associations without appropriate statistical protection against chance findings
    decision analysisa formalized approach to making complex medical decisions that relies on plotting (in a “tree”) the alternatives and rating each in terms of probability and utility
    decision treethe plot of clinical options and the associated probabilities and utilities used in a decision analysis
    degrees of freedomused in various tests of statistical significance, a measure of the number of observations in a data set that contribute to random variation
    dependent variableoutcome variable
    detection biaswhen differential rates of disease or condition detection between groups are attributable to differing levels of investigation or scrutiny between groups
    dichotomoushaving two possible values (e.g., yes/no)
    dichotomous datadata that are expressed with two possible values
    double-blindthe concealment of treatment status from both subject and investigator in a trial
    ecological fallacythe inference that an association in a population is an association within individual members of that population when in fact it is not
    ecological studya study in which exposure and outcome variables are assessed at a population rather than individual level, often by use of vital statistics
    effect modificationwhen a third variable, linked to both putative cause and effect variables, changes the relationship between them
    effect modifierthe variable that produces effect modification (see above)
    effectivenessthe influence of treatment on outcome under real-world circumstances
    efficacythe influence of treatment on outcome under ideal, or near ideal, circumstances
    expected valuethe distribution of values that would occur by chance under the conditions of the null hypothesis (i.e., lack of association)
    external validitygeneralizability; the degree to which study outcomes pertain to other individuals or populations
    false-negative error ratethe number of negative test results divided by the total number of (truly) positive cases [c/(a + c)]
    false-negative indexthe ratio of false to true negatives (c/d)
    false-positive error ratethe number of positive test results divided by the total number of (truly) negative cases [b/[b + d)]
    false-positive indexthe ratio of false- to true-positives (b/a)
    frequency distributiona plot of the values of a particular variable against the frequency of occurrence of each
    generalizabilityexternal validity; the degree to which study outcomes pertain to other individuals or populations
    hard measurean outcome measure not considered subjective or open to interpretation
    Hawthorne effectattribution of some or all of the observed outcome to differential levels of contact with the investigators or their associates between groups, rather than to the specific intervention per se; controlled for by establishing comparable levels of “contact” for all study groups
    hypothesisassertion of an association believed, but not known, to be true
    hypothesis testinga statistical approach to confirming or refuting a hypothesis
    incidence ratethe number of incident cases during a defined time period, divided by the population at risk at the midpoint of that period
    incidencethe number of new cases of a condition of interest in a defined population during a specified period of time
    independent variablepredictor or causal variable
    individualized benefit indexNNH/NNT when NNH is the larger number (also established by dividing the absolute risk reduction by the absolute risk increase); a measure of the number of patients helped by an intervention for every one harmed
    individualized harm indexNNT over NNH when NNT is the larger number (also established by dividing the absolute risk increase by the absolute risk reduction); a measure of the number of patients harmed by an intervention for every one helped
    intention-to-treat analysisthe analysis of outcomes in the groups derived from a randomization process, regardless of whether subjects in each group actually adhered to the assigned treatment (also referred to as “analyzing as randomized”)
    internal validitythe degree to which a study reliably measures the association it purports to measure without bias
    interval dataordinal data for which the distance between consecutive values is consistent
    intervention triala study in which one group of subjects receives a treatment or procedure
    Kaplan-Meier methoda method of survival analysis in which intervals of observation are determined by the timing of death (or outcome) as opposed to the fixed intervals of the actuarial method
    Koch's postulatesa set of conditions defining causality, most applicable to infectious diseases
    lead time biasthe apparent increase in survival time after diagnosis resulting from earlier time of diagnosis rather than later time of death
    length biasthe tendency, in a population screening effort, to detect preferentially the longer, more indolent cases of any particular disease
    life table methodsanalytical methods for survival analysis
    likelihood ratio (grand)the ratio of the likelihood ratio positive to the likelihood ratio negative; equivalent to the odds ratio; a measure of how much more likely it is that a positive test result is true than a negative result false; provides a measure of test reliability that is independent of disease prevalence
    likelihood ratio negativethe ratio of the false-negative error rate to specificity; a measure of how likely a false-negative test result is as compared to a true-negative result
    likelihood ratio positivethe ratio of sensitivity to the false-positive error rate; a measure of how likely a true-positive test result is as compared to a false-positive result
    linear regressiona method of statistical analysis that can be used to determine the amount of movement in a dependent variable expected for a given movement in an independent variable (e.g., the amount of increase in HDL cholesterol for hour of exercise per week)
    log-rank testa method of statistical significance testing for survival analysis
    major topica heading indicating major divisions within the MeSH language (see Medical subject heading (MeSH) term)
    meanthe average value of a sample
    measurement biaswhen differences in the finding of interest between groups are due to systematic differences in methods applied to the measurement of the finding
    measures of central tendencythose measures that characterize the clustering of observations in a data set, such as mean, median, and mode
    medianthe middle value in a set of observations arranged in ascending order
    medical subject heading (MeSH) termdesignations in a language developed by the National Library of Medicine; used to define the scope of an online search in MEDLINE or related services
    meta-analysisquantitative synthesis of the results of multiple smaller studies into a single analysis
    modethe one or more most frequently occurring values in a data set
    multivariate analysisa term typically used to denote the analysis of multiple predictor variables in relation to a single outcome variable; technically, multivariable analysis denotes multiple predictors and single outcome, while multivariate denotes both multiple predictors and outcome variables
    necessary causean exposure that must occur if the outcome is to occur; may or may not be sufficient (see Sufficient cause)
    negative confoundingwhen a confounding variable obscures a true causal relationship between two other variables
    negative predictive valuethe proportion of those with negative test results that is truly free of disease [d/(c + d)]
    nominal datadiscontinuous (categorical data) with no implied direction (e.g., blood types)
    nonparametric datadata that cannot be characterized by a mean and standard deviation (i.e., parameters)
    nonparametric methodsmethods of statistical analysis designed for nonparametric data, based largely on ranking the outcome data
    null hypothesisthe stipulation of no association that is the conventional basis for hypothesis and significance testing; statistical significance is predicated on finding evidence that allows for the rejection of the null hypothesis with a specified degree of confidence
    number needed to harmthe number of patients that need to be exposed to a procedure with a defined risk of a particular adverse effect before, on average, one individual experiences the adverse effect; 1/ARI, where ARI is absolute risk increase
    number needed to treatthe number of patients that need to be exposed to a procedure with a defined probability of a particular beneficial effect before, on average, one individual experiences the beneficial effect; 1/ARR, where ARR is absolute risk reduction
    observational cohort studya cohort study in which no intervention is performed; subjects with differing exposures are simply observed with regard to outcome
    odds ratiooften used as a measure of outcome in case-control studies; the relative odds of exposure in those with to those without the disease/outcome; [(a/c)/(b/d)]
    one-tailed testjustified when the direction in which an outcome will deviate from the null is predictable based on prior study, an approach to hypothesis testing that places the entire rejection region (or tail) of the hypothetical plot of sample means to one side
    operating characteristicsa term often used to denote the performance measures of a diagnostic test, such as sensitivity and specificity
    ordinal datadiscontinuous (categorical) data with implied direction (e.g., cardiac murmurs graded from I to VI)
    parametric datadata for which the frequency distribution is char-acterizable by mean and standard deviation (parameters)
    parametric methodsmethods of statistical analysis limited to parametric data
    Pearson correlation coefficientdenoted by r, a measure of the degree to which movement in one continuous variable corresponds with movement in a second variable
    performance characteristicssee “operating characteristics”
    population attributable riskthe degree to which the risk of a particular outcome in a population is ascribable to a particular factor; risk in the population, minus the risk in the unexposed
    population attributable risk percentthe percent of total population risk for a particular outcome ascribable to a particular factor; risk in the population, minus risk in the exposed, divided by the risk in the population, then multiplied by 100
    positive confoundingwhen a confounding variable creates the appearance of an association between two variables unassociated in reality
    positive predictive valuethe proportion of those with a positive test result that truly has the disease; [a/(a + b)]
    posterior probabilitythe probability of disease following the performance of a diagnostic test
    post-test oddsthe odds of disease following the performance of a diagnostic test
    post-test probabilitysee posterior probability
    powerthe complement of β, the degree of false-negative error (1 − β); the probability that an outcome effect will be detected if it exists
    precisionthe degree to which a test produces the same or similar results when repeated; reproducibility
    pretest oddsthe odds of disease before the performance of a given diagnostic test
    pretest probabilitythe probability of disease before the performance of a given diagnostic test
    prevalencethe number (or proportion) of individuals in a defined population with a condition of interest at a specified time
    primum non nocerefirst, do no harm
    prior probabilitythe probability of disease before the performance of a given diagnostic test
    proportion misleading negativesthe proportion of those with negative test results that truly has disease
    proportion misleading positivesthe proportion of those with positive test results that is truly disease-free
    prospective cohort studya cohort study in which subjects are followed from the present for a future outcome event
    publication biasthe tendency of studies with positive (significant) outcomes to be published preferentially relative to those with negative outcomes; presupposes an appropriate or “unbiased” pattern of publication that has not been clearly defined
    publication typespecifies the type of article to be retrieved in a MEDLINE search
    qualifiersterms used to narrow the scope of a MEDLINE search
    qualitative meta-analysisa systematic review of the literature on a give topic without synthesis of the data from the multiple individual trials
    quantitative meta-analysisa systematic review of the literature on a give topic followed by synthesis of the data from the multiple individual trials selected on the basis of formal evaluation and abstraction methods
    randomizationthe allocation of subjects to treatment assignments on the basis of chance by use of any of several methods, most commonly a computer algorithm
    randomized clinical trialconsidered the “gold standard” in the clinical study of humans, a trial in which subjects are randomly assigned to treatment groups, and in which, typically, there is a placebo control arm, and blinding of both subjects and investigators to treatment status (double-blinding)
    ratio datacontinuous data with a true 0 point for reference (e.g., the Celsius temperature scale)
    RCTthe standard abbreviation for randomized clinical trial
    recall biasdifferential recollection of exposure status as a result of different outcome status among subjects in a case-control study
    regression to the meansee Statistical regression effect
    rejection regionthe portion of the hypothetical distribution of trial outcomes that corresponds to alpha and represents a low enough probability of false positivity to permit rejection of the null hypothesis
    relative riskalso known as the risk ratio, and frequently used as a measure of outcome in cohort studies, the ratio of risk of the outcome in those with to those without the exposure; [{a/(a + b)}/{c/(c + d)}]
    reliabilitythe degree to which repeated measures of the same phenomenon correspond; reproducibility; precision
    reproducibilitythe degree to which repeated measures of the same phenomenon correspond; reliability
    responsivenessthe tendency of a test result to vary with changes in the status of the patient
    retrospective cohort studya cohort study in which groups are assembled on the basis of exposure status at some time in the past, then followed through the past for the occurrence of outcome
    risk differencerisk in the exposed minus risk in the unexposed
    risk ratiosee Relative risk
    sampling biasinclusion in a study of subjects not fully representative of the underlying population
    screeningefforts directed at detecting occult disease in some at-risk portion of the general population not otherwise actively under medical care or evaluation (as opposed to case finding)
    selection biasalso known as allocation bias, when there is differential assignment of subjects to treatment groups based on subject characteristics
    sensitivitythe capacity of a test to detect disease when it is truly present; [a/(a + c)]
    signal to noise ratioa descriptive characterization of the critical ratio used in hypothesis testing
    single-blindthe concealment of treatment status from subjects, but not investigators, in an intervention study
    soft measurea subjective outcome variable open to interpretation (e.g., level of contentment)
    specificitythe capacity of a test to exclude disease when it is truly absent; [d/(b + d)]
    spectrum biaswhen a test performs differently among patients with differing levels/stages of a disease
    standard deviationa measure of the average dispersion of observations in a sample from the sample mean; Sqrt where Sqrt is the square root, Xi is the observation value associated with the ith observation, is the mean of the sample, and n is the sample size; the square root of the variance; designated as s
    standard errorthe standard deviation (see above) divided by the square root of the sample size
    statistical regression effect (regression to the mean)the tendency, on the basis of random variation, for extreme test values to shift toward the population mean when repeated
    statistical significancethe characterization of a study outcome as less probably attributable to chance than the prespecified limit of risk for false-positive error (alpha); typically taken to mean that an apparent association is “real”
    sufficient causean exposure that will inevitably lead to the outcome; may or may not be necessary (see necessary cause)
    sum of squaresa measure used to characterize the dispersion of observations about the mean of a sample; where xi is the observation value associated with the ith observation, and is the mean of the sample
    survival analysisa method of analysis applied to studies with a dichotomous outcome measure, such as survival/death, that accounts for differences between groups in the distribution of the outcome over time
    therapeutic thresholdthe minimal ratio of probable benefit to risk that justifies an intervention; 1/(B : R + 1), where B is benefit and R is risk measured in the same units
    treatment biasdifferential care of subjects based on their group assignment other than with regard to the intervention of interest
    two-tailed testthe conventional approach to hypothesis testing in which the rejection region of the hypothetical distribution of trial outcomes is divided between the lower and upper tails of the curve
    type I errorfalse-positive error; alpha
    type II errorfalse-negative error; beta
    under-powereda study unlikely (typically due to inadequate sample size) to detect a meaningful outcome effect even when there truly is one
    univariate analysisthe initial step in data analysis, the characterization of variables with regard to the range and distribution of values, one at a time
    URLthe standard abbreviation for uniform resource locator, the typical designation of an “address” on the World Wide Web
    validitysynonymous with accuracy; a measure of the degree to which the results reflect the truth
    variancea measure of the average dispersion of observations in a sample from the sample mean; where xi is the observation value associated with the ith observation, is the mean of the sample, and n is the sample size; the square of the standard deviation; designated as s2
    variationthe dispersion of observations in a sample
    Z-valuethe distance of an observation from the sample mean in units of standard deviations

    Text Sources

    Dawson-SaundersB, TrappRG. Basic & Clinical Biostatistics.
    2nd ed
    . Norwalk, CT: Appleton & Lange; 1994.
    EddyDM. Clinical Decision Making. From Theory to Practice. Sudbury, MA: Jones and Bartlett Publishers; 1996.
    FletcherRH, FletcherSW, WagnerEH. Clinical Epidemiology. The Essentials.
    3rd ed
    . Baltimore, MD: Williams & Wilkins; 1996.
    FriedlandDJ, (ed). Evidence-Based Medicine. A Framework for Clinical Practice. Stamford, CT: Appleton & Lange; 1998.
    FriedmanLM, FurbergCD, DeMetsDL. Fundamentals of Clinical Trials.
    3rd ed
    . St. Louis, MO: Mosby; 1996.
    GrayJAM. Evidence-Based Healthcare. New York: Churchill Linvingstone; 1997.
    HulleySB, CummingsSR, BrownerWS, GradyD, HearstN, NewmanTB. Designing Clinical Research.
    2nd ed
    . Philadelphia, PA: Lippincott Williams & Wilkins; 2001.
    JekelJF, ElmoreJG, KatzDL. Epidemiology, Biostatistics, and Preventive Medicine. Philadelphia, PA: Saunders; 1996.
    KatzDL. Epidemiology, Biostatistics, and Preventive Medicine Review. Philadelphia, PA: Saunders; 1997.
    MotulskyH. Intuitive Biostatistics. New York: Oxford University Press; 1995.
    PetittiDB. Meta-Analysis, Decision Analysis, and Cost-Effectiveness Analysis. Methods for Quantitative Synthesis in Medicine.
    2nd ed
    . New York: Oxford University Press; 2000.
    PocockSJ. Clinical Trials. Chichester, UK: Wiley; 1993.
    RosserWW, ShafirMS. Evidence-Based Family Medicine. Hamilton, Ontario: B.C. Decker Inc; 1998.
    SackettDL, HaynesRB, GuyattGH, TugwellP. Clinical Epidemiology. A Basic Science for Clinical Medicine.
    2nd ed
    . Boston, MA: Little, Brown and Company; 1991.
    SackettDL, RichardsonWS, RosenbergW, HaynesRB. Evidence-Based Medicine. How to Practice & Teach EBM. New York: Churchill Livingstone; 1997.
    SpeicherCE. The Right Test.
    3rd ed
    . Philadelphia, PA: Saunders; 1998.
    U.S. Preventive Services Task Force. Guide to Clinical Preventive Services.
    2nd ed
    . Alexandria, VA: International Medical Publishing; 1996.
    Users' Guides to the Medical Literature
    BarrattA, IrwigL, Glasziou et al. Users' guides to the medical literature: XVII. How to use guidelines and recommendations about screening: Evidence-Based Medicine Working Group. JAMA. 1999;281:2029–2034.
    BucherHQGuyattGH, CookDJ, HolbrookA, McAlisterFA. Users' guides to the medical literature: XIX. Applying clinical trial results: A. How to use an article measuring the effect of an intervention on surrogate end points: Evidence-Based Medicine Working Group. JAMA. 1999;282:771–778.
    DansAL, DansLF, GuyattGH, RichardsonS. Users' guides to the medical literature: XIV How to decide on the applicability of clinical trial results to your patient: Evidence-Based Medicine Working Group. JAMA. 1998;279:545–549.
    DrummondMF, RichardsonWS, O'BrienBJ, LevineM, HeylandD. Users' guides to the medical literature: XIII. How to use an article on economic analysis of clinical practice: A. Are the results of the study valid? Evidence-Based Medicine Working Group. JAMA. 1997;277:1552–1557.
    GiacominiMK, CookDJ. Users' guides to the medical literature: XXIII. Qualitative research in health care: A. Are the results of the study valid? Evidence-Based Medicine Working Group. JAMA. 2000a;284:357–362.
    GiacominiMK, CookDJ. Users' guides to the medical literature: XXIII. Qualitative research in health care: B. What are the results and how do they help me care for my patients? Evidence-Based Medicine Working Group. JAMA. 2000b;284:478–482.
    GuyattGH, HaynesRB, JaeschkeRZ, et al. Users' Guides to the Medical Literature: XXV. Evidence-based medicine: Principles for applying the Users' Guides to patient care: Evidence-Based Medicine Working Group. JAMA. 2000;284:1290–1296.
    GuyattGH, NaylorCD, JuniperE, HeylandDK, JaeschkeR, CookDJ. Users' guides to the medical literature: XII. How to use articles about health-related quality of life: Evidence-Based Medicine Working Group. JAMA. 1997;277:1232–1237.
    GuyattGH, SackettDL, CookDJ. Users' guides to the medical literature: II. How to use an article about therapy or prevention: A. Are the results of the study valid? Evidence-Based Medicine Working Group. JAMA. 1993;270:2598–2601.
    GuyattGH, SackettDL, CookDJ. Users' guides to the medical literature: II. How to use an article about therapy or prevention: B. What were the results and will they help me in caring for my patients? Evidence-Based Medicine Working Group. JAMA. 1994;271:59–63.
    GuyattGH, SackettDL, SinclairJC, HaywardR, CookDJ, CookRJ. Users' guides to the medical literature: IX. A method for grading health care recommendations: Evidence-Based Medicine Working Group. JAMA. 1995;274:1800–1804.
    GuyattGH, SinclairJ, CookDJ, GlasziouP. Users' guides to the medical literature: XVI. How to use a treatment recommendation: Evidence-Based Medicine Working Group and the Cochrane Applicability Methods Working Group. JAMA. 1999;281:1836–1843.
    HaywardRS, WilsonMC, TunisSR, BassEB, GuyattG. Users' guides to the medical literature: VIII. How to use clinical practice guidelines: A. Are the recommendations valid? The Evidence-Based Medicine Working Group. JAMA. 1995;274:570–574.
    HuntDL, JaeschkeR, McKibbonKA. Users' guides to the medical literature: XXI. Using electronic health information resources in evidence-based practice: Evidence-Based Medicine Working Group. JAMA. 2000;283:1875–1879.
    JaeschkeR, GuyattG, SackettDL. Users' guides to the medical literature: III. How to use an article about a diagnostic test: A. Are the results of the study valid? Evidence-Based Medicine Working Group. JAMA. 1994a;271:389–391.
    JaeschkeR, GuyattGH, SackettDL. Users' guides to the medical literature: III. How to use an article about a diagnostic test: B. What are the results and will they help me in caring for my patients? The Evidence-Based Medicine Working Group. JAMA. 1994b;271:703–707.
    LaupacisA, WellsG, RichardsonWS, TugwellP. Users' guides to the medical literature: V How to use an article about prognosis: Evidence-Based Medicine Working Group. JAMA. 1994;272:234–237.
    LevineM, WalterS, LeeH, HainesT, HolbrookA, MoyerV. Users' guides to the medical literature: IV. How to use an article about harm: Evidence-Based Medicine Working Group. JAMA. 1994;271:1615–1619.
    McAlisterFA, LaupacisA, WellsGA, SackettDL. Users' Guides to the Medical Literature: XIX. Applying clinical trial results: B. Guidelines for determining whether a drug is exerting (more than) a class effect. JAMA. 1999;282(14): 1371–1377.
    McAlisterFA, StrausSE, GuyattGH, HaynesRB. Users' guides to the medical literature: XX. Integrating research evidence with the care of the individual patient: Evidence-Based Medicine Working Group. JAMA. 2000;283:2829–2836.
    McGinnTG, GuyattGH, WyerPC, NaylorCD, StiellIG, RichardsonWS. Users' guides to the medical literature: XXII. How to use articles about clinical decision rules: Evidence-Based Medicine Working Group. JAMA. 2000;284:79–84.
    NaylorCD, GuyattGH. Users' guides to the medical literature: X. How to use an article reporting variations in the outcomes of health services: The Evidence-Based Medicine Working Group. JAMA. 1996a;275:554–558.
    NaylorCD, GuyattGH. Users' guides to the medical literature: XI. How to use an article about a clinical utilization review: Evidence-Based Medicine Working Group. JAMA. 1996b;275:1435–1439.
    O'BrienBJ, HeylandD, RichardsonWS, LevineM, DrummondMF. Users' guides to the medical literature: XIII. How to use an article on economic analysis of clinical practice: B. What are the results and will they help me in caring for my patients? Evidence-Based Medicine Working Group. JAMA. 1997;277:1802–1806.
    OxmanAD, CookDJ, GuyattGH. Users' guides to the medical literature: VI. How to use an overview: Evidence-Based Medicine Working Group. JAMA. 1994;272:1367–1371.
    OxmanAD, SackettDL, GuyattGH. Users' guides to the medical literature: I. How to get started: The Evidence-Based Medicine Working Group. JAMA. 1993;270:2093–2095.
    RandolphAG, HaynesRB, WyattJC, CookDJ, GuyattGH. Users' Guides to the Medical Literature: XVIII. How to use an article evaluating the clinical impact of a computer-based clinical decision support system: JAMA. 1999;282:67–74.
    RichardsonWS, DetskyAS. Users' guides to the medical literature: VII. How to use a clinical decision analysis: A. Are the results of the study valid? Evidence-Based Medicine Working Group. JAMA. 1995a;273:1292–1295.
    RichardsonWS, DetskyAS. Users' guides to the medical literature: VII. How to use a clinical decision analysis: B. What are the results and will they help me in caring for my patients? Evidence Based Medicine Working Group. JAMA. 1995b;273:1610–1613.
    RichardsonWS, WilsonMC, GuyattGH, CookDJ, NishikawaJ. Users' guides to the medical literature: XV How to use an article about disease probability for differential diagnosis: Evidence-Based Medicine Working Group. JAMA. 1999;281:1214–1219.
    RichardsonWS, WilsonMC, WilliamsJWJr, MoyerVA, NaylorCD. Users' guides to the medical literature: XXIV. How to use an article on the clinical manifestations of disease: Evidence-Based Medicine Working Group. JAMA. 2000;284:869–875.
    WilsonMC, HaywardRS, TunisSR, BassEB, GuyattG. Users' guides to the Medical Literature: VIII. How to use clinical practice guidelines: B. What are the recommendations and will they help you in caring for your patients? The Evidence-Based Medicine Working Group. JAMA. 1995;274:1630–1632.
    Other Articles
    DetskyAS, NaglieG, KrahnMD, NaimarkD, RedelmeierDA. Primer on medical decision analysis: Part 1—Getting started. Med Decis Making. 1997;17:123–125.
    DetskyAS, NaglieG, KrahnMD, RedelmeierDA, NaimarkD. Primer on medical decision analysis: Part 2—Building a tree. Med Decis Making. 1997;17:126–135.
    EisemanB, JonesR, McClatcheyM, BorlaseB. Cost-effective diagnostic test sequencing. World J Surg. 1989;13:272–276.
    ErakerSA, EeckhoudtLR, VanbutseleRJ, LebrunTC, SaillyJC. To test or not to test—to treat or not to treat: The decision-threshold approach to patient management. J Gen Intern Med. 1986;1:177–182.
    KrahnMD, NaglieG, NaimarkD, RedelmeierDA, DetskyAS. Primer on medical decision analysis: Part 4—Analyzing the model and interpreting the results. Med Decis Making. 1997;17:142–151.
    MerzCN, BermanDS. Imaging techniques for coronary artery disease: Current status and future directions. Clin Cardiol. 1997;20(6):526–532.
    NaglieG, KrahnMD, NaimarkD, RedelmeierDA, DetskyAS. Primer on medical decision analysis: Part 3—Estimating probabilities and utilities. Med Decis Making. 1997;17:136–141.
    NaimarkD, KrahnMD, NaglieG, RedelmeierDA, DetskyAS. Primer on medical decision analysis: Part 5—Working with Markov processes. Med Decis Making. 1997;17:152–159.
    PattersonRE, ChurchwellKB, EisnerRL. Diagnosis of coronary artery disease in women: Roles of three dimensional imaging with magnetic resonance or positron emission tomography. Am J Card Imaging. 1996;10:78–88.
    PattersonRE, HorowitzSF, EisnerRL. Comparison of modalities to diagnose coronary artery disease. Semin Nucl Med. 1994;24:286–310.
    RathbunSW, RaskobGE, WhitsettTL. Sensitivity and specificity of helical computed tomography in the diagnosis of pulmonary embolism: A systematic review. Ann Intern Med. 2000;132:227–232.
    ResnikDB. To test or not to test: A clinical dilemma. Theor Med. 1995;16:141–152.


    A leading clinician and clinical epidemiologist received the bad news that her 76-year-old father had suffered a large anterior wall myocardial infarction. She rushed to the hospital, where she learned he was hypotensive and in renal failure. She was confronted with desperate options, daunting decisions—intra-aortic balloon pump, emergency bypass surgery, or hemodialysis. Her father had been clear that he never wanted “heroic measures” if and when it came to that. Yet, she thought, that pertained when his condition was truly irreversible. Was it? His cardiac function might improve over time. His kidneys might recover from acute tubular necrosis. Struggling with the possibilities, her own emotions and those of her family, and the weightiness of the decisions to be made, the clinician turned to a trusted friend and colleague for advice.

    “But,” said the friend, “you're a leading clinical epidemiologist. You know all there is to know about the probabilities of different outcomes with different procedures, the potential risks and benefits involved, the operating characteristics of various tests and procedures. If anyone has the means to make a tough decision like this one, it's you.”

    “Oh, come on!” replied the clinician, clearly drained and exasperated. “This is important!”

    With thanks to: David Paltiel, PhD, Yale University School of Medicine, Department of Epidemiology and Public Health.

    About the Author

    David L. Katz, MD, MPH, is Associate Clinical Professor of Epidemiology and Public Health & Medicine, and Director of Medical Studies in Public Health at the Yale University School of Medicine. He is board certified in Internal Medicine and Preventive Medicine/Public Health. He earned his BA from Dartmouth College, his MD from the Albert Einstein College of Medicine, and his MPH from the Yale University School of Medicine. He directs courses at the Yale Schools of Medicine and Public Health in biostatistics/clinical epidemiology, public health/epidemiology, preventive medicine, health policy, and nutrition, and has served as clinical preceptor to both medical students and residents. A former preventive medicine residency director, he helped establish a unique, integrated Internal Medicine/Preventive Medicine training program at a Yale affiliated hospital in Derby, CT, and was recognized with the Rising Star Award of the American College of Preventive Medicine in 2001. He founded and directs the Yale School of Public Health's CDC-funded Prevention Research Center, where he oversees a staff of 15, and is principal investigator of numerous studies in chronic disease prevention, including both clinical and community-based trials. Dr. Katz has published numerous scientific articles and chapters on topics including chronic disease prevention, nutrition, obesity, clinical epidemiology, and cardiovascular disease. In addition, he has authored several previous books, including Epidemiology, Biostatistics & Preventive Medicine Review (W.B. Saunders, 1997) and Nutrition in Clinical Practice (Lippincott Williams & Wilkins, 2000).

    • Loading...
Back to Top

Copy and paste the following HTML into your website