# Clinical Epidemiology & Evidence-Based Medicine: Fundamental Principles of Clinical Reasoning & Research

Books

### David L. Katz

• Chapters
• Front Matter
• Back Matter
• Subject Index
• ## Dedication

I dedicate this book to my wife, Catherine: the reason why.

## Preface

Evidence has securely claimed its place among the dominant concepts in modern medical practice. To the extent possible, clinicians are expected to base their decisions (or recommendations) on the best available evidence. Physicians may see this as one possible silver lining in the dark cloud of managed care. Insurers competing for clientele and revenue have increasingly made it a practice to include in benefits packages only those items for which there is convincing proof of benefit. Moreover, these items must provide their benefit at reasonable and acceptable cost. Thus, when applying evidence to the practice of medicine the benefit of the evidence must be measurable and definable, the cost must be measurable, and, perhaps the subtlest challenge of all, evidence itself must be defined and measured.

Despite current efforts to bridge the gap between medicine and public health through the Medicine-Public Health Initiative,1,2 the philosophical divide between a discipline devoted to the concerns of populations and one devoted to the advocacy of an individual seems impassable. However, the consistent application of evidence to clinical decision making is the bridge between the concerns of clinical practice and the goals of public health.

Evidence-based practice is population-based practice. Evidence applied clinically is derived from the medical literature, where the standards of evidence, and therefore practice, continuously evolve. But what is reported in the literature is not the experience of an individual patient (other than in case reports, a modest although time-honored and often important source of evidence, or in n-of-1 experiments), and certainly not the experience of our individual patient, but rather the experience of a population of patients. Therefore the practice of evidence-based medicine requires the application of population-based data to the care of an individual patient whose experiences will be different in ways both discernible and not, from the collective experience reported in the literature. All evidence-based decisions made on behalf of (or preferably, with) individual patients are extrapolation or interpolation from the prior experience of other patients. Clinical medicine is evidence-based only if it is population-based.

This may or may not seem a provocative concept, but consider the alternative. To base clinical decisions for an individual on the individual alone, the outcome of an intervention would need to be known in advance. In other words, medicine would need to borrow from astrology or some other system of predicting future events. The choice of an initial antihypertensive drug for a hypertensive patient cannot be based, before the drug is prescribed, on the response of the patient in question. Nor can the benefits to the patient be known in advance. The drug is chosen based on the published results of antihypertensive therapy in other patients. The particular drug is selected based on how closely the characteristics of our patient match those of others who have benefited from specific therapies. Once the drug is selected, while the therapeutic effect on the surrogate measure (e.g., blood pressure) is detectable, any outcome benefit to our patient (e.g., stroke prevention) remains unknowable. We can never identify the stroke we have prevented in an individual. The strokes we prevent by prescribing antihypertensives, the myocardial infarctions we prevent by prescribing aspirin or statins, are statistical events. We know the rate at which such conditions occur in particular populations, and research demonstrates how these rates can be changed. By applying the intervention to our patient, we expect the risk of the event to decline comparably. But unless an intervention eliminates the risk of a clinical event entirely (few, if any, do), our patient may suffer the event despite intervention. Alternatively, our patient may have appeared to be at risk, but would have never suffered the event even without intervention. We can never know. We never base what we do for an individual on the outcomes particular to that individual. We base what we do on the experience of populations, and the probability that our patient will share that experience. Astute medical care is predicated on the capacity to identify similarities between a single patient and the particular population whose collective experience is most likely to inform and anticipate the single patient's experience. In his Poetics, Aristotle considers this “eye for resemblances,” or “intuitive perception of the similarity in dissimilars,” a mark of genius.3 If so, it is a genius the clinician frequently has cause to invoke.

The science of applying the principles of population-based (epidemiologic) evidence to the management of individual patients has come to be known as clinical epidemiology. While epidemiology characterizes the impact of health related conditions on populations, clinical epidemiology applies such data to individual patient care. Clinicians are traditionally uncomfortable with the notion of catering to populations rather than individual patients. Clinical epidemiology asserts that the two are effectively the same, or at least inextricably conjoined. Individual patient care is informed by the interpretation of population-based data. When populations are well served by the health care they receive, the individual members of those populations are (generally) well served. When individuals are well served by the clinical care they receive, in the aggregate, the pattern of that care becomes (usually) the sound practice of health care delivery to populations.

Implicit in the concept of evidence being the derivative of a populations experience is the need to relate that experience back to the individual patient. The inapplicability of some evidence to some patients is self-evident. Studies of prostate cancer are irrelevant to our female patients; studies of cervical cancer are irrelevant to our male patients. Yet beyond the obvious exclusions is a vast sea of gray. If our patient is older than, younger than, sicker than, healthier than, ethnically different from, taller, shorter, simply different from the subjects of a study, do the results pertain? As our individual patient will never be entirely like the subjects in a study (unless they were a subject, and even then their individual experience might or might not reflect the collective experience), can the results of a study ever be truly pertinent? Clinical epidemiology is a sextant, or in more modern but equally nautical terms, the geographical positioning system (GPS), on a vast sea of medical uncertainty. And, to extend the metaphor, the skills of piloting can be acquired and increase the reliability with which a particular destination (i.e., diagnosis, therapeutic outcome) is achieved. Yet each crossing will be unique, often with previously unencountered challenges and hazards. No degree of evidence will fully chart the expanse of idiosyncrasy in human health and disease. Thus, to work skillfully with evidence is to acknowledge its limits. Judgment must be prepared to cross those seas as yet uncharted by evidence.

It is expected that some of the material in this text, particularly the more statistically involved, will diverge from what we would accept as intuitive. However, comfort can be taken from the fact that we are all de facto clinical epidemiologists. As clinicians, we decide which information pertains to a particular patient every day of practice: who does and does not get prescribed an antibiotic (and if so, which one); who does and does not get treated with insulin, metformin, a sulfonylurea, a thiazolidinedione; who does and does not get advised to be x-rayed, injected, phlebotomized, cannulated, or instrumented. Cognizant or not of the subtleties as they play out, in each such decision we are comparing our patient to others that have come before; others in our own practice, relying as we tend to do (though we tell one another we should not) on the compelling lessons of personal anecdote, or others whose experience has been more formally conveyed, in the tables and graphs of a peer-reviewed article. While the choices we ultimately make in clinical testing and management are a product of our interaction with patients, our shared and disparate values, beliefs, and preferences, the decisions that delineate those choices are largely the product of clinical epidemiology.

Because we all practice clinical epidemiology, an understanding of this tool (or array of tools) we use is incumbent upon us all. If every clinical decision derives in whole or in part (and it does) from the tacit comparison of our patient to a population of patients, then the skill with which that comparison is made is fundamental to the skill with which medicine is practiced. Integral to that comparison is the capacity to recognize the defining characteristics of both patients and populations as the basis for defining the bounds of similarity and dissimilarity. The physician's capacity to evaluate the context in which “evidence” was gathered is equally important. The ability to evaluate the quality as well as the pertinence of evidence is essential. Of course, finding the best available evidence when one is uncertain about a clinical decision is prerequisite to its interpretation.

Viewed with the cool glare of reductionism, the practice of evidence-based medicine requires a discrete and modest skill set. One must be able to find the available evidence. One must be able to evaluate the relevance and quality of evidence. And one must be able to interpret evidence presented in terms pertinent to populations so that the same data may inform patient care decisions. These skills, like any others, can be learned and mastered. The various tools of our trade—stethoscopes and sphygmomanometers—were handed to us along with the lessons that made us competent in their use. While the tools of evidence-based practice have become enjoined among the more highly valued items in our proverbial black bags, most of us have had no formal instruction in their use. Consequently, many of us are likely using these tools less effectively than we might.

While clinical choices (for both testing and treatment) are predicated on, at a minimum, the knowledge, judgment, values, preconceived notions, experiences, preferences and fears of both clinician and patient, clinical decision-making is greatly influenced by three considerations: probability, risk, and alternative. Probability is fundamental to such decisions, as we evaluate and treat patients only for a given condition or conditions it seems they might have. We do not order CT scans of every patient's brain, yet we do order some. The distinction is derived from our estimate of the probability of finding relevant pathology. A clinical decision cannot be reached without a semiquantitative estimate of probability. A patient either seems likely enough, or not likely enough, to need a particular test or treatment, to result in our recommending it. This is a truism for any test applied only to some patients.

Some low probability diagnoses are pursued because they pose such high risk. Here, too, the natural tendencies of our minds are in alignment with clinical epidemiology. We admit some patients to the hospital to “rule out MI” even though we believe the probability of myocardial infarction (MI) to be low, because the risk associated with undetected MI is high. We have all been taught to do a lumbar puncture (LP) whenever we wonder “should I do an LP?” because of the devastating consequences of missing meningitis.

Finally, we factor in alternatives: alternative treatments, alternative tests, alternative diagnoses. When chest pain seems atypical for angina, but no alternative explanation is at hand, we are more apt to treat the pain as angina. When pneumonia is present to explain shortness of breath, we will be less inclined to work up pulmonary embolism (PE), despite pleuritic chest pain and tachycardia. When we have excluded the impossible we are apt to focus on what remains, however improbable.4 By a process to which we are, for the most part, comfortably incognizant, we make every decision factoring in considerations of probability, risk, and alternatives.

But an unconscious process is a process that cannot be optimally regulated. By knowing that our decisions are borne on our musings over probability, risk, and alternative, these parameters should become of sufficient interest to us to warrant conscious monitoring. Each of these parameters is population-based. There is no probability of genuine relevance to an individual: there is the rate of occurrence in populations, and the degree of concordance between individual and population characteristics. There is no true individual risk; for an individual, an event occurs (100% risk) or does not (0% risk). There, is however, the comparability of the patient to groups in whom the event rate in question is higher or lower. The alternatives available for an individual patient are those options and interventions applied under similar circumstances to other patients, with varying degrees of success.

Similar principles underlie the research that constitutes the evidence base (or its greater portion) for clinical practice. As is detailed later in the text, studies are constructed in an effort to establish the collective experience of a few (the study subjects) as representative of the many. While the clinician looks for correspondence between patient and study participants, the investigator must consider the relevance of the study to the larger population of potential future patients. Just as the probability of outcomes, good and bad, guides clinical management, the probabilities of outcomes, good and bad, false and true, are estimated and factored into the statistical stipulations and design of a study. The appropriateness of a particular methodology depends as much on alternatives as does the appropriateness of a clinical intervention. As is expressed by the conventional safeguards against false-positive and false-negative error (see Chapter 9) and the application of progressively stringent standards of human subject protection,5,6,7,8 thorough consideration of risk is intrinsic to the research process. Even less rigorous means of conveying evidence, such as case reports and case series, depend for their interest and relevance on probability, alternative, and risk. Such reports are meaningful only when the putative association is apparent and convincing; the clinical need nontrivial; the risks of application acceptable; alternative explanations unlikely; and the pertinence to our patients probable. Probability, alternative, and risk influence one another within the disciplines of clinical practice and clinical research, and these disciplines in turn interact. The needs, insights and frustrations of practice are an important source of hypotheses and attendant methods in clinical research. The evidence generated by such studies naturally serves to inform clinical practice. These interactions are displayed in Table 1.

TABLE 1 The Influence of Probability, Risk, and Alternatives on Clinical Research and Clinical Reasoning, and the Salient Interactions

Ultimately, then, while judicious practice depends on evidence, the derivation of evidence depends on many of the same principles as judicious practice. The thoughtful and diligent practitioner remains abreast of the literature to apply the best practice an evolving evidence base supports. The diligent and thoughtful investigator exploits evolving methodologies to generate evidence most conducive to advances in science and practice. The highest standards of evidence-based practice are achieved not only when evidence is well applied, but also when that evidence is well produced. Part of the burden for the responsible cultivation of higher standards and better outcomes in medicine falls, naturally, to researchers and those that screen and publish their findings. But application is ultimately the responsibility of the clinician, who is obligated to consider not only the pertinence of particular evidence to his or her practice but the adequacy and reliability of the evidence itself. At every step, from the design of a study to clinical counseling, probability, alternative, and risk must be addressed. For evidence to be well applied the correspondence of this one patient to those that came before must be considered, the compatibility of prior knowledge with current need revisited.

We cannot, therefore, practice clinical medicine and avoid population-based principles. We cannot practice clinical medicine and avoid the practice of clinical epidemiology. But the discipline of evidence-based practice/clinical epidemiology (the terms might be used interchangeably) is not one in which most of us have had any formal initiation. All of the art and all of the science of medicine depend on how artfully and scientifically we as practitioners reach our decisions. The art of clinical decision-making is judgment, an even more difficult concept to grapple with than evidence. As the quality and scope of evidence to support clinical interventions is, and will likely always remain, limited in comparison to the demands of clinical practice, the practice of evidence-based medicine requires an appreciation for the limits of evidence, and the arbiters of practice at and beyond its perimeters. Judgment fortified by the highest standards of decision-making science is a force to be reckoned with, enabling us each to extract the best possible results from a process to which we are naturally inclined. Ultimately that is the validation of evidence-based practice, or population-based practice, or clinical epidemiology—the outcomes to which such concepts contribute. Rigorous reasoning is the means, desirable outcomes the ends.

For Whom is This Book Intended?

This book is about concepts, or rather the methodology of arriving at robust clinical decisions that binds together an array of concepts. This book is not about the facts, or current fund of medical knowledge, on which such decisions rest. The life span of medical facts is short and shortening further all of the time. Fortunately the methods for extracting optimal performance from the prevailing facts of the day are enduring. The intent here is to provide a basic mastery of such methods, that is, the capacity to harness the power of our intrinsic heuristics (decision-making pathways) and apply it to a constantly evolving body of knowledge. The medical literature and clinical vignettes will be referenced as required to demonstrate applications of the methods described. But the message is in the methods rather than their application to any particular study or article or case.

The intended audience for this text is anyone who makes, or will make, clinical decisions. Worth noting are a number of excellent texts on the subjects of clinical epidemiology and evidence-based medicine already available, many of which I have used liberally as sources (see Text Sources). Compared to most of these, this text is intended to be more clinician-friendly and assumes less prior knowledge. Every effort has been made to present material in a simple and uncluttered manner. Most tables in the text, for example, should be interpretable at a glance.

One of the important distinctions I have made while writing this text is to endeavor to teach less and clarify more. The contention on which this text is based is that clinicians are intuitive clinical epidemiologists, and therefore don't really need to learn to function as such. This text is designed to help reveal the influence and application of this intuition. By doing so, it should illuminate the processes of converting evidence to decisions, research to practice. The more we understand the ways in which we approach evidence and make decisions, the more reliably we can control these processes, and their attendant outcomes.

While a fair amount of statistics is included, the use of a calculator in clinical practice is certainly not intended. Rather, as quantitative principles already underlie clinical reasoning, one is well advised to have a basic familiarity with those principles. Fundamentals of practice truly hang in the balance. A positive or negative test result is at times highly reliable, at other times highly unreliable. A bit of number crunching demonstrates how different clinical conclusions can, and should, be under different circumstances. The numbers need not be recalled for the importance of the concepts to be retained.

The consistent application of the basic principles of clinical epidemiology infuses with the strengths of science the decision making that presupposes all else in clinical practice, including its outcomes. That science and evidence are limited and are dependent upon judgment for their application is implicit in the text everywhere it is not explicit. Also implicit throughout the text is that the medical decisions reached by clinicians serve only to provide patients—the ultimate decision makers—with good information upon which to base their decisions.

I am grateful to the many accomplished clinicians and clinical epidemiologists whose contributions I have drawn on so heavily, both in the drafting of this text and in my own clinical and research efforts. I acknowledge with appreciation and humility that in drafting this text I have followed where many luminaries have led. That said, if I have wandered off the trails blazed by the leaders of this field, I can blame no one but myself. Any misstep—ambiguity, miscalculation, or distortion—is of course my responsibility. While hoping that none is found, I apologize and offer my sincere regret in advance on the chance that any is.

With a great reverence for the unique burdens and privileges of clinical practice, I submit the principles of this text in the belief and hope that they will enhance your ability to obtain the best possible outcomes for your patients.

References
. Medicine and public health. Pursuing a common destiny. JAMA. 1999;276:1429–1430. http://dx.doi.org/10.1001/jama.1996.03540170073036
. Topics for our times: The medicine/public health initiative. Am J Public Health. 1997;87:1098–1099. http://dx.doi.org/10.2105/AJPH.87.7.1098
, (ed). The Complete Works of Aristotle.2Princeton, NJ: Princeton University Press; 1984:2335.
. The sign of four. In: The Complete Sherlock Holmes. New York: Doubleday; 1930:87–138.
. Improving the protection of human research subjects. Acad Med. 2000;75:718–720. http://dx.doi.org/10.1097/00001888-200007000-00013
. Children's rights in clinical research. J Nurs Scholarsh. 2000;32:179–184. http://dx.doi.org/10.1111/j.1547-5069.2000.00179.x
. Primary care research and protection for human subjects. JAMA. 1999;281:1697–1698. http://dx.doi.org/10.1001/jama.281.18.1697
, . Ethical and legal issues in conducting research involving elderly subjects. Behav Sci Law. 1995;13:319–335. http://dx.doi.org/10.1002/bsl.2370130302

## Acknowledgments

I am grateful to Dr. Ralph Horwitz, chairman of medicine at the Yale School of Medicine, for setting the standard so many of us strive (without much hope of success) to meet.

I sincerely appreciate the vision of Dan Ruth, and the guidance and support of C. Deborah Laughton, at Sage. The transition from idea to book is a relay race, in which their laps were very well run indeed.

I acknowledge with thanks the contributions of my collaborators, Dr. Laura Greci, a senior resident in preventive medicine/internal medicine at Griffin Hospital in Derby, CT, and Dr. Haq Nawaz, associate director of the same preventive medicine residency.

I am grateful to my parents, Dr. Donald Katz and Susan Katz, for never (well, hardly ever…) discouraging me when, as a child, I incessantly asked, “why?” To my Dad, I also add appreciation for the walks at Horse Heaven; I do my best thinking there.

I am deeply indebted to Jennifer Ballard, administrator of the Yale Prevention Research Center, who makes me wonder every day how I managed before!

I am grateful to my children, Rebecca, Corinda, Valerie, Natalia, and Gabriel, for their patience and unconditional love, seemingly unattenuated by the countless times I have turned from them to the computer, and turned down their invitations to play.

Above all, I am grateful to my wife, Catherine, my best editor as well as my best friend, for the love and the coffee and the kind words as much as for the ruthlessly honest (and always constructive) criticism.

## Appendix A: Getting at the Evidence

Appendix A.1: Accessing the Medical Literature: How to Get There from Here

Online search engines offer the advantages of up-to-date, tailored information access. The disadvantages of online searching include the potential time lost in futile efforts and the need to weigh the quality of evidence from a diverse array of sources.

Online medical databases are collections of published medical articles from peer-reviewed (and sometimes non peer-reviewed) journals. The National Library of Medicine's (NLM) MEDLINE is the premier example. For the generalist clinician, MEDLINE tends to be the most useful database. It is appropriate for the efficient identification of available answers to most clinical questions. There are numerous searchable databases with more targeted application as well.

NLM Online Databases and Databanks

The National Library of Medicine maintains a computerized database bank, collectively called MEDLARS® (MEDical Literature Analysis and Retrieval System). MEDLARS contains several online databases including some 20 million references (and growing). Some of the more important databases maintained by the NLM are listed in Table A1.1. Table A1.2 provides the characteristics of some of the more commonly used NLM databases.

TABLE A1.1 Some Databases Maintained by the National Library of Medicine
AIDSDRUCSHSDB®
AIDSLINE®HSRPROJ
AIDSTRIALS®IRIS
AVLINEMEDLINE®
BIOETHICSLINE®MEDLINEplus
CancerLit®MeSH Vocabulary®
CATLINE®OLDMEDLINE
CCRISPOPLINE®
ChemlDPlusPreMEDLINE
DART®RTECS
DIRLINE®SDILINE®
DOCLINESERLINE®
DOCUSER®SPACELINE
EMICTOXLINE®
ETICBACKTOXNET
GENE-TOXTRI
HISTLINE®TRIFACTS
Source: National Library of Medicine website (http://www.nlm.nih.gov)
TABLE A1.2 Databases Maintained by the NLM and Their Characteristics

MEDLINE

MEDLINE covers many broad areas in the fields of medicine, nursing, dentistry, veterinary medicine, public health, and health care systems, among other clinical and basic sciences. The MEDLINE database consists of citations and abstracts from approximately 4,300 journals as of mid 2000. The majority of journals published in the US are included in MEDLINE along with journals from almost 70 other countries. MEDLINE contains foreign language citations to articles with an English abstract. MEDLINE'S ever-expanding pool of citations reached 11 million in 2000, of which about three-quarters provide abstracts. The database, however, is limited to articles published as of 1966 or after. Citations prior to 1966 are available in a database called OLDMEDLINE, but do not provide abstracts or MeSH (see below) term indexing.

PreMEDLINE represents a preliminary MEDLINE, providing basic citation information and abstracts before the completed records are prepared and added to MEDLINE. Each record is identified with a Unique Identification (UI) number at this stage. Each article is then indexed according to a predetermined indexing system known as Medical Subject Heading (MeSH) terms, or “mesh terms.” Subsequently, indexing for Publication Type (e.g., Randomized Controlled Trial, Review, etc.) is added to enhance searchability. Once this process is completed, a citation is added to MEDLINE. As the record is transferred to MEDLINE, it is deleted from PreMEDLINE. More than 8,000 of these citations are transferred every week.

MeSH Terminology

The NLM indexes each article with a standardized vocabulary system. The MeSH vocabulary contains approximately 17,000 terms. Each MeSH term represents a solitary concept in medicine. New terms are entered as they develop in the medical literature. MeSH is the major means by which a search engine will retrieve articles. Each article published in biomedical journals is indexed with several MeSH terms that represent the overall theme and contents of the article. MeSH terms themselves are grouped under major divisions called trees. For example, atrial fibrillation is a MeSH term, but it also is a subcategory of cardiovascular disease which itself is a branch of disease as illustrated in Figure A1.1. Each MeSH term is linked to other terms in a vocabulary hierarchy.

Figure A1.1. Illustration of a major tree, MeSH term, and MeSH subheading in descending order.

Major MeSH terms (MeSH subheadings) are further categorized into MeSH subtopics to further refine the indexing. A searcher can narrow his or her search by choosing more specific terms down the line, or may expand the search to be more inclusive by choosing terms higher up in the MeSH hierarchy. In the above instance, a search with the more general term cardiovascular diseases would yield far more records than a search with the term atrial fibrillation.

For most clinicians, the term atrial fibrillation is apt to be broader than the goals of a particular search. The addition of qualifiers to the MeSH headings or sub-headings results in a narrower search. Qualifiers further dissect a concept into categories such as etiology, pathology, diagnosis, etc. Available qualifiers for “atrial fibrillation” are listed in Table A1.3.

TABLE A1.3 Listing of Qualifiers Used for MeSH Term atrial fibrillation
bloodnursing
chemically inducedpathology
classificationphysiopathology
complicationsprevention & control
diagnosispsychology
enzymologysurgery
epidemiologytherapy
etiologyultrasonography
geneticsmortality
metabolismveterinary
Source: National Library of Medicine website (http://www.nlm.nih.gov)

A listing of all (current) qualifiers included in Medline is provided in Table A1.4. Qualifiers are designated by an intervening forward slash (/) character. For instance, the term atrial fibrillation/etiology indicates a search targeting articles on the etiology of atrial fibrillation. A search with no qualifiers added will include all of the qualifiers that exist for that term. As shown in Table A1.4, two-letter abbreviations may be used for qualifier terms. Table A1.5 lists “Publication Type” terms for MEDLINE.

TABLE A1.4 Subheadings (Qualifer terms) Used with MeSH Terms in MEDLINE and Their Standard Abbreviations
Abnormalities ABEmbryology EMPhysiopathology PP
Adverse Effects AEEpidemiology EPPrevention and Control PC
Agonists AGEthnology EHPsychology PX
Analogs and Derivatives AAEtiology ETRadiation Effects RE
Anatomy and Histology AHGrowth and Development GDRadionuclide Imaging RI
Antagonists and Inhibitors AlHistory HIRadiotherapy RT
Biosynthesis BIImmunology IMRehabilitation RH
Blood Supply BSInjuries INSecondary SC
Blood BLInnervation IRSecretion SE
Cerebrospinal Fluid CFInstrumentation ISStandards ST
Chemical Synthesis CSIsolation and Purification IPStatistics and Numerical Data SN
Chemically Induced CILegislation and Jurisprudence LJSupply and Distribution SD
Chemistry CHManpower MASurgery SU
Classification CLMetabolism METherapeutic Use TU
Complications COMethods MTTherapy TH
Congenital CNMicrobiology MlToxicity TO
Contraindications CTMortality MOTransmission TM
Cytology CYNursing NUTransplantation TR
Deficiency DFOrganization andTrends TD
Diagnostic Use DUParasitology PSUltrastructure UL
Diet Therapy DHPathogenicity PYUrine UR
Drug Effects DEPathology PAUtilization UT
Drug Therapy DTPharmacokinetics PKVeterinary VE
Economics ECPharmacology PDVirology VI
Education EDPhysiology PH
Source: National Library of Medicine website (http://www.nlm.nih.gov)
TABLE A1.5 Terms Used for “Publication Type” in MEDLINE
AddressesLegal Cases [includes law review, legal case study]
BibliographyLetter [includes letters to editor]
BiographyMeeting Abstract
Classical Article [for republished seminal articles]Meta-Analysis [quantitative summary combining results of independent studies]
Clinical Conference [for reports of clinical case conferences only]Multicenter Study
Clinical Trial [includes all types and phases of clinical trials]News [for medical or scientific news]
Clinical Trial, Phase IIOverall [collection of articles; consider Meeting Report]
Clinical Trial, Phase IIIPeriodical Index [for cumulated indexes to journals]
Clinical Trial, Phase IVPractice Guideline [for specific health care guidelines]
Congresses Controlled Clinical TrialPublished Erratum [consider Corrected and Republished Article]
Randomized Controlled TrialRetracted Publication [article later retracted by author]
Comment [for comment on previously published article]Retraction of Publication [author's statement of retraction]
Consensus Development ConferenceReview [includes all reviews; consider specific types]
Consensus Development Conference, NIHReview, Academic [comprehensive, critical, or analytical review]
Corrected and Republished Article [consider Published Erratum]Review, Multicase [review with epidemiological applications]
Dictionary Directory Duplicate Publication [duplication of material published elsewhere]Review of Reported Cases [review of known cases of a disease]
Editorial Festschrift [for commemorative articles]Review Literature [general review article; consider other reviews]
Guideline [for administrative, procedural guidelines in general]Review, Tutorial [broad review for nonspecialist or student]
Historical Article [for articles about past events]Scientific Integrity Review [U.S. Office of Scientific Integrity reports]
InterviewTechnical Report
Journal Article [excludes Letter, Editorial, News, etc.]Twin Study [for studies of twins]
Lectures
Source: National Library of Medicine web sites (http://www.nlm.nih.gov)

MEDLINE is available on CD-ROM and web-based search engines. Some private companies download NLM databases and then market them on CD-ROMs, mostly to colleges and universities (e.g., Silver Platter), where closed hospital computer systems may not allow for Internet access.

NLM offers two free web-based programs, PubMed and Internet Grateful Med (IGM) to which many websites link. Most MEDLINE vendors charge for full text retrievals and document deliveries but some (e.g., OVID), offer a majority of the articles online as full text.

PubMed and Internet Grateful Med (IGM)

PubMed and Internet Grateful Med (IGM) are the two main search engines provided by the NLM. Both are web-based, user-friendly, relatively quick, and are offered free of charge without registration. Both allow MEDLINE searches, and IGM has the added feature of allowing searches of most of the other NLM databases (see Table A1.2) by selecting that database on the first page. Both have an automatic MeSH mapping feature. PubMed also provides links to journal sites on the web (when available) which may allow access to the full text. PubMed's “advanced search mode” allows quick searches (by author, date, page number, journal, volume, etc.).

Searching MEDLINE

MEDLINE records can be searched using MeSH terms, title or text words or phrases, author name, journal name, publication type, dates, language, study subjects or any combination of these. As stated above, MeSH terms are a vocabulary of medical and scientific terms assigned to most documents in MEDLINE by the indexers at the NLM. Text terms are strictly word searches for the exact words or phrases found in the title and abstract of a document. Whenever possible, it is advantageous to use MeSH terms over text terms for several reasons:

• MeSH terms are assigned on the theme subject of the entire document and not just the citation text (title and abstract). Thus, a search using a MeSH term can find relevant documents even when the exact term in question is not found in the citation. This feature is especially important for documents that do not have abstracts, because titles are very short and often omit important terms.
• Some MeSH terms are assigned as Major Topic terms, meaning that the indexer has determined that these terms represent the major thrust of the document. Using the Major Topic field in MEDLINE searches can help discard documents that are less relevant to your search. These terms are designated in the citation and MEDLINE reports with an asterisk (*).
• The use of MeSH terms and MeSH subheadings can allow searches that are very narrow and specific, and can help to speed the search process and avoid the retrieval of irrelevant material.

Both PubMed and IGM have links to the NLM's MeSH Browser through which these vocabulary trees can be visualized, and searches focused or expanded. Automatic MeSH mapping refers to the process where words or queries entered in the search fields are automatically checked against the NLM's MeSH directory and appropriately replaced in the search. This system is not foolproof, however, and matching to a proper MeSH term is sometimes not achieved.

Searches can be conducted by use of Boolean operators (mathematical set language) or natural language. Boolean search operators are more exact.

Boolean Search Operators
 AND is used for combining terms or phrases. Only those records that match with all the specific terms separated by AND are retrieved. The above field will result in retrieval of only those records that contain all these words in the title or the abstract. This is the most commonly used search operator being used (Hint: it happens automatically when you enter several words) OR Searches for records containing either of the words it separates. Using OR will lead to a higher number of articles retrieved than using AND. In the above, the preferred search term is unknown, therefore OR is used to conduct a search based on the alternatives. NOT Searches for records containing the query word preceding it without containing the word following it. That is, it discards records containing the terms after the not. For example: Would retrieve all articles on congestive cardiomyopathy except those pertaining to hypertrophic cardiomyopathy * is a wild card operator to match part of a word or phrase. Examples: micro* matches microorganism, microsome, etc. Similarly, by placing* before a word or part of a word will match all words ending with that word (e.g., *itis will match carditis, arthritis, encephalitis, etc.)

While there are texts available with useful instruction in accessing medical information online,1,2,3 printed information about accessing the published evidence base on-line will be somewhat dated by the time it is published. The pertinent websites (e.g., links to Pubmed or Internet Grateful Med off of the National Library of Medicine home page, http://www.nlm.nih.gov) generally provide tutorials and/or guidance, with the advantage of remaining constantly current.

Appendix A.2 requires internet access, and provides step-by-step guidance through several illustrative searches.

Appendix A.2: A Walking Tour of MEDLINE

Use of this appendix requires internet access. Log on to the National Library of Medicine (http://nlm.nih.gov) home page, or select some alternative means to access PubMed to proceed through the illustrative searches provided below. Conduct the searches as detailed below, and refer to the screen for the content discussed. Note that the exact number of citations retrieved at each step of the searches will likely have changed over time; this will not interfere with the demonstration of search techniques.

Search 1: Basic PubMed Search

A 30-year-old Brazilian women presents to the emergency room with a one month history of fatigue, and a three day history of left pleuritic chest pain and dull epigastric pain. Examination reveals stable vital signs and is notable only for a slightly enlarged spleen. Laboratory results show liver enzymes to be elevated (AST 400 IU/L, ALT 350 IU/L, total bilirubin 1.8 mg/dL, alkaline phosphatase 333 IU/L, and Prothrombin time 15s). An abdominal ultrasound confirms splenomegaly and suggests hepatic fibrosis. A nuclear liver-spleen scan demonstrates portal hypertension with colloid shift. The diagnosis of autoimmune hepatitis is strongly suggested by a positive ANA titer (1:2560, homogeneous pattern).

When this information is conveyed to the patient, she asks what the treatment options are. You conduct a search.

Step I: Formulate a Question and a Search Plan

In this case, the question would be

And, as a corollary

Initial search terms from these questions might include

Step II: Identify Appropriate MeSH Terms
• Log on to the NLM's Entrez-PubMed Start Page (http://www.ncbi.nlm.nih.gov/pubMed/).
• Next, click on the link to the MeSH Browser.
• Type the search words or phrase into the “search for” field. This is where natural language is transcribed into MeSH terminology. Autoimmune hepatitis is not a MeSH term. However, Hepatitis, Autoimmune is, and it will be automatically replaced in the search. A brief definition of the term and the year it was introduced are also provided. Note also that the MeSH tree is displayed, that the “Detailed Display” button provides all of the qualifiers for the term, and that the “Add” button allows for the insertion of search operators (And, Or, Not).
• As we are particularly interested in the treatment of autoimmune hepatitis, we select the “detailed display” link to view the qualifers.
• From the available qualifiers, we select “drug therapy,” “therapy,” and “restrict to Major Topic headings only.” We then select the “Add” button to reformulate the search.

Notice the shaded area at the top now includes technical information for the appropriate MeSH terms (hepatitis, autoimmune) with the appropriate qualifiers (listed as/drug therapy and/therapy) as the major topic area ([MAJR]). Select “PubMed Search” to return to the original PubMed screen with the newly adopted search terms incorporated.

While autoimmune hepatitis therapy in natural language pulls up over 400 citations, the use of MeSH terms as detailed above yields approximately 45 articles.

Step III: Setting Limits

At this point, the articles retrieved can be reviewed, or the search can be further refined by applying limits. (Note that limits could have been incorporated into the search right from the beginning.) Select the “Limits” button below the search term field.

Assign the following limits to the search: publication type (review), age (all adult: 19+ years), subject type (human), publication dates (1995–2000). Entrez dates (5 years) is an alternative to limiting on the basis of publication dates. Conduct the search.

The limited search yields three very relevant articles. Note that the limits applied are listed at the top and that these limits can be applied all at once (as we have done) or one at a time to whittle down the number of articles retrieved.

If at this point search results were not satisfactory (e.g., if the search retrieved no articles), there are several options. The first is to expand some of the stipulated limits (e.g., remove age limits, or expand the acceptable period of publication). Alternatively, were any articles retrieved, select the “related articles” feature to expand the search.

Selecting the “related articles” hyperlink next to the first article of the three retrieved in the above search identifies over 100 related papers.

At this point you can either apply limits to the search as before or continue to retrieve “related articles” to those most pertinent and useful.

Search 2: PubMed Searching with Search Operators

A 65-year-old woman is brought to your office by her daughter for increasing memory loss. History and examination suggest early dementia, probably of the Alzheimer type. After you reveal your suspicion to the daughter, she asks whether her mother should use ginkgo biloba. Since the patient's daughter is a nurse, she wants to know if there is any good evidence that ginkgo biloba really works. You tell her that you are unable to answer a question that specific, but promise to look into the issue and get back to her.

Step I: Formulate a Question and a Search Plan

Step II: Find Appropriate MeSH Terms

Search terms: Ginkgo biloba, dementia Other related terms: cognition, memory

Initial PubMed search of MEDLINE entering just ginkgo biloba yields some 80 articles.

Step III: Setting Limits

Dementia, memory and cognition are similar, but separate MeSH terms. Using each of the terms with the search operator “AND” is illustrated in Table A2.1. Conduct these searches with and without publication type limited to “clinical trial.” As shown in Table A2.1, these constraints result in the retrieval of a few highly relevant articles. (Note again that the actual retrieval may have changed by this time.)

TABLE A2.1 Using Search Operators and Limits to Narrow a MEDLINE Search
Search Terms UsedLimitsNumber of Articles Retrieved
ginkgo bilobanone80
ginkgo biloba AND dementianone15
ginkgo biloba AND dementiaclinical trial2
ginkgo biloba AND memorynone9
ginkgo biloba AND memoryclinical trial1
ginkgo biloba AND cognitionnone4
ginkgo biloba AND cognitionclinical trial1
Search 3: Retrieving a Particular Article

Often, a citation in a lecture or discussion will be of interest. Equally often, the information provided, particularly in the context of informal discussion or clinical rounds, will be incomplete. For example, one might hear a comment such as, “Well of course that's true given that famous actual causes of death paper…”

When interested in retrieving an article alluded to in such a way, the most expedient method is to use the “Single Citation Matcher” of PubMed. Select the “Single Citation Matcher” along the left side of the PubMed Services on the PubMed homepage (http://www.ncbi.nlm.nih.gov/PubMed/). Use of this feature requires that some specific information about the article be available, e.g., the year, journal and an author.

If the necessary identifying information to use the “Single Citation Matcher” is unavailable, we can limit the search to match text terms in the title. Select limits and change the “All Fields” to “Title Text.” Search results based on progressive inclusion of more of the terms from the “allusion” above are shown in Table A2.2.

TABLE A2.2 Progressive Text Term Search by Title Text
Search Terms AddedLimitsNumber of Articles Retrieved
Actualtitle text1504
Actual causestitle text9
Actual causes of deathtitle text6
Actual causes of death in the United Statestitle text6
94017136UID*1

*See Figure A2.1.

Other known terms listed under “All Fields” can also be used to perform focused searching of partial citations (for example, by author or journal name). PubMed offers specific selections on the start page which allow you to “Enter one or more search terms,” “Enter author names,” or “Enter journal titles.”

Note that every citation in MEDLINE is labeled with a MEDLINE Unique Identifier (UI) number, generally listed at the end of a citation. Some libraries may require this number, or the PubMed identification number (the PMID) to locate the citation. An example of a typical MEDLINE citation providing these identifiers is shown in Figure A2.1.

Figure A2.1. Use of MEDLINE Ul number and PubMed ID.

By any of several approaches to the search, the article of interest (McGinnis JM, Foege WH. Actual causes of death in the United States. JAMA. 1993;270:2207–2212) can be retrieved.

Appendix A.3: Publication Bias: The Limits of Accessible Evidence

Any search of the literature, however sophisticated, is only as good as the literature it is searching. One of the principal concerns about the quality of the published literature is its susceptibility to publication bias. Publication bias is the preferential publication of studies with significantly positive findings, as compared to those with negative (nonsignificant) findings.4

There are two interpretations of bias that are germane to this topic. The first, referred to as the use of “bias” in the vernacular, implies preference. It is perhaps understandable that editors and reviewers might prefer studies that demonstrate an association over those that fail to do so. The second interpretation, more related to customary use of “bias” in medicine, is the introduction of a systematic distortion from the truth. The published literature is biased if it distorts reality.

There are two ways in which the literature could distort reality—by describing associations that do not truly exist, or refuting those that do. The first represents false-positive error, the second false-negative error. To some degree, the susceptibility of the literature to systematic distortions resides in the conventional thresholds of tolerance for type I (false-positive) and type II (false-negative) error.

Prevailing convention stipulates that alpha, the threshold for type I error, be set at 0.05, and beta, the threshold for type II error, at 0.2. Thus, when these conventions are honored, there is a 1 in 5 chance that any negative study is misleading, and a 1 in 20 chance that any positive study is misleading. As systematic distortion from the truth requires consideration of what constitutes truth, some conjecture about the “unbiased” medical literature is, at least theoretically, prerequisite to the assessment of publication bias. If all studies performed, regardless of outcome, were equally likely to be submitted for publication, one could posit that the unbiased literature should include 4 positive studies for every negative one, so that the probability of a false conclusion were equal between positive and negative reports.

There are problems with this position, however. Almost certainly, the probability of submission does vary with outcome, so that the journals are less likely to see negative studies.5,6,7 This distorts the denominator of papers reviewed, and alters the ratio of publication needed to achieve unbiased representation. Further, if 4 positive papers were published for every negative paper (so that the probability of a single misleading report in either category were the same), the weight of published evidence would still favor the positive outcome (i.e., there would be a preponderance of positive papers in the literature, even with no bias). There would, as well, be no way to know which positive outcome were true, and which a fluke.

This problem might be addressed if no paper were published without considering the ratio of positive to negative papers assessing the same outcome or association. One could only be confident of a true positive association when the positive papers outnumbered the negative by more than 4 to 1, and confident of the converse when negative papers occurred more often than 1 for each 4 positive papers. This approach would require systematic submission of all completed studies to a single evaluator who would see “the big picture” and publish the pattern of truth that emerged. However, this approach is not remotely feasible. And, extreme as it is, it is also flawed. Negative studies might be less likely than positive studies to be conducted to completion. Also compounding the situation is actual results that are well beyond the stipulated threshold for tolerable error. For example, p-values much lower than an alpha of 0.05 denote a risk of false-positive much less than 5%.

While publication bias refers to the preferential publication of positive studies, there is a distortion of the literature when studies purport to be negative but are perhaps not. This occurs because many studies, particularly prior to the 1990's, did not address adequately the issue of power. A negative outcome form an under-powered study is not truly negative; rather, it is as if a study were simply not done. When such findings are published, they suggest a conclusion about lack of association that is not justified.8,9,10,11,12,13

It is almost certainly true that there are distortions from “the truth” in the published literature. What they are, and how to detect, judge, or measure them is unclear. An appreciation for the limitations of even the most diligent search of the published evidence is useful for, if nothing else, conferring the humility evidence-based practice requires. Even if evidence is available, even if it can be found, even if it can be interpreted, even if it seems conclusive, and even if it pertains to the clinical need at hand, it may not reflect the truth, the whole truth, and nothing but the truth of association. To appreciate and apply evidence is to appreciate and respect its many limitations.

References
, , (eds). Handbook of Medical Informatics. Springer Verlag; 1997.
, The Internet for Physicians (Book with CD-ROM). Springer Verlag; 1999.
, (eds). Clinical Decision Support Systems: Theory and Practice. Springer Verlag; 1998.
Meta-Analysis, Decision Analysis, and Cost-Effectiveness Analysis. Methods for Quantitative Synthesis in Medicine.
2nd ed
. New York: Oxford University Press; 2000.
, , , , Empirical assessment of effect of publication bias on meta-analyses. BMJ. 2000;320:1574–1577. http://dx.doi.org/10.1136/bmj.320.7249.1574
, Publication bias in meta-analysis: Its causes and consequences. J Clin Epidemiol. 2000;53:207–216. http://dx.doi.org/10.1016/S0895-4356%2899%2900161-4
, , , Do certain countries produce only positive results? A systematic review of controlled trials. Control Clin Trials. 1998;19:159–166. http://dx.doi.org/10.1016/S0197-2456%2897%2900150-5
, How to detect effects: Statistical power and evidence-based practice in occupational therapy research. Am J Occup Ther. 1999;53:181–188. http://dx.doi.org/10.5014/ajot.53.2.181
, The statistical power of family practice research. Fam Pract Res J. 1993;13:105–111.
Statistical power of psychological research: What have we gained in 20 years?J Consult Clin Psychol. 1990;58:646–656. http://dx.doi.org/10.1037/0022-006X.58.5.646
, , The problem of the type II statistical error. Obstet Gynecol. 1995;86:857–859. http://dx.doi.org/10.1016/0029-7844%2895%2900251-L
, Empowering research: Statistical power in general practice research. Fam Pract. 1997;14:324–329. http://dx.doi.org/10.1093/fampra/14.4.324
, , , Low power, type II errors, and other statistical problems in recent cardiovascular research. Am J Physiol. 1997;273(1 Pt2):H487–H493.
, . Understanding and using the medical subject heading (MeSH) vocabulary to perform literature searches. JAMA. 1994;271:1103–1108. NLM URL: http://www.nlm.nih.gov Internet URL: http://www.nlm.nih.gov/pubs/factsheets/factsheets.html. http://dx.doi.org/10.1001/jama.1994.03510380059038

## Appendix B: The Constraint of Resource Limitations

Considering Cost in Clinical Practice

In applying quantitative methods and a population-based perspective to patient care, no factor is more naturally, yet less comfortably, drawn into consideration than cost. Cost is an unavoidable element in both clinical practice and public health,1 and a convenient quantifier of efficiency, and to some degree even quality of care.2,3 Yet any notion of equating pain and suffering, living and dying, with dollars causes the Hippocratic hairs on a clinician's neck to lift in suspicion and disdain.

One might wish that we were at liberty to ignore cost, that resources were unlimited. If considerations of cost were truly damaging to clinical care, to devotion to quality, then it's elimination from clinical decision making would be a good thing. But is that so?

Consider a situation in which all health care is free. Not just free to a particular consumer or payer, but actually free, due to limitless resources. In such a context, a patient with a poor clinical response could, and perhaps should, be sent to every relevant expert, receive every relevant therapy. Yet what if the probability of a response from each successive referral, each successive treatment, were diminished relative to the last? With unlimited resources, a declining probability of benefit would not, in and of itself, be cause to withhold any service. Perhaps the only reason to do so would be increasing risk from those very services.

But if the disease in question were serious, the risk of nonintervention could be so high that the risk of any intervention might be justified. But what of effectiveness?4 In an environment of unlimited resources, effectiveness, the reliability with which a given intervention produces the intended effect under real-world conditions, would be in the eyes of the beholder. Because it is almost impossible to prove conclusively that something does not work, there is always some small chance, even if it has never worked before, that it could. In situations of clinical desperation, one might be inclined to try something—anything—no matter how nominal the chances of benefit. At some point in this approach to care, unsound interventions will be brought into practice. There will be risk imposed without the justifying concurrence of potential benefit. And the ethical principals underlying medical practice will be violated.

The notion that cost could be the arbiter of medical ethics may seem far-fetched, but some currency, some medium is required so that the relative effectiveness of treatments can be compared.1 Alternatives to dollars could be considered. Cost could be measured in terms of risk or time or inconvenience, but dollars tend to be a surrogate for these. Benefit could be measured in functional status or survival or quality of life or quality-adjusted life years. But both the “costs” of medical interventions, and the attendant benefits, need to be measured in some way so that the relative merits of alternative interventions are comparable. In the real world, where resources are in fact limited, the dollar costs of procedures and therapies become a convenient, and appropriate, basis for such comparisons.2

Once the notion that medical practice standards require a measure of cost becomes acceptable, the distinction between public and private cost arises. Cost is of profound importance to public health, where the impact of resource limitations is felt acutely.5 In the care of the individual patient, resource limitations are generally an abstraction in the US health care system where money to pay for interventions is the only medium likely to be in short supply. Yet the aggregated costs of individual patient care become the costs of caring for the public, and practice patterns in the aggregate have an enormous impact on expenditures. Ultimately, interventions of great potential benefit, both to individuals and the public, are abandoned due to resource limitations.

Cost-Benefit Analysis

The simplest and generally least acceptable means of assessing cost tradeoffs in clinical practice is cost-benefit analysis. The objectionable element in such an approach is the need to measure both cost and benefit in dollars. Thus, if the benefit expected is clinical, such as symptom resolution or extension of life, a translation of the benefit into dollar terms would be required to determine the cost-benefit implications of the intervention. Cost-benefit analysis is used to determine if an intervention will save or cost money.

There are situations in which a cost-benefit analysis makes sense. In an occupational setting, there is likely to be a strong interest in overall profitability. Health-related interventions for employees will be fine up to a point, but beyond a certain point they would be prohibited by cost and the erosion of profit. But if an intervention, such as stress reduction counseling for employees, translated into fewer days missed and greater productivity, the costs of the intervention might be more than offset by the resultant gains. In such a situation, consideration of cost might lead to interventions that would otherwise never be made available.

Another situation in which cost considerations can enhance services is prevention. Clinical preventive services all cost something. But in many instances, the cost of not providing them is higher still. An example is provided by a cost-benefit analysis of influenza vaccination in the occupational setting.6 When a clinical benefit is associated with an actual and measurable reduction in costs, a compelling argument is advanced for the provision of new services.

Cost-Effectiveness Analysis

Often, dollar savings from an intervention are not expected. A clear example is renal dialysis.7 The intervention is life prolonging, even life saving, but costly. These costs are sustained and accumulated over time, with little to no economic return on the investment. At the same time, the value of such costly clinical interventions is clear in human terms: reduced suffering, increased survival.

One means of comparing financial cost to clinical benefit is to assess cost-effectiveness. Cost-effectiveness considers the cost in dollars per pertinent outcome, such as life saved or illness prevented. Such an analysis is undertaken when the clinical outcome has been established as a social priority. Cost-effectiveness permits comparisons among alternative strategies to determine which achieves the desired outcome at the lowest cost.8 The greatest generation of pertinent outcomes for each dollar spent, or alternatively the methods leading to the least dollars spent per unit outcome achieved, are considered the most cost-effective. Such an analysis does not require that clinical endpoints be cast in dollar terms, nor that an intervention can produce savings, but merely that its costs be justified by the magnitude of its clinical benefit.

Of note, cost-effectiveness and quality measures often, although not always, correlate well. Poor clinical outcomes and complications of suboptimal care are costly. The efficient achievement of clinical recovery through optimal use of available resources tends to be both desirable professionally and cost-effective. The lowest cost means of achieving a clinical endpoint is apt to be the most direct, most efficient, most reliable means, and as such clinically and financially preferable to the alternatives.

Standards of cost-effectiveness are contextually defined and bound. In the US, interventions with costs in the range of $50,000 per year of life extended are widely supported. Costlier interventions are generally considered unattractive.9 Programs are adopted by the World Health Organization (WHO) for developing countries only with evidence of dramatically greater cost-effectiveness, in the range of$10 per year of life saved.10 Obviously the adoption of a global standard of cost-effectiveness in health care delivery would require massive reallocations of resources.

Cost-Utility Analysis

A variation on the concept of cost-effectiveness is cost utility. Whereas cost-effectiveness assumes that the clinical outcome is constant and seeks to demonstrate the least costly means of achieving it, a utility measure permits the outcome side of the formula to vary as well. The prevailing utility measure is a composite of both quality and length of life gained, or quality adjusted life years (QALY's).3,11 Typically, a year of perfect health is expressed as 1, with reductions in either time or the degree of intact health yielding portions of a quality adjusted life year (QALY). For example, living 10 years with a 50% reduction in quality of life due to a disability would result in a QALY measure of 5. Living 5 years in perfect health would yield the same measure, as would living 20 years with such severe functional limitations that the quality of life were reduced by 75% (to “0.25” the quality of intact health). Dollars spent per QALY gained is a measure of cost utility, allowing variation in both costs and benefits to be considered without specifying a dollar value for those benefits.

Marginal Costs and Benefits

The relevance of cost considerations to clinical decision making is mediated by the influence of marginal cost and benefit, or utility. That some additional benefit might result from some additional expenditure is often the case, but once conventional practices have been applied, both the probability and magnitude of additional benefit from additional spending tend to wane. Thus, the benefit at the margin of usual practice, or the marginal benefit, will decline with the extent of the evaluation or treatment. Often it is in just such situations that the costs per additional intervention rise steeply—as the more extravagant the management plan, the more costly its components. As noted above, failure to consider cost at all can result in an inattention to the imbalance between marginal costs and utilities, so that very expensive interventions of little potential value are applied as routinely as low cost interventions of high potential value. In situations where cost and potential benefit rise together, or fall together, considerations of cost-effectiveness are required to determine how to proceed. In situations where costs rise and marginal utilities fall, the intervention is apt to be of questionable merit. But when marginal utility rises as costs fall, there is a clear indication to proceed.

The tension between costs and utilities can be captured in a 2 × 2 table used to portray “cost-consequence space,” as shown in Box B.1.

Other Considerations

Among other important considerations in cost evaluations are perspective, externalities, discounting, and projections. Perspective refers to the group (or individual) for whom a particular cost (or benefit) is relevant. For example, society (or a given population) might benefit if a pharmaceutical company made a drug or vaccine available free in a developing country, while the company (or its consumers) would bear the cost. The trade-offs between cost and benefit would be quite different depending on whose perspective were considered. Externalities refer to costs or benefits that occur as an indirect result of a particular intervention, often affecting a different individual or group. For example, if the care of a premature baby in the neonatal intensive care unit requires one or both parents to miss work, the lost productivity is an additional, indirect, or external cost. Discounting refers to the erosion of monetary benefits or costs that are postponed until some future time. Projections in economic analysis are an effort to anticipate the financial impact of time trends; Markov models (see Chapter 10) are used for this purpose.

Economic analysis in medicine and health care is far too expansive a topic to receive comprehensive treatment here. The intent of this appendix is to highlight the importance of cost considerations in evidence-based practice and to provide examples of application. The reader interested in a thorough discussion of the subject should consult other sources.12

References
, , , . Evidence-based medicine and cost-effectiveness. J Health Care Finance. 1999;26:14–23.
. Evidence-based medicine: Is translating evidence into practice a solution to the cost-quality challenges facing medicine?Jt Comm J Qual Improv. 1999;25:480–485.
, , , et al. Systematic overview of cost-utility assessments in oncology. J Clin Oncol. 2000;18:3302–3317.
, . Defining quality health care with outcomes assessment while achieving economic value. Top Health Inf Manage. 2000;20:44–54.
. The once and future application of cost-effectiveness analysis. Jt Comm J Qual Improv. 1999;25:455–461.
, , , et al. Economic impact of providing workplace influenza vaccination. A model and case study application at a Brazilian pharma-chemical company. Pharmacoeconomics. 1999;16(5 Pt 2):563–576. http://dx.doi.org/10.2165/00019053-199916050-00012
, , . Economic evaluation and end-stage renal disease: From basics to bedside. Am J Kidney Dis. 2000;36:12–28. http://dx.doi.org/10.1053/ajkd.2000.9013
, . Testing the validity of cost-effectiveness models. Pharmacoeconomics. 2000;17:501–513. http://dx.doi.org/10.2165/00019053-200017050-00007
, , , et al. Cost effectiveness of early discharge after uncomplicated acute myocardial infarction. N Engl J Med. 2000;342:749–755. http://dx.doi.org/10.1056/NEJM200003163421101
. Economic aspects of vaccines and immunizations. C R Acad Sci III. 1999;322:989–994. http://dx.doi.org/10.1016/S0764-4469%2800%2987198-X
, , . Cost-utility analysis of screening intervals for diabetic retinopathy in patients with type 2 diabetes mellitus. JAMA. 2000;283:889–896. http://dx.doi.org/10.1001/jama.283.7.889
. Meta-Analysis, Decision Analysis, and Cost-Effectiveness Analysis. Methods for Quantitative Synthesis in Medicine.
2nd ed
. New York: Oxford University Press; 2000.

## Appendix C: Clinically Useful Measures Derived from the 2 × 2 Contingency Table

 a = true-positive cases b = false-positive cases c = false-negative cases d = true-negative cases (a + b) = all test-positives (a + c) = all disease-positives (b + d) = all disease-negatives (c + d) = all test-negatives (a + b + c + d) = sample size (a + c)/(a + b + c + d) = prevalence (b + d)/(a + b + c + d)= proportion disease free (1 − prevalence) a/(a + c) = sensitivity (the proportion1of those with disease correctly detected by the test) d/(b + d) = specificity (the proportion1 of those without disease correctly identified as disease-free by the test) c/(a + c) = false-negative error rate (the proportion1 of those with disease with a negative test result; 1 − sensitivity) b/(b + d) = false-positive error rate (the proportion1 of those truly disease-free with a positive test result; 1 − specificity) a/[a + b) = positive predictive value (PPV) (the proportion1 of those with a positive test who truly have disease) b/(a + b) = proportion misleading positives2 (PMP) (the proportion1 of positive test results that are incorrect; 1 − PPV) d/(c + d) = negative predictive value (NPV) (the proportion1 of those with a negative test result who are truly disease-free) c/(c + d) = proportion misleading negatives2 (PMN) (the proportion1 of negative test results that are incorrect; 1 − NPV) b/a = false-positive index2 (FPI) (the ratio of false- to true-positives) c/d = false-negative index2 (FNI) (the ratio of false- to true-negatives) (a/c)/(b/d) = odds ratio [a/[a + b)]/[c/(c + d)] = risk ratio (relative risk)

Measures influenced by prevalence: positive predictive value; negative predictive value; proportion misleading positives; proportion misleading negatives; false positive index; false negative index

1 While identified as a proportion, the measure is typically expressed as a percentage.

2 Newly established term/measure.

Specific Measures Are More Readily Recalled if the Denominator Is Considered First
DenominatorMeasure
disease positives (a + c)sensitivity false-negative error rate
disease negatives (b + d)specificity false-positive error rate
test positives (a + b)positive predictive value proportion misleading positives
test negatives (c + d)negative predictive value proportion misleading negatives

## Text Sources

Books
, . Basic & Clinical Biostatistics.
2nd ed
. Norwalk, CT: Appleton & Lange; 1994.
. Clinical Decision Making. From Theory to Practice. Sudbury, MA: Jones and Bartlett Publishers; 1996.
, , . Clinical Epidemiology. The Essentials.
3rd ed
. Baltimore, MD: Williams & Wilkins; 1996.
, (ed). Evidence-Based Medicine. A Framework for Clinical Practice. Stamford, CT: Appleton & Lange; 1998.
, , . Fundamentals of Clinical Trials.
3rd ed
. St. Louis, MO: Mosby; 1996.
. Evidence-Based Healthcare. New York: Churchill Linvingstone; 1997.
, , , , , . Designing Clinical Research.
2nd ed
. Philadelphia, PA: Lippincott Williams & Wilkins; 2001.
, , . Epidemiology, Biostatistics, and Preventive Medicine. Philadelphia, PA: Saunders; 1996.
. Epidemiology, Biostatistics, and Preventive Medicine Review. Philadelphia, PA: Saunders; 1997.
. Intuitive Biostatistics. New York: Oxford University Press; 1995.
. Meta-Analysis, Decision Analysis, and Cost-Effectiveness Analysis. Methods for Quantitative Synthesis in Medicine.
2nd ed
. New York: Oxford University Press; 2000.
. Clinical Trials. Chichester, UK: Wiley; 1993.
, . Evidence-Based Family Medicine. Hamilton, Ontario: B.C. Decker Inc; 1998.
, , , . Clinical Epidemiology. A Basic Science for Clinical Medicine.
2nd ed
. Boston, MA: Little, Brown and Company; 1991.
, , , . Evidence-Based Medicine. How to Practice & Teach EBM. New York: Churchill Livingstone; 1997.
. The Right Test.
3rd ed
U.S. Preventive Services Task Force. Guide to Clinical Preventive Services.
2nd ed
. Alexandria, VA: International Medical Publishing; 1996.
Users' Guides to the Medical Literature
, , et al. Users' guides to the medical literature: XVII. How to use guidelines and recommendations about screening: Evidence-Based Medicine Working Group. JAMA. 1999;281:2029–2034. http://dx.doi.org/10.1001/jama.281.21.2029
, , , . Users' guides to the medical literature: XIX. Applying clinical trial results: A. How to use an article measuring the effect of an intervention on surrogate end points: Evidence-Based Medicine Working Group. JAMA. 1999;282:771–778. http://dx.doi.org/10.1001/jama.282.8.771
, , , . Users' guides to the medical literature: XIV How to decide on the applicability of clinical trial results to your patient: Evidence-Based Medicine Working Group. JAMA. 1998;279:545–549. http://dx.doi.org/10.1001/jama.279.7.545
, , , , . Users' guides to the medical literature: XIII. How to use an article on economic analysis of clinical practice: A. Are the results of the study valid? Evidence-Based Medicine Working Group. JAMA. 1997;277:1552–1557. http://dx.doi.org/10.1001/jama.1997.03540430064035
, . Users' guides to the medical literature: XXIII. Qualitative research in health care: A. Are the results of the study valid? Evidence-Based Medicine Working Group. JAMA. 2000a;284:357–362. http://dx.doi.org/10.1001/jama.284.3.357
, . Users' guides to the medical literature: XXIII. Qualitative research in health care: B. What are the results and how do they help me care for my patients? Evidence-Based Medicine Working Group. JAMA. 2000b;284:478–482. http://dx.doi.org/10.1001/jama.284.4.478
, , , et al. Users' Guides to the Medical Literature: XXV. Evidence-based medicine: Principles for applying the Users' Guides to patient care: Evidence-Based Medicine Working Group. JAMA. 2000;284:1290–1296. http://dx.doi.org/10.1001/jama.284.10.1290
, , , , , . Users' guides to the medical literature: XII. How to use articles about health-related quality of life: Evidence-Based Medicine Working Group. JAMA. 1997;277:1232–1237. http://dx.doi.org/10.1001/jama.1997.03540390062037
, , . Users' guides to the medical literature: II. How to use an article about therapy or prevention: A. Are the results of the study valid? Evidence-Based Medicine Working Group. JAMA. 1993;270:2598–2601. http://dx.doi.org/10.1001/jama.1993.03510210084032
, , . Users' guides to the medical literature: II. How to use an article about therapy or prevention: B. What were the results and will they help me in caring for my patients? Evidence-Based Medicine Working Group. JAMA. 1994;271:59–63. http://dx.doi.org/10.1001/jama.1994.03510250075039
, , , , , . Users' guides to the medical literature: IX. A method for grading health care recommendations: Evidence-Based Medicine Working Group. JAMA. 1995;274:1800–1804. http://dx.doi.org/10.1001/jama.1995.03530220066035
, , , . Users' guides to the medical literature: XVI. How to use a treatment recommendation: Evidence-Based Medicine Working Group and the Cochrane Applicability Methods Working Group. JAMA. 1999;281:1836–1843. http://dx.doi.org/10.1001/jama.281.19.1836
, , , , . Users' guides to the medical literature: VIII. How to use clinical practice guidelines: A. Are the recommendations valid? The Evidence-Based Medicine Working Group. JAMA. 1995;274:570–574. http://dx.doi.org/10.1001/jama.1995.03530070068032
, , . Users' guides to the medical literature: XXI. Using electronic health information resources in evidence-based practice: Evidence-Based Medicine Working Group. JAMA. 2000;283:1875–1879. http://dx.doi.org/10.1001/jama.283.14.1875
, , . Users' guides to the medical literature: III. How to use an article about a diagnostic test: A. Are the results of the study valid? Evidence-Based Medicine Working Group. JAMA. 1994a;271:389–391. http://dx.doi.org/10.1001/jama.1994.03510290071040
, , . Users' guides to the medical literature: III. How to use an article about a diagnostic test: B. What are the results and will they help me in caring for my patients? The Evidence-Based Medicine Working Group. JAMA. 1994b;271:703–707. http://dx.doi.org/10.1001/jama.1994.03510330081039
, , , . Users' guides to the medical literature: V How to use an article about prognosis: Evidence-Based Medicine Working Group. JAMA. 1994;272:234–237. http://dx.doi.org/10.1001/jama.1994.03520030076032
, , , , , . Users' guides to the medical literature: IV. How to use an article about harm: Evidence-Based Medicine Working Group. JAMA. 1994;271:1615–1619. http://dx.doi.org/10.1001/jama.1994.03510440075037
, , , . Users' Guides to the Medical Literature: XIX. Applying clinical trial results: B. Guidelines for determining whether a drug is exerting (more than) a class effect. JAMA. 1999;282(14): 1371–1377. http://dx.doi.org/10.1001/jama.282.14.1371
, , , . Users' guides to the medical literature: XX. Integrating research evidence with the care of the individual patient: Evidence-Based Medicine Working Group. JAMA. 2000;283:2829–2836. http://dx.doi.org/10.1001/jama.283.21.2829
, , , , , . Users' guides to the medical literature: XXII. How to use articles about clinical decision rules: Evidence-Based Medicine Working Group. JAMA. 2000;284:79–84. http://dx.doi.org/10.1001/jama.284.1.79
, . Users' guides to the medical literature: X. How to use an article reporting variations in the outcomes of health services: The Evidence-Based Medicine Working Group. JAMA. 1996a;275:554–558. http://dx.doi.org/10.1001/jama.1996.03530310060034
, . Users' guides to the medical literature: XI. How to use an article about a clinical utilization review: Evidence-Based Medicine Working Group. JAMA. 1996b;275:1435–1439. http://dx.doi.org/10.1001/jama.1996.03530420063038
, , , , . Users' guides to the medical literature: XIII. How to use an article on economic analysis of clinical practice: B. What are the results and will they help me in caring for my patients? Evidence-Based Medicine Working Group. JAMA. 1997;277:1802–1806. http://dx.doi.org/10.1001/jama.1997.03540460066034
, , . Users' guides to the medical literature: VI. How to use an overview: Evidence-Based Medicine Working Group. JAMA. 1994;272:1367–1371. http://dx.doi.org/10.1001/jama.1994.03520170077040
, , . Users' guides to the medical literature: I. How to get started: The Evidence-Based Medicine Working Group. JAMA. 1993;270:2093–2095. http://dx.doi.org/10.1001/jama.1993.03510170083036
, , , , . Users' Guides to the Medical Literature: XVIII. How to use an article evaluating the clinical impact of a computer-based clinical decision support system: JAMA. 1999;282:67–74. http://dx.doi.org/10.1001/jama.282.1.67
, . Users' guides to the medical literature: VII. How to use a clinical decision analysis: A. Are the results of the study valid? Evidence-Based Medicine Working Group. JAMA. 1995a;273:1292–1295. http://dx.doi.org/10.1001/jama.1995.03520400062046
, . Users' guides to the medical literature: VII. How to use a clinical decision analysis: B. What are the results and will they help me in caring for my patients? Evidence Based Medicine Working Group. JAMA. 1995b;273:1610–1613. http://dx.doi.org/10.1001/jama.1995.03520440064038
, , , , . Users' guides to the medical literature: XV How to use an article about disease probability for differential diagnosis: Evidence-Based Medicine Working Group. JAMA. 1999;281:1214–1219. http://dx.doi.org/10.1001/jama.281.13.1214
, , , , . Users' guides to the medical literature: XXIV. How to use an article on the clinical manifestations of disease: Evidence-Based Medicine Working Group. JAMA. 2000;284:869–875. http://dx.doi.org/10.1001/jama.284.7.869
, , , , . Users' guides to the Medical Literature: VIII. How to use clinical practice guidelines: B. What are the recommendations and will they help you in caring for your patients? The Evidence-Based Medicine Working Group. JAMA. 1995;274:1630–1632. http://dx.doi.org/10.1001/jama.1995.03530200066040
Other Articles
, , , , . Primer on medical decision analysis: Part 1—Getting started. Med Decis Making. 1997;17:123–125. http://dx.doi.org/10.1177/0272989X9701700201
, , , , . Primer on medical decision analysis: Part 2—Building a tree. Med Decis Making. 1997;17:126–135. http://dx.doi.org/10.1177/0272989X9701700202
, , , . Cost-effective diagnostic test sequencing. World J Surg. 1989;13:272–276. http://dx.doi.org/10.1007/BF01659033
, , , , . To test or not to test—to treat or not to treat: The decision-threshold approach to patient management. J Gen Intern Med. 1986;1:177–182. http://dx.doi.org/10.1007/BF02602334
, , , , . Primer on medical decision analysis: Part 4—Analyzing the model and interpreting the results. Med Decis Making. 1997;17:142–151. http://dx.doi.org/10.1177/0272989X9701700204
, . Imaging techniques for coronary artery disease: Current status and future directions. Clin Cardiol. 1997;20(6):526–532. http://dx.doi.org/10.1002/clc.4960200604
, , , , . Primer on medical decision analysis: Part 3—Estimating probabilities and utilities. Med Decis Making. 1997;17:136–141. http://dx.doi.org/10.1177/0272989X9701700203
, , , , . Primer on medical decision analysis: Part 5—Working with Markov processes. Med Decis Making. 1997;17:152–159. http://dx.doi.org/10.1177/0272989X9701700205
, , . Diagnosis of coronary artery disease in women: Roles of three dimensional imaging with magnetic resonance or positron emission tomography. Am J Card Imaging. 1996;10:78–88.
, , . Comparison of modalities to diagnose coronary artery disease. Semin Nucl Med. 1994;24:286–310. http://dx.doi.org/10.1016/S0001-2998%2805%2980020-0
, , . Sensitivity and specificity of helical computed tomography in the diagnosis of pulmonary embolism: A systematic review. Ann Intern Med. 2000;132:227–232.
. To test or not to test: A clinical dilemma. Theor Med. 1995;16:141–152. http://dx.doi.org/10.1007/BF00998541

## Epilogue

A leading clinician and clinical epidemiologist received the bad news that her 76-year-old father had suffered a large anterior wall myocardial infarction. She rushed to the hospital, where she learned he was hypotensive and in renal failure. She was confronted with desperate options, daunting decisions—intra-aortic balloon pump, emergency bypass surgery, or hemodialysis. Her father had been clear that he never wanted “heroic measures” if and when it came to that. Yet, she thought, that pertained when his condition was truly irreversible. Was it? His cardiac function might improve over time. His kidneys might recover from acute tubular necrosis. Struggling with the possibilities, her own emotions and those of her family, and the weightiness of the decisions to be made, the clinician turned to a trusted friend and colleague for advice.

“But,” said the friend, “you're a leading clinical epidemiologist. You know all there is to know about the probabilities of different outcomes with different procedures, the potential risks and benefits involved, the operating characteristics of various tests and procedures. If anyone has the means to make a tough decision like this one, it's you.”

“Oh, come on!” replied the clinician, clearly drained and exasperated. “This is important!”

With thanks to: David Paltiel, PhD, Yale University School of Medicine, Department of Epidemiology and Public Health.