Measuring Patient Outcomes


Marie T. Nolan & Victoria Mock

  • Citations
  • Add to My List
  • Text Size

  • Chapters
  • Front Matter
  • Back Matter
  • Subject Index
  • Copyright

    View Copyright Page


    To my husband, Patrick G. Nolan, and to my parents, Alice M. Gould, RN, BA, and James P. Gould, MD, my first and best example of the multidisciplinary team.

    —Marie T. Nolan

    To my husband, Quent, and my son, Grey, for their loving support and the joy they bring to my life.

    —Victoria Mock

    Finally, to the clinical nurses and graduate students who vigilantly measure patient outcomes to continually improve the care of their patients.


    It is a privilege to introduce a book that deals with the practical issues and basic tools of measuring patient outcomes. This book is a clear, step-by-step primer to organize the reader's thinking about kinds of outcome assessment, the nature and scope of measurement, and the context and processes of interdisciplinary care—all necessary components to understand before engaging in the tasks entailed in patient outcomes research. The clinical examples operationalize the how-to's into real-life patient care situations. To advance practice nurses everywhere: This is the book you have been waiting for! It is the guide for measuring patient outcomes as we enter the 21st century.

    Throughout the 1980s and the 1990s, advance practice nurses (APNs) and nurse researchers employed in clinical settings struggled to appropriately identify, elicit, aggregate, and illustrate data from existing clinical sources to present the most accurate depiction of patient care outcomes. Often, they were faced with clinical data that did not translate across patient populations, that were incomplete or missing, and that were suspect as to their conceptual clarity when viewed within the context of diagnostic groups, gender, or age.

    Two challenges were paramount for the pioneers of patient outcomes research. The first challenge was to understand fully, embrace, and integrate the concept of measurement into the clinical arena. This meant moving the clinical environment beyond the proliferation of unreliable and invalid questionnaires to one of devising or selecting reliable, valid, and practical tools for use with patients. The second challenge was to understand fully, embrace, and integrate the concept of cost into the practice environment. This meant transforming from holding negative ideas about cost-related decisions into learning how to collect, analyze, and relate cost data to practice and care outcome variables. During the 1980s and 1990s, we evolved from an opinion-related process of writing letters to influence decisions for or against an impending architectural or practice change that affected the care we gave to our patients to a data-driven process of explicitly and systematically collecting data to influence decisions.

    As I read this book, I was reminded of the influences of the previous decades that have led us to measure patient outcomes in the 21st century. The atmosphere of the 1980s was one focused on the integration of research in practice, including partnerships of APNs and academicians, with a huge influx of doctorally prepared nurses hired into clinical settings to lead this integration. While the scientific community sent out a plea for the development of reliable and valid tools to measure abstract care concepts, desperately needed in clinical research, we celebrated the congressional approval of the National Center for Nursing Research (NCNR) at the National Institutes of Health (NIH). More than 100 APNs, administrators, educators, and nurse researchers were mentored in their development and testing of psychometrically sound instruments through the Measurement of Clinical and Educational Nursing Outcomes Project, administered by Drs. Carolyn Waltz and Ora Strickland at the University of Maryland, School of Nursing and funded by the Division of Nursing, Special Projects Branch. These tools now serve as a rich resource for today's studies of patient outcomes. Under the leadership of NCNR Director Dr. Ada Sue Hinshaw, NCNR led the way in examining a research agenda for outcomes research in the early 1990s. It was during this time that NCNR became an institute (NINR) within the NIH. NCNR invited collaboration with the Agency for Health Care Policy and Research, and together they sponsored the first national conference on patient outcomes research: Examining the Effectiveness of Nursing Practice. This NIH-supported conference brought together scholars from throughout the United States to discuss the seedlings of the fruit we bear today. The visibility and influence of nurse scholars as viable members of the interdisciplinary scientific health community in all these events set the stage for a collaborative effort in addressing the measurement of patient outcomes in the clinical arenas of the 21st century.

    Doctors Marie Nolan and Victoria Mock have given us a wonderful gift to guide us into the 21st century. This book brings together the experience of health care professionals and the latest knowledge about what is necessary to

    • Identify and measure patient outcomes successfully and practically
    • Analyze and illustrate reliable and valid outcomes data using the latest technology and computer packages
    • Use the results of systematic, interdisciplinary outcomes studies to improve care and plan future studies

    Measuring Patient Outcomes delivers an enthusiastic message about the strength of interdisciplinary teams and the power of reliable and valid data to make decisions that affect the care outcomes of our patients. This is the message of patient outcomes research in the 21st Century!

    Veronica F.Rempusheski, PhD, RN, FAAN, Associate Professor, University of Rochester School of Nursing, Rochester, New York


    In our roles as nurse researchers at The Johns Hopkins Hospital and faculty members at The Johns Hopkins University School of Nursing, we are commonly called on to assist case managers, clinical nurse specialists, nurse managers, and students in the measurement of patient outcomes. We noted that although there is a great deal of literature describing managed care systems, case management, and the role of the advance practice nurse and nurse manager in the coordination of care, there was little that explained how to measure outcomes in a practical sense. Over time, we created materials to guide nurses and other clinicians through this process. These materials included information on how to select outcomes to improve, how to find instruments to accurately measure these outcomes, and how to select the statistical procedures to analyze these outcomes. Because successful measurement of patient outcomes requires a minimal level of data competence, we also have taught clinicians and students how to use the statistical analysis computer program, SPSS®, to analyze patient outcomes data and create graphics to summarize these data. This book is a compilation of the materials that we have used to teach patient outcomes measurement with additional information to help the reader get started.

    The first part of this book serves as a primer for nurses and other health professionals who are ready to begin measuring patient outcomes. The second part of the book includes a series of case studies describing successful patient outcomes projects. The examples we have selected range from the straightforward assessment of length of stay and readmission rate before and after the introduction of a care pathway to the more complicated efforts to assess the impact of swimming on central venous catheter infection in children with cancer. Some chapters represent the starting point for outcomes measurement whereas other chapters are built on previous work. Because we are continually impressed with the resourcefulness of clinicians who overcome barriers to the measurement of outcomes, we have included projects that highlight their strategies for success. Although some project leaders accessed existing clinical databases, two project leaders were assisted by graduate students who conducted chart reviews and one project leader was assisted by a medical student who performed data entry. One project leader used an informational cancer web site to identify an outcome of importance to cancer patients. Another coordinated the efforts of health professionals across five settings to study factors that were related to leg wound complications following coronary artery bypass graft. We know readers will benefit from these examples of outcomes measurement, and we wish our readers much success in their own outcomes measurement achievements.

    About the Authors

    Marie T. Nolan, DNSc, is Assistant Professor at The Johns Hopkins University School of Nursing and Nurse Researcher for Nursing Administration at The Johns Hopkins Hospital. Her research focuses on patient outcomes in serious illness. She has studied the stress experienced by patients and their families awaiting heart transplantation and patient decision making in patients with terminal illness. She teaches graduate courses in patient outcomes measurement and case management. She is coeditor (with Sharon M. Augustine) of Transplantation Nursing: Acute and Long-Term Management (1995). Her recent work has been published in the Journal of Professional Nursing, Nursing Outlook, Nursing Economics, Dimensions of Critical Care, Heart & Lung, and Academic Medicine.

    Victoria Mock, DNSc, RN, AOCN, is Associate Director at The Johns Hopkins University School of Nursing, and at the School of Medicine. She is also Director of Nursing Research at the Johns Hopkins Comprehensive Cancer Center. She was appointed an American Cancer Society Professor of Oncology Nursing in 1999. Her research focuses on symptom management and quality of life. She is principal investigator of a multi-institutional clinical trial investigating the effects of exercise on fatigue and other symptoms during cancer treatment. She is also coinvestigator of another multisite study—the Oncology Nursing Society (ONS) Acute Care/Radiation Therapy Outcomes Research Project. She received the 1997 ONS Schering Excellence in Cancer Nursing Research Award and the 1998 ONS Ortho Biotech FIRE (Fatigue Initiative in Research and Education) Cancer Excellence Award. Her work appears in numerous books and journals.

    List of Case Study Authors

    Donna L. Brannan, RN, MSN

    Nursing Staff Assistant

    Neurosciences and Psychiatry Nursing

    The Johns Hopkins Hospital

    Baltimore, MD

    Laura J. Burke, RN, PhD, FAAN

    Director of Nursing Research

    Aurora Health Care—Metro Region

    West Allis, WI

    Susan M. Cohen, DSN, RN

    Associate Professor

    Director, Adult and Family Nurse Practitioner Program

    School of Nursing

    Yale University

    New Haven, CT

    JoAnn Coleman, RN, MS, ACNP-CS, AOCN

    Acute Care Nurse Practitioner, Pancreas and Biliary Surgery

    Department of Surgical Nursing

    The Johns Hopkins Hospital

    Baltimore, MD

    Philene Cromwell, MS, RN

    Advanced Practice Nurse

    Associate Clinical Professor

    University of Rochester School of Nursing

    University of Rochester Medical Center

    Division of Pediatric Hematology/Oncology

    Rochester, NY

    Christine A. Engstrom, RN, MS, CRNP, AOCN

    Adult Nurse Practitioner

    Department of Medical Oncology

    Baltimore Veterans Administration Medical Center

    Baltimore, MD

    Maura Goldsborough, RN, MSN

    Research Nurse Program Coordinator

    Cardiac Surgery

    The Johns Hopkins Hospital

    Baltimore, MD

    Andrea O. Hollingsworth, PhD, RN

    Associate Professor

    Director, Undergraduate Nursing Program

    College of Nursing

    Villanova University

    Villanova, PA

    Marinell H. Jernigan, RN, EdD

    Thesis Chair

    University of North Carolina at Charlotte

    Charlotte, NC

    David N. Korones, MD

    Assistant Professor of Pediatrics

    University of Rochester School of Medicine and Dentistry

    University of Rochester Medical Center

    Division of Pediatric Hematology/Oncology

    Rochester, NY

    Penny Marschke, RN, MSN

    Clinical Research Program Coordinator

    Department of Urology

    The Johns Hopkins University School of Medicine

    Doctoral Candidate in Nursing

    University of Maryland

    Baltimore, MD

    Barbara F. Paegelow, A.R.T.

    Clinical Data Specialist

    Aurora Health Care – Metro Region

    Milwaukee, WI

    Jacqueline Robbins, MS, RN CPNP

    Pediatric Nurse Practitioner

    Associate Clinical Professor

    University of Rochester School of Nursing

    University of Rochester Medical Center

    Division of Pediatric Hematology/Oncology

    Rochester, NY

    Judith M. Rohde, RN, MSN

    Director of Nursing

    Neuroscience and Psychiatry Nursing

    The Johns Hopkins Hospital

    Baltimore, MD

    Pamela T. Rudisill, RN, MSN, CCRN, ANP

    Associate Executive Director of Nursing

    Lake Norman Regional Medical Center

    Mooresville, NC

    Suzanne J. Rumble, RN, MSN, CCRN

    Nursing Instructor

    Presbyterian Hospital School of Nursing

    Charlotte, NC

  • Glossary

    Agency for Health Care Policy and Research—a federal agency that supports and disseminates research dealing with patient outcomes; provides online data on the Internet on length of stay and mean charges by diagnostic-related groups.

    ANOVA/ANCOVA (analysis of variance/analysis of covariance)—a statistical procedure to determine differences among two or more group means. In ANCOVA, the subjects are made statistically equal on a potentially confounding variable (the covariable).

    Benchmark—to compare the care processes and outcomes of one group of patients with those of a similar group of patients in another organization known for having the best practice in this type of care. This method can be used to identify an organization's areas of weakness and to prioritize the patient outcomes on which to focus.

    Boxplot—a summary plot based on the median, quartiles, and extreme values. The box represents the interquartile range that contains the 50% of values. The whiskers are lines that extend from the box to the highest and lowest values, excluding outliers. A line across the box indicates the median.

    Capitation—a method of reimbursement for health care services that includes a flat fee per subscriber regardless of the amount of health services provided.

    Case manager—a person who oversees the care of groups of patients through collaboration with other health professionals, coordination of services, and elimination of inefficiencies in the delivery of care.

    Chi-square2)—a statistical test used to determine if there is a difference between frequencies of two categorical variables, such as the difference between groups on gender, education, types of employment, or religion.

    Contingency table—a cross-tabulation table exhibiting the frequencies of two variables within a sample.

    Copyright—the legal right to publish, reproduce, or market a creative work, such as a computer program or an artistic or literary composition.

    Correlation—describes the strength of the relationship between two variables.

    Correlation coefficient—numerical account of the relationship between two variables; values can range from −1 to 1, where 0 = no relationship, 1 = the strongest positive relationship (A + B increase together), and −1 = strongest negative or inverse relationship (as A increases, B decreases).

    Critical pathway—a written schedule of treatments and interventions for patients within the same diagnostic group or who are undergoing a similar procedure. The objective of the pathway is to decrease fragmentation of services and cost of care.

    Cronbach's alpha—a reliability index that assesses the extent to which items in an instrument, such as questions on a survey, are related. On a scale of 0 to 1.0, higher values reflect a higher degree of internal consistency.

    Descriptive statistics—statistics used to describe and summarize data (e.g., frequency, mean, and percentages).

    Diagnostic-related group—a system of categorizing patients for the purpose of reimbursement according to factors such as diagnosis, procedure, and comorbidities; first developed within the Medicare program.

    Effectiveness of care—the extent to which interventions achieve the desired outcomes when applied in the practice setting.

    Efficacy of care—the extent to which interventions achieve the desired outcome in a controlled setting.

    External validity—the generalizability of the study's results to other samples or settings.

    Fee-for-services—a method of reimbursement for health care services in which payment is provided for the health care services delivered.

    Frequency distribution—a display of numerical values from lowest to highest with a count of the number of times each value was observed.

    Health maintenance organization (HMO)—a health care organization that uses primary physicians to control subscriber access to care within designated health care facilities; the objective is to decrease costs by limiting the care provided.

    Heterogeneity—the degree to which objects are dissimilar or varied with respect to some attribute.

    Histogram—displays the distribution of a quantitative variable by showing the relative concentration of data points along different intervals or sections of the scale on which the data are measured.

    Homogeneity—the degree to which objects are similar or equivalent with respect to some attribute.

    Hypothesis—a statement predicting the relationship between two or more variables.

    Inferential statistics—used for making generalizations from samples to populations and in estimating or testing population parameters or hypotheses using samples of the population.

    Instruments, tools, and measures—the methods or devices used to collect data (e.g., questionnaire).

    Internal validity—the extent to which it can be inferred that the independent variable in a study, rather than extraneous factors, is responsible for the outcomes (dependent variables).

    Interrater reliability—the extent that two independent raters assign the same value for a characteristic (a specific variable) being measured.

    Interval—level of measurement in which an attribute of a variable is rank ordered with equal numerical distances between points on a scale (e.g., temperature).

    Kruskall-Wallis test—a nonparametric statistical test used to detect differences among three or more independent groups when the data are an ordinal level of measurement.

    Level of significance (or alpha)—the risk of rejecting a null hypothesis when it is true (Type I error). The cutoff point resolves whether the sample tested is of the same or a different population.

    Mann-Whitney U test—a nonparametric statistical test used to detect differences between two independent groups based on ordinal data.

    Mean—the arithmetic average score computed by adding all scores and dividing by the number of scores.

    Median—the middle score in a set of numbers; half of the set is higher than and half is lower than the middle score.

    Mode—most frequently occurring score in a data set.

    National Committee for Quality Assurance—the private, not-for-profit organization that accredits managed care organizations. This organization examines several key indicators of quality of care, such as patient satisfaction and frequency of childhood immunization.

    Nominal—lowest level of measurement. Variables with similar characteristics are assigned a label. If a number label is used, it is arbitrary (not used for arithmetic purposes).

    Nonparametric test—statistical test used with a sample when the rigorous assumptions about a normal distribution of scores on a variable are not met; generally used with data measured on nominal and ordinal scales.

    Normal distribution—a continuous frequency distribution of scores that is bell-shaped, symmetrical, and unimodal.

    Null hypothesis—states that there is no relationship between the variables being studied; a form of the hypothesis used for statistical testing.

    Ordinal—level of measurement that involves rank ordering of data where the intervals (differences in order of magnitude between ranks) are not equal.

    Outcome—the end result of care or a measurable change in the health status or behavior of patients. Outcomes may be clinical, functional, financial, or perceptual.

    Outlier—extreme numerical score that lies remotely from the center of a distribution of scores.

    p Value or alpha level—the level of probability that the obtained results are a result of chance (probability of a Type I error). Generally, .05 alpha level means that generalizing to the real world or in repeating the study one should find these same differences 95 out of 100 times; thus, it is unlikely that the differences are due to chance.

    Parametric test—a statistical test used when the assumption of normal distribution of scores or values can be met and the level of measurement is interval or ratio; considered more powerful than nonparametric tests (e.g., t test).

    Pearson's r (Pearson's product-moment correlation coefficient)—a parametric statistical test used to determine magnitude and direction of the relationship between variables.

    Population (target)—the total set of objects that have some common attribute and meet the specific criteria for a research study.

    Power—the probability that a statistical test will detect a significant relationship among variables if one exists.

    Preferred provider organization—a health care network involving contracts with groups of physicians and other health professionals to provide care for a reduced rate in return for an increased volume of patients; less restrictive than an HMO because no gatekeeper physician is used to limit access to care.

    Range—a measure of variability obtained by subtracting the lowest from the highest score in a distribution of scores.

    Ratio—highest level of measurement. There are equal numerical distances between points on the scale and a true meaningful zero point (e.g., heart rate).

    Reliability—the consistency or repeatability with which an instrument measures the attribute it is designed to measure.

    Sample—the portion of the population that is selected to participate in a research study.

    Scatterplot—a graph in which one numeric variable is plotted against another.

    Selectivity—the ability of an instrument to correctly identify the presence of an attribute being measured and to distinguish it from similar attributes.

    Sensitivity—the ability of an instrument to detect differing amounts of the attribute being measured.

    Spearman's rho (Spearman's rank-order correlation coefficient)—a non-parametric statistical test used to determine relationships between two variables measured at the ordinal level.

    Standard deviation (square root of the variance)—the average deviation of scores from the mean of all scores in the data set.

    Statistical significance—the extent to which the results are not due to chance at some specific level of probability.

    Subjects—individuals participating in a research study.

    t test—parametric statistical test used for analyzing differences between the mean scores of two groups.

    Type I error—error that occurs when the null hypothesis is rejected when it is actually true (a relationship is concluded to exist when one does not exist).

    Type II error—error that occurs when the null hypothesis is accepted when it should have been rejected (a relationship is concluded not to exist when one does exist in reality).

    Validity—the degree to which an instrument measures what it is supposed to measure.

    Variable—any characteristic or attribute that changes or varies within the population under study.

    Variance—The square of the standard deviation. A large variance means that the data are widely distributed.

    • Loading...
Back to Top

Copy and paste the following HTML into your website