Risk Adjustment of Outcomes

Risk adjustment facilitates meaningful comparisons of outcomes of different groups of individuals by accounting for differences across the groups in baseline characteristics that could affect their outcomes. Groups can be defined in countless ways depending on comparisons of interest, such as patients admitted to one hospital versus another, individuals receiving Treatment X versus Treatment Y, and persons in one socioeconomic stratum versus other strata. In observation studies, individuals are not randomly assigned to the groups, and, for reasons that are sometimes poorly understood, those in one group may differ in significant ways from those in other groups. These differences might affect the likelihood that the individuals will experience the outcomes of interest. For example, if patients at the neighborhood hospital are older on average than those admitted to the downtown teaching facility, it is unclear whether the higher [Page 986]mortality rate at the community hospital is caused by worse care or older patients. In these types of observation studies, the goal of risk adjustment is to take into account—or adjust for—the effect of important risk factors, so that analysts can more confidently attribute differences in outcomes to the variable of interest (e.g., hospital quality) than to underlying characteristics of the individuals in the groups (e.g., older vs. younger age).

The gold standard for determining the effects of an intervention is the randomized controlled trial (RCT). Randomization ensures that, on average, individuals assigned to receive an inter vention have similar baseline characteristics (unmeasured as well as measured attributes) to those randomized to the control group that does not receive the intervention. Thus, an RCT that uses an intention-to-treat analysis—that is, compares outcomes of all persons assigned to the intervention versus control group regardless of the treatments patients actually receive—provides a theoretically sound basis for concluding that differences in outcomes are caused by the intention to use a different treatment.

However, RCTs are neither ethical nor practical in many situations. The key ethical concern is an unwillingness to substitute an untried treatment for one that is widely used, even if its utility is largely untested, or a treatment with well-documented but modest benefits. The numerous practical impediments include high costs and lengthy time horizons required to conduct RCTs, challenges of human subjects protections and obtaining truly informed consent, and refusals of physicians and patients to participate in randomized studies even when widely used standard therapies have little rigorous scientific evidence supporting their benefits. Another difficulty in planning RCTs is the tension between answering a narrowly defined question well (e.g., by excluding patients with extensive comorbidities) versus the ability to generalize findings more broadly (e.g., by including the full range of patients who might be candidates for the treatment should the RCT find it effective). Finally, RCTs are not well suited for answering many important questions where it is infeasible to randomly assign individuals to different groups, such as comparing the quality of care at Hospital X versus Hospital Y by contrasting patients' outcomes at these respective institutions. Randomly assigning patients to different hospitals is not possible in today's environment. When data from RCTs are not available, inferences about the value of interventions must come from observational studies. Risk-adjusted comparisons have become a standard method for using observational data to study treatment effectiveness, as well as to facilitate quality monitoring and support other health policy initiatives, such as pay-for-performance programs being implemented by government and private payers. These value-based purchasing programs aim to pay higher quality and more efficient providers more than providers offering lower value: Credible judgments about provider quality or efficiency require adjusting for differences in the types of patients seen by each provider.

...