Decision Curve Analysis

Biostatistical Approaches and Their Drawbacks

Biostatistical analysis of prediction models, diagnostic tests, and molecular markers is largely concerned with accuracy. Such metrics have been criticized by decision analysts as having little clinical value. An accurate test, prediction model, or marker is, in general, more likely to be useful than one less accurate, but it is difficult to know for any specific situation whether the accuracy of a test, prediction model, or marker is high enough to warrant implementation in the clinic. For example, if a new blood marker for prostate cancer increased the area under the curve (AUC) of an established prediction model from .77 to .79, would this be sufficient to justify its clinical use?

Decision Analytic Approaches and Their Drawbacks

Decision analysis formally incorporates the consequences of test results and can therefore be used to determine whether use of a prediction model, diagnostic test, or molecular marker to aid decision making would improve clinical outcome. A typical approach is to construct a decision tree as shown in Figure 1. We denote probabilities and values of each health outcome, respectively, as pxy and as bxy, where x is an indicator for the test result and y is the indicator for disease. To determine the optimal decision, the values of each outcome are multiplied by their probability and summed for each decision; the decision with the highest expected value is chosen.

To obtain pxys for a statistical model or molecular marker, the analyst has to choose a cut point in order to dichotomize results into positive and negative. Different analysts can disagree about the appropriate cut point, entailing that the analysis may need to be run several times for a range of [Page 270]reasonable alternatives. Choice of bxys can be even more difficult. A bxy may require data from the literature that can be hard to come by or controversial; moreover, a bxy may require judgments that may reasonably vary from patient to patient. The need for additional data may be one of the reasons why the number of biostatistical evaluations of tests, prediction models, and markers dwarfs the number of decision analyses: In one systematic review of more than 100 papers on cancer markers, researchers failed to find a single decision analysis.

...