Receiver Operating Characteristic (Roc) Curve

The receiver operationg characteristic (ROC) curve is a two-dimensional measure of classification performance depicting the trade-off between sensitivity and specificity. It is used in the analysis of a diagnostic test or screening test that classifies experimental units into two categories such as diseased (D)ornondiseased (D). Screening and laboratory test results are usually reported as a continuous variable. For example, the risk variable serum concentration of creatine phosphokinase for myocardial infarction (D) is approximately normally distributed varying from less than 100 units/ml to greater than 4,000 units/ml. The serum concentration of creatine phosphokinase for those without myocardial infarction (D) also has an approximate normal distribution but has a different mean (see Figure 1). Suppose that we dichotomize the serum concentration by some cutpoint so that values above it represent positive (+) test results and values below it represent negative (−) test results. We may now define the following misclassification rates: falsepositiverateP(+ |D) is the probability of classifying a noncase as positive, true-positive rate (sensitivity) P(+ |D) is the probability of classifying a case as positive, false-negative rate P(− |D) is the probability of classifying a case as negative, and true-negative rate (specificity) P(− |D) is the probability of classifying a noncase as negative. As shown in the table below, different cutpoints lead to tests with different levels of misclassification rates. For example, when the cutoff value of the serum concentration is chosen to be 5.4, calculation from the two normal distributions gives P(+ |D) = .725747 (see the shaded area in Figure 1), P(− |D) = .27425, P(− |D) = 991802, and P(+ |D) = .008198 (see the small double-shaded area in Figure 1). Note that if we lowered the cutoff value, we would decrease the false-negative rate, but we would also increase the false-positive rate. Similarly, if we raised the cutoff value, we would decrease the falsepositive rate, but we would increase the false-negative rate (see Figure 1).

An ROC curve is obtained by plotting the falsepositiverate(1 − specificity) against the true-positive rate (sensitivity) for a series of cutpoints defined by the test (see Figure 2). It shows the trade-off between the true-positive rate and the false-positive rate of a test (any increase in sensitivity will be accompanied by a decrease in specificity and conversely). In statistical terminology, it is the plot of Type I error against the power. This ROC plot is representative of those plotting one conditional distribution function against another found to be useful in epidemiology and other health sciences, which includes plotting the posttest [Page 896]probability of disease given the test is positive against the pretest probability of disease, plotting the positive predictive value against the point prevalence rate, and plotting the total time on test against the distribution function of the duration time. The closer the curve follows the left-hand border and then the top border of the ROC space, the more accurate the test. The closer the curve comes to the 458 diagonal of the ROC space, the less accurate the test. The slope of the tangent line at any point on the ROC curve may be accurately estimated by spline interpolation and differentiation. The slope of the tangent line at a cutpoint gives the likelihood ratio (LR) for that value of the test. So, by choosing the slope of the tangent to the ROC curve to equal the LR that will minimize the total cost of making false-positive and false-negative errors, one can identify the optimal cutoff values. Such LR turns out to be the ratio of the product of the net cost of treating nondiseased patients and the pretest probability of no disease to the product of the net benefit of treating diseased patients and the pretest probability of disease.

...