Receiver Operating Characteristic Curve

Reviewed and revised 27 August 2015

OVERVIEW

• Receiver Operating Characteristic (ROC) curves plot sensitivity versus false positive rate for several values of a diagnostic test
• By convention, sensitivity (the proportion of true positive results) is shown on the y axis, going from 0 to 1 (0–100%) and 1-specificity (the proportion of false positive results) is shown on the x axis, going from 0 to 1 (0–100%)
•  illustrates the trade-off between sensitivity and specificity in tests that produce results on a numerical scale, rather than as an absolute positive or negative result

USES

• determination of the cut-off point at which optimal sensitivity and specificity are achieved (decision thresholds)
• visual and quantitative (AUC) assessment of the diagnostic accuracy of a test (can be used for comparisons)
• Can be used to generate confidence intervals for sensitivity and specificity and likelihood ratios

ADVANTAGES

• Simple and graphical
• Represents accuracy over the entire range of the test
• Independent of prevalence
• Tests may be compared on the same scale
• Allows comparison of accuracy between several tests

CUTOFF VALUES

• these are chosen according to whether one wants to maximise the sensitivity (e.g. D-dimer) or specificity (e.g. CTPA) of the test
• e.g. Troponin T levels in the diagnosis of MI
  — several different TNT plasma concentrations would have been chosen and compared against a gold standard in diagnosing MI (ECHO: regional wall abnormalities)
  — the sensitivity and specificity of each chosen TNT level would have been plotted
• the ideal cut off is one which picks up a lot of disease (high sensitivity) but has very few false positives (high specificity)
• one method assumes that the best cut-off point for balancing the sensitivity and specificity of a test is the point on the curve closest to the (0, 1) point, i.e. high up on the left-hand side of the graph resulting in a large AUC method
• an alternative method is to use the Youden index (J), where J is defined as the maximum vertical distance between the ROC curve and the diagonal or chance line

ACCURACY AND AREA UNDER THE CURVE (AUC)

• the higher the AUC, the more accurate test
• AUC = 0.5 means the test is no better than chance alone (plotted as a straight diagonal line)
• AUC = 1.0 means the test has perfect accuracy

LIKELIHOOD RATIOS

• the tangent at a point on the ROC curve corresponds to the likelihood ratio for a single test value represented by that point
• the slope between the origin and a point on the curve corresponds to the positive likelihood ratio using the point as a criterion for positivity;
• the slope between two points on the curve corresponds to the likelihood ratio for a test result in a defined level bounded by the two points

References and Links

LITFL

• CCC — Understanding diagnostic tests

Journal articles

• Akobeng AK. Understanding diagnostic tests 3: Receiver operating characteristic curves. Acta Paediatr. 2007 May;96(5):644-7. Epub 2007 Mar 21. PMID: 17376185. [Free Full Text]
• Choi BC. Slopes of a receiver operating characteristic curve and likelihood ratios for a diagnostic test. Am J Epidemiol. 1998 Dec 1;148(11):1127-32. PMID: 9850136 [Free Full Text]
• Fan J, Upadhye S, Worster A. Understanding receiver operating characteristic (ROC) curves. CJEM. 2006 Jan;8(1):19-20. PMID: 17175625. [Free Full Text]