Receiver Operating Characteristic Curve

Reviewed and revised 27 August 2015

OVERVIEW

  • Receiver Operating Characteristic (ROC) curves plot sensitivity versus false positive rate for several values of a diagnostic test
  • By convention, sensitivity (the proportion of true positive results) is shown on the y axis, going from 0 to 1 (0–100%) and 1-specificity (the proportion of false positive results) is shown on the x axis, going from 0 to 1 (0–100%)
  •  illustrates the trade-off between sensitivity and specificity in tests that produce results on a numerical scale, rather than as an absolute positive or negative result

USES

  • determination of the cut-off point at which optimal sensitivity and specificity are achieved (decision thresholds)
  • visual and quantitative (AUC) assessment of the diagnostic accuracy of a test (can be used for comparisons)
  • Can be used to generate confidence intervals for sensitivity and specificity and likelihood ratios

ADVANTAGES

  • Simple and graphical
  • Represents accuracy over the entire range of the test
  • Independent of prevalence
  • Tests may be compared on the same scale
  • Allows comparison of accuracy between several tests

CUTOFF VALUES

  • these are chosen according to whether one wants to maximise the sensitivity (e.g. D-dimer) or specificity (e.g. CTPA) of the test
  • e.g. Troponin T levels in the diagnosis of MI
    — several different TNT plasma concentrations would have been chosen and compared against a gold standard in diagnosing MI (ECHO: regional wall abnormalities)
    — the sensitivity and specificity of each chosen TNT level would have been plotted
  • the ideal cut off is one which picks up a lot of disease (high sensitivity) but has very few false positives (high specificity)
  • one method assumes that the best cut-off point for balancing the sensitivity and specificity of a test is the point on the curve closest to the (0, 1) point, i.e. high up on the left-hand side of the graph resulting in a large AUC method
  • an alternative method is to use the Youden index (J), where J is defined as the maximum vertical distance between the ROC curve and the diagonal or chance line

ACCURACY AND AREA UNDER THE CURVE (AUC)

  • the higher the AUC, the more accurate test
  • AUC = 0.5 means the test is no better than chance alone (plotted as a straight diagonal line)
  • AUC = 1.0 means the test has perfect accuracy

LIKELIHOOD RATIOS

  • the tangent at a point on the ROC curve corresponds to the likelihood ratio for a single test value represented by that point
  • the slope between the origin and a point on the curve corresponds to the positive likelihood ratio using the point as a criterion for positivity;
  • the slope between two points on the curve corresponds to the likelihood ratio for a test result in a defined level bounded by the two points

References and Links

LITFL

Journal articles

  • Akobeng AK. Understanding diagnostic tests 3: Receiver operating characteristic curves. Acta Paediatr. 2007 May;96(5):644-7. Epub 2007 Mar 21. PMID: 17376185. [Free Full Text]
  • Choi BC. Slopes of a receiver operating characteristic curve and likelihood ratios for a diagnostic test. Am J Epidemiol. 1998 Dec 1;148(11):1127-32. PMID: 9850136 [Free Full Text]
  • Fan J, Upadhye S, Worster A. Understanding receiver operating characteristic (ROC) curves. CJEM. 2006 Jan;8(1):19-20. PMID: 17175625. [Free Full Text]

CCC 700 6

Critical Care

Compendium

Chris is an Intensivist and ECMO specialist at the Alfred ICU in Melbourne. He is also the Innovation Lead for the Australian Centre for Health Innovation at Alfred Health and Clinical Adjunct Associate Professor at Monash University. He is a co-founder of the Australia and New Zealand Clinician Educator Network (ANZCEN) and is the Lead for the ANZCEN Clinician Educator Incubator programme. He is on the Board of Directors for the Intensive Care Foundation and is a First Part Examiner for the College of Intensive Care Medicine. He is an internationally recognised Clinician Educator with a passion for helping clinicians learn and for improving the clinical performance of individuals and collectives.

After finishing his medical degree at the University of Auckland, he continued post-graduate training in New Zealand as well as Australia’s Northern Territory, Perth and Melbourne. He has completed fellowship training in both intensive care medicine and emergency medicine, as well as post-graduate training in biochemistry, clinical toxicology, clinical epidemiology, and health professional education.

He is actively involved in in using translational simulation to improve patient care and the design of processes and systems at Alfred Health. He coordinates the Alfred ICU’s education and simulation programmes and runs the unit’s education website, INTENSIVE.  He created the ‘Critically Ill Airway’ course and teaches on numerous courses around the world. He is one of the founders of the FOAM movement (Free Open-Access Medical education) and is co-creator of litfl.com, the RAGE podcast, the Resuscitology course, and the SMACC conference.

His one great achievement is being the father of two amazing children.

On Twitter, he is @precordialthump.

| INTENSIVE | RAGE | Resuscitology | SMACC

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.