De testimonio
I recently discovered the online slideshow (with audio) version of Professor Sir Michael Rawlins’ 2008 Harveian Oration titled “De testimonio: On the evidence for decisions about the use of therapeutic interventions”.
Now, I’ve nodded off in more than my share of lectures on medical statistics and clinical trials, so believe me when I claim that this lecture is FANTASTIC. This lecture is a must for anyone interested in the evidence base of medicine and the problems (and future) of medical research.
Rawlins shows that the randomised controlled trial (RCT) is not the “be-all and end-all” of evidence in medicine. He shows that the hierarchical grading of levels of evidence (RCTs at the top with case reports at the bottom) is overly simplistic and irrational. He advocates using case-by-case judgement to assess the value of a given study:
Such judgements relate to – the extent to whether each component of the evidence base is “fit-for-purpose”. Is it reliable? Is it likely to be generalisable? Do the benefits outweigh the harms?
Professor Sir Michael Rawlins, from the 2008 Harveian Oration
Rawlins (author of 19 RCTs) addresses the important limitations of the ‘gold standard’ RCTs, namely:
- Inappropriateness
- Utility of the null hypothesis
- Theories of probability
- Generalisability of the results
- Resource implications
In particular, he convincingly argues for the use of a Bayesian approach (one that takes into account pre-existing knowledge) rather than the ‘Frequentist‘ approach of starting with the ‘null hypothesis‘ when interpreting the significance of the results of a clinical trial. After all, if bookmakers use the Bayesian approach it must work… Goodbye p-values!
Rawlins goes on to show that evidence from ‘lower down’ the hierarchy (such as observational studies), when ‘fit for purpose’, can negate the need for an RCT. After all, if we needed an RCT to prove the value of every medical intervention we wouldn’t be using penicillin for infections, thyroxine for myxoedema, or insulin for diabetes mellitus. Rawlins’ examples here are fascinating.
Any belief that the controlled trial is the only way would mean not that the pendulum had swung too far but that it had come right off the hook
Austin Bradford Hill, the architect of the RCT.
If you’ve just read what I’ve written and it all seems like ‘gobbledy-gook’, don’t fret. Rawlins will make it all crystal clear for you.
God forbid that truth should be confined to mathematical demonstration.
William Blake
The lecture can also be downloaded in print form (thanks to the Royal College of Physicians). The quotations above are all stolen from Rawlins’ superb talk.
- Rawlins M. De Testimonio: on the evidence for decisions about the use of therapeutic interventions. Clin Med (Lond). 2008 Dec;8(6):579-88.
Chris is an Intensivist and ECMO specialist at the Alfred ICU in Melbourne. He is also a Clinical Adjunct Associate Professor at Monash University. He is a co-founder of the Australia and New Zealand Clinician Educator Network (ANZCEN) and is the Lead for the ANZCEN Clinician Educator Incubator programme. He is on the Board of Directors for the Intensive Care Foundation and is a First Part Examiner for the College of Intensive Care Medicine. He is an internationally recognised Clinician Educator with a passion for helping clinicians learn and for improving the clinical performance of individuals and collectives.
After finishing his medical degree at the University of Auckland, he continued post-graduate training in New Zealand as well as Australia’s Northern Territory, Perth and Melbourne. He has completed fellowship training in both intensive care medicine and emergency medicine, as well as post-graduate training in biochemistry, clinical toxicology, clinical epidemiology, and health professional education.
He is actively involved in in using translational simulation to improve patient care and the design of processes and systems at Alfred Health. He coordinates the Alfred ICU’s education and simulation programmes and runs the unit’s education website, INTENSIVE. He created the ‘Critically Ill Airway’ course and teaches on numerous courses around the world. He is one of the founders of the FOAM movement (Free Open-Access Medical education) and is co-creator of litfl.com, the RAGE podcast, the Resuscitology course, and the SMACC conference.
His one great achievement is being the father of three amazing children.
On Twitter, he is @precordialthump.
| INTENSIVE | RAGE | Resuscitology | SMACC