Making RCTs Useful: The Role of Heterogeneity
Jack Iwashyna (@iwashyna) gave a fascinating presentation at the last ANZICS annual scientific meeting titled ‘Making RCTs Useful: The Role of Heterogeneity’. If you are interested in clinical research, have used a clinical guideline or believe yourself to be an evidence-based practitioner, then you should listen to this. It has already taken out the LITFL Review 213 ‘Ripper of the Week’ however I think it deserves its own post:
Jack and his colleagues have now published the paper that this talk is based on:
Iwashyna TJ, Burke JF, Sussman JB, Prescott HC, Hayward RA, Angus DC. Implications of Heterogeneity of Treatment Effect for Reporting and Analysis of Randomized Trials in Critical Care. American journal of respiratory and critical care medicine. 192(9):1045-51. 2015. [pubmed]
Randomized clinical trials (RCTs) are conducted to guide clinicians’ selection of therapies for individual patients. Currently, RCTs in critical care often report an overall mean effect and selected individual subgroups. Yet work in other fields suggests that such reporting practices can be improved. Specifically, this Critical Care Perspective reviews recent work on so-called “heterogeneity of treatment effect” (HTE) by baseline risk and extends that work to examine its applicability to trials of acute respiratory failure and severe sepsis. Because patients in RCTs in critical care medicine-and patients in intensive care units-have wide variability in their risk of death, these patients will have wide variability in the absolute benefit that they can derive from a given therapy. If the side effects of the therapy are not perfectly collinear with the treatment benefits, this will result in HTE, where different patients experience quite different expected benefits of a therapy. We use simulations of RCTs to demonstrate that such HTE could result in apparent paradoxes, including: (1) positive trials of therapies that are beneficial overall but consistently harm or have little benefit to low-risk patients who met enrollment criteria, and (2) overall negative trials of therapies that still consistently benefit high-risk patients. We further show that these results persist even in the presence of causes of death unmodified by the treatment under study. These results have implications for reporting and analyzing RCT data, both to better understand how our therapies work and to improve the bedside applicability of RCTs. We suggest a plan for measurement in future RCTs in the critically ill.
Huge kudos to ANZICS for releasing talks from the last annual scientific meeting as ‘FOAM’ vodcasts — look for more on the ANZICS Youtube channel. Remember to sign up and become a member of ANZICS if you work as a doctor in an Australasian ICU — trainees get the first year free 🙂
Further Reading
- A Practical Approach to Running a Scarce Resource Allocation Team (SRAT). Jack Iwashyna
- Should we put multiple COVID-19 patients on a single ventilator? Jack Iwashyna
- Making RCTs Useful: The Role of Heterogeneity. Jack Iwashyna
- Mastering Intensive Care 020 with Jack Iwashyna
SMILE 2
Better Healthcare
Chris is an Intensivist and ECMO specialist at the Alfred ICU in Melbourne. He is also a Clinical Adjunct Associate Professor at Monash University. He is a co-founder of the Australia and New Zealand Clinician Educator Network (ANZCEN) and is the Lead for the ANZCEN Clinician Educator Incubator programme. He is on the Board of Directors for the Intensive Care Foundation and is a First Part Examiner for the College of Intensive Care Medicine. He is an internationally recognised Clinician Educator with a passion for helping clinicians learn and for improving the clinical performance of individuals and collectives.
After finishing his medical degree at the University of Auckland, he continued post-graduate training in New Zealand as well as Australia’s Northern Territory, Perth and Melbourne. He has completed fellowship training in both intensive care medicine and emergency medicine, as well as post-graduate training in biochemistry, clinical toxicology, clinical epidemiology, and health professional education.
He is actively involved in in using translational simulation to improve patient care and the design of processes and systems at Alfred Health. He coordinates the Alfred ICU’s education and simulation programmes and runs the unit’s education website, INTENSIVE. He created the ‘Critically Ill Airway’ course and teaches on numerous courses around the world. He is one of the founders of the FOAM movement (Free Open-Access Medical education) and is co-creator of litfl.com, the RAGE podcast, the Resuscitology course, and the SMACC conference.
His one great achievement is being the father of three amazing children.
On Twitter, he is @precordialthump.
| INTENSIVE | RAGE | Resuscitology | SMACC