Making RCTs Useful: The Role of Heterogeneity
Jack Iwashyna (@iwashyna) gave a fascinating presentation at the last ANZICS annual scientific meeting titled ‘Making RCTs Useful: The Role of Heterogeneity’. If you are interested in clinical research, have used a clinical guideline or believe yourself to be an evidence-based practitioner, then you should listen to this. It has already taken out the LITFL Review 213 ‘Ripper of the Week’ however I think it deserves its own post:
Jack and his colleagues have now published the paper that this talk is based on:
Iwashyna TJ, Burke JF, Sussman JB, Prescott HC, Hayward RA, Angus DC. Implications of Heterogeneity of Treatment Effect for Reporting and Analysis of Randomized Trials in Critical Care. American journal of respiratory and critical care medicine. 192(9):1045-51. 2015. [pubmed]
Randomized clinical trials (RCTs) are conducted to guide clinicians’ selection of therapies for individual patients. Currently, RCTs in critical care often report an overall mean effect and selected individual subgroups. Yet work in other fields suggests that such reporting practices can be improved. Specifically, this Critical Care Perspective reviews recent work on so-called “heterogeneity of treatment effect” (HTE) by baseline risk and extends that work to examine its applicability to trials of acute respiratory failure and severe sepsis. Because patients in RCTs in critical care medicine-and patients in intensive care units-have wide variability in their risk of death, these patients will have wide variability in the absolute benefit that they can derive from a given therapy. If the side effects of the therapy are not perfectly collinear with the treatment benefits, this will result in HTE, where different patients experience quite different expected benefits of a therapy. We use simulations of RCTs to demonstrate that such HTE could result in apparent paradoxes, including: (1) positive trials of therapies that are beneficial overall but consistently harm or have little benefit to low-risk patients who met enrollment criteria, and (2) overall negative trials of therapies that still consistently benefit high-risk patients. We further show that these results persist even in the presence of causes of death unmodified by the treatment under study. These results have implications for reporting and analyzing RCT data, both to better understand how our therapies work and to improve the bedside applicability of RCTs. We suggest a plan for measurement in future RCTs in the critically ill.
Huge kudos to ANZICS for releasing talks from the last annual scientific meeting as ‘FOAM’ vodcasts — look for more on the ANZICS Youtube channel. Remember to sign up and become a member of ANZICS if you work as a doctor in an Australasian ICU — trainees get the first year free 🙂
Further Reading
- A Practical Approach to Running a Scarce Resource Allocation Team (SRAT). Jack Iwashyna
- Should we put multiple COVID-19 patients on a single ventilator? Jack Iwashyna
- Making RCTs Useful: The Role of Heterogeneity. Jack Iwashyna
- Mastering Intensive Care 020 with Jack Iwashyna
SMILE 2
Better Healthcare
Chris is an Intensivist and ECMO specialist at The Alfred ICU, where he is Deputy Director (Education). He is a Clinical Adjunct Associate Professor at Monash University, the Lead for the Clinician Educator Incubator programme, and a CICM First Part Examiner.
He is an internationally recognised Clinician Educator with a passion for helping clinicians learn and for improving the clinical performance of individuals and collectives. He was one of the founders of the FOAM movement (Free Open-Access Medical education) has been recognised for his contributions to education with awards from ANZICS, ANZAHPE, and ACEM.
His one great achievement is being the father of three amazing children.
On Bluesky, he is @precordialthump.bsky.social and on the site that Elon has screwed up, he is @precordialthump.
| INTENSIVE | RAGE | Resuscitology | SMACC