De testimonio

I recently discovered the online slideshow (with audio) version of Professor Sir Michael Rawlins’ 2008 Harveian Oration titled “De testimonio: On the evidence for decisions about the use of therapeutic interventions”.

Now, I’ve nodded off in more than my share of lectures on medical statistics and clinical trials, so believe me when I claim that this lecture is FANTASTIC. This lecture is a must for anyone interested in the evidence base of medicine and the problems (and future) of medical research.

Rawlins shows that the randomised controlled trial (RCT) is not the “be-all and end-all” of evidence in medicine. He shows that the hierarchical grading of levels of evidence (RCTs at the top with case reports at the bottom) is overly simplistic and irrational. He advocates using case-by-case judgement to assess the value of a given study:

Such judgements relate to – the extent to whether each component of the evidence base is “fit-for-purpose”. Is it reliable? Is it likely to be generalisable? Do the benefits outweigh the harms?

Professor Sir Michael Rawlins, from the 2008 Harveian Oration

Rawlins (author of 19 RCTs) addresses the important limitations of the ‘gold standard’ RCTs, namely:

1. Inappropriateness
2. Utility of the null hypothesis
3. Theories of probability
4. Generalisability of the results
5. Resource implications

In particular, he convincingly argues for the use of a Bayesian approach (one that takes into account pre-existing knowledge) rather than the ‘Frequentist‘ approach of starting with the ‘null hypothesis‘ when interpreting the significance of the results of a clinical trial. After all, if bookmakers use the Bayesian approach it must work… Goodbye p-values!

Rawlins goes on to show that evidence from ‘lower down’ the hierarchy (such as observational studies), when ‘fit for purpose’, can negate the need for an RCT. After all, if we needed an RCT to prove the value of every medical intervention we wouldn’t be using penicillin for infections, thyroxine for myxoedema, or insulin for diabetes mellitus. Rawlins’ examples here are fascinating.

Any belief that the controlled trial is the only way would mean not that the pendulum had swung too far but that it had come right off the hook

Austin Bradford Hill, the architect of the RCT.

If you’ve just read what I’ve written and it all seems like ‘gobbledy-gook’, don’t fret. Rawlins will make it all crystal clear for you.

God forbid that truth should be confined to mathematical demonstration.

William Blake

The lecture can also be downloaded in print form (thanks to the Royal College of Physicians). The quotations above are all stolen from Rawlins’ superb talk.

• Rawlins M. De Testimonio: on the evidence for decisions about the use of therapeutic interventions. Clin Med (Lond). 2008 Dec;8(6):579-88.