Meta-analysis and Systematic Review
Reviewed and revised 17 May 2014
OVERVIEW
- Meta-analysis is a tool for quantitative systematic review of observational studies and controlled trials that weights available evidence based on the numbers of patients included, the effect size, and often statistical tests of agreement with other trials
- Systematic reviews involve a detailed and comprehensive plan and search strategy derived a priori, with the goal of reducing bias by identifying, appraising, and synthesizing all relevant studies on a particular topic (as opposed to a narrative review)
- Systematic reviews may or may not involve meta-analysis
- Meta-analysis avoids Simpson’s paradox, in which a consistent effect in constituent trials is reversed when results are simply pooled
ROLES IN EBM
- Increases statistical power through increased sample size
- Resolve uncertainty when studies disagree
- Improve estimates of effect size
- Establish questions for future RCTs
PROS AND CONS OF META-ANALYSIS
Advantages
- pooled estimate of effect
- allows for an objective appraisal of evidence
- may reduce the probability of false negative results
- heterogeneity between study results may be explained
- avoids Simpson’s paradox
Disadvantages
- heterogeneity of study demographics, methods, results, quality
- selection of studies and data from studies may be biased
- use of summary data rather than individual data
- inclusion and exclusion criteria may not be detailed
- publication bias (many negative studies are not published)
BASIC METHOD
- systematic literature search performed and research into possible unpublished trials
- data analysed in terms of quality and heterogeneity
- large trials weighted most heavily
- OR used (most commonly) and combined using random effects model
- graphical displays of OR, CI’s and pooled OR (Forrest Plot – see below)
- Funnel plot used to identify possible publication bias
- findings presented as NNT
- positive meta-analysis findings should be confirmed with large RCT
EXAMPLE OF FORREST PLOT
- The horizontal lines – confidence intervals of the OR
- The position of the square – point estimate of the OR
- The size of the square – the weight of the study
-> the depicted data suggests the treatment is not more effective than control as the 95% confidence limits of the combined OR crosses the vertical line
ASSESSING QUALITY OF THE META-ANALYSIS
Assess for:
- Research questions clearly defined
- Definition of inclusion criteria for studies
- adequacy of the search protocol
- assessment of methodological quality of the included studies
- calculation of a pooled estimate
- plot of the results (Forest Plot)
- measurement of heterogeneity
- assessment of publication bias (Funnel Plot)
References and links
Journal articles
- Reade MC, Delaney A, Bailey MJ, Angus DC. Bench-to-bedside review: avoiding pitfalls in critical care meta-analysis–funnel plots, risk estimates, types of heterogeneity, baseline risk and the ecologic fallacy. Crit Care. 2008;12(4):220. PMC2575558.
FOAM and web resources
- Absolutely maybe — 5 key things to know about meta-analysis (2014)
Critical Care
Compendium
Chris is an Intensivist and ECMO specialist at the Alfred ICU in Melbourne. He is also a Clinical Adjunct Associate Professor at Monash University. He is a co-founder of the Australia and New Zealand Clinician Educator Network (ANZCEN) and is the Lead for the ANZCEN Clinician Educator Incubator programme. He is on the Board of Directors for the Intensive Care Foundation and is a First Part Examiner for the College of Intensive Care Medicine. He is an internationally recognised Clinician Educator with a passion for helping clinicians learn and for improving the clinical performance of individuals and collectives.
After finishing his medical degree at the University of Auckland, he continued post-graduate training in New Zealand as well as Australia’s Northern Territory, Perth and Melbourne. He has completed fellowship training in both intensive care medicine and emergency medicine, as well as post-graduate training in biochemistry, clinical toxicology, clinical epidemiology, and health professional education.
He is actively involved in in using translational simulation to improve patient care and the design of processes and systems at Alfred Health. He coordinates the Alfred ICU’s education and simulation programmes and runs the unit’s education website, INTENSIVE. He created the ‘Critically Ill Airway’ course and teaches on numerous courses around the world. He is one of the founders of the FOAM movement (Free Open-Access Medical education) and is co-creator of litfl.com, the RAGE podcast, the Resuscitology course, and the SMACC conference.
His one great achievement is being the father of three amazing children.
On Twitter, he is @precordialthump.
| INTENSIVE | RAGE | Resuscitology | SMACC