Meta-analysis and Systematic Review

Reviewed and revised 17 May 2014

OVERVIEW

• Meta-analysis is a tool for quantitative systematic review of observational studies and controlled trials that weights available evidence based on the numbers of patients included, the effect size, and often statistical tests of agreement with other trials
• Systematic reviews involve a detailed and comprehensive plan and search strategy derived a priori, with the goal of reducing bias by identifying, appraising, and synthesizing all relevant studies on a particular topic (as opposed to a narrative review)
• Systematic reviews may or may not involve meta-analysis
• Meta-analysis avoids Simpson’s paradox, in which a consistent effect in constituent trials is reversed when results are simply pooled

ROLES IN EBM

• Increases statistical power through increased sample size
• Resolve uncertainty when studies disagree
• Improve estimates of effect size
• Establish questions for future RCTs

PROS AND CONS OF META-ANALYSIS

Advantages

• pooled estimate of effect
• allows for an objective appraisal of evidence
• may reduce the probability of false negative results
• heterogeneity between study results may be explained
• avoids Simpson’s paradox

Disadvantages

• heterogeneity of study demographics, methods, results, quality
• selection of studies and data from studies may be biased
• use of summary data rather than individual data
• inclusion and exclusion criteria may not be detailed
• publication bias (many negative studies are not published)

BASIC METHOD

• systematic literature search performed and research into possible unpublished trials
• data analysed in terms of quality and heterogeneity
• large trials weighted most heavily
• OR used (most commonly) and combined using random effects model
• graphical displays of OR, CI’s and pooled OR (Forrest Plot – see below)
• Funnel plot used to identify possible publication bias
• findings presented as NNT
• positive meta-analysis findings should be confirmed with large RCT

EXAMPLE OF FORREST PLOT

• The horizontal lines – confidence intervals of the OR
• The position of the square – point estimate of the OR
• The size of the square – the weight of the study

-> the depicted data suggests the treatment is not more effective than control as the 95% confidence limits of the combined OR crosses the vertical line

ASSESSING QUALITY OF THE META-ANALYSIS

Assess for:

• Research questions clearly defined
• Definition of inclusion criteria for studies
• adequacy of the search protocol
• assessment of methodological quality of the included studies
• calculation of a pooled estimate
• plot of the results (Forest Plot)
• measurement of heterogeneity
• assessment of publication bias (Funnel Plot)

References and links

Journal articles

• Reade MC, Delaney A, Bailey MJ, Angus DC. Bench-to-bedside review: avoiding pitfalls in critical care meta-analysis–funnel plots, risk estimates, types of heterogeneity, baseline risk and the ecologic fallacy. Crit Care. 2008;12(4):220. PMC2575558.

FOAM and web resources

• Absolutely maybe — 5 key things to know about meta-analysis (2014)