Reviewed and revised 26 August 2015
- Types of study design arranged by level of evidence, from low to high
LEVELS OF EVIDENCE
- no control group
- no control group
- useful for uncommon events as cases can be collected over a long period of time and studied
- control patients are ‘matched’ for using some criteria (age, gender)
- begins with a definition of outcome or interest
- then looks backwards to identify risk factors associated with outcomes
- measures exposure to risk factors
- outcome = in case and controls
- odds ratio used to quantify risk
- data collected prospectively
- requires a long time to acquire data or outcome of interest
- relatively inefficient
- looks at exposures and then the development of disease
- relative risk used to quantify risk
- each patient acts as their own control
- all patients characteristics affecting outcome are equalised
- patients cross over from one treatment to the next
- usually done in a randomised fashion to diminish bias
- problems include: a wash out period required to eliminate effect of first treatment, carry over effects, period effect (deterioration over time), sequence effect (order of treatment effects outcome), patient drop out
- each patient acts their own control.
- all patients characteristics affecting outcome and equalized.
- effect of new drug or treatment on group of patients with base line data obtained then repeated after treatment.
RANDOMISED CONTROL TRIAL
- the RCT is a ‘gold standard’
- level I and II evidence
- allocates volunteers or subjects to one of two groups (ie. control & treatment group)
- subjects are , thus overcoming bias when samples are compared
- ensures descriptive characteristics are randomly distributed among groups and that any difference is due to chance alone.
Types of randomisation
- simple – no restriction on allocation (groups may be unequally sized)
- block – allocation is performed in blocks, so that groups are equally sized within each block.
- stratified – factors such as age, sex… are randomised separately, so that they are equally distributed among the groups.
- computer-generated random numbers are usually used to determine group that patient goes into.
- only level of evidence able to establish causation
- ability to assign and administer treatment or intervention in a precise, controlled way
- decreases selection bias and minimises confounding due to unequal distribution in a chosen population
- measurements can be chosen precisely making it easier to make observations consistently (especially parametric data)
- blinding is easier improving credibility and decreasing patient or observer bias
- controlling of group allocations enhances similarity of baseline features so it is easier to form basis for statistical hypothesis
- can make trial large -> may detect clinically relevant conclusions
- can have subgroup analysis enhancing usefulness for clinical practice
- a successful RCT with conclusive or inconclusive results is eminently publishable
- increased expense
- increased time – clinical practice may have evolved by the time the study is published
- difficulty organising/supervising if multiple sites & locations
- results may not always mimic real life treatment situation
- risk of choosing treatments or subjects whose consent is not valid or unethical treatment is involved
- is a small trial has very stringent parameters -> type II errors decreased at the expense of applicability for a chosen population
SYSTEMATIC REVIEW AND META-ANALYSIS
- meta-analysis = the mathematical process of combining numeric data from studies using similar treatments in a systematic manner
- the whole process = a systematic review
- pooled estimate of effect
- allows for an objective appraisal of evidence
- may reduce the probability of false negative results
- heterogeneity between study results may be explained
- heterogeneity of study demographics, methods, results, quality.
- selection of studies & data from studies may be biased
- use of summary data rather than individual data
- inclusion & exclusion criteria may not be detailed
- publication bias (many negative studies are not published)
- literature performed + research into possible unpublished trials
- data analysed in terms of quality and heterogeneity
- large trials weighted most heavily
- OR used & combined using random effects model
- graphical displays of OR, CI’s and pooled OR (Forrest Plot)
- findings presented as NNT
- positive meta-analysis findings should ideally be confirmed with large RCT
References and Links
- CCC — Diagnostic Tests in Research
- CCC — Levels and Grades of Evidence
- CCC — Randomised Controlled Trials
- CCC — Animal and laboratory studies
- CCC — Retrospective studies and chart reviews
- CCC — Before-and-after studies
- CCC — Cluster cross-over trials (TBC)
- CCC — Adaptive trial designs
- CCC — Meta-analysis and Systematic Review
Chris is an Intensivist and ECMO specialist at the Alfred ICU in Melbourne. He is also a Clinical Adjunct Associate Professor at Monash University. He is a co-founder of the Australia and New Zealand Clinician Educator Network (ANZCEN) and is the Lead for the ANZCEN Clinician Educator Incubator programme. He is on the Board of Directors for the Intensive Care Foundation and is a First Part Examiner for the College of Intensive Care Medicine. He is an internationally recognised Clinician Educator with a passion for helping clinicians learn and for improving the clinical performance of individuals and collectives.
After finishing his medical degree at the University of Auckland, he continued post-graduate training in New Zealand as well as Australia’s Northern Territory, Perth and Melbourne. He has completed fellowship training in both intensive care medicine and emergency medicine, as well as post-graduate training in biochemistry, clinical toxicology, clinical epidemiology, and health professional education.
He is actively involved in in using translational simulation to improve patient care and the design of processes and systems at Alfred Health. He coordinates the Alfred ICU’s education and simulation programmes and runs the unit’s education website, INTENSIVE. He created the ‘Critically Ill Airway’ course and teaches on numerous courses around the world. He is one of the founders of the FOAM movement (Free Open-Access Medical education) and is co-creator of litfl.com, the RAGE podcast, the Resuscitology course, and the SMACC conference.
His one great achievement is being the father of three amazing children.
On Twitter, he is @precordialthump.