Power and Sample Size
Reviewed and revised 26 August 2015
OVERVIEW
- It is unethical and a waste of time and resources to embark on a study when there is a high chance of a false negative result (Type II error)
- The commonest cause of this is having a sample size that is too small
- The larger the sample size, the more likely it is that the true effect of the intervention will be demonstrated
‘What is the smallest sample I need to be almost certain of producing the true result?’
POWER
Definition of power
- Power is the chance of a study successfully demonstrating the ‘true’ result
- it is the probability of detecting a significant difference if one exists
- Power = 1 – the false negative rate
- Power = 1 – beta error
- it is the ability of avoiding a false negative result
- it is the likelihood of correctly rejecting the null hypothesis when it is false
- normally power is 80%, there is a 80% probability of detecting a difference if one exists and a 20% chance of a false negative result
Importance
- helps to determine sample size
- need to have adequate power to make sure patients aren’t subjected to risk without cause (time, economic and ethical reasons)
- 3. ensures we don’t recruit too many patients
- 4. adds to publishability
SAMPLE SIZE
Factors determining sample size
- alpha value = level of significance (normally 0.05, lower alpha requires larger sample size)
- beta-value = power (normally 0.05-0.2, smaller beta/higher the power then the larger sample size required)
- statistical test used (students T if n < 60, normal distribution if n > 60)
- variance of population (the greater the variance, the larger the sample required)
- effect size (the smaller the effect size sought, the larger the sample size required) — this should be based on previous studies, current clinical practice and clinical relevance
An ongoing issue in critical care research is that sample size calculations tend to under-estimate the sample size required because overall ICU mortality is improving.
- power calculations are often based on previous studies, with higher mortality rates in the past
- to have equivalent power, with lower overall mortalities, a larger sample size is typically necessary
- this bias is termed ‘delta inflation’ and results in false negative’ results (Type II error)
References and Links
Online Calculators
- Statistics Calculators — Power calculations and Sample Size
Journal articles
- Aberegg SK, Richards DR, O’Brien JM. Delta inflation: a bias in the design of randomized controlled trials in critical care medicine. Critical care (London, England). 14(2):R77. 2010. [pubmed] [free full text]
- Arnold BF, Hogan DR, Colford JM Jr, Hubbard AE. Simulation methods to estimate design power: an overview for applied research. BMC Med Res Methodol. 2011 Jun 20;11:94. PMC3146952.
- Godwin M. Hypothesis: the research page. Part 3: Power, sample size, and clinical significance. Can Fam Physician. 2001 Jul;47:1441-3, 1450-3. PMC2018536.
- Jones SR, Carley S, Harrison M. An introduction to power and sample size estimation. Emerg Med J. 2003 Sep;20(5):453-8. Review. Erratum in: Emerg Med J. 2004 Jan;21(1):126. PMC1726174.
Critical Care
Compendium
Chris is an Intensivist and ECMO specialist at the Alfred ICU in Melbourne. He is also a Clinical Adjunct Associate Professor at Monash University. He is a co-founder of the Australia and New Zealand Clinician Educator Network (ANZCEN) and is the Lead for the ANZCEN Clinician Educator Incubator programme. He is on the Board of Directors for the Intensive Care Foundation and is a First Part Examiner for the College of Intensive Care Medicine. He is an internationally recognised Clinician Educator with a passion for helping clinicians learn and for improving the clinical performance of individuals and collectives.
After finishing his medical degree at the University of Auckland, he continued post-graduate training in New Zealand as well as Australia’s Northern Territory, Perth and Melbourne. He has completed fellowship training in both intensive care medicine and emergency medicine, as well as post-graduate training in biochemistry, clinical toxicology, clinical epidemiology, and health professional education.
He is actively involved in in using translational simulation to improve patient care and the design of processes and systems at Alfred Health. He coordinates the Alfred ICU’s education and simulation programmes and runs the unit’s education website, INTENSIVE. He created the ‘Critically Ill Airway’ course and teaches on numerous courses around the world. He is one of the founders of the FOAM movement (Free Open-Access Medical education) and is co-creator of litfl.com, the RAGE podcast, the Resuscitology course, and the SMACC conference.
His one great achievement is being the father of three amazing children.
On Twitter, he is @precordialthump.
| INTENSIVE | RAGE | Resuscitology | SMACC