Power and Sample Size

Reviewed and revised 26 August 2015

OVERVIEW

• It is unethical and a waste of time and resources to embark on a study when there is a high chance of a false negative result (Type II error)
• The commonest cause of this is having a sample size that is too small
• The larger the sample size, the more likely it is that the true effect of the intervention will be demonstrated

‘What is the smallest sample I need to be almost certain of producing the true result?’

POWER

Definition of power

• Power is the chance of a study successfully demonstrating the ‘true’ result
• it is the probability of detecting a significant difference if one exists
• Power = 1 – the false negative rate
• Power = 1 – beta error
• it is the ability of avoiding a false negative result
• it is the likelihood of correctly rejecting the null hypothesis when it is false
• normally power is 80%, there is a 80% probability of detecting a difference if one exists and a 20% chance of a false negative result

Importance

• helps to determine sample size
• need to have adequate power to make sure patients aren’t subjected to risk without cause (time, economic and ethical reasons)
• 3. ensures we don’t recruit too many patients
• 4. adds to publishability

SAMPLE SIZE

Factors determining sample size

• alpha value = level of significance (normally 0.05, lower alpha requires larger sample size)
• beta-value = power (normally 0.05-0.2, smaller beta/higher the power then the larger sample size required)
• statistical test used (students T if n < 60, normal distribution if n > 60)
• variance of population (the greater the variance, the larger the sample required)
• effect size (the smaller the effect size sought, the larger the sample size required) — this should be based on previous studies, current clinical practice and clinical relevance

An ongoing issue in critical care research is that sample size calculations tend to under-estimate the sample size required because overall ICU mortality is improving.

• power calculations are often based on previous studies, with higher mortality rates in the past
• to have equivalent power, with lower overall mortalities, a larger sample size is typically necessary
• this bias is termed ‘delta inflation’ and results in false negative’ results (Type II error)

References and Links

Online Calculators

• Statistics Calculators — Power calculations and Sample Size

Journal articles

• Aberegg SK, Richards DR, O’Brien JM. Delta inflation: a bias in the design of randomized controlled trials in critical care medicine. Critical care (London, England). 14(2):R77. 2010. [pubmed] [free full text]
• Arnold BF, Hogan DR, Colford JM Jr, Hubbard AE. Simulation methods to estimate design power: an overview for applied research. BMC Med Res Methodol. 2011 Jun 20;11:94. PMC3146952.
• Godwin M. Hypothesis: the research page. Part 3: Power, sample size, and clinical significance. Can Fam Physician. 2001 Jul;47:1441-3, 1450-3. PMC2018536.
• Jones SR, Carley S, Harrison M. An introduction to power and sample size estimation. Emerg Med J. 2003 Sep;20(5):453-8. Review. Erratum in: Emerg Med J. 2004 Jan;21(1):126. PMC1726174.