STATISTICS An introduction to power and sample size …

1 STATISTICSAn introduction to power and sample size estimationS R Jones, S Carley, M Med J2003;20:453 458 The importance of power and sample size estimation forstudy design and Understand power and sample size Understand why power is an important part ofboth study design and Understand the differences between sample sizecalculations in comparative and diagnostic Learn how to perform a sample size calculation. (a) For continuous data (b) For non-continuous data (c) For diagnostic testsPOWER AND sample size ESTIMATIONP ower and sample size estimations are measuresof how many patients are needed in a all clinical studies entail studying a sampleof patients with a particular characteristic ratherthan the whole population.

2 We then use this sam-ple to draw inferences about the whole previous articles in the series on statisticspublished in this journal, statistical inference hasbeen used to determine if the results found aretrue or possibly due to chance alone. Clearly wecan reduce the possibility of our results comingfrom chance by eliminating bias in the studydesign using techniques such as randomisation,blinding, etc. However, another factor influencesthe possibility that our results may be incorrect,the number of patients studied. Intuitively weassume that the greater the proportion of thewhole population studied, the closer we will get totrue answer for that population. But how many dowe need to study in order to get as close as weneed to the right answer?

3 WHAT IS power AND WHY DOES ITMATTERP ower and sample size estimations are used byresearchers to determine how many subjects areneeded to answer the research question (or nullhypothesis).An example is the case of thrombolysis in acutemyocardial infarction (AMI). For many years cli-nicians felt that this treatment would be ofbenefit given the proposed aetiology of AMI,however successive studies failed to prove thecase. It was not until the completion of ad-equately powered mega-trials that the smallbut important benefit of thrombolysis these trials compared thrombolysiswith placebo and often had a primary outcomemeasure of mortality at a certain number of basic hypothesis for the studies may havecompared, for example, the day 21 mortality ofthrombolysis compared with placebo.

4 There aretwo hypotheses then that we need to consider:1 The null hypothesis is that there is no differencebetween the treatments in terms of The alternative hypothesis is that there is a dif-ference between the treatments in terms of trying to determine whether the two groupsare the same (accepting the null hypothesis) orthey are different (accepting the alternativehypothesis) we can potentially make two kinds oferror. These are called a type error and a type type error is said to have occurred when wereject the null hypothesis incorrectly (that is, it istrue and there is no difference between the twogroups) and report a difference between the twogroups being type error is said to occur when we acceptthe null hypothesis incorrectly (that is, it is falseand there is a difference between the two groupswhich is the alternative hypothesis) and reportthat there is no difference between the can be expressed as a two by two table(table 1).

5 power calculations tell us how many patientsare required in order to avoid a type or a type 1 Two by two tableActual truthTreatmentbenefitNo treatmentbenefitClinical trial result Treatment benefitCorrect resultType errorFalse positive resultNo treatment benefit Type errorCorrect resultFalse negative resultSee end of article forauthors to:Dr S R Jones, EmergencyDepartment, ManchesterRoyal Infirmary, OxfordRoad, ManchesterM13 9WL, on September 16, 2022 by guest. Protected by Med J: first published as on 3 September 2003. Downloaded from on September 16, 2022 by guest. Protected by Med J: first published as on 3 September 2003. Downloaded from on September 16, 2022 by guest. Protected by Med J: first published as on 3 September 2003.

6 Downloaded from on September 16, 2022 by guest. Protected by Med J: first published as on 3 September 2003. Downloaded from The term power is commonly used with reference to allsample size estimations in research. Strictly speaking power refers to the number of patients required to avoid a type error in a comparative study. sample size estimation is a moreencompassing term that looks at more than just the type error and is applicable to all types of studies. In common par-lance the terms are used AFFECTS THE power OF A STUDY?There are several factors that can affect the power of a should be considered early on in the development of astudy. Some of the factors we have control over, others we precision and variance of measurements withinany sampleWhy might a study not find a difference if there truly is one?

7 For any given result from a sample of patients we can onlydetermine a probability distribution around that value thatwill suggest where the true population value lies. The bestknown example of this would be 95% confidence size of the confidence interval is inversely proportional tothe number of subjects studied. So the more people we studythe more precise we can be about where the true populationvalue 1 shows that for a single measurement, the moresubjects studied the narrower the probability distributionbecomes. In group 1 the mean is 5 with wide confidence inter-vals (3 7). By doubling the number of patients studied (but inour example keeping the values the same) the confidenceintervals have narrowed ( ) giving a more preciseestimate of the true population probability distribution of where the true value lies isan integral part of most statistical tests for comparisonsbetween groups (for example,ttests).

8 A study with a smallsample size will have large confidence intervals and will onlyshow up as statistically abnormal if there is a large differencebetween the two groups. Figure 2 demonstrates how increas-ing the number of subjects can give a more precise estimate magnitude of a clinically significant differenceIf we are trying to detect very small differences between treat-ments, very precise estimates of the true population value arerequired. This is because we need to find the true populationvalue very precisely for each treatment group. Conversely, if wefind, or are looking for, a large difference a fairly wideprobability distribution may be other words if we are looking for a big difference betweentreatments we might be able to accept a wide probability dis-tribution, if we want to detect a small difference we will needgreat precision and small probability distributions.

9 As thewidth of probability distributions is largely determined byhow many subjects we study it is clear that the differencesought affects sample size 1 Change in confidenceinterval width with increasingnumbers of 2 Effect of confidenceinterval reduction to demonstrate atrue difference in means. Thisexample shows that the initialcomparison between groups 1 and 3showed no statistical difference as theconfidence intervals overlapped. Ingroups 3 and 4 the number ofpatients is doubled (although themean remains the same). We see thatthe confidence intervals no longeroverlap indicating that the differencein means is unlikely to have occurredby , Carley, on September 16, 2022 by guest. Protected by Med J: first published as on 3 September 2003.

10 Downloaded from When comparing two or more samples we usually havelittle control over the size of the effect. However, we need tomake sure that the difference is worth detecting. For example,it may be possible to design a study that would demonstrate areduction in the onset time of local anaesthesia from 60seconds to 59 seconds, but such a small difference would be ofno clinical importance. Conversely a study demonstrating adifference of 60 seconds to 10 minutes clearly would. Statingwhat the clinically important difference is a key componentof a sample size important is a type or type error for the studyin question?We can specify how concerned we would be to avoid a type ortype error. A type error is said to have occurred when wereject the null hypothesis incorrectly.