Power and Sample Size - Vanderbilt University

1 Power andSample SizeIntroductionDefinitionsRelationships CalculationsMargin ofErrorSummaryPower and Sample SizeChris Slaughter, DrPHAssistant Professor, Department of BiostatisticsVanderbilt University School of MedicineGI Research ConferenceJune 19, 2008 Power andSample SizeIntroductionDefinitionsRelationships CalculationsMargin ofErrorSummaryOutline1 Introduction2 Definitions3 Factors that Impact Power4 Sample size Calculations5 Margin of Error6 Conclusions and AdvicePower andSample SizeIntroductionDefinitionsRelationships CalculationsMargin ofErrorSummaryPower and Sample SizeFirst question asked (and the last answered)How many subjects do I need in my study?If I enroll # subjects in a treatment and control group,how likely am I to detect a significant difference betweenthe two groups?

2 Calculations depends onScientific goalsStudy designAnalysis methodPractical limitations: budget, timePower andSample SizeIntroductionDefinitionsRelationships CalculationsMargin ofErrorSummaryExpectationsWhat to expect from a Sample size calculationEstimate of theapproximatenumber of subjects for agiven study designConduct early at design phase when changes still possibleOpportunity to plan data analysis before collecting anydataWhat not to expectHigh accuracy if inputs (informed guesses) are not accurateA quick answerPost-hoc Power analysisPower andSample SizeIntroductionDefinitionsRelationships CalculationsMargin ofErrorSummaryHypothesis TestingHypothesis: usually a statement to be judged of the form population value = specified constant Null hypothesis (H0)Usually a hypothesis of no effectH0is often a straw man; something you hope to disproveH0: 1 2= 0 Alternative hypothesis (H1)H1: 1 26= 0 Power and Sample size calculation require you specify thealternative hypothesis too; : 1 2= 10 Power andSample SizeIntroductionDefinitionsRelationships CalculationsMargin ofErrorSummaryErrors in Hypothesis TestingType I error ( )Prob.

3 Ofrejectingyour null hypothesis when it istrueDeclaring that a significant association exists betweenXandYwhen, in truth,XandYare not related = usuallyType II error ( )Prob. offailing to rejectyour null hypothesis when it isfalseNot finding a significant association exists betweenXandYwhen, in truth,XandYare relatedPower= 1 = usuallyPower andSample SizeIntroductionDefinitionsRelationships CalculationsMargin ofErrorSummaryMore DefinitionsEffect size : How large of a difference you expect to seebetween groups ( a treatments and control group)Difference in means, difference in proportions, odds ratios,relative riskWhat is a clinically relevant difference?PrecisionAbsence of random errorVariable has nearly the same value when measuredmultiple timesHigh precision leads to decreased variability and higherpowerAccuracyDegree to which a variable accurately measures what it issupposed to measureIncreases validity of conclusionsPower andSample SizeIntroductionDefinitionsRelationships CalculationsMargin ofErrorSummaryEffect size and PrecisionPower andSample SizeIntroductionDefinitionsRelationships CalculationsMargin ofErrorSummaryPower and Sample size RelationshipsPower whenAllow larger type I error (.)

4 Tradeoff between type I and IIerrors)Larger effect observedVariability n (andn1n2= 1)Required Sample size (n) Allow larger type I errorLarger effect observedVariability Allow larger type II error ( Power ) Power andSample SizeIntroductionDefinitionsRelationships CalculationsMargin ofErrorSummaryPower versus Sample SizePower andSample SizeIntroductionDefinitionsRelationships CalculationsMargin ofErrorSummaryTypes of outcomes and predictorsSpecific Power calculation will depend on the analysismethodContinuous outcome, binary predictorPercent of time below pH 4 in a treatment and controlgroup2-samplet-test, Wilcoxon rank sum testBinary outcome, binary predictorAny improvement (yes/no) in the steroid group comparedto the steroid plus dilation groupDichotomize percent of time below pH 4 2test, test of proportions, odds ratioContinuous outcome, continuous predictorCorrelation, linear regressionLots of other analysis andSample SizeIntroductionDefinitionsRelationships CalculationsMargin ofErrorSummaryCalculation MethodsSoftware: PS, web, ~rlenth/ Power /#1 and #4 on google searchFormulasRepeated measuresUnusual designsSimulationStudy-specificPower andSample SizeIntroductionDefinitionsRelationships CalculationsMargin ofErrorSummaryExample.

5 Percent of time below pH 4 Continuous outcome, binary predictorTreatment group spends 40% of time below pH 4 Control group spends 50% of time below pH 4 Standard deviation of 10% Power andSample SizeIntroductionDefinitionsRelationships CalculationsMargin ofErrorSummaryExample: Dichotomize percent of time below pH 4 Binary outcome, binary predictor Abnormal if more than half of time is spent below pH 4 Treatment group: 16% AbnormalControl group: 50% AbnormalStandard deviation determined by above percentagesPower andSample SizeIntroductionDefinitionsRelationships CalculationsMargin ofErrorSummaryComparison of binary and continuous outcomesSame data assumptionsTreatment: 40% of time below pH 4 ( = 10%) wouldgive 16% AbnormalControl: 50% of time below pH 4 ( = 10%) would give50% AbnormalNumber of subjects needed (each group)Continuous outcome: 22 subjectsBinary outcome: 38 subjectsNeed estimate of the variability for continuous outcomesFor binary outcomes, variability is largest forp= 50% Power andSample SizeIntroductionDefinitionsRelationships CalculationsMargin ofErrorSummarySample size and Margin of ErrorGoal.

6 Plan a study so that the margin of error issufficiently smallThe margin of error is defined to be half of the confidenceinterval widthBasing the Sample size calculations on the margin of errorcan lead to a study that givesscientificallyrelevant resultseven if the results are andSample SizeIntroductionDefinitionsRelationships CalculationsMargin ofErrorSummaryMargin of Error ExampleInfection rate in a population is 50% and a reduction to40% is believed to clinically significant Enroll enough subjects so that the margin of error is 5%.Consider these two possible outcomes:1 The new treatment is found to decrease infections by 6%(95% CI: [11%,1%]).P-value< ( significant )2 The new treatment decreases infections by only 4% (95%CI: [9%, 1%]).

7 P-value> ( not significant ) Power andSample SizeIntroductionDefinitionsRelationships CalculationsMargin ofErrorSummaryAdvantagesAdvantages of planning for precision rather than power1 Many studies are powered to detect a miracle and nothingless; if a miracle doesn t happen, the study providesnoinformationPlanning on the basis of precision will allow the resultingstudy to be interpreted if theP-value is large, because theconfidence interval will not be so wide as to include bothclinically significant improvement and clinically significantworseningSee Borenstein M:J Clin Epi1994; 47 andSample SizeIntroductionDefinitionsRelationships CalculationsMargin ofErrorSummaryUsing Correlation (r) to Compute Sample SizeContinuous outcomes, continuous predictorsWithout knowledge of population variances, etc.

8 ,rcan beuseful for planning studiesChoosenso that margin for error (half-width of ) forris acceptablePrecision ofrin estimating is generally worst whenpopulation correlation is 0 This margin for error is shown in the following figure belowPower andSample SizeIntroductionDefinitionsRelationships CalculationsMargin ofErrorSummaryUsing Correlation (r) to Compute Sample SizeMargin for error (length of longer side of asymmetric confidence interval) forrin estimating , when = 0 (solid line) and = (dotted line). Calculations are based on Fisher sztransformation andSample SizeIntroductionDefinitionsRelationships CalculationsMargin ofErrorSummaryOther considerationsOther factors that can impact required Sample sizeDropouts (missing data)Correlation: Paired observations or repeated measuresMultiple testing and interim analysesEquivalence testingBetter analysis optionsPower andSample SizeIntroductionDefinitionsRelationships CalculationsMargin ofErrorSummaryBad IdeasDo retrospective Power calculationsCalculate standardized effect sizes (Cohen)Standardize measure.

9 Small , medium , and large effectsIgnores important parts of study planning, sciencePower andSample SizeIntroductionDefinitionsRelationships CalculationsMargin ofErrorSummaryGood IdeasDo ..Use Power calculations prospectively to plan future studiesPutsciencebeforestatisticsDesign your study to meet scientific goalsClinically important effect sizesStatistics help identify a plan that is effective in meetingscientific goals not the other way aroundConduct pilot studiesUseful for estimating varianceUse continuous variables when possibl