What is meta-analysis?

1 What is meta- analysis ?Allison Shorten,1 Brett Shorten2 When clinicians begin their search for the best availableevidence to inform decision-making, they are usuallydirected to the top of the evidence pyramid tofind outwhether a systematic review and meta- analysis havebeen conducted. The Cochrane Library1is fastfilling withsystematic reviews and meta-analyses that aim to answerimportant clinical questions and provide the most reliableevidence to inform practice and research. So what ismeta- analysis and how can it contribute to practice?What is meta- analysis ?Meta- analysis is a research process used to systematic-ally synthesise or merge thefindings of single, inde-pendent studies, using statistical methods to calculate anoverall or absolute does notsimply pool data from smaller studies to achieve a largersample size. Analysts use well recognised, systematicmethods to account for differences in sample size, vari-ability (heterogeneity) in study approach andfindings(treatment effects) and test how sensitive their resultsare to their own systematic review protocol (study selec-tion and statistical analysis ).

2 23 The Five-step processThere is debate about the best practice for meta- analysis ,however there arefive common 1: the research questionA clinical research question is identified and a hypothesisproposed. The likely clinical significance is explained andthe study design and analytical plan are 2: systematic reviewA systematic review (SR) is specifically designed toaddress the research question and conducted to identifyall studies considered to be both relevant and of suffi-ciently good quality to warrant inclusion. Often, onlystudies published in established journals are identified,but identification of unpublished data is important toavoid publication bias or exclusion of studies meta-analyses only considerrandomised control trials (RCTs) in the quest for highestquality evidence. Other types of experimental and quasi-experimental studies may be included if theysatisfy the defined inclusion/exclusion 3: data extractionOnce studies are selected for inclusion in themeta- analysis , summary data or outcomes are extractedfrom each study.

3 In addition, sample sizes and measuresof data variability for both intervention and controlgroups are required. Depending on the study and theresearch question, outcome measures could includenumerical measures or categorical measures. Forexample, differences in scores on a questionnaire or dif-ferences in a measurement level such as blood pressurewould be reported as a numerical mean. However,differences in the likelihood of being in one categoryversus another (eg, vaginal birth versus cesarean birth)are usually reported in terms of risk measures such asOR or relative risk (RR).Step 4: standardisation and weighting studiesHaving assembled all the necessary data, the fourthstep is to calculate appropriate summary measures fromeach study for further analysis . These measures areusually calledEffect Sizesand represent the differencein average scores between intervention and controlgroups.

4 For example, the difference in change in bloodpressure between study participants who used drug Xcompared with participants who used a placebo. Sinceunits of measurement typically vary across includedstudies, they usually need to be standardised in orderto produce comparable estimates of this effect. Whendifferent outcome measures are used, such as whenresearchers use different tests, standardisation isimperative. Standardisation is achieved by taking, foreach study, the mean score for the intervention group,subtracting the mean for the control group and divid-ing this result by the appropriate measure of variabilityin that data results of some studies need to carry moreweight than others. Larger studies (as measured bysample sizes) are thought to produce more precise effectsize estimates than smaller studies. Second, studies withless data variability, for example, smaller SD or narrowerCIs are often regarded as better quality in study weighting statistic that seeks to incorporate both thesefactors, known asinverse variance, is commonly 5:final estimates of effectThefinal stage is to select and apply an appropriatemodel to compareEffect Sizesacross different most common models used areFixed EffectsandRandom Effectsmodels are basedon the assumption that every study is evaluating acommon treatment effect.

5 5 This means that the assump-tion is that all studies would estimate the sameEffectSizewere it not for different levels of sample variabilityacross different studies. In contrast, theRandom Effectsmodel assumes that the true treatment effects in theindividual studies may be different from each other .5and attempts to allow for this additional source of inter-study variation inEffect Sizes. Whether this lattersource of variability is likely to be important is oftenassessed within the meta- analysis by testing for heterogeneity .Forest plotThefinal estimates from a meta- analysis are oftengraphically reported in the form of a Forest Plot .In the hypothetical Forest Plot shown infigure 1, foreach study, a horizontal line indicates the standardisedEffect Sizeestimate (the rectangular box in the centre of Nursing, YaleUniversity, New Haven,Connecticut, USA2 Informed Health Choices Trust,Wollongong, New South Wales,AustraliaCorrespondence to:Dr Allison ShortenYale University School ofNursing, 100 Church StreetSouth, PO Box 9740,New Haven, CT 06536, Based NursingJanuary 2013| volume 16| number 1|3 Research made simple on May 17, 2022 by guest.)

6 Protected by Based Nurs: first published as on 23 November 2012. Downloaded from each line) and 95% CI for the risk ratio used. For eachof the studies, drug X reduced the risk of death (the riskratio is less than ). However, thefirst study was largerthan the other two (the size of the boxes represents therelative weights calculated by the meta- analysis ).Perhaps, because of this, the estimates for the twosmaller studies were not statistically significant (thelines emanating from their boxes include the value of 1).When all the three studies were combined in themeta- analysis , as represented by the diamond, we get amore precise estimate of the effect of the drug, wherethe diamond represents both the combined risk ratioestimate and the limits of the 95% to practice and researchManyEvidence Based Nursingcommentaries featurerecently published systematic review and meta-analysisbecause they not only bring new insight or strength torecommendations about the most effective healthcarepractices but they also identify where future researchshould be directed to bridge the gaps or limitations incurrent evidence.

7 The strength of conclusions frommeta- analysis largely depends on the quality of the dataavailable for synthesis. This reflects the quality of individ-ual studies and the systematic review. Meta- analysis doesnot magically resolve the problem of underpowered orpoorly designed studies and clinicians can be frustrated tofind that even when a meta- analysis has been conducted,all that the researchers can conclude is that the evidenceis weak, there is uncertainty about the effects of treatmentand that higher quality research is needed to betterinform practice. This is still an importantfinding and caninform our practice and challenge us tofill the evidencegaps with better quality research in the Cochrane (accessed 23 Oct 2012). M,Davey Smith G. Meta- analysis : potentials ;315:1371 systematic reviews Dis Child2005;90:845 J,Egger M, Davey Smoth G.

8 Investigating and dealingwith publication and other biases in ;323:101. (accessed 23 Oct 2012). JPT,Green S handbook for systematicreviews of interventions version [updated March 2011].The Cochrane Collaboration, 2011. (accessed 23 Oct 2012).Figure 1 Hypothetical Forest Plot4 Evidence Based NursingJanuary 2013| volume 16| number 1|Research made simple on May 17, 2022 by guest. Protected by Based Nurs: first published as on 23 November 2012. Downloaded from