Converting Among Effect Sizes - Meta-analysis

1 CHAPTER 7 Converting Among Effect SizesIntroductionConverting from the log odds ratio todConverting fromdto the log odds ratioConverting fromrtodConverting fromdtorINTRODUCTIONE arlier in this Part we discussed the case where different study designs were usedto compute the same Effect size. For example, studies that used independent groupsand studies that used matched groups were both used to yield estimates of thestandardized mean difference,g. There is no problem in combining these estimatesin a Meta-analysis since the Effect size has the same meaning in all , however, the case where some studies report a difference in means,which is used to compute a standardized mean difference. Others report a differencein proportions which is used to compute an odds ratio. And others report a correla-tion. All the studies address the same broad question, and we want to include themin one Meta-analysis .

2 Unlike the earlier case, we are now dealing with differentindices, and we need to convert them to a common index before we can question of whether or not it is appropriate to combine Effect Sizes fromstudies that used different metrics must be considered on a case by case basis. Thekey issue is that it only makes sense to compute a summary Effect from studies thatwe judge to be comparable in relevant ways. If we would be comfortable combiningthese studies if they had used the same metric, then the fact that they used differentmetrics should not be an example, suppose that several randomized controlled trials start with thesame measure, on a continuous scale, but some report the outcome as a meanand others dichotomize the outcome and report it as success or failure. In thiscase, it may be highly appropriate to transform the standardized mean differencesIntroduction to Meta-analysis .

3 Michael Borenstein, L. V. Hedges, J. P. T. Higgins and H. R. Rothstein 2009 John Wiley & Sons, Ltd. ISBN: 978-0-470-05724-7and the odds ratios to a common metric and then combine them across contrast, observational studies that report correlations may be substantiallydifferent from observational studies that report odds ratios. In this case, even ifthere is no technical barrier to Converting the effects to a common metric, it may bea bad idea from a substantive this chapter we present formulas for Converting between an odds ratio andd,orbetweendandr. By combining formulas it is also possible to convert from an oddsratio, viad,tor(see Figure ). In every case the formula for Converting the effectsize is accompanied by a formula to convert the we convert between different measures we make certain assumptionsabout the nature of the underlying traits or effects. Even if these assumptions donot hold exactly, the decision to use these conversions is often better than thealternative, which is to simply omit the studies that happened to use an alternatemetric.

4 This would involve loss of information, and possibly thesystematicloss ofinformation, resulting in a biased sample of studies. A sensitivity analysis tocompare the Meta-analysis results with and without the converted studies wouldbe outlines the mechanism for incorporating multiple kinds of data in thesame Meta-analysis . First, each study is used to compute an Effect size and varianceof itsnativeindex, the log odds ratio for binary data,dfor continuous data, andrforcorrelational data. Then, we convert all of these indices to a common index, whichwould be either the log odds ratio,d,orr. If the final index isd, we can move fromthere to Hedges g. This common index and its variance are then used in dataContinuous dataCorrelational dataLog odds ratioStandardizedMean Difference(Cohen s d )Fisher s zBias-correctedStandardizedMean Difference(Hedges g)Figure Among Effect Size and PrecisionCONVERTING FROM THE LOG ODDS RATIO TOdWe can convert from a log odds ratio (LogOddsRatio) to the standardized meandifferencedusingd5 LogOddsRatio ffiffiffi3pp; 7:1 wherepis the mathematical constant (approximately ).

5 The variance ofdwould then beVd5 VLogOddsRatio 3p2; 7:2 whereVLogOddsRatiois the variance of the log odds ratio. This method was originallyproposed by Hasselblad and Hedges (1995) but variations have been proposed(see Sanchez-Meca, Marin-Martinez, & Chacon-Moscoso, 2003; Whitehead,2002). It assumes that an underlying continuous trait exists and has a logisticdistribution (which is similar to a normal distribution) in each group. In practice,it will be difficult to test this example, if the log odds ratio with a , thend50:9069 ffiffiffi3p3:141650:5000with varianceVd50:0676 33:1416250:0205: Converting FROMdto the log odds ratioWe can convert from the standardized mean differencedto the log odds ratio(LogOddsRatio) usingLogOddsRatio5dpffiffiffi3p; 7:3 wherepis the mathematical constant (approximately ). The variance ofLogOddsRatiowould then beVLogOddsRatio5 Vdp23: 7:4 For example, thenLogOddsRatio50:5000 3:1416ffiffiffi3p50:9069.

6 Chapter 7: Converting Among Effect SizesandVLogOddsRatio50:0205 3:14162350:0676:To employ this transformation we assume that the continuous data have the FROMrTOdWe convert from a correlation (r)to a standardized mean difference (d) usingd52rffiffiffiffiffiffiffiffiffiffif fiffiffi1 r2p: 7:5 The variance ofdcomputed in this way (converted fromr)isVd54Vr1 r2 3: 7:6 For example, , thend52 0:50ffiffiffiffiffiffiffiffiffiffiffiffi ffiffiffiffiffiffiffi1 0:502p51:1547and the variance ofdisVd54 0:00581 0:502 350:0550:In applying this conversion we assume that the continuous data used to computerhas a bivariate normal distribution and that the two groups are created by dichot-omizing one of the two FROMdTOrWe can convert from a standardized mean difference (d) to a correlation (r) usingr5dffiffiffiffiffiffiffiffiffiffiff iffiffid2 ap 7:7 whereais a correction factor for cases wheren16 n2,a5 n1 n2 2n1n2: 7:8 The correction factor (a) depends on the ratio ofn1ton2, rather than theabsolute values of these numbers.

7 Therefore, ifn1andn2are not known precisely,usen15n2, which will yielda54. The variance ofrcomputed in this way(converted fromd)is48 Effect Size and PrecisionVr5a2 Vdd2 a 3: 7:9 For example, ifn15n2, , thenr51:1547ffiffiffiffiffiffiffiffiffif fiffiffiffiffiffiffiffiffiffiffiffiffiff iffi1:15472 4p50:5000and the variance ofrconverted fromdwill beVr542 0:05501:15472 4 350:0058:In applying this conversion assume that a continuous variable was dichotomized tocreate the treatment and control we transform between Fisher szanddwe are making assumptions aboutthe independent variable only. When we transform between the log odds ratio anddwe are making assumptions about the dependent variable only. As such, the two setsof assumptions are independent of each other, and one has no implications for thevalidity of the other. Therefore, we can apply both sets of assumptions and trans-form from Fisher szthroughdto the log odds ratio, as well as the POINTS If all studies in the analysis are based on the same kind of data (means, binary,or correlational), the researcher should select an Effect size based on that kindof data.

8 When some studies use means, others use binary data, and others use correla-tional data, we can apply formulas to convert Among Effect Sizes . Studies that used different measures may differ from each other in substantiveways, and we need to consider this possibility when deciding if it makes senseto include the various studies in the same 7: Converting Among Effect Sizes