Lee A. Becker …

1 Lee A. Becker < >Effect Size (ES) 2000 Lee A. BeckerI. OverviewII. Effect Size Measures for Two Independent difference between two measures of effect examplesIII. Effect Size Measures for Two Dependent Meta AnalysisV. Effect Size Measures in Analysis of VarianceVI. ReferencesEffect Size CalculatorsAnswers to the Effect Size Computation QuestionsI. OverviewEffect size (ES) is a name given to a family of indices that measure the magnitude ofa treatment effect. Unlike significance tests, these indices are independent of samplesize. ES measures are the common currency of meta-analysis studies that summarizethe findings from a specific area of research. See, for example, the influential meta-analysis of psychological, educational, and behavioral treatments byLipsey andWilson (1993).There is a wide array of formulas used to measure ES.

2 For the occasional reader ofmeta-analysis studies, like myself, this diversity can be confusing. One of myobjectives in putting together this set of lecture notes was to organize and summarizethe various measures of general, ES can be measured in two ways:a) as the standardized difference between two means, orb) as the correlation between the independent variable classification and theindividual scores on the dependent variable. This correlation is called the "effect sizecorrelation" (Rosnow & Rosenthal, 1996).These notes begin with the presentation of the basic ES measures for studies with twoindependent groups. The issues involved when assessing ES for two dependent groupsare then Effect Size Measures for Two Independent Groups1. Standardized difference between two 's dd = M1 - M2 / where = [ (X - M) /N]where X is the rawscore,M is the mean,andN is the number (1988) definedd as the difference between themeans, M1 - M2, divided by standard deviation, , ofeither group.

3 Cohen argued that the standard deviation ofeither group could be used when the variances of the twogroups are meta-analysis the two groups are considered to be theexperimental and control groups. By convention thesubtraction, M1 - M2,is done so that the difference ispositive if it is in the direction ofimprovement or in thepredicted direction and negative if in the direction ofdeterioration or opposite to the predicted is a descriptive = M1 - M2 / pooled pooled = [( 1 + 2 ) / 2]In practice, the pooled standard deviation, pooled, iscommonly used (Rosnow and Rosenthal, 1996).The pooled standard deviation is found as the root meansquare of the two standard deviations (Cohen, 1988, ). That is, the pooled standard deviation is the squareroot of the average of the squared standard the two standard deviations are similar the rootmean square will be not differ much from the simpleaverage of the two = 2t / (df)ord =t(n1 + n2) /[ (df) (n1n2)]d can also be computed from the value of thet test of thedifferences between the two groups (Rosenthal andRosnow, 1991).

4 In the equation to the left "df" is thedegrees of freedom for thet test. The "n's" are the numberof cases for each group. The formula without the n'sshould be used when the n's are equal. The formula withseparate n's should be used when the n's are not = 2r / (1 - r )d can be computed from r, the ES =g (N/df)d can be computed from Hedges' interpretation of Cohen'sdCohen'sStandardEffectSizePercent ileStandingPercent (1988) hesitantly definedeffect sizes as "small,d = .2,""medium,d = .5," and "large,d =.8", stating that "there is a certainrisk in inherent in offeringconventional operationaldefinitions for those terms for usein power analysis in as diverse afield of inquiry as behavioralscience" (p. 25).Effect sizes can also be thought ofas the average percentile standingof the average treated (orexperimental) participant relativeto the average untreated (orcontrol) participant.

5 An ES of that the mean of thetreated group is at the 50thpercentile of the untreated ES of indicates that themean of the treated group is at the79th percentile of the untreatedgroup. An effect size of that the mean of thetreated group is at the of the untreated sizes can also be interpretedin terms of the percent ofnonoverlap of the treated group'sscores with those of the untreatedgroup, seeCohen (1988, pp. 21-23) for descriptions of additionalmeasures of An ES indicates that the distributionof scores for the treated groupoverlaps completely with thedistribution of scores for theuntreated group, there is 0% ofnonoverlap. An ES of indicatesa nonoverlap of in the twodistributions. An ES of a nonoverlap of inthe two 'sgg = M1 - M2 /SpooledwhereS= [ (X - M) / N-1]andSpooled = MSwithinHedges'sg is an inferential is normally computed by using thesquare root of the Mean Square Errorfrom the analysis of variance testingfor differences between the 'sg is named for Gene , one of the pioneers of =t (n1 + n2) / (n1n2)org = 2t / NHedges'sg can be computed from thevalue of thet test of the differencesbetween the two groups (Rosenthaland Rosnow, 1991).

6 The formula withseparate n's should be used when then's are not equal. The formula withthe overall number of cases, N,should be used when the n's are equal. pooled = Spooled (df / N)were df = the degrees of freedom forthe MSerror, andN = the total number of pooled standard deviation, pooled,can be computed from the unbiasedestimator of the pooled populationvalue of the standard deviation,Spooled,and vice versa, using the formula onthe left (Rosnow and Rosenthal, 1996,p. 334).g =d / (N / df)Hedges'sg can be computed fromCohen' = [r / (1 - r )] / [df(n1 + n2) / (n1n2)]Hedges'sg can be computed from r,the ES 's delta = M1 - M2 / controlGlass's delta is defined as the mean difference betweenthe experimental and control group divided by thestandard deviation of the control Correlation measures of effect sizeThe ES correlation, rY rY = rdv,ivThe effect size correlation can becomputed directly as the point-biserial correlation between thedichotomous independent variableand the continuous = dv with ivThe point-biserial is a special case ofthe Pearson product-momentcorrelation that is used when one ofthe variables is dichotomous.

7 AsNunnally (1978) points out, the point-biserial is a shorthand method forcomputing a Pearson product-momentcorrelation. The value of the point-biserial is the same as that obtainedfrom the product-moment can use the CORR procedure inSPSS to compute the ES = = ( (1) / N)The ES correlation can be computedfrom a single degree of freedom ChiSquare value by taking the squareroot of the Chi Square value dividedby the number of cases, N. This valueis also known as = [t / (t + df)]The ES correlation can be computedfrom thet-test = [F(1,_) /(F(1,_) + df error)]The ES correlation can be computedfrom a single degree of freedomF testvalue ( , a oneway analysis ofvariance with two groups).rY =d / (d + 4)The ES correlation can be computedfrom Cohen' = {(g n1n2) /[g n1n2+(n1 +n2)df]}The ES correlation can be computedfrom Hedges' relationship betweend, r, and r Cohen'sStandarddrr.

8 Noted in the definition sectionsabove,d and be converted tor andvice example, thed value of .8corresponds to anr value of ..000 The square of the r-value is thepercentage of variance in thedependent variable that is accountedfor by membership in the independentvariable groups. For ad value of .8,the amount of variance in thedependent variable by membership inthe treatment and control groups meta-analysis studiesrs aretypically presented rather thanr .3. Computational ExamplesThe following data come fromWilson, Becker , and Tinker (1995). In that studyparticipants were randomly assigned to either EMDR treatment or delayed EMDR treatment. Treatment group assignment is called TREATGRP in the analysis dependent measure is the Global Severity Index (GSI) of the Symptom CheckList-90R. This index is called GLOBAL4 in the analysis below.

9 The analysis looks atthe the GSI scores immediately post treatment for those assigned to the EMDR treatment group and at the second pretreatment testing for those assigned to thedelayed treatment condition. The output from the SPSS MANOVA andCORR(elation) procedures are shown below. Cell Means and Standard Deviations Variable .. GLOBAL4 GLOBAL INDEX:SLC-90R POST-TEST FACTOR CODE Mean Std. Dev. N 95 percentConf. Interval TREATGRP TREATMEN .589 .645 40 . TREATGRP DELAYED .628 40 . For entire sample .797 .666 80 . * * * * * * * * * * * * * A n a l y s i s o f V a r i a n c e -- Design 1 * * ** * * * * * * * * Tests of Significance for GLOBAL4 using UNIQUE sums of squares Source of Variation SS DF MS F Sig of F WITHIN CELLS 78.

10 41 TREATGRP 1 .005 (Model) 1 .005 (Total) 79 .44 - - Correlation Coefficients - - GLOBAL4 TREATGRP .3134 ( 80) P= .005 Look back over the formulas for computing the various ES estimates. This SPSS output has the following relevant information: cell means, standard deviations, and ns,the overall N, and MSwithin. Let's use that information to compute ES = M1 - M2 / [( 1 + 2 )/ 2] = - / [( + ) / 2] = / [( + ) / 2] = / ( / 2) = / = = .65 Cohen'sdCohen'sd can be computed using thetwo standard is the magnitude ofd, accordingto Cohen's standards?The mean of the treatment group is atthe _____ percentile of the = M1 - M2 / MSwithin = - / = / =.