On generalized Simes critical constants

1 Biometrical Journal56(2014) 6, 1035 1054 DOI: generalized Simes critical constantsJiangtao Gou1andAjit C. Tamhane ,21 Department of Statistics, Northwestern University, 2006 Sheridan Road, Evanston, IL 60208, USA2 Department of Industrial Engineering and Management Sciences, Northwestern University, 2145 Sheridan Road, Evanston, IL 60208, USAR eceived 4 November 2013; revised 26 March 2014; accepted 18 April 2014We consider the problem treated by Simes of testing the overall null hypothesis formed by the intersectionof a set of elementary null hypotheses based on orderedp-values of the associated test statistics. TheSimes test uses critical constants that do not need tabulation. Cai and Sarkar gave a method tocompute generalized Simes critical constants which improve upon the power of the Simes test whenmore than a few hypotheses are false.

2 The Simes constants can be viewed as the first order (requiringsolution of a linear equation) and the Cai-Sarkar constants as the second order (requiring solution ofa quadratic equation) constants . We extend the method to third order (requiring solution of a cubicequation) constants , and also offer an extension to an arbitrarykth order. We show by simulation thatthe third order constants are more powerful than the second order constants for testing the overallnull hypothesis in most cases. However, there are some drawbacks associated with these higher orderconstants especially fork>3, which limits their practical :Multiple hypotheses; Power; Simes test; Type I error. Additional supporting information may be found in the online version of this articleat the publisher s web-site1 IntroductionConsidern 2 null hypotheses,H1.

3 ,Hn, and denote their associatedp-values byp1,.., (1) p(n)denote the orderedp-values andH(1),..,H(n), the corresponding null this paper we consider the problem of testing the overall null hypothesisH0= ni=1Hi. We assumethat thepiare independent uniform [0,1] random variables underH0. The dependence case will bestudied in a separate Simes (1986) test is based on the identityP n i=1 p(i) i n = ,(1)where the probability is computed underH0(as are all the type I error probabilities in this paper).Thus it rejectsH0at level (0,1)if at least onep(i) i /n(1 i n). It is more powerful than theBonferroni test, which rejectsH0if at least onepi /n. Corresponding author: Phone:+1-847-491-3577, Fax:+1-847-491-8005C 2014 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim1036J. Gou and A. C. Tamhane: On generalized Simes critical constantsCai and Sarkar (2008) defined generalized Simes critical constants as any set ofci(1 i n)thatsatisfyP n i=1 p(i) ci = (2)subject to the monotonicity condition:c1 cn.

4 (3)In this notation, the Simes critical constants areci=i/n(note that we use a different notation forcritical constants from that used by Cai and Sarkar). The test based on the generalized constantsrejectsH0ifp(i) ci for at least onei(1 i n).(4)The monotonicity condition (3) is necessary for this test to be valid as will be seen in the the method given by Cai and Sarkar (2008) to compute these constants , the Simes constants canbe viewed as the first order (requiring solution of a linear equation) and the Cai-Sarkar constants asthe second order (requiring solution of a quadratic equation) constants . By recursive application ofthe Cai-Sarkar method we derive third order constants and study their properties in detail. We alsopresent a general result on thekth order constants . Finally, we compare different choices of constantsin terms of power via simulation and show that the third order constants improve the power of the testcompared to the first and second order constants in a majority of the cases , Klein, and Hommel (2004) have given a nice review of the literature on global andmultiple test procedures based onp-values.

5 The following global tests discussed there use special casesof generalized Simes constants . In the case of independentp-values, Bauer (1989) proposed the so-called(n,k, )-test which usesc1= =ck 1=0andck= =cn=cwherec>0 is determinedfrom the equationn i=k ni (c )i(1 c )n i= ,wherekis prespecified. R ohmel and Streitberg (1987) showed that if thep-values are arbitrarilydependent then the -level is controlled ifnn i=1(ci ci 1)/i constants that satisfy this condition are (i) Bonferroni:c1= =cn=1/n, (ii) R uger (1978):c1= =ck 1=0andck= =cn=k/nwherekis prespecified and (iii) Hommel (1983):ci=i/(n nj=1j 1).Generally, a global test does not control the familywise error rate (FWER) if used as a multiple testprocedure (MTP). For example, the Simes test does not control the FWER if used to reject anyH(i)forwhichp(i) i /n(1 i n).

6 An MTP can be derived by constructing a closed procedure (Marcus,Peritz, and Gabriel, 1976) which uses an -level global test for all intersection (1996) showed under what conditions this closed procedure has a stepwise shortcut. Towardthis end, denotecibycinto indicate its dependence onn. Wei (1996) showed that, if the closedprocedure uses (4) to test all nonempty subset intersections ofHis of sizem nwith constantsci=cim, thencim=cm(1 i m)is a necessary and sufficient condition for the closed procedureto have a step-down shortcut andcim=ci+1,m+1(1 i m)to have a step-up shortcut. The Holm(1979) procedure is the step-down shortcut to a closed procedure that uses the Bonferroni test for allC 2014 WILEY-VCH Verlag GmbH & Co. KGaA, Journal56(2014) 61037intersection hypotheses. The Hommel (1988) procedure is the closed procedure that uses the Simes testfor all intersection hypotheses.

7 But the Simes constants do not satisfy either of Wei s conditions; hencethe Hommel procedure does not have a simple stepwise shortcut. Hochberg s (1988) step-up testingprocedure can be shown to be based on a conservative choice of the constants ,cim=1/(m i+1),which satisfy Wei s outline of the paper is as follows. In Section 2 we review the derivation of second order Section 3 we extend the method to third order constants and study their properties. Section 4 givesa general result about thekth order constants . Section 5 gives tables of the second and third orderconstants for selected values ofc1and(c1,c2), Section 6 compares the powers of the generalized Simestest for different choices of constants . Conclusions are given in Section 7. Proofs of all the results aregiven in the Second order generalized Simes constantsWe assume throughout that the generalized Simes constants satisfy the type I error rate condition (2).

8 Define the probabilities:An(i)= P(p(1)>c1 ,..,p(n)>cn )i=0,P(p(i) ci ,p(i+1)>ci+1 ,..,p(n)>cn )i=1,..,n 1,P(p(n) cn )i=n.(5)Note that ni=0An(i)=1 and hencen i=1An(i)=1 An(0)=P n i=1{p(i) ci } = .(6)In the sequel we use a recursion which involves, for fixedn, expressingAn(i)in terms ofAn 1(i),An 2(i), etc. These lower dimensional probabilities are given byAn m(i)= P(p(1)>cm+1 ,..,p(n m)>cn )i=0,P(p(i) cm+i ,p(i+1)>cm+i+1 , .. ,p(n m) cn )i=1,..,n m 1,P(p(n m) cn )i=n m.(7)Note that when computingAn m(i)forn m<n,p(1)is compared withcm+1 , not withc1 ;p(2)iscompared withcm+2 , not withc2 , etc. The latter would be the case if we change the notation so thatthe index ofciis changed fromiton i+ and Roters (1994) showed that under the monotonicity condition (3), the following recurrencerelation holds:An(i)=nci iAn 1(i 1)(i=1.)

9 ,n).(8)Since this recurrence relation lies at the core of the computation of generalized Simes constants , theirvalidity (in terms of controlling the type I error) requires that the monotonicity condition (3) must substituting this recurrence relation in (6) we getn i=1nci iAn 1(i 1)= .(9)If we setncii= 1(1 i n),(10)C 2014 WILEY-VCH Verlag GmbH & Co. KGaA, Gou and A. C. Tamhane: On generalized Simes critical constantsand note that ni=1An 1(i 1)= n 1i=0An 1(i)=1 then we obtain from (9) that 1n 1 i=0An 1(i)= 1= = 1= 1=1 back in (10) yields the Simes constantsci=i/n. Observe that they do not requirepredetermining and Sarkar (2008) applied the recurrence relation (8) a second time by puttingAn 1(i 1)=n 1i 1ci An 2(i 2)(i=2,..,n)in (9) to obtain the equationnc1 +n(n 1) 2n i=2cii 1 cii c1 An 2(i 2)=.

10 (11)If we setcii 1 cii c1 = 2(12)and note that ni=2An 2(i 2)=1, we obtain from (11) that 2=1 nc1n(n 1) .Substituting 2back in (12) we obtain the quadratic equation:c2ii(i 1) cic1i 1 1 nc1n(n 1) = roots of this equation depend on unlike the Simes constants . Furthermore, they depend onc1,which needs to be specified. Cai and Sarkar (2008) limited the range ofc1to 0 c1 1/nin whichcase the admissible root is given byci=c1i2+ c21i24+(1 nc1) i(i 1)n(n 1).(13)We can show that the range ofc1can be extended to 2/[n(1+ 1 )]>1/n. However, the secondorder constants obtained by this extension do not result in any significant power gain. Therefore, weomit the details of this extension. Note that if we putc1=1/nin (13) then we get the Simes constantsci=i/nand if we putc1=0thenwegetci= [i(i 1)]/[n(n 1) ].