Gatekeeping procedures with clinical trial applications

1 PHARMACEUTICAL STATISTICSP harmaceut. ;6: 171 180 Published online 21 June 2007 in Wiley InterScience( ) DOI: procedures with clinicaltrial applicationsAlex Dmitrienko1,*,yand Ajit C. Tamhane21 Eli Lilly and Company, Indianapolis, IN, USA2 Northwestern University, Evanston, IL, USAThe objective of this paper is to give an overview of a relatively new area of multiplicity research thatdeals with the analysis of hierarchically ordered multiple objectives. Testing procedures for thisproblem are known as Gatekeeping procedures and have found a variety of applications in clinicaltrials. This paper reviews main classes of these procedures , including serial and parallel gatekeepingprocedures, and tree Gatekeeping procedures that account for logical restrictions among multipleobjectives. We focus on procedures based on marginalp-values; extensions to procedures that exploitthe joint distribution of thep-values are also noted.

2 clinical trial examples are used to illustrate theprocedures and their important properties. Copyright#2007 John Wiley & Sons, :multiple comparisons; multiple endpoints; clinical trials1. INTRODUCTIONIt is becoming increasingly common to considerdesigns with multiple endpoints, analyses andobjectives in registration studies because addi-tional information on the efficacy and safetyprofiles of an experimental drug helps patients,prescribing physicians and payers better under-stand its properties. More complicated studydesigns give rise to more sophisticated analysismethods. In the area of multiple comparisons,these problems motivated research on novel testingstrategies for hierarchically ordered objectives[1 4].This paper gives an overview of recent develop-ments in this area with emphasis on testingstrategies for multiple families of analyses are often related to multipleendpoints (both primary and secondary) but canalso represent dose control comparisons, nonin-feriority and superiority tests or inferences atseveral time points.

3 Testing strategies consideredhere are commonly referred to asgatekeepingstrategies. This terminology highlights the factthat the families of analyses are examined sequen-tially and each one serves as a gatekeeper forthe subsequent families. The sequential testingapproach reflects the hierarchical nature ofthe problem and improves the power of theyE-mail: to: Alex Dmitrienko, Lilly ResearchLaboratories, Eli Lilly and Company, Lilly Corporate Center,Drop Code 2233, Indianapolis, IN 46285, #2007 John Wiley & Sons, important analyses placed early in paper is organized as follows. Section 2introduces the basic concepts about gatekeepingand establishes notation. Section 3 describesserial Gatekeeping proceduresand Section 4 de-scribesparallel Gatekeeping procedures . A general-ization of the two approaches, termed thetreegatekeeping approach, is considered in Section Gatekeeping procedures enable clinicaltrial researchers to construct testing strategiesthat combine serial and parallel gatekeepersand also account for logical restrictions amongmultiple analyses conducted in a clinical procedures discussed in Sections 3 5 arebased on marginalp-values.

4 In Section 6 webriefly discuss methods that exploit the jointdistribution of thep-values; these includeresampling and normal theory methods. Section7 gives a summary and references for downloadingthe SAS macros for applying the above proce-dures. clinical trial examples are providedto illustrate key properties of BASIC CONCEPTS ANDNOTATIONThis paper assumes that the reader is familiar withthe key concepts in the theory of multiplecomparisons. For more information about multi-ple comparison procedures , see Hochberg andTamhane [5]. To introduce the concepts under-lying Gatekeeping testing strategies, consider aclinical trial with multiple objectives. Each objec-tive is associated with a null hypothesis of notreatment effect and each hypothesis is testedusing an appropriate significance test. The objec-tives are hierarchically ordered, for example,primary, secondary and tertiary objectives aredefined. To account for the hierarchical structureof the testing problem, the hypotheses aregrouped into families.

5 Considermfamilies denotedbyF1;..;Fmand letHi1;..;Hinidenote thehypotheses included inFi;i 1;..;m:Further,letn n1 nmdenote the total number ofhypotheses. The families are examined sequentiallybeginning withF1that corresponds to the mostimportant objectives. Inferences in this family areperformed without adjusting for tests of hypoth-eses in the other families. However, when sig-nificance tests are carried out in the subsequentfamilies, one needs to introduce a multiplicityadjustment to account for the previously of the firstm 1 families serves as agatekeeper for the families placed later in thesequence. A family is termed aserial gatekeeperifand only if (iff) one must reject all hypotheses inthis family to test subsequent families. Serialgatekeeping procedures were considered byMaureret al.[1], Baueret al.[2] and Westfalland Krishen [3]. As an example, in clinical trialsfor Alzheimer s disease, two primary endpoints aregenerally required: Alzheimer s Disease Assess-ment Scale-Cognitive Subscale (ADAS-Cog) andClinical Global Impression of Change (CGIC).

6 The trial is declared successful only if the treat-ment effect on both endpoints is , the primary gatekeeper can be passedonly if the null hypotheses for both the endpointsare concept of aparallel gatekeeperwasintroduced in Dmitrienkoet al.[4]. To pass aparallel gatekeeper, one needs to reject at least onehypothesis in the family. The acute respiratorydistress syndrome (ARDS) trial in [4] provides anexample of a parallel gatekeeper. It had twoprimary endpoints (mortality and lung functionendpoints). A significant treatment effect withrespect to either of these two endpoints allowedthe researcher to test for efficacy with respect tosecondary pointed out in [6], the serial gatekeepingapproach is analogous to intersection uniontesting [7] in which the union of several componenthypotheses is rejected iff all of them arerejected. Likewise, the parallel Gatekeeping ap-proach is similar to union intersection testing [8]in which the intersection of several componenthypotheses is rejected iff at least one of them #2007 John Wiley & Sons, ;6: 171 180 DOI: A.

7 Dmitrienko and A. C. TamhaneGatekeeping procedures are defined as multipletesting procedures that meet the following condi-tions:*Type I error rate control: The familywise errorrate (FWER) associated with the null hypoth-eses inF1;..;Fmis controlled in the strongsense at a prespecifiedalevel [5].*Serial and parallel Gatekeeping conditions: Con-siderFi;i 1;..;m 1:IfFiis a serialgatekeeper, then hypotheses inFi 1are testediff all hypotheses inFiare rejected. In otherwords, ifepi1;..;epiniare multiplicity adjustedp-values inFi;then the hypotheses inFi 1aretested iffmax epi1;..;epini 4aIfFiis a parallel gatekeeper, then hypotheses inFi 1are tested iff one or more hypotheses inFiare rejected, epi1;..;epini 4aThe untested hypotheses are automaticallyaccepted.*Independence condition: Inferences inFi;i 1;..;m 1;are independent of thep-values forthe hypotheses inFi 1;..;Fm:The independence condition plays a key role inclinical applications [9, Section ].

8 It ensures thatmore important analyses ( analysis of primaryendpoints) will not depend on the results of lessimportant analyses ( analysis of secondaryendpoints). However, relaxing this condition,when it is consistent with the objectives of aclinical trial , can result in some power gains, seeChenet al.[10], Dmitrienkoet al.[9, Section ]and Hommelet al.[11].3. SERIAL GATEKEEPINGPROCEDURESS erial Gatekeeping procedures have a straightfor-ward stepwise form that facilitates their use inmultiplicity problems arising in clinical studies. Asan example, consider a trial with two families ofhypotheses,F1andF2;the first of which is a serialgatekeeper, and suppose the overall FWER is tobe controlled at thealevel. Since it is required thatall hypotheses inF1must be rejected to test thehypotheses inF2;a powerful procedure to usewould be the intersection union test of Berger [7].This test tests each hypothesis inF1at inF2can be tested using any multipletest that controls the FWER for that family at thealevel, the Holm test [12] or the Hochberg test[13] (assuming that conditions under which theHochberg test controls the FWER are satisfied, seeSarkar and Chang [14]).

9 This simple setting is easily extended to thegeneral case ofmfamilies in which the firstm 1families are serial gatekeepers. Since any coherentgatekeeping procedure can be expressed as aclosed testing procedure [15], this serial gate-keeping procedure protects the FWER in thestrong an illustration, consider a clinical trialin patients with Alzheimer s disease that wasconducted to evaluate the efficacy and safety of asingle dose of an experimental drug comparedto placebo. The primary objective of the trialwas to assess the effect of the experimental drugon two endpoints, P1 (ADAS-Cog) and P2(CGIC). The null hypotheses associated with theprimary endpoints were included inF1:This familyserved as a serial gatekeeper forF2whichcontained two hypotheses related to the secondaryendpoints, S1 (a biochemical endpoint) and S2 (animaging endpoint). The hypotheses in bothfamilies were equally weighted and the FWERwas set ata 0:05:The rawp-values produced bythe primary and secondary tests are given inTable a serial gatekeeper and the hypothesescorresponding to P1 and P2 are tested by using anintersection union test that does not require anadjustment for multiplicity.

10 Thep-values inF1aresignificant at the level and thus the primaryobjective of the trial is met. Since the primary testsare both significant, the testing procedure passesthe serial gatekeeper and can now examine thehypotheses inF2:The secondary tests are carriedout using the Holm test. Comparing the Holm-adjustedp-values inF2to , it is easy to see thatGatekeeping procedures 173 Copyright#2007 John Wiley & Sons, ;6: 171 180 DOI: S1 is significant. The overall conclusion isthat the experimental drug is significantly differentfrom placebo with respect to P1, P2 and S1 at PARALLEL GATEKEEPINGPROCEDURESThe most basic parallel Gatekeeping procedure,derived from the Bonferroni test, was proposed inDmitrienkoet al.[4]. This procedure was for-mulated as a closed testing procedure and guar-anteed strong control of the FWER due to theclosed testing principle [16]. To satisfy the parallelgatekeeping and independence conditions, aweighted Bonferroni test was defined for eachindividual intersection hypothesis in the closedfamily induced by the null hypotheses inF1.