Analyzing multiple endpoints in clinical trials of …

1 Review and recommendationsAnalyzing multiple endpoints in clinical trials of paintreatments: immpact recommendationsDennis C. Turka,*, Robert H. Dworkinb, Michael P. McDermottb, Nicholas Bellamyc,Laurie B. Burked, Julie M. Chandlere, Charles S. Cleelandf, Penney Cowang,Rozalina Dimitrovah, John T. Farrari, Sharon Hertzd, Joseph F. Heysee, Smriti Iyengarj,Alejandro R. Jadadk, Gary W. Jayl, John A. Jermanom, Nathaniel P. Katzn,Donald C. Manningo, Susan Martinp, Mitchell B. Maxq, Patrick McGrathr,Henry J. McQuays, Steve Quessyt, Bob A. Rappaportd, Dennis A. Revickiu,Margaret Rothmanv, Joseph W. Staufferw, Ola Svenssonx,Richard E. Whitey, James WitterzaDepartment of Anesthesiology, University of Washington, Box 356540, Seattle, WA 98195, USAbDepartment of Anesthesiology, University of Rochester, Rochester, NY, USAcMayne Medical School, University of Queensland, Brisbane, Queensland, AustraliadUnited States Food and Drug Administration, Silver Spring, MD, USAeEpidemiology, Merck & Co.

2 , Blue Bell, PA, USAfDepartment of Symptom Relief, MD Anderson Cancer CentergAmerican Chronic pain Association, Rocklin, CA, USAhClinical Research, Allergan, Inc., Irvine, CA, USAiCenter for clinical Epidemiology & Biostatistics, University of Pennsylvania, Philadelphia, PA, USAjLilly Corporate Center, Eli Lilly & Co, Indianapolis, IN, USAkCenter for Global Health, University of Toronto, Toronto, Ontario, CanadalSchwarz Biosciences, Research Triangle Park, NC, USAmNeurogesX, Inc., San Carlos, CA, USAnAnalgesic Research, Needham, MA, USAoCelgene Corporation, Summit, NJ, USApPfizer, Inc., Ann Arbor, MI, USAqDepartment of Anesthesiology, University of Pittsburgh, Pittsburgh, PA, USArDepartment of Psychology, Dalhousie University, CanadasPain Relief, Oxford University, Oxford, UKtGlaxoSmithKline, Research Triangle Park, NC, USAuUnited Biosource Corporation, Bethesda, MD, USAvJohnson & Johnson, Raritan, NJ, USAwAlpharma, Piscataway, NJ, USAxAstraZeneca R&D, Sodertalje, SwedenyEndo Pharmaceuticals Inc, Chadds Ford, PA USAzUnited States Food and Drug Administration, now at United States National Institutes of Health, USAR eceived 12 March 2008; received in revised form 11 June 2008; accepted 30 June 20080304-3959/$ 2008 International Association for the Study of pain .

3 Published by Elsevier All rights *Corresponding author. Tel.: +1 (206) 616 2626; fax: +1 (206) 543 Turk). 139 (2008) 485 493 AbstractThe increasing complexity of randomized clinical trials and the practice of obtaining a wide variety of measurements from studyparticipants have made the consideration of multiple endpoints a critically important issue in the design, analysis, and interpretationof clinical trials . Failure to consider important outcomes can limit the validity and utility of clinical trials ; specifying multiple end-points for the evaluation of treatment efficacy, however, can increase the rate of false positive conclusions about the efficacy of atreatment. We describe the use of multiple endpoints in the design, analysis, and interpretation of pain clinical trials , and reviewavailable strategies and methods for addressing multiplicity.

4 To decrease the probability of a Type I error ( , the likelihood ofobtaining statistically significant results by chance) in pain clinical trials , the use of gatekeeping procedures and other methods thatcorrect for multiple analyses is recommended when a single primary endpoint does not adequately reflect the overall benefits oftreatment. We emphasize the importance of specifying in advance the outcomes and clinical decision rule that will serve as the basisfor determining that a treatment is efficacious and the methods that will be used to control the overall Type I error rate. 2008 International Association for the Study of pain . Published by Elsevier All rights : multiple endpoints ; Multiplicity; clinical trials ; treatment outcomes; Chronic pain ; Acute pain ; Sampling error; Type I error1. IntroductionTo facilitate meta-analyses and systematic reviews ofclinical trials of pain treatments , the Initiative on Meth-ods, Measurement, and pain Assessment in clinical Tri-als ( immpact ) has recommended a set of coreoutcome domains[53] and measures[18], strategies fordeveloping improved measures[54], and methods fordetermining clinical importance of changes in outcomemeasures[19].

5 Based on reviews of the literature andconsensus discussions, six core outcome domains wererecommended for consideration for chronic pain clinicaltrials: pain , physical functioning, emotional functioning,participant ratings of global improvement, symptomsand adverse events, and participant disposition[53].The use of multiple outcome measures has also been rec-ommended for evaluations of the efficacy of treatmentsfor other chronic pain conditions, for example, rheuma-toid arthritis[20], osteoarthritis[5], low back pain [14],and neuropathic pain [10]. More generally, multiplepatient-reported, clinician-rated, laboratory test, anddevice measurement endpoints are often used in evalua-tions of treatment impact for diverse clinical conditions,and are commonly reported in product labeling[61].A major concern with conducting multiple tests ofsignificance (often referred to as the problem of multi-plicity ) of different endpoints in a clinical trial involvesthe so-called Type I error, the probability that a nullhypothesis is rejected when the null hypothesis is actu-ally true.

6 The greater the number of statistical tests per-formed, the greater the probability that one or more ofthem will yield a statistically significant result by chancealone. One consequence of conducting multiple analysesis that it increases the likelihood of false positive results,making it possible for an investigator to choose the mostfavorable result from among many analyses that havebeen performed[9]. For example, if individual statisticaltests, each using a significance level (a) of , are per-formed for four specific measures recommended byIMMPACT for the pain , physical functioning, emo-tional functioning, and participant-rated globalimprovement outcome domains[18], the chance of falselyrejecting at least one null hypothesis of no treatment dif-ference is This example assumes that the fourmeasures are uncorrelated, which is very unlikely formeasures of these outcome domains, and the problemof multiplicity is reduced when the outcome measuresare positively multiple endpoints are often necessary toadequately evaluate the benefits of pain treatment [5,10,14,20,53], consideration must be given to control-ling the overall probability of a Type I error and the riskof false positive conclusions in designing clinical trials ofthe efficacy and effectiveness of pain treatments .

7 Regula-tory agencies[9,55], biostatisticians[2], CONSORT guidelines[1], and scientific journals often advocatethe use of appropriate adjustments to control the overallprobability of a Type I error when multiple endpointsare included in clinical trials , and a single primary end-point does not adequately reflect diverse benefits oftreatment. The objective of this article is to discuss mul-tiplicity and describe strategies for minimizing the riskof false positive conclusions in pain clinical trials withmultiple efficacy Consensus meeting procedureAn immpact consensus meeting was held thatincluded an international group of 33 participants fromuniversities, governmental agencies, a patient self-helporganization, and the pharmaceutical industry. Partici-pants were selected on the basis of their research, clini-cal, or administrative expertise relevant to the designand evaluation of pain treatment outcomes.

8 An attemptwas made to include broad representation of variousdisciplines and expertise while limiting the size of themeeting to promote frank and efficient discussion. Toensure that all attendees were familiar with the Turk et al. / pain 139 (2008) 485 493advances in addressing multiplicity, six articles review-ing important issues and strategies involving multipleanalyses and endpoints were circulated prior to themeeting[7,12,16,23,39,44]. In addition, background lec-tures were presented at the meeting that examined (1)general issues regarding multiple endpoints and multipleanalyses in clinical trials (JFH, TP, Lemuel A. Moye ,III), (2) responder analyses and state attainment criteriain studies of rheumatic diseases (NB), and (3) regulatoryperspectives on multiple endpoints (LB).3. Classification of endpointsThe primary objectives of most clinical trials includeevaluating whether a treatment provides clinical benefitin a sample drawn from a population to which the resultswill be generalized.

9 clinical benefit should be defined andassessed as unambiguously as possible because it pro-vides the basis for determining whether the results ofthe clinical trial have demonstrated evidence of treat-ment efficacy. The procedure for determining whetherthe results of the trial have demonstrated efficacy, whichhas been termed the clinical decision rule [7], must bespecified prior to beginning data to the recommendation that multiple out-come measures should be used in chronic pain clinicaltrials to adequately evaluate clinical benefit[53]willinvolve multiple analyses and, as a consequence, thepossibility of an increased risk of false positive conclu-sions for one or more of the outcome measures. Thismust be addressed in the design of the clinical trialand in its statistical plan, which must specify whetherstatistically significant improvements for one, several,or all of these endpoints are required for the trial to havedemonstrated clinical benefit of the treatment .

10 AlthoughIMMPACT recommended multiple outcome domainsand measures[18,53], minimal guidance was providedregarding whether these should be primary, co-primary,or secondary endpoints , and no attention was paid tomethods for addressing multiplicity in pain clinical trials ( , the clinical decision rule for interpreting results formultiple endpoints ). In addition, many chronic painclinical trials do not clearly specify the clinical decisionrule and which endpoints are primary and secondary;without this information it is often impossible to deter-mine whether a treatment has convincingly demon-strated efficacy relative to a control condition[7]. Primary endpointThe primary endpoint in a clinical trial has beendefined as the variable capable of providing the mostclinically relevant and convincing evidence directlyrelated to the primary objective of the trial [55].