Evaluation of follow-on companion diagnostics assays ...

1 Evaluation of follow -on companion diagnostic assays : bridge from the clinical trial assay to a follow -on assay via external concordance data Jingjing Ye, Gene Pennello Food and Drug Administration (FDA) May 19, 2015 MBSW 1 Outline companion diagnostics follow -on companion diagnostics Scenarios for Evaluation Proposed approach Results Discussion 2 companion diagnostics FDA guidance (2014) defines that an IVD companion diagnostics device is an in vitro diagnostics device that provides information that is essential for the safe and effective use of a corresponding therapeutic product. allows the therapeutic product s benefits to exceed its risks 3 companion diagnostics Safety and effectiveness Identify patients most likely benefit from the therapeutic product Identify patients likely to be at increased risk for serious adverse reactions from the therapeutic product Monitor response to treatment for the purpose of adjusting regimens such as schedule, dose or discontinuation 4 Study Design for companion diagnostics Retrospective analysis of all-comer randomized clinical trial (RCT) Enrichment design 5 Retrospective analysis of all-comer randomized clinical trial (RCT) Apply Dx to stored specimens Evaluate treatment effect in marker subgroups ( , +) 6 Enrichment Design Specimens in marker may or may not be stored 7 FDA Guidance, Draft.

2 Enrichment Strategies for clinical Trials to Support Approval of Human Drugs and Biological Products, December 2012. follow -on companion diagnostics Test 1: FDA-approved companion diagnostic for a therapeutic product; select a subset for a drug Test 1 approved with a therapeutic clinical trial; drug efficacy ( HR) for outcome Y available Test 2 ( follow -on): New manufacture intended for the same therapeutic indication 8 Scenarios of follow -on companion Diagnostic Test Samples available from clinical trial for Test 1 to test with follow -on Test 2: (1)All (Scenario I) (2)Some (Scenario II) (3)None (Scenario III) 9 Te s t 1 follow -on Test 2 Outcome Y clinical Study (1), (2) concordance study (3) Only for (3), sponsor of follow -on may have only access to summary level data in clinical study. Evaluation of follow -on companion Diagnostic Test Analytical studies: LoB, LoD, repeatability, reproducibility, cross reactivity Performance on archived samples should also be evaluated clinical studies clinical Study of Test 1 for Outcome Y Agreement study of follow -on Test 2 ( 2) with approved Test 1 ( 1) E(Y | 2 +)= E(Y| 2+, 1+ )*P( 1+| 2+)+ E(Y| 2+, 1 )*P( 1 | 2+) 10 Scenario I, All available All-comers study design clinical trial of Test 1 ( 1) Retest clinical trial samples with follow -on Test 2 ( 2) E(Y | 2 +) can be evaluated 11 1 2 Outcome Y clinical Study ( concordance study) Scenario I, All available (cont d) Enrichment design clinical trial on patients who were Test 1 + ( 1+) Retest samples with follow -on Test 2 for agreement with Test 1 and clinical performance E(Y | 2 +, 1+ ) Subjects who are 1 do not have outcome and samples may not be available to re-test.

3 May need to evaluate banked samples for 2+, 1 and sensitivity analysis on E(Y| 2+, 1 ) 12 1 2 Outcome Y clinical Study ( concordance ) 1+ concordance Study 1 Scenario II, Some available Specimens missing due to lack of consent, inadequate sample volume, insufficient samples quality, etc. Intended to Diagnose (ITD) Impute missing test results 13 Scenario II, Some available All-comers study design clinical trial of Test 1 ( 1) Retest clinical trial samples with follow -on Test 2 ( 2) E(Y | 2 +)= E(Y| 2+, 1+ )*P( 1+ | 2+)+ E(Y| 2+, 1 )*P( 1 | 2+) 14 1 2 Outcome Y clinical Study ( concordance ): clinical Study: Scenario II, Some available (cont d) Enrichment design clinical trial on patients who were Test 1 + ( 1+) Retest samples with follow -on Test 2 for agreement with Test 1 and clinical performance E(Y | 2 +, 1+ ) Samples with 1 do not have outcome and not all may be available to re-test.

4 Sensitivity analysis on E(Y| 2+, 1 ) 15 1 2 Outcome Y clinical Study ( concordance ) 1 + clinical Study 1 + clinical Study( concordance ) 1 16 16 Intent to Diagnose (ITD) In statistical analysis, include all patients on whom a diagnosis could have been attempted: Report number (percent) of subjects without results (invalid, unevaluable, etc.). Report number (percent) of subjects with equivocal result When appropriate, consider imputation of missing test results. FDA Statistical Guidance on Reporting Results from Studies Evaluating Diagnostic Tests, Final 2007. Campbell, Pennello, and Yue, 2011, Missing data in the Regulation of Medical Devices, J Biopharm Stat, 21(2), 180-195 Scenario II, Some available (cont d) 1)Identify a set of covariates which can affect test result ( , use logistic regression or linear model of test result on covariates). 2)Check for imbalance in the covariates between samples in test available analysis set and in test not available analysis set.

5 3)Impute test results assuming they are Missing at random Missing not at random (nonignorable missingness) Missing result not associated with response 17 Variables Patient Characteristics Disease characteristics Handling and processing factors Specimen Characteristics Outcome 18 Patient characteristics Gender Race Age Baseline ECOG PS Baseline weight Marker status by reference method 19 Disease characteristics Months from first histological diagnosis to randomization Number of disease sites Presence of metastases (yes or no) Number of previous therapies Prior therapy (yes or no) 20 Handling, processing factors Enrollment site Region ( , Canada, Non-Canada) Age of sample at testing Sampling method (biopsy, resection) 21 Characteristics of sample Tumor type (primary or metastatic) If metastatic, then site of metastasis Area of tumor tissue (mm ) Tumor content in sample (%) Macro-dissection of sample (yes or no) Necrosis score in tumor area (0, 1, 2 or 3) H&E staining slide evaluable (yes or no) 22 Tipping point analysis 2 Result 1 Result Total 1 1+ 2 N00 N01 N00+ N01 2 + N10 N11 N10+ N11 Total N00+ N01 N01+ N11 N 23 k k Denne, , Pennello, G.

6 , Zhao, L., Chang, , Althouse, S. Identifying a subpopulation for a tailored therapy: Bridging clinical efficacy from a laboratory-developed assay to a validated in vitro diagnostic test kit. Statistics Biopharm Res, 2014; 6(1): 78-88. The number k was increased until a tipping point was reached at which the result of clinical performance became insignificant Scenario III, None available All-comers study design clinical trial of Test 1 ( 1) external concordance data between 2 and 1 Goal: infer clinical outcome Y on 2 24 1 2 Outcome Y clinical Study concordance study Line- data ? No KRAS IVD companion Diagnostic Population: Colorectal cancer patients Drug: BMS Lilly Cetuximab CoDx: Qiagen therascreen KRAS RGQ PCR Kit Pivotal clinical Study, Drug: CRYSTAL RCT Pivotal clinical Study, Device: NCIC CA225-025 572 subjects in All Randomized population 394 ( ) evaluated by bi-directional Sequencing.

7 484 ( ) had tissue available for re-testing. 453 ( ) evaluated by Qiagen therascreen . 25 NCIC CA225-025 RCT Karapetis et al. Kras mutations and benefit from cetuximab in advanced colorectal cancer. N Engl J Med 2008; 359: 1757-65. Hazard ratios by sequencing result: Mut (Wt): OS HR , 95% CI , p< Mut+ (Mt): OS HR , 95% CI , p= Mut (Wt): PFS HR , 95% CI , p< Mut+ (Mt): PFS HR , 95% CI , p= Pretend Qiagen s test is a follow -on test to sequencing 26 27 28 Model Parameters clinical Outcome Study external concordance (EC) Study Goal: Estimate 29 S=0 S=1 Count Hazard=h(Y) )1Pr(==S ..(| )ssEhSs ==S=0 S=1 Q=0 n00 n01 Q=1 n10 n11 Pr(1|)spQSs== =..(| )qqEhQq ==Non-Differential Misclassification Assume misclassification of S by Q is non-differential to Y, that is, Equivalently, That is, given test result S, follow -on test result Q is uninformative for outcome Y ( , hazard ratio) That is, Q and Y are conditionally independent given S.

8 30 =|,|QSY QS=|,|Y QS Y SNon-differential Misclassification Hazard ratios by Q, S: Under non-differential misclassification error (NDME), the hazard ratios become 31 S=0 S=1 Q=0 Q=1 S=0 S=1 Q=0 Q=1 Non-differential Misclassification Under NDME, hazard ratios for Q are Where 1=Pr =1 =1= = 1( 1+1 0) 1 32 .01(1) = + (1) = +qqNDME 0=Pr =1 =0=1 = (1 1)( 1 1+(1 )(1 0)) 1 NDME Attenuation Result NDME is a conservative assumption: , 0. ( .0 .1)( 1 0) That is, the difference in hazard ratio between subsets defined by Q is attenuated relative to the corresponding difference for S . 33 (|0) (|1)= =Eh QEh Q[] [](|0) (| 1)1E h SE h SNPVPPV= = = + Provided PPV 1 NPV , , Q not negatively informative for S, so that 0 PPV + NPV 1 1. (Pennello, Clin Trials 2013 Oct;10(5):666-76 .) Cox Model For treatments T = 0, 1, hazard model for S is 34 0( | , )()STy STye =log hazard ratio for sSs =={}( | , )| , ,Ey ST QT Y y = {}0()| , ,STy E eQT Y y = {}0()| ,STy E eQT ()0010()TTTqyee e = + 010( ) (1)TTqqy ee = + {}(|,)(|,,)|,,y QTEy Q ST QT Y y = NDME Rare Disease Cox Model Same approximate attenuation holds for log hazard ratio difference.

9 Approximation is OK for rare enough outcome. More investigation is needed. Prentice, 1982. Pepe, Self, Prentice, 1989. 35 ()0010( | , )()TTTqy QTy eee + ( |1,1)( |0,1)yQTyQT == ==()()01010()ye e = []()10( |1,1)( |0,1)YSTYST = == == Bayesian Model EC data Outcome data 36 S=0 S=1 Q=0 n00 n01 Q=1 n10 n11 S=0 S=1 Count Hazard=h(Y) Parameters )1Pr(==S ..(| )ssEhSs ==Pr(1|)spQSs== = data Model Priors 1~( ,) mBin m1~ (,) sssnBin np()2log() ~log(), (log()) ssshNseh,~( , ) spBeta()log() ~0,1000 sNGibbs Sampler Full Conditional Posteriors Monitor 37 38 Parametric Bootstrap clinical Study external concordance S=0 S=1 Count Hazard=Y S=0 S=1 Q=0 n00 n01 Q=1 n10 n11 )1Pr(==S ..(|)ssEh S s ==Pr(1|)spQSs== =*11 ~(, ), /mBin mmm =*11 ~(,), /sss ss snBin nppnn =()*2log() ~log(), (log()) ssshNhseh39 data for Model to Estimate HRs for Test Q, OS.

10 NCIC 225-025 Study OS, S=0 (Wt), S=1 (Mt), NEJM data external concordance data (PMA SSED) S=0 S=1 Count 230 164 Hazard ratio (95% CI) ( , ) ( , ) S=0 S=1 S=NE Q=0 80 2 0 Q=1 3 52 1 Q=NE 5 0 0 To tal 88 54 1 HR Results for OS Te s t Estimate N Wt Mt Trt Diff Attenuation+ S Observed 394 ( , ) ( , ) Q Observed 453 ( , ) ( , ) Q Model 394 ( , ) ( , ) 40 41 data for Model to Estimate HRs for Test Q, PFS. NCIC 225-025 Study PFS, S=0 (Wt), S=1 (Mt), NEJM data external concordance data (PMA SSED) S=0 S=1 Count 230 164 Hazard ratio (95% CI) ( , ) ( , ) S=0 S=1 S=NE Q=0 80 2 0 Q=1 3 52 1 Q=NE 5 0 0 To tal 88 54 1 HR Results for PFS Te s t Estimate N Wt Mt Trt Diff Attenuation+ S Observed 394 ( , ) ( , ) Q Observed 453 ( , ) ( , ) Q Model 394 ( , ) ( , ) 42 Simulation 43 Simulation Results, OS, P PA N PA Coverage Prob. Q=0 Mean Est. OS Q=0 True OS Q=0 Coverage Prob.