METHOD VALIDATION & VERIFICATION

1 METHOD VALIDATION & VERIFICATION . APEC-PTIN Project: CHILE 2013. 1 2012 3M. All Rights Reserved. OBJECTIVES. Provide definitions of VALIDATION & VERIFICATION from key resources List some key performance characteristics for METHOD VALIDATION Provide an overview of 3rd party METHOD VALIDATION schemes Distinguish VALIDATION and VERIFICATION List some key performance characteristics for METHOD VERIFICATION Distinguish VERIFICATION : METHOD VERIFICATION vs continuous VERIFICATION Provide links to references for further use / information 2 2012 3M. All Rights Reserved. Main VALIDATION & VERIFICATION Definition Sources: CODEX - CAC/GL 74-2010: Guidelines on performance criteria and VALIDATION of methods for detection, identification and quantification of specific DNA sequences and specific proteins in foods ISO/DIS 16140-1: Microbiology of food and animal feed, METHOD VALIDATION , Part 1: Vocabulary Currently under revision USDA FSIS: Guidance for Test Kit Manufacturers, Laboratories: Evaluating the Performance of Pathogen Test Kit methods ; and Establishment Guidance for the Selection of a Commercial or Private Microbiological Testing Laboratory US FDA: Guidelines for the VALIDATION of Analytical methods for the Detection of Microbial Pathogens in Foods Health Canada: The Compendium of Analytical methods , Volume 1, Development of methods ISO/IEC 17025:2005 General requirements for the competence of testing and calibration laboratories 3 2012 3M.

2 All Rights Reserved. WHAT is VALIDATION ? Reference VALIDATION Definition Codex Process to establish the performance characteristics and limitations of an analytical METHOD : which analytes, in what kind of matrices, in the presence of which interference. CAC/GL 741 Result = precision and trueness values of a certain analytical METHOD under the examined conditions. 1 [pg 2]. ISO 16140-12 Establishment of the performance characteristics of a METHOD and provision of objective evidence that the performance requirements for a specified intended use are [pg 2]. USDA FSIS3 Process to measure performance characteristics of a particular test, with the goal of determining whether the test is equivalent to the reference test for the intended conditions of use. Equivalent = the performance characteristics are statistically [pg 2]. US FDA4 Demonstration that adequate confidence is provided when the results obtained by the alternative METHOD the commercially available kit, are comparable to or exceed those obtained using the reference METHOD using the statistical criteria contained in the approved VALIDATION [pg 16].

3 Health Canada5 Evaluation of the performance parameters of a new METHOD in comparison to an accepted reference METHOD using paired or unpaired samples. In the context of relative VALIDATION , the results of the reference METHOD are assumed to reflect the true microbiological status of the samples and the performance parameters of the alternative METHOD are calculated in relation to this. (Source: Part 4). 4. ISO 17025:20056 The confirmation by examination and the provision of objective evidence that the particular 2012 3M. All Rights Reserved. requirements for a specific intended use are Summary of ALL VALIDATION reference definitions: VALIDATION : 1. Performance characteristics are defined, then . 2. compared to a reference METHOD , and then . 3. statistically evaluated to determine equivalence . A METHOD is validated for the matrices [& sample sizes] included in the VALIDATION 5 2012 3M. All Rights Reserved. Categories of Microbiological Test methods Performance Characteristics Included in a VALIDATION Study1.

4 Performance Characteristic Quantitative Qualitative (presence/. absence). Relative Accuracy (% Recovery) Yes No Matrix Effects Yes Yes Precision Yes No Sensitivity Yes Yes Specificity Yes Yes Inclusivity Yes Yes Exclusivity Yes Yes False Positive Rate Yes Yes False Negative Rate Yes Yes LOD [POD or LOD50] Yes Yes LOQ Yes No Ruggedness Yes Yes Linearity/Range Yes No Dr. Michael Brodsky 6 AOAC International 2013 Annual Meeting 2012 3M. All Rights Reserved. methods requiring VALIDATION : New or original methods Modifications to validated methods Extension of scope to include additional analytes, matrices, or changes in intended use Changes involving new technology or automation Significant parameter changes: reagents/apparatus, time /temperature incubation periods, enrichment media, etc. It is possible that small changes/matrix extensions to a related product etc would not need a full VALIDATION ' but may need some level 7.

5 Of testing to assess for unintended differences. 2012 3M. All Rights Reserved. WHY are Validated methods important? Validated methods are required for conducting Official Testing . Official = adjective, relating to an authority or public body and its activities and responsibilities [Oxford online dictionary]. European Union regulations specifically stipulate validated methods that have been collaboratively studied Internationally, regulatory agencies require, or at least prefer, to use methods that have been validated and collaboratively studied. Some global regulatory agencies require AOAC Official methods of Analysis to receive Final Action, before they may be used as an Official METHOD 8 2012 3M. All Rights Reserved. REQUIREMENTS of methods for Official use Agency METHOD VALIDATION Requirements Codex Whenever possible, only methods for which the reliability (accuracy, reproducibility, inter- and intra-laboratory variation) has been statistically established in comparative or collaborative studies CAC/GL 218 in several laboratories should be used.

6 (EC) No The use of alternative analytical methods is acceptable when the methods are validated against the reference METHOD in Annex I and if a proprietary METHOD , certified by a third party in accordance 2073/20079 with the protocol set out in EN/ISO standard 16140 or other internationally accepted similar protocols, is used. Pg 9. USDA FSIS11 All analytical methods described in FSIS-MLG have been scientifically validated and are considered fit for their intended purpose. Laboratories may also use other validated testing methods that differ from the methods described in FSIS-MLG. For the intended conditions of use, the performance characteristics of the new METHOD and the well characterized METHOD should be statistically indistinguishable Pg 13. US FDA4 FDA-BAM presents the agency's preferred laboratory procedures for microbiological analyses of foods and cosmetics. methods fully validated in a collaborative study monitored and evaluated by an independent accrediting body AOAC-OMA, AFNOR, etc.

7 Require in-house VERIFICATION for the first use . Health Canada5 HPB methods and Laboratory Procedures are generally used to determine compliance with the Food and 9. Drugs Act and Regulations. In general, methods published in the Compendium of Analytical methods 2012 3M. All Rights Reserved. for which the "application" section is appropriate for the intended purpose are used for microbiological testing of food. What is considered an OFFICIAL METHOD ? Official methods = methods used or accepted for use by a regulatory authority Traditional, cultural methods [may be jurisdiction specific]. ISO METHOD ISO 16140 Validated methods AOAC Official methods FDA Bacteriological Analytical methods USDA FSIS Microbiology Laboratory methods HC Compendium of Analytical methods , Volumes 1 to 5. Any of the above methods MAY be accepted for use by regulatory agencies, IF: They have been compared to an acceptable reference METHOD They have been validated as fit for purpose for the matrices analyzed 10 2012 3M.

8 All Rights Reserved. VALIDATION Schemes ISO. AFNOR: French National Organization for Standardization MicroVal: European certification body for microbiology methods NordVal: Nordic certification body: Denmark, Finland, Iceland, Norway and Sweden AOAC Research Institute [RI]: International Standards Organization Performance Tested METHOD (PTM). Official METHOD of Analysis (OMA). Health Canada: Canadian certification through Microbiology methods Committee (MMC). Official methods (MFOs). HPB methods (MFHPBs). Laboratory Procedures (MFLPs). 11 2012 3M. All Rights Reserved. AFNOR and MicroVal Certification schemes based on EN/ISO 16140: Comparison Study to reference METHOD 1. Independent expert laboratory co-ordinates VALIDATION 2. Inter-laboratory study [collaborative study]. 3. All Foods Claim Regular audit of the quality system of the manufacturer [based on EN 29002. standard (ISO 9002)]. Regular VERIFICATION of the certified methods post certification [every 4 years].

9 12 2012 3M. All Rights Reserved. AOAC Research Institute AOAC Performance Tested METHOD [PTM]. Pre-collaborative + 1 Independent lab: I/E; Ruggedness testing, paper review of quality system. Yearly renewal paper process only NO all foods claim only for matrices evaluated AOAC PTM / Official METHOD of Analysis [OMA] Harmonization Complete AOAC PTM program / Initiate collaborative study to achieve OMA First Action status;. Final Action status after 2 yrs use in the industry without issue AOAC PTM / OMA/ MicroVal Harmonization Both programs run simultaneously 1 collaborative study 13 2012 3M. All Rights Reserved. Health Canada Micro Food Lab Procedure [MFLP]. Independent lab [pre-collaborative study]. All Foods Claim Reviewed every 5 years Paper review of quality system Health Protection Food Branch [MFHPB] methods Must be an MFLP METHOD for 1 year prior to collaborative study 14 2012 3M. All Rights Reserved. METHOD Scheme Websites list validated methods including claimed matrices, sample sizes AFNOR MicroVal AOAC.

10 A498-19b9763d11b4&hkey=33b744f6-f71e-456 a-9305-552469587666&CCO=5. Health Canada (Volumes 1 to 5). NordVal 15 2012 3M. All Rights Reserved. Harmonization ISPAM. International Stakeholder Panel on Analytical methods A global Stakeholder panel was formed [through AOAC RI]. Performance characteristics between agencies/schemes were compared 5 areas for harmonization: 1. Reference methods so much variation, some matrix specific 2. Definition of Fractional Positives 25-75% positive out of 20 samples 3. Selection of Food/Category Chart created for incorporation into ISO 16140-2. 4. Statistical Analysis RLOD, Chi-Square and/or POD models 5. Data Sets for Qualitative Collaborative Study/Sample Size 10 labs / 25 g samples Recent addition: All Foods claim now = Broad range of foods . 16 2012 3M. All Rights Reserved. Comparison of VERIFICATION definitions Reference VERIFICATION Definition Codex Could not find a METHOD VERIFICATION definition Codex documents [only process VERIFICATION ].