Evidence-Based Training Implementation Guide - IATA

1 Evidence-Based Training Implementation Guide July 2013. 1st Edition Evidence-Based Training Implementation Guide July 2013. International Air Transport Association Montreal Geneva 1st Edition NOTICE. DISCLAIMER. The information contained in this publication is subject to constant review in the light of changing government requirements and regula- tions. No subscriber or other reader should act on the basis of any such information without referring to applicable laws and regulations and/or without taking appropriate professional advice. Although ev- ery effort has been made to ensure accuracy, the International Air Transport Association shall not be held responsible for any loss or damage caused by errors, omissions, misprints or misinterpretation of the contents hereof. Furthermore, the Interna- tional Air Transport Association expressly disclaims any and all liability to any person or entity, whether a purchaser of this publication or not, in respect of anything done or omitted, and the consequences of anything done or omitted, by any such person or en- tity in reliance on the contents of this publication.

2 International Air Transport Association. All Rights Reserved. No part of this publication may be reproduced, recast, reformatted or trans- mitted in any form by any means, electronic or mechanical, including photocopying, record- ing or any information storage and retrieval sys- tem, without the prior written permission from: Senior Vice President Safety and Flight Operations International Air Transport Association 800 Place Victoria Box 113. Montreal, Quebec CANADA H4Z 1M1. Evidence-Based Training Implementation Guide ISBN 978-92-9252-191-2. 2013 International Air Transport Association. All rights reserved. Montreal Geneva Evidence-Based Training . Implementation Guide . Table of Contents GLOSSARY OF TERMS .. iv DEFINITIONS .. v EXECUTIVE LETTER .. vii DISCLAIMER .. viii BACKGROUND ..1. 1 INTRODUCTION AND RATIONALE.

3 5. Pilot competencies ..5. evidence ..6. Operator s EBT Instructors and examiners ..6. 2 KEY ELEMENTS ..7. General Principles ..7. Staged Baseline EBT Program ..8. Enhanced EBT Programs ..11. Collection and Analysis of Training Integration of Analysis ..14. Enhanced EBT Program Development ..15. Gaining Regulatory Partnering for Results ..20. 3 COMPETENCIES AND BEHAVIORAL INDICATORS ..21. Definition ..21. Example derived from the ICAO definition ..21. Uses of competencies in EBT recurrent Training ..21. Components of competencies ..22. Example past models ..22. Competencies ..23. Comparison of data ..23. Different systems ..23. ITQI the new safety EBT competency measurement example ..23. Guidelines for the consistency of assessments ..24. Guidelines for developing instructor standardization and Training modules.

4 24. i Evidence-Based Training . Implementation Guide . Guidelines for developing crew effectiveness modules .. 25. Implementation guidelines for EBT instructors .. 25. Guidelines for maintaining standards and inter-rater reliability .. 25. Guidelines for improving competency and measurement system .. 26. 4 INSTRUCTOR Training .. 27. The EBT instructor .. 27. Effective instruction .. 28. Instruction and 29. Facilitation skills .. 32. Questioning .. 32. Active listening .. 32. Body language .. 33. Observation of behavior .. 33. Giving and receiving criticism .. 33. Continuous development .. 33. Some advice for effective facilitation in a debrief .. 33. Important characteristics of EBT instructor Training .. 35. Assessment .. 35. Standardizing EBT Training and assessment .. 35. Expanding practice .. 36. Debriefing.

5 36. Methodology of debriefing .. 36. 5 PILOT UNDERSTANDING OF EBT AND Training .. 37. 37. Philosophy of EBT history .. 37. Recurrent EBT programs .. 38. 6 Training METRICS .. 40. 40. Process .. 40. Generation of criteria .. 41. Grouping of Criteria .. 41. Development of grading options .. 43. Techniques to be applied in grading .. 45. 7 PROGRAM DEVELOPMENT .. 46. 46. ii Evidence-Based Training . Implementation Guide . Background from the analysis ..47. EBT recurrent assessment and Training matrix ..47. Implementation of a baseline EBT program ..49. General ..49. Construction of EBT modules ..49. 8 CONTINUOUS IMPROVEMENT ..57. Introduction ..57. Development of the Baseline Enhanced EBT Operational data monitoring ..58. Operational characteristics ..58. Competency framework ..58. Safety Management Systems (SMS) and EBT.

6 59. Reporting Systems ..59. Flight data analysis (FDA) ..59. Flight deck observation ..60. Training Data ..60. Participation in data sharing groups ..63. Integration of analysis ..64. Conclusions and recommendations ..64. APPENDIX A Competencies and Behavioral Indicators ..65. APPENDIX B Instructor Training ..67. APPENDIX C Threat and Error Management (TEM) ..70. APPENDIX D Example EBT Module ..78. APPENDIX E Malfunction Clustering ..94. APPENDIX F Baseline Program Priorities ..98. APPENDIX G Training Program Development Guidance Generation 4 (Jet) ..101. APPENDIX H Training Program Development Guidance Generation 3 (Jet) ..110. APPENDIX I Training Program Development Guidance Generation 3 (Turboprop) ..119. APPENDIX J Training Program Development Guidance Generation 2 (Jet) ..130. APPENDIX K Training Program Development Guidance Generation 2 (Turboprop).

7 139. APPENDIX L Training Program Development Guidance Generation 1 (Jet) ..148. APPENDIX M Example Grading Scales ..149. APPENDIX N Frequently Asked Questions ..153. iii Evidence-Based Training . Implementation Guide . GLOSSARY OF TERMS. A/C Aircraft ACAS Airborne Collision Avoidance System APP Approach AQP Advanced Qualification Program ATA Air Transport Association ATC Air Traffic Control ATO Approved Training Organization ATQP Alternative Training and Qualification Program CAA Civil Aviation Authority CLB Climb CRM Crew Resource Management CRZ Cruise DES Descent EASA European Aviation Safety Agency EBT Evidence-Based Training DA Decision altitude FAA Federal Aviation Administration (United States of America). FL Flight level FMS Flight Management System FOQA Flight Operations Quality Assurance FSTD Flight Simulation Training Device GA or G-A Go-around GND Ground IOE Initial Operating Experience IRR Inter-rater reliability LDG Landing LOE Line Oriented Evaluation LOFS Line Orientated Flight Scenario LOFT Line Oriented Flight Training LOSA Line Operations Safety Audit MEL Minimum equipment list MPL Multi-crew pilot license Neo New engine option PF Pilot Flying PIC Pilot-in-Command PM Pilot Monitoring PNF Pilot Not Flying (former term for PM).

8 QAR Quick Access Recorder SOP Standard Operating Procedure TEM Threat and Error Management TO Take-off iv Evidence-Based Training . Implementation Guide . DEFINITIONS. Assessment. The determination as to whether a candidate meets the requirements of the competency standard. ATA Chapters. The chapter numbering system controlled and published by the Air Transport Association, which provides a common referencing standard for all commercial aircraft documentation. Behavior. The way a person responds, either overtly or covertly, to a specific set of conditions, which is capable of being measured. Behavioral indicator. An overt action performed or statement made by any flight crew member that indicates how the crew is handling the event. Competency. A combination of skills, knowledge and attitudes required to perform a task to the prescribed standard.

9 Competency- based Training . Training and assessment that are characterized by a performance orientation, emphasis on standards of performance and their measurement and the development of Training to the specified performance standards. Core competencies. A group of related behaviors, based on job requirements, which describe how to effectively perform a job. They describe what proficient performance looks like. They include the name of the competency, a description, and a list of behavioral indicators. Critical flight maneuvers. Maneuvers that place significant demand on a proficient crew. Critical system malfunctions. Aircraft system malfunctions that place significant demand on a proficient crew. These malfunctions should be determined in isolation from any environmental or operational context. Evidence-Based Training (EBT).

10 Training and assessment that is characterized by developing and assessing the overall capability of a trainee across a range of competencies rather than by measuring the performance of individual events or maneuvers. EBT instructor. A person, who has undergone a screening and selection process, successfully completed an approved course in delivering competency- based Training , and is subsequently authorized to conduct recurrent assessment and Training within an approved EBT program. EBT module. A session or combination of sessions in a qualified FSTD as part of the 3-year cycle of recurrent assessment and Training . EBT session. A single defined period of Training in a qualified FSTD that normally forms part of an EBT. module. EBT scenario. Part of an EBT session encompassing one or more scenario elements, constructed to facilitate real time assessment or Training EBT scenario element.