Evidence-Based Training Implementation Guide

1 Evidence-Based Training Implementation Guide July 2013. 1st Edition Evidence-Based Training Implementation Guide July 2013. International Air Transport Association Montreal Geneva 1st Edition NOTICE. DISCLAIMER. The information contained in this publication is subject to constant review in the light of changing government requirements and regula- tions. No subscriber or other reader should act on the basis of any such information without referring to applicable laws and regulations and/or without taking appropriate professional advice. Although ev- ery effort has been made to ensure accuracy, the International Air Transport Association shall not be held responsible for any loss or damage caused by errors, omissions, misprints or misinterpretation of the contents hereof.

2 Furthermore, the Interna- tional Air Transport Association expressly disclaims any and all liability to any person or entity, whether a purchaser of this publication or not, in respect of anything done or omitted, and the consequences of anything done or omitted, by any such person or en- tity in reliance on the contents of this publication. International Air Transport Association. All Rights Reserved. No part of this publication may be reproduced, recast, reformatted or trans- mitted in any form by any means, electronic or mechanical, including photocopying, record- ing or any information storage and retrieval sys- tem, without the prior written permission from: Senior Vice President Safety and Flight Operations International Air Transport Association 800 Place Victoria Box 113.

3 Montreal, Quebec CANADA H4Z 1M1. Evidence-Based Training Implementation Guide ISBN 978-92-9252-191-2. 2013 International Air Transport Association. All rights reserved. Montreal Geneva Evidence-Based Training . Implementation Guide . Table of Contents GLOSSARY OF TERMS .. iv DEFINITIONS .. v EXECUTIVE LETTER .. vii DISCLAIMER .. viii BACKGROUND ..1. 1 INTRODUCTION AND RATIONALE ..5. Pilot competencies ..5. Evidence ..6. Operator s EBT Instructors and examiners ..6. 2 KEY ELEMENTS ..7. General Principles ..7. Staged Baseline EBT Program ..8. Enhanced EBT Programs ..11. Collection and Analysis of Training Integration of Analysis.

4 14. Enhanced EBT Program Development ..15. Gaining Regulatory Partnering for Results ..20. 3 COMPETENCIES AND BEHAVIORAL INDICATORS ..21. Definition ..21. Example derived from the ICAO definition ..21. Uses of competencies in EBT recurrent Training ..21. Components of competencies ..22. Example past models ..22. Competencies ..23. Comparison of data ..23. Different systems ..23. ITQI the new safety EBT competency measurement example ..23. Guidelines for the consistency of assessments ..24. Guidelines for developing instructor standardization and Training modules ..24. i Evidence-Based Training .

5 Implementation Guide . Guidelines for developing crew effectiveness modules .. 25. Implementation guidelines for EBT instructors .. 25. Guidelines for maintaining standards and inter-rater reliability .. 25. Guidelines for improving competency and measurement system .. 26. 4 INSTRUCTOR Training .. 27. The EBT instructor .. 27. Effective instruction .. 28. Instruction and 29. Facilitation skills .. 32. Questioning .. 32. Active listening .. 32. Body language .. 33. Observation of behavior .. 33. Giving and receiving criticism .. 33. Continuous development .. 33. Some advice for effective facilitation in a debrief.

6 33. Important characteristics of EBT instructor Training .. 35. Assessment .. 35. Standardizing EBT Training and assessment .. 35. Expanding practice .. 36. Debriefing .. 36. Methodology of debriefing .. 36. 5 PILOT UNDERSTANDING OF EBT AND Training .. 37. 37. Philosophy of EBT history .. 37. Recurrent EBT programs .. 38. 6 Training METRICS .. 40. 40. Process .. 40. Generation of criteria .. 41. Grouping of Criteria .. 41. Development of grading options .. 43. Techniques to be applied in grading .. 45. 7 PROGRAM DEVELOPMENT .. 46. 46. ii Evidence-Based Training . Implementation Guide .

7 Background from the analysis ..47. EBT recurrent assessment and Training matrix ..47. Implementation of a baseline EBT program ..49. General ..49. Construction of EBT modules ..49. 8 CONTINUOUS IMPROVEMENT ..57. Introduction ..57. Development of the Baseline Enhanced EBT Operational data monitoring ..58. Operational characteristics ..58. Competency framework ..58. Safety Management Systems (SMS) and EBT ..59. Reporting Systems ..59. Flight data analysis (FDA) ..59. Flight deck observation ..60. Training Data ..60. Participation in data sharing groups ..63. Integration of analysis.

8 64. Conclusions and recommendations ..64. APPENDIX A Competencies and Behavioral Indicators ..65. APPENDIX B Instructor Training ..67. APPENDIX C Threat and Error Management (TEM) ..70. APPENDIX D Example EBT Module ..78. APPENDIX E Malfunction Clustering ..94. APPENDIX F Baseline Program Priorities ..98. APPENDIX G Training Program Development Guidance Generation 4 (Jet) ..101. APPENDIX H Training Program Development Guidance Generation 3 (Jet) ..110. APPENDIX I Training Program Development Guidance Generation 3 (Turboprop) ..119. APPENDIX J Training Program Development Guidance Generation 2 (Jet).

9 130. APPENDIX K Training Program Development Guidance Generation 2 (Turboprop) ..139. APPENDIX L Training Program Development Guidance Generation 1 (Jet) ..148. APPENDIX M Example Grading Scales ..149. APPENDIX N Frequently Asked Questions ..153. iii Evidence-Based Training . Implementation Guide . GLOSSARY OF TERMS. A/C Aircraft ACAS Airborne Collision Avoidance System APP Approach AQP Advanced Qualification Program ATA Air Transport Association ATC Air Traffic Control ATO Approved Training Organization ATQP Alternative Training and Qualification Program CAA Civil Aviation Authority CLB Climb CRM Crew Resource Management CRZ Cruise DES Descent EASA European Aviation Safety Agency EBT Evidence-Based Training DA Decision altitude FAA Federal Aviation Administration (United States of America).

10 FL Flight level FMS Flight Management System FOQA Flight Operations Quality Assurance FSTD Flight Simulation Training Device GA or G-A Go-around GND Ground IOE Initial Operating Experience IRR Inter-rater reliability LDG Landing LOE Line Oriented Evaluation LOFS Line Orientated Flight Scenario LOFT Line Oriented Flight Training LOSA Line Operations Safety Audit MEL Minimum equipment list MPL Multi-crew pilot license Neo New engine option PF Pilot Flying PIC Pilot-in-Command PM Pilot Monitoring PNF Pilot Not Flying (former term for PM). QAR Quick Access Recorder SOP Standard Operating Procedure TEM Threat and Error Management TO Take-off iv Evidence-Based Training .