Software Service History Report

1 DOT/FAA/AR-01/125 Office of Aviation Research Washington, 20591 Software Service History Report January 2002 Final Report This document is available to the public through the National Technical Information Service (NTIS), Springfield, Virginia 22161. Department of Transportation Federal Aviation Administration NOTICE This document is disseminated under the sponsorship of the Department of Transportation in the interest of information exchange. The United States Government assumes no liability for the contents or use thereof. The United States Government does not endorse products or manufacturers. Trade or manufacturer s names appear herein solely because they are considered essential to the objective of this Report .

2 This document does not constitute FAA certification policy. Consult your local FAA aircraft certification office as to its use. This Report is available at the Federal Aviation Administration William J. Hughes Technical Center s Full-Text Technical Reports page: in Adobe Acrobat portable document format (PDF). Technical Report Documentation Page 1. Report No. DOT/FAA/AR-01/1252. Government Accession Recipient s Catalog No. 4. Title and Subtitle Software Service History REPORT5. Report Date January 2002 6. Performing Organization Code 7. Author(s) Uma D. Ferrell and Thomas K. Ferrell 8. Performing Organization Report No. 9.

3 Performing Organization Name and Address Ferrell and Associates Consulting, Inc. 1261 Cobble Pond Way10. Work Unit No. (TRAIS) Vienna, VA 22182 11. Contract or Grant 12. Sponsoring Agency Name and Address Department of Transportation Federal Aviation Administration13. Type of Report and Period Covered Final Report Office of Aviation Research Washington, 2059114. Sponsoring Agency Code AIR-13015. Supplementary NotesThis Report is one of two deliverables under this contract. The first being the Software Service History Handbook. The FAA William J. Hughes COTR is Charles Abstract The safe and reliable operation of Software within civil aviation systems and equipment has historically been assured through the application of rigorous design assurance applied during the Software development process.

4 Increasingly, manufacturers are seeking ways to use Software that has been previously developed for other domains or that has been previously certified for use in lower criticality aviation applications. Product Service History is one method for demonstrating that such Software is acceptable for use in the new application domain. In theory, product Service History would seem to be a fairly simple concept, both to understand and to apply. However, in practice, such use has proved extremely problematic, as questions of how to measure the historic performance and the relevance of the provided data have surfaced. This Report represents the results of research in this area, for use by the Federal Aviation Administration in formulating new guidance and prioritizing future research work in the area of product Service History .

5 17. Key Words DO-178B, SC-190, DO-248, Product Service History , Software reliability, Airborne systems and equipment, Error rates18. Distribution Statement This document is available to the public through the National Technical Information Service (NTIS) Springfield, Virginia Security Classif. (of this Report ) Unclassified20. Security Classif. (of this page) Unclassified21. No. of Pages 8822. Price Form DOT F (8-72) Reproduction of completed page authorized TABLE OF CONTENTS Page EXECUTIVE SUMMARY vii 1. INTRODUCTION 1 Purpose 1 Scope 1 Background 1 The Research Effort 3 Related Activities/Documents 4 Document Structure 5 2.

6 DO-178B FRAMEWORK 6 The Definition of Product Service History 6 Analysis of Product Service History in DO-178B 7 Relationship With Previously Developed Software 15 Product Service History Versus Software Reliability 15 3. THE ELEMENTS OF PRODUCT Service History 16 Questions of Problem Reporting 16 Individual Questions to Consider 17 Identified Gaps and Gap Closure Problem Reporting 21 Questions of Operation 24 Individual Questions to Consider 26 Identified Gaps and Gap Closure Operation 27 Questions of Environment 28 Individual Questions to Consider 30 Identified Gaps and Gap Closure Environment 32 Questions of Time 33 Individual Questions to Consider 35 Identified Gaps and Gap Closure Time 36 4.

7 ADEQUACY OF DEVELOPMENT PROCESS 38 5. ESTABLISHMENT OF EQUIVALENT SAFETY 40 iii 6. RESEARCH SUMMARY 41 Research Findings 41 Follow-On Prioritized Research Needs 43 7. CONCLUSION 44 APPENDICES A Data Collection and Synthesis B Service History Literature Search LIST OF FIGURES Figure Page 1 Overall Research Effort Design 3 2 Research Taxonomies 4 3 Operation 24 4 Environment 29 5 Timeline 33 6 Calculation of Service History Duration 34 7 Product Service History Multiple Operating Copies 38 LIST OF TABLES Table Page 1 Analysis of DO-178B.

8 Section 8 2 Worksheet for Questions of Problem Reporting 18 3 Worksheet for Questions of Operation 26 4 Worksheet for Questions of Environment 31 5 Worksheet for Questions of Time 35 iv LIST OF ACRONYMS AC Advisory Circular ACO Aircraft Certification Office ATM Air Traffic Management CAST Certification Authorities Software Team CNS Communication, Navigation, Surveillance COTR Contracting Officer s Technical Representative COTS Commercial Off-The-Shelf DO Document DOT Department of Transportation ED European Document EUROCAE European Organization for Civil Aviation Equipment FAA Federal Aviation Administration FAR Federal Aviation Regulations HBAW Airworthiness Handbook ICSE International Conference on Software Engineering IEEE Institute of Electrical and Electronic Engineers ISO/IEC International Organization for Standardization/International Electrotechnical Commission JAA Joint Airworthiness

9 Authorities MTTR Mean Time to Repair MoD Ministry of Defense, United Kingdom NASA National Aeronautics and Space Administration OSED Operational services and Environment Definition PDF Portable Document Format PDS Previously Developed Software PSAC Plan for Software Aspects of Certification RTCA formerly Radio Technical Commission for Aeronautics SAS Software Accomplishment Summary SC Special Committee SCM Software Configuration Management SOUP Software of Unknown Pedigree SSA System Safety Assessment SSAC Streamlining Software Aspects of Certification TGL Temporary Guidance Leaflet v/vi EXECUTIVE SUMMARY The safe and reliable operation of

10 Software within civil aviation systems and equipment has historically been assured through the application of rigorous design assurance applied during the Software development process. Increasingly, manufacturers are seeking ways to use Software that has been previously developed for other domains, has been previously certified for use in lower criticality aviation applications, or has been certified to earlier versions or different standards than those currently employed. Product Service History is one method for demonstrating that such Software is acceptable for use in a new application. In theory, product Service History would seem to be a fairly simple concept, both to understand and to apply.