Technology Readiness Assessment Guide

1 From August 11, 2016 to August 10, 2017, GAO is seeking input and feedback on this Exposure Draft from all interested parties. See page 9 for more 2016 GAO-16-410 GAssessment GuideTechnology ReadinessBest Practices for Evaluating the Readiness of Technology for Use in Acquisition Programs and Projects Page 1 DRAFT Preface .. 8 Introduction .. 11 The Guide s Case Studies .. 13 The Guide s Readers .. 13 Acknowledgments .. 14 Chapter 1 .. 15 What Is a Technology Readiness Assessment (TRA)? .. 15 Definitions and Overview.

2 16 TRAs Inform Technology Development and Identify Potential Concerns .. 18 Overview of Technology Development and Its Relationship to Acquisition Programs .. 18 Technology Development Models .. 23 TRA s Relationship to Program Management and Oversight .. 24 Tailoring TRAs for Different Purposes .. 25 Chapter 2 .. 26 Why TRAs Are Important and Understanding Their Limitations .. 26 Maturity of Technology at Program Start Is an Important Determinant of Success .. 28 TRA Process Is a Mechanism That Informs Important Acquisition Functions.

3 29 Understanding TRAs Can Help Practitioners and Consumers of Information .. 31 TRAs Are Snapshots in Time .. 33 Advancements in Technology Can Pose Challenges in Applying 33 Organizational Experience, Culture, and Bias Can Affect TRAs .. 34 Page 2 DRAFT TRAs Depend on the Quality and Availability of Credible Data .. 36 Chapter 3 .. 38 Best Practice: A Reliable Process for Conducting Credible TRAs .. 38 More Frequent Evaluations of Technology Maturity .. 40 High Quality TRAs .. 41 Chapter 4 .. 43 Best Practice: Including Technology Maturity Assessments in the Program Strategy, Designing the TRA Plan, and Determining the 43 Technical Maturity Assessment Strategy.

4 43 The Assessment Team .. 46 The TRA Plan .. 48 Chapter 5 .. 52 Best Practice: Selecting Critical 52 Critical Technologies Defined .. 52 Challenges in Selecting Critical Technologies .. 54 Steps for Selecting Critical Technologies .. 55 Identifying Other Important Technologies and Programmatic Issues .. 65 Chapter 6 .. 68 Best Practice: Evaluating Critical Technologies .. 68 Steps for Evaluating Critical Technologies .. 68 Relevant Information Must Be Used to Evaluate Critical Technologies .. 74 Operational Environment Is Key to Evaluating Critical Technologies.

5 74 Creating Critical Technology Subsets for Exceptionally Large or Complex Programs .. 75 Page 3 DRAFT Chapter 7 .. 77 Best Practice: Preparing the TRA Report .. 77 The TRA Report .. 77 Steps for Preparing and Coordinating the TRA Report .. 80 Response to the TRA Report .. 84 How Dissenting Views Are Documented and Submitted .. 84 Chapter 8 .. 86 Best Practice: Using the TRA Results .. 86 How TRA Reports Are Used .. 86 TRAs for Governance Decisions .. 87 TRAs as Knowledge- building Exercises .. 88 Identification of Potential Areas of Concern and Risk.

6 89 Early Technology Development .. 91 TRA Process Facilitates Information Sharing Opportunities .. 92 Basis for Developing a Technology Maturation Plan (TMP) for Immature Technologies .. 93 Chapter 9 .. 95 Best Practice: Preparing a Technology Maturation Plan (TMP) .. 95 Steps for Preparing a Technology Maturation Plan .. 96 Updating a Technology Maturation Plan .. 98 The Technology Maturation Plan Template .. 101 Chapter 10 .. 106 Practices Are Evolving in Evaluating Software Systems and Systems Integration Using TRAs.

7 106 Applying TRAs to Software Systems .. 106 Software Embedded Technologies versus Software-only Technologies .. 108 Page 4 DRAFT Development of System-level Readiness Metrics .. 109 Appendix I: Key Questions to Assess How Well Programs Followed the Six Step Process for Developing Credible TRAs .. 112 Appendix II: Auditing Agencies and Their Websites .. 117 Appendix III: Case Study Backgrounds .. 119 Case Study 1: Immature Technologies Increases Risk, GAO-08-408 119 Case Study 2: Assessments Provide Key Information, GAO-10-675.

8 120 Case Study 3: Space Programs Often Underestimate Costs, 120 Case Study 4: Program Updates Can Change Critical Technologies, GAO-02-201 .. 121 Case Study 5: Identifying Back-up Critical Technologies, GAO-08-467SP .. 121 Appendix IV: Experts Who Helped Develop This Guide .. 123 Appendix V: Contacts and Acknowledgments .. 130 GAO Contacts .. 130 Other Leadership Provided for This Project .. 130 Acknowledgments .. 130 Appendix VI: Examples of Various TRL Definitions and Descriptions by Organization .. 131 Appendix VII: Other Types of Readiness Levels.

9 137 Appendix VIII: Agency Websites Where TRA Report Examples Can Be Found .. 142 References .. 143 Image Sources .. 146 Page 5 DRAFT Tables Table 1: Cost and Schedule Experiences for Products with Mature and Immature Technologies 29 Table 2: Six Steps for Conducting a Technology Readiness Assessment (TRA) 39 Table 3: Characteristics of High Quality Technology Readiness Assessments (TRA) 41 Table 4: Software Implementation Project Work Breakdown Structure 60 Table 5: Technology Readiness Levels (TRLs) Supporting Information for Hardware and Software 72 Table 6.

10 Example Program Management Tools Used with Technology Readiness Assessments (TRAs) 90 Table 7: Auditing Agency Websites 117 Table 8: GAO Reports Used As Case Study in the TRA Guide 119 Table 9: Experts Who Made Significant Contributions 123 Table 10: Experts Who Made Noteworthy Contributions 124 Table 11: DOD Technology Readiness Levels (2011) 131 Table 12: DOD Software Technology Readiness Levels (2009) 132 Table 13: NASA Hardware Technology Readiness Levels (2013) 133 Table 14: NASA Software Technology Readiness Levels (2013) 134 Table 15: DOE Technology Readiness Levels (2011) 135 Table 16: DOD Manufacturing Readiness Levels 137 Table 17: Integration Readiness Levels 140 Table 18: System Readiness Levels 141 Figures Figure 1: Technology Readiness Levels 17 Figure 2: Phased Acquisition Cycle with Decision Points 19 Page 6 DRAFT Figure 3.