COST ESTIMATING AND ASSESSMENT GUIDE

1 March 2020 GAO-20-195 GCOST ESTIMATING AND ASSESSMENT GUIDEBest Practices for Developing and Managing Program Costs Page i GAO-20-195G cost ESTIMATING and ASSESSMENT GUIDE Preface 1 Introduction 3 Chapter 1 Why Government Programs Need cost Estimates and the Challenges in Developing Them 8 cost ESTIMATING Challenges 9 Chapter 2 cost Analysis and cost Estimates 17 Types of cost Estimates 17 Significance of cost Estimates 22 cost Estimates in Acquisition 22 The Importance of cost Estimates in Establishing Budgets 24 cost Estimates and Affordability 25 Chapter 3 The Characteristics of Credible cost Estimates and a Reliable Process for Creating Them 31 The Four Characteristics of a Reliable cost Estimate 31 Best Practices Related to Developing and Maintaining a Reliable cost Estimate 32 cost ESTIMATING Best Practices and the ESTIMATING Process 33 Chapter 4 Step 1: Define the Estimate s Purpose 38 Scope 38 Including All Costs in a Life Cycle cost Estimate 39 Survey of Step 1 40 Chapter 5 Step 2: Developing the ESTIMATING Plan 41 Team Composition and Organization 41 Study Plan and Schedule 42 cost ESTIMATING Team 44 Certification and Training for cost ESTIMATING and EVM Analysis 46 Survey of Step 2 46 Chapter 6 Step 3.

2 Define the Program - Technical Baseline Description 48 Definition and Purpose 48 Process 48 Contents 49 Contents Page ii GAO-20-195G cost ESTIMATING and ASSESSMENT GUIDE Key System Characteristics and Performance Parameters 52 Survey of Step 3 54 Chapter 7 Step 4: Determine the ESTIMATING Structure - Work Breakdown Structure 56 WBS Concepts 56 Common WBS Elements 61 WBS Development 63 Standardized WBS 64 WBS and Scheduling 65 WBS and EVM 66 WBS and Risk Management 67 WBS Benefits 67 Survey of Step 4 69 Chapter 8 Step 5: Identify Ground Rules and Assumptions 71 Ground Rules 71 Assumptions 71 Global and Element-Specific Ground Rules and Assumptions 73 Assumptions, Sensitivity, and Risk Analysis 76 Survey of Step 5 79 Chapter 9 Step 6: Obtain the Data 81 Data Collection 81 Types of Data 84 Sources of Data 86 Data Applicability 89 Validating and Analyzing the Data 90 Data Bias 92 EVM Data Reliability 92 Data Normalization 93 Recurring, Nonrecurring, Fixed, and Variable Costs 94 Inflation Adjustments 96 Selecting the Proper Indexes 97 Data Documentation 98 Survey of Step 6 99 Chapter 10 Step 7.

3 Develop the Point Estimate 102 cost ESTIMATING Methods 102 Page iii GAO-20-195G cost ESTIMATING and ASSESSMENT GUIDE Pulling the Point Estimate Together and Comparing to an Independent cost Estimate 123 Survey of Step 7 127 Chapter 11 Step 8: Conduct Sensitivity Analysis 130 Sensitivity Analysis in cost ESTIMATING 130 Steps in Performing a Sensitivity Analysis 134 Identifying Factors for the Sensitivity Analysis 134 The Limitations of Sensitivity Analysis 136 Survey of Step 8 137 Chapter 12 Step 9: Conduct Risk And Uncertainty Analysis 139 Risk and Uncertainty 141 The Need for cost Risk and Uncertainty Analysis 142 Conducting a cost Risk and Uncertainty Analysis 144 cost Risk Analysis Data and Methods 146 cost Risk Analysis with Risk Drivers 149 Collecting Risk and Uncertainty Data 151 Correlation between cost Elements 155 cost Contingency 158 Allocating, Phasing, and Converting a Risk Adjusted cost Estimate 160 Updating and Documenting a Risk and Uncertainty Analysis 163 Survey of Step 9 164 Chapter 13 Step 10: Document the Estimate 167 Elements of cost Estimate Documentation 170 Other Considerations 173 Survey of Step 10 173 Chapter 14 Step 11.

4 Present the Estimate to Management 176 Survey of Step 11 178 Chapter 15 Step 12: Update the Estimate 180 Update the Program cost Estimate with Actual Costs 180 Survey of Step 12 183 Page iv GAO-20-195G cost ESTIMATING and ASSESSMENT GUIDE Chapter 16 Auditing and Validating the cost Estimate 184 The Four Characteristics of a Reliable cost Estimate and their Best Practices 184 Assessing the Reliability of a cost Estimate 197 Assessing the Extent to Which cost ESTIMATING Best Practices Are Met 201 Assessing the Thoroughness of an Organization s ESTIMATING Guidance 203 Chapter 17 Earned Value Management 207 What EVM Is 207 Other Benefits of EVM 215 Obstacles to EVM 216 Federal and Industry Guidelines for Implementing EVM 220 Chapter 18 Earned Value

5 Management Process 223 EVM Process 223 Chapter 19 Earned Value Management Execution 241 Validating the Performance Measurement Baseline and Integrated Baseline Reviews 241 Contract Performance Reports 243 Analyze Performance 252 Project Future Performance 264 Continue EVM until the Program is Complete 270 Chapter 20 Earned Value Management Validation, Surveillance, and Over Target Baselines 272 Validating the EVM System 272 Using System Surveillance to Keep the Performance Management Baseline Current 276 Developing a Corporate Surveillance Plan 278 Overtarget Baselines and Schedules 284 Key Steps of the Overtarget Budget-Overtarget Schedule Process 290 Keep Management Updated 296 Auditing EVM Systems and Data 297 Survey of EVM 299 Page v GAO-20-195G cost ESTIMATING and ASSESSMENT GUIDE Appendix I Objectives, Scope.

6 And Methodology 305 Appendix II Case Study Backgrounds 306 Appendix III Experts Who Helped Develop This GUIDE 340 Appendix IV Data Collection Instrument and Rationale 344 Appendix V ESTIMATING Software Costs 347 Appendix VI Examples of Work Breakdown Structures 360 Appendix VII Learning Curve Analysis 378 Appendix VIII Technology Readiness Levels 386 Appendix IX Integrated Baseline Review 388 Appendix X Common Probability Distributions 397 Appendix XI Best Practices for the Analysis of Alternatives Process 399 Appendix XII The cost ESTIMATING Process and Internal Control 417 Page vi GAO-20-195G cost ESTIMATING and ASSESSMENT GUIDE Appendix XIII GAO Contacts and Staff Acknowledgments 427 Glossary 428 References 436 Image Sources 462 Tables Table 1: Life Cycle cost Estimates 20 Table 2: Other Types of cost Estimates 20 Table 3: The Twelve Steps and their Best Practices 36 Table 4: Typical Technical Baseline Elements 49 Table 5: General System Characteristics 52 Table 6: Common Elements in Work Breakdown Structures 61 Table 7: Basic Primary and Secondary Data Sources 86 Table 8: Three cost ESTIMATING Methods Compared 103 Table 9: An Example of the Analogy cost ESTIMATING Method 104 Table 10: An Example of the Engineering Build-Up cost ESTIMATING Method 108 Table 11: An Example of the Parametric cost ESTIMATING Method 111 Table 12.

7 Ranges of cost by WBS 146 Table 13: Air Vehicle Risks, Likelihood, and cost Effects 149 Table 14: Air Vehicle Uncertainty and cost Effects 150 Table 15: cost Estimate Documentation Elements 171 Table 16: Select cost ESTIMATING and EVM Criteria for Federal Agencies: Laws and Regulations 200 Table 17: The Twelve Steps and their Associated Tasks 203 Table 18: Key Benefits of Implementing EVM 214 Table 19: Eleven Common Concerns about EVM 217 Table 20: EVM Implementation Guides 221 Table 21: Typical Methods for Measuring Earned Value Performance 231 Table 22: Contract Performance Report Data Elements: Format 1 245 Table 23: EVM Performance Indexes 257 Page vii GAO-20-195G cost ESTIMATING and ASSESSMENT GUIDE Table 24: Best Predictive Estimate at Completion (EAC) Efficiency Factors by Program Completion Status 265 Table 25: Basic Program Management Questions That Earned Value Management (EVM) Data Help Answer 271 Table 26: Sample Factors in Selecting Projects 279 Table 27: Common Indicators of Poor Program Performance 288 Table 28: Overtarget Budget Funding Implications by Contract Type 289 Table 29: Options for Treating Variances in Performing a Single Point Adjustment 292 Table 30: EVM Best Practices 299 Table 31: Case Studies Drawn from GAO Reports Illustrating This GUIDE 306 Table 32.

8 Common Data Collection Items 344 Table 33: Supplementary Common Data Collection Items 345 Table 34: Aircraft System Work Breakdown Structure 360 Table 35: Missile/Ordnance Systems Work Breakdown Structure 363 Table 36: Sea System Work Breakdown Structure 365 Table 37: Space System Work Breakdown Structure 367 Table 38: Ground Vehicle System Work Breakdown Structure 368 Table 39: Information Systems/Defense Business Systems (Investment) Work Breakdown Structure 370 Table 40: Technology Development Project Work Breakdown Structure 372 Table 41: Particle Accelerator System Work Breakdown Structure 372 Table 42: Environmental Management Work Breakdown Structure 373 Table 43: Pharmaceutical Management Work Breakdown Structure 374 Table 44: Process Plant Construction Work Breakdown Structure 376 Table 45: Telecom Work Breakdown Structure 376 Table 46: The Anderlohr Method for the Learning Lost Factor 382 Table 47: Integrated Baseline Review Risk Categories 391 Table 48: Common Probability Distributions 397 Table 49: The Four Characteristics of the AOA Process and Their Corresponding Best Practices 414 Table 50: cost ESTIMATING Steps Mapped to Selected Components and Principles of Internal Controls 425 Figures Figure 1: cost ESTIMATING Challenges and Mitigations 9 Figure 2.

9 A Representative Life Cycle cost Estimate for a System 19 Page viii GAO-20-195G cost ESTIMATING and ASSESSMENT GUIDE Figure 3: Cone of Uncertainty 24 Figure 4: An Affordability ASSESSMENT 26 Figure 5: The cost ESTIMATING Process 34 Figure 6: Disciplines and Concepts in cost Analysis 42 Figure 7: A Product-Oriented Work Breakdown Structure 57 Figure 8: A Work Breakdown Structure with Common Elements 62 Figure 9: A Contract Work Breakdown Structure 63 Figure 10: A Learning Curve 121 Figure 11: A Sensitivity Analysis that Creates a Range around a Point Estimate 132 Figure 12: Tornado Chart for a Sensitivity Analysis 136 Figure 13: Notional Changes in cost Ranges across the Life Cycle 143 Figure 14: A Cumulative Distribution Function, or S Curve 145 Figure 15: Air Vehicle cost Cumulative Probability Distribution from a Three-Point Estimate 148 Figure 16: Air Vehicle cost Cumulative Probability Distribution from a Risk Driver Simulation 151 Figure 17: Cumulative Probability Distributions with and without Correlation 157 Figure 18: ESTIMATING Steps and Best Practices Related to a Comprehensive Estimate 185 Figure 19: ESTIMATING Steps and Best Practices Related to a Well-Documented Estimate 188 Figure 20: ESTIMATING Steps and Best Practices Related to an Accurate Estimate 191 Figure 21: Est