TEST AND EVALUATION MANAGEMENT GUIDE - …

1 TEST AND EVALUATION . MANAGEMENT GUIDE . JANUARY 2005. FIFTH EDITION. PUBLISHED BY. THE DEFENSE ACQUISITION UNIVERSITY PRESS. FORT BELVOIR, VA 22060-55654. i ii ACKNOWLEDGMENTS. This January 2005 update to the Defense Acquisition University's Test and EVALUATION MANAGEMENT GUIDE includes updates from the Military Services, Defense Agencies, and other organizations, as well as changes in the May 2003 DoD 5000 series. Inputs were collated and finalized by DAU. Program Director for Test and EVALUATION Dr. John Claxton. Freelance writer Christina Cavoli and Defense AT&L editor-in-chief Collie Johnson edited the final document. Freelance designer and cartoonist Jim Elmore designed the cover, and final desktop publishing and layout was accomplished by Bartlett Communications and DAU Visual Information Specialist Deborah Gonzalez.

2 Iii iv FOREWORD. This book is one of many technical MANAGEMENT educational guides written from a Department of Defense perspective; , non-Service specific. They are intended primarily for use in courses at Defense Acquisition University and secondarily as a desk reference for program and project man- agement personnel. These guidebooks are written for current and potential acquisition MANAGEMENT personnel who are familiar with basic terms and definitions employed in program offices. The GUIDE - books are designed to assist government and industry personnel in executing their MANAGEMENT re- sponsibilities relative to the acquisition and support of defense systems.

3 The objective of a well-managed test and EVALUATION program is to provide timely and accurate in- formation. This GUIDE has been developed to assist the acquisition community in obtaining a better understanding of whom the decision-makers are and determining how and when to plan test and EVALUATION events. Dr. John D. Claxton Program Director, T&E. Defense Acquisition University 4. v vi CONTENTS. Page Acknowledgments .. iii Foreword .. v MODULE I MANAGEMENT of Test and EVALUATION Chapter 1 Importance of Test and EVALUATION Introduction .. 1-1. Testing as a Risk MANAGEMENT Tool .. 1-1. The T&E Contribution at Major Milestones .. 1-4. 1-8. Chapter 2 The Test and EVALUATION Process Introduction.

4 2-1. Defense System Acquisition Process .. 2-1. T&E and the Systems Engineering Process (SEP) .. 2-5. Definitions .. 2-9. The DoD T&E Process .. 2-11. 2-12. Chapter 3 T&E Policy Structure and Oversight Mechanism Introduction .. 3-1. The Congress .. 3-1. OSD Oversight Structure .. 3-2. Service T&E MANAGEMENT Structures .. 3-5. The T&E Executive Agent Structure .. 3-11. 3-12. Chapter 4 Program Office Responsibilities for Test and EVALUATION Introduction .. 4-1. Relationship to the Program Manager .. 4-1. Early Program Stages .. 4-1. PMO/Contractor Test MANAGEMENT .. 4-4. vii Integrated Product Teams for Test and EVALUATION .. 4-4. Test Program Funding/Budgeting.

5 4-4. Technical Reviews, Design Reviews, and Audits .. 4-5. Contractor Testing .. 4-5. Specifications .. 4-5. Independent Test and EVALUATION 4-6. PMO Responsibilities for Operational Test and EVALUATION .. 4-6. 4-10. Chapter 5 Test-Related Documentation Introduction .. 5-1. Requirements Documentation .. 5-1. Program Decision 5-4. Program MANAGEMENT Documentation .. 5-4. Test Program Documentation .. 5-6. Other Test-Related Status Reports .. 5-9. Summary .. 5-10. Chapter 6 Types of Test and EVALUATION Introduction .. 6-1. Development Test and EVALUATION .. 6-1. Operational Test and EVALUATION .. 6-3. Multi-Service Test and EVALUATION .. 6-6. Joint Test and EVALUATION .

6 6-6. Live Fire Testing .. 6-7. Nuclear, Biological, and Chemical (NBC) Weapons Testing .. 6-8. Nuclear Hardness and Survivability (NHS) Testing .. 6-9. Summary .. 6-9. MODULE II Development Test and EVALUATION Chapter 7 Introduction to Development Test and EVALUATION Introduction .. 7-1. DT&E and the System Acquisition 7-1. DT&E Responsibilities .. 7-4. Test Program Integration .. 7-5. Areas of DT&E Focus .. 7-6. System Design for Testing .. 7-7. viii Impact of Warranties on T&E .. 7-8. DT&E of Limited Procurement Quantity Programs .. 7-9. 7-9. Chapter 8 DT&E Support of Technical Reviews and Milestone Decisions Introduction .. 8-1. DT&E and the Review Process.

7 8-1. Configuration Change Control .. 8-6. 8-6. Chapter 9 Combined and Concurrent Testing Introduction .. 9-1. Combining Development Test and Operational Test .. 9-1. Concurrent Testing .. 9-3. Advantages and Limitations .. 9-4. 9-4. Chapter 10 Production-Related Testing Activities Introduction .. 10-1. Production MANAGEMENT .. 10-1. Production Readiness Review (PRR) .. 10-2. Qualification Testing .. 10-2. Transition to Production .. 10-2. Low Rate Initial Production (LRIP) .. 10-4. Production Acceptance Test and EVALUATION (PAT&E) .. 10-4. 10-5. MODULE III Operational Test and EVALUATION Chapter 11 Introduction to Operational Test and EVALUATION Introduction.

8 11-1. Purpose of OT&E .. 11-1. Test Participants .. 11-1. Types of OT&E .. 11-2. Test Planning .. 11-4. Test Execution .. 11-5. Test Reporting .. 11-5. 11-7. ix Chapter 12 OT&E to Support Decision Reviews Introduction .. 12-1. Concept Refinement (CR) and Technology Development (TD) .. 12-1. System Development and Demonstration (SDD) .. 12-2. Production and Deployment (P&D) .. 12-4. Operations and Support (O&S) .. 12-4. 12-5. MODULE IV Test and EVALUATION Planning Chapter 13 EVALUATION Introduction .. 13-1. Difference Between Test and EVALUATION .. 13-1. The EVALUATION 13-1. Issues and Criteria .. 13-3. Measures of Effectiveness (MOEs) .. 13-5. EVALUATION Planning.

9 13-5. Evaluating Development and Operational tests .. 13-6. 13-8. Chapter 14 Modeling and Simulation Support to T&E. Introduction .. 14-1. Types of Models and Simulations .. 14-1. Validity of Modeling and Simulation .. 14-3. Support to Test Design and Planning .. 14-4. Support to Test Execution .. 14-7. Support to Analysis and Test Reporting .. 14-9. Simulation 14-9. Simulation Planning .. 14-9. 14-10. Chapter 15 Test Resources Introduction .. 15-1. Obtaining Test Resources .. 15-1. Test Resource 15-4. Test Resource Funding .. 15-8. 15-10. x Chapter 16 Test and EVALUATION Master Plan Introduction .. 16-1. TEMP Development .. 16-1. TEMP 16-5. 16-7. MODULE V Specialized Testing Chapter 17 Software Systems Testing Introduction.

10 17-1. Definitions .. 17-2. Purpose of Software Test and EVALUATION .. 17-2. Software Development Process .. 17-3. T&E in the Software Life Cycle .. 17-4. 17-8. Chapter 18 Testing for Vulnerability and Lethality Introduction .. 18-1. Live Fire Testing .. 18-1. Testing for Nuclear Hardness and Survivability .. 18-5. 18-7. Chapter 19 Logistics Infrastructure T&E. Introduction .. 19-1. Planning for Logistics Support System T&E .. 19-1. Conducting Logistics Support System T& 19-4. Limitations to Logistics Support System T&E .. 19-8. 19-8. Chapter 20 EC/C4 ISR Test and EVALUATION Introduction .. 20-1. Testing EC Systems .. 20-1. Testing of C4 ISR Systems.