Assessing, Monitoring, and Evaluating Army Security ...

1 Angela O Mahony, Ilana Blum, Gabriela Armenta, Nicholas Burger, Joshua Mendelsohn, Michael J. McNerney, Steven W. Popper, Jefferson P. Marquis, Thomas S. SzaynaAssessing, Monitoring, and Evaluating Army Security CooperationA framework for ImplementationCORPORATIONL imited Print and Electronic Distribution RightsThis document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited. Permission is given to duplicate this document for personal use only, as long as it is unaltered and complete. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial use. For information on reprint and linking permissions, please visit RAND Corporation is a research organization that develops solutions to public policy challenges to help make communities throughout the world safer and more secure, healthier and more prosperous.

2 RAND is nonprofit, nonpartisan, and committed to the public interest. RAND s publications do not necessarily reflect the opinions of its research clients and RANDMake a tax-deductible charitable contribution at more information on this publication, visit of Congress Cataloging-in-Publication Data is available for this BN: 978-0-8330-9941-9 Published by the RAND Corporation, Santa Monica, Calif. Copyright 2018 RAND CorporationR is a registered cooperation is an important instrument of foreign policy, and is employed extensively to accomplish a diverse set of objectives, such as building relationships that pro-mote Security interests, developing partners capabilities for self-defense and multina-tional operations, and securing access for forces. However, Evaluating the effectiveness of Security cooperation to meet these objectives has proven help the Army increase the effectiveness of its Security cooperation activities and to more effectively evaluate its future Security cooperation activities, the Army Deputy Chief of Staff, G-3/5/7, asked the Arroyo Center to address two questions.

3 First, when can Army Security cooperation have the greatest impact? Second, how should the Army assess, monitor, and evaluate Security cooperation? Taken together, the answers to these two questions can help the Army prioritize and evaluate future Security cooperation I of this report addresses the first question. It focuses on understanding the factors that are associated with Security cooperation effectiveness and examines the extent to which recent Army Security cooperation activities reflect those factors. Part II addresses the second question. It presents a framework to help the Army implement an assessment, monitoring, and evaluation (AM&E) process in line with good practice and emerging Department of Defense guidance. The framework can enable accountability for planners, implementers, and stakeholders and institutionalize a learning process within the study results will assist the Army to better plan and execute Security cooperation activities.

4 The primary users of this framework will be Army Service Component Command planners. The Army s focus on deepening assessment, monitoring, and evaluation procedures at the Security cooperation activity level can serve as a model for the Department of Defense (DoD) to link its strategic guidelines to the operational planning level. The Army s bottom-up operational approach can be a valuable complement for the Office of the Secretary of Defense s top-down strategic approach. Taken together, these approaches form the basis for a compre-hensive approach to Security cooperation, encompassing strategic, operational, and tactical report documents research and analysis conducted as part of a project titled Devel-oping a framework to Assess Expected Impact of Army Security Cooperation, sponsored by the United States Army Deputy Chief of Staff, G-3/5 Project Unique Identification Code for the project that produced this document is R A research was conducted within the R AND Arroyo Center s Strategy, Doctrine, and Resources Program.

5 R AND Arroyo Center, part of the R AND Corporation, is a federally funded research and development center (FFRDC) sponsored by the United States AND operates under a Federal-Wide Assurance (FWA00003425) and complies with the Code of Federal Regulations for the Protection of Human Subjects Under United States Law (45 CFR 46), also known as the Common Rule, as well as with the implementa-tion guidance set forth in DoD Instruction As applicable, this compliance includes reviews and approvals by R AND s Institutional Review Board (the Human Subjects Protec-tion Committee) and by the Army. The views of sources utilized in this study are solely their own and do not represent the official policy or position of DoD or the AND s publications do not necessarily reflect the opinions of its research clients and.

6 IiiFigures ..viiTables ..ixSummary ..xiAcknowledgments ..xxvAbbreviations ..xxviiPA RT ON EUnderstanding Army Security Cooperation Effectiveness ..1 CHAPTER ONEI ntroduction ..3 Partner Engagement Is an Important Foreign Policy Tool ..3 Evaluating Security Cooperation Is Challenging but Necessary ..4 Objectives, Approach, and T WOLessons from Security Cooperation and International Development Assistance ..9 Why Compare Security Cooperation and Development? ..9 Considerations Affecting International Assistance ..11 Approach ..13 Strategic Considerations (Political Importance) ..14 Strategic Considerations (Economic Importance)..15 Political Considerations ..17 Absorptive Capacity Considerations ..20 Cultural Considerations ..21 Financial Assistance Considerations ..22 Conclusion ..24 CHAPTER THREEArmy Security Cooperation Empirics ..27 Approach.

7 28 Overview ..31 Strategic Considerations ..33 Political Considerations ..36 Absorptive Capacity Considerations ..38vi assessing , Monitoring, and Evaluating Army Security CooperationCultural Considerations ..40 Financial Aid Considerations ..41 Conclusion ..42PA RT T WOImplementing an Army AM&E framework ..43 CHAPTER FOURAM&E Developments and Implications for the Army ..45AM&E Developments and Requirements Are Accelerating ..45 The Army s Independent Approach to Security Cooperation AM&E ..49 Applying Army AM&E Policy and Doctrine to Security Cooperation Processes ..52 Potential Improvements in Army AM&E ..54 CHAPTER FIVEGood Practices in Development AM&E ..57AM&E Trajectory in International Development ..58 The Use of AM&E in Practice ..60 Conclusion ..76 CHAPTER SIXF ramework for assessing , Monitoring, and Evaluating Army Security Cooperation.

8 77 ASCC Planning for Security Cooperation ..77 ASCC Initial Country Assessments ..79 TOC for Army Security Cooperation Activities ..80 Army Security Cooperation M&E ..87 Conclusion ..97 CHAPTER SEVENP ilot Tool for Evaluating Security Cooperation Portfolios ..99 Security Cooperation Portfolio Tool: Inputs and Outputs ..100 Sample Portfolio Analysis for Bandaria ..104 Conclusion ..113 CHAPTER EIGHTC onclusion ..115 APPENDIXESA. What Is Security Cooperation? ..117B. Regression Model Methodology ..125C. Dimensions of Partner Capabilities Assessment ..129D. Examples of Three Notional Activity-Level TOCs ..133E. Activity-Level AM&E Decision Tree and Supporting Annexes ..145F. Works Included in Literature Review ..155G. Further Reading Suggestions on AM&E and Impact Evaluation Practices ..161 References ..163viiFigures Potential Army AM&E Improvements.

9 Xvii Army Security Cooperation AM&E framework Components ..xviii ASCC Country Assessment Dimensions ..xix Causal Logic of a TOC ..xx Activity-Level M&E Decision Tree ..xxi Exemplar Activity-Level M&E Decision Tree ..xxii First Comparison of Security Cooperation Portfolios ..xxiv Interquartile Range of Activity Attendance Rates by GCC ..32 Average Portfolio Share Percentage for Each Activity Type in Each GCC ..33 New Developments and Requirements Affecting AM&E ..46 Comparing Army Operations and DoD AM&E Constructs ..50 Potential Army AM&E Improvements ..54 AM&E and Prioritization ..55 Pyramid for AM&E ..62 AM&E Cycle ..63 ASCC Security Cooperation Planning Process ..78 Causal Logic of an Activity-Level TOC ..83 Components Involved in a TOC ..87 Activity-Level M&E Decision Tree ..93 Exemplar Activity-Level M&E Decision Tree.

10 94 First Comparison of Security Cooperation Portfolios ..106 Comparison of Costs and Presumed Outcomes of Two Alternative Portfolios ..107 Second Comparison of Security Cooperation Portfolios ..108 Peeling Trajectory of Alternative Specifications ..110 Third Comparison of Security Cooperation Six Categories of Security Cooperation Activities ..121 Notional TOC for MEDCAP to Djibouti Village ..135 Notional TOC for Colombian Participants in MCCC ..139 Notional TOC for Southern Accord 2016 (Malawi) ..142 AM&E Decision Tree for Three Example Activities ..14 6ixTables Factors Associated with Assistance Provision and Variation in Effectiveness of Security Cooperation Across Key Characteristics ..xvi Factors Associated with Assistance Provision and Predicted Activity Attendance, as Strategic Importance Varies ..35 Predicted Difference in Portfolio Share, Given Strategic Importance.