Ten Steps to a Results- Based Monitoring - OECD

1 THE WORLD BANKJody Zall Kusek Ray C. RistTen StepsTen Stepsto aResults-BasedMonitoringandEvaluationSys temto aResults-BasedMonitoringandEvaluationSys temA Handbook for Development PractitionersTen Stepsto aResults- Based Monitoring andEvaluationSystemA Handbook for Development PractitionersJody Zall Kusek Ray C. RistTHE WORLD BANKW ashington, Handbook for Development PractitionersTen Steps to aResults- Based MonitoringandEvaluationSystemTen Stepsto aResults- Based Monitoring andEvaluationSystem 2004 The International Bank for Reconstruction and Development / The World Bank1818 H Street, NWWashington, DC 20433 Telephone 202-473-1000 Internet rights findings, interpretations, and conclusions expressed herein are those ofthe author(s)

2 And do not necessarily reflect the views of the Board ofExecutive Directors of the World Bank or the governments they World Bank does not guarantee the accuracy of the data included inthis work. The boundaries, colors, denominations, and other informationshown on any map in this work do not imply any judgment on the part ofthe World Bank concerning the legal status of any territory or the endorse-ment or acceptance of such and PermissionsThe material in this work is copyrighted. Copying and/or transmitting portions or all of this work without permission may be a violation of applicable law.

3 The World Bank encourages dissemination of its work andwill normally grant permission permission to photocopy or reprint any part of this work, please send arequest with complete information to the Copyright Clearance Center, Inc.,222 Rosewood Drive, Danvers, MA 01923, USA, telephone 978-750-8400,fax 978-750-4470, other queries on rights and licenses, including subsidiary rights, should be addressed to the Office of the Publisher, World Bank, 1818 HStreet NW, Washington, DC 20433, USA, fax 202-522-2422, e-mail of Congress Cataloging-in-Publication DataKusek, Jody Zall, 1952 Ten Steps to a results- Based Monitoring and evaluation system : a hand-book for development practitioners / Jody Zall Kusek and Ray C.

4 Bibliographical references and 0-8213-5823-51. Government productivity developing countries Performance standards developing countries Evaluation. 3. Totalquality management in government developing countries Public administration developing countries Evaluation. I. Rist, RayC. II. 5 dc222004045527 Preface xiAbout the Authors xivIntroductionBuilding a Results- Based Monitoring and Evaluation System 1 Part I New Challenges in Public Sector Management 2 International and External Initiatives and Forces for Change 3 National Poverty Reduction Strategy Approach 8 Internal Initiatives and Forces for Change 10 Part 2 Results- BasedM&E APowerfulPublicManagementTool 11 Monitoring and Evaluation: What Is It All About?

5 12 Key Features of Traditional Implementation-Focused and Results- Based M&E Systems 15 Many Applications for Results- Based M&E 17 Political and Technical Challenges to Building a Results- Based M&E System 20 Introducing the 10-Step Model for Building a Results- Based M&E System 23 Where to Begin: Whole-of-Government, Enclave, or Mixed Approach 24 Part 3 M&E Experience in Developed and developing Countries 27M&E Experience in Developed and OECD Countries 27 Special M&E Challenges Facing developing Countries 32M&E Experience in developing Countries 35 Chapter 1 Step 1: Conducting a Readiness Assessment 39 Part 1 Why Do a Readiness Assessment?

6 40 Part 2 The Readiness Assessment: Eight Key Questions 43 Part 3 Readiness Assessments in developing Countries: Bangladesh, Egypt, and Romania 48 ContentsPart 4 Lessons Learned 49 Chapter 2 Step 2: Agreeing on Outcomes to Monitor and Evaluate 56 The Importance of Outcomes 56 Issues to Consider in Choosing Outcomes to Monitor and Evaluate 57 The Importance of Building a Participatory and Consultative Process involving Main Stakeholders 58 The Overall Process of Setting and Agreeing upon Outcomes 59 Examples and Possible Approaches 61 Chapter 3 Step 3.

7 Selecting Key Performance Indicators to Monitor Outcomes 65 Indicators Are Required for All Levels of Results- Based M&E Systems 66 Translating Outcomes into Outcome Indicators 66 The CREAM of Good Performance Indicators 68 The Use of Proxy Indicators 70 The Pros and Cons of Using Predesigned Indicators 72 Constructing Indicators 74 Setting Indicators: Experience in developing Countries 75 Chapter 4 Step 4: Setting Baselines and Gathering Data on Indicators 80 Establishing Baseline Data on Indicators 81 Building Baseline Information 82 Identifying Data Sources for Indicators 83 Designing and Comparing Data Collection Methods 84 The Importance of Conducting Pilots 86 Data Collection: Two developing Country Experiences 89 Chapter 5 Step 5.

8 Planning for Improvement Selecting Results Targets 90 Definition of Targets 90 Factors to Consider When Selecting Performance indicator Targets 91 Examples of Targets Related to Development Issues 93 The Overall Performance- Based Framework 94 Chapter 6 Step 6: Monitoring for Results 96 Part 1 Key Types and Levels of Monitoring 98 Links between Implementation Monitoring and Results Monitoring 101 Part 2 Key Principles in Building a Monitoring System 103 Achieving Results through Partnership 105 Needs of Every Results- Based Monitoring System 106 The Data Quality Triangle: Reliability, Validity, and Timeliness 108viContentsAnalyzing Performance Data 111 Pretesting Data Collection Instruments and Procedures 112 Chapter 7 Step 7.

9 The "E" in M&E Using Evaluation Information toSupport a Results- Based Management System 113 Uses of Evaluation 115 The Timing of Evaluations 118 Types of Evaluations 121 Characteristics of Quality Evaluations 126 Examples of Evaluation at the Policy, Program, and Project Levels 128 Chapter 8 Step 8: Reporting the Findings 129 The Uses of Monitoring and Evaluation Findings 130 Know and Target the Audience 130 Presentation of Performance Data in Clear and Understandable Form 132 What Happens If the M&E System Produces Bad Performance News?

10 136 Chapter 9 Step 9: Using the Findings 138 Uses of Performance Findings 138 Additional Benefits of Using Findings: Feedback, Knowledge, and Learning 140 Strategies for Sharing Information 146 Chapter 10 Step 10: Sustaining the M&E System within the Organization 151 Six Critical Components of Sustaining Results- Based M&E Systems 152 The Importance of Incentives and Disincentives in Sustaining M&E Systems 155 Possible Problems in Sustaining Results- Based M&E Systems 155 Validating and Evaluating M&E Systems and Information 160M&E: Stimulating Positive Cultural Change in Governments and Organizations 160 Last Reminders 160 Chapter 11 Making Results- Based M&E Work for You and Your Organization 162 Why Results- Based M&E?