monitoring and evaluation (M&E) guide

1 Lives, changing monitoring and evaluation (M&E) guideAcknowledgementsThis guide was developed by the Planning and evaluation Department (PED) of the IFRC Secretariat. It would not have been possible without the invaluable review and feedback from National Societies. In particular, we want to express our thanks to the British Red Cross, the Danish Red Cross, the Norwegian Red Cross, the Swedish Red Cross, the Finnish Red Cross, the American Red Cross, the Australian Red Cross, and the Canadian Red Cross. Also, special thanks to Julie Smith for her creative cartoons and M&E sense of humour. International Federation of Red Cross and Red Crescent Societies, Geneva, 2011 Copies of all or part of this guide may be made for noncommercial use, providing the source is acknowledged The IFRC would appreciate receiving details of its use.

2 Requests for commercial reproduction should be directed to the IFRC at designations and maps used do not imply the expression of any opinion on the part of the International Federation or National Societies concerning the legal status of a territory or of its authorities. All photos used in this guide are copyright of the IFRC unless otherwise indicated. Cover photo, from left to right, clockwise: Benoit Matsha-Carpentier/IFRC, Arzu Ozsoy/IFRC, Alex Box 372CH-1211 Geneva 19 SwitzerlandTelephone: +41 22 730 4222 Telefax: +41 22 733 0395E-mail: site: monitoring and evaluation (M&E) guide1000400 E 3,000 08/2011 Strategy 2020 voices the collective determination of the International Federation of Red Cross and Red Crescent Societies (IFRC) to move forward in tackling the major challenges that confront humanity in the next decade.

3 Informed by the needs and vulnerabilities of the diverse communities with whom we work, as well as the basic rights and freedoms to which all are entitled, this strategy seeks to benefit all who look to Red Cross Red Crescent to help to build a more humane, dignified and peaceful the next ten years, the collective focus of the IFRC will be on achieving the following strategic aims:1. Save lives, protect livelihoods, and strengthen recovery from disasters and crises 2. Enable healthy and safe living 3. Promote social inclusion and a culture of non-violence and peace1 Table of ContentsAcknowledgements inside coverAbbreviations and Acronyms 4 Introduction 5 PART 1: M&E concepts and considerations 9 Results-based management (RBM) 9 M&E and the project/programme cycle 10 What is monitoring ?

4 11 What is evaluation ? 13 Baseline and endline studies 17 Comparing monitoring , evaluation , reviews and audits 19 M&E standards and ethics 20 Attention to gender and vulnerable groups 21 Minimize bias and error 22 PART 2: Six key steps for project/programme M&E 25 STEP 1 Identify the purpose and scope of the M&E system 27 Review the project/programme s operational design (logframe) 27 Identify key stakeholder informational needs and expectations 29 Identify any M&E requirements 30 Scope of major M&E events and functions 30 STEP 2 Plan for data collection and management 32 Develop an M&E plan table 32 Assess the availability of secondary data 33 Determine the balance of quantitative and qualitative data 35 Triangulate data collection sources and methods 36 Determine sampling requirements 36 Prepare for any surveys 38 Prepare specific data collection methods/tools 38 Establish stakeholder complaints and feedback mechanisms 40 Establish project/programme staff/volunteers review mechanisms 42 Plan for data management 43 Use an indicator tracking table (ITT) 45 Use a risk log (table)

5 47 STEP 3 Plan for data analysis 48 Develop a data analysis plan 49 Follow the key data analysis stages 50 STEP 4 Plan for information reporting and utilization 57 Anticipate and plan for reporting 58 Plan for information utilization 66 STEP 5 Plan for M&E human resources and capacity building 692 International Federation of Red Cross and Red Crescent SocietiesProject/programme monitoring and evaluation guide Assess the project/programme s human resources capacity for M&E 69 Determine the extent of local participation 69 Determine the extent of outside expertise 72 Define the roles and responsibilities for M&E 72 Plan to manage project/programme team s M&E activities 73 Identify M&E capacity-building requirements and opportunities 73 STEP 6 Prepare the M&E budget 74 Itemize M&E budget needs 74 Incorporate M&E costs in the project/programme budget 74 Review any donor budget requirements and contributions 75 Plan for cost contingency 75 ANNEXES 77 Annex 1: Glossary of key terms for M&E 77 Annex 2: M&E resources 83 Annex 3: Factors affecting the quality of M&E information 88 Annex 4: Checklist for the six key M&E steps 90 Annex 5: IFRC s logframe definition of terms 92 Annex 6: Example M&E stakeholder assessment table 93 Annex 7: Example M&E activity planning table 95 Annex 8.

6 M&E plan table template and instructions 96 M&E plan example 97 M&E plan purpose and compliance 98 M&E plan instructions 98 Annex 9: Closed-ended questions examples 100 Annex 10: Key data collection methods and tools 101 Annex 11: Project/programme feedback form template 103 Annex 12: Complaints log 104 Annex 13: Staff/volunteer performance management template 105 Annex 14: Individual time resourcing sheet 106 Annex 15: Project/programme team time resourcing sheet 107 Annex 16: Indicator tracking table (ITT) examples and instructions 108 Annex 17: Example risk log 113 Annex 18: Reporting schedule 114 Annex 19: IFRC s project/programme management report template and instructions 115 Annex 20: Example tables (logs) for action planning and management response 122 Annex 21: Example M&E job description 123 Annex 22: M&E training schedule 127 List of tables, boxes and diagrams Table 1: Common types of monitoring 12 Table 2: Summary of major evaluation types 15 Table 3: The IFRC s framework for evaluation criteria and standards 17 Table 4: Comparing key features of monitoring /review, evaluation and audit 20 Table 5: Example of indicator tracking table for one quarter only 46 Table 6: Comparing data analysis terms: findings, conclusions, recommendations and actions 563 International Federation of Red Cross and Red Crescent SocietiesProject/programme monitoring and evaluation guide Box 1: Principle Nine of the Conduct for International Red Cross and Red Crescent Movement and NGOs in Disaster Relief 6 Box 2: monitoring best practices 13 Box 3.

7 The challenge of measuring impact 18 Box 4: Principle Five of the Code of Conduct for International Red Cross and Red Crescent Movement and NGOs in Disaster Relief 21 Box 5: M&E in emergency settings 27 Box 6: Types of industry (standard) indicators 28 Box 7: Examples of IFRC s key stakeholders and informational needs 29 Box 8: Specific evaluation requirements for the IFRC s secretariat-funded projects/programmes 30 Box 9: Examples of key M&E activities 31 Box 10: Is an M&E plan worth all the time and effort? 33 Box 11: Comparing quantitative versus qualitative data 35 Box 12: Minimizing data collection costs 40 Box 13: The IFRC s guide for stakeholder feedback 42 Box 14: Formats can reinforce critical analysis and use 44 Box 15: The importance of target setting 47 Box 16: Benefits of involving multiple stakeholders in data analysis 50 Box 17: Data analysis questions to help describe the data 52 Box 18: Using traffic lights to highlight data 55 Box 19: Criteria of good reporting 58 Box 20: Internal versus external reporting 60 Box 21: Example reporting formats 62 Box 22: Report writing tips 63 Box 23: IFRC s project/programme management report outline (refer to Annex 19 for full template) 64 Box 24: Reporting roadblocks and solutions 65 Box 25: Key categories of information use 66 Box 26.

8 Key mediums of information dissemination 66 Box 27: Principle Seven of the Conduct for International Red Cross and Red Crescent Movement and NGOs in Disaster Relief 70 Box 28: Considering participatory M&E 71 Box 29: Adhering to human resources codes and standards People in Aid 73 Box 30: How much money should be allocated for M&E? 75 Diagram 1: Key M&E activities in the project/programme cycle 10 Diagram 2: monitoring questions and the logframe 11 Diagram 3: evaluation questions and the logframe 14 Diagram 4: An example of information flows in project/programme reporting 61 Diagram 5: The participatory continuum 704 International Federation of Red Cross and Red Crescent SocietiesProject/programme monitoring and evaluation guideAbbreviations and Acronyms DAC Development Assistance Committee FWRS Federation-Wide Reporting System HNS Host National Society HR human resources ICRC International Committee of the Red Cross IFRC International Federation of Red Cross and Red Crescent Societies IT information technology ITT indicator tracking table M&E monitoring and evaluation MoU Memorandum of Understanding NGO non-governmental organization OECD Organization for Economic Co-operation Development ONS Operational National Society PED planning and evaluation department PMER planning, monitoring .

9 evaluation and reporting PNS Participating National Society RBM results-based management RTE real-time evaluation SMART specific, measurable, achievable, relevant, time-bound SWOT strengths, weaknesses, opportunities and threats ToR terms of reference VCA vulnerability and capacity assessment 5 International Federation of Red Cross and Red Crescent SocietiesProject/programme monitoring and evaluation guideWhat is this guide ?The purpose of this guide is to promote a common understanding and reliable practice of monitoring and evaluation (M&E) for IFRC project/programmes. It is meant to be a desktop reference that supplements the more concise and field-friendly IFRC PMER Pocket guide . Therefore, this guide is not intended to be read from cover to cover; the reader can refer to specific topics for more detail when guide does not provide detailed guidance on conducting evaluations; this is pro-vided in separate IFRC Instead, emphasis is placed on establishing and implementing a project/programme monitoring and related reporting system.

10 However, as evaluation is integrally linked to monitoring , an overview of evaluation is included for planning evaluation events within the overall M&E is the intended audience?This guide is intended for people managing projects/programmes in National Red Cross and Red Crescent Societies and the secretariat. However, it has been de-signed to be understood by multiple other users as well, including IFRC staff and volunteers, donors and partners. Although it has been designed for use at the country level, the basic principles can be applied to projects/programmes at other levels. Introduction1 A guide for managing evaluations will be available from the IFRC s planning and education department (PED).6 International Federation of Red Cross and Red Crescent SocietiesProject/programme monitoring and evaluation guideWhy is M&E important?