monitoring and evaluation (M&E) guide - IFRC

1 Lives, changing monitoring and evaluation (M&E) guideAcknowledgementsThis guide was developed by the Planning and evaluation Department (PED) of the IFRC Secretariat. It would not have been possible without the invaluable review and feedback from National Societies. In particular, we want to express our thanks to the British Red Cross, the Danish Red Cross, the Norwegian Red Cross, the Swedish Red Cross, the Finnish Red Cross, the American Red Cross, the Australian Red Cross, and the Canadian Red Cross. Also, special thanks to Julie Smith for her creative cartoons and M&E sense of humour.

2 International Federation of Red Cross and Red Crescent Societies, Geneva, 2011 Copies of all or part of this guide may be made for noncommercial use, providing the source is acknowledged The IFRC would appreciate receiving details of its use. Requests for commercial reproduction should be directed to the IFRC at designations and maps used do not imply the expression of any opinion on the part of the International Federation or National Societies concerning the legal status of a territory or of its authorities. All photos used in this guide are copyright of the IFRC unless otherwise indicated.

3 Cover photo, from left to right, clockwise: Benoit Matsha-Carpentier/IFRC, Arzu Ozsoy/IFRC, Alex Box 372CH-1211 Geneva 19 SwitzerlandTelephone: +41 22 730 4222 Telefax: +41 22 733 0395E-mail: site: monitoring and evaluation (M&E) guide1000400 E 3,000 08/2011 Strategy 2020 voices the collective determination of the International Federation of Red Cross and Red Crescent Societies (IFRC) to move forward in tackling the major challenges that confront humanity in the next decade. Informed by the needs and vulnerabilities of the diverse communities with whom we work, as well as the basic rights and freedoms to which all are entitled, this strategy seeks to benefit all who look to Red Cross Red Crescent to help to build a more humane, dignified and peaceful the next ten years, the collective focus of the IFRC will be on achieving the following strategic aims:1.

4 Save lives, protect livelihoods, and strengthen recovery from disasters and crises 2. Enable healthy and safe living 3. Promote social inclusion and a culture of non-violence and peace1 Table of ContentsAcknowledgements inside coverAbbreviations and Acronyms 4 Introduction 5 PART 1: M&E concepts and considerations 9 Results-based management (RBM) 9 M&E and the project/programme cycle 10 What is monitoring ?

5 11 What is evaluation ? 13 Baseline and endline studies 17 Comparing monitoring , evaluation , reviews and audits 19 M&E standards and ethics 20 Attention to gender and vulnerable groups 21 Minimize bias and error 22 PART 2: Six key steps for project/programme M&E 25 STEP 1 Identify the purpose and scope of the M&E system 27 Review the project/programme s operational design (logframe) 27 Identify key stakeholder informational needs and expectations 29 Identify any M&E requirements 30 Scope of major M&E events and functions 30 STEP 2 Plan for data collection and management 32 Develop an M&E plan table 32 Assess the availability of secondary data 33 Determine the balance of quantitative and qualitative data 35 Triangulate data collection sources and methods 36 Determine sampling requirements 36 Prepare for any surveys 38 Prepare specific data collection methods/tools 38 Establish

6 Stakeholder complaints and feedback mechanisms 40 Establish project/programme staff/volunteers review mechanisms 42 Plan for data management 43 Use an indicator tracking table (ITT) 45 Use a risk log (table) 47 STEP 3 Plan for data analysis 48 Develop a data analysis plan 49 Follow the key data analysis stages 50 STEP 4 Plan for information reporting and utilization 57 Anticipate and plan for reporting 58 Plan for information utilization 66 STEP 5 Plan for M&E human resources and capacity building 692 International Federation of Red Cross and Red Crescent SocietiesProject/programme monitoring and evaluation guide Assess the project/programme s human resources capacity for M&E 69 Determine the extent of

7 Local participation 69 Determine the extent of outside expertise 72 Define the roles and responsibilities for M&E 72 Plan to manage project/programme team s M&E activities 73 Identify M&E capacity-building requirements and opportunities 73 STEP 6 Prepare the M&E budget 74 Itemize M&E budget needs 74 Incorporate M&E costs in the project/programme budget 74 Review any donor budget requirements and contributions 75 Plan for cost contingency 75 ANNEXES 77 Annex 1: Glossary of key terms for M&E 77 Annex 2: M&E resources 83 Annex 3: Factors affecting the quality of M&E information 88 Annex 4: Checklist for the six key M&E steps 90 Annex 5: IFRC s logframe definition of terms 92 Annex 6: Example M&E stakeholder assessment table 93 Annex 7: Example M&E activity planning table 95 Annex 8: M&E plan table template and instructions 96 M&E plan example 97 M&E plan purpose and compliance 98 M&E plan instructions 98 Annex 9.

8 Closed-ended questions examples 100 Annex 10: Key data collection methods and tools 101 Annex 11: Project/programme feedback form template 103 Annex 12: Complaints log 104 Annex 13: Staff/volunteer performance management template 105 Annex 14: Individual time resourcing sheet 106 Annex 15: Project/programme team time resourcing sheet 107 Annex 16: Indicator tracking table (ITT) examples and instructions 108 Annex 17: Example risk log 113 Annex 18: Reporting schedule 114 Annex 19: IFRC s project/programme management report template and instructions 115 Annex 20: Example tables (logs) for action planning and management response 122 Annex 21: Example M&E job description 123 Annex 22: M&E training schedule 127 List of tables, boxes and diagrams Table 1: Common types of monitoring 12 Table 2: Summary of major evaluation types 15 Table 3: The IFRC s framework for evaluation criteria and standards 17 Table 4: Comparing key features of monitoring /review, evaluation and audit 20 Table 5.

9 Example of indicator tracking table for one quarter only 46 Table 6: Comparing data analysis terms: findings, conclusions, recommendations and actions 563 International Federation of Red Cross and Red Crescent SocietiesProject/programme monitoring and evaluation guide Box 1: Principle Nine of the Conduct for International Red Cross and Red Crescent Movement and NGOs in Disaster Relief 6 Box 2: monitoring best practices 13 Box 3: The challenge of measuring impact 18 Box 4: Principle Five of the Code of Conduct for International Red Cross and Red Crescent Movement and NGOs in Disaster Relief 21 Box 5: M&E in emergency settings 27 Box 6: Types of industry (standard) indicators 28 Box 7: Examples of IFRC s key stakeholders and informational needs 29 Box 8: Specific evaluation requirements for the IFRC s secretariat-funded projects/programmes 30 Box 9: Examples of key M&E activities 31 Box 10: Is an M&E plan worth all the time and effort?

10 33 Box 11: Comparing quantitative versus qualitative data 35 Box 12: Minimizing data collection costs 40 Box 13: The IFRC s guide for stakeholder feedback 42 Box 14: Formats can reinforce critical analysis and use 44 Box 15: The importance of target setting 47 Box 16: Benefits of involving multiple stakeholders in data analysis 50 Box 17: Data analysis questions to help describe the data 52 Box 18: Using traffic lights to highlight data 55 Box 19: Criteria of good reporting 58 Box 20: Internal versus external reporting 60 Box 21: Example reporting formats 62 Box 22: Report writing tips 63 Box 23: IFRC s project/programme management report ou