Monitoring and Evaluation How to develop M&E plans

1 Monitoring and Evaluation How to develop M&E plansAction Against Desertification programmeWorkshopon socio-economic baseline assessment and M&E planningOuagadougou, Burkina Faso, 21st 25thMarch 201612 INTRODUCTION Monitoring and Evaluation (M&E) is necessary and efficient process to deliver development results M&E is a part of the programme, not something to be added afterwards Designing an effective M&E system helps programmemanagers toomake sure that objectives are well understood and achievableoplan for data collectionomake sure that the data collected is used for decision-makingoto improve the programme, year after year, but also to better design and implement the next programmeDEFINITIONSMONITO-RINGis a continuous process of collecting data to verify whether the objectives of your programme are likely to be attainedis done by identifying expected results at each step of the programme and the indicators to assess these resultsrelies on data that you can easily and regularly retrieve EVALUA-TIONis the systematic assessment of a project, programme or policy, its design.

2 Implementation and resultsis usually done once or twice in the lifetime of a programmerelies on advanced database analyses, surveys, and interviews with stakeholders, focus groups, case studies at understanding how and why, as much as whether or not your programme managed to achieve its objectivesThough they are planned together, Monitoring and evaluationdiffer in terms of audience, timing, purposes and actors3 OUTLINE OF THEPRESENTATION Choosing what to monitor Basic concepts on M&E indicators Choosing indicators Planning for M&E activities4 CHOOSING WHAT TOMONITOR[1] The first step to set up a M&E system is to review the logical frameworkof the project, by clarifying and updating oProject activitiesoExpected results (in terms of targets)oExpected outcomes (specific objectives) (in terms of targets) Specifyassumptionsand risksoAssumptions are the variables or factors that need to be in place for results to be achievedoAssumptions shall be revised all along the project, they are likely to be easier to define after a few months of implementationoRisks correspond to a potential future event, fully or partially beyond one's control that may (negatively) affect the achievement of results5 CHOOSING WHAT TOMONITOR[2] Focusing on expected results and specific objectives of the logframe, define good M&E questions for each of them oHow can I measure the achievement of this result/objective?

3 OWhich data do I need?oWho may be concerned?oWhen shall I retrieve the data? Stakeholdersmight be involved in the design and implementation of the M&E system, especially when revising the logframe and defining the right M&E questionsoFocus groups to focus on M&E questionsoInclude stakeholders in the M&E Committee6 BASICCONCEPTS ONM&E INDICATORS[1] Selecting the right indicators is crucial to develop good M&E plans Indicators measure the achievement ofoGeneral objectives of the programme IMPACT INDICATORSoSpecific objectives of the project OUTCOME INDICATORSoExpected results PERFORMANCE INDICATORS Indicators may be numerical (quantitative) and non-numerical (categoricalor descriptors)oExample of quantitative indicator Number of trees planted within a reforestation projectoExample of categorical indicator Species plantedoExample of descriptor Attitude of the people to the project7 BASICCONCEPTS ONM&E INDICATORS[2] Numerical indicators are preferred to answer to M&E questions, because they are simpleand easily understandable, but some non-numerical information may be useful to better understand what happened Indicators can be specific, constructed specifically for the M&E, or generic, they are generally agreed to be able to measure a certain phenomenon Both specific and generic indicators are needed, they should be balanced within the M&E system It is recommended that different projects belonging to the same programme share a similar set of indicators, so that their results are comparable8 BASICCONCEPTS ONM&E INDICATORS[3]

4 COUMPOUND INDICATORSAre calculated as the weighted sum of several elementary or derived indicatorsDERIVED INDICATORSAre based on the calculation of the ratio between two elementary indicatorsELEMENTARY INDICATORSP rovide basic information on which other indicators can be built9 BASICCONCEPTS ONM&E INDICATORS[4]COUMPOUND INDICATORSC omposite indicator on physical capital, encompassing tractors, type of house, water points INDICATORSR atio of households holding a tractor on the total number of households involved in farmingELEMENTARY INDICATORSN umber of households holding a tractor10 CHOOSINGINDICATORS[1] Indicators aim to answer to the M&E questions They are always an approximation, therefore it is important to choose the best approximation Each indicator should be described in detail, in terms ofoDefinition: of how it should be measuredoFrequency: how often it should be measuredoSource: where to collect the data needed for the calculation (internal data collected within the project, external data or survey data) Each indicator should include a baselineand a targetoBaseline values are usually defined through a baseline study, to be performed at the beginning of the projectoTargets are defined at the beginning but they may be revised during the lifetime of the project11 CHOOSINGINDICATORS[2] All these information are summarized in the indicator tableName of the indicatorPercentage of extension officers trained on the strategies for reforestationDefinitionIndicator = (A/B*100)%, where:A= number of extension officers who completed the trainingB= estimated number of extension officers who are likely to be involved in the implementation of reforestation strategiesFrequencyEvery yearSourcesFor elementary indicator A: activity reports of the programmeFor elementary indicator B.

5 Needs assessment performed in the design phase of the programmeBaseline15% (54 extension officers were trained in a previous project, against an estimated target of 360)Targetyear 3: 50%; year 5 (end of programme): 75%12 CHOOSINGINDICATORS[3] How to choose the right indicators? Indicators should be few enough so that the M&E system does not become a burden to the very programme it is supposed to serve and numerous enough so that all the major results and objectives of the programme are covered. Usually, indicators are selected if the answer the following questions is positiveoAm I able to collect and analyse data?oIs the indicator really necessary?oWhat indicators are used by the organization?oWhat is available from the project level?oDo I have enough indicators to cover all the results and objectives?13 CHOOSINGINDICATORS[4] When the set of indicators is finally defined, it is recommended to check its quality The quality of indicators can be checked through the SMART criteria; SMART indicators areoSpecific: you can make a decision based on it;oMeasurable: it is possible to retrieve the data needed to calculate them;oAchievable: they have a target value and this target may be attained;oRelevant: they answer the information needs.

6 OTime-phased: the target value evolves in time depending on the time needed to achieve the expected FORM&E ACTIVITIES[1] Once the indicators are defined, it s time to plan M&E activities, namelyoData collectionoAnalysis and reportingoResourcesoData management15 PLANNING FORM&E ACTIVITIES[2]Testimoniesmay be collected via interviews, workshops and surveysMain issues in surveys:- Sampling- QuestionnaireAnnual data collection for Monitoring and Evaluation purposes Planning for data collectionQuestionnairestraightforwardQu estionnaireshould be relevant to calculateindicators, be short and straightforwardthe Sampling:- probabilisticor not-Sizeto be calculated atthe time of the surveyBaseline studyfor a first measurementof the indicators16 PLANNING FORM&E ACTIVITIES[3]Indicators may be below, close or above the targetUsing the traffic light method for reporting maybe usefulAnnualreports Annual reports containupdated valuesof indicatorscompared to the expectedtargets Planning data analysis and reportingFigures shouldalway be accompaniedby interpretation!

7 17 PLANNING FORM&E ACTIVITIES[4]Define the tasks to be completedCommon tasks Common tasks in M&E: - data collection- calculation of indicators- interpretation- reportingDefine roles and responsibilities within the M&E team Planning for resourcesfor staff, Cost for staff, reporting, expertise, data collectionrelated to M&E should be estimatedTraining, coaching and knowledgesharing mayhelp to buildM&E team scapacitiesHuman resourcesneeded for M&E should be planned asearly aspossible18 PLANNING FORM&E ACTIVITIES[5]use should be pData availability: intended users should have access to data; unauthorized use should be preventedInformation technology: to systematize the recording, storage and use of data Data format: standardizedformats and templates are recommended Planning for data managementData security: Data security: identifiyresponsiblesfor data management and secureconfidentialityData qualitycontrol: data checking and cleaning, management of missingvaluesData organization: organizeinformation into logicaleasy to use categories19 Thanks for your attention !

8 20