1 Impact Assessment Impact Assessment is a means of measuring the effectiveness of organisational activities and judging the significance of changes brought about by those activities. It is neither Art or Science, but both. Impact assessment is intimately linked to Mission, and, in that sense, ripples through the organisation. Being able to assess and articulate Impact is a powerful means of communicating, internally and externally, the contribution of activities to the Mission of IFRC and NS. Impact is seen as the positive and negative, intended or unintended long-term results produced by an IFRC operation or National Society, either directly or indirectly.
2 Impact should be seen as the contribution of the intervention to the overall goal. The Problem of Impact Impact assessment is straightforward in development projects. There is a large literature underpinning Impact , a wealth of experience and accepted norms and practises. This is not the case for the humanitarian sector. The rationale for Impact arises from the introduction of Results Based Management techniques and a mindset that sees Impact practice in the developmental sector as transferable to the humanitarian sector. It is not. And this is the nub of the problem. Until there are agreed norms and standards across the humanitarian sector then Impact assessment will remain a contentious issue.
3 A start has been made with SPHERE. And how should this problem be addressed? Firstly both the donor community and the humanitarian sector have to recognise the problem and agree to look at Impact from a different perspective. Given the difficulty of measurement in complex and chaotic environments a more intuitive approach is needed. Impact is a function of the effectiveness, relevance and sustainability of the intervention. Evaluation for Impact should be directed at looking across projects as a whole and asking, quite simply, Did it meet real needs? This raises the problem of verification and brings me to the second point.
4 The simplest way of verifying is to ask the beneficiaries. Participatory approaches must be embedded within the humanitarian sector, but done in a way that gives confidence to donors. This raises the problem of norms and standards. Humanitarian organisations could demonstrate this by showing how much of the budget is used to develop this capacity. But in the longer term a standardised approach to training with external accreditation would give 1. confidence to both donors and the humanitarian sector. Trained field workers will have the confidence and ability to make project changes as and when needed.
5 Evaluation will then be able verify the efficacy of changes. Impact assessment comes down to developing two proxies. The first is asking if the intervention met real needs and the second is accredited training in participatory techniques for field workers. Impact assessment is concerned with making judgements about the effect on beneficiaries of humanitarian interventions. It is a function of the results chain and is an integral part of Monitoring and Evaluation and should be incorporated into the design of a Monitoring and Evaluation framework (see Module 2). In order to consider the full extent of Impact , questions have to be raised internally and externally.
6 Internally 1. Assess the relevance of strategies are current strategies contributing to the overall goal? 2. Evaluate the effectiveness of organizational structures and management systems are there areas where structures impede the realisation of the overall goal? 3. Evaluate communications are messages effectively communicated throughout the organisation, is information being fed to appropriate places at the right times, is everyone aware and informed, is the need to ensure significant and positive impacts embedded within the culture of the organisation, are we doing all that we can to ensure that we are effective?
7 2. Externally 1. Show donors the Impact of their contributions make the link between donation and changes in the lives of beneficiaries. 2. Show the wider public the effectiveness, relevance and efficiency of operations . Impact can generate wider support. 3. Demonstrate to potential funders the focus of the organisation that is on realising change as opposed to a focus on process. 4. Increase advocacy be an effective voice for the vulnerable in ensuring that their voices are heard. 5. Embrace transparency to show an effective and well functioning organisation, able to mobilize support and realize the overall goal 6.
8 Open up to scrutiny for independent verification When should Impact Assessment take place? Impact is a measure of the changes made and Impact assessment seeks to establish a causal connection between inputs and changes in terms of magnitude or scale or both. The Logframe (see Module 1) provides the starting pointing for integrating Impact assessment with Monitoring and Evaluation. Impact assessment should be seen as the contribution of the Outputs and Outcomes to Purpose and Overall Goal. Impact is not usually incorporated into the Logframe, Table 1 illustrates where it can be placed within the structure of the Logframe.
9 Table 1: Placement of Impact within the Logframe Logframe Performance Indicators Means of Assumptions Hierarchy Verification and Risks Goal Purpose Impacts Impact Indicators the changes Quantitative and assessment of (positive, negative, intended, qualitative changes made by unintended) made by techniques intervention(s) intervention(s). Outcomes Outputs Process Indicators Activities Inputs Costs 3. Planning for Impact Assessment for Field Operations The approach to Impact assessment is very similar to the planning that is required to establish a monitoring system see Module The main difference is in the type of information and data that is needed to be able to assess Impact .
10 This can be summarised in the Logframe structure. Table 2 (building on table in what monitoring data are required? ). shows the relationship between the Logframe and Impact questions:- Table 2: Logframe and Impact Questions Logframe Level Monitoring Questions Impacts To what extent has the intervention contributed to positive changes in the lives of beneficiaries? Have there been any unintended or negative changes that can be attributed to the intervention? Can beneficiaries identify the changes made by the intervention? Are there any trends (morbidity and mortality rates) that the intervention has influenced?