Qualitative Impact Protocol (QUIP)

1 Page | 1 assessing Rural Transformations (ART)A DFID/ESRC funded projectQualitative Impact Protocol ( quip )Guidelines for field useOverviewHow can the Impact of development activities intended to benefit poor men, women and children caught up in complex processes of rural transformation best be assessed? There is growing demand for Impact evidence within development agencies, both to support their own learning on effectiveness, and for public accountability. Independent Impact assessment of projects is increasingly used to complement internal monitoring, but doubts remain over the reliability and/or cost-effectiveness of the many different alternative approaches to Impact assessing Rural Transformations (ART) project addressed the issue of how specific interventions can be attributed to different stakeholders in a way that is reliable, timely and cost-effective, without distorting or constraining the development activity being assessed.

2 These guidelines lay out the resulting Impact assessment Protocol which uses quantitative monitoring of key indicators alongside Qualitative attribution of Impact based on self-reported assessment of Qualitative Impact Protocol ( quip ) uses semi-structured household interviews and focus group discussions to assess Impact based on self-reported attribution. This document contains guidance for all those involved in carrying out a quip in the field, as well as the household and focus group interview schedules used in the ART project (these can be adapted for use in different contexts). forDevelopment StudiesProf James Copestake (PI):Tel: +44 (0)1225 383859 For more information:Email: @CDS_BathWeb: work was supported by the Economic and Social Research Council; grant number ES/J018090/1 Page | 2 Contents1.

3 Introduction 32. Which method? Deciding what data is needed and how to collect it 4 Why use the quip rather than other forms of Impact assessment? 4 The issue of bias 53. Carrying out the quip 7 The quip Commissioner 7 Broad scoping of the study 7 Recruiting the lead evaluator and lead field researcher 8

4 Sample selection 8 Quality assurance, dissemination and use 11 The Lead Evaluator 11 Recruiting the lead field researcher 11 Refining data collection instruments

5 12 Briefing the lead field researcher 12 Moving in the field and arranging interviews 13 Data quality control, debriefing and data analysis 13 Using the findings 15 The Lead Field Researcher 16 The key principles of the quip 16

6 Before field work starts 17 Conducting household level interviews 18 Conducting focus group discussions (FGDs) 19 After field work 20 List of outputs required 21 Field Researchers 50

7 Code of Conduct 504. quip : Individual Household Interview 225. quip : Focus Group Guidelines 37 Page | 3 IntroductionThese guidelines aim to assist in planning and using Qualitative methods to evaluate the Impact of a development project.

8 Section 1 is addressed primarily to the commissioner of a study. The commissioner can be anyone who (a) identifies the need for a credible reality check on how the project is affecting its intended beneficiaries, and (b) is in a position to authorise work to meet this need. Section 1 covers what kind of Impact evidence might be needed and different options for obtaining it using this Qualitative Impact Protocol (or quip ) being one. Section 2 explains the practicalities of carrying out a quip , going into more detail about the specific roles of commissioner, lead evaluator and lead field researcher in turn. Appendices include illustrative data collection schedules and a final boxes provide further insight into how to use the quip by sharing experiences of piloting its use as part of the ART project.

9 This was an action research project, led by the University of Bath, into assessing rural transformations. For more about this project and how to contact us see These guidelines are continuously being developed and tested, so we welcome TermsQUIP stands for Qualitative Impact Protocol . These guidelines are designed specifically to guide Qualitative assessment of the Impact of rural livelihood projects. At the same time, they can also be adapted to address other kinds of development project is defined broadly to include any planned activity funded by one organisation or a group, and intended to improve the wellbeing of a defined group of intended beneficiaries. The quip can be used to assess the performance of on-going as well as time-bound is also defined broadly to include systematic enquiry into how a project worked or is working: how far, how cost-effectively and how sustainably it is realising its intended goals, and the appropriateness of these goals.

10 This can incorporate Impact assessment, but goes beyond it. Theory of change - The causal mechanism by which the sponsor of a project expects to achieve intended impacts, subject to clearly specified and managed | 42. Which method? Deciding what data is needed and how to collect itThere are many methods for obtaining evidence about the Impact of development projects. Each offers a different take and there are good reasons for using more than one approach. The quip aims primarily to collect rich and credible narrative descriptions of the causal links between project activities and changes in the wellbeing of intended beneficiaries. It does this by providing intended beneficiaries with the opportunity to describe their experience in an open way that places a high value on their personal perceptions and priorities.