Impact evaluation methods - economia.uniroma2.it

1 Impact evaluation methods Gianluigi Nico Food and Agriculture Organization. Economic and Social Development Department (Statistics Division) Email: Introduction Identifying the effect of a policy or programme is a complex and challenging task. This issue is particularly prominent in certain economy, where government are under pressure to implement policies and programme aimed at recharging growth and reducing poverty rates. Monitoring and evaluation are at the heart of evidence-based policy and programme making. They provide a core set of tools that can be used to verify the casual Impact of a given policy or programme in practice, thus improving the quality, efficiency, and effectiveness of the implemented interventions at various stages of implementation. Broadly speaking, monitoring and evaluation assess whether or not a programme has achieved its intended results.

2 The key policy question behind Impact and evaluation methods revolves around the casual effect of a programme on an outcome of interest, which is: what is the causal effect of scholarships on school attendance and academic achievement? What is the Impact on access to health care of contracting out primary care to private providers? Do improved roads increase access to labour markets and raise households income? The approach to Impact evaluation in this course is meant to be largely intuitive, through an attempt to minimize technical notation. After having introduced the concepts that underpin any Impact evaluation method, the course explores a core set of Impact evaluation empirical tools (econometric models) that can be applied to real-world development operations.

3 The methods are drawn directly from applied research in the social sciences. In this sense, Impact evaluation brings the empirical research tools widely used in economics and other social sciences together with the operational and political-economy realities of policy implementation and development practice. In this course, empirical econometric tools presented will also be explored through empirical exercises, by using econometric software ( stata ). The overall goal of these lectures is to provide students with knowledge and skills on Impact evaluations that will open up the possibility to land jobs in academic research, government agencies, NGOs as well as private firms. Duration: 12h (9 front + 3 stata ) Course structure: 1. Basic issues of Impact and evaluation Introduction: Monitoring versus evaluation Indicators within a Monitoring and evaluation Framework Quantitative versus qualitative Impact Assessment 2.

4 Causal Inference and Counterfactuals The problem of counterfactual The problem of selection bias 3. Randomized Selection methods Randomized Assignment of the Treatment Randomized Impact evaluation in practice 4. Impact evaluation empirical tools Regression Discontinuity Design Difference-in-Differences Matching 5. stata exercises Introduction: stata for M&E Difference-in-difference model (empirical application using stata ) Impact estimation using regression discontinuity design (empirical application using stata ) Materials: Do. files and associated datasets will be provided during lectures. Materials: Chapter 2 and 3: World Bank Handbook: Impact evaluation . Quantitative methods and practices Chapter 4 to 7 Paul J. Gertler, Sebastian Martinez, Patrick Premand, Laura B. Rawlings, Christel M.

5 J. Vermeersc: Impact evaluation in Practice . Suggested Readings Arcand, , and L. Bassole (2009) Does Community Driven Development Work? Evidence from Senegal. Graduate Institute and the World Bank, April. Rosenbaum, P., and D. Rubin (1983): The Central Role of the Propensity Score in Observational Studies for Causal Effects. Biometrika, 70(1), 41-55. ~ryantibs/ Buddelmeyer, Hielke, and Emmanuel Skoufi as. 2004. An evaluation of the Performance of Regression Discontinuity Design on PROGRESA. World Bank Policy Research Working Paper 3386, IZA Discussion Paper 827, World Bank, Washington, DC. Duflo, E., and R. Chattopadhyay (2004): Women as Policy Makers: Evidence from a Randomized Policy Experiment in India. Econometrica, 72(5), 1409-1443. Miguel, E., and M. Kremer (2004): Worms: Identifying Impacts on Education and Health in the Presence of Treatment Externalities.

6 Econometrica, 72(1), 159-217. de Janvry, Alain, Frederico Finan, Elisabeth Sadoulet, and Renos Vakis. 2006. Can Conditional Cash Transfer Programs Serve as Safety Nets in Keeping Children at School and from Working When Exposed to Shocks? Journal of Development Economics 79 (2): 349 73. ~ James J. Heckman: Sample Selection Bias as a Specification Error