IMPACT EVALUATION - ws.mbri.ac.ir

1 1 Money and Banking Research Institute IMPACT EVALUATION Summer 2014 Instructor: Mahdi Rastad Assistant professor of Finance, California Polytechnic State University ( ) Office Location: Office Hours By appointment Email: COURSE OBJECTIVE Students passing this course will learn: The basic issues pertaining to an EVALUATION of an intervention to reach certain targets and goals. The fundamental challenge with program/ IMPACT EVALUATION ; endogeneity problem. The experimental approach in designing an IMPACT EVALUATION study; the key role of randomization Non-experimental methods used in observational studies including: o Difference in difference o Matching and propensity score method o Instrumental variable o Regression discontinuity method o Quantile regression Comparison of usefulness of experimental and non-experimental methods in policy making.

2 BERIEF COURSE DESCRIPTION The course provides an opportunity for students to gain a deep understanding of methods that concerns policy makers when evaluating public programs to see if they reach the targets they are designed for. The main theme of this course is a review of the methods to evaluate whether those programs actually work and how they IMPACT the beneficiaries. Here are some of the textbook examples: Has the Grameen Bank, for example, succeeded in lowering consumption poverty among the rural poor in Bangladesh? Can conditional cash transfer programs in Mexico and Latin America improve health and schooling outcomes for poor women and children? Does a new road actually raise welfare in a remote area in Tanzania, or is it a "highway to nowhere?" To benefit from learning by doing students are required to develop an IMPACT EVALUATION report as a project for the course.

3 2 COURSE METHODS The course involves a combination of teaching and learning methods: 1. Readings of book chapters to preview the material (mandatory) 2. Short lectures to review specific concepts and the toolbox of IMPACT EVALUATION 3. In class problems to be discussed and solved in groups 4. Final project and presentations TEXT and MATERIALS Required Textbook Handbook on IMPACT EVALUATION : Quantitative Methods and Practices, by Khandker, Shahidur R., Gayatri B. Koolwal, and Hussain A. Samad. World Bank Publications, 2010. (KKS) You can download a free copy of the textbook directly from here or from the worldbank s website: Also, you can access STATA codes and data files for STATA exercises in the book from here: Recommended Textbooks Mostly Harmless Econometrics: An Empiricist's Companion, by Angrist, Joshua D.

4 , and J rn-Steffen Pischke, Princeton university press, 2008. FINAL PROJECT As an application of the skills learned in the course you will build an IMPACT EVALUATION report step by step as the course progress. In here, you identify a policy related question, locate related literature, find relevant data basis, illustrate your identification strategy using an experimental or quasi-experimental design and finally estimate your model and comments on the effectiveness of the program. PREREQUISITES You need to have access to a computer/laptop and STATA ( ) during the class time. Basic knowledge of Econometrics is essential or you must be very devoted to teach yourself an intense introductory econometric course along the course. An excellent place to start is Introductory Econometrics: A Modern Approach, by Wooldridge, Jeffrey.

5 Cengage Learning, 2012. Introductory Econometrics by Wooldridge. A preliminary acquaintance with STATA is a plus, but not required. Public Disclosure AuthorizedPublic Disclosure AuthorizedPublic Disclosure AuthorizedPublic Disclosure Authorized 3 COURSE SCHEDULE This is a tentative outline for the course. Changes may be made as the course progresses. Throughout the course, I may cover materials that are not from the textbook, including papers and book chapters. Topic 1: Basic Issues of EVALUATION Chapter 2 of KKS Monitoring versus EVALUATION Quantitative versus Quantitative IMPACT Assessments Quantitative IMPACT Assessment: Ex Post versus Ex Ante IMPACT Evaluations The Problem of the Counterfactual Basic Theory of IMPACT EVALUATION : The Problem of Selection Bias Topic 2: Randomization Chapter 3 of KKS Setting the Counterfactual Statistical Design of Randomization Calculating Treatment Effects Different Methods of Randomization Concerns and Difficulties with Randomization Topic 3: Propensity Score Matching (PSM) Chapter 4 of KKS PSM and Its Practical Uses What Does PSM Do?

6 PSM Method in Theory Application of the PSM Method Critiquing the PSM Method PSM and Regression-Based Methods Topic 4: Double Difference (DD) Chapter 5 of KKS Addressing Selection Bias from a Different Perspective: Using Differences as Counterfactual DD Method: Theory and Application Advantages and Disadvantages of Using DD Alternative DD Models Topic 5: Instrumental Variable Estimation Chapter 6 of KKS Two-Stage Least Squares Approach to IVs Concerns with IVs Sources of IVs Topic 6: Regression Discontinuity and Pipeline Methods Chapter 7 of KKS Regression Discontinuity in Theory Sharpe vs. Fuzzy RD Advantages and Disadvantages of the RD Approach Pipeline Comparisons Topic 7: Quantile Regression and Measuring Distributional Program Effects a Chapter 8 of KKS Structural versus Reduced-Form Approaches Why Quantile Regression?

7 Average Treatment Effect (ATE) vs. Quantiel Treatment Effect (QTE) Modeling the Effects of Policies Assessing the Effect of Policies in a Macroeconomic Framework Modeling Household Behavior in the Case of a Single Treatment: Case Studies on School Subsidy Programs.