Impact Evaluation in Practice - Fapesp

1 Impact Evaluation in PracticeSecond EditionPlease visit the Impact Evaluation in Practice book website at .org/ieinpractice. The website contains accompanying materials, including solutions to the book s HISP case study questions, as well as the corresponding data set and analysis code in the stata software; a technical companion that provides a more formal treatment of data analysis; PowerPoint presentations related to the chapters; an online version of the book with hyperlinks to websites; and links to additional book has been made possible thanks to the generous support of the Strategic Impact Evaluation Fund (SIEF).

2 Launched in 2012 with support from the United Kingdom s Department for International Development, SIEF is a partnership program that promotes evidence-based policy making. The fund currently focuses on four areas critical to healthy human development: basic education, health systems and service delivery, early childhood development and nutrition, and water and sanitation. SIEF works around the world, primarily in low-income countries, bringing Impact Evaluation expertise and evidence to a range of programs and policy-making Evaluation in PracticeSecond EditionPaul J.

3 Gertler, Sebastian Martinez, Patrick Premand, Laura B. Rawlings, and Christel M. J. Vermeersch 2016 International Bank for Reconstruction and Development / The World Bank1818 H Street NW, Washington, DC 20433 Telephone: 202-473-1000; Internet: rights reserved1 2 3 4 19 18 17 16 The fi nding, interpretations, and conclusions expressed in this work do not necessarily refl ect the views of The World Bank, its Board of Executive Directors, the Inter- American Development Bank, its Board of Executive Directors, or the governments they represent.

4 The World Bank and the Inter-American Development Bank do not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgement on the part of The World Bank or the Inter-American Development Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Nothing herein shall constitute or be considered to be a limitation upon or waiver of the privileges and immunities of The World Bank or IDB, which privileges and immunities are specifi cally and PermissionsThis work is available under the Creative Commons Attribution IGO license (CC BY IGO) http://creativecommons.

5 Org/licenses/ Under the Creative Commons Attribution license, you are free to copy, distribute, transmit, and adapt this work, including for commercial purposes, under the following conditions:Attribution Please cite the work as follows: Gertler, Paul J., Sebastian Martinez, Patrick Premand, Laura B. Rawlings, and Christel M. J. Vermeersch. 2016. Impact Evaluation in Practice , second edition. Washington, DC: Inter-American De velopment Bank and World Bank. License: Creative Commons Attribution CC BY IGOT ranslations If you create a translation of this work, please add the following disclaimer along with the attribution: This translation was not created by The World Bank and should not be considered an offi cial World Bank translation.

6 The World Bank shall not be liable for any content or error in this If you create an adaptation of this work, please add the following disclaimer along with the attribution: This is an adaptation of an original work by The World Bank. Views and opinions expressed in the adaptation are the sole responsibility of the author or authors of the adaptation and are not endorsed by The World content The World Bank does not necessarily own each component of the content contained within the work. The World Bank therefore does not warrant that the use of any third-party-owned individual component or part contained in the work will not infringe on the rights of those third parties.

7 The risk of claims resulting from such infringement rests solely with you. If you wish to re-use a component of the work, it is your responsibility to determine whether permission is needed for that re-use and to obtain permission from the copyright owner. Examples of components can include, but are not limited to, tables, fi gures, or queries on rights and licenses should be addressed to the Publishing and Knowledge Division, The World Bank, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2625; e-mail: (paper): 978-1-4648-0779-4 ISBN (electronic): 978-1-4648-0780-0 DOI: : C.

8 Andres Gomez-Pena and Michaela WieserCover Design: Critical StagesLibrary of Congress Cataloging-in-Publication DataNames: Gertler, Paul, 1955- author. | World : Impact Evaluation in Practice / Paul J. Gertler, Sebastian Martinez, Patrick Premand, Laura B. Rawlings, Christel M. J. : Second Edition. | Washington, : World Bank, 2016. | Revised edition of Impact Evaluation in Practice , ers: LCCN 2016029061 (print) | LCCN 2016029464 (ebook) | ISBN 9781464807794 (pdf ) | ISBN 9781464807800 | ISBN 9781464807800 ()Subjects: LCSH: Economic development projects Evaluation .

9 | Evaluation research (Social action programs)Classifi cation: LCC 2016 (print) | LCC (ebook) | DDC dc23LC record available at v vPreface xvAcknowledgments xxiAbout the Authors xxiiiAbbreviations xxviiPART ONE. INTRODUCTION TO Impact Evaluation 1 Chapter 1. Why Evaluate?

10 3 Evidence-Based Policy Making 3 What Is Impact Evaluation ? 7 Prospective versus Retrospective Impact Evaluation 9 Effi cacy Studies and Effectiveness Studies 11 Complementary Approaches 13 Ethical Considerations Regarding Impact Evaluation 20 Impact Evaluation for Policy Decisions 21 Deciding Whether to Carry Out an Impact Evaluation 26 Chapter 2. Preparing for an Evaluation 31 Initial Steps 31 Constructing a Theory of Change 32 Developing a Results Chain 34 Specifying Evaluation Questions 36 Selecting Outcome and Performance Indicators 41 Checklist: Getting Data for Your Indicators 42 PART TWO.