Logic models for program design, implementation, and ...

1 Tools May 2015 Logic models for program design , implementation , and evaluation: Workshop toolkit Karen Shakman Sheila M. Rodriguez Education Development Center, Inc. Overview This Logic model Workshop Toolkit is designed to help practitioners learn the overall purpose of a Logic model , the different elements of a Logic model , and the appropriate steps for developing and using a Logic model for program evaluation. This toolkit includes a facilitator workbook, a participant workbook, and a slide deck. At Education Development Center, Department of Department of Education Arne Duncan, Secretary Institute of Education Sciences Sue Betka, Acting Director National Center for Education Evaluation and Regional Assistance Ruth Curran Neild, Commissioner Joy Lesnick, Associate Commissioner Amy Johnson, Action Editor Chris Boccanfuso, Project Officer REL 2015 057 The National Center for Education Evaluation and Regional Assistance (NCEE) conducts unbiased large-scale evaluations of education programs and practices supported by federal funds.

2 Provides research-based technical assistance to educators and policymakers; and supports the synthesis and the widespread dissemination of the results of research and evaluation throughout the United States. May 2015 This report was prepared for the Institute of Education Sciences (IES) under Contract ED-IES-12-C-0009 by Regional Educational Laboratory Northeast & Islands administered by Education Development Center, Inc. The content of the publication does not neces sarily reflect the views or policies of IES or the Department of Education nor does mention of trade names, commercial products, or organizations imply endorsement by the Government.

3 This REL report is in the public domain. While permission to reprint this publication is not necessary, it should be cited as: Shakman, K., & Rodriguez, S. M. (2015). Logic models for program design , implementation , and evaluation: Workshop toolkit (REL 2015 057). Washington, DC: Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Northeast & Islands. Retrieved from This report is available on the Regional Educational Laboratory website at ncee/edlabs. Contents Facilitator workbook Introduction to workshop 3 Session I.

4 Learning about Logic models (2 hours) 5 Session II. From Logic models to program and policy evaluation ( hours) 30 Appendix A. Simple Logic model A-1 Appendix B. Template for simple Logic model B-1 Appendix C. Sample Logic model for college readiness program C-1 Appendix D. Example of a Logic model for an educator evaluation system theory of action D-1 References and resources Ref-1 Participant workbook Introduction to workshop 3 Session I. Learning about Logic models (2 hours) 4 Session II. From Logic models to program and policy evaluation ( hours) 26 Appendix A. Simple Logic model A-1 Appendix B.

5 Template for simple Logic model B-1 Appendix C. Sample Logic model for college readiness program C-1 Appendix D. Example of a Logic model for an educator evaluation system theory of action D-1 References and resources Ref-1 i

6 Facilitator workbook Contents Introduction to workshop 3 Session I. Learning about Logic models (2 hours) 5 Agenda 5 Pre-assignment 6 Goals 7 Introducing the cases 8 Additional case examples 9 What is a Logic model ?

7 11 Elements of a Logic model 16 Additional activity: Strategies and activities in sequence 23 The Logic in a Logic model 26 Next steps 28 Final thoughts 29 Session II. From Logic models to program and policy evaluation ( hours) 30 Agenda 30 Pr e-a s si g n ment 31 Goals 32 Example cases revisited 33 Additional case examples 34 Review of Logic models 35 Introducing evaluation 37 Moving from Logic model to evaluation questions 39 Generating indicators 43 Building an evaluation design 48 Additional activity: Data collection framework 52 Putting it all together 53 Review 55 Appendix A. Simple Logic model A-1 Appendix B.

8 Template for simple Logic model B-1 Appendix C. Sample Logic model for college readiness program C-1 Appendix D. Example of a Logic model for an educator evaluation system theory of action D-1 References and resources Ref-1 1 Activities Discussion of cases 10 Inputs outputs outcomes 15 Problem statement 17 Focus on outcomes 19 Intangible resources 24 Uncovering internal and external assumptions 25 If-then statements 27 Next steps 28 How will you know? 37 Formative and summative evaluation 40 Generating questions for different audiences 42 Generating indicators of the flu 43 Process and outcome indicators 45 Consider data sources 49 Table 1 Traditional audiences, evaluation questions, and uses 41 2 Introduction to workshop A Logic model is a visual representation of a theory of action or program Logic guiding the design and implementation of a program or policy and can be used as a tool for build ing a relevant evaluation design .

9 The Regional Educational Laboratory (REL) Northeast & Islands administered by Education Development Center created this workshop to help groups, such as the research alliances affiliated with the 10 RELs, as well as individual alliance members, learn about and build Logic models to support program designs and eval uations. Based on feedback from alliance members, REL Northeast & Islands learned that many of its district- and state-based members would like to build their capacity to develop Logic models for both evaluating their own programs and working more effectively with evaluators whom they engage to conduct evaluations on their behalf.

10 This workshop provides a primer on Logic modeling and demonstrates how to use Logic models as a tool for program evaluation. The overarching goals of the workshop are to: Introduce Logic models as an effective tool for program or policy design , implemen tation, and evaluation. Practice the elements of a Logic model . Provide guidance in appropriate steps for building a Logic model for a program or initiative. Practice using Logic models to develop evaluation questions and indicators of success. Provide guidance in how to determine the appropriate evaluation for a specific program or policy.