Improving Teaching Effectiveness - rand.org

1 Improving Teaching EffectivenessBRIAN M. STECHER, DEBORAH J. HOLTZMAN, MICHAEL S. GARET, LAURA S. HAMILTON, JOHN ENGBERG, ELIZABETH D. STEINER, ABBY ROBYN, MATTHEW D. BAIRD, ITALO A. GUTIERREZ, EVAN D. PEET, ILIANA BRODZIAK DE LOS REYES, KAITLIN FRONBERG, GABRIEL WEINBERGER, GERALD PAUL HUNTER, JAY CHAMBERSCORPORATION final reportThe INTENSIVE PARTNERSHIPS for EFFECTIVE Teaching Through 2015 2016 This work is licensed under a Creative Commons Attribution International License. All users of the publication are permitted to copy and redistribute the material in any medium or format and transform and build upon the material, including for any purpose (including commercial) without further permission or fees being required. For additional information , please visit rand Corporation is a research organization that develops solutions to public policy challenges to help make communities throughout the world safer and more secure, healthier and more prosperous.

2 rand is nonprofit, nonpartisan, and committed to the public interest. rand s publications do not necessarily reflect the opinions of its research clients and RANDMake a tax-deductible charitable contribution at of Congress Cataloging-in-Publication Data is available for this : 978-1-9774-0079-6 For more information on this publication, visit by the rand Corporation, Santa Monica, Calif. Copyright 2018 rand CorporationR is a registered : julief514/GettyImages with Bill & Melinda Gates Foundation launched the Intensive Part-nerships for Effective Teaching (IP) in the 2009 2010 school After thorough screening, the foundation identified seven IP sites three school districts and four charter management organizations to implement teacher-evaluation systems and related human-relations reforms patterned on a common design over a six-year The foundation also selected the R AND Corporation and its partner, the American Institutes for Research (AIR), to evaluate the IP efforts.

3 The evaluation began in July 2010 and collected data annually for six school years, from 2010 2011 through 2015 R AND/AIR team produced three interim reports and two journal articles:3 Brian M. Stecher, Michael S. Garet, Laura S. Hamilton, Elizabeth D. Steiner, Abby Robyn, Jeffrey Poirier, Deborah Holtzman, Eleanor S. Fulbeck, Jay Chambers, and Iliana Brodziak de los Reyes, Improving Teaching Effectiveness : Implementation The Intensive Partnerships for Effective Teaching 1 In the interest of concision, we use the year pair alone, without school year, to indicate a school year ( , 2009 2010 is the school year running from the fall of 2009 through the spring of We use the word site to describe any of the three school districts or four charter manage-ment organizations that received funding from the foundation to implement the IP We also prepared a series of internal reports to the foundation and the Improving Teaching Effectiveness : Final ReportThrough 2013 2014, Santa Monica, Calif.)

4 : R AND Corporation, RR-1295-BMGF, 2016 Matthew D. Baird, John Engberg, Gerald Hunter, and Benjamin Master, Improving Teaching Effectiveness : Access to Effective Teaching The Intensive Partnerships for Effective Teaching Through 2013 2014, Santa Monica, Calif.: R AND Corporation, RR-1295/4-BMGF, 2016 Italo A. Gutierrez, Gabriel Weinberger, and John Engberg, Improving Teaching Effectiveness : Impact on Student Outcomes The Intensive Partnerships for Effective Teaching Through 2013 2 014, Santa Monica, Calif.: R AND Corporation, RR-1295/3-1-BMGF, 2016 Gema Zamarro, John Engberg, Juan Esteban Saavedra, and Jennifer Steele, Disentangling Disadvantage: Can We Distin-guish Good Teaching from Classroom Composition?

5 Jour-nal of Research on Educational Effectiveness , Vol. 8, No. 1, 2015, pp. 8 4 111 Brian Stecher, Mike Garet, Deborah Holtzman, and Laura Ham-ilton, Implementing Measures of Teacher Effectiveness , Phi Delta Kappan, Vol. 94, No. 3, November 2012, pp. 39 present report encompasses the full IP initiative, describ-ing its design, implementation, and impact on Teaching Effectiveness and student outcomes. This study was undertaken by R AND Educa-tion and AIR. The Bill & Melinda Gates Foundation sponsored the research. It should be of interest to researchers, policymakers, and prac-titioners who want to understand the potential benefits and challenges of adopting new teacher-evaluation systems and associated strategic human-relations AND Education is a unit of the R AND Corporation that conducts research on prekindergarten, kindergarten 12th grade, and higher education issues, such as assessment and accountability, choice-based and standards-based school reform, vocational training.

6 And the value of arts education and policy in sustaining and promoting well-rounded vMore information about R AND can be found at Questions about this report should be directed to and questions about R AND Education should be directed to in 1946, AIR is an independent, nonpartisan, not-for-profit organization that conducts behavioral and social science research on important social issues and delivers technical assistance, both domestically and internationally, in the areas of education, health, and workforce information about AIR can be found at ..iiiFigures ..xiiiTables ..xxiiiSummary ..xxvAcknowledgments ..liAbbreviations ..lvCHAPTER ONEI ntroduction ..1In Brief ..1 Overview of the IP Initiative ..2 The IP Theory of Action.

7 4 Overview of the IP Evaluation ..12 Remainder of the Report ..19 Next Steps ..21 CHAPTER T WOThe IP Sites ..23In Brief ..23 Introduction ..23 Demographic Characteristics of the IP Sites ..24 The Districts ..24 CMOs ..40 CHAPTER THREEM easures of Effective Teaching ..53viii Improving Teaching Effectiveness : Final ReportIn Brief ..53 Introduction ..54 Best Practices for Measuring TE as Part of a Teacher-Evaluation System ..54IP Sites TE Measures ..57 Distribution of TE Ratings over Time ..75 Implementation Challenges ..78 Teacher and SL Reactions to Sites Evaluation Systems ..83 Summary ..109 CHAPTER FOURR ecruitment, Hiring, Placement, and Transfer Policies ..113In Brief ..113 Introduction ..114 Best Practices with Respect to Recruitment and Hiring ..115IP Recruitment, Hiring, Placement, and Transfer Challenges.

8 133 Teacher and SL Reactions to Recruitment, Hiring, Placement, and Tr a n sfer Pol icie s ..136 Teacher Outcomes Associated with Recruitment, Hiring, Placement, and Transfer Policies ..151 Summary ..152 CHAPTER FIVET enure and Dismissal Policies ..155In Brief ..155 Introduction ..15 6 Best Practices for Tenure and Dismissal ..15 6IP Tenure and Dismissal Policies ..162 Implementation Challenges ..172 Teacher and SL Reactions to Tenure and Dismissal Policies ..175 Summary ..196 CHAPTER SIXPD Policies ..199In Brief ..199 Contents ixIntroduction ..200 Best Practices for Evaluation-Linked PD ..200 Intensive Partnership PD Policies ..206 Implementation Challenges ..216 Teacher and SL Reactions to PD Policies ..219 Summary ..243 CHAPTER SEVENC ompensation Policies.

9 247In Brief ..247 Introduction ..248 Best Practices for Teacher Compensation ..249IP Compensation Policies ..252 Implementation Challenges ..266 Teacher and SL Reactions to Compensation Policies ..268 Did the IP Compensation Policies Increase Pay for Effective Te a c h e r s ? ..282 Summary ..289 CHAPTER EIGHTCL Policies ..291In Brief ..291 Introduction ..292 Best Practices with Respect to CLs ..293IP CL Policies and How the Sites Defined Them ..296 Implementation Challenges ..313 Teacher and SL Reactions to CL Outcomes Associated with CL Policies ..330 Summary of the CL Lever ..330 CHAPTER NINER esources Invested in the IP Initiative ..333In Brief ..333 Introduction ..334 Evidence on Resources Needed to Implement the Best Practices Described in Previous Chapters.

10 335x Improving Teaching Effectiveness : Final ReportTotal Spending on the IP Initiative ..340 Proportion of Sites Total Budgets Spent on the IP Initiative in 2015 2016 ..344 Change over Time in Funding and Funding Source ..347 Spending by Implementation Lever ..350 Teachers and SLs Allocation of Time ..353 Comprehensive Cost of Operating the Teacher-Evaluation System in 2014 2015 ..362 Summary ..364 CHAPTER TENThe Effectiveness of Newly Hired Teachers ..367In Brief ..367 Introduction ..367IP Policies Designed to Influence New-Hire Effectiveness ..368SL Perceptions ..370 Changes in the Measured Effectiveness of New Hires ..375 Summary ..388 CHAPTER ELEVENR etention of Effective Teachers ..389In Brief ..389 Introduction ..390IP Policies Designed to Influence Retention and Issues Assessing Their Impact.