Approaches to Evaluating Teacher Effectiveness: A Research ...

1 Approaches to Evaluating Teacher Effectiveness: A Research Synthesis June 2008 Laura Goe, Courtney Bell, Olivia Little ETS 1100 17th Street NW, Suite 500 Washington, DC 20036-4632 877-322-8700 202-223-6690 Copyright 2008 National Comprehensive Center for Teacher Quality, sponsored under government cooperative agreement number S283B050051.

2 All rights reserved. This work was originally produced in whole or in part by the National Comprehensive Center for Teacher Quality with funds from the Department of Education under cooperative agreement number S283B050051. The content does not necessarily reflect the position or policy of the Department of Education, nor does mention or visual representation of trade names, commercial products, or organizations imply endorsement by the federal government. The National Comprehensive Center for Teacher Quality is a collaborative effort of Education Commission of the States, ETS, Learning Point Associates, and Vanderbilt University.

3 3018_08/08 Acknowledgments The authors wish to thank Arie Van der Ploeg, Amy Holcombe, Matthew Springer, Jeffrey Max, Jane Coggshall, Tricia Coulter, Waverely Van Winkle, Sarah Ohls, Carol Dwyer, Sabrina Laine, and Cynthia Tocci for helpful comments and support. Contents Page Rationale and Goals of This Study ..3 Important Definitions and Specifications ..4 Defining Teacher Critiques of the Dominant Teacher Effectiveness Considering a More Comprehensive Definition of Teacher Effectiveness.

4 8 Data Collection and General Approach ..10 Stages of Development ..10 Validity and Considerations in Measuring Teacher Methods of Measuring Teacher Classroom Observations ..20 Principal Evaluations ..26 Analysis of Classroom Artifacts ..28 Self-Reports of Teacher Practice ..35 Students Ratings ..39 Value-Added Toward a Comprehensive View of Teacher Considering Teaching Using Teacher Effectiveness Results to Improve A Final Note About Policy Recommendation and Implications ..52 Appendixes Appendix A.

5 Tools for Measuring Teacher Appendix B. Technical Consideration in Assessing Teacher Effectiveness ..75 Appendix C. Outcomes of Interest in Teacher Evaluation ..78 Appendix D. Comprehensive List of Studies With Summaries ..81 Abstract This Research synthesis examines how Teacher effectiveness is currently measured. By Evaluating the Research on Teacher effectiveness and the different instruments used to measure it, this Research synthesis contributes to the discussion of appropriate rigor and relevance of measures for different purposes ( , formative vs.)

6 Summative evaluation). The findings are presented along with related policy implications. In addition, the synthesis describes how various measures have been evaluated, explains why certain measures are most suitable for certain purposes (high-stakes evaluation vs. formative evaluation, for instance), and suggests how the results of the study might be used to inform the national conversation about Teacher effectiveness. A comprehensive definition of the components and indicators that characterize effective teachers is provided, extending this definition beyond teachers contribution to student achievement gains to include how teachers impact classrooms, schools, and their colleagues as well as how they contribute to other important outcomes for students.

7 Through this synthesis, the National Comprehensive Center for Teacher Quality (TQ Center) hopes to provide some practical guidance in how best to evaluate Teacher effectiveness. National Comprehensive Center for Teacher Quality Approaches to Evaluating Teacher Effectiveness 1 Introduction The process of Evaluating the effectiveness of teachers has changed over time along with the definition of what effective teaching is, due in part to increasing state and federal attention to school -level and classroom-level accountability for student learning.

8 Effective teaching has been defined in many ways throughout the years (Campbell, Kyriakides, Muijs, & Robinson, 2003; Cheng & Tsui, 1999; Cruickshank & Haefele, 1990; Good, 1996; Muijs, 2006), and methods for measuring teachers have changed as definitions and beliefs about what is important to measure have evolved. Although there is a general consensus that good teaching matters and that it may be the single most important school -based factor in improving student achievement (Darling-Hammond, 2000; Wright, Horn, & Sanders, 1997), measuring Teacher effectiveness has remained elusive in part because of ongoing debate about what an effective Teacher is and does.

9 In a discussion of Research -based indicators of effective teaching, Cruickshank and Haefele (1990) stated, An enormous underlying problem with Teacher evaluation relates to lack of agreement about what constitutes good or effective teaching (p. 34). Besides a lack of clear consensus on what an effective Teacher is and does or perhaps because of it there is not a generally agreed-upon method for Evaluating Teacher effectiveness. Commonly used methods include classroom observations designed to measure Teacher practices against some standard of effective teaching and value-added models that set out to measure the contribution of individual teachers to their students achievement gains.

10 This Research synthesis, describes the various ways in which effective teaching can be conceptualized and measured and consists of the following sections: Rationale and Goals of This Study Important Definitions and Specifications Proposal of a Comprehensive Definition of Teacher Effectiveness Data Collection and Methods Validity and Considerations in Measuring Teacher Effectiveness Methods of Measuring Teacher Effectiveness Considering a Comprehensive Measure of Teacher Effectiveness Policy Recommendations and Implications National Comprehensive Center for Teacher Quality Approaches to Evaluating Teacher Effectiveness 2