A systematic review of the impact of summative assessment ...

1 review June 2002 EPPI-Centre A systematic review of the impact of summative assessment and tests on students' motivation for learning review conducted by the assessment and Learning Research Synthesis Group Evidence for Policy and Practice Information and Co-ordinating Centre The EPPI-Centre is part of the Social Science Research Unit, Institute of Education, University of London iiiAUTHORS AND INSTITUTIONAL BASES This work is a review of the assessment and Learning Research Synthesis Group (ALRSG).

2 It was conducted following the procedures for systematic review developed by the EPPI-Centre and in collaboration with David Gough and Dina Kiwan, and with help from other members of the EPPI-Centre education team. Principal authors Wynne Harlen, Graduate School of Education, University of Bristol. Ruth Deakin Crick, Graduate School of Education, University of Bristol. ALRSG members Professor Patricia Broadfoot, University of Bristol Professor Richard Daugherty, University of Wales, Aberystwyth Professor John Gardner, Queen s University, Belfast Professor Wynne Harlen, University of Bristol Dr Mary james , University of Cambridge Dr Gordon Stobart, Institute of Education, University of London (The above are also members of the assessment review Group.)

3 Mr P Dudley, Head of School Improvement and Lifelong Learning, Redbridge, and member of AAIA Mr R Bevan, Deputy Headteacher, king Edward VI Grammar School, Chelmsford Ms P Rayner, Headteacher, Caldecote Primary School, Cambridge Dr Ruth Deakin Crick, Researcher EPPI-Centre members Dr David Gough, Deputy Director Ms Dina Kiwan, Education Research Officer Expert advisers to the ALRSG The ALRSG is advised by the following international experts: Dr Steven Bakker, ETS International, The Netherlands Dr Dennis Bartels, President, TERC, Cambridge, MA.

4 USA Professor Lorrie Shepard, President, AERA, 1999-2000, University of Colorado Professor Eva Baker, co-director of CRESST, University of California, USA Dr T Crooks, Director, EARM, University of Otago, Dunedin, New Zealand Thanks The authors would like to thank the above and members of the EPPI-Centre team for their guidance and support during the conduct of the review . ivAcknowledgements This review was carried out with funding from the Nuffield Foundation and from the EPPI-Centre (Evidence for Policy and Practice Information and Coordinating Centre).

5 Submission date: 30/05/2002 This report should be cited as: Harlen W, Deakin Crick R (2002). A systematic review of the impact of summative assessment and tests on students' motivation for learning (EPPI-Centre review , version *). In: Research Evidence in Education Library. Issue 1. London: EPPI-Centre, Social Science Research Unit, Institute of Education. * This review has been updated since it was published originally. This update involves minor amendments only and has not changed the substantive findings of the review .

6 Copyright Authors of the systematic reviews on the EPPI-Centre Website ( ) hold the copyright for the text of their reviews. The EPPI-Centre owns the copyright for all material on the Website it has developed, including the contents of the databases, manuals, and keywording and data extraction systems. The Centre and authors give permission for users of the site to display and print the contents of the site for their own non-commercial use, providing that the materials are not modified, copyright and other proprietary notices contained in the materials are retained, and the source of the material is cited clearly following the citation details provided.

7 Otherwise users are not permitted to duplicate, reproduce, re-publish, distribute, or store material from this Website without express written permission. vTABLE OF CONTENTS 1 1. BACKGROUND TO THE review The increase in summative assessment and 9 assessment and raising 9 impact on students and 10 Motivation for 11 Relationship between assessment and aspects of 14 Differential impact on minority 15 2. AIMS OF THE review AND review QUESTION Aims of the 17 review 18 3.

8 IDENTIFYING AND DESCRIBING STUDIES: METHODS Applying inclusion and exclusion 19 Methods for identifying 21 Methods for characterising included studies: 21 Methods for quality 21 4. IDENTIFYING AND DESCRIBING STUDIES: RESULTS Numbers of hits and studies included at each 22 Characteristics of included 23 5. IN-DEPTH review : METHODS Moving from broad characterisation to in-depth 26 Methods for extracting data and evaluating weight to be given to 26 Methods for synthesising 28 28 Methods for quality 29 6.

9 IN-DEPTH review : RESULTS Description of included 31 Synthesis across studies: overall research 33 Synthesis across studies: the subsidiary review 48 7. DISCUSSION Re-statement of principal 61 Strengths and weaknesses of the 65 Relationship with other 68 Meaning of the review for different user 69 Unanswered 69 8. CONCLUSIONS AND RECOMMENDATIONS Conclusions and recommendations: assessment practice and 70 Conclusions and recommendations: assessment 72 vi9.

10 REFERENCES Included 75 Excluded 76 Other references used in the 87 APPENDIX A: Search 90 APPENDIX B: Keywords .. 94 APPENDIX C: Summary of extracted 96 APPENDIX D: Conference 140 Summary A systematic review of the impact of summative assessment and tests on students' motivation for learning 1 SUMMARY Background The current widespread use of summative assessment and tests is supported by a range of arguments. The points made include that not only do tests indicate standards to be aimed for and enable these standards to be monitored, but that they also raise standards.