TALIS 2018 Technical Report - OECD

1 TALIS 2018 Technical ReportTEACHING AND LEARNING INTERNATIONAL SURVEY 1 TALIS 2018 Technical Report OECD 2020 000 TALIS TALIS 2018 Technical Report PUBE This work is published under the responsibility of the Secretary-General of the OECD. The opinions expressed and arguments employed herein do not necessarily reflect the official views of the OECD member countries. This document, as well as any data and any map included herein, are without prejudice to the status of or sovereignty over any territory, to the delimitation of international frontiers and boundaries and to the name of any territory, city or area.

2 The statistical data for Israel are supplied by and under the responsibility of the relevant Israeli authorities. The use of such data by the OECD is without prejudice to the status of the Golan Heights, East Jerusalem and Israeli settlements in the West Bank under the terms of international law. Note by Turkey The information in this document with reference to Cyprus relates to the southern part of the Island. There is no single authority representing both Turkish and Greek Cypriot people on the Island. Turkey recognises the Turkish Republic of Northern Cyprus (TRNC).

3 Until a lasting and equitable solution is found within the context of the United Nations, Turkey shall preserve its position concerning the Cyprus issue . Note by all the European Union Member States of the OECD and the European Union The Republic of Cyprus is recognised by all members of the United Nations with the exception of Turkey. The information in this document relates to the area under the effective control of the Government of the Republic of Cyprus. Photo credits: Cover Hill Street Studios/Gettyimages You can copy, download or print OECD content for your own use, and you can include excerpts from OECD publications, databases and multimedia products in your own documents, presentations, blogs, websites and teaching materials, provided that suitable acknowledgement of OECD as source and copyright owner is given.

4 All requests for public or commercial use and translation rights should be submitted to Requests for permission to photocopy portions of this material for public or commercial use shall be addressed directly to the Copyright Clearance Center (CCC) at or the Centre fran ais d exploitation du droit de copie (CFC) at 3 TALIS 2018 Technical Report OECD 2019 Acknowledgements The OECD Teaching and Learning International Survey ( TALIS ) is the outcome of collaboration among the participating countries and economies, the OECD Secretariat, the European Commission and an international consortium led by the International Association for the Evaluation of Educational Achievement (IEA).

5 The OECD recognises the significant contributions of members of the IEA Hamburg, Germany, IEA Amsterdam, the Netherlands, as well as Statistics Canada, who authored this Report . The principal authors of the respective chapters were as follows: Chapter 1: Steffen Knoll and Ralph Carstens Chapter 2: John Ainley and Ralph Carstens Chapter 3: Ralph Carstens Chapter 4: Michelle Djeki , David Ebbs, Sandra Dohr, Jan-Philipp Wagner, Steffen Knoll and Viktoria B hm Chapter 5: Jean Dumais and Yves Morin Chapter 6: Viktoria B hm, Alena Becker and Christine Busch Chapter 7: Michelle Djeki , David Ebbs, Sandra Dohr, Jan-Philipp Wagner, Viktoria B hm, Alena Becker and Steffen Knoll Chapter 8.

6 Alena Becker and Christine Busch Chapter 9: Jean Dumais and Yves Morin Chapter 10: Jean Dumais, Alena Becker, Jan-Philipp Wagner, David Ebbs and Steffen Knoll Chapter 11: Agnes Stancel-Pi tak, Justin Wild, Minge Chen, Mojca Rozman, Plamen Mirazchiyski and Hynek Cigler Chapter 12: Ann-Kristin Koop, Falk Brese and Agnes Stancel-Pi tak. Paula Wagemaker copy-edited the Report on behalf of the IEA. OECD Directorate for Education and Skills (EDU) analysts Pablo Fraser, Gabor Ful p and No mie Le Donn provided additional editorial support.

7 Timothy L. Kennel, Bart Meuleman and Christian Monseur provided valuable comments during the development of some chapters. Emily Groves (EDU) provided additional proofreading and managed the layout process, with support from Eleonore Morena, who verified the final layout, and from Henri Pearson (EDU communications team). Emily Groves also provided administrative assistance. We would like to gratefully acknowledge the contribution to TALIS of the late Fons van de Vijver, who was Chair of the Technical Advisory Group and an advisor to TALIS since the first cycle in 2008.

8 4 TALIS 2018 Technical Report OECD 2019 Table of contents Acknowledgements .. 3 Abbreviations and acronyms .. 13 Three-digit operational codes of TALIS 2018 participants .. 15 Executive summary .. 17 Governance, management and structure .. 17 Participants, international options and survey phases .. 18 Framework and questionnaire development, translation .. 18 Sampling of schools and teachers .. 19 Main survey administration and quality observations .. 19 Data collection, processing, weighting and adjudication .. 20 Scaling, analysis and results table production.

9 20 Differences between TALIS 2018 and the 2008 and/or 2013 21 Chapter 1. Introduction .. 22 TALIS in brief .. 23 Participating countries and economies .. 24 Managing the survey internationally .. 26 Working with national project managers .. 27 Standardised procedures .. 29 Key survey phases .. 30 References .. 33 Notes .. 33 Chapter 2. Development of the conceptual framework .. 34 Introduction .. 35 General purpose of TALIS and policy focus for 2018 .. 35 Developing and refining the conceptual framework .. 37 Connecting the conceptual framework with proposed analyses.

10 46 References .. 48 Notes .. 49 Chapter 3. Development of the teacher and principal questionnaires .. 50 General aims and principles .. 51 Timeline .. 54 Questionnaire Expert Group (QEG) role, membership and collaboration .. 56 Pilot phase .. 58 Field trial phase .. 62 Main survey phase .. 68 References .. 78 Notes .. 79 Chapter 4. Preparation of national survey instruments .. 81 Overview of adaptation and translation verification .. 82 Instruments requiring adaptation and translation .. 83 5 TALIS 2018 Technical Report OECD 2019 Identifying the target language.