I SA - OECD.org

1 PISA 2018 DRAFT ANALYTICAL FRAMEWORKS MAY 2016 P 2 I S A 0 1 8 1 TABLE OF CONTENTS I. PISA 2018 DRAFT READING LITERACY FRAMEWORK _____ # INTRODUCTION _____ # A. Reading as the major domain _____ # B. Changes in the nature of reading literacy _____ # C. The continuity and change in the framework from 2000 to 2015 _____ # D. Revising the framework for PISA 2018 _____ # E. The importance of digital reading literacy _____ # F. Reading motivation, practices and metacognition _____ # G. The structure of the reading literacy framework _____ # 1. DEFINING READING LITERACY _____ # 2. ORGANISING THE DOMAIN _____ # A. Processes _____ # B. Texts _____ # 3.

2 ASSESSING READING LITERACY _____ # A. Scenarios _____ # B. Tasks _____ # C. Factors affecting item difficulty _____ # D. Factors improving the coverage of the domain _____ # E. Responses formats _____ # F. Assessing the ease and efficiency of reading simple texts for understanding _____ # G. Assessing students' reading motivation, reading practices and awareness of reading strategies _____ # H. Considerations for adaptive testing _____ # 4. REPORTING PROFICIENCY IN READING _____ # A. Reporting scales _____ # B. Interpreting and using the scales _____ # C. Defining levels of reading literacy proficiency _____ # APPENDIX A. MAIN CHANGES IN THE READING FRAMEWORK, 2000-2015 _____ # APPENDIX B.

3 SAMPLE TASKS _____ # REFERENCES _____ # 2 II. FINANCIAL LITERACY FRAMEWORK _____ # INTRODUCTION _____ # A. Policy interest in financial literacy _____ # B. Expected benefitsof financial education and improved levels of financial literacy _____ # C. The need for data _____ # D. The measurement of financial literacy in PISA _____ # 1. DEFINING FINANCIAL LITERACY _____ # 2. ORGANISING THE DOMAIN _____ # A. Content _____ # B. Processes _____ # C. Contexts _____ # D. Policy interest in financial literacy _____ # 3. ASSESSING FINANCIAL LITERACY _____ # A. The structure of the assessment _____ # B. Response formats and coding _____ # C. Distribution of score points _____ # D.

4 The impact of other domain knowledge and skills on financial literacy _____ # 4. REPORTING FINANCIAL LITERACY _____ # NOTES _____ # REFERENCES _____ # III. PISA 2018 DRAFT QUESTIONNAIRE FRAMEWORK _____ # INTRODUCTION _____ # A. Aims of the Programme for International Student Assessment _____ # B. PISA 2018 _____ # C. Outline of the Framework _____ # 1. DEFINING THE QUESTIONNAIRE CORE IN PISA 2018 _____ # A. Outline of content covered in questionnaires: a) general framework of the 2018 study and modular approach, b)constructs to be covered, c) global competence _____ # B. Previous use of PISA context data: Measures that have been and will be important for analysis and reporting _____ # 3 2.

5 COVERAGE OF POLICY ISSUES FOR 2018 _____ # A. Assessing non-cognitive and metacognitive constructs _____ # B. Assessing student background _____ # C. Assessing teaching and learning processes _____ # D. Assessing school policies and governance _____ # E. Dealing with response bias _____ # REFERENCES _____ # IV. FRAMEWORK DEVELOPED FOR PISA 2018 : MEASURING ADOLESCENT WELL-BEING IN PISA _____ # EXECUTIVE SUMMARY _____ # INTRODUCTION _____ # 1. WELL-BEING AS A MULTI-DIMENSIONAL CONSTRUCT _____ # 2. ADDRESSING MEASUREMENT CHALLENGES _____ # A. Balance single-item measures with multi-item indices _____ # B. Use meaningful number of scale points and clearly distinguishable scale labels _____ # C.

6 Select measures that maximize cross-cultural comparability _____ # D. Consider item formats beyond traditional self-reports _____ # E. Consider alternative questionnaire designs to reduce respondent burden _____ # 3. SUGGESTED QUALITY OF LIFE INDICATORS _____ # A. Quality of Life as a Whole _____ # B. Self-related well-being _____ # C. School-environment: School-related well-being _____ # D. Out-of school environment: Well-being outside of school _____ # E. Possible composite indicators _____ # REFERENCES _____ # APPENDIX: DETAILED ITEM-LEVEL ASSESSMENT PLAN _____ # BOXES Box 1. The definition of reading literacy in earlier PISA cycles _____ # Box 2. Characteristics used to classify texts in the PISA 2009 reading framework _____ # Box 3.

7 Text availability and its impact on comprehension in the design of tasks_____ # Box 4. Categorisation of situations _____ # Box 5. Classification of texts _____ # Box 6. The status of writing skills in PISA 2018 reading literacy assessment _____ # Box 7. OECD activities in relation to financial education _____ # 4 FIGURES Figure 1. Reading Literacy Sources of Influence _____ # Figure 2. PISA 2018 Reading Framework Processes _____ # Figure 3. A model for organising the domain for an assessment framework _____ # Figure 4. Item for the unit AT THE MARKET _____ # Figure 5. Item for the unit SPENDING CHOICES _____ # Figure 6. Item for the unit MOTORBIKE INSURANCE _____ # Figure 7.

8 Item for the unit BANK ERROR _____ # Figure 8. Item for the unit PAY SLIP _____ # Figure 9. Item for the unit SHARES _____ # Figure 10. Item for the unit AT THE MARKET _____ # Figure 11. Item for the unit NEW OFFER _____ # Figure 12. Item for the unit TRAVEL MONEY _____ # Figure 13. Item for the unit NEW BANK CARD _____ # Figure 14. Relationship between the content of financial literacy and mathematical literacy in PISA _____ # Figure 15. PISA 2018 Questionnaire Modules _____ # Figure 16. Framework Overview _____ # Figure 17. Proposed PISA 2018 Well-being Framework _____ # TABLES Table 1. Mapping of 2018 process typology to 2018 reporting scales and to former 2009-2015 cognitive aspects _____ # Table 2.

9 Approximate distribution of tasks per targeted process and text availability _____ # Table 3. Item difficulty for tasks _____ # Table 4. An overview of reading proficiency levels as they were described in the PISA 2012 study _____ # Table 5. Approximate distribution of score points in financial literacy, by content _____ # Table 6. Approximate distribution of score points in financial literacy, by processes _____ # Table 7. Approximate distribution of score points in financial literacy, by contexts _____ # 5 I. PISA 2018 DRAFT READING LITERACY FRAMEWORK INTRODUCTION A. Reading as the major domain PISA 2018 marks the third time reading is a major domain and the third time that the framework receives a major revision.

10 Such a revision must reflect the changing definition of reading literacy as well as the contexts in which reading is used in citizens lives. Thus, the present revision of the framework builds on contemporary and comprehensive theories of reading literacy as well as considers how students acquire and use information across broad contexts. We live in a rapidly changing world, in which both the quantity and variety of written materials are increasing and where more and more people are expected to use these materials in new and increasingly complex ways. It is now generally accepted that our understanding of reading literacy evolves along with changes in society and culture.