I SA - OECD.org

1 PISA 2018 DRAFT ANALYTICAL FRAMEWORKS MAY 2016 P 2 I S A 0 1 8 1 TABLE OF CONTENTS I. PISA 2018 DRAFT READING LITERACY FRAMEWORK _____ # INTRODUCTION _____ # A. Reading as the major domain _____ # B. Changes in the nature of reading literacy _____ # C. The continuity and change in the framework from 2000 to 2015 _____ # D. Revising the framework for PISA 2018 _____ # E. The importance of digital reading literacy _____ # F. Reading motivation, practices and metacognition _____ # G. The structure of the reading literacy framework _____ # 1. DEFINING READING LITERACY _____ # 2.

2 ORGANISING THE DOMAIN _____ # A. Processes _____ # B. Texts _____ # 3. ASSESSING READING LITERACY _____ # A. Scenarios _____ # B. Tasks _____ # C. Factors affecting item difficulty _____ # D. Factors improving the coverage of the domain _____ # E. Responses formats _____ # F. Assessing the ease and efficiency of reading simple texts for understanding _____ # G. Assessing students' reading motivation, reading practices and awareness of reading strategies _____ # H. Considerations for adaptive testing _____ # 4. REPORTING PROFICIENCY IN READING _____ # A. Reporting scales _____ # B. Interpreting and using the scales _____ # C.

3 Defining levels of reading literacy proficiency _____ # APPENDIX A. MAIN CHANGES IN THE READING FRAMEWORK, 2000-2015 _____ # APPENDIX B. SAMPLE TASKS _____ # REFERENCES _____ # 2 II. FINANCIAL LITERACY FRAMEWORK _____ # INTRODUCTION _____ # A. Policy interest in financial literacy _____ # B. Expected benefitsof financial education and improved levels of financial literacy _____ # C. The need for data _____ # D. The measurement of financial literacy in PISA _____ # 1. DEFINING FINANCIAL LITERACY _____ # 2. ORGANISING THE DOMAIN _____ # A. Content _____ # B. Processes _____ # C.

4 Contexts _____ # D. Policy interest in financial literacy _____ # 3. ASSESSING FINANCIAL LITERACY _____ # A. The structure of the assessment _____ # B. Response formats and coding _____ # C. Distribution of score points _____ # D. The impact of other domain knowledge and skills on financial literacy _____ # 4. REPORTING FINANCIAL LITERACY _____ # NOTES _____ # REFERENCES _____ # III. PISA 2018 DRAFT QUESTIONNAIRE FRAMEWORK _____ # INTRODUCTION _____ # A. Aims of the Programme for International Student Assessment _____ # B. PISA 2018 _____ # C. Outline of the Framework _____ # 1. DEFINING THE QUESTIONNAIRE CORE IN PISA 2018 _____ # A.

5 Outline of content covered in questionnaires: a) general framework of the 2018 study and modular approach, b)constructs to be covered, c) global competence _____ # B. Previous use of PISA context data: Measures that have been and will be important for analysis and reporting _____ # 3 2. COVERAGE OF POLICY ISSUES FOR 2018 _____ # A. Assessing non-cognitive and metacognitive constructs _____ # B. Assessing student background _____ # C. Assessing teaching and learning processes _____ # D. Assessing school policies and governance _____ # E. Dealing with response bias _____ # REFERENCES _____ # IV.

6 FRAMEWORK DEVELOPED FOR PISA 2018 : MEASURING ADOLESCENT WELL-BEING IN PISA _____ # EXECUTIVE SUMMARY _____ # INTRODUCTION _____ # 1. WELL-BEING AS A MULTI-DIMENSIONAL CONSTRUCT _____ # 2. ADDRESSING MEASUREMENT CHALLENGES _____ # A. Balance single-item measures with multi-item indices _____ # B. Use meaningful number of scale points and clearly distinguishable scale labels _____ # C. Select measures that maximize cross-cultural comparability _____ # D. Consider item formats beyond traditional self-reports _____ # E. Consider alternative questionnaire designs to reduce respondent burden _____ # 3.

7 SUGGESTED QUALITY OF LIFE INDICATORS _____ # A. Quality of Life as a Whole _____ # B. Self-related well-being _____ # C. School-environment: School-related well-being _____ # D. Out-of school environment: Well-being outside of school _____ # E. Possible composite indicators _____ # REFERENCES _____ # APPENDIX: DETAILED ITEM-LEVEL ASSESSMENT PLAN _____ # BOXES Box 1. The definition of reading literacy in earlier PISA cycles _____ # Box 2. Characteristics used to classify texts in the PISA 2009 reading framework _____ # Box 3. Text availability and its impact on comprehension in the design of tasks_____ # Box 4.

8 Categorisation of situations _____ # Box 5. Classification of texts _____ # Box 6. The status of writing skills in PISA 2018 reading literacy assessment _____ # Box 7. OECD activities in relation to financial education _____ # 4 FIGURES Figure 1. Reading Literacy Sources of Influence _____ # Figure 2. PISA 2018 Reading Framework Processes _____ # Figure 3. A model for organising the domain for an assessment framework _____ # Figure 4. Item for the unit AT THE MARKET _____ # Figure 5. Item for the unit SPENDING CHOICES _____ # Figure 6. Item for the unit MOTORBIKE INSURANCE _____ # Figure 7.

9 Item for the unit BANK ERROR _____ # Figure 8. Item for the unit PAY SLIP _____ # Figure 9. Item for the unit SHARES _____ # Figure 10. Item for the unit AT THE MARKET _____ # Figure 11. Item for the unit NEW OFFER _____ # Figure 12. Item for the unit TRAVEL MONEY _____ # Figure 13. Item for the unit NEW BANK CARD _____ # Figure 14. Relationship between the content of financial literacy and mathematical literacy in PISA _____ # Figure 15. PISA 2018 Questionnaire Modules _____ # Figure 16. Framework Overview _____ # Figure 17. Proposed PISA 2018 Well-being Framework _____ # TABLES Table 1.

10 Mapping of 2018 process typology to 2018 reporting scales and to former 2009-2015 cognitive aspects _____ # Table 2. Approximate distribution of tasks per targeted process and text availability _____ # Table 3. Item difficulty for tasks _____ # Table 4. An overview of reading proficiency levels as they were described in the PISA 2012 study _____ # Table 5. Approximate distribution of score points in financial literacy, by content _____ # Table 6. Approximate distribution of score points in financial literacy, by processes _____ # Table 7. Approximate distribution of score points in financial literacy, by contexts _____ # 5 I.