PISA 2009 Assessment Framework - OECD

1 PISA 2009 Assessment FrameworkKey competencies in reading, mathematics and science Programme for International Student Assessment3 PISA 2009 Assessment Framework KEY COMPETENCIES IN READING, MATHEMATICS AND SCIENCE OECD 2009 ForewordThe OECD Programme for International Student Assessment (PISA), created in 1997, represents a commitment by the governments of OECD member countries to monitor the outcomes of education systems in terms of student achievement, within a common internationally agreed Framework . PISA is a collaborative effort, bringing together scientific expertise from the participating countries and steered jointly by their governments on the basis of shared, policy-driven interests. Participating countries take responsibility for the project at the policy level. Experts from participating countries also serve on working groups that are charged with linking the PISA policy objectives with the best available substantive and technical expertise in the field of internationally comparative Assessment .

2 Through involvement in these expert groups, countries ensure that the PISA Assessment instruments are internationally valid and take into account the cultural and curricular context of OECD member also have strong measurement properties, and place an emphasis on authenticity and educational validity. PISA 2009 represents a continuation of the data strategy adopted in 1997 by OECD countries. As in 2000, reading literacy is the focus of the PISA 2009 survey, but the reading Framework has been updated and now also includes the Assessment of reading of electronic texts. The Framework for assessing mathematics was fully developed for the PISA 2003 Assessment and remained unchanged in 2009. Similarly, the Framework for assessing science was fully developed for the PISA 2006 Assessment and remained unchanged in 2009. This publication presents the guiding principles of the PISA 2009 Assessment , which are described in terms of the skills students need to acquire, the processes that need to be performed and the contexts in which knowledge and skills are applied.

3 Further, it illustrates the Assessment domains with a range of sample tasks. These have been developed by expert panels under the direction of Raymond Adams, Juliette Mendelovits, Ross Turner and Barry McCrae from the Australian Council for Educational Research (ACER) and Henk Moelands (CITO). The reading expert group was chaired by Irwin Kirsch of Educational Testing Service in the United States. The mathematics expert group was chaired by Jan de Lange of the University of Utrecht in the Netherlands and the science expert group was chaired by Rodger Bybee of the Biological Science Curriculum Study in the United States. The questionnaire expert group was chaired by Jaap Scheerens of University of Twente in the Netherlands. The members of the expert groups are listed in Annex C of this publication. The frameworks have also been reviewed by expert panels in each of the participating countries. The chapters on reading, mathematics and science were drafted by the respective expert groups under the direction of their chairs, Irwin Kirsch (reading), Jan de Lange (mathematics) and Rodger Bybee (science).

4 The chapter on the questionnaire Framework was drafted by Henry Levin of Teachers College, Columbia University, New York, and is based on a review of central issues, addressed in conceptual papers for the PISA Governing Board, prepared by Jaap Scheerens in collaboration with the questionnaire expert group. The publication was prepared by the OECD Secretariat, principally by Andreas Schleicher, Karin Zimmer, Juliet Evans and Niccolina Clements. The report is published on the responsibility of the Secretary-General of the OECD. Table of conTenTs5 PISA 2009 Assessment Framework KEY COMPETENCIES IN READING, MATHEMATICS AND SCIENCE OECD 2009 Table of ContentsForeword ..3 Executive Summary ..9 Basic features of PISA 2009 ..11 What makes PISA unique ..13An overview of what is being assessed in each domain ..13 Assessing and reporting PISA 2009 ..15 The context questionnaires and their use ..16 Collaborative development of PISA and its Assessment 1 PISA 2009 Reading Framework .

5 19 Introduction ..20 Continuity and change in the reading literacy Framework ..20 The structure of the reading literacy Framework ..20 Reading literacy as a foundational skill ..21 The importance of electronic texts ..22 Motivational and behavioural elements of reading literacy ..22 Defining reading literacy ..23 Organising the domain ..25 Situation ..25 Text ..27 Aspect ..34 Summary of the relationship between printed and electronic reading texts and tasks ..43 Assessing reading literacy ..45 Building tasks in the print medium ..45 Building tasks in the electronic medium ..60 Motivational and behavioural constituents of reading literacy ..69 Reading engagement ..69 Metacognition in reading ..72 Reporting proficiency in reading ..75 Interpreting and using the data ..75 Reporting PISA 2009 reading literacy ..76 Conclusion ..78 References ..80 Table of conTenTs6 PISA 2009 Assessment Framework KEY COMPETENCIES IN READING, MATHEMATICS AND SCIENCE OECD 2009 CHAPTER 2 PISA 2009 Mathematics Framework .

6 83 Introduction ..84 Definition of the domain ..84 Theoretical basis for the PISA mathematics Framework ..85 Organisation of the domain ..90 Situations and context ..91 Mathematical content the four overarching ideas ..93 Mathematical processes ..105 Assessing mathematics in PISA ..116 Task characteristics ..116 Assessment structure ..119 Aids and tools ..120 Reporting proficiency in mathematics ..120 Conclusion ..122 References ..123 CHAPTER 3 PISA 2009 Science Framework ..125 Introduction ..126 Defining the domain ..127 Scientific literacy ..128 Organising the domain ..129 Situations and context ..130 Scientific competencies ..137 Scientific knowledge ..138 Attitudes towards science ..141 Assessing Science in PISA ..141 Test characteristics ..141 Science Assessment structure ..142 Reporting proficiency in science ..145 Conclusion ..146 References ..148 Table of conTenTs7 PISA 2009 Assessment Framework KEY COMPETENCIES IN READING, MATHEMATICS AND SCIENCE OECD 2009 CHAPTER 4 PISA 2009 questionnaire Framework .

7 149 Introduction ..150 Types of background information and their purposes ..151 Educational system as a whole ..153 School level ..155 Instructional settings ..158 Student level ..161 Contents of the questionnaires ..162 School Student questionnaire ..163 Parent questionnaire (international option) ..163 questionnaire on educational career (international option) ..164 questionnaire on student familiarity with ICT (international option) ..164 Information for in-depth investigations ..164 System level indicators ..165 Effective learning environments in reading ..166 School effectiveness and school management ..167 Educational equity ..168 References ..170 ANNEX A1: Print reading sample tasks ..173 ANNEX A2: Electronic reading sample tasks ..233 ANNEX B: Background questionnaires ..249 ANNEX C: PISA expert groups ..289 InTroducTIon9 PISA 2009 Assessment Framework KEY COMPETENCIES IN READING, MATHEMATICS AND SCIENCE OECD 2009 Executive SummaryParents, students, teachers, governments and the general public all stakeholders - need to know how well their education systems prepare students for real-life situations.

8 Many countries monitor students learning to evaluate this. Comparative international assessments can extend and enrich the national picture by providing a larger context within which to interpret national performance. They can show what is possible in education, in terms of the quality of educational outcomes as well as in terms of equity in the distribution of learning opportunities. They can support setting policy targets by establishing measurable goals achieved by other systems and help to build trajectories for reform. They can also help countries work out their relative strengths and weaknesses and monitor progress. In response to the need for cross-nationally comparable evidence on student performance, the Organisation for Economic Co-operation and Development (OECD) launched the OECD Programme for International Student Assessment (PISA) in 1997. PISA represents a commitment by governments to monitor the outcomes of education systems through measuring student achievement on a regular basis and within an internationally agreed common Framework .

9 It aims to provide a new basis for policy dialogue and for collaboration in defining and implementing educational goals, in innovative ways that reflect judgements about the skills that are relevant to adult life. PISA is a collaborative effort undertaken by its participants the OECD member countries as well as over 30 non-member partner economies to measure how well students, at age 15, are prepared to meet the challenges they may encounter in future life. Age 15 is chosen because at this age students are approaching the end of compulsory education in most OECD countries. PISA, jointly guided by the participating governments, brings together the policy interests of countries with scientific expertise at both national and international levels. PISA has been measuring the knowledge, skills and attitudes of 15-year-olds over the last ten years and is therefore able to give some insight into how countries are faring over time. The PISA Assessment takes a broad approach to measuring knowledge, skills and attitudes that reflect current changes in curricula, moving beyond the school-based approach towards the use of knowledge in everyday tasks and challenges.

10 It is based on a dynamic model of lifelong learning in which new knowledge and skills necessary for successful adaptation to a changing world are continuously acquired throughout life. PISA focuses on things that 15-year-old students will need in the future and seeks to assess what they can do with what they have learned reflecting the ability of students to continue learning throughout their lives by applying what they learn in school to non-school environments, evaluating their choices and making decisions. The Assessment is informed, but not constrained, by the common denominator of national curricula. Thus, while it does assess students knowledge, PISA also examines their ability to reflect, and to apply their knowledge and experience to real-life issues. For example, in order to understand and evaluate scientific advice on food safety an adult would need not only to know some basic facts about the composition of nutrients, but also to be able to apply that information.