PISA 2012 Assessment and Analytical Framework - …

1 PISA 2012 Assessment and Analytical Framework MAtheMAtIcS, ReAdIng, ScIence, problem solving And FInAncIAl lIteRAcyProgramme for international Student AssessmentPISA 2012 Assessment and Analytical FrameworkMatheMatics, Reading, science, problem solving and Financial liteRacyPISAThe statistical data for Israel are supplied by and under the responsibility of the relevant Israeli authorities. The use of such data by the OECD is without prejudice to the status of the Golan Heights, East Jerusalem and Israeli settlements in the West Bank under the terms of international credits: khoa vu /Flickr /Getty Images Shutterstock /Kzenon Simon Jarratt /CorbisCorrigenda to OECD publications may be found on line at: OECD 2013 You can copy, download or print OECD content for your own use, and you can include excerpts from OECD publications, databases and multimedia products in your own documents, presentations, blogs, websites and teaching materials, provided that suitable acknowledgement of OECD as source and copyright owner is given.

2 All requests for public or commercial use and translation rights should be submitted to Requests for permission to photocopy portions of this material for public or commercial use shall be addressed directly to the Copyright Clearance Center (CCC) at or the Centre fran ais d exploitation du droit de copie (CFC) at work is published on the responsibility of the Secretary-General of the OECD. The opinions expressed and arguments employed herein do not necessarily reflect the official views of the Organisation or of the governments of its member document and any map included herein are without prejudice to the status of or sovereignty over any territory, to the delimitation of international frontiers and boundaries and to the name of any territory, city or cite this publication as: OECD (2013), PISA 2012 Assessment and Analytical Framework : Mathematics, Reading, Science, problem solving and Financial Literacy, OECD 978-92-64-19052-8 (print)ISBN 978-92-64-19051-1 (PDF)ForewordPISA 2012 Assessment AND Analytical Framework OECD 20133 The OECD programme for international Student Assessment (PISA), created in 1997 , represents a commitment by the governments of OECD member countries to monitor the outcomes of education systems in terms of student achievement, within a common internationally agreed Framework .

3 PISA is a collaborative effort, bringing together scientific expertise from the participating countries and steered jointly by their governments on the basis of shared, policy-driven interests. Participating countries take responsibility for the project at the policy level. Experts from participating countries also serve on working groups that are charged with linking the PISA policy objectives with the best available substantive and technical expertise in the field of internationally comparative Assessment . Through involvement in these expert groups, countries ensure that the PISA Assessment instruments are internationally valid and take into account the cultural and curricular context of OECD member countries. They also have strong measurement properties, and place an emphasis on authenticity and educational validity. PISA 2012 represents a continuation of the data strategy adopted in 1997 by OECD countries. As in 2003, mathematical literacy is the focus of the PISA 2012 survey.

4 The Framework for assessing mathematics was fully revised for the PISA 2012 Assessment and introduces three new mathematical processes that form the basis of developments in the reporting of PISA mathematics outcomes. A computer-based Assessment of mathematics was also included in the 2012 cycle. The Framework for assessing science was revised for PISA 2006 while the Framework for assessing reading was revised for PISA 2009. Both of these frameworks remained unchanged in PISA 2012 . The analytic Framework that formed the basis of the development of the various questionnaire instruments was also redeveloped for PISA to the PISA 2012 Assessment include a computer-based Assessment of problem solving and an Assessment of financial literacy. In 2003, problem solving became an Assessment domain in PISA but was not reintroduced in the PISA 2006 and 2009 cycles. However, a new Framework was devised for problem solving in PISA 2012 and additional Assessment methodologies were implemented, allowing for the real-time capture of students capabilities.

5 In particular, the PISA 2012 Assessment of problem solving was computer-based, and interactivity of the student with the problem is a central feature of the Assessment . Financial literacy was included for the first time in the PISA Assessment . Its Framework provides a common language for discussion about financial literacy, a working definition of the domain, an articulated plan for developing items, and defines the relevant content, processes and contexts for the Assessment of 15-year-old students in this publication presents the guiding principles of the PISA 2012 Assessment , which are described in terms of the skills students need to acquire, the processes that need to be performed and the settings in which knowledge and skills are applied. Further, it illustrates the Assessment domains with a range of sample Framework development for mathematics was undertaken jointly by the Australian Council for Educational Research (ACER), and Achieve, Inc., a USA-based educational development organisation.

6 The Framework development for all other cognitive domains, as well as the context questionnaire, was undertaken by the Australian Council for Educational frameworks were developed by the expert panels, with the guidance of Raymond Adams, Barry McCrae, Petra Lietz, Juliette Mendelovits, Dara Ramalingam and Ross Turner from ACER. The mathematics expert group was chaired by Kaye Stacey from the University of Melbourne. The problem solving expert group was chaired by Joachim Funke from the University of Heidelberg. The reading expert group was chaired by Irwin Kirsch of Educational Testing Service in the United States of America. The science expert group was chaired by Rodger Bybee, formerly of the Biological Science Curriculum Study in the United States. The financial literacy expert group was chaired by Annamaria Lusardi of The George Washington University School of Business, in the United States of America. The questionnaire expert group was chaired by Eckhard Klieme of the German Institute for international Educational Research (DIPF) in Germany.

7 The members of the expert groups are listed in Annex B of this publication. The frameworks have also been reviewed by expert panels in each of the participating countries. The chapters were drafted by the respective expert groups under the direction of their chairs. The publication was prepared by the OECD Secretariat, principally by Michael Davidson, Sophie Vayssettes, Pablo Zoido, Giannina Rech, Elisabeth Villoutreix, Marilyn Achiron and Elizabeth Del report is published on the responsibility of the Secretary-General of the of Contents PISA 2012 Assessment AND Analytical Framework OECD 20135 INTRODUCTION ..13 Overview .. 13 Basic features of PISA 2012 .. 14 What makes PISA unique .. 16An overview of what is being assessed in each domain .. 17 Assessing and reporting PISA 2012 .. 18 The context questionnaires and their use .. 20 Collaborative development of PISA and its Assessment Framework .. 20 CHAPTER 1 PISA 2012 MATHEMATICS Framework .. 23 Introduction.

8 24 Defining mathematical literacy .. 24 A view of students as active problem solvers in PISA 2012 .. 25 An explicit link to a variety of contexts for problems in PISA 2012 .. 27 A visible role for mathematical tools, including technology in PISA 2012 .. 27 Organising the domain .. 27 Mathematical processes and the underlying mathematical capabilities .. 28 Mathematical content knowledge .. 31 Contexts .. 37 Assessing mathematical literacy .. 38 Structure of the PISA 2012 mathematics Assessment .. 38 Reporting proficiency in mathematics .. 41 Attitudes towards mathematics .. 42 Optional computer-based Assessment of mathematics .. 43 Summary .. 44 Illustrative PISA mathematics items .. 46 Notes .. 56 References .. 57 CHAPTER 2 PISA 2012 READING Framework .. 59 Introduction .. 60 Defining reading literacy .. 60 Organising the domain .. 62 Situation .. 62 Text ..63 Aspect ..66 Summary of the relationship between printed and digital reading texts and tasks.

9 69 Assessing reading literacy .. 69 Building tasks in the print medium .. 69 Building tasks in the digital medium .. 71 Reporting proficiency in print and digital reading .. 78 Print reading .. 78 Digital reading .. 80 Summary .. 80 TABLE OF CONTENTS6 OECD 2013 PISA 2012 Assessment AND Analytical FRAMEWORKPISA 2012 Assessment AND Analytical Framework OECD 20137 Illustrative PISA print reading items .. 81 Notes .. 94 References .. 95 CHAPTER 3 PISA 2012 SCIENCE Framework .. 97 Introduction ..98 Defining scientific 99 Explanation of the definition .. 100 Organising the domain .. 101 Situations and context .. 102 Illustrative PISA science items .. 103 Scientific competencies .. 107 Scientific knowledge .. 108 Attitudes towards science .. 110 Assessing scientific literacy .. 111 Test characteristics .. 111 Science Assessment structure .. 112 Reporting proficiency in science .. 113 Summary .. 114 Notes .. 116 References .. 117 CHAPTER 4 PISA 2012 problem - solving Framework .

10 119 Introduction .. 120 Background to the 2012 Assessment .. 120 problem solving in the OECD survey of adult skills .. 121 Defining problem - solving competency .. 121 Definition of a problem .. 121 Definition of problem solving .. 122 Definition of problem - solving competency .. 122 Scope of the Assessment .. 123 Organising the domain .. 124 problem context .. 124 Nature of problem situation .. 124 problem - solving processes .. 125 Assessing problem - solving competency .. 127 Structure of the Assessment .. 127 Functionality provided by computer delivery .. 127 Task characteristics and difficulty .. 127 Distribution of items .. 130 Reporting problem - solving competency .. 130 Summary .. 131 Illustrative PISA problem - solving items .. 131 Notes .. 135 References .. 136 CHAPTER 5 PISA 2012 FINANCIAL LITERACY Framework ..139 Introduction .. 140 The importance of financial literacy .. 140 Expected benefits of financial education and improved levels of financial literacy.