PISA 2015 Assessment and Analytical Framework: Science, Reading, Mathematic and Financial Literacy

PISA 2015 Assessment
and Analytical Framework

SCIENCE, READING, MATHEMATIC
AND FINANCIAL LITERACY
IS
PA
P r o g r a m m e f o r I n t e r n a t i o n a l S t u d e n t A s s e s s m e n t
PISA
PISA 2015 Assessment

and Analytical Framework
SCIENCE, READING, MATHEMATIC
AND FINANCIAL LITERACY
This work is published under the responsibility of the Secretary-General of the OECD. The
opinions expressed and arguments employed herein do not necessarily reflect the official
views of the OECD member countries.
This document and any map included herein are without prejudice to the status of or
sovereignty over any territory, to the delimitation of international frontiers and boundaries
and to the name of any territory, city or area.
Please cite this publication as:

OECD (2016), PISA 2015 Assessment and Analytical Framework: Science, Reading, Mathematic and Financial
Literacy, PISA, OECD Publishing, Paris.
http://dx.doi.org/10.1787/9789264255425-en
ISBN (print) 978-92-64-25541-8

ISBN (PDF) 978-92-64-25542-5
ISSN (print): 19908539

ISSN (online): 19963777
Photo credits:
© Geostock / Getty Images
© Hero Images Inc. / Hero Images Inc. / Corbis
© LIUSHENGFILM / Shutterstock
© RelaXimages / Corbis
© Shutterstock /Kzenon
© Simon Jarratt/Corbis
Corrigenda to OECD publications may be found on line at: www.oecd.org/about/publishing/corrigenda.htm.

© OECD 2016
You can copy, download or print OECD content for your own use, and you can include excerpts from OECD publications, databases
and multimedia products in your own documents, presentations, blogs, websites and teaching materials, provided that suitable
acknowledgment of the source and copyright owner is given. All requests for public or commercial use and translation rights should
be submitted to rights@oecd.org. Requests for permission to photocopy portions of this material for public or commercial use shall be
addressed directly to the Copyright Clearance Center (CCC) at info@copyright.com or the Centre français d’exploitation du droit de copie
(CFC) at contact@cfcopies.com.
Foreword
The OECD Programme for International Student Assessment (PISA), created in 1997, represents a commitment by the
governments of OECD countries to monitor the outcomes of education systems, in terms of student achievement, within
a common, internationally agreed framework. PISA is a collaborative effort, bringing together scientific expertise from
the participating countries/economies and steered jointly by their governments on the basis of shared policy interests.
Experts from participating countries also serve on working groups that are charged with linking the PISA policy objectives
with the best available substantive and technical expertise in the field of internationally comparable assessments. Through
involvement in these expert groups, countries ensure that the PISA assessment instruments are internationally valid and
take into account the cultural and curricular context of the PISA-participating countries and economies.
PISA 2015 is the sixth cycle of the triennial assessment. For the first time, PISA 2015 delivers the assessments of all subjects –
science, reading, mathematics and financial literacy – via computer. However a paper-based assessment instrument,
consisting only of trend items, is provided for countries/economies that choose not to test their students on computer.
As in 2006, scientific literacy is the main focus of this survey. The framework for assessing science was fully revised for
the PISA 2015 assessment and introduces a refined notion of “knowledge about science” that has been split into two
components – procedural knowledge and epistemic knowledge. In addition, the construct “Support for scientific enquiry”
has been changed to “Valuing scientific approaches to enquiry”, which is essentially a change in terminology to better
reflect what is measured. In addition, the contexts in PISA 2015 have been changed from “personal, social and global”
in the 2006 assessment to “personal, local/national and global” to make the headings more coherent across the domains.
The framework for assessing reading was revised for PISA 2009 while the frameworks for assessing mathematics and
financial literacy were revised for PISA 2012. These frameworks remained unchanged in PISA 2015. The analytic
framework on which the development of the various questionnaires was based was also redeveloped for PISA 2015.
This publication presents the guiding principles behind the PISA 2015 assessment, which are described in terms of
the knowledge and competencies students need to acquire and use to solve scientific problems, the contexts in which
knowledge and competencies are applied, and students’ attitudes towards science. Sample tasks are also included.
The framework for assessing science was developed by the scientific literacy expert group with the guidance of John de
Jong, Rose Clesham, Christine Rozunick, Peter Foltz, Mark Robeck and Catherine Hayes from Pearson. The scientific
literacy expert group was chaired by Jonathan Osborne from Stanford University.
The framework for the PISA 2015 questionnaires was developed by the questionnaire expert group with the guidance of
Eckhard Klieme from the German Institute for International Educational Research (DIPF) in Germany. The questionnaire
expert group was chaired by David Kaplan from the University of Wisconsin, United States. Other experts who contributed
to the development of the questionnaire framework are Sonja Bayer, Jonas Bertling, Bieke de Fraine, Art Graesser,
20163 PISA 2015 ASSESSMENT

© OECD 2016 PISA 2015 ASSESSMENT AND ANALYTICAL
AND ANALYTICAL FRAMEWORK:FRAMEWORK: SCIENCE,MATHEMATIC
SCIENCE, READING, READING, MATHEMATIC AND FINANCIAL LITERACY
AND FINANCIAL © OECD 20163
LITERACY
Silke Hertel, Nina Jude, Franz Klingebiel, Susanne Kuger Patrick Kyllonen, Leonidas Kyriakides, Katharina Müller,
Manfred Prenzel, Christine Sälzer, Tina Seide, Anja Schiepe-Tiska, Svenja Vieluf and Nadine Zeidler.

AND ANALYTICAL FRAMEWORK:
FRAMEWORK: SCIENCE, MATHEMATIC
LITERACY
FOREWORD
The frameworks have also been reviewed by expert panels in each of the participating countries. The chapters were drafted
by the respective expert groups under the direction of their chairs. The members of the expert groups are listed in Annex B.
The publication was prepared by the OECD Secretariat, principally by Sophie Vayssettes, Marilyn Achiron, Sophie Limoges
and Hélène Guillou.
The report is published under the responsibility of the Secretary-General of the OECD.

LITERACY
FOREWORD
Table of Contents
CHAPTER 1 WHAT IS PISA? �� 9

What makes PISA unique �� 10
The PISA 2015 test �� 12
An overview of what is assessed in each domain �� 12
The evolution of reporting student performance in PISA �� 14
The context questionnaires �� 15
A collaborative project �� 16
CHAPTER 2 PISA 2015 SCIENCE FRAMEWORK �� 17

Defining scientific literacy�� 18
• The competencies required for scientific literacy �� 20
• The evolution of the definition of scientific literacy in PISA�� 22
Organising the domain of science �� 23
• Contexts of assessment items �� 23
• Scientific competencies �� 24
• Scientific knowledge �� 26
• Sample items �� 29
• Attitudes �� 36
Assessing scientific literacy �� 38
• Cognitive demand �� 38
• Test characteristics�� 40
• Assessment structure �� 41
• Reporting proficiency in science �� 42
References �� 44
CHAPTER 3 PISA 2015 READING FRAMEWORK �� 47

Defining reading literacy �� 49
Organising the domain of reading �� 50
• Situation �� 51
• Text �� 52
• Aspect �� 54
Assessing reading literacy �� 56
• Factors affecting item difficulty�� 56
• Response formats �� 57
• Coding and scoring �� 57
• Transition from paper-based to computer-based delivery �� 58
• Reporting proficiency in reading �� 58
Note �� 60
CHAPTER 4 PISA 2015 MATHEMATICS FRAMEWORK�� 63

Defining mathematical literacy �� 64
Organising the domain of mathematics �� 65
• Mathematical processes and the underlying mathematical capabilities �� 66
• Mathematical content knowledge �� 70

LITERACY
TABLE OF CONTENTS TABLE OF CONTENTS
Assessing Mathematical literacy �� 75

• Structure of the survey instrument �� 75
• Response formats �� 75
• Item scoring �� 75
• Computer-based assessment of mathematics �� 75
• Reporting proficiency in mathematics �� 76
Note �� 78
CHAPTER 5 PISA 2015 FINANCIAL LITERACY FRAMEWORK �� 79

Policy interest in financial literacy �� 80
• Demographic and cultural shifts �� 80
• Risk shift and increased individual responsibility �� 80
• Greater supply of a wide range of financial products and services �� 81
• Increased demand for financial products and services �� 81
• Expected benefits of financial education and improved levels of financial literacy�� 81
• Focus on youth �� 82
• The need for data �� 83
• Measuring financial literacy in PISA �� 84
Defining financial literacy �� 85
Organising the domain of financial literacy �� 86
• Content �� 87
• Planning and managing finances �� 88
• Risk and reward �� 88
• Processes �� 89
• Contexts�� 90
• Non-cognitive factors �� 92
Assessing financial literacy �� 93
• The structure of the assessment �� 93
• Response formats and coding�� 93
• Distribution of score points �� 94
The impact of other domain knowledge and skills on financial literacy �� 95
Reporting financial literacy �� 96
Notes �� 98
CHAPTER 6 PISA 2015 CONTEXT QUESTIONNAIRES FRAMEWORK �� 101

Defining the core of context assessment in PISA �� 103
• Outline of core content: Constructs to be covered �� 103
• Previous use of PISA context data: Measures that were important for analysis and reporting �� 105
• Selecting and organising the core content �� 105
Expanding the framework for broader coverage of policy issues �� 107
• Modular approach to the PISA design �� 107
• Assessing non-cognitive outcomes �� 108
• Assessing teaching and learning processes �� 111
• Assessing school policies and governance �� 115
• Assessing student background (Modules 7 – 9) �� 118
Notes �� 120
Annex 6.A1 �� 127
ANNEX A PISA 2015 BACKGROUND QUESTIONNAIRES �� 129
ANNEX B PISA 2015 EXPERT GROUPS �� 197

LITERACY
BOXES
Box 1�1 Key features of PISA 2015
�� 11
Box 1�2 2015 mode study

�� 12
Box 1�3 Definitions of the

domains �� 13
Box 2�1 Scientific knowledge: PISA 2015

terminology �� 19
Box 2�2 The 2015 definition of scientific literacy
�� 20
Box 3�1 The 2015 definition of reading literacy

�� 49
Box 4�1 The 2015 definition of mathematical literacy

�� 65
Box 5�1 OECD activities in relation to financial education

�� 82
Box 5�2 The 2015 definition of scientific literacy

�� 85
Box 6�1 Measures based on questionnaires used in PISA 2009 Results: What Students Know and Can
Do �� 105
FIGURE
Figure 2�1 Aspects of the scientific literacy assessment framework for PISA 2015
�� 23
Figure 2�2 Inter-relations between the four aspects

�� 23
Figure 2�3 Contexts in the PISA 2015 scientific literacy assessment

�� 24
Figure 2�4a PISA 2015 scientific competencies: Explain phenomena

scientifically �� 24
Figure 2�4b PISA 2015 scientific competencies : Evaluate and design scientific
enquiry �� 25
Figure 2�4c PISA 2015 scientific competencies: Interpret data and evidence scientifically
�� 25
Figure 2�5 Knowledge of the content of

science�� 26
Figure 2�6 PISA 2015 procedural knowledge
�� 27
Figure 2�7 PISA 2015 epistemic

knowledge �� 28
Figure 2�8 Framework categorisation for GREENHOUSE question
1�� 30
2�� 30
3�� 30

AND ANALYTICAL FRAMEWORK:FRAMEWORK: SCIENCE, MATHEMATIC
LITERACY
Figure 2�11 GREENHOUSE presented onscreen: Stimulus page

1�� 31
Figure 2�12 GREENHOUSE presented onscreen: Stimulus page
2�� 31
Figure 2�13 GREENHOUSE presented onscreen: Question
1 �� 32
2 �� 32
3 �� 33
Figure 2�16 SMOKING: Question 1
�� 33
Figure 2�17 Framework categorisation for SMOKING question 1

�� 34
Figure 2�18 SMOKING: Question 2

�� 34
Figure 2�19 Framework categorisation for SMOKING question 2

�� 35
Figure 2�20 ZEER POT:

Stimulus �� 35
Figure 2�22 Framework categorisation for ZEERPOT question
1�� 36
Figure 2�21 ZEER POT: Question
1 �� 36
Figure 2�23 PISA 2015 Framework for Cognitive Demand
�� 39
Figure 2�24 A tool for constructing and analysing assessment units and
items �� 40
Figure 2�25 Initial draft of proficiency scale descriptions for
science�� 42
Figure 3�1 Relationship between the reading framework and the aspect
subscales �� 54
Figure 3�2 Summary description of the seven levels of reading proficiency in PISA 2015 �� 59

AND ANALYTICAL FRAMEWORK:FRAMEWORK: SCIENCE, MATHEMATIC
LITERACY

PISA 2015 Assessment and Analytical Framework: Science, Reading, Mathematic and Financial Literacy

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

PISA 2015 Assessment and Analytical Framework: Science, Reading, Mathematic and Financial Literacy

Uploaded by

Copyright:

Available Formats

PISA 2015 Assessment

and Analytical Framework

PISA 2015 Assessment

Please cite this publication as:

ISBN (print) 978-92-64-25541-8

ISSN (print): 19908539

Corrigenda to OECD publications may be found on line at: www.oecd.org/about/publishing/corrigenda.htm.

20163 PISA 2015 ASSESSMENT

20164 PISA 2015 ASSESSMENT

20165 PISA 2015 ASSESSMENT

20166 PISA 2015 ASSESSMENT

20167 PISA 2015 ASSESSMENT

Box 1�2 2015 mode study

Box 1�3 Definitions of the

Box 2�1 Scientific knowledge: PISA 2015

Box 3�1 The 2015 definition of reading literacy

Box 4�1 The 2015 definition of mathematical literacy

Box 5�1 OECD activities in relation to financial education

Box 5�2 The 2015 definition of scientific literacy

Figure 2�2 Inter-relations between the four aspects

Figure 2�3 Contexts in the PISA 2015 scientific literacy assessment

Figure 2�4a PISA 2015 scientific competencies: Explain phenomena

Figure 2�5 Knowledge of the content of

Figure 2�7 PISA 2015 epistemic

20168 PISA 2015 ASSESSMENT

Figure 2�11 GREENHOUSE presented onscreen: Stimulus page

Figure 2�17 Framework categorisation for SMOKING question 1

Figure 2�18 SMOKING: Question 2

Figure 2�19 Framework categorisation for SMOKING question 2

Figure 2�20 ZEER POT:

20169 PISA 2015 ASSESSMENT

You might also like