Mathematics Education Research Journal (2019) 31:175–195

https://doi.org/10.1007/s13394-018-0248-4
ORIGINAL ARTICLE

The different mathematics performances in PISA 2012

and a curricula comparison: enriching the comparison by an
analysis of the role of problem solving in intended
learning processes

Safrudiannur 1,2 & Benjamin Rott 2

Received: 12 July 2017 / Revised: 5 June 2018 / Accepted: 10 September 2018 /

Published online: 17 September 2018
# Mathematics Education Research Group of Australasia, Inc. 2018

Abstract
The results of international comparative studies like PISA and TIMSS attract re-
searchers to conduct studies comparing math contents of curricula directly to each
other. Unlike those studies, we compared math contents of Indonesian and Singaporean
curricula based on math contents tested in PISA 2012 items. We also enrich the
comparison with the examination of the role of problem solving in the intended
learning processes of the two curricula. The results of the comparison of math contents
show that there are differences in the breadth of the math contents of the two curricula.
The results also indicate that the Singaporean curriculum covers math contents tested in
the PISA 2012 items which are missing in the Indonesian curriculum. However, it is
still difficult to identify possible reasons for the distinct performances between Indo-
nesia and Singapore in the PISA 2012 study from the comparison. Enriching the
comparison with the examination of the role of problem solving in the intended
learning processes of the two curricula gives us additional insight into a possible
reason. The results show that the intended learning process of the Singaporean curric-
ulum highlights problem solving much more than does that of the Indonesian curric-
ulum. These results are suggestive of the need for further research.

Keywords Comparative study . Mathematics curricula . PISA 2012 . Problem solving

* Safrudiannur
safrudiannur.safrudiannur@uni-koeln.de

1
Department of Mathematics Education, Faculty of Teacher Training and Education, Mulawarman
University, Kampus Gn. Kelua, Samarinda 75119, Indonesia
2
Institute of Mathematics Education, Faculty of Mathematics and Sciences, University of Cologne,
Gronewaldstr. 2, D-50931 Cologne, Germany
176 Safrudiannur, B. Rott

Introduction and background

In recent years, international comparison studies, especially the Programme for Inter-
national Student Assessment (PISA) and Trends in International Mathematics and
Science Study (TIMSS), receive a lot of attention—scientifically as well as politically
(ACARA 2017). The results of those comparative studies categorise their participating
countries as low-, mid-, or high-performing. The differences in performances attract
researchers to do studies comparing curricula with different purposes. For example,
Jane et al. (2011), Ruddock et al. (2008), as well as the Department for Education
(2012), investigated the alignment between mathematics contents of a curriculum and
other curricula of high-performing countries to improve the curriculum. Another
purpose is to find a possible explanation for the different performances between
countries, see, for example, Ibrahim and Othman (2010).
Ibrahim and Othman (2010) compared mathematics curricula of Singapore and
Malaysia to find possible reasons for the different achievements of Singaporean and
Malaysian students in the TIMSS tests from 1995 to 2007. In their approach, they
focused on examining the correspondence of the mathematics contents of two
curricula. They compared both curricula directly to each other without aligning
their contents with the mathematics contents tested in the TIMSS studies. In doing
so, they found that the curriculum of Malaysia covered all mathematics topics that
were in the curriculum of Singapore. Moreover, they found that the curriculum of
Malaysia included more content, namely mathematical reasoning, which the
curriculum of Singapore did not. Nevertheless, the TIMSS results showed that the
mathematics performances of Singaporean students were higher than those of
Malaysian students, which could not be explained by the curricula comparison.
We argue that the contrast between the results of the study by Ibrahim and Othman
(2010) and the TIMSS results indicates the following: It is necessary to investigate
the correspondence of mathematics contents in curricula and those tested in
international comparison studies such as TIMSS or PISA. Kiselova and Kangro
(2009) argue that examining the correspondence may help us to find possible
explanations regarding students’ achievement in the PISA or TIMSS tests.
Furthermore, the negative result of the study by Ibrahim and Othman also indicates
that a curricula comparison should not focus only on mathematics contents. A math-
ematics curriculum usually consists of a set of processes in addition to a list of
mathematics contents (Anderson 2009). Since both TIMSS and PISA examine how
good a student solves mathematical problems, Anderson (2009) and the Department for
Education (2012) suggest that it is essential to look at the role of problem solving in the
intended learning processes of curricula.
We conduct a curricula comparison as a part of a study to find possible reasons for
the different mathematics performances of countries in international comparison stud-
ies, particularly in the PISA 2012 study. We chose to compare mathematics curricula of
Indonesia and Singapore. Comparing those two countries promises to be very interest-
ing as they had very different mathematics performances in all PISA studies in which
they participated (see Fig. 1), even though Indonesia is geographically very close to
Singapore.
Unlike other curricula comparison studies, which compare curricula directly to each
other, we examine the alignment between mathematics contents of the curricula of
The different mathematics performances in PISA 2012 and a curricula... 177

OECD Indonesia Singapore

562 573 564
600 500 500 498 496 494 490
367 360 391 371 375 386
400
Score

200

0
2000 2003 2006 PISA 2009 2012 2015

Fig. 1 Average scores of mathematics performances of Indonesia and Singapore in PISA studies (summarised
from OECD 2013a, 2013b, 2004, 2007, 2010, 2014, 2016, respectively). Singapore participates in PISA from
2009

Indonesia and Singapore based on contents tested in the PISA 2012 study. Moreover,
we enrich the comparison of mathematics contents with the examination of the role of
problem solving in the intended learning processes of the two curricula (see Fig. 2). The
reason why we chose the PISA 2012 study is elaborated further in the section below.

The comparison of performances from Indonesia and Singapore in international

comparison studies

The results of the PISA studies from 2000 to 2015 on the mathematics part show that
Indonesian students have always performed very poorly. For example, the 15-year-old
Indonesian students’ mean score in the PISA 2012 study is ranked second to last in
comparison to 64 other countries (OECD 2014). In PISA 2015 (OECD 2016), the mean
score is better but still significantly below the mean value (for results from other years,
see Fig. 1). Kementerian Pendidikan dan Kebudayaan Indonesia (the Ministry of
Education and Culture of Indonesia) offers an explanation for these results by referring
to the contents of the mathematics curriculum of Indonesia:

The Indonesian participation in the TIMSS and PISA studies since 1999 shows
that the performances of Indonesian students are unsatisfying. It happens because
lots of contents tested in PISA and TIMSS do not exist in the Indonesian
Curriculum. (Kementerian Pendidikan dan Kebudayaan Indonesia 2014, p. 2,
translated by the first author)

Mathematics contents of PISA 2012 items

Intended mathematics contents Intended mathematics contents

The curriculum of Singapore The curriculum of Indonesia

Intended learning processes Intended learning processes

The role of problem solving

Fig. 2 The framework of this curricula comparison study

178 Safrudiannur, B. Rott

The claim of the ministry indicates that one of the reasons for the poor achievements of
Indonesian students in the TIMSS and PISA studies is that the Indonesian curriculum
omits certain mathematics contents that are tested in TIMSS or PISA items.
Contrary to this claim, TIMSS is designed to closely match the curricula of
participating countries (Mullis et al. 2004, 2008). The Test-Curriculum Matching
Analysis (TCMA) on the mathematics part for grade 8 was conducted for both TIMSS
2003 and 2007. The results indicate that there was a very close match between
mathematics contents of the Indonesian curriculum with mathematics contents of the
TIMSS 2003 test (209 of 213 score points, see Mullis et al. 2004) and the TIMSS 2007
test (236 of 236 score points, see Mullis et al. 2008). While the TCMA results for
Singapore indicate a slightly lower curriculum fit in TIMSS 2003 (206 of 213 score
points, see Mullis et al. 2004) and 2007 (221 of 236 score points, see Mullis et al.
2008). However, Singaporean students had higher performances in TIMSS 2003 and
2007 (with mean scores of 605 and 593, respectively) than did Indonesian students
(with mean scores of 411 and 397, respectively). The achievements of Indonesian
students in TIMSS 2003 and 2007 indicate that a curriculum covering mathematics
contents in the TIMSS tests is not a guarantee for getting high achievements in the
TIMSS tests.
TIMSS adopts traditional mathematics content domains, namely number, algebra,
measurement, geometry and measurement, as well as data and chance. Unlike TIMSS,
the mathematical contents of the PISA tests are based on mathematical phenomena
underlying broad classes of problems in the real world (OECD 2013a). Wu (2010)
expresses that TIMSS covers a wider range of curriculum mathematics contents than
PISA. However, Stacey (2011) argues that PISA contents correspond with school
curricula in a broad sense. She claims that if PISA intends to measure how well 15-
year-old students are prepared at school to use what they have learned to solve
problems in a variety of situations in life, then PISA items should be well represented
in school curricula. Since there are no curriculum matching analyses for PISA, we
match the mathematics contents of the Indonesian and Singaporean curricula with
mathematics contents of PISA problems. This is the reason why we chose PISA
instead of TIMSS. Further, we decided to select the PISA 2012 study since its
central part was mathematics, and the OECD (2013b) released several items used in
the study.
The PISA test is administered to 15-year-old students, a choice which mostly
addresses students in the ninth year of school education after kindergarten (Grade 9)
in the selected countries for this study. Therefore, we chose to compare mathematics
curricula for the secondary school in Indonesia and Singapore. As the educational
systems of these two countries are very different, we start this part by shortly discussing
the secondary education system and the curricula of each of those two countries.

Overview of secondary education system and the curricula

Indonesia

There are four levels of formal education in Indonesia, namely Primary School (Grade
1–6), Junior High School/Lower Secondary School (Grade 7–9), Senior High School/
Upper Secondary School or Vocational School (Grade 10–12), and Higher Education
The different mathematics performances in PISA 2012 and a curricula... 179

(University/Institute/High School/Academy/Polytechnic). Two curricula are being im-

plemented in the lower secondary school, namely 2006 Curriculum (C06) and 2013
Curriculum (C13). C13 is currently being implemented in some selected schools and is
going to replace C06 comprehensively in all schools until 2019. However, we preferred
C06 to C13 in this curricula comparison because of choosing PISA 2012. C06 was the
implemented curriculum when PISA 2012 took place.
The goal of mathematics lessons in C06 is to enable students to:

1. Understand mathematical concepts, explain the connection between the concepts,

and apply the concepts and algorithms flexibly, accurately, efficiently, and appro-
priately in problem solving;
2. Use reasoning on mathematical patterns and properties and do mathematical
manipulations in generalising, proving, or explaining mathematical ideas and
statements;
3. Solve problems encompassing the abilities to understand problems, creating math-
ematical models, solving the models, and interpreting the solutions;
4. Communicate symbols, tables, diagrams, or other media to explain situations or
problems; and
5. Have attitudes appreciating mathematics utilities in human life, namely curiosity,
concern and interest in learning mathematics, perseverance and confidence in
problem solving.(BNSP 2006, p. 140, translated by the first author)

The mathematics contents of C06 are sorted by strands, namely Number, Algebra,
Geometry and Measurement, and Statistics and Probability (BNSP 2006). The contents
of each strand are written in the form of the Competency Standards which are
elaborated in the form of Basic Competencies. These Basic Competencies become
the guidelines for mathematics teachers to decide what mathematics topics they teach in
their classrooms.
In addition to the mathematics contents, the Ministry of Education and Culture also
published the learning process standard of C06 as guidelines for teachers to design a
learning process in their class. This document is intended not only for mathematics
lessons but also for other lessons as well. There are three phases of the intended
learning process, namely opening, core, and closing (BNSP 2007). Three learning
activities should be implemented in the core phase, namely exploration, elaboration,
and confirmation (BNSP 2007).

Singapore

The education system in Singapore consists of 6 years in Primary Education (Primary

1–6), 4–5 years in Secondary Education (Secondary 1–4), and 1–6 years in Post-
Secondary Education. Singaporean students must take the Primary School Leaving
Examination (PSLE) to graduate from a primary school. The results of the PSLE assign
the students to different courses in secondary schools (Kaur 2014).
There are three courses in the secondary education system, namely Special, Express,
and Normal courses (Kaur 2014). Students in the Normal course sit in either the
Normal Academic [N(A)] course or the Normal Technique [N(T)] course (ibid). The
Ministry of Education of Singapore (MOE) has published a mathematics curriculum for
180 Safrudiannur, B. Rott

all courses that has been implemented since 2013. In this article, however, we chose the
curriculum of secondary schools published in 2006 and implemented in 2007 (SC) due
to the results of PISA 2012.
The aims of mathematics education stated in SC are to enable students to:

1. Acquire the necessary mathematical concepts and skills for everyday life, and for
continuous learning in mathematics and related disciplines;
2. Develop the necessary process skills for the acquisition and application of math-
ematical concepts and skills;
3. Develop the mathematical thinking and problem solving skills and apply these
skills to formulate and solve problems;
4. Recognise and use connections among mathematical ideas, and between mathe-
matics and other disciplines;
5. Develop positive attitudes towards mathematics;
6. Make effective use of a variety of mathematical tools (including information and
communication technology tools) in the learning and application of mathematics;
7. Produce imaginative and creative work arising from mathematical ideas; and
8. Develop the abilities to reason logically, to communicate mathematically, and to
learn cooperatively and independently. (MOE 2006, p. 1)

The mathematics contents of SC are divided into three levels, namely O-level for the
special and express courses, N(A)-level for the N(A) course, and N(T)-level for the
N(T) course. Nonetheless, the curriculum for all levels consists of three similar
mathematics content strands, (1) Number and Algebra; (2) Geometry and Measure-
ment; as well as (3) Statistics and Probability (MOE 2006).
Unlike C06, SC has a framework for mathematics lessons designed as the direction
for teaching, learning, and assessments of mathematics (MOE 2006). In the framework
(see Fig. 3), mathematical problem solving is in the centre of teaching and depends on
five interrelated components, namely Concepts, Skills, Processes, Attitudes, and Meta-
cognition (MOE 2006). This framework shows that Singapore has a mathematics
curriculum in which problem solving is the primary goal of learning mathematics
(Anderson 2009) and the core of the mathematics curriculum (Dindyal 2006).

Fig. 3 The framework of Singapore’s mathematics curriculum (MOE 2006, p. 2, redrawn by the first author)
The different mathematics performances in PISA 2012 and a curricula... 181

Theories of the interpretation of problem solving

The curricula section shows that problem solving is an essential part of the educational
goals of the mathematics curricula of both Indonesia and Singapore. This is also true for
a lot of countries. For example, problem solving is one of the proficiencies incorporated
in the Australian curriculum (Sullivan 2011) as well as one of six process-related
competencies in Germany (KMK 2003). The NCTM (2000) also suggests that teachers
implement mathematics teaching and learning in a way that emphasises the use of
problem solving. Seemingly, it is widely accepted that problem solving is essential for
mathematics learning (cf. Rott 2012). Nevertheless, we should be careful to interpret the
role of problem solving in a curriculum. The meaning of the term Bproblem solving^ in
the context of mathematics is still open to different interpretations as it ranges from
working on a routine task to solving a puzzling situation (Rott 2012). Besides, the
different interpretations of problem solving may also happen since the use of problem
solving in mathematics learning (Schroeder and Lester 1989) or the view related what
things are important in problem solving (Branca 1980) can be different.
In accordance with the literature, we argue that a task should be considered a Broutine
task (not a problem)^ if teachers tell students what they are supposed to do to solve it
(Pehkonen 2017). The task should also be considered a routine task if students know an
algorithm or a method to solve it and are able to apply it (Eisenmann et al. 2015; Rott
2012). Eisenmann et al. (2015) refer to this condition as the straight-forward way. The task
can be considered to be a B(non-routine) problem^ if the student cannot straightforwardly
solve the task and the student may need to apply a heuristic strategy to solve it (Eisenmann
et al. 2015). However, working on a routine task can be considered as a problem-solving
activity if a teacher allows and encourages students to express their own strategies and
gives them opportunities to communicate with each other to share and justify their own
strategies or their friends’ strategies (Hiebert et al. 1996). Therefore, whether a task is
considered as a problem or a routine task depends on the solver and the situation.
A distinction is made between three types of addressing problem solving in a lesson,
namely teaching about problem solving, teaching for problem solving, and teaching via
problem solving (Schroeder and Lester 1989). In teaching about problem solving,
students are explicitly taught about the phases of solving a problem, namely under-
standing the problem, devising plan to solve the problem, carrying out the plan, and
looking back (ibid). Specifically, teachers use the Pólya-based approach to teach
problem solving (Chapman 2017). In teaching for problem solving, students are
explicitly taught about mathematical concepts and procedures to solve problems
(Schroeder and Lester 1989). In this category, teachers help students to implement
specific methods (translation-based approach) to solve problems (Chapman 2017). At
last, in teaching via problem solving, students are expected to construct their knowl-
edge and get a deep understanding of the knowledge during the process of problem
solving (Schroeder and Lester 1989). This category seems to be in line with teaching
using an inquiry-based approach proposed by Chapman (2017) which teachers help
students to develop mathematical knowledge using problems.
Branca (1980) differentiates three common interpretations of problem solving, name-
ly problem solving as a goal, as a process, and as a basic skill. He stresses that the first
interpretation—problem solving as a goal—is related to the view that mathematics is a
useful object to solve problems. He refers this interpretation to the document of NCSM
182 Safrudiannur, B. Rott

(1977) that learning mathematics means learning to solve problems. Some researchers
point out that it is essential to teach students to be able to solve non-routine problems
(Arslan and Altun 2007) or real-world problems (Geiger et al. 2017).
The second interpretation—problem solving as a process, according to Branca’s
(1980) explanation—is related to the view that students can apply mathematical
knowledge as well as strategies, methods, or approaches that they have learnt to solve
problems. In other words, Branca (1980) expresses that problem solving is a process of
the application of knowledge. However, Hiebert et al. (1996) argue that students also
gain new knowledge in the process of problem solving. Specifically, solving problems
can help students to acquire knowledge (NCTM 2000; Pehkonen 2017).
The focus of learning mathematics in the last interpretation—problem solving as a
basic skill—is related to the view that students must learn essentials of problem solving
such as understanding the contents of problems and deciding what techniques should be
used to solve the problems (Branca 1980). Branca (1980) cites the documents of SMSG
(1972) and NCSM (1977) to explain this interpretation. SMSG (1972) emphasises that
students should know how to formulate a problem and create appropriate models or
representations for given situations in the problem. It is essential since creating repre-
sentations can help students to develop deep understanding (Makar 2012).
Further, in the view of problem solving as a basic skill, students should have
opportunities to learn a variety of strategies for solving problems and then develop
the skills of using them flexibly (SMSG 1972; NCTM 2000). Eisenmann et al. (2015)
suggest that it is essential to teach students heuristic strategies in problem solving since
they may not have or remember necessary mathematical knowledge or procedures to
solve problems. Moreover, students should also have opportunities to make arguments
to justify their strategies (Makar 2012; Fielding-Wells 2015). The NCSM (1977) also
stresses that students should be able to translate and interpret their results to the
situation in the problem and examine the results to arrive at a valid conclusion.

Research questions

In this comparison study, we compare Indonesian 2006 curriculum (C06) and Singa-
porean curriculum published in 2006 (SC). We examine the alignment of the mathe-
matics contents of C06 and SC based on mathematics contents of PISA 2012 items as
well as the role of problem solving in the intended learning process of the two curricula.
The questions of the study at hand are:

1. What are the differences between C06 and SC in relation to the emphasis on
mathematics contents of PISA 2012 items?
2. What are the differences between C06 and SC in relation to the role of problem
solving in the intended learning processes of the two curricula?

Method

The systems of the secondary education of Indonesia and Singapore are very different.
The designs of the mathematics curricula of those two countries are also different, for
The different mathematics performances in PISA 2012 and a curricula... 183

example their goals, competencies, contents, and standards of the intended learning
processes. Therefore, we need a framework to make the comparison.
Both countries participated in the PISA 2012 study. This indicates that they both
accepted the PISA 2012 framework to assess their educational systems. Thus, we chose
the mathematical contents of the mathematics framework of PISA 2012 as a basis,
which is external to both countries, for our content analysis. The categories of the
mathematical contents of PISA 2012 items are (1) change and relationship, (2) space
and shape, (3) quantity, as well as (4) uncertainty and data (OECD 2013a). However,
OECD (2013a) lists illustrative mathematics contents included in the PISA 2012
survey:

Functions, algebraic expressions, equations and inequalities, coordinate systems,

relationships within and among geometrical objects in two and three dimensions,
measurement, numbers and units, arithmetic operations, percents, ratios and
proportions; counting principles; estimation; data collection, representation and
interpretation; data variability and its description; samples and sampling; and
chance and probability. (OECD 2013a, p. 36)

To analyse the term of problem solving in the curricula of those two countries, we
chose the theory suggested by Branca (1980) who expresses that there are three
common interpretations of problem solving: as a goal, as a process, and as a basic skill.
We created codes for each analysis by following steps suggested by Creswell (2014).
We outline the steps from Creswell (2014, p. 197–199) below:

1. Organising and preparing documents for analysis. The documents are (1) the
content standards of C06 for lower secondary school (BSNP 2006) and the
learning process standard of C06 (BSNP 2007), and (2) Secondary Mathematics
Syllabus of Singaporean Curriculum (MOE 2006).
2. Writing a list of topics/sub-topics and then creating codes for them. We created a
table to identify the main topic, sub-topic, and sub-sub-topic of mathematics
contents listed in the PISA 2012 framework and gave codes for each of them,
namely Codes for Mathematics Contents (CMC) 1. We also created a table to
identify the sub-category and sub-sub-category of each interpretation of problem
solving and gave codes for each of them, namely Codes for Problem-solving
Interpretation (CPI) 1.
3. Improving codes. We matched CMC 1 to the PISA 2012 released items in OECD
(2013b). If there is content in an item uncovered by CMC 1, we put the content on
a topic or as a new topic. The name of the revised codes is CMC 2. To improve CP
1, we matched it to chapters about problem solving both in SMSG (1972) and
NCSM (1977) since Branca (1980) uses those chapters to illustrate his point of
view about problem solving. If there is content in the chapters uncovered by CP 1,
we put the content on a sub-category or as a new sub-category. The name of the
revised codes is CPI 2.
4. Performing preliminary analysis. The first author used CMC 2 to code mathemat-
ics contents of C06 and of O-level, N(A)-level and N(T)-level of SC and used CPI
2 to code the learning process standard of C06 and the explanation of the
framework of SC.
184 Safrudiannur, B. Rott

5. Checking the coding consistency. Three coders (both authors and a colleague from
Indonesia) independently rated selected parts of the mathematics contents of both
curricula using CMC 2. By using the formula for calculating the percentage of
agreement (Neuendorf 2002), the percentage was at least 91.4%. Since it was high,
we did not revise CMC 2.

Both authors independently rated selected parts of the framework of SC by using CPI 2.
The percentage of agreement for the sub-category codes was 90%, while the percentage
for the sub-sub-category codes was below 90%. We discussed and revised CPI 2
together to reach a better agreement. Then, we coded the selected parts again together
by using the revised codes (namely CPI 3). After that, we independently rated selected
parts of the learning process standard of C06 by using CPI 3. The percentage of
agreement was higher than 90%.

6. Re-coding existing data. The independent coders on the fifth step only coded
selected parts. Thus, re-coding the other parts of each document besides the
selected parts by using the revised codes (CMC 2 and CPI 3) was necessary

We present examples of how we coded using CMC 2 and CPI 3 in Table 1. Regarding
the content analysis, each code represents a sub-sub-topic. If we did not find a code in a
curriculum, we decided that the curriculum does not cover the sub-sub-topic represent-
ed by the code.
Regarding the analysis of the role of problem solving, we summarised the results of
the coding by displaying the number of sentences related to a code. The standard of the
learning process of C06 published by BSNP (2007) consists of 20 pages divided into
five parts. We only analyse Part III (the implementation of the learning process) on
pages 13–15. Secondary Mathematics Syllabus of Singaporean Curriculum (SC) pub-
lished by MOE (2006) consists of 40 pages divided into six parts. We only analysed
Part 3 on pages 2–5 consisting of the discussion of the mathematics framework of SC.

Results

The comparison of mathematics contents

We will now present the results of the comparison of the statistics and the geometry topics
as well as the measurement on two-dimensional (2D) shape topics. We chose these two
topics because we found significant differences between C06 and SC on these two topics.

Comparison of the statistics topic

The sub-topics identified in the PISA 2012 framework and released items are Bdata
collection, representation, and interpretation^, Bdata variability^, and Bsampling and
sample^ (see Table 2). Three questions of the BCharts^ item in the PISA 2012 test are
examples which are related to the statistics topic (see Fig. 4). Specifically, the mathe-
matics content of all questions of this item is related to the sub-sub-topics Bdata
representation^ and Bdata interpretation^.
The different mathematics performances in PISA 2012 and a curricula... 185

Table 1 Examples of the coding of C06 and SC curricula using CMC 2 and CPI 3, respectively

No Selected parts [together with codes]

1 Regarding mathematical contents of C06 and SC:

a. C06 (Document: BSNP 2006, p. 142, 143, 145)
- Represent data in the form of tables, bar graphs, line graphs, and phi charts (Grade9 (G9) page (p.)
145) [Codes: (Statistics: data representation)]
- Calculate the average, median, and mode of a set of data including the interpretation (G9 p. 145)
[Codes: (Statistics: central tendency), (Statistics: the interpretation of central tendency)]
- Identify the properties of a triangle based on its sides and angles (G7 p. 142) [Codes: (Geometry:
properties of a triangle)]
- Use of Pythagoras’ theorem to calculate the length of three sides of a right triangle (G8 p. 143)
[Codes: (Geometry: Pythagorean theorem)]
b. SC (Document: MOE 2006, p. 8, 11)
- Construction and interpretation of tables, bar graphs, pictograms, line graphs, phi charts,
histograms (Secondary one (S1) p. 8) [Codes: (Statistics: data representation), (Statistics: data
interpretation)]
- Interpretation and analysis of dot diagrams, histograms, stem- and leaf- diagrams (S2 p. 11)
[Codes: (Statistics: data representation), (Statistics: data interpretation)]
- Mean, mode, and median as averages (S2 p. 11) [Codes: (Statistics: central tendency)]
- Properties of triangles and special quadrilaterals (S2 p. 8) [Codes: (Geometry: properties of a
triangle), (Geometry: properties of a quadrilateral)]
- Use of Pythagoras’ theorem (S2 p. 11) [Codes: (Geometry: Pythagorean theorem)]
2 Regarding learning process related problem solving (PS) mandated by C06 and SC
a. C06 (Document: BSNP 2007, p. 13, 14)
- The core phase encompasses three processes namely exploration, elaboration, and confirmation (p.
13) [Codes: No codes (not explicitly about PS)]
- In the exploration, a teacher (p. 13):
- uses a variety of learning approaches, learning media, and learning sources [Codes: No codes]
- In the elaboration, a teacher (p. 13):
- gives students opportunities to think, analyse, solve problems and act without fear (p. 14)
[Codes: (PS as a goal)]
- In the confirmation, a teacher (p. 14):
- facilitates students to get meaningful experiences to reach the basic competencies by helps
students in solving problems [Codes: (PS as a goal)]
b. SC (Document: MOE 2006, p. 2, 4, 5)
- Mathematical problem solving is central to mathematics learning. It involves the acquisition and
application of mathematics concepts and skills in a wide range of situations, including
non-routine, open-ended and real-world problems (p. 2) [Codes: (PS as a process: the application
of knowledge, the acquisition of knowledge), (PS as a goal: solving non-routine problems, open
problems, real-world problems)]
- Students should use various thinking skills and heuristics to help them solve mathematical
problems (p. 4) [Codes: (PS as a basic skill: using variety strategies), (PS as a goal)]
- The following activities may be used to develop the metacognitive awareness of students and to
enrich their metacognitive experience (p. 5):
- Encourage students to seek alternative ways of solving the same problem and to check the
appropriateness and reasonableness of the answer [Codes: (PS as a basic skill: using various
strategies, evaluating solutions)]
186 Safrudiannur, B. Rott

Table 2 The comparison table on the statistics and the geometry and measurement on 2D shape topics

Contents of PISA 2012 item Indonesia Singapore

C06 N(T)-Level N(A)-Level O-Level

Statistics (main topic):

a. Data collection, representation, and interpretation (sub-topic):
- Data collections V V V V
(in G9) (in S1, S2, S3/4) (in S1, S2, S3/4) (in S1, S2, S3/4)
- Data representations V V V V
(in G9) (in S1, S2, S3/4) (in S1, S2, S3/4) (in S1, S2, S3/4)
- Data interpretation X V V V
(in S1, S2, S3/4) (in S1, S2, S3/4) (in S1, S2, S3/4)
b. Data variability (sub-topic):
- Central tendency V V V V
(in G9) (in S2, S3/4) (in S2, S3/4) (in S2, S3/4)
- Interpretation of central tendency V V V V
(in G9) (in S2, S3/4) (in S2, S3/4) (in S2, S3/4)
- Variability X V V V
(in S3/4) (in S3/4) (in S3/4)
- Interpretation of variability X V V V
(in S3/4) (in S3/4) (in S3/4)
c. Samples and Sampling (sub-topic)
- Sampling and population X X X X
- Simple inferences based on X X X X
properties of samples
Geometry and measurement on 2D shapes (main topic):
a. Angles and Measurement (sub-topic):
- Concepts on angles (kind of an V V V V
angle, etc.) (in G7) (in S1) (in S1) (in S1)
- Measurements of angle V V V V
(in G7) (in S1) (in S1) (in S1)
b. Planes and Measurements (sub-topic):
- Properties of regular shapes V V V V
(square, triangle, etc.) (in G7) (in S2) (in S2) (in S1)
- Measurements on Polygonal shapes X X V V
(in S2) (in S1)
- Measurements on regular shapes V V V V
(in G7) (in S1) (in S1) (in S1)
- Measurements on composite X V V V
figures (in S1) (in S1) (in S1)
c. Circle and Measurements (sub-topic)
- Elements of a circle (arc, sector, V V V V
etc.) (in G8) (in S1, S3/4) (in S1, S3/4) (in S1, S3/4)
- Measurements (centre angle of a V V V V
sector, area and circumference of (in G8) (in S1, S3/4) (in S1, S3/4) (in S1, S3/4)
a circle, arc length, etc.)
d. Similarity and Congruence (sub-topic)
- Similarity V V V V
The different mathematics performances in PISA 2012 and a curricula... 187

Table 2 (continued)

Contents of PISA 2012 item Indonesia Singapore

C06 N(T)-Level N(A)-Level O-Level

(in G9) (in S2, S3/4) (in S2, S3/4) (in S2)
- Congruency V V V V
(in G9) (in S2, S3/4) (in S2, S3/4) (in S2)
e. Pythagorean Theorem (sub-topic) V V V V
(in G8) (in S2) (in S2) (in S2)

V covered, X not covered, G9 Grade 9, S1 Secondary 1, S2 Secondary 2, S3/4 Secondary 3/4, C06 Indonesian
2006 Curriculum, SC Singaporean Curriculum, N(T) Normal (Technical), N(A) Normal (Academic), O
Express

Table 2 shows that the statistics topic in C06 is introduced in Grade 9 for the first
time. In contrast, the statistics topic in all levels of SC (O-level, N(A)-level, and N(T)-
level) is introduced from the beginning of the secondary education, Secondary 1 (S1).
Furthermore, Table 2 shows that SC also covers missing sub-sub-topics in C06, for
example Bdata interpretation^. Moreover, SC discusses the sub-sub-topic Bdata
interpretation^ in all grades and all level of secondary school.
We also found that all levels of SC oblige more data representation than does C06.
C06 only explicitly mentions tables, bar charts, line charts, and pie charts for the sub-
sub-topic Bdata representations^. While all levels of SC specifically include not only
those four data representations but also pictograms, dot diagrams, histograms, stem-
and-leaf-diagrams, cumulative frequency diagrams, as well as box-and-whisker plots.

Fig. 4 Questions of BCharts^ item of PISA 2012 (OECD 2013b, p. 9, redrawn by the first author)
188 Safrudiannur, B. Rott

Comparison of the geometry and measurement on two-dimension (2D) shape topic

The sub-topics identified for this topic are Bangle and measurement^, Bplanes and
measurements^, Bcircle and measurements^, Bsimilarity and congruence^, and Bthe
Pythagorean Theorem^ (see Table 2). Question 3 of the BSailing Ships^ item (see
Fig. 5) and Question 1 of the BRevolving Door^ item (see Fig. 6) are two examples of
questions in the PISA 2012 study which are related to the topic Bgeometry and
measurement on 2D shapes^. Specifically, the mathematics content of the BSailing
Ships^ item is related to the sub-sub-topics Bmeasurement on angles^, Bthe properties
of an isosceles right triangle^, and Bthe Pythagorean Theorem^. While the mathematics
content of Question 1 of the BRevolving Door^ item is related to the sub-sub-topics
Belements of a circle^ and Bmeasurements on a circle^. Similar to the statistics topic,
Table 2 shows that SC covers more mathematics contents of PISA 2012 items than
does C06. SC explicitly discusses the content related to polygonal shapes and mea-
surements of composite figures.

Comparison of the examination of the role of problem solving

The number of the sentences that we coded in C06 and SC is 35 and 50, respectively.
Each sentence may have more than one code. We display the results of our coding in
the form of the frequency (the number of sentences containing a code).
Table 3 shows that the intended learning processes of C06 and SC support the idea
of problem solving as a goal. However, the intended learning process of C06 does not
mention the type of the problems. Meanwhile, SC explicitly recommends Singaporean
teachers to give their students opportunities to solve problems, for example, real-world
problems:

It is important that students apply mathematical problem-solving skills and

reasoning skills to tackle a variety of problems, including real-world problems.
(MOE 2006, p. 4)

Through mathematical modelling, students learn to use a variety of representa-

tions of data and to select and apply appropriate mathematical methods and tools
in solving real-world problems. (MOE 2006, p. 4)

Moreover, Table 3 shows that only SC emphasises problem solving as a process and
as a basic skill. The framework of SC highlights problem solving as a process where

Fig. 5 Question 3 of BSailing Ships^ item of PISA 2012 (OECD 2013b, p. 13, redrawn by the first author)
The different mathematics performances in PISA 2012 and a curricula... 189

Fig. 6 Question 1 of BRevolving Door^ item of PISA 2012 (OECD 2013b, p. 33, redrawn by the first author)

problem solving plays important roles as a place for students to apply and build their
knowledge:

Mathematical problem solving is central to mathematics learning. It involves the

acquisition and application of mathematics concepts and skills in a wide range of
situations, including non-routine, open-ended and real-world problems. (MOE
2006, p. 2)

They should be given a variety of learning experiences to help them develop a

deep understanding of mathematical concepts and to make sense of various
mathematical ideas […]. (MOE 2006, p. 3)

Furthermore, the framework of SC also highlights problem solving as a basic skill.

The framework recommends Singaporean teachers design mathematical learning
in which students can acquire skills in problem solving. For instance, the frame-
work expects that students are able to use various methods/heuristics to solve
problems.

BStudents should use various thinking skills and heuristics to help them solve
mathematical problems.^ (MOE 2006, p. 4)

BAllow students to discuss how to solve a particular problem and to explain the
different methods that they use for solving the problem.^ (MOE 2006, p. 5)

BEncourage students to seek alternative ways of solving the same problem and to
check the appropriateness and reasonableness of the answer.^ (MOE 2006, p. 5)

The last quote above also indicates that the framework of SC expects that
Singaporean students are able to evaluate solutions in problem solving (skills in
examining the results).
190 Safrudiannur, B. Rott

Table 3 The results of the coding about the examination of the role of problem solving

No The codes Frequency

C06 SC

1 Problem solving as a goal (main category)

a. Learning to solve problems (sub-category)
- real-world problems – 3
- open problems (more than one solution) – 1
- non-routine problems – 1
others related to (1a) 2 12
2 Problem solving as a process (main category)
a. Problem solving and knowledge (sub-category)
- application of knowledge – 7
- acquisition of knowledge – 5
others related to (2a) – 1
3 Problem solving as a basic skill (main category)
a. Skill to understand/formulate a problem (sub-category)
- able to identify mathematics aspects in the problem – 5
- able to represent the situation in the problem – 5
others related to (3a) – –
b. Skill to solve a problem (sub-category)
- able to use a variety of strategies/heuristics/methods – 7
- able to explain/justify strategies/heuristics/methods – 4
others related to (3b) – 1
c. Skill to examine the results of solving a problem (sub-category)
- able to evaluate the solution – 2
- able to evaluate the strategies/heuristics/methods to reach the solution – 1
others related to (3c) – –

A sentence coded Bothers related to a sub-category code^ means that the sentence contains the sub-category,
but no sub-sub-category codes are appropriate. For example, if a sentence is about solving a problem without
specifically mention the type of the problem, it will be coded as Bothers related to 1a^ (see Table 1)

Discussions and concluding remarks

As we mentioned in the introduction, this curricula comparison is a part of a study with

the goal of finding possible reasons for the different mathematics performances be-
tween countries like Indonesia and Singapore in international comparison studies with a
focus on the PISA 2012 study. Unlike other curricula comparison studies which
compare mathematics contents of curricula directly, we compared mathematics con-
tents of Indonesian 2006 Curriculum (C06) and Singaporean Curriculum (SC) not to
each other, but to the contents tested in PISA 2012. With this approach, we found two
significant differences between C06 and SC.
The first significant difference is the difference in the breadth of mathematics
contents of statistics as well as geometry and measurement topics and the grade at
The different mathematics performances in PISA 2012 and a curricula... 191

which the mathematics contents are taught. For example, both C06 and SC suggest that
teachers teach students contents related to the statistics topic such as Bdata collection^,
Bdata representation^, and Bcentral tendency^. However, SC introduces the statistics
topic earlier and for all grades in the secondary school (starting with Secondary 1),
whereas C06 presents the statistics topic only at Grade 9, the last grade of the secondary
school of Indonesia (see Table 2). Moreover, we also found that SC suggests that
Singaporean teachers introduce more different representations to present data than C06
does for Indonesian teachers.
The second significant difference is related to the emphasis on mathematics contents
tested in the PISA 2012 study. SC covers mathematical contents tested in the PISA
2012 study which are missing in C06. For example, in the case of the statistics topic,
SC covers Bdata interpretations^ and Bdata variability^, but C06 does not. In the case of
the geometry and measurement topic, only SC explicitly mandates Singaporean
teachers to teach Bmeasurements on composite figures^ (see Table 2). Seemingly, the
results of the comparison on mathematics contents support the Ministry of Education
and Culture of Indonesia’s claim that many mathematics topics tested in PISA do not
exist in Indonesian Curriculum (Kementerian Pendidikan dan Kebudayaan Indonesia
2014, p. 2). According to the ministry, it is one of the factors causing the low
performances of Indonesian tests in PISA studies.
However, we argue that a curriculum covering contents of a PISA item is not a
guarantee for students to be able to solve the item. Thus, we take a close look at a small
excerpt of the data of PISA 2012 to justify the argument.
The content of Question 5 of the BCharts^ item is Bdata interpretation^ for the
statistics topic. Our curricula comparison shows that Indonesian C06 does not cover
this content, but SC does (see Table 2). Table 4 shows that the percentage of Singa-
porean students who solved this question is much higher than that of Indonesian
students. These data seem to support the Indonesian Ministry’s claim. However, Table 4
also shows that only 19.80% of Indonesian students correctly solved Question 2 of the
BSailing Ships^ item. In contrast, the percentage of Singaporean students is 77.19%.
Whereas, Table 2 shows that both C06 and SC cover the mathematics contents of the
question. These data indicate that solely aligning and comparing mathematical contents
of C06 and SC might not be sufficient to get insight into possible reasons for the
different performances of Indonesia and Singapore in the PISA 2012 study.
The second aspect that distinguishes this comparison from other curricula compar-
ison studies is that in addition to examining mathematical contents, we also analysed
the intended learning processes of the two curricula with a special focus on mathemat-
ical problem solving as a goal, as a process, and as a basic skill. This examination also

Table 4 The percentage of students providing the correct answers to the selected items

Selected items Percentage*

Indonesia Singapore OECD

Question 5 of Charts (see Fig. 4) 48.16 85.63 76.68

Question 3 of Sailing Ships (see Fig. 5) 19.80 77.19 49.81

*The percentages are taken from data sets: a compendium of the cognitive item responses (OECD n.d.)
192 Safrudiannur, B. Rott

shows significant differences between C06 and SC. The results indicate that SC
highlights problem solving much more than does C06. Table 3 indicates that SC
emphasises the use of problem solving for all categories (problem solving as a goal,
problem solving as a process, and problem solving as a basic skill). C06, on the other
hand, seems only to mandate Indonesian teachers to teach their students to solve
problems (problem solving as a goal).
Besides Singaporean students are expected to be able to solve problems, SC
explicitly recommends that Singaporean teachers design learning processes where
problem solving has the role to help the students to be able to build their mathematical
knowledge as well as to apply it to real-world situations (problem solving as a process).
SC expects that Singaporean students can develop a deep understanding of mathemat-
ical concepts from problem solving activities (MOE 2006). This recommendation is in
line with the suggestions from the NCTM (2000) standards that learning with under-
standing is better than only memorising.
Moreover, SC also emphasises that Singaporean students should be able to acquire
necessary skills in problem solving such as understanding and representing the situation
in problems and solving them in various strategies (problem solving as a basic skill).
SC stresses that Singaporean students should be able to solve problems with a variety
of strategies/methods/heuristics (MOE 2006). These results are in line with the findings
of the study from Jane et al. (2011) adopting the method for curriculum analysis
developed by Porter (2002). The study found that SC has a great focus on solving
non-routine problems. According to Porter (2002), solving non-routine problems is
related to the expectation, for example, that students are able to apply and adapt a
variety of strategies to solve non-routine problems. Furthermore, the findings from the
examination of problem solving in the learning process of SC seem to be in line with
the arguments from Dindyal (2006) and Kaur (2014) that Singaporean Mathematics
Curriculum Framework (SMCF) strongly emphasises problem solving at a very young
age.

Limitations and further studies

A limitation of this study is that OECD did not release all mathematics items of the
PISA 2012 survey. However, the released items seem to be sufficient to elaborate
mathematical contents of the framework of the PISA 2012 study and to guide this
comparison study. At least, the published items are adequate to identify that SC covers
some of the mathematical contents of PISA 2012 that C06 does not cover. Thus, there
are two benefits of the usage of the mathematical contents of the PISA 2012 items as a
basis to examine the mathematics contents of curricula. The first benefit is that it may
enable us to identify the lack of mathematics contents of a curriculum compared to
curricula of high-performing countries. The second benefit is that it may also enable us
to identify the lack of the mathematics contents of the curriculum compared to those
tested in the PISA 2012 study.
Further, enriching the examination of the mathematics contents of curricula with the
analyses of the role of problem solving in the intended learning processes of the
curricula also gives additional benefits. The analyses may enable us to evaluate the
different degree of the emphasis on problem solving in the intended learning processes
of the curricula. Moreover, the varying degree of the emphasis between the curricula of
The different mathematics performances in PISA 2012 and a curricula... 193

Indonesia and Singapore that we have found is suggestive of the need for further
research.
The results of the analyses lead us to the hypothesis that the actual learning
processes related to problem solving may be another possible reason for the different
performances between Indonesia and Singapore. Therefore, for our next study, we
choose to investigate the implemented learning processes in classrooms related to
problem solving including teachers’ beliefs of mathematics and problem solving. The
reason why we include the beliefs is that teachers may differently enact the learning
process mandated by a curriculum since they may have different beliefs about math-
ematics and problem solving.

Acknowledgement This research is supported by Directorate General of Resources for Research, Technol-
ogy and Higher Education of Indonesia (Project number 101.20/E4.4/2015).

References

Anderson, J. (2009). Mathematics curriculum development and the role of problem solving. Paper presented at
the 2009 ACSA National Biennial Conference Curriculum, Canberra, ACT, http://www.acsa.edu.
au/pages/images/Judy%20Anderson%20-%20Mathematics%20Curriculum%20Development.pdf.
Retrieved on November 16, 2015.
Arslan, Ç., & Altun, M. (2007). Learning to solve non-routine mathematical problems. Elementary Education
Online, 6(1), 50–61.
Australian Curriculum, Assessment and Reporting Authority [ACARA]. (2017). Literature review:
Contemporary approaches to comparative education research, http://curriculumuat.azurewebsites.
net/media/3576/literature-review.pdf. Retrieved on February 28, 2018.
Badan Standar Nasional Pendidikan (BSNP). (2006). Standar isi untuk satuan pendidikan dasar dan
menengah: Standar kompetensi dan kompetensi dasar SMP/MTs [content standards of primary and
secondary school: Competency standards and basic competencies of junior high school]. Jakarta: BSNP.
Branca, N. A. (1980). Problem solving as a goal, process, and basic skill. In S. Krulik & R. E. Reys (Eds.),
Problem solving in school mathematics (pp. 3–8). National Council of Teachers of Mathematics.
BSNP. (2007). Peraturan Menteri Pendidikan Nasional Republik Indonesia Nomor 41 tahun 2007 tentang
standar proses untuk satuan pendidikan dasar dan menengah [the decree of the minister of National
Education of Indonesia no. 41 year 2007 about the process standard for elementary and secondary
school]. Jakarta: BSNP.
Chapman, O. (2017). Mathematics teachers ‘ways of supporting students’ learning of problem solving. In M.
Stein (Ed.), A life’s time for mathematics education and problem solving (pp. 45–69). Munster: WTM.
Creswell, J. W. (2014). Research design: Qualitative, quantitative and mixed method approaches. In Thousand
oaks. California: Sage Publications.
Department of Education. (2012). Review of the National Curriculum in England: What can we learn from the
English, mathematics and science curricula of high-performing jurisdictions?. Research Report DFE-
RR178, https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/184064/DFE-
RR178.pdf. Retrieved on February 23, 2018.
Dindyal, J. (2006). The Singaporean mathematics curriculum: Connections to TIMSS. In P. Grootenboer, R.
Zevenbergen, & M. Chinappan (Eds.), Identities, Cultures, and Learning Spaces: Proceedings of the 29th
annual conference of the Mathematics Education Research Group of Australasia (pp. 179–186),
http://www.merga.net.au/documents/RP182006.pdf. Retrieved on November 23, 2015.
Eisenmann, P., Novotnà, J., Přibyl, J., & Břehovský, J. (2015). The development of a culture of problem
solving with secondary students through strategies. Mathematics Education Research Journal, 27(4),
536–562.
Fielding-Wells, J. (2015). Identifying core elements of argument-based inquiry in primary mathematics
learning. In M. Marshman, V. Geiger, & A. Bennison (Eds.), Mathematics education in the margins
194 Safrudiannur, B. Rott

(proceedings of the 38th annual conference of the mathematics education research Group of Australasia)
(pp. 229–236). Sunshine Coast: MERGA.
Geiger, V., Stillman, G., Brown, J., Galbriath, P., & Niss, M. (2017). Using mathematics to solve real word
problems: The role of enablers. Mathematics Education Research Journal, 30(1), 7–19.
Hiebert, J., Carpenter, T. P., Fennema, E., Fuson, K., Human, P., Murray, H., Olivier, A., & Wearne, D. (1996).
Problem solving as a basis for reform in curriculum and instruction: The case of mathematics.
Educational Researcher, 25(4), 12–21.
Ibrahim, Z. B., & Othman, K. I. (2010). Comparative study of secondary mathematics curriculum between
Malaysia and Singapore. Procedia Social and Behavioral Sciences, 8(1), 351–355.
Jane, G., Wilson, B., & Zbar, V. (2011). Curriculum mapping project phase 4a: Comparing international
curricula against the Australian curriculum, final report. Australian Curriculum, Assessment and
Reporting Authority, https://acaraweb.blob.core.windows.net/resources/International_Mapping_Project_
Phase_4a_v1_file_2.pdf. Retrieved on August 8, 2017.
Kaur, B. (2014). Mathematics education in Singapore – An insider’s perspective. IndoMS-JME, 5(1), 1–16.
Kementerian Pendidikan dan Kebudayan Indonesia. (2013). Lampiran Peraturan Menteri Pendidikan dan
Kebudayaan Indonesia no. 65 tahun 2013: Standar proses untuk satuan pendidikan dasar dan menengah
[The attachment of the decree of the Minister of Education and Culture of Indonesia no. 65 year 2013:
The process standard for elementary and secondary school]. Available from http://jdih.kemdikbud.go.
id/new/public/produkhukum. Accessed on November 16, 2015.
Kementerian Pendidikan dan Kebudayan Indonesia. (2014). Lampiran Peraturan Menteri Pendidikan dan
Kebudayaan Indonesia no. 58 tahun 2014: Kurikulum 2013 Sekolah Menengah Pertama/Madrasah
Tsanawiyah [The attachment of the decree of the Minister of Education and Culture of Indonesia no. 58
year 2014: 2013 Curriculum for junior high school]. Available from http://jdih.kemdikbud.go.
id/new/public/produkhukum. Accessed on December 18, 2015.
Kementerian Pendidikan Nasional Indonesia. (2006). Lampiran Peraturan Menteri Pendidikan Nasional
Indonesia nomor 22 tahun 2006: Standar isi untuk satuan pendidikan dasar dan menengah [The
attachment of the decree of the Minister of National Education of Indonesia no. 22 year 2006: The
content standards of primary and secondary school]. Available from http://jdih.kemdikbud.go.
id/new/public/produkhukum. Accessed on December 22, 2015.
Kiselova, R. & Kangro, A. (2009). Mathematics curriculum in Latvia and PISA assessment(symposium869).
Paper presented at the ECER 2009 conference, http://www.eera-ecer.de/ecer-programmes/conference/2
/contribution/3253/. Accessed on June 20, 2017.
Kultusministerkonferenz (KMK). (2003). Beschlüsse der Kultusministerkonferenz: Bildungsstandards im
Fach Mathematik für den Mittleren Schulabschluss (Resolutions by the Standing Conference of
Ministers of Education and Cultural Affairs: Educational standards in mathematics for the intermediate
school leaving certificate), http://www.kmk.org/fileadmin/veroeffentlichungen_beschluesse/2003
/2003_12_04-Bildungsstandards-Mathe-Mittleren-SA.pdf. Retrieved on February 27, 2018.
Makar, K. (2012). The pedagogy of mathematical inquiry. In R. Gillies (Ed.), Pedagogy: New developments in
the learning sciences (pp. 371–397). Hauppauge, NY: Nova Science.
Mullis, I. V. S., Martin, M. O., Gonzalez, E. J., & Chrostowski, S. J. (2004). TIMSS 2003 international
mathematics report: Findings from IEA’s trends in international mathematics and science study at the
fourth and eighth grades. Chestnut Hill, MA: TIMSS and PIRLS International Study Center.
Mullis, I. V. S., Martin, M. O., Foy, P., Olson, J. F., Preuschoff, C., Erberber, E., Arora, A., & Galia, J. (2008).
TIMSS 2007 international mathematical report: Findings from IEA’s trends in international mathematics
and science study at the fourth and eighth grades. Chestnut Hill, MA: TIMSS and PIRLS International
Study Center.
National Council of Supervisors of Mathematics (NCSM). (1977). Position paper on basic mathematics skills.
Washington D.C: National Institute of Education.
National Council of Teachers of Mathematics (NCTM). (2000). Principles and Standards of School
Mathematics. Reston, VA: NCTM.
Neuendorf, K. A. (2002). The content analysis guidebook. In Thousand oaks. California: Sage Publications.
OECD. (2004). Learning for tomorrow’s world: First results from PISA 2003, https://doi.org/10.1787
/9789264006416-en.
OECD. (2007). PISA 2006: Science competencies for tomorrow’s world (volume 1: Analysis), https://doi.
org/10.1787/9789264040014-en.
OECD. (2010). PISA 2009 results: What students know and can do – Student performance in reading,
mathematics and science (volume I), https://doi.org/10.1787/9789264091450-en.
OECD (2013a). PISA 2012 assessment and analytical framework: Mathematics, reading, science, problem
solving and financial literacy, https://doi.org/10.1787/9789264190511-en.
The different mathematics performances in PISA 2012 and a curricula... 195

OECD. (2013b). PISA 2012 released mathematics items, http://www.oecd.org/pisa/pisaproducts/pisa2012-

2006-rel-items-maths-ENG.pdf. Retrieved on November 16, 2015.
OECD. (2014). PISA 2012 results : What students know and can do - student performance in mathematics,
reading and science, https://doi.org/10.1787/9789264201118-en.
OECD. (2016). PISA 2015 Results (Volume I): Excellence and Equity in Education, https://doi.org/10.1787
/9789264266490-en
OECD. (n.d.). Data base-PISA 2012: Downloadable data [Data sets: Compendium for the cognitive item
responses]. Available from http://www.oecd.org/pisa/pisaproducts/pisa2012database-downloadabledata.
htm
Organisation for Economic Co-operation and Development [OECD]. (2003). Literacy skills for the world of
tomorrow: Further results from PISA 2000, https://doi.org/10.1787/9789264102873-en
Pehkonen, E. (2017). Teaching mathematics via problem solving. In M. Stein (Ed.), A life’s time for
mathematics education and problem solving (pp. 45–69). Munster: WTM.
Porter, A. C. (2002). Measuring the content of instruction: uses in research and practice. Educational
Researcher, 31(7), 3–14.
Rott, B. (2012). Problem solving processes of fifth graders: An analysis of problem solving types. In B. Ubuz
(Ed.), Proceedings of the 35th conference of the International Group for the Psychology of mathematics
education, Vol. 4 (pp. 65–72). Turkey: Ankara.
Ruddock, G. & Sainsbury, M. with Clausen-May, T., Vappula, H., Mason, K., Patterson, E. W., Pyle, K.,
Kispal, A., Siddiqui, R., Mc Naughton, S., & Rees, F. (2008). Comparison of the core primary curriculum
in England to those of other high performing countries. National Foundation for Educational Research,
http://www.uquebec.ca/observgo/fichiers/48461_GSE-2.pdf. Retrieved on August 8, 2017.
School Mathematics Study Group (SMSG). (1972). Minimum goals for mathematics education. Newsletter
No. 38, section 2. Stanford, Calif: SMSG.
Schroeder, T. L., & Lester, F. K. (1989). Understanding mathematics via problem solving. In P. Trafton (Ed.),
New directions for elementary school mathematics (pp. 31–42). Reston, VA: National Council of
Teachers of Mathematics.
Stacey, K. (2011). The PISA view of mathematical literacy in Indonesia. IndoMS Journal on Mathematics
Education, 2(2), 95–126.
Sullivan, P. (2011). Teaching mathematics: Using research-informed strategies. In Camberwell. Victoria:
Australian Council for Educational Research.
The Ministry of Education of Singapore (MOE). (2006). Secondary mathematics syllabuses. Singapore:
Curriculum Planning and Development Division.
Wu, M. (2010). Comparing the Similarities and Differences of PISA 2003 and TIMSS. OECD Education
Working Papers, No. 32, OECD Publishing, https://doi.org/10.1787/5km4psnm13nx-en