See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/346167653

DEVELOPMENT OF ASSESSMENT TOOLS OF CRITICAL THINKING IN

MATHEMATICS IN THE CONTEXT OF HOTS

Article · October 2020

DOI: 10.37418/amsj.9.10.91

CITATION READS

1 169

4 authors:

Syaiful Hendra Tanjung Supriadi Nababan

Universitas Tadulako Telkom University
2 PUBLICATIONS   1 CITATION    3 PUBLICATIONS   1 CITATION   

SEE PROFILE SEE PROFILE

Cholis Sa'dijah Subanji Subanji

State University of Malang State University of Malang
87 PUBLICATIONS   232 CITATIONS    182 PUBLICATIONS   597 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

ASESMEN PEMECAHAN MASALAH OPEN-ENDED UNTUK MENGUKUR PROFIL BERPIKIR KREATIF MATEMATIS SISWA BERDASAR GENDER View project

PROSES KONEKSI MATEMATIKA SISWA SMK PGRI 7 MALANG DALAM MENYELESAIKAN MASALAH BERDASARKAN PEMAHAMAN SKEMP View project

All content following this page was uploaded by Subanji Subanji on 28 June 2021.

The user has requested enhancement of the downloaded file.

 Advances in Mathematics: Scientific Journal 9 (2020), no.10, 8659–8667
ADV MATH
SCI JOURNAL ISSN: 1857-8365 (printed); 1857-8438 (electronic)
https://doi.org/10.37418/amsj.9.10.91

DEVELOPMENT OF ASSESSMENT TOOLS OF CRITICAL THINKING IN

MATHEMATICS IN THE CONTEXT OF HOTS

HENRA SAPUTRA TANJUNG1 , SITI AMINAH NABABAN, CHOLIS SA’DIJAH, AND SUBANJI

A BSTRACT. The present was purposed to determine whether or not students have pos-
sessed critical thinking skills through assessment. In general, the study was aimed at
developing tools for assessing mathematical critical thinking skills, specifically in the
context of higher-order thinking skills (henceforth, HOTS). To this day, the primary ob-
jective of the assessment is to determine students’ competencies before and after learn-
ing processes, thus allowing teachers to get feedback. The teachers will benefit from
the feedback to improve the instructional media, including learning approaches, activi-
ties, and resources. In addition, the feedback also serves to disseminate information to
parents regarding the effectiveness of educational activities at schools. This study relied
on Dick and Carrey’s research and development model; the stages comprised literature
study, planning of the product, development of the product, expert validation, trials, re-
vision, and finalization of the final product. There were two trials in this research: trial
I was carried out in class IX-A, and trial II was conducted in class IX-B. According to the
results, (1) the designed instructional media were deemed effective. This was evident
from the results of individual and classical learning mastery, student activities (within
the bounds set by a specific regulation), and students’ positive reception to the learning
media. (2) From trial I to trial II, the improvement in students’ critical thinking skills
was 0.32 on average, and the classical learning mastery was measured at 21.4%.

1. I NTRODUCTION

Education refers to efforts in boosting the competencies of human resources. The suc-
cess of educational processes is determined by teachers’ assessment skills. Such skills

1
corresponding author
2020 Mathematics Subject Classification. 97D70, 97D60.
Key words and phrases. Assessment, Critical Thinking Skills, Higher-Order Thinking Skills.
8659
8660 H. S. TANJUNG, S. A. NABABAN, C. SA’DIJAH, AND SUBANJI

are essential to determine whether or not the objectives of learning, stated in the cur-
riculum, are attained. Furthermore, assessment skills are central to actualizing quality
teaching and learning processes. This notion also corresponds to The Standard of Aca-
demic Qualification and Competencies of Mathematics Teachers in junior high schools
and its equivalent [1]. With regard to the issues of critical thinking skills, the present
study had carried out a need analysis on selected junior high schools in West Aceh. The
result shows that: 1) Cognitive assessment instruments tend to emphasize the students’
memorizing skills, rather than higher-order thinking skills; tests for examining these
skills are yet available. 2) According to a report by PISA, the thinking skills of children
in Indonesia are considered low. This situation blames several factors, one of which is
that the students are not accustomed to solving contextual tests, which demand their
reasoning, argument skills, and creativity. 3) Teachers have low competence in devel-
oping assessment instruments. 4) The assessment instruments for measuring students’
critical thinking, especially in the context of HOTS, are yet available. Another concern
is the fact that Curriculum 2013 focuses on promoting students’ skills in observing, ask-
ing, reasoning, and communicating everything they have learned. For this reason, it is
suggested that the instruments for assessing mathematical critical thinking in the con-
text of HOTS should enable the students to practice the skills previously mentioned. In
Aceh, little is known about the assessment instruments for measuring students’ critical
thinking in HOTS context, since the only cognitive assessment instruments tend to em-
phasize the students’ memorizing skills, rather than higher-order thinking skills. Such
tests or assignments are, in fact, essential to help students retain their concentration in
the class and, at the same time, discover their potential [2]. The enhancement of HOTS
has been one of the main goals of teaching mathematics. In junior high schools, the
students are supposed to start practicing thinking skills. Indeed, the skills should fit the
capabilities of people in the students’ age [3]. Assessment is simply defined as a compre-
hensive process to identify students’ performance. This process is central to learning; it
is regarded as one of the efforts to improve the quality of education. Recently, there has
been a shift in the standard of learning assessment, from everything assessable to every-
thing that must be assessed [4]. Assessment comes from a psychometric / measurement
perspective, and is primarily concerned with scores of groups orindividuals, rather than
examining students’ thinking and communication processes [5]. Apsychometric per-
spective is concerned with reliably measuring the outcome of learning, rather than the
learning itself [6]. The development of critical thinking skills assessment instruments is
an effort to meet the needs of teachers in assessing critical thinking skills of students.
In mathematics, development efforts are also carried out, given the need for critical
thinking skills itself [7]. Critical thinking is needed, where every day individuals face
 DEVELOPMENT OF ASSESSMENT TOOLS OF CRITICAL THINKING. . . 8661

unlimited information, complex problems, rapid technological and social changes [8].
Furthermore, assessment instruments play a strategic role in the decision-making pro-
cess of teachers and schools regarding student learning outcomes, including the aspect
of HOTS. Chief among the components of teaching are the mathematical critical think-
ing skills. These skills are not granted for granted. In fact, a student should go through
intensive practices to master these skills [9]. Before teaching the students, a teacher is
demanded to demonstrate his or her mathematical critical thinking skills well. Simply
put, critical thinking skills are defined as the highest intellectual activities in human
interaction, which allow an individual to get involved in a meaningful decision-making
process [10]. HOTS are a set of thinking processes at a high cognitive level developed
from various concepts and cognitive processes in the learning taxonomy, such as critical
thinking skills, Bloom’s taxonomy, and taxonomy of learning, teaching, and assessment
[11]. Through mastering HOTS, students can differentiate ideas and opinions, deliver
good arguments, solve problems, construct explanations, formulate hypotheses, and
comprehend complex ideas and interpret them into clear, more straightforward ideas
[12]. Students are considered to fully master HOTS once they demonstrate the capabil-
ities to correlate new information with the one that they have stored in their memory.
On top of that, the students are able to reconstruct and develop everything they have
learned to meet a particular goal or to find a solution to a specific, complex problem
[13]. The primary objective of teaching HOTS is to enhance students’ thinking skills,
bringing the skills to a new level, with an emphasis on critical thinking skills in learning
information in various contexts [14]. Today’s learning evaluation is performed through
the actualization of quality education in terms of the assessment standards. To this day,
the improvement programs are carried out by adapting international assessment mod-
els gradually. The designed assessment is expected to assist students in shaping their
HOTS better since these skills stimulate the way the students think everything they got
from the class comprehensively [15]. HOTS are a part of the revised Bloom’s taxonomy;
the skills comprise several action verbs, namely analyze (C4), evaluate (C5), and create
(C6). All of which are applicable in designing a test.

2. M ETHOD

The present work applied a research and development (R&D) model by Dick and Car-
rey and was carried out in selected junior high schools in West Aceh regency. All quanti-
tative and qualitative data were retrieved from the results of expert validation and trials,
respectively. These data functioned to provide an overview of the developed products.
The procedures encompassed (1) the analysis of learning assessment, i.e., determining
8662 H. S. TANJUNG, S. A. NABABAN, C. SA’DIJAH, AND SUBANJI

the design of the developed assessment based on the HOTS and digital literacy prin-
ciples. (2) Development of assessment tools of critical thinking in mathematics in the
context of dan digital literacy the students. (3) Validation of the developed assessment;
its steps encompassed empirical trials, limited trials, and field trials. (4) Evaluation of
the data of trials. In this research, the data were retrieved from the validation sheets.
The sheets were completed by material experts, instructional media experts, and assess-
ment experts. Another data collection instrument involved questionnaires distributed
to the students. Stages in collecting the data were: (1) designing research instruments,
e.g., tests, scoring and assessment rubrics, (2) determining the validity, (3) revising the
product based on the input from the validators, (4) trials of the research instruments,
(5) examining the reliability and determining the difficulty and distinguishing features
of the test items, and (6) revision based on the results of the trials.

3. R ESULTS AND D ISCUSSION

The primary goal of the present study is to report the results of the development of
assessment tools of critical thinking skills in the context of HOTS. It was also aimed
at reporting the student activities when using the developed instructional media, the
students’ responses regarding the media, and the critical thinking learning outcomes.
Observation of student activities is performed during learning processes for three times
in trial I and trial II since the classes were also in three meetings. The result of the
analysis of the student activities in trial I is displayed in Table 1, while the following
Table 2 provides the data of Trial II.

TABLE 1. Description of the table

Meeting 1 Meeting 2 Meeting 3

Activity
Percentage Percentage Percentage
Frequency Frequency Frequency
(%) (%) (%)
a 57 26.39 58 26.85 100 28.49
b 30 13.89 28 12.96 43 12.25
c 67 31.02 80 37.04 113 32.19
d 43 19.91 35 16.2 66 18.8
e 18 8.33 12 5.556 21 5.983
f 1 0.46 3 1.389 8 2.279
Sum
216 100 216 100 351 100
Total
 DEVELOPMENT OF ASSESSMENT TOOLS OF CRITICAL THINKING. . . 8663

Description:
(a) Paying attention to the teacher’s/friend’s explanation.
(b) Reading/comprehending contextual problems in the textbooks.
(c) Solving problems/formulating solutions.
(d) Discussing/asking others and/or teachers.
(e) Drawing conclusions of a particular procedure or concept, and presenting re-
sults.
(f) Irrelevant behaviors during the class.
The student activities, meeting 1, of each category is measured at 26.39%, 13.89%,
31.02%, 19.91%, 8.33%, and 0.46%. The percentage of student activities in the aspect
of paying attention to teacher’s or friend’s explanations in 90 minutes scores 26.39%.
This is based on the result of dividing the score of the frequency of activity of 12 stu-
dents’ activities in category (a), or 57, and 216. Further, the result of the division is
multiplied by 100%. The value 216 refers to the result of the division of the total dura-
tion of the learning process in meeting 1 (90 minutes) with the duration of observation
per class (5 minutes); the result is multiplied by 12, or the number of the students ob-
served. These calculations are also used to determine the percentage of other activities
in each meeting since the duration and number of students in one observation session
always vary (90 minutes or 135 minutes; there were 12 students in meeting 1 and 2,
while in meeting three, there were 13 students). For this reason, the percentage of each
category is subject to change, depending on the duration and number of students who
participated in one observation session.

TABLE 2. Observation Results of Student Activities in Trial II

Meeting 1 Meeting 2 Meeting 3

Activity Percentage Percentage Percentage
Frequency Frequency Frequency
(%) (%) (%)
a 68 26.98 64 27.35 90 27.8
b 33 13.1 26 11.11 46 14.2
c 98 38.89 90 38.46 115 35.5
d 37 14.68 34 14.53 41 12.7
e 14 5.56 19 8.12 25 7.72
f 2 0.79 1 0.43 7 2.16
Sum
252 100 234 100 324 100
Total
8664 H. S. TANJUNG, S. A. NABABAN, C. SA’DIJAH, AND SUBANJI

The student activities, meeting 2, of each category is measured at 26.98%, 13.1%,

38.89%, 14.68%, 5.56%, and 0.79%. The percentage of student activities in the aspect
of paying attention to teacher’s or friend’s explanations in 90 minutes scores 26.98%.
This is based on the result of dividing the score of the frequency of activity of 14 stu-
dents’ activities in category (a), or 68, and 252. Further, the result of the division is
multiplied by 100%. The value 252 refers to the result of the division of the total dura-
tion of the learning process in meeting 2 (90 minutes) with the duration of observation
per class (5 minutes); the result is multiplied by 14, or the number of the students ob-
served. These calculations are also used to determine the percentage of other activities
in each meeting as the duration and number of students in one observation session al-
ways vary (90 minutes or 135 minutes; there were 14 students in meeting 1, 13 students
in meeting 2, and 12 students in meeting 3). By that, the percentage of each category
is subject to change, depending on the duration and number of students participated in
one observation session. Provided in Table 3 is the percentage of each category in all
three meetings, in trial I and trial II. On average, the percentage of each category of the
student activities in trial I is measured at 27.37%, 12.8%, 37.61%, 13.96%, 7.13%, and
1.13%. This is based on the division of the percentage of each category and the total
meetings (three meetings). For instance, the percentage of category (a) of the student
activities in trial I gets 26.39%, 26.85%, and 28.49%. Each percentage is divided by 3.
This method also applies during calculating the percentage of other activity categories.

TABLE 3. The Average Percentage of Student Activity

Percentage of Activity (%)

Activity
Trial 1 Trial 2
a 27.37 27.24
b 12.80 13.03
c 37.61 33.42
d 13.96 18.30
e 7.13 6.62
f 1.13 1.38
Sum
100 100
Total

On average, the percentage of the duration of each activity in the class is displayed in
the following Figure 1. The activity that the students spent the most is category (c), i.e.,
formulating solutions to a specific problem. The percentage of this category is 37.61%
 DEVELOPMENT OF ASSESSMENT TOOLS OF CRITICAL THINKING. . . 8665

and 33.42% in trial I and trial II, respectively. Such results signify that, for the students,
solving mathematical problems in the worksheets is time-consuming compared to other
activities.

F IGURE 1. The Average Percentage of Student Activity

F IGURE 2. The Outcomes of the Test of Students’ Critical Thinking Skills

8666 H. S. TANJUNG, S. A. NABABAN, C. SA’DIJAH, AND SUBANJI

Figure 2 highlights the significant increase in the number of students who pass the
standard in cycle II, thus implying the decline in the number of students who score
below the minimum mastery. The students’ critical thinking skills, from trial I to trial II,
are improved by 21.4%.

4. C ONCLUSION

From the above discussion, this research concludes that: (1) The designed assessment
instruments for assessing HOTS are deemed effective. This was evident from the results
of individual and classical learning mastery, student activities (within the bounds set
by a specific regulation), and students’ positive reception to the learning media. (2)
From trial I to trial II, the improvement in students’ critical thinking skills was 0.32 on
average, and the classical learning mastery was measured at 21.4%

ACKNOWLEDGMENT

We would like to express our gratitude to the Ministry of Research and Technol-
ogy/National Research and Innovation Agency for their financial support to this re-
search through a research grant Hibah Penelitian Kerja Sama Perguruan Tinggi (PKPT)
2020, No. Decree: No. 8/E1/KPT/2020. Also, we thank the research partners from
Universitas Negeri Malang for their guidance and cooperation throughout the course of
this present study.

R EFERENCES
[1] P ERMENDIKBUD: Penilaian Hasil Belajar oleh Satuan Pendidikan dan Penilaian Hasil Belajar oleh
Pemerintah, Permendikbud, No 4, 2018.
[2] B UDIMAN , A. JAILANI: Pengembangan Instrumen Asesmen, Higher Order Thinking Skill, (HOTS)
Pada Mata Pelajaran Matematika Smp Kelas VIII Semester 1”, Jurnal Riset Pendidikan Matematika,
2(1), 2014.
[3] B ADAN S TANDAR N ASIONAL P ENDIDIKAN (BNSP): Standar Isi Untuk Satuan Pendidikan Dasar
Dan Menengah, Permendikbud, 21, 2018.
[4] L.D. G LORIA: Pentingnya Asesmen Alternatif Dalam Meningkatkan Kemampuan Berpikir Dan Mem-
baca Ilmiah Siswa Pada Pembelajaran Biologi, Jurnal Scienteai Education, 1(1), 2012.
[5] C. S UURTAMM , D. R. T HOMPSON , R. Y OUNG K IM , L. D IAZ M ORENO, N. S AYAC , S. S CHUKA -
JLOW, P. V OS : Assessment in mathematics education: Large-scale assessment and classroom assess-
ment, Springer Nature, 2016.
[6] J. B AIRD , T. N. H OPFENBECK , P. N EWTON , G. S TOBART, A. T. S TEEN -U THEIM: State of the
field review: Assessment and learning, Norwegian Knowledge Centre for Education. Oxford, England:
Oxford University Centre for Educational Assessment, 2014.
 DEVELOPMENT OF ASSESSMENT TOOLS OF CRITICAL THINKING. . . 8667

[7] M. I. H AKIM , K. K ARTONO, L. WAHYU: Developing Assessment Instruments of Students’ Critical

Thinking Ability in Trigonometry Material Mathematics Subjects, Journal of Research and Educational
Research Evaluation, 8(1) (2019), 39-47.
[8] S. A. V ONG , W. K AEWURAI: Instructional Model Development to Enhance Critical Thinking and
Critical Thinking Teaching Ability of Trainee Students at Regional Teaching Training Center in Takeo
Province, Cambodia, Kasetsart Journal of Social Sciences, 38(1) 2017, 88-95.
[9] A. A KBAR: Membudayakan Literasi dengan Program 6M di Sekolah Dasar, Jurnal Pendidikan Sekolah
Dasar, 3(1) (2017),
[10] H. L IBERNA: Peningkatan Kemampuan Berpikir Kritis Matematis Siswa Melalui Penggunaan Metode
Improve Pada Materi Sistem Persamaan Linear Dua Variabel, Formatif: Jurnal Ilmiah Pendidikan MIPA,
2(3) (2015), 190-197.
[11] H. S APUTRA: Pengembangan Mutu Pendidikan Menuju Era Global: Penguatan Mutu Pembelajaran
dengan Penerapan HOTS. High Order Thinking Skills, Bandung: SMILE’s Publishing, 2016.
[12] T. W IDODO, S. K ADARWATI: High Order Thinking Berbasis Pemecahan Masalah Untuk Meningkatkan
Hasil Belajar Berorientasi Pembentukan Karakter Siswa, Cakrawala Pendidikan, 32(1) (2013), 161-
171.
[13] D. KURNIATI: Kemampuan Berpikir Tingkat Tinggi Siswa SMP Di Kabupaten Jember Dalam Menyele-
saikan Soal Berstandar PISA. Penelitian dan Evaluasi Pendidikan, 20(2) (2016), 142-155.
[14] S. Y EN , T. DAN H ALILI , S. H AJAR: Effective teaching of higher-order thinking (hot) in education,
The Online Journal of Distance Education and e-Learning (TOJDEL), 3(2) (2015), 41-47.
[15] M. Z. FANANI: Strategi pengembangan soal hots pada kurikulum 2013. Edudeena 2.1.2018. Badan
Standar Nasional Pendidikan (BNSP). Standar Isi Untuk Satuan Pendidikan Dasar Dan Menengah,
Permendikbud, 21 (2018), 57-76.

D EPARTMENT OF M ATHEMATICS E DUCATION

STKIP B INA B ANGSA M EULABOH
A CEH B ARAT, I NDONESIA
Email address: hnrsaputra@gmail.com

D EPARTMENT OF M ATHEMATICS E DUCATION

STKIP B INA B ANGSA M EULABOH
A CEH B ARAT, I NDONESIA
Email address: sitinababan28@gmail.com

D EPARTMENT OF M ATHEMATICS E DUCATION

U NIVERSITAS N EGERI M ALANG
M ALANG, I NDONESIA
Email address: cholis.sadijah.fmipa@um.ac.id

D EPARTMENT OF M ATHEMATICS E DUCATION

U NIVERSITAS N EGERI M ALANG
M ALANG, I NDONESIA
Email address: subanji.fmipa@um.ac.id

View publication stats