Vol 21 No 8 August 2022

IJLTER.
ORG
International Journal
of
Learning, Teaching
And
Educational Research
p-ISSN: 1694-2493
e-ISSN: 1694-2116
Vol.21 No.8
International Journal of Learning, Teaching and Educational Research
(IJLTER)
Vol. 21, No. 8 (August 2022)
Print version: 1694-2493
Online version: 1694-2116
IJLTER
International Journal of Learning, Teaching and Educational Research (IJLTER)
Vol. 21, No. 8
This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part
of the material is concerned, specifically those of translation, reprinting, re-use of illustrations,
broadcasting, reproduction by photocopying machines or similar means, and storage in data banks.
Society for Research and Knowledge Management

The International Journal of Learning, Teaching and Educational

Research is a peer-reviewed open-access journal which has been
established for the dissemination of state-of-the-art knowledge in the
fields of learning, teaching and educational research.
Aims and Objectives
The main objective of this journal is to provide a platform for educators,

teachers, trainers, academicians, scientists and researchers from over the
world to present the results of their research activities in the following
fields: innovative methodologies in learning, teaching and assessment;
multimedia in digital learning; e-learning; m-learning; e-education;
knowledge management; infrastructure support for online learning;
virtual learning environments; open education; ICT and education;
digital classrooms; blended learning; social networks and education; e-
tutoring: learning management systems; educational portals, classroom
management issues, educational case studies, etc.
Indexing and Abstracting

Research is indexed in Scopus since 2018. The Journal is also indexed in
Google Scholar and CNKI. All articles published in IJLTER are assigned
a unique DOI number.
Foreword
We are very happy to publish this issue of the International Journal of

Learning, Teaching and Educational Research.

Research is a peer-reviewed open-access journal committed to
publishing high-quality articles in the field of education. Submissions
may include full-length articles, case studies and innovative solutions to
problems faced by students, educators and directors of educational
organisations. To learn more about this journal, please visit the website
http://www.ijlter.org.
We are grateful to the editor-in-chief, members of the Editorial Board

and the reviewers for accepting only high quality articles in this issue.
We seize this opportunity to thank them for their great collaboration.
The Editorial Board is composed of renowned people from across the

world. Each paper is reviewed by at least two blind reviewers.
We will endeavour to ensure the reputation and quality of this journal

with this issue.
Editors of the August 2022 Issue

VOLUME 21 NUMBER 8 August 2022
Table of Contents
Mathematical Knowledge for Teaching in Further Education and Training Phase: Evidence from Entry Level
Student Teachers’ Baseline Assessments ............................................................................................................................. 1
Folake Modupe Adelabu, Jogymol Kalariparampil Alex
Exploring the Use of Chemistry-based Computer Simulations and Animations Instructional Activities to Support
Students’ Learning of Science Process Skills ..................................................................................................................... 21
Flavia Beichumila, Eugenia Kafanabo, Bernard Bahati
Issues Surrounding Teachers’ Readiness in Implementing the Competency-Based ‘O’ Level Geography Syllabus
4022 in Zimbabwe ................................................................................................................................................................. 43
Paul Chanda, Tafirenyika Mafugu
Exploring Headteachers, Teachers and Learners’ Perceptions of Instructional Effectiveness of Distance Trained
Teachers ................................................................................................................................................................................. 58
Vincent Mensah Minadzi, Ernest Kofi Davis, Bethel Tawiah Ababio
The Role of Middle Managers in Strategy Execution in two Colleges at a South African Higher Education
Institution (HEI) .................................................................................................................................................................... 75
Ntokozo Mngadi, Cecile N. Gerwel Proches
Learners’ Active Engagement in Searching and Designing Learning Materials through a Hands-on Instructional
Model...................................................................................................................................................................................... 92
Esther S. Kibga, Emmanuel Gakuba, John Sentongo
The Development of e-Reading to Improve English Reading Ability and Energise Thai Learners’ Self-Directed
Learning Strategies ............................................................................................................................................................. 109
Pongpatchara Kawinkoonlasate
The Role of Mother’s Education and Early Skills in Language and Literacy Learning Opportunities ................... 129
Dyah Lyesmaya, Bachrudin Musthafa, Dadang Sunendar
Exploring Assessment Techniques that Integrate Soft Skills in Teaching Mathematics in Secondary Schools in
Zambia.................................................................................................................................................................................. 144
Chileshe Busaka, Septimi Reuben Kitta, Odette Umugiraneza
Generic Competences of University Students from Peru and Cuba ............................................................................ 163
Miguel A. Saavedra-López, Xiomara M. Calle-Ramírez, Karel Llopiz-Guerra, Marieta Alvarez Insua, Tania Hernández
Nodarse, Julio Cjuno, Andrea Moya, Ronald M. Hernández
Representation of Nature of Science Aspects in Secondary School Physics Curricula in East African Community
Countries .............................................................................................................................................................................. 175
Jean Bosco Bugingo, Lakhan Lal Yadav, K.K Mashood
Using Graphic Oral History Texts to Operationalize the TEIL Paradigm and Multimodality in the Malaysian
English Language Classroom ............................................................................................................................................ 202
Said Ahmed Mustafa Ibrahim, Azlina Abdul Aziz, Nur Ehsan Mohd Said, Hanita Hanim Ismail
Remote Teaching and Learning at a South African University During Covid-19 Lockdown: Moments of
Resilience, Agency and Resignation in First-Year Students’ Online Discussions ...................................................... 219
Pineteh E. Angu
Enhancing Upper Secondary Learners’ Problem-solving Abilities using Problem-based Learning in Mathematics
............................................................................................................................................................................................... 235
Aline Dorimana, Alphonse Uworwabayeho, Gabriel Nizeyimana
The Development of Mobile Applications for Language Learning: A Systematic Review of Theoretical
Frameworks ......................................................................................................................................................................... 253
Kee-Man Chuah, Muhammad Kamarul Kabilan
The Effect of Professional Training on In-service Secondary School Physics 'Teachers' Motivation to Use Problem-
Based Learning .................................................................................................................................................................... 271
Stella Teddy Kanyesigye, Jean Uwamahoro, Imelda Kemeza
Knowledge of Some Evidence-Based Practices Utilized for Managing Behavioral Problems in Students with
Disabilities and Barriers to Implementation: Educators' Perspectives ........................................................................ 288
Hajar Almutlaq
Exploring Virtual Reality-based Teaching Capacities: Focusing on Survival Swimming during COVID-19 ........ 307
Yoo Churl Shin, Chulwoo Kim
Math Anxiety, Math Achievement and Gender Differences among Primary School Children and their Parents
from Palestine ...................................................................................................................................................................... 326
Nagham Anbar, Lavinia Cheie, Laura Visu-Petra
Investigating the Tertiary Level Students’ Practice of Collaborative Learning in English Language Classrooms,
and Its Implications at Public Universities and at Arabic Institutions ........................................................................ 345
Md Anwar, Md Nurul Ahad, Md. Kamrul Hasan
Navigating the New Covid-19 Normal: The Challenges and Attitudes of Teachers and Students during Online
Learning in Qatar ................................................................................................................................................................ 368
Caleb Moyo, Selaelo Maifala
The Classical Test or Item Response Measurement Theory: The Status of the Framework at the Examination
Council of Lesotho .............................................................................................................................................................. 384
Musa Adekunle Ayanwale, Julia Chere-Masopha, Malebohang Catherine Morena
Addressing the Issues in Democratic Civilian Control in Ukraine through Updating the Refresher Course for
Civil Servants ...................................................................................................................................................................... 407
Valentyna I. Bobrytska, Leonid V. Bobrytskyi, Andriy L. Bobrytskyi, Svitlana M. Protska
Supervisory Performance of Cooperative Teachers in Improving the Professional Preparation of Student Teachers
............................................................................................................................................................................................... 425
Ali Ahmad Al-Barakat, Rommel Mahmoud Al Ali, Mu’aweya Mohammad Al-Hassan, Omayya M. Al-Hassan
1

Vol. 21, No. 8, pp. 1-20, August 2022
https://doi.org/10.26803/ijlter.21.8.1
Received Mar 7, 2022; Revised May 31, 2022; Accepted Jul 17, 2022
Mathematical Knowledge for Teaching in

Further Education and Training Phase: Evidence
from Entry Level Student Teachers’ Baseline
Assessments
Folake Modupe Adelabu*
Walter Sisulu University, Nelson Mandela Drive, Mthatha, South Africa
Jogymol Kalariparampil Alex

Walter Sisulu University, Nelson Mandela Drive, Mthatha, South Africa
Abstract. This paper investigates entry-level student teachers'

mathematical knowledge for teaching in the Further Education and
Training phase (FET) through Baseline Assessment. The study employed
a quantitative research technique. The data collection instrument was a
mathematics subject knowledge test (Baseline Assessment) for FET phase
student teachers. Purposive and convenient sampling methods were
employed in the study. The study enlisted the participation of 222 first-
year mathematics education student teachers from a rural Higher
Education Institution (HEI) specialising in FET phase mathematics
teaching. One hundred and seventy-five (175) student teachers completed
the Baseline Assessment for all grades in this study (10, 11, and 12). The
Baseline Assessment findings were examined using descriptive statistics.
The results revealed that student teachers have a moderate knowledge of
mathematics topics in the FET phase at the entry-level. In addition, an
adequate level of understanding for teaching Grades 10 and 12 Patterns,
Functions, Algebra, Space and Shape (Geometry), and Functional
Relationships. While the elementary level of understanding for teaching
grade 10 Measurement, Grade 11 Patterns, Functions, Algebra, and
Trigonometry and Grade 12 Space and Shape (Geometry). There is no
level of understanding for teaching FET phase Data and Statistics and
Probability. The paper suggests that student teachers must develop a
comprehensive understanding of the mathematics curriculum with the
assistance of teacher educators in HEIs.
Keywords: Mathematical knowledge; student teachers’ entry-level;

Baseline Assessments; Further Education and Training
*
Corresponding author: Folake Modupe Adelabu, fadelabu@wsu.ac.za
©Authors
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0
International License (CC BY-NC-ND 4.0).
2
1. Introduction
There was much emphasis on the teachers’ content knowledge in mathematics in
2013, according to Julie (2019). The focus on content knowledge was due to the
Diagnostic Measures for the Trends in Mathematics and Science Study (TIMSS)
2011, which focused mainly on student and mathematics teacher performance in
public schools. Based on the test results, Reddy et al. (2016) concluded a need for
significant improvement in teachers' content knowledge of classroom
mathematics. They found that most teachers' lack of mathematical content
knowledge is a contributing factor to learners' poor mathematics performance in
most South African schools. According to research, several studies in developed
countries and developing countries suggest that teachers’ content knowledge for
teaching mathematics contributes significantly and is a good predictor of student
achievement (Mullens, Murnane and Willett, 1996; Altinok, 2013). (e.g. Norton
2019, Shepherd, 2013) (Monk, 1994; Wayne & Youngs, 2003; Hill, Rowan & Ball,
2005; Rivkin, Hanushek & Kain, 2005). This paper presents the findings of a
baseline assessment that investigated the mathematical subject content knowledge
of entry-level student teachers who are being trained to teach mathematics in the
Further Education and Training (FET) phase in South Africa.
The South African educational system is divided into three hierarchical phases:
General Education and Training (GET), Further Education and Training (FET), and
s Higher Education (HE). The national matriculation examination takes place at
the end of Grade 12 to mark the shift from the GET to the FET phase of schooling
(DBE, 2011). Secondary school is known as the FET phase, where learners' abilities
are improved to prepare them for careers of their choice. During this stage, learners
lay the groundwork for future success. At the end of the FET phase, learners
prepare to transition into university and higher education. According to the DBE
(2011), it is expected that all learners will have a sound foundational grasp of the
fundamentals that will assist them in choosing courses or study programmes at a
higher education institution. Therefore, at this stage, learners concentrate on
course selections consistent with their unique professional objectives and goals,
whether in Commerce, Humanities, or Sciences.
To advance to the HE level for Bachelor’s degree in South Africa, learners must
attain at least 40% minimum passes in three or four subjects, including one official
home language in the national matriculation and school-leaving examination
(DBE, 2012). Therefore, the teachers who specialise in the FET phase during the
Bachelor of Education degree teach subjects in the FET phase in secondary schools.
For example, a student teacher with a degree in FET phase mathematics learns how
to teach mathematics to learners in Grades 10 to 12. As a result, the student-teacher
devotes themselves to mathematics as a subject specialist. The student-teacher
concentrates on merging basic mathematics knowledge with efficiently
communicating the knowledge to prospective Grades 10 to 12 learners. According
to DBE (2011), the link between the Senior Phase and the Higher Education band
is FET. Therefore, all learners who complete this phase gain a functional
understanding of mathematics, allowing them to make sense of society. FET
learners get exposed to various mathematical experiences that provide them with
numerous possibilities to build mathematical reasoning and creative skills in
http://ijlter.org/index.php/ijlter
3
preparation for abstract mathematics in higher education. In this regard, the

student teachers for the FET phase need to be prepared for the task and the
comprehensive role ahead since studies show that learners' poor performance in
mathematics is due to the teachers' poor mathematical content knowledge (Pino-
Fan, Assis & Castro, 2015; Reddy et al., 2016; Siyepu & Vimbelo, 2021; Verster,
2018). In recent times, there has been increasing attention to investigating
knowledge that mathematics teachers should have to execute an adequate control
of the learners’ learning. Hence, to quantify the mathematical knowledge content
for teaching and understanding level of the student-teacher for the FET phase, this
paper reports on the Baseline Assessment that investigated the mathematical
content knowledge of entry-level FET phase student teachers for teaching
mathematics in South Africa.
Specifically, it sought an answer to the following questions:

1. What is the mathematical content knowledge of student teachers for
teaching FET phase Mathematics through baseline assessment?
2. What is the level of understanding of the entry-level student teachers'
mathematical content knowledge for teaching FET phase mathematics
through baseline assessment?
2. Literature review and theoretical framework

2.1 Mathematical Knowledge for Teaching
The theoretical framework that underpins this study is Mathematical Content
Knowledge. Mathematical Content Knowledge (MCK) was built on Shulman’s
pedagogical content knowledge by Ball and colleagues in 2005. Mathematical
Content Knowledge (MCK) is an essential factor to consider when teaching
mathematics because it influences teachers' decisions towards teaching and
learning mathematics. The entry-level mathematics subject knowledge of the
student teachers for teaching in the FET phase is crucial because it determines the
student achievement in mathematics (Reddy et al., 2016). Jacinto & Jakobsen (2020)
argues that several studies highlight that teachers should be able to teach what
they know and comprehend. Jakimovik (2013) further supports this, who states
that teachers should have the appropriate MCK for effective teaching and learning.
(According to Narh-Kert (2021), effective mathematics teachers know the
mathematics relevant to the grade level and the value of the mathematics courses
they teach. Therefore, the authors believe that the quality of FET mathematics
teaching depends on teachers' knowledge of the content in the phase.
Deborah Ball and colleagues in Michigan created a test for mathematics teachers'
professional expertise aimed at elementary school teachers in the United States
(Ball, Hill & Bass, 2005) to assess their MCK for the grades they teach. The test was
a multiple-choice measure of number and operation, pattern, function, algebra and
geometry. This test became a measure and was used to evaluate the MCK of
mathematics educators, mathematicians, professional developers, project staff,
and classroom teachers. Ball et al. (2005) discovered that teachers lack sound
mathematical knowledge and skills. The test results led to the definition of
mathematical content knowledge and its two components, Common Content
Knowledge (CCK) and Specialised Content Knowledge (SCK) (Ball et al. 2005).
4
These researchers further explained that most of the in-service mathematics

teachers in the U.S are graduates of a weak system. Therefore, there is a dire need
to improve the mathematical knowledge of educators. Ball et al. (2005) state that
the system clarifies that these in-service teachers learned mathematics with
irregularity and insufficient mathematical knowledge, leading to many teachers'
weak mathematical knowledge. To improve teachers' MCK, Ball et al. (2005) test
approach is embedded in Shulman's (1986, 1987) taxonomy of teacher knowledge.
Shulman makes a theoretical distinction between pedagogical content knowledge

(PCK), which is the knowledge of how to make the subject accessible to others, and
content knowledge (CK), which is the knowledge of deep comprehension of the
domain itself (Shulman 1986). As a result, Shulman (1986, 1987) and Ball et al.
(2005) use mathematical subject knowledge to assess teachers’ performance. Both
rely on a distinct teaching philosophy that emphasises teachers' capacity to
translate content knowledge into pedagogical strategies that help students learn
effectively. Jacinto and Jakobsen (2020) state that Mathematical Knowledge for
Teaching (MKfT) also provides a long-term theoretical foundation and practical
ramifications for teacher preparation programs. (. Hence, the theory of MKfT
proposed by Ball, Thames, and Phelps (2008) is used in this study.
According to the MK, the following domains are the key focus: common content
knowledge (CCK), horizon content knowledge (HCK), specialised content
knowledge (SCK), knowledge of content and students (KCS), knowledge of
content and teaching (KCT), and knowledge of content and curriculum (KCC)
(Jacinto & Jakobsen, 2020).
• The first domain (CCK) refers to mathematical knowledge that is
frequently utilised and created in various settings, including outside of
formal education. This form of knowledge consists of questions that can be
answered by those who know mathematics rather than specialised
understandings (Ball et al., 2008).
• CCK is demonstrated by using an algorithm to solve an addition problem.
• Horizon content knowledge (HCK) is the knowledge of "how the content
being taught fits into and is connected to the larger disciplinary domain."
This domain includes knowing the origins and concepts of the subject and
how useful it may be to students' learning. HCK allows teachers to "make
judgements about the value of particular concepts" raised by students, as
well as address "the discipline with integrity, all resources for balancing
the core goal of linking students to a large and highly developed area" (Ball
et al., 2008: 400; Jacinto & Jakobsen, 2020).
• Specialized Content Knowledge (SCK) is defined as "the mathematical
knowledge specific to the teaching profession." It entails an unusual form
of mathematical unpacking that is not required in environments other than
education. It necessitates knowledge that extends beyond a thorough
understanding of the subject matter. Teachers' roles include being able to
present mathematical ideas during instruction and responding to students'
queries, both of which necessitate mathematical expertise specific to
teaching mathematics (Ball et al., 2008: 400; Jacinto & Jakobsen, 2020).
5
• Knowledge of content and students (KCS) was another sub-construct that

needed to be redefined because it did not fit the criterion for one-
dimensionality. For instance, respondents such as teachers, non-teachers,
and mathematicians used standard mathematical procedures to answer the
items designed to reflect KCS, according to cognitive tracing interviews.
Furthermore, the use of multiple-choice items in KCS measurement was
reviewed in favour of open-ended questions.
Teachers utilise CCK to plan and teach mathematics concepts, allowing them to
evaluate students' answers, respond to concept definitions, and complete a
mathematical approach. Therefore, any adult with a well-developed CCK but not
the knowledge required to educate, such as new student teachers entering Higher
Education Institutions (HEI), may have a well-developed CCK but lack the
necessary knowledge to teach. Hence, this study investigates the mathematical
content knowledge of entry-level student teachers in the FET phase training phase
for teaching mathematics through Baseline Assessment in South Africa.
2.2. Educational assessment

Educational assessment supports knowledge, skills, attitudes, and beliefs, usually
in measurable terms. Assessment is an essential component of a coherent
educational experience (Sarka, Lijalem & Shibiru, 2017). According to Sarka et al.
(2017), assessment methods considerably influence the breadth and depth of
students' learning, that is, the approach to studying and retention, with either a
strong influence or a lack thereof. Assessments are used in a variety of ways, which
include motivating students and focusing their attention on what is essential,
providing feedback on the students' thinking, determining what understandings
and ideas that are within the zone of proximal development, and gauging the
effectiveness of teaching, including identifying parts of lessons that could be
improved. (Patterson, Parrott & Belnap, 2020).
Assessment is a process of collecting, analysing and interpreting information to

assist teachers, parents and other stakeholders in making decisions about the
progress of learners (DBE, 2011). Therefore, assessment serves a wide range of
functions, including permission to progress to the next level, classifying students'
performance in ranked order, improving their learning and evaluating the success
of a particular technique for improvement (Sarka et al., 2017). Furthermore, the
assessment goals include curriculum development, teaching, gathering data to aid
decision-making, communication with stakeholders, instructional improvement,
program support, and motivation (Pattersonet et al., 2020; Sarka et al., 2017;
Wilson, 2018).
According to the DBE (2011), there are various types of assessments. These include
formative assessment, summative assessment, diagnostic and baseline assessment.
Formative assessment is assessing students' progress and knowledge regularly to
identify learning needs and adapt teaching accordingly (Wilson, 2018). The Centre
for Educational Research and Innovation (CERI) (2008) states that teachers who
use formative assessment methods and strategies are better equipped to address
the requirements of a wide range of students. This can be done by differentiating
6
and adapting their instruction to enhance students' achievement to achieve more

significant equity in their learning outcomes. Formative assessment can also be
defined as the activity that supports learning by giving information that can be
utilised as feedback by teachers and students to evaluate themselves and each
other to improve the teaching and learning activities. Therefore, formative
assessment is one of the primary core activities in teachers' work (Wilson, 2018).
Summative assessments are used to determine what students have learned at the
end of a unit and are used as a measure for promotion purposes. Dolin et al. (2018)
state that summative assessment ensures that students have fulfilled the
requirements to achieve certification for school completion or admittance into
higher education institutions or occupations. In addition, when an assessment
activity is used to provide a summary of what a student knows, understands, and
can do rather than to aid in the modification of the teaching and learning activities
in which the student is engaged by providing feedback, it is considered summative
(CERI, 2008; Wilson, 2018). Summative assessments are used in education for a
variety of reasons. Individual students and their parents discuss progress and
receive an overall assessment that includes praise, inspiration, and guidance for
what has been accomplished. Summative assessments provide a comprehensive
guide to the effectiveness of the students' work, which may be externally
standardised ((Dolin et al., 2018; Wilson, 2018). Wilson (2018) agrees that
summative assessments assist schools in making the best possible grouping and
subject choices for the learners. Both a school and a public authority employ
summative assessments to inform teachers and the school’s accountability. As a
result, a common element of summative assessments is that the results are utilised
to guide future decisions.
The initial assessment occurs when a student begins a new learning program. The
initial assessment is a comprehensive process in which students start to piece
together a picture of an individual's accomplishments, abilities, interests, prior
learning experiences, ambitions, and the learning requirements associated with
those ambitions. The information from the initial assessment is used to negotiate a
program or course (Quality Improvement Agency (QI), 2008). Diagnostic
assessment supports the identification of individual learning strengths and
weaknesses. It provides learning objectives and the necessary teaching and
learning strategies for achievement. This is necessary because many students excel
in some areas but struggle in others. Diagnostic evaluation occurs at the start of a
learning program and again when required. It has to do with the specialised talents
needed for specific tasks. The information acquired from the initial examination is
supplemented by diagnostic testing (QIA, 2008).
Baseline assessment commonly used in early childhood education gathers

information regarding a child's development or achievement as they transition to
a new environment or grade. These assessments are conducted in various ways,
ranging from casual observations to standardised examinations. The information
gathered from these assessments assists educators in fulfilling the learner's
requirements, highlighting their strengths and areas for improvement. All these
assessments are helpful in their capacity to assess the learners. Baseline
7
assessments assist schools in understanding the students' requirements. It also aids

in determining learners' learning capability and potential and assessing the
influence the schools have on learners. Information from baseline assessment
facilitates schools in customising planning, teaching, and learning, including
determining the most effective resource allocation to track learners’ progress
throughout the school year. According to Khuzwayo and Khuzwayo (2020) and
Tomlinson (2020), the baseline assessment findings provide information to the
teacher regarding the learners’ abilities and knowledge gaps. This evidence assists
the teacher in organising learning content, selecting, and matching teaching and
learning approaches with the learning needs of individual students or groups of
students.
The three assessments (The Initial, Diagnostic, and Baseline assessments) are
interrelated in education. The assessments are always administered at the
beginning or entry of students into the school, measure the strengths and
weaknesses, and deduce places for improvement in a learner. The assessments are
embedded in formative assessment.
The baseline assessment (CAMI) utilised in this paper is in accordance with the
Curriculum and Assessment Policy Statement (CAPS) for Further Education and
Training in South Africa. The licensed online Computer Aided Mathematics
Instruction (CAMI) software is used to program the baseline assessments. CAMI
is a high-productivity software system that can improve mathematics grades in a
minimal amount of time. One of the software's functions is to correct extension
work for a more advanced student. CAMI employs the computer as a "Drill and
Practice" system rather than a tutoring system because it focuses on knowledge
retention (see www.cami.co.za).
The main mathematics topics in the FET phase are Functions; Number Patterns,
Sequences, and Series; Finance, growth, and decay; Algebra; Differential Calculus;
Probability; Euclidean Geometry and Measurement; Analytical Geometry;
Trigonometry; and Statistics. The topics constitute Papers 1 and 2 of the national
examinations in South Africa. The weighting of content areas is shown in Table 1
below:
Table 1: The weight of content areas description of FET’s mathematics topics

The weighting of Content Areas
Description Grade 10 Grade 11 Grade 12
Paper 1 (Grades 12: bookwork: maximum 6 marks)
Algebra and Equations (and Inequalities) 30 ± 3 45 ± 3 25 ± 3
Patterns and Sequences 15 ± 3 25 ± 3 25 ± 3
Finance and Growth 10 ± 3
Finance, growth, and decay 15 ± 3 15 ± 3
Functions and Graphs 30 ± 3 45 ± 3 35 ± 3
Differential Calculus 35 ± 3
Probability 15 ± 3 20 ± 3 15 ± 3
TOTAL 100 150 150
8
Paper 2: Grade 11 and 12: theorems and /or trigonometric proofs: maximum 12 marks
Description Grade 10 Grade 11 Grade 12
Statistics 15 ± 3 20 ± 3 20 ± 3
Analytical Geometry 15 ± 3 30 ± 3 40 ± 3
Trigonometry 40 ± 3 50 ± 3 40 ± 3
Euclidean Geometry and Measurement 30 ± 3 50 ± 3 50 ± 3
TOTAL 100 150 150
(Source: CAPS Documents, DBE, 2011)
3. Research methodology
3.1 Research design and sampling
A quantitative research design and methodology were used in this study. The data
collection instrument was a mathematics subject knowledge test (Baseline
Assessment by CAMI) for FET phase student teachers. The Baseline Assessment
was used to assess the entry-level student teachers' mathematical content
knowledge through online Computer Aided Mathematics Instruction (CAMI)
software. The CAMI programme is part of the ongoing research conducted in the
Mathematics Education and Research Centre established in rural higher education
(HEI) in South Africa. Two hundred and twenty-two (222) first-year mathematics
student teachers specialising in FET phase mathematics teaching participated in
the study. This paper included 175 student teachers who completed the Baseline
Assessment for all grades (10, 11, and 12). Purposive and convenience samplings
were utilised to collect data. Participation in the CAMI Baseline Assessment was
done in a controlled environment in an invigilated computer lab for two weeks.
The majority of the student teachers enrolled in the FET Teaching Bachelor of
Education Course came from rural secondary schools and had not experienced
computer-assisted learning.
3.2 Data collection

3.2.1 Baseline Assessment through CAMI
Computer-Aided Mathematics Instruction (CAMI) baseline assessment is an
online assessment available in the CAMI EduSuite program (further information
is available from www.cami.co.za). The FET baseline assessment consisted of a 60-
minute online test with 25 items that student teachers can easily access through
internet connectivity. CAMI was installed on the lab computers, and all student
teachers participating in the FET Mathematics courses were given credentials to
log in and access the FET Baseline Test (Grades 10, 11, and 12). After completing
the Baseline Assessment, the teacher can access their results.
The navigation to the FET Baseline Test on the CAMI package is illustrated in the
figure below.
9
Figure 1: The navigation to the senior phase Baseline Test on the CAMI package
After logging into the system, student teachers should go to the Assessment box
and click ‘Do assessment’, which will bring up the Baseline and Grades
assessments. After that, the student teachers choose Grades 10, 11, and 12 from the
Baseline Assessment and complete the test items one by one, as shown in figure 1.
Each of the Baseline Assessments for Grades 10, 11, and 12 has 25 items.
3.3. Data analysis

The findings of the Baseline Assessment were analyses using descriptive statistics.
The frequency distributions were used to establish the mathematical content
knowledge and the level of understanding of the contents for teaching
mathematics in the FET phase. One-way ANOVA was used to establish the
variability of the mean performance of the student teachers from grade to grade.
Because the program includes the Baseline Assessment, all the questions on each
grade are valid. All ethical requirements were completed, and the student teachers
participated (Ethical Clearance Number: FEDSRECC001-06-21).
Below are some of the sample items from the CAMI Baseline Assessment.
Figure 2: assessment items no. 9 and 16 (source: www.cami.co.za)
10
Figure 3: assessment items no. 20 and 25 (source: www.cami.co.za)
According to international benchmarks, 60 per cent was used as the

understanding level of mathematical content knowledge in the FET phase in this
study. The national codes and descriptions of the percentages that qualify learner
performance can be found in Table 2 (DBE, 2011).
Table 2: Codes and percentages for recording and reporting in Grades R-12
performances
Achievement level Achievement description Marks %
7 Outstanding achievement 80 – 100
6 Meritorious achievement 70 – 79
5 Substantial achievement 60 – 69
4 Adequate achievement 50 – 59
3 Moderate achievement 40 – 49
2 Elementary achievement 30 – 39
1 Not achieved 0 – 29
(Source: DBE, 2011)
According to the benchmarking, "Substantial achievement" was the minimum

score for student teachers' subject content knowledge mastery at a specific grade
level.
4. Results
4.1. Baseline assessment of the mathematical content knowledge of student
teachers for teaching FET phase Mathematics
The mean of the Baseline Assessment in the three grades of the FET phase was
determined using a one-way single factor ANOVA. The following tables depict the
outcome:
Table 3: ANOVA Summary table
Groups Count Sum Average Variance
Grade 10 175 7832 44.75429 113.3588
Grade 11 175 6580 37.6 166.9885
Grade 12 175 5756 32.89143 171.7985
11
Table 4: One-way ANOVA single factor

Source of Sum of Df Mean F P-value F crit
Variation Squares Squares
Between Groups 12488.11 2 6244.053 41.42947 2E-17 3.012991
Within Groups 78673.37 522 150.7153
Total 91161.48 524
Notes: Df – Degree of freedom; P-value: p < 0.05
As shown in Table 3, the mean strengths range from 32.89 for Grade 12 to 44.75 for
Grade 10, indicating that the sample means are different. That is to say; the average
score is not the same. Table 4 shows that the p-value of 2 ×10-17 is less than the
significant level of 0.05, implying that the Baseline Assessment mean scores for
FET student teachers are not equal. This means that student teachers' average
performance in the FET phase varies from grade to grade. The mean percentage
scores of student teachers in the FET phase Baseline Assessment are shown in the
graph below.
Figures 4: The mean percentage scores of the student teachers in the FET phase
Baseline Assessment according to content areas
The mean percentage scores of student teachers in the FET phase Baseline
Assessment according to the content areas are shown in Figure 4 above. The results
revealed that the students' mean percentage in Space and Shape (Geometry) Grade
10 was 59.18%, Patterns, Functions, and Algebra at 50.96%, measurement at 36.48
per cent, data and statistics at 19.13%, and probability at 2.55%. Patterns,
Functions, and Algebra (39.59%), Trigonometry (30.35%), and Space and Shape
(Geometry) (27.69%) are the average percentage scores of student teachers in
Grade 11. The student teachers had the highest mean percentage in Functional
Relationships Grade 12 (54.28%), 38.07%percent in Space and Shape (Geometry),
27.60% in Trigonometry, and 22.68% in Patterns, Functions, and Algebra (see
Figure 4).
12
According to the above findings, student teachers scored better in Grade 10

concepts than in Grades 11 and 12 during the FET phase. Patterns, Functions, and
Algebra in Grade 12 and Measurement and Space and Shape (Geometry) in Grade
11 were all below average. Students in Grades 11 and 12 should study
trigonometry and functional relationships, whereas, in Grade 10, students should
study Data Statistics and Probability. The frequency distribution of the student
teachers' achievements was analysed to corroborate the study's findings. Figure 5
depicts the frequency distribution of student-teacher marks for the FET phase:
Figure 5: Frequency distribution of the student teachers’ achievements in Grades 10,

11, and 12
The percentage marks from the CAMI Baseline Assessment for Grades 10, 11, and
12 for entry-level student teachers are shown in different percentiles in Figure 5.
As indicated in the graph, most student teachers' achievements for Grades 10, 11,
and 12 are within 40% and 49% of each other, corresponding to 66, 56, and 48 in
Grades 10, 11, and 12, respectively. For the three FET Grades (10, 11 and 12), the
number of student-teacher marks above 50% is 61, 29, and 18. The number of
student teachers with scores below 30% in Grades 10, 11, and 12 is 16, 56, and 71.
In Grades 10, 11, and 12, - 32, 34, and 38, student teachers within 30 per cent and
39 per cent, respectively. In Grades 10 and 12, no student-teacher receives a score
higher than 70%. In Grade 11, just two student teachers receive a score of more
than 70%. The signal denotes moderate achievement in Grades 10, 11, and 12.
According to national codes and descriptions (DBE, 2011), the number of 'not
achieved' student teachers in the FET phase Baseline Assessment is 16, 56, and 71
in Grades 10, 11, and 12, respectively as shown in Figure 5 above. In Grades 10, 11,
and 12; 32, 34, and 38 of the student teachers have elementary achievement, 66, 56,
and 48 have moderate achievement, 43, 18, and 17 have adequate achievement, 18,
9 and 1 have substantial achievement, respectively and just two have meritorious
achievement at Grade 11 level. In the FET phase Baseline Assessment, no student-
teacher achieved the outstanding achievement (80% and above). According to the
findings, the student teachers have a moderate level of accomplishment. As a
13
result, student teachers' entry-level mathematical content knowledge in the FET

phase is of modest achievement.
4.2. Level of understanding of student teachers’ mathematical content

knowledge for teaching FET phase mathematics through baseline assessment
Table 5 shows the student teachers' mathematical content knowledge level for
teaching the FET phase in each grade according to the content areas.
Table 5: The understanding level of student teachers’ mathematical content knowledge

for teaching FET phase mathematics according to content areas
Achievement Achievement Grade 10 Grade 11 Grade 12
level description
7 Outstanding - - -
achievement
6 Meritorious - - -
achievement
5 Substantial - - -
achievement
4 Adequate Patterns, - Functional
achievement Functions, and Relationships
Algebra; Space
and Shape
(Geometry)
3 Moderate - - -
achievement
2 Elementary Measurement Patterns, Space and Shape
achievement Functions, and (Geometry)
Algebra;
Trigonometry
1 Not achieved Data and Data and Data and
statistics; statistics; Space statistics;
Probability; and Shape Measurement;
Trigonometry; (Geometry); Probability;
Functional Measurement Patterns,
Relationships Probability; Functions, and
Functional Algebra;
Relationships Trigonometry
The results given in Table 5 show the level of understanding of the student teachers
according to the content areas. The findings revealed that student teachers have an
adequate level of understanding of Patterns, Functions, Algebra and Space and
Shape (Geometry) in Grade 10 and Functional Relationships in Grade 12.
Furthermore, the student teachers have an elementary level of understanding of
Measurement in Grade 10, Patterns, Functions, and Algebra, Trigonometry in
Grade 11, and Space and Shape (Geometry) in Grade 12. The student teachers have
no level of understanding of Data and statistics and Probability in any of the
grades, that is, Grade 10, 11 and 12. The finding indicated that the level of
understanding of the student teachers’ mathematical content knowledge for
teaching Grade 10 Patterns, Functions, and Algebra, as well as Space and Shape
(Geometry) and Grade 12 Functional Relationships, is adequate level. While the
level of understanding of the student teachers’ mathematical content knowledge
14
for teaching Grade 10 Measurement, Grade 11 Patterns, Functions, Algebra, and

Trigonometry and Grade 12 Space and Shape (Geometry) is elementary level. In
addition, the results revealed that the student teachers did not have sufficient
understanding of the mathematical content knowledge for teaching FET phase
Data and Statistics and Probability.
5. Discussion
The evidence can be drawn from the findings that the entry-level student teachers'
mathematical knowledge for the FET phase is at the 'moderate level' of
achievement. In contrast, the actual level of understanding was not attainable.
However, the findings in table 5 revealed an adequate level of understanding of
the entry-level student teachers’ mathematical content knowledge for teaching
grades 10 and 12 Patterns, Functions, and Algebra, Space and Shape (Geometry)
Functional Relationships. Elementary level of understanding for teaching Grade
10 Measurement, Grade 11 Patterns, Functions, and Algebra, including
Trigonometry and Grade 12 Space and Shape (Geometry). The entry-level student
teachers do not have adequate mathematical content knowledge for teaching FET
phase Data and Statistics and Probability.
The result of the mean percentage from the Baseline Assessment (Figure 4)
determined the mathematical content knowledge of the student teachers to be in
Grade 10 Space and Shape (Geometry) and Patterns, Functions, and Algebra with
(59.18%) and (50.96%) respectively as well as Grade 12 Functional Relationships
with (54.28%). Similar results were obtained by Fonseca, Maseko, and Roberts
(2018) in their study ‘Students’ mathematical knowledge in a Bachelor of
Education (Foundation or intermediate phase) programme’ that there is a good
distribution of attainment for the first-year students in their pilot test. In contrast,
the findings in this study disagree with Alex and Roberts (2019), where low
percentage performance and poor mathematical knowledge for teaching were
recorded in their research. There is a need to improve entry-level first-year student
teachers’ mathematical content knowledge. The finding also revealed that none of
the student teachers achieved the “outstanding achievement", and only two have
“meritorious achievement” at Grade 11 level.
The results of the student teachers' level of understanding are in agreement with
Reid and Reid (2017). They found that student teachers had difficulty
understanding mathematical content knowledge, such as probability and standard
algorithms. According to the above researchers, the student teachers performed
below the expected standard. As a result, student teachers must have a strong
understanding of mathematical concepts and be able to express and explain them
in a variety of ways in their future teaching.
According to studies, the primary purpose of a baseline assessment in teaching

and learning is to get to know students at the entry level of a new school year
(Khuzwayo & Khuzwayo, 2020; Nguare, Hungi & Matisya, 2018; Tiymms, 2013;
Tomlinson, 2020). Therefore, the goal of baseline assessment in this study is to
assist HEIs teacher educators in developing learning activities inclusive of various
learning styles. This would also assist in detecting student teachers' special needs
15
at an early stage so that a remediation program can be implemented (DBE, 2019).

Taylor (2021) states that in South Africa, a vicious-cycle system problem is evident.
Due to the negative public perception of teaching, ITE programs cannot attract
competent matriculants to study for a teaching qualifications. Most of the students
intending to study teaching as a career are often rejected in their first and second
choices at the university level. Often universities are forced to recruit a lesser
quality of pre-service teachers into the programme, which demands a reduction in
the rigour of their training. A lower-quality or competent teacher is thus deployed
into schools, resulting in poor quality teaching, thus lowering the learner
performance and the prestige of the teaching profession. Matriculant quality while
also lowering the perceived prestige of teaching. Taylor & Robinson (2016) opine
that the inability to recruit qualified pre-service teachers enhances the cycle of poor
quality teaching and learning.
According to Deacon (2016), the entrance requirements for Initial Teacher

Education programs are generally lower than most other entry-level degree
programs. The evidence suggested that the weakest students enter education
faculties as a last resort, motivated by a desire to earn a university qualification
rather than a desire to make a difference in students' lives. Taylor (2021) supported
his claim with data from the Centre for Educational Testing for Access and
Placement's National Benchmark Tests (NBTs) (CETAP, 2020). Most university
applicants take the NBTs, which require a minimum to gain admission into a
particular programme. However, this is not applicable to most Initial Teacher
Development Programme. Over 75 000 university applicants took the Academic
Literacy (AL) and Quantitative Literacy (QL) examinations, while over 58 000 took
the Mathematics Test (MAT) during the 2019 NBT entry cycle. Candidates
planning to study Education had the second-lowest average score of all
applications to all faculties, with only those intending to study Allied Healthcare
or Nursing having a lower average (CETAP, 2020). Basic, Intermediate, and
Proficient are the three tiers of NBT scores, with applicants in the Basic band
defined as:
“Test performance reveals serious learning challenges: it is predicted that
students will not cope with degree-level study without extensive and
long-term support, perhaps best provided through bridging programmes
(i.e., non-credit preparatory courses, special skills provision) or FET
provision. Institutions admitting students performing at this level need
to provide such support themselves.” (CETAP, 2020, p. 18).
Due to the low mathematics achievement of students entering teacher education

programs, the goal of creating a deep understanding of mathematics required for
teaching should become an essential aspect of the mathematics course design and
implementation (Jakimovik, 2013). Furthermore, Jakimovik (2013) claims that the
complete lack of a link between mathematics and methods courses is a long-
standing trend in teacher preparation programs. The only stipulation is that
students complete the mathematics course’s exams before enrolling in the methods
courses. The mathematics courses are taught by university mathematicians and
academics who teach the techniques courses, which place less emphasis on the
interaction between subject matter expertise and teaching.
16
According to Ma (1999), teachers should possess a Profound Understanding of

Fundamental Mathematics (PUFM). This means teachers' mathematical content
knowledge should be a thorough understanding of mathematics that has breadth,
depth, connectedness, and thoroughness, not on the average level. Jakimovik
(2013) maintains that one of the most critical aspects of teaching is understanding
what will be taught. In addition, mathematics is one of the fundamental realms of
human thought and investigation. Learners need to build intellectual resources for
knowing about and actively engaging in mathematics. The above researcher
explains that the future teachers must use their mathematical knowledge in
conducting classroom discourse in a learning community, mentioning students'
educational needs by involving them in genuine mathematics learning, analysing
students' productions, examining students' mathematical knowledge and skills in
lesson preparation, or in evaluating curriculum materials. Consequently, to
provide successful learning for future teachers, educators must establish
specialised instructional methodologies in the HEIs.
According to Burghes and Geach (2011), the requirements for being a good
mathematics teacher are confidence, competency, commitment and a passion for
mathematics at a level much higher than the one being taught. Furthermore,
knowledge of the topic to be taught is a significant factor in determining the quality
of training. Goldsmith, Doerr and Lewis (2014) believe that teacher’s capacity to
recognise and analyse student’s thinking also their ability to engage in effective
professional conversations are hampered by a lack of mathematical content
understanding.
6. Conclusions
In conclusion, to become a FET mathematics teacher, student teachers must be
exposed to many mathematical experiences. They should be offered a variety of
opportunities to hone their mathematical reasoning and creative abilities in
preparation for teaching mathematics in the FET phase. Their low level of
mathematical knowledge and understanding may make it difficult for the student
teachers to teach the FET phase in the future. To teach in the FET phase, student
teachers must have mathematical solid foundational knowledge and
understanding. Since FET is the link between the Senior Phase and the Higher
Education band, the student teachers should have an appropriate achievement
level, namely, adequate, substantial, meritorious, and outstanding achievement
level, to link FET learners to the Higher Education band.
Consequently, student teachers will need to improve their ability to teach

mathematics effectively and ensure that it is meaningful for learners. They will be
able to effectively teach mathematics in the future, even further than their current
level of knowledge and ability. Then the mathematics performance of the learners
will improve.
7. Recommendations
This paper showed that the mathematical content knowledge of the student
teachers at the entry-level is at a moderate level, and the level of understanding
was low. Therefore, this paper recommends that with the assistance of teacher
17
educators in HEIs, student teachers must gain a thorough understanding of the

mathematics curriculum. Furthermore, the mathematics appropriate to the grade
level and mathematical courses that the student teachers are responsible for
teaching should be known and well understood. This study also recommends that
only those students who have attained substantial achievement in mathematics
should be allowed to study FET mathematics at higher education institutions. HEI
should consider those students who have applied for teaching as their 1st option
rather than their last option as an entry requirement. Stricter entry-level to FET
teaching programmes should be implemented at HEIs, such as good mathematics
attainment levels in the matriculation examination. Finally, every university
should build into their entry-level programme a 'Baseline assessment’ for all
students intending to study towards teaching mathematics in the FET phase.
8. Implications and contribution of the study

In conclusion, the authors believe that teachers with a low entry-level and a low
level of understanding will have poor content knowledge of mathematics. As a
result, there will be ineffective classroom teaching and poor mathematics
performance in secondary schools. Therefore, for learner performance
improvement, HEIs and the Department of Higher Education and Training
(DHET) should ensure that student teachers have a solid entry-level level of
understanding of the mathematics curriculum. Student teachers' entry level
should be investigated for all educational system stages, including general
education and training, further education and training, and higher education for
future studies.
9. Limitations of the study

This research is confined to student teachers who enrolled in a FET mathematics
teaching programme and came from poor, disadvantaged backgrounds. The
majority of the student teachers had not experienced computer-assisted learning,
which may have contributed to their performance in the baseline test.
10. References
Alex, J., & Roberts, N. (2019). The need for relevant initial teacher education for primary
mathematics: Evidence from the Primary Teacher Education project in South
Africa. In Proceedings of the 27th Conference of the Southern African Association for
Research in Mathematics, Science and Technology Education (59-72).
Altinok, N. (2013). The Impact of Teacher Knowledge on Student Achievement in 14 Sub-Saharan
African Countries. Retrieved from: http://unesdoc.unesco.org/images/0022/
002258/225832e.pdf
Ball, D. L., Hill, H.C., & Bass, H. (2005). Knowing mathematics for teaching: Who knows
mathematics well enough to teach third grade, and how can we decide? American
Educator, 29(1), 14-17, 20-22, 43-46.
https://deepblue.lib.umich.edu/bitstream/handle/2027.42/65072/Ball_F05.pdf
Ball, D. L., Thames, M. H., & Phelps, G. (2008). Content knowledge for teaching: What
makes it special. Journal of teacher education, 59(5), 389-407. https://www.cpre.org/
Behrman, J. R., Ross, D., & Sabot, R. (2008). Improving quality versus increasing the
quantity of schooling: Estimates of rates of return from rural Pakistan. Journal of
Development Economics, 85, 94–104.
https://reader.elsevier.com/reader/sd/pii/S0304387806001301
18
Burghes, D., & Geach, R. (2011). International comparative study in mathematics training:
Recommendations for initial teacher training in England. CfBT Education Trust.
https://www.nationalstemcentre.org.uk/res/documents/page/International%2
0comparative%20study%20in%20mathematics%20teacher%20training.pdf/.
Centre for Educational Research and Innovation (CERI) 2008. Assessment for Learning
Formative Assessment. OECD/CERI International Conference “Learning in the
21st Century: Research, Innovation and Policy 15 – 16 May 2008
CETAP, (2020). The national benchmark tests. National report: 2019 intake cycle, Centre for
Educational Testing for Access and Placement, University of Cape Town.
https://nbt.uct.ac.za/sites/default/files/NBT%20
National%20Report%202019.pdf.
Deacon, R. (2016). The initial teacher education research project: Final report.
Johannesburg: JET Education Services. https://www.admin.jet.org.za
Department of Basic Education. (2011). Curriculum assessment policy statements Grades 10 –
12 Mathematics.
Department of Basic Education. (2012). Curriculum assessment policy statements.
http://www.education.gov.za/Curriculum/NCSGradesR12/CAPS/tabid/420/
Default.aspx
Department of Basic Education. (2019). Mathematics Teaching and Learning Framework for
South Africa: Teaching and Learning mathematics for understanding. Department
Printers: Pretoria RSA. https://www.jet.org.za
Dolin, J., Black, P., Harlen, W., & Tiberghien, A. (2018). Exploring Relations Between
Formative and Summative Assessment. In J. Dolin & R. Evans (Éd.), Transforming
Assessment (Vol. 4, p. 53 80). Cham, Switzerland: Springer International
Publishing. https://doi.org/10.1007/978-3-319-63248-3
Fonseca, K., Maseko, J., & Roberts, N. (2018). Students’ mathematical knowledge in a
Bachelor of Education (Foundation or intermediate phase) programme.
In Govender, R. & Jonquière, K. (2018) Proceedings of the 24th Annual National Congress
of the Association for Mathematics Education of South Africa (124-139).
Goldsmith, L., Doerr, H., & Lewis, C. (2014). Mathematics teachers’ learning: A conceptual
framework and synthesis of research. Journal of Mathematics Teacher Education,
17, 5–36. https://link.springer.com/content/pdf/10.1007/s10857-013-9245-4.pdf
Hill, H. C., Rowan, B., & Ball, D. L. (2005). Effects of Teachers’ Mathematical Knowledge
for Teaching on Student Achievement. American Educational Research Journal, 42,
371–406. https://www.jstor.org/stable/pdf/3699380.pdf
Jacinto, E. L., & Jakobsen, A. (2020). Mathematical Knowledge for Teaching: How do
Primary Student Teachers in Malawi Understand it? African Journal of Research in
Mathematics, Science and Technology Education, 24(1), 31-40.
Jakimovik, S. (2013). Measures of mathematical knowledge for teaching and university
mathematics courses design. u: Kiryakova V.[ur.]. In Complex Analysis and
Applications' 13'(Proc. Intern. Conf. Sofia, 2013), Institute of Mathematics and
Informatics, Bulgarian Academy of Sciences (118-138).
Julie, C. (2019). Assessment of teachers’ mathematical content knowledge through large-
scale tests: What are the implications for CPD? Caught in the act: reflections on
continuing professional development of mathematics teachers in a collaborative
partnership, chapter 3, 37-53.
Khuzwayo, M. E., & Khuzwayo, H. B. (2020). Baseline Assessment in the Elementary
Mathematics Classroom: Should it be Optional or Mandatory for Teaching and
Learning? International Journal of Learning, Teaching and Educational Research, 19(8),
330-349. https://doi.org/10.26803/ijlter.19.8.18
Ma, L. (1999). Knowing and teaching elementary mathematics: Teachers’ understanding of
fundamental mathematics in China and the United States. Mahwah: Lawrence
Erlbaum Associates Inc.
19
Mullens, J. E., Murnane, R. J., & Willett, J. B. (1996). The Contribution of Training and
Subject Matter Knowledge to Teaching Effectiveness: A Multilevel Analysis of
Longitudinal Evidence from Belize. Comparative Education Review, 40(2), 139–157.
https://www.jstor.org/stable/pdf/1189048.pdf
Narh-Kert, M. (2021). Predictive Validity of Entry-Level Mathematics: Mathematical Knowledge
of Student Teachers for Teaching Basic School Mathematics in Ghana. LAP LAMBERT
Academic Publishing.
Nguare, M. W., Hungi, N., & Mutisya, M. (2018). Assessing Learning: How can classroom-
based teachers assess students' competencies in Numeracy. Journal of Assessment in
Education, 26(2), 222-244. https://doi.org/10.1080/0969594X.2018.1503156
Patterson, C. L., Parrott, A., & Belnap, J. (June 2020). Strategies for assessing mathematical
knowledge for teaching in mathematics content courses. In A. Appova, R. M.
Welder, and Z. Feldman, (Eds.), Supporting Mathematics Teacher Educators’
Knowledge and Practices for Teaching Content to Prospective (Grades K-8)
Teachers. Special Issue: The Mathematics Enthusiast, (17) 2 & 3, 807–842. Scholar
Works: University of Montana. https://scholarworks.umt.edu/tme
Pino-Fan, L. R., Assis, A., & Castro, W. F. (2015). Towards a methodology for the
characterization of teachers’ didactic-mathematical knowledge. EURASIA Journal
of Mathematics, Science and Technology Education, 11(6), 1429-1456.
Reddy, V., Visser, M., Winnaar, L., Arends, F., Juan, A., Prinsloo, C. & Isdale, K. (2016).
TIMSS 2015: Highlights of Mathematics and Science Achievement of Grade 9 South
African Learners (Nurturing green shoots). Pretoria: Human Sciences Research
Council.
Reid, M., & Reid, S. (2017). Learning to be a Math Teacher: What Knowledge is Essential?
International Electronic Journal of Elementary Education, 9(4), 851-872.
https://www.generationready.com/white-papers/what-is-effective-teaching-of-
mathematics
Rivkin, S. G., Hanushek, E. A., & Kain, J. F. (2005). Teachers, Schools, and Academic
Achievement. Econometrica, 73, 417–458.
https://onlinelibrary.wiley.com/doi/pdf/10.1111/j.1468-0262.2005.00584
Sarka, S., Lijalem, T., & Shibiru, T. (2017). Improving Academic Achievement through
Continuous Assessment Methods: In the Case of Year Two Students of Animal and
Range Sciences Department in Wolaita Sodo University, Ethiopia. Journal of
Education and Practice, 8(4), 1-5. https://files.eric.ed.gov/fulltext/EJ1133029.pdf
Shepherd, D. L. (2013, June). The impact of teacher subject knowledge on learner
performance in South Africa: A within-pupil across-subject approach.
In International Workshop on Applied Economics of Education, Cantanzaro (1-32).
http://www.iwaee.org/papers%20sito%202013/Shepherd.pdf
Siyepu, S. W., & Vimbelo, S. W. (2021). Student teachers’ mathematical engagement in
learning about the total surface areas of geometrical solids. South African Journal of
Education, 41(2), 1-13. https://doi.org/10.15700/saje.v41n2a1837
Taylor, N. (2021). The dream of Sisyphus: Mathematics education in South Africa. South
African Journal of Childhood Education 11(1), a911.
https://doi.org/10.4102/sauce.v11i1.911
Taylor, N., & Robinson, N. (2016). Towards teacher professional knowledge and practise
standards in South Africa. A report commissioned by the Centre for Development and
Enterprise. https://www.researchgate.net/profile/Nick-Taylor-
23/publication/339592811
Tomlinson, C. A. (2020). Fulfilling the promise of the differentiated classroom: Strategies and tools
for responsive teaching. Alexandria. Retrieved from
http://www.ascd.org/publications
20
Tymms, P. (2013). Baseline Assessment and Monitoring in Primary schools: Achievement,

Attitudes, and Value-added indicators. New York: Routledge. -
https://www.api.taylorfrancis.com
Verster, J. (2018). Experiences of learning to become a further education and training mathematics
teacher’ case study (Doctoral dissertation, Cape Peninsula University of
Technology).
Wayne, A. J., & Youngs, P. (2003). Teacher Characteristics and Student Achievement Gains:
A Review. Review of Educational Research, 73, 89–122.
https://www.jstor.org/stable/pdf/3516044.pdf
Wilson, M. (2018). Making measurement important for education: The crucial role of
classroom assessment. Educational Measurement: Issues and Practice, 37(1), 5-20.
https://onlinelibrary.wiley.com/doi/full/10.1111/emip.12188
21

Vol. 21, No. 8, pp. 21-42, August 2022
Received Mar 30, 2022; Revised July 12, 2022; Accepted July 27, 2022
Exploring the Use of Chemistry-based Computer

Simulations and Animations Instructional Activities to
Support Students’ Learning of Science Process Skills
Flavia Beichumila
African Centre of Excellence for Innovative Teaching and Learning Mathematics
and Science (ACEITLMS), University of Rwanda, College of Education
Eugenia Kafanabo
University of Dar es Salaam, School of Education, Tanzania
Bernard Bahati
University of Rwanda, College of Education, Rwanda
Abstract. This study aimed at exploring the instructional activities that

could support students’ learning of science process skills by using
chemistry-based computer simulations and animations. A total of 160
students were randomly selected and 20 teachers were purposively
selected to participate in the study. Data were gathered in both qualitative
and quantitative formats. This was accomplished through the use of a
classroom observation checklist as well as a lesson reflection sheet. The
qualitative data were analyzed thematically, while the quantitative data
were analyzed using percentages. The key findings from the study
indicated that chemistry-based computer simulations and animations
through instructional activities, particularly formulating hypotheses,
planning experiments, identifying variables, developing operational
definitions and interpretations, and drawing conclusions, support
students in learning science process skills. It was found that during the
teaching and learning process, more than 70% of students were able to
perform well in the aforementioned types of instructional activities, while
60% performed well in planning experiments. On the other hand, as
compared to other instructional activities, planning experiments was least
observed among students and teachers. Students can be engaged in
knowledge construction while learning science process skills through the
use of chemistry-based computer simulations and animations
instructional activities. Therefore, the current study strongly
recommends the use of chemistry-based computer simulations and
animations by teachers to facilitate students’ learning of chemistry
concepts in Tanzanian secondary schools.
Keywords: chemistry-based computer simulations; instructional

activities; science process skills
©Authors
22
1. Introduction
The possibility of involving students in the acquisition of knowledge and scientific
skills, particularly science process skills (SPSs), has grown in importance in
chemistry curricula globally (Aydm, 2013; Bete, 2020). This is owing to the science
process skills' alignment with students' learning and application in everyday life.
As a result, different countries' chemistry curricula include science process skills
in both basic and integrated SPSs. Basic SPSs includes observing, classifying,
measuring, calculating, inferring, and communicating. Integrated SPSs include
formulating hypotheses, identifying and controlling variables, designing
experiments, data recording and interpretation (Abungu et al., 2014; Athuman,
2019; Aydm, 2013). During chemistry teaching and learning, effective
instructional strategies that engage students in inquiry activities are essential for
the development of science process skills. Therefore, inquiry-based approaches to
teaching and learning, such as practical work and hands-on activities, are critical
for engaging students in active learning (Abungu et al., 2014; Irwanto et al., 2018;
& Seetee et al., 2016).
Chemistry includes abstract concepts such as chemical kinetics, equilibrium and

energetics which students find difficult to learn (Lati et al., 2012). Along the same
line, teacher-centeredness dominates chemistry teaching and learning in
Tanzanian classrooms, with the teacher remaining the primary source of
information through the chalk-and-talk technique. Moreover, inquiry learning
tasks such as observations, hypotheses, testing, data collection, interpretations,
discourse, and conclusions are similarly restricted in the learning process (Kalolo,
2015; Kinyota, 2020). Consequently, memorization learning persists, and there is
little effort to support learners with science process skills (Mkimbili et al., 2018;
Kinyota, 2020; Semali & Mehta, 2012). In this regard, inappropriate teaching
strategies which rely on teacher-centeredness and occasional practical work,
shortages of laboratories and teaching aids, as well as large class size, are among
the contributing causes (Mkimbili et al., 2018; Semali & Mehta, 2012).
Chemistry-based computer simulations and animations are examples of an

information and communication technology (ICT) invention that has been
explored and used as alternative teaching and learning resources in classrooms
globally. Computer simulations are computational models of real or hypothesized
situations or natural phenomena that allow users to explore the implications by
manipulating or changing parameters within them (Nkemakolam et al., 2018). In
addition, animations are dynamic displays of graphics, images, and colors that
are used to create certain visual effects over a series of frames (Trindade et al.,
2002). Computer simulations and animations include virtual laboratories and
visualizations of phenomena. Further, the interactivity feature of computer
simulations in involving students in hands-on activities has promoted their
importance as they are essential for inquiry learning and a learner-centered
environment in the classroom (Moore et al., 2014; Plass et al., 2012). Based on the
significance of ICT, the competence-based curriculum in Tanzania recommends
the availability and use of ICT, including computer simulations and animations.
This is to ensure smooth teaching and learning as well as giving learners real-
world experience in learning (MoEST, 2015; MoEST, 2019).
23
Despite Tanzania's government's initiatives to integrate ICT into classrooms, little

is known about how chemistry content may be presented effectively in an inquiry-
based setting (Ngeze, 2017). ICT uses encompasses specific instructional strategies
that support students in learning science process skills through inquiry learning
in the chemistry classroom. This follows the fact that blending proper
instructional activities when using computer simulations is an important factor in
engaging students in learning chemistry concepts and specific science process
skills (Çelik, 2022). The reviewed literature (Beichumila et al., 2022; Çelik, 2022;
Moore et al., 2014) advocates the use of computer simulations and animations in
chemistry learning to improve students’ acquisition of science process skills.
In the above regard, Çelik (2022) and Sreelekha (2018) emphasize teaching
strategies for students to acquire science process skills through computer
simulations and animations. In such a learning context, little is known about
instructional strategies that support the learning of these integrated science
process skills through computer simulations and animations. Therefore, the goal
of this study was to investigate the chemistry-based computer instructional
activities used to engage students in building integrated science process skills
during chemistry teaching and learning. The study sought to address the
following research question: What are the chemistry-based computer simulation
and animation instructional activities used to engage students in building
integrated science process skills during chemistry teaching and learning?
2. Literature Review
2.1 Chemistry-computer simulations, animations and science process skills
development
The interactivity feature of computer simulations and animations has ability to
enable students to observe process, events, and activities during learning
(Smetana & Bell, 2012). As students interact with computer simulations and
animations, they become engaged in the exploration of the world around them
through inquiry activities (Moore et al., 2014). In this sense, students get the
opportunity to engage in inquiry learning and gather scientific evidence that are
important for learning science concepts. Through computer simulations students
develop scientific knowledge as well as science process skills (Beichumila et al.,
2022; Çelik, 2022; Supriyatman & Sukarino, 2014). However, aspects of inquiry are
not the focus in most of the lessons in science classrooms. As a result, instructional
strategies as advocated by Yadav and Mishra (2013) in teaching and learning
processes are critical towards using any inquiry-based approach, including
computer simulations and animations to develop science process skills. Students
learn less in terms of science process skills by using computer simulations in a
teacher-centered format in which students’ complete recipe-type tasks that require
them to verify solutions (Çelik, 2022; Smetana & Bell, 2012). Thus, instructional
activities for inquiry learning are important.
2.2 The importance of instructional activities and development of science

process skills
Instructional activities relate to all activities that support the teaching and learning
process (Akdeniz, 2016). These instructional activities are teaching and learning
activities and assessment activities that play a significant role in engaging
24
students in the construction of knowledge and the acquisition of skills.

Instructional activities that engage teachers in explaining or lecturing students
while students are passive listeners do not help students to acquire science process
skills. One way to develop the science process skills among students is to use
appropriate instructional activities that engage students in inquiry activities (Bete,
2020; Coil et al., 2010; Irwanto et al., 2018; Seok, 2010). Activating students'
background knowledge, offering analogies, asking questions, and encouraging
students to use alternative forms of representation are some of the teaching
strategies. According to Supriyatman and Sukarino, (2014), teachers can use
computer simulations to assist students in predictions to generate inquiry.
Furthermore, Brien and Peter (1994) and Jiang and McComas (2015) advocated the
need for instructional activities that integrate well into lessons for inquiry
learning. The approach allows students to gain a deeper and broader
understanding of science content with real-world applications, as well as learning
about the scientific inquiry process. This includes developing general
investigative skills (such as posing and pursuing open-ended questions,
synthesizing information, planning and conducting experiments, analyzing, and
presenting results). For example, during classroom lessons, students were
engaged in tasks such as making observations and inferences, planning
experiments, and generating predictions (Abungu et al., 2014., Chebii et al., 2012,
Rauf et al. 2013, Saputri, 2021). As a consequence of involving students in these
learning activities, they work collaboratively in groups, interact with each other
through discussion and carrying out experiments under the guidance of the
teacher. In addition, the instructional activities mentioned develop critical
thinking skills and learning curiosity among learners (Higgins & Moeed, 2017;
Pradana et al., 2020). Thus, in the Tanzanian context it was important to explore
instructional activities that support students’ learning of science process skills
while using computer simulations and animations to learn chemistry concepts.
2.3 Theoretical Framework

This study was framed within social constructivism theory by Vygotsky (1978)
who believed that knowledge construction is an active process conducted through
social interaction among learners themselves, learners and teachers or learners
and materials. This indicates that scientific knowledge and skills are socially
constructed and verified under social constructivism in science learning. As a
result, Onwioduokit (2013) suggested that when students are taught science, they
should participate in inquiry activities. This becomes possible when learners are
encouraged to learn by doing something as a means of learning instead of only
listening (Demirci, 2009). In essence, these instructional activities are essential to
enable teachers and learners to interact with computer simulations and
animations during teaching and learning.
Vygotsky (1978) explained the role of teachers in using instructional activities and
learner-centered strategies to enable students to construct knowledge and skills.
Therefore, using social constructivism theory, it was believed that it could help to
understand instructional activities that engage learners in knowledge
construction and learning science process skills as they learn using computer
simulations. These are essential learning environments to create a social learning
25
environment that facilitates students' construction of knowledge and skills that

can be applied from a classroom context to real life experiences.
3. Methodology
3.1 Participants, sampling and sample size
The study was carried out at four secondary schools from the Dodoma and
Singida regions of Tanzania's central part. The area was chosen because students
perform poorly in science, including chemistry, and there is a shortage of
instructional materials (MoEST, 2019, 2020). The selection of schools was based on
the availability of computer laboratories and other ICT equipment or tools such
as projectors. The assumption was that by using computer laboratories, students
could be subjected to the teaching and learning of chemistry using computer
simulations as one way to engage learners in hands-on activities.
The challenging topic of chemical kinetics, equilibrium, and energetics was the
focal point of the current study (Beichumila et al., 2022; Lati et al., 2012), which is
taught at level three of secondary education in Tanzania (MoEVT, 2010). This
served the choice of 160 Form Three students (level 3 of ordinary secondary
education), who were rondomly selected to be involved in this study.
Furthermore, 20 chemistry teachers were purposely involved in the study based
on the criteria that they had prior training in ICT integration in the classroom.
3.2 Research approach and design

The study employed a mixed method through both quantitative and qualitative
approaches to collect data. This was done through classroom observations
focusing on both teachers' and students' learning activities (Cresswell, 2013;
Cresswell & Clark, 2018). In addition, a lesson reflection sheet was used to explore
students’ insights on lesson instructional activities. The focus was to explore the
instructional activities that could support students’ learning of science process
skills by using chemistry-based computer simulations and animations. This
generated information that helped the research team to explore the instructional
strategies that could engage students in learning chemistry concepts using
computer simulations and animations. The use of both classroom observation and
a lesson reflection sheet was considered as triangulation of information (Cohen et
al., 2011). The design of the study followed two steps, namely pre-intervention
and post-intervention.
3.3 Data Collection Procedure

Step 1: Pre-intervention
The first four sessions, which were utilized as a pre-intervention, focused on the
topics of chemical kinetics, equilibrium, and energetics, with conducted one
lesson per school being conducted. The four lessons in pre-intervention were
purposely used to capture an actual picture of instructional activities used by
teachers to support students’ learning of science process skills through computer
simulations. This was a baseline setting. At this stage a classroom observation
checklist was used as a data collection tool. The classroom observation checklist
was developed by the researcher from existing literature, for example, Chebii et
al. (2012). Classroom observation was chosen as the method since it provides first-
26
hand evidence of what the teacher and students perform in class as compared to
a questionnaire (Atkinson & Bolt, 2010).
Step 2: Post-intervention
In post-intervention, seven consecutive series of lessons were conducted at school
level, making a total of 28 lessons in four secondary schools. Teachers and
researchers were involved in the process of lesson planning, classroom teaching,
and reflection. During lesson planning, teachers collaborated to prepare a lesson.
It was to ensure that the lesson was prepared based on inquiry learning, focusing
on achieving science process skills. Classroom teaching involved observations of
different instructional activities and how students were learning chemistry
concepts as well as science process skills. During lesson reflection, students were
given a lesson reflection sheet on which they identified their favorite learning
activities from the lesson. This was also time for the research team to reflect on the
lesson and plan for the next one. Therefore, in this study, students were required
to acquire knowledge as well as to formulate hypothesis, plan experiments,
identify variables, define operationally, make interpretations, and draw
conclusions. Table 1 indicates the nature of teaching strategies that accompanied
the lessons adapted from Jiang and McComas’s (2015) framework on inquiry
instructional strategies for learning science concepts and process skills in the
classroom context. This was to engage students in a more discursive context, as
supported by chemistry-based computer simulations and animations in each
lesson.
Table 1: Instructional strategies and science process indicators

Item Instructional strategies in classroom context Indicators of science process skills
1 Students were required to formulate a Formulating a hypothesis
hypothesis in relation to the question under
investigation
2 Students were required to think of scientific Identifying procedures and planning
procures, plan an investigation, and for investigation
conduct experiments for the purpose of
testing the hypothesis
3 Students were required to identify associated Identifying variables
variables of the investigation that could be
controlled variables, dependent or
independent variables
4 Students were required to make Making interpretations and
interpretations of the collected evidence or conclusions
data through tables, graphs, or words in
order to obtain meaningful information and
thereafter draw conclusions basing on
collected evidence
5 Students were required to develop Developing operational definitions
statements presenting a concrete description
of an event that indicates what to observe/do
as the evidence towards their observations
and conclusion in relation to the question
under investigation
27
Furthermore, computer simulations from Yenka chemistry

(https://www.yenka.com/en/Yenka_Chemistry), and one model of PhET
simulation of reactions and rates
(https://phet.colorado.edu/en/simulations/reactions-and rates) were used
during the teaching and learning process in this study. Figures 1 and 2 are samples
of these simulations in which students were engaged to learn chemical kinetics,
equilibrium and energetics.
Figure 1: Computer simulation of the effect of a catalyst on the rate of reaction
Figure 2: Computer simulation of the effect of concentration on the rate of reaction
3.4 Validity and reliability of data collection tools

In the case of ensuring validity, the classroom observation checklist and reflection
sheets were evaluated by three chemistry teachers. Later on, the tools were piloted
in two secondary schools that were not part of the selected schools in the study.
This helped to identify and remove irrelevant items. In addition, inter-observer
reliability which is a measure of consistency between two or more observers of
the same construct was calculated (Cohen, 1988). The value of the Kappa
coefficient (ka) across three observer pairs was found to be 0.80, 0.78, and 0.79
which are acceptable. The use of three observers (the researcher and two assistant
researchers) independently during classroom observation helped to improve the
internal reliability of the findings from classroom observation (Cresswell, 2013).
3.5 Data analysis

For the quantitative data, percentages (Pallant, 2020) were used to show the
number of students and teachers in relation to instructional activities and science
28
process skills indicators in the teaching and learning process. The qualitative data
generated from classroom observations were thematically analyzed according to
Braun and Clarke (2012). Information from the classroom observation and
reflection sheet were transcribed and coded after a thorough discussion among
the research team. This included notes and comments from observers on specific
instructional activities that engaged students to learn science process skills
through computer simulations. Finally, the agreed themes were used to conclude
specific instructional activities supporting the learning of chemistry concepts with
computer simulations and animations.
4. Results and Discussion

The general findings from this study indicate that instructional activities,
particularly formulating hypothesis, planning experiments, identifying variables,
developing operational definitions, making interpretations, and drawing
conclusions, support students in learning integrated science process skills using
chemistry-based computer simulations. It was found that during the teaching and
learning process, generally more than 70% of students were able to perform the
aforementioned activities well while 60% performed well in planning
experiments. On the other hand, as compared to other instructional activities,
planning experiments was the least observed among students and teachers.
Tables 2-6 indicate the findings under each instructional activity.
Formulating hypothesis
The findings from this study indicated that the hypothesis formulation as an
instructional activity involved students in predictions skill as 75% of students in
post-interventions were able to formulate hypotheses. It was observed that,
initially, 70% of students had no idea on how to hypothesize; however, their
ability improved as they were involved in this learning activity. The activity
helped students to make their predictions that could be scientifically tested. It was
found in this study that using chemistry-based computer simulations to learn and
understand chemical kinetics, equilibrium, and energetics made students more
engaged in the teaching and learning process. Students were more involved in the
lesson when they were asked to formulate a hypothesis in relation to the
experiment’s aim, rather than doing experiments by following predetermined
sequence of procedures, as is the case in most science classrooms (Table 2).
Table 2: Formulating hypothesis

Indicators of science
process skills in the
Teaching activities Learning activities classroom context
90% of teachers guided Students in small groups Before:

students in small groups of 3-5 students were
of 3-5 students through the required to think and The majority of students
process of writing down write down the question (70%) were not able to
the aim of the experiment to be investigated and the formulate a hypothesis
to be explored. aim of the experiment. correctly. For example, one
of students in school C
Then teachers guided Students discussed in wrote:
students to observe the groups what the
29
computer simulation hypothesis could be in “Surface area and rate of

models, for example, the relation to the aim of the reaction are related”.
simulation that exhibited experiment they
the effect of temperature determined by observing After:
and rate of reaction. They the computer simulations. 75% of students could
began writing down their formulate a hypothesis.
hypothesis in relation to
the question being Captured sentences from
investigated. For example, students formulating a
investigating how the hypothesis:
temperature affect the rate
“The higher the temperature,
of reaction.
the higher the rate of a
The teacher used probing The majority of students chemical reaction”
questions to help students (75%) were able to think
In another group:
use their prior knowledge and discuss in their small
to understand how they groups how collision “The higher the temperature,
could formulate the theory relates with the fast the chemical reaction”
hypothesis before further temperature and rate of
activity, for example: any chemical reaction. “Temperature affects the rate
of a chemical reaction”
“From the collision theory
what do you think will Another group in another
happen if the temperature is lesson:
lower or high in the reaction
“The presence of catalyst will
of calcium carbonate and
speed up the decomposition of
hydrochloric acid?”
hydrogen peroxide”
Observations from
students: “Increasing the
rate of a reaction means
increasing the number of
fruitful collisions between
particles, therefore increasing
the temperature will increase
the rate of reaction”.
The findings from this study support Seok (2010), who found that engaging
students in formulating a hypothesis on the question to be investigated in the
science classroom helps develop this science process skill. Moreover, the findings
indicated that through this instructional activity students developed a sense of
collaboration and ownership of the lesson. This was revealed through learning
from each other and arguing to reach a conclusion on the kind of hypothesis being
formulated. This helped students to construct knowledge while at the same time
developing a hypothesis-formulation skill. Darus and Saat (2014) found that
teaching strategies that could be used by teachers to help students in hypothesis
formulation to generate inquiry include activating students’ background
knowledge, providing analogies, questioning, and encouraging students to use
alternative forms of representation. Thus, hypothesizing as learning with
computer simulations in science classroom is one way to promote active learning
and reasoning among students (Moore et al., 2014; Sreelekha, 2018).
30
Furthermore, collaboration is discussed under social-constructivism theory as one

of the essential elements in the learning process as it changes the dynamics of the
classroom by encouraging discussion among the learners. Vygotsky (1978) further
explained that collaboration impacts students’ learning. As a result, one of active
learning strategies that promote students' curiosity in learning chemistry is their
ability to make predictions. As has been suggested in the literature, students'
interest in the subject matter contributes significantly to their ability to learn the
subject when they are exposed to a social learning environment through active
learning activities (Anderhag et al., 2015; Higgins & Moeed, 2017).
Planning experiment
The findings revealed that students (60%) learned to plan experiments through
interaction with their peers during the investigation process since students could
brainstorm with each other and work cooperatively in their small group to ensure
that they come up with a good procedure to test their hypothesis. For example,
when investigating how a catalyst affects the rate of reaction, a student told his
group members that they needed to use the same amount of hydrogen peroxide
in both test tubes, but one test tube needed to be added with a catalyst while the
other did not, so that they could observe the difference. This is because some
students understand the procedures more easily than others. Therefore, it was
observed that this process helped students to share their ideas in the lesson which
was also another way of being aware of the procedures and important related
aspects such as materials, variables to consider and how to conduct their
experiment (Table 3).
Table 3: Planning experiment

Teaching activities Learning activities Indicators of science
classroom context
60% of teachers guided 60% of students in Evidence from students’
students in groups of 3-4 to groups of 3-4 students work in one of the
devise procedures to were able to discuss and groups:
investigate the scientific critically think of the best “We have to put the same
question being explored to plan they could use for amount of hydrogen
test their the procedure to test peroxide in two test tubes,
hypotheses/predictions. For their hypothesis with the then in one of the test tubes
instance, in a scientific computer simulation. put certain amount of
question where students were catalyst manganese (IV)
to investigate how the catalyst oxide, then we will start the
affects the rate of a chemical reaction and observe the
reaction, time taken for the reaction
teachers guided students to between the two test tubes
use their plans and computer to complete.”
simulations to conduct simple
experiments, make In another group
observations, record data and “We will put 25mls of
write simple reports. hydrogen peroxide (H2O2)
The majority of teachers were Students were able to in two test tubes, then we
insisting students use specific discuss and decide the will add 2g of manganese
(IV) oxide (catalyst) and
measurements to obtain amount of solution or
justifiable scientific solute to be used in their
31
conclusions, for example one experiment to come up observe the reaction in both
of the teachers: “Do you think with scientific test tubes.”
if you use a different amount of conclusion.
hydrogen peroxide in the two test Observations from
tubes and a different amount of students’ group
catalyst you will come up with a discussion “… no, we
good scientific conclusion?” need to take the same
amount of hydrogen
60% of teachers used probing Students were listening peroxide in both test tubes
questions to help students to teacher’s questions and measure specific
understand how to plan and trying to think of amount of catalyst to be
scientific and give examples of added in one of the test
investigation/experiments by short plans for scientific tubes”.
relating various concepts of investigation or Another student: “Yes,
kinetics in daily life activities experiments from daily this is good, let us use 2g of
in their homes. life experiences in manganese (IV) oxide as a
society. catalyst.”
Observations from
students:
“We can scientifically
investigate a good soap to
remove stains on clothes if
we use same amount and
types of water, the same
clothes but we vary the
soaps.”
It was found that planning and performing experiments as an instructional
activity enabled students to use concrete activities through computer simulations
to test their hypotheses and come up with evidence. Students could learn other
skills such as measuring substances, knowing when to mix chemicals and start the
reaction, making observations, keeping records on what they observed, either in
tables or in words and making relevant decisions. Irwanto et al. (2018) and Seetee
et al. (2016) suggested that students’ experimenting skill is developed when a
science teacher guides them to write out detailed steps to their procedure and
determine the variables, including what needs to be controlled, and thinking of
the data to be collected. The capacity to design an experiment is essential for
comprehending the scientific process and developing critical thinking abilities
(Pradana et al., 2020).
In addition, experimentation, a process which engages students directly with the

physical world has been found to be effective in developing various students’
science process skills (Chebii et al., 2012). Moreover, the study mentioned did not
explain the students’ abilities to plan experiments based on their own experience
and understanding rather than following predetermined procedures. The use of
these instructional procedures during practical activities is teacher-centered and
does not match directly with social-constructivist theory as used in the context of
the present study. As a result, the current study has revealed that experimentation
instructional activity through computer simulations is one way to enable students
to think critically and devise procedures to test their hypotheses. As students
engage in these learning activities, they learn to reason and think critically.
32
According to Coil et al. (2010) and Pradana et al. (2020), this encompasses
designing experimental skills that involve students in critical thinking and
reasoning abilities.
Identifying and controlling variables

It was observed that initially, the majority of students (80%) were not able to
identify variables. The findings in post-interventions from this study indicated
that using the instructional activity of identifying variables involved students in
learning to understand concepts of variables in scientific investigations.
Therefore, it was observed that 70% of students could identify variables that can
affect an experimental outcome, keeping variables constant while manipulating
only the independent variables. Students could explain the independent variables
as they manipulated computer simulations. For example, when learning the effect
of concentration on reaction rate, one of the students explained that the
concentration of acid was an independent variable since it was the one that was
manipulated, whereas the rate of a chemical reaction was a dependent variable
because it was the one that was measured. Table 4 provides more specific
examples.
Table 4: Identifying and controlling variables

classroom context
Teachers were guiding Students, in their small Before intervention:
students to be aware of groups of 3-4, were able to
variables associated with discuss and identify the 80% of students had
the investigation they were variables as they were wrong answers. For
conducting. For example, observing the computer example, one student in
teachers were guiding simulations. school A said:
students to identify the “…the controlled variable in
variables through probing our experiment is time
questions. because in every test tube
time taken for the reaction to
For instance, where
complete was different”
students were investing
the effect of concentration After intervention:
on the rate of chemical
reaction with computer One of the student’s words
simulation, the teacher from school A:
asked what the controlled
“…in this experiment
variables or factors in the
amount of calcium carbonate
experiment were as well as and temperature are the
what the dependent and controlled variables because
independent variables in all three test tubes there is
were. 0.6g of calcium carbonate and
temperature is 250c in all test
tubes but there are different
concentrations of
hydrochloric acid which is
Again, some teachers The majority of students 1M, 1.5M and 2M.
through probing questions (70%) were able to Concentration of
techniques guided brainstorm in their groups hydrochloric acid in this
33
students to understand the why some factors were experiment is independent

logic behind these kept constant while some variable because it is the one
variables and their role in were varied in their being varied. The rate of
scientific investigations. experiments. reaction is dependent variable
as it depends on the
For instance, in an concentration of the acid in
investigation of how the the rate of reaction”
temperature affect the rate
of reaction, one of the
teachers asked “Why do you
Observations by from
think i) the amount of calcium
students:
carbonate was maintained or
kept constant in both test “...because we wanted to
tubes?” know how the temperature
affect the rate of a chemical
ii) “…the temperature in both
reaction, so we had to vary
the test tubes was varied?”
the temperature only in both
reactions while keeping other
factors constant like amount
of reacting substance in order
to come up with scientific
evidence that real the
temperature affect the rate of
reaction”.
Another observation was

that students were also
active and eager to ask
questions, for example,
one student asked “What
will happen if we don’t
control other variables?”
Therefore, the study findings support the teaching and learning methodologies of
Athuman (2017) and Irwanto et al. (2018) for the skill of identifying and
manipulating variables during the learning process. They include asking learners
how they would decide on the set-up of the inquiry that would result in the most
complete answer to the problem, as well as leading learners to the conclusion that
they will only need to compare one component at a time. This implies that this is
the kind of learning that situates students in knowledge construction rather than
cramming the concepts. Moreover, Beichumila et al. (2022) and Saat (2004) found
that the use of computer simulations-based environment improved students’
ability to identify variables. In this case, identifying variables as a teaching and
learning activity in chemistry is critical because it exposes students to the reality
of these variables and their implications in scientific investigation as
recommended in the chemistry competence curriculum.
Even though Ardac and Sezen (2002) and Beichumila et al. (2022) acknowledge
the importance of identification and manipulation of variables in learning science,
there is much more to be added. This includes the questions around how students
are provided with opportunities to explain their understanding of variables in
relation to daily life experiences. The focus of the curriculum is on students'
34
understanding of the idea of variables in many situations in everyday life

activities so that they may apply what they have learned in chemistry to other
contexts (MoEVT, 2010). In this sense, it was critical for the chemistry students in
this study to grasp the concepts of variables as they learned chemical kinetics,
equilibrium, and energetics, all of which are closely tied to numerous daily
activities. Moreover, as students are involved in asking questions and assessing
their responses, they use their rational and logical thinking (Harrison, 2014).
Making interpretations and drawing conclusions

From the study findings, students (75%) were able to engage in learning activities
such as providing meaning to the obtained data in order to comprehend the
patterns and relationships that lead to the formation of conclusions. Students
collaborated to organize their information either in tables, words or graphs as they
interacted with computer simulations and animations. It was observed that
students had an excellent opportunity to discuss in small groups and reach
agreement with one another during this teaching and learning activity, and then
present their agreed-upon results to the rest of the class. As a result, students
learned to develop their own scientific explanations, as opposed to being passive
in a teacher-centered classroom where teachers do all the work of explaining and
writing notes on the chalkboard. Table 5 provides more insights and examples.
Table 5: Making interpretations and conclusion

Teaching activities Learning activities Indicators of science process skills in the
classroom context
90% of teachers 75% of students in Evidence from students works
guided students to groups of 3-4
make interpretations students were able to
of the collected data make interpretations
with computer on the observations
simulations. made by making
Teachers supported simple tables to
students to do it show the pattern of
better by probing the data, and
questions in the reading the graphs
data, from the computer
for example, “Why simulation Students’ observation when explaining the
do the balloons experiments. table above: “When the temperature of a reaction
connected to the test Moreover, increases, the rate of reaction increases.
tube with a high explanations given Temperature and rate of reactional are directly
concentration of acid from the proportional.”
inflate faster and burst observations when
faster than the rest?” performing simple
experiments.
Teachers were Students in their Observations from students:
guiding students to groups were able to “The rate of the reaction increases as the
make conclusions on make conclusions temperature increases, hence the temperature
their own from their based on the affects the rate of a chemical reaction”.
observations in collected evidence or
relation to the observations made. Another student observation:
hypothesis which
35
was also presented “The higher the temperature, the fast the reaction;
to the whole class the lower the temperature, the slow the reaction.
Hence temperature affects the rate of reaction”.
Therefore, the findings have revealed the role of interpretation and conclusion as
an instructional activity for students learning to interpret the collected
information and constructing the meaning from them. The findings support the
views of Coil et al. (2010) and Rauf et al. (2013), namely that the central part of the
teacher’s role in developing interpreting and conclusion skills is to ensure that
results are used, and that students do not rush from one activity to another
without discussing and thinking through what the results mean, for example,
communicating the units such as centigrade, seconds per time, molarity, grams as
they report what they did, observed and found. It was also an opportunity for
students learning to read tables and graphs for them to interpret these more easily
depending on the patterns.
Furthermore, it was found that discussing their results involved students in

meaningful learning as they interacted with real activities, the teacher, and the
learners themselves. It was therefore observed that students discussing and
making conclusions on their own is very important in the teaching and learning
process. It enables students to communicate their results but also to improve their
communicative skills in interpreting what they have observed through words,
graphs, or tables and drawing their own conclusions depending on the patterns
or relationship of their results. In addition, Rauf et al. (2013) and Saputri (2021)
explained that when students are involved in a discursive process, their science
process skills become more developed. However, the studies are based on
biological and physics phenomena rather than the current, which is based on
chemistry, particularly chemical kinetics, equilibrium, and energetics. The
common link between the previous mentioned studies and the present study is
that both agree on the role of a teaching strategy that involves students in making
relevant interpretations through words, graphs, or tables to make meaning. This
supports social-constructivism theory as employed in this study that involving
students in discursive processes promotes student interaction.
Making operational definitions

From the findings (Table 6), it was found that defining operationally is one of the
instructional activities that involves students (70%) in learning. This includes
what to observe or measure when conducting a scientific investigation. During
scientific investigations, students are encouraged to use logic and critical thinking,
and to participate in knowledge construction. Table 6 gives more evidence.
Table 6: Making operational definitions

classroom context
80% of teachers were able 70% of students were able The observation of one of
to guide students to make to make operational the students answering the
operational definitions definitions of their question:
which engaged students in observations to justify their
critical thinking on the “The time taken for the green
balloon to inflate and burst,
36
observations made as they conclusions to answer the which was placed at the
performed experiments. questions. reaction with a high
temperature, was very short
For example, students compared to the red balloon
were recording the time for placed at the reaction with a
Teachers used probing each reaction to be low temperature.”
questions. completed, observing the
For example: “How do you color changes, the
know that the rate of reaction differences in chemical
Another student indicated:
was fast or slow?" reactions, the formation
and disappearance of “Because the green balloon
bubbles as they mixed took less time to fill and burst
chemicals through (8 seconds) than the red
computer simulations. balloon, the green balloon's
reaction was fast, but the red
balloon's reaction was
delayed”.
Athuman (2017) and Ngozi (2021) claimed that by providing a variety of materials
and resources to aid students' investigations, posing thoughtful questions,
encouraging dialogue among students and with the teacher, and maintaining
students' natural curiosity throughout the process students’ development of
science process skills can be promoted. In addition, Athuman (2019) explained
that involving students directly to develop understanding could deflect them
from cramming information. Instead, they are involved in the process of
understanding what is happening.
Furthermore, similar findings from students’ lesson reflection sheets (Figures 3

and 4) indicated that learning using a chemistry-based computer strategy exposes
students to instructional activities. This implies that the instructional activities are
important in helping students to learn the associated science process skills.
Figure 3: Students’ lesson reflection in school C
37
Figure 4: Students’ lesson reflection in school A
Because chemistry is so closely connected to our daily lives, learning these science
process skills makes it a meaningful subject. Students get the opportunity to
discuss and evaluate their solutions with classmates and teachers while acquiring
these science process skills. Students utilize their logical and reasonable thinking
while encouraging higher order thinking skills in the act of generating
assessments and discourse (Harrison, 2014; Pradana et al., 2020). Although, the
current study has demonstrated that students can be engaged in the process of the
learning science process to some extent, more work needs to be done on the
exploration and development of chemistry computer simulations and animations.

The findings of this study revealed that instructional activities such as formulating
hypotheses, planning experiments, identifying variables, compiling operational
definitions, making interpretations, and drawing conclusions can support
students in learning integrated science process skills. However, the study was
limited to the exploration of the chemistry-based computer simulations and
animations instructional activities supporting science process skills learning. To
extend the scope beyond this sudy, further studies may be conducted on students’
perceptions towards the use of computer simulations and animations in chemistry
teaching and learning at secondary school level.
6. Conclusion
The study revealed that during the teaching and learning process, instructional
activities as used in this study can support students in learning integrated science
process skills using chemistry-based computer simulations and animations. This
implies that instructional activities that focus not only on scientific content but
also on transferable skills such as hypothesis formulation, designing experiments,
identifying variables, interpreting results, and drawing conclusions are needed to
prepare students more effectively to apply chemistry concepts in daily life. The
same activities are essential in students’ daily lives as they need to observe or
predict different phenomena they come across. Furthermore, the use of
38
chemistry-based computer simulations is one way to engage students in active

teaching and learning processes, which creates a favorable learning environment
for students to construct knowledge in the classroom at the same time that they
acquire science process skills.
7. Recommendations
The findings of this study suggest that chemistry-based computer simulation
instructional strategies that focus on engaging students in both scientific content
and process skills such as formulating hypotheses, designing experiments,
identifying variables, interpreting information, and drawing conclusions are
essential in the learning process. Therefore, teachers need to consider the use of
these instructional activities through chemistry-based computer simulations to
facilitate students' learning of chemistry concepts in secondary schools.
Acknowledgement
The authors would like to thank the management of the African Centre of
Excellence for Innovative Teaching and Learning Mathematics and Science
(ACEITLMS) at the University of Rwanda, College of Education, which provided
the funding for the entire study. We also acknowledge the participation of all the
teachers, students, and the researchers’ team.
8. References
Abungu, H. E., Okere, M. I., & Wachanga, S. W. (2014). The effect of science process skills
teaching approach on secondary school students’ achievement in chemistry in
Nyando district, Kenya. Journal of Educational and Social Research, 4(6), 359-372.
Ardac, D., & Sezen, A.H. (2002). Effectiveness of computer-based chemistry instruction in
enhancing the learning of content and variable control under guided versus
unguided conditions. Journal of Science Education and Technology, 11(1), 39-48.
https://doi.org/10.1023/A:1013995314094
Akdeniz, C. (2016). Instructional process and concepts in theory and practice. Springer
Science and Business Media.
Anderhag, P., Hamza, K. M., & Wickman, P.O. (2015). What can a teacher do to support
students’ interests in science? A study of the constitution of taste in a science
classroom. Research in Science Education, 45(5), 749-784.
https://doi.org/10.1007/s11165-014-9448-4
Athuman, J. J. (2019). Assessing knowledge levels of selected integrated science process
skills of Morogoro Biology secondary students – Tanzania. International Journal of
Science and Research, 8(9), 254-261.
http://www.suaire.sua.ac.tz/handle/123456789/2973
Athuman, J. J. (2017). Comparing the effectiveness of an inquiry-based approach to that of
conventional style of teaching in the development of students’ science process skills.
International Journal of Environmental and Science Education, 8(9), 1797-1816.
Atkinson, J., & Bolt, S. (2010). Using teaching observations to refelect upon and improve
teaching practice in higher education. Journal of the Scholarship of Teaching and
Learning, 10(3), 1-9.
Aydm, A. (2013). Representation of science process skills in the chemistry curriculum for
grades 10, 11 and 12 in Turkey. International Journal of Education and Practice, 1(5), 51-
63. https://doi.org/10.18488/journal.61/2013.1.5/61.5.51.63
Beichumila, F., Bahati, B., & Kafanabo, E. (2022). Students’ acquisition of science process
skills in chemistry through computer simulations and animations in secondary
39
schools in Tanzania. International Journal of Learning, Teaching and Educational Research,

21(3), 166-195. https://doi.org/10.26803/ijlter.21.3.10
Bete, A. O. (2020). Students’ knowledge and process skills in learning grade-8 chemistry.
Journal of Research, Policy & Practice of Teachers and Teacher Education, 10(1), 1-13.
https://doi.org/10.37134/jrpptte.vol10.1.1.2020
Braun, V., & Clarke, V. (2012). Successful qualitative research: A practical guide for beginners.
Sage Publishing.
Brien, O., & Peter, G. (1994). The effects of four instructional strategies on integrated
science process skills and skill achievement of pre-service elementary teachers
having different cognitive development levels. Journal of Elementary Science
Evaluation, 28(7), 593–607. https://doi.org/10.1007/BF03170648
Çelik, B. (2022). The effects of computer simulations on students’ science process skills:
Literature review. Canadian Journal of Educational and Social Studies, 2(1), 16–28.
https://doi.org/10.53103/cjess.v2i1.17
Cohen, L., Manion, L., & Morrison, K. (2011). Research methods in education (7th ed).
Routledge
Coil, D., Wenderoth M. P., Cunningham M., & Dirks C. (2010). Teaching the process of
science: Faculty perceptions and an effective methodology. Journal of CBE Life
Sciences Education 9(1), 524–535. https://doi.org/10.1187/cbe.10-01-0005
Chebii, R., Wachanga, S., & Kiboss, J. (2012). Effect of science process kills mastery learning
approach on tudents’ acquisition of selected chemistry practical skills in schools.
Creative Education, 3(8), 167-176. http://dx.doi.org/10.4236/ce.2012.38188
Cresswell, J. W., & Clark, V. L. (2018). Designing and conducting mixed methods research.
Sage Publishing.
Cresswell, J. W. (2013). Research design: Qualitative, quantitative and mixed method approaches.
Sage Publishing.
Darus, F., & Saat, R. M. (2014). How do primary school students acquire the skill of making
hypothesis. The Malaysian Online Journal of Educational Science 2(2), 20-26.
Demirci, C. (2009). Constructivist learning approach in science teaching. HUJ Education,
37(1), 24-35.
Dixon, J. K., Adams, T. L., & Hynes, M. (2001). Controlling variables. Journal of Teaching
Children Mathematics, 8(3), 160-164.
Harrison, C. (2014). Assessment of inquiry skills in the SAILS project. Science Education
International, 25(1), 112–122.
Higgins, J., & Moeed, A. (2017). Fostering curiosity in science classrooms: Inquiring into
practice using co-generative dialoguing. Science Education International, 28(3), 190–
198.
Irwanto, I., Rohaeti, E., & Prodjosantoso, A. K. (2018). Undergraduate students’ science
process skills in terms of some variables: a perspective from Indonesia. Journal of
Baltic Science Education, 17(5), 751-764.
Jiang, F., & McComas, W. F. (2015). The effects of inquiry teaching on student science
achievement and attitudes: Evidence from propensity score analysis of PISA data.
International Journal of Science Education, 37(3), 554-576.
Kalolo, J. F. (2015). Is the provision of quality science education in crisis? Evidence from
the Tanzanian junior secondary schools. International Journal of Current Innovation
Research, 1(9), 213-224.
Kinyota, M. (2020). The status of and challenges facing secondary science teaching in
Tanzania: A focus on inquiry-based science teaching and the nature of science.
https://doi.org/10.1080/09500693.2020.1813348
40
Lati, W., Supasorn, S., & Promarak, V. (2012). Enhancement of learning achievement and
integrated science process skills using science inquiry learning activities of chemical
reaction rates. Procedia Social and Behavioral Sciences, 46, 4471-4475.
https://doi.org/10.1016/j.sbspro.2012.06.279
Ministry of Education and Vocational Training (MoEVT). (2010). Chemistry syllabus for
secondary schools. Dar es Salaam: Tanzania Institute of Education.
Ministry of Education, Science and Technology (MoEST). (2015). Education and training
policy. Dar es Salaam: Ministry of Education, Science, and Technology.
Ministry of Education, Science and Technology (MoEST). (2019). National Basic Education
Statistics: National Data. Dar es salaam: MoEST.
Ministry of Education, Science and Technology (MoEST). (2019). National curriculum
Framework for Basic and Teacher Education. Dar es Salaam: Tanzania Institute of
Education
Statistics: National Data. Dar es salaam: MoEST.
Mkimbili, S. T., Tiplic, D., & Ødegaard, M. (2017). The role played by contextual challenges
in practising inquiry-based science teaching in Tanzania secondary schools. African
Journal of Research in Mathematics, Science and Technology Education, 21(2), 211–221.
https://doi.org/10.1080/18117295.2017.1333752
Moore, E. B., Chamberlain, J. M., Parson, R., & Kerkins, K. (2014). PhET interactive
simulations: Transformative tools for teaching chemistry. Journal of Chemical
Education, 91(8), 1190-1197 https://doi.org/10.1021/ed4005084
Ngeze, L. V. (2017). ICT integration in teaching and learning in secondary schools in
Tanzania: Readiness and way forward. International Journal of Information and
Education Technology, 7(6), 424–427.
Ngozi, P.O. (2021). Enhancing Science Process Skills Acquisition in Chemistry among
Secondary School Students through Context-Based Learning. Science Education
International 32(4), 323-330. https://doi.org/10.33828/sei.v32.i4.7
Nkemakolam, O. E., Chinelo, O. F., & Jane, M. C. (2018). Effect of computer simulations
on secondary school students’ academic achievement in chemistry in
Anambra State. Asian Journal of Education and Training, 4(4), 284–289.
https://doi.org/10.20448/journal.522.2018.44.284.289
Onwioduokit, F. A. (2013). The ordeal of science teaching in the contemporary society: A
need for paradigm shift for the new generation. A professorial Inaugural Lecture
(series 36th) Delivered 31st, 2013. University of Uyo, Nigeria.
Pallant, J. (2020). SPSS survival manual: A step-by-step Guide to data analysis using IBM
SPSS (7th ed.). Routledge. https://doi.org/10.4324/9781003117452
Plass, J. L., Milne, C., Homer, B. D., Schwartz, R. N., Hayward, E. O., Jordan, T., Verkuilen,
J., Ng, F., Wang, Y., & Barrientos, J. (2012). Investigating the effectiveness of
computer simulations for chemistry learning. Journal of Research in Science Teaching,
49(3), 394−419. https://doi.org/10.1002/tea.21008
Pradana, D., Nur, M., & Suprapto, N. (2020). Improving critical thinking skills of junior
high school students through science process skills-based learning. Jurnal Penelitian
Pendidikan IPA, 6(2), 166–172. https://doi.org/10.29303/jppipa.v6i2.428
Rauf, R.A.A,, Rasul, M.S., Mansor, A.N., Othman, Z., & Lyndon. N. (2013). Inculcation of
Science Process Skills in a Science Classroom. Asian Social Science, 9(8), 47-57.
http://dx.doi.org/10.5539/ass.v9n8p47
Saat, R. M. (2004). The acquisition of integrated science process skills in a web-based
learning environment. Research in Science & Technological Education, 2(21), 23-40.
Seetee, N., Coll, R. K., Boonprakob, M., & Dahsah, C. (2016). Exploring integrated science
process skills in chemistry of high school students. Veridian E-Journal, 9 (4), 247–259.
41
Seok, Oh. (2010). How can teachers help students formulate scientific hypotheses? Some
strategies found in abductive inquiry activities of earth science. International Journal
of Science Education, 32(4), 541-560. https://doi.org/10.1080/09500690903104457
Saputri, A. A. (2021). Student science process skills through the application of computer
based scaffolding assisted by PhET simulation. At-Taqaddum, 13(1), 21-38.
https://doi.org/10.21580/at.v13i1.8151
Semali, L. M., & Mehta, K. (2012). Science education in Tanzania : Challenges and policy
responses. International Journal of Educational Research, 53(1), 225–239.
https://doi.org/10.1016/j.ijer.2012.03.012
Sreelekha, J. (2018). Effects of computer simulation and video-based instructional
strategies on students’ acquisition of skills in practical physics. International Journal
of Educational Research and Technology, 9(1), 1-8.
Supriyatman, A., & Sukarno, S. (2014). Improving science process skills (SPS) science
concepts mastery (SCM) prospective student teachers through inquiry learning
instruction model by using interactive computer simulation. International Journal of
Science and Research, 3(2), 6–9.
Trindade, J., Fiolhais, C. & Almedia, L. (2002). Science learning in virtual environments: A
descriptive study. British Journal of Educational Technology, 33(1), 1- 18.
Vygotsky, L. S. (1978). Mind in society: The development of higher mental process. Harvard
University Press.
Yadav, B., & Mishra, S. K. (2013). A study of the impact of laboratory approach on
achievement and process skills in science among IS standard students. International
Journal of Scientific Publications, 3(1), 2250-3153.
9. Appendix A
Classroom observation checklist
Directions: In each item below, place a tick (√) depending on what you will
observe by using the given scales. Also indicate any supporting evidence or
comments depending on your observation.
Item no
activities
learning
teaching and
students’
and
Teachers’
(1)
Never
(2)
Rarely
(3)
not clear
Sometime but
but clear (4)
Sometimes
and clearly (5)

always
Almost
as observed
any evidence
Comment/
1 Teacher guides students to

make predictions or
hypothesis of an investigation
2 Teacher forms groups for
students to work
collaboratively during
teaching and learning process
at different stages of the lesson
e.g., to formulate hypotheses,
identify procedures, conduct
investigations, and interpret
and communicate results
3 Teacher guides students’
participation in identifying
42
and writing procedures to

conduct a certain investigation
4 Teacher guide students
through questions to identify
variables associated with an
investigation
plan and perform experiments
to collect relevant data
observe changes taking place
in reactions, for example, color
changes, formation and
disappearance of bubbles,
changes in the volume of gases
being emitted when
performing experiments,
record observations and ideas
when doing activities
develop operational
definitions during
investigative activities
9 The teacher provides an
opportunity for students to
discuss with each other in
groups to describe the results
of an investigation
communicate the results of an
investigation using words or
graphs through whole class
presentation, group
presentation or report writing
11 Students discuss and compare
the results of the investigations
to predictions/hypothesis
made prior to investigations
12 Others (specify below)………
10. Appendix B
Students’ lesson reflection sheet
1) In the table below, indicate learning activities in which you have been involved
during the teaching and learning process. For each activity, write
explanations of how it was done.
Favored learning activities Explanations
43

Vol. 21, No. 8, pp. 43-57, August 2022
Received Apr 22, 2022; Revised July 27, 2022; Accepted Aug 23, 2022
Issues Surrounding Teachers’ Readiness in

Implementing the Competency-Based ‘O’ Level
Geography Syllabus 4022 in Zimbabwe
Paul Chanda
Midlands State University: Gweru, Zimbabwe
Tafirenyika Mafugu
University of the Free State, South Africa
Abstract. The qualitative study, which involved a multiple case study

design, focused on the issues surrounding the readiness of teachers to
implement the competency-based O-Level Geography Syllabus 4022 in
the Zimbabwe secondary school system. In adopting a multiple case
study design, the research sought to solicit the opinions of all
Geography teachers in the Kwekwe district of Zimbabwe on their
readiness to implement the competency-based O-Level Geography
Syllabus 4022. A technique involving an analysis of primary documents
published by MoPSE was done, and ten in-depth interviews with
Geography teachers drawn from two secondary schools and two Focus
Group Discussions (FDGs) from the same schools were adopted to
generate data. The study established that the updated O-Level
Geography Syllabus 4022 was introduced without enough consideration
of the readiness of teachers for its implementation. It also emerged from
the study that the breadth and depth of the issues surrounding teacher
readiness to implement an updated syllabus require action to be taken
from several fronts to ensure that the subject community is ready for its
rapid delivery. Finally, the research paper recommends massive
advocacy and sensitisation of O-Level Geography teachers who are at
the helm of the user system to allow for effective delivery of the O-Level
Geography Syllabus 4022 in the secondary school sector in Zimbabwe.
Keywords: readiness; educator; competency-based; syllabus; geography
1. Introduction
The introduction of the competency-based curriculum in Zimbabwe posed a
repertoire of challenges as classroom practitioners grappled with coming to
terms with the underlying assumptions, goals, content, and principles that guide
the operation of the new practice in both the primary and secondary schools
system. Thus, most of the world’s knowledge societies considering adopting
reform initiatives in their education systems have primarily focused on holistic
and integrated professional and pedagogical training concepts as a step toward
©Authors
44
promoting basic theoretical levels of knowledge, skills, and personal qualities in

teachers (Mizambaeva & Baimyrzaev, 2019). However, contemporary research
evidence has revealed that introducing reforms in some less Economically
Developed Countries (LEDCs) is not always perfect and, as such, often results in
adverse effects on the envisaged curriculum package. The problem has been
attributed to policies and mandates introducing innovations with little regard
for teacher readiness before the implementation phase. For instance, the state of
teacher readiness for implementing the updated O-Level Geography Syllabus
4022 in Zimbabwe seems to project a negative picture following its adoption,
which cynics claim was impromptu. Georgescu (2015) reports that at the
inception of the competency-based curriculum, the Government of Zimbabwe
(GoZ) committed itself to taking the curriculum seriously and promised that it
would give teachers total support to make curriculum implementation more
effective. As a priority, the Ministry of Primary and Secondary Education
(MoPSE) arranged for teacher capacity development programmes meant to
provide direction for classroom instruction and effective implementation of the
new practice (Mipfide & Mapolisa, 2021 and Simba, 2021). An operational plan
was established based on the terms of reference that included:
❖ provision of new syllabuses to all teachers for reviewing so that adjustments
are made, if necessary,
❖ provision of professional development for teachers, pre-service training to
help teachers learn to develop lessons and material resources, follow-up
support through clusters, district, provincial, and national staff,
❖ preparation, printing and delivering new learning materials including
textbooks and giving first preference to the poorest schools
❖ introduction of continuous assessment system (CA) and
❖ Monitoring and feedback.
The operational plan was sound and laudable, although its implementation
failed to live up to policy expectations due to pushback and lack of buy-in by the
people who were to deliver or benefit from the scheme. Meanwhile, a gap in
practice emerged because teachers were not well grounded in the philosophy
that informs the curriculum syllabus in question. This scenario is believed to
have demotivated teachers to engage the updated curriculum with learners in
their classrooms (Moyo and Hadebe, 2018). For Dube & Jita (2018) the
competency-based curriculum in Zimbabwe faced serious challenges of being
resisted by teachers, parents, and even learners themselves. Along the same
thread, the Progressive Teachers' Union of Zimbabwe (PTUZ) alleged that
teachers and parents were not consulted, so schools were inadequately prepared
to implement the curriculum framework (Newsday Zimbabwe, 9 January 2017).
PTUZ expressed concern about the lack of coherent programmes to train
teachers to operationalise the competency-based curriculum, particularly
continuous assessment (CA), despite the government’s insistence that the new
assessment should be adopted in schools. This decision of the government
sparked a lot of noise from different stakeholders, leading to the suspension of
CA without formal communication to schools. Thus, the then minister of
education publicly announced:
45
We have totally scrapped the issue of tasks, so we are saying we will no

longer have tasks in our schools. From my study, these tasks are not part
of the syllabus, and even if one would look at our syllabus, it has no
tasks. Above all, the tasks were too much a burden to both teachers and
learners because if a learner is doing ten subjects, they are required to do
at least one task per subject, meaning ten tasks for the ten subjects per
term or simply 30 tasks per year. We will compress the curriculum
especially for lower levels, because the learning areas are too much
(Sunday Times Zimbabwe, 4 March 2018).
The announcement in the national media implied that the implementation of the
competency-based curriculum was abortive at this point in time and that this
was due to the lack of support and collaboration within the system. This also
justifies the assertion of Barrow & Delisle (2010) that the inherent lack of support
from stakeholders makes implementation more difficult. Several other studies
reveal that teachers were uncomfortable with the way the new curriculum was
adopted, citing as reasons that the introduction was impromptu without giving
teachers opportunities to prepare for operationalising it. In a study conducted by
Mangwaya, Blignaut & Pillay (2016), the authors allege that there was lack of
consultation on the part of teachers who for this reason found it difficult
fulfilling their roles effectively. Elsewhere in the literature, it is further alleged
that the introduction of the competency-based curriculum created challenges for
teachers who were not well acquainted with the demands of the new practice,
especially its mode of assessment (Newsday Zimbabwe, 9 January 2017). In a
survey conducted by Dube & Jita (2018), the authors reported that the
introduction of CA brought more pain, friction and agony between teachers,
school heads, parents, as well as curriculum planners themselves.
Above all, there is overwhelming evidence to support the observation that the
introduction of the competency-based curriculum faced many challenges that
impeded its effective implementation. In that regard, there is therefore definite
need for research to be done in this area to build on the body of knowledge
about these challenges with respect to the O-Level Geography Syllabus 4022 in
the Zimbabwean context.
2. Review of the literature

This section sets the scene to provide the theoretical discourse surrounding
issues about teacher readiness to implement the competency-based O-Level
Geography Syllabus 4022 in the Zimbabwean secondary school sector. The
section begins by laying out the theoretical dimensions of the research and looks
at the nature and dynamics of the O-Level Geography Syllabus 4022 and finally
ends by examining the teacher readiness for the updated O-Level Syllabus 4022.
2.1 Theoretical framework

This research article contextualises the study within the theoretical discourse of
the Readiness Theory, mooted by Colney (2008). This theoretical discourse
alludes to the readiness discourse to imply a high level of professional
competence, which involves a combination of professional and personal
qualities necessary for high quality work in educational endeavours (Ozhegov,
46
2007; Belin & Gonchar, 2013; Lane & Bourke, 2019). Authors such as
Mizambaeva and Baimyrzaev (2019) use professional competence to mean the
theoretical and practical readiness of a teacher to carry out educational activities
at the highest level of professionalism. In the current study, the research
problem has been approached through a multidimensional and interconnected
component classification model involving gnoseological, axiological,
praxiological, and professional-personal components presented by Mizambaeva
& Baimyrzaev (2019) in Table 1.
Table 1: Classification of teacher readiness components

(Mizambaeva & Baimyrzaev, 2019)
Components Contents Competencies
The content involves various What follows are the

aspects of the Knowledge competencies developed from
System as indicated below: the proposed component
classification.
Gnoseological Knowledge of Geography: The
focus of the content is on: Readiness to use knowledge of
modern science and education
Understanding the role of problems in solving
Geography in the modern professional tasks.
scientific worldview and its
value. Readiness to interact with the
parties of the educational
Knowledge of basic Geography
process and social partners, to
concepts, relationships, theories,
lead the team, with a tolerant
cause-and-effect relationships,
perception of social ethno
laws, and regularities
confessional and cultural
underlying school Geography
differences.
courses.
Knowledge of ecology: this

aspect of content is concerned
with developing:
Knowledge of environmental
issues as a cross-cutting line of
Geography.
Knowledge in the field of

education and methods of
teaching Geography,
Knowledge of modern teaching

aids, professional methods, and
techniques.
47
Geographic knowledge of the

teacher:
The aim is to develop the
-Geographical worldview,
geographical thinking,
Geography methods,
Geography terms.
The aspect emphasises the Readiness to recognise the

development of values of social significance of their
Axiological individual teachers with respect future profession and to have
to their educational needs, the motivation to carry out
interests, emotions, and professional activities.
attitudes to teaching, as well as
the development of readiness Ability to carry out
for self-education and self- professional and personal self-
development. education, to design further
learning routes and
professional career.
Ability to develop the

trajectory of your professional
growth and personal
development.
Professional Skills: The main Ability to analyze the results

thrust is to develop the of scientific research, apply
following readiness skills: them when solving specific
research problems in the field
Gnostic Skills: ability to work of science and education, carry
with scientific papers on out independent scientific
Geography, education, research, the ability to apply
methodology; the ability to modern methods and
master the methodology of techniques to organise the
pedagogical research, etc. learning process, diagnose,
and evaluate the quality of the
Design Skills: the ability to educational process within
measure, observe, predict, and various educational
model natural and social programmes.
processes and phenomena in
time and space, etc. Ability to develop and
implement methodological
Constructive Skills: creating a models, methods, training
lesson plan, selecting the best techniques, to analyse the
teaching methods and results of their application in
techniques, and using modern educational organisations.
teaching tools.
Praxiological Ability to systematise,
Organizational Skills: The synthesise, and disseminate
ability to organize interaction international methodological
48
with students, the ability of experience related to teaching.

teachers to control themselves
and be tactful in unforeseen Readiness to conduct
situations, etc. professional communication to
solve problems related to
Communicative Skills: the teaching.
ability to carry out a
conversation or a discussion, the Readiness to interact with the
ability to establish rapport with parties to the educational
students, teachers, parents, etc. process and social partners to
lead the team.
Reflexive Skills: Self-analysis
and self-esteem, the ability to
adequately assess the activities
of students, etc.
Professional- Emotional and Intellectual Readiness to work as a

Personal Features of Geography teacher.
Teachers, as well as Operational
and Voluntary Qualities.
The study used the component classification presented in Table 1 in relation to

the readiness model developed by Colney (2008) to provide a theoretical basis
for interrogating the issues surrounding teacher readiness to implement the
competency-based O-Level Geography Syllabus 4022 in Zimbabwe. The Colney
(2008) model focuses on three related key facets directly linked to teacher
readiness development, namely cognitive strategies, content knowledge,
academic behaviours, and contextual skills (Mizambaeva & Baimyrzaev (2019).
The following paragraphs provide an apt exposition of the key facets of the
model and subsequent action plans necessary to ensure that teachers are ready
to deliver a new practice in an expeditiously timely manner.
Colney’s cognitive strategies represent the praxiological component in the

Mizambaeva & Baimyrzaev component classification (Table 1). The authors are
of the idea that readiness relates to the functional level mechanics of an
innovation, and therefore adopting cognitive strategies during teacher capacity
development allows them to integrate constructive skills, such as those required
for creating lesson plans, selecting the best teaching methods and techniques,
and using modern teaching tools (Colney, 2008). Apparently, key cognitive
strategies are necessary tools for developing professional skills that
subsequently lead to the development of various competencies, including the
teacher’s ability to apply modern methods and techniques, organising the
learning process, diagnosing, and evaluating the quality of the educational
process within various educational programmes (Mizambaeva & Baimyrzaev,
2019).
In addition, Colney (2008) considers the acquisition of key content knowledge as

critical to enhancing teacher readiness to deliver the content for the criterion-
referenced assessment. The refined consensus model considers five pillars to be
49
critical for a teacher to effectively deliver subject content. The five include
curricular knowledge, assessment knowledge, content knowledge, pedagogical
knowledge, and students' knowledge (Carlson & Daehler, 2019). The assumption
is that an appropriate praxiological plan of action should be adopted during
teacher capacity development to equip them with adequate knowledge that is
intrinsically specific to the subject taught in the school curriculum (Young, 2012).
Similar conception finds expression in the classification of the gnoseological
component, which involves teacher readiness development covering issues such
as readiness for the knowledge of modern science problems, as well as skills
required to solve professional tasks, team leadership and tolerance, perception
of social ethno-confessional and cultural differences (Mizambaeva &
Baimyrzaev, 2019). Elsewhere in the literature, Colney (2008) perceives
metacognitive skills as critical for developing teacher curriculum
implementation readiness. The author draws distinctive categories of
metacognitive skills to develop teacher readiness that includes self-awareness,
self-monitoring, and self-control. These skills relate quite well to the classes of
the praxiological and personal components presented in Table 1. The
assumption is that the possible outcomes from a teacher readiness development
programme necessitate the development of axiological and praxiological related
skills such as organisational skills, communicative skills, and reflective skills (see
Table 1).
Finally, Colney (2008)'s readiness model values the development of contextual

knowledge readiness for the development of systemic readiness in addition to
the generic understanding of the norms, values, and conventions of a new
practice. Colney (2008) further argues that contextual knowledge promotes the
teacher's ability to cope with and adapt to new systems. Thus, according to
Colney, teachers who do not understand or who are not ready to adopt the
institutions' norms, values, and expectations are more likely to feel alienated and
thus foster intentions to withdraw from the system voluntarily. Similarly,
Colney (2008) sees early readiness for organisational change as one of the most
important factors that affect the successful implementation of change. For this
reason, many teachers are expected to acquire readiness as an important
construct for the first step in the change process. According to Vakola (2013),
once teachers attain readiness, they become confident and motivated to
implement change. The literature further acknowledges that change can best be
made when members of an institution are ready to implement it. The
assumption is that if the change is accepted and understood by every
institutional member, then the members become motivated and committed to
provide support to implement it effectively. However, if change is ignored and
resisted, obstacles that inhibit the success of the planned change often emerge
(Vakola, 2013).
2.2 The Nature and Dynamics of the Geography Syllabus 4022

One of the major changes in education (post 2015) was the shift toward a
competency-based curriculum. More precisely, economic and political
challenges overwhelmed the need for a new curriculum; one that responds to
the external pressures and lived realities of the 21st century world (Mipfide &
Mapolisa, 2021; Simba, 2021). The upgraded O-Level Geography Syllabus 4022
50
was one such reform initiative. In essence, the upgraded O-Level Geography
Syllabus 4022 was developed from the MoPSE-designed Curriculum Framework
for Primary and Secondary Education (CFPSE) template. The syllabus adopted
competency-based approaches that allow learners to manipulate geographical
data and make informed decisions in their day-to-day experiences. Such
approaches are learner-centred and can be managed through the application of
orthodidactic principles and multisensory approaches to teaching (Zimbabwe
Schools Examination Council [ZIMSEC] O-Level Geography Syllabus, 2015-
2022). It is worth mentioning that in its programmatic whole; the upgraded O-
Level syllabus 4022 has been designed to cover the study of Geography from
Forms One to Four progressively, unlike the previous one, which only focused
on Forms Three and Four. Basically, programmes grounded within the
competency-based phenomenon have a link with what learners need to know,
what they need to learn to do, and to be able to live and work with other people
(MoPSE, 2021). Simply put, effective learning under a competency-based system
is one that endures and is capable of being put to good use in differing situations
in the future (Georgescu, 2015). Despite this educationally sound initiative,
teachers seem to be still not in terms with the best practices for delivering the
new practice yet. This has overwhelmed the need to ask questions about teacher
readiness to implement the O-Level Geography Syllabus 4022. The ensuing
section attempts to explain the concept of readiness, which is the theoretical
basis for this research paper.
2.3 Teacher readiness for the updated Geography Syllabus 4022

Teacher readiness is the premier phase of curriculum innovation; hence,
curriculum change should be framed by telos, which means by ‘sense of
purpose’ (Biesta, 2012). The implication is that developing readiness in teachers
requires a deliberate decision to ensure that they are ready before the start of the
implementation phase. Most importantly, Biesta is of the idea that ongoing
professional development campaigns should be adopted to get teachers ready to
use approaches that match the new practice. Hall & Hord (2006) use the analogy
of an implementation bridge to express the need for a thorough thought-out
strategy to direct teacher readiness development. The analogy of the
implementation bridge represents teacher readiness development programmes
whose purpose is to prevent teachers from failing to achieve the desired results
of their teaching.
Just like crossing a bridge, the implementation of an innovation is problematic

when teachers lack the praxiological skills that are important for maintaining the
use of a curriculum syllabus. In Zimbabwe, for example, some important
stakeholders, including teachers’ unions and PTUZ particularly, concede that at
the start of the new curriculum, professional development programmes were not
properly executed to promote pedagogical skills, making it difficult to transfer
the required syllabus competencies into practice. Evidence is available that
shows that teachers could not embrace the three competency-based pedagogical
skill related goals namely, helping students acquire important information and
work-related skills, making meaning of the content, and effectively transferring
the learning to new situations both within school and beyond (by Wiggins &
McTighe, 2008). In line with the challenges mentioned before, Whitehead in
51
Brown (2019) proposed the rhythm-of-learning model as an intervention avenue

that (Geography) teachers may adopt to make their teaching enjoyable.
Whitehead argues that courses and curricular evolve around three well-meshed
phases of learning, namely romance, precision, and generalisation stages; all
which characterise the rhythm of learning. Thus, the ideal introduction to a
curriculum subject is to be romantic, full of excitement and attractive, making it
alive, real, stimulating, and worth studying. In summary, the rhythm of learning
emphasises the acquisition of knowledge and skills that continue to sustain
interest, eventually leading to the generalisation phase. For Brown (2019), in the
rhythm of learning, learners need not be pressured to cover the topic, but should
be invited to explore it. Certainly, the rhythm of learning model exhibits
ascertainable general traits which are valid for most pupils, as it links quite well
with the component classification and therefore, teachers need to receive
appropriate training that makes them ready to adapt their teaching to suit the
stages in the rhythm to which learners have advanced (Brown, 2019).
3. Methodology
The approach to empirical research adopted for this study was one of a
qualitative multiple case study design. The technique drew data from analysis of
the primary documents published by MoPSE and ten in-depth key informant
interviews involving five Geography teachers from each of the two purposively
selected secondary schools. In addition, Focus Group Discussions (FGDs) were
conducted with two groups consisting mainly of O-Level Geography teachers
from each of the two secondary schools. To investigate the research problem, the
interview with Geography teachers took 10 to 15 minutes and 30 to 40 minutes
for each FGD. In all instances, data were generated through asking questions
based on a set of three categories that emerged as subthemes upon which the
discussion of results is centred. In addition, literature reviews were specifically
used to provide conceptual and theoretical support as well as to demonstrate to
the readers how this research particularly fits into the larger field of inquiry.
4. Limitations
Although the purpose of this research article was to solicit the opinions of all
Geography teachers in the Kwekwe district of Zimbabwe regarding their
readiness to implement the competency-based O-Level Geography Syllabus
4022, this was not possible due to several limitations. The potential limitations
were its small population size. Often, the generalisability of findings is
problematic if the data are not representative of wider contexts (Creswell, 2014).
The deliberate selection of such a sample arose from the small establishment of
the schools that were used as research sites, limitation of time, financial
constraints, as well as the Covid-19 restrictions on movement.

In soliciting participation in this study, informants were made aware that the
research was not going to use actual names. Instead, pseudonyms were used to
conceal the identity of the participants. The informants were identified by letters:
A B C D E while schools by numbers 1 and 2. The reader will find that the results
are presented and then immediately discussed. This approach has been adopted
52
for this study to avoid unnecessary repetition that usually results when
presentation of results and their discussion are separated (Mouton, 2008). In
addition, a thematic analysis approach has also been adopted in which the
presentation is structured into three major themes, viz:
o Dissemination of the O-Level Geography Syllabus 4022,
o Availability of Resources and
o Teacher readiness for implementation of the O-Level Geography
Syllabus.
These three themes formed the basis for discussing the research findings. In
adopting this approach, McMillan & Schumacher (2010) stated that it is prudent
for the researcher to interpret the collected data organised according to themes
and categories.
5.1 The Dissemination of Competency-Based Geography 4022

The way a syllabus is distributed is a critical factor that cannot be ignored in any
serious analysis of issues such as those surrounding teachers’ readiness to adopt
and subsequently implement a new curriculum package. For instance, an
analysis of the primary documents published by MoPSE suggests that the
adoption and dissemination of the updated O-Level Geography Syllabus to
secondary schools around the country was a fait accompli. In that case, teachers
were only asked to implement the syllabus without prior training. In
confirmation of this evidence, reference is made to a circular published by
MoPSE, part of which reads:
“This circular deliberates on the direction to the implementation of
Phase 2 of the new Curriculum commencing January 2017. Therefore, it
is incumbent upon every school to effectively implement the provisions
of the circular.”
It emerged from documentary evidence that the proposed change was more
revolutionary (in the sense that it was introduced using circular/it came as a
directive) rather than evolutionary (Mangwaya et al., 2016). This was a flawed
process to develop the readiness of the teacher to adopt the new practice, and
surprisingly, the change was resisted. In fact, the use of a circular is a clear
indication that no meaningful programmes were put in place to prepare teachers
for the updated O-Level Geography Syllabus 4022. One of the key informants
did not hide to say:
“We need thorough training, and this should be done by subject
managers from ZIMSEC. The idea of using circulars dilutes the quality
of information that ends us up with something different altogether as
circulars are prone to different interpretations by individual teachers. I
even doubt very much if the education inspectors themselves are
knowledgeable of what is supposed to be done. I am reliably told that
during a one-day seminar with school heads, the inspectors told the
school heads to go back to their schools and continue with what they
thought was the correct thing.”
This is consistent with the position of Stronge (2018) that teachers need an
induction of some sort prior to the implementation phase. If they are not ready
53
as curriculum end users, they can reject all planned changes in schools and can
act in negative ways such as being hesitant and fearful, and they can even
attempt to undermine the efforts of the changes that are to be implemented.
During a focus group discussion, Participant A from school 1 expressed
resentment by saying:
“I never received any training; instead, circulars were read to us during
a briefing after assembly, at which time we had different interpretations
of what the curriculum intended to achieve. The school head who usually
read the circular avoided questions at all costs fearing he wouldn’t be in
a better position to explain satisfactorily.”
The consensus from participants in School 1 is that the new curriculum was
brought in without consultation and sufficient guidelines; a situation which
could have triggered some confusion along the way. Teachers struggled to
understand the procedure, particularly on the issue of tasks and projects.
Teachers who participated in the focus group discussions at school 2, generally
agreed in their observation that pilot testing of the curriculum and a needs
analysis were not carried out. Pilot testing and needs analysis are critical
practices for effective implementation of reform initiatives as pointed out by
Ndawi & Maravanyika (2011) who acknowledged that these practices identify
the exact nature of the deficiency to be addressed and eventually single out the
exact change to be instituted. In some instances, needs analysis may reveal that
some conceived changes are not fundamentally different from the present
practice or cannot be instituted in a system for some reason or another.
Therefore, the omission of a need analysis during the decision to disseminate an
innovation often creates a disjunction that jeopardises the implementation of a
reform initiative. Teachers from both schools confirmed that they experienced a
disjunction in engaging the new curriculum in their classrooms.
5.2 Availability of Resources

An analysis of issues surrounding the readiness of teachers to implement O-
Level Geography would be incomplete without an examination of the
availability of resources, since their availability determines the extent to which
teachers are ready to use them for successful operationalisation of any
instructional practice carried out in the school with learners. Although it is true
that the Government of Zimbabwe (GoZ) provided material resources as
indicated in its terms of reference, the resources were grossly inadequate.
Participant C from School 1 remarked:
“When the new curriculum was introduced, the Government promised
it would provide resources to all public schools. I am appalled by the
existing state of affairs as not much in terms of material resource
support has been offered. There are no funds to purchase textbooks and
other essential resources like project files which are a requirement for
every subject in addition to several exercise books we used to ask for.
This has put an extra uncalled-for burden on the parents who are
already struggling to raise money for tuition.”
54
The available evidence suggests that the GoZ paid little attention to the
provision of adequate resources, as initially promised. To confirm this
observation, Participant A from school 1 made the following remarks.
“Considering the urgency that was given to introduce the curriculum,
most of us teachers were expecting the government to provide a school
with adequate resources and to train us on the proper handling and use
of curriculum material but that was not the case. We definitely need
resources in order for us to operationalise the new curriculum
successfully. Most of the concepts that were included to the curriculum
cannot be implemented without or with limited knowledge of the
resource use. For instance, you cannot just walk into the computer lab
and teach learners without having first to be trained.”
5.3 Teacher readiness for implementation of the O-Level Geography Syllabus

Teachers are the main hub around which the successful implementation of the
new curriculum revolves. However, it is particularly troubling to find that there
was an inherent lack of orientation for Geography teachers; therefore, the
implementation of the syllabus was made difficult. For example, teachers who
participated in the FDGs at both schools shared the sentiment that they were
asked to implement the updated curriculum immediately without
understanding how they were going to do it, as no explanations were provided
as advance information. As a consequence, the teachers involved experienced
serious problems in interpreting the new syllabus, the hallmarks of daily
classroom delivery, and practice. Based on the FGDs, the teachers of the two
schools did not receive a proper orientation to the new curriculum. An informant
who participated in FGDs at school 2 indicated that he was committed.
“I was introduced to the competence-based curriculum at a one-day
cluster workshop. The training I received was not adequate. I still do not
understand certain aspects of the competency-based curriculum to
effectively implement it. The trainers who happened to be fellow teachers
were not very much conversant with the competency-based system. The
training was too theoretical for teachers already in the field to
understand yet it was supposed to be hands-on to facilitate
understanding.”
All FGD participants agreed in their opinions that there was a need for ongoing
reconfiguration of professional development approaches that correspond to the
new practice, which should be done through a positive engagement of teachers
already in the field. The issue of positive engagement is supported by Ndawi &
Maravanyika (2011), who pointed out that policy makers and leaders must
positively engage teachers by fostering values of respect and justice to promote
the new idea and change the attitude of teachers. One of the findings of this
study is that the method used to induct teachers was dictatorial and full of
threats, and because of that, teachers fearing victimisation did not voice their
concerns, but just accepted what was dictated by the authority. The consensus of
the teachers who participated in the FGDs was that those who facilitated the
workshops were not knowledgeable, so they responded to the questions of the
participants in a threatening way. Participant E from school 1 expressed concern:
55
“Some schoolteachers could not attend workshops organised by MoPSE

as their schools could not manage the travel and subsistence allowances
as schools were left to sponsor their teachers to go for training. Instead of
sending a reasonable number of teachers, only one or no representative
would be sent. We were lucky as a cluster centre that all our teachers
received training although it was superficial to say the least.”
One of the most significant findings to emerge from this study is that teachers
were expected to implement the new curriculum with too little support,
guidance, and coherent orientation programmes and their readiness was
doubtful. In fact, this situation has been blamed for stifling gnoseological
competencies that are necessary to promote teacher readiness to effectively
deliver the O-Level Geography Syllabus 4022 as suggested by Mizambaeva &
Baimyrzaev (2019).The results of the study support the idea that change can best
be done when members of an institution are ready to implement it. Meanwhile,
Vakola (2013) agrees that if change is accepted and understood by every
institutional member, then members become motivated and committed to
provide support to implement it effectively, but if change is ignored and
resisted, obstacles that inhibit the success of the planned change often emerge.
Teacher B who participated in the FGD at school 2 had no kind words about the
way the new practice was introduced. The following comment was made:
“The cluster workshops we attended were actually useless to say the
least. They were facilitated by HoDs who were sufficiently confused and
were not confident. When asked to give more explanation, they failed.
We were left with no option except to do what we used to do.”
The general observation made in the study is that some certain aspects of the
updated syllabus were resisted because teachers were prematurely ready. The
study also went some way towards making it understood that when readiness
for curriculum change must be developed in teachers, action must be taken from
several fronts and that intervention strategies that must be instituted must be
done so through positive engagement of teachers to ensure expeditious delivery.
6. Conclusions
This study has taken into account the issues surrounding teachers' readiness to
implement the competency-based O-Level Geography Syllabus 4022 in
Zimbabwe. In general, the study found that the competency-based O-Level
Geography curriculum was introduced with insufficient consideration of teacher
readiness for its implementation. The study also found that the breadth and
depth of the issues surrounding teacher readiness to implement the competency-
based O-Level Geography Syllabus 4022 requires action to be taken from several
fronts to ensure that the subject community is ready for its rapid delivery. One
such action is to have in place an ongoing reconfiguration of professional
development approaches that correspond to the new practice. To be more
successful, this must be done through positive engagement of teachers already
in the field.
56
7. Recommendations
Taking into account the findings of the study on the issues surrounding teacher
readiness to implement the competency-based O-Level Geography Syllabus 4022
in the Zimbabwean secondary school system, this research recommends ongoing
intensive training to get teachers ready to ensure high levels of performance. In
addition, the study implores policy makers to make massive advocacy and
sensitisation of Geography teachers, the end users of the updated competency-
based O-Level Geography Syllabus 4022 for more effective implementation.
8. Acknowledgement
The authors are heartily grateful to all the people who, through various
contributions, made possible the success of this study. We are thankful to the
Geography teachers who participated in this study. Without their participation
and cooperation, this study would not have been a success. Finally, we are
grateful to our colleagues and friends who contributed immensely to this study
with moral support.
9. References
Barrow, D., & Delisle, J. (2010). A qualitative evaluation of some teachers’ Concerns and
levels of use of the lower secondary SEMP Science curriculum of Trinidad and
Tobago. Caribbean Educational Research Journal, 2(1), 3-16.
Belin, V. L., & Gonchar, E. A. (2013). Integration of cultural competence-based
approaches in education of intending geography teachers. Acmeology of Education
Developmental Psychology, 12(2), 90-93.
Biesta, G. J. J. (2015). How does a competent teacher become a good teacher? London: Brunel
University.
Brown, D. W. (2019). Whitehead’s Rhythm of Education. Middletown RI.
https://restlessageconomist.info/wp-content/uploads/2019/02/Whiteheads-
Rhythm-of-Education-1.pdf
Carlson, J. & Daehler, K. R. (2019). The Refined Consensus Model of Pedagogical
Content Knowledge in Science Education: Singapore, Springer Singapore
Creswell, J. W. (2014). Research Design: Qualitative & Quantitative Approaches. London:
Sage.
Conley, D. T. (2008). What Makes College Students Ready? Expecting Excellence .
Educational leadership Magazine, 66: 87-88
Dube, B., & Jita, T. (2018). Rethinking health school relations for curriculum change in
Zimbabwe: A relational leadership approach. Issues in Educational Research, 28(1),
901-917. https://search.informit.org/doi/abs/10.3316/ielapa.022365270326426
Georgescu, D. (2015). The Zimbabwe Review: Concept Paper. UNESCO, Beirut
Hall, G., & Hord, S. (2006). Implementing change: Patterns, principles and potholes. New
York, NY: Pearson.
Hume, A., Cooper, R & Borowski, A. (Eds.) (2019). Repositioning Pedagogical Content
Knowledge in Teachers’ Knowledge for Teaching Science. Singapore, 2019; pp.77−92.
https://doi.org/10.1007/978-981-13-5898-2.
Lane, R., & Bourke, T. (2019). Assessment in geography education: a systematic
review. International Research in Geographical and Environmental Education, 28(1),
22-36.
Mangwaya, E., Blignaut, S., & Pillay, S.K. (2016). The readiness of schools in Zimbabwe
for the implementation of early childhood education. South African Journal of
Education, 36(1), 792.
57
McMillan, J. H., & Schumacher, S. (2001). Research in Education; Conceptual Introduction.

New York: Longman.
McTighe, J., & Seif, E. (2011). Teaching for meaning and understanding: A summary of
underlying theory and research. Pennsylvania Educational Leadership, 24(1), 6-14.
Mipfide, I., & Mapolisa, T. (2021). School’s Administrations Role in Improving Learner
Performance at Ordinary Level Public Examinations in Zimbabwean Secondary
Schools. Indiana Journal of Humanities and Social Sciences, 2(2), 11-24.
Mizambaeva, F. K., & Baimyrzaev, K. M. (2019). Conditions for the formation of
professional competence of an intending geography teacher. Revista
ESPACIOS, 40(09).
Moyo, L., & Hadebe, L. B. (2018). The relevance of teacher education as a trajectory in
developing and sustaining inclusivity in the digital classroom. European Journal
of Open Education and E-learning Studies.
Mouton, J. (2008). How to succeed in your Masters and Doctoral studies. South African
Guide and Resource Book. Pretoria: Van Schaik.
Ndawi, O. P., & Maravanyika, O. E. (2011). Curriculum and its building blocks: Concepts and
processes. Mambo Press.
Ozhegov, S. I. (2007). Russian dictionary: Approx. 53000 words. М.: LLC" Publishing
house Оnix": LLC" Publishing house" Word and Education"[in Russian].
Simba, S. (2021). A feminist critique of ubuntu: Implications for citizenship education in
Zimbabwe
Stronge, J. H. (2018). Qualities of Effective Teachers (3rd ed.). Alexandria, VA: ASCD.
The Newsday. (2017, 9 January). http://www.newsday.co.zw
The Newsday. (2021, 9 September). http://www.newsday.co.zw
The Sunday Times. (2018, 4 March). https//www.sundaytimes.co.zw
Vakola, M. (2013). Multilevel readiness to organizational change: A conceptual
approach. Journal of change management, 13(1), 96-109.
Young, M. (2012). Education, globalization and the ‘voice of knowledge’. In The
knowledge economy and lifelong learning (pp. 335-347). Brill.
Zimbabwe Curriculum Framework for Primary and Secondary Education (2015-2022).
Harare: Government Printers.
Zimbabwe Schools Examination Council, Ordinary Level Geography Syllabus (2015-
2022). Harare: Government Printers.
Zimbabwe Ministry of Primary and Secondary Education (2021). Continuous
Assessment. Training Manual. Harare
Zimbabwe Ministry of Primary and Secondary Education, (2017). Secretary of education’s
report, Harare: Government Printers.
58

Vol. 21, No. 8, pp. 58-74, August 2022
Received May 18, 2022; Revised Aug 5, 2022; Accepted Aug 12, 2022
Exploring Headteachers, Teachers and Learners’

Perceptions of Instructional Effectiveness of
Distance Trained Teachers
Vincent Mensah Minadzi
College of Distance Education, University of Cape Coast, Cape Coast
Ernest Kofi Davis

College of Education Studies, University of Cape Coast, Cape Coast
Bethel Tawiah Ababio

College of Education Studies, University of Cape Coast, Cape Coast
Abstract. This study examined the perceptions of headteachers, teachers,

and learners regarding instructional effectiveness of University of Cape
Coast distance trained teachers. The convergent parallel mixed-method
design was employed to conduct the study. Research participants
comprising 25 headteachers, 667 distance trained teachers, and 138
learners were selected through purposive sampling, stratified random
sampling, and simple random sampling techniques respectively. Mean
and standard deviations, independent sample t-test, and thematic
analysis were employed to analyse the data. The study revealed that
headteachers, teachers and learners perceived distance trained teachers
as instructionally effective because they demonstrated professional
knowledge, professional practice and professional values and attitudes in
their instructions. Results also indicated that there was statistically
significant difference between male and female teachers with regard to
perceived instructional effectiveness. It is recommended that
stakeholders such as parents and prospective applicants should consider
UCC distance education programme as an alternative to the conventional
education.
Keywords: distance education; headteachers; instructional effectiveness;

pre-service teacher
1. Introduction
Instructionally effective teachers play important roles in the lives of learners
across the globe. It is against this background that Mtetesha (2017) asserted that
an effective education system in any society hinges on the caliber of its teaching
staff. This indicates that teachers who are effective in their instruction form part
©Authors
59
of the critical human resource as highlighted by Davis, Yarkwah, Beccles, and

Ayebi-Arthur (2019) that quality and effective teachers are important in the
preparation of quality human force. What constitutes effective instruction and
from whose perspectives instructional effectiveness is measured has been the
subject of concern for academics, researchers and school administrators. We
enumerated in this study three domains of instructional effectiveness from the
perspectives of headteachers, distance trained teachers and learners. Evaluating
effective teaching based on the views of headteachers, teachers and learners has
always been a contested area since researchers have not concluded on the validity
within the education enterprise (Almutairi & Shraid, 2021). Notwithstanding,
their perspectives on effective teaching could provide insights into the classroom
practices.
Training instructionally effective teachers falls within the domain of higher

education institutions. However, there came a time in Ghana’s history where
public universities and teacher training institutions were unable to meet the
teacher training needs as expected. This was due partly to unavailability of space
to accommodate ever-increasing youths who wanted to acquire higher education
(Kumi-Yeboah, Blankson, & Young, 2014). This with other possible factors
culminated in the birth of distance education in the country where a number of
both public and private tertiary education institutions engaged in the training of
teachers. Distance education is an educational experience during which
instructors and students are separated in time and space; and consequently, it
employs various technological gadgets so as to connect students with their
teachers (Tzivinikou, Charitaki, & Kagkara, 2021). We conceptualised distance
education as a teaching and learning phenomenon where study materials
(modules) are used in combination with technology to bridge the geographical
vacuum between the learner and the instructor.
The College of Distance Education (CoDE), a college dedicated to the training of

teachers at the University of Cape Coast has since its establishment in 1997,
trained thousands of teachers. The College trains different groups/cohorts of
teachers principally for the basic education system. We have pre-service teachers
who had not been professionally trained prior to enrolling on the programme
while the other enrolled on the programme to upgrade from diploma degree to
bachelor degree. The current study focused on the former cohort as the first group
to be given professional training. The focus on this cohort was to help make a case
for or otherwise of the contribution of distance programme to perceived
instructional effectiveness of teachers. Having trained many of this cohorts since
the inception of the distance programme, little is known about their perceived
instructional effectiveness. This study therefore sought to examine headteachers,
teachers and pupils’ perspectives through the mixed methods approach. The
study was guided by the following research question and hypotheses:
1. How do headteachers, teachers and learners perceive instructional
effectiveness of teachers trained through distance education?
H1: There is no statistically significant difference in teachers’ perception about
instructional effectiveness based on gender.
60
H2: There is a statistically significant difference in teachers’ perceptions about

instructional effectiveness based on gender
The raging debate about whether distance trained products can match those who
enrolled on traditional face-to-face programmes seems to be far from over. All
things being equal, distance trained and traditionally trained teachers who
pursued the same programme should exhibit similar instructional competences
in their teaching. Per the teaching standards developed by the National Teaching
Council (NTC) in Ghana, teachers irrespective of the training mode are expected
to demonstrate three minimum characteristics namely, professional knowledge,
professional practice and professional values and attitudes. Professional
knowledge is a collection of knowledge pre-service teachers acquired through
initial professional and continuous training as well as engaging in active
participation in professional collaboration (Wang, Lai, & Lo, 2014). Effective
teachers must possess vast knowledge about learners and how they should be
taught. Professional knowledge also covers knowledge of the official school
curriculum and the learning outcomes that are to be stated in the lesson notes.
Additionally, effective instruction covers demonstration of adequate subject
matter, knowing how to transmit the knowledge to the learner and knowing how
best to transmit the knowledge so that the learner in particular can learn better
(Nopriyeni, Prasetyo, & Djukri, 2019). Understanding the way learners grow and
learn in varied settings and how it should be applied in teaching to enhance
learners’ academic work is crucial to a good teacher (National Council for Tertiary
Education [NCTE], 2010).
Moreover, an effective teacher demonstrates professional practice which

comprises managing the learning space, teaching and learning and evaluation.
Managing the learning space is one of the surest ways to improve academic
outcomes of learners. Hence, an effective teacher ensures a safe and enabling
learning context that facilitates teaching and learning (Dhanapala, 2021). For
Danielson (2013), it is appropriate that teachers establish and monitor daily
routines for the smooth running of classroom activities including use of time. She
adds that, for learners to immensely benefit from what is taught, the classroom
environment must be orderly; a business-like atmosphere should exist and
dictatorial tendencies should be avoided. It is imperative therefore that an
effective teacher engages in a number of activities that promote learners’ academic
outcomes. For instance, an instructionally effective teacher adopts a variety of
teaching approaches that give learners encouragement and motivation to
participate thereby developing in them critical thinking. Assessment is very
critical in teaching and learning for it is one of the best approaches to determine
how learners have progressed. Assessment affords teachers the opportunity to
establish if learning has actually taken place (Msimanga, 2017).
Professional values and attitudes characterize how a teacher engages in activities

to project the image of the teaching profession within the school and beyond. This
may include behaviours such as establishing a cordial relationship with learners,
colleague staff, and parents. An effective and well-trained teacher should be seen
61
as someone who recognizes that the school is a professional learning community

(PLC) where knowledge is shared for the improvement of teaching and learning.
Being instructionally effective is not limited to classroom teaching but goes
beyond to include, for instance, serving as a role model for learners. Under
professional values and attitudes, an effective and qualified teacher should keenly
take part in school-wide activities such as Parent- Teacher Association/ School
Management Committee (PTA/SMC) events, sporting and cultural activities.
Effective teachers should also keep leaners records to serve as reference points in
monitoring learners’ progress.
Perceived instructional effectiveness, in general, and in distance education in

particular continues to receive the attention of scholars and researchers. Åhslund
and Boström’s (2018) study describes how primary school teachers perceive
differences in behaviour and learning of boys and girls, and showed that they had
positive view of their teaching. A case study was conducted by Jangu (2015) in the
Upper West region of Ghana to evaluate the perception of headteachers on the
instructional performance of distance trained teachers. Results of the study
showed that headteachers were satisfied with the instructional performance of
distance trained teachers. It can further be explained that teachers who obtained
their certificates through DE were perceived as effective in delivering the
curriculum in the classroom. The finding was in line with Samkange (2016) whose
study showed that headteachers perceived that there was no major difference
between traditional face-to-face teaching approach of training teachers and the
open and distance learning (ODL) means of training teachers. This means that
teachers who were produced through DE were perceived to be as good as
graduates from the conventional mode.
Moreover, Fernandez-Garcia, Maulana, Inda-Caro, Helms-Lorenz and Garcia-

Perez (2019) conducted a study concerning students’ evaluation of their teachers.
The purpose of the study was to examine students’ perception of teaching
behaviour and teacher characteristics that explain teacher effectiveness. The study
revealed that students perceived teachers’ learning climate, efficient classroom
management and instructional clarity (less complex task) as good while activating
teaching, differentiation and teaching learning strategies (more complex task) as
sufficient.
Another variable of interest to researchers and educationists is whether gender

has influence on perceived instructional effectiveness of teachers. A gender and
perceived instructional effectiveness study was conducted by El-Emadi, Said and
Friesen (2019). The study examined the motivation factors and attitudes toward
and interest in science among Qatari students. It was shown that there were
differences in the teaching styles of male and female teachers. The finding that
there is difference in perceived instructional effectiveness based on gender finds
expression in the study by Ahmed, Ambreen and Hussain (2018). Their study
revealed that female teachers exhibited more classroom management skills on
four out of six dimensions of classroom management than the male teachers.
However, the finding was inconsistent with the work of Singh and Attri (2020),
62
who found out that there was no significant difference in the perceived
instructional effectiveness of male and female teachers.
The current study was underpinned by social constructivism theory developed

by Vygotsky (1968). The theory is premised on the assumption that humans
experience, communicate and understand reality through two sources – language
and culture. According to Vygotsky, culture and language play a pivotal role in
cognitive development and how humans conceptualise the world around them.
It indicates that learning concepts and principles are transferred through
communication, interpreted and understood by experience and interactions
within a particular social setting. Within this context, it takes a group of
individuals to have language and culture to construct cognitive structures.
Knowledge, therefore, is not only socially constructed but co-constructed (Akpan,
Igwe, Mpamah & Okoro, 2020). The connection, therefore, is that while the
constructivist sees knowledge as what students construct among themselves
based on the experiences they gather from their environment, the social
constructivist sees knowledge as what students do in collaboration with other
students, teachers and peers (Akpan, et al. 2020). An effective teacher adopts
collaborative approach where learners are encouraged to learn in groups through
sharing ideas and knowledge. This easily deepens learners’ understanding of the
knowledge they are constructing in the classroom.
In addition, the study applied the theory of self-efficacy, a psychological construct

which is rooted in the framework of social cognitive theory propounded by Albert
Bandura in 1977. It is conceptualised as people’s conviction about their capability
to produce acceptable and desired results within an organisation. Self-efficacy
beliefs provide direction as to how people feel, reason, and encourage themselves
to take certain action (Bandura, 1994). A person who has positive orientation
towards self-efficacy is believed to succeed because the chances are that there will
be perseverance towards an activity until it is completed. However, a person with
low perception of self-efficacy foresees failure and is less likely to make conscious
efforts or persist in challenging the status quo (Harahsheh, 2017). Self-efficacy can
be equated to “I can do this” attitude. Researchers such as Clement and
Rencewigg (2020) investigated into the characteristics of effective teachers and
discovered that good teachers are optimistic for academic excellence and
behaviour for their learners to give off their best. The theory of self-efficacy is
premised on the assumption that expectation of personal capability and ability to
make progress is depended on whether a person will be ready to take a particular
cause of action. It is also presumed that an individual’s ability to demonstrate a
specific action lies basically on the persons’ psychological status, including the
drive, readiness to exhibit the behaviour, endurance in the face of setbacks,
dedication, success in foreseeable future, and the positive mind.
By way of filling the research gap, all the studies reviewed were different from
the current study in terms of multiple respondents in a single study as in the case
of the present study. Headteachers, teachers and learners occupy different
positions within the teaching and learning context to the extent that when their
views converge, such views are deemed to be valid. Moreso, we also employed a
63
mixed-methods design to seek different perspectives to the topic under study

since it is one of the complex human experiences. A single approach may not yield
the kind of results that may be encompassing.
3. Methods
3.1 Research design
The researchers adopted a convergent parallel mixed-method design to carry out
the study. The mixed-method was employed on the grounds that social
phenomena such as teaching and learning processes can best be understood by
adopting both qualitative and quantitative data collection and analysis methods
in the same research (Creswell & Creswell, 2018). The researchers believe that this
approach would enable them to understand the complex nature of human aspects
(Plano Clark, Foote & Walton, 2018; Minadzi, Gyimah & Ankoma-Sey, 2019) like
teaching and learning. In this milieu, the study was carried out by employing
different data collection instruments to seek how headteachers, teachers and
learners perceived instructional effectiveness of distance trained teachers from the
University of Cape Coast. We merged the data sets (point of interface or
integration) during the interpretation stage where the results from the qualitative
and quantitative data were compared for confirmation, corroboration and
complementation or disconfirmation.
3.2 Participants, sample and sampling techniques

Respondents comprised teachers who graduated from the College of Distance
Education and had up to six years of teaching experience in public basic schools
at the time of the study. The year was limited to six since the impact of good or
poor training stays a bit longer and can be examined for at least few years after
the pre-service teachers have completed the programme (Wendel, 2000).
Headteachers and learners in whose schools’ distance trained teachers teach were
included in the study. The inclusion of the headteachers and learners was to
triangulate distance trained teachers’ views with respect to their perceived
instructional effectiveness. Research respondents were selected from three out of
sixteen randomly selected regions in Ghana. A region each was selected from the
three administrative zones of the College of Distance Education (CoDE) namely,
Ashanti region from the Middle Zone, Volta Region from the Southern Zone, and
Northern Region from the Northern Zone. Record from Students’ Records and
Management Unit (SRMU, 2020) showed that a total of 15,671 comprising 8997
males and 6674 females graduated within the period. In all, 399 males
(representing 59.8%) while 268 females (representing 40.2%) were selected
through the stratified random sampling technique. Their average age ranges from
21 to 41. The majority (56.5%) of the teachers indicated they had 5-6 years
teaching experience while 24.8% had less than 3 teaching experience
respectively. Finally, 18. 7% stated they had 3-4 years teaching experience.
Twenty-five (18 males and 7 females) headteachers with average age ranging from
31 to 57 were selected through the purposive sampling technique. Headteachers
were chosen because they had deep knowledge and experience in classroom
practices and could provide rich information (Cresswell and Plano Clark, 2011)
about the perceived instructional effectiveness of distance trained teachers. Their
inclusion in the study, therefore, was important since they play roles such as
64
planning, supervision of staff and vetting of lesson notes. These activities equip
them to be able to provide valid information concerning effective teaching. For
the headteacher to be included in the study, his or her teacher who went through
the UCC distance programme willingly agreed to be observed in the classroom
(we were unable to carry out classroom observation due to COVID-19 pandemic
leading to the closure of schools). Hence, headteachers were excluded on the basis
that their teachers were unwilling to be observed in the classroom. Moreover,
headteachers who also pursued distance programme from the same institution
were excluded from the study. Learners (67 males and 71 females) whose average
age ranged from 11 to 14 years were also included in the study because they
occupy a critical position in the teaching and learning process and are at the
receiving end of quality or poor teaching. They could tell whether the teacher is
good or otherwise (McCoy, Smyth, Watson, & Darmody, 2014) in terms of
classroom practices. Six learners each were selected through random sampling
from the schools where headteachers and distance trained teachers were selected.
In total, 23 FGDs were conducted across the three regions.
3.3 Data collection instruments

Three instruments namely, interview guide, questionnaire and focus group
discussion (FGD) guide were used to collect data. Questionnaire was used to data
from distance trained teachers. Part A of the questionnaire had seven items which
dealt with the bio data of respondents while Part B covered the perceived
instructional effectiveness of distance trained teachers. Part B of the questionnaire
was adapted from National Teachers’ Standards (NTS, 2017) for teaching, a
framework designed by the National Teaching Council in Ghana to assess
teachers’ instructional practices. The framework has three domains namely,
professional knowledge, professional practice, and professional values and
attitudes. Coincidentally, the framework aligns well with the widely used
Danielson (2013) framework, a framework used to assess teachers’ professional
practices. Danielson’s framework has four domains which are planning and
preparation, the classroom environment, instruction and professional
responsibilities. The framework was used because it is rooted in the social
constructivist theory of learning developed by Vygotsky (1978) which is premised
on interaction, discussion and sharing of ideas among students (Akpan, Igwe,
Mpamah & Okoro, 2020). Additionally, the NTS framework was used because it
was the only locally developed standard document to assess teachers’
performance in Ghanaian basic schools. Finally, Part C of the questionnaire was
adapted from Nsamba’s (2016) instrument which yielded reliability coefficient of
0.89. The questionnaire was designed to find out students’ perception about the
service quality provided by the University of South Africa (UNISA). This part of
the questionnaire has four subheadings which included student support service,
face-to-face teaching session, teaching practice, and assessment/evaluation
procedures. Interview and FGD guide were used to collect data from
headteachers and learners respectively. The interview guide and FGD were
designed to cover the three domains of teachers’ classroom practices namely
professional knowledge, professional practice and professional values and
attitudes.
65
3.4 Validity and reliability

Prior to pretesting the data collection instruments on the field, they were validated
through seeking the assessment of two experts at the College of Education Studies
who had vast experience in the area of teacher education. This was done to seek
their inputs regarding the challenges with the items. Their inputs helped to fine
tune the instruments for optimal results. The questionnaire was pre-tested in the
Central region of Ghana before final administration for the study. Fifty (50)
distance trained teachers were selected for the pre-test. This sample was good for
pre-test because Ornstein (2022) suggested a sample between 20 to 50. The
Cronbach Alpha for professional knowledge, professional practice and
professional attitudes and value was .974, .973 and .973 respectively. After the pre-
test, words or phrases which were not clear to the respondents were modified or
deleted. The interview and FGD were pre-tested on two headteachers and six
learners respectively to enable fine tune the instruments before the final
administration. The initial four questions in the interview guide and FGD were
reduced to three questions because they appeared to lack clarity.
3.5 Data collection and data analysis

Prior to the data collection, clearance was obtained from the Institutional Review
Board at the University of Cape Coast. Apart from the IRB clearance, an
introduction letter was obtained from the three regional directorates of Ghana
Education Service that permitted us to have access to the selected schools. The
researchers self-administered the questionnaire in the three regions. Data
collection assistants were recruited and given brief training about questionnaire
administration. For example, explaining to respondents the purpose of the study,
when to administer it and how to handle the questionnaire after filling them to
ensure no respondent was identified. Researchers themselves conducted the
interview and FGD in the respective schools of the respondents. The interview
was conducted in the headteachers’ offices while the FGDs were conducted in the
classrooms during the time learners were on break. The classroom enabled the
learners to feel relaxed since they were used to the environment and that it also
provided a conducive atmosphere devoid of disruptions. Data collection lasted
for 3 months.
Research question one sought to find out how headteachers, teachers and learners
perceived instructional effectiveness of CoDE trained teachers. Descriptive
statistics such as mean and standard deviation were used to analyse the data from
respondents. Specifically, data from distance trained teachers were analyzed
using mean, standard deviation, and independent sample t-test. Qualitative data
from headteachers and learners was analysed using thematic approach. The
qualitative data were reduced to manageable units through coding into themes or
categories based on emerging patterns or similarity among the codes. The
reduction exercise was helpful to us because we were able to edit the data,
summarize it, and make it presentable. For example, after the coding exercise,
major themes namely professional knowledge, professional practice, professional
values and attitudes emerged from minor themes. Some of the minor themes were
knowledge of the learners, knowledge of the curriculum, managing the learning
environment, teaching and learning, assessment and school as a learning
community.
66
4. Results
In all, six hundred and sixty-seven (667) distance trained teachers comprising 399
males and 268 females completed the questionnaire. Moreover, twenty-five (25)
headteachers whose school distance trained teachers teach were purposively
selected for interviews. Finally, 6 learners each were also randomly sampled to
engage in FGD in schools where the headteachers were selected.
Standard deviations which range from 0.70 to 0.85 were moderate and closer to
each other indicating the non-dispersion in a widely-spread distribution. The
moderateness of the standard deviations of the distribution implies that the
perspectives of the respondents were coming from a moderately homogeneous
group. This indicates, to large extent, that the group had similar characteristics or
understanding with regard to the issues being investigated. In other words,
teachers’ views on perceived instructional effectiveness were an approximation to
a normal distribution.
Responses to the close-ended items, used in collecting data on perceived

instructional effectiveness, were measured on a four-point unilinear scale. This
ranged from one to four where one indicated the strongest disagreement to the
items while four indicated the strongest agreement to the items. Based on the four-
point numerical scale used, the study interpreted the results using
recommendation of Sarstedt and Mooi (2019) which states that in a unilinear scale
items, the responses to the items can be interpreted, using mathematical
approximation. Therefore, the study adopted mathematical approximation
techniques to interpret the mean scores. These were Strongly Agree (3.5 – 4.0),
Agree (2.5 – 3.4), Disagree (1.5–2.4), and Strongly Disagree (1.0–1.4). The
quantitative results on teachers’ perceptions of instructional effectiveness of
distance trained teachers are presented in Table 1.
Table 1: Summary of CoDE trained teachers’ perception of perceived instructional

effectiveness
Items Mean SD
Professional knowledge 3.29 0.76
Professional practice 3.21 0.74
Professional values and attitudes 3.19 0.73
Mean of all means (perceived instructional effectiveness) 3.23 0.74
Source: Field survey (2020) (N = 667)
Results from Table 1 showed that respondents perceived professional knowledge

of teachers in positive terms (Mean = 3.29, SD = 0.76). The results suggest that
distance trained teachers possessed knowledge of the curriculum framework and
subject matter, and understood learners’ individual needs. The results from the
Table further showed that respondents (CoDE trained teachers) perceived
professional practice within the context of managing the learning environment,
teaching and learning as well as assessment positively (Mean = 3.21, SD = 0.74).
In all, the results seem to indicate that through the UCC distance programme,
teachers demonstrated professionalism in terms of managing learning in the
classroom effectively. The results revealed that teachers perceived themselves as
effective in handling teaching and learning activities during lessons. In addition,
67
the results indicated that the programme equipped distance trained teachers with
professional values and attitudes. They were able to appreciate the need to
participate regularly in workshops/in-service training, develop love for
the teaching profession to serve as role model to their learners and the need
to delegate work to both male and female learners in their respective
classes. Meanwhile, the overall mean (Mean =3.23, SD = 0.74) signifies that
largely CoDE trained teachers perceived themselves to demostrate professional
knowledge, professional practice, and professional values and attidues in the
discharge of classroom activites.
Analysis of interview data showed that 23 out of 25 headteachers appear to

confirm teachers’ perceptions. Interview number 18 reflects the confirmation from
headteachers.
Her output of work, in fact I would say is perfect. You go to meet her
teaching then you realise that she has the technical knowhow of the
subject that she is teaching. Sometimes you meet somebody teaching a
particular topic then when you observe the person, you realise that the
person doesn’t have the technical knowhow or subject knowledge
[Interview 18, 11/3/2020].
Analysis of learners’ data showed that 21 out of 23 FGD corroborated

teachers’ views concerning professional knowledge as for instance:
Before he starts teaching, he reviews with us the previous lesson and talks
briefly about what he will teach. He reviews the previous lesson because
he thinks that we might forget what we learnt previously. This will tell
him whether we are learning what he is teaching [Participant 5,
5/2/2020].
Again, teachers’ perceptions on professional practice with specific reference to

managing the learning environment were confirmed by all the 25 headteachers.
For example, interview 1 indicated:
During lessons his classroom environment can be described as friendly.
He teaches Mathematics and Ghanaian Language. Someone came and
we gave Mathematics to him and now he teaches Ghanaian Language.
And you know children want to identify with subjects they can easily
relate with. Even when he was teaching Mathematics the same situation
existed. So, it tells you that he is friendly. His approach to the subject is
good [Interview 1, 4/2/2020].
Twenty-one (21) out of 23 FGDs from learners were consistent with the
perceptions of teachers concerning managing the classroom environment
by CoDE trained teachers. For instance, FGD 5 affirmed teachers’ views:
He is friendly so if he gives us exercise and you did not get all, he will
find out the problem. If you are learning and it is not giving results, he
will advise you what you should do to improve [Participant 5,
5/2/2020].
68
More so, results from Table 1 showed that respondents (CoDE trained teachers)
agreed that CoDE programme had helped them in boosting their professional
values and attitudes (Mean = 3.19, SD = 0.73). This appears to suggest that through
the CoDE programme, they were able to develop their professional attitudes and
values towards the teaching profession. 18 out of 25 headteachers corroborated
with what teachers said. Interview 7 reflects the general observation from
headteachers:
If she is not interested in upgrading herself, she would not have enrolled
on the UCC programme. For instance, I was there and she came to find
out something on the net. For me, she is trying to find something to
improve on herself [Interview 7, 7/2/2020].
Consistent with teachers’ perceptions, 19 out of 23 FGD indicated that learners

perceived CoDE trained teachers as demonstrating professional values and
attitudes. For example, FGD 1 reflected this:
Our teacher is very enthusiastic about teaching and that enable us to
enjoy his lessons. He does not want to waste his time or period in doing
things that are not necessary [Participant 4, 4/2/2020].
Though there seemed to be positive ratings for CoDE trained teachers with respect
to professional values and attitudes, there was negative reaction from one of the
headteachers. It was revealed that the teacher did not show interest in school wide
activities such as PTA and SMC meetings:
I don’t think he is active in the participation of PTA/SMCs. I don’t
because sometimes you organise and you want all teachers to be present
and he would not come with no permission. I hope you are a teacher. You
know the way teachers are. Someone may not even come to school, no call,
no anything. How long would one keep on asking you? One can hardly
see him around.
The head teacher’s comments showed that this particular teacher did not live up
to expectation as a professional teacher. A professional teacher is expected to
involve in all school wide activities and seek permission if one is to absent him or
herself from these activities.
H10: There is no statistically significant difference in teachers’ perception about
teacher effectiveness based on gender.
H2: There is a statistically significant difference in teachers’ perceptions about
instructional effectiveness based on gender.
Table 2 presents summary of teachers’ perceptions about instructional

effectiveness based on their gender.
69
Table 2: Summary of independent sample t-test on CoDE teachers‘ perception of

instructional effectiveness by gender
Variables Gender N Mean SD t-value p-value η2
Professional knowledge Male 399 3.345 .736
Female 268 3.203 .793 2.365* .018 .008
Professional practice Male 399 3.238 .719
(managing the learning
environment) Female 268 3.125 .736 1.964* .047 .006
(teaching and learning) Female 268 3.141 .735 1.948 .052
(assessment) Female 268 3.155 .759 2.389* .017 .009
Professional values and Male 399 3.215 .712
attitudes Female 268 3.142 .763 1.265 .206
Perceived instructional Male 399 3.268 .685
effectiveness Female 268 3.153 .720 2.085* .037 .007
Source: Field survey (2020) *p<0.05 df = 665 (N = 667)
Where η2 = Eta Square, SD = standard deviation
Results from Table 2 showed that there was a statistically significant gender
difference in male (Mean = 3.345, SD = .736) and female (Mean = 3.203, SD = .793)
teachers’ perception regarding development of professional knowledge [t = 2.365,
df = 665, p = .018]. Similarly, there was a statistically significant gender differences
in male and female teachers’ perception regarding their professional practices in
the area of managing the learning environment [t = 1.964, df = 665, p = .047] and
assessment [t = 2.389, df = 665, p = .017]. In addition, the results showed that there
was no significant difference in perceptions between male and female regarding
professional practice (teaching and learning) [t = 1.948, df = 665, p = .052]. It can
be further observed that there was significant statistical difference in perception
between male and female in terms of professional values and attitudes [t = 1.265,
df = 665, p = .206]. Overall, the results revealed that there was statistically
significant difference between male and female as far as their perceived
instructional effectiveness is concerned. Based on the findings, the researchers
reject the hypothesis that there is no statistically significant difference in teachers’
perceptions concerning instructional effectiveness based on gender. This means
that the perceptions of teachers on their instructional effectiveness differed from
male to female.
5. Discussion
The study sought to examine how headteachers, teachers and learners viewed
instructional effectiveness of University of Cape Coast distance trained teachers.
Results from the analysis of data showed that UCC distance trained teachers were
perceived by stakeholders as instructionally effective in the three domains
namely, professional knowledge, professional practice, and professional values
and attitudes. It was evident from the findings that CoDE trained teachers were
perceived to possess professional knowledge since they demonstrated knowledge
of learners, knowledge of the subject being handled and knowledge of curriculum
70
framework. Effective teachers use a variety of teaching methods to meet different

learning needs of learners, and review learners’ previous knowledge to identify
the learning needs. Again, effective teaching includes asking learners further
questions for clarifications during lessons, having broad subject knowledge to
relate lessons to learners’ real experience, and demonstrating deep understanding
of topics.
The revelation that CoDE trained teachers were perceived to demonstrate

professional knowledge is situated within the theory of self-efficacy. This theory
states that teachers who are self-efficacious know their learners, create classroom
environments that are conducive to motivate learners to learn (O’Connor et al.,
2017), and are responsible for students with special learning needs (Laninga-
Wijnen, Ryan, Harakeh, Shin, and Vollebergh, 2018). Furthermore, it emerged
from the findings that UCC distance trained teachers were perceived to possess
professional practice because they demonstrated how to manage the learning
environment, adapt appropriate teaching and learning strategies as well as
conduct appropriate assessment.
The findings were situated in the theory of social constructivism which views
knowledge construction as a social interplay of people, interactions that involve
sharing, comparing and debating among learners while the teacher plays a
facilitation role. Instructionally effective teachers used the social constructivist
approach where group work is effectively used on to drive the teaching and
learning process. Headteachers, teachers and learners in their responses indicated
that they were encouraged to do class work together to boost their understanding.
Similarly, teachers’ demonstration of professional values and attitudes is very
crucial for effective instructions in the classroom. Teachers who are effective
demonstrate love for participating in professional programmes such as
workshops, in-service training, and PTA activities. They are also expected to value
policies and regulations that govern the teaching profession. Moreover, teachers
are supposed to join colleague staff to undertake activities that bring about
changes in their work place. During these events, new ideas and knowledge are
shared which ultimately could improve teachers’ effectiveness. Results from the
study showed that UCC distance trained teachers were involved in these activities
and observe the rules and regulations governing the teaching profession.
It was also indicated from the study that UCC distance trained teachers were
perceived to demonstrate characteristics that make them effective in their
instructions. This was in line with what Stronge (2018) outlined as constituting
qualities of effective teaching. He pointed out that effective teachers demonstrate
professional knowledge, instructional planning, instructional delivery,
assessment, and learning environment. The finding further agreed with the
assertion by Ko and Sammons (2013) that effective teachers should possess good
subject knowledge, classroom organisation, good questioning skills, and use of
appropriate group work. The results that UCC distance trained teachers rated
themselves positively aligned with Åhslund and Boström, (2018). Their study
indicated that teachers perceived their teaching as positive. The finding
corroborates the later study by Samkange (2016) who indicated that teachers
71
trained through open distance education in Zimbabwe were perceived to be

effective in the classroom. Also, the findings were consistent with Jangu’s (2015)
study which found that headteachers perceived classroom performance of UCC
distance trained teachers to be effective. The finding that learners described CoDE
teachers as demonstrating effective teaching skills is also consistent with the
earlier study by Fernandez-Garcia, Maulana, Inda-Caro, Helms-Lorenz and
Garcia-Perez (2019) which revealed that learners perceived teachers’ classroom
environment, classroom management, and instructional clarity, differentiation,
and teaching learning strategies as sufficient. However, the finding was
inconsistent with that of Iqbal, Ibraheem and Hussain (n.d) who found out that
the professional competencies of distance trained teachers from education
programmes do not meet the set standards of quality in Pakistan.
The results of the study further gave credence to the statement read on behalf of
the former Minister of Education Prof. Naana Opoku-Agyemang, that the
distance education programme offered by UCC is comparable to that of the
regular programmes (CoDE Digest, n. d). The then Executive Secretary to the
National Council for Tertiary Education (NCTE) Prof. Mahama Duwiejua who
read the minister’s statement added that “distance education is not inferior to the
mainstream and by the time you complete the programme you would have
developed some attitudes that would equip you for the challenge ahead” (p. 8).
The programme at the College is designed by experts in their area of specialization
to equip pre-service teachers with the necessary teaching skills to be able to
deliver the curriculum in the classroom. This means that graduates (teachers)
from the College should demonstrate pedagogical skills similar to other graduates
(teachers) from different teacher training institutions.
The study also examined whether gender of distance trained teachers had
influence on their perceived instructional effectiveness. Drawing conclusion on
whether gender has influence on perceived instructional effectiveness is difficult.
This is so because gender and perceived effective teaching tend to be varried in
different context. In some societies, there is the notion that women are the
nurturing type and therefore suit caring for children at the elementary school level
while in other societies, men are considered to be better teachers. Analysis of data
showed that apart from professional professional (teaching and learning), there
was signifcant difference in the perceptions of teachers with regard to
instructional effectiveness within the context of gender. Overall, there was a
statistically significant difference between male teachers with regard to the
perception of instructional effectiveness. Specifically, male teachers perceived
instructional effectiveness more positively than female teachers. This shows that
gender has an effect on the way teachers perceive their instructional effectiveness.
The reason for the difference in perception between male and female distance
trained teachers is difficult to ascertain. Some scholars have attributed the
difference to the fact that male and female teachers possess different teaching
styles (Islahi & Nasreen, 2013). The positive outlook in the perceptions in favour
of male teachers is strange because the long-held view is that teaching at the
elementary school level is seen as female profession. This resurrects the raging
debate whether teachers perceived differently the way they teach based on
72
gender. The finding that gender has an effect on the perception of teachers
regarding instructional effectiveness supports El-Emadi, Said and Friesen (2019);
Ahmed, Ambreen and Hussain (2018) whose investigations showed that there
was significant difference between the male and female teachers in terms of their
instructional practices. On the other hand, it contradicts Singh and Attri’s (2020)
investigation which found out that male and female school teachers do not differ
significantly in their instructional effectiveness. So, clearly there seems to be no
agreement whether instructional effectiveness can be influenced by the gender of
the teacher.
6. Conclusion
This study explored perceptions of headteachers, teachers and pupils concerning
instructional effectiveness of distance trained. We adopted mixed method design
to undertake the study. The study revealed that headteachers, distance trained
teachers and learners generally agreed that UCC distance trained teachers are
instructionally effective. It showed that distance trained teachers possess
professional knowledge, professional practice, and professional values and
attitudes. The findings in this study therefore appeared to suggest that distance
education programme had a positive influence on the classroom practices of
teachers. It further indicates that the UCC distance education programme is
effective and as such could be used as an alternative to traditional mode of
training teachers for the education system in Ghana. Moreso, the study revealed
that gender had influence on perceived instructional effectiveness of teachers in
the classroom. This suggests that gender is a major variable within the context of
teachers’ perceptions as regards their effective instruction in the school. It is
recommended that stakeholders such as parents, prospective applicants and the
Ghana Education Service should consider UCC distance education programme as
an alternative to the conventional education. This is due to the fact that the
distance programme is able to equip service teachers with the minimum
instructional competences.
The outcome of the study has significant practical implications for distance
education discourse, in general, and teacher education, in particular. Distance
education institutions involved in training teachers pay attention to the three
domains of teachers’ instructional competence. Literature indicated that if
teachers possess these characteristics and demonstrate it in their instructions,
learners make significant gains in their learning outcomes. Teachers need to
apprise themselves of the curriculum they transmit to students, understand that
learners are different, creating enabling environment, and assessing how learners
progress. Moreso, teachers need to demonstrate enthusiasm and share knowledge
within the school community. Distance training institutions equip teacher trainees
with these essential characteristics to be able to function well in the classroom.
Limitation of the Study: A limitation of the study was our inability to carry out
classroom observations to triangulate stakeholders’ perceptions. Our inability
was due to the closure of schools close to a year due to COVID-19 pandemic,
hence, the findings were based on perceptions.
73
7. References
Ahmed, M., Ambreen, M., & Hussain, I. (2018). Gender differentials among teachers’
classroom management strategies in Pakistani context. Journal of Education and
Educational Development, 5(2), 178 – 193.
Åhslund, I, & Boström, L. (2018). Teachers’ perceptions of gender differences: What about
boys and girls in the classroom? International Journal of Learning, Teaching and
Educational Research, 17(4), 28 – 44. https://doi.org/10.26803/ijlter.17.4.2.
Almutairi, T. S., & Shraid, N.S. (2021). Teacher evaluation by different internal evaluators:
Head of departments, teachers themselves, peers and students. International
Journal of Evaluation and Research in Education, 10(2), 588–596.
https://doi.org/10.11591/ijere.v10i2.20838.
Akpan, V. I., Igwe, U. A., Mpamah, I. B. I., & Okoro, C. O. (2020). Social constructivism:
implications on teaching and learning. British Journal of Education, 8(8), 49–56.
Clement, A., & Rencewigg, R. (2019). Qualities of effective teachers: Students’
perspectives. International Journal of Advances in Engineering and Management, 2(10),
365-368.
Creswell, J. W. & Creswell, J. D. (20218). Research Design: Qualitative, quantitative, and mixed
methods approaches (5th Ed). SAGE Publications, Inc. Los Angeles.
Cresswell, J. W, & Plano Clark, V. L. (2011). Designing and conducting mixed method
research. 2nd Sage; CA: Thousand Oaks.
Danielson, C. (2013). The framework for teaching evaluation instrument. The Danielson Group.
https://focus.teachscape.com/pdf/2013-framework-for-teaching-evaluation-
instrument.pdf.
Davis, E. K., Yarkwah, C., Beccles, C., & Ayebi-Arthur, K. (2019). Motivation to become a
teacher among pre-service teachers in Colleges of Education in Ghana. Bangladesh
Education Journal, 18(2), 7–22.
Dhanapala, R. M. (2021). The effect of learning environment on academic performance
from students’ perspective. Global Scientific Journal, 9(3), 1527 – 1534.
https://doi.org/10.11216/gsj.2021.03.49602.
El-Emadi, A. A., Said, Z., & Friesen, H. L. (2019). Teaching style differences between male
and female science teachers in Qatari schools: Possible impact on student
achievement. Journal of Mathematics, Science and Technology Education, 15(12), 1–16.
https://doi.org/10.29333/ejmste/109236
Harahsheh, H. (2017). Perceived self-efficacy and its relationship to achievement
motivation among parallel program students at Prince Sattam University. International
Journal of Psychological Studies, 9(3), 21 – 34. https://doi.org/10.5539/ijps.v9n3p21
Iqbal, M., Ibraheem, M. K., & Hussain, M. (n. d.) Professional competencies of prospective
teachers: Investigation of teacher education programs of distance education in
Pakistan. Journal of Elementary Education, 25(2), 139–152.
Islahi, F., & Nasreen, M. (2013). Who make effective teachers, men or women? An Indian
perspective. Universal Journal of Educational Research, 1(4), 285–293.
Jangu, A. (2015). The Perception of headteachers on the instructional performance of
teachers by distance learning in the Upper West Region of Ghana. International
Journal of Humanities Social Sciences and Education, 2(6), 141–149.
Ko, J. & Sammons, P. (2013). Effective teaching: A review of research and evidence. CfBT
Education Trust.
Kumi-Yeboah, A., Blankson, H., & Young, W. (2014). Emerging trends in distance education
and teachers’ education in Ghana. IGI Global.
Laninga-Wijnen, L., Ryan, A.M., Harakeh, Z., Shin, H., & Vollebergh, W.A. (2018). The
moderating role of popular peers’ achievement goals in 5th-and 6th-graders’
achievement-related friendships: A social network analysis. Journal of Educational
Psychology, 110(2), 289–307.
74
McCoy, S., Smyth, E. Watson, D., & Darmody, M. (2014). Leaving school in Ireland: A
longitudinal study of post school transitions.
http://www.esri.ie/UserFiles/publications/RS36.pdf
Minadzi, V. M., Gyimah, E., & Ankoma-Sey, V. R. (2019). Towards teacher preparation at
the College of Distance Education: Mentors’ challenges and managing strategies.
West East Journal of Social Sciences, 8, 109–121.
Msimanga, M. R. (2017). Teach and assess: A strategy for effective teaching and learning in
economic and management sciences [Ph. D Dissertation, University of Free State
Bloemfontein].
Mtetesha, N. (2017). Teacher education and the student teacher as an agent of change.
https://www.academia.edu/4941833/Teacher_Education_and_the_student_tea
cher_as_an_agent_of_Change
National Council for Tertiary Education [NCTE] (2012). Summary of basic statistics on public
tertiary education institutions. Accra, Ghana.
National Council for Tertiary Education [NCTE] (2010). Summary of basic statistics on public
tertiary education institutions. Accra, Ghana.
National Teachers’ Standards for Ghana (2017): Guidelines. Ministry of Education, Ghana.
Nopriyeni, Prasetyo, Z. K., & Djukri. (2019). The implementation of mentoring-based
learning to improve pedagogical knowledge of prospective teachers. International
Journal of Instruction, 12(3), 529-540. https://doi.org/10.29333/iji.2019.12332a
Nsamba, A. N. (2016). Exploring the quality of students’ support services in distance learning
environments. [Ph. D Dissertation]. University of South Africa, South Africa.
Plano Clark, V. L., Foote, L. A., & Walton, J. B. (2018). Intersecting mixed methods and
case study research: Design possibilities and challenges. International Journal of
Multiple Research Approaches,10(1), 14–29.
Samkange, W. (2016). Training teachers at a distance: Perceptions and challenges of open
and distance learning in teacher education: The Zimbabwean experience. Turkish
Online Journal of Distance Education, 14(4), 222–234.
Sarstedt, M., & Mooi, E. (2019). A concise guide to market research: The process, data, and
methods using IBM SPSS statistics (3rd ed.). Berlin, Germany: Springer-Verlag
GmbH. https://doi.org/10.1007/978-3-662-56707-4
Singh, K., & Attri, A. K. (2020). A study of teacher effectiveness of secondary school
teachers in relation to their gender, locale, educational qualification and teaching
experience. International Journal of Creative Research Thoughts, 8(12), 1126–1133.
Tzivinikou, S., Charitaki, G., & Kagkara, D. (2021). Distance Education Attitudes (DEAS)
during Covid-19 crisis: Factor structure, reliability and construct validity of the
brief DEA scale in Greek-speaking SEND teachers. Tech Know Learn 26, 461–479.
https://doi.org/10.1007/s10758-020-09483-1
Vygotsky, L. (1978). Mind in Society. London: Harvard University Press.
Wang, L., Lai, M., & Lo, L. N. (2014). Teacher professionalism under the recent reform of
performance pay in Mainland China. Prospect, 44, 429–443.
https://doi.org/10.1007/s11125-014-9315-0
Wendel, T. (2000). Creating equity and quality: A literature review of school effectiveness and
improvement. Society for the Advancement of Excellence in Education, Canada.
75

Vol. 21, No. 8, pp. 75-91, August 2022
Received Apr 10, 2022; Revised Jul 23, 2022; Accepted Aug 12, 2022
The Role of Middle Managers in Strategy

Execution in two Colleges at a South African
Higher Education Institution (HEI)
Ntokozo Mngadi
Graduate School of Business and Leadership, University of KwaZulu-Natal
Durban, South Africa
Cecile N. Gerwel Proches

Graduate School of Business and Leadership, University of KwaZulu-Natal
Durban, South Africa
Abstract. This research examined the role of middle management in

executing strategic plans in two colleges in a South African higher
education institution (HEI). Strategy execution is a complex operational
process. Flawed sense-making of the strategy could lead to strategy
breakdown, and challenges may occur concerning understanding and
executing the strategic plans. Therefore, the main aim of this study was
to examine how middle management in the two colleges at the HEI
engaged in the sense-making of the strategic plan. The study set out to:
examine how middle management participated in the sense-making of
the two colleges' strategic goals; determine how middle management
communicated the strategy to employees at the two colleges, and identify
the challenges middle management faced with strategy execution at the
two colleges. The research employed a qualitative research approach.
Interviews were conducted with a purposive non-probability sample of
ten middle managers in the two colleges at the HEI. Primary data were
collected through semi-structured interviews. Data were analysed using
thematic analysis. The interviews were conducted at the middle
management’s offices and lasted between 20-45 minutes. The study
findings revealed that each college engaged in the sense-making
processes differently and that there are different understandings of the
role of middle management in executing strategic plans. The study
highlights the need for middle management in the two colleges in the HEI
to engage in training in executing strategic plans. The findings may help
us understand how middle management in the two colleges executes
strategy and how sense-making occurs. In addition, it could assist with
role clarification of university service units in implementing the strategic
initiatives, as most literature focuses on strategy formulation rather than
execution.
Keywords: Higher Education Institutions (HEIs); middle management;

South Africa; stakeholders; strategic plans and execution
©Authors
76
1. Introduction
Higher education institutions (HEIs) in South Africa were a major part of the post-
apartheid social transformation process that changed the landscape of the South
African education system through mergers of various institutions in the higher
education sector (Popescu, 2015). This transformation was implemented to fulfil
the criteria for the sector to compete with higher learning institutions locally and
internationally, thus contributing to the internationalization of the education
system. Badat (2017) states that post-1994, various transformative advantages
have been pursued to transform South African institutions of higher learning.
Transformation is an indication of social progress (Mzangwa, 2019). One of the
changes in implementing strategic plans is to guide the transformation process. In
delivering well-structured strategic plans, universities align with the institutional
annual performance plan, contributing to national higher education policy.
Middle management in organizations are the primary drivers of strategic plans;

however, they often do not execute these as they should because of
misinterpretation of the objectives of a strategic plan, or sometimes having limited
knowledge (Van Rensburg et al., 2014). Challenges exist regarding understanding
and executing strategic initiatives by middle management in the two colleges.
Consequently, misunderstanding and unguided descriptions of their role occur in
each college institution. The misconception influences their ability to influence
subordinates to comply. Previous studies indicated that a lack of direction and
alignment of a strategic plan with higher education policy hinders progress and
success, which are crucial in highlighting the role of higher education in national
development (Van Schalkwyk et al., 2022). This study plays a vital role in
contributing to the body of knowledge on management. It will assist in
understanding how middle management in the university context executes
strategy and how flawed sense-making may lead to strategy breakdown.
The South African HEI considered in this study embarked on a new strategic plan
that builds on the HEIs historical strengths and allows for further improvements
to be made. The latest strategic plan provides an aspirational and robust
framework for the HEI to continue to thrive in the uncertain but exciting period
that lies ahead for the higher education sector in South Africa. Furthermore, the
plan is a basis for developing annual operational strategies in South African HEIs.
Though, there are challenges faced in middle management’s understanding and
execution of the HEI’s strategic initiatives. A lack of contextualisation of strategy
and validation of roles in middle management occurs. These challenges call for
new models of leadership that will withstand the changes in the higher learning
institutions and intricate systems implemented.
Middle management executes strategy by interpreting established strategy into

action plans and objectives (Van Rensburg et al., 2014). However, middle
management translates the information top management gives and should make
available the resources needed to execute the strategic plans properly. According
to Davies and Davies (2004), there is a growing need for middle management to
exercise a decisive influence on organizations. Middle management at the two
colleges of the HEI comprises administrative and technical staff members who
77
run the university's strategic plans. Like corporate organizational structures,

middle management is treated as front-line supervisors in the two colleges (John
et al., 2019).
Middle management’s lack of understanding and their inability to execute the

HEI’s strategic initiatives influenced their ability to persuade subordinates to
comply. The strategic goals might not have resonated with all staff members’
values (Sutphen et al., 2018), and it was suggested that the strategy lacked context.
In addition, middle management could not validate their roles due to the
challenges they faced during strategic execution, neither could they exercise vital
power during strategic execution because executives held the power and dictated
the strategic direction (Burgelman et al., 2018).
Strategic plans are implemented in all South African higher education sectors.
Strategies are indicative of the differences between organizations and institutions,
showing how competitive they are and highlighting the institutions' uniqueness
(Burgelman et al., 2018). Higher education institutions need to indicate to their
stakeholders their uniqueness through their strategic plans by outlining their
objectives to take advantage of the opportunities that globalization and
internationalization have to offer, and consequently increase global
competitiveness. Internationalization was identified as the critical goal of the
strategic plan linked with the Fourth Industrial Revolution (4IR).
Higher education institutions throughout the country are undergoing

reorganization to align their strategic plans with globalization and
internationalization. As a result, the HEI concerned has positioned itself by
investing in initiatives that respond to the global changes and the economic
landscape so that it can contribute in the form of research and resources. Although
there are challenges with executing strategic goals, South African higher
education has been striving to be visible in Africa and globally. The challenges
also affect the staff in charge of strategy execution. In this light, strategic plans
often look good on paper. but fail at the execution level, because of lack of
resources, poor participation from all stakeholders, and the lack of understanding
of the reasons for the strategic goals (Adobor, 2019). This study is relevant because
it exposed the roles of middle management in strategy execution as they are
supposed to be part of the critical stakeholders for implementing strategic plans
at higher education institutions.
Therefore, the main aim of this study was to examine how middle management
in two colleges at this South African HEI engages in making sense of the
University’s strategic plan. Middle management in HEIs generally executes
strategy by interpreting established strategy and implementing action plans based
on the strategy’s objectives. According to Netz et al. (2019), there is a growing
need for middle management to make fast strategic decisions under extreme
pressure in organizations. Thus, challenges are experienced regarding
understanding and executing HEI strategic initiatives by middle management,
especially under the forms of pressure that may be experienced in HEIs.
78
The definition of strategy has changed over the decades because of the evolution
of organizations and their values to the point where the concept of strategy has
now replaced policymaking and organizational behaviour. Moreover, scholars'
lack of unanimity has led to ambiguity in defining strategy concepts. In addition,
there is a lack of understanding and analysis in the evolution of strategy,
especially in the HEI context, because the idea of strategy in the past was based
solely on operational effectiveness.
There are generic modes of vital application tools to implement strategy, used
differently based on the problem. Examples of strategic tools are resource-based
approaches, competitive analysis, balanced scorecards, and portfolio analysis
models (Rengarajan et al., 2021). But, of course, a good strategic planning process
does not guarantee success until it is implemented (Tawse & Tabesh, 2021). These
vital tools encapsulate all the concepts, ideas, strategies, and techniques used for
strategic decision-making. Making a long-term strategic decision in the Volatile,
Unpredictable, Complex, and Ambiguous (VUCA) environment requires
ownership of the strategic goals (Putro et al., 2022). Strategic decision-making also
requires significant resources that relate to organizational changes.
Over the years, managers have adapted strategic tools, such as Mintzberg’s Five
P’s and Porter’s Five Forces, to facilitate strategy interactions. “Strategic thinking
is a distinct way of thinking that utilizes intuition and creativity, with the outcome
being ’an integrated perspective of the enterprise’” (Shaik & Dhir, 2020). Porter’s
strategy model is widely used because of the belief that competition is aligned
with strategic decisions (Kemp, 2021). Further, Nurlansa and Jati (2016) discuss
Porter’s model of strategy based on microeconomics, which, despite criticism
from Mintzberg and others, is still used in HEIs. Porter’s five competitive forces
model of strategy is grounded on understanding a firm’s business-level strategy.
One of the core roles of middle management is to implement organizational

strategy; thus, a strategy is a practice made possible by middle management.
According to Thompson and Wolstencroft (2018), corporate organizational
strategy takes an operational view compared to when discussed within an HEI
because of the level of accountability that middle management has to take on in
the corporate workplace to the educational sector. Furthermore, middle
management's role resonates more with the organizational expectations of what
they are required to perform. In addition, middle management administers and
focuses on systems, accepts the status quo, and takes a short-term perspective
(Thompson & Wolstencroft (2018).
Several perspectives of what middle management is, have been explored in the
literature. According to Van Rensburg et al. (2014), middle management's roles
depict them as potential strategy drivers and mediators between individuals from
different organizational units. On the other hand, when strategic plans do not
succeed, middle management is viewed as hesitant to change, despite their role
in staff continuity and chaos prevention (Mair & Thurner, 2008). Furthermore,
Kazmi and Naaranoja (2015) confirm that senior management’s inability to apply
79
strategic thinking may create issues with executing strategic plans; thus, these
functions are delegated to middle management. Despite their crucial roles, it is
not common to find a position termed as ‘middle management’ in an
organization; instead, the positions may be termed as ‘operational manager’ or
‘supervisor.’
Hambrick and Lovelace (2018) argue that the concept of sense giving was
discovered after the challenges that face leaders in the organization in a change
management task began to be recognized. Hambrick (2007, p. 335) explains sense
giving as “the process of attempting to influence the sense making and meaning
construction of others toward a preferred redefinition of organizational reality.”
The meaning construction implies that how middle management's information is
interpreted and communicated to subordinates is the organization's decision.
Logemann et al. (2019) emphasize that leadership shapes and directs the
interpretation of new strategies and realities.
Middle management’s roles involve communicating and translating the

information from the organizational strategy to the organizational staff. Van
Rensburg et al. (2014) state that the strategy often needs to be broken down and
filtered to the team. This includes using slogans and catchphrases to highlight the
essential aspects of the strategic goals which guide middle management when
presenting ideas to employees. Rouleau (2005) defines sense making as a method
whereby middle management constructs and reconstructs strategy to make sense
of it for themselves and then interprets the same to the staff. This definition
suggests that middle management must first break down the themes from a
strategy and then apply it to relevant areas of concern. Logemann et al. (2019) add
that sense-giving attempts to affect sense making and meaning construction of
others towards a model redefinition of organizational reality. It is a crucial stage
before the execution of the strategy. If issues arise during implementation, middle
management can resolve them efficiently because they would have understood
the basis of the strategy.
Rouleau and Balogun (2011) argue that sense-making requires not only middle
management’s input and expertise, but multiple stakeholders are involved in
sense making of the strategy. Therefore, stakeholder analysis should occur to
understand how stakeholders make sense of strategy and how they affect middle
management decisions during the sense-making process. McKiernan et al. (2018)
concur that stakeholders are essential in the strategy process and contend that
stakeholders' impact and influence on strategy are not discussed thoroughly
enough in the strategy research. While stakeholders can be invisible during
strategic execution, McKiernan et al. (2018) argue that stakeholders are essential
in strategic management and that their influence should not be ignored.
Globalization has led to growth in interdependence, interconnectedness, and

flexibility across global communities. Xing (2019) argues that HEIs must be
innovative in addressing specialized skills and creating excellent research
opportunities driven by private and public partnerships. Universities exist in
changing environments with a need to be adaptive and to respond with
80
exceptional strategies to meet external pressures (Ahmed et al., 2015). Rotaru et

al. (2019) state that managers know the importance of strategic objectives on
performance.
Strategic planning is about implementing organizational changes and creating a

valuable environment to excel in a specific field. However, Vuorinen (2018)
highlights limited literature on the strategic tools, models, frameworks, and
methods used to form strategies. In addition, strategy execution is an even less
researched topic than strategy development (Vuori, 2016). Organizational
changes, the absence of relevant literature on strategy development, execution
and middle management, and a lack of sense-making skills are some of the main
issues that lead to the failure of strategic plans.
Organizational change can cause anxiety among employees and result in a

reluctance to perform exceptionally towards the strategic plans. McKnight and
Hawkrigg (2005) state that the lack of full participation from employees may be
due to the lack of understanding of the strategic plans, where employees do not
feel a sense of urgency in executing a strategy and thus do not feel inspired to
work towards the strategy. Poor communication also may contribute significantly
to the collapse of a strategic plan. Change management is crucial as it impacts
sense-making processes (Li, 2018). Middle management plays an essential role in
change management because they are both recipients and implementers of change
(Mair & Thurner, 2008).
The best way to execute a strategy well is to be transparent about the strategic
intent. Strategic intent refers to the influential position an organization wishes to
take in the industry and the ‘roadmap’ to achieve this position. Unfortunately, not
all organizations deliver their intent clearly to the public because they miss the
essential elements of strategic execution and may focus on reorganization and
structural changes (Neilson et al., 2008).
The strategic intent needs to be supported by three critical aspects: vision, mission,
and organizational values. In an organization, it is essential to clarify the decision
flow to ensure that the information flows to all execution levels. Neilson et al.
(2008) concur that strategic plans can be executed well if service delivery matches
the competitors’ level. However, there may be a misalignment between the
organization's objectives and the execution of the strategy, which would result in
the organization not achieving the position it wants to.
Studies show that as middle-management roles are redefined, the activities

involved will require specific expertise, such as the sense-making process and
how best to apply the available resources to communicate the strategy.
Communication is the most significant part of strategy execution, and a lack of
this skill can result in the collapse of the organization's strategic goals.
Organizational changes may affect the strategy, and the inability to involve all
stakeholders during the developmental stages of the strategic objectives may lead
to failure.
81
3. Research Methodology
For the study we employed a qualitative research approach, drawing on a case
study. A qualitative research approach was suitable as it aligned with the
objectives of the study. The aim was to determine the in-depth views and opinions
of participants. An interpretivist approach was followed. Semi-structured
interviews were conducted with a purposive sample of ten (10) middle managers
at the relevant HEI, namely school operating managers, principal programme
officers, and technical managers at their respective campuses in Durban. The
interviews comprised open-ended questions, which were developed after
analysing the literature. Semi-structured interviews permitted participants to
express their opinions and views, and data collected from the interviews were
analysed using thematic analysis. Thematic analysis is a descriptive, qualitative
approach to data analysis. The study also focused on identifying themes and
patterns of behaviour when middle management executed strategy to gain insight
into and an overview of specific areas related to strategy (McTavish, 2006). The
first step in collecting data is to draw on transcribed conversations, either in the
form of direct quotes or paraphrased.
Thematic data analysis was conducted, (cf. Vaismoradi et al., 2013), according to
the following steps:
• Transcribe the interview and read the transcripts several times.
• Generate codes systematically across the data set according to the
subcategories.
• Collate codes into relevant research themes; the term theme must be
associated with a unit or domain.
• Generate a thematic map based on the codes.
• Define and name themes; additionally, discover themes and sub-themes
that are part of analysing the text (Ryan & Bernard, 2003).
• Select extracts relating to the objectives and research questions and draft a
report.
A covering letter was obtained from the HEI, and the Research Office granted
ethical clearance approval for this study at the university. All principles relating
to ensuring participants' anonymity and confidentiality were strictly adhered to.
Interviews were recorded and then accurately transcribed, facilitating credibility
and trustworthiness.

Four themes were identified, and patterns of behaviour were noted when the
strategy was executed by middle management: (1) strategy; (2) position; (3) sense-
making; (4) training and development. Each of these themes is discussed below.
4.1 Strategy
Knowledge of the strategy is a theme that was evident because participants could
explain the strategic concepts and definitions and knew the details of the strategic
plans. Participants themselves needed to comprehend the strategy at the
operations level.
82
The participants viewed the strategic plan as a vision, a five-year plan, with the
values of an institute for higher learning and operations aligned to the institution.
Participants did not feel the strategic plan belonged to them and they should own
it. However, they acknowledged the multiple stakeholders involved in
consultation before executing the strategy. Participants expressed the opinion
that the more they knew about how the strategic goals fitted in with all employees,
the better they could communicate these to the staff confidently. The participants’
knowledge of the strategic plan was technical and operational; they did not fully
identify it with their positions.
Participation in strategic plans is a sub-theme that emerged after the participants

had explained their participation in and inputs regarding the strategic plan
initiatives and projects. Middle management in the two colleges highlighted that
they had to apply their skills to be full participants in the university's strategic
planning, despite little training.
I support this one [strategic plan] because it encompasses both supports
of professional staff and academic, previously [it] supported academics,
and professional staff would not know how they fit in. (R2)
When the strategic plan was first drafted, the leadership of the university had not
shown any interest in including professional service staff in the strategic plan
engagements. Engagements related to the planning of the strategy had initially
not included middle management; therefore, it was not easy for middle
management to participate fully with enthusiasm and to know the roles they then
had to play in executing the strategy.
The overwhelming majority of participants indicated that discussing the strategy

with staff was part of their job profile and a critical factor in strategy execution.
While a process should be adopted to communicate the initiatives taken to
implement the strategy between the two colleges, no consistent method was used.
Most participants preferred to have meetings to discuss the policies and
documents with staff input, using PowerPoint presentations provided by Human
Resources, or created their PowerPoint presentations based on the policy
documents. One of the respondents said:
We had a meeting to present. Staff was allowed to internalize and ask
where they fit in, as leadership, we then explained where each staff fits in
and what it means to them. (R2)
Participants felt that the meetings could not be profitable if middle management
did not understand the content of the strategic goals and, most importantly, how
each staff member could participate in the strategy. Participants confirmed that
each college understood the strategic goals' narrative differently and
communicated separately to its staff members. Nevertheless, the standard means
of communication in both colleges were engagement in different forums and
thorough interrogation of the strategic plan with staff members.
Findings reveal that middle management in the two colleges emphasized specific
strategic goals because of the drive that came from each college’s leadership team
and their passion for a particular purpose. Higher education changes are
83
revolutionary; globalization and internationalization are significant drivers for

developing and executing strategic goals. Deans and heads of schools in the two
colleges are driven by these trends of advancing research regionally and globally.
However, other strategic goals involving professional service staff should also be
operated with the same passion.
This subtheme emerged when participants spoke about the challenges middle
management in the two colleges experienced during the strategy execution.
Initially, challenges were encountered when the strategy was planned and
introduced. One of these was that staff unions had opportunities for input and
made their views known regarding some of the strategic goals, which posed a
threat to the strategic plans. Afterward, participants also encountered challenges
in selecting from the action list during execution of the strategy, because
leadership of the two colleges did not consider how each member contributed to
the strategic plan's success. Participants indicated that the staff was only doing
the bare minimum because there was mixed information from the unions and
their line managers. For instance, one of the participants said:
There are some members of staff that take it seriously, there is a staff who
do the bare minimum, and those are problematic ones, they take it to the
unions, and they are supported. (R3)
4.2 Position
The theme was to show how much consultation was done across all middle
management and how many participated in formulating and implementing the
strategic goals. Institutions of higher learning are faced with challenges of
delivering excellent service to all stakeholders: students, government, and
communities. Participants mentioned that as drivers of strategic plans, they
recognized that they were under pressure to participate fully in the plans to
achieve the university's goals. All these plans spoke to the university landscape,
which was unstable and problematic. Participants added that full participation of
middle management is integrally linked to teaching and learning excellence.
Participants mentioned the leadership of the two colleges, as the core participants
in the implementation of the strategy, and they revealed the role of leadership in
the execution of strategic goals as paramount. They believed that college leaders
should play the role of facilitators and guides in strategy execution. Participants
assumed that for a strategic plan to materialize, leadership should have been
visible and open about their plans. Also, the two colleges' leadership at the HEI
needed to take centre stage in communicating and providing as much information
and direction as possible for middle management to better execute the strategy.
Transformational leaders act as a bridge between leaders and followers to develop
a clear understanding of followers’ interests, values, and motivations.
The strategy was well thought off because we were involved; the VC was
more involved and did campus visits, and looked at the challenges…
review company goals. (R7)
It was evident that participants understood their role, specifically school

operations managers, and why they were part of the strategy execution. However,
84
they felt much more responsibility than the college deans and heads of school who
were part of the strategy development.
As School Managers, we double as Dean’s assistants. Deans should run
with the strategy, [as] I take the baton from the Dean and make sure there
is buy-in with the staff. He communicates with me as his assistant and [I]
communicate to various schools. (R2)
Role clarification was a subtheme developed when it became clear that middle
management endured extra workloads and a lack of clarity on how the service
units contributed to the strategic goals. It was evident that participants
understood their role, specifically school operations managers, and why they
were part of the strategy execution. However, they felt that there was much more
responsibility in their position compared to the Dean/HoS, who was part of the
development of the strategy.
4.3 Sense-making
Interpretation of the strategy was recognized because of the explanation of sense-
making as a concept and a process that middle management in the two colleges
had to operationalize. Participants engaged in forums or cluster meetings during
staff meetings to interpret what was communicated by top management. The
participants in one of the colleges applied the same method of meetings and
forums, during which they went through the strategic documents sent by human
resources and corporate affairs. Nevertheless, they did not understand the
concept of sense-making.
I meet with my staff frequently at once and present whatever is important
from the strategy because not all the information there is relevant to all
staff. I then send emails and a reminder as we go over the year.
Constantly, the staff needs to be reminded of what is happening in the
University and how they can benefit and be of help to the strategy. (R9)
However, participants were expected to interpret a concept they did not

understand. One of the participants indicated that they did not fully understand
the viewpoint of the college is regarding the comprehensive university strategy.
The respondent felt there was no indication of the link between the college-
specific strategies and the university strategy.
It is difficult, because most of the time they say, ’management said’, so
staff members lose sight from CHS, we should have the vision to support
the university vision and filter it down to our school ………. Even though
we have an overall strategy, CHS needs to have our vision aligned with
the University. It must be unique from other colleges. (R7)
4.4 Training and Development

Most participants had to convince staff that the new strategy was inclusive across
all staff. However, this was proven difficult because the communication or
information provided was mainly geared to academic staff. Additionally, the
emphasis on research outputs excluded professional staff in the two colleges,
because they did not understand how it concerned them.
85
They felt like it is for academics, they do not know how they fit into the
strategic plan, I tried to explain to them, but they feel not much is said
about technical staff. (R1)
The participants mentioned a lack of training and development before executing

the strategic plan. Training and development before the execution of the strategy
would ensure efficiency in executing the strategic goals. The participants
indicated that staff development was strongly linked to executing the college and
the university's strategic plans. Middle management in the two colleges required
adequate resources and information to execute the strategy efficiently.
Participants emphasized the need for training to enable them to execute the
strategy well in a standard manner across the university. The training was
considered vital to prevent strategic goals from being misinterpreted or
interpreted differently across the university.
Capacitate on how to do things and communicate the strategy. (R8)
The responses indicated that performance management in the two colleges was
connected to executing the university's strategic goals. A few participants
emphasized meeting their crucial performance area (KPA) targets linked to the
university's strategic objectives. They stated that they would be viewed as poor
performers if they did not meet these. They stressed that college leadership
emphasized strategic goals, which put tremendous pressure on middle
management to perform optimally.
It will affect performance management in a hard way. Listen, we are judged
on how well you execute a challenge or a task. Therefore, I will not get any
high scoring if I do not take this strategy seriously. I will not have performed
to the expected level, and that can ruin my job in the University. (R9)
The study revealed that middle management in the HEI colleges was a strategy
driver and had a strategic role in executing the strategy. Leadership in the two
colleges, therefore, should, provide vision and direction to staff at the institution.
However, the literature fails to highlight how middle management interprets and
codes the information delegated to them by leadership. The study's findings
indicate a high responsibility for middle management to excel at their operational
duties and execute the university's strategic plans.
The research findings were contextualized by comparison with other studies on

sense-making and strategic plans available in the academic literature. This study
further showed that the government's institutional arrangements shape how the
institution should be structured, regulated, and financed. The concept of sense-
giving is mainly understood in an organizational context, with leaders providing
a view of reality to their network (Mirbabaie et al., 2020)
It was found that the concept of ‘sense-giving’ came about after organizational
change of management tasks because each time there was a change in leadership,
new strategic plans were given to the organization. It was also found that sense-
giving changes how information is written and articulated. This information
indicates that putting meaning into action needs a collective understanding and
cognitive application. The findings further showed that there were planners and
86
doers in the organization and that a breakdown of information happens if there is

no continuous learning and skills development. Failing of strategic plans might
be caused by a lack of understanding by executors of the plan, namely middle
management.
Communication channels, such as e-mails and staff meetings, were available for
middle management. However, understanding the strategic plan was more
important, because this allowed the easy flow of information between middle
management and staff during engagement sessions. As stated, strategy tools and
frameworks are crucial for managers to navigate their business environment and
formulate strategies. Middle management at the HEI did not use strategic tools;
however, the leadership of the colleges used these when developing the strategic
plan. Therefore, some scholars argue that there is a need for tools and decision-
making frameworks that would flow to middle management and are suited to
these specific changes in the business environment (Rengarajan et al., 2021)
There was no motivation for the staff to participate fully in the strategic plans
because they did not understand their roles and how they fitted into the execution
of the university's strategic plans. Strategy development should be an inclusive
process. Middle management in the two colleges had a big task, especially in
explaining how each member was part of the bigger plan and how they could
benefit from the strategic plan.
The study findings indicate that not only were the objectives set, achieved, but an
understanding was gained of the workload that middle management faced.
Further, the study revealed that all colleges differ regarding their information
flow. The results showed how much responsibility middle management carried
in executing strategic plans.
The study examined the role of middle management during the execution of the
strategic plan at an HEI. HEIs throughout the country are undergoing
reorganization; increasingly, their strategic plans are aligned with globalization
and internationalization. As a result, HEIs have positioned themselves by
investing in initiatives that respond to global changes and the economic landscape
to contribute to research and resources. Although there are challenges with
executing strategic goals, South African higher education has been striving to be
visible in Africa and globally. The challenges faced in this reorganization affect
staff in charge of strategy execution. In view of this, strategic plans are often sound
on paper, but fail at the execution level because of a lack of resources, lack of
participation from all stakeholders, and, primarily, poor understanding of the
reasons for the strategic goals (Adobor, 2019). This study is relevant because it
investigated the roles of middle management in strategy execution in the HEI
context. This study is especially relevant as universities have become more
‘corporatized’ over the years. Middle management should be part of the critical
stakeholders in implementing strategic plans at an HEI.
87
5.1 Recommendations
5.1.1 Contextualising the strategy and validating middle management’s roles
Middle management in the two colleges should attend the management
committee meetings with the college deans and heads of school to get first-hand
information to pass down to the staff, and should not merely rely on what the
college deans and heads of school have passed on to them. Further, middle
management in the two colleges can provide insights to these committees on
pressing matters, because they have first-hand information on these issues.
Middle management in the two colleges was found not to have a competitive
mindset aligned with the strategic plans. They ought to be strategic executors
based on their position at the university and their position as line managers of
professional support staff.
This study revealed that a lack of top management support influenced the drive
for executing a strategic plan. Strategic thinking should be applied - a cultivated
skill developed and mastered over time. Middle management should combine
analysis, exploration, and understanding elements to solve complex issues and
build planning capacity. Developing planning capacity guarantees teamwork
across all teams and between the colleges' middle management and leadership. A
strategy requires a new set of behaviour, resulting from training workshops and
engagement with the strategic policy documents. Strategic plans are best executed
if the behaviour change has not been initiated by middle management.
5.1.2 Constant communication regarding the stages in the strategic plan

The communication of the strategic plans was conducted on online platforms, and
no face was attached to the plans. Institutions of higher learning have advanced
technology that can apply operational strategies to communicate with employees
consistently. The staff could not identify who was part of the strategic plan. The
strategic initiatives and projects are numerous, and employees, therefore, become
confused about what other staff do and how they support these initiatives.
Strategic plans applied in different spheres and at different stages should be

visible to all staff in the institution. The spheres are from the planning stages until
the strategy's complete execution. The stages are embedded in the organizational
culture and collective experience of previous and current strategies. Strategies are
based on position and resources; therefore, emphasis on objects and people
working on particular goals should be visible and transparent. Leaders at HEIs
must attach a face to the strategy by doing campus visits and roadshows to show
that the strategy is happening. They should also emphasize that all the initiatives
require full participation from everyone. However, in this study, there was not
enough information provided to staff regarding at which stage the strategic plan
initiatives in the colleges were. Some initiatives were not visible to all staff
members because they were not communicated fully throughout the colleges.
Constant communication will prevent the challenges ensuing from unclear

strategy. Added to the flawed strategic thinking, lack of top management support
in the university initiatives, and unclear delineation of planning and executing the
strategic goals hamper implementation. A perception exists that there are too
many initiatives ensuing from different service units, re-allocation of resources,
88
and a lack of understanding of the organizational structure and middle

management role in the strategic plan.
5.2 Recommendations for future studies

The current study used a purposive non-probability sample of middle
management in specific colleges in a higher education institution. For future
studies, it will be beneficial to use a mixed-methods approach. A representative
sample could examine the role of middle management in other HEIs. There are
several focused areas that this study did not address, and they include strategy
development in educational institutions and sense-making as a concept. Future
studies could be conducted at different universities in South Africa to investigate
the main drivers of the strategy and the degree to which middle management is
involved in driving the strategic plan.
It will also be valuable to research the importance of leadership visibility or

presence during strategy execution. Linked to this is the need to examine how the
various stakeholders (academics, administrators, and leadership) should work
together during strategy formulation and execution. Finally, it will be worthwhile
to research how organizations process the management of knowledge or practices
to sustain a competitive advantage during the COVID-19 pandemic and other
natural disasters (Mahdi & Nassar, 2021).
6. References
Adobor, H. (2019). Opening up strategy formulation: Benefits, risks, and some
suggestions. Business Horizons, 62(3), 383-393.
Ahmed, J. U., Ahmed, K. U., Shimul, M. A. S. & Zuñiga, R. (2015). Managing Strategies for
Higher Education Institutions in the UK:An Overview. Higher Education for the
Future, 2(1), 32-48.
Badat, S. (2017). The Idea of Higher Education as an Instrument for Social Mobility and
Societal Transformation: A Critique of Nelson Mandela. In Nelson Mandela (pp.
125-136). Brill Sense.
Burgelman, R. A., Floyd, S. W., Laamanen, T., Mantere, S., Vaara, E. & Whittington, R.
(2018). Strategy processes and practices: Dialogues and intersections. Strategic
Management Journal, 39(3), 531-558.
Davies, & Davies, B. (2004). Strategic leadership. School Leadership & Management, 24(1),
29-38. https://doi.org/10.1080/1363243042000172804
Hambrick. (2007). Upper Echelons Theory: An Update. Academy of Management Review,
32(2), 334-343.
Hambrick., & Lovelace. (2018). The Role of Executive Symbolism in Advancing New
Strategic Themes in Organizations: A Social Influence Perspective. Academy of
Management Review, 43(1), 110-131.
John, A., Qadeer, F., Shahzadi, G., & Jia, F. (2019). Getting paid to be good: How and when
employees respond to corporate social responsibility? Journal of Cleaner Production,
215, 784-795.
Kazmi, S. A. Z., & Naaranoja, M. (2015). Cultivating strategic thinking in organizational
leaders by designing supportive work environment! Procedia-Social and Behavioral
Sciences, 181, 43-52.
Kemp, N. (2021). Toward an Ontology of Strategy in an Enterprise Context. International
Journal of Business Strategy and Automation (IJBSA), 2(4), 1-23.
89
Li, P.-Y. (2018). Top management team characteristics and firm internationalization: The
moderating role of the size of middle managers. International Business Review,
27(1), 125-138.
Logemann, M., Piekkari, R., & Cornelissen, J. (2019). The sense of it all: Framing and
narratives in sensegiving about a strategic change. Long Range Planning, 52(5),
101852.
Mahdi, O. R., & Nassar, I. A. (2021). The business model of sustainable competitive
advantage through strategic leadership capabilities and knowledge management
processes to overcome Covid-19 pandemic. Sustainability, 13(17), 9891.
Mair, J., & Thurner, C. (2008). Going global: how middle managers approach the process
in medium-sized firms [Article]. Strategic Change, 17(3/4), 83-99.
McKiernan, P., Ackermann, F., & Eden, C. (2018). Stakeholders in strategy:
their'invisible'and yet an inevitable presence? EURAM 2018: 18th Annual
Conference of the European Academy of Management,
McKnight, R., & Hawkrigg, J. (2005). Leading the human side of change. Canadian HR
Reporter, 18(21), 17-19.
McTavish, D. (2006). Further Education Management Strategy and Policy:Institutional
and Public Management Dimensions. Educational Management Administration &
Leadership, 34(3), 411-428.
http://journals.sagepub.com/doi/abs/10.1177/1741143206065273
Mirbabaie, M., Bunker, D., Stieglitz, S., Marx, J., & Ehnis, C. (2020). Social media in times
of crisis: Learning from Hurricane Harvey for the coronavirus disease 2019
pandemic response. Journal of Information Technology, 35(3), 195-213.
Mzangwa, S. T. (2019). The effects of higher education policy on transformation in post-
apartheid South Africa. Cogent Education, 6(1), 1592737.
Neilson, G. L., Martin, K. L., & Powers, E. (2008). The secrets to successful strategy
execution. Harvard business review, 86(6).
Netz, J., Svensson, M., & Brundin, E. (2019). Business disruptions and affective reactions:
A strategy-as-practice perspective on fast strategic decision making. Long Range
Planning, 101910.
Nurlansa, O., & Jati, H. (2016). Analysis porter’s five forces model on Airbnb. Elinvo
(Electronics, Informatics, and Vocational Education), 1(2), 84-96.
Popescu, F. (2015). South African Globalization Strategies and Higher Education. Procedia
- Social and Behavioral Sciences, 209, 411-418.
Putro, S., Rianto, R., & Wibisana, B. H. (2022). Making Business Policies and Strategies in
the Vuca Era with Technology Development: A Literature Review. International
Journal of Innovative Technologies in Social Science(1 33).
Rengarajan, S., Moser, R., & Narayanamurthy, G. (2021). Strategy tools in dynamic
environments – An expert-panel study. Technological Forecasting and Social Change,
165, 120560.
Rotaru, K., Fehrenbacher, D. D., Liang, M. H., & Schulz, A. K.-D. (2019). Causal Inference
in Judgment Using the Balanced Scorecard. Journal of Management Accounting
Research.
Rouleau, L. (2005). Micro‐practices of strategic sensemaking and sensegiving: How
middle managers interpret and sell change every day. Journal of Management
studies, 42(7), 1413-1441.
Rouleau, L., & Balogun, J. (2011). Middle managers, strategic sensemaking, and discursive
competence. Journal of Management studies, 48(5), 953-983.
Ryan, G. W., & Bernard, H. R. (2003). Techniques to Identify Themes. Field Methods, 15(1),
85-109. http://journals.sagepub.com/doi/abs/10.1177/1525822X02239569
Shaik, A. S., & Dhir, S. (2020). A meta-analytical review of factors affecting the strategic
thinking of an organization..
90
Sutphen, M., Solbrekke, T. D., & Sugrue, C. (2018). Toward articulating an academic praxis
by interrogating university strategic plans. Studies in Higher Education, 1-13.
Tawse, A., & Tabesh, P. (2021). Strategy implementation: A review and an introductory
framework. European Management Journal, 39(1), 22-33.
Thompson, C., & Wolstencroft, P. J. R. i. P.-C. E. (2018). Trust into mistrust: the uncertain
marriage between public and private sector practice for middle managers in
education. Foresight 23(2), 213-230.
Vaismoradi, M., Turunen, H., & Bondas, T. (2013). Content analysis and thematic analysis:
Implications for conducting a qualitative descriptive study. Nursing & Health
Sciences, 15(3), 398-405.
Van Rensburg, M. J., Davis, A., & Venter, P. (2014). Making strategy work: The role of the
middle manager. Journal of Management & Organization, 20(2), 165-186.
Van Schalkwyk, F. B., van Lill, M. H., Cloete, N., & Bailey, T. G. (2022). Transformation
impossible: policy, evidence and change in South African higher education.
Higher Education, 83(3), 613-630. https://doi.org/10.1007/s10734-021-00687-7
Vuori, J. (2016). Towards Strategic Actorhood? The Execution of Institutional Positioning
Strategies at Finnish Universities of Applied Sciences [Article]. Higher Education
Quarterly, 70(4), 400-418.
Vuorinen, T. (2018). Mapping the landscape of strategy tools: A review on strategy tools
published in leading journals within the past 25 years. Long Range Planning, 51(4),
586.
Xing, B. (2019). Towards a Magic Cube Framework in Understanding Higher Education
4.0 Imperative for the Fourth Industrial Revolution. In Handbook of Research on
Challenges and Opportunities in Launching a Technology-Driven International
University (pp. 107-130). IGI Global.
91
Appendix A: Interview questions
1. What is the University’s strategy for the next five years?

2. How informed do you feel about the University's Strategic Goals?
3. Have you attended any of the presentations regarding the Strategic Plan?
What are your thoughts?
4. How does the Strategic Plan fit into your current position in middle
management?
5. Do you feel that leadership of the University communicates enough with
you about your Strategic Plans?
6. How do you engage in the sense-making of the strategic plan?
7. How do you communicate the strategy to employees?
8. What are the challenges that you face with strategy execution?
9. What recommendations can be provided for middle management to
improve the strategy execution process?
10. What are the existing initiatives that are strategically aligned in the
University?
11. Do you think the strategy-aligned initiatives are clear to the Schools in
your College?
12. What is the level of enthusiasm or participation from your staff in strategic
initiatives of the University?
92

Vol. 21, No. 8, pp. 92-108, August 2022
Received Apr 8, 2022; Revised Aug 11, 2022; Accepted Aug 23, 2022
Learners’ Active Engagement in Searching and

Designing Learning Materials through a Hands-
on Instructional Model
Esther S. Kibga*
University of Rwanda-College of Education,
and Science (UR-CE, ACEITLMS), Rwamagana, Rwanda.
Emmanuel Gakuba
University of Rwanda-College of Education (UR-CE), Rwamagana, Rwanda
John Sentongo
Makerere University, Department of Science, Technical and Vocational
Education (DSTVE), Kampala, Uganda
Abstract. When engaging learners in searching and designing, the

learning materials are of paramount importance, in order to help the
learners in the achievement of academic goals and objectives. However,
learners being the primary consumers of chemistry content delivered by
the teachers, ought to beware of the learning materials that can facilitate
their learning. This can well be done if learners participate fully in
searching and designing learning materials that correspond with the
lesson content. The purpose of this research was to use a Hands-on
Instructional Model (HIM) designed to help secondary school teachers
actively to engage learners in searching, designing, and manipulating
locally made learning materials, in order to facilitate the face-to-face
learning of chemistry concepts. The study employed Design-Based
Research (DBR) in designing, refining, and implementing HIM, as well as
the learning materials designed by the learners following a pragmatic
philosophical world-view. The data were collected and analysed by using
qualitative research techniques; and the research instruments included
Lesson-Observation Protocol, semi-structured interviews, and Focus-
Group Discussions. The study involved three intact senior-science classes
deliberately selected from three Dar es Salaam community-secondary
schools. The results revealed that learners become actively engaged in the
lesson, when the learning materials designed by themselves are used.
They used materials, like empty water bottles of different sizes, syringes
*
Corresponding author: Esther Samwel Kibga, jeyden.janell@gmail.com
©Authors
This work is licensed under a Creative Commons Attribution-Non-Commercial-Non-Derivatives
93
of different sizes, rubber bands, and pegs to prepare locally made

apparatus that served as beakers, burettes, droppers, and funnels in
titration hands-on activities. In this regard, we recommend teachers
involve learners in searching and designing the learning materials to be
used in the teaching and learning process, in order to enhance chemistry
content mastery and the acquisition of soft learning skills.
Keywords: community secondary schools; hands-on activities; learners’

active engagement; learning materials
1. Introduction
Learning materials have been used in the chemistry classrooms since ancient
times. Learning materials are the tools that can be used by instructors and teachers
within the classroom context, in order to facilitate the learning and understanding
of concepts among learners (Arop et al., 2019; Khalil & Elkhider, 2016; Wang,
2021). These materials are mostly used to support teachers in achieving the
objectives that are set for a particular lesson (Choppin et al., 2020; Kanellopoulou
& Darra, 2018).
According to Hayat et al. (2017), these materials make learning real, enjoyable,
practical, and pleasurable for the learners. Also, learning materials facilitate the
illustration and reinforce the acquisition of skills, viewpoint, perspective, and
ideas (Arop et al., 2019). Furthermore, research has indicated that a large number
of learners are not interested in perusing science subjects at the secondary school
level (Nbina & Mmaduka, 2014; O-saki, 2007). One of the noted causes is the
inappropriate and insufficient learning materials used to facilitate the learning of
these concepts.
This may result in the inability to understand these concepts (Ko et al., 2013).
Notwithstanding the need for learning materials, Mafumiko (2006), pointed out
that the Tanzanian government supplies limited instructional resources to
community secondary schools; and as such, there cannot be enough for all the
learners in all the schools (Machumu, 2011; Nbina & Mmaduka, 2014). Therefore,
there is a need to involve learners in designing locally made learning materials
that are appropriate for teaching and learning chemistry. This would go a long
way to improve the quality of chemistry teaching, as stipulated in SDG4 (Jackson
et al., 2013).
1.1 The Literature Review

Successful implementation of the chemistry curriculum is dependent on the
learning materials available for both teachers and learners in lesson sessions.
According to Khalil and Elkhider (2016), the term learning material means all
theoretical, practical and skill-oriented resources, which are accessible and
available to facilitate the learning acquisition of various learning skills. In
addition, learning materials bring the hope of delivering educational facts and
experiences vividly and widely with realism that the printed media could hardly
achieve (Arop et al., 2019).
94
For instance, a chemistry teacher can barely explain and describe a pipette, burette
or any other learning equipment in chemistry; but it is hard to tell the learners
what exactly a pipette or a burette look like, without a picture or physical
equipment for clarity (Choppin et al., 2020). The picture of a pipette or burette is
considered a learning material that would help the students to comprehend the
concepts behind their use. Also, these materials are among the materials needed
by teachers and other instructors to assess the knowledge acquired by their
learners from the lesson (Ko et al., 2013). Therefore, learning materials have a vital
impact on the learning process and the acquisition of various skills.
Research in chemistry education has indicated that the introduction of modern

and innovative methods and teaching-learning have led to developments in the
overall system of education (Aydin-Günbatar & Demirdöğen, 2017; Stammes et
al., 2020). However, it is assumed that using the hands-on strategy motivates
learners’ active engagement in the lesson by making learning a more realistic and
exciting experience. Working through hands-on in line with a popular proverb,
which states ‘I hear, I forget’; ‘I see, I remember and ‘I do, I understand’. Besides,
Cirenza et al. (2018) and Holstermann et al. (2010) identified experimentation, the
manipulation of symbols and objects, as well as learners’ interaction, as being
among the learners’ hands-on activities that can assist in the learning of chemistry.
According to them, the respective hands-on activities reflect on how they can
enhance learning. Schwichow et al. (2016) further highlighted that learning can be
accomplished through a careful and thoughtful selection of appropriate teaching
strategies that would help in promoting students’ ability to create the scientific
and mathematical meaning of concepts rather than the passive reception of ideas.
Therefore, the learning of the subject matter can be strengthened and emphasized
when a learner experiences a learning activity as enjoyable, pleasant, stimulating,
and relevant through the design process (Stammes et al., 2020).
Searching and designing learning materials increases learners' knowledge,

thereby enabling them to learn how to think scientifically and understand how
scientists work in natural life (Alkan, 2019). According to Ibe et al. (2021),
deliberate practice and the continuous engagement of chemistry students in the
design process are needed, so that learners become familiar with the content.
According to Valdez et al. (2015), these practices are more effective when a learner
is in an interactive environment and critically analyses the problem. An
interactive classroom environment gives room for learners to exercise their ideas,
knowledge and competencies (Holstermann et al., 2010; Wood, 2006). Wood
(2006) asserts that group discussions create an active learning environment that
improves students’ ability to work and communicate with others, as well as to
develop awareness and control of their thinking. In addition, (Holstermann et al.,
2010) highlighted that when learners work in groups, it is possible to predict
various solutions for a given task; since they share experiences.
The fact that learners learn through their experiences is not new. John Dewey
(1859–1952) in his pragmatic theory of education posited that the experiences
brought by learners in a classroom setting, from the outside environment, have a
95
great impact on their learning (Sikandar, 2016). For Dewey, the generation of
knowledge takes place in real and meaningful situations, through the
spontaneous activities done by learners (Rizk, 2011). Besides, Levy Vygotsky
(1920s) in the socio-cultural theory stated that learners' experiences are sharpened
in the Zone of Proximal Development (ZPD), in the presence of a mentor (Fani
and Ghaemi, 2011; Lui, 2012).
The major idea is that learners learn best when working together with others in
collaboration; and it is by such shared endeavours with more experienced persons
that learners learn and internalize new concepts, and skills (Fani and Ghaemi,
2011; Glassman, 2001; Lui, 2012).
1.2 The statement of the Problem

Chemistry, as a subject, is a real-life science subject, based on the concepts that
comprise it. By its nature, most concepts in chemistry are practically oriented and
its teaching and learning really require the use of teaching and learning materials.
However, the literature has shown that teachers have been depending on the
excessive use of words to express and convey chemical ideas (Stammes et al., 2020;
Udogu & Enukora, 2017), theories, principles and fact-related skills and
competencies to learners during teaching, which is completely teacher-centred via
the lecture method (Sevian & Talanquer, 2014).
Additionally, teachers have played a role in ensuring that learning materials are
available in chemistry lessons, although not enough for all the learners, especially
in classrooms with a large number of learners. This method of teaching denies
learners their active engagement in learning chemistry, which makes some
students consider it as a white-man's "magic' (Udogu & Enukora, 2017), yet
chemistry is the science that they experience in their everyday life. Also, the
majority of individual learners in chemistry lessons only end up observing the
learning materials, but not by using them to enhance their learning of various
chemistry topics.
Nevertheless, the Tanzanian competence-based curriculum emphasizes the

involvement of learners in practical exercises during science teaching and
learning, by using various kinds of material resources. But researchers like (Nbina
& Mmaduka, 2014), have reported that there are inadequate materials for teaching
chemistry in schools. The above assertion prompted us to acknowledge the need
to try out the use of locally made learning materials during the teaching of
concepts. The problem of this work is to find out whether secondary school
learners can search and design learning materials by using the materials available
in the environment, in order to enhance their active engagement in hands-on
activities during chemistry lessons.
1.3 The Research Questions

This research project answered the following questions:
1. How do learners engage s in searching and designing locally available
learning materials through a Hands-on Instructional model?
96
2. How do learners engage themselves in chemistry hands-on activities by

using learning materials designed from locally available learning
materials?
2. The Methodology
2.1 The Research Design
In this study, a Hands-on Instructional Model (HIM) was designed through
Design-Based Research DBR) when following a pragmatic philosophical view. A
HIM prototype designed in this study guided chemistry teachers to lead students
in designing locally made instructional materials that were used in the hands-on
activities of chemistry lessons. However, the four DBR stages suggested by Reeves
(2000), which combine research, design, and practice (Bowler & Large, 2008) were
preferred.
Bowler and Large (2008) highlighted that Design-Based Research holds promise,
as a research design that can bridge the theory/practice gap in the real educational
world. The four steps helped to connect the learners’ class activities and the use
of the designed materials, in order to meet the chemistry content outcomes.
2.2 The Research Site and Sample

This study was performed in three community secondary schools in Dar es
salaam, Tanzania. The sample for this study involved students’ purposely
selected from three intact science classes from the selected community-secondary,
schools. Furthermore, the names of the students were not used in the data
analysis, but rather they were identified by using pseudonyms. The
implementation of this study was facilitated by three teachers (two females and
one male), with an average teaching experience of five years.
2.3 Research Instruments and Data-Collection Procedures

This research was conducted from March to June 2019; and it employed a
qualitative research approach. The research instruments for the data-collection
process included Lesson-Observation Protocol (LOP), Focus-Group Discussions
(FGDs) as guides for students (Appendix A), and semi-structured interviews as a
guide for teachers (Appendix B).
A total of 42 face-to-face lessons (14 lessons in each school) were observed in all
three schools. The researchers acted as non-participatory observers, in order to
avoid influencing the process of data collection in the course of the lesson-
observation. Also, during each lesson observed, we identified the resources used
by the teachers; and we observed the learners, as they were interacting with the
instructional materials locally made by using the materials from the home
environment during hands-on activities.
Furthermore, the FGDs in each school were conducted once a week after lesson
observation; and a total of 21 FGD interviews were conducted in all three schools,
seven per school. Each FGD comprised six students; and it lasted for
approximately 30-45 minutes on average; and this time was considered sufficient
to reduce any initial anxiety. In addition, a total of 21 interviews with the teacher
97
were guided by the interview guide; and they were conducted, corresponding to
seven interviews per school.
All the teacher interviews were conducted after every observation of a lesson, in
order to evaluate the lesson and the trend of the designed intervention. All FGDs
and the interviews were audio-recorded; and this helped the researchers to elicit
both students' and teachers’ experiences during the chemistry lessons.
2.5 The Data Analysis

The credibility of all the instruments was checked by two experienced science
educators, in order to ascertain their ability to produce credible outcomes and
their inter-rater reliability was established. To establish the inter-rater reliability,
the external researchers were ‘raters’ that were familiar with qualitative research.
They rated the instruments and recommended some changes. Their
recommendations were effected after reaching a consensus before data the
collection. Furthermore, the conformability of the information obtained was
observed through member checking (Basit, 2003; Yin, 2009), in order to ensure that
all the information obtained was based on the participants’ responses.
Furthermore, audit trials were done throughout the data-analysis process, in

order to ensure that the study’s findings portray accurately the respondents'
views (Yin, 2009). Also, triangulation of the research information obtained by
using different instruments (Cohen et al., 2007; Creswell, 2014; Mertens, 2010; Yin,
2009) was done, to ensure the credibility and the ,accuracy of the research
findings.
The analysis of the data collected in this study was done concurrently with the
data-collection process (Creswell, 2014) daily. Constant reflection on the
information obtained from the interviews, the FGDs, and lesson observations
were done to monitor the ongoing process of data collection and to identify those
issues that needed clarity and follow-up during the intervention process. Then,
the analysis was performed thematically (Braun et al., 2016) in which the whole
process began by transcription of the audio data, translation of some transcripts
and field notes from Swahili to the English language, as well as organization of
all the data, according to their types, thereby forming a database for the inductive-
coding process (Yin, 2009).
Generally, the entire coding was done by one of the researchers, and all the codes
and the themes were assessed independently by two raters, who were part of the
research team. The coded information was sorted and sifted through, in order to
identify similar and coherent phrases (Braun et al., 2016), as well as the
relationships between variables and patterns, in order to differentiate distinct and
common sequences of categories in line with the research question (Basit, 2003;
Baxter & Jack, 2008). Also, triangulation of information from lesson observations,
teacher interviews, and students' FGDs was done, in order to capture the different
dimensions of the same theme (Braun et al., 2016) and to minimize the researchers’
biases. Lastly, meaningful information that gave a better interpretation of the data
about the research questions was obtained from the developed themes and sub-
themes (Baxter & Jack, 2008).
98
3. The Results
The analysis of the data from students’ FGDs, observation protocol, and teachers’
interview transcripts resulted in the generation of three themes for better
presentation and interpretation of the gathered information. These themes
included the lesson plan and presentation, the search and the design. As
mentioned earlier, the questions were asked of both the teachers and the students;
these focused on the active engagement strategies and resources, rather than on
the chemistry content.
Theme 1: The Lesson plan and the Presentation thereof

All the lessons were planned and presented, according to a competency-based
framework from the Ministry of Education and Vocational Training (MoEVT).
The teachers prepared a written lesson plan for each lesson that was to be
observed. Additionally, the learning objectives (success criteria and the learning
intentions) for each lesson were well stated in the lesson plan; and these were
shared with students before and after every lesson. The sharing of the success
criteria and the learning intentions was seldom done by the teachers before the
beginning of the study.
The observations done at the beginning of the teaching and learning process using
the designed intervention revealed that the teachers mostly preferred the lecture
method. For example, it was observed that teacher A was largely using a textbook
to teach; while teachers B and C used notes, which are not currently available.
When they come to chemistry lessons they mostly talk, write on the board, and
rarely involve any of the students.
“……Surely with this number of students, I just use my Oxford textbook to prepare notes
to at least cover the content in the allocated time. The preparation of the materials is a
challenge for some reasons. …..Firstly, it consumes a lot of time, which could have been
that of used to teach ….therefore, with this number of students in the class, the preparation
of the learning materials is another disturbance. ….”again we have too much to cover”
(Interview, Teacher C).
“….Frankly, I cannot pretend that it is not hard for us to prepare the lesson notes with
their corresponding learning materials; since we have too much to cover. To do all those
activities requires one to search from various sources, in order to understand those
materials that correspond with the lesson content” (Interview, Teacher B).
The students were only involved when they were supposed to answer questions,
such as “Are we together? Understood? Is it clear? Can I carry on? However, the
answers given by the students were also general, which could not be enough to
verify whether the concept was clear. However, the teacher could carry on with
the lesson. The students in FGDs expressed a similar view, as the statements
below demonstrate.
Carol: “Our teacher normally teaches and gives us notes to write”.
Jeff: “The learning style you introduced is new; I can say we are not familiar with it”.
Halima: ”We are many in the class, so we cannot be involved in classroom activities”.
Khauthal: “Our teacher normally talks and writes notes on the blackboard for us to
copy…..It is very rare to be asked questions, or for us to get involved in discussions among
ourselves.”
99
However, with time, interactive teaching methods like group discussions, paired
with some hands-on activities, were adopted. Teachers allocated students in
respective groups before the lessons based on students' needs, and abilities, as
well as ensured gender balance. During the group discussions, students were
observed to be able to design various learning materials and solve some problems
in the given activities of the lessons within their groups; and they were sometimes
told to answer directly, or to make attempts on the blackboard.
“….first allocating students in groups makes them come together, and to combine their
ideas to work for the materials and attempt various working activities that are given to
them. These days, I don’t use much energy to engage the students in the lesson. Together
in their groups, they prepare and present some concepts related to these topics that you
have put more emphasis on in your work. In their groups, they co-operated and used the
learning materials they design to ensure the questions given to them are well prepared and
ready for presentation to the whole class.” (Interview, Teacher C)
Also, students were able to freely move around to check on the materials prepared
by other peers and how they were able to implement the prepared equipment.
Besides, individual students could collaborate with other group members to
ensure that the equipment is designed, based on the learning intentions and the
objectives of a particular lesson.
Theme 2: Search
The learning intentions and the objectives for a lesson were always shared by the
teachers before and after the lessons, in order to give students the prerequisite
knowledge of content to be learnt in the coming lesson; and to be able to associate
the content with the materials in their environment that could enhance their
learning. The students prepared themselves for the next lesson by going through
what was to be taught and searching for the instructional materials that
corresponded with the specific chemistry content. However, as time went by
during the intervention, both the teachers and the students seemed to enjoy the
use of hands-on activities; because they both realized that the teaching strategy
was possible to implement; and it was fruitful. This means that there was a
gradual paradigm shift; from being more teacher-centred to becoming learner-
centred; and the students had great joy while searching for their instructional
learning materials.
This matter is amplified in the teachers’ and students’ statements below.

“After the guidance given to my class during the orientation, searching for materials
related to what is supposed to be learned has been engaging, especially when students are
aware of what is going to be learned in the next lesson”(Interview, Teacher C).
“I have learnt that it is important for the materials used in teaching and learning to be
interesting and to motivates learning in classroom sessions; and that it can continuously
be used by students outside the classroom” (Interview, Teacher B).
“…if the objectives of the lessons are shared with the students, they normally take their
time to search for appropriate learning materials….You see these days, the syllabus is
100
available to the students; and they always know what is going to be taught in advance.
Therefore, being familiar with the content, designing of materials does not become an issue
to the students.” (interview, Teacher A)
To complement the two statements given by the teachers, Jeff and Vanesa (not real
names) in different FGDs sessions said:
Jeff: “….Normally at the end of each lesson, our teacher informs us about what is going
to be learned in the coming chemistry lesson…. By doing that, we can easily look for
materials that would facilitate active participation in the lesson and co-operation in our
specific groups.”
Vanesa: “…..Interestingly, these days you can pace on your own and interact with the
content, while thinking of the materials that correspond with it….. Frankly, these days it
is not a hustle to understand what the teacher teaches…..because when the teacher comes
to the class, it becomes like repetition…… this kind of repetition strengthens our
understanding.”
Theme 3: The design

The lesson observations revealed that different materials obtained from the
learners’ environment were partly used to engage learners in mole-concept
lessons and to design/prepare various volumetric analytical apparatus. It was
also realized that when students get the proper teacher guidance, they fully
engage themselves in hands-on activities, as well as searching for quality learning
materials.
The teachers said in the interview:

“…..imagine what happens when students, who are the major concern of the learning
processs, are positioned in a role of a manufacturer…..this time manufacturing equipment
that is going to be used for their learning. You would find them concentrating and they
would dedicate much of their efforts to designing neat equipment. ….for instance, when
you assign them to make a pipette from fabricated materials, students would do their best
effort to make a well-calibrated apparatus.” (Interview, Teacher A)
“…when students are given a task to design learning materials, they take time to think,
read, explore and come out with materials appropriate for their learning. Sometimes, the
task of designing the materials is taken as competition between themselves, whereby each
group desires to design better and more durable materials than any other group.”
(Interview, Teacher B)
“……Well, if students are engaged in various activities in the lesson, they feel
belongingness; and they tend to own the learning process, so, yeah, they seem to enjoy the
making of learning materials; and they can show that they’ve participated in the process
by looking at how they practically use those materials in hands-on activities.” (Interview,
Teacher C)
The students managed to make some apparatus on their own by using home-
based materials. They used materials like empty waterbottles of different sizes,
syringes of different sizes, rubber bands, and pegs to prepare the locally made
apparatus used during volumetric analytical lessons. Some of the apparatus
101
served as beakers, burettes, droppers, and funnels. Some of these materials are
published in (https://doi.org/10.29333/ejmste/10856 and
https://doi.org/10.36681/tused.2021.93).
With the appropriate apparatus made by the students, teachers prepared

solutions for simple titration experiments, which students used to perform the
experiments in their respective groups. This was directly expressed in the FGDs
excerpts as follows:
Katoto: “….I enjoy using materials, which I had used my own ideas to make.”
Marina: “…if the system like this continues, I think we will be able to design even complex
equipment that could not be imagined, simply by using the materials we see around us.”
Niki: “….these days I ask my father a lot of questions, if I fail in some steps or I don’t get
an idea that could guide me throughout the process. This is because I want to design good
materials. Sometimes I ask my eldest sister; or I search through the internet until I succeed
in designing the materials I want.”
Lincoln: I managed to make a burette, simply by using a transparent pipe, rubber, the
tube of a pen and a clothes’ peg. I only faced some challenges in labelling the exact volume.
But when I came to school I used the pipette in the laboratory to calibrate the volume.”
Considering the responses from both teachers and students, as well as the
observations made in different lessons, it is clear that when students are well
guided to engage in hands-on activities, they can own the learning process and
help each other throughout the lesson. Not only so, but they have also participated
in lesson preparation by searching and designing learning materials.
4. Discussion
This paper gives evidence that it is possible to successfully engage students in the
chemistry-lesson process by involving them in searching and designing the
appropriate learning materials, in order to render the learning process active and
engaging. Engaging students in searching for appropriate materials relevant to a
particular subject matter makes the process of learning continuous. Teaching
chemistry content should not only be the priority of learning; but chemistry
teachers can value the design of learning materials as a way to present chemistry
content knowledge to their students (Stammes et al., 2020).
Students can continue being in a learning mood outside the classroom

environment, especially when they associate different materials in their
immediate environment with the subject of interest. Searching and designing
learning materials is a very useful and important aspect of the learning process,
despite this not being included in the learning standards (Vos et al., 2010). For
instance, thinking of what materials can be used to make apparatus like conical
flasks, pipettes, burettes, and measuring cylinders remains a challenge in
students’ minds all the time, until they successfully make such apparatus. In the
context of this study, therefore, designing learning materials was considered to be
a part of the learning process. (Stammes et al., 2020)
102
Students who participated in the study had no previous experience in searching

and designing instructional materials. Teachers used to design only a few learning
materials that were only used for demonstration of the lesson content. The few
learning materials designed by teachers could not be enough for all the students
in one classroom session. This was evident at the beginning of the intervention. In
the interviews and FGD conducted in the first week of the intervention, both the
students and the teachers reported how, initially, they saw the intervention as an
inconvenience, interference, and time-consuming exercise. As the intervention
progressed, both the teachers and the students became accustomed to the
intervention; and eventually, they enjoyed the lessons.
However, researchers recommend the use of design to help students develop an

understanding of any new concepts (Alkan, 2019). This has led to some countries
dedicating effort to the design of learning materials, in order to stimulate context-
based chemistry education (Prins et al., 2018). During the intervention, students
managed to prepare the apparatus used for titration procedures by using
materials from their immediate environment. The locally made apparatus was a
solution to the scarcity of laboratory equipment, which hindered students’
frequent experimentation (Galabawa, 2008; Machumu, 2011).
Previously, the available equipment was reserved for examination classes

(Mafumiko, 2006). Therefore, with the locally made apparatus, titration activities
were conducted practically, thereby making the process of learning a reality.
The findings from the lesson observation and FGDs indicated some improvement
in the learners’ active engagement in the lesson, whereby learners seemed to be
more active a few weeks after the intervention than at the onset of the
intervention. Teachers should persist in traditional views of chemistry education
concerning students’ chemistry-content knowledge (Stammes et al., 2020), rather
however. they should take a more contemporary perspective by valuing design
as an approach to address soft skills like curiosity, creativity, meta-cognition and
problem-solving skills (Ibe et al., 2021).
This is also consistent with previous research reports (by (Freeman et al., 2014;
Jensen & Lawson, 2011; Prince, 2004). The research report by Freeman et al. (2014),
indicates that the instructional method had a minute effect on the learners'
achievement when measured by using Bloom's taxonomy. Some effects were
detected on learners’ active learning, for instance, students gained the ability to
learn independently, which was minimal before the intervention. Besides, it is
clearly stated in ‘pragmatism’ by Dewey that the ability of the individual student
is strengthened through shared experiences (Sikandar, 2016).
Engaging students in hands-on activities enhances students’ active involvement

in chemistry lessons at the individual level.
Also, with time, students gained the ability to search for learning materials, and
to actively use them for learning the respective content during chemistry lessons.
In addition to acquiring skills to search for materials and active engagement in the
103
lesson, the intervention enabled students to improve their reasoning and ability
to respond to given activities with confidence, as proposed by Khoiriyah et al.,
2015; Pirttimaa et al., 2017. Thus, it can be noted from the findings that the
intervention contributed to students’ reasoning, exploratory ability and active
engagement in-class activities.
Although all the teachers went through a similar orientation and the students
from the participated classes went through training before the commencement of
the intervention, the students from school C seemed to grasp the instructions
given by the teacher more quickly than the students from the other two schools.
As Prins et al. (2018) found, we saw some variation in the learning materials
designed by the students. Perhaps, teacher guidance (Cirenza et al., 2018) might
have been the cause of the difference in the quality of learning materials made by
students from school C. Also, students’ active participation in school C was
possibly due to close mentorship and scaffolding of the teacher, as suggested in
Vygotsky’s ZPD concept of socio-cultural theory (Fani & Ghaemi, 2011).
Proper guidance of the teacher enhances students’ collaboration with other peers
and better interaction with learning materials (Fani & Ghaemi, 2011; Glassman,
2001; Lui, 2012). To the teachers, design and the searching of learning materials
seemed to mean having the opportunity to address mentorship and scaffolding in
chemistry education which highly motivated them to include design practices in
their teaching (Stammes et al., 2020). Additionally, students can perform above
their current level of knowledge, while collaborating with other peers of higher
ability, rather than when they work independently (Fani & Ghaemi, 2011). Indeed,
active learning takes place and less memorization is expected, when students
collaboratively learn together with others (Jensen & Lawson, 2011). According to
these authors, active learning is due to the helping behaviour to effect that which
occurs within the groups. This finding is in line with the findings of this study,
where the peers in the groups helped each other with the tasks done within the
groups and showed less memorization.
Given the above differing empirical results and the theoretical perspectives
employed in this work, we evaluated the success of the intervention in the use of
home-based learning materials designed by students to be used in hands-on
activities during chemistry lessons, to have an impact on the development of
learners’ active learning and their acquisition of soft skills. Furthermore, the
findings of this study contribute to the improvement of educational practices,
instructional designs, and other related literature in chemistry education.
Little work is evident on active engagement in searching and designing chemistry

learning materials in the context of community schools. Therefore, this work adds
to the literature to improve educational practice, as well as the need to engage
students in activities, consequently putting them at the centre of the learning
process.
Regarding the limitations of this research, the learning materials developed for
this study and the content thereby facilitated, were based on two topics in the
104
Chemistry senior three content syllabus. This is because an in-depth

understanding of the association between the learning materials and the content
was required. The iterative nature of the DBR design used in the study also
required developing prototypes of the same materials. Furthermore, since the
study allowed successive reviews of the materials, it was possible to identify and
overcome weaknesses in the design of the materials.
If the procedures to make these materials are clearly outlined, it would be easy to
design more materials from other topics in the Chemistry syllabus and other
science subjects. Additionally,, the study covered only hands-on activities in
Chemistry, particularly the learners’ ability to search and design learning
materials. This is because many Chemistry concepts can be demonstrated through
hands-on activities, as well as both practical and theoretical concepts.
Additionally, the study was limited to only three community secondary schools,
in order to establish a deep understanding of the identified problem and to ensure
that the research participants became accustomed to the designed intervention.
In this regard, future researchers should increase the population of the sample to
improve the reliability and the generalizability of the research results. Thus,
triangulation of multiple methods were utilised, in order to minimize the
mentioned limitations.
5. Conclusion
With this study, a body of knowledge on searching and designing learning
materials in chemistry has been built; because the learning materials designed
were useful for chemistry learning in community schools. This study was carried
out to enhance senior three chemistry students' ability to search for and design
chemistry learning materials that could be used in hands-on activities, to
stimulate students’ active engagement in the learning process.
What differentiates the present study from others in the same field is the fact that
the study focuses on students’ ability to work independently to make learning
materials from home-based materials. Other studies mostly engage teachers to
enhance the learning process; and they rarely emphasize the need for the learners
to work on their learning materials, especially in the context of community
secondary schools.
From the results, we can conclude that engaging students in the process of
searching and designing the learning materials to be used in chemistry lessons
enhances learning to become a continuous process from the classroom to the
outside environment; and it prepares the students for lifelong learning. Based on
the key findings obtained from this research, it is recommended that more
emphasis should be placed on learners’ abilities to search and design the learning
materials – not only in chemistry – but also in other science subjects of a similar
educational context to the research participants, as those of this study.
The study brought about a paradigm shift from teachers being the centre of all
aspects of the learning process, by empowering learners to take control of their
learning and to reduce the workload of the teachers. Consequently, making
105
learners the centre of the learning process can create a meaningful learning
environment which in turn would enhance the nurturing of a generation of
independent learners, equipped with skills essential for the fast-growing 21st-
century world economy.
6. References
Alkan, F. (2019). Examining the instructional materials' motivation of prospective
chemistry teachers in the laboratory. SHS Web of Conferences, 66, 01006.
https://doi.org/10.1051/shsconf/20196601006
Arop, B. A., Umanah, F. I., & Effiong, O. E. (2019). Effect of instructional materials on the
teaching and learning of basic science in junior secondary schools in Cross River
State, Nigeria. Global Journal of Educational Research, 14(1), 67.
https://doi.org/10.4314/gjedr.v14i1.9
Aydin-Günbatar, S., & Demirdöğen, B. (2017). Chemistry Teaching Method Course for
Secondary-Science Teacher Training. In A. J. Sickel & S. B. Witzig (Eds.), Designing
and Teaching the Secondary-Science Methods Course (pp. 129–148). Sense Publishers.
https://doi.org/10.1007/978-94-6300-881-5_8
Basit, T. (2003). Manual or electronic? The role of coding in qualitative data analysis.
Educational Research, 45(2), 143–154.
Baxter, P., & Jack, S. (2008). Qualitative case study methodology: Study design and
implementation for novice researchers. The Qualitative Report, 13(4), 544–559.
Bowler, L., & Large, A. (2008). Design-based research for LIS. Library & Information Science
Research, 30(1), 39–46.
Braun, V., Clarke, V., & Weate, P. (2016). Using thematic analysis in sport and exercise
research. In Routledge handbook of qualitative research in sport and exercise (pp. 213–
227). Routledge.
Choppin, J., Roth McDuffie, A., Drake, C., & Davis, J. (2020). The role of instructional
materials in the relationship between the official curriculum and the enacted
curriculum. Mathematical Thinking and Learning, 1–26.
https://doi.org/10.1080/10986065.2020.1855376
Cirenza, C. F., Diller, T. E., & Williams, C. B. (2018). Hands-On Workshops to Assist in
Students’ Conceptual Understanding of Heat Transfer. Journal of Heat Transfer,
140(9).
Cohen, L., Manion, L., & Morrison, K. (2007). Research methods in education (6th edn).
Routledge.
Creswell, J. W. (2008). Educational research: Planning, conducting and evaluating
quantitative and qualitative research. Upper Saddle River, NJ: Merril. Creswell,
JW (2009). Research Design. Qualitative, and Mixed-Methods Approaches, 570–590.
Creswell, J. W. (2014a). Research design: Qualitative, quantitative, and mixed-methods
approaches (4th ed). SAGE Publications.
Creswell, J. W. (2014b). Research design: Qualitative, quantitative, and mixed-methods
approaches (4th edn). SAGE Publications.
Fani, T., & Ghaemi, F. (2011). Implications of Vygotsky’s zone of proximal development
(ZPD) in teacher education: ZPTD and self-scaffolding. Procedia-Social and
Behavioral Sciences, 29, 1549–1554.
Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., &
Wenderoth, M. P. (2014). Active learning increases student performance in
science, engineering, and mathematics. Proceedings of the National Academy of
Sciences, 111(23), 8410–8415. https://doi.org/10.1073/pnas.1319030111
106
Glassman, M. (2001). Dewey and Vygotsky: Society, experience, and inquiry in

educational practice. Educational Researcher, 30(4), 3–14.
Hayat, M., Hasan, R., Ali, S. I., & Kaleem, M. (2017). Active learning and student
engagement using Activity-Based Learning. 2017 International Conference on
Infocom Technologies and Unmanned Systems (Trends and Future Directions) (ICTUS),
201–204.
Holstermann, N., Grube, D., & Bögeholz, S. (2010). Hands-on activities and their influence
on students’ interest. Research in Science Education, 40(5), 743–757.
Ibe, N., Obikezie, M., Awka, & Chikendu, R. (2021). Effect of Improvised Instructional
Materials on Chemistry Students’ Academic Retention in Secondary School.
International Journal of Research in Education and Sustainable Development, 19–31.
https://doi.org/10.46654/IJRESD.1520
Jackson, K., Garrison, A., Wilson, J., Gibbons, L., & Shahan, E. (2013). Exploring
Relationships between Setting up Complex Tasks and Opportunities to Learn in
Concluding Whole-Class Discussions in Middle-Grades Mathematics Instruction.
Journal for Research in Mathematics Education, 44(4), 646–682.
https://doi.org/10.5951/jresematheduc.44.4.0646
Jensen, J. L., & Lawson, A. (2011). Effects of Collaborative Group Composition and Inquiry
Instruction on Reasoning Gains and Achievement in Undergraduate Biology.
CBE—Life Sciences Education, 10(1), 64–73. https://doi.org/10.1187/cbe.10-07-
0089
Kanellopoulou, E.-M., & Darra, M. (2018). The Planning of Teaching in the Context of
Lesson Study: Research Findings. International Education Studies, 11(2), 67.
https://doi.org/10.5539/ies.v11n2p67
Khalil, M. K., & Elkhider, I. A. (2016). Applying learning theories and instructional design
models for effective instruction. Advances in Physiology Education, 40(2), 147–156.
https://doi.org/10.1152/advan.00138.2015
Khoiriyah, U., Roberts, C., Jorm, C., & Van der Vleuten, C. P. M. (2015). Enhancing
students’ learning in problem-based learning: Validation of a self-assessment
scale for active learning and critical thinking. BMC Medical Education, 15(1), 140.
https://doi.org/10.1186/s12909-015-0422-2
Ko, J., Sammons, P., CfBT Education Trust (Great Britain), Hong Kong Institute of
Education (China), & University of Oxford (England), D. of E. (2013). Effective
Teaching: A Review of Research and Evidence.
Lui, A. (2012). Teaching in the Zone: An introduction to working within the Zone of
Proximal Development (ZPD) to drive effective early childhood instruction.
Children’s Progress, 1–10.
Machumu, H. (2011). The Growing Impetus of Community Secondary Schools in Tanzania:
Quality concern is debatable.
http://content.grin.com/document/v181095.pdf
Mafumiko, F. (2006). Micro-Scale Chemistry Experiments as a Catalyst for Improving the
Chemistry Curriculum in Tanzania. Unpublished PhD Thesis, University of Twente,
The Netherlands.
Mertens, D. M. (2010). Research and evaluation in education and psychology: Integrating
diversity with quantitative, qualitative, and mixed methods (3rd ed). Sage.
Nbina, J. B., & Mmaduka, O. A. (2014). Enhancing chemistry teaching in secondary
schools: An alternative teaching approach. AFRREV STECH: An International
Journal of Science and Technology, 3(2), 127–135.
O-saki, K. (2007). “Science and Mathematics Teacher Preparation in Tanzania: Lessons
from Teacher Improvement Projects in Tanzania, 1965-2006. International
Education Cooperation, 2(1), 51–64.
107
Pirttimaa, M., Husu, J., & Metsärinne, M. (2017). Uncovering procedural knowledge in
craft, design, and technology education: A case of hands-on activities in
electronics. International Journal of Technology and Design Education, 27(2), 215–231.
https://doi.org/10.1007/s10798-015-9345-9
Prince, M. (2004). Does Active Learning Work? A Review of the Research. Journal of
Engineering Education, 93(3), 223–231. https://doi.org/10.1002/j.2168-
9830.2004.tb00809.x
Prins, G. T., Bulte, A. M. W., & Pilot, A. (2018). Designing context-based teaching materials
by transforming authentic scientific modelling practices in chemistry. International
Journal of Science Education, 40(10), 1108–1135.
https://doi.org/10.1080/09500693.2018.1470347
Reeves, T. C. (2000). Enhancing the worth of instructional technology research through
“design experiments” and other development research strategies. International
Perspectives on Instructional Technology Research for the 21st Century, 27, 1–15.
Rizk, L. (2011). Learning by doing: Toward an experiential approach to professional
development. Cairo, Egypt.
Schwichow, M., Zimmerman, C., Croker, S., & Härtig, H. (2016). What students learn from
hands-on activities: HANDS-ON VERSUS PAPER-AND-PENCIL. Journal of
Research in Science Teaching, 53(7), 980–1002. https://doi.org/10.1002/tea.21320
Sevian, H., & Talanquer, V. (2014). Rethinking chemistry: A learning progression on
chemical thinking. Chem. Educ. Res. Pract., 15(1), 10–23.
https://doi.org/10.1039/C3RP00111C
Sikandar, A. (2016). John Dewey and his philosophy of education. Journal of Education and
Educational Development, 2(2), 191–201.
Stammes, H., Henze, I., Barendsen, E., & de Vries, M. (2020). Bringing design practices to
chemistry classrooms: Studying teachers’ pedagogical ideas in the context of a
professional learning community. International Journal of Science Education, 42(4),
526–546. https://doi.org/10.1080/09500693.2020.1717015
Udogu, M.-A. E., & Enukora, E. (2017). The Efficacy of Locally Improvised Materials and
Home Activities on Students Achievement and Retention in Chemistry.
International Digital Organization for Scientific Research, 2(3), 30–41.
Valdez, A. V., Lomoljo, A., Dumrang, S. P., & Didatar, M. M. (2015). Developing critical
thinking through activity-based and co-operative learning approach in teaching
high-school chemistry. International Journal of Social Science and Humanity, 5(1), 139.
Vos, M. A. J., Taconis, R., Jochems, W. M. G., & Pilot, A. (2010). Teachers implementing
context-based teaching materials: A framework for case-analysis in chemistry.
Chem. Educ. Res. Pract., 11(3), 193–206. https://doi.org/10.1039/C005468M
Wang, C. X. (2021). CAFE: An Instructional Design Model to Assist K-12 Teachers to Teach
Remotely during and beyond the Covid-19 Pandemic. TechTrends, 65(1), 8–16.
https://doi.org/10.1007/s11528-020-00555-8
Wood, C. (2006). The development of creative problem solving in chemistry. Chemistry
Education Research and Practice, 7(2), 96–113.
Yin, R. K. (2009). Case study research: Design and methods (4th edn). Sage Publications.
108
APPENDIX A
Focus group discussion (FGDs) guide
The focus-group discussion was mainly for students; and the following
questions guided them throughout the discussion.
a. Introduction of the researcher and the students, who were involved

b. Researcher gives guidelines of the discussion
c. Are you taking chemistry and what do you think is the importance of
studying chemistry?
d. What should teachers do to improve the way you are learning chemistry?
e. What can you say about this way of teaching, in which the teacher
involves you in hands-on activities? (Probe, depending on the answer)
f. Do you think it is good to search and for your own learning materials?
(Why do you say that?)
g. In what ways can this strategy be used to help you acquire desired
competences and learning skills?
h. What suggestions do you have for improving the next lessons?
i. What else would you wish to add?
APPENDIX B
Semi-structured interview guide for teachers
Interviews for all teachers started with the introduction of each other, in order
to establish rapport with teachers and to make them free to talk. Thereafter, the
following questions guided the interview:
a. What can you say about this learner-centred teaching approach, which
involves learners in different activities during the lesson?
b. Has this strategy been helpful to you (probe; how)?
c. How do students perceive the issue of involving them in hands-on
activities?
d. What about being involved in searching and the designing of learning
materials?
e. How are students interacting with learning materials?
f. What could be done to improve this instructional strategy and to make it
more useful to your teaching practice?
g. What challenges have you encountered in using this instructional
strategy?
h. Are there any other things related to chemistry, instructional strategy
and instructional materials that you think are important to consider?
(What are they?)
109

Vol. 21, No. 8, pp. 109-128, August 2022
Received Apr 10, 2022; Revised Jul 23, 2022; Accepted Aug 12, 2022
The Development of e-Reading to Improve

English Reading Ability and Energise Thai
Learners’ Self-Directed Learning Strategies
Pongpatchara Kawinkoonlasate
English Department, Faculty of Liberal Arts,
Huachiew Chalermprakiet University, Samut Prakarn, Thailand
Abstract. Technological education has changed lifestyles, and

educational institutions need to prepare their learners with the necessary
tools to be successful in real life. The objectives of the study are: 1) to
explore learner’s achievements in reading comprehension with the use of
e-Reading and traditional teaching methods, 2) to analyse the satisfaction
of learners using e-reading and traditional teaching methods, and 3) to
observe the effects of the learner’s reading comprehension while using e-
reading methods versus traditional teaching methods based on a
standard 80/80 criterion. The participants were sophomore learners
enrolled in English courses. Thirty learners were selected by
nonprobability sampling and divided into two groups, the experimental
and control. The experimental group received lessons using an e-Reading
programme, the control group was taught using a traditional teaching
method. The research tools were pre- and post-tests, lesson plans,
learners’ perception questionnaires, and interviews. Data collected
through quantitative means were analysed using a t-test standard and
average deviation programme. The interview data were analysed using a
content analysis method. Data from the pre- and post-tests show that
reading comprehension improved in the experimental group, who used
the e-Reading programme. They also showed a “positive” attitude
regarding their learning satisfaction and self-directed learning. The
learners’ satisfaction and self-directed learning were higher in the e-
Reading programme. Standard 80/80 criteria were met with an efficiency
of 81.6/82.3 in the e-Reading programme. This research could be used as
reference for further study and application of the e-Reading programme.
Keywords: e-Reading programme; English reading ability; Self-directed

Learning; Thai EFL Learners; Motivation
1. Introduction
Global communication is largely carried out through the internet, and the
majority of users on the internet communicate in English. To be competitive in the
global economy, individuals need to have proficient skills in English and,
©Authors
This work is licensed under a Creative Commons Attribution-Non-Commercial-Non-Derivatives
110
especially, English reading comprehension. With a strong grasp of the English

language, people in every country have access to larger audiences and potential
business opportunities. English is also useful in the daily lives of most working
individuals (Kruawan & Dennis, 2017).
With English being taught in almost every country, globalisation is fuelled by
English as a world language and, in Thailand, English is the language most taught
and studied because of its use throughout the world (Nagi, 2012). Since English is
a foreign language (EFL) among Thailand learners, instructors need access to
multiple teaching techniques. Thai language instructors need to find new and
creative ways to teach English to their learners. As a doctor needs different types
of medication to treat different kinds of ailments, so language teachers need
different kinds of teaching techniques to motivate and improve their learners’
learning. Traditional Thai instructors have used a teacher-centred approach to
teaching English; however, a more learner-centred approach is becoming more
popular. Technology can help promote learner-centred learning in and outside
the classroom (Chen, 2010). Educators should try to employ technology in their
classrooms whenever possible to help support learner-centred environments.
Not only is reading in English critical for necessary for global communication, but
it is also necessary for testing. Being able to read and to understand English text
is very valuable (Ellis & Shintani, 2014) for Thai learners. A learner who is
proficient in reading in English will be able to expand their writing, listening, and
speaking skills more easily in English (Shang, 2015). Reading is one of the most
valuable tools a learner can acquire for their educational and professional careers.
Being able to analyse and understand textbooks, journals, and online material is
important for Thai EFL learners, and reading in English is valuable for graduate
learners because most research is published in English journals. Learners with
strong reading comprehension skills can spend more time reflecting on English
text rather than trying to piece words together to understand each word. Reading
is one of the most valuable tools a learner can acquire for their educational and
professional careers.
However, learners and instructors can find it difficult to learn and teach English
as a second or foreign language. Thai university learners have been found to have
low English reading skills, and even EFL learners struggle with English text. Thai
learners struggle with remembering vocabulary and understanding sentence
structures, which makes it difficult for them to understand what they are reading
(Uamduang 2012; Kottong, 2014). Low reading comprehension makes it very
difficult for Thai learners to rapidly skim and scan text for relevant information.
When learners struggle with reading comprehension, they also have difficulty
using teacher resources such as worksheets, short stories, and sentence structure
lessons. One reason for Thai learners’ lack of reading ability is a limited
vocabulary and understanding of sentence structure. Vocabulary understanding
enables the learner to understand the meaning of words while understanding
sentence structure helps learners recognise word order and patterns in texts.
When learners encounter a text which is not relevant to their studies or personal
life, it can be hard for them to feel motivated to try and understand the text
thoroughly. All of these issues can affect a learner's reading comprehension. Self-
111
directed learning is a learner’s ability to successfully gain more knowledge about

a topic or subject without the assistance of a teacher or colleague (Shakeri, 2012)
and such self-directed learning can also be influenced by a learner’s reading
comprehension. For a learner to truly be autonomous, they need to be able to
search out information on their own. If they lack reading skills it can be difficult
for them to find relevant information.
This study aims to aid language instructors, to investigate learners' achievements

in reading comprehension using e-Reading and traditional teaching methods, to
analyse the satisfaction of learners using e-reading and traditional teaching
methods, and to observe the effects of the learner’s reading comprehension while
using e-Reading methods versus traditional teaching methods, based on the
standard 80/80 criterion. Accordingly, three research questions were formulated
as follows:
1. How can an e-Reading programme help learners improve their skills in reading
comprehension?
2. What are the effects of the e-Reading programme compared to a traditional
teaching method regarding the learner's satisfaction?
3. How does the e-Reading programme improve the learner’s reading
comprehension?
2.1 Reading Comprehension
Reading skills and attitudes toward reading with electronic text have become a
popular research topic in recent years, one which has been investigated in many
recent studies (Larson, 2010; Short, 2010; Anderson, 2012; Huang, 2013).
For EFL and ESL learners, reading is critical because most tests and evaluations in
English courses are conducted by reading text and writing answers; further, being
able to understand and interpret English text is a valuable skill for most jobs and
careers (Khruawan & Dennis, 2017). Learners have a variety of strategies for
developing their reading comprehension; those who find it difficult to improve
their reading comprehension can seek guidance from instructors, but ultimately,
it is only the learner’s discipline and motivation that can truly help them succeed.
The strategies approach has proved effective in improving learners’ reading
comprehension (McNamara, 2004; McNamara, 2007; Zhang, 2008; McKeown,
Beck & Blake, 2009;). Oakley (2011) describes cognitive strategies which can be
employed by EFL or ESL instructors to improve reading comprehension in
learners. These include summarising, visualising, questioning, making inferences,
and predicting. Connecting a learner’s background knowledge to the language
lesson to make the lesson more engaging and personal, encouraging predictions,
organising thoughts into visual charts, and answering questions are a few
strategies that can be used.
Reading models can be used to explain what processes are taking place during
reading and can explain how a learner perceives text with their eyes and then
forms an understanding of the text. Educators need to be fluent in different types
of theoretical models and approaches when trying to understand a learner's
112
ability to read (Tracey & Morrow, 2012). Tracey and Morrow (2012) emphasise
social constructivist, constructivist, behaviourist, and information processing
cognitive theories. What learners are thinking and doing while reading can be
shown in the cognitive-processing theory. The way learners build from the text
while interacting with it can be explained in the constructivist theory (Auer, 2016).
Cognitive-processing and constructivist theories will be used in this research
paper regarding reading and can be used to describe EFL learners' cognitive
reading aspects. The following table shows the learners reading models and
strategies.
Table 1: Reading models and related reading strategies
Reading model Examples of reading Strategy

strategies description
1. Participative Model 1. Drawing conclusions. 1. Use world knowledge
to identify the meanings
of words.
2. Construction 1. Activating background 1. Think about prior
Integration Model knowledge. knowledge to help to
2. Identifying the main understand.
points. 2. Focus on the main
points of the text.
3. Schema Theory 1. Creating reading goals. 1. Understand why
reading is valuable.
4. Metacognitive Theory 1. Organising and assessing. 1. Identify key ideas,
review while reading,
and understand what
you are reading.
2.2 Technology Tools that Support Reading Comprehension

The tools used to interact with digital text such as electronic hardware and
software are what this study refers to as e-Reading technology. Computers,
smartphones, tablets, and e-Readers are some of the tools used in e-Reading
(Biancarosa & Griffiths, 2012). The e-Reading programme and all the hardware
can be used to create a convenient and practical way for learners to practise
reading in English. A study by Mangen, Bente and Bronnick (2013) explains that
text displayed electronically might be more useful than text displayed on paper.
However, a text which is found online might be more complex for the learners to
access than a printed text. Almekhlafi (2020) studied the use of e-Reading books
on EFL learners and concluded that e-book reading projects had a positive effect
on perceptions of the learners’ content learning, reading ability, and English
learning.
The use of e-Reading devices is becoming more common in educational

institutions as English reading lessons that include e-Reading technology can
create better opportunities for learners that struggle with reading and help
increase self-directed learning (Pastore, 2008). The features present in e-books can
help promote processing and memory as well as increase the learner's motivation
and attitude toward the reading lessons which employ reading technology
(Ertem, 2010; Korat, 2010; Park & Kim, 2011; Chen, Chen, Chen & Wey, 2013;
113
Eicker-Nel & Matthee, 2014; Reid, 2016). Traditional reading lessons with drills
are not as engaging as newer electronic reading lessons. With relevant and
interesting text more widely available in online reading lessons, learners are more
able and willing to expand their literacy and to engage more meaningfully in the
language lessons (Ertem, 2010; Korat, 2010). Ciampa (2012) and Huang (2013)
found that an e-Reading programme increases the convenience for the readers
and can improve the learners’ reading speed as well as their engagement. Recent
studies have proved how reading speed, comprehension, and accuracy can all be
improved with electronic reading devices and programmes. Studies by Mangen,
Bente and Bronnick (2013) and Almekhlafi (2020) showed that digital text is more
useful than printed text.
All these studies imply that teachers who implement electronic reading in their
lessons will see improvements in learners' attitudes, engagement, and
achievements. The 21st century is bringing newer and more affordable technology
which can be used in the classroom. When technology is included in a language
programme’s curriculum, the learners should be more successful in reading. For
these reasons this study examines the impact of portable e-reading programme
use on EFL learners’ perceived reading skills (comprehension) and attitudes
(enjoyment), and identifies the learners’ levels of satisfaction when using an e-
Reading instructional design programme.
2.3 Technology for Reading Comprehension and Motivation

With all the new and affordable technology introduced each year, academic
institutions have an opportunity to add reading technology to the classroom to
promote learner literacy rates. Reading comprehension, motivation, and fluency
have all been found to be affected by technology. Furthermore, Chen (2010)
believes that using technology in the classroom can raise the learner's abilities. A
wide variety of technology plays a part in most Thai learners’ daily lives. Most
learners already have electronic devices that can access the internet, so enabling
schools to create assignments and activities that can be done in the learner's home
or anywhere the learner might find convenient. Using electronic devices in a
learning context excites, interests, and motivates learners (Ciampa, 2012; Fox,
2014; Szabo & Long, 2016; Ingram, 2020). However, Sackstein, Spark and Jenkins
(2015) explain that teachers must be able to use the technology themselves if they
want to implement electronic devices in their classroom. The success of a lesson
using electronic device hinges on the ability of the teacher’s knowledge of the
technology and their creativity in using it in interesting and engaging ways.
Reading comprehension can be improved if electronic tools are correctly

implemented in the curriculum. Different electronic applications can affect a
learner's reading comprehension and background knowledge of the desired text
(Crum, 2017). More access to computers and electronic devices can improve
learners’ reading and writing skills and their motivation toward the language
being studied (Linik, 2012). Many teachers prefer using electronic devices in their
classrooms (Keengwe, Onchwari & Agamba 2014). Therefore, many technology-
based lessons and programmes have been created and studied to support reading
in an academic environment. Technology can help replace a more teacher-centred
learning environment and allow the learners to become more autonomous.
114
2.4 Self-directed Learning in Language Learning Development

Self-directed learning is a collaborative learning approach. The teacher must
provide the learner with the tools and guidance necessary to learn and practise on
their own although the learner should seldom have the impression that they are
learning entirely on their own (Khodabandehlou et al., 2012). Many learners
might have difficulty when reading, such as trying to find an answer, or not
understanding what they are reading. Self-directed learning can help learners to
recognise their learning techniques and learn how to set goals for personal
development and to improve their academic achievements.
Many factors can affect a learner's language ability such as learning strategies,
motivation, social anxiety, and learning beliefs (Gregersen & Horwitz, 2002;
Osanai, 2000). In ESL and EFL classrooms, self-directed learning should be
promoted to encourage learners to become life-long language learners. With
regard to life-long language learning, the learner’s interests and determination
can be important indicators for future success and learning. Holec (1996) showed
that language learning programmes need to encourage self-directed learning as
much as possible. When learners can learn on their own, they are more likely to
be successful language learners and users. In the following figure, a
comprehensive model shows how motivation can lead to self-directed learning.
Motivation
Self-Monitoring Self-
Management
Self-Directed
Learning
Figure 1: Self-Directed Learning toward a Comprehensive Model (Garrison, 1997).
3. Research Methodology
3.1 Investigative Techniques
The current study employed experimental research. The research design of this
study is the plan, structure, and strategy of investigation used to find the solutions
to the research questions. A system was created to study the potential of the e-
Reading course. This system was split into two parts. The first part of this system
was devoted to reading strategies. The second part introduced ten reading
activities with practice assignments. The learners then responded to
questionnaires that detailed their experiences and thoughts. Finally, the selected
learners were interviewed to elicit more qualitative data on their experiences. The
goal of the interviews was to support data from the questionnaires. A more
thorough picture emerges from examining both the qualitative and quantitative
115
data. Figure 2 shows the set-up and steps implemented during the conduct of the
study.
Group A: The e-Reading Learning Group B: Traditional
Teaching
Group A Learners Group B Learners
1. Pre-test and 10 reading topics start 1. Pre-test and 10 reading topics start
from W1-W.10
from W1-W.10
2. Worksheets are developed

2. e-Reading lesson plans created
3. Start the lessons

3. Start the lessons
4. Scores are collected

4. Scores are collected
5. Post-test
5. Post-test
6. Questionnaire
6. Questionnaire
7. Interview: Learners’ attitudes
7. Interview: Learners’ attitudes
8. Data are collected and analysed
8. Data review
Figure 2: The steps of the experimental and control groups
3.2 Research Instruments

The researcher conducted the learning programme along the following lines:
3.2.1 Lesson Plans

Two types of lesson plans were created. Lesson plans for the e-Reading
programme were created first, then the traditional teaching lesson plans. The
lesson plans in this study covered 15 classes with a focus on reading
comprehension strategies, ten reading assignments with practice activities, and a
reading test. Next, the steps taken to create the lesson plans are explained.
116
3.2.2. Model for e-Reading Instructional Design Programme

The researcher divided the emerging e-Reading instructional design approach
into two parts. Lessons on reading techniques were covered in the first section,
while reading passages with exercises were covered in the second. Five experts
reviewed the effectiveness of the e-Reading lessons once they were completed to
check for material correctness, lesson design, and to make other suggestions for
improvement. Finally, the e-Reading instructional design model was tested with
forty students to establish the validity and accuracy of content and the efficacy of
the lessons.
3.2.3 Learners’ Perception Questionnaire

The researcher created a perception questionnaire to gauge how happy the
students were with their education and these various teaching strategies. There
were two sections to the questionnaire: demographic questions were asked in the
first section to learn more about the students, and students were asked for their
thoughts and reactions to the various teaching strategies in the second section.
Each question was answered by the learners using a five-point Likert scale, with
the options being ‘strongly agree’ to ‘strongly disagree’. After completion, the
questionnaire was given to the experts to review for correctness and suitability.
Once the questionnaire had been piloted with the learners, the result of Item
Objective Congruence Index by three experts showed a mean value at 0.93.
3.2.4 Interview Questions

Interviews were used as an additional way to consolidate the data. The basis of
interviews is a set of prepared questions that allow the researcher to delve deeply
into the subject and provide a wealth of information for review. The data collected
from the interviews were used to further support or explain the data from the
questionnaires. Experts were used to review and edit the interview questions. Ten
questions were created and used in each interview.
3.2.5 The Pre- and Post-examinations

A reading section from the TOEFL test was used for the examinations. The test
was designed to gauge the learner’s reading ability and understanding of the
content. The pre-examination was given to the learners before the lessons were
taught, and the post-examination took place at the end of the learning
programmes. The findings of both tests are compared and evaluated by using the
t-test for independent samples to see whether this model enhanced the learners’
reading ability or not.
3.3 Participants of the Study

The learners who took part in this research were all second-year English major
students. Two groups were created with 30 learners in each group. A non-
probability sampling method was used. The first group, Group A, was the
experimental group which was exposed to the e-Reading programme. The second
group (Group B) was taught using traditional teaching methods. Participants
were chosen based on their weak performance in reading comprehension. All the
learners took a reading proficiency test and completed a reading strategy
117
questionnaire before being selected and put into groups to ensure that their
English proficiency levels matched or were in the same range.
3.4 Data Collection

3.4.1 Treatment
The learners were exposed to both teaching methods for 15 weeks. Each week the
learners took part in one session, for a total of 15 sessions. All the sessions took
place in the second semester of the January 2020 to May 2021 school year at
Huachiew Chalermprakiet University (HCU) which permitted this study to be
conducted on campus. Data collection was done during the learning sessions. The
results of the study did not affect the learner's academic report card. Each group
spent the following amount of learning time:
1. The e-Reading group (Group A) received 3 hours per week with a total of 45
hours using e-Reading electronic programmes.
2. The control group (Group B) also received 3 hours per week with a total of 45
hours using traditional textbooks and paper worksheets.
In the first week of the programme, all participants in both groups took the pre-
test. Sections from the TOEFL test were used for both pre- and post-examination,
with topics for the reading passages chosen based on the learners' preferences.
When the students in Group A began their lessons, the ten reading topics were
deployed using the e-Reading instructional design programme. The conventional
teaching approach was used to teach lessons to Group B students. Each learner's
scores were recorded once they had finished all the tasks in their programmes. All
of the students took the post-test at the conclusion of their reading programme. It
was explained to all the participants in each group that the scores from these
programmes would not be included in their grades. The participants’ perceptions
of the benefits and drawbacks of learning English using these instructional
approaches were then elicited using questionnaires. Learners were chosen at
random to take part in interviews during the final lesson of each learning
programme so they could share, express, or make comments about their
experiences in the e-Reading programme and the traditional teaching
programme.
3.5 Data Analysis

Both quantitative and qualitative analyses of the data were performed in this
study. All data were analysed using the following steps.
1. A t-test was used to compute and convert the pre-test and post-test scores into
mean values. Each group’s mean and standard deviation were used to determine
whether there was a statistically significant difference between the mean scores
on the pre-and post-tests.
2. The scores of the effectiveness of the e-Reading instructional design programme

were calculated with the efficiency of the exercises using the Efficiency Standard
Criterion on an 80/80 basis (E1/E2).
118
3. The scores of the perception questionnaires were analysed in terms of the mean
( X ) and standard deviation (S.D.).
4. Data from the interview questions were read, then read again, and finally
coded. The raw data were coded into useful categories. The researcher rearranged
the data into increasingly specific groups as the coding process went on. All the
data were coded using the final list of codes.
4. Results
4.1 e-Reading programme approach and students’ achievement in reading
comprehension
The effect of e-Reading programme on students’ achievement in reading
comprehension were measured through pre- and post-tests.
The pre-test and post-test results for the experimental and control groups are
given in Table 4.1.1. As shown in the table during the pre-test only 1 (3.3.%) in the
experimental group reach an excellent level of performance while 3 (10%) in the
control group. The data presented in Table 2 show that the control group
performed better than the experimental group in the pre-test since there are 2
(6.7%) in the experimental group in the ‘poor performance’ level. However, after
exposing the two groups to the different methods of teaching, results in the post-
test scores reveal that both groups reach excellent (100%) level performance. This
implies that e-reading instructional material is of great help in improving the
performance level of the experimental group, and the traditional approach did the
same for the control group.
Table 2: Level of scores during pre-test scores and post-test of both groups
Evaluation Lists Experimental Group Control Group
Pre-test Post-test Pre-test Post-test
N % N % N % N %
Excellent 1 3.3 30 100.0 3 10.0 30 100.0
Good 16 53.3 - - 20 66.7 - -
Fair 11 36.7 - - 7 23.3 - -
Poor 2 6.7 - - - - - -
Very poor - - - - - - - -
Total 30 100 30 100 30 100 30 100
To analyse in which particular exercises the experimental group improved their

performance, each exercise was subjected to statistical analysis, as shown in Table
3.
119
The experimental group learners had an average score of “excellent” (40.80%),

and the control group learners had an average score of “good” (39.83%). This
difference was not statistically significant, but it cannot be denied that the
experimental group performed better than the control group after they were
exposed to e-Reading lessons. Analysis of the association between learning
groups and the various levels of the pre-test and post-test scores make it clear that
the learners in both groups (experimental and control) had higher post-test scores
regardless of the teaching delivery method that they received. However, the
experimental group and the control group had somewhat different mean scores
for the 10 exercises, according to the average scores for all 10 exercises.
Covid-19 restrictions during the data collection affected the delivery of the
teaching programmes. As a result, teaching and learning management was
modified to an online format. Using traditional teaching methods in the control
group’s classroom was not practical, so the control group’s data collection was
adjusted to an online format. It was difficult to control or force learners to search
for information from outside resources, so learners in the control group were able
to search for knowledge and practise their reading skills using various types of
technological media, which may explain the similarity of the control group’s
scores to those of the experimental group. The average exercise scores of the
experimental group were higher than the exercise average scores of the control
group when each exercise is examined separately. This implies that the results of
this study found that the e-Reading programme approach can assist learners in
improving their achievements in reading comprehension.
Table 3: The scores of each exercise for both groups

Exercise Experimental Group Control Group
xˉ SD. Meaning xˉ SD. Meaning
Exercise 1 (5 Points) 3.20 1.424 Fair 3.37 .556 Fair
Exercise 2 (5 Points) 4.30 .988 Excellent 2.37 1.245 Poor
Exercise 3 (5 Points) 4.53 .860 Excellent 4.43 1.006 Excellent
Exercise 4 (5 Points) 4.77 .679 Excellent 4.43 .728 Excellent
Exercise 5 (5 Points) 3.20 1.215 Fair 3.87 1.042 Good
Exercise 6 (5 Points) 3.07 1.760 Fair 4.43 1.073 Excellent
Exercise 7 (5 Points) 4.23 .858 Excellent 4.33 1.124 Excellent
Exercise 8 (5 Points) 4.50 .572 Excellent 4.00 .947 Good
Exercise 9 (5 Points) 4.70 .535 Excellent 3.97 .928 Good
Exercise 10 (5 Points) 4.30 .596 Excellent 4.63 .669 Excellent
Total (50 Points) 40.80 9.487 Excellent 39.83 9.318 Good
120
4.2 Effects of standard teaching approach and e-reading programme on

students’ pleasure and self-directed learning
The first segment described how students perceived the delivery method used to
convey their instruction. Overall results in Table 4 show that the level of the
learners’ satisfaction in the experimental group with teaching and learning
courses had an average of 3.97. Similarly, the control group had a total average of
3.41, which was also a “good" level. These results are evidence that both groups
experienced the same satisfaction with the teaching methods they were exposed
to.
Table 4: Learners’ satisfaction with the two teaching delivery methods
Learners’ Satisfaction Experimental Group Control Group
1. Learners are satisfied 3.93 .365 Good 3.57 .568 Good

with the teaching
methodology.
2. The teaching 4.17 .592 Good 3.43 .679 Good

methodology is suitable for
learning English reading.
3. Learners often learn with 4.10 .481 Good 3.57 .504 Good
their lessons and practise
being responsible.
4. Learners like to practise 3.73 .583 Good 3.30 .702 Fair

reading with their teaching
method.
5. Learners gain more 3.90 .548 Good 3.57 .679 Good

academic knowledge with
their teaching method.
6. The teaching methods 3.47 .681 Good 2.83 .986 Fair

affect the learner’s study.
7. Learners have no limit to 4.20 .610 Good 3.47 .681 Good

study and can study any
time and place.
8. It is convenient for 4.13 .507 Good 3.57 .679 Good

learners to review the
lesson outside the
classroom using their
teaching media.
9. Using this teaching media 3.90 .662 Good 3.23 .774 Fair
allows learners to meet
their learning objectives.
121
Learners’ Satisfaction Experimental Group Control Group
10. Learners have the 4.20 .664 Good 3.60 .724 Good
freedom to study from their
teaching method.
Total 3.97 .569 Good 3.41 .698 Good
According to the overall results shown in Table 5, the level of the learners’ self-
directed learning in the experimental group towards teaching and learning had
an average of 3.83, which was a “good” level. The control group had a total
average of 3.42, which was also a “good” level. According to these findings, the
level of self-directed learning toward the instruction in the learners in the
experimental group was, by a difference of 0.05, marginally greater than the
control group.
Table 5: Level of learners’ self-directed learning in the two groups

Learners’ Self-Directed Experimental Group Control Group
Learning
1. The teaching methods 3.83 .531 Good 3.43 .728 Good

create self-directed
learning atmosphere.
2. Learners are happy 3.60 .675 Good 3.30 .702 Fair

and have fun in their
learning.
3. Learners enjoy 3.60 .621 Good 3.37 .809 Fair

learning with the
teaching media.
4. The teaching methods 3.97 .615 Good 3.67 .711 Good

are suitable for learner-
centred learning.
5. The teaching methods 4.13 .507 Good 3.33 .994 Fair

motivate learners to
improve reading skills.
Total 3.83 .590 Good 3.42 .789 Good
4.3 Effects of e-Reading programme on learners’ reading comprehension

The researcher used the Efficiency Standard Criterion on the 80/80 basis (E1/E2)
to assess the effectiveness of the programme. The results are presented in Table 6
where E1 is the average score of the participants who were able to complete the
class exercises with a score of at least 80%. The average score (E2) is that of the
individuals who were able to pass the tests with a score of at least 80%.
122
According to Table 6, the e-Reading programme was effective on the efficiency

standard criterion at 81.6/82.3. The efficiency value of exercises (E1) was 81.6%
(Mean = 40.8) whereas the efficiency value of the test (E2) was 82.3% (Mean =
16.46). Thus, the e-Reading programme proved efficient and allowed the learners
to learn effectively on the efficiency standard criterion of 80/80.
Table 6: Effectiveness value of exercises and achievement test (n = 30)

Scores from exercises/ achievement Total Scores X Percentage
test
Scores from exercises (E1) 50 40.8 81.6
Scores from the achievement test (E2) 20 16.46 82.3
5. Discussion
During the interviews, the students were asked about their teaching and learning
experiences with the e-Reading software, and they reported that they had tailored
learning opportunities, enhanced their English reading ability, and understood
their reading processes. The e-Reading programme not only brought greater
learning access to the classroom, but also enabled the instructor to adapt the lesson
to each learner's learning style. The instructor could also customise lessons and
homework to fit the learner's unique approaches and optimise their results. This
helped turn traditionally dull subjects into interactive and fun activities.
e-Reading can help learners to take responsibility for their education. In the
future, learners will be able to use e-Reading programmes to guide themselves
through lessons, learn at their own pace, or identify areas of personal interest. The
learners expressed their attitudes toward the e-Reading programme in terms of
helping them to study outside the classroom after they had finished the class
lesson. They also prepared for each lesson before the class met, which helped them
to understand the content before the class discussions.
The experimental group learners' exercise scores were higher than those of the
control group students. These results are in line with the study by Biancarosa and
Griffiths (2012) who believe that electronic tools in the classroom can lead to
higher learner motivation and achievements. Teerapaksiri, Watanabut and Sopa
(2017) agree that using technology can help learners increase their language
abilities. Their findings showed that the learners' communication post-test scores
were, with a 0.05 difference, higher than their pre-test scores. The e-Reading
programme was convenient for the learners to use in the classroom and at home,
which is in line with findings by Ciampa (2012), Huang (2013) and Almekhlafi
(2020). Reading programmes that use computers, phones, and other electronic
devices proved to be more effective than traditional programmes, according to
Ingram (2020). The findings show that technology is an excellent learning and
teaching tool for reading. Nevertheless, some limitations of this study must be
acknowledged; since the Covid-19 pandemic occurred during data collection, the
teaching method was changed from a traditional teaching method to an online
format.
According to the study's findings, students in both the experimental group and
control group scored better on the post-test than they had on the pre-test.
123
However, when the students in the control group were questioned about their
experiences with the standard instructional delivery technique, they expressed
negative feelings towards the traditional teaching method. First, the learners
explained that they had wanted to get more learning experiences from this course,
that it had been hard for them to gain more learning experience. Second, they had
no idea how to create their learning atmosphere. Finally, the learners wanted
more time to study and practise. In their opinion, the traditional teaching
approach may not be appropriate for learners who take a passive approach to
learning, where the instructor is in charge of everything in the classroom.
Additionally, they suggested that undergraduate students study online rather

than with chalk, discussion and paying attention to their professors' instructions.
During the Covid-19 pandemic, the learners had to study via an online channel,
so paper-based forms were not appropriate for them to study. The students
offered their comments based on the improvement of the teaching strategy while
concentrating on the advice from the traditional classroom. They advised the
instructor to create an online course and use technology in educational instruction
to hone their media-teaching skills. In conclusion, traditional learning methods
allow the instructors to decide how they convey information to their learners, but
online learning provides more flexibility for the learners, especially during the
Covid-19 pandemic. Most learners provided suggestions in terms of applying
technological devices to the classroom and designing an online course.
In response to the second research question, “What are the effects of the e-Reading
programme compared to a traditional teaching method regarding the learner's
satisfaction?” the outcomes of the perception surveys and interview questions
revealed the impact of the traditional teaching technique and the e-Reading
programme on learners' satisfaction and self-directed learning. The experimental
group's level of student satisfaction with the teaching and learning of the courses
was higher than that of the control group, confirming that the learners were
satisfied with the e-Reading programme, which offered them learning benefits
and reading comprehension practice. The findings of this study were similar to
the findings of Vaish (2016) and Klauda and Guthrie (2015) who found that
reading speeds, reading comprehension, and interest could all be improved with
e-reading programmes. Szabo and Long (2016) also found that electronic books
and e-reading programmes could be used to create positive feelings towards
reading and improve the learner’s achievements. These are similar to the results
found in this study. Confirming the results of this research was the research
carried out by Fox (2014) and Sackstein, Spark and Jenkins (2015), who found that
e-Reading programmes were convenient and affordable for language learners.
Another study with similar results was that of Teerapaksiri, Watanabut, and Sopa
(2017) which identified electronic books as effective at raising the satisfaction
levels of learners. In their interviews, the students in the experimental group
insisted that the e-Reading programme promoted their self-directed learning. The
learners tended to find the idea of adopting technology-assisted self-directed
learning in language classrooms to be difficult. New methods and technologies
that fit learners' lifestyles and those things that attracted and motivated them to
study independently were closely tied to innovations in teaching and learning. In
124
this study, the learners were satisfied with the e-Reading programme because it
promoted self-directed learning, and they had a great chance to study, think, and
share knowledge with their peers. The learners also claimed that the e-Reading
programme helped them and their parents to save on textbook costs. During the
Covid-19 crisis, any financial help towards learning and living was welcomed.
In the interview results, the learners expressed satisfaction in terms of

independence for learning, reduction of pressure, suitable and useful learning
media, and enjoyable learning. The learners claimed that the e-Reading
programme helped them to reduce stress and pressure during their learning. They
felt relaxed and spent more time studying and practising their lessons. Both inside
and outside of the classroom, they felt free to study. Furthermore, the learners
could share the learning experience with their peers. Therefore, in general, online
reading programmes like the e-Reading programmes instruct learners in key
literacy areas within a one-on-one learning environment with instruction being
inherently motivational and fun. The learners who participated in the e-Reading
programme achieved significantly higher learning outcomes over those in the
traditional classrooms and were equally as satisfied with the learning
environment.
Responses to Research Question 3, “How does the e-Reading programme improve

the learner’s reading comprehension?” showed the efficiency standard criterion
on an 80/80 basis (E1/ E2) to find the effectiveness of the e-Reading programme
was discussed as follows: the e-Reading programme was effective on the
efficiency standard criterion at 81.6/82.3; it was efficient and allowed the learners
to learn effectively, findings which were congruent with the study by Kongvimon
(2016) who had looked into the production of instructional material for
elementary-level instructors using the Photoshop application to create electronic
books (e-Books). The findings found that the developed electronic book (e-Book)
with the Photoshop Programme had an efficiency of 91.17/90.00 which met the
set 90/90 criteria. In similar research, a study by Singha-o-pas, Nuchmee and
InThanin (2016) proved that an electronic book on Information Technology for
education for undergraduate learners at Thaksin University showed an
effectiveness of 88.00/89.11, which is in line with the set criteria. Another study
by Teerapaksiri, Watanabut and Sopa (2017) showed that the index of the
electronic book efficiency was 89.60/87.24 which was higher than the assigned
criteria efficiency.
In summary, applying technological devices to teaching and learning might be a

good way for learners to study for several reasons: the approach creates new paths
for learners to search out knowledge and learn by themselves. The learners also
mentioned that studying with the e-Reading programme saved them the cost of
textbooks. Furthermore, technology supports collaborative learning and aids in
the individualization and personalisation of education as well as reducing the
burden of learning for the learners and teaching for teachers. It makes education
more interesting and creates a more fulfilling learning atmosphere. The learners
in the experimental group expressed positive attitudes towards the e-Reading
programme which provided them a chance to study, to practise more on their
own, and to improve their reading comprehension skills.
125
6. Conclusion and Recommendation

The e-Reading programme is a powerful tool for instructors in capturing the
attention of today’s learners. However, the literature indicates that not many
instructors are utilising e-Reading programmes in the classroom, suggesting the
need for more research to assess their value. It is recommended that there should
be more research and development of e-Reading media formats in other subjects
as well. These could be in the form of online media which can be linked to other
content resources on websites for dissemination of lessons.
The learning media should have a modern format that meets the needs of the
learners. Current educational technology is considered important in teaching and
learning and provides an opportunity for learners to learn at any time.
Technology is an effective tool and very much part of language learning
throughout the world at all different levels. Learners must use technology as a
significant part of their learning process and in their careers. Additionally, when
used properly and successfully, technology may be a useful approach to help
learners create authentic and entertaining environments and enable them to
actively interact while learning a language outside of the classroom. Modern
technological learning systems, like the e-Reading software used in this study, can
give students more engaging material and encourage better language
accomplishment. These applications can also help educators interact with
younger generations and provide pedagogical tools. Consequently, language
instructors worldwide should try to incorporate modern technology into their
curricula to promote technical skills and develop their learner’s language
achievements.
7. References
Almekhlafi, A. G. (2020). The effect of E-books on Preservice learner teachers’ achievement
and perceptions in the United Arab Emirates. Education and Information
Technologies, 26(1), 1001–1021. https://doi.org/10.1007/s10639-020-10298-x
Anderson, T. L. (2012). Examining elementary learners' use of electronic readers for independent
reading (Doctoral dissertation). University of Tennessee.
http://trace.tennessee.edu/utk_graddiss/1262
Auer, N. (2016). Scaffolding foreign language learners’ reading strategies using tablet computers
at two secondary schools in Denmark [PhD Thesis, University of Leicester].
http://www.gymnasieforskning.dk.
Biancarosa, G., & Griffiths, G.G. (2012). Technology tools to support reading in the digital
age. The Future of Children / Center For the Future of Children, the David and Lucile
Packard Foundation. 22: 139–60. https://doi.org/10.1353/foc.2012.0014
Ciampa, K. (2012). Reading in the digital age: Using electronic books as a teaching tool for
beginning readers. Canadian Journal of Learning and Technology, 38(2).
https://doi.org/10.21432/t2nk5n
Chen, C. N., Chen, S. C., Chen, S- H. E., & Wey, S. C. (2013). The effects of extensive reading
via e-books on tertiary level EFL learners' reading attitude, reading
comprehension and vocabulary. Turkish Online Journal of Educational Technology,
12(2). 303–312. https://doi.org/10.1002/rrq.152
Chen, R. (2010). Investigating models for preservice teachers’ use of technology to support
learner-centred learning. Computers & Education, 55(1), 32–42.
https://doi.org/10.1016/j.compedu.2009.11.015
126
Crum, C. E. (2017). Influence of technology on English language learners’ vocabulary, reading,

and comprehension [PhD Thesis, General Education].
https://scholarworks.waldenu.edu/dissertations/
Eicker-Nel S., & Matthee M. (2014). The adoption of tablet-based e-textbooks in a South
African private school. In Proceedings of the e-skills for knowledge South African
Journal of Education, 35(4),1–14. http://proceedings.e-
skillsconference.org/2014/e-skills109-123EickerMatthee715.pdf
Ellis, R., & Shintani, N. (2014). Exploring language through second language acquisition
research. London: Routledge.
Ertem, I. (2010). The effect of electronic storybooks on struggling fourth-graders’ reading
comprehension. Turkish Online Journal of Educational Technology - TOJET, 9(4), 140–
155. https://doi.org/10.17051/ilkonline.2020.763772
Fox, L. C.C., (2014). Effects of Technology on Literacy Skills and Motivation to Read and Write.
Education and Human Development. [Master's Thesis].
http://digitalcommons.brockport.edu/ehd_theses/522
Garrison, J. (1997). Self-directed learning toward a comprehensive model. Science &
Education, 6(6), 543–554. https://doi.org/10.1023/a:1008645503209
Gregersen, T., & Horwitz, E.K. (2002). Language learning and perfectionism: Anxious and
non-anxious language learners' reactions to their oral performance. Modern
Language Journal, 86, 562–570. https://doi.org/10.1111/1540-4781.00161
Huang, H. C. (2013). Online reading strategies at work: What teachers think and what
learners do. ReCALL, 25(3), 340–358. https://doi.org/10.1017/s0958344013000153
Holec, H. (1996). Self-directed learning: an alternative form of training. Language Teaching,
29, 89–93. https://doi.org/10.1017/s0261444800008387
Ingram, N. B. (2020). The use of E-books, computers, and tablets to enhance reading skills [PhD
Thesis]. https://digitalcommons.kennesaw.edu/teachleaddoc_etd/41
Keengwe, J., Onchwari, G., & Agamba, J. (2014). Promoting effective e-learning practices
through the constructivist pedagogy. Education and Information Technologies, 19(4),
887–898. https://doi.org/10.1007/s10639-013-9260-1
Khodabandehlou, M., Jahandar, S., Seyedi, G., Mousavi, D. A. R. (2012). The impact of
self-directed learning strategies on reading comprehension. International Journal of
Scientific & Engineering Research 3(7), 1–9. https://doi.org/10.5539/elt.v5n7p129
Klauda, S., & Guthrie, J. (2015). Comparing relations of motivation, engagement, and
achievement among struggling and advanced adolescent readers. Read Write, 28,
239–269. https://doi.org/ 10.1007/s11145-014-9523-2
Khruawan, P. & K. Dennis, N. (2017). A study of English reading comprehension using
content-based instruction approach. International Journal of Research -
Granthaalayah, 5(1), 368–375. https://doi.org/10.5281/zenodo.266416.
Kongvimon, W. (2016). The development of electronic books (E-Book) on Photoshop programme
for instructional media production for primary level teachers [Master’s Thesis, Burapha
University].
Korat, O. (2010). Reading Electronic Books as a Support for Vocabulary, Story
Comprehension, and Word Reading in Kindergarten and First Grade. Computers
and Education 55(1), 24–31. https://doi.org/10.1016/j.compedu.2009.11.014
Kottong, K. (2014). The development of English reading exercises based on Asian folktales through
the storyline method for prathomsuksa six learners of Banpao (Samranchaiwittaya) school.
[Master’s Thesis, Silpakorn University]. http://www.thapra.lib.su.ac.th.
Larson, L. (2010). Electronic reading workshop: Beyond books with new literacies and
instructional technologies. Journal of Adolescent and Adult Literacy, 52, 121–131.
https://doi.org/10.1598/jaal.52.2.3
127
Linik, J. R. (2012). Literacy 2.0: Teaching learners the skills needed to succeed in our
information economy. Education Digest, 78(3), 25–29.
Mangen, A., Bente, R., & Bronnick, K. (2013). Reading linear texts on paper versus
computer screen: Effects on reading comprehension. International Journal of
Educational Research, 58, 61–68. https://doi.org/10.1016/j.ijer.2012.12.002
Mckeown, M. G., Beck, I. L., & Blake, R. (2009). Rethinking reading comprehension
instruction: A comparison of instruction for strategies and content approaches.
Reading Research Quarterly, 44(3), 218–253. https://doi.org/10.1598/rrq.44.3.1
Mcnamara, D. S. (2004). SERT: Self-explanation reading training. Discourse Processes, 38(1),
1–30. https://doi.org/10.1207/s15326950dp3801_1
Mcnamara, D. S. (2007). Reading comprehension strategies: Theories, Interventions and
Technologies. New Jersey: Erlbaum.
Nagi, K. (2012). Learning English language in Thailand: Hype or Necessity? The Nation.
http://www.nationmultimedia.com/opinion/Learning-English-language-in-
Thailand-Hype-or-Nece-30179150.html.
Oakley, G. (2011). The assessment of reading comprehension cognitive strategies:
Practices and perceptions of Western Australian teachers. Australian Journal of
Language & Literacy, 34, 279–293. http://www.alea.edu.au/documents/item/265
Osanai, D. (2000). Differences in language learning strategies between male and female, and also
between Asian and Latino ESL learners [Doctoral dissertation. University of Texas,
Austin].
Park, H., & Kim, D. (2011). Reading-strategy use by English as a second language learners
in online reading tasks. Computers & Education, 57(3), 2156–2166.
Pastore, M. (2008). 30 benefits of e-books. Epublishers Weekly, January 28, 2008.
http://epublishersweekly.blogspot.com/2008/02/30-benefits-ofebooks. html
Reid, C. (2016). eBooks and Print Books Can Have Different Effects on Literacy Comprehension.
https://www.semanticscholar.org/paper/eBooks-and-Print-Books-Can-Have-
Different-Affects Reid/2e98a77a9438d086a05d1ce1c7ef837c2376b443
Sackstein, S. Spark, L., & Jenkins, A. (2015). Are e-Books effective tools for learning?
Reading speed and comprehension: ipad vs. paper. South African Journal of
Education, 35(4), 1–14. https://doi.org/10.15700/saje.v35n4a1202
Shang, H-F. (2015). An investigation of scaffolded reading on EFL hypertext
comprehension. Australasian Journal of Educational Technology, 31(3), 293–312.
https://doi.org/10.14742/ajet.1735
Shakeri, H. (2012). The relationship among EFL learners’ autonomy, tolerance of ambiguity,
reading strategies, and reading comprehension [Master’s Thesis, Islamic Azad
University].
Short, L. A. (2010). Using electronic books to increase elementary learners’ motivation to read
[Doctoral dissertation, Dominican University of California, San Rafael].
Singha-o-pas, S., Nuchmee, K. C., & InThanin, W. T. (2016). Development of an electronic
book on information technology for education for undergraduate learners at
Thaksin University. Valaya Alongkorn Review (Humanities and Social Science), 6(2),
79–91. https://doi.org/10.18178/ijke.2016.2.4.071
Szabo, S., & Long, D. (2016). E-readers and the effects on learners’ reading motivation,
attitude, and comprehension during guided reading. Cogent Education, 3.
http://dx.org/10.1080/2331186x.2016.1197818.
Teerapaksiri, K., Watanabut, L., & Sopa, N. (2017). Development of English
communication skills for the ASEAN Community by using electronic books for
mattayom suksa 4. The 3rd National Conference on Technology and Innovation
Management, 3,1-7. https://doi.org/10.23977/etmhs.2017.1011
128
Tracey, D. H., & Morrow, L. M. (2012). Lenses on reading: an introduction to theories and
models (2nd ed). New York: The Guilford Press.
Uamduang U. (2012). The development of English for career reading exercises for first-
year electrical engineering technology learners [University of Technology,
Rattanakosin].
Vaish, V. (2016). Learner Engagement in Reading. Education Innovation Series, 135–151.
http://link.springer.com/chapter/10.1007%2F978-981-287-967-7_9
Zhang, L. J. (2008). Constructivist pedagogy in strategic reading instruction: Exploring
pathways to learner development in the English as a Second Language (ESL)
classroom. Instructional Science: An International Journal of the Learning Sciences,
36, 89–116. https://doi.org/10.1007/s11251-007-9025-6
129

Vol. 21, No. 8, pp. 129-143, August 2022
Received Jun 12, 2022; Revised Aug 11, 2022; Accepted Aug 24, 2022
The Role of Mother’s Education and Early Skills

in Language and Literacy Learning
Opportunities
Dyah Lyesmaya*
Primary Education Department, Universitas Pendidikan Indonesia, Bandung,
Indonesia
Bachrudin Musthafa
English Language Education Department, Universitas Pendidikan Indonesia,
Bandung, Indonesia
Dadang Sunendar
Indonesian Language Education Department, Universitas Pendidikan Indonesia,
Bandung, Indonesia
Abstract. The purpose of this study was to examine the relationship

between language and literacy learning opportunities, early skills,
socioeconomic status, and mother’s education in children’s early
education. In addition, we attempted to determine the language domains
needed for training as well as the ideal duration for language and literacy
learning for children. We used a mixed methods design to assess early
childhood language and literacy skills. The study involved 2550 early
childhood learners, 235 teachers, and parents of learner participants from
112 kindergartens. The standard measure of language proficiency was
used to measure the learners’ early language and literacy skills. Mother’s
education and learners’ early abilities were positively correlated with
learners’ language and literacy learning opportunities. Learners who
have mothers with a high level of education receive significant
opportunities to learn language and literacy compared to those with
mothers with a low level of education. Therefore, their language skills are
superior. There are 12 language domains needed for training in early
childhood and the ideal duration of language learning for children is 26
minutes per day or 182 minutes a week. The language domains are
determining factors for obtaining reading skills in the future. Mother’s
education and child’s early skills really help children in language and
literacy learning. The implication of this research is that literacy programs
are needed for early childhood learners who have low initial abilities and
children with mothers with a low level of education.
©Authors
130
Keywords: early skills; literacy and language learning; socioeconomic

status; mother’s education
1. Introduction
Early childhood education requires the introduction of various types of language
knowledge in order to improve the oral language and literacy skills of children.
This includes knowledge of the alphabet, phonology, practical vocabulary, and
printed letters. These abilities are a determining factor for children’s success in
obtaining reading skills in the future (Beisly et al., 2020; Farley & Piasta, 2020).
Evidence has reinforced that those children who show good early literacy skills
have the potential to become good readers, whereas children who have poor early
literacy skills tend to have difficulty in reading in the future. Children’s success in
language and literacy learning at an early age is also influenced by several factors,
including socioeconomic status. Aspects included in determining socioeconomic
status are income, social class, occupation, and education (Aarnoutse et al., 2005;
Aram, 2008; Hammer et al., 2020). This research related to language and literacy
learning, parental education, and early skills of children is very important because
it can assist stakeholders in drafting policy to support the success of children’s
language and literacy learning in the family. It also provides an overview of the
role of parents in supporting the success of children’s language and literacy
learning. The implication of this research is that teachers, parents, and
stakeholders may know the standard measure of language learning and early
literacy that must be possessed by early childhood. In addition, this standard
measure might be useful for policy makers who will conduct literacy programs.
Several previous studies have proven that a better socioeconomic status of

children contributes to their ability to learn language and early literacy (Alshatti
et al., 2020; Hunkin, 2021). Children’s language learning difficulties can be
observed from the pre-school period and will continue until school age. As such,
it is necessary to equip the parents with some early language recognition actions
in a regular and patterned manner (Bailey & Bulotsky-Shearer, 2022; Bohlmann &
Downer, 2016). Several studies have examined language learning and early
literacy in early childhood. These include the research of Cabell et al. (2021), which
proved that the level of children’s early skills in spoken language and reading
ability in early childhood greatly determines their writing ability when they enter
elementary school in the early grades. In addition, Lee-Hammond and McConney
(2017) examined the impact of early literacy programs on early childhood in rural
areas and compared it with the early literacy abilities of early childhood learners
in urban areas. Their findings showed that children’s early literacy skills in
literacy programs are superior for reading literacy and numeracy literacy despite
living in rural areas (Bracken & Fischel, 2008; Hadianto et al., 2022). This proves
that targeted interventions can be effective in improving children’s early literacy
skills. Farley and Piasta’s research (2019) has also proven that early childhood
literacy is positively correlated with maternal education. However, these studies
did not explore the relationship of early childhood literacy with socioeconomic
status and did not explore language skills in depth. The difference between the
current study and previous studies is that the current study explored early
childhood language and literacy skills that are associated with the ability of the
131
mother when considering socioeconomic status and education. In addition, this

study also determined the standard measure of language learning and early
literacy that must be possessed by early childhood. This standard measure is
useful for policy makers who will conduct literacy programs.
The purpose of this study was to look at early childhood language and literacy
skills as related to the mother’s education and the socioeconomic status of the
family. This study reveals the components of language and literacy learning and
the ideal duration that is needed for early language and literacy learning in early
childhood. Several previous studies have shown that socioeconomic status
contributes to children’s language and literacy skills, but in this study, we
correlated it with the mother’s education variable. This provides knowledge to
stakeholders in determining what early literacy skills should be conveyed to
children and makes it easier for stakeholders to determine which children need
early literacy programs. Several research questions have been formulated to
address this problem:
1. What is the relationship between mother’s education, early abilities, and
socioeconomic status with language learning abilities and early literacy in
early childhood?
2. What are the domains of language learning and early literacy that need to
be applied to children both at school and by their families at home?
3. What is the ideal duration for the ideal early childhood language and
literacy learning to take place?
2.1 Opportunities to Learn Language and Literacy in Early Childhood
Many researchers have studied early childhood programs in various countries,
finding that the programs have a positive impact on future academic abilities
(Jones, 2014). One such program is an early literacy program that is applied to
children of low socioeconomic status and has received government attention in
developed countries. This program can improve the literacy skills of children
whose language and literacy skills are delayed so that they have adequate abilities
when they enter school age (Gibson, 2008; Shemwell & Furtak, 2010). The idea of
learning opportunities first emerged in 1999 to assist children who are less
academically capable. The term learning opportunities was then developed and
applied in early childhood schools to increase students’ learning opportunities in
addition to what they receive at home with their parents. Teachers should be able
to adapt instructional learning in early childhood if learners have academic
abilities that lag behind that of other learners (Bathgate et al., 2015; Cinici, 2016).
Educational institutions rarely ensure implementation of these different methods,
making it necessary to equalize academic abilities in early childhood by providing
many language learning opportunities at home.
Greater early language and literacy learning opportunities become very

important for young children of low socioeconomic status considering they
receive less language and literacy experience at home. This causes their abilities
in school to be less developed than that of their peers who are of higher
socioeconomic status (Gilles & Buck, 2020; Hadianto et al., 2021a, 2021b; Koffman
132
et al., 2017). Such cases often occur in schools in Indonesia, considering that there
are still many people who are at a low socioeconomic level and parents who do
not have the ability to acquire a higher education. This needs to be rectified with
the literacy program for early childhood (Curenton & Justice, 2008; Gonzalez et
al., 2017). Such literacy program needs to be supported by research to serve as a
guide indicating the components of language learning and the ideal time by which
learners in early childhood education must possess these and the ideal level they
must be at. This concept is reinforced by research showing that children from
families of lower socioeconomic status start kindergarten with fewer early literacy
skills (Haak et al., 2012; Hartas, 2012). However, the standard deviation between
these children and those of a higher socioeconomic status is 0.5–1 (Chatterji, 2006).
It can thus be concluded that the intensity at which children learn language and
literacy is based on the language learning they receive both in their school
environment and at home with their parents.
2.2 Socioeconomic Status

Socioeconomic status refers to the social and economic level of the family in the
community. This includes profession, family income, and education level of
parents (Hu et al., 2019; Janus et al., 2019). The socioeconomic status greatly
determines the pattern of language and literacy learning that mothers follow for
their children’s education at home. Several previous studies have proven that
socioeconomic status contributes to children’s academic abilities, including
language skills and early literacy, showing that socioeconomic mapping of the
community is an important component that the government must consider in
implementing a literacy program (Kalia & Reese, 2009; Krahe et al., 2021).
Results of previous research that examined the provision of opportunities to learn

mathematics and science to children of low socioeconomic status showed that
children of high socioeconomic status performed better. This is due to the better
teaching programs provided by parents of high socioeconomic status (Lee-
Hammond & McConney, 2017; Lewis et al., 2016). Other evidence has shown that
children of low socioeconomic status receive limited language learning
opportunities and that their learning is dominated by rote activities, so that their
creativity and critical thinking abilities are not explored (Marcella et al., 2014;
Mashburn et al., 2010). The research has also shown that language and literacy
learning opportunities are highly variable, not only so between classes but also
within the same class. Previous research has shown that learners can get 0 to 81
minutes of early language and literacy learning opportunities per day regardless
of socioeconomic aspects. This is not impossible if educators provide language
learning opportunities according to children’s needs. In different teaching
frameworks, teachers are expected to be able to adapt teaching to the early skills
of early childhood. Thus, teachers should provide more language learning
opportunities for children who have low initial skills.
2.3 Early Skills

Early skills are early childhood language and literacy skills that are acquired
naturally in the family. These early language and literacy skills include the ability
to recognize the alphabet, phonological abilities, and early childhood oral
language skills (Neumann et al., 2013; Poolman et al., 2017). Alphabet knowledge
133
includes names and pronunciations, phonological abilities include the ability to

pronounce letters and words, while oral language skills are children’s vocabulary
knowledge. Several studies have specifically examined the differences in
language learning opportunities and early childhood literacy provided to children
of various levels of socioeconomic status or early skills (Reilly et al., 2020; Skibbe
et al., 2013). This study examines the relationship between the variables of family
socioeconomic status, mother’s education, and children’s early skills.
Other studies have shown that children from low-income families are more likely
to use individualized instruction in the learning process. These children usually
receive fewer language learning opportunities than children in other
socioeconomic groups in the class. Children whose mothers have higher levels of
education tend to be able to follow group instruction and work independently.
These children also tend to receive better teacher response and support. The same
is true in the early literacy skills of children. Children who have lower initial skills
are also more suited to individualized instruction. So, it can be concluded that the
children who most need the opportunity to learn language are those who have
fewer initial skills (Slot et al., 2018; Varghese & Wachen, 2016).
3. Methodology
We used mixed methods research to study the relationship between the variables
of language and literacy learning opportunities, mother’s education, and the
child’s early skills. We used mixed methods research to combine quantitative data
strengthened by qualitative data. Quantitative data were gathered on the
relationship between variables and are described by inferential statistics, while
qualitative data were gathered from observations and questionnaires to
strengthen and complete the quantitative data. This study involved 2550 early
childhood learners, 235 teachers, and parents of learner participants. The early
childhood learners who were involved in this study were 4–6 years of age and
attended 112 kindergarten schools in the city of Bandung, Indonesia. Sample
selection of the early childhood learners was done using a purposive sampling
technique mapped based on the variables that were to be studied. Teachers and
parents of learners were involved in this study to assist us in analyzing language
and literacy learning opportunities as well as the learners’ language skills both at
home and at school. The average age of the teachers was 42 years (SD: 10.71) and
their average teaching experience was 10 years (SD: 8.12). Regarding education
level of the teachers, 20% had a diploma, 50% had a bachelor’s degree, and 18%
had a master’s degree. The participating schools were from several areas in
Bandung, including urban (30%), suburban (30%), and rural areas (40%). The
learners had a median age of 5.4 years (SD: 0.42) and were almost equally divided
according to gender (51% male; 49% female). The average family income was
between IDR 2 million and IDR 10 million per month.
3.1 Data Collection Tools and Procedure

Classroom observations mere made to determine the opportunities to learn
language in the classroom. In addition, a researcher-made questionnaire was used
to determine the mothers’ level of education and family socioeconomic status. The
instrument was made by adopting the language domains as presented in Table 1.
134
The validity test was carried out through the judgment of five PhD-qualified
experts, while the reliability test was carried out through an empirical test on
some learners who became research participants.
3.1.1 Opportunity to learn language and literacy

Observation of language learning opportunities was carried out in the classroom
and at home. Observation at home was carried out by asking parents’ help in
filling out an observation grid. Observation of the teaching process in the
classroom was carried out directly and facilitated using a camera for two
semesters within one year. Observations were made every day to obtain an
overview of language learning practices. The scale used to measure the
vocabulary of spoken language, ranging from vocabulary that is commonly used
to that which is not commonly used by children, ranged from 1 to 5. The average
score on this assessment was 5.21 (SD: 0.74). Each class had an average duration
of one day (M: 82;27 minutes; SD: 22.57; range: 30–60 minutes). Our classroom
observation was facilitated by using a camera to document teaching activities. In
addition, we also used observation notes to observe language and literacy
learning opportunities in the classroom. The components of the assessment of
language learning and early literacy were taken from the theory showing what
level of language learning and early literacy must be possessed by early
childhood. We used these components as instructions to assess the outcomes of
early language and literacy learning as a guide. Table 1 presents the components
and instructions followed for the early language and literacy learning assessment.
Table 1. The language domains assigned for the teaching process and evaluation
Domain Method Example
Phoneme awareness Distinguishing the Producing rhyming or alliterative
pronunciation of the letters phrases
individually or in words
Morpheme Introducing basic and Adding prefixes to alternate
awareness compound words and affixes phrase meanings
Word identification/ Spelling the letters in the Reading sight phrases from a
decoding word phrase wall
Word identification/ Spelling the letters in the Spelling through matching
encoding word sounds and letters
Grapheme-phoneme Listening or reading word Identifying letters related to a
correspondence for word couple of sounds
Fluency Practicing the quick Reading sight phrases with
pronunciation of words increased speed
Print and text Writing printed letters and Modeling ebook orientation and
concepts sounding them directionality
Spoken language Speaking and introducing Using role-play conversation to
new vocabulary practice the use of vocabulary
phrases that belong to the
language domain
Pattern vocabulary Writing new words and Defining homonyms and noting
understanding their spelling patterns
meanings
135
Understanding Reading text and Using earlier expertise to better

understanding their recognize an ebook
meanings
Text reading Reading text Listening to an ebook on tape
Writing Writing simple sentences Participating in shared writing
from familiar words throughout class time
After the observations had been made, the learning videos were analyzed by
coding each action using the individualizing student instruction (ISI) coding
scheme. The ISI code consists of 12 language learning and early literacy domains,
namely phoneme awareness, morpheme awareness, word
identification/decoding, word identification/encoding, grapheme-phoneme
correspondence, fluency, print and text concepts, spoken language, pattern
vocabulary, understanding, text reading, and writing (see Table 1). We calculated
the time used for early language and literacy learning that included the 12
domains in the classroom and at home of learners who were observed with the
help of their parents and assessed the results at the end of the semester. This was
done to determine whether the opportunity to learn a language within a certain
timespan was ideally used and whether the results of the assessment were related
to the socioeconomic status variables and the education level of the learners’
mothers.
3.1.2 Mother’s education

In this study, we used mother’s education level and socioeconomic status as
variables that influenced the level of language learning ability and early literacy.
As has been done in previous studies, the mother’s level of education is a variable
that can predict the success of the child’s academic level and is more predictive
than the level of family socioeconomic status. In this study, 10% of the mothers of
learners who were participants were admitted to higher education but did not
finish, 15% had a master’s degree, 40% had a bachelor’s degree, 30% had a
diploma, and 5% had a doctoral degree.
3.1.3 Initial skills level

To determine the learners’ early language skills, we conducted a series of tests in
collaboration with participating schools. The components tested included
knowledge of the alphabet and phonological and spoken skills. Each of these skills
was analyzed and associated with the level of learners’ reading ability. The
components assessed are in line with the components of the ISI scheme in the 12
standard domains of language learning and early literacy.
3.1.4 Alphabet knowledge

We assessed learners’ alphabet knowledge by testing their usage of uppercase and
lowercase letters and pronunciation. The test sheet had the 26 letters of the
alphabet arranged randomly, with one point awarded for each correct letter, for a
total of 55 points for the maximum score. Internal consistency for the composite
scale was high (0.98).
3.1.5 Phonological awareness

We assessed learners’ phonological abilities through rhyming and repetition
word tests. Assessment was done by instructing the learners to identify a picture
136
and give it a name. Learners were asked to choose words that do not rhyme and
words that do not start with the same vowel during the alliteration test. One point
was assigned for each correct answer and there were a possible twenty-four points
to be received in total. Internal consistency for each subtest was high (0.85).
3.1.6 Oral language skills

Learners’ oral language skills were assessed through sentence, phrase, and
vocabulary tests. For the sentence test, learners were asked to choose a picture that
matched the teacher’s oral explanation. For the word structure test, learners were
asked to complete the missing sentences. For the vocabulary test, learners were
asked to identify the object and provide a description of the word that
corresponds to the object. One point was awarded for each correct answer. The
total score for the 3 subtests was 70 points. Internal consistency for the subtests
ranged from 0.8 to 0.97.
4. Results
To answer the first research question, we tested the relationship between mother’s
education level, initial skills, and opportunities to learn language and literacy, as
shown in Table 2.
Table 2. Relationship between, mother’s education, early skills, and opportunity to

learn language and literacy
1 2 3 4 5 n M SD Min Max
Alphabet – 2550 26.71 19.37 0 55
knowledge
Phonological .405** – 1606 8.51 4.63 0 30
awareness
Spoken .450** .465** – 1732 40.16 11.51 0 70
language
Mother’s .367** .245** .315** – 1762 3.45 1.57 0 9
education
Knowledge .121** .086** .088** .051* – 1754 15.07 9.31 0 65
opportunities
Analysis revealed that the learners’ early skills showed a significant relationship
with language learning opportunities. Likewise, mother’s education had a
significant relationship with spoken language and knowledge opportunities.
Language learning opportunities showed a significant relationship with mother’s
education and early skills. On average, learners whose mothers had a higher
education and who had better early skills tended to show better language skills
(r2 0.02; see Table 3). Table 3 shows that mother’s education and child’s early skills
greatly affect spoken language, alphabet knowledge, and phonological
awareness. The results of this study indicate that although children are given the
opportunity to learn language and literacy at school in the same amount and time,
the results will vary depending on the mother’s education and the child’s initial
skills that had been obtained at home.
137
Table 3. Processing results based on the linear hierarchy model to predict language
and literacy learning opportunities
Phonological
Oral language Alphabet knowledge
awareness
Parameter
Coeffi SE Coeffi SE Coeffi SE
cient cient cient
Intercept (b00) 15.12 *** 0.50 15.02 *** 0.50 15.19 *** 0.42
Classroom level
Mother’s education _0.14 0.35 _0.06 0.32 _0.15 0.33
(b01)
Learner level
Early skill (b10) 0.05 ** 0.05 0.02 0.02 0.05 0.04
Mother’s education 0.22 * 0.11 0.25 * 0.11 0.24 * 0.12
(b20)
Alteration
components
Learner level ® 17.19 4.08 16.75 4.11 17.25 4.25
Classroom level 68.36 *** 9.22 71.36 *** 9.22 69.13 *** 9.15
(u0)
SE = Effective contribution; *** p < .001; ** p < .01; * p < .05
For the second research question, data from observations in the classroom and at
learners’ homes and combined with the theory of language skills and early
literacy in early childhood were used. The data were used to determine the
domains of language learning and early literacy that are needed to train learners
both through literacy programs and learning patterns carried out at home. These
domains were phoneme awareness, morpheme awareness, word
identification/decoding, word identification/encoding, grapheme-phoneme
correspondence, fluency, print and text concepts, spoken language, pattern
vocabulary, understanding, text reading, and writing. Learners’ ability to
differentiate between phonemes can be improved by distinguishing the
pronunciation of the letters themselves or in words. Improving the ability to
identify morphemes can be done by introducing basic and compound words and
affixes. The ability to decipher words can be improved by spelling the letters in
the word. Similarly, the ability to identify words can be achieved by spelling the
letters in the word. Grapheme-phoneme skills can be practiced by listening or
reading word for word. Fluency can be improved by practicing the quick
pronunciation of words. To introduce the concept of print and text, learners can
be trained by writing printed letters and sounding them. To improve spoken
language skills, learners can be introduced to new vocabulary and practice
speaking it. Furthermore, written vocabulary can be increased by training learners
in writing new words and understanding their meanings. Lastly, to enhance
writing text skills, learners can be trained to write simple sentences from familiar
words.
To answer the third research question, we observed opportunities to learn

language and literacy in the classroom and at home. The learners had the
opportunity to learn language while in the school environment for an average of
80 minutes per day, whereas their opportunity to learn language at home differed
remarkably. Regarding language learning opportunities obtained while at home,
138
the results showed that learners with mothers with a low level of education only
received additional language learning opportunities of an average of five minutes
per day. To improve spoken language skills, these learners received an additional
10 minutes of language learning opportunities per day. This is low when
compared to learners with highly educated mothers, who received an additional
30 minutes per day, totaling 210 minutes per week. The average amount of time
for language learning a child generally needs is 26 minutes per day or 182 minutes
a week. Of course, it is not surprising that children with mothers who have higher
levels of education have different language learning patterns that are applied to
them every day.
5. Discussion
This study sought to determine the relationship between the variables of language
and literacy learning opportunities, mother’s education, and early skills by
providing knowledge of language and literacy learning in schools and ending
with an evaluation. The evaluation used included the 12 domains of language
learning and early literacy that have been mentioned previously. The results
showed that the mother’s education and the child’s early skills greatly affect the
child’s language skills, even though the material and implementation method are
the same. Learners with highly educated mothers tend to have superior language
skills. This is due to the learning pattern applied by the mother at home. In
addition, based on the questionnaire, the results showed that the duration of
language learning for learners whose mothers are highly educated is longer than
that of learners with mothers with lower levels of education. Based on the theory,
learners have the opportunity to learn language both in the family and at school
for an average of 85 minutes per day (Poolman et al., 2017; Reilly et al., 2020). This
skills gap must be overcome by the pre-school literacy program, which must be
able to equip parents on how to train their children in language skills and early
literacy in their home environment (Krahe et al., 2021; Lane et al., 2014). Of course,
this can help better prepare children for school. Stakeholders must include
children who have mothers with lower levels of education into the equalization
program for language and literacy learning (Poolman et al., 2017; Reilly et al.,
2020).
The results of this study also indicated that children with low levels of maternal
education and early skills have more difficulty in understanding material and
learning in the school environment (Meng, 2015; Neumann et al., 2013). Children’s
oral language ability is another indicator of poor language skills. Oral language
skills are basic skills and abilities that will greatly determine a child’s success in
reading and other academic abilities. The results showed that children belonging
to the lower socioeconomic groups tended to lag behind in their oral language
skills. In addition, it also tended to be difficult to improve their oral language skills
through classroom instruction (Skibbe et al., 2013; Slot et al., 2018). Of course,
these skills need to be promoted through the learning pattern that must be applied
by the mother at home. The findings of this study are in accordance with the real
situation in society. Children with mothers who have higher levels of education
and have a high ability for communication will have better language and literacy
skills. The findings of this study are also in accordance with previous studies
which reinforce that the mother’s level of education has a very important role in
139
the success of children in language learning (Gilles & Buck, 2020; Koffman et al.,
2017).
The findings of this study are also in line with that of other studies (Mulyati &
Hadianto, 2022; Slot et al., 2018). However, it should be noted that language and
literacy learning opportunities in schools always favour children of higher
socioeconomic status. One alternative to overcome this problem, as recommended
by literature, is that pre-school education should be done individually (Varghese
& Wachen, 2016; Wasik & Hindman, 2010). Needless to say, this will require quite
a lot of teachers. One alternative option is that pre-school education should be
grouped based on factors that greatly affect learners’ academic abilities, which
factors include maternal education, learners’ initial abilities, and socioeconomic
status (Weigel et al., 2006; Wickstrom & Pelletier, 2021). The aim here is to provide
children who have fewer language skills with more intensive learning compared
to other groups of children.
The 12 language domains used in this study also need to be considered as main
focus areas in which children require training by their parents and in the school
environment (Varghese & Wachen, 2016; Yazejian & Peisner-Feinberg, 2009). The
domain of language learning and early literacy is believed to be the most decisive
component of children’s success in reading and other academic abilities when
entering elementary school. The findings of this study concur with this, namely
that children who have more opportunities to learn language have superior oral
language skills (Reilly et al., 2020; Skibbe et al., 2013). These findings reinforce the
fact that the more intensively children are introduced to language and literacy
learning at an early age, the better their academic abilities will be at school age.
These findings are reinforced by literature (Lewis et al., 2016; Meng, 2015). This
research has the short-term goal to strengthen and clarify the role of parents and
teachers in supporting the success of children’s language and literacy learning in
terms of domain, method, and duration. The long-term goal is that the findings of
the study provide guidelines for literacy-program makers to improve the literacy
skills of children starting from an early age. These programs will certainly have
an impact on the literacy level of children when they reach adulthood.
6. Conclusions, Limitations, and Recommendations

Mother’s education, early skills, and socioeconomic status are determining factors
in a child’s success in acquiring language and literacy skills. Therefore, it is
necessary to map the mother’s education, the child’s initial skills, and
socioeconomic status as material for implementing early education in schools and
education through literacy programs outside of school. In language learning and
early literacy, attention should also be paid to the 12 domains identified in this
study, as these are the basis of children’s success in the future. The number of
language learning opportunities also greatly determine children’s oral language
skills; hence, individual instruction is needed that provides language learning
opportunities for children with mothers with a low level of education. Promoting
the language and literacy learning of children both in school and at home can instil
character values in children. This can be done by the mother when giving advice,
during study time, or through casual dialogue. Therefore, the mother’s role is very
140
important in the success of children in language and literacy learning. Mothers

must be active in providing language learning opportunities by paying attention
to the language domains in this study. Opportunities can be created in various
situations, whether through chatting, studying, or playing.
This study had several limitations, including linking children’s language skills
with only the variables of mother’s education and socioeconomic status. Perhaps
there are other factors that also determine the success of children’s language
learning, so research can thus be expanded to include other variables.
Furthermore, participants were from only one area in Indonesia, and it would
have been better had children from various regions in Indonesia been involved.
Third, it is possible that we excluded other important aspects of instructional
opportunities in relation to mother’s education. Despite these limitations, the
main finding of this study was that the language and literacy learning
opportunities provided during early childhood are largely determined by the
mother’s level of education and early spoken language skills. In addition, through
this study, we also formulated the language domains and estimated duration
needed to assist parents in their children’s language learning. For future research,
we recommend adding more variables and selecting a wider range of participants
to identify the factors that determine the success of children’s language learning.
7. References
Aarnoutse, C., Van Leeuwe, J., & Verhoeven, L. (2005). Early literacy from a longitudinal
perspective. Educational Research and Evaluation, 11(3), 253–275.
https://doi.org/10.1080/08993400500101054
Alshatti, T., Al-Sulaihim, N., & Abdalla, F. A. (2020). Home-based literacy practices of
Arab mothers from Kuwait. Speech, Language and Hearing, 23(3), 133–145.
https://doi.org/10.1080/2050571X.2019.1581464
Aram, D. (2008). Parent-child interaction and early literacy development. Early Education
and Development, 19(1), 1–6. https://doi.org/10.1080/10409280701838421
Bailey, J., & Bulotsky-Shearer, R. J. (2022). Learning behaviors mediate associations
between parent-reported peer play skills and literacy and mathematics skills for
low-income preschool children. Early Education and Development, 33(3), 523–540.
https://doi.org/10.1080/10409289.2021.1904625
Bathgate, M., Crowell, A., Schunn, C., Cannady, M., & Dorph, R. (2015). The learning
benefits of being willing and able to engage in scientific argumentation.
International Journal of Science Education, 37(10), 1590–1612.
https://doi.org/10.1080/09500693.2015.1045958
Beisly, A., Kwon, K. A., & Jeon, S. (2020). Executive function and learning behaviors:
Associations with academic skills among preschoolers. Early Child Development
and Care, 190(15), 2469–2483. https://doi.org/10.1080/03004430.2019.1585347
Bohlmann, N. L., & Downer, J. T. (2016). Self-regulation and task engagement as predictors
of emergent language and literacy skills. Early Education and Development, 27(1),
18–37. https://doi.org/10.1080/10409289.2015.1046784
Bracken, S. S., & Fischel, J. E. (2008). Family reading behavior and early literacy skills in
preschool children from low-income backgrounds. Early Education and
Development, 19(1), 45–67. https://doi.org/10.1080/10409280701838835
Cabell, S. Q., Lomax, R. G., Justice, L. M., Breit-Smith, A., Skibbe, L. E., & McGinity, A. S.
(2010). Emergent literacy profiles of preschool-age children with specific language
impairment. International Journal of Speech-Language Pathology, 12(6), 472–482.
https://doi.org/10.3109/17549507.2011.492874
141
Chatterji, M. (2006). Reading achievement gaps, correlates, and moderators of early

reading achievement: Evidence from the Early Childhood Longitudinal Study
(ECLS) kindergarten to first grade sample. Journal of Educational Psychology, 98(3),
489–507. https://doi.org/10.1037/0022-0663.98.3.489
Cinici, A. (2016). Balancing the pros and cons of GMOs: Socio-scientific argumentation in
pre-service teacher education. International Journal of Science Education, 38(11),
1841–1866. https://doi.org/10.1080/09500693.2016.1220033
Curenton, S. M., & Justice, L. M. (2008). Children’s preliteracy skills: Influence of mothers’
education and beliefs about shared-reading interactions. Early Education and
Development, 19(2), 261–283. https://doi.org/10.1080/10409280801963939
Farley, K. S., & Piasta, S. B. (2020). Examining early childhood language and literacy
learning opportunities in relation to maternal education and children’s initial
skills. Journal of Education for Students Placed at Risk, 25(3), 183–200.
https://doi.org/10.1080/10824669.2019.1689506
Gibson, K. (2008). Analogy in scientific argumentation. Technical Communication Quarterly,
17(2), 202–219. https://doi.org/10.1080/10572250701878868
Gilles, B., & Buck, G. (2020). Preservice teachers’ use of discourse to shape the construction
of scientific arguments. Journal of Science Teacher Education, 31(3), 291–310.
https://doi.org/10.1080/1046560X.2019.1696005
Gonzalez, J. E., Acosta, S., Davis, H., Pollard-Durodola, S., Saenz, L., Soares, D.,
Resendez, N., & Zhu, L. (2017). Latino maternal literacy beliefs and practices
mediating socioeconomic status and maternal education effects in predicting child
receptive vocabulary. Early Education and Development, 28(1), 78–95.
https://doi.org/10.1080/10409289.2016.1185885
Haak, J., Downer, J., & Reeve, R. (2012). Home literacy exposure and early language and
literacy skills in children who struggle with behavior and attention problems.
Early Education and Development, 23(5), 728–747.
https://doi.org/10.1080/10409289.2011.565721
Hadianto, D. S., Damaianti, V., Mulyati, Y., & Sastromiharjo, A. (2022). Effectiveness of
literacy teaching design integrating local culture discourse and activities to
enhance reading skills. Cogent Education, 9(1), 2016040.
https://doi.org/10.1080/2331186X.2021.2016040
Hadianto, D., Damaianti, V. S., Mulyati, Y., & Sastromiharjo, A. (2021a). Does reading
comprehension competence determine level of solving mathematical word
problems competence? Journal of Physics: Conference Series, 1806(1), 012049.
https://doi.org/10.1088/1742-6596/1806/1/012049
Hadianto, D., Damaianti, V. S., Mulyati, Y., & Sastromiharjo, A. (2021b). Enhancing
scientific argumentation skill through partnership comprehensive literacy. Journal
of Physics: Conference Series, 2098(1), 012015. https://doi.org/10.1088/1742-
6596/2098/1/012015
Hammer, C. S., Cycyk, L. M., Scarpino, S. E., Jury, K. A., & Sawyer, B. E. (2020).
Development of the CECER-DLL child and family questionnaire: A new tool for
documenting the language and literacy experiences of Latino/a dual language
learners. International Journal of Bilingual Education and Bilingualism, 25(6), 2018–
2040. https://doi.org/10.1080/13670050.2020.1840507
Hartas, D. (2012). Inequality and the home learning environment: Predictions about seven-
year-olds’ language and literacy. British Educational Research Journal, 38(5),
859−879. https://doi.org/10.1080/01411926.2011.588315
Hu, J., Torr, J., Degotardi, S., & Han, F. (2019). Educators’ use of commanding language to
direct infants’ behaviour: Relationship to educators’ qualifications and
implications for language learning opportunities. Early Years, 39(2), 190–204.
https://doi.org/10.1080/09575146.2017.1368008
Hunkin, E. (2021). Devastating impacts? Investigating ‘edu-quality’ discourse in early
142
childhood policy and its implications. Journal of Education Policy, 36(2), 196–210.
https://doi.org/10.1080/02680939.2019.1690709
Janus, M., Labonté, C., Kirkpatrick, R., Davies, S., & Duku, E. (2019). The impact of speech
and language problems in kindergarten on academic learning and special
education status in Grade three. International Journal of Speech-Language Pathology,
21(1), 75–88. https://doi.org/10.1080/17549507.2017.1381164
Jones, L. (2014). Developing deaf children’s conceptual understanding and scientific
argumentation skills: A literature review. Deafness and Education International,
16(3), 146–160. https://doi.org/10.1179/1557069X13Y.0000000032
Kalia, V., & Reese, E. (2009). Relations between Indian children’s home literacy
environment and their English oral language and literacy skills. Scientific Studies
of Reading, 13(2), 122–145. https://doi.org/10.1080/10888430902769517
Koffman, B. G., Kreutz, K. J., & Trenbath, K. (2017). Integrating scientific argumentation
to improve undergraduate writing and learning in a global environmental
change course. Journal of Geoscience Education, 65(3), 231–239.
https://doi.org/10.5408/16-232.1
Krahe, M., O’Leary, K., Reilly, S., Ellis, K., & Walsh, K. (2021). An assessment of speech,
language, and literacy of children of young mothers attending supported
playgroup. Early Child Development and Care, 192(9), 1–13.
https://doi.org/10.1080/03004430.2021.1891050
Lane, C., Prokop, M. J. S., Johnson, E., Podhajski, B., & Nathan, J. (2014). Promoting early
literacy through the professional development of preschool teachers. Early Years,
34(1), 67–80. https://doi.org/10.1080/09575146.2013.827157
Lee-Hammond, L., & McConney, A. (2017). The impact of village-based kindergarten on
early literacy, numeracy, and school attendance in Solomon Islands. European
Early Childhood Education Research Journal, 25(4), 541–560.
https://doi.org/10.1080/1350293X.2016.1155256
Lewis, K., Sandilos, L. E., Hammer, C. S., Sawyer, B. E., & Méndez, L. I. (2016). Relations
among the home language and literacy environment and children’s language
abilities: A study of head start dual language learners and their mothers. Early
Education and Development, 27(4), 478–494.
https://doi.org/10.1080/10409289.2016.1082820
Marcella, J., Howes, C., & Fuligni, A. S. (2014). Exploring cumulative risk and family
literacy practices in low-income Latino families. Early Education and Development,
25(1), 36–55. https://doi.org/10.1080/10409289.2013.780504
Mashburn, A. J., Downer, J. T., Hamre, B. K., Justice, L. M., & Pianta, R. C. (2010).
Consultation for teachers and children’s language and literacy development
during pre-kindergarten. Applied Developmental Science, 14(4), 179–196.
https://doi.org/10.1080/10888691.2010.516187
Meng, C. (2015). Home literacy environment and head start children’s language
development: The role of approaches to learning. Early Education and Development,
26(1), 106–124. https://doi.org/10.1080/10409289.2015.957614
Mulyati, Y., & Hadianto, D. (2022). Relation between behavioural problems and reading
skills in gender contexts : Forecasting longitudinal data from kindergarten to Fifth
grade. Journal of Educational and Social Research, 12(4), 120–130.
https://doi.org/10.36941/jesr-2022-0101
Neumann, M. M., Hood, M., & Ford, R. (2013). Mother-child referencing of environmental
print and its relationship with emergent literacy skills. Early Education and
Development, 24(8), 1175–1193. https://doi.org/10.1080/10409289.2013.753567
Poolman, B. G., Leseman, P. P. M., Doornenbal, J. M., & Minnaert, A. E. M. G. (2017).
Development of the language proficiency of five- to seven-year-olds in rural areas.
Early Child Development and Care, 187(3–4), 756–777.
https://doi.org/10.1080/03004430.2016.1203787
143
Reilly, S. E., Johnson, A. D., Luk, G., & Partika, A. (2020). Head start classroom features
and language and literacy growth among children with diverse language
backgrounds. Early Education and Development, 31(3), 354–375.
https://doi.org/10.1080/10409289.2019.1661935
Shemwell, J. T., & Furtak, E. M. (2010). Science classroom discussion as scientific
argumentation: A study of conceptually rich (and poor) student talk. Educational
Assessment, 15(3), 222–250. https://doi.org/10.1080/10627197.2010.530563
Skibbe, L. E., Hindman, A. H., Connor, C. M., Housey, M., & Morrison, F. J. (2013). Relative
contributions of prekindergarten and kindergarten to children’s literacy and
mathematics skills. Early Education and Development, 24(5), 687–703.
https://doi.org/10.1080/10409289.2012.712888
Slot, P. L., Bleses, D., Justice, L. M., Markussen-Brown, J., & Højen, A. (2018). Structural
and process quality of Danish preschools: Direct and indirect associations with
children’s growth in language and preliteracy skills. Early Education and
Development, 29(4), 581–602. https://doi.org/10.1080/10409289.2018.1452494
Varghese, C., & Wachen, J. (2016). The determinants of father involvement and
connections to children’s literacy and language outcomes: Review of the
literature. Marriage and Family Review, 52(4), 331–359.
https://doi.org/10.1080/01494929.2015.1099587
Wasik, B. A., & Hindman, A. H. (2010). Understanding the home language and literacy
environments of head start families: Testing the family literacy survey and
interpreting its findings. NHSA Dialog, 13(2), 71–91.
https://doi.org/10.1080/15240751003737885
Weigel, D. J., Martin, S. S., & Bennett, K. K. (2006). Contributions of the home literacy
environment to preschool-aged children’s emerging literacy and language skills.
Early Child Development and Care, 176(3–4), 357–378.
https://doi.org/10.1080/03004430500063747
Wickstrom, H., & Pelletier, J. (2021). Understanding the relation between behavioral
self-regulation, academic learning, and children’s drawings in kindergarten. Early
Child Development and Care, 191(6), 906–920.
https://doi.org/10.1080/03004430.2019.1651304
Yazejian, N., & Peisner-Feinberg, E. S. (2009). Effects of a preschool music and movement
curriculum on children’s language skills. NHSA Dialog, 12(4), 327–341.
https://doi.org/10.1080/15240750903075255
144

Vol. 21, No. 8, pp. 144-162, August 2022
Exploring Assessment Techniques that Integrate

Soft Skills in Teaching Mathematics in Secondary
Schools in Zambia
Chileshe Busaka*
and Science (ACEITLMS), University of Rwanda, Rwanda
Septimi Reuben Kitta

University of Dar es Salaam, Tanzania
Odette Umugiraneza
Rwanda Polytechnic
Abstract. This study investigated the assessment techniques mathematics

teachers use that integrate soft skills in secondary schools in Mazabuka
District in Zambia. A total of 91 teachers, who were purposively selected,
took part in the study, of which 81 completed a questionnaire and four
observers evaluated 124 video-recorded lessons which were taken of 31
teachers. A sequential multi-phase design was used to collect data. The
data were analyzed using frequencies, means, standard deviations and
chi-square statistics. The findings revealed that the assessment techniques
mathematics teachers used did not assess soft skills. In addition,
mathematics teachers’ gender was not found to have influenced teachers’
choice of assessment techniques in the teaching and learning process but
the type of schools where teachers were teaching, though the effect size
was weaker. The consequence of this may be that secondary school
leavers may not be good communicators, innovators, creators and critical
thinkers. Therefore, it is recommended that mathematics teachers be
upskilled on how to assess soft skills in the teaching and learning of
Mathematics if the integration of soft skills is to be realized as espoused
in the Zambia Education Curriculum Framework of 2013.
Keywords: assessment techniques; soft skills; integration; mathematics;

mixed methods
*
Corresponding author: Chileshe Busaka, chilebusaka2006@yahoo.co.uk
©Authors
145
1. Introduction
1.1 Background to the Problem
Assessment is an essential part of the teaching and learning process. Through
assessment evidence of an individual, student’s learning progress is collected,
interpreted and later judgments are made about the students’ accomplishments
following some set standards (Guevara-Bazán et al., 2020). Guevara-Bazán et al.
(2020); the Ministry of Education [MINEDUC] and Rwanda Education Board
[REB] (2015b) also concurred that the purpose of assessment is for monitoring
students’ progress, providing feedback on students’ performance, guidance on
future progressions, promotion, selection and certification. The Curriculum
Development Centre [CDC] (2013e) and Tejeda and Gallardo (2017) stated that
assessment must match the aims of the curriculum so that it supports teaching
and learning. When this is achieved, teachers focus consistently on the intended
learning outcomes, as opposed to teaching to test.
Assessing knowledge, skills, values and attitudes, in totality, are foundational to

having students that will be equipped with relevant competences. However, the
various forms of assessment used in mathematics education are reported to have
failed to provide an insight into what students understand, know and can achieve
(Gallego & Dandis, 2014). This is probably due to technological advancements that
have increased the need for students to develop soft skills, such as problem-
solving, critical thinking, collaboration, communication, and the ability to analyze
and present data orally and in a written format (Dochy, 2001; Gallardo, 2020;
Schwab, 2017; Putra et al., 2020). This has brought about thinking of how to assess
these skills, as they are not normally captured in the standardized tests commonly
administered to measure students’ performance. Thus, mathematics teachers are
required to use assessment techniques, such as rubrics, portfolios, concept maps,
self-assessments, and group assessments, to determine what students know and
where they are in the learning process (Rebecca, 1998; Birgin, 2011).
Lesson observation in mathematics education research has become important

partly because the researcher has an opportunity to get the actual teacher and
student interactions which impact student learning significantly as opposed to
what teachers claim to do in class (Bostic et al., 2021; National Council of Teachers
of Mathematics [NCTM], 2014). Bostic et al. (2021) reviewed 114 peer-reviewed
articles that involved classroom observations between 2000 and 2015 using the
cross-comparative method of research. The authors concluded that to understand
classroom interaction in Mathematics, researchers require methods like
observation that would capture classroom events.
Furthermore, assessment practices are key in the integration of soft skills that
contributes to the individual students’ achievements in all aspects of life. The
Zambia Education Curriculum Framework (ZECF) of 2013 provides for the
development of six soft skills of problem-solving or critical thinking, cooperation,
communication, entrepreneurship, creativity and innovations, and self-
management (CDC, 2013e). Thus, this study examined mathematics teachers’
assessment approaches to establish the extent to which they assessed soft skills in
Mathematics in secondary schools in Mazabuka district in Zambia.
146
1.2 Research Questions

The following were the research questions that guided the study:
i. To what extent do the assessment techniques mathematics teachers use
in secondary schools in Zambia assess soft skills?
ii. Is there any relationship between mathematics teachers’ demographic
factors and the assessment techniques they used in the teaching and
learning of Mathematics?
2.1 Assessment of Soft Skills
The assessment of soft skills proves to be a challenge in the teaching and learning
of mathematics. Similar sentiments are affirmed by Care et al. (2018), who stated
that it was a challenge to measure most human social and non-cognitive
capabilities directly and, hence, constructs such as critical thinking, problem-
solving, communication and collaboration can be measured with tools designed
to capture indicators of these skills. Since most of the soft skills are seen through
behaviors, the challenge is to capture them accurately. Similarly, Mahasneh and
Thabet (2015) also admitted that it was difficult to develop and measure soft skills,
despite it proving increasingly valuable.
The challenges of assessing soft skills in schools are linked with determining the
suitable methods for assessing them. It is therefore imperative to scrutinize the
appropriate strategies for assessing soft skills (Durowoju & Onuka, 2014). It is
argued that the use of rubrics can help in assessing soft skills. In support,
Bargainnier (2003) reiterated that rubrics are tools, which teachers can use to come
to similar conclusions about soft skills. Further, the author contended that rubrics
provide clear descriptions of the work associated with each component, at varying
levels of mastery, and it is an appropriate tool for measuring outcomes or
competencies. Hence, teachers must go a step further to try and address the
challenge by trying different methods of assessing soft skills during teaching and
learning of Mathematics.
It is an indisputable fact that teaching Mathematics to students helps enhance

knowledge, skills, and positive attitudes. It also helps them to connect what they
learn to real-life situations. Furthermore, it enables students to acquire the
competences needed for improved living in society (Holmes & Hwang, 2016;
Moser et al., 1948). It is evident that when teachers adopt appropriate assessment
approaches, it may help in developing students' mathematical soft skills
(Gallardo, 2020; Stewart et al., 2020). It is essential for teachers to know when, how
and what to assess. Teachers should choose the assessment that helps to develop
students’ critical thinking and problem-solving.
Subsequently, soft skills are seen through behavior and they therefore require
appropriate assessment tools to test them. Assessment in formal education
systems is undertaken through written tests and this is too narrow in scope to
evaluate soft skills. However, REB (2015) and Care et al. (2018) advocated that soft
skills can be measured by assessing the stages students go through to find
solutions to a given task, establishing the reasoning behind resolving a situation,
147
and using evidence to determine the students’ advancement towards fulfilling

performance tasks. In addition, the CDC (2013d) highlighted that soft skills are
difficult to measure, and thus require tools designed to capture indicators of these
skills, such as projects, portfolios and performance assessments. Therefore, the
quality assessment of any performance depends on the accurate and reliable
measurement of key performance factors.
2.2 Approaches in Soft Skills Assessment

Assessment is an important tool which is used to determine how teaching and
learning had taken place. In this respect, various researchers and institutions
acknowledge that, besides standardized tests, other forms of assessments are used
to gauge students’ achievement, what they know or can do (CDC, 2013a;
Hamilton et al., 2021; Mustofa et al., 2020). The 2013 revised curriculum
framework in Zambia, other researchers and documents have also suggested
performance tasks, rubrics, assignments, and standard-based projects as
alternative assessment strategies among others (CDC, 2013d, 2013c, 2013b;
Danielson & Marquez, 2016; Perlman, 2003). Thus, the following paragraphs,
present some of the suggested assessment techniques, particularly those that can
be used to assess soft skills in the teaching and learning of Mathematics.
2.2.1 Performance assessment tasks

Performance assessment is a compilation of performance tasks, which are
structured in such a way that stimulus materials and a demand for action are
presented to a student, who generates responses that can be rated for quality using
explicit standards (Stecher, 2010; Tejeda & Gallardo, 2017). In other words,
performance assessment means an assessment that involves an evaluation of
students’ writing, products, or conduct (Danielson & Marquez, 2016).
Precisely, performance assessment includes all assessments except for
multiple-choice, matching, true/false testing, or problems with a single
correct answer. Classroom-based performance assessment includes all
such assessment that occurs in the classroom for formative or summative
purposes and is evaluated by teachers as distinct from large-scale, state-
wide testing programs (Danielson & Marquez, 2016, p. 1).
Additionally, performance assessments provide an effective way to measure

students’ abilities, such as communication, collaboration, thinking critically and
solving problems (Perlman, 2003). Assessment tasks, such as oral presentations,
projects, open-ended Mathematics problems and those requiring a critical
evaluation, are examples of performance tasks (Perlman, 2003).
Tejeda and Gallardo (2017) conducted a study in Mexico to establish students’

perceptions of performance assessment tasks in helping students reach good
levels of skills and abilities. Interviews were used to collect data using a mixed-
methods design involving 20 students. The results were that performance
assessment was an effective approach for understanding students’ strengths and
weaknesses in teaching and learning. Furthermore, the authors indicated that the
performance assessment technique helps to link what a student learns in school
to real-life situations.
148
2.2.2 Rubrics
Rubrics appeal to teachers and students for many reasons. A rubric is described
as a “rule, guide, criterion, or description that is used to assess the progress of
students in their academic subjects, as well as the grading system for assessing
each criterion” (Cooper & Gargan, 2011, p. 54). Furthermore, several researchers
have argued that a rubric is a powerful tool for teaching and assessment; improves
students’ performance; makes teachers’ expectations clear; and guides students
on how to meet what is expected of them (Gallardo, 2020; Goodrich, 1997; Stewart
et al., 2020). Goodrich (1997) also argued that the use of rubrics permits teachers
to accommodate classes that are heterogeneous and reduces the amount of time
teachers would spend evaluating students’ work. Thus, the quality of students’
performance is marked by improvements and rubrics make it easier for teachers
to share the students’ evaluations with various stakeholders.
Rubrics can be used in different subjects, including Mathematics; what

differentiates them is the purpose of the rubric the teacher wants to use. Guskey
(1994) indicated that rubrics “are specific guidelines that can be used to describe
students’ work in reading, writing, mathematics, and other content areas” (p. 25).
Using rubrics provides teachers with a good context to give feedback on a
student’s performance level and indicates the next course of action for
improvement. Similarly, Ash and Levitt (2003) state that rubrics can be used to
diagnose the students’ learning levels in the classroom, which provides teachers
an opportunity to clarify learning targets by giving appropriate feedback to the
students.
Mustofa et al. (2020) reported that among the various assessment tools used to
measure and assess soft skills is the rubric. Furthermore, rubrics have been found
to improve instruction, increase student achievement, evaluate courses and to
assess soft skills (Khuzzan & Mahdzir, 2020; Reddy & Andrade, 2010). In addition,
when rubrics are used as part of a student-centered approach, they have the
potential to help students understand the targets for their learning and the
standards of quality for a particular assignment, as well as making dependable
evaluations and judgments about their work that can inform revision and
improvement (Reddy & Andrade, 2010).
2.2.3 Projects-based learning technique

Project-based learning is a process where students work collaboratively to find
solutions to the problems around them (Holmes & Hwang, 2016). Problem-based
learning is identified as promoting students’ skills in problem-solving, decision
making, and investigation (Thomas, 2000, p. 1). Project-based learning is
grounded on the constructivist theory where students attain a deeper
understanding while actively constructing their ideas, and engaging in real and
meaningful problem-solving activities (Krajcik & Blumenfeld, 2006).
The NCTM (2000) and Remijan (2016) recommended mathematics teachers to

provide classroom activities that are relevant, such as projects, which provide
students with opportunities to engage in real-life problem solving and allow for
many avenues to demonstrate what they have learned. The project-based learning
technique assists students studying secondary Mathematics to be motivated to
149
learn, and apply their content knowledge and skills, such as collaboration,
problem-solving and critical thinking (Holmes & Hwang, 2016; Thomas, 2000). In
support, Hope and Allen (2009) reported that “When students are engaged in
project-based learning, which is rich in collaboration and problem-solving,
learning becomes more authentic” (p. 3853).
During project-based learning techniques, students work in groups to deal with

curriculum-based, authentic, thought-provoking problems and often students
decide what activities to pursue in dealing with a problem (Solomon, 2008).
Furthermore, Solomon (2008) stated that students benefit from this technique in
gathering information from a variety of sources, synthesizing, analyzing and
deriving knowledge. As a result, students learn how to demonstrate the
knowledge they have acquired and what they have learned is judged by how well
they can communicate that new knowledge. The teachers’ role is not to direct and
manage students but to advise and guide them. These characteristics make
project-based learning an appropriate assessment technique for soft skills as
learning is assessed by how much students can demonstrate what they have
learned.
A study on project-based learning conducted by Ravitz et al. (2012) employed a

quasi-experimental design with 60 teachers who were selected based on having
published peer-reviewed projects on problem-based learning and soft skills. The
teachers selected taught Mathematics, Science, English, or Social Studies from
grades 4 to 11. The findings were that through project-based learning, students
gain more skills such as collaboration, communication and critical thinking,
leading to learning deep content knowledge (Ravitz et al., 2012). Additionally,
Ravitz et al. (2012) reported that teachers who used project-based learning were
reported to have managed to develop and assess soft skills. This was established
through a study that explored the impact of this technique (project-based
learning) on the teachers’ capacity to teach and assess soft skills.
A study by Holmes and Hwang (2016) , in Holland, Michigan, USA, examined the
benefits of project-based learning to secondary-mathematics students' skills
development and techniques for learning. The study employed a mixed-method,
longitudinal design using a test, interviews and a survey to collect data. A total of
532 secondary students from grades 8 and 9 participated in the study. The
findings were that project-based learning intrinsically motivated the students in
learning mathematics, and they showed an increase in their critical thinking
abilities. The study further reported a decrease in the achievement gap among
different demographic clusters and achievement levels in secondary mathematics
among students. The study, which reviewed literature to address two subjects,
regarding project-based learning in Mathematics and its effect on developing and
assessing soft skills, concluded that “topic-specific assessments tended to show
gains in achievement” (Jacques, 2017, p. 430).
Holmes and Hwang (2016) further reported that gains were noted to have been
transferred to state assessments with the use of project-based learning. Research
has established that the use of the project-based learning technique, depends on
150
the subject area and grade level, as well as carefully planning, managing, and
assessing the connection between academic content and soft skills (Ravitz et al.,
2012). Thus, it is the teachers’ responsibility to determine the kind of projects
appropriate to the grade level and how soft skills can be assessed.
2.2.4 The portfolio assessment model

Students' work may also be evaluated through portfolios. A portfolio is defined
as an accumulation of a student’s work collected over a period and varies in form,
function and content (Cicmanec & Viechnicki, 1994). Applying the definition to
Mathematics, a portfolio is “a showcase for student work, a place where many
types of assignments, projects, reports, and writings can be collected as well as
students’ progress in, attitudes toward, and understanding of Mathematics are
monitored comprehensively” (Stenmark & NCTM., 2007, p. 35).
The student's portfolio is one of the assessment tools that weigh the progress made
and the work that has been accomplished (Khuzzan & Mahdzir, 2020; Wilson,
2014). Over the years “mathematics teachers have used portfolios in their
classrooms to make instructional decisions” (Wilson, 2014, p. 698). Also, Fukawa-
Connelly and Buck (2010) reported that "students have developed the ability to
read and write about Mathematics through portfolios and have submitted more
self-directed, higher-quality work on other assignments” (p. 650). Fukawa-
Connelly and Buck (2010) admitted that the use of a portfolio as an assessment
tool is challenging and consumes time. However, the tool has enhanced students'
ability to read and write in Mathematics. Hence, besides assessing the progress
made and work accomplished by the students' portfolio, the assessment tool can
be used to assess communication skills.
A study by Cicmanec and Viechnicki (1994) evaluated the use of portfolio tools
for assessing students' learning in Mathematics and reported that there was a
“weak support for claims that Mathematics portfolio assessments enhance
student learning and promote effective communication among teachers, students,
and parents” (p. 167). The authors recommended that a strong rationale needs to
be established for selecting a portfolio as a tool for assessment in Mathematics.
However, recent studies reported the contrary. For instance, it is argued in a study
which investigated students’ mathematical thinking through the use of portfolio
tools, that students' achievement proficiency improved in Mathematics and that
the “portfolio assessments played a major role in the successes” (Fukawa-
Connelly & Buck, 2010, p. 650).
The process of generating portfolios to monitor students' progress raises their

expectations, which attracts appropriate instruction and support from the teacher.
It also shows that teachers do care about the students’ learning and that working
hard was key to success (Fukawa-Connelly& Buck, 2010; Khuzzan & Mahdzir,
2020). Therefore, mathematics teachers are urged to embrace the use of portfolios
in teaching and learning Mathematics, to be able to assess soft skills like
communications skills, which are not assessed through standardized tests.
151
Therefore, assessing what students know and can do is not only done through
standardized tests but also performance assessment tasks, standard-based
projects, rubrics and assignments (Care et al., 2018; Ministry of General Education,
2019). Nevertheless, even after the revision of the curriculum in Zambia, test items
in the assessment papers for Mathematics and other subjects are still based on the
cognitive domain (Ministry of General Education, 2019). The cognitive domain
largely covers cognitive aspects of the student (Anderson et al., 2001), yet the
revised curriculum covers a full spectrum of the students’ development, including
soft skills captured under the affective domain developed by David Krathwohl
(Krathwohl et al., 1964). This shows that the soft skills and competences on focus
may not be adequately assessed when Bloom’s taxonomy of the cognitive domain
is used as the assessment guide. Based on the reviewed studies, this study intends
to examine mathematics teachers’ practices which involve the assessment of soft
skills. Since the implementation of the ZECF of 2013, very little is known about
how mathematics teachers assess soft skills in teaching and learning Mathematics.
Therefore, this study was necessary since there had been no such a study
conducted in Zambian secondary schools.
3. Theoretical Framework
The control theory guided the study, which takes into consideration the vital
concerns of the cognitive and emotional dimensions of learning in the soft skills
assessment (Gibb, 2014). Carver and Scheier (1982) expounded that control theory
provides a model of self-regulation required to operate effectively and is useful in
the analysis of human behavior. It is argued that when assessment is done well, it
can reveal gaps and prompt curative measures to reduce the discrepancy (Gibb,
2014). The important aspect of self-regulation in control theory is the feedback
loop. The feedback loop is defined as “Information about the gap between the
reference level and the actual level of a system parameter used to close up the
gap” (Gibb, 2014, p. 8). The theory fits well with this study because it aimed at
examining the extent to which the assessment techniques that mathematics
teachers use in secondary schools assessed soft skills. This is because the sole
purpose of assessment, among many others, is to diagnose student learning
difficulties, provide feedback, help in planning how to carry out instruction, and
maintain social balance in the learning process (Guevara-Bazán et al., 2020). Gibb
(2014) summarizes that in control theory, the assessor (teacher) can diagnose the
lack of capacity in the attainment of soft skills on the part of the recipient (student)
and adjust instructions and provide feedback to close the gap.
4. Methodology
4.1 Research Design
A sequential multi-phase design, involving multiple phases of data collection and
analysis by Saunders et al. (2016), guided the study. A sequential multi-phase
research design is a research process, which recognizes that mixed methods
research is interactive, where one phase subsequently informs and directs the next
phase of data collection and analysis (Ridenour & Newman, 2008; Teddlie &
Tashakkori, 2009).
152
Data were collected in two phases. The first phase involved the collection of data
through a questionnaire. The second phase involved the collection of data using
a lesson observation schedule involving 124 videos.
4.2 Participants and Their Demographic Factors

The participants, purposively selected for the study, were mathematics teachers
from secondary schools in Mazabuka District in Zambia. A total of 91 participants
were drawn from 22 secondary schools, which included non-public and public
schools. Of the 91 participants, 81 returned the questionnaires after completing
them and 31 had four of their lessons captured through a video recording. The
distribution of the mathematics teachers according to the type of school, gender,
age and experience is shown in Table 1.
Table 1: Distribution of mathematics teachers according to their gender, age, teaching

experience, qualifications and type of school (n=91)
Factors Description Frequency
Gender Male 48 (53%)
Female 43 (47%)
Age (in years) 25 ≤ X ≤ 34 36 (40%)

35 ≤ X ≤ 44 35 (38%)
45 ≤ X ≤ 54 20 (22%)
Teaching experience (in years) ≤ 10 50 (55%)

˃ 10 41 (45%)
Qualifications Diploma 46 (51%)

Bachelor 45 (49%)
Type of school Non-public schools 45 (49%)

Public schools 46 (51%)
The gender of mathematics teachers, as indicated in Table 1, were 48 (53%) male

and 43 (47%) female. Mathematics teachers who participated in the study were of
ages ranging from 25 years to 54 years. The study took place in two different types
of schools namely public and non-public. There were 45(49%) from Non-public
schools and 46(51%) from public schools.
4.3 Research Instruments

A questionnaire and observation schedule were used in this study. The items in
the questionnaire and observation schedule were adapted from a Validation of
Modified Soft Skills Assessment Instrument (MOSSAI) (Aworanti et al., 2015) and
the Reformed Teaching Observation Protocol [RTOP](Piburn et al., 2000). The
adapted items were measured using literature and standards set in the
Framework for 21st Century Learning developed by the Partnership for 21st
Century Learning (P21) (Global Partnership for Education [GPE], 2020). In
particular, the adapted items were the learning and innovation skills comprising
creativity and innovation, critical thinking and problem solving, communication
and collaboration.
153
4.4 Validity and Reliability of the Instruments

The 5-point Likert scale used in the study was found to have a good internal
consistency, with a Cronbach’s alpha coefficient of .725. The Cronbach's alpha
coefficient for the scale used to establish the assessment techniques that
mathematics teachers use to assess soft skills in Mathematics was slightly above
0.7. The scales were considered reasonably reliable based on the sample. For the
data from the observations, inter-rater reliability of the observers was done using
Spearman’s rho to establish the level of agreement among the observers (Saunders
et al., 2016). The results for the computed Spearman correlation coefficient are
presented in Table 2.
Table 2: Spearman correlation coefficient among the four observers on the extent to
which the assessment techniques mathematics teachers used captured soft skills
Evaluator Evaluator Evaluator Evaluator
Correlations
01 02 03 04
Correlation
1.000 .989** .994** .983**
Evaluator Coefficient
01
Sig. (2-
tailed)
Correlation
.989** 1.000 .994** .972**
02
Sig. (2-
Spearman's tailed)
rho
Correlation
.994** .994** 1.000 .977**
03
Sig. (2-
tailed)
Correlation
.983** .972** .977** 1.000
04
Sig. (2-
tailed)
**. Correlation is significant at the 0.01 level (2-tailed).
4.5 Data Analysis

The data from the questionnaire and the observation schedule were analyzed and
presented in frequencies, percentages, mean (M), standard deviations (SD) and
chi-square statistics, using the Statistical Package for Social Sciences (SPSS) (IBM
Corp., 2015).
4.6 Limitation of the Study

The study explored assessment techniques that integrate soft skills in teaching
mathematics in secondary schools in Zambia and were subject to a sample size
limitation. This is because the 91 participants were not representative of all the
teachers in the country. The implication is that the results may not be generalized.
154
5. Results
5.1 Research Question 1
To what extent do the assessment techniques mathematics teachers use in
secondary schools in Zambia assess soft skills?
5.1.1 Data from the self-reporting questionnaire

The assessment techniques that mathematics teachers used to assess soft skills
To establish the extent to which the assessment techniques mathematics teachers,
used in secondary schools in Zambia to assess soft skills, mathematics teachers
were asked to put a tick (√) in the appropriate box that suited their response to the
given statement in the self-reporting questionnaire. Table 3 reports the
frequencies, percentages, means (M) and standard deviations (SD) where SD, D,
U, A and SA represent strongly disagree, disagree, uncertain, agree and strongly
agree, respectively.
Table 3: Frequencies, percentages, mean and standard deviation according to

mathematics teacher’s assessment of soft skills in Mathematics (n = 81)
Frequency
SN Statement M SD
SD D U A SA
I assess students’
development of soft skills
3 4 11 46 17
1 through Mathematics long 3.86 0.93
(4%) (5%) (14%) (57%) (21%)
term tasks that I administer
to the class.
I assess and monitor
students’ progress in
1 8 34 38
2 Mathematics by keeping 4.35 0.71
(1%) (10%) (42%) (47%)
records of the students’
development of soft skills.
performance by 1 2 43 33
3 2 (2%) 4.30 0.75
administering Mathematics (1%) (2%) (53%) (41%)
assignments.
I use Mathematics tailored
rating scale in class to assess 2 2 25 44 8
4 3.67 0.79
students’ development of (2%) (2%) (31%) (54%) (10%)
soft skills.
I administer Mathematics
3 5 36 32
5 quizzes to assess students’ 5 (6%) 4.10 1.02
(4%) (6%) (44%) (40%)
performance.
4 11 17 38 11
6 mathematical understanding 3.51 1.05
(5%) (14%) (21%) (47%) (14%)
by administering games.
I administer mathematical
activities to assess students’ 1 28 51
7 1 (1%) 4.59 0.59
critical thinking/problem- (1%) (35%) (63%)
solving skills.
The analysis portrayed in Table 3 shows a high mean score on the assessment
approaches mathematics teachers claimed to use in the integration of soft skills,
ranging from performance tasks, project-based learning, quizzes, rubrics,
portfolio, and assignments. The mean score, on average, was between M ≥3.51 and
M ≤ 4.59. The high mean score could indicate that mathematics teachers used the
155
assessment approaches that allow the integration of soft skills in the teaching and
learning of Mathematics. However, a variation in responses, ranging from
strongly disagree to strongly agree may indicate a lack of clarity on the assessment
approaches Mathematics teachers use as to whether they assess soft skills or not.
5.1.2 Data from the observation schedule

The study captured and evaluated 124 video-recorded lessons from 31
mathematics teachers. Each of the 31 teachers had four of their lessons captured.
The evaluation was done by the researcher and other three evaluators who
together made 496 observations.
5.1.3 The extent to which assessment techniques mathematics teachers used in

Mathematics assess soft skills
After establishing the Spearman correlation coefficient among the four observers
which was ≥ .972, data were analyzed and presented in Table 4.
Table 4: Frequencies, percentages, Mean and standard deviation according to

Mathematics teacher’s assessment of soft skills in Mathematics (n = 496)
Frequency
Statement M SD
Yes No
1. The teacher assessed students’ performance
496
through Mathematics projects that he 0.000 0.000
assigned to the class.
(100%)
2. The teacher assessed and monitored students’ 496
0.000 0.000
progress in Mathematics using portfolios. (100%)
379 117
through administering Mathematics class 0.764 0.425
exercise.
(76%) (24%)
491
through administering Mathematics 5 (1%) 0.010 0.100
assignments.
(99%)
5. The teacher used Mathematics tailored
496
rubrics in class to assess students’ 0.000 0.000
performance.
(100%)
6. The teacher administered Mathematics
496
quizzes to the class to assess students’ 0.000 0.000
performance.
(100%)
7. The teacher assessed students’ mathematical 496
0.000 0.000
understanding by administering games. (100%)
8. The teacher administered mathematical
496
activities to assess students’ critical 0.000 0.000
thinking/problem-solving skills.
(100%)
Table 4 indicates a comparatively low mean on the assessment approaches

mathematics teachers used in the teaching and learning of mathematics based on
the 496 observations made on the 124 videos. The results from the observations
have shown a mean score on average between M ≥ 0.00 and M ≤ 0.76. The mean
M = .76 was the class exercises the teachers administered during the teaching and
learning of Mathematics which does not elicit high-order thinking skills, such as
problem-solving, critical thinking and creativity and innovation (Minarni & Elvis,
2019; Ministry of General Education, 2019). The small mean indicates that from
the videos observed the assessment techniques mathematics teachers use in
secondary schools in Zambia do not assess soft skills.
156
5.2 Research Question 2

Is there any relationship between mathematics teachers’ demographic factors and
the assessment techniques they used in the teaching and learning of Mathematics?
This research question was explored by examining the effect of teachers’ gender
and the type of school where they were teaching based on the data from the
observation schedule. To establish the effect of demographic factors on the
teachers’ choice of assessment techniques, a chi-square statistic was computed
(Morgan et al., 2011).
5.2.1 Teachers’ demographic factors and the assessment techniques they used in the
teaching and learning of Mathematics
Gender
The study investigated whether male and female mathematics teachers differed
in the assessment techniques they used in the teaching and learning of
Mathematics (ref. Table 4). The findings suggest that there was no statistical
difference between them (2 = 2.794, df = 1, N = 496, p-value=.095).
Type of school where teachers were teaching

The study also examined the effect of the type of school where mathematics
teachers who participated in the study were teaching and the assessment
techniques they were using in the teaching and learning of Mathematics, as
reported in Table 4. The findings showed that mathematics teachers from non-
public schools were likely to slightly differ from those from public schools (2 = 5.
163, df = 1, N = 496, p-value=.023).
6. Discussions
6.1 The Extent to Which the Assessment Techniques Mathematics
Teachers, Used in Secondary Schools in Zambia to Assess Soft Skills
The results from the observations show a mean score on average between M ≥ 0.00
and M ≤ 0.76. The mean M = .76 was the class exercises the teachers administered
during the teaching and learning of Mathematics, which does not elicit high-order
thinking skills such as problem-solving, critical thinking and creativity and
innovation (Minarni & Elvis, 2019; Ministry of General Education, 2019). The
small mean indicates that from the videos observed, the assessment techniques
mathematics teachers use in secondary schools in Zambia do not assess soft skills.
This is not far from the results obtained by Blom et al. (2017), who reported that
curricula in Zambia, Botswana and Lesotho presented little evidence that soft
skills, such as problem-solving, collaboration, entrepreneurship, and self-
management, were assessed in the teaching and learning. Similarly, a study by
Mkimbili and Kitta (2019), that critically analyzed the assessment of competencies
in secondary schools in Tanzania, reported that teachers were still using written
tests as assessment tools, which did not develop competencies, such as soft skills,
in students. The study recommended that attaining a better assessment level
requires restructuring the assessment techniques to include portfolio assessments,
oral examinations and projects (Mkimbili & Kitta, 2019). These techniques ensure
that students are engaged in critical thinking skills (Mkimbili, 2018).
157
The teachers are well-positioned to take up a very important role in developing

students’ soft skills, which demands a high level of thinking (Soh et al., 2012).
Although the ZECF of 2013 demands this, the assessment tasks in Mathematics at
both school and national levels do not incorporate adequate tasks that demand
high-level thinking (Ministry of General Education, 2019). Thus, teachers are
constrained to focus on integrating soft skills that enhance higher levels of
thinking.
Mathematics teachers require knowledge on techniques that embrace student-

centered approaches to develop soft skills and employ performance-based
assessments. However, teachers have continued with the same assessment
methods despite the curriculum being changed from content-based to
competency-based. The consequence of not assessing soft skills could be that
mathematics teachers may not see the need to focus on the development of soft
skills.
6.2 Mathematics Teachers’ Demographic Factors and the Assessment

Techniques They Used in the Teaching and Learning of Mathematics
The study investigated the relationship between mathematics teachers’
demographic factors and the assessment techniques they used in the teaching and
learning of Mathematics. The results show in the choice of assessment techniques
used in the teaching and learning of Mathematics, males are not more likely to
differ from females. The phi was .075, indicating a weaker association between the
two variables (Morgan et al., 2011). Furthermore, the results suggest that
mathematics teachers from non-public schools slightly differed in the choice of
assessment techniques they were using in the teaching and learning of
Mathematics from teachers from public schools. However, the phi was -0.102,
showing a weaker association between the two variables (Morgan et al., 2011).
7. Conclusion
This study investigated assessment techniques mathematics teachers used that
signify the integration of soft skills in secondary schools in Mazabuka District in
Zambia. In conclusion, the study established that the assessment techniques
mathematics teachers were using did not assess soft skills based on the data from
the observation. The findings further suggest that mathematics teachers’ choice of
assessment techniques used in the teaching and learning of Mathematics was not
related to gender but to the type of school where teachers were teaching, even
though the association between the two variables was weaker. These findings
could be an indication that secondary school mathematics teachers do not focus
on the assessment of soft skills. The consequence of this may be that secondary
school leavers may not be good communicators, innovators, creators and critical
thinkers. However, the results may not be generalized as the sample was not
representative of the whole country. The importance of assessing soft skills cannot
be over overlooked as it is the only way of confirming that soft skills are integrated
in the teaching and learning of Mathematics. Therefore, it is recommended that
mathematics teachers be upskilled in the use of a variety of assessment techniques
such as performance-based tasks, rubrics, assignments, observation, portfolio and
standard-based projects that would capture soft skills.
158
Acknowledgement
The authors would like to thank the African Centre of Excellence for Innovative
Teaching and Learning Mathematics and Science (ACEITLMS), the University of
Rwanda, for financial support, and Opanga David (Lecturer) at St. John's
University of Tanzania for proofreading the manuscript.
8. References
Anderson, L. W., Krathwohl, D. R., Airasian, P. W., Cruikshank, K. A., Mayer, R. E.,
Pintrich, P. R., Raths, J., & Wittrock, M. C. (2001). A taxonomy for learning, teaching,
and assessing: A revision of Bloom’s taxonomy of educational objectives (abridged edition).
Pearson Education Group.
https://www.bibsonomy.org/bibtex/29311293e9c7da8b40127405e31c7e8b0/yish
Anderson, R. S. (1998). Why talk about different ways to grade? The shift from traditional
assessment to alternative assessment. New Directions for Teaching and Learning,
1998(74), 5–16. https://doi.org/10.1002/tl.7401
Ash, D., & Levitt, K. (2003). Working within the zone of proximal development: Formative
assessment as professional development. Journal of Science Teacher Education, 14(1),
23–48.
https://www.researchgate.net/publication/225968533_Working_within_the_Zon
e_of_Proximal_Development_Formative_Assessment_as_Professional_Developm
ent
Aworanti, O. A., Taiwo, M. B., & Iluobe, O. I. (2015). Validation of modified soft skills
assessment instrument (MOSSAI) for use in Nigeria. Universal Journal of Educational
Research, 3(11), 847–861. https://doi.org/10.13189/ujer.2015.031111
Bargainnier, S. (2003). Fundamentals of rubrics. Supervision, 1–4.
Birgin, O. (2011). Pre-service mathematics teachers’ views on the use of portfolios in their
education as an alternative assessment method. Educational Research and Reviews,
6(11), 710–721.
Blom, A., Cao, X., Andriamihamina, H., & Akinlawon, T. (2017). Job-ready graduates of
secondary education in Botswana, Lesotho and Zambia: Reforming instruction, curriculum,
assessment, and structure to teach vocational and 21st Century skills. World Bank.
Bostic, J., Lesseig, K., Sherman, M., & Boston, M. (2021). Classroom observation and
mathematics education research. Journal of Mathematics Teacher Education, 24(1), 5–
31. https://doi.org/10.1007/s10857-019-09445-0
Care, E., Kim, H., Vista, A., & Anderson, K. (2018). Education system alignment for 21st
century skills: Focus on assessment. Center for Universal Education at the Brookings
Institution. https://www.brookings.edu/research/education-system-alignment-
for-21st-century-skills/
Carver, C. S., & Scheier, M. F. (1982). Control theory: A useful conceptual framework for
personality-social, clinical, and health psychology. Psychological Bulletin, 92(1), 111–
135. https://doi.org/10.1037/0033-2909.92.1.111
CDC [Curriculum Development Centre]. (2013a). 2013 Zambia education curriculum
framework. MESVTEE.
http://www.ibe.unesco.org/fileadmin/user_upload/archive/curricula/zambia/
za_alfw_2013_eng.pdf
CDC [Curriculum Development Centre]. (2013b). Additional mathematics syllabus grade 10-
12. Zambia Educational Publishing House.
CDC [Curriculum Development Centre]. (2013c). Mathematics syllabus (Grades 8 and 9).
MESVTEE. Zambia Educational Publishing House.
CDC [Curriculum Development Centre]. (2013d). O-level mathematics syllabus grade 10-12.
MESVTEE. Zambia Educational Publishing House.
159
CDC [Curriculum Development Centre]. (2013e). Teachers’ curriculum implementation

guide: guidance to enable teachers to make best use of the Zambia education curriculum
framework 2013 (No. 1). Ministry of General Education. Zambia Educational
Publishing House.
Cicmanec, K. M., & Viechnicki, K. J. (1994). Assessing mathematics skills through
portfolios: Validating the claims from existing literature. Educational assessment, 2(2),
167–178. https://doi.org/10.1207/s15326977ea0202_4
Cooper, B. S., & Gargan, A. (2011). Rubrics in education: Old term, new meanings.
Educational Horizons, 89(4), 6–8. https://doi.org/10.1177/0013175x1108900402
Danielson, C., & Marquez, E. (2016). Performance tasks and rubrics for high school
Mathematics: Meeting rigorous standards and assessments (2nd ed.). Routledge.
https://doi.org/10.4324/9781315695297
Dochy, F. (2001). A new assessment era: Different needs, new challenges. Learning and
Instruction, 10(1), 11–20. https://doi.org/10.1016/s0959-4752(00)00022-0
Durowoju, E. O., & Onuka, A. O. U. (2014). Best practices and experiences in soft skills
assessment in Oyo town, Oyo state schools. Journal of Emerging Trends in Educational
Research and Policy Studies (JETERAPS), 5(5), 607–613.
https://pdfs.semanticscholar.org/90f8/20bc3bf6ca6afbcdc067db147eda4465810e.
pdf
Fukawa-Connelly, T & Buck, S. (2010). Using portfolio assignments to assess students’
mathematical thinking. National Council of Teachers of Mathematics, 103(9), 649–654.
Gallardo, K. (2020). Competency-based assessment and the use of performance-based
evaluation rubrics in higher education: Challenges towards the next decade.
Problems of Education in the 21st Century, 78(1), 61–79.
https://doi.org/10.33225/pec/20.78.61
Gallego, M. J., & Dandis, M. (2014). Rubrics in a secondary Mathematics class. International
Electronic Journal of Mathematics Education, 9(1), 73–82.
Gibb, S. (2014). Soft skills assessment: Theory development and the research agenda.
International Journal of Lifelong Education, 33(4), 455–471.
https://doi.org/10.1080/02601370.2013.867546
Goodrich, H. (1997). Understanding Rubrics. Educational Leadership, 54(4), 14–17.
GPE [Global Partnership for Education]. (2020). 21st century skills: What potential role for the
Global Partnership for Education? (Issue January).
https://www.globalpartnership.org/sites/default/files/document/file/2020-01-
GPE-21-century-skills-report.pdf
Guevara-Bazán, I. A., Mata Martínez, K. L., & Martínez-Cortés, Jorge. (2020). Perceptions
of objective formative assessment in language learning. Revista Lengua y Cultura,
2(3), 22–30. https://doi.org/10.29057/lc.v2i3.6144
Guskey, T. R. (1994). Making the grade: What benefits students? Educational Leadership,
52(2), 14–20. http://www.burlesonisd.net/Forms/cur/Making the Grade.pdf
Hamilton, D., McKechnie, J., Edgerton, E., & Wilson, C. (2021). Immersive virtual reality
as a pedagogical tool in education: A systematic literature review of quantitative
learning outcomes and experimental design. Journal of Computers in Education, 8(1),
1–32. https://doi.org/10.1007/s40692-020-00169-2
Holmes, V. L., & Hwang, Y. (2016). Exploring the effects of project-based learning in
secondary mathematics education. Journal of Educational Research, 0(0), 1–15.
https://doi.org/10.1080/00220671.2014.979911
Hope, J. T., & Allen, W. J. (2012). Student competitive events: A strategy for integrating a
STEM block in public schools. Society for Information Technology & Teacher Education
International Conference, 3852–3856. http://www.learntechlib.org/noaccess/40204
IBM Corp. (2015). IBM SPSS statistics for windows, Version 23.0. IBM Corp.
160
Jacques, L. A. (2017). What does project-based learning (PBL) look like in the mathematics
classroom? American Journal of Educational Research, 5(4), 428–433.
https://doi.org/10.12691/education-5-4-11
Khuzzan, S. M. S., & Mahdzir, M. (2020). Assessment for the 21st Century in higher
education institution: Way forward. International Journal of Advanced Research in
Education and Society, 2(2), 35–46.
Krajcik, J. S., & Blumenfeld, P. C. (2006). Project-based learning. In R. K. Sawyer (Ed.), The
Cambridge handbook of the learning sciences (pp. 317–333). Cambridge University
Press.
Krathwohl, D., Bloom, B., & Masia, B. (1964). Taxonomy of educational objectives, the
classification of educational goals. Handbook II: Affective domain. David McKay Co.
Mahasneh, J. K., & Thabet, W. (2015). Rethinking construction curriculum: A descriptive
cause analysis for soft skills gap among construction graduates. 51st ASC Annual
International Conference Proceedings, August.
Minarni, A., & Elvis, N. E. (2019). Learning approach and soft-skills contribution toward
mathematical higher-order thinking skills of junior high school students. American
Journal of Educational Research, 7(12), 925–929. https://doi.org/10.12691/education-
7-12-5
MINEDUC [Ministry of Education] & [REB] Rwanda Education Board. (2015). Mathematics
syllabus for advanced level S4–S6. MINEDUC.
Ministry of General Education. (2019). Republic of Zambia ministry of education educational
statistical bulleting 2018.
https://www.parliament.gov.zm/sites/default/files/documents/committee_rep
orts/EDUCATION%20-
%20REPORT%20ON%20EDUCATION%202018%20FINAL.pdf
Mkimbili, S & Kitta, S. R. (2019). The Rationale of continuous assessment for development
of competencies in Tanzania secondary schools. Advanced Journal of Social Science,
6(1), 64–70. https://doi.org/10.21467/ajss.6.1
Mkimbili, S. T. (2018). Learner-centred science teaching in community secondary schools in
Tanzania. [Doctoral dissertation, University of Oslo].
https://www.duo.uio.no/bitstream/handle/10852/60279/PhD-Selina-Mkimbili-
2018.pdf?sequence=1
Morgan, G. A., Leech, N. L., Gloeckner, G. W., & Barrett, K. C. (2011). IBM SPSS for
introductory statistics: Use and interpretation (4th ed.). Routledge.
http://books.google.com/books?id=yhgGfAEACAAJ&pgis=1
Moser, E. H., Kinney, L., & Purdy, C. (1948). Aims and purposes in the teaching of
Mathematics. Review of educational research. Review of Education Research, 18(4),
315–322.
Mustofa, G. G., Ana, A., Widiaty, I., & Muktiarni, V. D. (2020). Improving learning
performance by applying multi-platform e-rubric as a soft skills assessment
instrument in the creative-art area of Indonesian vocational education. Journal of
Engineering Science and Technology, 6(2020), 61–69.
NCTM [National Council of Teachers of Mathematics]. (2000). Principles and standards for
school mathematics. NCTM.
NCTM [National Council of Teachers of Mathematics]. (2014). Principles to actions:
Ensuring mathematical success for all. The National Council of Teachers of
Mathematics.
Perlman, C. C. (2003). Performance assessment: Designing appropriate performance tasks
and scoring rubrics. In Measuring up: Assessment issues for teachers, counsellors, and
administrators (pp. 497–506). Educational Resources Information Center (ERIC).
161
Piburn, M., Sawada, D., Turley, J., Falconer, K., Benford, R., & Bloom, I. (2000). Reformed
teaching observation protocol (RTOP) reference manual. ACEPT Technical Report No.
IN00-3.
http://www.public.asu.edu/~anton1/AssessArticles/Assessments/Chemistry
Assessments/RTOP Reference Manual.pdf
Putra, A. S., Novitasari, D., Asbari, M., Purwanto, A., Iskandar, J., Hutagalung, D., &
Cahyono, Y. (2020). Examine relationship of soft skills, hard skills, innovation and
performance: The mediation effect of organizational learning. International Journal
of Science and Management Studies (IJSMS), 3(3), 27–43. www.ijsmsjournal.org
Ravitz, J., Hixson, N., English, M., & Mergendoller, J. (2012). Using project-based learning
to teach 21st century skills: Findings from a statewide initiative. Annual Meetings of
the American Educational Research Association., April, 1–9.
REB [Rwanda Education Board]. (2015). Curriculum framework: Pre-primary to upper
secondary 2015. In Competence Based Curriculum. REB/MINEDUC.
Reddy, Y. M., & Andrade, H. (2010). Assessment and evaluation in higher education: A
review of rubric use in higher education. Assessment & Evaluation in Higher
Education, 35(4), 435–448. https://doi.org/10.1080/02602930902862859
Remijan, K. W. (2016). Project-based learning and design-focused projects to motivate
secondary mathematics students. The Interdisciplinary Journal of Problem-Based
Learning, 11(1), 1–15. https://doi.org/10.7771/1541-5015.1520
Ridenour, C. S., & Newman, I. (2008). Mixed methods research: Exploring the interactive
continuum. South Illinois University Press.
Saunders, M., Lewis, P., & Thornhill, A. (2016). Research methods for business students
(Seventh). Pearson.
Schwab, K. (2017). The fourth industrial revolution (1st ed.). Crown Publishing Group.
Soh, T. M. T., Osman, K., & Arsad, N. M. (2012). M-21CSI: A validated 21st century skills
instrument for secondary science students. Asian Social Science, 8(16), 38–44.
https://doi.org/10.5539/ass.v8n16p38
Solomon, G. (2008). Project-based learning: A primer. Technology & Learning.
http://www.techlearning.com/db_area/archives/TL/2003/01/project.php
Stecher, B. (2010). Performance assessment in an era of standards-based educational
accountability. Stanford Center for Opportunity Policy in Education.
Stenmark, J. K., & National Council of Teachers of Mathematics [NCTM]. (2007).
Mathematics assessment: Myths, models, good questions, and practical suggestions (17th
ed.). National Council of Teachers of Mathematics.
Stewart, C., Marciniec, S., Lawrence, D., & Joyner-McGraw, L. (2020). Thinkubator
approach to solving the soft skills gap. American Journal of Management, 20(2), 78–89.
Teddlie, C., & Tashakkori, A. (2009). Foundations of mixed methods research: Integrating
quantitative and qualitative approaches in the social and behavioural sciences. Sage.
Tejeda, S., & Gallardo, K. (2017). Performance assessment on high school advanced
algebra. International Electronic Journal of Mathematics Education, 12(3), 777–798.
Thomas, J. W. (2000). A review of research on project-based learning (pp. 1–46). The Autodesk
Foundation. http://www.autodesk.com/foundation
Wilson, L. (2014). Using students’ portfolios to assess mathematical understanding. The
Mathematics Teacher, 87(9), 698–701.
162
Appendix
A SCALE FROM THE QUESTIONNAIRE
INSTRUCTIONS:
Put a tick (√) in the appropriate box that suits your response to the given statement where
SD (Strongly Disagree), D (Disagree), U (Uncertain), A (Agree), and SA (Strongly
Agree)
SN Statement SD D U A SA
I assess students’ development of soft skills through
1
Mathematics long-term tasks that I administer to the class.
I assess and monitor students’ progress in Mathematics by
2
keeping records of the students’ development of soft skills
I assess students’ performance by administering
3
Mathematics assignments
I use Mathematics tailored rating scale in class to assess
4
students’ development of soft skills.
I administer Mathematics quizzes to assess students’
5
performance.
I assess students’ mathematical understanding by
6
administering games.
I administer mathematical activities to assess students’
7
critical thinking/problem-solving skills
A SCALE FROM THE LESSON OBSERVATION SCHEDULE
Part 3: Assessment approaches that capture soft skills in the teaching and
learning Mathematics
Instructions: Tick (√) either YES or NO against the statements regarding the
lesson observed concerning the approaches used in the assessment of soft skills.
Give a brief description of the lesson observed concerning the assessment of soft
skills
SN STATEMENT NO YES
The teacher assessed students’ performance through Mathematics projects
1
that he assigns to the class.
The teacher assessed and monitored students’ progress in Mathematics
2
using portfolios.
The teacher assessed students’ performance by administering Mathematics
3
class exercise
The teacher assessed students’ performance by administering Mathematics
4
assignments
The teacher used Mathematics tailored rubrics in class to assess students’
5
performance.
The teacher administered Mathematics quizzes to the class to assess
6
students’ performance.
The teacher assessed students’ mathematical understanding by
7
administering games.
The teacher administered mathematical activities to assess students’ critical
8
thinking/problem-solving skills
163

Vol. 21, No. 8, pp. 163-174, August 2022
Generic Competences of University Students

from Peru and Cuba
Miguel A. Saavedra-López
Universidad Continental, Cusco, Perú
Xiomara M. Calle-Ramírez
Universidad Nacional de Tumbes, Tumbes, Perú
Karel Llopiz-Guerra
Universidad Central "Marta Abreu" de Las Villas, Santa Clara, Cuba
Marieta Alvarez Insua

Tania Hernández Nodarse

Julio Cjuno
Universidad Peruana Unión, Lima, Perú
Andrea Moya
Universidad Peruana Unión, Lima, Perú
Ronald M. Hernández*
Universidad Católica Santo Toribio de Mogrovejo, Chiclayo, Perú
Abstract. Generic competences are considered to be the ability of people

to adapt to changes, relate to others and work collaboratively. The
objective of this study was to compare the generic competences of
university students from Peru and Cuba. The study sample consisted of
248 female students from public universities with current enrollment in
Peru (127) and Cuba (121). The data collection was carried out using the
questionnaire on generic competences of university students (CCGEU),
the Cronbach's Alpha reliability value was .979, showing a high reliability
of the instrument. The results show that the mean of generic competences
in Peru is 3.88 and in Cuba is 3.84, which are adequate levels. In addition,
* Corresponding author: Ronald M. Hernández, ronald.hernandez@outlook.com.pe
©Authors
164
the dimensions Systemic competence, Interpersonal competence and

Instrumental competence show sufficient levels in both countries. It is
concluded that the generic competences in Peru and Cuba reach adequate
levels, so they were developed in university students in a similar way.
These results may be due to the fact that the evaluated samples have
adequately developed these competences that promote the relationship,
integration and effective communication among peers in a constant social
cooperation, teamwork and sense of leadership.
Keywords: generic competences, university students, higher education,

learning, Peru, Cuba
1. Introduction
At present, the higher education system worldwide is undergoing a stage of
restructuring and transformation. Higher education is a fundamental space for
training specialists to achieve the development of countries (Zahner et al., 2021).
However, there is a gap between academic training and activity in the workplace.
While it is true that it is important for higher education students to have a
favorable attitude and skills towards scientific research in order to consolidate
their profession as a science and perform well (Hernández et al., 2022), it is also
important to have generic competencies related to collaborative work,
interpersonal relationships and emotional management (Vélez et al., 2018).
In recent years, the term “generic competences” has been used in the university
environment, therefore, it is necessary to understand this concept. For this reason,
it can be said that when reference is made to competence in the professional
environment, it refers to the capabilities, aptitudes, skills and abilities that the
individual has to perform a task related to his/her profession (García-Atarés et
al., 2021). It also refers to people's performance, i.e., the ability to do, act or operate
in a certain way in different areas of life (McClelland, 1973). Based on the above,
some tests are conducted on students in order to evaluate these competences, as
they serve to know how learning is taking place in the individual (Garrido et al.,
2021). The generic competences also encompass the competences of the subject
and increase the autonomy and learning that he/she has, on which the
achievement of different objectives and solutions to problems typical of the
university stage will depend (Garrido et al. 2021), in order to be able to adapt in
the workplace, be flexible to change jobs and expand the knowledge learned
(Yorke and Harvey, 2005).
Research studies on generic competences of university students were developed

taking into account that the university education aims to enable students to
transcend and strengthen their specific skills related to the profession and generic
competences as a result of university experience and previous experience
(Nabaho, 2017). A previous study conducted on medical students showed that the
bidirectional feedback given through the tutor helps the development of generic
competences (Godoy et al., 2021). Another study conducted on professors in Spain
showed that methodological acceptance has an inverse relationship with the
perceived competence construct (slope of -0.295), which indicates that the higher
165
the values of the perceived competence construct, the less reluctant they are to
accept and perform teachers tutoring (Donoso et al, 2021). Therefore, the
professor's ability to adjust to the competences that the student needs to build is
essential (Koster et al., 2005). In addition, a survey of 1272 students from Osijek-
Baranja County in Croatia concluded that there is a positive correlation between
extracurricular activities and the entrepreneurial environment, and the level of
entrepreneurial skills of the students (Perić et al., 2020).
In Colombia, an Academic Support Plan was designed based on critical reading

and mathematics competences. The results showed that 22% of the students did
not exhibit progress in their mathematical competences in contrast to 78% who
had a significant contribution. In relation to critical reading, it was observed that
9.3% of the students did not show improvement in their reading competences in
contrast to 90.7% who reached adequate levels of critical reading. Significant
differences were also identified between the pre- and post-test. For example, in
the pretest, 4 students obtained a grade lower than 1.0. below 1.0 and after the
intervention intervention, the results of the post-test the post-test results show
that only one student a grade lower than 1.0 (Castellar et al., 2021).
The generic competences can be better understood from three dimensions that
explain this construct as instrumental competence which refers to methodological
(environment management, such as problem solving or learning methods),
cognitive (ability to control thoughts and ideas), linguistic and technological skills
(ability to use technological devices), favoring the basic training of the student
(Aguado et al., 2017; Amor, 2018). In addition, it is known that recent graduates
think that the capacity for analysis and synthesis, basic general knowledge and
knowledge related to their profession, computer skills and decision making,
among others, are fundamental, promote liaison, integration and communication
between two or more interlocutors, as well as social cooperation, specific or
multidisciplinary teamwork and a sense of leadership (Corominas, 2001;
Corominas et. al, 2006; Solanes et al., 2012). Finally, systemic competences include
understanding, sensitivity and knowledge, autonomous learning, adaptation to
changes and the creativity that the individual develops (Zabala and Arnau, 2008;
Solanes et. al, 2012).
In Peru, the National System of Evaluation, Accreditation and Certification of

Educational Quality aims that the profile of the graduate of higher education
institutions is related to the needs of the country (SINEACE. 2017). In other words,
professionals are trained with specific competencies related to the capabilities of
the specialty and generic competencies related to the ability to relate optimally in
the workplace. Likewise, Cuba's higher education institutions aim at the
comprehensive training of higher education students, where the graduating
student is competent, with scientific preparation and broad humanistic
development and social commitment (Vega and De Armas, 2009). In both
countries, the development of specific and generic competencies in higher
education students is taken into account; however, in higher education, priority
is given to specific competencies, considering them to be of greater relevance in
professional training, so it is important to know the generic competencies present
166
in university students in Peru and Cuba. Therefore, the objective of this study was
to compare the generic competencies of university students in Peru and Cuba.
2. Methodology
2.1 Type and design of research
The research is cross-sectional with a non-experimental design since the studied
variable was not manipulated. It is a comparative and descriptive study on the
state of the studied variable in a given population (Hernández-Sampieri and
Mendoza, 2018). Moreover, it is basic research since it facilitated the collection of
information on a reality that contributed to the increase of scientific knowledge
and understanding of a specific problem (Barriga, 1971).
The variable “generic competences” of female students from public universities
in Peru and Cuba was studied in this research.
2.2 Sample
The sample consisted of 248 female students from public universities with current
enrollment in Peru (127) and Cuba (121), in this sense, the female population was
considered, given that in Peru there is a higher percentage of women (17.2%) than
men (15.5%) who reach higher education (National Institute of Statistics and
Informatics [NISI], 2021). Due to the health emergency situation, the sample was
a convenience sample by using technology (filling out a Google form).
2.3 Instruments
The instrument applied is called Questionnaire on generic competences of
university students (CCGEU, by its Spanish initials) (Villanueva, 2014). The
instrument consists of 61 items distributed in three dimensions, in turn these
dimensions are subdivided into subdimensions, as follows: 1. Instrumental
competence: a. Organization and planning. b. Information management. c.
Solution of critical situations. d. Decision making. e. Communication.
Communication. 2. Interpersonal: a. Teamwork. b. Critical thinking. c. Self-
management. 3. Systemic: a. Learning orientation. b. Leadership. c. Motivation to
learn. d. Leadership. Leadership. c. Motivation to quality. d. Flexibility. The
instrument has levels of competence that are insufficient, sufficient and excellent.
Regarding percentages (%) of response, those that are equal to or higher than 30%
are classified as excellent according to frequency.
Finally, the Cronbach's Alpha reliability value was .979, which showed a high
reliability of the instrument applied.
2.4 Data Analysis

The statistical package Statistical Package for Social Sciences (SPSS V. 25.0) and
Microsoft Excel were used to process the study data. In addition, the analysis was
carried out with tables of frequencies, percentages and mean. Finally, the
Cronbach's Alpha reliability analysis was used.
3. Results
The population of this study is female students from public universities. In
relation to their professional careers, in Peru the majority of them belong to Social
167
Sciences with 67.7% and in Cuba they belong to Others with 90.1%. In relation to
the reasons they had for choosing the professional career they are studying, the
response was “It is the one I like it” in both countries, 70.1% in Peru and 68.6% in
Cuba. In relation to their age, more than 70% of the participants are between 18
and 21 years old, 78.7% (Peru) and 72.3% (Cuba). When asked about employment
status, it can be observed that 70.9% (Peru) and 82.9% (Cuba) are dedicated
exclusively to university studies. Finally, to get an idea of family conditions, the
level of education of the parents was taken into account: in the case of Peru, 52.8%
of mothers and 52% of fathers have basic education. In the case of Cuba, 68.2% of
mothers and 43% of fathers have university education (Table 1).
Table 1. Characteristics of the study population by country

Country of origin
Peru Cuba
% fx % fx
The Economic Sciences 7.1% 9 0.0% 0
professional Health Sciences 4.7% 6 0.8% 1
career belongs Other 9.4% 12 90.1% 109
to: Political Science 11.0% 14 0.0% 0
Social Sciences 67.7% 86 9.1% 11
Reasons for It is a family tradition 2.4% 3 0.8% 1
choosing the It is easy 0.0% 0 0.8% 1
career: It offers good job 11.8% 15 5.0% 6
opportunities
It is the one I like 70.1% 89 68.6% 83
I was not admitted to 2.4% 3 13.2% 16
another professional
program
No special reason 6.3% 8 7.4% 9
Other 7.1% 9 4.1% 5
Age: 18-21 78.7% 100 72.7% 88
21+ 21.3% 27 27.3% 33
Job: No 70.9% 90 82.6% 100
Yes 29.1% 37 17.4% 21
Mother's No 7.9% 10 0.0% 0
studies: Basic 52.8% 67 23.1% 28
Higher (university or high 37.0% 47 68.6% 83
school)
Other 2.4% 3 8.3% 10
Father's No 9.4% 12 0.0% 0
studies: Basic 52.0% 66 42.1% 51
Higher (university or high 33.1% 42 43.0% 52
school)
Other 5.5% 7 14.9% 18
%=Percentage; fx= Frequency
The mean of generic competences in Peru is 3.88 and in Cuba 3.84, which means
that both countries show Good enough level. 38.7% of them in Peru are almost
always, compared to Cuba, where 37% are always in terms of frequency of
demonstrating their competences (Table 2).
168
Table 2. Generic competences in Peruvian and Cuban students

Country Generic Competences
Peru Mean 3.88

Level Good enough
% of response N 0.7
CN 6
M 26
CS 38.1
S 29.2
Cuba Mean 3.84
Level Good enough
% of response N 2.7
CN 9.2
M 24.1
CS 27
S 37
%= Percentage; N=Never; CN=Nearly Never; M=Moderate; CS=Almost Always;
S=Always.
Table 3. Instrumental competence and subdimensions in Peru and Cuba

Country Instrumental Organization Information Solution of Decision Commu
Competence and planning management critical making nication
situations
Peru Mean 3.82 3.73 3.87 3.87 3.77 3.89
Level Good Good Good Good Good Good
enough enough enough enough enough enough
% of N 0.4 0.47 0.63 0.49 0.26 0.00
response CN 6.4 6.14 6.61 6.79 5.51 6.69
M 29.4 33.70 28.98 26.57 32.55 25.20
CS 38.2 38.98 33.07 37.89 40.42 40.55
S 25.7 20.71 30.71 28.25 21.26 27.56
Cuba Mean 3.73 3.56 3.67 3.79 3.60 4.02
Level Good Good Good Good Good Excelle
enough enough enough enough enough nt
% of N 4.1 5.45 2.98 2.27 3.03 2.07
response CN 10.7 11.24 13.22 10.74 11.29 7.02
M 24.5 29.26 28.10 27.17 32.78 19.42
CS 29.7 30.41 25.62 25.72 28.93 29.34
S 31.0 23.64 30.08 34.09 23.97 42.15
%= Percentage; N=Never; CN=Nearly Never; M=Moderate; CS=Almost Always; S=Always.
Table 3 shows that instrumental competence reaches a sufficient level, with a

mean of 3.82 (Peru) and 3.73 (Cuba). Regarding the frequency of demonstrating
instrumental competence, 67.6% of the Peruvian students answered “moderate”
and “almost always”, compared to Cuba, where 60.7% answered “almost always”
and “always”. 1. Organization and Planning subdimension: both countries
present sufficient levels, with Peru showing percentages above 30% in moderate
and almost always categories, compared to Cuba, which presents only 30.41% in
almost always category. 2. Information Management subdimension: the level of
169
this competence is sufficient in both countries. According to the percentages of

responses, the frequency of demonstrating this competence in Peru is almost
always and always, and in Cuba is always. 3. Solution of Critical Situations
subdimension: the level is sufficient in both countries. The frequency of
demonstrating the competence in Peru is almost always (37.89%) and in Cuba is
always (34.09%). 4. Decision Making subdimension: the level is sufficient in both
countries. Peru shows moderate (32.55%) and almost always (40.52%) and Cuba
shows moderate (32.78%). 5. Communication subdimension: Cuba shows an
excellent level in always category, with 42.15%, as opposed to Peru, which shows
a sufficient level in almost always category, with 40.55%.
Table 4. Interpersonal competence and subdimensions in Peru and Cuba

Country Interpersonal Teamwork Critical Self-
Competence Thinking Management
Peru Mean 3.88 3.94 3.72 3.97
Level Sufficient Sufficient Sufficient Sufficient
% of N 0.94 1.57 0.98 0.26
response CN 6.09 5.51 8.17 4.59
M 26.45 22.64 31.79 24.93
CS 37.50 37.80 36.12 38.58
S 29.01 32.48 22.93 31.63
Cuba Mean 3.84 3.78 3.74 4
Level Sufficient Sufficient Sufficient Sufficient
% of N 2.74 2.89 4.24 1.10
response CN 9.11 9.30 10.33 7.71
M 26.34 27.69 27.38 23.97
CS 25.05 27.48 23.55 24.10
S 36.75 32.64 34.50 43.11
S=Always.
Table 4 shows that the mean of interpersonal competence is 3.88 in Peru and 3.84
in Cuba, which means that both countries show a sufficient level. The frequency
of demonstrating this competence in Peru is almost always (37.5%) and in Cuba
always (36.75%). 1. In the Teamwork subdimension, students of both countries
present a sufficient level. Frequency of demonstrating this competence in Peru is
almost always (37.8%) and always (32.48%), and in Cuba is always (32.64%). 2. In
the Critical Reasoning subdimension, both countries show a sufficient level.
However, the frequency of demonstrating this competence in Peru is moderate
(31.79%) and almost always (36.12%), and in Cuba is always (34.5%). 3. Self-
Management subdimension: both countries show a sufficient level, with the
frequency in Cuba being always (43.11%) and in Peru almost always (38.58%) and
always (31.63%).
170
Table 5. Systemic Competence and subdimensions in Peru and Cuba.

Country Systemic Learning Leadership Quality Flexibility
Competence orientation motivation
Peru Mean 3.97 3.89 3.96 3.95 4.09

Level Sufficient Sufficient Sufficient Sufficient Excellent
% of N 0.73 0.98 0.79 0.94 0.20
response CN 5.63 5.91 4.27 7.40 4.92
M 22.14 24.02 24.41 21.42 18.70

CS 38.78 40.94 39.71 36.06 38.39
S 32.73 28.15 30.82 34.17 37.80
Cuba Mean 4 3,88 3,91 4.09 4.11
Level Sufficient Sufficient Sufficient Excellent Excellent
% of N 1.91 2.48 1.77 1.32 2.07
response CN 7.66 8.06 9.56 7.44 5.58
M 21.58 25.62 23.02 18.68 19.01
CS 26.32 26.24 27.63 25.79 25.62
S 42.53 37.60 38.02 46.78 47.73
S=Always.
Table 5 shows that in the systemic competence dimension, the mean of the
students in Peru is 3.97, and in Cuba the mean is 4. They show a sufficient level.
In relation to the frequency of demonstrating this competence, Peru almost always
(38.78%) and always (32.73%) demonstrates this competence and Cuba always
(42.53%) demonstrates this competence. 1. Learning Orientation subdimension:
both countries show a sufficient level. There are marked differences in the
frequency of demonstrating this competence. Cuba always has presented this
quality (37.60%), while the frequency in Peru is almost always (40.94%). 2. In the
Leadership subdimension, both countries show a sufficient level. In the case of
Peru, more than 60% of the students almost always and always demonstrate this
competence. In the case of Cuba, it is observed that the frequency is always
(38.02%). 3. In relation to Quality Motivation, Cuba stands out with a mean of 4.09
and shows an excellent level. Always category has the highest percentage in terms
of demonstration of this competence (46.78%). 4. In relation to the Flexibility
Competence, both countries show an excellent level. It is also observed that more
than 70% of the Peruvian students almost always (38.39%) and always (37.8%)
demonstrate this competence. Finally, Cuban students always (47.73%)
demonstrate this competence.
4. Discussion
For the University, the integral formation of its students constitutes a challenge,
taking as a priority challenge, by incorporating in its Educational Models the
integral formation, declaring to the generic competencies those that are desired
that all students of the institution work (Salazar Botello et al, 2019). Currently,
higher educational institutions consider relevant the strengthening of
competences for adequate performance in the labor market (Pukelis and
Pileicikiene, 2012). Within the development of competences, the production of
171
scientific knowledge through the publication of research papers (Hernández et al.,

2021), as well as the training of competent human resources for the transition to
knowledge societies and for socially sustainable economic development are
fundamental (Hernández et al., 2021) (UNESCO, 2017).
The purpose of the study was to compare the generic competences of university
students from Peru and Cuba. It was found that the mean of generic competences
in Peru is 3.88 and in Cuba 3.84, which is sufficient. In addition, 38.7% of students
in Peru almost always demonstrate the competences and in Cuba 37.0% of
students always demonstrate them. These results may be due to the fact that the
evaluated samples have adequately developed these competences that promote
relationship, integration and effective communication among peers, in a constant
social cooperation, teamwork and sense of leadership (Corominas, 2001; Solanes
et al., 2012). Based on the above, generic competences also comprise the
competences of the subject and increase the autonomy and learning that he/she
has, on which the achievement of different objectives and solutions to problems
typical of the university stage will depend (Garrido et al., 2021; Romaní-Pillpe and
Macedo-Inca, 2022). The evaluation of generic competencies should contemplate
thatstudents mobilize their previous knowledge in a realistic situation,
categorizing the student's performance in levels, proposing progressively more
complex activities and with more demanding criteria, which favor the acquisition
and mobilization of learning (Villarroel Aand Bruna, 2014).
Another important result was that in the systemic competence dimension, the
mean of the students in Peru was 3.97 and in Cuba the mean was 4, with a
sufficient level in both countries. The fact that the results of generic competences
in this dimension reach adequate levels can be due to the fact that both Peru and
Cuba are developing countries, they are in the same Latin American region and
therefore, the education policies and teaching are similar. This may have
influenced the finding of similar results (Vargas, 2011). Regardless of the country,
these generic competences are evaluated based on the graduate profile in public
and private universities in order to meet the trends and demands of the company
and the needs of the global market (Sandoval and Ormazábal, 2021). Huaiquilaf-
Jorquera et al. (2021) found that graduates were unable to attribute meaning to
the generic competencies described in the graduate profile because they were not
clear about what each one implies. It is important that within each subject,
teachers can transmit to students the development of competencies at different
levels and how they nurture the graduate profile.
Another important result reported in this study was in the interpersonal

competence dimension where the mean was 3.88 in Peru and 3.84 in Cuba, which
places both countries at the sufficient level. This means that university students in
both countries have managed to develop adequate levels of interpersonal
development, i.e., they have the skills to communicate effectively with their peers
and achieve favorable outcomes in social interaction (Hsiao, 2021). A study that
showed students with excellent social skills was conducted on university students
from Osijek-Baranja County in the country of Croatia and it concluded that there
is a positive correlation between extracurricular activities, the entrepreneurial
environment, and the level of entrepreneurial skills of the students (Perić et al.,
172
2020), showing that the entrepreneurial attitude was more important in those
university students with better social skills. Illesca-Pretty et al. (2022) indicate that
the students of the Nutrition and Dietetics course revealed greater importance to
the generic interpersonal competencies, emphasizing teamwork, ethical
commitment, critical capacity and self-criticism.
In addition, this study has some limitations in the level of analysis, which is
descriptive and comparative, so that it only compares and describes the main
developments in generic competences. However, since these developments have
not been reported in previous studies, it is important to highlight them as a
pioneering study. Another limitation is that the variable is subjective in nature, so
measurements are based on the responses of the participants of the study that may
or may not be honest. Thinking about it, in this study, at the beginning of the data
collection, we asked each participant to be honest when answering, so the results
maintain their scientific value. Additionally, this variable was collected with an
instrument with excellent psychometric properties.
Finally, it is concluded that the generic competencies in Peru and Cuba reach
adequate levels and were developed in a similar way in university students.
Further studies should verify the factors that best predict the development of
generic competences.
Likewise, it was found that the average of the dimensions of the generic
competencies variable of students in Peru and Cuba are located in the sufficient
and excellent levels, which could be due to the fact that both countries have
similar policies in higher education, considering the development of generic and
specific competencies to be important.
5. References
Aguado, D., González, A., Antúnez, M., & De Dios, T. (2017). Evaluación de Competencias
Transversales en Universitarios. Propiedades Psicométricas Iniciales del
Cuestionario de Competencias Transversales. Revista Iberoamericana sobre Calidad,
Eficacia y Cambio de Educación, 15(2), 129-152.
https://doi.org/10.15366/reice2017.15.2.007
Amor, I., & Serrano, R. (2018). Análisis & Evaluación de las Competencias Genéricas en la
Formación Inicial del Profesorado. Estudios pedagógicos, 44(2), 9-19.
http://dx.doi.org/10.4067/S0718-07052018000200009
Barriga, C. (1974). Manual autoinformativo metodología de la investigación. Universidad
Continental.
Castellar, A., Villadiego, D., Gamero, H., & Gamarra, J. (2021). Plan de acompañamiento
académico: Incidencia en el desarrollo de competencias genéricas en estudiantes
universitarios. Revista de Ciencias Sociales, 27(2), 256 - 271.
http://dx.doi.org/10.31876/rcs.v27i2.35917
Corominas, E., Tesouro, M., Capell, D., Teixidó, J., Pélach, J., & Cortada, R. (2006).
Percepciones del profesorado ante la incorporación de las competencias genéricas
en la formación universitaria. Revista de Educación, 341, 301 - 336.
Corominas E. (2001). Competencias Genéricas en la formación universitaria. Revista de
Educación, 325, 299-321.
173
Crespi, P., & García-Ramos, J. (2021). Competencias genéricas en la universidad.

Evaluación de un programa formativo. Educación XX1, 24(1), 297-327.
http://dx.doi.org/10.5944/educXX1.26846
Donoso, J., Serrano, F., & Camúñez, J. (2021). Modelling, using structural equations, the
level of satisfaction of university accounting teachers with final year dissertations.
Revista de Contabilidad Spanish Accounting Review, 24(1), 135-151.
http://dx.doi.org/10.6018/RCSAR.368451
García-Atarés, N., Santiago, R., & Jimeno, N. (2021). Comparación entre las Competencias
Transversales/Genéricas del Logopeda en España y Europa. Propuesta de
Reformulación. Revista de Investigación en Logopedia, 12(1), 1-11.
https://dx.doi.org/10.5209/rlog.76525
Garrido, A., Salgado, F., Paz, C., & Blanc, P. (2021). Competencias genéricas en la práctica
profesional de la carrera Ingeniería Civil Eléctrica. El discurso entre la academia
y la industria. Revista chilena de ingeniería, 29(2), 214-228.
http://dx.doi.org/10.4067/S0718-33052021000200214
Godoy, J., Vidal, A., Illesca, M., Espinoza, E., & Flores, L. (2021). Rol de Tutor par:
experiencia de estudiantes de medicina. Revista Médica Chile, 149, 765-772.
http://dx.doi.org/10.4067/s0034-98872021000500765
Hernández-Sampieri, R., & Mendoza, C. (2018). Metodología de la investigación: Las rutas
cuantitativa, Cualitativa y Mixta. Mc Graw–Hill Interamericana.
Hernández, R., Saavedra-López, M., Wong-Fajardo, E., Campos-Ugaz, O., Calle-Ramírez,
X., & García-Pérez, M. (2021). Producción científica iberoamericana sobre TIC en
el contexto educativo. Propósitos y Representaciones, 9(3), e1443.
http://dx.doi.org/10.20511/pyr2021.v9n3.1443
Hernandez, R., Montes-Valer, E., Mamani-Benito, O., Ortega-Pauta, B., Saavedra-Lopez,
M., & Calle-Ramirez, X. (2022). Index of Attitude towards Scientific Research in
Peruvian Psychology Students. International Journal of Education and Practice, 10(2),
204-213. http://dx.doi.org/10.18488/61.v10i2.3046
Huaiquilaf-Jorquera, S. Illesca-Pretty, M., González-Osorio, L., & Godoy-Pozo, Jessica.
(2021). Competencias genéricas: opinión de egresados de Kinesiología. FEM:
Revista de la Fundación Educación Médica, 24(4), 191-197.
https://dx.doi.org/10.33588/fem.244.1135
Illesca-Pretty, M., Barraza-Tello, A., Hernández-Díaz, A., González-Osorio, L., & Godoy-
Pozo, J. (2022). Competencias genéricas: opinión de estudiantes de la carrera de
Nutrición y Dietética, Universidad de Atacama-Chile. Archivo Médico Camagüey,
26, e8645. http://www.revistaamc.sld.cu/index.php/amc/article/view/8645
Instituto Nacional de Estadística e Informática (INEI, 2021). Perú: Mujeres y hombres
ocupadas/os, según nivel de educación alcanzado y ámbito geográfico.
Consultado el 20 de julio del 2022.
Koster, B., Brekelmans, M., Korthagen, F., & Wubbels, T. (2005). Quality requirements for
teacher educators. Teach-ing and Teeacher Education, 21, 157–176.
https://doi.org/10.1016/j.tate.2004.12.004
Nabaho, L. (2017). Developing Generic Competences in Life Sciences: The Untold Story of
the Makerere University College of Health Sciences in Uganda. Tuning Journal for
Higher Education, 4(2), 389-406. https://doi.org/10.18543/tjhe-4(2)-2017pp389-
406.
McClelland, D.C. (1973). Testing for Competence Rather Than for “Intelligence”. American
Psychologist, 28(1), 1-14. https://doi.org/10.1037/h0034092
Perić, Oberman, & Getoš (2020). The role of vocational education in developing
entrepreneurial competences of students. Ekonomski pregled, 71(5), 463 - 492.
https://doi.org/10.32910/ep.71.5.2
174
Pukelis, K., & Pileicikiene, N. (2012). Matching of Developed Generic Competences of

Graduates in Higher Education with Labour Market Needs. The Quality of Higher
Education, 9, 140-167. https://doi.org/10.7720/1822-1645.9.6.
Romaní-Pillpe, G., & Macedo-Inca , K. S. (2022). Aprendizaje basado en problemas para el
desarrollo de las competencias generales en estudiantes universitarios. Revista
Educación, 20(20), 49–56.
https://doi.org/10.51440/unsch.revistaeducacion.2022.20.234
Sandoval, L., & Ormazábal, M. (2021). The Generic skills challenge for higher education
institutions: Experience of public universities in Chile. Tuning Journal for Higher
Education, 8(2), 55–83. https://doi.org/10.18543/tjhe-8(2)-2021pp55-83
Salazar Botello, C. M., Olguín Gutiérrez, C. E., & Muñoz Jara, Y. A. (2019). Competencias
genéricas en la educación superior: una experiencia en la Universidad del Bío-Bío,
Chile. Revista Venezolana de Gerencia, 24(87), 807-825.
ttps://doi.org/10.37960/revista.v24i87.24638
SINEACE (2017). Modelo de acreditación para programas de estudios de educación
superior universitaria [Internet]. Perú. https://www.sineace.gob.pe/
Solanes, Á., Núñez, R., & Rodríguez, J. (2012). Elaboración de un cuestionario para la
evaluación de competencias genéricas en estudiantes universitarios. Apuntes de
Psicología, 30(1-3), 513 - 522.
United Nations Educational, Scientific and Cultural Organization (2017). UNESCO's
Strategy for Technical and Vocational Education and Training (2016-2021).
Consultado el 10 de junio del 2022.
http://www.unevoc.unesco.org/go.php?q=UNESCO+TVET+Strategy+2016-
2021&context=
Vargas, J. V. (2011). Estudio comparativo de estructuras de educación y atención a la
infancia en siete países: Australia, Bélgica, Brasil, Canadá, Cuba, Francia y Perú.
Educación, 20(39), 93–113.
https://revistas.pucp.edu.pe/index.php/educacion/article/view/2489
Vélez, A., Delgado L., Sánchez W. (2018). Análisis prospectivo de las competencias
genéricas Tuning-Alfa en la ciudad de Medellín al 2032. El Ágora USB, 18(1), 130-
151. https://doi.org/10.21500/16578031.3446
Vega, R., & De Armas, R. (2009). Tuning-América Latina y su compatibilidad con el
modelo curricular cubano. Reencuentro. Análisis de Problemas Universitarios, (54),
73-82. https://www.redalyc.org/articulo.oa?id=34012025007
Villanueva, G. (2014). Competencias en estudiantes universitarios: elaboración y validación de
un instrumento para la mejora de la mejora de la calidad universitaria (tesis doctoral).
Universidad Complutense de Madrid.
Villarroel, V., & Bruna, D. (2014). Reflexiones en torno a las competencias genéricas en
educación superior: Un desafío pendiente. Psicoperspectivas, 13(1), 22-34.
https://dx.doi.org/10.5027/psicoperspectivas-Vol13-Issue1-fulltext-335
Yorke, M., & Harvey, L. (2005). Graduate attributes and their development. In R. A.
Voorhees & L. Harvey (Eds.), Workforce development and higher education: a
strategic role for institutional research (pp. 41–58). Jossey-Bass.
https://doi.org/10.1002/ir.162.
Zabala, A., & Arnau, L. (2008). Once ideas clave. Cómo aprender y enseñar competencias. Graó.
Zahner, D., Van Damme, D., Benjamin, R., & Lehrfeld, J. (2021). Measuring the generic
skills of higher education students and graduates: Implementation of CLA+
international. In S. A. Nolan, C. M. Hakala, & R. E. Landrum (Eds.), Assessing
undergraduate learning in psychology: Strategies for measuring and improving student
performance (pp. 219–241). American Psychological Association.
https://doi.org/10.1037/0000183-015
175

Vol. 21, No. 8, pp. 175-201, August 2022
Representation of Nature of Science Aspects in

Secondary School Physics Curricula in East
African Community Countries
Jean Bosco Bugingo*

and Science (ACEITLMS),
University of Rwanda- College of Education (UR-CE), Rukara Campus, Rwanda
Lakhan Lal Yadav

University of Rwanda- College of Education (UR-CE), Rukara Campus, Rwanda
Department of Mathematics, Science and Physical Education, UR-CE, Rukara
Campus, Rwanda
K. K Mashood
Homi Bhabha Centre for Science Education, TIFR, Mumbai, India
Abstract. For several decades, nature of Science (NOS) has been

advocated as the fourth dimension of science teaching and is a
fundamental source of in-depth learning and teaching. In addition to
improving learning and teaching of science, the explicit inclusion of NOS
in science curricula helps the creation of a responsible citizenry. Here, we
analyze the representation of NOS aspects in science curricula,
particularly in the physics syllabi in four East African Community (EAC)
countries: Burundi, Rwanda, Tanzania, and Uganda. These EAC
countries have been purposively selected because of sharing similar
culture and history as neighboring countries. To compare NOS
representation in the physics content, five major topic areas (mechanics,
heat and thermodynamics, oscillations and waves, electricity, and atomic
physics) were randomly selected from the syllabi used in advanced level
secondary schools. The paper critically analyzes the representation of
NOS aspects throughout front matter (introductions and rationales) and
back matter (appendices and references), content, teaching methods, and
assessment procedures proposed in these physics’ syllabi. Based on the
analysis of data, the findings reveal that NOS aspects are not explicitly
represented in the four physics syllabi analyzed. This study also found
that in four syllabi reviewed, competencies were given much attention
*
Corresponding author: Jean Bosco Bugingo, bugingo2012@gmail.com
©Authors
176
without any overt connection to the work of scientists. Finally, we suggest

possible ways to improve NOS representation in the science curriculum.
Keywords: East African Community (EAC) countries; The Nature of

Science (NOS); NOS aspects; secondary schools; science curriculum
1. Introduction
The nature of science (NOS) is considered a vital component in science education
(Jenkins, 2013). Many science education reform documents around the world
documented a strong correlation between scientific literacy and the
understanding of the nature of science (NGSS Lead States, 2013). According to
Das et al. (2019), understanding NOS is the cornerstone of informed views that
can stimulate students’ understanding of science. Lederman et al. (2002)
emphasized that understanding the NOS assures students’ abilities to assess
scientific knowledge, which then acts as a driving force in engaging students in
using inquiry skills (Liang et al., 2008).
A lot of efforts have been made in science education, but some challenges still
exist, and they inhibit the quality of teaching and learning (Lederman, 2007;
Hipkins, 2012). One of the challenges is that many science curricula give much
attention to content (Cheung, 2020) rather than to the process of knowledge
construction (McDonald & Abd-El-Khalick, 2017; Çetin & Kahyaoğlu, 2022).
Second, a strange and persistent habit of viewing science as an irrelevant and
difficult subject was reported as a strong obstacle to students learning science
(Hipkins, 2012). In addition, the NOS content in the science curriculum is not yet
represented in an informed manner (Vesterinen et al., 2013; Schrauth, 2009).
Furthermore, lack of explicit NOS instructions, reluctance to the positive change
toward NOS, and limited practical examples for easy NOS instructions in
classrooms hold back science education progress (Boe et al., 2011).
Including NOS aspects in the science curriculum has been considered a practical
solution to overcome the challenges mentioned above (Hipkins, 2012; Lederman,
2007; Martín‐Díaz, 2006). For example, science educators view NOS as a new lens
to allow a type of learning that gives much attention to both active engagement
and equity among students from different backgrounds. In addition, having the
NOS component in the science curriculum create a responsible citizenry capable
of making rational decisions and positively impacts students to pursue a career in
science (Boe et al., 2011; Lederman et al., 2013). Furthermore, it can help or guide
the development of teaching and learning packages such as textbooks and other
learning materials (Olson, 2018) to support the advancement of science education.
It is within this regard, that science curriculum developers from some Middle East
countries (Yeh et al., 2019) and those from developed countries were inspired to
develop a curriculum that clearly states how NOS aspects should be integrated
and taught (Taber, 2008). Some recommendations were stated for the recognition
and inclusion of NOS teaching in the national curriculum of England and Wales.
Science curricula are the principal teaching resources that play a big role in giving
a clear direction to the teaching and learning process (Olson, 2018). Science
177
curricula play a key role in defining what science teachers are supposed to teach
and thereby guiding learning experiences (Chiappetta et al., 2006). In this regard,
the modern science curricula should not only be focused on the content and
practical work, but also on social, historical, and philosophical aspects, which are
referred as NOS aspects (Childs, 2015). Therefore, effective teaching and learning
of NOS aspects would be easier if NOS concepts were explicitly defined in science
curricula.
Explicit teaching and learning of NOS, aspects have been advocated as important
issues in the science curriculum at the different levels of education (Bell et al.,
2011; NRC, 2012; NGSS Lead States, 2013). For example, after the development of
the NOS benchmarks, the United States of America (USA), through its National
Academy of Sciences, established standards on how NOS could be integrated into
the science curriculum (Taber, 2008). Likewise, New Zealand also developed six
strands of the science curriculum in which four strands discuss science content
while the two remaining focus on incorporating NOS to develop scientific skills
and attitudes (Hipkins, 2012). However, a number of researchers still claim that
there is a lack of explicitely stated NOS aspects in science curricula (Caramaschi
et al., 2022). Therefore, there is a need for clarification and specification of NOS
aspects in science curricula and other educational documents to break down NOS
content into a simplified form to allow easy understanding among teachers and
students.
Although there are little literature interventions available on NOS in African

science curricula, a few African countries have tried to refine their science
curricula by introducing NOS components. For example, Ogunniyi (2006) reports
that the 2005 South African natural science curricula was identified as one that
may help learners develop NOS understanding. However, the implicit approach
was found dominating (Upahi et al., 2020).
Science curricula reforms were carried out in East African countries from around
the 1960s after their independence (Mbonyiryivuze et al., 2018). Since then, science
has continued to be a top school priority (Cairns, 2019), and it has been given
much attention to facilitating the central goal of economic development
(UNESCO, 2009). However, a few efforts to promote science in the region were
put in place but could not last long due to the political instabilities, wars,
destruction of infrastructures, a big loss of human capacity in the EAC (UNESCO,
2009), and the severe impact of Covid-19 on the education system (Tugirinshuti et
al. 2021). In addition to this, efforts to promote NOS aspects in the science
curricula in East African countries are very limited (Kinyota, 2020). This situation
of very little literature and limited interventions on NOS in teaching aids materials
is worrisome, particularly in East African Community countries (Kinyota &
Rwimo, 2022). Referring to the vital role of NOS in improving informed views
among students, Ramnarain and Chanetsa (2016) urged that the current school
science curricula should be designed to help students to learn NOS. Therefore,
there is a need for study on representation of NOS in the learning and teaching
materials used in the region.
178
Purpose
The purpose of this study is to investigate the representation of NOS concepts in
advanced physics syllabi from East African Community (EAC) countries.
Specifically, it sought an answer to the following research question: “How are
NOS aspects represented in the front matter and back matter, learning outcomes,
content being taught, teaching and assessment methods in selected physics syllabi
in EAC countries?”
2. Methodology
Research Approach Design
The main research approach used to get in-depth understanding of NOS
representations in this paper is document analysis (Bower, 2009). This research
design is used to determine the NOS aspects that have been integrated into the
front matter and back matter, expected learning outcomes of students, content to
be taught, and methods of teaching and assessment of analyzed syllabi. The
analytical framework in this review includes eight NOS aspects that have been
documented in several studies (Lederman, 2007; Lederman et al., 2002; Chaisri &
Thathong, 2014).
Selection of Countries, Syllabi, and Major Topic Areas Studied

We reviewed the physics syllabi of grades 10, 11, and 12 from Burundi, Rwanda,
Tanzania, and Uganda, as these countries are neighbors and share similar cultures
and history. The syllabi analyzed include an advanced level physics syllabus from
Rwanda (REB, 2015); three physics teacher’s guides from senior four to senior six
used in Burundi (Ministère de l’ Education, de l’ Enseignement Superieur et de la
Recherche Scientifique, 2016; 2017 and Ministère de l’ Education, de la Formation
Technique et Professionnelle, 2018) [Ministry of Education, Higher Learning and
Scientific Research, 2016; 2017 and Ministry of Education, Technical Training and
Professional, 2018]; teaching syllabi for physics and mathematics, volume 2 from
Uganda (NCDC, 2013); and Physics syllabus for advanced secondary Education,
form V to VI from Tanzania (Ministry of Education and Vocational Training,
2017).
The authors randomly selected five major topic areas from grades 10, 11, and 12
physics syllabi. Major topic areas selected to be analyzed for their NOS
representation are mechanics, heat and thermodynamics, oscillations and waves,
electricity, and atomic physics. Google translate and experts in science education,
particularly physicists with bilingual skills, were used to translate the French
content of analysed syllabi into English.
Participants of the Study

We investigate the representation of NOS aspects in four physics syllabi that are
used in East African Community countries. Three researchers/experts,
participated in this study in which every researcher investigated and analyzed
independently physics syllabi considered for allowing another opinions and
discussion and come to consensus.
179
Data Analysis
Any statement that addresses process skills or competencies in the physics syllabi
from EAC countries but does not make an explicit connection to the work of
scientists was not considered a NOS statement. For example, engaging someone
in using process skills like imagination and creativity without any connection to
the scientific enterprise does not mean that he/she is knowledgeable about NOS
aspects. In other words, to be counted the statements should reflect historical,
philosophical, social, and psychological perspectives of science.
The NOS statement to be considered as explicitly represented had to meet the

following criteria: (i) the statement should be an informed representation of NOS
aspects, as described in current NOS reform documents in science education, and
(ii) any statement has to be consistent throughout the whole physics syllabi in
addressing the targeted NOS aspects. In other words, NOS statements were
viewed throughout the whole physics syllabi in such a way that they are clearly
defined in any section of the selected physics syllabi, and they were described as
“explicitly presented and coded as (+)”. For any NOS statement to be viewed as
implicitly represented in the selected physics syllabi, the following criteria were
supposed to be met: (i) an informed representation of the NOS aspect could be
inferred from the physics syllabi materials (e.g., statements from the front matter
and back matter, content or any assessment tool that could infer the NOS
representation) and (ii) for any NOS statement that possibly would be learned or
taught as teachers teach normal content was taken as “implicitly presented and coded
as (×).” Finally, any NOS aspect which does not appear either explicitly or
implicitly was described as “not represented and coded as (-).”
3. Findings
3.1 NOS Representations in Front Matter and Back Matter of Physics Syllabi
The front matter of physics syllabi analyzed in this paper is made up of
backgrounds and rationales of the syllabi, competencies, general or broad aims of
science education, particularly related to teaching physics. At the same time, back
matters are extended parts such as references, additional readings, and
appendices of these syllabi to help the user understand the content package of
these syllabi. The analysis found that NOS concepts are not explicitly represented
in the analyzed documents.
Table 1 shows that there is very little representation of NOS aspects throughout
the four physics syllabi analyzed. Few concepts and statements which can
implicitly support teaching NOS aspects were identified, particularly in front
matters and students’ learning outcomes sections, as shown in Table 1. Among
eight NOS aspects that this study focused on, only six NOS aspects (tentative;
observations and inferences; the relationship between theories and laws; creative
and imaginations; scientific method and social and cultural embeddedness) were
supposed to be taught effectively if these concepts and statements identified are
explicitly linked to the work of scientists and supported throughout the physics
syllabi analyzed. In addition, Table 1 also shows that no statement identified may
support either explicit or implicit teaching of empirical and theory-laden aspects.
180
Table 1. Summary of NOS representation in the front matter, learning outcomes, and
back matter of four selected physics syllabi from EAC (FM – Front Matter, LO – Student
Learning Outcomes, and BM – Back Matter).
NOS aspects Burundi Rwanda Tanzania Uganda

FM LO BM FM LO BM FM LO BM FM LO BM
Tentative × – – – × – – – – – – –
Empirical – – – – – – – – – – – –
Observations – × – – × – – – – – – –
and inferences
Theory – laden – – – – – – – – – – – –
Creative and – × – – – – – – – – – –
imaginations
Relationship + – – – × – × × – – – –
between
theories and
laws
Social and × – – × – – – – – × – –
cultural
embeddedness
Scientific – – – – – – – × – – – –
method
+: explicitly presented, –: not represented, and ×: Implicitly presented
The NOS representation through the introductions of these syllabi revealed that
they mainly focused on shifting from content-based syllabi to competencies-based
syllabi. They also encourage a learner-centered approach, cross-cutting issues,
and advocate the removal of outdated or irrelevant content for facilitating smooth
and deep learning and teaching of physics subjects (REB, 2015; MoETV, 2017;
NCDC, 2013 and Ministère de l’ Education, de l’ Enseignement Superieur et de la
Recherche Scientifique, 2017). The above description of the main purpose of the
syllabi may be considered to promote the teaching and learning of NOS if
mentioned content, competencies, and skills are explicitly connected to the work
of scientists. Interestingly, it was not the case in the context of the four syllabi
mentioned above.
Although a few statements in front matter seem to show that NOS may be learned,
but still, these statements do not contain NOS concepts as described in educational
reform documents, and they are also not overtly discussed in the content to be
taught or in students’ learning outcomes to assure smooth learning of NOS. For
example, “the ambition of the new Physics syllabus in Rwanda is to develop a knowledge-
based society and hence promotes science and technology” (REB, 2015, p. viii). In other
words, this syllabus tries to link science, society, and technology together as NOS
targets too. Tanzania’s A-Level physics syllabus indicates that learners should
focus “on investigating natural phenomena and then applying patterns, principles,
theories, and laws to explain the physical behavior of the universe” (MoEVT, 2017, p. iii.
It appears, from this statement, that teaching and learning NOS may be possible
if these natural phenomena and related theories and laws are linked to the
scientific enterprise. However, there is no clear trace in proposed content and
181
student learning outcomes showing how knowledge of scientific enterprise

would be integrated, so NOS was indicated as not overtly emphasized in this case.
In the physics syllabus of grade 11 from Burundi, a statement described in the

preface section states that “integration of this reform will help the student to be a change
agent and a source of social and scientific development through teaching and learning
activities found in the learning package that is currently in place” (Ministère de l’
Education, de l’ Enseignement Superieur et de la Recherche Scientifique, 2017, p.
3). In addition to this, another statement related to NOS representation was
identified in the introduction section of the grade 10 physics syllabus, where it
states that “physics is a study of the external world, how physics laws change and hence
the evolution of physics” (Ministère de l’ Education, de l’ Enseignement Superieur
et de la Recherche Scientifique in Burundi, 2016, p.5). In other words, the above
statements give an impression of NOS consideration, but it is not the case because
it is not supported throughout the syllabus.
In Uganda’s physics syllabus, a statement, which seems to implicitly promote

NOS understanding, states that “one of the broad aims of education in Uganda is to
promote scientific, technical, cultural knowledge, skills, attitudes needed to promote
development” (NCDC, 2013, p. viii). A major problem with physics syllabi analyzed
from East African Countries is that they do not specify how these statements
would be connected to the work of scientists throughout different sections made
up of these syllabi of physics. Analysis of the back matter of these syllabi shows
that there is no appendix or additional reading which would support explicitly
teaching and learning of NOS, and this confirms the low level of representation
of NOS and no attention is given to NOS component in syllabi of physics in the
East African Community region.
3.2 NOS Representations in the learning outcomes from A-Level Physics syllabi
In context of the East African Community countries, expected learning outcomes
refer to the knowledge, skills, and aptitudes which every student should be able
to demonstrate at the end of the physics curriculum.
Table 2. Students’ expected learning outcomes in EAC countries

EAC country Students learning outcomes Observations related to
the representation of
NOS aspects
Burundi By the end of the physics syllabus, the Curiosity, imagination,
(Ministère de student should be able to: and creativity with an open
l’Education, • Analyze, interpret and solve problems mind seem to be linked to a
de related to states of matter, static solids, NOS aspect, but it is not
l’Enseigneme heat, fluids, cosmology, environment, explicitly described how
nt Superieur vapor and humidity, viscosity, and this aspect should be
et de la light. linked to the scientific
Recherche enterprise while viewing
Scientifique, • Solve current and important problems the whole physics syllabi.
2016, pp. 5–6; related to electricity, electromagnetism,
2017, p. 1) and mechanics (kinetics).
[Ministry of
Education,
182
Higher • Solve current and important problems

Learning, related to periodic phenomena,
and Scientific alternative current, fluids mechanics,
Research, atomic physics, and electronics
2016, pp. 5–6;
2017, p. 1] Candidates should also be able to
develop the following attitudes and
(Ministère de competencies:
l’Education, • Observation and communication skills
de la • Curiosity, imagination, and creativity
Formation skills with an open mind
Technique et • Accuracy and precision
Professionnel • Critical thinking skills
le, 2018, p. 3) • Self-respect and respect views of others
[Ministry of • Interest in scientific and technical
Education, development to serve society
Technical • Awareness of food security laws
training, and • Environmental awareness
Professional, • A citizen who can continue further
2018, p. 3] studies
• Relationship between physics and other
scientific disciplines
Rwanda Knowledge and understanding Some concepts of NOS,
Candidates should be able to such as scientific facts,
(REB, 2015, demonstrate knowledge and laws, scientific processes,
pp. xix–xxi) understanding of: pattern recognition,
• Scientific phenomena, facts, laws, observation, interpretation
definitions, concepts, and theories. of data, prediction, etc., are
• Scientific vocabulary, terminology, and weakly represented in
conventions (including symbols, learning outcomes, and the
quantities, and units). link between science and
• Scientific instruments and apparatus society is very weakly
used in Physics, including techniques of shown. Moreover, these
operation and aspects of safety. are not coherently
• Scientific quantities and their presented, and their link
determination. with the scientific
• Scientific and technological enterprise is not explicitly
applications, with their social, made. So, there is a weak
economic, and environmental representation of NOS
implications. aspects.
Handling information and solving
problems
Candidates should be able to handle
information and solve problems, using
written, symbolic, graphical, and
numerical forms of presentation to:
• Locate, select, organize and present
information from a variety of sources.
• Translate information from one form to
another.
• Manipulate numerical and other data.
• Use the information to identify patterns,
report trends, and draw conclusions.
183
• Give reasoned explanations for

phenomena, patterns, and
relationships.
• Make predictions and hypotheses.
• Apply knowledge, including principles,
to new situations.
• Demonstrate an awareness of the
limitations of physics theories and
models.
• Solve problems.
Experimental skills and investigations

Candidates should be able to:
• Observe, give feedback, and plan
experiments and investigations.
• Collect, record, and present
observations, measurements and
estimates.
• Analyze and interpret data to reach
conclusions.
• Evaluate methods and quality of data
and suggest possible improvements.
• Use ICT in solving problems.
Tanzania By the end of the physics syllabus, the Few concepts of NOS, such
(Ministry of student should be able to: as scientific theories and
Education • Understand the language of Physics. laws are mentioned. A
and • Explain theories, laws, and principles of scientific method has been
Vocational Physics. identified, but it is not
Training, • Understand the scientific method in supported throughout the
2017, p. v) solving problems. physics syllabus. In other
• Promote scientific and technological words, it is not linked to
knowledge and skills in management, the work of scientists.
conservation, and sustainable use of the
environment.
• Promote manipulative skills to manage
various technological appliances.
• Promote self-study for self-
advancement in new frontiers of
Physics.
• Appreciate the role of ICT in the process
of learning Physics.
Uganda At the end of the Physics syllabus, the A few concepts of NOS,
(NCDC, 2013, learners should be able to: such as scientific theories,
p. 6) • Recognize problems that can be dealt models, scientific
with using methods, concepts, processes, investigation,
principles, models, and theories of and experimentation, are
physics very poorly represented in
• Recognize the use of and manipulation learning outcomes, the link
of the apparatus and equipment between science and
common in a physics laboratory society is not shown.
• Design and carry out practical Moreover, their link with
investigations and experiments, the scientific enterprise is
not made. So, there is a
184
describe and explain the procedures very poor representation of

used as well as their effectiveness and NOS aspects.
limitations
• Handle all practical work with the
accuracy required to obtain the desired
results
• Define terms related to various concepts
in physics and explain their relationship
to materials and phenomena in the
environment.
• Discuss the use and effectiveness of
theories or models in explaining
physical phenomena as well as events in
the laboratory and the environment.
Table 2 shows that the NOS component in learning is represented very poorly.
For example, referring to NOS dimensions recommended to be in the science
curriculum (Lederman et al., 2002) and expected learning outcomes (REB, 2015;
NCDC, 2013 and MoEVT, 2017, it is very clear that NOS is not formally and
explicitly recognized in these syllabi. In other words, these expected learning
outcomes mainly focus on understanding the content and practical work. In
addition, it is also not easy to see clearly in any of the above-mentioned learning
outcomes how a student may be helped to develop skills related to scientific
knowledge construction.
3.3 NOS Representations in Content to be taught to students

We randomly selected five topic areas from grades 10, 11, and 12 physics syllabi
of all countries that have been considered in this paper. Major topic areas to be
analyzed for their NOS representation are mechanics; heat and thermodynamics,
oscillations and waves; electricity; and atomic physics. The analysis of the content
throughout these four physics syllabi shows that there is very little representation
of NOS aspects in almost all units discussed in those curricula (Tables 3 to 7).
We found that all physics students of grade 11 from Rwanda, Uganda, and
Tanzania and physics students of grade 12 from Burundi are supposed to study
‘waves.’ For example, in Rwanda, the term wave is under a unit called
“oscillations and waves,” in Uganda, it is under a unit named ‘Waves’ while in
Tanzania, the term “wave” is under a unit called ‘vibrations and waves’ as shown
in Table 5 below. In Burundi, the term “wave” is described only in grade 12 under
two units named: “interferences and stationary waves” and “sound waves.” The
topic of heat and thermodynamics is discussed to grade 10 students from Rwanda
and Burundi, while the same topic is taught to grade 11 and 12 students from
Tanzania and Uganda, respectively as shown in Table 4.
Projectile motion is under mechanics, and is taught to all students of grade 10 from
Rwanda and grade 11 from Tanzania, Burundi, and Uganda ( Table 3). The topic
on electricity is taught to grades 10 and 11 students from Rwanda, grade 12 from
Tanzania and Uganda, and grade 11 from Burundi ( Table 6). Atomic physics is
taught to students of grade 11 from Rwanda and all students of grade 12 from
Tanzania, Burundi, and Uganda (Table 7).
185
Differences were noted during the selection of the topics included in this paper.
First of all, it was noted that there is a difference in students’ levels. For example,
in Rwanda and Burundi, the advanced level of secondary school is from grade 10
to 12, while in Uganda and Tanzania, students are in the advanced level of
secondary school from grade 11 to grade 12. Second, it was identified that some
topics were taught from grade 10 up to grade 12 in some countries but not all in
other countries. For example, the topic of astrophysics and environmental physics
is taught to advanced-level students in Rwanda, Burundi, and Tanzania but not
taught to the same level students in Uganda.
Table 3. Selected content of projectile motion to be taught to A-level students from

EAC
EAC Countries Topic areas Unit to Unit objectives Content Observations
be related to the
taught representation of
NOS aspects
Burundi Mechanics Projectile • study the • composition of None of the NOS
motion composition of two uniform statements was
(Ministère de two uniform rectilinear identified.
l’Education, de rectilinear movements
l’Enseignement movements • Derivation of
Superieur et de la • demonstrate that projectile
Recherche the range of the motion
Scientifique, flight results from parameters
2017, pp. 215– the composition of
216) two movements
[Ministry of • calculate the
Education, horizontal shot
Higher Learning, range
and Scientific
Research, 2017,
pp. 215–216]
186
Rwanda Projectile • Define and explain • Definition of None of the NOS

motion terms used in projectile aspects identified.
(REB, 2015, pp. projectile motion and
14–15 ) • Relate projectile related terms.
motion to linear. • Applications of
• Appreciate projectile
applications of motion.
projectile • Graphs of
• Resolve projectile projectile
motion in motion.
horizontally and • Expressions of
vertically projectile
components. motion
• Derive equations (horizontal
of projectile range and
motion. maximum
• Determine the height).
maximum height
and horizontal
range in projectile
motion
Tanzania Projectile • Describe projectile • Projectile None of the NOS

motion motion parameters motion statements
(Ministry of • Derive projectile parameters identified.
Education and motion parameters • Derivation of
Vocational • Describe the projectile
Training, 2017, p. applications of motion
6) projectile motion parameters
• Applications of
projectile
motion
Uganda Projectile • Define flight and • Projectiles None of the NOS
motion range. (projectile statements
(NCDC, 2013, p. • Calculate time for motion on an identified.
28) flight, maximum inclined plane
height, and range. is beyond the
• Describe the scope)
applications of • Time of flight,
profile motion. maximum
height, and
range (range
along a
horizontal
plane only)
• Applications of
projectile
motion
It would be vital if students were helped to learn the history of mechanics from
Aristotle to Newton’s period and understand why some of the scientific
explanations given by different physicists were refuted. For example, it would be
187
better to discuss how projectile motion posed a problem to the second type of
motion proposed by Aristotle, which is known as the “violent motion” of a body
and was defined as a compulsory motion caused by an external influence (Rovelli,
2015). In addition, it was not easier to understand why a projectile continues to
move while it is separated from its launcher (Barahona et al., 2014). Contrary to
this, the unit objectives and content of projectile motion proposed by curriculum
developers in physics syllabi from EAC encourage memorization of the content.
For example, the verbs like define, calculate and derive, most of the time, help
students cram how different mechanical concepts would be derived rather than
linking these concepts to the work of knowledge construction, which in the end
may help learners interpret or understand natural phenomena in mechanics in a
better way.
Table 4. Selected content of heat and thermodynamics to be taught to A-level students

from EAC
EAC Topic Unit to be Unit objectives Content Observations
Countries areas taught related to the
representation of
NOS aspects
Burundi Heat and State of • Interpret different • Temperatur Temperature

Thermody matter and temperature scales es scales scales might be
(Ministère de namics Calorimetr • Specific heat used to teach
l’Education, de y • Distinguish different capacity NOS, however, it
l’Enseignemen states of matter • States of is not explicitly
t Superieur et • Differentiate matter mentioned in this
de la •
different phenomena Changes in content.
Recherche in changes in states states of
Scientifique, matter
of matter
2016, pp. 167– • Expansion of
168) • Explain the solids,
[Ministry of expansion in solids, liquids and
Education, liquids and gases. gases
Higher
Learning, and
Scientific
Research, 2016,
pp. 167–168]
Rwanda Applicatio • Differentiate the • Internal None of the NOS
ns of internal energy and energy and aspects identified.
(REB, 2015, pp. thermodyn total energy total energy
19 – 20 ) amics laws
• Explain the work • Work done
done by
expanding
• The state first, gas
second laws of
thermodynamics • First and
and their second laws
applications of
thermodyna
• Solve problems mics
related to Carnot
188
cycle, Carnot engine, • The

diesel engine, applications
refrigerators, of the first
and second
laws of
thermodyna
mics
• The
efficiency of
the heat
engine
• Heat engine
and climate
change
Tanzania Heat • Describe the • Thermomete None of the NOS

thermometric rs statements was
(Ministry of properties of a • Heat identified.
Education and substance and scale transfer
Vocational of temperature • The first law
Training, 2017, of
pp 18 – 22) • Explain ways of thermodyna
thermal heat transfer mics
• Explain
thermodynamics
processes
• Identify specific heat

capacity of gases
• Establish the first

law of
thermodynamics
and its applications
Uganda Thermody • Definer internal • Internal None of NOS

namics energy of an ideal energy statements
(NCDC, 2013, gas • Work done identified.
pp 73 – 74) • Derive the work by
done expanding
• Explain ideal gas
thermodynamics • Thermometr
processes ic processes
• Define principal • Principal
specific heat specific
capacities Cp and Cv capacities
of an ideal gas • The first law
• Define the first law of
of thermodynamics thermodyna
and its applications mics and its
applications
189
Looking at the proposed content in Table 4, it is very clear that this content may
not help students to develop a philosophical background in thermodynamics. In
addition, it has been identified that NOS concepts are not explicitly included.
Although the topic does not define how content is linked to the work of scientists,
such as the historical development of concepts of heat, calorific properties, as well
as thermometers, temperature scales, such linking could be very useful in NOS
teaching.
Table 5. Content of waves to be taught to grades 11 and 12 students from EAC

EAC Topic areas Unit to be Unit objectives Content Observations
Countries taught related to the
representation of
NOS aspects
Burundi Oscillations • Define and • Interference None of the NOS
and waves interpret the phenomena statements
(Ministère interference • Constructive identified.
de Interferenc phenomenon and destructive
l’Educatio es and • Distinguish interferences
n, de la stationary constrictive and • Application of
Formation waves destructive the interference
Technique interferences • Stationary wave
et • Apply interference • Different
Profession phenomenon stationary wave
nelle, 2018, • Define stationary phenomena
pp. 136 - wave
173) • Interpret
[Ministry phenomena of
of stationary waves
Education, • Identify different
Technical stationary waves
Training, • Define sound wave • Sound wave
and • Determine the • Nature of sound
Profession longitudinal nature wave
al , 2018, Sound of sound wave • Characteristics
pp. 136– waves • Identify of a sound
173] characteristics of a • Properties of
sound sound waves
• Identify properties
of sound waves
Rwanda Propagati • Explain the wave • Wave concept. None of the NOS
on of concept. statements
(REB, 2015, mechanica • Types of waves. identified.
p. 28) l waves • Explain the terms
amplitude, • Waves Terms.
frequency, • Characteristics
displacement, of waves.
wavelength, and
wave phase. • Relationship
between
• Explain the terms wavelength,
transverse and frequency
longitudinal waves.
190
• Explain the terms (Period), and

progressive and velocity.
stationary waves.
• Properties of
• Explain the phase waves
of vibration. (Reflection,
refraction,
• Explain reflection, interference,
refraction, diffraction).
diffraction and
interference of • Young double-
waves. slit experiment.
• Explain Young • Progressive and

double-slit stationary
experiment. waves.
• Equation of a
progressive
wave.
• Example of
progressive
wave on a
vibrating string.
Tanzania Vibrations Waves • Distinguish • Progressive and None of the NOS

and waves progressive and stationary waves statements
Tanzania stationary waves • Expressions for identified.
(Ministry • Deduce the progressive and
of expressions for stationary waves
Education progressive waves • Principle of
and and stationary superposition
Vocational waves
Training, • Deduce the
2017, p. 23) principle of
superposition of
waves
Uganda Waves Waves • Relate the different • Concept of wave None of the NOS
wave properties • The terminology statements
(NCDC, and use them to used in waves identified.
2013, p. 50) explain the • Relationships
different wave between
behaviors frequency,
• Explain the period,
formations of wavelength
stationary waves velocity
• Explain the • Transverse and
occurrences of longitudinal
resonance, beats, waves
Doppler Effect and
polarization.
As shown in Table 5, none of the NOS aspect was represented in content of waves
in physics syllabi. It would be better if the developers of these syllabi added
191
content elements related to history, sociology, and philosophy of science. For

example, if students were exposed to historical aspects of waves, their social
aspects, and empirical evidence leading to the evolution of the concepts of the
waves, this could help them understand the philosophical view behind it and its
application in daily life. In addition to this, the content presented under the wave
unit encourages memorization of wave concepts rather than understanding the
reason for learning these concepts. Therefore, the content provided in Table 5 is
evidence of low representation of NOS.
Table 6. Selected content of electricity to be taught to A-level students from EAC

EAC Topic Unit to be Unit objectives Content Observations
Countries areas taught related to the
representation of
NOS aspects
Burundi Electricity Nature of • Identify factors • Factors influence None of the NOS
current of The conduction of statements
(Ministère de electricity conduction of electricity in identified.
l’ Education, electricity in metals
de l’ metals • Mechanism of
Enseignemen • Interpret the electric
t superieur et mechanism of conduction
de la electric • A generator
Recherché conduction in • Quantity of
Scientifique, metals, electricity
2017, p. 28) electrolytes, • Unit of current
[Ministry of and gases intensity
Education,
Higher Learning,
• Define a
and Scientific generator
Research, 2017, p. • Determine the
28] quantity of
electricity
• Define and
measure the
current
intensity
• Define and use
a unit of
quantity of
electricity
Rwanda Kirchhoff’s • Recall sources • Review elements None of the NOS
laws and of electric of a simple statements
(REB, 2015, p. electric current, EMF electric circuit identified.
10) circuits electric, and and state the
receptors/appl application.
iance. • Definition of
• Describe electromotive
components of force.
a simple • The voltage or
electric circuit. terminal potential
• State and
Kirchhoff’s electromotive
laws force
192
• Explain the • Sources of electric

difference current and
between electric
potential receptors/applia
difference and nces.
electromotive • Internal and
force. external
• Apply resistance, the
Kirchhoff’s potential
laws to difference across
problems in a cell.
electric circuits. • Connection of
• Acquire electrical current
practical skills source and
resistors either in
series or parallel
or mix-up.
• Kirchhoff’s laws
(loop rule and
junction rule).
• Application of
Kirchhoff’s laws
to simple circuits.
Tanzania Electric • Describe the • Mechanism of None of the NOS
conduction mechanism of electric statements
(Ministry of in metals electric conduction in identified.
Education and gases conduction in gases
and metals • The resistivity of a
Vocational • Determine the conductor
Training, resistivity of a • Temperature
2017, pp. 37– conductor coefficient of
38) • Investigate the resistance
temperature • Electrical
coefficient of networks
resistance • Optical spectra
• Analyze for gases
electrical • The applications
networks of conduction of
• Investigate the electricity in gases
conduction of
electricity in
gases
• Explore optical
spectra for
gases
• Identify the
applications of
conduction of
electricity in
gases
Uganda Current • Define the • The coulomb None of the NOS
electricity coulomb • Charge statements
identified.
193
(NCDC, 2013, • Define electric • Potential

p. 85) current difference
• Explain the • Significance of
concept and potential
significance of difference
the potential • The volt
difference • emf
• Define volt, • Resistance
emf, resistance, • The ohm
and ohm • Verification of
• State and verify ohm’s law
ohm’s law • Kirchhoff’s laws
• State • Circuits problems
Kirchhoff’s using Kirchhoff’s
laws of laws
electricity
• Solve circuits
problems using
Kirchhoff’s
laws
Table 6 provides another piece of evidence of the very low representation of NOS
aspects in EAC physics syllabi. Both experimental observations and theories can
be used to integrate the historical perspective of electricity and its evolution or
develop imagination and creativity through building-model of electrical circuits
and electric concepts and phenomena. Contrary, unit objectives and content under
electricity from East African countries’ physics syllabi lack explicit connection to
the scientists’ work or the scientific enterprise. In addition to this, its content does
not provide direction on how NOS aspects are shown in educational reform
documents.
Teaching and learning electricity should always help students understand

macroscopic level observations from microscopic level theories, which is the solid
foundation of scientific reasoning skills and the current initiatives in technology.
This may help them interpret complex ideas of electrical phenomena and models
for understanding electromagnetic phenomena rather than defining, deriving,
and calculating concepts in electricity. It may be a good stimulus to learn the
evolution of electricity theories, which in the end lead to the advancement of the
study of electricity. For example, “electric effluvia” was refuted due to its inability
to give scientific explanations of electric repulsion or electric transmission
(Barahona et al., 2014).
194
Table 7. Selected content of atomic physics to be taught to A-level students from EAC
EAC Topic Unit to be Unit objectives Content Observations related
Countries areas taught to the representation
of NOS aspects
Burundi Atomic Introducti • Define the • The None of the NOS
Physics on to objective and objective of statements identified.
(Ministère atomic importance of atomic
de l’ physics atomic physics physics
Education, • Distinguish the • Physics of
de la physics of an an atom and
Formation atom from physics
Technique et physics with with atoms
Professionne atoms • Application
lle, 2018, p. • Identify s of atomic
307) applications of physics
[Ministry of atomic physics • Atomic
Education, • Get knowledge nucleus
Technical
about the Constituents
Training, and
Professional dimensions of of matter
2018, p. 307] the atomic
nucleus and the
constituents of
matter
Rwanda Atomic • Recall the duality • Structure of Explanation of

models nature of light. an atom. evidence of energy
(REB, 2015, • Explain the • Atomic levels in an atom, and
p. 36) structure of the models Rutherford and Bohr’s
atom. (Rutherford models may be used to
• Explain atomic ’s atomic teach some of the
radiation spectra. model and aspects of NOS.
• Explain Bohr’s However, it is not
evidence of atomic mentioned explicitly.
energy levels in model) So, there is no or
an atom. • Energy utmost very weak
levels and representation of NOS
spectral aspects.
lines.
Tanzania Structure • Describe the • Rutherford Rutherford and Bohr’s
of an atom Rutherford and and Bohr models may be used to
(Ministry of Bohr models of models of teach some of the
Education an atom an atom aspects of NOS.
and • Analyze atomic • Atomic However, it is not
Vocational energy levels energy mentioned explicitly.
Training, levels So, there is no or
2017, p. 46) utmost very weak
representation of NOS
aspects.
Uganda Charged • Describe the • Discharge None of NOS the
particles discharge tube tube statements identified.
phenomena as phenomena
(NCDC, pressure is • Production
2013, p. 102) reduced and
• Describe the properties
production of of cathode
cathode rays and rays
positive rays • Production
and
195
• State the properties

properties of of positive
cathode rays and cathode
positive rays rays
• Define specific • Cathode
charge rays and ion
beams in
electric and
magnetic
fields
• Specific
charge
Even though most units’ objectives in Table 7 focus on memorizing concepts about
the atom, two unit objectives under atomic physics were identified as the ones
which might help in developing NOS knowledge among students either
implicitly or explicitly taught. For example, the following unit objective: “explain
evidence of energy levels in the atom” was identified in grade 12 physics syllabus
from Rwanda and may help in the teaching of NOS aspects, such as tentativeness,
empirical, and the role of observations and inferences in developing theories
related to energy levels. Another unit objective which is “Rutherford and Bohr’s
models” was also identified in physics syllabi of Rwanda and Tanzania and this
statement may support development of creativity and imagination aspect. Table
8 shows that targeted NOS aspects in the study are almost not represented in the
five topic areas selected from all four physics syllabi considered. Contrary to other
topic areas selected in this study, atomic physics has been identified as an only
topic that contains few statements which might help teachers teach tentative and
empirical aspects implicitly, as shown in Table 8 below.
Table 8. Summary of NOS representation in the selected topic areas from physics
syllabi from EAC countries (Topic 1: Mechanics, Topic 2: Heat and thermodynamics,
Topic 3: Oscillations and waves, Topic 4: Electricity, and Topic 5: Atomic physics).
NOS aspects Burundi Rwanda Tanzania Uganda

1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5
Tentative – – – – × – – – × – – – – × – – – – – –
Empirical – – – – × – – – × – – – – – – – – – – –
Observations – – – – – – – – – – – – – – – – – – – –
and inferences
Theory – laden – – – – – – – – – – – – – – – – – – – –
Creative and – – – – – – – – – – – – – – – – – – – –
imaginations
Relationship – – – – – – – – – – – – – – – – – – – –
between theories
and laws
Social and – – – – – – – – – – – – – – – – – – – –
cultural
embeddedness
Scientific method – – – – – – – – – – – – – – – – – – – –
+: explicitly presented, –: not represented, and ×: Implicitly presented
196
3.4 NOS representations through teaching and assessment methods

Teaching and assessment methods are important in defining what relevant skills
or knowledge a student may demonstrate and how. It is the role of curriculum
developers to propose how intended concepts might be taught and assessed.
Since, the teaching of NOS aspects combines psychology, history, sociology, and
philosophy of science (McDonald & Abd-El-Khalick, 2017) as important skills a
student needs to demonstrate, it is mandatory for any teaching package such as
curricula or syllabi to describe how these NOS aspects would be taught and
assessed. Contrary to it, the present syllabi in EAC do not explicitly specify how
these important aspects should be taught and assessed. All four physics syllabi
analyzed, proposed teaching methods that would help learner-centered
pedagogy as the best way of promoting competencies and skills. But these
teaching methods lack explicit and reflective-based pedagogy to explicitly
improve students’ knowledge about the nature of science. Assessment methods
discussed in these syllabi respond to the knowledge or content assessment rather
than reasoning skills that may improve students’ understanding of the nature of
science.
4. Discussions of the findings

The analysis of the four A-level physics syllabi indicates that the topics or units to
be taught are good in respect of inclusion of NOS aspects. For example, topics
such as atomic physics, astrophysics, environmental physics, mechanics,
oscillations, and waves are good topics to teach NOS. If these topics are well
linked to the works of scientists or scientific enterprise, it will help students
acquire skills related to scientific knowledge construction.
In general, we found that there is a lack of NOS representation in both front matter
and back matter and learning outcomes. The critical issue of low representation
of NOS aspects in these syllabi comes out while looking in detail throughout unit
objectives and content. In other words, it is not clearly defined how students may
gain skills related to knowledge construction or how scientific enterprise operates.
The representation of targeted NOS aspects is missing or poorly presented in the
content of five topic areas selected from all four physics syllabi considered. Unlike
other topic areas selected in this study, atomic physics has been identified as the
only topic which contains a few statements which might help in teaching tentative
and empirical aspects implicitly. The syllabi in EAC do not explicitly specify how
these important NOS aspects should be taught and assessed. The results agree
with that of Kinyota (2020), who urged that the NOS was not given much attention
in Tanzania’s science curriculum, where the term “nature of science” was not
identified throughout the whole curriculum analyzed. The findings are also
consistent with the results with Arumit and Akerson (2022) where NOS aspects
were reported as a negligible content in Turkey middle schools’ science
curriculum.
According to Lederman and Lederman (2014), the teaching of NOS aspects would
be helpful for deeper understanding of science rather than rote memorization. In
addition to this, it is very important to have physics syllabi that would help
students develop new ideas freely, innovate new solutions, free science from rigid
197
rules, and attract attention and curiosity among students and teachers to be
engaged in the scientific enterprise (Al-Abdali & Al-Balushi, 2016). Therefore, it
would be better if the proposed content under the atomic physics unit could
support the teaching of aspects of NOS and hence promote a deep understanding
of science.
Insufficient representation of NOS aspects in the teaching resources is not only an

issue of this study, but it is widely known (Chaisri & Thathong, 2014; Yeh et al.,
2019). For example, in a study by Taber (2008), it was identified that the national
curriculum of England lacked a more explicit model to teach NOS aspects.
Ferreria and Morais (2013) reported that science construction knowledge was
mostly absent in the Portuguese science curriculum. In addition, a study of
analysis of Turkey’ science curriculum reveals that there is not only a low
representation of NOS aspects, but also a connection between the curriculum and
textbooks is inadequate to support NOS teaching in the classroom (Izci, 2017).
Similar results were reported in the study of Caramaschi et al. (2022), where the
aspects of NOS in the Italian advanced secondary school curriculum are not
explicitly represented. A recent study on representations of NOS in the science
curriculum in Norway showed that social values and scientific practices are
emphasized in the curriculum (Mork et al., 2022), which contrasts with other
related studies. NOS aspects have also been included explicitly in the science
curriculum in China, however, the majority of the NOS aspects are represented
implicitly in five textbooks analyzed and the ‘scientific method’ is inconsistently
or poorly represented in three of the five books (Zhuang et al., 2021).
Furthermore, Olson (2018) revealed that NOS aspects rarely occurred from
standards documents of nine diverse countries (Australia, Canada, Colombia,
Indonesia, Lebanon, Mexico, Thailand, South Africa, and the USA), with the
notable exception of Australia. And these standard documents do not clearly
define pedagogical support to break down NOS content into meaningful
experiences for the students. Even though little attention to NOS representation
in science curricula is known worldwide, this study shows that the situation is
very scary and worrying in EAC countries.
5. Conclusions and Recommendations

This study reveals that, in general, NOS concepts are not explicitly represented in
all statements from front matter and back matter (Table 1), learning outcomes
(Table 2), and physics content (Tables 3 to 7) based on the analysis of four physics
syllabi of EAC. The findings reveal that four reviewed physics syllabi mainly
focus on shifting from content-based syllabi to competencies-based syllabi. In
these syllabi, competencies were given much attention without an overt
connection to the work of scientists. In other words, having a science curriculum
that addresses curricular competencies, practical work skills, and good content
does not necessarily mean integrating NOS. Proposed teaching and assessment
methods in physics syllabi analyzed mainly focus on promoting competences and
memorizing content rather than understanding the process of the scientific
enterprise.
198
This paper also identified that the physics syllabus from Burundi contains few
NOS statements from its front matter compared to physics syllabi from Rwanda,
Tanzania, and Uganda. In addition, even though these few NOS statements might
promote the teaching of NOS concepts, particularly in the physics syllabus from
Burundi, these statements were not supported through learning outcomes,
content, teaching, and assessment methods. Among five major topic areas
considered, the atomic physics topic was identified as an area that may support
implicit teaching of NOS aspects in EAC countries. Furthermore, tentative and
empirical aspects were implicitly presented in the topics of atomic physics and
electricity in physics syllabi in Rwanda, Burundi, and Tanzania. Other remaining
targeted NOS aspects were identified as neither implicitly nor explicitly presented
in all physics syllabi from the EAC countries. We found that both teaching and
assessment methods do not explain how NOS concepts might be taught or
assessed.
A low representation of NOS aspects and a notable decline of NOS aspects in

many standards documents (Lederman & Lederman, 2014; Olson, 2018), is an
indicator of losing an important stimulus that is an insight for learning science. It
is now time for science educators to raise their voices to save this important aspect
of science. Therefore, the recommendation to the curriculum developers from East
African countries is to revisit current advanced physics syllabi by integrating NOS
concepts in an explicit manner. In addition, there is an urgent need for other
related learning materials, such as physics textbooks which might awaken physics
teachers to consider NOS components while integrating NOS concepts and to
guide students for easy learning of NOS aspects without much support from their
teachers.
Furthermore, due to little available literature related to NOS education in general

in the region, we recommend that scientists and researchers from the East African
countries study NOS representations in the different learning materials used in
the region for good interventions which can overtly link the described contents
from advanced level physics syllabi to the work of scientists or the development
of knowledge construction.
Acknowledgments
Financial support from the African Centre of Excellence for Innovative Teaching
and Learning Mathematics and Science (ACEITLMS) is thankfully acknowledged.
6. References
Al-Abdali, N. S., & Al-Balushi, S. M. (2016). Teaching for Creativity by Science Teachers
in Grades 5–10. International Journal of Science and Mathematics Education, 14, 251–
268. https://doi.org/10.1007/s10763-014-9612-3
Erumit, A. B., & Akerson, V. L. (2022). Using children’s literature in the middle school
science class to teach nature of science. Science & Education, 31(3), 713-737.
https://doi.org/10.1007/s11191-021-00274-3
Barahona, A., Chamizo, J. A., Garritz, A., & Slisko, J. (2014). The history and philosophy
of science and their relationship to the teaching of sciences in Mexico. In
International Handbook of Research in History, Philosophy and Science Teaching.
https://doi.org/10.1007/978-94-007-7654-8_69
199
Bell, R.L., Matkins, J.J. and Gansneder, M.B. (2011). Impacts of Contextual and Explicit
Instruction on Preservice Elementary Teachers’ Understandings of the Nature of
Science. Journal of Research in Science Teaching, 48(4), 414-436.
Boe, M., Hendriksen, E., Lyons, T., & Schreiner, C. (2011). Participation in science and
technology: Young people’s achievement-related choices in late-modern societies.
Studies in Science Education, 47(1), 37–71.
Bower, G.A. (2009). Document Analysis as a qualitative Research Method. Qualitative
Research Journal, 9(2), 27–40. https://doi.org/10.3316/QRJ0902027
Cairns, D. (2019). Investigating the relationship between instructional practices and
science achievement in an inquiry-based learning environment. International
Journal of Science Education, 41(15), 2113–2135.
https://doi.org/10.1080/09500693.2019.1660927
Caramaschi, M., Cullinane, A., Levrini, O., & Erduran, S. (2022). Mapping the nature of
science in the Italian physics curriculum: from missing links to opportunities for
reform. International Journal of Science Education, 44(1), 115-135.
https://doi.org/10.1080/09500693.2021.2017061
Çetin, A., & Kahyaoğlu, M. (2022). Investigation of Teacher Candidates' Nature of Science
Beliefs In Terms of Gender, Program, and Class Level. Dinamika Ilmu, 22(1) 53-72.
https://doi.org/10.21093/di.v22i1.4321
Chaisri, A., & Thathong, K. (2014). The Nature of Science represented in Thai Biology
Textbooks under the topic of evolution. Social and Behavioural Sciences, 116,
621-626.
Cheung, K. K. C. (2020). Exploring the inclusion of nature of science in biology curriculum
and high-stakes assessments in Hong Kong. Science & Education, 29(3), 491-512.
https://doi.org/10.1007/s11191-020-00113-x
Childs, P.E. (2015). Curriculum development in science: Past, Present, and Future.
LUMAT, 3(3), 381-400.
Chiappetta, E.L., Ganesh, T.G., Lee, Y.H., & Phillips, M.C. (2006). Examination of science
textbook analysis research conducted on textbooks published over the past 100
years in the United States. Paper presented at the annual meeting of the National
Association for Research in Science Teaching, San Francisco, CA.
Das, P.M., Faikhamta, C., & Punsuvon, V. (2019). Bhutanese students’ views of nature of
science: A case study of culturally rich country. Research in Science Education, 49(2),
391–412. https://doi. org/10.1007/s11165-017-9611-9
Erumit, B. A., & Akerson, V. L. (2022). Using Children’s Literature in the Middle School
Science Class to Teach Nature of Science: Preservice Teachers’ Development of
Sources. Science & Education, 31(3), 713–737. https://doi.org/10.1007/s11191-021-
00274-3
Ferreria, S., & Morais, A. M. (2013). The nature of science in science curricula: Methods
and concepts of analysis, International Journal of science Education, 35(16), 2670–
2691. https://www.tandfonline.com/doi/abs/10.1080/09500693.2011.621982
Hipkins, R. (2012). Building a Science Curriculum with an effective nature of science Component.
New Zealand Council for educational Research, Ministry of Education.
Izci, K. (2017). Nature of science as portrayed in the middle school science and technology
Curriculum: The case of Turkey. Journal of Education in Science, Environmental and
Health (JESEH), 3(1), 14–28.
Jenkins, E. W. (2013). The “nature of science” in the school curriculum: the great survivor.
Journal of Curriculum Studies, 45(2), 132–151.
Kinyota, M. (2020). The status of and challenges facing secondary science teaching in
Tanzania: a focus on Inquiry-based science teaching and the nature of science.
International Journal of Science Education.
https://doi.org/10.1080/09500693.2020.1813348
200
Kinyota, M., & Rwimo, B. S. (2022). Developing Student Teacher’s Conceptions of the
Nature of Science: Assessment of a pre-service Science Teacher Programme in
Tanzania. African Journal of Research in Mathematics, Science and Technology
Education, 25(3), 269–279. https://doi.org/10.1080/18117295.2022.2041788
Lederman, N.G., Abd-El-Khalick, F., Bell, R. L., & Schwartz, R. (2002). Views of nature of
science questionnaire: toward valid and meaningful assessment of learners’
conceptions of nature of science. Journal of Research in Science Teaching, 39(6), 497-
521.
Lederman, N.G. (2007). Nature of science: Past, present, and future. In S. Abell & N. Lederman
(Eds.), Handbook of research on science education, (pp. 831-879). Mahwah, NJ:
Lawrence Erlbaum.
Lederman, N. G., & Lederman, J. S. (2014). Is nature of science going, going, going, gone?
Journal of Science Teacher Education, 25, 235-238.
Lederman, N.G., Lederman, J.S., & Antink, A. (2013). Nature of science and scientific
inquiry as Contexts for the learning of science and achievement of scientific
literacy. International Journal of Education in Mathematics, Science and Technology,
1(3), 138-147.
Liang, L. L., Chen, S., Chen, X., Kaya, O. N., Adams, A. D., Macklin, M., & Ebenezer, J.
(2008). Assessing preservice elementary teachers’ views on the nature of scientific
knowledge: A dual response instrument. Asia-Pacific Forum on Science Learning and
Teaching, 9, 1-20.
McDonald, C.V., & Abd-El-Khalick, F. (2017). Representations of nature of science in school
science textbooks: A global perspective. New York: Routledge.
Martín‐Díaz, M.J. (2006) Educational Background, Teaching Experience and Teachers’
Views on the Inclusion of Nature of Science in the Science Curriculum,
https://doi.org/10.1080/09500690500439504
Mbonyiryivuze, A., Kanamugire, C., Yadav, L. L., & Ntivuguruzwa, C. (2018). Reforms in
Science Curricula in last six decades: Special reference to Physics. African Journal
of Educational Studies in Mathematics and Sciences, 14, 153-165.
Statistics in Tanzania (BEST). Government Press.
Ministère de l’ Education, de l’ enseignement supérieur et de la Recherche Scientifique.
(2016). Guide d’ Enseignant: Première Année Post-Fondamentale/Section
Sciences/Option Maths–Physique/Physique, République du Burundi. [Ministry of
Education, Higher Learning and Scientific Research. (2016). Teacher’s Guide: First
Year of post–Fundamental/Science Section/Physics, Republic of Burundi].
Ministère de l’ Education, de l’ enseignement supérieur et de la Recherche Scientifique.
(2017). Guide d’ Enseignant: Deuxième Année Post-Fondamentale/Section
Sciences/Option Maths-Physique/Physique, République du Burundi. [Ministry of
Education, Higher Learning and Scientific Research. (2017). Teacher’s Guide: Second
Year of post–Fundamental/Science Section/Maths-Physics option/Physics, Republic of
Burundi].
Ministère de l’ Education, de la formation Technique et Professionnelle. (2018). Guide
d’Enseignant: Troisième Année Post-Fondamentale/Section Sciences/Option Maths-
Physique/Physique, République du Burundi. [Ministry of Education, Technical
Training and Professional. (2018). Teacher’s Guide: Third Year of post–
Fundamental/Science Section/Maths-Physics option/Physics, Republic of Burundi].
Mork, S. M., Haug, B. S., Sørborg, Ø., Parameswaran Ruben, S., & Erduran, S. (2022).
Humanising the nature of science: an analysis of the science curriculum in
Norway. International Journal of Science Education, 1-18.
https://doi.org/10.1080/09500693.2022.2088876
201
National Research Council. (2012). A Framework for K-12 Science Education: Practices, cross-
cutting concepts, and core ideas. Washington, DC: The National Academies Press.
National Curriculum Development Centre (NCDC). (2013). Uganda Advanced Certificate of
Education: Teaching Syllabi for Physics and Mathematics. Volume 2. Republic
of Uganda.
NGSS Lead States. (2013). Next generation science standards: For states, by states.
Washington, DC: The National Academies Press.
Ogunniyi, M. (2006). Effects of a discursive course on two science teachers' perceptions
of the nature of science. African Journal of Research in Mathematics, Science and
Technology Education, 10(1), 93-102.
Olson, J.K. (2018). The Inclusion of the Nature of Science in Nine Recent International
Science Education Standards Documents. Science and Education, 27, 637-660.
Ramnarain, U. D., & Chanetsa, T. (2016). An analysis of South African Grade 9 Natural
sciences textbooks for their representation of nature of science. International
Journal of Science Education, 38(6), 922-933.
https://doi.org/10.1080/09500693.2016.1167985
Rovelli, C. (2015). Aristotle’s Physics: A Physicist’s Look. Journal of the American
Philosophical Association, 1(1), 23–40. https://doi.org/10.1017/apa.2014.11
Rwanda Education Board. (2015). Advanced level Physics syllabus: Advanced Level Physics
Competency-Based Curriculum. Republic of Rwanda.
Schrauth, B. A. (2009). The portrayal of the nature of science in early childhood
instructional materials. Graduate Theses and Dissertations.
http://lib.dr.iastate.edu/etd/10989
Taber, K. S. (2008). Towards a curricular model of the nature of science. Science and
education, 17, 179- 218. https://doi.org/10.1007/s11191-006-9056-4
Tugirinshuti, G. J., Mugabo, L. R., & Banuza. A (2021). Integrating Video-Based
Multimedia in Teaching Physics in Context of Covid-19 in Rwandan Secondary
Schools. International Journal of Learning, Teaching and Educational Research, 20(12),
49–63. https://doi.org/10.26803/ijlter.20.12.4
United Nations Educational, Scientific and Cultural Organization (2009). Current
Challenges in Basic Science Education, UNESCO, Paris.
Upahi, J. E., Ramnarain, U., & Ishola, I. S. (2020). The nature of science as represented in
chemistry textbooks used in Nigeria. Research in Science Education, 50(4),
1321-1339.
Vesterinen, V-M., Akesla, M., & Lavonen, J. (2013). Quantitative analysis of
representations of nature of science in Nordic upper secondary school textbooks
using framework of analysis based on philosophy of chemistry. Science &
Education, 22, 1839–1855. https://doi.org/10.1007/s11191-011-9400-1
Yeh, Y. F., Erduran, S., & Hsu, Y. S. (2019). Investigating coherence about nature of science
in science curriculum documents in Taiwan. Science & Education, 28(3), 291-310.
https://doi.org/10.1007/s11191-019-00053-1
Zhuang, H., Xiao, Y., Liu, Q., Yu, B., Xiong, J., & Bao, L. (2021) Comparison of nature of
science representations in five Chinese high school physics textbooks.
https://doi.org/10.1080/09500693.2021.1933647
202

Vol. 21, No. 8, pp. 202-218, August 2022
Using Graphic Oral History Texts to

Operationalize the TEIL Paradigm and
Multimodality in the Malaysian English
Language Classroom
Said Ahmed Mustafa Ibrahim , Azlina Abdul Aziz , Nur Ehsan Mohd
Said , Hanita Hanim Ismail
Universiti Kebangsaan Malaysia, Bangi, Malaysia
Abstract. The adoption of commercial language-teaching materials

produced in the United Kingdom, or the United States, is a common
practice in English language teaching (ELT) worldwide. This is due to the
wide perception of British and American English as the standard,
favorable models of the English language. This practice, however, does
not support the increasing and urgent shift to the teaching English as an
international language (TEIL) paradigm, and it further perpetuates the
hegemony of these Western countries over the market of ELT materials.
In this paper, we seek to not only problematize the adoption of global
commercial materials but also propose a conceptual model for composing
effective local ELT materials for the Malaysian English language
classroom. In doing so, we refer to the relevant literature and previous
research. The proposed conceptual model embraces the TEIL paradigm
as well as multimodality and executes them by utilizing oral history and
the graphic novel as two powerful pedagogical tools. By combining these
two pedagogies, the model accentuates the acknowledged pedagogical
value of both oral history and the graphic novel and results in local-
context and culture-based texts that are also consistent with the current
nature of texts being visual and multimodal. Furthermore, the paper
showcases some samples of graphic oral history texts composed by
Malaysian English language teachers and student teachers in two
projects.
Keywords: ELT materials; graphic novels; multimodality; oral history;

teaching English as an international language (TEIL)
1. Introduction
Language-teaching materials are the nucleus of English language teaching (ELT)
(Damayanti et al., 2018), contributing immensely to the teaching and learning of
the English language (Bouckaert, 2019; Bouckaert et al., 2018; Tomlinson, 2016).
©Authors
203
ELT materials, especially textbooks, constitute the primary source of the English
language input and practice that the students receive within the classroom. Thus,
it can be said that the quality of English language education is shaped, to a great
extent, by the quality of the teaching materials utilized in the classroom.
Accordingly, the Malaysian Ministry of Education endeavored to upgrade the
quality of ELT in its public schools where English is taught as a second language.
They decided to cease the use of local Malaysian textbooks and to import global
English coursebooks designed and produced by renowned international
publishers in the United Kingdom for both primary and secondary English
language classrooms (Abdul Aziz & Makhtar, 2021; Abdul Aziz et al., 2019; Abdul
Rahim & Jalalian Daghigh, 2019). The assumption underlying the government’s
decision is that imported ELT materials from native English-speaking countries,
especially the United Kingdom or the United States, will expose Malaysian
English language learners to content written in high-quality, standard English
superior to that of the local materials. Nonetheless, we agree with many of the
critics of the government’s move and contend that the attempt involves many
issues and is, in fact, problematic.
Similar to all commercial ELT materials, which are usually perceived as irrelevant
and unsatisfactory (Tomlinson & Masuhara, 2013), the imported coursebooks
disregard the Malaysian context and culture (Abdul Rahim & Jalalian Daghigh,
2019). That is not surprising, as commercial global coursebooks produced by
international publishers are often considered as being designed for everyone yet
satisfying no one (Tomlinson & Masuhara, 2004). They are designed to be useable
by different groups of English language learners from all over the globe in diverse
contexts and cultures. Thus, such coursebooks comprise content based on the
cultural elements and aspects relevant to the English-speaking countries without
taking into consideration the local ELT settings where these coursebooks are
going to be used. Furthermore, the imported materials are imposed upon the
teachers and the students. Teachers continue to be passive consumers of materials,
and students find difficulty in learning the language due to the foreign content
and shy away from practicing the language or engaging in the English lessons
(Can et al., 2020). What is worse is that the move reflects how we still consider
ELT materials produced by native speakers as being of superior quality in
comparison to locally developed materials. It also reflects our perception of native
speakers as the sole and supreme authority of the English language. Those
perceptions denote moving backward to revering the British culture and language
and imply that Malaysians have not yet moved beyond the colonial mindset
(Abdul Aziz et al., 2019).
This move by the Malaysian Government is, however, situated in a universal

paradigm in ELT which favors Western perspectives and knowledge over local
systems of knowledge (the perception of the West as the best!). They thus
maintain the positioning of the West, especially the United Kingdom, as the global
center of the English language dominating the field of ELT, while the rest of the
world serve as the recipient and are center dependent. The center perpetuates its
domination over the English language by generating all the approaches,
strategies, principles, and techniques regarding English language learning and
204
teaching. In addition, the center formulates the teaching materials based on the
ideas and principles they have established and continues to monopolize the
process of materials development (Pennycook, 2017). This hegemonic approach
involves a form of linguistic imperialism. The English language is utilized as a
means to reproduce and perpetuate imperial power relations between the United
Kingdom and its former colonies. This is achieved through a wide variety of ELT
practices, such as the assessment criteria, curricular plans, teaching methods, and
teaching materials, meant to perpetuate the hegemony of the center over the non-
native speakers (Kumaravadivelu, 2016). However, it is mainly through the
center-produced materials and teaching methods that the marginality of the vast
majority is sustained (Kumaravadivelu, 2016). Therefore, many of the materials
produced in these Western countries are considered no more than tools of
imperialism or neo-imperialism, with content romanticizing and promoting
Western cultures and values while ignoring and repressing local cultures
(Kanoksilapatham, 2018; Khodadady & Shayesteh, 2016; Lekawael et al., 2018;
Pennycook, 2017).
What exacerbates the problem is the conformation of the non-natives and

ex-colonies to the hegemonic paradigm and their submission to the subordinate
dominated role. They keep devotedly and comfortably depending on the one-way
flow of information and the ready-made prescriptions prepared and mandated by
the “authority” of the English language. Their minds are, therefore, still colonized
and cannot think that they can have their own new ideas and produce materials
relevant to their own needs and contexts. In a country with a colonial history, such
as Malaysia, there should be sincere efforts to effectively disrupt the hegemonic
paradigm imposed by the West on ELT practices and knowledge production.
Postcolonial Malaysia should adopt a decolonized mindset (Mignolo, 2007) that
is free of the perceptions and attitudes of colonialism. They should adopt a
decolonial mindset that is aware of the Eurocentric tendencies and is critical of
mainstream commercial materials that are meant to propagate the domination of
the native-speaker model. We need to recognize the significance of English as the
language of globalization and strive to teach it effectively without overlooking its
historical realities as an instrument of colonial imperialism.
Changing the mindset is not enough, though. Malaysian educators need to

become proactive towards decolonizing ELT practices and materials by
composing local ELT materials relevant to the goals and objectives of Malaysians
and reflecting the richness of the Malaysian multicultural and multilingual
landscape. Those materials should also be innovative and significantly contribute
to effective English as an international language teaching. In response, we propose
this conceptual model to serve as a guide for local professionals (materials
designers and English language teachers) to develop compelling, home-made ELT
texts.
2. Conceptual Model
Producing effective ELT materials that contribute to the effective learning of
English as an international language requires paying attention to not only the
content of the materials but also the format of these materials. This conceptual
205
model, shown in Figure 1, considers this aspect by espousing the TEIL paradigm
(informing the content) and multimodality (informing the format).
TEIL Paradigm Multimodality

Local context/culture Multimodal texts
Oral Graphic
History Novel
Local Content Multimodal Format

Personal narratives, Words & visuals
memories, or accounts
Graphic Oral History

Local content in multimodal-format texts
Figure 1: Conceptual model: Innovative fusion resulting in local multimodal English

language-teaching materials
The two-sided model therefore addresses the lack of TEIL-oriented materials

(Lwin & Marlina, 2018; Sifakis & Bayyurt, 2018) as well as the still limited
incorporation of multimodal texts in many ELT contexts (Ganapathy, 2011; Yusof
et al., 2017). The TEIL paradigm reflects the need for local, culturally responsive
materials that capitalize on students’ culture, identity, and communities. To
operationalize the TEIL paradigm, oral history will be utilized as a pedagogical
tool. By conducting interviews with local figures or community members who are
relevant to a certain topic or phenomenon, we will create English content that
capitalizes on students’ local culture and is more rooted in our everyday life and
inner values. On the other side, the pedagogy of multimodality entails the use of
multimodal texts. The graphic novel as a multimodal text comprising words,
pictures, and other conventions, such as panels, gutters, etc., will be used in the
model to enact multimodality. Presenting the content redeemed by oral history in
a graphic novel format will thus result in local visual narratives that can serve as
effective, innovative instructional ELT texts responsive to TEIL and the visual
multimodal world in which we live today. The following sections address each
component of the conceptual model.
206
2.1 The Teaching English as an International Language Paradigm

The supremacy of the native speaker as an ideal linguistic example or model is
not adequate anymore; the vast majority of English language users and learners
all over the world are bilingual or multilingual speakers. It is therefore
unreasonable to subject the majority, who utilize English mainly as a language of
expanded interaction and communication in conjunction with one or even more
other local languages they speak, to the minority’s norms and particular linguistic
features of the language. Kachru (1985) contended that the English native
speakers have lost their entitlement or authority over the language as they are the
minority. He further argued that it is incumbent upon the world community to
recognize this sociolinguistic truth and its implication. The TEIL paradigm
acknowledges that truth and perceives English as the world’s global lingua franca
regardless of any political or cultural associations, and more importantly without
much attention to which countries speak it as a first language. Moreover, it
recognizes many different local versions of English, which are collectively known
as World Englishes or WE (Kachru, 1985). In that context, English is no more a
single, monomodal entity; it is an international polycentric tool of communication
with a wide variety of lexis, grammatical structures, and other discourse
conventions (Marlina, 2018). In such a way, the TEIL paradigm represents an ELT
perspective that steers away from the idea of native-speakerism embedded in
colonialism. It also shifts the focus of ELT pedagogy from traditional Anglo-
centric standards and norms of English as a native language towards localized
usages and varieties (Matsuda, 2012; McKay, 2012).
Postcolonial Malaysia needs to cease seeing the English language from the
colonial perspective and teach it from a professional perspective by adopting the
TEIL paradigm. The paradigm ultimately aims at helping students to
communicate in English in a globalized world characterized by being culturally
and linguistically diverse. Conveniently, the paradigm recognizes Malaysian
English as a brand or a variety of World Englishes that has emerged in the
postcolonial era and makes a remarkable contribution to the English language as
a cultural resource with which people express their cultural values, perspectives,
and worldviews (Marlina, 2018). McKay (2003) argued that the TEIL paradigm
recognizes the significance of containing topics that represent the local culture and
supports the use of a methodology relevant to the local teaching and learning
contexts. By embracing the TEIL paradigm, Malaysians will confidently work
towards localizing ELT practices and materials; they will recognize the
contribution of its multilingual and cultural diversity to the English language and
produce local content comprising their local version of English. The local content
will value the traditional indigenous knowledge over colonial global influences,
and it will favor the day-to-day realities of Malaysians in their actual local settings.
In doing so, we will capitalize on local knowledge and cultures and use them as
pedagogical content to make the English language a vehicle for locals to
communicate their worldviews, cultural values, and socio-cultural realities.
The focus on the local context and culture in ELT entailed by the TEIL paradigm
(Lwin & Marlina, 2018) resonates with the massive body of research and literature
that advocates local ELT materials in comparison to materials that depend on the
207
target language’s culture. Global commercial materials might cause unnecessary

difficulty to English language learners due to the social and cultural dissimilarities
and this, in turn, can result in the learners’ inability to understand or engage with
teaching materials (Grabe, 2014). For language teaching to be successful, ELT
materials need to be associated with the learner’s home culture and provide them
with authentic activities that enable them to communicate about their own real
lives (Toledo-Sandoval, 2020). Several studies (Can et al., 2020; Jabeen & Shah,
2011; Kanoksilapatham, 2018; Kirkpatrick, 2007) have shown that local-culture-
based ELT materials have a more positive influence on the English language
learners than foreign-culture-based materials and texts. In addition, some of these
studies demonstrated that the learners’ cultural identity could be marginalized if
they were to be exposed to ELT materials that solely focus on the culture of the
target language. Therefore, by adopting the TEIL paradigm, the conceptual model
seeks to develop local content for ELT materials; that local content will be obtained
using oral history.
2.2 Oral History: A Pedagogical Tool Contributing to the TEIL Paradigm

Oral history is the collection of spoken stories, memories, and personal
commentaries through recorded interviews of people who lived through a
historical event or social phenomenon (Ritchie, 2003). The recordings of the
interviews are transcribed, edited, and then placed in a library or archives and
may be used for research, publications, documentaries, museum exhibitions, or
any other form of public presentation. By recording and documenting the spoken
stories of people who lived through past events or significant phenomena, oral
history serves as a research method that plays a significant role in writing the
histories of societies and communities (Ritchie, 2003). It enables us to learn about
those past events and thus have a deeper understanding of them. Oral history has
come to prominence as a tool in writing history due to the appearance of social
historians such as Paul Thompson, who called for “history from below”. This type
of history revolves around and gives priority to common people’s lives as part of
their writing. This is opposed to conventional history writing, which focused
mostly on the elite group of people with high social status, such as politicians and
diplomats, while neglecting ordinary people, such as farmers, workers, and
women (Boon Kheng, 2007). Thompson (2000), one of the pioneers of oral history
as a research methodology in the social sciences, considered oral history as the
history of ordinary people and marginalized groups. Utilizing oral history can be
beneficial, especially to those individuals who are historically and traditionally
marginalized within the dominant culture and society because of race, class,
gender, sexual orientation, or religious beliefs. Therefore, by including the
unheard and unseen within the great majority, oral history helps to constitute
history that revolves around society as a whole. In doing so, oral history attempts
to steer history readers away from what imperialism has left behind and
represented as symbols of greatness (Yeoh, 2003).
Besides being a research method contributing to the construction of histories, oral

history has been implemented in the classroom as an effective pedagogical tool
for enhancing the study of history-related topics and other important learning
skills (Dutt-Doner et al., 2016). It has been used extensively in classroom contexts
208
as a significant teaching strategy since the late 1960s to enhance the teaching and
learning of social studies and history at all school levels (Montero & Rossi, 2012).
In these contexts, students were exposed to oral history related to their immediate
families and local communities. Vodniza (2016) used traditional storytelling and
oral history as an effective pedagogy in his classes to support students in
discovering identities and traditional values. He found that oral history supported
his teaching to a great extent, especially in terms of communicating indigenous
traditional knowledge and cultural practices from generation to generation by
helping students to learn stories from their parents, grandparents, and great-
grandparents. By connecting students to their traditional practices and fostering
a deeper understanding of their community and culture, students learn to
recognize and celebrate their culture and heritage. Therefore, oral history research
can be a revolutionary pedagogical instrument (Ayers & Ayers, 2013) that is
culturally responsive (Gay, 2018). It validates the students’ local cultural heritage,
context, and prior experiences, and this results in the learning process being more
adequate and successful.
In the ELT context, Montero and Rossi (2012) argued that utilizing oral history
leads to a culturally responsive pedagogy that legitimizes and endorses the
learners’ lived experiences, stories, and histories to serve as content within the
English language classroom. Jones (1998) conducted a study in an undergraduate
course at the National Tsinghua University of Taiwan to investigate the
significance of English as a foreign language (EFL) students’ drafting of oral
histories in their academic writing. The study reported many pedagogical benefits
of writing family oral histories in the EFL writing classroom and concluded that
oral history is an excellent way to spark students’ interest and to teach them
academic writing. Furthermore, a recent study was conducted in Saudi Arabia
(Strachan & Winkel, 2020) to investigate the development of an oral project for
undergraduate EFL students in a Saudi private university. The project aimed at
reconnecting the undergraduates to their familial heritage while providing them
with a distinctive educational experience. The project not only enabled the
university EFL students to acquire a deeper understanding of their local heritage,
but it also led to the improvement of their English language skills.
Therefore, given the potential of oral history as a powerful, culturally responsive

pedagogy, we believe that it is the practical, ideal tool to redeem the local-context-
and culture-based content necessitated by the TEIL paradigm. Oral history will
provide content based on local issues, familial lived experiences, and local
communities of the learners, and that will motivate the learners and engage them
in the learning process (Christodoulou, 2016). Use of oral history corresponds
with research by Canagarajah (2016) and Kachru (1986), who were in favor of
localizing the teaching content and utilizing ELT materials that are culturally
responsive to the local learners’ needs by relying on their experiences in their local
environments and settings. In the context of the proposed conceptual model,
Malaysian materials designers or English language teachers will take on the role
of oral historians and follow some of the procedures and best practices for
conducting oral history research approved by the Oral History Association (OHA,
2022). They will study a local topic by conducting recorded (audio/video)
209
interviews, based on a well-prepared interview guide comprising relevant

questions, with members of the local community whose experiences are
specifically relevant to the topic. They will then transcribe the recording and,
finally, convert the interview transcript into a narrative or story that will serve as
the local content of the ELT materials. The question now is: How can we best
present that local content?
2.3 Multimodality
To answer the previous question, the conceptual model considers the types of
texts in which students are mostly engaged nowadays. Students are immersed in
a visually oriented world where visuals are a crucial component in almost all sorts
of communications. What we perceive as “text” is broader now and includes
various blends of modes (print, images, etc.). This change in the nature of texts
has, in turn, impacted how we now conceptualize literacy, with the foremost
literacy of the 21st century being visual (Burmark, 2008). Therefore, it is no longer
sufficient to solely rely on and use traditional printed texts; learners need to learn
to comprehend and produce both words and images. To cater to this shift in how
we now perceive texts and literacy and to help students develop the new set of
competencies required to succeed in the classroom as well as in their everyday
lives, the New London Group (1996) convened and introduced the document A
pedagogy of multiliteracies: Designing social futures. The document reflected a change
in the pedagogy of literacy. It centered on the interplay between the various
modes of meaning and brought about the transition from traditional print-based
texts towards acknowledging a more broadened and pluralistic concept of literacy
that incorporates the many modes used in today’s communications.
The New London Group’s pedagogical approach to multiliteracies signaled the

need for students to acquire new literacy competencies. That need has given rise
to multimodality and multimodal approaches in the field of education and
teaching curricula. According to Siegel (2012), multimodality is the concurrent use
of more than one mode in a single text or event. Each of these modes, such as
words, images, sound, movement, etc., has its affordances, or ways of creating
meaning (Kress, 2009). While conventional literacy practices focused on the
printed word, multimodal literacy recognizes the current dominance of
technology and its reliance on image, sound, and animation along with the text.
To underscore the significance of multimodality in English language learning and
teaching, Royce (2007) presented the term multimodal communicative competence, a
concept concerned with how English learners can become proficient in terms of
comprehending and composing meanings multimodally. Therefore, to address
multimodality within the English language classroom so that we teach useful
literacy practices for today’s age, we need to use multimodal texts such as
websites, picture books, graphic texts, and graphic novels.
2.4 The Graphic Novel: An Innovative Multimodal Pedagogical Tool

To pedagogically address the current multimodal nature of texts, the proposed
model utilizes the graphic novel as a multimodal pedagogical tool (Jamil & Abdul
Aziz, 2021). A graphic novel is a fiction or nonfiction narrative presented in a
comic book format, and it is also known as sequential art (Eisner, 2008). It
210
integrates words with pictures, and thus represents a multimodal text that caters
to both print literacy and visual literacy. According to Gallo and Weiner (2004), a
graphic novel presents the same reading experience of the traditional literary text
in addition to images and words working cumulatively, making a graphic novel
a text that one not only reads but also sees. It is worth noting that graphic novels
are not a genre; they are a format for recounting a story or communicating
information (Woolston, 2014). Moreover, they do not have to be novels; they
include both fictional and nonfictional work, as well as full-novel-length and short
narratives or stories. The way graphic novels integrate print and visuals provides
an engaging reading experience in reading (Begoray & Fu, 2015). They are popular
with teenagers (Griffith, 2010) and have gained more and more acceptance and
respect as quality literature over the years (Pantaleo, 2015). An example of a
popular graphic novel is American born Chinese (Yang & Pien, 2006). In this award-
winning graphic novel for excellence in young adult literature, the author
managed to address some serious issues, such as racism against Chinese
immigrants, culture, racial identity, and stereotyping.
The unique mix of the two abundant modes – the linguistic mode and the visual
mode – in graphic novels have made educators realize their potential as a
pedagogical tool (Seglem & Witte, 2009). A significant body of research has
highlighted the significance and value of graphic novels as multimodal texts in
the English language classroom. The novels have been found to provide
motivation and engagement for struggling and hesitant readers (Brozo et al., 2013;
Öz & Efecioğlu, 2015); improve learners’ language learning strategies, critical
thinking, and comprehension (Basol & Sarigul, 2013; Öz & Efecioğlu, 2015; Sabbah
et al., 2013); and aid teachers in teaching new lexical items (Basal et al., 2016; Öz
& Efecioğlu, 2015). The appealing illustrations of graphic novels offer contextual
clues to the written text; thus, they provide support and a sense of confidence to
struggling or remedial readers. In that regard, Krashen (2004) considered graphic
text materials an optimal instrument contributing to enhanced student
engagement and self-confidence in reading. He further indicated that the use of
graphic materials could boost language learners’ interest in the language and thus
increase their foreign- or second-language acquisition and learning.
Furthermore, graphic novels cater to a wide variety of learners and their learning
styles (Öz & Efecioğlu, 2015); they contain words and pictures, so they appeal to
visual as well as verbal learners. Besides, these visual texts can be acted out or
played by kinesthetic or spatial learners, who can also perform some of the actions
in the novel. Teachers could also involve their students in activities by asking
them to prepare and conduct interviews or role play some of the characters’ facial
expressions in the graphic novel. To engage tactile learners, the teacher could ask
them to sketch a drawing based on the theme or some of the actions taking place
in the graphic novel. Teachers could also assign learners to design or develop a
small representation or model of the setting in the graphic novel. Evidently,
numerous activities can be fully used in the English language classroom based on
a graphic novel, and they can all help to address the different learning styles and
the individual needs of the learners. Therefore, considering the many advantages
of the graphic novel as a pedagogical tool, materials designers or English language
211
teachers will transform the local oral history narratives into a graphic novel
format; they will combine multimodal resources (words, images, and graphic
novel conventions) to construct meaning. The outcome of the model is graphic
oral history narratives that could be utilized in Malaysian ELT contexts as
instructional texts.
2.5 Graphic Oral History Texts: An Innovative Fusion

On its own, oral history is an innovative language pedagogy (Burgo, 2016). Using
oral history will generate authentic local narratives founded on learners’
community, capturing some of the salient cultural elements from the many
communities that form Malaysia. It provides a platform to assemble anecdotes,
life experiences, and traditional knowledge from family or community members,
which cannot be found in global commercial coursebooks (Abdul Aziz et al.,
2019). The use of local cultural content in ELT materials provides learners with
content familiar to their reading schemata and can enhance English proficiency
and cultural awareness (Yahya et al., 2017). Moreover, as Harmer (2012)
contended, the learners become involved in content that is based on themselves
and their own real lives and in such a way they learn the English language better.
Furthermore, since 21st century students are exposed to multimodal texts, the
local content will be converted into a graphic novel format. The graphic novel is
an innovative multimodal pedagogical tool (Kwon, 2020) that is popular, visual,
motivating, engaging, and overall increases second language acquisition. By
developing graphic oral history narratives and using them as teaching texts, we
are meshing the two innovations and their educational strengths. The texts will
not only appreciate students’ local culture and experiences. They will also be
consistent with the recent, updated content standards in English language arts
and literacy, expecting all students to use a wide variety of modes beyond
language. Pedagogically, graphic oral history texts will serve as effective ELT
materials contributing to the effective teaching of English as an international
language. In conjunction with effective teaching practices, those texts will be
delivering quality content in a multitude of modalities and will have strong
pedagogical significance.
3. Projects and Samples

3.1 Personal Narratives of Malaysian Youths’ Schooling Experiences
The first project was conducted in the context of teacher education with Year 3
teaching English as a second language (TESL) students in the Faculty of
Education, Universiti Kebangsaan Malaysia (UKM). The student teachers
composed their narratives on the topic “Memorable high school experiences”.
Then, they reconstructed the personal narratives into a graphic novel form. The
outcome of the project was a published book containing 16 personal graphic
stories that represent real, significant high schooling experiences (Abdul Aziz &
Chang, 2021). The stories dealt with issues such as daydreaming in an
uninteresting class, receiving undesired attention from the opposite gender,
failing to live up to your parents’ expectations, and being subject to bullying by
friends at school. By using the real lived experiences of Malaysian student
teachers as ELT materials, English learners will be reading authentic familiar
212
content to which they can relate. Besides, presenting this content in a graphic story
format results in engaging multimodal texts. Teacher trainees’ written reflections
on their perceived learning outcomes from the project showed that they valued
the project as a practical, engaging activity that enhanced their writing skills,
creativity, and self-confidence in their ability to develop local multimodal ELT
materials. Figure 2 shows a sample from the trainees’ graphic oral history
narrative “The bitter taste of popularity”, in which the narrator recounted her
inspiring story of being subject to bullying at school and how she managed to
overcome that traumatic experience.
Figure 2. A sample from the graphic oral history narrative “The bitter taste of
popularity”
3.2 The Covid-19 Pandemic From the Local Perspective: Malaysian Front-
Liners’ Stories
The second project was conducted in the context of English language teacher
professional development with in-service English language teachers doing their
Master of Education in TESL, UKM. After receiving some input sessions on
designing and evaluating ELT materials, oral history, and the conventions of
graphic novels, 105 in-service English language teachers were assigned to work
in groups to develop and evaluate graphic oral history texts and teaching
activities for their English classrooms. The teachers conducted oral history
interviews with Malaysian front-liners (doctors, nurses, food delivery drivers,
213
cleaners, etc.) who had been boldly battling against Covid-19. The teachers then
presented the front-liners’ Covid-19 pandemic accounts graphically. Moreover,
they cooperated with a reputable local animation studio and turned the graphic
oral histories into 2D animations. The project could serve as an attempt to
document this current global phenomenon from the local Malaysian perspective,
and the produced graphic oral histories could be made available to the public to
raise their awareness and appreciation of front-liners. In addition, as the teachers
reported in their reflections on the project and their developed materials, the
project was an invaluable professional development experience. It actively
engaged them with their community and led to the creation of interesting local
multimodal ELT texts which, according to them, are more relevant and interesting
to their students than the coursebooks they are currently using. Figure 3 is a
sample from a group’s graphic oral narrative “Suffering in silence”. The narrative
was obtained from a male nurse who recounted what he and other healthcare
workers had been through during the pandemic.
Figure 3. Snapshot from the graphic oral history narrative “Suffering in silence”
214
4. Conclusion
Adopting commercial, globalized ELT materials, which are usually mass
produced in the West, is problematic; those materials are generic and may contain
cultural or social elements that may not be convenient for local students.
Furthermore, those imported materials and coursebooks are usually Anglo-
centric and undermine the local varieties of English and have not satisfactorily
conceptualized English as an international language. Therefore, Malaysians must
not undermine the contribution of their local culture and languages to the English
language, and they should not doubt themselves, their expertise, or their
capability of developing efficient local ELT materials. They must strive to teach
English effectively while maintaining the balance between the perceived
importance of English as an international language and the passion for nation
building and preserving its local cultural heritage. Thus, the proposed conceptual
model in this paper is of relevance here. It provides Malaysian materials designers
and English language teachers with a guiding framework that enables them to
develop effective local multimodal ELT materials in the form of graphic oral
history texts. By developing graphic oral history materials, these professionals
will be enacting the TEIL paradigm and multimodality in their English language
classrooms. Even though this paper addressed the Malaysian context particularly,
the proposed model could be beneficial and relevant to many other similar ELT
settings and contexts that aspire to innovatively teach English as an international
language.
Acknowledgments
We would like to thank UKM for its support for this research via its Grants GG-
2021-008 Dana Khas Penyelidikan FPEND Pembelajaran Futuristik, TAP-K017393
Tabung Agihan Penyelidikan (TAP), and GGPM-2017-126.
5. References
Abdul Aziz, A., & Chang, S.Y. (2021). Hysterical daydreaming: Personal narrative of Malaysian
youths schooling experience. Penerbit UKM.
Abdul Aziz, A., & Makhtar, R. (2021). TESL in-service teachers’ experiences when
evaluating an oral history workbook. 3L: Language, Linguistics, Literature, 27(3),
56–72. https://doi.org/10.17576/3L-2021-2703-04
Abdul Aziz, A., Narayanasamy, J. A. P., Mohamad, M., & Hameed, H. A. (2019). Student-
teachers’ experiences in creating oral history texts. 3L: Language, Linguistics,
Literature, 25(3), 79−89. https://doi.org/10.17576/3L-2019-2503-06
Abdul Rahim, H., & Jalalian Daghigh, A. (2020). Locally-developed vs. global textbooks:
An evaluation of cultural content in textbooks used in ELT in Malaysia. Asian
Englishes, 22(3), 317–331. https://doi.org/10.1080/13488678.2019.1669301
Ayers, W., & Ayers, R. (2013). Foreword: Every person is a philosopher/every day is
another story. In C. Mayotte (Ed.), The power of the story: The voice of witness teacher’s
guide to oral history (pp. 6–7). Voice of Witness.
Basal, A., Aytan, T., & Demir, I. (2016). Teaching vocabulary with graphic novels. English
Language Teaching, 9(9), 95. https://doi.org/10.5539/elt.v9n9p95
Basol, H. C., & Sarigul, E. (2013). Replacing traditional texts with graphic novels at EFL
classrooms. Procedia – Social and Behavioural Sciences, 70, 1621–1629.
215
Begoray, D., & Fu, H. (2015). Reading chasing adland: Middle years’ students in pursuit
of critical media health literacy. In D. Begoray & E. Banister (Eds.), Adolescent
health literacy and learning (pp. 25–44). Nova Science Publishers.
Boon Kheng, C. (2007). New theories and challenges in Malaysian history. In C. Boon
Keong (Ed.), New perspectives and research on Malaysian history: Essays on Malaysian
historiography (pp. 147–158). The Malaysian Branch of the Royal Asiatic Society
(MBRAS).
Bouckaert, M. (2019). Current perspectives on teachers as materials developers: Why,
what, and how? RELC Journal, 50(3), 439–456.
https://doi.org/10.1177/0033688218810549
Bouckaert, M., Konings, M., & van Winkelhof, M. (Eds.). (2018). Meaning-focused materials
for language learning. Cambridge Scholars Publishing.
Brozo, W. G., Moorman, G., Meyer, C., & Stewart, T. (2013). Content area reading and
disciplinary literacy: A case for the radical center. Journal of Adolescent & Adult
Literacy, 56(5), 353−357. https://doi.org/10.1002/JAAL.153
Burgo, C. (2016). Oral history as an innovative language teaching technique for Spanish
heritage language learners. International Journal of Teaching and Learning in Higher
Education, 28(3), 451–459. http://www.isetl.org/ijtlhe/
Burmark, L. (2008). Visual literacy. What you get is what you see. In N. Frey, & D Fisher
(Eds.), Teaching visual literacy: Using comic books, graphic novels, anime, cartoons and
more to develop comprehension and thinking skills (pp. 5–25). Sage.
Can, T., Frazier, S., McManus, C., & Rey, A. (2020). Glocalisation in action: ‘Less is more’
English coursebook series. Training, Language and Culture, 4(2), 56–66.
https://doi.org/10.22363/2521-442X-2020-4-2-56-66
Canagarajah, S. (2016). TESOL as a professional community: A half-century of pedagogy,
research, and theory. TESOL Quarterly, 50(1), 7–41.
https://doi.org/https://doi.org/10.1002/tesq.275
Christodoulou, N. (2016). Oral history as an effective pedagogical practice in high school:
Two examples of inspiration and engagement. In Our Schools/Our Selves (pp. 118–
126).
https://www.policyalternatives.ca/sites/default/files/uploads/publications/
National%20Office/2016/02/osos122_oralhistory.pdf
Damayanti, D., Fauzi, A., & Inayati, A. M. (2018). Learning materials: The “nucleus” of
language teaching. Journal of English Education, 3(1), 1–8.
https://doi.org/10.31327/jee.v3i1.417
Dutt-Doner, K. M., Allen, S., & Campanaro, K. (2016). Understanding the impact of using
oral histories in the classroom. The Social Studies, 107(6), 257–265.
https://doi.org/10.1080/00377996.2016.1221792
Eisner, W. (2008). Graphic storytelling and visual narrative: Principles and practices from the
legendary cartoonist. W.W. Norton & Company.
Gallo, D., & Weiner, S. (2004). Bold books for innovative teaching: Show, don’t tell:
Graphic novels in the classroom. The English Journal, 94(2), 114–117.
https://doi.org/10.2307/4128785
Ganapathy, M. (2011). The effects of the multiliteracies approach on the writing performance of
EFL students in a Chinese school in Penang: Teacher and student perceptions [PhD
thesis]. Universiti Sains Malaysia.
Gay, G. (2018). Culturally responsive teaching: Theory, research, and practice (3rd ed.).
Teachers College Press.
Grabe, W. (2014). Key issues in L2 reading development. 4th CELC Symposium Proceedings
[symposium].
https://www.nus.edu.sg/celc/research/books/4th%20Symposium%20proceed
216
ings/2).%20William%20Grabe.pdf
Griffith, P. (2010). Graphic novels in the secondary classroom and school libraries. Journal
of Adolescent & Adult Literacy, 54(3), 181–189.
https://www.proquest.com/scholarlyjournals/graphic-novels-secondary-
classroom-school/docview/791763088/se2?accountid=7098
Harmer, J. (2012). Essential teacher knowledge: Core concepts in English language teaching.
Pearson Longman.
Jabeen, F., & Shah, S.K. (2011). The role of culture in ELT: Learners’ attitude towards the
teaching of target language culture. European Journal of Social Sciences, 23(4), 604–
613.
https://www.researchgate.net/publication/260045647_The_Role_of_Culture_in
_ELT_Learners'_Attitude_towards_the_Teaching_of_Target_Language_Culture
Jamil, N. A., & Abdul Aziz, A. (2021). The use of multimodal text in enhancing students’
reading habit. Malaysian Journal of Social Sciences and Humanities (MJSSH), 6(9),
487–492. https://doi.org/10.47405/mjssh.v6i9.977
Jones, D. W. (1998). Distressing histories and unhappy interviewing. Oral History, 26(2),
49–56. http://www.jstor.org/stable/40179521
Kachru, B. B. (1985). Standards, codification, and sociolinguistic realism: The English
language in the outer circle. In R. Quirk and H.G. Widdowson (Eds.), English in
the world: Teaching and learning the language and literatures (pp. 11–30). Cambridge
University Press.
Kachru, B. B. (1986). The power and politics of English. World Englishes, 5(2–3), 121–140.
https://doi.org/10.1111/j.1467-971X.1986.tb00720.x
Kanoksilapatham, B. (2018). Local context-based English lessons: Forging northern Thai
knowledge, fostering English vocabulary. 3L: Language, Linguistics, Literature,
24(2), 127−142. https://doi.org/10.17576/3L-2018-2402-10
Khodadady, E., & Shayesteh, S. (2016). Cultural and linguistic imperialism and the EIL
movement: Evidence from a textbook analysis. Issues in Educational Research, 26(4),
604−622.
https://www.researchgate.net/publication/309768088_Cultural_and_linguistic
_imperialism_and_the_EIL_movement_Evidence_from_a_textbook_analysis
Kirkpatrick, A. (2007). Teaching English across cultures: What do English language
teachers need to know to know how to teach English? English Australia Journal,
23(2), 10–36. http://hdl.handle.net/10072/58563
Krashen, S. D. (2004). The power of reading: Insights from the research. Heinemann.
Kress, G. (2009). Multimodality: A social semiotic approach to contemporary communication.
Routledge. https://doi.org/10.4324/9780203970034
Kumaravadivelu, B. (2016). The decolonial option in English teaching: Can the subaltern
act? TESOL Quarterly, 50(1), 66–85. https://doi.org/10.1002/tesq.202
Kwon, H. (2020). Graphic novels: Exploring visual culture and multimodal literacy in
preservice art teacher education. Art Education, 73(2).
https://doi.org/10.1080/00043125.2019.1695479
Lekawael, J. R. F., Emzir, & Rafli, Z. (2018). The cultural values in texts of English
coursebooks for junior high school in Ambon, Moluccas-Indonesia. Advances in
Language and Literary Studies, 9(2), 24–30.
https://doi.org/10.7575/aiac.alls.v.9n.2p.24
Lwin, S. M., & Marlina, R. (2018). Using folktales as a way to operationalise the paradigm
of teaching English as an international language. Asian Englishes, 20(3), 206–219.
https://doi.org/10.1080/13488678.2018.1440279
Marlina, R. (2018). Teaching English as an international language: Implementing, reviewing, and
re-envisioning world Englishes in language education. Routledge.
217
Matsuda, A. (2012). Teaching materials in EIL. In L. Alsagoff, S.L. Mckay, G. Hu, &
W.A. Renandya (Eds.), Principles and practices for teaching English as an international
language. Routledge. https://doi.org/10.4324/9780203819159
McKay, S. (2003). Toward an appropriate EIL pedagogy: Re-examining common ELT
assumptions. International Journal of Applied Linguistics (United Kingdom), 13(1), 1–
22. https://doi.org/10.1111/1473-4192.00035
McKay, S. L. (2012). Teaching materials for English as an international language. In A.
Matsuda (Ed.), Principles and practices of teaching English as an international language
(pp. 70–83). https://doi.org/10.21832/9781847697042-007
Mignolo, W. D. (2007). Delinking: The rhetoric of modernity, the logic of coloniality and
the grammar of de-coloniality. Cultural Studies, 21(2–3), 449–514.
https://doi.org/10.1080/09502380601162647
Montero, M., & Rossi, M. (2012). Exploring oral history methodology as a culturally
responsive way to support the writing development of secondary English
language learners. Oral History Forum d’histoire Orale, 32, 1–26.
http://www.oralhistoryforum.ca/index.php/ohf/article/viewArticle/430
New London Group. (1996). A pedagogy of multiliteracies: Designing social futures.
Harvard Educational Review, 66(1), 60–92.
https://doi.org/10.17763/haer.66.1.17370n67v22j160u
Oral History Association (OHA). (2022). Principles and best practices.
https://oralhistory.org/about/principles-and-practices-revised-2009/
Öz, H., & Efecioğlu, E. (2015). Graphic novels: An alternative approach to teach English as
a foreign language. Dil ve Dilbilimi Çalışmaları Dergisi: Uluslararası Çevrimiçi
Dergisi, 11(1), 75–90.
https://www.researchgate.net/publication/274737896_Graphic_Novels_An_Al
ternative_Approach_to_Teach_English_as_a_Foreign_Language
Pantaleo, S. (2015). Language, literacy and visual texts. English Education, 49(2), 113–129.
https://doi.org/10.1111/eie.12053
Pennycook, A. (2017). The cultural politics of English as an international language. Routledge.
Ritchie, D. A. (2003). Doing oral history: A practical guide. Oxford University Press.
Royce, T. D. (2007). Multimodal communicative competence in second language contexts.
In T.D. Royce, & W. Bowcher (Eds.), New directions in the analysis of multimodal
discourse (pp. 361–403). Lawrence Erlbaum.
Sabbah, M., Masood, M., & Iranmanesh, M. (2013). Effects of graphic novels on reading
comprehension in Malaysian year 5 students. Journal of Graphic Novels and Comics,
4(1), 146–160. https://doi.org/10.1080/21504857.2012.757246
Seglem, R., & Witte, S. (2009). You gotta see it to believe it: Teaching visual literacy in the
English classroom. Journal of Adolescent & Adult Literacy, 53(3), 216–226.
Siegel, M. (2012). New times for multimodality? Confronting the accountability culture.
Journal of Adolescent and Adult Literacy, 55(8), 671–681.
https://doi.org/10.1002/JAAL.00082
Sifakis, N., & Bayyurt, Y. (2018). ELF-aware teaching, learning and teacher development.
In J. Jenkins, W. Baker, & M. Dewey (Eds.), The Routledge handbook of English as a
lingua franca (Chapter 36). Routledge.
Strachan, L. M., & Winkel, C. (2020). Oral history: An unpredictably effective strategy for
teaching ESL to Saudi Arabian students. Academic Journal of Interdisciplinary
Studies, 9(1), 158–169. https://doi.org/10.36941/ajis-2020-0014
Thompson, P. (2000). The voice of the past: Oral history. Oxford University Press.
Toledo-Sandoval, F. (2020). Local culture and locally produced ELT textbooks: How do
teachers bridge the gap? System, 95, 102362.
218
https://doi.org/https://doi.org/10.1016/j.system.2020.102362
Tomlinson, B. (Ed.). (2016). SLA research and materials development for language learning (1st
ed.). Routledge. https://doi.org/10.4324/9781315749082
Tomlinson, B., & Masuhara, H. (2004). Developing language course materials. SEAMEO
Regional Language Center.
Tomlinson, B., & Masuhara, H. (2013). Adult coursebooks. ELT Journal, 67(2), 233–249.
https://doi.org/10.1093/elt/cct007
Vodniza, G. (2016). Stories that are not told: Everyone has a story, or a story that matters.
In E. Shaker (Ed.), Our schools/our selves: Oral history education, political engagement,
and youth. Canadian Center for Policy Alternatives.
Yahya, Z., Ibrahim, N., Ramli, R., Md Yusof, N., & Othman, Z. (2017). Your language my
culture (YLMC): Developing a local-culture elt matrix for schools in Terengganu,
Malaysia. Kajian Malaysia, 35(2). https://doi.org/10.21315/km2017.35.2.3
Yang, G., & Pien, L. (2006). American born Chinese. First Second.
Yeoh, B. S. (2003). Changing conceptions of space in history writing. In A.T. Ahmad, & T.
Liok Ee (Eds), New terrains on Southeast Asian history (pp. 30–55). Singapore
University Press and Ohio University Press.
Yusof, S. M., Lazim, Z. M., & Salehuddin, K. (2017). Teacher trainees’ perspectives of
teaching graphic novels to ESL primary schoolers. 3L: Language, Linguistics,
Literature, 23(3), 81−96. https://doi.org/10.17576/3L-2017-2303-06
Woolston, J. M. (2014). Graphic novels and comics in the classroom: Essays on the
educational power of sequential art Carrye Kay Syma and Robert G. Weiner,
(Eds). Jefferson, NC: McFarland, 2013. The Journal of American Culture, 37(2).
https://doi.org/10.1111/jacc.12179
219

Vol. 21, No. 8, pp. 219-234, August 2022
Received May 31, 2022; Revised Aug 12, 2022; Accepted Aug 18, 2022
Remote Teaching and Learning at a South

African University During Covid-19 Lockdown:
Moments of Resilience, Agency and Resignation
in First-Year Students’ Online Discussions
Pineteh E. Angu
Unit for Academic Literacy, Faculty of Humanities, University of Pretoria,
Pretoria, South Africa
Abstract. The aim of article is to analyse the remote teaching and learning
experiences of students, as expressed in online discussions among first-
year students at a South African university, to enhance understanding of
how learning remotely during Covid-19 lockdown affected students’
lives and their academic commitment. The article draws on data collected
from 45 students, mainly aged between 18 and 22, registered for an
academic literacy course. These students came from diverse racial,
socioeconomic and schooling backgrounds, although most were black
students. These participants’ Covid-19 and remote learning experiences
are used in this case study as qualitative datasets. To generate rich
narratives, a set of open-ended questions were designed and posted on
the course’s Blackboard page. The questions focused on the
socioeconomic, psychological and academic effects of Covid-19 during
the lockdown. At the end of the course, a thematic analytical approach
was used to identify and categorise participants’ online discussions into
key themes. The datasets were then interpreted through the lens of
predetermined concepts such as resilience, resilience and agency. The
data show that some students saw remote learning as an opportunity to
be creative and innovative, but for others, their socioeconomic situation
negatively affected their lives and academic commitment. Students’
remote teaching and learning experiences revealed moments of resilience
and agency, but also a deep sense of resignation to the socioeconomic and
psychological burdens of Covid-19. Based on these findings, this article
discusses some implications for the future of higher education in South
Africa and recommends possibilities for further research.
Keywords: Covid-19 lockdown; remote teaching/learning; resilience;

resignation; South Africa
©Authors
220
1. Introduction and Background

Full remote teaching and learning under the Covid-19 lockdown in 2020 and
continuing restrictions in 2021 has provided an opportunity for the academic
community in South Africa to reflect on the state of the country’s higher
education. The pandemic also created, or rather enforced, an unprecedented
opportunity for contact universities to devise innovative, flexible and inclusive
ways of teaching and assessment (Hedding et al., 2020; Mhlanga & Moloi, 2020).
As staff and students reconfigured their social lives to adjust to the “new normal”,
there were ongoing conversations about the gains, opportunities and challenges
of online teaching and learning during the pandemic. These conversations were
driven partly by the uncertainties and unpredictability surrounding the Novel
Corona virus (Covid-19) and the future implications of the shift to online delivery
for higher education teaching and learning (Hedding et al., 2020; Le Grange,
2020b; Marongwe & Garidzirai, 2021; Motala & Menon, 2020).
As the debate on the use of different digital platforms grew, it became clear that
the pandemic had “add[ed] a new layer of complexity and uncertainty to an
already volatile and contested higher education” situation (Motala & Menon,
2020, p. 82). Moreover, the pandemic came at a time when South African
universities are still grappling with several items of unfinished business, such as
transformation and curriculum decolonisation. Issues of inclusivity and social
justice were again at the epicentre of these discussions, forcibly reminding us that
South African universities are still spaces for both the haves and have-nots, or to
put it differently, they are learning spaces where the privileged and the less-
privileged co-exist. Here, the familiar narrative of the rural South Africa students
with no food and living in deplorable conditions, who have no digital devices and
who can’t afford data dominated the conversations (Czerniewicz et al., 2020,
Dube, 2020). The severity of the Covid-19-related teaching and learning
challenges varied from university to university, with the most severely affected
students based at the historically disadvantaged universities and in vulnerable
communities (Le Grange, 2020b; Marongwe & Garidzirai, 2021; Mbhiza, 2021).
Because these universities were under-resourced to begin with, staff and students
encountered more technological and infrastructural challenges when the higher
education sector was unexpectedly forced to resort to remote teaching and
learning by the pandemic (Alex, 2022; Mtshweni, 2022; Odularu et al., 2022).
The learning challenges of particularly black South African students are well
known, and they resurfaced in the national consciousness as schools and
universities grappled with the academic disruptions induced by Covid-19. The
exacerbation of these long-standing challenges suggests that these challenges
have been approached and addressed in a very tokenistic style by South African
universities (Angu et al., 2020; Czerniewicz et al., 2020). Therefore, when the non-
distance universities were forced to migrate to virtual teaching, the first concern
of academic and support staff members, and university managements, was to
lessen the Covid-19 burden on already vulnerable students. They used existing
quantitative data to devise plans and to manage the challenges associated with
remote teaching and learning, especially during the national lockdown
(Agormedah et al., 2020; Hedding et al., 2020; Le Grange, 2020a; Motala & Menon,
221
2020). Consequently, many scholarly publications that emanated from these

conversations neglected the voices of students, as they focused on how
universities tried to navigate the Covid-19-induced emergency online teaching
and learning landscape. They also tended to focus primarily on the negative
effects of Covid-19, often ignoring students’ ability to be resilient and the ways in
which they exercised a sense of agency during this life-threatening crisis (see
Gumede & Badriparsad, 2021; Madiope & Mendy, 2021; Mtshweni, 2022).
Given that teaching and learning in higher education is negotiated between

students, staff and university management, student voices are critical for our
understanding of the impact of full remote teaching and learning during the
Covid-19 lockdown on students’ academic journeys. Against this background,
this article uses qualitative data gleaned from online discussions with 45 first-year
students in a Health Sciences academic literacy course to understand how Covid-
19 and online teaching and learning have affected the lives and academic journeys
of this cohort of students. It argues that while some of these students saw remote
learning as an opportunity to be creative and innovative, epitomising moments of
resilience and agency, for others, the socioeconomic demands that they faced had
a negative effect on their lives and ultimately their academic commitment,
resulting in a deep sense of resignation to the socioeconomic and psychological
burdens of Covid-19.
To make meaning of these discussions, I first review the literature on online

teaching and learning during Covid-19 lockdown with a focus on South Africa.
Then I explain how the data was collected and analysed, and discuss the key
themes that emerged from the data. Finally, I consider the future implications for
teaching and learning in South Africa and suggest possibilities for further
research.
2. Online Teaching and Learning During Covid-19 – the South African

Context
Covid-19 has produced a new body of knowledge in South African higher
education about emergency online teaching and learning during a pandemic. The
large range of articles published during the national lockdown in South Africa
focused on different topical issues, such as the relationship between the fourth
industrial revolution (4IR), the challenges and opportunities of emergency remote
teaching and social inequalities in South African higher education during and
after Covid-19 (Alex, 2022; Mhbiza, 2022; Mtshweni, 2022). These studies sought
to make sense of how the pandemic has shaped social and academic lives, while
inadvertently exposing the deepening social inequalities in South African
universities and communities (Czerniewicz et al., 2020; Hedding et al., 2020;
Marongwe & Garidzirai, 2022; Motala & Menon, 2020; Mtshweni, 2022). Recurring
references in these studies to the “new normal” suggest that the world will never
be the same again, and predict that modes of teaching and research in the higher
education landscape will change drastically after the pandemic. Many authors
believe that, despite the devastating effects of Covid-19 on our mind and souls,
the pandemic has provided an opportunity to reimagine, rethink and recreate the
university space (Du Preez & Le Grange, 2020; Hedding et al., 2020). One has to
222
continue to ask how, in finding ways to reimagine the South African university
space, these institutions will address the social issues, which they have neglected
for years, and which have resurfaced so clearly during this pandemic.
When the 2020 academic year in South Africa began, even though news of a new
deadly virus spread globally, it was business as usual on university campuses.
Lectures and social activities continued uninterrupted, as the government and
university management cautiously monitored the rapid spread of the virus.
However, the always buzzing campuses were deserted when the South African
government announced a full national lockdown in March 2020 and students had
to return home. Once it became clear that the lockdown would not end after the
initial three weeks, universities resorted to full online teaching and learning to
salvage the academic project amidst mounting fears, anxieties and uncertainties,
as the virus spread insidiously throughout South Africa. The announcement of a
national lockdown prompted university leaderships to respond speedily, as
events from the recent past tell us that “higher education in South Africa is no
stranger to crises, having experienced several of these at different junctures in its
history” (Motala & Menon, 2020, p. 85). For example, the #Rhodesmustfall and
#feesmustfall had already changed the higher education landscape in South
Africa drastically, with universities investing large sums in educational
technologies that can support massive hybridised teaching and learning, either
synchronously or asynchronously.
Although these landmark student protests transformed the South African higher
education landscape in unimaginable ways, it would be naive to compare these
events to the unprecedented disruptions caused by Covid-19. Unlike with Covid-
19, the managements of universities could predict that there would be explosive
student protests because tensions had been simmering on campuses for years
(Czerniewicz et al., 2020; Hedding et al., 2020; Madiope & Mendy, 2021; Motala &
Menon, 2020). In the case of the #mustfall movements, the management of
universities long ignored visible warning signs, but once the movements started
at some universities, others could hurriedly prepare. However, because the
emergence and spread of Covid-19 was so abrupt and unpredictable, universities
were forced into emergency remote teaching and learning without adequate
planning and preparation. This meant students and staff had to experiment with
different technologies to teach, and universities chopped and changed strategies,
requirements and expectations on the fly as more research evidence about the
mutations of the virus and the likelihood of an extended international crisis
became available (Adedoyin & Soykan, 2020; Czerniewics et al., 2020; Mhlanga &
Moloi, 2020). At some universities, especially those in the rural parts of South
Africa, remote teaching and learning challenges were exacerbated by inadequate
resources and lack of staff with the right skillsets (Alex, 2022; Marongwe &
Garidzirai, 2021; Mtshweni, 2022).
In fact, staff and students at most universities had been using learning
management systems such as Blackboard for years. However, they had mainly
been used to complement contact teaching and learning. As a result, many
academic staff members were not completely prepared for remote learning
223
because “academic staff at contact universities typically have little, if any

experience of training in the pedagogy or delivery of online learning” (Hedding
et al., 2020, p. 1). In addition, students from both privileged and disadvantaged
communities were left to struggle with access to devices, data and internet
connectivity problems in urban and rural areas, without the quality of student
support they were accustomed to, and these problems were more severe in
vulnerable communities (Alex, 2022; Dison et al., 2022; Le Grange, 2020b;
Mtswheni, 2022). To alleviate the burdens of emergency remote learning, many
South African universities provided devices and data to students in need. Despite
such support from universities, many students found themselves at the
intersection of social challenges such as structural poverty and inequality, as well
as the continued emotional and psychological challenges related to Covid-19
(Adedoyin & Soykan, 2020; Czerniewics et al., 2020; Mhlanga & Moloi, 2020).
On the positive side, the pandemic has definitely ignited a renewed passion to
reimagine and rethink modes of teaching and learning in the South African higher
education context. However, Le Grange (2020a) cautions that this process might
open up space for the “uberfication” of the university: as South African
universities shift towards online pedagogy, higher education studies will be
commodified and students will become clients, choosing what to learn, how to
learn and where to learn and “the lecturer becomes an on-demand worker” (Le
Grange, 2020a, p. 6). This means the university will become a marketplace and it
will entrench an already corporatized leadership style aimed primarily at
gratifying the clients while “academic staff members are increasingly subjected to
forms of surveillance, including through performance management” (Le Grange,
2020a, p. 8). If the “uberfication” of the university becomes a reality, this will
further deepen the social inequalities already prevalent in South African
universities, as there will be no “equitable benefits from the affordances of new
technologies” (Du Preez & Le Grange, 2020a, p. 90).
Thinking through the idea of “uberfication” of universities, one is again reminded

that South African universities are still spaces defined by different forms of
discrimination, often captured in the paradoxical learning experiences of students
in impoverished townships and suburban homes. The class structures that have
defined our universities for decades have been made increasingly visible,
epitomising the disparate worlds of our students (Angu, 2019; Dube, 2020;
Mhlanga & Moloi, 2020). Under Covid-19 conditions, some participated in remote
learning, undisturbed, in the comfort of their homes, with unlimited data, a
dedicated study space and reliable devices, but others had to study in shared
communal spaces with family, with very limited data and out-dated devices. This
meant, according to several studies, that many students struggled with online
learning because of the cost of data, poor connectivity or simply because they had
no dependable electronic devices (Gumede & Badriparsad, 2021; Marongwe &
Garidzirai, 2021; Mbhiza, 2021). For example, Alex (2022), Mtshweni (2022) and
Odularu et al. (2022) argue that resource-constrained and historically
disadvantaged universities in rural areas have been affected more than those in
urban areas, revealing clearly the digital inequalities in the South African higher
education sector. Many students’ experiences during the national lockdown are
224
therefore tales of those living on the fringes of survival – the voices from the
margins – as well as tales of those living in luxury and comfort. These paradoxes
tell us that South African universities’ “capacity to respond [to Covid-19] is thus
uneven along the predictable fault lines of class, race and gender” (Manderson &
Levine, 2020, p. 368). These experiences have had several implications for the
academic performance of students during the national lockdown from March
2020.
3. Method of Data Collection

The data used in this article emerged from online discussions with 45 students
registered for an academic literacy course offered by a Department of Family
Medicine at a South African university. They were mainly between the ages 18
and 22 and from different racial, social, economic, and schooling backgrounds.
However, the majority were black South African students. These students were
therefore the project participants and constituted the unit of analysis for this case
study. Their stories about Covid-19, the lockdown and remote learning are used
in this article as qualitative datasets. To sustain participation and generate rich
narratives during the online discussions, a set of open-ended questions were
designed and posted on the course Blackboard page and students were regularly
reminded to participate through announcements. The questions focused on the
social, economic, psychological and academic effects of Covid-19 during
lockdown. The course convened once a week for one hour 40 minutes (a double
period). The first hour was used for normal lectures and class activities. To enliven
the lectures, the last 40 minutes were reserved for discussion. In addition, regular
communication through announcements ensured that discussion continued after
lectures.
At the end of the course, a thematic analytical approach was used to identify and
categorise participants’ online discussions into key themes. The datasets were
then interpreted through the lens of predetermined concepts: agency, resilience
and resignation. This approach provided a prism to avoid generalising about the
South African Covid-19 and remote learning experiences, and to access and make
meaning from participants’ subjective perspectives on remote teaching and
learning. The analysis and interpretation of these personalised experiences
revealed that, although some of the participants saw remote learning during
lockdown as an opportunity to be creative and innovative, for others, the social
and economic demands of learning remotely during pandemic exerted various
pressures on their lives. Consequently, they could not cope with academic
expectations, resulting in de-registration (Angu, 2019; Dube, 2020; Mhlanga &
Moloi, 2020). Drawing on these findings, the following sections have used the
concept of resilience, agency and resignation as frames to analyse and make
meaning from the data collected from the participants.
To use students’ online discussions as research data, ethics clearance was obtained
from the university where the study was done. This clearance granted permission
to use students’ learning activities and assignments for research purposes. In line
with the ethical requirements, the participants have been anonymised and direct
225
quotations from the online discussions are cited verbatim (in italics) in this article
simply as “student response”.
4. Presentation and Analysis of Students’ Discussions in a Virtual

Space
The discussion forum on Blackboard provided a space for this group of students
to share with their peers, without any restrictions, their Covid-19 and remote
learning experiences. As the students navigated the challenges of full remote
learning, this space became an outlet to share their frustrations, disillusionments
or their impassioned commitment to their studies, driven by a sense of resilience
and agency. The data collected from these online discussions provided an
opportunity for different interpretations because of the uniqueness of their stories
about Covid-19, lockdown and remote teaching and learning in South Africa
(Angu, 2019; Motala & Menon, 2020).
4.1 Memories of the Social Realities in South Africa

Many scholars have represented Covid-19 as the “great equaliser”, arguing that
the virus is race-, class- and ethnicity-blind. However, the stories of this group of
students tell us that it would be misleading to ignore intersections of race and
class in trying to understand how the virus has affected the lives of students. For
instance, in a country that was already grappling with different forms of
inequality years before the viral outbreak, it is disingenuous to fail to focus on
social structures that continue to drive inequalities in South Africa and how these
affected student learning during lockdown-induced remote learning (Angu, 2019;
Czerniewicz et al., 2020; Sokhulu, 2020). The online discussions allowed students
to remember, through the lens of their remote learning experiences, the
digital/social inequality in South Africa, even though they were students at the
same university. One student narrated her remote learning experience as follows:
My worries regarding the online learning is that some of us live in a
crowded space with many family members and there is a lot of noise which
might deprive a person from focusing. We cannot go to libraries or any
quiet place as a result of this lockdown due to Covid-19. Besides what I
have aforementioned, data will be a problem and some of us live in places
where network connection is not stable hence there will be chances of
missing out on online teaching. But if it is the only option to continue
with our academics then we might as well use it as it will benefit us, and
if ever a person missed a session due to any problems then he/she can catch
up with any friend who was in the online teaching. (Student response)
This student lives several kilometres from the university and her comment
highlights the social differences between students who live in rural areas and
those in urban areas, and how their social spaces became impediments to learning
during remote learning. Also, because the student’s experience of remote learning
happened in a vulnerable rural area, it can be read as a process of remembering
“how issues of equity and inequality have played out in the ‘pivot’ to remote
teaching and learning” (Czerniewicz et al., 2020, p. 947). In this testimony, two
forms of inequality (digital and social) are shown to intersect to influence this
student’s remote learning efforts. This student experience is not new in South
Africa, but Covid-19 has helped to excavate memories of a post-apartheid South
226
Africa still reeling from different forms of inequalities (Du Preez & Le Grange,
2020; Motala & Menon, 2020).
Given that this South African university’s student population is demographically

diverse, discrepancies in students’ lived experiences in rural and urban areas and
how they affected their remote learning efforts featured strongly in the online
discussions. In the following comment, one participant remembers poor living
conditions, digital constraints and the learning disruptions caused by this
student’s situation:
I would like to address how this pandemic is affecting my learning. Like I
said before I'm in the village with my mother right now and she is
currently sick so I have to care for her since there's only the two of us here.
The connection is not stable and my phone does not have enough space to
download all your slides. It's very painful what this pandemic is really
doing but I hope for the best. I would like you to share everything on
Whatsapp because it’s very easy to access it. (Student response)
Here, we see how this student’s access to remote learning during a viral pandemic
is hamstrung by broader societal problems that the South African government has
failed to address in its grand transformative agenda designed to “redress
inequalities of access, participation and success” (CHE, 2004, p. 24). This remote
learning experience captures how Covid-19 and its wider effects feed on endemic
structural vulnerabilities and inadvertently compromise students’ access to
learning in remote environments (Angu, 2019; Team & Manderson, 2020).
For years now, the gap between rich and poor has continued to widen. The ANC-
led government is often criticised for failing to address social challenges,
especially in vulnerable communities. For example, in the middle of the
pandemic, the government was heavily lambasted “for its poor handling of
emergency public procurement and the distribution of financial support as it
became evident that the procurement had not been able to avoid endemic
corruption in government” (Van Schalkwyk, 2021, p. 46; see also Baloyi, 2020).
The handling of procurement processes exposed the government’s ineptitude,
which provided a fertile ground for more corruption in government departments,
delaying the provision of protective equipment and learning devices for learners
in need (Du Preez & Le Grange, 2020; Van Schalkwyk 2021). Universities managed
their own procurement processes, but the government’s inability to efficiently
manage Covid-19 relief support and the implications for remote learning recurred
in the discussions. One participant commented:
The first thing that we should look at, is that the Covid-19 exposed our
government. We now know that our government didn't invest much into
our education, they didn't have a backup plan for the nation to keep on
with the school work during such outbreaks and now they have made a
resolution whereby most of the students will get devices to use as a backup
to learn which is a good thing and through this, the inequalities in the
education system are fading bit by bit because even poor students will be
able to get them and learning will continue. I believe that our government,
universities and colleges have to learn their lesson that they should invest
227
more in online learning and also covering every student poor or rich….
(Student response)
Several studies have documented the South African government’s failures,

including a lack of political will to address educational inequalities in basic and
higher education. It is well-known that while schools in urban and opulent
suburbs are resource-rich, many schools in black communities are severely under-
resourced (see, for example, Alex 2022; Marongwe & Garidzirai, 2021; Mtshweni
2022; Shava 2022). In the above comment, the student’s memory of the school
system in South Africa is a snapshot of the two disparate worlds that continue to
influence the scholastic experiences of students and the failure of the state to
address socioeconomic constraints in the schooling system. For this student, a
dysfunctional schooling system and remote learning challenges during lockdown
are symptomatic of an ailing state, which has failed to address its social problems
(Czerniewicz et al., 2020; Motala & Menon, 2020).
4.2 Student Discussions as Moments of Resignation or Resilience

The online discussion by students also contained vignettes of resignation or
resilience as they grappled with emergency remote learning during the national
lockdown. Here, this group of students displayed their personal dispositions,
either to confront the challenges of remote learning or to allow their
socioeconomic vulnerabilities to influence their determination to succeed
(Chisadza et al., 2021; Mahendher et al., 2021; Vaugh, 2020):
I am trying to get used to these online platforms that I have never used
and I am equally concerned that technology might fail me. But because of
the Covid-19, we must do it, not only to complete the academic year but
most importantly to protect lives. (Student response).
Most of us are all worried about this whole online thing and if it will ever
be fruitful because on my side, I am back in the village and the internet
connection is very poor. I'm currently able to use my phone as a device to
interact with everyone, no laptop or bigger device for the upcoming
lessons. I am even scared of what is going to happen when it's time for
tests and exams since they'll be conducted online because my phone is
giving me problems as well. So, it is really overwhelming but I am trying
my best to not miss anything because we cannot afford to lose the whole
year because of this pandemic. (Student response)
Despite the multiple challenges associated with emergency online learning during
lockdown, universities’ rapid pedagogical response to Covid-19 disruptions was
construed as empowering for many students, regardless of their social
backgrounds (Chisadza et al., 2021; Makgahlela et al., 2021). These responses
expose a strong determination to succeed despite the challenges with devices and
connectivity. Although many of these students experience different forms of
exclusion, they still imagined their education as critical for their social
transformation. For these respondents, losing an academic year would only
worsen their material conditions in the aftermath of Covid-19. Here, we see
students who have been emboldened by pre-existing social conditions and are
ready to endure the impact of Covid-19. In this context, students’ sense of
228
resignation or resilience was influenced by individual motivation, self-awareness

and adaptive behaviour, as well as the physical and mental strength to study
(Chisadza et al., 2021; O’Sullivan, 2006).
In the following two responses, the need to be resilient was driven by students’
sense of empathy, stemming from the recognition that Covid-19 is a human crisis
that has affected both students and lecturers (Le Grange, 2020b; Mbiydzebyuy &
Silungwe, 2020). The dominant narrative that someone’s race and class or
privilege cannot shield the person from Covid-19 seemed to have instilled a
renewed sense of survival and willingness to overcome the challenges associated
with remote learning during lockdown:
This pandemic is a first for many of us. It's overwhelming in many
concepts for both lecturers and students. Online teaching and learning
have its pros and cons. Many of us might have the data and means to
connect to the internet but our signal in the area might not be good. In all
honesty, Sir, I personally feel that at this point in time we can't say
whether online teaching will have a positive or negative effect on both
students, lecturers and the university as a whole and most importantly
the effect it will have on those who move on to second year (following year
of study). (Student response)
We are as equally concerned as you are regarding this online method of

teaching but I believe we don’t have any alternative at this moment.
Instead we need to be open-minded towards it so that it doesn’t become
tedious...I suggest we become patient with each other as this type of
learning is new to all of us. (Student response)
There is a sense of collective pain and suffering in these responses, which serves
as an additional incentive to resist remote learning challenges. The students’
acknowledgement of the lecturers’ frustrations and anxieties, as well as the
impacts of the pandemic on university resources seemed to have provided an
additional reason to be resilient.
4.3 Student Online Discussions as Expressions of Agency

The collection of online discussions showed clearly that although South African
students experience different forms of marginalisation, many come to university
with knowledge and lived experiences that can be used to confront their
challenges in higher education (see Clark et al., 2016; Luo et al., 2019; Vaugh,
2020). These online discussions became prisms through which we can understand
how a group of students was able to exercise their sense of agency as they studied
remotely during the national lockdown. Here, we see these students’ attempts to
reimagine and rethink their learning strategies as they navigated the unexpected
learning disruptions caused by the national lockdown. Students’ enactment of
agency is captured in the following two responses:
I think we should use this opportunity to familiarize ourselves with
online learning as I think this will be an important part of studying in the
future and this will help to digitalise some classes. Digitalization of
classes will help a lot with time management and learning at your own
pace. I am concerned that new test dates are not made readily available by
229
all our subjects so I feel afraid that I would not have adequate time to
prepare. (Student response)
To succeed with online learning, one must be disciplined enough to not

only attend the classes and passively absorb the information but to
actively participate. In addition to self-discipline, one must also have
adequate time management skills so that you do not feel overwhelmed
when classes and assignments start to pile up. (Student response)
In these discussions, the students cited above approached remote learning as an

opportunity to remake their social identities, learning activities and digital
experiences (Lindgren & McDaniel 2012; Vaugh, 2020). In the quotations, the
students capture a shift to new and creative learning ideas employed to “give
significance to the world in purposeful ways with the aim of creating, impacting
and/or transforming themselves and/or the conditions of their lives” (Basu et al.,
2009, p. 345). The display of student agency in these discussions was also
evidenced in the way the students envisioned a future digitized higher education
space and the role that Covid-19 remote learning could play in preparing them
for the future higher education environment. Remote learning during lockdown
is constructed here as an opportunity for students “to make choices, act on their
intentions, and take actions in their efforts to develop their own stance in [this]
learning context” (Vaugh, 2020, p. 116).
Emergency teaching and learning during national lockdown in South Africa

meant students and lecturers needed more support to complete the set
curriculum. Unlike the #mustfall movements, no one was physically, technically
or emotionally prepared for full remote learning in a viral pandemic situation (see
Du Preez & Le Grange, 2020; Laher et al., 2021; Mahlaba, 2020; Van Wyk, 2021).
In the online discussions, there was evidence of peer support and learning
amongst students using different social media platforms. Students’ sense of
agency in this context was shown in their ability to think collaboratively as they
confronted different online learning challenges such as self-motivation, time
management and self-discipline:
Guys, first thing that you have to do is to set your own time table that
literally works for you, but remember that a day is made of 24 hours you
cannot make a time table whereby you gonna study for 4 hours and sleep,
watch movies and Television for 20 hours. Try studying for more than 6
hours but with breaks in between just for brains to rest or you can even
get some nap in those breaks, but you have to tell your siblings or parents
to wake you up after how many minutes. (Student response)
In this excerpt, agency is about sharing ideas and strategies for meaningful online
learning with peers. It is about building a relationship of collaboration as students
were able to use the discussion forums to reach out to their peers (Angu 2019;
Clark et al., 2016). In the context of online learning during lockdown, effective
learning was not restricted to this group of students independently taking
ownership of their learning. In fact, the online space provided an outlet for
collaborative learning and peer-to-peer support. Here, agency is about collective
230
thinking and engagement with classroom activities, learning strategies and

coping mechanisms (Angu, 2019; Luo et al., 2019; Vaugh, 2020).
5. Discussion
As the Covid-19 virus continues to mutate in very unpredictable ways, this
pandemic is expected to continue to disrupt the lives of South African students in
the (un)foreseeable future. The economic and psychosocial effects of the
pandemic have continued to influence teaching and learning, even after the
national lockdown (Chisadza et al., 2021; Laher et al., 2021). As staff and students
concluded the 2021 academic year, South African university campuses are yet to
return to the old normal, although universities are no longer operating in an
emergency mode. The majority of South African students have continued to study
remotely, but with relatively restricted support in the form of devices and data
from universities, compared to the support they received during emergency
remote learning in 2020 (Le Grange, 2020b; Mahlaba, 2020; Sokhulu, 2020). This
means that in the context of widening social inequalities and the digital divide,
many of these students are still struggling to access meaningful teaching and
produce quality academic projects (Laher et al., 2021; Makgahlela et al., 2021;
Mtshweni, 2022; Odularu et al., 2022).
As universities continued to operate remotely into 2021, there was an increase in

scholarly publications on students’ perceptions and experiences of Covid-19-
induced online learning. An increasing number of studies now focus on
understanding the perspectives of students as universities grapple with the
deepening effects of Covid-19 on the academic project (see Laher et al., 2021;
Makgahlela et al., 2021). Although these studies seek to understand student
experiences, their findings tend to focus on the dominant narrative of the
vulnerable poor student, living in miserable conditions, which are not conducive
for learning. These studies continue to perpetuate the familiar victimhood
narrative, often neglecting the transformative power of destitution and under-
privilege shown in the multiple ways in which students have exercised their
agency and resilience. Put differently, students from impoverished backgrounds
are repeatedly portrayed as fragile, and therefore incapable of exercising any
agency, while learning remotely (Makgahlela, 2021; Mtshweni, 2022; Odularu et
al., 2022; Vaugh, 2020). There are several merits to this argument, if one considers
the existential social equalities and the implications for teaching and learning in
South Africa. However, history has taught us that the social ills of society are also
catalysts of resilience and agency.
Based on the data discussed above, this article makes two key arguments. Firstly,
although the narrative of vulnerable poor students with no device, data and/or
connectivity dominates in earlier publications on emergency remote teaching and
learning, many of these vulnerable students in fact saw this mode of learning as
an opportunity to reimagine and rethink their learning approaches (Dison et al.,
2022; Mbhiza, 2021). This category of students was able to experience fully the
transformative potential of online learning during lockdown. As a result, they
attempted to exercise a form of resilience and agency as they grappled with the
challenges of remote learning. Secondly, while students believed that every dark
cloud has a silver lining, there were also some students who displayed resignation
231
to the Covid-19 and remote learning challenges. This category of students

capitalised on how the negative effects of remote learning and the Covid-19
lockdown, as well as the worsening socioeconomic problems in post-apartheid
South African communities did indeed affect their academic performance (Alex,
2022; Dison et al., 2022; Shava, 2022).
6. Conclusion
The #Rhodesmustfall and #feesmustfall experiences have shown that South
African students, including those from poorer backgrounds, have learned to
survive in disruptive learning environments. Although it would be naïve to
compare the experiences of Covid-19 with these two critical moments in South
African higher education, online discussions collected from this study’s sample of
students have shown that students who have increasingly been regarded as
fragile and vulnerable can in fact be extremely resilient. Although institutional
and academic staff support were necessary during emergency remote teaching
and learning in 2020, a key predictor of success during emergency remote
teaching and learning was a strong determination to succeed against all odds. This
sample of students’ remote teaching and learning experiences epitomised
moments of resilience and agency as they navigated the academic disruptions
caused by the pandemic. However, online discussions also revealed some
students’ sense of resignation to the economic and psychosocial burdens of Covid-
19 (Angu, 2019; Dison et al., 2022; Hemson, 2019; Holton, 2019; Mbhiza, 2021).
In the second half of the 2022 academic year, the South Africa university at which
this study was done returned to full contact mode after the government lifted all
Covid-19 restrictions. There is general excitement around its campuses, but
students and staff have to readjust to socialising without wearing masks or
maintaining social distance. For some first-year students, it is an opportunity to
participate in their long-awaited “university experience”; for others, it is an
overwhelming experience with its own fears and anxieties. For university
managements, Covid-19 has provided an opportunity to reimagine the idea of
higher education, to respond to new, efficient and seemingly more productive
ways of learning and working revealed by Covid-19 (Dison et al., 2022; Mbhiza,
2021; Shava, 2022).
7. Recommendation for future research

Since this article is based on qualitative data gleaned from a non-representative
sample, the findings, analysis and discussions are not generalisable. Nor can they
be used to make recommendations for the South African higher education
context. With students back on campus, a large-scale quantitative study on
student experiences during and after the lockdown is likely to produce findings
that can influence new teaching and learning policies as South African
universities enter a post-Covid-19 era.
8. References
Adedoyin, O. B., & Soykan, E. (2020). Covid-19 pandemic and online learning: The
challenges and opportunities. Interactive Learning Environments.
https://doi.org/10.1080/10494820.2020.1813180
232
Agormedah, E. K., Henaku, E. A., Ayite, D. M. K., & Ansah, E. A. (2020). Online learning
in higher education during Covid-19 pandemic: A case of Ghana. Journal of
Educational Technology and Online Learning, 3(3), 182-210.
https://doi.org/10.31681/jetol.726441
Alex, J. K. (2022). Impact of the COVID-19 pandemic on the academic life of higher
education students: A rural South African perspective from a global study. South
African Journal of Higher Education, 36(1), 20-40. http://dx.doi.org/10.20853/34-4-
3607.
Angu, P. E. (2019). Students’ experiences of teamwork: Dealing with conflicting identities
during a team-based academic writing project in a clinical medical practice
programme. International Journal of Diversity in Education, 19(1), 11-25.
https://doi.org/10.18848/2327-0020/CGP
Angu, P. E., Boakye, N., & Eybers, O. O. (2020). Rethinking the teaching of academic
literacy in the context of calls for curriculum decolonization in South Africa. The
International Journal of Pedagogy and Curriculum, 27(1), 1-16.
Angu, P. E. (2019). Understanding voices from the margins: Social injustice and agency in
first year students’ literacy narratives. Journal of Further and Higher Education,
43(8), 1152-1162. https://doi.org/10.1080/0309877X.2018.1458977
Baloyi, T. (2020, September 9). SA will never triumph due to corruption – Treasury DG.
The South African. https://www.thesouthafrican.com/news/sa-will-
nevertriumph-due-to-corruption-treasury-dg/
Basu, S., Calabrese Barton, A., Clairmont, N., & Locke, D. (2009). Developing a framework
for critical science agency through case study in a conceptual physics context.
Cultural Studies of Science Education, 4, 345-371.
https://link.springer.com/content/pdf/10.1007/s11422-008-9135-8.pdf
Council on Higher Education (CHE), 2004, South African Higher Education in the
First Decade of Democracy, Pretoria. https://www.che.ac.za
Chisadza, C., Clance, M., Mthembu, T., Nicholls, N., & Yitbarek, E. (2021) Online and face
to face learning: Evidence from students’ performance during the Covid-19
pandemic. African Development Review, 33, S114-S125.
https://doi.org/10.1111/1467-8268.12520
Clark, S. N., Howley, I., Resnick, L., & Rose, C.P. (2016). Student agency to participate in
dialogic science discussions. Learning, Culture and Social Interaction, 10, 27-39.
https://doi.org/10.1016/j.lcsi.2016.01.002
Czerniewicz, L., Agherdien, N., Badenhorst, J., Belluigi, D., Chambers, T., Chili, M., De
Villiers, M., Felix, A., Gachago, D., Gokhale, C., Ivala, E., Kramm, N., Madiba, M.,
Mistri, G., Mgqwashu, E., Pallitt, N., Prinsloo, P., Solomon, K., Strydom, S.,
Swanepoel, M., Waghid, F., & Wissing, G. (2020). A wake-up call: Equity,
inequality and Covid-19 emergency remote teaching and learning. Postdigital
Science and Education, 2(3), 946-967. https://doi.org/10.1007/s42438-020-00187-4
Dison, L., Padayachee, K., De Klerk, D., Conradie, W., MacAlister, F., Moch, S., & Krull,
G. (2022). Reframing purpose and conceptions of success for a post-COVID-19
South African higher education. Scholarship of Teaching and Learning (SOTL), 6(1),
33-54. https://doi.org/10.36615/sotls.v6i1.222
Dube, B. (2020). Rural online learning in the context of Covid-19 in South Africa.
Evoking an inclusive educational approach. REMIE Multidisciplinary Journal of
Educational Research, 10(2), 1-24. https://doi.org/10.17583/remie.2020.5607
Du Preez, P., & Le Grange, L. (2020). The Covid-19 pandemic, online teaching /learning,
the digital divide, and epistemological access. In L. Ramrathan, N. Ndimande-
Hlongwa, N. Mkhize, & J. A. Smit (Eds.), Rethinking the Humanities Curriculum in
Times of Covid-19 (pp. 90-106). Alternation African Book Scholarship Series, 1.
https://www.researchgate.net/publication/347974339_The_COVID-
233
19_Pandemic_Online_Teaching_Learning_the_Digital_Divide_and_Epistemolog
ical_Access
Gumede, L., & Badriparsad, N. (2022). Online teaching and learning through the
students’ eyes – Uncertainty through the COVID-19 lockdown: A qualitative
case study in Gauteng province, South Africa. Radiology, 28, 193-198. http://doi:
10.1016/j.radi.2021.10.018.
Hedding, D. W., Greve, M., Breetzke, G. D., Bel, W., & Van Vuuren, B. J. (2020). Covid-19
and the academe in South Africa: Not business as usual. South African Journal of
Science, 116(7/8), 1-3. https://doi.org/10.17159/sajs.2020/8298
Hemson, C. (2019). Agency, resilience and innovation in overcoming educational failures.
Perspectives in Education, 36(2), 61-74.
https://doi.org/10.18820/2519593X/pie.v36i2.6
Holton, M. (2019). Debating the geographies of contemporary higher education students:
Diversity, resilience and resistance? Children’s Geographies, 17(1), 13-16.
https://doi.org/10.1080/14733285.2018.1536777
Laher, S., Bain, K., Bemath, N., De Andrade, V., & Hassem, T. (2021). Undergraduate
psychology student experiences during COVID-19: Challenges encountered and
lessons learnt. South African Journal of Psychology, 51(1), 215-228.
https://doi.org/10.1177/0081246321995095
Le Grange, L. (2020a). Could the Covid-19 pandemic accelerate the uberfication of the
university? South African Journal of Higher Education, 34(4), 1-10.
http://dx.doi.org/10.20853/34-4-4071
Le Grange, L. (2020b). Covid-19 pandemic and the prospects of education in South Africa.
Prospects, 51, 425-436. https://doi.org/10.1007/s11125-020-09514-w
Lindgren, R., & McDaniel, R. (2012). Transforming online learning through narrative and
student agency. Journal of Educational Technology & Society, 15(4), 344-355.
https://www.jstor.org/stable/jeductechsoci.15.4.344?seq=1&cid=pdfreference#
references_tab_contents
Luo, H., Yang, T., Xue, J., & Zuo, M. (2019). Impact of student agency on learning
performance and learning experience in a flipped classroom. British Journal of
Educational Technology, 50(2), 819-831. https://doi.org/10.1111/bjet.12604
Madiope, M. & Mendy, J. (2021). Analysis of current trends in distance education during
COVID-19: A South African higher education context. The Online Journal of
Distance Education and E-learning. 9(1), no page numbers.
https://eprints.lincoln.ac.uk/id/eprint/43951/
Mahendher, S., Doreswamy, H., Shenoy, V., & Uttam, R. (2021). Covid-19: Lockdown-
perception of faculty and students towards life, society, teaching and learning.
Journal of Contemporary Issues in Business and Government, 27(1), 2811-2833.
https://www.researchgate.net/publication/349882064_Title_COVID-
19_Lockdown-
Perception_of_Faculty_and_Students_towards_Life_Society_Teaching_and_Lear
ning
Mahlaba, S. C. (2020). Reasons why self-directed learning is important in South Africa
during the Covid-19 pandemic. South African Journal of Higher Education, 3(6),
120-136. https://doi.org/10.20853/34-6-4192
Makgahlela, M., Mothiba, T. M., Mokwena, J. P., & Mphekgwana, P. (2021). Measures to
enhance student learning and well-being during the COVID-19 pandemic:
Perspectives of students from a historically disadvantaged university. Education
Sciences, 11(5), 212-227. https://doi.org/10.3390/educsci11050212
Manderson, L., & Levine S. (2020). COVID-19, risk, fear, and fall-out. Medical
Anthropology, 39(5), 367-370. https://doi.org/10.1080/01459740.2020.1746301
Marongwe, N., & Garidzirai, R. (2021). Together but not together: Challenges of remote
learning for students amid the COVID-19 pandemic in rural South African
234
universities. Research in Social Sciences and Technology, 6(3), 213-226.

https://doi.org/10.46303/ressat.2021.39
Mbhiza, H. W. (2021). Shifting paradigms: Rethinking education during and post-
COVID-19 pandemic. Research in Social Sciences and Technology, 6(2), 279-289.
https://doi.org/10.46303/ressat.2021.31
Mbiydzenyuy, N. E., & Silungwe, D. (2020). Teaching and learning in resource-limited
settings in the face of the COVID-19 pandemic. Journal of Educational Technology &
Online Learning, 3(3), 211-223. https://doi.org/10.31681/jetol.732077
Mhlanga, D., & Moloi, T. (2020). Covid-19 and the digital transformation of education:
What are we learning on 4IR in South Africa? Education Sciences, 10(180), 1-11.
https://doi.org/10.3390/educsci10070180
Motala, S., & Menon, K. (2020). In search of the ‘new normal’: Reflections on teaching and
learning during Covid-19 in a South African university. Southern African Review of
Education, 26(1), 80–99. https://hdl.handle.net/10520/ejc-sare-v26-n1-a6
Mtshweni, B. V. (2022). COVID-19: Exposing un matched historical disparities in the South
African institutions of higher learning. South African Journal of Higher Education,
36(1), 234-250.
Odularu, O. I., Puzi, M. E., Ngqila, K. H., & Olatoye, T. A. (2022). Transformations in
higher educational institutions: A review of the post-COVID-19 era. Journal of
Culture and Values in Education, 5(1), 183-194.
O’Sullivan, M. (2006). Professional lives of Irish physical education teachers: Stories of
resilience, respect and resignation. Physical Education and Sport Pedagogy, 11(3),
265-284. https://doi.org/10.1080/17408980600986314
Shava, E. (2022). Reinforcing the role of ICT in enhancing teaching and learning post
COVID-19 in tertiary institutions in South Africa. Journal of Culture and Values in
Education, 5(1), 78-91.
Sokhulu, L. H. (2020). Students’ experiences of using digital technologies to address their
personal research needs during Covid-19. African Identities, 19(4), 436-452.
https://doi.org/10.1080/14725843.2020.1801384
Team, V., & Manderson, L. (2020). How COVID-19 reveals structures of
vulnerability. Medical Anthropology, 39(8), 671-674.
https://doi.org/10.1080/01459740.2020.1830281
Van Schalkwyk, F. (2021). Reflections on the public university sector and the covid-19
pandemic in South Africa. Studies in Higher Education, 46(1), 44-48.
https://doi.org/10.1080/03075079.2020.1859682
Van Wyk, M. M. (2021). Academic support under COVID-19 lockdown: What students
think of online support e-tools in an ODeL course. Interactive Technology and Smart
Education, 18(2), 137-157. https://doi.org/10.1108/ITSE-08-2020-0121
Vaugh, M. (2020). What is student agency and why is it needed now more than ever?
Theory Into Practice, 59(2), 109-118.
https://www.researchgate.net/publication/337853481_What_is_Student_Agen
cy_and_Why_is_it_Needed_Now_More_than_Ever
235

Vol. 21, No. 8, pp. 235-252, August 2022
Received Mar 31, 2022; Revised Jul 29, 2022; Accepted Aug 8, 2022
Enhancing Upper Secondary Learners’ Problem-

solving Abilities using Problem-based Learning
in Mathematics
Aline Dorimana
and Science (ACEITLMS), University of Rwanda College of Education (URCE),
Kayonza, Rwanda
Alphonse Uworwabayeho
School of Education, University of Rwanda College of Education (URCE),
Kayonza, Rwanda
Gabriel Nizeyimana
School of Education, University of Rwanda College of Education (URCE),
Kayonza, Rwanda
Abstract. Developing problem-solving abilities is a major objective of

learning mathematics at school. However, learners’ problem-solving
abilities are still critical. The main purpose of this study was to
investigate how the problem-based learning model could enhance
learners’ problem-solving abilities in mathematics. The study used
quasi-experimental research with one group pre-test-post-test design.
The population in this study consisted of fifty-four grade eleven learners
(aged between 16 to 19 years old) from one school in Kayonza District in
Rwanda. Data were collected using mathematical problem-solving tests
and interviews and were analysed using paired t-tests for dependent
sample means and descriptive analysis. The study results indicate that
problem-based learning potentially impacts learners’ problem-solving
abilities. It is shown from learners’ work in problem-solving that all
indicators of problem-solving abilities, namely understanding the
problem, planning ways to approach the problem, monitoring the
progress while tackling the problem and reviewing the solution process,
emerged as being fairly well improved. In addition, based on the
interview results from some learners and their teachers, they like the
PBL model because embedded tasks helped them to apply the
knowledge that can improve their reasoning, creativity and thinking
capability. The study recommends that schools encourage teachers to
adopt PBL for enhancing learners’ problem-solving abilities.
Additionally, researchers are urged to use findings from this study as a
reference for further research. Furthermore, researchers could conduct
similar research on a large scale using different methodologies.
©Authors
236
Keywords: mathematical problem-solving; problem-solving abilities;

problem-solving processes; problem-based learning; secondary school
learners; Rwandan schools
1. Introduction
The teaching of mathematics for problem-solving has been central to
mathematics education since the movement of curricula reform around the
1980s in the United States and has expanded worldwide (Rosli et al., 2013). The
ultimate goal is to engage learners in conducting mathematical investigations by
themselves and for them to be able to identify the application of mathematics
they have learned in a real-world situation (Stacey, 2005). The introduction of
Rwanda's competence-based curriculum (CBC) was meant to address this
educational reform (REB, 2015). However, learners’ problem-solving abilities
still require attention. Research of international assessments such as the
Programme for International Student Assessment (PISA) and Trends in
International Mathematics and Science Study (TIMSS) shows that the levels of
learners’ problem-solving abilities are still weak, even in developed countries
(Novita et al., 2012; Stacey, 2005). Therefore, the situation in developing
countries such as Rwanda can only be imagined. Factors such as mathematics
learning and the evaluation process contribute to low levels of problem-solving.
Thus, it is essential to update the learning and evaluation process to ensure
quality education is oriented to problem-solving.
Problem-solving refers to mathematical tasks that have the potential to provide

intellectual challenges for enhancing learners’ mathematical development
(Novita et al., 2012). These tasks are understood by learners but cannot be solved
by an already established procedure because they have a hurdle that is not
immediately realised. Learners must then think and reason through a situation.
It is good to note that what seems a problem to someone is not necessarily a
problem to others: it depends on the situation and the problem solver
(Schoenfeld, 1992). The solving of problems can foster the development of 21st-
century skills essential for the workplace (Gravemeijer et al., 2017; Szabo et al.,
2020) and stimulate the interest, motivation and curiosity of learners (Căprioară,
2015). In addition, having to solve an appropriate problem can affect the
enhancement of learners’ problem-solving abilities in identifying how they can
use mathematics skills to solve daily problems they face in life. It is essential to
expose learners to meaningful problems to practise sense making (Gravemeijer
et al., 2017; Polya, 1945; Voskoglou, 2011; Yu et al., 2014).
Problem-solving abilities are fundamentals of mathematics education in schools.

The enhancement of problem-solving abilities can equip learners to think
logically, analytically, critically, and creatively (Surya et al., 2017). Therefore,
learners’ problem-solving abilities will be enhanced if real-life problems are used
in the classroom. Yu et al. (2014) contended that problem-solving abilities are
better cultivated when concrete problems are taught in the classroom. Klegeris
and Hurren (2011) further pointed out that skills necessary to solve a problem
from available knowledge and information are developed while critically
analysing the contextual problems in a small collaborative group. Moreover,
237
Setiawan and Supiandi (2018) pointed out that successful problem-solving

abilities are emphasized when learners develop their own knowledge and apply
concepts. In this regard, this study used problem-based learning (PBL) as a
learning model because it offers many opportunities for learners to work on
solving concrete problems during learning. It has been used in an effort to help
learners improve their problem-solving abilities in mathematics (Prastiti et al.,
2020; Setiawan & Supiandi, 2018).
PBL is a learner-centred pedagogical model that offers a learning environment

with a problem-solving focus (Klegeris & Hurren, 2011). The general pattern of
instruction with PBL is to present concrete real-life problem (related to the
content and the broader curricular goals) to learners at the start of the lesson.
Learners are then given time to work collaboratively in small groups to identify
what they need to know to solve the problem while engaging in self-directed
learning (SDL) to look for answers and report back to the group and apply the
new knowledge to the problem. Finally, groups present their proposed solution
to the problem and conclude the activity by reflecting on what they have learned
as well as the effectiveness of the strategies employed. The entire process of
learning within PBL occurs around solving problems (Savery, 2019). Thus, PBL
is used in this study to provide learners with plenty of opportunities to solve
problems during learning.
Some studies documented the effectiveness of PBL model in different contexts

(Astriani et al., 2017; Merritt et al., 2017; Savery, 2019; Siregar, 2017; Valdez &
Bungihan, 2019). They have shown that PBL promotes conceptual
understanding and improves higher order thinking skills, confidence,
knowledge retention, motivation, academic achievement, and problem-solving.
Klegeris and Hurren (2011) point out that learners develop transferable skills
while analysing the problem, constructing their knowledge and applying
concepts in PBL (Setiawan & Supiandi, 2018). However, other studies have
shown that PBL can lead to poor performance when it is practised in a fact-based
learning environment (Aksela & Haatainen, 2019; Craig & Marshall, 2019). PBL
can also disengage learners unfamiliar with problems from participating in
group activities.
Parallel to this, some studies have employed different learning strategies and
focused on key factors to enhance grades five through to eight learners'
problem-solving abilities in mathematics (Bostic, 2011; Sigurdson et al., 1994;
Verschaffel et al., 1999; Yu et al., 2014). These studies implemented teaching
strategies that supplemented daily mathematics instruction with a problem-
solving focus for at least four weeks. The findings of these studies revealed
enhanced learners' problem-solving abilities in terms of understanding the
problem, implementing the plan for solution and verification of the answer, as
well as improved achievement and attitudes. Several other studies (Darma, 2018;
Klegeris & Hurren, 2011; Setiawan & Supiandi, 2018) exposed learners to
learning mathematics using PBL to improve problem-solving abilities. The
common outcome of their studies is a positive impact of PBL on learners'
problem-solving abilities. The latter show evidence of the suitability of PBL in
238
mathematics for small and large classroom settings. However, to the authors'
knowledge, few studies have applied PBL in upper secondary schools to
enhance learners' problem-solving abilities. Additionally, no study has used PBL
to strengthen problem-solving abilities in Rwanda. Thus, the current research
will document literature in this aspect and add knowledge to the existing body
of knowledge.
The reviewed literature revealed that the current learning and the evaluation
process of mathematics hinders the provision of quality mathematics education.
In addition, learning contributes nothing to learners’ problem-solving abilities
when tasks are meaningless. Consequently, learners will not be able to solve
problems successfully in mathematics or beyond and will lack the significance of
education in general, particularly mathematics. However, solving problems is
the primary means of doing mathematics at school (Stacey, 2005). Thus, this
study intends to investigate whether problem-based learning can enhance the
problem-solving abilities of grade eleven learners in Kayonza District in
Rwanda. The following research questions guided the study:
1. Will problem-based learning models improve learners’ problem-solving
abilities?
2. What do teachers and learners experience while learning mathematics with
PBL model?
Theoretical approach
According to Polya (1945), problem-solving is an attempt to find a way out of
the difficulty in achieving a goal which is not immediately achievable. The entire
process of problem-solving comprises identifying and understanding the
problem, planning ways to approach the problem, monitoring the progress
while tackling the problem and then reviewing the solution to the problem
(Novita et al., 2012). Polya (1945) suggested four stages for problem-solving: (1)
understanding the problem (what is known and needs to be done, and then
deciding what information is important and what seems unimportant); (2)
devising a plan (reflecting on ideas that can be brought to the problem); (3)
carrying out the plan (following through with the approach selected, carefully
taking each step along the way) and looking back (verifying whether the
solution makes sense or fits the given data). These processes are used in this
study as indicators of problem-solving abilities.
The PBL strategy in the classroom reflects the constructivism learning theory,
specifically Vygotsky's social constructivism theory (Vygotsky & Cole, 2018).
This theory states that knowledge is constructed and reconstructed socially
between teachers and learners where learners learn from one another in small
groups (Vygotsky, 1978). In this learning, the teacher acts as a facilitator or a
guide and catalyst. Learners actively engage in the learning process while
making interpretations and constructing new knowledge through teamwork and
collaboration (Cohen et al., 2003). Social interaction support learners to learn
concepts more quickly and efficiently than when working alone. This interaction
creates a better opportunity for learners to build their problem-solving abilities
(Vygotsky & Cole, 2018).
239
In the PBL, learning opportunities are offered to learners at the beginning of the
lesson because learners are not empty-minded (Razieh, 2016). Learning is
maximized in the so-called zone of proximal development (ZPD). According to
Vygotsky (1978), the ZPD refers to the distance between the actual and the
potential development of an individual. This is why the ZPD could not be
ignored in this study. Teachers should monitor individual learners to promote
discussions that support interactive learning or conceptual understanding. PBL
emphasizes collaborating and sharing of information, which positively impact
learners’ problem-solving abilities.
2. Methodology
This study is a mixed-method research of quasi-experiment with one group
pretest-posttest design. Qualitative data were gathered from interviews while
quantitative data were obtained from mathematical problem-solving tests. The
choice of mixed methods was guided by the belief of pragmatic philosophers
that mono-paradigm orientation research is not enough to interpret and
understand human behaviours (Alise & Teddlie, 2010). More practical and
pluralist approaches should allow a combination of research methods regarded
as most appropriate to address the research questions (Kivunja & Kuyini, 2017).
The epistemological and ontological views of the research problem support
mixed research methods. Thus, for this study, mixed methods were considered
appropriate.
Using non-probability purposive sampling techniques, eighty-two (82) grade-

eleven learners (45 females and 37 males) aged between 16 to 19 years old from
one school in Kayonza District in Rwanda were selected. The school
accommodates both core and subsidiary mathematics in the combinations and is
situated in the district in which the University of Rwanda – College of Education
(UR-CE) operates. These learners have one year left to complete their secondary
education and were divided into two combinations; forty-seven in Mathematics-
Economics-Geography (MEG) and thirty-five in Physics-Chemistry-Biology
(PCB). These learners received a PBL intervention for 14 weeks. Therefore, their
teachers could not be excluded from the study. They were given a test before
and after the intervention. However, the study considered only fifty-four
learners (28 females and 26 males) who completed both pre-test and post-test.
Tests comprised seven questions and were selected from a bunch of questions
prepared for a large doctoral project by the first author based on their fit for
enhancing problem-solving abilities as this study intended. Researchers
formulated questions based on the algebra content of grade eleven learners (see
students’ book on REB website) on units of sequences, logarithmic and
exponential equations, and solving equations and puzzle problems. To avoid
bias, questions from the pre-test were slightly modified or replaced in the post-
test (see Table 1).
240
Table 1: Pre-and post-test mathematics items arrangement

Units Question Question Comments
number for number for
pre-test post-test
Unit 2: Sequences 1,2 1,5 The two questions were similar
except that they were arranged in
different positions.
Unit 3: Logarithm 7 7 These two questions were the same.
and exponential
equations
Unit 4: Solving 6 2,3 Question 3 in the post-test covered
equations the unit of solving equations using
concepts of matrices. Q6 and Q2
were different.
Puzzle problems 3,4,5 4,6 Puzzle problems for the pre-test
were completely dissimilar to those
of the post-test.
Two experts, both PhD holders in mathematics education, validated the tests
before their administration. They rated the question items as very easy, easy,
and difficult. In addition, they provided comments on whether they fit research
objectives, and these were used to revise the questions. Additionally, a reliability
coefficient was calculated using a test-retest method. It was found to be .72 from
a sample of 30 learners who were not part of the study in the actual data
collection.
This research received an ethical clearance letter from UR-CE through the
research and innovation unit. This letter was used to seek permission to collect
data from the district education office where the school is located. Before
commencing the data collection process, participants were explained the
purpose of the study to encourage their participation in the research. They were
given time to ask questions and fill out consent forms to ensure voluntary
participation. After that, a pre-test was given to monitor learners’ level of
problem-solving abilities prior to the intervention.
Five mathematics teachers (1 female and 4 males) at the school of intervention

attended a two-day workshop on PBL. The workshop focused on PBL concepts,
patterns of instruction within PBL and the formulation of PBL activities. At the
end of the workshop, teachers prepared and performed two microteachings
using PBL and reflected on their teaching in order to make better use of PBL.
Two out of five teachers implemented PBL for 14 weeks and were assisted in
refining the intervention during the intervention period. Teachers encouraged
learners to identify and understand the problem presented to them, to plan ways
to approach it, to monitor the progress while tackling the problem, and to
review the solution. Learners learnt by doing problems before receiving input
from the teachers (see introduction section for more detail).
241
Box 1. Example of activities done as a teaching intervention

Activity:
A father wants to build a tower of ten rows of blocks starting from the bottom.
Each row will have two blocks fewer than the previous. He would like to know
the total number of blocks he will need to purchase before starting with the
bottom row comprising 1025 blocks. What is the total number of blocks he will
need?
Steps followed:
The problem was presented to learners at the start of the lesson (the teacher
wrote the problem on the chalkboard) (5 minutes). They were then given time to
read the problem and brainstorm ideas critically, and difficult words were
explained (10 minutes). The teacher assigned learners in small groups to analyse
the problem (2 minutes). During group work, learners did research to identify
what they needed to know to solve the problem (using textbooks), propose
solutions (20 minutes), and then report back to the group to apply the new
knowledge to the problem. The entire process of problem-solving (1-understand,
2-plan, 3- implement, and 4-check the solution) was written on the chalkboard in
one corner so that even those who might have forgotten the steps could verify
them. Learners were given time to present their findings to the whole class,
where reflections were given by classmates (15 minutes) (group presentation
was given to those who solved the problem in different ways; not all the groups
in the class). A decision was taken for the whole class, and the teacher linked the
problem solution with the lesson's learning objective. The remaining time was
given to learners to foster the concepts by working on various textbook exercises
which the teacher indicated in their book.
Note:
*Many of the problems used during the intervention were designed based on the
content, and textbooks were used as well. Questions were used for sequences
(the lesson was designed for 80 minutes).
*The lesson's learning objective was to be able to apply a formula to determine
the nth term of arithmetic sequences and find the sum of n in terms of arithmetic
progressions.
*Learners had no idea about the formula for finding the sum of nth arithmetic
progressions or even how to find any term of the sequence; it was a new lesson.
The formulas were clarified (the general formula was given in case learners did
not find that formula).
Every Friday, teachers in the intervention and researchers met to evaluate the
intervention's progress, discuss difficulties, and prepare the next lesson by
focusing mostly on the PBL activity. To recap, researchers made suggestions and
contributed to lesson preparation at the end of the meeting. These meetings were
organized and conducted for mentorship purposes. At the end of each week,
researchers conducted semi-structured interviews with teachers and FGDs with
selected learners to hear their appreciation of the intervention. Field notes were
also taken and were used to refine the learning material and improve teacher
242
questioning strategies. After intervention, a post-test was administered to the

same group of learners to monitor their knowledge and skills gained in
problem-solving and any change in their problem-solving abilities. Researchers
supported learners to understand the language used in both pre-test and post-
test. Each test was conducted for approximately 60 minutes.
Before data analysis, every question on the answer sheet of learners was checked
and marked. Each question was allocated ten marks following indicators of
problem-solving abilities (see details in Table 2), as Szetala and Nicol (1992)
suggested. The ability to understand the problem scored 0 to 2, the ability to
plan the solution strategy scored 0 to 2, the ability to implement the plan was
scored from 0 to 4, and the ability to check whether the solution makes sense
scored 0 to 2.
Table 2: Details of the indicators and scoring of problem-solving abilities

Indicators Details Scores
1 Understanding If there is no identification of known, asked 0
the problem elements of the data needed and underlining of
keywords in the question
If there is the identification of known, asked 1
elements, or underlining of keywords but are
incomplete
If there is the identification of known, asked 2
elements, or underlining of keywords and are
complete and adequate
2 Planning the There is no plan at all, or it is unclear, or it is erased 0
solution strategy There is a plan with errors 1
There is a correct plan 2
3 Implementing There is no answer, or it is erased 0
the plan Some solutions are following the plan but are wrong 1
Some solutions are following the plan with errors 2
and are incomplete
There are correct solutions that follow the plan but 3
are incomplete
There are correct solutions that follow the plan 4
4 Interpreting or There is no conclusion or interpretation of the 0
checking the solution obtained
solution obtained There is proof of the solution, or there is 1
interpretation with errors
There is correct proof of the solution obtained and 2
interpretations
Microsoft Excel was used as a tool to analyse data. Data from interviews were
analysed using descriptive analysis (presented in text and quotes) while data
from tests were analysed using charts and inferential statistics of the paired t-
test. The test was computed to determine whether learning mathematics with
PBL has an impact on learners’ problem-solving abilities. The authors wanted to
find out whether there is a significant difference between means of learners'
scores in the pre-test and post-test at a .05 level of significance. Finally,
normalized learning gains were computed from pre-test and post-test results
across each indicator of problem-solving abilities.
243
3. Findings
The results of the study are presented based on the order of the research
questions examined.
Question one
To establish how learning mathematics with PBL improved learners’ problem-
solving abilities, data from the pre-test and post-test results for each question
were analysed according to indicators of problem-solving abilities described in
Table 2. Learners’ problem-solving performance on each of seven questions
according to problem-solving indicators is displayed in Figure 1. Results
indicate an average improvement of learners' problem-solving abilities from 37%
to 61%.
100%
Percentage of learners' performance
90%
80%
70%
60%
50%
40%
30%
%
20%
10%
0%
Plan
Plan
Plan
Plan
Plan
Plan
Plan
Average
Implement
Implement
Implement
Implement
Implement
Implement
Implement
Understand
Verify
Understand
Verify
Understand
Verify
Understand
Verify
Understand
Verify
Understand
Verify
Understand
Verify
Qn 1 Qn 2 Qn 3 Qn 4 Qn 5 Qn 6 Qn 7
Pre-test Post-test
Figure 1: Percentage of learners’ problem-solving performance on each question

according to indicators of problem-solving abilities (Qn: Question)
The performance percentage on each question according to problem-solving

indicators reveals that learners performed better in the post-test than in the pre-
test. This means that there is an improvement in learners' problem-solving
abilities. Also, a paired t-test for the dependent sample mean was computed (see
Table 3) to explore whether there is a significant difference between the means of
learners’ scores in the pre-test and post-test. The results revealed 𝑡 𝑠𝑡𝑎𝑡 =
−12.08 < 𝑡 𝑐𝑟𝑖𝑡𝑖𝑐𝑎𝑙 𝑡𝑤𝑜 𝑡𝑎𝑖𝑙 = 2.01, which means that PBL instruction affects
learners' problem-solving abilities positively, and the impact is significant with
𝑝 < .05.
Table 3: t-Test: Paired two samples for means

Pearson P(T<=t) t Critical
Mean Variance N Correlation df t Stat two-tail two-tail
Pre-test 22.11 37.50 54 0.27 53 -12.08 0.00 2.01
Post-test 37.30 76.85 54
244
Based on the results displayed in Figure 1, there is little impact on learners’

abilities for question one after the intervention. The figure shows that the first
three indicators (understand, plan and implement) on question one are higher in
the pre-test compared to those of the post-test. This could be because the
question was too short, with few words embedded. Also, learners could have
been accustomed to the type of questions which seem closed with few terms that
generally provide an immediate procedure to apply for solving the problem. An
example of a learner response in Figure 2 shows the application of looking for a
pattern strategy to find a solution in the pre-test. However, in the post-test, the
learner could apply more than one strategy for an answer, although s/he was
not successful.
Figure 2: A learner response on question one in the pre-test and post-test
Based on responses shown in Figure 2, a learner made better progress in the

post-test than in the pre-test of problem-solving ability. The first attempt shows
that the learner knew the formula and relied on it to find the solution. However,
for the second attempt, the learner answered the question freely, showed all the
workings, and performed better than on way to the solution.
The effect of PBL on learners’ problem-solving abilities was evaluated according

to gender by minimum, maximum, quartiles, median, mean, range, interquartile
range, p-value, and learning gains (see Table 4).
Table 4: Effect of problem-based learning on learners’ mathematical problem-solving

abilities
Pre-test Post-test Pre-test Post-test Pre-test Post-test
[All, [All, [Female, [Female, [Male, [Male,
N=54] N=54] N=26] N=24] N=28] N=28]
Minimum 14% 30% 19% 30% 14% 31%
245
Quartile 1 24% 43% 25% 43% 25% 46%

Median 31% 51% 30% 49% 33% 59%
Quartile 3 37% 61% 38% 59% 39% 68%
Maximum 49% 77% 49% 74% 49% 77%
Mean 31% 53% 32% 51% 32% 56%
Range 34% 47% 30% 44% 35% 46%
Intra
Quartile
Range 13% 19% 13% 16% 15% 22%
p-value <.001 <.001 <.001
Learning
gains 32% 28% 36%
The learning gain (N-gain) for males is 36%, while that of females is 28%. The
results indicate that the overall learning N-gain is 32%; however, male learners
outperformed female' learners in both pre-test and post-test. Table 4 presents the
detailed results.
Furthermore, the average N-gains across each indicator of problem-solving

abilities were computed. The results indicate an overall N-gain of 38%, which
shows to what extent changes occurred in learners' problem-solving abilities.
Results show a higher N-gain of 43% of the "verify” indicator. This means that
after PBL, learners learn to make sense of the solution to the problem. However,
it was not easy to identify a high N-gain for the "implement” stage because
learners mostly combine the “plan” and “implement” phase when solving the
problem. Thus, effort should be made to indicate the implementing phase
during the problem-solving process. Figure 3 shows the graph of indicators of
problem-solving abilities against percentages (values).
80%
70%
70% 66%
61%
60% 55% 56%
52%
50% 42% 43%
41% 38%
40% 37%
33% 34%
28%
30% 22%
20%
10%
0%
Understand Plan Implement Verify Average
Pre-test Post-test N-gain
Figure 3: Average N-gain across all questions for each indicator of problem-solving
abilities
Moreover, the study compared indicators of learners’ problem-solving abilities

based on the question content, sequence, and series, solving questions using
numerical methods, logarithm and exponential equations, and puzzle problems,
as shown in Figure 3. The results indicates an improvement of learners’
246
problem-solving abilities in solving equations, puzzle problems, logarithms and

exponential equations and a slight decrease of learners’ problem-solving abilities
in sequence and series.
80%
70%
60%
50%
40%
30%
20%
10%
0%
a b c d a b c d a b c d a b c d
Sequence and series Solvings equations Puzzle problems Logarithms and
exponential equations
Pretest Posttest
Figure 4: Learners’ problems-solving abilities according to the content
Question two
Results from the interview data indicate some PBL benefits that support the
enhancement of learners’ problem-solving abilities. The first finding relates to
the PBL environment which provides a learning atmosphere that can help
learners to develop their understanding. Both teachers and learners indicated
that they were able to value the importance of understanding a problem before
they can solve it. The PBL activity requires learners to reason through the
situation. An example of a transcript from Anna is given below:
“Usually, the teacher gives us notes before learning, and we get tired
before learning new knowledge, but with this learning style, we learn
more meaningfully with understanding."
Anna continues:
“Normally when you are given a challenging activity before learning, it
first challenges you before getting the right answer…it requires us to
use our brain as compared to how we usually do… normally we
memorize the formula so that if they change the question a little bit, you
cannot think deeply, but now we are required to think first so that once
you face the similar challenge, you are not afraid to solve it”.
This finding indicates that PBL learning assisted learners in developing

mathematical understanding. PBL learning offered a space for the learners to
learn easily as they had enough time to reason and think without getting tired or
bored. The second finding relates to PBL activity which can support learners in
planning ways of approaching the challenge since the activity did not require
them to use any readymade procedure. Many learners were of the view that the
PBL strategy made the learning of mathematics more concrete and enjoyable
than being abstract, where formula and rote learning are given less attention.
Frank described this as follows:
247
“Activities are severe and good because they reflect the real-life situation
that we are familiar with and are not forcing us to use the formula from
the notebook, but the brain and different strategies”.
Many learners indicated that they had received different guidance from their
teachers, which again encouraged them to think critically. This means that both
teachers and learners could pay attention to processes; teachers could not expect
short or final answers from the learners. Rather, they could shift their teaching to
focus on problem-solving. The usual teaching process was as follows:
“Teachers often give us exercises and homework sometimes and leave
them uncorrected… consequently, engaged learners cannot know if what
they have done is correct or know the wrong step; those unengaged
learners get nothing”.
The results indicated that being given challenging activities helped learners to be
engaged and reduced the teacher’s talking in the classroom. For instance,
Hawah indicated:
“I wish to learn in this way…. to learn first before taking notes and
therefore we could take notes of what we understand well”.
Anna was referring to being given an engaging activity to think about and
identify the learning gap. The teacher should intervene to fill the knowledge
gap. Fausto complimented Hawah, saying the following:
“Actually, we saw more content of what we are learning here in senior
five (grade eleven); what we need most is little guidance from the teacher
depending on what he needs us to achieve.”
Fausto continued: “Normally, when one learner takes the role of the
teacher, or when we work in small groups, we are not afraid to ask
questions than we can do to the teacher.”
The third finding relates to difficulties of learning and implementing PBL.

Generally, participants reported positive experiences that are similar to previous
studies such as the improvement of learners’ attitudes, longer retention of
knowledge, improved self-directed learning practices, enhanced problem-
solving abilities and critical thinking skills. They also developed a teamwork
spirit as well as their presentation and communication skills. However,
participants reported negative experiences of PBL, namely that it is time
consuming, requires time, and needs infrastructure and equipment, as well as
the Internet. Therefore, the PBL learning process could impact on teachers’
workload. Below is a transcript of a teacher interview. Kalisa said the following:
“You see, you need to understand the content very well to design
appropriate task… sometimes you need to search on the internet, and I
have three classes to teach…. Besides, I cannot teach very fast the
content I have to cover in 9 weeks of this term".
Kalisa added that:

"The other is that when you give learners much time to talk in the
classroom, they shout out. It becomes difficult to keep one conversation
in the room.” He further said that: “you find yourself spending much
248
time managing the classroom instead of managing the 40 minutes of the

lesson period.”
Kalisa was complemented by his colleague Kamana, who said:

“PBL methodology is good to the extent I saw learners going beyond the
planned learning expectations…but they may not achieve the learning
objective, then you delay covering the content as planned on the scheme
of work.”
4. Discussion
The results of the mathematical problem-solving test revealed that scores were
higher in the post-test than pre-test, with average performance increasing from
37% to 61%, which confirms the value of the t-test. This result demonstrates that
PBL instruction was effective in assisting learners in improving their problem-
solving abilities. Across all questions, learners showed a significant increase in
how they demonstrate understanding of the problem, plan ways to approach the
problem, monitor their progress as they tackle the problem and review the
solution to check whether all conditions of the problem had been met. These
findings are in line with those of Padmavathy and Mareesh's (2013) study which
found positive results in learners' understanding and abilities to use concepts in
real life after learning with PBL. Sigurdson et al. (1994), Verschaffel et al. (1999)
and Yu et al. (2014) pointed out that a problem-solving focus should supplement
daily mathematics teaching to improve learners’ achievement, attitudes and
problem-solving abilities.
In addition, the improvement of problem-solving abilities using PBL models

follows the social constructivism theory of learning (Purwoko et al., 2019). As
Vygotsky (1978) stated, learning is central to students, and new knowledge is
constructed owing to social interactions. Students engaged in and sharing the
activity are facilitated by their teacher through appropriate tasks and monitoring
productive discussions. Thus, students' experience when solving problems in
small groups during learning improves their thinking and mental function.
The PBL model effectively cultivates students’ abilities to understand and

analyse the problem then select and develop solutions. This is why other studies
(e.g., Mukuka et al., 2020; Ukobizaba et al., 2021) have emphasized the need for
teachers to turn their attention to instructions oriented to problem solving and
assessment in order to address learners’ learning needs. The study revealed
better results of learning, namely N-gain of 32% of the participants. This finding
shows that learners benefitted from the PBL method, especially the girls. Boys
(N-gain =28%) outperformed girls (N-gain=36%) for both pre-test and post-tests.
This finding is in line with the study conducted by Siregar (2017) who found a
higher increase in the problem-solving ability of male learners in both control
and experimental group. This could be because mathematics is perceived as a
male subject and difficult (Osman & Kriek, 2021). However, both male and
female need problem-solving experience in mathematics that encourages them
to think differently since the labour market is open equally to both.
249
Changes in learners' problem-solving abilities were high (N-gain=38%). Surya et

al. (2017) point out that high N-gain boosts learners' confidence, interest and
motivation. A higher N-gain was on the "verify” indicator with 43% compared
to other indicators. This means that after PBL, learners are able to make sense of
the solution to the problem. It helped them to value the importance of the
process in problem-solving. Learners showed their understanding rather than
directly solving the problem; they showed their plan on the answer sheet rather
than using another sheet as well as showing whether all the conditions to the
problem had been satisfied. This indicates how learning mathematics with a
problem-solving focus should take place, with minimal attention and focus
being given to short and final answers (Novita et al., 2012). The situation then
calls for teachers to pay attention to problem-solving processes to help learners
to be creative. NCTM (2010) points out that exposing learners to meaningful
problems fosters understanding and mathematical development.
To develop the learners' problem-solving abilities in challenging tasks, guidance

in feedback on the worked examples and scaffolding were of primary
importance (Setiana et al., 2021). As in the zone of proximal development
(Vygotsky & Cole, 2018), this helped learners be engaged, reduced teacher talk
in the classroom, and enhanced learning and cognitive development. The study
conducted by Simamora and Saragih (2019) found that learning using culture
context materials improved learners' problem-solving ability and self-efficacy. In
addition, according to Szetala and Nicol (1992), paying attention to the process
and explanations of learners’ solutions rather than focussing on the answers
learners give will be more beneficial to learners’ communicative thinking. The
PBL model accommodates these facilitations. During the problem-solving
activity, the teacher visited learners in their small groups to guide the problem-
solving process as an example of facilitation. In addition, learners presented
findings to communicate their thinking on every step taken to other learners.
Through problem-solving, learners can improve their reasoning. An

improvement in N-gain influences learners’ abilities, confidence, interest and
motivation to learn mathematics (Surya et al., 2017). In other words, meaningful
mathematical tasks can equip learners to think logically, analytically,
systematically, critically and creatively. Results from interviews indicate
participants have positively experienced PBL strategies that are in line with
previous studies (Osman & Kriek, 2021; Setiawan & Supiandi, 2018). The PBL
strategy made the learning of mathematics more learner-centred and active
while providing learners with the opportunity to develop interpersonal
relationship skills, critical thinking and reasoning skills, improved attitudes, and
enhanced self-directed learning. Also, PBL provided learners with the
opportunity to associate academic life with their everyday experience, which
substantively increased their awareness of the usefulness of mathematics.
5. Conclusion
Based on the findings and discussions, the study concludes that problem-based
learning impacts learners’ problem-solving abilities positively in terms of
understanding the problem, planning ways to approach the problem,
250
monitoring the progress while tackling the problem and reviewing the solution
process in order to check whether all conditions of the problem have been
satisfied. In addition, the PBL learning activities provided learners with
opportunities to apply knowledge. Furthermore, although PBL presents a
negative experience, it nevertheless also has many positive experiences. Thus,
teachers should apply the PBL model to prepare learners better for the future
and develop their problem-solving abilities. The focus is on how learners
approach the problem at the initial stage. The study invites further research to
explore the effectiveness of PBL on learners’ problem-solving abilities using a
different methodology on large scale.
Limitations of the study

The study was limited to two classes of grade eleven learners from one school in
Kayonza District in Rwanda. It has used one group pre-test and post-test design.
Therefore, there could be other factors that could have influenced the increase in
learners’ problem-solving abilities. This could have been avoided if an
experimental and control group had been used.
Acknowledgements
We acknowledge the African Centre of Excellence for Innovative Teaching and
Learning Mathematics and Science (ACEITLMS) for funding this research and
Juliette Itangishatse for her assistance in data collection. We thank all research
participants from Kayonza District for making this research possible.
6. References
Aksela, M., & Haatainen, O. (2019). Project-based learning (PBL) in practice : Active
teachers’ views of its advantages and challenges. Proceedings of the 5th International
STEM in Education Conference, 9–16.
Alise, M. A., & Teddlie, C. (2010). A continuation of the paradigm wars? Prevalence rates
of methodological approaches across the social/behavioral sciences. Journal of
Mixed Methods Research, 4(2), 103–126. https://doi.org/10.1177/1558689809360805
Astriani, N., Surya, E., & Syahputra, E. (2017). The effect of problem-based learning to
students’ mathematical problem-solving ability. International Journal Of Advance
Research and Innovative Ideas in Education, 3(2), 3441–3446.
Bostic, J. D. (2011). The effects of teaching mathematics through problem-solving contexts on
sixth-grade students' problem-solving performance and representation use [Doctoral
dissertation]. University of Florida.
Căprioară, D. (2015). Problem solving - Purpose and means of learning mathematics in
school. Procedia - Social and Behavioral Sciences, 191, 1859–1864.
Cohen, D. K., Raudenbush, S. W., & Ball, D. L. (2003). Resources, instruction, and
research. Educational Evaluation and Policy Analysis, 25(2), 119–142.
https://doi.org/10.3102/01623737025002119
Craig, T. T., & Marshall, J. (2019). Effect of project-based learning on high school
Students ’ state-mandated, standardized math and science exam performance.
Journal of Research in Science Teaching, 56(10), 1461–1488.
https://doi.org/10.1002/tea.21582
Darma. (2018). Improving mathematical problem-solving ability through problem-based
learning and authentic assessment for the students of Bali State Polytechnic.
Proceedings of the 2nd International Joint Conference on Science and Technology (IJCST),
1–11.
251
Desi, Lesmini, B., & Hidayat, I. (2019). Enhancing student problem solving skills through
worksheets-assisted problem-based learning. Journal of Physics: Conference Series,
1166(1). https://doi.org/10.1088/1742-6596/1166/1/012005
Gravemeijer, K., Stephan, M., Julie, C., Lin, F. L., & Ohtani, M. (2017). What mathematics
education may prepare students for the society of the future? International Journal of
Science and Mathematics Education, 15(1), 105–123. https://doi.org/10.1007/s10763-
017-9814-6
Kivunja, C., & Kuyini, A. B. (2017). Understanding and applying research paradigms in
educational contexts. International Journal of Higher Education, 6(5), 26.
https://doi.org/10.5430/ijhe.v6n5p26
Klegeris, A., & Hurren, H. (2011). Impact of problem-based learning in a large classroom
setting : Student perception and problem-solving skills. Advances in Physiology
Education, 35, 408–415. https://doi.org/10.1152/advan.00046.2011.
Merritt, J., Lee, M. Y., Rillero, P., & Kinach, B. M. (2017). Problem-based learning in K – 8
mathematics and science education : A literature review. The Interdisciplinary
Journal of Problem-Based Learning, 11(2), 5–17.
Mukuka, A., Balimuttajjo, S., & Mutarutinya, V. (2020). Applying the SOLO taxonomy in
assessing and fostering students' mathematical problem-solving abilities. Problems
of Education in the 21st Century, 77(6), 13.
Novita, R., Zulkardi, & Hartono, Y. (2012). Exploring primary students' problem-solving
ability by doing tasks like PISA’s question. IndMS. J.M.E, 3(2), 133–150.
Osman, A., & Kriek, J. (2021). Science teachers’ experiences when implementing
problem-based learning in rural schools. African Journal of Research in Mathematics,
Science and Technology Education, 0(0), 1–12.
https://doi.org/10.1080/18117295.2021.1983307
Padmavathy, R. D., & Mareesh, K. (2013). Effectiveness of problem-based learning in
mathematics. International Multidisciplinary E-Journal, II(I), 45–51.
www.shreeprakashan.com
Polya, G. (1945). How to solve it mathematical method (2nd ed.). Doubleday Company.
Prastiti, T. D., Dafik, & Azkarahman, A. R. (2020). The application of problem-based
learning in mathematics education on several South East Asia high schools.
Pancaran, 9(4), 75–89. https://doi.org/10.25037/pancaran.v9i4.317
Purwoko, R. Y., Nugraheni, P., & Instanti, D. (2019). Implementation of pedagogical
content knowledge model in mathematics learning for high school. Proceedings of
the 1st UPY International Conference on Applied Science and Education 2018, 1–6.
https://doi.org/10.1088/1742-6596/1254/1/012079
Razieh, T. N. (2016). Bandura’ s social learning theory & social cognitive learning theory.
Journal of Personality and Social Psychology, 1, 589.
Republic of Rwanda. Rwanda Education Board (REB). (2015). Competency-based
curriculum -- summary of curriculum framework pre-primary to upper secondary.
https://mudarwan.files.wordpress.com/2015/08/curriculum_framework_final_pr
inted-compressed.pdf
Rosli, R., Goldsby, D., & Capraro, M. M. (2013). Assessing students ’ mathematical
problem-solving and problem-posing skills. Asian Social Science, 9(16), 54–60.
Savery, J. R. (2019). Comparative pedagogical models of problem-based learning. The
Wiley Handbook of Problem-Based Learning, 81–104.
https://doi.org/10.1002/9781119173243.ch4
Schoenfeld, A. H. (1992). Learning to think mathematically: Problem solving,
metacognition and sense making in mathematics. In D.A. Grouws (Ed.), NCTM
Handbook of research on mathematics teaching and learning (pp. 334–370). Macmillan.
Setiana, D. S., Purwoko, Y. R., & Sugiman, S. (2021). The application of mathematics
learning model to stimulate mathematical critical thinking skills of senior high
252
school students. European Journal of Educational Research, 10(1), 509–523.

https://doi.org/10.12973/eu-jer.10.1.509
Setiawan, B., & Supiandi, M. I. (2018). The contribution of metacognitive skills and
reasoning skills on problem-solving ability based on problem-based learning (PBL )
model. Anatolian Journal of Education, 3(2), 75–86.
Sigurdson, S. E., Olson, A. T., & Mason, R. (1994). Problem solving and mathematics
learning. Journal of Mathematical Behavior, 13, 361–388.
Simamora, R. E., & Saragih, S. (2019). Improving students’ mathematical problem-
solving ability and self-efficacy through guided discovery learning in local culture
context. International Electronic Journal of Mathematics Education, 14(1), 61–72.
https://doi.org/https://doi.org/10.12973/iejme/3966
Siregar, N. (2017). Problem-solving ability of students mathematics in problem-based
learning. Journal of Educational Science and Technology, 3(3), 185–189.
Stacey, K. (2005). The place of problem solving in contemporary mathematics curriculum
documents. Journal of Mathematical Behavior, 24(3–4), 341–350.
Surya, E., Putri, F. A., & Mukhtar. (2017). Improving mathematical problem-solving
ability and self-confidence of high school students through contextual learning
model. Journal on Mathematics Education, 8(1), 85–94.
https://doi.org/10.22342/jme.8.1.3324.85-94
Szabo, Z. K., Körtesi, P., Guncaga, J., Szabo, D., & Neag, R. (2020). Examples of problem-
solving strategies in mathematics education supporting the sustainability of 21st-
century skills. Sustainability Article, 12(10113), 1–28.
Szetala, W., & Nicol, C. (1992). Evaluating problem-solving in mathematics. Educational
Leadership, 49(8), 42–45.
Ukobizaba, F., Nizeyimana, G., & Mukuka, A. (2021). Assessment strategies for
enhancing students’ mathematical problem-solving skills : A review of literature.
Eurasia Journal of Mathematics, Science and Technology Education, 17(3), 2–10.
Valdez, J. E., & Bungihan, M. E. (2019). Problem-based learning approach enhances the
problem solving skills in chemistry of high school students. Journal of Technology
and Science Education, 9(3), 282–294. https://doi.org/10.3926/JOTSE.631
Verschaffel, L., De Corte, E., Lasure, S., Van Vaerenbergh, G., Bogaerts, H., & Ratinckx,
E. (1999). Learning to solve mathematical application problems: A design
experiment with fifth graders. Mathematical Thinking and Learning, 1(3), 195–229.
https://doi.org/10.1207/s15327833mtl0103_2
Voskoglou, M. G. (2011). Problem-solving from Polya to nowadays : A review and future
perspectives. In A.R. Baswell (Ed). Advances in mathematics research, 12(1), 1-18.
Nova Publishers.
Vygotsky, L., & Cole, M. (2018). Lev Vygotsky: Learning and social constructivism. Learning
theories for early years practice, 58.
Vygotsky, L. S. (1978). Minds in society: The development of higher pschological processes.
Harvard University Press. https://doi.org/10.2307/j.ctvjf9vz4
Yu, K.-C., Fan, S.-C., & Lin, K. (2014, November). Enhancing students ’ problem-solving
skills through context-based learning. International Journal of Science and Mathematics
Education. https://doi.org/10.1007/s10763-014-9567-4
253

Vol. 21, No. 8, pp. 253-270, August 2022
The Development of Mobile Applications for

Language Learning: A Systematic Review of
Theoretical Frameworks
Kee-Man Chuah*
Faculty of Language and Communication, Universiti Malaysia Sarawak,
Sarawak, Malaysia
School of Educational Studies, Universiti Sains Malaysia, Pulau Pinang,
Malaysia
Muhammad Kamarul Kabilan

School of Educational Studies, Universiti Sains Malaysia, Pulau Pinang,
Malaysia
Abstract. This systematic review was conducted to examine theories

that were adopted or considered in the development of mobile
applications for language learning. It aims to provide a holistic overview
of major theoretical principles that underpin each developmental study
to identify trends and gaps in the synthesised literature. The studies
were collected from the Scopus and Web of Science databases as they
were the main sources of reputable journals. Primary searches between
2011 and 2020 revealed approximately 158 studies related to the topic
under investigation. After further filtering based on the inclusion criteria
and removal of duplicates, 39 studies matched the research criteria and
were used for further analysis. The analysis revealed that researchers
tend to choose to solve problems pertaining to vocabulary learning and
learners’ motivation through the development of mobile applications for
language learning. They preferred to use constructivist-based theories
such as situated learning and collaborative learning in guiding their
development though behaviourist principles are also dominant.
However, very few studies used theories related to language acquisition
and learning in the design and development process. Hence, this gap
should be given priority in future developmental research within the
same scope as the generic learning theories may not be accurately
addressing the language learning problems.
Keywords: mobile applications; theories; language learning; mobile

learning; mobile development
*
Corresponding author: Kee-Man Chuah, kmchuah@unimas.my
©Authors
254
1. Introduction
The advancement of mobile technologies has led the shift towards ubiquitous
learning, where learning can occur anytime, anywhere. Over the past decades,
mobile devices are no longer confined to communication purposes but are
increasingly used as an essential learning tool across all levels of education
(Crompton & Burke, 2018; Heil et al., 2016; Rajendran & Md Yunus, 2021; Shin et
al., 2011). The same development can be observed in language teaching, mainly
in the second language and foreign-language learning contexts. As mobile-
assisted language learning (MALL) becomes more prevalent, it has motivated
researchers to explore various mobile technologies for the purpose of simulating
authentic language learning experiences (Adams et al., 2021; Chuah, 2014),
elevating the opportunities to use the target language and increasing language
gains (Kukulska-Hulme, 2016; Persson & Nouri, 2018; Shadiev et al., 2020).
Studies about MALL have evolved from empirical investigations on the

effectiveness of existing mobile applications to design-based research that
centred on the development of mobile applications to address specific areas of
concern within language learning (Botero et al., 2018; Viberg & Grönlund, 2012).
Despite the encouraging development of MALL, many studies on its
implementation focused on short-term pre- and post-test results (Gutiérrez-
Colón et al., 2020; Hsu & Liu, 2021). Burston (2015) highlighted this shortcoming,
which could be a result of poorly designed research studies. There is also a
commonly cited concern about the novelty effect of introducing mobile
applications to learners as it often leads to biased positive results in most of the
studies reported (Björkman et al., 2019; Jeno et al., 2019). In addition, previous
studies on MALL (Heil et al., 2016) were predominantly incorporating mobile
applications or tools which were not intended for language learning (e.g., social
media, chat, and quiz applications). Some studies investigated the usefulness
and effectiveness of commercially available language learning applications such
as Duolingo (Garcia, 2013; Shortt et al., 2021) and Busuu (Shibata, 2020). This
trend has prompted researchers to embark on design-based or developmental
research (Chuah & Kabilan, 2021) in creating applications that are more relevant
to language learning, especially in recent years.
In terms of development research related to mobile applications for language

learning, researchers tend to emphasise the testing of diverse technologies such
as multimedia, virtual and augmented reality, conversational agents and
artificial intelligence-based systems. Moreover, previous reviews on mobile-
assisted language learning were focusing on factors and effects of using mobile
tools. For instance, Elaish et al. (2019) reviewed 69 papers on mobile English
language learning according to constructs that included types of mobile learning
technologies, language learning problems, sample information, purposes and
assessment methods. Most of the studies selected were in the form of
investigation or review. There is an apparent need to investigate further the
development process by scrutinising the fundamental theories that were
selected. As reiterated by Traxler (2017), the underlying problem in mobile
learning is the lack of discourse on necessary theoretical considerations as the
spotlight is often on the technological aspect.
255
There is currently very limited systematic review driven by the need to examine
theoretical underpinnings of development research on mobile applications in
language learning. Most studies were on the general overview of mobile
learning adoption (Crompton & Burke, 2018; Osman, 2021). Existing reviews
that cover theories and models in mobile learning are limited to integration and
acceptance models (Alkhezzi & Ahmed, 2020; Kumar & Chand, 2019) or the
general design of mobile learning (Churchill et al., 2015), which are not specific
to developmental studies. As learning theories lay the foundation for any
development, examining and comparing the most commonly used theories in
the selected literature from 2011 to 2020 (a decade of literature) would be
beneficial as the findings could guide future development. Based on these gaps
in the current related reviews and research reports, it is necessary to examine
learning theories and principles that were considered in the development of
mobile applications for language learning.
Hence, this systematic review aims to answer the following research questions:
i. Which language learning problems were addressed in the studies on
mobile application development for language learning?
ii. What were the learning theories or principles used in guiding the
development of mobile applications for language learning?
2. Methods
A systematic review was conducted to answer the research questions. It was
based on the procedures mentioned by Khan et al. (2003) and Sarkis-Onofre et al.
(2021) which followed the PRISMA review protocol. The process consisted of the
following four phases:
2.1 Formulation of inclusion and exclusion criteria

In order to ensure the review covers relevant and accurate studies, this
systematic review used the inclusion and exclusion criteria as shown in Table 1.
These criteria were formulated in line with the aim and research questions set
for this review. Each research article must have fulfilled all the criteria to be
selected for further analysis.
Table 1: Inclusion and Exclusion Criteria

Inclusion Criteria Exclusion Criteria
• Explains the process of developing the • Used commercial applications or
mobile application for language readily available tools (e.g., Duolingo,
learning/teaching (design and Busuu) as this review focuses on
development) development research
• Includes explanations on learning • Reporting only empirical findings
theories or principles in guiding (e.g., pre- and post-test results)
development • Not related to language learning or
• Across all levels of education (K-12, teaching
higher education) • Article was published before 2011 or
• Article was published in peer- after 2020. This criterion is to focus on
reviewed publications the decade period of the related works.
• Article was written in English • Explanation on theories is too minimal
• Article’s full text is accessible and cannot be synthesised
256
2.2 Development of a Search Strategy

Several key terms and combination of the terms were used and then shortlisted
for better search results. The terms were “development of mobile applications”,
“design of mobile applications”, “language learning”, “development studies”,
“mobile learning applications”, “design-based research”, “developing mobile
applications”, “language learning apps”, “mobile apps”, and “language learning
applications”. Using these key terms with Boolean operators such as AND, OR,
and NOT (Jahan et al., 2016) as well as the database’s filtering function, the most
relevant articles were able to be shortlisted. Quotation marks were also used to
find the exact phrase. The chosen strategy focused on the article title, abstract or
summary, and on keywords listed in the article so that relevant articles could be
filtered efficiently.
The term “learning theories” or “learning theories for mobile application

design” were not added to the search process as the intention of this systematic
review is to derive those theories through the analysis of reports gathered on
related mobile learning applications. The review of related language learning
theories, however, was done separately in formulating the coding frame for the
analysis.
2.3 Identification of Relevant Publications

Two main databases (Scopus and the Web of Science) were used to identify
publications according to the criteria and search strategy defined in the earlier
phases. These databases were chosen as they were widely accepted as the main
sources of reputable journals that have a proper review process. The abstracts of
articles obtained were carefully read and evaluated for relevance. If the title was
relevant but the abstract provided had very limited information, the full text was
skimmed, scanned and checked according to the inclusion criteria. This
identification procedure was repeated twice to ensure no pertinent articles were
left out. Articles from the same project or by the same authors were also filtered
and only the one with the most information was included in the analysis.
From the initial search results using keywords, 158 studies were found to meet
the scope for further screening. A total of 79 did not fulfil the criteria stated in
Table 1, as the majority of these studies mainly reported empirical findings with
minimal details on the design and development process. Some also used existing
language learning applications though the titles seem relevant. Twelve studies
were excluded as they were duplicates or related to the same projects. The
remaining 67 were then screened thoroughly to ensure they met all the criteria in
Table 1. However, 28 of those studies had to be excluded owing to the fact that
the theoretical part lacked sufficient details or was unclear. Hence, the total
number of articles included for further analysis was 39. The identification and
screening procedures are illustrated in Figure 1.
257
Figure 1: Procedures in Screening Relevant Publications for Review
2.4 Coding and analysis procedures

In this review, the studies were first coded for essential information, namely
publication year and rate of publication, the geographical distribution of the
research area, and the research context or setting. To answer the research
questions (RQs) of the study, the studies were read and coded according to the
qualitative data coding guidelines suggested by Linneberg and Korsgaard
(2019). A deductive coding approach was employed whereby codes were pre-
defined for the important elements included in the two research questions.
These codes were theoretical concepts or themes derived from relevant
literature. The codes were compared during the coding process, and elements
that were not within this coding frame would then be added.
Table 2 shows the coding frame used in the analysis process. RQ1 covers
linguistic competence and communicative competence that are accepted to be
fundamental to language learning (Celce-Murcia, 2001; Larsen-Freeman &
Anderson, 2013) and other related difficulties such as motivation, lack of
exposure. RQ2 includes the three broad groups of learning theory
(behaviourism, cognitivism and constructivism) and their sub-groups that could
be in the forms of principles, models or approaches (Mowrer & Klein, 2000) as
well as theories drawn from language learning, especially within second
language acquisition (Mitchell et al., 2019), which are not part of the broad
groups. The coding process was repeated three times to ensure reliability and
validity.
Table 2: Coding Frame used for Data Analysis

RQ1: Language Learning Problems or Difficulties
Linguistic competence • Grammar
• Vocabulary
• Syntax
Communicative competence • Speaking skills
• Listening skills
• Reading skills
• Writing skills
• Interpersonal skills
• Intercultural communication
258
Other issues • Learner’s motivation

• Lack of authentic language learning opportunities
• Lack of support/guidance
RQ2: Underlying Learning Theories

Behaviourism • Direct instruction
• Programme instruction
Cognitivism • Attribution theory
• Elaboration theory
• Cognitive development
• Condition of learning
• Information processing theory
• Cognitive theory of multimedia learning
Constructivism • Activity theory
• Cognitive apprenticeship
• Contextual learning
• Collaborative learning
• Discovery learning
• Inquiry-based learning
• Problem-based learning
• Situated learning
• Personalised learning
Second Language • Connectionism
Acquisition • Chomsky’s Universal Grammar
• Krashen’s input hypothesis
• Krashen’s monitor model
• Interactionist theory
• Schmidt’s noticing hypothesis
• Halliday’s systemic functional grammar (SFG)
For research papers that do not specifically mention the theories, the underlying
principles explained in the papers were extracted and analysed. The researchers
then categorised them according to the most relevant theory or approach.
However, it is worth noting that there were only a few of these papers and only
those with evident indicators were included in the analysis. Papers that
contained insufficient details on the theories were excluded during the screening
phase.

The primary aim of this systematic review is to provide an overview of the
theoretical foundation chosen by researchers in their development of mobile
applications for language learning. This section reports the results obtained from
the analysis of 39 relevant studies, which are also discussed in relation to
pertinent literature.
3.1 Distribution of Publication within the Selected Period (2011-2020)

Figure 2 shows the distribution of research papers by year of publication within
the selected ten-year period. The average number of studies reported per year is
approximately four. As found by Elaish et al. (2019), studies in the area of mobile
learning are dominated by the purpose of general investigation and review of
259
existing applications. However, the number of development research studies

related to mobile learning applications for language learning shows an
increasing trend, perhaps due to the greater interest in developing applications
for specific group of learners.
Figure 2: Number of Studies by Year of Publication
3.2 Geographical Distribution of Research Area

The studies selected for analysis covered four regions (Asia, Europe, North
America and Africa) and 21 countries as shown in Figure 3. Researchers from
Asia, particularly those from Taiwan, were active in reporting their
developmental research related to mobile applications for language learning.
They were followed by researchers in Europe. However, the selected pool of
studies still managed to represent a global view of the research questions
investigated in this systematic review.
Figure 3: Geographical Distribution of Research Area of Selected Studies
3.3 Research Contexts

Figure 4 illustrates the number of publications by research contexts or settings.
Similar to other systematic reviews that focused on mobile-assisted language
learning, much of the development research was conducted in higher education
260
contexts. Nevertheless, it is interesting to note there were some development

projects that were done specifically for the general public such as the one by
Bradley et al. (2020), which focused on migrants. This seems to suggest that the
development of mobile applications for language learning is no longer restricted
to educational settings.
Figure 4: Number of Publications by Research Contexts
3.4 Language Learning Problems

Table 3 summarises language learning problems mentioned in the selected
studies. These problems were then used as the research problems to be
addressed through the development of mobile applications. The researchers in
the selected studies had given more attention to developing learners’ linguistic
competence, especially in terms of vocabulary acquisition (n=21). As the
building blocks of language, vocabulary mastery is regarded as very important
and any means to promote it, including through mobile applications, is often
welcome (Brahin et al., 2020; Wang & Suwanthep, 2017).
Another commonly cited problem across the selected studies is the lack of
motivation among the learners to learn the target language (n=11), especially
when conventional teaching strategies were used. There is also an excitement
among learners to use mobile devices to increase the opportunities to learn the
language by means of authentic tasks and resources (n=9). As mentioned by Hao
et al. (2019), mobile applications enable learners to be engaged with language
learning resources beyond the classroom. Furthermore, researchers focussed
more on receptive skills (listening and reading) than productive skills since
mobile applications are useful in displaying multimedia resources seamlessly.
Writing skills were not the main focus of the studies, perhaps owing to the fact
that it is a difficult skill to be taught via mobile applications. Li and Hegelheimer
(2013) highlighted this issue when developing their application for second
language writing (called Grammar Clinic), as they still focus on the grammar
aspect. Only three studies mentioned the problem of intercultural
communication, and they were embedded within the learning of speaking skills.
261
Table 3: Language Learning Problems Mentioned in the Studies

Problems Studies Total
Vocabulary Ogata et al. (2011); Veenhof et al. (2012); Lu et al. (2014); 21
Sandberg et al. (2014); Milutinović et al. (2013); Kim and
Smith (2015); Chachil et al. (2015); Kidu (2015); Rosell-
Aguilar and Qian (2015); Böhm and Constantine (2016);
Wang and Suwanthep (2017); Ou-Yang and Wu (2017);
Rawendy et al. (2017); Wilken et al. (2018); Samur (2019);
Al-Razgan and Alshaarri (2019); Jalaluddin (2020);
Bradley et al. (2020); Lai et al. (2020); Brahin et al. (2020);
Chen and Hsu (2020)
Learners' Connolly et al. (2011); Ibáñez et al. (2011); Ogata et al. 11

Motivation (2011); Hsu et al. (2013); Sandberg et al. (2014); Wang
and Suwanthep (2017); Ou-Yang and Wu (2017); Samur
(2019); Hao et al. (2019); Bradley et al. (2020); Brahin et
al. (2020)
Listening Ibáñez et al. (2011); Chachil et al. (2015); Rosell-Aguilar 10

and Qian (2015); Liu et al. (2018); Wilken et al. (2018);
Shadiev et al. (2018); Hao et al. (2019); Wang et al. (2019);
Bourekkache dan Kazar (2020); Chen and Hsu (2020)
Lack of Connolly et al. (2011); Ogata et al. (2011); Fallahkhair 9

Authentic (2012); Rosell-Aguilar and Qian (2015); Wong et al.
Language (2016); Böhm and Constantine (2016); Wang and
Learning Suwanthep (2017); Lee (2020); Hao et al. (2019)
Opportunities
Reading Ibáñez et al. (2011); Hsu et al. (2013); Chachil et al. 9
(2015); Mustapa et al. (2018); Shadiev et al. (2018); Hao et
al. (2019); Wang et al. (2019); Bourekkache dan Kazar
(2020); Chen and Hsu (2020)
Grammar Ogata et al. (2011); Li and Hegelheimer (2013); Haristiani 6

et al. (2019); Al-Razgan and Alshaarri (2019); Refat et al.
(2020); Lin et al. (2020)
Speaking Ibáñez et al. (2011); Lu et al. (2014); Wilken et al. (2018); 6

Shadiev et al. (2018); Hao et al. (2019); Bourekkache dan
Kazar (2020)
Writing Ibáñez et al. (2011); Li and Hegelheimer (2013); Rosell- 5

Aguilar and Qian (2015); Shadiev et al. (2018); Hao et al.
(2019)
Intercultural Ogata et al. (2011); Ibáñez et al. (2011); Bradley et al. 3

Communication (2020)
3.5 Learning Theories and Principles in Guiding Development

The outcome from the analysis on learning theories reveals that 25 of the 39
studies adopted constructivism in guiding their design and development of
262
mobile applications for language learning (refer to Figure 5). This finding is to be
expected since 21st century learning has been extensively promoted for the past
ten years. Technological advancement such as virtual and augmented reality
and web-based interactive tools has also contributed to the greater acceptance of
constructivist principles. However, it is noteworthy that behaviourism still has a
place in the development of mobile applications, especially in vocabulary and
grammar drills. The application designed by Kidu (2015), for example, is based
on structural programmed instruction that teaches learners the basic vocabulary
and grammatical rules of a minority language in Ethiopia. It comprises many
drills and exercises to enhance memorisation of the words and rules. Clearly, it
serves its intended purpose though guided by the less-popular behaviourist
approach.
Apart from that, researchers were not paying much attention to adopting
theories specific to language learning. Only six studies (Li & Hegelheimer, 2013;
Lin et al., 2020; Mustapa et al., 2018; Moreno & Vermeulen, 2015; Shadiev et al.,
2018; Wilken et al., 2018) selected theories within the scope of second language
acquisition (e.g., interactionist, Halliday’s systemic functional grammar and
Krashen’s input hypothesis). Four studies reported the use of the interactionist
theory in which the communicative approach is the preferred choice among the
researchers as it fits the scope of language learning.
In addition, contemporary theories related to the information age such as

connectivism and challenge-based learning were not mentioned at all in the
reviewed studies. This preference could mean that researchers are inclined to
rely on “tried-and-tested” theories in their developmental work rather than
exploring uncharted zones of newer theories. It could also be due to the general
belief that newer theories (e.g., connectivism) are essentially derived from the
core principles of constructivism. Nevertheless, this finding means that there is
room to investigate whether the theories or principles spawned from 21st century
learning could be more useful in the process of designing and developing
mobile learning applications.
Figure 5: Frequency Count of the Theory selected to guide Design and Development
263
Table 4 shows the overview of the main principles or approaches adopted in

each study based on the selected learning theory. Within constructivism,
situated learning, contextual learning, collaborative learning and game-based
learning are popular across the time period of 2011 to 2020. Activity theory and
cognitive apprenticeship are rarely mentioned; however, Wang et al. (2019) and
Hao et al. (2019) demonstrated how these two theories can be integrated in
developing applications for language learning. For cognitivism, the information
processing and mixed-modality theory are commonly used in applications that
contain multimedia elements as learning resources; however, they lack
interactivity. As cognitivism deals with human working memory, the principles
of this theory are useful in the development of applications that emphasise
meaningful memorisation of vocabulary based on a given context.
Table 4: Learning Theories and Principles Mentioned in Each Study

No. Study Learning Theory Main Principle/Approach
1 Connolly et al. (2011) Constructivism Situated learning
2 Ogata et al. (2011) Constructivism Situated learning;
Metacognition
3 Ibáñez et al. (2011) Constructivism Situated learning;
Collaborative learning
4 Fallahkhair (2012) Constructivism Collaborative learning
5 Veenhof et al. (2012) Constructivism Game-based learning
6 Li and Hegelheimer (2013) Interactionist SLA - Noticing hypothesis
7 Hsu et al. (2013) Constructivism Personalised learning;
Collaborative learning
8 Lu et al. (2014) Constructivism Activity theory;
RASE model
9 Sandberg et al. (2014) Cognitivism Information processing
10 Milutinović et al. (2013) Behaviourism Programme instructions;
Drills
11 Kim and Smith (2015) Constructivism Situated learning
12 Chachil et al. (2015) Constructivism Contextual learning
13 Kidu (2015) Behaviourism Structural; Direct instruction
14 Rosell-Aguilar and Qian Constructivism Personalised learning;
(2015) Game-based learning
15 Moreno and Vermeulen Interactionist Communicative approach;
(2015) Task-based language teaching
16 Wong et al. (2016) Constructivism Contextual learning
17 Chang et al. (2016) Constructivism Inquiry-based learning; ARCS
18 Böhm and Constantine Constructivism Contextual learning;
(2016) Personalised learning
19 Wang and Suwanthep Constructivism Contextual learning
(2017)
20 Ou-Yang and Wu (2017) Cognitivism Mixed-modality
21 Rawendy et al. (2017) Cognitivism Information processing;
Mnemonic
22 Liu et al. (2018) Constructivism Collaborative learning
23 Wilken et al. (2018) Interactionist Communicative approach
24 Mustapa et al. (2018) Krashen’s input Narrow reading (NR)
hypothesis approach
25 Shadiev et al. (2018) Interactionist Communicative approach;
264
Task-based language teaching

26 Samur (2019) Constructivism Game-based learning
27 Hao et al. (2019) Constructivism Cognitive apprenticeship
28 Haristiani et al. (2019) Behaviourism Programme instructions
29 Wang et al. (2019) Constructivism Activity theory model
(Engestrom)
30 Al-Razgan and Alshaarri Constructivism Game-based learning
(2019)
31 Jalaluddin (2020) Cognitivism Cognitive theory of
multimedia
learning
32 Bradley et al. (2020) Constructivism Situated learning
33 Refat et al. (2020) Cognitivism Information processing
34 Bourekkache dan Kazar Constructivism Situated learning
(2020)
35 Lee (2020) Constructivism Game-based learning;
collaborative learning
36 Lai et al. (2020) Constructivism Contextual learning
37 Brahin et al. (2020) Constructivism Game-based learning
38 Chen and Hsu (2020) Constructivism Situated learning;
Game-based learning
39 Lin et al. (2020) Halliday’s SFG Seven functions of language
development
The findings from this systematic review seem to point toward the emerging
concepts catalysed by constructivism. It is clear that as mobile technologies
become more advanced, features that enable authentic learning of a target
language become the key interest of mobile application developers. Related
principles or approaches such as situated learning and collaborative learning are
regarded to be effective in engaging learners while enhancing the mobile
language learning experience.
4. Implications and Recommendations

This systematic review is restricted to studies published in journals indexed by
the two databases (Scopus and Web of Science) pertaining to the development of
mobile applications for language learning with a specific interest in the learning
theories selected in each study. The review was limited to the articles screened
according to the criteria mentioned earlier and may not cover all works
published within the same scope as some could have been published in other
languages or indexed by other databases. Furthermore, during the screening
process, there were developmental studies that had to be removed from analysis
as they did not mention the underlying theories in the published article (e.g.
focusing only on the technical aspect). Despite this limitation, the results from
the review have provided a sufficient overview of the popular theoretical
frameworks adopted by researchers globally.
In addition, this review has also shown the emphasis on developing applications
for vocabulary learning. The ubiquitous nature of mobile applications is seen as
an excellent way to increase exposure to the target language by highlighting
words in context (Ogata et al., 2011). There is, however, a shift of focus towards
265
a more holistic approach in creating applications that not only motivate learners
to learn but also simulate a meaningful environment for interactions. The
development works by Lin et al. (2020) and Wilken et al. (2018) are among the
examples of how language learning through mobile applications can go beyond
the repeated memorisation of words and grammar.
Future research could examine the choice of theoretical underpinnings and its
impact on other relevant variables such as learners’ satisfaction and learning
gain so as to provide a clearer understanding of how a theoretical foundation
can influence the outcome of development. Moreover, this systematic review
also reveals that most of the studies were conducted in higher education
contexts although mobile applications for language learning could benefit school
students more, particularly at a younger age. This calls for researchers or
developers to make school settings their priority in future developmental works
to maximise the potential of MALL. During the review process, it is noted that
some theories were used without proper mapping of how each design element
can match the principles of the selected theory. Future research could therefore
address this issue by investigating how each feature or element in the
application is linked to the theoretical foundation.
5. Conclusion
This systematic review contributes to the related body of MALL research by
highlighting the current trends within the development of mobile applications
for language learning. It aims to address the gap in previous reviews by focusing
on the theoretical underpinnings of each study. Out of the 39 studies screened
for analysis, 64% (n=25) employed constructivism as their guiding theory in
designing and developing mobile applications. The dominance of constructivism
is anticipated; however, the lack of attempts to include theories related to
language learning as well is surprising. Only six studies specifically made use of
second language acquisition theories or at the very least, were guided by a
general understanding of how language learning works. Since the applications
were meant for language learning, it is a concern that the development process is
guided only by generic pedagogical approaches such as situated learning and
collaborative learning. Though these studies are beneficial in outlining how the
application should function, they may not be accurately solving the problems
identified in language learning. In conclusion, this systematic review has
highlighted the need to document the inclusion of learning theories properly in
the development of mobile applications for language learning, which can guide
other researchers who intend to evaluate the most appropriate theories to be
used.
6. References
Adams, D., Chuah, K. M., Mohamed, A., Sumintono, B., Moosa, V., & Shareefa, M.
(2021). Bricks to clicks: Students’ engagement in e-learning during the COVID-19
pandemic. Asia Pacific Journal of Educators and Education, 36(2), 99–117.
https://doi.org/10.21315/apjee2021.36.2.6
Al-Razgan, M., & Alshaarri, S. (2019, October). Design and development of a mobile
spelling game for elementary students using genetic algorithms. Proceedings of the
266
2019 11th International Conference on Education Technology and Computers (pp. 205-
209).
Alkhezzi, F., & Ahmed, M. S. (2020). A review of mobile learning technology integration:
Models, frameworks, and theories. College Student Journal, 54(4), 491-504.
Björkman, A. S., Spångeus, A., & Woisetschläger, M. (2019). Mobile learning device
increased study efficiency for radiology residents but with risk of temporary
novelty effect. Acta Radiologica Open, 8(11).
http://dx.doi.org/10.1177/2058460119889871
Böhm, S., & Constantine, G. P. (2016). Impact of contextuality on mobile learning
acceptance: An empirical study based on a language learning app. Interactive
Technology and Smart Education, 13(2), 107-122. https://doi.org/10.1108/ITSE-02-
2016-0003
Botero, G. G., Questier, F., Cincinnato, S., He, T., & Zhu, C. (2018). Acceptance and usage
of mobile-assisted language learning by higher education students. Journal of
Computing in Higher Education, 30(3), 426-451. https://doi.org/10.1007/s12528-
018-9177-1
Bourekkache, S., & Kazar, O. (2020). Mobile and adaptive learning application for
English language learning. International Journal of Information and Communication
Technology Education (IJICTE), 16(2), 36-46. https://doi.org/
10.4018/IJICTE.2020040103
Bradley, L., Bartram, L., Al-Sabbagh, K. W., & Algers, A. (2020). Designing mobile
language learning with Arabic speaking migrants. Interactive Learning
Environments, 1-13. https://doi.org/10.1080/10494820.2020.1799022
Brahin, N. M. A., Nasir, H. M., Jidin, A. Z., Zulkifli, M. F., & Sutikno, T. (2020).
Development of vocabulary learning application by using machine learning
technique. Bulletin of Electrical Engineering and Informatics, 9(1), 362-369.
Burston, J. (2015). Twenty years of MALL project implementation: A meta-analysis of
learning outcomes. ReCALL, 27(1), 4-20.
https://doi.org/10.1017/S0958344014000159
Celce-Murcia, M. (2001). Language teaching approaches: An overview. Teaching English
as a Second or Foreign Language, 2(1), 3-10.
Chachil, K., Rias, R. M., Engkamat, A., & Sarkawi, A. (2015). Interactive multimedia-
based mobile application for learning Iban language. Jurnal Teknologi, 75(3).
https://doi.org/10.11113/jt.v75.5041
Chang, C., Chang, C.-K., & Shih, J.-L. (2016). Motivational strategies in a mobile inquiry-
based language learning setting. System, 59, 100–115.
https://doi.org/10.1016/j.system.2016.04.013
Chen, Y. L., & Hsu, C. C. (2020). Self-regulated mobile game-based English learning in a
virtual reality environment. Computers & Education, 154,
https://doi.org/10.1016/j.compedu.2020.103910
Chuah, K. M. (2014). Luring the lurkers: Increasing participations in the online
discussions of a blended learning course. International Journal on E-Learning
Practices, 1(1), 49-58.
Chuah, K. M., & Kabilan, M. K. (2021). Teachers' views on the use of chatbots to support
English language teaching in a mobile environment. International Journal of
Emerging Technologies in Learning, 16(20), 223-237.
Churchill, D., Lu, J., Chiu, T. K., & Fox, B. (Eds.). (2015). Mobile learning design: Theories
and application. Springer.
Connolly, T. M., Stansfield, M., & Hainey, T. (2011). An alternate reality game for
language learning: ARGuing for multilingual motivation. Computers & Education,
57(1), 1389-1415. https://doi.org/10.1016/j.compedu.2011.01.009
267
Crompton, H., & Burke, D. (2018). The use of mobile learning in higher education: A
systematic review. Computers & Education, 123, 53-64.
Elaish, M. M., Shuib, L., Ghani, N. A., & Yadegaridehkordi, E. (2019). Mobile English
language learning (MELL): A literature review. Educational Review, 71(2), 257-276.
Fallahkhair, S. (2012). Development of location-based mobile language learning system
to support geolearners. Ubiquitous Learning: An International Journal, 4(1), 1-10.
https://doi.org/10.18848/1835-9795/CGP/v04i01/40288
Khan, K. S., Kunz, R., Kleijnen, J., & Antes, G. (2003). Five steps to conducting a
systematic review. Journal of the Royal Society of Medicine, 96(3), 118-121.
Kukulska-Hulme, A. (2016). Mobile assistance in language learning: A critical appraisal.
In A. Palalas & M. Ally (Eds.), The international handbook of mobile-assisted language
learning (pp. 138–160). China Central Radio & TV University Press.
Garcia, I. (2013). Learning a language for free while translating the web. Does Duolingo
work? International Journal of English Linguistics, 3(1), 19-25.
http://dx.doi.org/10.5539/ijel.v3n1p19
Gutiérrez-Colón, M., Frumuselu, A. D., & Curell, H. (2020). Mobile-assisted language
learning to enhance L2 reading comprehension: A selection of implementation
studies between 2012–2017. Interactive Learning Environments, 1-9.
https://doi.org/10.1080/10494820.2020.1813179
Hao, Y., Lee, K. S., Chen, S. T., & Sim, S. C. (2019). An evaluative study of a mobile
application for middle school students struggling with English vocabulary
learning. Computers in Human Behavior, 95, 208-216.
https://doi.org/10.1016/j.chb.2018.10.013
Haristiani, N. U. R. I. A., Danuwijaya, A. A., Rifa’i, M. M., & Sarila, H. (2019). Gengobot:
A chatbot-based grammar application on mobile instant messaging as language
learning medium. Journal of Engineering Science and Technology, 14(6), 3158-3173.
Heil, C. R., Wu, J. S., Lee, J. J., & Schmidt, T. (2016). A review of mobile language
learning applications: Trends, challenges, and opportunities. The EuroCALL
Review, 24(2), 32-50. https://doi.org/10.4995/eurocall.2016.6402
Hsu, K. C., & Liu, G. Z. (2021). A systematic review of mobile-assisted oral
communication development from selected papers published between 2010 and
2019. Interactive Learning Environments, 1-17.
https://doi.org/10.1080/10494820.2021.1943690
Hsu, C. K., Hwang, G. J., & Chang, C. K. (2013). A personalised recommendation-based
mobile learning approach to improving the reading performance of EFL students.
Computers & Education, 63, 327-336.
Ibáñez, M. B., García, J. J., Galán, S., Maroto, D., Morillo, D., & Kloos, C. D. (2011).
Design and implementation of a 3D multi-user virtual world for language
learning. Journal of Educational Technology & Society, 14(4), 2-10.
Jahan, N., Naveed, S., Zeshan, M., & Tahir, M. A. (2016). How to conduct a systematic
review: a narrative literature review. Cureus, 8(11).
https://dx.doi.org/10.7759%2Fcureus.864
Jalaluddin, I., Ismail, L., & Darmi, R. (2020). Developing vocabulary knowledge among
low achievers: Mobile augmented reality (MAR) practicality. International Journal
of Information and Education Technology, 10(11), 813-819.
Jeno, L. M., Vandvik, V., Eliassen, S., & Grytnes, J. A. (2019). Testing the novelty effect of
an m-learning tool on internalisation and achievement: A self-determination
theory approach. Computers & Education, 128, 398-413.
Kidu, H. (2015). A mobile-based Tigrigna language learning tool. International Journal of
Interactive Mobile Technologies, 9(2). http://dx.doi.org/10.3991/ijim.v9i2.4322
268
Kim, Y., & Smith, D. (2017). Pedagogical and technological augmentation of mobile
learning for young children interactive learning environments. Interactive Learning
Environments, 25(1), 4-16. http://dx.doi.org/10.1080/10494820.2015.1087411
Kumar, B. A., & Chand, S. S. (2019). Mobile learning adoption: A systematic review.
Education and Information Technologies, 24(1), 471-487.
Lai, C. H., Jong, B. S., Hsia, Y. T., & Lin, T. W. (2020). Integrating flash cards with
narratives for mobile learning of English vocabulary. International Journal of
Interactive Mobile Technologies, 14(4). https://doi.org/10.3991/ijim.v14i04.11723
Larsen-Freeman, D., & Anderson, M. (2013). Techniques and principles in language
teaching (3rd ed.). Oxford handbooks for language teachers. Oxford University Press.
Lee, J. (2020). Problem-based gaming via an augmented reality mobile game and a
printed game in foreign language education. Education and Information
Technologies, 1-29. https://doi.org/10.1007/s10639-020-10391-1
Li, Z., & Hegelheimer, V. (2013). Mobile-assisted grammar exercises: Effects on self-
editing in L2 writing. Language Learning & Technology, 17(3), 135-156.
Lin, J. J., & Lin, H. (2019). Mobile-assisted ESL/EFL vocabulary learning: A systematic
review and meta-analysis. Computer-Assisted Language Learning, 32(8), 878-919.
https://doi.org/10.1080/ 09588221.2018.1541359
Lin, C. J., Hwang, G. J., Fu, Q. K., & Cao, Y. H. (2020). Facilitating EFL students’ English
grammar learning performance and behaviors: A contextual gaming approach.
Computers & Education, 152, 103876.
Linneberg, M. S., & Korsgaard, S. (2019). Coding qualitative data: A synthesis guiding
the novice. Qualitative Research Journal, 19(3), 259-270.
https://doi.org/10.1108/QRJ-12-2018-0012
Liu, G. Z., Chen, J. Y., & Hwang, G. J. (2018). Mobile-based collaborative learning in the
fitness center: A case study on the development of English listening
comprehension with a context-aware application. British Journal of Educational
Technology, 49(2), 305-320. https://doi.org/10.1111/bjet.12581
Lu, J., Meng, S., & Tam, V. (2014). Learning Chinese characters via mobile technology in
a primary school classroom. Educational Media International, 51(3), 166-184.
https://doi.org/10.1080/09523987.2014.968448
Milutinović, M., Labus, A., Stojiljković, V., Bogdanović, Z., & Despotović-Zrakić, M.
(2015). Designing a mobile language learning system based on lightweight
learning objects. Multimedia Tools and Applications, 74(3), 903-935.
https://doi.org/10.1007/s11042-013-1704-5
Mitchell, R., Myles, F., & Marsden, E. (2019). Second language learning theories. Routledge.
Moreno, A. I., & Vermeulen, A. (2015). Profiling a MALL app for English oral practice: A
case study. Journal of Universal Computer Science, 21(10), 1339-1361.
Mowrer, R. R., & Klein, S. B. (Eds.). (2000). Handbook of contemporary learning theories.
Psychology Press.
Mustapa, A. M., Rahman, Z. A., Ghani, M. Z. A., Saad, M. F. M., & Mohamed, F. A.
(2018). Qiraahbot's prototype development for an extensive reading activity.
International Journal of Civil Engineering and Technology, 9(9), 1494-1503.
Ogata, H., Li, M., Hou, B., Uosaki, N., El-Bishouty, M. M., & Yano, Y. (2011). SCROLL:
Supporting to share and reuse ubiquitous learning log in the context of language
learning. Research & Practice in Technology-Enhanced Learning, 6(2).
Osman, S. Z. M. (2021). A visual pattern of two decades of literature on mobile learning:
A bibliometric analysis. International Journal of Learning, Teaching and Educational
Research, 20(10). https://doi.org/10.26803/ijlter.20.10.16
Ou-Yang, F. C., & Wu, W. C. V. (2017). Using mixed-modality vocabulary learning on
mobile devices: Design and evaluation. Journal of Educational Computing Research,
54(8), 1043-1069. https://doi.org/10.1177%2F0735633116648170
269
Persson, V., & Nouri, J. (2018). A systematic review of second language learning with
mobile technologies. International Journal of Emerging Technologies in Learning, 13(2).
https://doi.org/10.3991/ijet.v13i02.8094
Rajendran, T., & Yunus, M. M. (2021). Chatterpix Kids: A potential mobile app for
helping primary ESL pupils to improve their speaking fluency. International
Journal of Learning, Teaching and Educational Research, 20(4), 18-42.
Rawendy, D., Ying, Y., Arifin, Y., & Rosalin, K. (2017). Design and development game
Chinese language learning with gamification and using mnemonic method.
Procedia Computer Science, 116, 61-67.
Refat, N., Kassim, H., Rahman, M. A., & Razali, R. B. (2020). Measuring student
motivation on the use of a mobile-assisted grammar learning tool. PloS One, 15(8),
https://doi.org/10.1371/journal.pone.0236862
Rosell-Aguilar, F., & Qian, K. (2015). Design and user evaluation of a mobile application
to teach Chinese characters. The JALT CALL Journal, 11(1), 19-40.
https://doi.org/10.29140/jaltcall.v11n1.182
Sarkis-Onofre, R., Catalá-López, F., Aromataris, E., & Lockwood, C. (2021). How to
properly use the PRISMA Statement. Systematic Reviews, 10(1), 1-3.
Samur, Y. (2019). Kes Sesi: A mobile game designed to improve kindergarteners'
recognition of letter sounds. Journal of Computer-Assisted Learning, 35(2), 294-304.
https://doi.org/10.1111/jcal.12331
Sandberg, J., Maris, M., & Hoogendoorn, P. (2014). The added value of a gaming context
and intelligent adaptation for a mobile learning application for vocabulary
learning. Computers & Education, 76, 119-130.
Shibata, N. (2020). The usefulness of Busuu online courses for foreign language
learning. Computer-Assisted Language Learning Electronic Journal, 21(2), 197-203.
Shin, D. H., Shin, Y. J., Choo, H., & Beom, K. (2011). Smartphones as smart pedagogical
tools: Implications for smartphones as u-learning devices. Computers in Human
Behavior, 27(6), 2207-2214.
Shadiev, R., Liu, T., & Hwang, W. Y. (2020). Review of research on mobile-assisted
language learning in familiar, authentic environments. British Journal of Educational
Technology, 51(3), 709-720. https://doi.org/10.1111/bjet.12839
Shadiev, R., Hwang, W. Y., Huang, Y. M., & Liu, T. Y. (2018). Facilitating application of
language skills in authentic environments with a mobile learning system. Journal
of Computer-Assisted Learning, 34(1), 42-52. https://doi.org/10.1111/jcal.12212
Shortt, M., Tilak, S., Kuznetcova, I., Martens, B., & Akinkuolie, B. (2021). Gamification in
mobile-assisted language learning: A systematic review of Duolingo literature
from public release of 2012 to early 2020. Computer-Assisted Language Learning, 1-
38. https://doi.org/10.1080/09588221. 2021.1933540
Traxler, J. (2017). Mobile learning: The philosophical challenges, problems and
implications of defining and theorising. Progressio, 39(1), 17-33.
Veenhof, G., Sandberg, J., & Maris, M. (2012, June). ZooQuest: A mobile game-based
learning application for fifth graders. International Conference on Intelligent Tutoring
Systems (pp. 687-688). Springer.
Viberg, O., & Grönlund, Å. (2012). Mobile assisted language learning: A literature
review. In M. Specht, M. Sharples, & J. Multisilta (Eds.). Proceedings of the 11th
world conference on mobile and contextual learning (pp. 9-16). University of Helsinki.
Wang, F., & Suwanthep, J. (2017). Constructivism-based mobile application for EFL
vocabulary learning. International Journal of Learning, 3(2), 106-112.
https://doi.org/10.18178/IJLT.3.2.106-112
270
Wang, H. Y., Lin, V., Hwang, G. J., & Liu, G. Z. (2019). Context-aware language-learning
application in the green technology building: Which group can benefit the most?
Journal of Computer Assisted Learning, 35(3), 359-377.
Wilken, I., Taljard, E., & De Wet, F. (2018). Language learning applications for Sepedi: A
user experience study. Southern African Linguistics and Applied Language Studies,
36(2), 85-104. https://doi.org/10.2989/16073614.2018.1450638
Wong, L. H., King, R. B., Chai, C. S., & Liu, M. (2016). Seamlessly learning Chinese:
Contextual meaning making and vocabulary growth in a seamless Chinese as a
second language learning environment. Instructional Science, 44(5), 399-422.
https://doi.org/10.1007/s11251-016-9383-z
271

Vol. 21, No. 8, pp. 271-287, August 2022
The Effect of Professional Training on In-service

Secondary School Physics 'Teachers' Motivation
to Use Problem-Based Learning
Stella Teddy Kanyesigye*
University of Rwanda College of Education (URCE), Rwanda
Jean Uwamahoro
University of Rwanda College of Education (URCE), Rwanda
Imelda Kemeza
Mbarara University of Science and Technology, Uganda
Abstract. Professional training for in-service teachers is at an utmost level

to improve their teaching practices. Therefore, this study aimed to assess
the effect of professional training on in-service secondary school physics
teachers' motivation to use Problem-Based Learning (PBL). A pre-and
posttest quasi-experimental design was used to conduct the study. A two-
day professional training on PBL was delivered as an independent
variable, while teachers' motivation to use PBL was conceived as the
dependent variable. The study involved fifty (50) in-service physics
teachers (20 in control and 30 in the experimental group) from 25
secondary schools in four districts of Southwestern Uganda. Data were
analyzed with SPSS v.23.0 using descriptive statistics, and paired, and
independent samples t-test. The findings indicated a high statistically
significant (p<.05) positive change and a greater motivation to use PBL
among teachers who received professional training in PBL compared to
those who did not receive it. It was recommended that the Government,
in conjunction with school administrators, regularly organize similar
training and, if possible, for all teachers national wide.
Keywords: physics teachers; problem-based learning; professional

training; secondary school; Uganda
* Corresponding author: Stella Teddy Kanyesigye; kanyesigyestella@gmail.com
©Authors
272
1. Introduction
The Science policy in Uganda, which took effect in 2006, made the study of science
subjects, namely: Physics, Chemistry, and Biology, compulsory for ordinary level
secondary school students. According to Uganda National Examination Board
(2017), the performance levels in all science subjects continue to be low, with about
55% of the candidates unable to exhibit the minimum required competency to be
graded; the worst performed science subject is Physics. Ugandan Ministry of
Education and Sport (2014) attached 'students' poor performance and weak
knowledge acquisition to 'teachers' employment of inappropriate pedagogical
skills despite the introduction of a number of programs, including the Secondary
Science and Mathematics (SESEMAT) program in 2005, to enhance the quality of
teaching and learning science and mathematics in secondary schools.
Traditionally, teachers use direct instruction in teaching, which does not

effectively promote 'students' understanding of the subject concepts (Samsudin et
al., 2019). Mineo et al. (2010) emphasized that teachers must employ innovative
teaching approaches that lead to observable and measurable positive changes in
'students' learning. In this regard, professional training becomes essential for the
change of attitude of teachers since innovative methods such as problem-based
learning (PBL) are newer techniques in educational institutions, as observed by
Singh et al. (2014). PBL is a teaching style that allows students to develop as
learning engines. It is a learner-centered pedagogy where students learn the
subject by experiencing the solution to an open-ended problem found in the
trigger material. PBL uses complex real-world topics as classroom material and
encourages students to develop problem-solving skills and learn concepts rather
than simply absorbing facts (Dorimana et al., 2021).
Although using PBL as a method of instruction makes students active and enables
them to develop cognitive skills (Sulaiman, 2010), the Ugandan physics syllabus
does not reflect the PBL components, leading to few schools practicing it. Research
by Mansor et al. (2015) showed that 'teachers' lack of experience in implementing
PBL and limited knowledge of the PBL approach could contribute to their lack of
motivation to use the pedagogy, which may affect the skills that students possess.
Based on this background, the study set out to evaluate the effect of Professional
Training on In-service Secondary School Physics Teachers' Motivation to Use
Problem-Based Learning. The study was guided by one research question: What
is the impact of professional training on in-service secondary school physics
teachers' motivation to use Problem-Based Learning? It followed the hypothesis
that there is no statistical difference in in-service secondary school physics
teachers' motivation to use Problem-Based Learning between those who received
professional training in PBL and those who did not.
2. Literature review
Concept of Problem-Based Learning (PBL)
Teachers and students in STEM tend to face challenges in formulating suitable
classroom problems which promote critical thinking, scientific writing,
communication, and problem-solving skills (Sulaiman, 2010). Classroom
problems presented in textbooks tend to be well-structured with specific solutions
that require the application of a limited number of rules and principles within
273
well-defined parameters limiting 'students' involvement (Sterling, 2013).

Involving students actively in classroom activities and enabling them to relate the
concepts with real-life make them find their concepts that impact improving
learning achievements (Mansor et al., 2015). Real-life problems are open-ended
and ill-defined, with multiple solutions that require various paths to find a
solution; they tend to have fewer parameters that can be manipulated and contain
uncertainty about which concepts, rules, and principles that are necessary for the
solution, or how they are organized and which solution is best (Christiansen,
Kuure, Morch & Lindstraom, 2013). Designing such problems into learning in an
authentic context is a key factor that distinguishes PBL from other instructional
methods (Sterling, 2013).
In PBL, students learn a concept through the experience of collaborative problem

solving achieved through interacting with peers and teachers to obtain a solution
or clarify the problem (Christiansen et al., 2013). It was meant to guide students
to become experts in their fields of study, identify existing problems, analyze
them, and come up with appropriate solutions. During the implementation of
PBL, hands-on and inquiry are adopted as pedagogical techniques to build
appropriate behaviors necessary for successful problem solving (Sterling, 2013).
In PBL, students are organized into small groups as they solve a problem, discuss
the results and find the best solution (Nikmah, 2018). They do not only need
instruction but also access to the world to relate the knowledge already possessed
with that in the real-world setting (Sterling, 2013). The effectiveness of the use of
PBL depends on several factors, including the knowledge teachers have about the
whole teaching and learning process (Dorimana et al., 2021).
Teacher professional training

Improving teacher knowledge is important since teachers are the ones expected
to improve 'learners' learning (Ndihokubwayo, Nyirigira, et al., 2020;
Ndihokubwayo & Murasira, 2019). Carpendale and Hume (2020) emphasize the
need to train teachers to sustain the content knowledge. Since students value
expertise, the best teachers are the subject specialists who understand the
curriculum and have excellent facilitation skills. Professional training is of great
importance in such a way that: they not only provide them with the support,
resources, and training needed to transform their courses to incorporate PBL and
related active learning strategies; but also enables them to acquire skills in
facilitating and managing group dynamics. Active learning, including the use of
PBL, not only allows learners to work in small groups during class and interact
with each other or their teacher (Lombardi et al., 2021) but also builds and
improves teamwork (Chien, 2020). Weizman et al. (2008) pointed out that
professional development and practice using PBL can help teachers develop the
ability to apply their knowledge in real classroom settings, assess the effectiveness
of their actions, and revise plans according to the evidence they collect and
interpret. In addition, PBL pieces of training help teachers develop effective group
skills and experience.
Theoretical framework
The study aimed at effecting change in knowledge of in-service physics teachers
in PBL through professional training. It targeted helping teachers to be conversant
274
with PBL knowledge and willingly applied it during the teaching and learning
process to improve students' academic achievements. It was hoped that the
teachers would be self-directing and innovative as they implement PBL in real
classrooms with minimal supervision. PBL dwells on principles of adult learning
theory which focus on motivating students, encouraging them to set their own
learning goals, and allowing them to make decisions concerning their learning.
Adult learning theory assumes that trainees have an existing base of knowledge
and life experiences; they seek out continuous learning based on personal
interests, wants, and needs; and they understand why they are learning. This
theory calls for active participation and puts the learner at the center of the
learning experience, emphasizing the teacher's role as purely that of a mentor; all
these are key elements of PBL (Kenner & Weinerman, 2011; Prusko, 2021). Thus,
the study was based on adult learning theory applicable to this study of in-service
teacher training.
The motivation for the study

A recent study has shown that teachers are motivated by being provided in-
service training, and such training improves their teaching practices. Oyo et al.
(2017) have investigated the outcome after implementing the massive computer
literacy open online courses (MOOC) through 'the teachers' e-learning portal. The
authors found that Ugandan teachers get interested and improve their
performance. It was found that irrespective of age, when teachers are adequately
supported internally by their schools and externally by a university, they can
improve their digital literacy and subsequently engage in life-long online
learning. Likewise, the high percentage of teacher 'participants' completions and
high volume of educational e-content generated confirm an effective, attractive,
and self-sustainability for in-service teachers. Therefore, the current study proves
the in-service 'teachers' change through PBL to sustain and improve classroom
teaching and learning atmosphere.
3. Methodology
Research design
In reference to Creswell (2014), a quantitative cross-sectional survey design was
used in this study in order to understand how professional training affects in-
service physics teachers' motivation to use PBL. Such design analyzes data from
a population, or a representative subset, at a specific point in time; researchers
measure the outcome and the exposures of the study participants at the same time.
The study was conducted among 50 in-service physics teachers hailing from four
districts in the southwestern region of Uganda. The four districts included
Bushenyi, Rubirizi, Mitooma, and Sheema. The schools were randomly selected
from clustered districts, but the teachers who participated were selected
purposively by the respective headteachers. There were two groups identified as
experimental (n = 30; 60%) and control (n = 20; 40%) groups depending on
whether they participated or not in the professional training in PBL organized by
the authors. These participants were all exposed to both pretest and posttest.
Respondents
Among the 50 participants, 14 were from Bushenyi district, 23 from Mitooma
district, five from Rubirizi district, and eight from Sheema district; 26 teachers
275
came from Government owned schools while 24 were from private owned
schools; 12 came from single-girls schools, 12 from single 'boys' schools and 31
from mixed (both girls and boys) schools; 13 were from schools which are
boarding only, and 37 were from schools which are both day and boarding; 39
came from rural based schools while 11 came from urban-based schools. Among
the 50 participants in the study, 34% were females while 66% were males; their
age was such that 12% were in the range of 20 -24 years, 18% were 25 – 29 years,
28% were 30 – 34 years, and 32% were 35 039 years, and 10% were at least 40 years
old. Sixteen percent had an experience of fewer than two years, 16% were 2 -5
years, 38% were in the range 6 – 10 years, 22% were 11 -15 years, and 8% had an
experience above 15 years. Those with a diploma as their highest academic
qualification constituted 6 % of the total participants, 82% had bachelor's degrees,
while 12% had postgraduate degrees such as masters.
Instruments
Among 50 participants, two groups were formed and identified as an
experimental group with 30 teachers (teachers who participate in the professional
training of PBL) and a control group with 20 teachers (teachers who did not
participate in the professional training of PBL). This study used a survey
questionnaire which was modified from the tool developed by Lee and Blanchard
(2019) as the main data gathering tool. The modification was in such that Item 1,
which initially was stated as 'The following statement best describes my training
for PBL (1= none; 2 = informal; 3 = formal; 4 = informal & formal) was restated as
'I have previously had training for 'PBL'; item 2 which read as 'I have had the
following amount of formal training PBL training (e.g., professional
development): (1 = ≤ 1 day; 2 = 2–5 days; 3 = 6–10 days; 4 = > 2 weeks)'. All items
were rated based on the scale of "1 - Strongly disagree, 2 - Disagree, 3- somewhat
disagree, 4 - somewhat agree, 5 - Agree, and 6 - Strongly agree." Section A of the
survey investigated 'participants' demographic information such as school of
origin, ownership of the school, category of the school, type of school, location of
the school, gender of the teacher, age of teacher, number of years spent teaching,
and the highest level of education. Part B of the survey looked at teachers'
experience with PBL (items 1 and 2), 'Teachers' general concept with PBL (items 3
- 5), 'teachers' perceived competence in practicing PBL (items 6 - 10), the perceived
value of PBL to teachers (items 11 – 15), the perceived value of PBL to students
(items 16 – 17), and perceived cost of implementing PBL (items 18 – 29).
Validation of instruments
The survey items, after their modification (Appendix A), were first presented to
two (2) educational research experts for review to assess their relevance to the
study. The difference in agreement between the two experts was compared using
McNemar Test and was found to be nonsignificant (p = 1.000) hence considering
the instrument valid. For reliability of the instrument, the survey was pilot-tested
among ten (10) physics teachers who were also part of the final study to assess the
correctness of the wordings. The results of the pilot study yielded a 'Cronbach's
Alpha reliability coefficient of 0.92; hence the survey items were considered
reliable.
276
Data collection procedures

The survey was first administered to all participants a week before the
commencement of the training to determine the 'teachers' competency gap in
using PBL. Results from the first testing indicated a lack of competency and
negative perception about using PBL. This necessitated a training which the
authors organized (see appendix B for the training schedule) and was attended by
30 randomly selected in-service physics teachers. After the training, the teachers
went back to their schools with the determination to practice the knowledge and
skills they had acquired. Two months after the trained teachers had gone back to
their schools, a posttest (similar to the pretest) was then administered to include
even teachers who had not trained.
Intervention
A two-day, six-hour professional training was then organized on 7th and 8th
February 2021, at Ruhinda Secondary School-Mitooma district and was attended
by 30 in-service physics teachers mentioned earlier. The main purpose of the
professional training was to enhance in-service secondary school physics teachers'
knowledge of what PBL is and how it can be effectively implemented in the
classrooms; the following objectives guided it: (a) To provide background
information on the origin and importance of PBL, (b) to provide skills on
generating PBL questions, (c) To provide skills on presentation of a PBL lesson,
and (d) to provide knowledge on the assessment of a PBL lesson.
The trainer for the Secondary Science and Mathematics (SESEMAT) program in
the western region of Uganda served as a facilitator for the formed groups. The
roles of the participants and the training leader were defined at the start of the
training. Groups of five participants were constituted. In formulating PBL
questions, the topic of waves was selected as agreed upon by all participants in
the training based on the fact that it was among the most challenging topic to
students. Since participants were taken as adult students, learning objectives were
first shared with them at the start of the training, along with methods to be utilized
during the course of the training. Using a projector, literature on the history and
importance of PBL was then presented to the participants. During the training,
specific attention was put on elaborating and refining prior knowledge of
participants on PBL, engaging them in self-directed learning via hands-on
activities based on the topic of waves, and regularly reflecting on how to put PBL
into practice. One of the formulated questions was: "Explain how a standing wave
is formed."
Data analysis
We used descriptive and inferential statistics to analyze data. Data was entered
into the computer using the Statistical Package for Social Science (SPSS) software,
version 23.0, to compute statistical significance. Modal responses were
determined, and bar charts were used to visualize differences in the pretest and
posttest between experimental and control groups. Then, a paired samples t-test
was used to find out if there was a statistically significant change in the use of PBL
between the pretest and postest for teachers that attended the professional
training and those that did not attend the professional training across all the
subsections of the survey. An independent samples test was run to determine
277
whether there was a statistical difference in the responses between the

experimental and control groups for each of the pretest and posttest. Thus, we
computed inferential statistics on the p-value. When this value is less than alpha
significance, there is a statistically significant difference in favor of a group with a
high mean score.
Ethical considerations
The research proposal was ethically cleared by the University of Rwanda Research
Ethics Committee, and thereafter, an authorization letter to do research in Uganda
was obtained from the Permanent Secretary-Ministry of Education.
4. Results
This study aimed to assess the effect of Professional Training on In-service
Secondary School Physics Teachers' Motivation to use Problem-Based Learning.
In the case of the pretest, we first considered the modal responses in each item for
both the experimental and control groups (Figure 1), and the results indicated that
the responses were generally similar for both groups. Most of the participants
indicated that they had hardly had any training in PBL (item 1), which was mostly
the reason they did not teach using PBL (item 2). They also lacked knowledge of
PBL concepts in addition to low perceived competence in practicing PBL. They
were not sure as to whether practicing PBL adds any value to either themselves
or the students, and they also felt that practicing PBL is costly. Generally, they
portrayed a low motivation to use PBL as their overall responses ranged from
somehow agree to disagree.
4.5
4
Pretest Modal Response
3.5
3
2.5
2
1.5
1
0.5
0
10
15
20
25
Item
11
12
13
14
16
17
18
19
21
22
23
24
26
27
28
29
5
1
2
3
4
6
7
8
9
Experimental group Control group
Figure 1: Modal Responses of participants in the pretest
A paired samples t-test was run (Tables 3 and 4) for the experimental and control
groups.
278
Table 3: Paired Samples Statistics for the experimental and control groups
Item Experimental group (N=30) Control group (N=20)
Posttest Pretest Posttest Pretest
Mean SD Mean SD Mean SD Mean SD
1 5.47 0.507 3.37 0.964 3.25 0.910 3.25 0.910
2 5.57 0.504 3.63 0.765 3.05 1.146 3.05 1.146
3 5.50 0.509 3.73 0.640 3.80 0.410 3.65 0.489
4 5.53 0.507 4.27 0.450 3.80 0.410 3.80 0.410
5 5.40 0.498 3.83 0.531 3.65 0.587 3.65 0.587
6 4.03 0.183 3.80 0.484 3.55 0.510 3.55 0.510
7 1.53 0.507 3.50 0.682 3.60 0.598 3.60 0.598
8 2.20 1.031 3.50 0.682 3.45 0.686 3.45 0.686
9 5.47 0.507 4.10 0.305 3.40 0.503 4.10 0.308
10 1.87 0.629 3.53 0.819 3.65 0.671 3.65 0.671
11 1.67 0.479 3.00 0.910 3.00 0.973 3.00 0.973
12 5.30 0.466 3.13 0.629 3.30 0.923 3.30 0.923
13 5.37 0.490 3.80 0.484 3.90 0.308 3.90 0.308
14 1.53 0.507 2.10 0.481 2.10 0.553 2.10 0.553
15 5.40 0.498 3.27 0.640 3.85 0.587 3.85 0.587
16 1.40 0.498 2.23 0.679 1.95 0.224 2.30 0.733
17 1.57 0.504 2.97 0.320 2.75 0.550 2.90 0.447
18 5.57 0.504 4.03 0.183 4.00 0.000 4.00 0.00
19 5.50 0.572 4.10 0.403 4.05 0.224 4.05 0.224
20 5.63 0.490 4.03 0.414 4.00 0.324 4.00 0.324
21 5.63 0.490 4.07 0.254 3.95 0.224 3.95 0.224
22 1.57 0.568 2.83 0.461 2.85 0.489 2.85 0.489
23 2.03 0.615 3.37 0.490 3.55 0.510 3.55 0.510
24 1.63 0.490 2.70 0.535 2.95 0.224 2.95 0.224
25 1.90 0.712 4.07 0.254 4.10 0.308 4.10 0.308
26 1.77 0.626 3.47 0.776 3.45 0.605 3.45 0.605
27 1.53 0.507 3.10 0.662 3.50 0.513 3.50 0.513
28 5.60 0.563 4.03 0.183 4.05 0.224 4.05 0.224
29 2.57 0.774 3.90 0.548 3.95 0.224 3.95 0.224
From Table 3, it is observed that the mean values between the pretest and posttest,
especially for the experimental group, were really different (inclined more to
agree in posttest than in pretest strongly), and this difference was statistically
significant (p < 0.00) for almost all items (Table 4) and the effect size for most of
the items was high (d > 0.8). However, these results were almost similar and
nonsignificant for the control group in almost all items (Figure 2), and as a result,
a paired samples T-test could not be computed for this group but was instead
computed for the experimental group.
279
Table 4: The paired samples t-test and effect size values for the experimental group
Ite Paired Differences t df p d
m Mea Std. Std. Error 95% Confidence
n Deviati Mean Interval of the
on Difference
Lower Upper
1 2.10 1.09 0.20 1.69 2.51 10.52 29 0.00 1.9
2 1.93 0.79 0.14 1.64 2.23 13.49 29 0.00 2.5
3 1.77 0.82 0.15 1.46 2.07 11.84 29 0.00 2.2
4 1.27 0.58 0.11 1.05 1.48 11.89 29 0.00 2.2
5 1.57 0.63 0.11 1.33 1.80 13.71 29 0.00 2.5
6 0.23 0.43 0.08 0.07 0.39 2.97 29 0.01 0.5
7 1.97 0.85 0.16 2.28 1.65 12.67 29 0.00 2.3
8 1.30 1.21 0.22 1.75 0.85 5.90 29 0.00 1.1
9 1.37 0.62 0.11 1.14 1.60 12.17 29 0.00 2.2
10 1.67 0.99 0.18 2.04 1.30 9.18 29 0.00 1.7
11 1.33 0.80 0.15 1.63 1.03 9.10 29 0.00 1.7
12 2.17 0.75 0.14 1.89 2.45 15.89 29 0.00 2.9
13 1.57 0.68 0.12 1.31 1.82 12.64 29 0.00 2.3
14 0.57 0.73 0.13 0.84 -0.30 4.26 29 0.00 0.8
15 2.13 0.73 0.13 1.86 2.41 16.00 29 0.00 2.9
16 0.83 0.65 0.12 1.08 0.59 7.05 29 0.00 1.3
17 1.40 0.56 0.10 1.61 1.19 13.61 29 0.00 2.5
18 1.53 0.51 0.09 1.34 1.72 16.55 29 0.00 3.0
19 1.40 0.68 0.12 1.15 1.65 11.37 29 0.00 2.1
20 1.60 0.56 0.10 1.39 1.81 15.56 29 0.00 2.8
21 1.57 0.57 0.10 1.35 1.78 15.10 29 0.00 2.8
22 1.27 0.64 0.12 1.51 1.03 10.85 29 0.00 2.0
23 1.33 0.71 0.13 1.60 1.07 10.27 29 0.00 1.9
24 1.07 0.64 0.12 1.31 0.83 9.13 29 0.00 1.7
25 2.17 0.75 0.14 2.45 1.89 15.89 29 0.00 2.9
26 1.70 0.95 0.17 2.06 1.34 9.78 29 0.00 1.8
27 1.57 0.82 0.15 1.87 1.26 10.50 29 0.00 1.9
28 1.57 0.57 0.10 1.35 1.78 15.10 29 0.00 2.8
29 1.33 0.76 0.14 1.62 1.05 -9.63 29 0.00 1.8
280
4.5
4
3.5
Mean Response
3
2.5
2
1.5
1
0.5
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
Items
Control group (N=20) Pretest Mean Control group (N=20) Posttest Mean
Figure 2: Mean responses between the pretest and posttest of the control group
The difference in the posttest responses between the experimental and control
groups can be visualized in Figure 3, where the modal response for each item was
represented using a bar group.
6
Posttest Modal Response
0
14
10
11
12
13
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
Item
7
1
2
3
4
5
6
8
9
Experimental group Control group
Figure 3: Posttest modal responses of both the experimental and control groups
It can be seen from Figure 3 that even though the control group's responses rotated
around somewhat agree for most of the items (implying little motivation to use
PBL), the responses for the experimental group were in the range of agreeing to
strongly agree to mean that this group had high motivation to use PBL.
5. Discussion of results
Professional training motivates teachers to use PBL. It was found from the pretest
that the teachers lack training in PBL; thus, they lack knowledge of the PBL
concept and do not use PBL as an instructional approach since they perceive it as
costly. The findings indicated that for the control group, the 'participants'
responses in both the pretest and posttest were generally similar (Table 4 and
281
Figure 2) and maintained a somewhat low motivation toward using PBL. From
Table 4, it is observed that attending the professional training increased in-service
physics teachers' motivation to use PBL since there was a positive shift in all items.
In addition, findings in Table 5 indicate that the change was statistically
significant (p < 0.05) with a high effect size (d > 0.8). This positive effect could be
related to the fact that during the training, these teachers were exposed to what
PBL is all about, how it is implemented and assessed, and its advantages in
enhancing 'students' understanding of various concepts as well as being aware
that using PBL is actually not time-consuming as pointed out by Salam et al. (2009)
and Weizman et al. (2008).
Our participants indicated their willingness to continue using PBL during

teaching and follow the appropriate procedures for its implementation. This
agrees with Shakoor et al. (2013) in their study about the effect of in-service
training on science teachers' working capacity and performance at the secondary
level. They found that successful completion of in-service training has a positive
and far-reaching impact on professional competence as it improves science
curricula implementation and raises science 'teachers' attitudes towards the
teaching profession. In the same line, Dorimana et al. (2021) found out that most
participants, after attending the training, acknowledged having acquired
knowledge such as simulation with the PBL process, researching learning topics,
and active discussion of how research materials are applied to solve problems.
This knowledge acquisition significantly changed 'teachers' initial negative
perceptions of PBL and motivated them to apply its principles in real-life
scenarios (Zaidi et al., 2010). This motivation, according to Singh et al. (2014), was
such that most participants, after attending professional training, were interested
in changing their role from teacher to facilitator and were generally more
enthusiastic about adopting PBL. Talvio et al. (2016) also observed that if teachers
are motivated and feel competent in teaching a particular pedagogy, they will find
ways to include the necessary content in their teaching. Likewise, Iqbal et al.
(2020) collected data from TVET teachers in South Asia and found a significant
effect of in-service training and motivation on job performance.
Our results were in line with adult learning theory. Ndihokubwayo, Uwamahoro,
et al. (2020) trained teachers on the usability of PhET simulations and YouTube
videos for physics classrooms and later found the effect on 'students' performance
(Ndihokubwayo, Uwamahoro, et al., 2020a) and conceptual understanding
(Uwamahoro et al., 2021) of geometric optics. We can there say that the quality of
education can be improved by prioritizing teaching methods and how teachers
spend time in their classrooms, as observed by Junejo et al. (2018). Therefore, our
results imply that Ugandan secondary school teachers were unaware of PBL but
showed interest in its use. They should continuously implement it in the
classroom to improve students learning outcomes.
6. Conclusion and recommendations

The use of active learning methods, including PBL during instruction, has become
a norm at almost all levels of education across the globe. In this study, it was found
that teachers generally have low motivation for the use of PBL. However, those
282
teachers who attended the professional training in PBL (experimental group) had
increased knowledge of PBL concepts, were more competent in those PBL, had a
greater perceived value for PBL, and generally looked at implementing PBL as
being less costly, as compared those teachers who did not receive the professional
training (control group). This study was limited to only 50 in-service secondary
school physics teachers (30 in experimental and 20 in control groups) selected
from just four districts in southwestern Uganda, and the professional training in
PBL was limited to only two days. Therefore, it is recommended that more PBL
training should be regularly organized for all teachers, including school
administrators. Such pieces of training should be funded by the Government
where possible for the affordability of all. Teacher education institutions should
design their curricula in such a way as to prioritize advanced methods of teaching,
including PBL. It is hoped that the insights derived from the study should form a
valuable baseline for conducting a longitudinal study to determine the extent to
which PBL impacts student achievements, especially those in science, technology,
and mathematics.
Acknowledgement
Our heartful thanks go to our instructors for their constant guidance and
encouragement all through the way. We are so indebted to the management of
Ruhinda Secondary School for providing space and other equipment at no cost
for the training. Our African Center of Excellence for Innovative Teaching and
Learning Mathematics and Science is also acknowledged.
7. References
Carpendale, J., & Hume, A. (2020). Content representations to support out-of-field physics
teachers. Physics Education, 55(6), 065021.
https://doi.org/https://doi.org/10.1088/1361-6552/abaf16
Chien, C.-W. (2020). 'Undergraduates' Learning on Elementary School English Education
through Problem Based Learning in Scenario Analysis. Current Issues in Education,
21(1), 1–17.
https://acces.bibl.ulaval.ca/login?url=https://search.ebscohost.com/login.aspx
?direct=true&db=eue&AN=144942914&amp%0Alang=fr&site=ehost-live
Christiansen, E. T., Kuure, L., Morch, A., & Lindstraom, B. (2013). Problem-Based Learning
for the 21st Century (1st ed.). Aalborg University Press. https://doi.org/ISBN: 978-
87-7112-117-9
Creswell, J. W. (2014). Research Design (4th ed.). SAGE Publications Pvt. Ltd.
https://doi.org/ISBN: 978-1-4522-2609-5
Dorimana, A., Uworwabayeho, A., & Nizeyimana, G. (2021). Examining Mathematical
Problem-Solving Beliefs among Rwandan Secondary School Teachers.
International Journal of Learning, Teaching and Educational Research, 20(7), 227–240.
Iqbal, N., Khan, M. M., Mohmand, Y. T., & Mujtaba, B. G. (2020). The Impact of in-Service
Training and Motivation on Job Performance of Technical & Vocational Education
Teachers: Role of Person-Job Fit. Public Organization Review Volume, 20, 529–548.
https://doi.org/https://doi.org/10.1007/s11115-019-00455-3
Junejo, M. I., Sarwar, S., & Ahmed, R. R. (2018). Impact of in-service training on
performance of teachers a case of STEVTA Karachi region. International Journal of
Experiential Learning & Case Studies, 2(2), 50–60.
283
Kenner, C., & Weinerman, J. (2011). Adult learning theory: Applications to non-traditional
college students. Journal of College Reading and Learning, 41(2), 87–96.
https://doi.org/https://doi.org/10.1080/10790195.2011.10850344
Lee, H., & Blanchard, M. R. (2019). Why Teach With PBL ? Motivational Factors
Underlying Middle and High School Teachers' Use of Problem-Based Learning.
Interdisciplinary Journal of Problem-Based Learning, 13(1).
Lombardi, D., Shipley, T. F., Bailey, J. M., Bretones, P. S., Prather, E. E., Ballen, C. J., Knight,
J. K., Smith, M. K., Stowe, R. L., Cooper, M. M., Prince, M., Atit, K., Uttal, D. H.,
LaDue, N. D., McNeal, P. M., Ryker, K., St. John, K., van der Hoeven Kraft, K. J.,
& Docktor, J. L. (2021). The curious construct of active learning. Psychological
Science in the Public Interest, 22(1), 8–43.
https://doi.org/10.1177/1529100620973974
Mansor, A. N., Abdullah, N. O., Wahab, J. A., Rasul, M. S., Nor, M. Y. M., Nor, N. M., &
Amnah, R. (2015). Managing Problem-based Learning : Challenges and Solutions
for Educational Practice. Asian Social Science, 11(4), 259–268.
Mineo, R. M., Fazio, C., & Tarantino, G. (2010). An Inquiry-Based Approach to Physics
Teacher Education: the Case of Sound Properties. Eaching and Learning Physics
Today: Challenges. https://www.semanticscholar.org/paper/An-Inquiry-Based-
Approach-to-Physics-Teacher-the-of-Mineo-
Fazio/6026b6d2eb95cb1b80d1955982f27c1bfca9ff6c
Ndihokubwayo, K., & Murasira, G. (2019). 'Teachers' training college 'learners'
expectations for their future teaching. LWATI: A Journal of Contemporary Research,
16(2), 1–10.
https://doi.org/https://www.ajol.info/index.php/lwati/article/view/185978
Ndihokubwayo, K., Nyirigira, V., Murasira, G., & Munyensanga, P. (2020). Is
Competence-Based Curriculum well Monitored ? Learning from Rwandan Sector
Education Officers. Rwandan Journal of Education, 5(1), 1–12.
Ndihokubwayo, K., Uwamahoro, J., & Ndayambaje, I. (2020a). Effectiveness of PhET
simulations and YouTube videos to improve the learning of optics in Rwandan
secondary schools. African Journal of Research in Mathematics, Science and Technology
Education, 24(2), 253–265. https://doi.org/10.1080/18117295.2020.1818042
Ndihokubwayo, K., Uwamahoro, J., & Ndayambaje, I. (2020b). Usability of Electronic
Instructional Tools in the Physics Classroom. EURASIA Journal of Mathematics,
Science and Technology Education, 16(11), 1–10.
https://doi.org/https://doi.org/10.29333/ejmste/8549
Nikmah, I. L. (2018). The Implementation of Problem Based Learning to Improve
'Students' Learning Achievements. International Education & Research Journal, 4(3).
https://doi.org/E-ISSN No : 2454-9916
OYO, B., KALEMA, B. M., & BYABAZAIRE, J. (2017). MOOCs for in-service teachers: The
case of Uganda and lessons for Africa. Revista Española de Pedagogía, 75(1), 121–
141. https://doi.org/10.22550/rep75-1-2017-14
Prusko, P. T. (2021). Shaping the Futures of Learning in the Digital Age Helping STEM
Students Thrive: Adult Learning Pathways. Current Issues in Education, 22(1), 1–6.
Samsudin, A., Suhandi, A., Rusdiana, D., Kaniawati, I., Fratiwi, N. J., Zulfikar, A.,
Muhaemin, M. H., Hermita, N., Mansur, Wibowo, F. C., Supriyatman, Malik, A.,
& Costu, B. (2019). Optimizing Students' Conceptual Understanding on Electricity
and Magnetism through Cognitive Conflict-Based Multimode Teaching ( CC-
BMT ). Journal of Physics: Conference Series, 7(1204). https://doi.org/10.1088/1742-
6596/1204/1/012027
Shakoor, A., Ghumman, M. S., & Mahmood, T. (2013). Effect of In-Service Training on the
284
Working Capacity and Performance of Science Teachers at Secondary Level.

Journal of Educational and Social Research, 3(3), 337–342.
https://doi.org/10.5901/jesr.2013.v4n3p337
Singh, A., Saxena, A., Bhambani, P., Nema, S. K., Gaur, R., & Ambey, R. (2014). Faculty
Perception and Attitude on Problem Based Learning ( PBL ) in Medical College
from Central India. British Journal of Medicine & Medical Research, 4(9), 1836–1843.
https://doi.org/https://doi.org/10.9734/BJMMR/2014/4659
Sterling, D. R. (2013). How to Develop a Problem-Based Science Unit Manual. In Virginia
Initiative for Science Teaching and Acheivement. Virginia Initiative for Science
Teaching and Achievement (VISTA).
Sulaiman, F. (2010). 'Students' perceptions of implementing problem-based learning in a
physics course. Procedia-Social and Behavioral Sciences, 7(C), 355–362.
Talvio, M., Berg, M., Litmanen, T., & Lonka, K. (2016). The Benefits of 'Teachers'
Workshops on Their Social and Emotional Intelligence in Four Countries. Creative
Education, 07(18), 2803–2819. https://doi.org/10.4236/ce.2016.718260
Uganda National Examination Board. (2017). Uganda National Examinations Board Report
on Work of Candidates UCE 2016.
Ugandan Ministry of Education and Sport. (2014). Teacher Issues in Uganda: A shared vision
for an effective teachers policy. UNESCO - IIEP Pôle de Dakar.
https://doi.org/ISBN:978-92-803-1378-9
Uwamahoro, J., Ndihokubwayo, K., Ralph, M., & Ndayambaje, I. (2021). Physics 'Students'
Conceptual Understanding of Geometric Optics: Revisited Analysis. Journal of
Science Education and Technology, 30, 706–718. https://doi.org/10.1007/s10956-
021-09913-4
Weizman, A., Covitt, B. A., Koehler, M. J., Lundeberg, M. A., Oslund, J. A., Low, M. R.,
Eberhardt, J., & Urban-Lurain, M. (2008). Measuring 'Teachers' Learning from a
Problem-Based Learning Approach to Professional Development in Science
Education. Interdisciplinary Journal of Problem-Based Learning, 2(2).
https://doi.org/10.7771/1541-5015.1081
Weizman, A., Covitt, B., Koehler, M. J., & Lundeberg, M. (2008). Measuring Teachers'
Learning from a Problem-Based Learning Approach to Professional Development
in Science Education. The Interdisciplinary Journal of Problem-Based Learning, 2(2),
29. https://doi.org/10.7771/1541-5015.1081
Zaidi, Z., Zaidi, S. M., Razzaq, Z., Luqman, M., & Moin, S. (2010). Training workshops in
problem-based learning: changing faculty attitudes and perceptions in a Pakistani
medical college. Education for Health, 23(3), 440.
285
Appendix A. Teachers' survey on their motivation to Use PBL

Section A: Background information (Tick appropriate response)
School ownership: Government Private
School category: Single-girls Single boys Both girls and boys
School type: Day only Boarding only Both day and oarding
School location: Rural-based Urban-based
Gender: Female Male
Age (years): 20 – 24 25 – 29 30 – 34 35 – 39 40 and above
Teaching experience (years): <2 2–5 6 – 10 11 – 15 Above 15
Qualification: Certificate Diploma Bachelors Postgraduate
Section B: Concepts on motivation to use PBL

The scale interpretation: 1 = Strongly Disagree, 2 = Disagree, 3 = Somehow disagree,
4 = Somehow Agree, 5 = Agree, 6 = Strongly Agree
Item Response (Tick )

1 2 3 4 5 6
I have previously had PBL.
I have been teaching using PBL.
In a PBL classroom, the teacher functions as a facilitator, and
therefore, no content teaching is necessary.
PBL gives too much responsibility to students.
PBL is especially effective for students with low ability.
I will be able to implement PBL successfully.
I do not feel competent to teach with a PBL approach.
I may not persist with PBL if my students struggle.
I feel confident that I can successfully assess students' learning
progress in a PBL setting.
I am not sure that I can teach with PBL in ways that meet state
and district standards.
I am not interested in implementing PBL.
Teaching with PBL could be enjoyable.
Teaching well with PBL is important for my career.
Teaching with PBL is not important for my professional
growth
The skills that I gain by implementing PBL may be useful
beyond the classroom.
PBL does not help students to obtain a deeper understanding
of the content knowledge than they do in a traditional
classroom.
Using PBL causes students to have negative attitudes toward
learning
286
In PBL, students engage in issues relevant to their

lives/communities
PBL stimulates students' creativity
PBL enhances students' collaboration and communication
skills
PBL promotes students' critical thinking
I am concerned that PBL can lead to students missing out on
learning important basic concepts
Preparing to implement PBL would require too much of my
time
Implementing PBL will make classroom management more
difficult
It will be too stressful for me to cover the mandated
curriculum if I implement PBL
I worry that PBL might have a negative impact on how my
students score on the end-of-course tests
I am concerned that implementing PBL might have a negative
impact on my teaching evaluation
I believe that the overall benefits of implementing PBL would
outweigh the costs
There are not many people at work who are willing to help me
with implementing PBL
Appendix B. The two-day schedule for the professional training in PBL

Time Activity Facilitator Supporting
(Hours) materials
Day 1
8:00-8:30 Arrival and registration Research assistant Registration
Forms
8:30-9:00 Individual introduction All members Attendance
sheets
9:00-9:30 Opening remarks (sharing Training leader Powerpoint
training objectives) slides
9:30-10:00 Pretest All Participants Survey forms
10:00-10:30 Commercial Break All members
10:30-11:30 Origin of PBL Training Leader Powerpoint
slides
11:30-12:30 Importance of PBL in Teaching Training Leader Powerpoint
and Learning slides
12:30-13:00 Open discussion All members Flip charts
13:00-14:00 Lunch Break All members
14:00-15:00 Formulating a PBL question Training Leader Powerpoint
slides
15:00-16:00 Class-activity - on Formulating Facilitated groups Flip charts
PBL questions and SESEMAT
trainer
16:00-16:30 Summary of the day' 's Training leader Powerpoint
activities and closure slides
287
Day 2
8:00-8:30 Arrival and registration Research assistant Registration
forms
8:300-9:30 Steps followed in the Training Leader Powerpoint
presenting a PBL lesson slides
9:30-10:00 Class activity- drafting PBL Participants and Flip charts
lessons SESEMAT trainer
10:00-10:30 Commercial break All members
10:30-13:00 Group presentations on PBL Group secretaries Flip charts
lessons
13:00-14:00 Lunch break All members
14:30-15:00 Assessing a PBL lesson Training Leader Powerpoint
slides
15:00-15:30 Open discussion Participants Flip charts
15:30-16:00 Summary of the day's activities Training leader Powerpoint
slides
16:00-16:30 Post-test and closure Participants and Survey forms
Training leader
288

Vol. 21, No. 8, pp. 288-306, August 2022
Knowledge of Some Evidence-Based Practices

Utilized for Managing Behavioral Problems in
Students with Disabilities and Barriers to
Implementation: Educators' Perspectives
Hajar Almutlaq*
Majmaah University, Riyadh, Saudi Arabia
Abstract. Evidence-based practices, including applied behavior analysis,

have been used to manage behavioral problems among students with
disabilities. Educators have found a lack of utilizing empirical practices
in the classroom. The purpose of this study is to understand educators’
perspectives on practices used to manage behavioral problems among
students with disabilities and to determine barriers that prevent them
from utilizing evidence-based practices in the classroom. This study
examined the most effective learning opportunities rated by participants,
educators who were voluntarily recruited from Midwestern U.S. state
elementary public schools to complete an online survey about practices
used to manage behavioral problems among students with disabilities.
One hundred and seventeen educators (85 special education, 7 general
education teachers, and 25 other educators) completed a questionnaire
using Likert-type scales to describe their experience dealing with students
with disabilities who have exhibited behavioral problems. The results
indicated that there was an association between educators’ specialties and
their experience of the intensity of behavioral problems among students
with disabilities. The results showed a high percentage of educators rated
punishment as the most effective behavior management strategy among
a variety of behavioral management and Applied Behavior Analysis
strategies. The results showed that educators received more professional
training during their in-service as compared to their pre-service period.
Educators reported that shortages of supplies and support were the
barriers that most prevented them from utilizing evidence-based
practices in the classroom. This study highlighted the most effective
training methods preferred by educators, and implications and future
directions are provided.
Keywords: evidence-based practice; applied behavior analysis;

professional development; student behavioral problems; pre- and in-
service teachers
*
Corresponding author: Hajar Almutlaq, H.almutlaq@mu.edu.sa
©Author
289
1. Introduction
The number of students with disabilities has increased dramatically in the last
decades according to the National Center for Education Statistics [NCES] (2022a).
In the United States, educators reported that disruptions including non-
compliance, verbal and physical aggression, out of seat, and disrespecting
teachers have been the most common behavioral problems exhibited by students
in the classrooms (Education Advisory Board [EAB], 2019). Another report has
been released by NCES (2022b) displaying the percentage of students who
attacked their public-school educators; six percent of public-school educators
confirmed that they were attacked by a student from their school, and ten percent
reported that a student threatened them with injuries. About half of public-school
educators face a variety of behavioral problems displayed by their students on a
daily basis (Sutherland et al., 2019; Westling, 2010). These behavioral problems
may include, but are not limited to, aggression, yelling, crying, not complying
with teacher instructions, and off-seat and off-task behaviors (Almutlaq, 2021;
Amstad& Müller, 2020; Westling, 2010). A high percentage of behavioral
problems were exhibited by students with disabilities due to a number of reasons,
including inability to determine the acceptable social behavior, lack of appropriate
communication skills such as expressing their needs, and lack of self-management
skills such as controlling their temper tantrums (Amstad& Müller, 2020; Cooper
et al., 2019; Crone et al., 2015).
A report published by NCES about the discipline problems in public schools

(2022c) showed that approximately 50% of educators acknowledged that
students’ behavioral problems negatively affect the classroom environment, and
educators often spent more time trying to cope with such problems. Additionally,
educators pointed out that students displaying behavioral problems in the
classroom conflicted with their work during their attempt to manage the situation.
Approximately 40% of educators spend more time dealing with student
behavioral problems than they do teaching, according to Public Agenda ]PA[
(2004). When students engage in behavioral problems in the classroom, this leads
to a disruption in the learning process by negatively impacting the students
themselves, as well as their teachers and peers. Displaying behavioral problems
in the classroom forces teachers to take action to manage the behaviors; in
addition, actions presented to intervene in the problem behaviors of one student
can drive their peers to pay close attention to the situation. This eventually leads
to a disruption in the learning process. Behavioral problems place great stress on
teachers when they are trying to reduce these behaviors (McLean et al., 2019).
Students with Disabilities

The reauthorization of the Individuals with Disabilities Education Act [IDEA]
(2004) and the No Child Left Behind Act [NCLB] (2002) required highly qualified
teachers to meet students’ different individual needs; thus, there is an urgent need
for continuous teacher training and professional development. Inadequately
prepared teachers could fail to cope with behavioral problems among students
with disabilities (Alotaibi, 2015; Haimour & Obaidat, 2013; Westling, 2010).
Student teachers (pre-service teachers) during their education and before
undertaking any official teaching, as well as in-service teachers who are currently
providing learning, are both in need of sufficient training to prepare them in
290
facing the variety of behavioral problems among students with disabilities.

Educators have been encouraged to utilize Evidence-Based Practices (EBPs), such
as teaching and behavioral modification methods, over the decades.
Evidence-Based Practices
There is a wide range of Evidence-Based Practices (EBPs), and these practices have
been tested in many studies to prove their effectiveness in modifying behavioral
problems or improving learning outcomes. Using EBPs increases the chance of
implementing effective behavioral interventions or teaching methods, because the
selection of a specific practice should be based upon backup data, and a rigorous
process should be put into place (Pring & Thomas, 2004; Scheeler et al., 2016;
Stahmer et al., 2015). Some of the EBPs have been proven to be effective in
managing behavioral problems among students, such as differential
reinforcement, response cost, the token economy, and punishment and error
correction (Simonsen et al., 2008). In fact, these EBP practices and others used to
manage behavior problems, such as shaping, modeling, positive and negative
reinforcement, and prompting, are mainly derived from Applied Behavior
Analysis (ABA) principles (Cooper et al., 2019).
Applied Behavior Analysis

Applied behavior analysis (ABA) is defined as
“The science in which tactics derived from the principles of behavior are
applied to improve socially significant behavior and experimentation is
used to identify the variables responsible for the improvement in behavior.”
(Cooper et al., 2019, p.20).
There are many positive behavior management strategies derived from

ABA principles and used to enhance human behavior. ABA strategies are
believed to be the most effective EBP practices in the field of education
(Cooper et al., 2019; Wolf, 1978). ABA supports an individual's behavior in
six different ways, including: (1) teaching new skills, (2) increasing
appropriate behaviors, (3) maintaining behaviors, (4) changing or replacing
responses, (5) decreasing inappropriate behaviors, (6) generalizing or
transferring behavior from one situation to another (Cooper et al., 2019).
ABA is one of the most effective strategies used to manage students with
disabilities and behavioral problems, and has become used in school-wide
strategies all over the world (Behavior Analyst Certification Board [BCBA],
2014). The importance of utilizing ABA strategies has led to several studies
that explore teachers’ knowledge and their experience in utilizing ABA
strategies in the classroom (Reeves, 2017). Another study has investigated
the implementation barriers and preferred future training methods among
special education teachers (Alotaibi, 2015). Teachers rated the observing
and mentoring teachers implementing the exact same behavioral strategy
as the most effective learning opportunity that they could receive in a
training (Alotaibi, 2015). On the other hand, teachers were found to have a
lack of knowledge and utilization of ABA strategies (Alotaibi, 2015; Reeves,
2017), and lacked support and supervision to implement ABA and other
EBPs in the classroom (Alotaibi, 2015; Khaleel, 2019).
291
1.2. Significance of the study

After reviewing the previous studies, it is critical to understand to what extent
educators use EBP and ABA strategies as well as the barriers to its implementation
(Alotaibi, 2015; Reeves, 2017; Westling, 2010). Few studies have examined the
professional training received during the pre-service period and compared to this
the in-service period (Khaleel, 2019; Kodak et al., 2018; Schloss & Smith, 1998).
Knowing the professional training that teachers have received could help to focus
on the quantity and quality of designing a future training program for both pre-
service and in-service teachers. In addition, there is a need to design a suitable
training program for educators in Midwestern U.S. state public schools by
identifying their knowledge of EBPs implemented to manage students with
disabilities’ behavioral problems, the received training programs during their pre-
and in-service periods, the barriers, and the preferred learning methods.
This study aims to investigate the association between educators’ specialties and
their experience of the intensity of students’ problem behaviors. This study rates
the educators' perspectives of basic behavior management strategies, which are
EBPs, including ABA. The sufficiency of the training that pre-and in-service
educators receive to deal with students with behavioral problems is highlighted
in this study. This study identifies the possible barriers faced by educators that
prevent them from fully utilizing EBP practices, including ABA. In addition, this
study aims to effectively provide practical suggestions from the educators’
perspectives about their preferences for future learning opportunities, and to
provide future suggestions to support the skills needed to manage students’
behaviors. This study contributes to understand the educators’ perspectives, in
order to assist building a professional training program in the future that fits both
their abilities and their needs by considering the reported barriers and then by
employing the most effective learning methods during training. Therefore, this
study examines the following questions:
1. Is there an association between the participants’ specialties and their experience
of intense student behavioral problems?
2. What are the educators’ perspectives on utilizing basic behavior management
strategies, including applied behavior analysis?
3. Do pre-and in-service educators receive appropriate training to deal with their
students with behavioral problems?
4. What are the barriers that educators face that are relevant to managing their
students with behavioral problems?
5. What are some future learning opportunities that can be effectively used, from
the educators' perspectives?
2. Methodology
2.1. Research Design
To collect the needed data, this study used a questionnaire that employed a web-
based Qualtrics survey method to target participants. The questionnaire partially
adopted the questionnaires from previous studies by Alotaibi (2015) and Reeves
(2017) and were modified to suit the purpose of this study in order to gain more
information about the support needed by the educators in a Midwestern state in
the United States regarding behavior management strategies for their students
with disabilities.
292
2.2. Sample and Population

Participants in the study include 85 teachers in special education, seven general
education teachers, six principals and administrators, and 19 specialists in other
related services (such as school psychologists and special education directors).
The educators were recruited from different elementary public schools in a
Midwestern state in the United States and were constituted using convenience
sampling to represent a useful source of data in this exploratory research (Holton
& Burnett, 2005). The study gathered information about the support needed by a
certain population and generalized the findings. All of the participants were
volunteers who were entered into a draw to get a monetary reward for completing
the questionnaire. More information about participants is provided in Tables 1
and 2. (Table 1 displays gender, ethnicity, professional position, years of
experience, education level, school district, school location; Table 2 shows type of
classroom, number of students exhibiting behavioral problems, and number of
teachers in the classroom.)
Table 1: Demographic information of the participants (n = 117)

Variables N Percentage
Gender
Female 109 93.2
Male 8 6.8
Ethnicity/Race
African 6 5.1
American
Hispanic/Latino 2 1.7
Native 9 7.7
American
White 100 85.5
Professional Position
Gen. Ed. 7 5.2
Teacher
Special Ed. 85 73.3
Specialists
Related services 19 16.2
specialists
Admin/ 6 5.2
Principal
Years of Experience
0-1 years 2 1.7
2-3 8 6.8
4-5 3 2.6
More than 5 104 88.9
years
Academic Degree
Doctor 2 1.7
Master 68 58.1
Bachelor 47 40.2
School District
Urban 40 34.5
Suburban 35 30.2
Rural 42 35.3
293
Location of School
Central 47 40.5
East 12 10.4
West 14 11.3
North 12 10.4
South 32 27.4
Table 2. Summary of educational units, number of students and teachers in classroom

Variables N Percentage
Type of the Classroom
Gen. Ed. Classroom 13 11.1
Self-Contained 15 12.8
Autism Class 3 2.6
Resource Room 47 40.2
Severe Profound 2 1.7
Other 37 31.6
Number of Students Exhibiting Behavioral Problems Per Class
1-3 33 28.7
4-7 40 34.8
8-9 17 14.8
More than 10 27 21.7
Number of Teachers
1 49 41.8
2 8 6.8
3 6 5.1
4 4 3.4
6 and more 33 28.2
2.3. Research instruments

The questionnaire employed in this study was partially adopted from studies by
Alotaibi (2015) and Reeves (2017), in order to meet the purpose of this study by
allowing participants to report their perspectives on students’ behavior
management strategies, including using some ABA. In addition, this
questionnaire provided participants with an opportunity to share their preferred
practical suggestions and future learning opportunities to support the skills
needed to manage students’ behaviors. A questionnaire entitled “Educators'
Knowledge in a Midwestern State”, was initially created using Qualtrics online
software and allowed participants to report their perceptions, through Likert-type
ratings, on four dimensions related to the educators' perspectives of behavioral
management strategies, evaluating basic behavior management strategies
including ABA, barriers to its implementation, and suggestions for future
learning opportunities. Some of the questionnaire items were illustrated in Table
3.
294
Table 3. Information about some sections in the questionnaire requiring participant

responses
Dimension No. of Examples or Response Type

Tittle Items Description of
Items
It is important to 5-point scale ranged
collect data on from: (1. strongly
Section 2
inappropriate disagree -
Educators' behaviors 5. strongly agree)
perspective Provide class-wide 5-point scale ranged
of behavioral rewards when the from: (1. always - 5.
management Section 2 class as a whole never)
strategies Item No. demonstrates good
5 behavior (e.g., extra
recess time, pizza
party)
3-point scale ranges
Barriers to from: (1. never a
implement Section 3 Feedback/guidance barrier to my use of
behavioral Item No. for using the behavior - 3. very
management 3 behavior strategy often a barrier to my
strategies use of a behavior
strategy)
4-point scale ranged
from: (1. I never
experienced this
Watching and
Suggestions learning opportunity
Section 4 reflecting on videos
for future in my teacher training
Item No. that show the
learning or any other training -
7 implementation of
opportunities 4. I experienced the
behavior strategies.
learning opportunity
and it was very
effective)
Information about sections of the questionnaire requiring participant responses.

The questionnaire was divided into four sections:
1) Section 1: Demographic Information
In this section, questions were asked about gender, ethnicity, position, years of
experience, academic degree obtained, rating the severity of students’ problem
behaviors that the respondents have encountered, and educators’ formal training
in managing students’ problem behaviors and ABA.
2) Section 2: Basic Behavior Management Strategies Including Applied Behavior
Analysis
In this section, there were three subsections including behavior management
strategies, applied behavior analysis practices, and the effectiveness of a variety
of behavior management strategies and ABA practices. The respondents
answered the questions regarding their opinions toward their knowledge upon
using basic behavior management strategies. The scales were ranked from
strongly agree, agree, neither agree nor disagree, and disagree, to strongly
disagree.
3) Section 3: Possible Barriers to the Use of Behavior Management Strategies
295
In this section, possible barriers to the use of behavior management strategies that
the respondents faced were asked, and the rating scale was never, rarely,
occasionally, often, and very often.
4) Section: Learning Opportunities
In this section, the respondents were asked to evaluate the effectiveness of
learning opportunities and rated them as never experienced, experienced but it
was not effective, it was somewhat effective, or it was very effective.
2.3.1 The Validity of The Questionnaire

Alotaibi (2015) and Reeves (2017) reported that their questionnaires were created
based on a review of relevant literature in the evidence-based practices. Alotaibi
(2015) reviewed and revised the questionnaire with faculty members in the field
of special education, while Reeves (2017) used a modified questionnaire found in
previous studies (Martin & Baldwin, 1993; Martin et al., 2007; McCormick, 2011;
Musgrove,1974; Randazzo, 2011; The Incredible Years, 2012). Thus, the modified
questionnaires used in this study were developed and validated in previous
studies (Alotaibi, 2015; Reeves, 2017).
2.4 Content validity

The draft questionnaire was created based on a review of the relevant literature
(Alotaibi, 2015; Martin & Baldwin, 1993; Martin et al., 2007; McCormick, 2011;
Musgrove, 1974; Randazzo, 2011; Reeves, 2017; The Incredible Years, 2012). The
questionnaire was sent to collaborators, parties, and a faculty member in the
special education department who were experts in the field of behavior
management strategies and applied behavior analysis was used to review the
survey's items. The reviewers provided a few recommendations regarding the
survey’s duration and terminology, and the author revised the survey's items
based on their comments.
2.5 Procedures
The recruitment process for participants began by using convenience sampling.
Cooperation between the author and other professional agencies was established
to deliver the questionnaire to potential participants who met the inclusion
criteria and who volunteered to respond to the survey items. The survey's link
was sent to the Director of Special Education Teachers in a Midwestern state, and
the survey's link was sent to an Educators List found in the Special Education
Support Center in the authors' university. Because of the length of the
questionnaire, educators who volunteered to complete it were told that they
would be entered into a draw, and that five of them would be paid $10 for the
time required to complete the survey. Educators who were willing to volunteer
were asked to sign a consent form on the first page in order to proceed to the
following items of the survey. The participants were not asked to provide names
or any other personal information, in order to protect their personal privacy. All
of the questionnaires were distributed and collected during September through
October 2021.
2.6 Data Analysis

Data were exported from the online Qualtrics survey software into SPSS (version
21) for statistical analysis. Qualtrics and SPSS software were used to descriptively
296
analyze the data. Incomplete responses and information from respondents who
declined to participate in the study were excluded.
3. Results
3.1.1 Demographic Information of The Participating Educators
Tables 1 and 2 present the demographic characteristics of the participants. The
majority of the participants (93%) were female and 85% were white. Most of the
participants (73%) were special education teachers, 89% with experience of over
five years, and 58% of the participants had earned a master degree.
Approximately 47% of the educational units were resource rooms, with about 35%
reporting an estimate of 4 to 7 students per classroom.
3.1.2 Participants’ Specialty and The Intensity of Student Behavior Problems

A chi-square test was performed to examine the relationship between the
specialties of the educators and the intensity of the students' problem behaviors
experienced by those educators. The relation between these variables was
significant, at 235.148. The p-value is < 0.00001 and the result is significant at p <
.05. There was significant association between educators' specialties and their
experience of rating their students' behavioral problems intensity. More details
can be found in Table 4 and Figure 1.
Table 4. Percentage of participants’ specialties rating the intensity of student behavior

problems
Special General Related Psychol-
Intensity
education education services gists or
of problem Total
teachers teachers teams (n directors
behavior
(n = 85) (n =7) =6) (n =19)
Severe 46% 43% 8% 3% 100%
Moderate 40% 50% 86% 22% 198%
Mild 14% 7% 6% 75% 102%
Total 100% 100% 100% 100% 400%
The intensity of student

Psychologists or directors (n = 19) behavior problems
Educators Speciality
Related services specialists (n = 6)

Severe
Moderate
General education teachers (n = 7)
Mild
Special education teachers (n = 85)
0% 20% 40% 60% 80% 100%
Figure 1. Educators’ specialty and the intensity of student behavior problems
3.1.3 Participants’ Perspective of Basic Behavior Management Strategies Including

Some of Applied Behavior Analysis Strategies
Table 5 shows the percentage of educators who rated the items regarding the basic
behavior management strategies including applied behavior analysis as strongly
297
agreed or agree. As can be seen in Table 5, the majority (96%) of the educators
indicated that they considered punishment to be an effective behavioral
management strategy, with negative punishment at 91%. It is noteworthy that
educators rated making intervention plans and reinforcing specific positive
behavior as strongly agreed or agreed by a majority of 87%. A range of 40% to
48% of the participants rated the items regarding behavior management
strategies, such as differential reinforcement, ignoring, and interdependent group
rewards, as strongly agreed or agree. The percentage of educators who reported
strongly agreed or agreed about the basic behavior management strategies
including applied behavior analysis in specific strategies such as positive
reinforcement and negative punishment (time-out) was low, at 4% to 6%.
Table 5. Percentage of participants who strongly agreed or agreed with items from
different sections of the questionnaire
Specific items Educators (n = 117)
Making intervention plans for repeated behavioral 87
problems
Collecting data on inappropriate behaviors 83
Understanding the underlying cause of a 75
student’s behavioral problem
Positive Reinforcement 4
Reinforcing specific positive behavior 87
Reinforcing students who are following the 11
expected rules in the classroom
Differential reinforcement 48
Differential reinforcement of alternative behavior 76
Identifying preferred rewords to a student 85
Token Economy 29
Ignoring student’s behavioral problem when it is 40
possible
Punishment (as the most effective way to change a 96
behavior)
Negative punishment 91
Negative punishment (time-out) 6
Interdependent group rewards 41
Note: Items from questionnaire are paraphrased.
3.1.4 Pre-and In- Service Teachers Training

Table 6 shows that the majority of educators (63%) had taken behavior
management courses prior to graduation from school. Pre-service teachers who
had not received training in appropriate behavior management strategies such
as ABA were at 61%, compared to those in their in-service period at 56%.
Table 6. Summary of responses on the professional development section by pre-and

in-service teachers
Professional Development Yes No
Behavior Management Course 63.2% 36.8%
Pre-service training in ABA 39.3% 60.7%
In-service training in ABA 43.6% 56.4%
298
3.1.5 Barriers for In-Service Educators

Table 7 shows that the most frequent barrier that teachers experience (18%) was
the supplies needed to use a behavior management strategy. The second most
significant barrier that teachers experience was support from colleagues (15%) to
use a behavior management strategy, followed by support from administrators
(14%). Colleagues’ experimentation found to be ineffective was considered as a
barrier by 12%. On the average, teachers reported that all the barriers somehow
affected teacher strategies to cope with student problem behavior or to apply
behavior management strategies.
Table 7. Frequencies of barriers faced by educators

Items Options
Never Sometimes Very often
1. My knowledge or skill 41 (35.3%) 69 (59.5%) 7 (5.2%)
regarding the behavior
strategy
2. Support from administration 55 (47.0%) 45 (38.5%) 17 (14.5%)
for using the behavior
strategy
3. Feedback/guidance for 49 (41.9%) 56 (47.9%) 12 (10.3%)
using the behavior strategy
4. Supplies for using the 28 (24.1%) 67 (57.8%) 21 (18.1%)
behavior strategy
5. Support from colleagues to 36 (31.3%) 61 (53.0) 18 (15.7%)
use the behavior strategy
6. My general approach does 63 (53.8%) 52 (44.4%) 2 (1.7%)
not help me
7. Takes too much time and 57 (48.7%) 52 (44.4%) 8 (6.8%)
effort to use the behavior
strategy
8. Perception that the behavior 49 (42.2%) 59 (50.9%) 8 (6.9%)
strategy is not effective
9. Other teachers have tried the 53 (45.3%) 49 (41.9%0 15 (12.8%)
behavior strategy and they
thought it was ineffective
10. The behavior strategy would 60 (51.7%) 51 (44.0%) 5 (4.3%)
not help me achieve my
work goals
11. The behavior strategy was 42 (35.9%) 63 (53.8%) 12 (10.3%)
not appropriate for the
students in my class(es) and
their specific problems
12. Other factors that influence 41 (39.0%) 61 (58.1%) 3 (2.9%)
your use of the behavior
strategies described in this
survey (explain and rate
those factors)
299
3.1.6 Future Learning Opportunities

Almost 47% teachers reported that observing mentors, or supervisors dealing
with behavior problems using behavior strategies offered the most effective
learning opportunities. The second greatest learning opportunity (40%) was
receiving feedback in implementing behavior strategies. Two learning
opportunities, completing a portfolio related to behavior (5%) and using other
instructional techniques (5%), were found to be the least learning effective
opportunities, as shown on Table 8.
Table 8. Frequencies of learning opportunities rated by educators

Items Options*
1 2 3 4
1 Reviewing case studies of 10 (8.8%) 13 (11.5%) 54 (47.8%) 36 (31.9%)
students with behavioral
problems
2 Listening to lectures about 11 (9.7%) 21 (18.6%) 58 (51.3%) 23 (20.4%)
behavior strategies
3 Reading books/articles about 6 (5.3%) 15 (13.3%) 68 (60.2%) 24 (21.2%)
behavior strategies
4 Observing mentors, teachers, 16 (13.7%) 6 (5.3%) 37 (32.7%) 54 (47.8%)
or supervising teachers
dealing with behavior
problems using these
behavior strategies
5 Receiving feedback on how I 23 (20.4%) 9 (8.0%) 35 (31.0%) 46 (40.7%)
implement behavior
strategies
6 Reflective journal writing 44 (38.9%) 23 (20.4%) 28 (24.8%) 18 (15.9%)
about behavior strategies
7 Watching and reflecting on 22 (19.6%) 13 (11.6%) 50 (44.6%) 27 (24.1%)
videos that show the
implementation of behavior
strategies
8 Role-playing scenarios about 35 (31.0%) 24 (21.2%) 35 (31.0%) 19 (16.8%)
using behavior strategies
9 Completing group projects 41 (36.3%) 27 (23.9%) 37 (32.7%) 8 (7.1%)
related to behavior strategies
10 Completing portfolios 62 (54.9%) 23 (20.4%) 22 (19.5%) 6 (5.3%)
related to behavior
management
11 Other instructional 62 (54.9%) 23 (20.4%) 22 (19.5%) 6 (5.3%)
techniques you’ve
experienced (describe them
and rate them)
*(1) I never experienced this learning opportunity in my teacher training or any other training. (2) I experienced the learning
opportunity, but I don’t feel the learning opportunity was effective. (3) I experienced the learning opportunity and was
somewhat effective. (4) I experienced the learning opportunity, and it was very effective.
3.2 Reliability Measures

Cronbach’s alpha is a measure of internal consistency reliability (Cronbach, 1951).
Cronbach’s alpha was used to assess the internal consistency of the questionnaire
items, based on the results of the participants in the present study. The internal
300
consistency of the four sections in the survey was α = 0.720 to 0.840, as seen in
Table 9, based on which an acceptable to good level of reliability is indicated
(George & Mallery, 2003).
Table 9. Reliability analysis

Scale Items Number of Cronbach
items Alpha
Section 2 (A): Basic Q16-Q29 14 .720***
Behavior Management
Strategies Including
Applied Behavior
Analysis
Section 2 (B): Basic Q30-Q36 7 .787***
Behavior Management
Applied Behavior
Analysis
Section 2 (C): Basic Q37-Q43 7 .742***
Behavior Management
Applied Behavior
Analysis
Section 3: Barriers Q44-Q55 12 .792***
Section 4: Learning Q56-Q66 11 .840***
Opportunities
***p  0.7
4. Discussion
The first research question concerned the relationship between the participants’
specialty and their experiences of intensive student behavior problems. In this
study, a positive relationship was found between participants’ expertise and their
experience of more student behavior problems. Consistent with other studies
(McLean et al., 2019; Sutherland et al., 2019; Westling, 2010), who found that half
of the educators were struggling with students with behavioral problems, over
46% of the participants with a specialty in special education reported that they
experienced a severe level of student behavior problems in the classroom, while
50% of the general educator participants reported that they experienced a
moderate level of students with behavioral problems in the classroom. Special
education teachers tended to have more students with severe disabilities and
problem behaviors in their classroom. Students with mild to moderate disability
and behavioral problems are more likely to be integrated into the regular
classroom with typically developing peers. Eighty-six percent of related services
team members reported they had experienced a moderate level of students’
problem behaviors. Related service providers such as psychologists and special
education directors often are present in the classroom for a short period of time
for supervision purposes (Scheeler et al., 2016). Due to the lack of interaction
between students and related services teams on a daily basis, specialists may have
an insufficient description of students’ behaviors, which require frequent
observation and recording data to describe accurately.
301
The second research question highlighted the educators’ perspective upon

utilizing some of the basic behavior management strategies, including ABA. The
result indicated that 90% and above of the participants reported punishment as
an effective behavioral management strategy. The result was consistent with other
studies that indicated that teachers preferred punishment (Almutlaq, 2021;
Alotaibi, 2015; Khasawneh, 2020; Koh & Shin, 2014; Reeves, 2017).
Teachers in the US and in other developing countries still favor punishment as a

way to cope with an undesirable behavior among students by removing a
favorable stimulus. This results in a decrease of the likelihood of a behavior
occurring; however, ethical considerations need to be considered before using
punishment (Cooper et al., 2019). In fact, students have the right to safe and
effective behavioral treatment and it is the professionals’ responsibility to use the
least restrictive procedures. Some teachers may mistake identifying the suitable
behavioral intervention for a student's behavior problem and implement a sort of
punishment as a primary strategy. On the other hand, the results indicated that,
at 87%, a high percentage of educators reported making intervention plans and
reinforcing specific positive behavior. This showed that educators agreed with the
significance of developing behavioral intervention plans for students in need and
also showed that they supported the positive behavior strategies. This is
consistent with Alotaibi (2015), who found that a high percentage of teachers rated
training on developing behavioral intervention plans as a desired future training.
Educators believe in the importance of developing intervention plans which can
be suitable for a specific behavioral problem that a student might have. Koh and
Shin’s (2014) study found that teachers in the US are directed to implement
positive behavior management strategies, similar to the result arrived at in this
study, which showed that a high percentage of educators rated reinforcing
specific positive behavior as desirable.
The third research question highlighted pre- and in-service training on ABA and
behavior modification courses that educators had received. Most of the
participants reported that they had enrolled in behavior modification courses
during their graduate or undergraduate academic programs, although 71% of pre-
service and 66% of in-service educators reported that they had not received
training in ABA. The findings are consistent with other studies that recommended
providing educators with appropriate training on empirical and EBPs (Almutlaq,
2021; Scheeler, et al., 2016; Westling, 2010) such as ABA to cope with student's
behavioral problems (Khaleel, 2019). Studies indicate that training teachers to
utilize EBP through implementing behavioral interventions or teaching methods
can decrease the level of stress and increase their skills to successfully manage the
classroom environment (Khaleel, 2019; Scheeler et al., 2016; Westling, 2010).
Participants reported some of the barriers that they face in-service relevant to
managing students with behavioral problems. The results indicated that 18% to
57% of the participants reported that a lack of supplies and professional support
in their school is, sometimes to very often, a barrier that prevents them from
effectively managing students with behavioral problems in the classroom. This
could be a significant reason that prevents educators from implementing EBP
302
strategies. Lack of support from colleagues and administrators was rated as a

frequent barrier by 15% of the participants. This is consistent with Scheeler et al.
(2016) who found that administrators and supervisors observe educators twice a
year only for the purpose of the teachers’ evaluation, and this results in educators
reporting a lack of feedback and support from professionals; hence, they are more
likely to be unmotivated to implement EBP.
The fifth research question concerned the future learning opportunities, and
educators reported the efficacy of each learning opportunity. Approximately half
of the participants who experienced observing professionals dealing with student
problem behaviors rated this item as a very effective learning opportunity. This
indicates that observing other professionals who are modeling appropriate
responses to cope with students with behavioral problems could be an
appropriate training method used to teach educators Forty-two percent of
participants reported that receiving feedback on how to implement a behavior
strategy was a very effective learning opportunity that they had experienced. In
fact, educators reported a lack of supervision and feedback to implement EBP
successfully (Scheeler et al., 2016). The results indicate that educators prefer direct
training such as modeling and receiving direct feedback and support to
implement an evidence-based practice such as ABA. Among other learning
opportunities, such as listening to lectures or reading books about behavior
management strategies, 50%-53% of participants reported these as moderately
efficacious learning strategies. Also, 48% to 42% of participants reported that
reviewing case studies, watching, and reflecting on videos that show the
implementation of behavior strategies were somewhat effective learning
opportunities that they had experienced. However, 35% of participants reported
that they had never experienced working in a group to complete a project related
to behavior strategies, while a similar percentage had experienced that and
reported this strategy as a moderate learning opportunity. In addition, 31% of
participants reported that they had never experienced role-playing scenarios
about using behavior strategies. Results suggest that those educators can
effectively learn from direct observation, feedback, and support provided by
professionals, while less interactive learning strategies, such as listening to
lectures, reading books, and reflecting on case studies, were deemed to be
insufficient training strategies.
5. Conclusion
EBPs are proven to be successful in managing student behavioral problems
including ABA strategies (Amstad & Müller, 2020; Scheeler et al., 2016; Westling,
2010). Special education teachers are in need of additional support and guidance
to implement EBPs to successfully cope with students with behavioral problems
and to build a safe and effective learning environment for all. Professional
development and training provided for educators during their pre- and in-service
periods are recommended, to enhance teachers’ knowledge and skills to fully
utilize EBPs. Barriers and future learning opportunities rated by educators should
be considered in order to provide suitable and effective training and support.
303
5.1. Limitations
There are several limitations that affect this study. First, it was conducted in a
specific geographic location in a Midwestern state in the United States with a non-
random, convenience sample. Thus, it is difficult to determine whether or not the
population of respondents is an appropriately representation due to the
specificity of the geographic location and its inability to be generalized to
educators across different geographical areas. Another limitation which must be
considered is that educators may interpret some of the items in the survey
differently, even though the reliability of this study was computed. However,
these results are also useful, since they provide an overview of the educators’
understanding of EBP and applied behavior analysis, and they determine the
barriers that prevent educators from utilizing EBP in the classroom. These results
identify possible future learning opportunities that educators rate as effective
training methods. These results assist in building a suitable training program
based on the educators’ needs in a particular area.
5.2. Implications of the Study

According to the results in this study, it is proven that special education teachers
experience a high to severe level of students’ behavioral problems. This study
indicates the need, for special education teachers in particular, for support and
supervision to utilize EBP practices in the classroom. In addition, this study
explicitly describes those educators who reported that the lack of support and
supervision are barriers that prevent them from appropriately utilizing EBPs in
the classroom. Thus, the results indicate the specific needs for direct support and
supervision that the educators need in order to utilize EBPs in their classrooms.
This study showed that pre- and in-service teachers receive insufficient training
on EBP, which leads to inappropriate utilization of EBP in the classroom. This
study provides implications that can be considered by policy makers and
researchers to improve teacher support programs. In fact, these results provide
specific training methods that are rated as effective by educators, such as training
participants using group strategies, including role-playing scenarios and
completing group projects.
5.3. Future directions

These results only provided an exploration of the perspectives on the EBP among
educators in a particular geographic location. Future study could be done to
determine the universal support needed by educators across a wide variety of
locations. Future teacher training programs could utilize direct and live training
methods in order to acquire and to generalize the target skills.
Acknowledgement
The author extend her appreciation to the deputyship for Research & Innovation,
Ministry of Education in Saudi Arabia for funding this research work through the
project number (IFP-2020-19).
6. References
Almutlaq, H. (2021). Saudi special educators' perceptions of applied behavior analysis for
students with autism. World Journal of Education, 11(4), 18-30.
https://doi.org/10.5430/wje.v11n4p18
304
Alotaibi, A. A. (2015). Knowledge and use of applied behavior analysis among teachers of students
with autism spectrum disorder in Saudi Arabia (Doctoral thesis, Washington State
University Pullman, WA).
https://rex.libraries.wsu.edu/view/pdfCoverPage?instCode=01ALLIANCE_W
SU&filePid=13349972000001842&download=true
Amstad, M., & Müller, C. M. (2020). Students' problem behaviors as sources of teacher
stress in special needs schools for individuals with intellectual
disabilities. Frontiers In Education, 4,159.
https://doi.org/10.3389/feduc.2019.00159
Behavior Analyst Certification Board. (2014). Professional and ethical compliance code for
behavior analysts. Littleton. https://doi.org/10.1016/b978-0-12-815914-9.00027-4
Cooper, J. O., Heron, T. E., & Heward, W. L. (2019). Applied behavior analysis (3rd edition).
Hoboken, NJ: Pearson Education.
Cornett, J., & Knight, J. (2009). Research on coaching. Coaching: Approaches and perspectives
(pp. 192-216). Thousand Oaks, CL: Corwin Press.
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika,
16(3), 297-334.
http://psych.colorado.edu/~carey/courses/psyc5112/readings/alpha_cronbac
h.pdf
Crone, D. A., Hawken, L. S., & Horner, R. H. (2015). Building positive behavior support
systems in schools: Functional behavioral assessment. New York, NY: Guilford.
Education Advisory Board. (2019). Educators report growing behavioral issues among young
students. https://eab.com/insights/press-release/district-leadership/press-
release-growing-behaviorial-disruptions/
George, D., & Mallery, P. (2003). SPSS for windows step by step: A simple guide and reference
(4th ed.). Boston: Allyn & Bacon.
Haimour, A. I., & Obaidat, Y. F. (2013). School teachers' knowledge about autism in Saudi
Arabia. World Journal of Education, 3(5), 45-56.
http://dx.doi.org/10.5430/wje.v3n5p45
Holton, E. F., & Burnett, M. F. (2005). Research in organizations: Foundations and methods of
inquiry. Berrett-Koehler Publishers.
https://www.bkconnection.com/static/Research_in_Organizations_EXCERPT.
pdf
Individuals with Disabilities Education Act, 20 U.S.C. § 1400 (2004)
Khaleel, Y. F. (2019). Assessing the knowledge level of teachers of children with autism
spectrum disorder about the importance of applied behavior analysis (ABA)
strategies in Zarka city. International Education Studies, 12(5), 120-132.
https://doi.org/10.5539/ies.v12n5p120
Khasawneh, M. A. S. (2020). The extent of bullying against students with learning
disabilities according to the age variable. International Journal of Learning, Teaching
and Educational Research, 19(6), 267-281. https://doi.org/10.26803/ijlter.19.6.16
Kodak, T., Cariveau, T., LeBlanc, B. A., Mahon, J. J., & Carroll, R. A. (2018). Selection and
implementation of skill acquisition programs by special education teachers and
staff for students with autism spectrum disorder. Behavior Modification, 42(1), 58-
83. https://doi.org/10.1177/0145445517692081
Koh, M. & Shin, S. (2014). A comparative study of elementary teachers’ beliefs and
strategies on classroom and behavior management in the USA and Korean school
systems. International Journal of Progressive Education, 10(3), 18-33.
https://dergipark.org.tr/en/download/article-file/258546
Martin, N. K., & Baldwin, B. (1993, April 12-16). Validation of an inventory of classroom
management style: Differences between novice and experienced teachers. Paper
305
presented at the Annual Meeting of the American Education Research

Association, Atlanta, GA. https://files.eric.ed.gov/fulltext/ED359240.pdfer
Martin, N. K., Yin, Z., & Mayall, H. (2007). The attitudes & beliefs on classroom control
inventory-revised and revisited: A continuation of construct validation. Journal of
Classroom Interaction, 42(2), 11-20. https://files.eric.ed.gov/fulltext/EJ829001.pdf
McCormick, J. A. (2011). Inclusive elementary classroom teacher knowledge of and attitudes
toward applied behavior analysis and autism spectrum disorder and their use of applied
behavior analysis. (Doctoral dissertation, Dowling College, Brookhaven, NY).
https://search.proquest.com/openview/929829c2e61ec734e25f69319944c67a/1?
pq-origsite=gscholar&cbl=18750
McLean, D., Eklund, K., Kilgus, S. P., & Burns, M. K. (2019). Influence of teacher burnout
and self-efficacy on teacher-related variance in social-emotional and behavioral
screening scores. School Psychology, 34(5), 503.
https://doi.org/10.1037/spq0000304
Musgrove, W. J. (1974). A scale to measure attitudes toward behavior modification.
Psychology in the Schools, 11, 392-396. https://doi.org/10.1002/1520-
6807(197410)11:4%3C392::aid-pits2310110403%3E3.0.co;2-l
National Center for Education Statistics. (2022a). Discipline problems reported by public
schools. https://nces.ed.gov/programs/coe/indicator/a07/discipline-
problems?tid=4
National Center for Education Statistics. (2022b). Students with disabilities.
https://nces.ed.gov/programs/coe/indicator/cgg.
National Center for Education Statistics. (2022c). Teachers threatened with injury or physically
attacked by students. https://nces.ed.gov/programs/coe/indicator/a05
No Child Left Behind Act of 2001, P.L. 107-110, 20 U.S.C. § 6319 (2002).
Pring, R., & Thomas, G. (2004). Evidence-based practice in education. McGraw-Hill Education
(UK).
Public Agenda. (2004). Teaching interrupted. do discipline policies in today's public schools foster
the common good? http://www.publicagenda.org
Randazzo, M. E. (2011). Elementary teachers’ knowledge and implementation of applied behavior
analysis techniques. (Doctoral dissertation, Rutgers University, New Brunswick,
NJ). https://rucore.libraries.rutgers.edu/rutgers-lib/33413/pdf/1/
Reeves, A. (2017). Teacher perceptions, use, and knowledge of applied behavior analysis-based
techniques (Doctoral dissertation, Appalachian State University, Boone, NC).
https://core.ac.uk/download/pdf/345084801.pdf
Scheeler, M. C., Budin, S., & Markelz, A. (2016). The role of teacher preparation in
promoting evidence-based practice in schools. Learning Disabilities: A
Contemporary Journal, 14(2), 171-187.
https://files.eric.ed.gov/fulltext/EJ1118433.pdf
Schloss, P., & Smith, M. (1998). Applied behavior analysis in the classroom. Boston, MA: Allyn
and Bacon. https://www.jstor.org/stable/41824077
Simonsen, B., Fairbanks, S., Briesch, A., Myers, D., & Sugai, G. (2008). Evidence-based
practices in classroom management: Considerations for research to practice.
Education and treatment of children, 351-380. https://doi.org/10.1353/etc.0.0007
Stahmer, A. C., Reed, S., Lee, E., Reisinger, E. M., Connell, J. E., & Mandell, D. S. (2015).
Training teachers to use evidence-based practices for autism: Examining
procedural implementation fidelity. Psychology in the schools, 52(2), 181–195.
https://doi.org/10.1002/pits.21815
Sutherland, K. S., Conroy, M. A., McLeod, B. D., Kunemund, R., & McKnight, K. (2019).
Common practice elements for improving social, emotional, and behavioral
306
outcomes of young elementary school students. Journal of Emotional and Behavioral

Disorders, 27(2), 76-85. https://doi.org/10.1177/1063426618784009
The Incredible Years (2012). Teacher classroom management strategies questionnaire.
https://incredibleyears.com/?wpfb_dl=535
Westling, D. L. (2010). Teachers and challenging behavior. Remedial and Special Education,
31(1), 48–63. https://doi.org/10.1177/0741932508327466
Wolf, M. M. (1978). Social validity: the case for subjective measurement or how applied
behavior analysis is finding its heart 1. Journal of Applied Behavior Analysis, 11(2),
203-214. https://doi.org/10.1901/jaba.1978.11-203
307

Vol. 21, No. 8, pp. 307-325, August 2022
Exploring Virtual Reality-based Teaching

Capacities: Focusing on Survival Swimming
during COVID-19
Yoo Churl Shin
Gachon University, Seongnam, South Korea
Chulwoo Kim
Gachon University, Seongnam, South Korea
Abstract. This study aims to analyse teaching capacities exhibited by

survival swimming instructors applying virtual reality (VR) devices in
the education field. We conducted in-depth interviews with swimming
instructors to obtain research data, which was then qualitatively
analysed. Based on the research results, the following capacities were
derived. First, as the VR and simulator-based survival swimming classes
utilise educational equipment, the ability to skilfully handle educational
equipment is considered a vital teaching capacity. Second, strong
communication skills are required to accurately explain the class
objectives and contents. To achieve the class goals, teachers should
practice using educational equipment before class, answer students'
queries during and after practice, and prepare the necessary materials so
that the class is well organised. Finally, the principal lecturer needs to be
able to quickly improvise in various situations as needed to ensure
students remain focused. Under the circumstances where survival
swimming education was restricted due to the COVID-19 pandemic, the
research results indicated that survival swimming education applying VR
and simulation devices allowed students to indirectly gain experience,
interest, and pleasure. Therefore, it is expected that VR-based education
can increase students’ interest and learning performance in survival
swimming. Finally, VR-based teaching capacities were derived based on
the research results.
Keywords: teaching capacity; virtual reality-based teaching; physical

education; elementary education; survival swimming
1. Introduction
In South Korea, due to continuous drowning accidents, there is an increasing
social consensus that learning survival swimming is necessary. The Korean
government therefore recently revised the national curriculum to include survival
©Authors
308
swimming education as part of water leisure safety and to facilitate mandatory

implementation of survival swimming education (Ministry of Education, 2016).
However, most schools avoid providing survival swimming education due to
several limitations in their surrounding environments such as possible accidents
and injuries, lack of swimming facilities, and transportation to get there (Jeon &
Cho, 2007). Accordingly, although practical training should form the core of
survival swimming education, teachers have had to utilise audio-visual materials
due to teacher’s insufficient swimming skills (Kwon & Jung, 2021). Moreover,
since swimming pool operations have been repeatedly halted and resumed due
to the COVID-19 pandemic, elementary schools have converted survival
swimming education to theoretical education to protect elementary school
students from exposure to the disease.
In response to these concerns, many people have expressed an increasing interest
in methods for providing physical education based on virtual reality (VR)
hardware and software to ensure the health and safety of students while also
providing appropriate education (Cheng, 2021). VR is a human-computer
interaction technology in which people use computer software to mimic real-
world experiences. This technology and relevant simulations have practical
applications in the education field to exert positive effects on learner motivation
and increase learners’ interest in and enjoyment of education (Bae et al. 2018;
Elmqaddem, 2019; Kavanagh et al., 2017). These tools also allow users to
experience situations that cannot be explored in the real world (Clark, 2006).
Given this perspective, this study analysed whether VR-based survival swimming
education could create positive changes in the learning attitudes of students
toward survival swimming, especially for students who fear water and are
reluctant to participate in such classes. VR-based swimming education can also be
applied as an alternative learning method to overcome the limitations of in-person
survival swimming education owing to the spread of COVID-19. VR technology
has been used mainly in physical education and safety experience education, and
multiple studies on the application of this technology have been carried out in
various fields (Araiza-Alba et al., 2021; Zhu & Kou, 2021; van Duijn et al., 2021).
Thus, applying VR technology in an education setting was evaluated as an
alternative method to increase the effectiveness of survival swimming education
policies formulated by the Korean government (Park, 2020).
To teach VR-based survival swimming classes, teachers need to develop the
necessary teaching capacities, which differ considerably from the skills required
for practical training. In addition, the instructors might struggle to access VR-
based survival swimming education reference. It may also be challenging for
them to receive in-service training designed to increase their teaching capacities
as such training programs have not yet been developed. Due to these issues, the
instructors have constantly struggled to develop their own teaching methods and
apply them directly and practically in classes, thus depending on trial and error
to determine effective strategies.
Therefore, this study conducted in-depth interviews with a focus on survival
swimming instructors who had been recognised by organisations, students, and
other teachers (in terms of their teaching capacities) and who had expanded their
teaching fields to include VR learning during the COVID-19 pandemic. This study
309
analysed their motivations for specialising in VR-based survival swimming

education and examined their capacities including goals, targets, and methods
employed to teach.
2. Theoretical Background
2.1. VR and experiential learning
It might appear that mutual circulation between the virtual world and the real
world can be facilitated using a display. However, even though the virtual world
can include objects from the real world, they are never identical. (Milgram &
Colquhoun, 1999). In other words, virtual objects that are produced to mimic
actual objects cannot exist as real objects and vice versa. VR refers to an artificial
environment that effectively or essentially exists in the same or a similar way to
the real world (Lee, 2004). VR enables people to perceive the same or a similar
experience as they would in the real world. A VR experience can therefore be a
direct experience of human beings despite ongoing discussions about the state of
practical existence of VR from an ontological view.
Prior to the active development and spread of immersive VR technology,
researchers performed several studies on experiential learning based on partial
VR application. These studies reported that people considered VR experiences to
be equal or similar to actual real-world experiences (Kamarainen et al., 2013;
Klopfer et al., 2005; Perry, 2014; Squire & Klopfer, 2007). Dunleavy (2014) analysed
previous studies on partial VR application and found that they adopted
constructivism and situated learning theory, which focused on social interactions
that play an important role in constructing knowledge (Cobb & Bowers, 1999), as
theoretical grounds. Partial VR application provided similar situations to the
actual environment and enabled users to form knowledge and meanings through
their virtual experiences (Dunleavy, 2014). The latest VR technology, which has
advanced in terms of lifelike interactions and vivid images, can provide users with
similar experiences to real life based on a strong sense of presence beyond the
simple VR experience of the past. Furthermore, users will more clearly recognise
the VR experience provided by the most advanced VR technology as direct
experience.
The existing experiential learning theory, which emphasized physical experience
and action learning, described the predominance of dichotomous thinking based
on direct and indirect experience obtained in the field and from media,
respectively (Dewey, 1938:13-27; Kolb, 1984:20). However, in recent times, people
find indirect experience obtained through media to be more vivid, realistic, and
similar to direct experience due to the development of virtual environment
technology, such as VR and augmented reality. For this reason, the difference
between direct and indirect experience has been gradually obscured. In general, a
sense of presence serves as a crucial element that enables people to consider
indirect experience to be similar or equal to direct experience (Parong et al., 2020;
Lackey et al., 2016; Servotte et al., 2020; Slater, 2018). A sense of presence refers to
someone feeling like they are still in a real space when they are actually in a virtual
environment without physical contact. It also indicates a perception of virtual
objects in the real space as real objects. Factors affecting a sense of presence
include interactions and vividness (Kim et al., 2021; Kim & Ko, 2019). Immersive
VR technology has advanced based on interactions and audio-visual vividness as
310
head-mounted displays (HMD) have become popular. In addition, researchers

have developed and commercialised methods that facilitate direct interactions
with virtual objects or simulate their movement. It is therefore expected that
future VR technology will enable users to more strongly perceive virtual
experiences as direct experiences.
VR technology can offer learning opportunities for users in artificially created
situations similar or equal to real situations by using visualisation and a sense of
interaction with learning targets (Piovesan et al., 2012). Sacks et al. (2013) reported
that a VR-based education method was more effective than existing education
methods. Some of the potential advantages of VR application in education and
training are as follows (Velev & Zlateva, 2017):
• Virtual platforms and headsets are the new tools for inspiring creative
learning.
• Education that is not possible in reality, will be possible in VR.
• Virtual game-based experience increases students’ motivation.
• Collaboration in the VR classroom fosters leaners’ social integration.
• Learning is achieved by direct interaction, not by mouse clicks.
• The results from the learning process are truly assessed.
2.2. VR-based survival swimming

Several researchers have conducted research on survival swimming instructors.
Kang et al. (2021) assessed social interest in the instructors and suggested a
systematic approach to support them. Kwon et al. (2019) pointed out the lack of
experienced survival swimming instructors. Lee and Kim (2020) investigated the
core capacities of instructors who provided survival swimming education based
on practical training and examined the elements and priorities of these core
capacities. Kim et al. (2018) investigated qualitative improvement of educational
contents on survival swimming rather than teaching capacities. Sun (2020)
examined whether female teachers in elementary schools were aware of survival
swimming. There have been few studies, however, that analyse the teaching
capacities required for survival swimming instructors who provide VR-based
survival swimming education, which is currently being implemented in the
education field.
Park (2020) conducted an experiment where students who had previously gained
experience through a physical learning session learnt survival swimming based
on instant interactions, a characteristic of VR, in a VR-based survival swimming
class. Students also learnt survival swimming in stages based on the active use
and manipulation of controllers in virtual space instead of passive video
watching. The experimental result indicated that students gained objective and
successful experience perceived in the staged survival swimming learning
processes. Accordingly, based on these students’ objective experiences, VR-based
survival swimming education increased students’ awareness of risks related to
water safety accidents in a VR environment that was similar or equal to real
environment. Furthermore, experience-based education on preventing water
safety incidents, which encouraged students to actively protect their own safety,
positively influenced learners’ perceptions of how to prevent water safety
accidents (Park, 2020).
311
To expand the research scope of previous studies, this study analysed the
motivation of survival swimming instructors who had recently added VR-based
lessons to their previous experience as survival swimming teachers. This study
also obtained information on how instructors specialising in VR-based survival
swimming education survived competition against numerous other instructor
applicants after they began working as survival swimming instructors applying
VR devices. Furthermore, this study investigated the pedagogic capacities of
survival swimming instructors who were applying VR devices to teach students
and compared these capacities with those of existing instructors to intensively
analyse how the differences in capacities affected students’ learning.
3. Research Methods
The theoretical basis for the qualitative method is phenomenological research. It
describes the experiential meaning of a concept or phenomenon that all research
participants have in common while experiencing the phenomenon (Neubauer et
al., 2019). For data collection, in-depth interviews were used to draw people's
thoughts, knowledge, and perspectives in more depth. Lastly, thematic analysis
was used to classify common themes among the collected data (Vaismoradi et al.,
2013).
3.1. Research participants

To select research participants in qualitative research, researchers should
prioritise whether the candidates can obtain sufficient appropriate data to meet
the research objectives (Creswell, 1998). This study selected four research
participants through the following process. First, we recruited candidates who
had recently worked as survival swimming instructors in elementary schools
located in different regions. As these experts had worked in swimming-related
fields for several years, they were regarded as “professional swimming
instructors.” Among the candidates, four were selected whose capacities as
instructors had been recognised by both peer instructors and students and who
had put forth consistent efforts to improve their personal teaching capacities,
including guiding students, developing and applying teaching contents and
methods, and evaluating teaching performance. Moreover, the selected research
participants possessed various types of swimming-related certificates. All
participants had acquired a lifeguarding certificate. Finally, they are 20-30s
females living in the metropolitan areas. Table 1, shown below, provides brief
information on the research participants. Detailed information on these
participants is described in the following sections.
Table 1. Personal characteristics of research participants and their personal

information
Teaching
Name
Sex experience Details
(pseudonym)
(years)
This instructor worked as a general swimming
instructor and a survival swimming instructor in a
Hahn Female 3
swimming pool after her retirement as a
professional fin swimmer.
312
This instructor worked as a general swimming

instructor and a survival swimming instructor for
Ham Female 8
several years after her retirement as a professional
fin swimmer.
This instructor gained experience in the field of
Ahn Female 6 water safety after beginning work as a lifeguard in
her early 20s.
This instructor taught general swimming and
survival swimming for several years based on her
Byeon Female 8
previous experience as a swimmer while she was a
student.
3.2. Data collection

In-depth interviews were conducted with the participants. In these intensive
interviews, participants provided responses to questions on their direct and
practical experience as survival swimming instructors, awareness of problems,
and methods for solving them. The selection process of participants in qualitative
research should prioritise whether they can properly exchange their information
with researchers (Lee, 2003). Thus, we selected four survival swimming
instructors who were already acquainted with the researchers, had established a
rapport with them, and satisfied research objectives and criteria as participants.
The individuals were contacted directly, provided explanations about the
research objectives and methods, after which they submitted their written
agreement to participate in the study.
The entire interview contents were transcribed, and the transcription result was
stored and managed as research data. Four individuals participated in this
interview processes for a total of three or more times from October to November
2021. During this period, the researchers contacted participants 12 or more times.
To carry out the full course of in-depth interviews with each participant, the
researchers directly visited participants’ workplaces. Each interview lasted
approximately 50–90 minutes.
The feasibility of the data collection processes and procedures was reviewed by
an expert group consisting of two employees of organisations related to survival
swimming and three elementary school teachers. While conducting the feasibility
analysis of data collection procedures from in-depth interviews, the expert group
considered whether the interview contents reflected the purposes and objectives
of this study and if there were any problems with the content. When certain
interview data were found to be problematic (e.g., contents irrelevant to the
research topic, involuntary participation, participants' identity exposure etc.)
during this process, the researchers and the expert group discussed the exclusion
of the data.
3.3. Data analysis

The data analysis procedure in qualitative research refers to the process of
determining the order and significance of the data collected. Qualitative research
is also a process of reducing the amount of data based on theme-based elements
and verifying and correlating analytic categories to identify the themes and
313
significance (Dey, 1993). This study analysed the data collected from in-depth
interviews based on inductive analysis (Patton, 2002).
During the transcription process, the participants agreed to have the interviews
transcribed to prevent data loss. Interviews were recorded electronically, and the
recorded data were coded and analysed. In the case that there were insufficient
contents, additional interviews were conducted.
The data collected from each participant was coded by considering characteristics
of the educational attempts they had made during their work as survival
swimming instructors. The minimum unit newly derived in the coding process,
was registered in a node. Accordingly, this study integrated similar words and
removed duplicated words in the coding process. The researchers had multiple
discussions to inspect words of the minimum unit that should be newly registered
or integrated. The expert group formed for this study reviewed whether the initial
extraction process and assumptions applied in the processes of adding and
integrating words was adequate. Through these processes, research data were
classified into four large areas and five medium areas. Table 2 describes the
categorisation of the research data.
Table 2. Categorisation and conceptualisation results of the research data
Large areas Medium areas Small areas
Motivation for Acquisition of a teaching certificate,

becoming a VR- expansion of a range of teaching methods,
based survival Becoming a teacher and motivation for having newly become a
swimming VR-based survival swimming instructor
instructor
Advantages of
applying VR and
simulators in Differences between VR-based and general
Operation of survival swimming
survival survival swimming classes
classes
swimming
classes
Educational Analysis of educational contents and

evaluation methods for education evaluation
Self-feedback on
Work as a Analysis of weaknesses and self-reflection
teaching in classes
survival
swimming
instructor Factors as an
Necessary capacities as an instructor
instructor
3.4. Research Integrity

This study conducted a review (Kim, 1997; Guba, 1981; Nelson, 1990) to increase
the research integrity by considering various characteristics of the research
processes applied. Specifically, during the review process, participants,
314
researchers, and the expert group revisited discussions on whether the data
collection process and data analysis result reflected the intended research
purpose. Specifically, it examined whether the raw data collected had been
accurately classified and analysed to evaluate the validity of the obtained data
and analytic results and if the data had been analysed from different perspectives.
In the first verification stage, the analytic results of the raw data were shown to
participants to confirm the results. In the second verification stage, the expert
group consisting was formed to review the analytical results.
To ensure research integrity and ethics, participants agreed to participate after
being clearly informed about the research purposes and subsequent use of the
research results. Participants were also assured that the interview contents would
only be used for research purposes. The interview contents were recorded after
considering the schedules of the individual research participants. They were also
checked for errors related to the researchers’ subjective judgments. Participants’
privacy was protected by disguising their personal information and allowing
them to use pseudonyms.
4. Results
The results indicated that the swimming instructors who participated in this study
demonstrated sufficient teaching capacities to implement effective survival
swimming education about educational goals and teaching and evaluation
methods. Schools have recently begun to prefer to provide survival swimming
education in indoor classes rather than swimming pools due to the COVID-19
pandemic. As a result, instructors who participated in this study had also held
survival swimming classes using VR and simulation devices. For this reason, the
capacity to handle various devices required for survival swimming education was
evaluated as a significant competence of the instructor. Participants also used
creative methods to teach students and implement an adjusted educational goal
according to the newly revised curriculum. Moreover, they constantly reflected
on their own teaching methods and attempted to improve them. These capacities
of participants were recognised as significant for instructors based on their
association with the establishment of educational goals, selection of educational
contents and methods, and educational evaluation and feedback processes.
4.1. Motivation for having newly become a survival swimming instructor

If instructors in different types of sports, including survival swimming, begin
playing a certain sport related to their interests, hobbies, or athletic career while
they were students, they tend to continue to play the sport as adults. This makes
them more likely to work as instructors in their chosen sport. According to an
interview with Byeon, a participant survival swimming instructor, she developed
her athletic career as a swimmer as a student and began working as a swimming
instructor in a sport centre at the age of 20 based on her previous experience with
and affection for children. As she began working as a swimming instructor
without a university degree, she was concerned about her life. However, peer
instructors supported her in adapting to the position of a swimming instructor
and she was employed as a swimming instructor at the time of the interview.
I really like children, so I began working in a swimming pool for children
when I was 20 years old. When I just worked as a swimming instructor, I
315
did not know anything about this job, but other instructors in the
swimming pool helped and taught me a lot. In fact, I was too young around
that time… I thought, “I am going to start a new job now but is it right to
work like this?” I just got this job at that time because I found it fun to
teach someone else and a teaching job matched my personality well. The
position of a swimming instructor has put a great amount of physical
pressure on my body, but I find this job is suitable for me. That is why I
have worked as a swimming instructor so far. (Byeon)
Participant Ahn stated that she showed an extraordinary interest in water
activities beginning in high school. Consequently, she wondered whether she
should be a swimming instructor and a lifeguard as an adult. She was offered the
position of swimming instructor by chance at a place that she visited to attend a
training session on teaching swimming. She began working as a swimming
instructor after accepting their offer and was able to constantly accumulate
knowledge of teaching methods.
I really wanted to be a swimming instructor from when I was in high
school. I worked as a lifeguard for a long time, but I kept thinking about
whether I should change my job to be a swimming instructor. Then, one
day, someone I knew told me about a place that provided a teacher training
in swimming. I went to that place to take the training, and a person there
suddenly asked me to swim. After watching me swimming, this person
unexpectedly offered me the position of a swimming instructor. I had just
come to take a training and I got offered to teach there. The place was a
swimming pool in a hotel. At first, I studied teaching methods intensively
for a week. I had also studied teaching methods a lot since high school. So,
it was not a problem for me to instantly start working as an instructor.
(Ahn)
Most research participants responded that they had begun swimming based on
their interest in it as students and that swimming experience naturally drove them
to work as survival swimming instructors as adults. In general, they exhibited a
high level of job satisfaction working as survival swimming instructors.
Participants, who were in their late 20s on average, indicated that they began
working as swimming instructors as soon as they retired as professional
swimmers. They designed their career paths based on jobs in the fields of water
activities from high school. As such, they considered that their tendencies and
talents matched the necessary capacities of a swimming instructor.
4.2. Advantages of VR-based survival swimming classes

As instructors provide general survival swimming education in swimming pools,
they should exert great efforts to manage students in every step of the way—from
taking a shower to entering the water. According to Ham, VR-based survival
swimming education has the advantage of reducing class preparation time
because indoor education is possible. Moreover, survival swimming education
based on VR experiences can create greater interest and more effective outcomes
for students who fear the water compared to classes based on blackboards and
theories.
It truly brings comfort for instructors. This type of education is so
comfortable that it can be called a class for instructors. It is really
316
uncomfortable to hold a survival swimming class wearing a swimsuit or a

wetsuit. It is also stressful to let tens of students enter a swimming pool
and watch them during a class. A swimming class performed in a
swimming pool is turmoil. Should I take a shower with children in the
shower room and take care of them at the same time? However, VR
survival swimming education is performed in an indoor class and teaching
contents for each class are established. I like this neat type of education. It
is also advantageous that I can use both VR contents and various types of
audio-visual materials to increase students’ level of understanding. I find
these theoretical contents necessary. Children who are not comfortable
with the water might learn survival swimming more effectively by
experiencing it based on VR contents in advance and later entering a
swimming pool than by simply watching videos and learning contents
written on a blackboard in a class. VR contents might increase the
children’s level of interest when they are provided such contents. (Ham)
Byeon stated that VR-based survival swimming education performed in an indoor

class can alleviate instructors’ fatigue because they can speak comfortably at a
normal volume and use VR devices to deliver educational contents based on
experience instead of physically delivering them. Moreover, students tend to
focus on what the survival swimming instructor says more intensely in a VR-
based survival swimming class than in a general swimming pool due to their
curiosity about VR.
The advantage of this type of education is that we instructors can speak at
a low volume. When I teach survival swimming for more than two or three
hours per day, my throat hurts. However, a VR-based survival swimming
class definitely provides comfort for instructors because students can learn
survival swimming based on VR contents. Children also tend to be quiet
and follow what I say so that they can experience VR contents more
quickly… The atmosphere of a VR-based survival swimming class is more
pleasant than that of a general survival swimming class.” (Byeon)
According to an interview with Ahn, students used to have difficulty
concentrating when they learnt cardiopulmonary resuscitation (CPR) in general
survival swimming classes. However, when she taught CPR to students using
simulators in a VR-based survival swimming class, students were able to monitor
information on the number of compressions, number of effective compressions,
number of compression failures, probability of revival, and depth of compression
in real time. As simulation devices can increase students’ interest in a topic, the
need for the instructor to control students may also be minimised.
I used to conduct CPR classes using only animation contents, but
students did not concentrate in these classes at all. On the contrary, when
I teach CPR by using simulators, students focus on these classes very well.
When I give them feedback, such as “press it more strongly” and “take
your pose like this,” they listen to my words carefully to increase the
number of their effective compressions for CPR displayed on a monitor. I
do not need to control external things outside of the class. (Ahn)
When instructors hold survival swimming classes in swimming pools, they must
speak to students at a comparatively higher volume to deliver educational
317
contents. For this reason, instructors who teach survival swimming in swimming
pools for a long time are likely to experience throat pain and eventually suffer
from vocal fold nodules. In contrast, survival swimming classes based on VR and
simulation devices can increase students’ interest in the lesson as well as the
quality of education. All participants agreed that VR-based survival swimming
education effectively improved the perceptions of students who feared the water
by allowing them to virtually experience survival swimming before entering an
actual swimming pool. Moreover, CPR classes based on VR and simulation
devices maximised educational effectiveness because these classes allowed
learners to monitor their scores in real time. In this regard, VR-based CPR classes
were evaluated to be more effective than general CPR classes where students were
asked to simply use animated contents.
4.3. Educational evaluation

Ham discussed how she conducted educational evaluation based on VR
experience in the form of quizzes after students finished learning. She controlled
the level of quiz difficulty according to students’ level of knowledge. At the end
of her classes, she emphasised that students can practically apply survival
swimming education contents that they have learnt based on VR experience to
other water-related places such as swimming pools and beaches. She assessed that
survival swimming education applying simulators had the advantage of
increasing students’ interest in the topic and the disadvantage of harming the
nature of survival swimming classes (i.e., the possibility of the class focused on
fun and interest rather than learning). To overcome this disadvantage, she
reminded students about the order of CPR and precautions when performing
chest compressions.
I let students undergo VR experience and solve quizzes. I show only
questions and select multiple students to ask them to answer the questions
and draw students’ attention. Children are not familiar with survival
swimming, so I evaluate their knowledge based on questions that are not
extremely difficult. Then, at the end of the class, I highlight that they will
guarantee their own safety and even save other people by recalling and
applying what they have learnt in the class when they go to beaches or
swimming pools in the future. I finish my class by emphasising that they
can save other people by remembering only the order of CPR beyond the
significance of finding the accurate location to do chest compressions.
(Ham)
Byeon also mentioned that she conducted educational evaluations based on
quizzes after students underwent the VR experience. She said that she provided
students detailed explanations about the quiz choices, specifically the reasons for
both correct and incorrect answers. When students had a high percentage of
correct answers for the quizzes, she allowed them to freely experience VR contents
during a break time of approximately three to five minutes.
When students solve quizzes and ask, “what is underwater swimming?”
I try to describe underwater swimming according to their level of
knowledge and clearly explain why this choice is a correct or incorrect
answer. Then I tell students that I will hold a VR class again when they
solve quizzes successfully. I reckon that quizzes are important. When
318
children follow my instructions well, I allow them to freely experience VR

contents for about three to five minutes.” (Byeon)
Students independently participate in a VR-based survival swimming class where

each student can use their own device. As teachers cannot check the learning
attitudes of students in real time in a VR-based survival swimming class, the
educational evaluation process is significant. Each participant employed quizzes,
including simple questions, to perform their educational evaluations of students’
knowledge after completing their VR experience. The participants completed
quizzes with students and helped them understand relevant contents more
deeply by providing detailed explanations of the answers. In addition, the use of
VR and simulation devices has an advantage of triggering student curiosity.
However, when these devices are used without appropriate controls, instructors
might fail to deliver educational contents that should be included in a VR-based
survival swimming class. However, all participants were aware of the problem
that VR-based survival swimming classes were mainly conducted based on
experience. To solve this problem, they completed a VR-based survival swimming
class by summarising the educational contents that had been covered in that
lesson.
4.4. Self-feedback on classes

For survival swimming classes in schools, instructors are generally required to
teach for 35 to 40 minutes per class period. However, the required teaching time
for survival swimming education per class period might change depending on the
school schedule. In survival swimming education applying VR and simulation
devices, instructors must spend a few minutes setting up and installing these
devices. For this reason, they encounter situations where they should teach classes
continuously without a break time. In addition, survival swimming instructors
are required to hold classes in different schools every day. Accordingly, they must
adapt to relevant factors such as the building arrangements and the atmosphere
of the schools in which they hold classes. Hahn indicated that she was once
affected by the movement and character of a homeroom teacher of the class in
which she was supposed to teach students. As a result, she works to quickly
identify the mood of the homeroom teacher and students in the corresponding
class.
When I hold a 40-minute-long class, I tend to be unconsciously anxious
for about five minutes in the beginning of the class. My voice shakes, and
I try to read the mind of the homeroom teacher. I hold classes in many
different schools, and the homeroom teachers in classes show different
preferences for teaching styles. When I first worked as a survival
swimming instructor, I was concerned about these issues a lot. It was the
time when I taught survival swimming education in an elementary school
located in Paju. The principal of the school told instructors including me
that we should not eat something or even drink water indoors. As the
principal had established an atmosphere like this, I was overwhelmed by
the atmosphere and could not speak well for about ten minutes. It is the
greatest weakness of my personality that I am easily affected by my
surroundings. (Hahn)
319
It has been less than a year as of this writing since survival swimming education
applying VR and simulation devices was practically implemented in schools.
Thus, the number of survival swimming instructors that have applied VR and
simulation devices, including the research participants, was significantly low.
Moreover, it can be reasonably assumed that most of these instructors had less
than one year experience as a VR-based survival swimming instructors. Before
these instructors began using VR and simulation devices, they would have also
taught students in swimming pools. After the beginning of the COVID-19
pandemic, they would have held indoor classes using monitors and audio-visual
materials. Ahn, who had maintained her previous teaching method for survival
swimming education for several years, encountered considerable difficulty in
converting this teaching method to a new one based on VR and simulation
devices. She stated that she spent a significant amount of time helping certain
students remain focused in her classes when they showed half-hearted learning
attitudes. She selected this characteristic as her simultaneous strength and
weakness.
I used to apply a standardised teaching method in classes, and it was very
difficult to convert this teaching method to a new teaching method. But I
should be aware of such difficulty all the time. I think that I should study
more in the future, and I am so meticulous that I cannot bear students
behaving half-heartedly in my class. Other instructors might pass over
certain problematic behaviours of students flexibly if these behaviours are
not too serious, but it is not the story of my life. In general, students cannot
perform even what they have learnt perfectly as soon as they have a
swimming class in general. They need more time to do so. Nevertheless, I
have a tendency of informing students about a great and sufficient number
of educational contents within a limited time. It is my concern that
students might lose interest in the subject that I teach because of my
tendency. (Ahn)
4.5. Crucial elements for an instructor

Experience based on educational devices is the core of survival swimming
education applying VR and simulation devices. However, the effects of such
experiences can be doubled only when students are clearly aware of the class
objectives and class contents prior to experiencing the VR contents and
simulation. Ham selected the capacity of an instructor to express opinions as a
necessary competence of a VR-based survival swimming instructor, given the
need to inform students about contents related to his or her class and methods for
experiencing VR contents. The instructors had recently taught students in indoor
classes due to the COVID-19 pandemic. However, Ham addressed a need for
survival swimming instructors to be flexible in responding to questions from
students regarding survival swimming classes and methods for using VR and
simulation devices.
Instructors’ tone of voice and accent are important. Instructors should
speak clearly enough to help students understand what they say well.
Students find it more difficult to understand what I say these days because
I have to wear a mask in a class. Instructors should also have the capacity
to flexibly manage a situation when students ask weird questions or when
a certain issue arises in a class. Instructors should develop their own
320
teaching strategies for helping students concentrate in their classes and

controlling students. (Ham)
In South Korea, certificates for survival swimming instructors are issued by
organisations such as the Republic of Korea National Red Cross and the Korea
Survival Swimming Association. To obtain these certificates, instructor
candidates must complete various types of theoretical education and practical
training and pass evaluation processes. In addition to possessing certificates, they
should also spend an appropriate amount of time analysing teaching methods to
help students understand what they learnt in class and increase students’ interest
in the class. In this way, they can facilitate smooth class operation in the practical
survival swimming education field. Ahn commented that instructors should
constantly conduct research on contents to be taught in classes and class operation
methods so they can respond swiftly to questions from students about the class
contents and establish a desirable class atmosphere.
I think that I should study relevant certificates and teaching methods by
myself. When I watch other instructors teaching students, I find a
difference between these instructors in detailed aspects. Experienced
instructors have a different range of capacities for responding to students’
questions from that of inexperienced instructors. Experienced instructors
are used to providing more satisfactory answers and solutions based on
their experience. For example, in CPR classes, experienced instructors
show similar CPR performance regardless of the surrounding conditions.
On the other hand, inexperienced instructors perform CPR at irregular
speeds. The speed of their CPR performance will inevitably increase or
decrease. (Ahn)
5. Discussion
The research results can be summarised as follows. First, research participants
began swimming to develop their career or hobbies as swimmers and naturally
came to work as survival swimming instructors in adulthood. Second, research
participants believed that using VR and simulation devices significantly
contributes to teaching survival swimming and the educational operation of their
classes. Third, research participants provided self-feedback on their classes and
analysed factors they should develop as instructors.
This study derived the following implications based on the research results. First,
as VR and simulator-based survival swimming classes utilise educational
equipment (VR, simulator), the ability to adeptly handle educational equipment
is considered a vital teaching capacity. If the focus of the VR devices is out of focus
or the field of vision is not clear, the instructor needs to set up the VR devices so
that the students can participate in the class.
Second, communication is a key capacity for instructors to effectively explain the
class objectives and contents. Solid communication skills are required to achieve
class goals and explain the use of educational equipment before commencing
practice, answer students' questions after the practice, and organise the class.
Practice education is conducted with one device per person due to the nature of
the class. Therefore, if the time lag for completing the experience increases, there
is a strong possibility of a setback in the class progress.
321
Third, instructors should possess the flexibility to deal with various unexpected
or uncomfortable situations. For example, due to the nature of the survival
swimming class, classes proceed in the classroom in the presence of homeroom
teachers. In general, the homeroom teacher does little to intervene in the class.
However, sometimes the homeroom teacher exerts a considerable influence in the
class compared to the principal lecturer. In this case, it may disperse the children's
attention. Therefore, the principal lecturer needs to be able to improvise to address
unexpected situations within a short time and reorient students so that they
concentrate on the class again.
The theoretical implication of the research results is that the use of VR devices
induces interaction and indirect experiences so that students can expect similar
learning effects without direct experiences. This has meaningful implications for
situated learning and experiential learning theory. In addition, the practical
implication is that by using VR devices, instructors can expect learning effects
while securing safety. (Araiza-Alba et al., 2021)
6. Conclusion
This study analysed the teaching capacities of survival swimming instructors
applying VR devices. In-depth interviews were conducted with four instructors
applying VR devices who described their motivations for having begun working
in this field and various cases from their experiences. Detailed interviews focused
on examining participants’ capacities for teaching survival swimming. An
inductive analysis of the research data obtained from these interviews was carried
out to classify the data into the categories of motivation for having begun working
as a survival swimming instructor and strategies for teaching survival swimming.
The research results showed what teaching capacities were required for effective
VR-based swimming lessons. The result also indicated that survival swimming
education applying VR and simulation devices indirectly provided students with
VR-based educational contents and allowed them to gain similar experience,
interests, and enjoyment to what they obtained in previous traditional training
classes for survival swimming. Therefore, it is expected that VR-based survival
swimming education can boost students’ learning performance and interest in
survival swimming education.
7. Limitations and Future Research

There are several limitations to the research method used in this study. Based on
the analytical results, this study presents several suggestions for follow-up studies
in relevant fields.
First, although random selection was employed, this study was carried out based
on only a few participants who were all female and working as survival
swimming instructors in metropolitan area. Therefore, the results cannot be
generalised to all survival swimming instructors using VR devices. Follow-up
research should pay attention to participant selection by matching the
characteristics of a group of participants with research purposes.
Second, this study intensively analysed the teaching capacities deemed necessary
according by the participants. However, this study did not collect or analyse
research data regarding teaching capacities for survival swimming instructors
322
who are not applying VR and simulation devices. Future studies could compare
swimming instructors using traditional methods with the research participants of
this study.
Finally, future studies should evaluate the practical performance of survival
swimming learners in actual swimming pools after the end of the COVID-19
pandemic to verify the effectiveness of VR-based survival swimming education.
8. References
Araiza-Alba, P. Keane, T., Matthews, B., Simpson, K., Strugnell, G., Chen, W.S., Kaufman,
J. (2021). The potential of 360-degree virtual reality videos to teach water-safety
skills to children, Computers & Education, 163,104096.
https://doi.org/10.1016/j.compedu.2020.104096.
Bae, Y., Park, P., Moon, G.S., Yoo, I., Kim, W., Lee, H., & Shin, S. (2018). An Instructional
design of STEAM programs using virtual reality equipment and analysis of its
effectiveness and attitude of learners. Journal of The Korean Association of
Information Education, 22, 593~603.
http://dx.doi.org/10.14352/jkaie.2018.22.5.593.
Cheng, J. (2021). Evaluation of physical education teaching based on web embedded
system and virtual reality. Microprocessors and Microsystems, 83, 103980.
https://doi.org/10.1016/j.micpro.2021.103980.
Clark, D. (2006). Games and e-learning. Epic. [Online]. Available: http://www.cedma-
europe.org/newsletter%20articles/misc/Games%20and%20e-
Learning%20(Nov%2006).pdf. Retrieved March 16, 2022.
Creswell, J. W. (1998). Qualitative inquiry and research design: Choosing among five traditions.
Sage.
Dewey, J. (1938). Experience and Education. Collier Books.
Dey, I. (1993). Qualitative data analysis: A user-friendly guide for social scientists. Routledge.
https://doi.org/10.4324/9780203412497.
Dunleavy, M. (2014). Design principles for augmented reality learning. Tech Trends, 58(1),
28–34. https://doi.org/10.1007/BF02766777.
Elmqaddem, N. (2019). Augmented reality and virtual reality in education. Myth or
reality? International Journal of Emerging Technologies in Learning, 14(3), 234-242.
https://doi.org/10.3991/ijet.v14i03.9289.
Guba, E. G. (1981). Criteria for assessing the trustworthiness of naturalistic inquiries.
Educational Communication and Technology Journal, 29, 75–92
https://doi.org/10.1007/bf02766777.
Jeon J., & Cho, S. (2007). 초등학교 체육수업에 대한 특별편입교사의 인식분석 [Cognition
analysis on the physical education teaching of elementary specially enlisted
teachers]. 학교보건체육연구소 논문집 [The Journal of Institute of School Health &
Physical Education], 14(1), 189–227.
Kamarainen, A. M., Metcalf, S., Grotzer, T., Browne, A., Mazzuca, D., Tutwiler, M. S., &
Dede, C. (2013). EcoMOBILE: Integrating augmented reality and probeware with
environmental education field trips. Computers & Education, 68, 545–556
https://doi.org/j.compedu.2013.02.018.
Kang M., Won, D. H., & Jo, W. (2021). 생존수영 의무교육에 대한 강사의 인식과 경험
[Instructor's perception and experience of compulsory education for survival
swimming]. 한국스포츠사회학회지 [Korean Journal of Sociology of Sport], 34(1), 23–
33. https://doi.org/10.22173/ksss.2021.34.1.2.
323
Kavanagh, S., Luxton-Reilly, A., Wuensche, B., & Plimmer, B. (2017). A systematic review
of virtual reality in education. Themes in Science and Technology Education, 10(2)2,
85-119. https://eric.ed.gov/?id=EJ1165633.
Kim, D., & Ko, Y. J. (2019). The impact of virtual reality (VR) technology on sport
spectators' flow experience and satisfaction, Computers in Human Behavior, 93, 346-
356, https://doi.org/10.1016/j.chb.2018.12.040.
Kim, J. (1997). 체육학 연구법[A method of physical education]. 대한 미디어 [Korean Media].
Kim, J.-H., Kim, M., Park, M., & Yoo, J. (2021). How interactivity and vividness influence
consumer virtual reality shopping experience: the mediating role of telepresence.
Journal of Research in Interactive Marketing, 15(3), 502-525.
https://doi.org/10.1108/JRIM-07-2020-0148.
Kim, Y., Kim, W., & Park, I. (2018). 생존수영 교육맥락의 질적 이해를 통한 교육내용과
지도자 전문성 개선 방안 탐색 [A study on the improvement of contents and
expertise through qualitative understanding of the survival swimming
educational context]. 한국스포츠교육학회지 [Korean Journal of Sport Pedagogy],
25(1), 75–103. https://doi.org/10.21812/kjsp.2018.01.25.1.75.
Klopfer, E., Perry, J., Squire, K., Jan, M. F., & Steinkuehler, C. (2005). Mystery at the
museum–A collaborative game for museum education. Proceedings of Conference
on Computer Support for Collaborative Learning, 316–320.
https://doi.org/10.3115/1149293.1149334.
Klopfer, E., Squire, K., & Jenkins, H. (2002, August). Environmental detectives: PDAs as a
window into a virtual simulated world. Proceedings of IEEE International Workshop
on Wireless and Mobile Technologies in Education, 95-98.
https://doi.org/10.1109/WMTE.2002.1039227.
Kolb, D.A. (1984). Experiential Learning: Experience As The Source Of Learning And
Development. Prentice-Hall.
Kwon, T., & Jung S. (2021). Elementary school teachers' survival swimming education
burdens and improvement plans. The Korean Journal of Elementary Physical
Education, 27(1), 51-71. https://doi.org/10.26844/ksepe.2021.27.1.51
Kwon, Y., Yeo, K., & Noh, Y. (2019). 델파이 기법을 활용한 생존수영 프로그램 개발을 위한
요인 도출에 관한 연구 [A study on derivation of factors for developing survival
swimming program using Delphi technique]. 한국체육정책학회지 [Journal of
Korean Sport Policy], 17(4), 143–160. https://doi.org/10.52427/kssp.17.4.10.
Lackey, S. J., Salcedo, J. N., Szalmam J.L., & Hancock, P.A. (2016). The stress and workload
of virtual reality training: the effects of presence, immersion and flow. Ergonomics,
59(8), 1060-1072. https://doi.org/10.1080/00140139.2015.1122234.
Lee, C. (2003). 여가학 연구방법론 [A methodology for the study of leisure studies]. 대경북스
[Daegyeong Books].
Lee, C. (2004). 가상현실, 별 만들기 [Virtual reality, making stars]. 동과서 [East and West].
Lee, Y., Kim, M. (2020). 생존수영 강사의 핵심역량 구성요소 및 우선순위 분석 [An analysis
of survival swimming instructors of core competencies of factors and priority].
한국체육교육학회지 [Journal of Korean Society for the Study of Physical Education],
25(2), 51–62. https://doi.org/10.15831/jksspe.2020.25.2.51.
Milgram, P., & Colquhoun, H. (1999). A taxonomy of real and virtual world display
integration. Mixed Reality: Merging Real and Virtual Worlds, 1(1999), 1–26.
https://doi.org/10.1007/978-3-642-87512-0_1.
324
Ministry of Education (2016). 2016 school sports promotion plan. Department of

Personality, Physical Education, and Arts.
Nelson, T. O. (1990). Metamemory: A theoretical framework and new findings. Psychology
of Learning and Motivation. 26, 25-173. https://doi.org/10.1016/s0079-
7421(08)60053-5.
Neubauer, B.E., Witkop, C.T., & Varpio, L. (2019). How phenomenology can help us learn
from the experiences of others. Perspectives on Medical Education, 8, 90–97.
https://doi.org/10.1007/s40037-019-0509-2.
Park, J. (2020). 코로나 19 상황에서 가상현실 (VR) 소프트웨어를 활용한 초등학생
생존수영교육이 운동자기효능감과 수상안전의식에 미치는 영향 [Effects of
elementary school students survival swimming education using virtual reality
(VR) software in the corona-19 situation on exercise self-efficacy and water safety
awareness]. 한국체육과학회지 [The Korea Journal of Sports Science], 29(6), 315-326.
https://doi.org/10.35159/kjss.2020.12.29.6.315.
Parong, J., Pollard, K. A., Files, B. T., Oiknine, A. H., Sinatra, A. M., Moss, J. D., Passaro,
A., & Khooshabeh, P. (2020). The mediating role of presence differs across types
of spatial learning in immersive technologies. Computers in Human Behavior, 107,
106290. https://doi.org/10.1016/j.chb.2020.106290.
Patton, M. Q. (2002). Qualitative evaluation and research methods (3rd ed.) Sage.
Perry, B. (2014). Gamifying French language learning: A case study examining a quest-
based, augmented reality mobile learning-tool. Procedia-Social and Behavioral
Sciences, 174, 2308–2315. https://doi.org/10.1016/j.sbspro.2015.01.892.
Piovesan, S. D., Passerino, L. M., & Pereira, A. S. (2012, Oct 19-21). Virtual reality as a tool
in the education. International Association for Development of the Information Society
International Conference on Cognition and Exploratory Learning in Digital Age, 295–
298. https://eric.ed.gov/?id=ED542830.
Sacks, R., Perlman, A., & Barak, R. (2013). Construction safety training using immersive
virtual reality. Construction Management and Economics, 31(9), 1005–1017.
https://doi.org/10.1080/01446193.2013.828844.
Servotte, J.-C., Goosse, M., Campbell, S. H., Dardenne, N., Pilote, B., Simoneau, I. L.,
Guillaume, M., Bragard, I., & Ghuysen, A. (2020). Virtual reality experience:
Immersion, sense of presence, and cybersickness. Clinical Simulation in Nursing,
38(C), 35-43. https://doi.org/10.1016/j.ecns.2019.09.006.
Slater, M. (2018). Immersion and the illusion of presence in virtual reality. British Journal
of Psychology, 109(3), 431-433. https://doi.org/10.1111/bjop.12305
Squire, K., & Klopfer, E. (2007). Augmented reality simulations on handheld computers.
Journal of the Learning Sciences, 16, 371–413.
https://doi.org/10.1080/10508400701413435.
Sun, H. (2020). 생존수영 교육에 대한 초등 여교사들의 인식 [Awareness of primary female
teachers in survival swimming education]. 한국스포츠학회 [The Korean Journal of
Sport], 18(1), 299–310.
Vaismoradi, M., Turunen, H., & Bondas, T. (2013). Content analysis and thematic analysis:
Implications for conducting a qualitative descriptive study. Nursing & Health
Sciences, 15(3), 398-405. https://doi.org/10.1111/nhs.12048.
van Duijn, T., Ng J.L., Burnay, C., Anderson, N., Uehara L., Cocker, K., & Button, C (2021).
The influence of equipment and environment on children and young adults
learning aquatic skills. Frontiers in Psychology, 12, 733489.
https://doi.org/10.3389/fpsyg.2021.733489.
325
Velev, D. & Zlateva, P. (2017). Virtual reality challenges in education and training.
International Journal of Learning and Teaching, 3(1), 33-37.
https://doi.org/10.18178/ijlt.3.1.33-37.
Zhu, X., & Kou, F. (2021). Three-dimensional simulation of swimming training based on
Android mobile system and virtual reality technology, Microprocessors and
Microsystems, 82, 103908. https://doi.org/10.1016/j.micpro.2021.103908.
326

Vol. 21, No. 8, pp. 326-344, August 2022
Math Anxiety, Math Achievement and Gender

Differences among Primary School Children and
their Parents from Palestine
Nagham Anbar , Lavinia Cheie and Laura Visu-Petra*
Babeș-Bolyai University, Department of Psychology
Research in Individual Differences and Legal Psychology (RIDDLE) Lab
Republicii Str. 37, 400015, Cluj-Napoca, Romania
Abstract. This study explored the math anxiety (MA) level and math
achievement of primary school children and the association of these
variables to their gender differences and parents’ math anxiety. Also, we
investigated the potential interaction between child MA and parental
variables on child math outcomes. The sample consisted of 230 students
in the 3rd and 4th grades (Mean age = 8.9; SD = .59), including one
parent for each child. The Scale for Early Math Anxiety, The
Mathematics Anxiety Rating Scale, The Children Test Anxiety Scale, and
the parental involvement survey was used for data collection.
Palestinian children reported lower levels of MA compared to previous
research, and the expected negative relation between students’ MA and
their math achievements was confirmed. Girls reported higher levels of
both MA and test anxiety than boys. A positive correlation was found
between mothers’ MA and daughters’ MA, while no association
between fathers’ MA and sons’ MA was found. Moreover, child MA,
parental MA and trait anxiety were found to significantly predict
children’s math achievement. This study contributed to a better
understanding of some factors affecting mathematics achievements and
future career orientations, such as Math anxiety, Test anxiety, and
possible gender differences. We suggest implementing new strategies to
reduce math anxiety, improve math achievement, and enhance females'
contribution to math-related fields in the Palestinian community.
Keywords: Math anxiety; math performance; primary school children;

parental influence; gender differences
1. Introduction
A core target of the educational system is to equip students with the essential
skills needed in their studying phase and afterward. Although schools are
considered the key vehicle for building children’s academic competence and life
*
Corresponding author: Laura Visu-Petra; laurapetra@psychology.ro
©Authors
327
aptitudes, parents also play a critical role in their children’s success (Jacobs &
Eccles, 2000). Parents are their children’s first and long-term teachers, but what if
parents are themselves anxious about the subjects their children are studying?
Studies suggest that this is fairly common when it comes to mathematics (Chang
& Beilock, 2016; Maloney et al., 2015) and that children’s math performance can
be hindered by both their own specific anxiety toward mathematics and by their
parents’ MA (Maloney et al., 2015). Additionally, children’s math attainment
varies as a function of both cultural (Daches Cohen & Rubinsten, 2017), and
gender-related differences (Devine et al., 2012). Consequently, our study set out
to explore both parental influences and gender-related differences in Palestinian
children’s MA and math achievement, in a society with inequitable gender
norms.
According to the results of the international men and gender equality survey,
Palestine represents a predominantly masculine society and displays inequitable
gender attitudes. For instance, 80% of men believe that a "woman’s most
important role is to take care of the home and to cook for the family", while 83%
of men reported that "when work opportunities are scarce, men should have
access to jobs before women" (El Feki et al., 2017). Moreover, 87% of women
stated that Palestinians need to do more to promote gender equality.
During the last 30 years, many studies have reported small or no actual gender
differences in math outcomes (Devine et al., 2012). According to the annual
national evaluation of primary schools’ students in Arabic, Math, and science
(2017-2018), the findings reported that the success percentage in mathematics for
the 4th grade was only 38%, meanwhile, mathematics average score was 47 for
girls and 42 for boys (the full score is 100). Also, according to the Palestinian
statistical centre, girls outperformed boys in the high school general
examinations for the last 5 years. For example, in 2018 the success percentage
was (91.8 % vs. 98%) in the scientific pathway, (77.7% vs. 91.2%) in the
commercial pathway, and (82.8% vs. 96.9%) in the vocational, for males vs.
females, respectively. However, as reported by the Palestinian Statistical Centre
(2017) females’ percentage of graduated with a diploma or bachelor’s in
educational sciences was 21%, compared to 6.5% of their male counterparts,
while in engineering it was 3%, compared to 9% of their male counterparts.
2. Literature and Previous Studies

2.1. Math anxiety
Across all ages, a widespread apprehension about mathematics is noticeable,
termed MA. It is defined as an overwhelming feeling of tension and worries
accompanied by negative thoughts that occur as a response to math-related
activities, and a corresponding tendency to appraise math as threatening stimuli
and to react with rising levels of anxiousness (Lyons & Beilock, 2011). A large
body of literature views MA as a subject-specific manifestation of test anxiety
(Ho et al., 2000; Ma, 1999), with the test anxiety defined as a set of negative
emotions and worries that occur especially during any evaluation context
(Hembree; 1990).
328
Devine et al. (2012) conducted a study to investigate the relations between

mathematics anxiety, test anxiety and math performance. The results showed a
positive correlation between MA and test anxiety, while a negative correlation
was found between MA and performance. Test anxiety was also negatively
correlated with math performance. Moreover, Joseph (2009) revealed a positive
correlation between MA and test anxiety. Another study conducted by Ashcraft
and Moore (2009), revealed a positive correlation between MA and trait anxiety.
As reported by Xie et al. (2018), MA was found to be positively correlated with
both test anxiety and with general anxiety, while it was negatively correlated
with math performance.
Numerous studies have indicated the negative impact of MA on math

performance (Rubinsten et al., 2018). Math-anxious students score significantly
less in mathematical exams compared to their non-anxious counterparts
(Ramirez et al., 2018), they tend to avoid math-related activities (Carey et al.,
2017) and are underrepresented in STEM domains (Maloney & Beilock, 2012).
2.2. Gender differences

Gender differences in math competence and MA have been frequently
investigated, with several findings reporting that females are more anxious than
males in math-related situations (Ashcraft, 2002; Else-Quest et al., 2010; Hopko
et al., 2003). In 2012, as stated by the Programme for International Student
Assessment (PISA), the data showed that in the majority of (OECD) countries
girls reported stronger MA than boys. For instance, Devine et al. (2012) study
among secondary school students in England, revealed no differences in
mathematics performance, although girls scored higher on the MA scale.
Similarly, findings were reported for German students (Frenzel et al., 2007) and
also for Chinese students (Huang et al., 2019) revealed that boys reported less
MA, while no significant gender differences were found in mathematics self-
efficacy.
On the other hand, Erturan and Jansen (2015) showed a significant effect of
gender on test anxiety, with girls reporting higher levels of test anxiety, while
math scores and MA did not differ based on gender. A similar finding has been
reported about the gender differences regarding test anxiety, with males
obtaining lower scores than females on evaluative tests (Popa et al., 2019). The
same results were also reported by Kavanagh et al. (2016), revealing higher
levels of test anxiety among females, compared to males. Trait anxiety was also
found to differ based on gender, according to Macher et al. (2011) study, higher
levels of trait anxiety among female students than males were confirmed. In
addition, many studies indicate that women tend to report higher trait anxiety
scores than men do (Putwain & Daly, 2014).
2.3. Parental Math anxiety

Research has confirmed that home the environment including parents’ feelings,
attitudes, and perceptions about their children has a notable impact on
children’s emotions, attitudes, self-esteem, and even their cognitive abilities
(Jameson, 2013; Mohr‐Schroeder et al., 2017; Anbar & Visu-Petra, 2021). It is also
noted that parents’ own perception of the value of mathematics has a significant
329
impact on their children’s motivation to pursue related fields in the future (Soni
& Kumari, 2017).
Math-anxious parents are more likely to pass their MA to their children,

particularly when trying to help their children with math homework frequently
(Maloney et al., 2015). Parental expressed attitudes toward math, such as ‘Oh, I
used to hate math as a child’ or 'doing math is difficult' are negatively related to
children’s success and attitudes toward math (Chang & Beilock, 2016). In
contrast, studies suggest that children who are more engaged in-home math-
related activities (e.g., board games, play with puzzles, cards) report more
positive attitudes and better math achievement than those involved in fewer
home math-related activities. Indeed, parents who try to enhance positive math
attitudes as much as possible in the home environment, regardless of their
emotions or their comprehension of math, are more likely to improve their
children's achievements in math and establish positive behaviours toward math
learning (Wilder, 2015).
A study conducted by Batchelor et al. (2017) indicated that children's MA is

related to parents' MA. More specifically, a positive association between parents'
MA and sons’ MA was identified, while no association between mothers’ and
daughters’ MA was found. The authors suggested that this could be justified by
the mediating effect of parental involvement in the child's homework. Studies
have investigated parental influences on children's math achievements, for
instance, parents' MA is considered a strong predictor of children's math
achievement (Casad et al., 2015; Maloney et al., 2015). Direct forms of parental
involvement were sometimes negatively related to children’s math achievement
(Vukovic et al., 2013). Indeed, children’s success in mathematics is not
contingent upon parents' help, given that most parents haven't received formal
teaching training. Besides, parents’ involvement doesn’t require parents to show
high skill in math, rather, they can promote children’s math performance by
simply adopting positive attitudes about math learning (Pugsley & Jill, 2018;
Vukovic et al., 2013).
To sum up, the role of parental involvement in children's achievement is still

debatable. Most findings reveal that parental involvement is positively related to
children’s math achievement and suggest that it may limit negative attitudes
toward mathematics (Mohr‐Schroeder et al., 2017). However, other meta-
analytic findings suggest that at-home parental involvement is negatively
related to children's achievement, as a negative correlation was found between
students’ academic performance and homework parental assistance (Wilder,
2015). Moreover, the gender gap in terms attitudes and anxieties transmission
has been investigated in several studies, findings emphasizing the role of gender
stereotype threats among adults and children (Chang & Beilock, 2016). For
instance, parents reported that girls need to do more effort in math learning than
boys do, while girls declared less confidence and less efficiency in their math
abilities than boys, following many years of exposure to this type of math
stereotype (Batchelor et al., 2017). It is very challenging to identify parental
influences on the relation between academic achievement and MA, due to
330
various several factors, such as family structure, parent educational level, family
income, parents’ occupation, and the history of parents’ performance in
mathematics (Soni & Kumari, 2017).
3. Current study
3.1. The importance of the study
The current study extends the existing literature on the relationship between
children’s MA and math achievement in several directions. For the first time, to
our knowledge, we measured both parents’ and child’s MA in the Palestinian
community, investigating potential gender differences in relation to their
mathematics achievement and the congruence with other forms of anxiety (trait
and test anxiety). Also, we explored the possible predictors of child mathematics
achievement. Finally, we attempted to investigate the possible interactive effects
of child MA and parental variables, such as parental involvement in children’s
math homework, parental history of parents’ math performance, and parental
MA on child math outcomes. To our knowledge, this is the first study to
investigate the interaction effect of parental variables on the relations between
child MA and child math performance.
3.2. Study questions

* Is there a positive association between various forms of anxiety e.g., Math
anxiety, Trait anxiety, and Test anxiety?
* Are there significant gender differences in Test/Math anxiety/Math
achievement, for children and in Math anxiety for parents?
* Is there a negative relation between child Math achievement and both Math
anxiety and Test anxiety?
* Is there a positive relation between a child's Math anxiety and parents' Math
anxiety?
* Does Math anxiety, Test anxiety, Trait anxiety, Gender, or Parental variables
play a role as a predictor of child's Math achievements?
* Are there possible interactive effects of child Math anxiety and parental
variables, such as parental involvement in children’s math homework, parental
history of parents’ math performance, and parental math anxiety on child math
outcomes?
3.3. Hypotheses
First, following evidence regarding the common ground of these anxiety
subtypes, a positive correlation among all forms of anxiety (math, test, and trait)
was expected. Second, following evidence showing gender-related differences in
MA, we predicted that female participants would report higher levels of MA,
trait anxiety and test anxiety. Third, in accordance with recent meta-analyses
revealing a moderate negative association between MA and math performance
(e.g., Barroso et al., 2021; Zhang, Zhao, & Ping Kong, 2019), we predicted a
replication of this result in our Palestinian sample. Importantly, regarding the
influence of parents, we expected parental MA to be positively related to
children’s MA and negatively predict children’s math achievement. Finally,
parental involvement, parents’ MA, and the history of parents’ math
331
performance during their own childhood were hypothesized to be moderators of

the relation between children's MA and their mathematics achievement.
4. Methodology
4.1. Participants
Participants were 230 students from four primary schools in Ramallah city (151
girls; Mean age = 8.9 years; SD = 0.59 Years) and from all students’ parents (N =
230, 74.8% mothers). From the total sample (N = 230), 104 participants (37 boys,
67 girls) were enrolled in the third grade, whereas 126 (42 boys, 84 girls) in the
fourth grade. All children were Palestinian, had intact or corrected vision, and
had Arabic as their primary language. Most children had a middle-class
background, with 88.8% of parents earning the average to above-average wage
per capita, 37.4% of the mothers and 25.3% of the fathers having a high-school
diploma, while 41.8 % mothers and 28.7% fathers had a college or university
degree.
4.2. Materials
4.2.1. Child’s measures
1- Math anxiety
The Scale for Early MA (SEMA; Wu, Amin, Barth, Malcarne, & Menon, 2012)
was used. Via its 20 items, children are asked to indicate on a five-point scale
how nervous they would feel if (1) they would have to answer certain math
questions (e.g., “George bought two pizzas that had six slices each. How many
total slices did George have to share with his friends?") and (2) in certain math-
related situations (e.g., "You are in class doing a math problem on the board").
Hence, the first 10 items assess children’s anxiety related to numeral processing
and the last 10 items assess their situational and performance anxiety (e.g., "You
are in class doing a math problem on the board"). The total score range is from
20 to 100, while the range of each subscale is from 10 to 50. A higher summed
score indicates greater MA. In the present sample, this scale had good reliability,
Cronbach’s α = .87.
2- Trait anxiety
We used the trait version of the State-Trait Anxiety Inventory (STAI-TC;
Spielberger, 1973). It consists of 20 items measuring children’s trait anxiety via
items such as “I am afraid to do things wrong”. Items are rated on a three-point
scale (1 = almost never, 2 = sometimes, 3 = often), with higher scores indicating
higher levels of trait anxiety. The scale showed high internal consistency in the
current sample, Cronbach’s α = .85.
3- Test anxiety
The Children Test Anxiety Scale (CTAS; Douglas & Jeri, 2004) is a 30-item self-
report questionnaire used to assess children’s test anxiety. A four-point Likert
scale is used to measure children’s frequency of autonomic reactions (physical
anxiety, e.g., “My heart beats fast”), off-task behaviours (e.g., “I play with my
pencil”), and worrisome thoughts (e.g., “I think about how poorly I am doing”)
while taking tests (1 = almost never, 2 = sometimes, 3 = often, 4 = almost
332
always). Higher scores indicate greater test anxiety. The scale showed good
internal consistency, Cronbach’s α = .88.
4- Math achievement
For students’ math achievement, we used the teachers’ final evaluation math
records at the end of a semester. Hence, these records represent students’
objective evaluation in math achievement at the end of the first school semester.
Performance scale range was between 0 – 100.
4.2.2. Parent’s measures

1- Math anxiety
The Mathematics Anxiety Rating Scale (MARS; Suinn &Winston, 2003) consists
of 30 items, with the first 15 items assessing math situational performance
anxiety (Math Test Anxiety) and the last 15 items assessing numerical processing
anxiety (Numerical Anxiety). A five-point Likert scale was used to indicate how
nervous the individual would feel (1 = I'm not nervous at all, 2 = I'm a little
nervous, 3 = I'm somewhat nervous, 4 = I'm very nervous, 5 = I'm very nervous)
during certain math-related situations, such as reading a cash register receipt.
The total score was the sum of answers on all items. Higher scores indicate
higher anxiety levels. In this sample, the scale showed a very good internal
consistency, Cronbach’s α = .93.
2- Parental involvement in children’s homework

We used the Parental involvement survey used by Maloney et al. (2015). Parents
completed a survey of five items, indicating how often they engage in certain
behaviours in order to help their child with their math homework (e.g., “Check
out the homework at the end”). A seven-point Likert scale was used to indicate
involvement frequency (from 1 = never to 7 = more than once a day). Total scale
score was computed summing all points received on the five items; hence,
higher scores are indicative of high involvement. In the current study, the
Cronbach’s α = .93.
3- Parental history of school performance

To investigate parents’ history of school performance, parents were asked to
complete a nine-items form about their performance in mathematics, Arabic
language, and general school performance in primary, middle and high school.
A five-point scale was used to indicate their level of performance, (1 = Poor to 5
= Excellent) for each subject (i.e., mathematics, Arabic, other) during each school
period (i.e., primary, middle school, high school). A higher score on indicated
higher school performance. The Cronbach’s alpha for the nine items scale was
.94.
4.3. Procedure
After permission was granted from the school authorities, written informed
consent was obtained from the parents whose children were in the 3rd and 4th
grades. While children's consent was obtained verbally, and their participations
were voluntary. Consequently, we organized a meeting during school hours
with the students whose parents provided their informed consent. Children
were also informed and assured about the confidentiality of their responses.
333
They were then asked to fill in the Arabic questionnaire in the following order:
MA, test anxiety, trait anxiety. The researcher collaborated with two teachers
during these meetings with the children, all children's questions were answered,
and the ones who needed help in reading were helped, while no time restrictions
were applied Parents who accepted to participate received the questionnaires
via their child or while they were picking the child from school, they completed
the forms in the following order: demographic information, parental
involvement, parental history of school performance, and the MA scale. The
researcher's contact details were provided for any clarifications.
5. Results
The SPSS statistics 25th version was used to process our raw data, the obtained
results are in section below

Descriptive statistics regarding children’s MA, trait anxiety test anxiety, math
academic achievement, as a function of gender are displayed in Table 1.
Table 1: Means and standard deviations for children’s measures

Boys Girls
N = 230
n= 79 n= 151
M M M t-test Cohen’s
(SD) (SD) (SD) (228) d
Math anxiety 29.09 26.92 30.20 -2.32* .33
(10.32) (8.36) (11.07)
Trait anxiety 30.18 28.55 31.03 -2.48* .35
(7.27) (6.73) (7.41)
Test anxiety 52.39 50.72 53.26 -1.23 -
(14.83) (14.72) (14.86)
Math achievement 84.70 85.99 1.34 -
(12.12) (11.81) (12.26)
Note. **p < .001, *p < .005
Looking at gender-related differences in parental reports, mothers reported higher
levels of MA, t (169) = -2.43, p = .016, Cohen’s d = 0.40, while nonsignificant
differences were found between mothers and fathers in terms of parental
involvement and history of math performance. Means and standard deviations
as function of parents’ gender are presented in Table 2.
Table 2: Means and standard deviations for parents’ measures

Fathers Mothers
n M(SD) n M(SD)
Parents’ Math anxiety 74 71.23 (25.69) 124 81.26 (23.40)
Parents’ involvement in homework 57 27.15 (6.13) 168 27.82 (6.28)
Parents’ history of math performance 47 10.61(2.90) 146 10.90 (2.98)
334
5.2. Correlations among study variables

Pearson correlations revealed positive moderate correlations among the three
forms of anxiety measured in children: MA, trait anxiety, and children test
anxiety (see Table 3), suggesting that while they all tap the same general
concept, they measure different faucets of anxiety. Negative correlations were
found between children's math achievement and their MA, and test anxiety,
respectively. Moreover, children’s math achievement was also negatively
associated with their parents’ own MA, but positively associated with parental
involvement in children’s math homework and history of math performance.
Nevertheless, the correlation between parents' MA and their children's MA did
not reach significant levels. Additionally, we found a negative moderate
correlation between parents’ MA and their own math performance history.
Table 3: Correlations among study variables
Measures 1 2 3 4 5 6
1 Math anxiety -
2 Trait anxiety .55** -
3 Test Anxiety .55** .52** -
4 Math achievement -.25** -.08 -.14* -
5 Parents' Math anxiety .13 .07 .09 -.27** -
6 Parents’ involvement in -.04 .03 -.04 .15* -.07 -
homework
7 Parents’ history of math -.15* -.01 -.06 .37** -.55** .23**
performance
Note. **p < .001, *p < .005
To check for potential different associations between the same-gender parent-
child dyads, additional zero order correlations were calculated between same-
gender parents' MA and child MA. Correlations computed between mothers and
daughters and between fathers and sons revealed a significant positive
association between mother’s MA levels and daughters’ levels of MA, (r = .25, p
= 0.02). In contrast, nonsignificant associations were found between fathers’ MA
levels and sons’ MA (p = .98)
5.3. Predictors of children’s math achievement

To analyse contributions of individual characteristics, parental factors, and
specific math-anxiety factors to children’s math achievement variation, a three-
step hierarchical multiple regression was conducted with children’s math
achievement as the criterion. Potential baseline individual differences (gender,
grade, trait anxiety, test anxiety) were entered Step 1. Based on previous results
and current correlations, parental variables (parents’ MA, parental involvement
in children’s math homework, and the history of parents' math performance)
were added in Step 2. To test whether children’s own MA levels contribute to
variations in math achievement over and above the influence of parental history
with math and homework involvement, children’s MA was added in Step 3 (see
Table 4).
335
Table 4: Summary of hierarchical regression analysis for potential predictors of

children’s math achievement
Predictor B ꞵ t R R2 ΔR2 p
Step 1 .23 .05 .05 .07
Gender -2.83 -.11 -1.42
Grade -3.12 -.12 -1.54
Trait anxiety .17 .09 .94
Test anxiety -.16 -.19 .04*
Step 2 .40 .16 .11 .00

Parents' history of math .77 .14 1.47
performance
Parents’ Math anxiety -.10 -.12 -2.12*
Parents’ involvement in .28 .17 1.69
homework
Step 3 .46 .21 .05 .03

Gender -3.54 -.13 -1.83
Grade -3.14 -.12 -1.67
Trait anxiety .41 .21 2.11*
Test anxiety -.07 -.08 -.92
Parents' history of math .55 .09 1.08
performance
Parents’ Math anxiety -.09 -.19 -2.12*
Parents’ involvement in .25 .12 1.57
homework
Math anxiety -.38 -.29 -3.32*
Note. *p < .05, **p < .01, ***p< .001
The hierarchical multiple regression revealed that baseline individual

differences predictors had no significant contribution to the regression model.
Introducing the parental variables explained an additional 11.1% of variation in
children math achievement and this change in R² was significant, F (3,158) =
4.35, p < .001. As visible in Table 4, parents’ MA was a significant predictor of
children’s math achievement, with higher levels of parental anxiety predicting
lower math achievement scores in children. On the other hand, parental history
in terms of math achievement and parental involvement with the child’s math
homework did not prove to be significant predictors of the child’s math
achievement. Adding children’s own MA to the regression model explained an
additional 4.6% of the variation in the dependent variable, F (1,157) = 5.14, p <
.001. When all independent variables were included in this third step of the
regression model, the significant predictors of children’s math achievement
were: baseline differences in children’s trait anxiety, parents’ MA, and children’s
MA. This final model accounted for 20.7% of the variance in children math
achievement.
336
5.4. The potential interaction effects between children’s Math anxiety and
parental variables on children’s math outcomes
Additional models analysing potential interaction effects between children’s MA
and parental variables (parent’s MA, parent involvement in child math
homework, and the history of parents’ math performance) revealed
nonsignificant interaction effects of child and parental MA on children’s math
outcome, b = 0.01, p = .08, BCa CI [-0.001, 0.011], of children’s MA and parents’
involvement in math homework, b = 0.01, p = .28, BCa CI [-0.01, 0.04], and of
children’s MA and parents’ math performance, b = 0.05, p = .16, BCa CI [-0.02,
0.13].
6. Discussion
The study aimed to investigate the prevalence of MA and identify gender
differences among primary school students and their parents, and explore the
relation between MA and math achievement as a function of parental factors.
The main findings included confirming higher levels of MA among
girls/mothers than among boys/fathers, a negative relation between children’s
MA and their math achievement, also a negative association between children’s
math achievement and their parent’s MA. In addition, we uncovered the
possible predictors of math achievement and the possible moderating role of
parental variables in the relation between children's MA and their math
achievement. Next, we will discuss these results, integrating them in the
growing body of literature on factors generating or minimizing MA in primary
school children.

6.1.1. Gender differences in Math anxiety
The results showed significant gender differences in MA, as girls reported
higher scores compared to boys, consistent with many previous studies
conducted in China, Poland, England, and Uganda (see Xie et al., 2018; Schnell et
al .2013; Devine et al .2012; Hunt et al., 2021). Similarly, mothers reported higher
MA compared to fathers (Else-Quest et al., 2010). Our findings are in line with
the results of a meta-analysis of 151 studies (Hembree, 1990), which found that
females tend to have higher levels of MA than males do, which may account for
some of the gender gaps in math achievement and math-related professions.
A similar finding was reported by Carey et al. (2017), among British children
aged 8–13, where a significantly higher levels of MA among girls compared to
boys were found. A study was conducted by Ho et al. (2000) among 6th-grade
students from the USA, China, and Taiwan revealed that there were significantly
higher scores of MA among Taiwanese girls, while no MA gender differences
were found among Chinese and American students. On the other hand, Birgin et
al. (2010) or Tapia (2004) reported no significant difference between boys and
girls in MA.
One potential explanation for girls/females generally experiencing greater levels

of MA than males may stem from the gender differences in socialization
practices. In particular, women are socialized to express their feelings and
emotions, this may result in the inclination for women to admit their fears more
337
than men do (Devine, 2012; Kavanagh et al., 2016). Another potential reason is
that math is traditionally seen as a male domain, so females may be socialized to
perceive their mathematical skills as less competent and therefore may even
avoid mathematical activities. It was expected for females in this study to report
higher levels of MA compared to males, according to the Arabic culture that
imposes such gender roles, where boys are raised to be tough and brave, which
can lead them to report lower levels of MA. Not surprisingly, females may be
more willing to admit their worries and anxiety (Kavanagh et al., 2016). Also,
there is less recent research examining the gender biases in the Palestinian
school mathematics textbooks indicating these textbooks are male-biased, with
females being less likely to be represented by names, pictures, verbs (actions),
pronouns and professions (Karama, 2020). Women’s underrepresentation in
science, technology, engineering, and mathematics (STEM) fields emerges from
these gender biases, Palestinian women avoid math-related positions due to
their beliefs that these areas are unimportant or even damaging to their self-
image as females (Rube & Ehrenfeld, 2020).
6.1.2. Gender differences in other forms of anxiety

Girls reported higher levels of trait anxiety than boys, confirming many similar
findings in the literature across various age groups (Macher et al., 2011; Putwain
& Daly, 2014), while no significant gender differences were found in test anxiety.
The results among Australian, American, Romanian, and Chinese students
confirmed our previous findings of no gender differences in test anxiety
(Kavanagh et al., 2016; Popa, 2019; Xie, 2018). However, our results are not in
line with previous studies that did find gender differences in test anxiety in
favour of boys (Erturan & Jansen, 2015) or in favour of girls (McDonald, 2001;
Putwain & Daly, 2014).
A cautionary note refers to the overall lower levels of test anxiety found in our
sample. In the Palestinian educational system, primary students during 1st to
4th grades are not exposed to formal examination sessions, but teachers still
have to evaluate children's performance using many strategies such as team or
pair work, homework, and class evaluative papers. The absence of standardized
tests or formal examination environments could explain the lower levels of test
anxiety compared to Wren and Benson (2004) findings, who used the same scale
of children's test anxiety.
6.1.3. Gender differences in Math achievement

Although the results indicated that girls were more math anxious than males in
math-related situations, no significant differences in math achievements were
documented. This result is confirming many previous findings suggesting that
the gender gap in math performance has been significantly minimized in the last
decades, especially within primary school students (Devine et al., 2012; Schnell
et al., 2013). One possible explanation for girls outperforming or performing
equally in math is that girls are more self-disciplined than boys, in general girls
tend to study longer hours and do more homework, while boys need more
monitoring to do their homework (Duckworth & Seligman, 2006). On the other
hand, our results do not confirm previous findings suggesting that males
outperform females in math (Else-Quest et al., 2010, see also, Erturan & Jansen,
2015; Osborne, 2006). Although many recent studies showed the gender gap
338
minimizing across the years (Gunderson et al., 2011; Hyde et al., 2008; Schnell et
al, 2013) fewer girls end up pursuing math courses or math-related career paths
(Eccles, 2009), a tendency visible in Palestine as well (Rubel & Ehrenfeld, 2020).
6.2. Correlations among study variables

6.2.1. Congruence between anxiety measures
Findings revealed a positive moderate correlation between the MA and the Trait
Anxiety. Similar findings were revealed by Ashcraft and Moore (2009), who
reported a positive correlation between MA and trait anxiety. Although we
confirmed this consistency among the mentioned scales, it also appears that each
one of them still measures different aspects of anxiety. A positive correlation
between MA and Test Anxiety was found, also a positive correlation between
Trait Anxiety and Test Anxiety. Similarly, Devine’s et al. (2012) study among
secondary school students in England, and Joseph (2009) study among
secondary students in Singapore also reported positive correlations between MA
and test anxiety. In fact, MA was conceptualized as a situation-specific anxiety
demonstrated in mathematics-related activities (Rubinsten et al., 2015), while
Test Anxiety was seen as a situation-specific personality trait, which specifically
arises in evaluative situations (Schnell et al. 2013).
6.2.2. Math anxiety and math performance

The results also found a negative correlation between child math performance
and child MA. In this respect, Hembree's meta-analysis (1990) showed that MA
negatively correlates with math achievement and math grades. Later, a similar
finding was confirmed by Ma (1999) in his meta-analysis. Also, Cipora et al.
(2015) and Schnell et al. (2013) findings revealed a negative association between
MA and math performance among Polish and German students, respectively.
6.2.3. Parent-child Math anxiety

On parent-child math anxiety results indicated that parents’ MA and child math
achievements were negatively correlated. Similar results were found by
Berkowitz et al. (2015), who conducted a study about math at home and child
achievements. Such results suggested when parents are more math anxious,
their children learned less math during first grade compared to children of less
math-anxious parents. Another study reported that children's math performance
was negatively associated with high parent MA, but only when both mothers
and daughters or when fathers and sons had high MA (Casad et al., 2015).
The results also showed significant associations between mothers' MA and their
daughter’s MA, while no correlation was found between fathers and sons. A
possible explanation for this gender effect arises from the fact that mothers and
girls in this study are found to be more math-anxious compared to males. Also,
the gender stereotype threat of math as a male domain may negatively affect
female’s MA levels across the lifespan.
In the line with the present findings, Casad et al. (2015) carried out a study
among students in the 6th to 8th grades. Their results confirmed that parents'
MA was related to children's MA and both variables interacted to predict
mathematics outcomes. Also, in a sample of Indian children aged 10 to 15 years,
339
Soni and Kumari (2017) confirmed that parents' MA was positively associated
with children’s MA and negatively affected their math attitude. In contrast,
Jameson (2013) examined the environmental factors relating to MA in 2nd-grade
students (aged 7–9 years) and found no significant correlation between parents'
MA and their children's MA. Another study conducted by Batchelor et al. (2017)
indicated that children's MA is related to parents' MA, more specifically, a
positive association between parents' MA and sons was calculated, while no
association with daughter’s levels of MA was found.
6.3. Predictors of child math achievement

A weak but positive correlation was found between parental involvement in
child math homework and child math achievement. Similar findings were
reported by Fan’s and Chen’s (2001) meta-analysis in which a positive
association was calculated between academic achievement in young children
and parental involvement at home and school. Our findings are not in line with
other studies indicating home parental involvement to be negatively related to
children's achievement (Wilder, 2015).
In order to identify the main predictors of child mathematics achievement, a
hierarchical multiple regression in 3 steps was conducted. The regression
analysis showed that child MA explained 4.6% of the variance in the math
achievement score, while parental variables (parent's MA, parent involvement in
child math homework, and the history of parents' math performance) explained
11% of the variance in children math achievement. Test anxiety was a predictor
of child mathematics achievement only in the first stage of the regression model,
while after controlling all other variables in the last stage, the ultimate predictors
of math achievements were children’s MA, parents’ MA, and child trait anxiety.
Findings revealed that children’s MA and parents’ MA were both predictors of

children’s math achievement, being consistent with the Maloney et al. (2015)
study, which confirmed the role of parents' MA as a predictor of children's math
achievement but only for children whose parents were involved in math
homework. Similar results were reported by Casad et al. (2015) who suggested
parents’ anxiety as a predictor of children’s math education outcomes. In
addition, a meta-analysis proposed that students’ MA levels can significantly
predict their mathematics performance (Ma, 1999). However, Hembree’s meta-
analysis (1990) revealed that MA was more predictive of math performance in
boys than in girls.
In contrast to our findings that are not revealing a predictive role of parents’
involvement in child math homework for their math performance, other studies
confirmed the suggestion of parental involvement as an important predictor of
math achievements (Harackiewicz et al., 2012; Hill & Taylor, 2004). Interestingly,
the way parents interact with their children and their spontaneous reaction to
math is a better predictor of children’s outcomes than the level of school parental
involvement (Wilder, 2015).
340
6.4. The potential interaction effects between children’s Math anxiety and
parental variables on children’s math outcomes
A moderation interaction analysis was run to examine if parental variables
(parent's MA, parent involvement in child math homework, and the history of
parents' math performance) moderate the relation between child MA and child
math performance. Findings of this study showed that none of them was playing
a moderator effect on the relation. It is important to note that regardless of
causal direction, parental variables didn’t moderate the relation between anxiety
and performance. In fact, a majority of parents in our sample tended to rate
themselves as highly involved, so there was little variation in the levels of
parent's involvement that could be a reason for making parental involvement
moderate the relation between child MA and child math performance. Due to
the self-reported measure used in this study, parents may have provided socially
desirable responses about their level of involvement in their child's math
homework, rather than indicating their actual parental involvement level
(Warren et al., 2018).
7. Conclusion
The findings of this study substantially contribute to the growing body of
literature addressing several factors affecting mathematics achievements and
future career orientations: Math anxiety, test anxiety and possible gender
differences. Moreover, the potential interaction between child MA and parental
variables on child math outcomes revealed significant gender differences in
Math levels, young girls in primary school and also their mothers, reported
higher levels of MA. In addition, the study showed that MA levels significantly
and negatively affected mathematics achievements. Both child’s MA and their
parent's MA were found to be strong predictors of children's math
achievements. With this, the current study offers the first systematic
investigation of MA in Palestinian primary schools, offering valuable insights
into the relation between math achievements and MA and gender differences, in
addition to parental role in the transmission of MA. Its results offer valuable
avenues to monitor and enhance the female presence in math-related fields in
the Palestinian workforce. Thus, it is recommended that further investigations of
the barriers to females' participation in the Palestinian STEM workforce could be
conducted.
8. Limitations
The study findings need to be interpreted in light of some limitations. The use of
self-report measures for children and parents, which may result in systematic
response distortions is considered the first limitation of the current study. A
second limitation is that child's math achievement was based on teachers'
records and, since the data are obtained from several schools, although the
grading system was identical, this can mask different standards of performance
for math achievement between schools. A third limitation refers to the sample
composition, since the majority of the participants were females such as girls
(65.6%), mothers (74.8%), the male sample size was rather small for the effect we
wanted to observe.
341
Acknowledgments
We would like to express our gratitude to everyone who contributed to the
realization of this work, teachers, children, parents, colleagues, and colleagues at
Babeș-Bolyai University. Our research was funded by a Romanian state doctoral
scholarship awarded to the first author (Contract no. DGRIAE
875/III/54/CMJ/28.09.2017) and a Young Teams Project, PN-III-P1-1.1-TE-2016-
2170, awarded to the supervisor (https://www.minimanx.com/).
9. References
Anbar, N., & Visu-Petra, L. (2021). Intersecting parent and child math anxiety, parental
math-gender stereotypes and children’s math performance: A scoping review.
Revista de Psihologie, 67(4).
Ashcraft, M. H. (2002). Math anxiety: Personal, educational, and cognitive consequences.
Current directions in psychological science, 11(5), 181-185.
https://doi.org/10.1111/1467-8721.00196
Ashcraft, M. H.; Moore, A. M. (2009). Mathematics Anxiety and the Affective Drop in
Performance. Journal of Psychoeducational Assessment, 27(3), 197–
205. https://doi.org/10.1177/0734282908330580
Batchelor, S., Gilmore, C., & Inglis, M. (2017). Parents’ and Children’s Mathematics
Anxiety. Understanding Emotions in Mathematical Thinking and Learning, Academic
Press, 315–336. https://doi.org/10.1016/b978-0-12-802218-4.00012-1
Berkowitz, T., Schaeffer, M. W., Maloney, E. A., Peterson, L., Gregor, C., Levine, S. C., &
Beilock, S. L. (2015). Math at home adds up to achievement in school. Science,
350(6257), 196–198. http://doi.org/10.1126/science.aac7427
Birgin, O., Baloğlu, M., Çatlıoğlu, H., & Gürbüz, R. (2010). An investigation of
mathematics anxiety among sixth through eighth grade students in Turkey.
Learning and Individual Differences, 20(6), 654-658.
https://doi.org/10.1016/j.lindif.2010.04.006
Carey, E., Hill, F., Devine, A., & Szűcs, D. (2017). The Modified Abbreviated MA Scale: A
Valid and Reliable Instrument for Use with Children. Frontiers in Psychology, 8,
11. https://doi.org/10.3389/fpsyg.2017.00011
Casad, B. J., Hale, P., & Wachs, F. L. (2015). Parent-child math anxiety and math-gender
stereotypes predict adolescents' math education outcomes. Frontiers in
psychology, 6, 1597. https://doi.org/10.3389/fpsyg.2015.01597
Chang, H., & Beilock, S. L. (2016). The Math anxiety-math performance link and its
relation to individual and environmental factors: a review of current behavioral
and psychophysiological research. Current Opinion in Behavioral Sciences, 10, 33-
38. https://doi.org/10.1016/j.cobeha.2016.04.011
Cipora, K., Szczygieł, M., Willmes, K., & Nuerk, H. C. (2015). Math anxiety assessment
with the abbreviated Math anxiety scale: applicability and usefulness: insights
from the polish adaptation. Frontiers in psychology, 6, 1833.
https://doi.org/10.3389/fpsyg.2015.01833
Daches Cohen, L., & Rubinsten, O. (2017). Mothers, Intrinsic Math Motivation,
Arithmetic Skills, and MA in Elementary School. Frontiers in psychology, 8, 1939.
https://doi.org/10.3389/fpsyg.2017.01939
Devine, A., Fawcett, K., Szűcs, D., & Dowker, A. (2012). Gender differences in
mathematics anxiety and the relation to mathematics performance while
controlling for test anxiety. Behavioral and brain functions, 8(1), 1-9.
https://doi.org/10.1186/1744-9081-8-33
342
Duckworth, A. L., & Seligman, M. E. (2006). Self-discipline gives girls the edge: Gender
in self-discipline, grades, and achievement test scores. Journal of Educational
Psychology, 98(1), 198. https://doi.org/10.1037/0022-0663.98.1.198
Eccles, J. (2009). Who Am I and What Am I Going to Do with My Life? Personal and
Collective Identities as Motivators of Action. Educational Psychologist, 44(2), 78–
89. https://doi.org/10.1080/00461520902832368
El Feki, S., Heilman, B. and Barker, G., Eds. (2017). Understanding Masculinities: Results
from the International Men and Gender Equality Survey (IMAGES) – Middle East and
North Africa. Cairo and Washington, D.C.: UN Women and Promundo-US.
Else-Quest, N. M., Hyde, J. S., & Linn, M. C. (2010). Cross-national patterns of gender
differences in mathematics: a meta-analysis. Psychological bulletin, 136(1), 103–
127. https://doi.org/10.1037/a0018053
Erturan, S., & Jansen, B. (2015). An investigation of boys’ and girls’ emotional experience
of math, their math performance, and the relation between these variables.
European Journal of Psychology of Education, 30(4), 421-435.
https://doi.org/10.1007/s10212-015-0248-7
Fan, X., & Chen, M. (2001). Parental involvement and students' academic achievement: A
meta-analysis. Educational psychology review, 13(1), 1-22.
https://doi.org/10.2307/23358867
Frenzel, A. C., Pekrun, R., & Goetz, T. (2007). Girls and mathematics—A “hopeless”
issue? A control-value approach to gender differences in emotions towards
mathematics. European Journal of Psychology of Education, 22(4), 497-514.
https://doi.org/10.1007/bf03173468
Gunderson, E. A., Ramirez, G., Levine, S. C., & Beilock, S. L. (2011). The Role of Parents
and Teachers in the Development of Gender-Related Math Attitudes. Sex Roles,
66(3-4), 153–166. https://doi.org/10.1007/s11199-011-9996-2
Harackiewicz, J. M., Rozek, C. S., Hulleman, C. S., & Hyde, J. S. (2012). Helping Parents
to Motivate Adolescents in Mathematics and Science. Psychological Science, 23(8),
899–906. https://doi.org/10.1177/0956797611435530
Hembree, R. (1990). The Nature, Effects, and Relief of Mathematics Anxiety. Journal for
Research in Mathematics Education, 21(1), 33. https://doi.org/10.2307/749455
Hill, N. E., & Taylor, L. C. (2004). Parental school involvement and children's academic
achievement: Pragmatics and issues. Current directions in psychological science,
13(4), 161-164. https://doi.org/10.1111/j.0963-7214.2004.00298.x
Ho, H.-Z., Senturk, D., Lam, A. G., Zimmer, J. M., Hong, S., Okamoto, Y., Chiu, S.-Y.,
Nakazawa, Y & Wang, C.-P. (2000). The Affective and Cognitive Dimensions of
Math anxiety: A Cross-National Study. Journal for Research in Mathematics
Education, 31(3), 362. https://doi.org/10.2307/749811
Hopko, D. R., Mahadevan, R., Bare, R. L., & Hunt, M. K. (2003). The Abbreviated Math
Anxiety Scale (AMAS): Construction, Validity, and Reliability. Assessment, 10(2),
178–182. https://doi.org/10.1177/1073191103010002008
Huang, X., Zhang, J. & Hudson, L. Impact of math self-efficacy, Math anxiety, and
growth mindset on math and science career interest for middle school students:
the gender moderating effect. European Journal of Psychology of Education, 34, 621–
640 (2019). https://doi.org/10.1007/s10212-018-0403-z
Hunt, T., Muwonge, C., Ashraf, F., Asanjarani, F., Kotera, Y., Vione, K., Sheffield, D.,
Mir, R., & Waldron, M. (2021). Psychological barriers in maths education:
Insights from Iran, Pakistan, and Uganda. University of Derby.
https://doi.org/10.13140/RG.2.2.36106.59844/1
343
Hyde, J. S., Lindberg, S. M., Linn, M. C., Ellis, A., and Williams, C. (2008). Gender
similarities characterize math performance, Science, 321, 494–495.
https://doi.org/10.1126/science.1160364
Jacobs, J. E., & Eccles, J. S. (2000). Parents, task values, and Real-Life achievement-related
choices. Intrinsic and Extrinsic Motivation, 405–439.
https://doi.org/10.1016/b978-012619070-0/50036-2
Jameson, M. M. (2013). Contextual Factors Related to Math anxiety in Second-Grade
Children. The Journal of Experimental Education, 82(4), 518–536. https://
doi.org/10.1080/00220973.2013.813367
Joseph, Y. K. K. (2009). Mathematics Anxiety and Test Anxiety of Secondary Two
Students in Singapore. Mathematics Education, 319–336.
https://doi.org/10.1142/9789812833761_0014
Karama, M. J. (2020). Gender Bias in School Mathematics Textbooks from Grade 1 to 12
in Palestine. Journal of International Women's Studies, 21(1), 162-171.
https://doi.org/10.31219/osf.io/u6pf4
Kavanagh, B. E., Ziino, S. A., & Mesagno, C. (2016). A comparative investigation of test
anxiety, coping strategies and perfectionism between Australian and united
states students. North American Journal of Psychology, 18(3), 555-570.
Lyons, I. M., & Beilock, S. L. (2011). Mathematics Anxiety: Separating the Math from the
Anxiety. Cerebral Cortex, 22(9), 2102–2110.
https://doi.org/10.1093/cercor/bhr28
Ma, X. (1999). A Meta-Analysis of the Relationship between Anxiety toward
Mathematics and Achievement in Mathematics. Journal for Research in
Mathematics Education, 30(5), 520–540. https://doi.org/10.2307/749772
Macher, D., Paechter, M., Papousek, I., & Ruggeri, K. (2011). Statistics anxiety, trait
anxiety, learning behavior, and academic performance. European Journal of
Psychology of Education, 27(4), 483–498. https://doi.org/10.1007/s10212-011-
0090-5
Maloney, E. A., & Beilock, S. L. (2012). Math anxiety: who has it, why it develops, and
how to guard against it. Trends in Cognitive Sciences, 16(8), 404–406.
https://doi.org/10.1016/j.tics.2012.06.008
Maloney, E. A., Ramirez, G., Gunderson, E. A., Levine, S. C., & Beilock, S. L. (2015).
Intergenerational Effects of Parents’ Math anxiety on Children’s Math
Achievement and Anxiety. Psychological Science, 26(9), 1480–1488.
https://doi.org/10.1177/0956797615592630
McDonald, A. S. (2001). The Prevalence and Effects of Test Anxiety in School Children.
Educational Psychology, 21(1), 89–101.
https://doi.org/10.1080/01443410020019867
Mohr‐Schroeder, M. J., Jackson, C., Cavalcanti, M., Jong, C., Craig Schroeder, D., &
Speler, L. G. (2017). Parents’ Attitudes Toward Mathematics and the Influence
on Their Students’ Attitudes toward Mathematics: A Quantitative Study. School
Science and Mathematics, 117(5), 214-222. https://doi.org/10.1111/ssm.12225
Osborne, J. W. (2006). Gender, stereotype threat, and anxiety: Psichophysiological and
cognitive evidence. Electronic Journal of Research in Educational Psychology, 4(1),
109-137. https://doi.org/10.1080/01443410601069929
Palestinian statistical center. (2020). Men and Women in Palestine, statistic book 2020.
Ramallah. Palestine.
PISA. (2012). Results: Ready to Learn (Volume III): Students’ Engagement, Drive and
Self-Beliefs. Paris: OECD Publishing; 2013.
http://dx.doi.org/10.1787/9789264201170-en
344
Popa, C., Bonchis, L., & Clipa, O. (2018). School assessment and test anxiety at primary
school pupils. In 4th International Conference on Lifelong education and
Leadership for all (pp. 867-874).
Pugsley, A., & Price, J. (2018). Back to School: A focus on Math anxiety.
Putwain, D., & Daly, A. L. (2014). Test anxiety prevalence and gender differences in a
sample of English secondary school students. Educational Studies, 40(5), 554–570.
https://doi.org/10.1080/03055698.2014.953914
Ramirez, G., Shaw, S. T., & Maloney, E. A. (2018). Math anxiety: Past research, promising
interventions, and a new interpretation framework. Educational Psychologist,
53(3), 145-164. https://doi.org/10.1080/00461520.2018.1447384
Rubel, L. H., & Ehrenfeld, N. (2020). Palestinian/Arab Israeli women’s experiences in
mathematics education: An intersectional analysis. International Journal of
Educational Research, 102, 101616. https://doi.org/10.1016/j.ijer.2020.101616
Rubinsten, O., Eidlin, H., Wohl, H., & Akibli, O. (2015). Attentional bias in math anxiety.
Frontiers in psychology, 6, 1539.https://doi.org/10.3389/fpsyg.2015.01539
Rubinsten, O., Marciano, H., Eidlin Levy, H., & Daches Cohen, L. (2018). A Framework
for Studying the Heterogeneity of Risk Factors in Math anxiety. Frontiers in
Behavioral Neuroscience, 12. https://doi.org/10.3389/fnbeh.2018.00291
Schnell, K., Tibubos, A. N., Rohrmann, S., & Hodapp, V. (2013). Test and Math anxiety:
A validation of the german test anxiety questionnaire. Polish Psychological
Bulletin, 44(2), 193. http://doi.org.am.e-nformation.ro/10.2478/ppb-2013-0022
Soni, A., & Kumari, S. (2017). The role of parental Math anxiety and math attitude in
their children’s math achievement. International Journal of Science and Mathematics
Education, 15(2), 331-347. https://doi.org/10.1007/s10763-015-9687-5
Suinn, R. M., & Winston, E. H. (2003). The Mathematics Anxiety Rating Scale, a Brief
Version: Psychometric Data. Psychological Reports, 92(1), 167–173.
http://doi.org/10.2466/pr0.2003.92.1.167
Tapia, M., & Marsh, G. E. (2004). The relationship of Math anxiety and gender. Academic
Exchange Quarterly, 8(2), 130-134.
Vukovic, R. K., Roberts, S. O., & Green Wright, L. (2013). From Parental Involvement to
Children's Mathematical Performance: The Role of Mathematics Anxiety. Early
Education & Development, 24(4), 446–467.
http://doi.org/10.1080/10409289.2012.693430
Warren, J. M., Locklear, L. A., & Watson, N. A. (2018). The Role of Parenting in
Predicting Student Achievement: Considerations for School Counseling Practice
and Research. Professional Counselor, 8(4), 328-340.
http://doi.org/10.15241/jmw.8.4.328
Wilder, S. (2015). Parental involvement in mathematics: giving parents a voice. Education
3-13, 45(1), 104–121. https://doi.org/10.1080/03004279.2015.1058407
Wren, D. G., & Benson, J. (2004). Measuring test anxiety in children: Scale development
and internal construct validation. Anxiety, Stress & Coping, 17(3), 227-240.
https://doi.org/10.1080/10615800412331292606
Wu, S., Amin, H., Barth, M., Malcarne, V., & Menon, V. (2012). Math anxiety in second
and third graders and its relation to mathematics achievement. Frontiers in
psychology, 3, 162. https://doi.org/10.3389/fpsyg.2012.00162
Xie, F., Xin, Z., Chen, X., & Zhang, L. (2018). Gender Difference of Chinese High School
Students’ Math anxiety: The Effects of Self-Esteem, Test Anxiety and General
Anxiety. Sex Roles. https://doi.org/10.1007/s11199-018-0982-9
345

Vol. 21, No. 8, pp. 345-367, August 2022
Investigating the Tertiary Level Students’

Practice of Collaborative Learning in English
Language Classrooms, and Its Implications at
Public Universities and at Arabic Institutions
Md Nurul Anwar*
English Language Institute (ELI), School of Humanities and Social Sciences, United
International University (UIU), Dhaka, Bangladesh
Md Nurul Ahad
Department of English, School of Humanities and Social Sciences, University of Asia
Pacific (UAP), Dhaka, Bangladesh
Md. Kamrul Hasan

English Language Institute
School of Humanities and Social Sciences, United International University (UIU),
Dhaka, Bangladesh
Abstract. The benefits of Collaborative Learning have been emphasized in

the ELT literature; however, its positive implications have remained few
and far between at the higher educational institutions in Bangladesh. CLT
has been introduced in Bangladesh; however, the majority of Public
Universities and Madrasa Systems conspicuously lack trained ELT
professionals who are aware of collaborative learning. Thus, the practical
use of CLT in classrooms still remains a myth let alone the
implementation of collaborative learning. The current study sought to
identify the impact of collaborative learning on students’ performance
rate in their presentation test results, and to identity their overall
perspectival response rates to (CL). The study analyzed three hundred
and ten responses relating to their experience of studying in collaboration
with their fellow classmates during the preparatory stages of knowledge
production and gathering information that involved gathering ideas and
having discussions on social media (i.e. Facebook) towards the
accomplishment of their project assignments. Then, the study compares
students’ previous presentation test results derived from presentations
conducted in the pre-collaborative stage to the test results derived from the
post-collaborative stage. Finally, the study ran several statistical tests, and
found that students responded positively to collaborative learning, and
they made significant improvements in the post-collaborative test.
* Corresponding author: Md Nurul Anwar; nurul@eli.uiu.ac.bd
©Authors
346
Keywords: collaborative learning; pre-collaborative state; post-

collaborative stage; communication skills
1. Introduction
After the British left Indian Sub-Continent, India was divided into two dominions,
the East and the West Pakistan. Throughout the Pakistani period, English was a
second language in Bangladesh as a result of British control. In the business,
professions, school, and other contexts, people who were educated or even
marginally educated were compelled to use English as English was a link
language between the West and East Pakistan (Hamid, 2016, p.28). The English
language, however, was downgraded to the status of a foreign language in
independent Bangladesh following the liberation battle in 1971. Almost
everything is done in Bengali in this monolingual country, and English is not
utilized in regular discourse. When they were forced to communicate in English,
they began to struggle. (Akbas, 2016).
In the late 18th century, a group of British people brought English education
to the subcontinent, establishing a number of English medium schools.
However, it was formalized in 1835 by Lord Macaulay's Minute. The approval
of this Minute by Lord William Bentinck formed the foundation for British
education policy in then-British India. After the liberation of Bangladesh, the
status of English as the second language was declined and remained
unidentified. It was rather neglected in the offices and educational institutes.
Bengali replaced English in all official communications except those in courts,

foreign missions, and armies which still use English as the official language.
To stress the individuality of “Bengali” in 1972, English was omitted from the
primary and tertiary level of education. Bengali became the only medium of
instruction in secondary and higher secondary schools; all schools including
the English medium institutes were brought under a general order to switch
to Bengali with immediate effect (Islam, 1975). But it was still taught as a
subject in the secondary level. Azam and Tatsuya (2018) stated, “a huge wave
went on in changing the names of places and institutes into Bengali and
translating books into Bengali” (p.107). This is owing to ardent nationalism
that everybody instantaneously grew animosity towards learning and using
English almost dramatically.
Without a doubt, Bangladesh has seen the prevalence of many streams of

education, such as Bengali medium, English medium, Madrasa education, and
other medium where discrete teaching-learning of English has been seen for
a long time. Because there are no defined instructions from the authorities,
particular implementation of appropriate methods such as Communicative
Language Teaching (CLT) is exceedingly challenging under these diverse
streams. Looking at Bengali medium schools and colleges, it is clear that CLT
is rarely employed due to a variety of factors such as current materials,
experienced teachers, and technological assistance (Ahmed, 2016, p.98).
Therefore, this study aimed at investigating the implementation of
347
collaborative learning, and making the teachers aware of its benefits in

developing English language skills, particularly speaking.
2. Collaborative Learning
The definition of collaborative learning is "the use of small groups in which
students work to accomplish shared objectives and maximize both their own and
others' potential" (JONY, 2019).
The teacher's duty is to foster an atmosphere in which students are
eager and able to collaborate. The setting must also provide many
chances and interesting situations for learners to collaborate with
others, as well as a safe space for them to share their growing ideas and
insights (JONY, 2019, p. 94).
Students frequently learn about things that are previously known in the
classroom. They only have to figure out how to make these discoveries when they
have a project of their own (Davis et. al., 2006). In order to promote teaching and
learning, educational institutions are continually integrating cutting-edge
technologies into their instructional strategies. SNS (Social Network Services)
have evolved into online learning environments for collaborative learning and
knowledge exchange in the current era (Qureshi et al. 2021).
As students today are much more advanced with the use of technology and social
media tertiary institutions such as mine have officially adopted Google Meet and
Facebook as one of the most potential communicative tools to ease learning
processes by posting dates of tests and quizzes, important events and other class-
related notices, lecture notes and materials etc. In this line, Collaborative learning
or CL is one of the methods that is now used to get students to work in
togetherness so as to create such an environment in class as to help them not learn
what is ready for them but to stimulate their thoughts towards learning through
inquisitiveness and discoveries. However, collaborative learning in Bangladeshi
tertiary education system has not seen a rampant spread and wide acceptance
among students.
Although there is limited literature on the relationship between collaborative
learning and students' cognitive development or development of critical thinking
(Loes & Pascarella, 2017), the study has revealed that there is still much to learn
from the relationship between these two entities that help students be critical and
make critical decisions during discussions with peers. The goal of this study is to
demonstrate that collaborative or group study has a positive impact on students'
speaking abilities as judged by presentation test scores.
In order for the phrase to serve a wider familiarity, the term Collaborative
Learning (CL) has been substituted with group work, as group work is the term
researchers have frequently chosen (Nguyen, 2013; Jabbarova, 2020; Herrmann,
2013 in Novitasari, 2019). It is now widely used as a legitimate and catalytic
technique for engaging kids in speaking, writing, and, in some cases, reading
assignments (Remedios et al., 2008; Wiener, 1986; Sembiring, 2018; Aloisi et. al.
2016; Pattanpichet, 2011; Ibrahim et. al. 2015). Collaborative learning activities
vary, but the majority focus on students' inquiry or application of course
348
information rather than the teacher's presentation or explanation (Smith et al.,

1992).
Collaborative learning makes students more active, and it helps students share
and communicate their ideas and they can solve the difficulty of the given
materials easily (Tampubolon, 2018; Kelson & Distlehorst, 2000; Sembiring, 2018;
Murda et. al. 2015; Almajed et. al., 2016; Laal & Ghodsi, 2012; Awerbuch &
Kleinberg, 2008). The subjects of the current study are taking Intensive English-1
where the course instructors emphasize speaking over the other skills since the
majority demonstrates weakness and difficulty in speaking rather than in other
skills. The status of English in Bangladesh is far more than a foreign language but
less than a second language. Though much emphasis is put on English from the
elementary level, many students fail to achieve the target level of proficiency
(Shuchi & Islam, 2016). Implementing the strategy's main purpose is to track
learners' cognitive growth as they learn by sharing knowledge, comparing and
contrasting old and new knowledge, and correcting misconceptions (Marsico et.
al., 2013).
Collaborative learning thus has a strong bearing on and indeed capitalizes on
constructivism and Vygotsky's Zone of Proximal Development (i.e. quoted in
literature) (Harland, 2003) with the goal of serving the same purpose of comparing
and contrasting prior learning with newly acquired knowledge to test and
measure degrees of improvement in any given area of learning.
3. Suggested Collaborative Learning Model

The collaborative learning model presupposes students’ active engagement in an
interactive way of learning where they get the opportunity to share ideas and
engage in critical thinking. It refers to the formation of such a learning
environment where they happen to capitalize on shared ideas and skills invested
by the participants in the learning group. Learners in collaborative learning
understand the connectedness and presence of their peers, which stimulates
inspiration in them to get involved in group tasks. The collaborative model is a
way of anticipating active discussion and shared instructions from all the
participants that facilitate active processing of course material and activate fruitful
learning.
349
Teacher’s
Instruction
on Pre-task
Discussion and
with peers
Feedback
Shared Performance
Collaborative Students’ of Learning
Instruction
Learning Active Task and Outcome
Use of
technology
Chart 1. Theoretical Model of the Study
Teacher’s instruction during the process of collaborative learning ensures

students’ active and organized engagement in learning, which ultimately ensures
better performance and satisfactory outcome towards the end of the activity.
4. Literature Review on Collaborative Learning

Collaborative learning is a substantial departure from the traditional lecture- or
teacher-centered environment found in college classes. The
lecture/listening/note-taking process may not fully vanish in collaborative
classrooms, but it coexists with other processes that are built on student
conversation and active engagement with the course material. Teachers who
employ collaborative learning strategies often consider themselves to be
professional intellectual experience designers for their students rather than expert
knowledge transmitters—coaches or midwives of a more emergent learning
process. The benefits of using collaborative learning method is evident, and often
reflected through its popularity among researchers.
“Johnson, Roseth, & Shin, 2014; La Rocca, Margottii, & Capobianco, 2014 suggest,
when university students work in collaborative groups, compared to students
who study individually, motivation and achievement increase, as evidenced in
university achievement tests” (Weinberger & Shonfeld, 2020, p. 128).
Method of Collaborative Learning CLM is the most effective method for involving
students in group work and conversation, as well as motivating them to engage
in more discussion-based cooperative learning (Murda & Flora, 2015; De Hei,
Strijbos, Sjoer & Admiraal, 2015). The idea of the Zone of Proximal Development
(ZPD) first came into being in the 1930s by Soviet psychologist Lev Vygotsky, is
frequently used as a synonym for and related to the theme of Collaborative
Learning (Beheshti et. al., 2000 in Rezaee & Azizi, 2012; Harland, 2003; De Marsico
et. al., 2013).
The ZPD is the difference between a learner's current degree of progress and what
he has already mastered. The ZPD is thought to highlight the difference between
350
a child's ability to solve problems on his own and his ability to solve difficulties
with help. (Schutz, 2004 in Rezaee, et al., 2012). Furthermore, “Vygotsky’s
educational model includes a strong bent towards social and collaborative
learning” (De Marsico et. al., 2013, p.12).
Likewise, the collaborative learning has been used here to determine the
difference between the learning and production of knowledge by students’
individual level of regular development reflected in the pre-test result and the
level reflected in the post test results derived from the collaborative studies driven
by the support from the peers. Vygotsky’s ZPD can be used to pinpoint a pivotal
development in between two stages.
Vygotsky's views on the nature of human development and the interrelationship
between learning and development are clearly reflected in the ZPD concept.
Learning, which is not the same as development, can lead to development, and
ZPD is the term for the mechanism and possible effect of learning on growth
(Rezaee & Azizi, 2012).
Students are put in a collaborative atmosphere where they are encouraged in

finding ways to express themselves and share ideas, thereby refining their own
ideas, according to the constructivist theory, which is widely acknowledged as
one of the most successful techniques of learning (De Marsico et. al., 2013). This is
expected to increase motivation and active collaboration during learning
activities, as well as improve learning efficiency and overall student satisfaction
(De Marsico et. al., 2013). “Bangladeshi students become quite demotivated in
speaking or practicing English for which they are gradually becoming hesitant
and afraid to speak before a large audience” (Suchona & Shorna, 2019, p.36). To
this end, there remains a great deal of necessity to rethink a more diplomatic and
student-friendly teaching method. In this respect, collaborative learning appears
to be the robust host. “Peer collaboration has also been largely studied in the L2
literature” (Nguyen, 2013, p.64).
The ambiguous status of English, as well as the vagueness of language policy,

hindered English teaching and learning. The most major hindrance to improving
the English teaching situation has been a serious shortage of English instructors,
which has resulted from the neglect of English instruction during the post-
liberation decades. Students who progress to higher education may have
improved their math or science skills, but they continue to struggle with English
and, in some ways, join higher education programs unprepared. “As a result, they
are unable to cope with the English necessary for Madrasa level” (Golam, 2018, p.
108).
“To support EFL teachers and learners in taking advantage of peer support in
developing learners’ spoken English ability, there needs to be more research data
in this area to inform practice” (Nguyen, 2013, p.64). It is understood that there is
a need for further research on collaborative learning to see its scope and strength
in connection with the development of students’ interaction and discussion and
proficiency in speaking. From a developmental point of view, it is social action in
groups that is primary, with working alone being a particular and derivative case.
351
Going further in this direction, Crook (2013) has argued that “the desire to ‘share’,
to achieve ‘mutuality’ is a basic and defining drive of human beings; being
motivated to ‘share’ our experiences with others, and sometimes taking pleasure
in this for its own sake, is part of what it means to be human, and not only a more
or less efficient strategy for achieving a task” (in Baker, 2015, p.3).
Johnson & Johnson (2009) and Stahl (2006) cite “Despite the widespread influence
of social constructivism that supports CL methods, in education teachers’
perceptions of their students' learning processes are not always indicative of a
concomitant internalization of these ideas” (in Weinberger& Shonfeld, 2020, p.
128). However, Weinberger & Shonfeld (2020) find that the students' grasp of
collaborative learning is shown in their active participation in conversations and
conversational processes, which allows them to construct new knowledge and
aids in the generation of meaning through speech. Thus, the recognition of the
signs of reliability of collaborative learning methods has been fully manifested.
5. Method
The current study applied quantitative data analysis method as the study used
questionnaire with multiple choice questions and numerical numbers.
Current study considers speaking skill as a befitting term to replace the term
presentation in the title; however, the term presentation will be
interchangeably used with speaking from time to time. The study bases its
findings on two popular theoretical frameworks, Constructivist Approach
and Vygotsky’s Zone of Proximal Development in order to demonstrate
collaborative learning in the line of the central hypothesis of this study
discussed in the Significance of the Study section. Literature highlighted the
positive impact of collaborative learning as opposed to self-study or
individual learning with specific focus on speaking.
The current study has, thus, put students in contextual and collaborative learning
process to contrast their individual prior knowledge with their recently acquired
knowledge under the rubric of collaborative learning and thereby their own
corrective measures.
6. Bangladeshi Context
As Bangladesh is a Muslim country and a developing country, people from
middle and lower middle-class families prefer sending their children to Madrasas
as a means to a spiritual sacrifice to their God. A significant body of students earns
education in a number of Madrasas around the country, and the number increases
every year. A total of 1.4 million pupils have been enrolled in the country's 13,902
Qawmi madrasas.
According to a study, Dhaka division has the most madrasas with 4,599, while
Barisal division has the fewest with 1,040. The research, published by the
Bangladesh Bureau of Educational Information and Statistics (BANBEIS) in response
to Prime Minister's Office orders, is the first such study of Qawmi madrasas in
352
Bangladesh (Correspondent, 2021). In addition, a lot of students are studying at

Public Universities where English is rarely practiced.
“In Bangladesh, the education system is divided into three sections -Bangla
Medium, English Medium, and Madrassa System” (Suchona & Shorna, 2019,
p.34). Chowdhury found,
English is a compulsory course in the syllabi designed for school, college,
and university level. In the public universities the non-English major
students are offered one or two compulsory English courses, whereas, in
the private universities the students need to do two to four English
courses (2012, p.43).
Yet, Madrasa or Arabic Institutions and Bengali Medium Institutions like public
colleges and universities do not have trained ELT teachers, and these teachers are
not familiar with collaborative. Most teachers follow grammar translation method
in which students are exposed to formal structures rather than a mixed
communication environment where students can share their thoughts and ideas
and can generate context-based language output.
In Public colleges English is taught as a mandatory subject, while Bengali is used
as the primary language of instruction in class across all disciplines. English
medium (A-Levels) schools and colleges deliver mixed modes of lectures both in
Bengali and in English, while English is highly encouraged in and outside the
classroom. “As the use of English is increasing day by day in different forms, there
is significant evidence of use of English along with Bangla as code-mixing and
code-switching” (Banu & Sussex, 2001 in Rahman, 2005, p. 29-55).
According to Hamid (2009), the government funds and oversees this instruction
through a national board of religious education. In this stream, Bangla is the
medium of teaching, with Arabic and English as required courses. Madrasa
education students are mostly drawn from lower and lower-middle class
households, and it serves around 16% of the school-age population (in Hamid, et.
al. 2016, n.p.).
Finally, there is the Madrasa, or Arabic Medium, where English is practically
never taught as a primary language. As a result, university students come from a
diverse range of backgrounds, and the majority of them are unwilling to talk in
English because they are shy, scared, or intimidated. Instructors are supposed to
employ the communicative approach as an antidote to such difficulties as worry,
fear, and psychological stress in order to minimize these problems with pupils.
(Suchona & Shorna, 2019).
The Grammar Translation Method is still used by the majority of academics at
Bangladeshi universities, although others aim to combine more student-centered
instruction CLT Communicative Language Teaching. “One of the strongest
criticisms leveled at the higher education system, and particularly at teacher
education, is that it does not provide students with the relevant professional
expertise demanded of their profession” (Libman, 2014 in Weinberger& Shonfeld,
2018, p.127).
353
The current study has capitalized on students’ collaborative study as a potential

candidate in order to facilitate such a student-friendly ambience that can
minimize fear and anxiety. “There are a number of benefits that are associated
with the concept of collaborative learning (CL)” (Laal & Ghodsi, 2012, p. 486).
Collaborative learning not only allows students to express themselves, but it also
allows them to better understand their own strengths while also valuing their
peers' contributions and abilities. However, some studies have been done on
collaborative learning, and the majority of the research on the outcomes related to
collaborative learning has been done at the primary and secondary levels of school
(Johnson, & Smith, 1991 in Loes & Pascarella, 2017).
Aside from the literature demonstrating collaborative learning's overall positive
impact on a variety of student outcomes, there is a smaller body of evidence
suggesting collaborative learning may influence college students' cognitive
development in general and critical-thinking skills in particular (Loes &
Pascarella, 2017).
Students from both Public Universities and Madras System participated in a two-
week long workshop on collaborative speaking practice following by formal
presentation. Students were led through a pre-lesson activity before their active
engagement in Active Task with their peers. Students were introduced to
individual work and group work and peer discussions and to using their previous
knowledge to collect ideas on their assigned topics. Students were instructed by
the instructor through email correspondence and Google meet video conference
sessions.
The samples used as subjects in the current study are first year undergraduate
students from 3 Arabic Medium or Madrasas and from 2 Public Universities.
Students from Madrasas are studying towards earning Fazil Degree, equivalent
to Bachelor’s Degrees at universities, in which English is a compulsory subject.
Students from public universities participated in the survey include the
Department of Business Administration (BBA), the Department of Economics
(BSECO) and the Department of Engineering (BSEEE) who are required to take at
least two English courses as foundation courses.
A total of 310 students participated voluntarily in the survey in a given period of
20 days. Survey has been conducted with Survey Questionnaire provided through
Google Drive link to the subjects. Among 310 students, 210 students from the
public universities filled out questionnaire; whereas, 100 students from madrasas
filled out the questionnaire.
Data Collection and Instruments

“In order to develop a survey/questionnaire first the researcher should decide
how to collect the required data” (Taherdoost, 2018 in Taherdoost, 2019, p.3). Data
was collected with 310 questionnaires through Google-Class as “questionnaire is
the most considered the most effective tool by researchers to collect survey data”
(Taherdoost, 2019, p. 2). A fixed time frame was given for the questionnaire to be
filled out with attention and carefulness and returned to the researcher. The main
purpose of the survey questionnaire was to collect date on open ended and close
354
ended questions. Students filled out the questionnaire providing responses in

accordance their personal perspectives. As this research conducted a quantitative
method, questionnaire was a useful method of data collection.
Teachers provided feedback on their individual work on the collection of ideas on
the given topic in the first three days of seven-day session followed by individual
presentation that was assessed on 10 points each. Then, a similar session was
executed guiding them through first 3 days followed by their group activities of
collecting information and the final group presentation assessed on 10 points
each. After the two sessions were successfully conducted, students were given
feedbacks on their sessions. Then, the students were given their respective
individual and group presentation test scores after three working days. In both
cases, in group and individual evaluations students were evaluated separately to
show their individual performance.
The Likert Scale has been used as an efficient measurement tool to assess the
numerical values of the responses by the study participants during the initial
phase of data collection. The Likert Scale is considered one of the most
fundamental and common scaling methods in social science research.
(Taherdoost, 2019). “It is also the means to collecting data with NON-FORCED
choice” (Polland, 1998, p. 11) using measurement scale.
Likert scale has been used as the measuring tool to scale the weight of responses
from Strongly Agree (SA)-Strongly Disagree (SD) using numeric values on 1 to 5
scale where 5=Strongly Agree, 4=Agree, 2=Disagree & 1= Strongly Disagree with
3=Neutral in the middle as per the degree of intensity in opinion and feelings of
the subjects on a given question. As there is no substantial standard on how many
points to be put on the Likert scale code [1-5] has been used to denote the degree
of intensity under the measuring.
7. Findings
Data have been presented in numbers and percentages using graphs appropriate
for the representation and analysis. The following table demonstrates that among
the students taking part, 67.7% are from public universities while 32.3% are from
madrasa system.
Table 1. Total number of participants and the ratio of Public and Madrasa participant
Institutions Number of Participants Ratio

Public University 210 67.70%
Madrasa System 100 32.30%
Total 310 100.00%
355
Chart-2 demonstrates the graphic representation of the above table. The subsequent
table-2 has exhibited all the responses from the total participants, which have been
represented on Likert Scale, Rensis Likert (1931), has been used to measure the
numerical values worth specific ranges from SA(5)=Strongly Agree to
SD(1)=Strongly Disagree and N(3)= Neutral has been used to determine the midpoint
to represent Neither Agree nor Disagree.
A Likert-type scale “requires an individual to respond to a series of statements by

indicating whether he or she strongly agrees (SA), agrees (A), is undecided (U),
disagrees (D), or strongly disagrees (SD)” (Jamieson, 2004; Croasmuns & Ostrom,
2011, p.19). “A Likert rating scale measurement can be a useful and a reliable
instrument for measuring self-efficacy” (Maurer, 1998 in Croasmun, & Ostrom, 2011,
p.19).
Chart 2. Representation of Public and Madrasa participants in the survey
Chart-2 represents the total number of participants [n=310] in the current survey
where [Public=210] & [Madrasa=100]. 310 questionnaires were dispatched to the
subjects of the current study and the study has received full attendance, 310 data on
the questionnaire.
Table 2. Representation of the variables and percentage of dependent variables
Variables Q.1 Q.2 Q.3 Q.4 Q.5 Q.6 Q.7 Q.8 Q.9 Q.10 Q.11 Q.12 Q.13 Q.14 Q.15 Q.16
SA 44% 45% 33% 34% 60% 48% 42% 29% 52% 59% 48% 61% 46% 37% 44% 35%
A 56% 55% 67% 62% 40% 48% 32% 63% 45% 41% 52% 39% 54% 56% 52% 58%
DA 0% 0% 0% 3% 0% 3% 21% 8% 3% 0% 0% 0% 0% 4% 4% 8%
SD 0% 0% 0% 0% 0% 0% 5% 0% 0% 0% 0% 0% 0% 4% 0% 0%
356
Chart 3. Representation of overall responses on the questionnaire
Table-2 consists of variables on the extreme left vertical and the questions on the top
row labeled as Q.1 to Q.16. The subjects of the current research were delivered a set
of 16 questions covering different aspects of collaborative learning related
predominantly to its predicted positive and negative impact on students’
performance on speaking / presentation tests. Students have been divided into six
different groups with five general members working under a selected team leader
towards accomplishing the assigned presentation project followed by a presentation
test. Table-2 represents the response ratio in percentile on the scale of [1-5] on each
and every 16 questions. The first two questions have been omitted from the above
table which has been displayed at a later stage.
According to the responses on Table-2, it is clearly evident that subjects of the study
have responded enthusiastically enough to get their voice heard. Some of the major
research questions from among the 16 have been discussed here. The first question is
about whether a collaborative learning environment creates a better opportunity for
learning. Among the total participants 44% responded (SA) and 56% responded (A),
while 0% students responded SD & DA.
For the second and third question-Collaborative learning helps you better
understand, and Collaborative learning improves students/ increases a student's
desire to excel-most students acknowledged the positive impact of collaborative
learning on their cognitive development (i.e. Zone of Proximal Development) which
helps them understand their peers who come from culturally different backgrounds
and they believe that collaborative learning improves or increases their desire to excel
in their work. For, Q.2& Q.3 45% & 33% said (SA) and 55% &67% responded (A),
while 0% students responded (D) & (SD) for both the questions respectively.
Survey questions [Q.6, Q9, and Q.10. & Q.11] have been intended to determine
whether collaborative learning helps students develop better communication skills,
speaking skills, think of new ideas, and create a better environment for speaking
practice. 48%, 52%, 59%, &48% students responded positively as they have strongly
357
agreed with the statements, while only 3% students think that collaborative learning
does not help them develop better communication skills and speaking skills, while
none disagreed with the statements.
According to Table-2, we can make a strong case that most students are in favor of
collaborative learning and they understand that collaborative learning helps perform
better in their speaking and presentation. At this stage, the research has undergone a
number of tests to determine the data consistency and reliability and to extract more
elaborate analysis of different variables in the data.
In addition to the fundamental questions to Collaborative Learning and its impact,
two introductory questions have been asked to measure the level of familiarity and
students’ overall preference to collaborative learning. The results on the two
questions have been generated below.
Table 3. Students’ familiarity and preference to Collaborative Learning
Responses Institutions Familiarity Presences
Yes Public University 74.4% 93.5%
No Madrasa System 22.6% 6.5%
Chart 4. Public and Madras students’ familiarity and preference to collaborative learning
Table 3 corresponds with Chart 4 as both are representing students’ reflection on the
questions, Are you familiar with collaborative learning or group learning. And do
you prefer studying in groups? In response to these questions, 77.4% and bulk 93.5%
Public University students have confirmed that they are familiar with collaborative
learning, and they prefer collaborative learning, as they love studying in groups
rather than individually.
358
Table 4. Comparative Test Scores and their sum total, mean value and standard dev.
Test Scores** Sum Mean* Standard Deviation*
Individual Test Scores 1694 5.464516129 1.328198632
Collaborative Test Scores 2281 7.358064516 0.928442791
Chart 5. Graphic representations of the comparative test score
Chart 5 displays graphical representation of table 4 both of which demonstrates the

total score earned in the individual test and in the collaborative test by Madras
students as well as Public University students. It also shows the mean difference and
standard deviation between the two test scores. It is evident that scores earned in
collaborative test outweighs the scores earned in the individual test. The mean
difference between the two tests is also remarkable together with the standard
deviation 0.92844 between the test scores of all students in their collaborative test,
which means that all students in the collaborative learning and collaborative test
scored equally consistently compared to 1.328 (standard deviation) between all the
scores earned in individual test scores both by Public university students and
Madrasa Students.
It is evident that students from both Arabic and Bengali medium institutions show
similar improvement upon collaborative learning. The study also suggests positive
improvement in students’ English language learning through collaborative learning.
8. Reliability Test
Numbers instead of texts have been used on Excel and SPSS Spreadsheet for the
analysis of data at different phases of the data incorporation and analysis at different
sections. A reliable validity testing tool has been adapted to test internal consistency
of the data. First, a validity test has been conducted following Cronbach’s alpha to
measure “internal consistency” reliability as “Cronbach’s alpha is the most widely
used tool to measure consistency in scientific research” (Bonett & Wright, 2014, p.3).
Lee Cronbach in (1951) provides a measure of the internal consistency and reliability
of a test or scale; it is expressed as a number between 0 and 1. Internal consistency
describes the extent to which all the items in a test measure the same concept or
construct and hence it is connected to the inter-relatedness of the items within the
test.
359
A two-sample F-test, Correlation, and Regression analysis also have been run to see
the internal correlation and variance between two sets of data variables (i.e. in
between dependent variables as well as between test scores of individual
performance & test scores of collaborative performance) gathered from students’
responses and test scores generated by individual effort and the post-test scores.
A two sample T-test has been conducted to test [t-stat] and the distinction between
M=mean values between two different variables. The T-test is followed by the
significant correlation and regression testing on ANOVA hypothesis tool to
determine r values (reliability coefficient) and p= (probability) of the positive or
negative relationship between two variables.
9. Pilot Study
The primary collection of data samples has been tested to see internal consistency of
the data against Cronbach Alpha (α ≥.75) and KR-21 formula see Reliability
Coefficients. It is common to see the reliability of instruments used in published
science education studies framed in terms of a statistic known as Cronbach’s alpha
(Taber, 2018) Cronbach’s alpha has been described as “one of the most important and
pervasive statistics in research involving test construction and use” (Cortina, 1993, p.
98 in p.1275).
Table 5. Summary of participants
Case Processing Summary
N %
Cases Valid 310 100.0
Excluded 0 .0
Total 310 100.0
Above table-5 displays a case processing summary of the number of participants with
0 excluded. At first, the collected data has been set in excel spreadsheet and then
transferred to SPSS for final testing. The raw data has been renamed with specific
numerical values in order for them to fit SPSS criteria. Data labels have been put
properly. For the first two questions on students’ familiarity and preference
numerical values Yes=1 and No=0 have been used on the SPSS spreadsheet. Likewise,
Public University=1 and Madrasa=0 have been put along with 5=Strongly Agree,
4=Agree, 3=neither Agree nor Disagree, 2=Disagree and 1=Strongly Disagree have
been set as data to run a reliability test on all the variables.
Following table-6 shows the Cronbach’s Alpha α=*.897 on N=19 items put on SPSS.
It is clear that the data set has successfully passed the important critical point of
Cronbach Alpha in order for the data to be recognized as statistically reliable. Alpha
value is considered Excellent at (0.93–0.94), strong (0.91–0.93), dependable (0.84–
0.90), robust (0.81), quite high (0.76–0.95), high (0.73–0.95), good (0.71–0.91),
somewhat high (0.70–0.77), moderate (0.61–0.65), satisfactory (0.58–0.97), acceptable
(0.45–0.98), sufficient (0.45–0.96), not satisfactory (0.45–0.96) (0.11) (Taber, 2018).
360
As α=*.897 is higher than 0.81 set by (Taber, 2018) and fits within [0.84-0.90] we can
consider it reliable. Hence, the internal consistency of the collected data is proven at
this stage. Table-6 measures the item-means and inter-item correlation summary
which shows that the average correlation between items is at significant level = *.377
which suggests that all the respondents responded positively towards collaborative
learning.
Table 6. Reliability Statistics
Reliability Statistics
Cronbach's Alpha Based on

Cronbach's Alpha Standardized Items N of Items
α = *.897 .920 19
Table 7. Demonstration of internal consistency and variance
Reliability test results showing Cronbach’s Alpha

SUMMARY
Groups Count Sum Average Variance
FAM/CL 310 24 0.774194 0.180645
PREF/CL 310 29 0.935484 0.062366
CL/BETTERLERENVIRON 310 132 4.258065 0.464516
CLLERCULTURE 310 135 4.354839 0.369892
CLLERDESIREEXCEL 310 129 4.16129 0.406452
CLRESPONSIBLE 310 130 4.193548 0.494624
CLIMPORTANCETEAMWORK 310 141 4.548387 0.322581
CLBETTERCOMSKILLS 310 134 4.322581 0.55914
CLCHAOS 310 109 3.516129 1.258065
CLHELPSCRITICAL 310 120 3.870968 0.716129
CLIMPSPEAK 310 135 4.354839 0.569892
CLNEWIDEAS 310 139 4.483871 0.391398
PRACTSPEAKCL 310 136 4.387097 0.378495
CLMOTIVATIONMORE 310 138 4.451613 0.455914
361
CLACTIVE 310 134 4.322581 0.425806
CLINTERESTNEWIDEAS 310 125 4.032258 0.898925
CLENERGY 310 127 4.096774 0.690323
CLMOREENGAGED 310 124 4 0.733333
CLPRESTEST 310 208.5 6.725806 1.813978
Table 7 demonstrates inter-item consistency along with mean variance in between

variables. Column on the very left (Groups) represents the shortened forms for the
each and every survey question with CLPRESTEST=Presentation Test after
Collaborative Learning at the very bottom. On the extreme right column variances
are presented. There is no significant variance among the variables except the first
two variables which are outstanding items with close-ended questions (Yes/No). The
reason perhaps is because for the first two items only two options / two numerical
values 1 & 0 were put in the processing sheet.
Study shows that ANOVA single factor P value is (p<0.05) we can say that there is no
significant difference between the variables, which means that there is internal
consistency in the responses from all the subjects of the research. Smaller (p=8.3E-
148) suggests that the data is statistically significant and it allows us to reject the null
hypothesis to accept the alternative hypothesis that there is a significant relationship
between the given variables. Hence, the data has been proven reliable according to
Bowling (2009) that “the reliability in quantitative research as synonymous to
dependability, consistency, reproducibility over time, over instruments and over
groups of respondents” (in Oluwatayo, 2012, p. 391).
10. Discussion
At first, the two sets of data collected on students’ individual performance in the test
scores [INDVPRESTEST] and the presentation test scores generated by collaborative
preparation and performance [CLPRESTEST). These two sets of data have undergone
a reliability test on SPSS to measure internal consistency and reliability on the basis
of Cronbach’s Alpha.
Table: 8 Reliability and Consistency testing of Individual Test and Collaborative Test
Reliability Statistics
Cronbach's Alpha Cronbach's Alpha Based on Standardized Items N of Items
*.991 .992 2
Above table-8 demonstrates reliability and internal consistency between two test
results by the students. The testing results of Cronbach Alpha stands (α= *.991) and
the number of items (N=2). According to statistics the Alpha complies with and
stands higher than Excellent = (0.93-0.94) set by Taber (2009). Therefore, both the data
362
sets are statistically reliable to conduct correlation and regression with other
variables. It also suggests that the students’ performance in both individual and
collaborative tests have consistency. In order to determine the points of fluctuation,
an F-test was conducted. In addition, a Regression analysis was done on the two sets
of results to pinpoint variance in data.
Regression analysis shows that the correlation between the two variables is
statistically significant as the P= possibility of null hypothesis that two variables are
different proves wrong and (p=0.0000) is obviously very small and p<0.05 which
suggests that we can easily reject the null hypothesis that these two variables are
different. Hence, there is no significant difference observed at this level.
It can be said that students in both cases performed quite well. However, the current
findings require a different test to measure the degree of improvements in the
collaborative test score in order for the responses to match the test results. The table
also demonstrates important clues for us to reject the null hypothesis and form an
alternative hypothesis at this stage. Table-10 which is part of table-11 also shows the
similar signs of correlation between these two items, where R-square =*0.968464389
which is statistically significant, which means that there is significant correlation
between the two items.
To determine any significant variance between the test after individual performance
and test after collaborative performance a T-test and an F-test have been conducted
to highlight the mean difference, which can provide an important sign of difference
between the two tests.
T-test exhibits the statistics found in t-test, which shows that (p=0.38252) which is
(*p>0.05) which is larger, and it suggests that we cannot reject the Null Hypothesis.
Also, t-stat is smaller than t-critical value which suggests that there is no significant
difference; however, there is a difference in mean in the t-test as well as in F-test which
suggests otherwise. Though the difference is small, students have performed better
and made better improvement in collaborative learning by contrast.
F test shows the same test results where there is a marked difference between the two
sets of tests-data, which suggests that though the p-value suggests otherwise there is
still a sign of better performance in the Collaborative performance results made
possible by students’ participation in the collaborative learning process. Hence, the
findings of table13 match with students’ responses on the Likert scale. The tests also
demonstrate the relationship between dependent variable CLPRESTEST and other
independent predictors that predict the improvement in performance and test-scores
after collaborative learning. Inter-item correlation has been displayed as a valid
manifestation of the positive impact of collaborative learning according to the
findings.
At this stage, the research has succeeded in replicating the kind of context the
Constructivist Approach of learning suggests through learners’ immersion in
collaborative engagements and sharing and refining their ideas reflected in chart-8.
363
In other words, “a research can be deemed valid if the extent to which a research
instrument consistently has the same results, if it is used in the same situation on
repeated occasions” (Heale, R., & Twycross, A., 2015, p.66).
In order to prove further consistency in the data the current study has represented
several other tests, and the following correlation test on SPSS has been conducted to
measure +/- relationship between Collaborative Test-Score and other indicators. For
Salvucci, Walter, Conley, Fink, and Saba, in terms of the range of reliability measure,
when the r value is less than 0.50, the reliability is considered low; if the r value is
between 0.50 and 0.80, the reliability is regarded as moderate whereas the r value is
greater than 0.80, the reliability is treated as high (1997, p. 121). The data have also
undergone a regression test to determine the p value to see whether p</>.05 to reject
the null hypothesis and reach an alternative hypothesis.
The correlation between the significant indicators and their consistent relationship
with CLPRESTEST=Collaboration Presentation Test. Based on the correlation
statistics, we see there are negative and positive correlations between the variables,
which will be further interpreted below; however, the correlations between inter-
items are statistically significant at (P<0.01) level and (P<0.05) level.
To further clarify the correlations one by one, the correlation between Collaborative
Presentation Test (Individual performer) and CLMOREENGAGED= Collaborative
Learning and More Engagement in lesson, CLINTERESTNEWIDEAS= Collaborative
Learning and More Interest in New Ideas, PRACTSPEAKCL= Collaborative Learning
and Practice Speaking, CLIMPSPEAK= Collaborative Learning and Importance of
Speaking, CLBETTERCOMSKILLS= Collaborative Learning and Communication
Skills, CLHELPSCRITICAL Collaborative Learning and Critical Thinking show that
the correlations are not statistically significant as the P value is larger than (P≥0.05)
at (P=0.312, 0.603, 0.423, 0.933, 0.398, and 0.126) respectively.
Though the test displays negative Pearson (r= *-0.188, *-0.097, * -0.149, -0.016, -0.157,
-0.281) respectively it suggests that the correlation is still statistically significant and
has bearing on the test results. According to Taylor (1990), the correlation's strength
is independent of its direction or sign. A negative correlation denotes an inverse
relationship, in which one variable rises while the other falls. One of the reasons for
the negative association is that the numerical values assigned to test scores are higher
than those assigned to other indicators, such as the Likert Scale options. (5-1).
Regression analysis demonstrates the findings on the regression analysis between
INDIVPRESTEST and other predictors, where INDIVPRESTEST is considered
Dependent Variable. Coefficient R2=*0.1 and R=*316 prove that the correlation
between the items is statistically at close variance and there is a relationship among
the items. Whereas, (P>.842) is not statistically significant. We can make a case that
the earlier between the two tests shows better statistically significant P value than the
latter, which suggest that students’ performance on CLPRESTEST is better than on
INDIVPRESTEST. That means students in collaborative learning perform better than
364
in individual learning. The study hence is ready to suggest collaborative would very
likely ensure better results.
11. Implications of the Study

The current study provides strong, positive implications for both Public University
students and Madrasa students that they are more active and responsible in
collaborative learning than in individual learning. “While the effectiveness of
collaborative learning cannot be negated or questioned, limitations to implementing
its full force is often ascribed to students’ inability to communicate effectively (i.e.
language) and unequal individual participation in group tasks not to Collaborative
Learning” (e.g. Freeman & Greenacre, 2010; Janssen et. al., 2007 in Le, Janssen, &
Wubbels, 2018, p. 104).
Thus, the current study provides successful data on ZPD together with a reflection
on students’ engagement in dialogues through contextual learning process as
suggested by Constructivist Approach. This study strongly suggests that the
collaborative learning method can facilitate more productive and ensure much better
learning compared with other methods and provide learners student-centered
learning environment in both Public Universities and Madrasas.
12. Conclusion
Collaborative learning can be used as a universal communicative teaching method
where both adults and children can get a proper ambience to learn through their
shared interests and previous knowledge. The study has shown that collaborative
learning is more productive if properly enacted irrespective of Bengali medium
universities and Arabic education systems. It not only helps school children do better
on exams but it also helps tertiary level students and help them develop their critical
thinking skills. “Students are not only expected to speak well but also expected to
address complex scientific and social issues, greater involvement of students in
dialogue, and an increased emphasis on collaborative discourse and argumentation,
have become essential modes of engagement and learning” (Harney, Hogan, &
Quinn, 2017 in Weinberger et. al., 2020, p.127).
Collaborative learning supports ZPD by allowing students to learn in context.
Collaboration is also mentioned, if indirectly, in UNESCO's document from the
summit on rethinking learning in the digital era (Shonfeld et al., 2017 in Weinberger,
2020). Yet, “despite the widespread influence of social constructivism that supports
CL methods in education” (Johnson & Johnson, 2009; Stahl, 2006), “teachers’
perceptions of their students’ learning processes are not always indicative of a
concomitant internalization” (Weinberger et al., 2020, p. 28).
13. Limitation of the study

However, the data from the present study has several limitations. 310 students
from two distinct universities, each using a different teaching method and
academic structure, participated in this study. This study demonstrates not just
the advantages of collaborative learning but also highlights parallels between
365
madrasa and university students' perceptions. To that aim, this study suggests
further research among college students at various levels from other schools in
order to acquire more information and develop a more comprehensive grasp of
the subject.
14. References
Ahmed, M. K. (2016). Communicative Language Teaching: A Practical Scenario in the Context
of Bangladesh. Advances in Language and Literary Studies, 7(5), 97–104.
Akbas, I. (2016). Difficulties of English Language Teaching in Bangladesh. International Journal
of English Language Education, 4(1), 17-36. https://doi.org/10.5296/ijele.v4i1.8575
Almajed, A., Skinner, V., Peterson, R., & Winning, T. (2016). Collaborative learning: Students'
perspectives on how learning happens. Interdisciplinary Journal of Problem-Based
Learning, 10(2), 9. https://doi.org/10.7771/1541-5015.1601
Aloisi, G., & Scana, M. (2016). Social networks and collaborative learning. European Journal of
Open Education and E-learning Studies.
Awerbuch, B., & Kleinberg, R. (2008). Competitive collaborative learning. Journal of Computer
and System Sciences, 74(8), 1271-1288. https://doi.org/10.1016/j.jcss.2007.08.004
Baker, M. J. (2015). Collaboration in collaborative learning. Interaction Studies, 16(3), 451-473.
https://doi.org/10.1075/is.16.3.05bak
Bonett, D. G., & Wright, T. A. (2015). Cronbach's alpha reliability: Interval estimation,
hypothesis testing, and sample size planning. Journal of Organizational Behavior, 36(1),
3-15. https://doi.org/10.1002/job.1960
Chowdhury, N. (2012). Classroom code switching of English language teachers at tertiary
level: A Bangladeshi perspective. Stamford Journal of English, 7, 40-61.
https://doi.org/10.3329/sje.v7i0.14462
Correspondent, S. (2021). 1.4m students study in 14,000 Qawmi madrasas.
https://en.prothomalo.com/bangladesh/14-lakh-students-study-in-14-000-Qawmi-
madrasas.
Croasmun, J. T., & Ostrom, L. (2011). Using Likert-Type Scales in the Social Sciences. Journal
of Adult Education, 40(1), 19-22.
Davis, L., McGraw, P., & Mileham, M. (2006). Collaborative Learning. Linfield Magazine, 3(1),
5.
De Hei, M. S. A., Strijbos, J. W., Sjoer, E., & Admiraal, W. (2015). Collaborative learning in
higher education: lecturers' practices and beliefs. Research Papers in Education, 30(2),
232-247. https://doi.org/10.1080/02671522.2014.908407
De Marsico, M., Sterbini, A., & Temperini, M. (2013). A strategy to join adaptive and
reputation-based social-collaborative e-learning, through the zone of proximal
development. International Journal of Distance Education Technologies, 11(3), 12-31.
https://doi.org/10.4018/jdet.2013070102
Golam, A. M., & Kusakabe, T. (2018). A qualitative study of English teaching in Bangladesh:
A case study of Madrasa education. US-China Education Review, 8(3), 106-122.
https://doi.org/10.17265/2161-623X/2018.03.003
Harland, T. (2003). Vygotsky's zone of proximal development and problem-based learning:
Linking a theoretical concept with practice through action research. Teaching in higher
education, 8(2), 263-272. https://doi.org/10.1080/1356251032000052483
366
Hamid, M. O., & Erling, E. J. (2016). English-in-education policy and planning in Bangladesh:
A critical examination. In English language education policy in Asia (pp. 25-48).
Springer, Cham. https://doi.org/10.1007/978-3-319-22464-0_2
Heale, R., & Twycross, A. (2015). Validity and reliability in quantitative studies. Evidence- based
nursing, 18(3), 66-67. https://doi.org/10.1136/eb-2015-102129
Ibrahim, N., Shak, M. S. Y., Mohd, T., Zaidi, A., & Yasin, S. M. A. (2015). The importance of
implementing collaborative learning in the English as a second language (ESL)
classroom in Malaysia. Procedia Economics and Finance, 31, 346-353.
https://doi.org/10.1016/S2212-5671(15)01208-3
Jabbarova, A. (2020). MODERN APPROACHES IN TEACHING SPEAKING.
АрхивНаучныхПубликаций JSPI, 1-5.
JONY, M. S. (2019). Exploring the effectiveness of cooperative learning at secondary level in
Bangladesh. International Journal of Educational Research Review, 5(1), 93-99.
https://doi.org/10.24331/ijere.669388
Laal, M., & Ghodsi, S. M. (2012). Benefits of collaborative learning. Procedia-social and behavioral
sciences, 31, 486-490.https://doi.org/10.1016/j.sbspro.2011.12.091
Le, H., Janssen, J., & Wubbels, T. (2018). Collaborative learning practices: teacher and student
perceived obstacles to effective student collaboration. Cambridge Journal of Education,
48(1), 103-122. https://doi.org/10.1080/0305764X.2016.1259389
Likert, R. (1931). A technique for the measurement of attitudes. Archives of Psychology. New
York: Columbia University Press.
Murda, L. A., & Flora, H. (2015). The Effectiveness of Collaborative Learning in Improving
Students Speaking Skill. Lampung University: Teacher Training and Education.
Loes, C. N., &Pascarella, E. T. (2017). Collaborative learning and critical thinking: Testing the
link. The Journal of Higher Education, 88(5), 726-753.
https://doi.org/10.1080/00221546.2017.1291257
Oluwatayo, J. A. (2012). Validity and reliability issues in educational research. Journal of
educational and social research, 2(2), 391-391.
Pattanpichet, F. (2011). The effects of using collaborative learning to enhance students English
speaking achievement. Journal of College Teaching & Learning (TLC), 8(11), 1-10.
https://doi.org/10.19030/tlc.v8i11.6502
Polland, J. Ronald. (1998). Essentials of survey research and analysis.
Qureshi, M. A., Khaskheli, A., Qureshi, J. A., Raza, S. A., & Yousufi, S. Q. (2021). Factors
affecting students' learning performance through collaborative learning and
engagement. Interactive Learning Environments, 1-21.
https://doi.org/10.1080/10494820.2021.1884886
Rahman, S. (2005). Orientations and motivation in English language learning: A study of
Bangladeshi students at undergraduate level. Asian EFL Journal, 7(1), 29-55.
Remedios, L., Clarke, D., & Hawthorne, L. (2008). Framing collaborative behaviors: Listening
and speaking in problem-based learning. Interdisciplinary Journal of Problem-Based
Learning, 2(1), 1-20. https://doi.org/10.7771/1541-5015.1050
Rezaee, A. A., & Azizi, Z. (2012). The role of zone of proximal development in the students'
learning of english adverbs. Journal of Language Teaching and Research, 3(1), 51.
https://doi.org/10.4304/jltr.3.1.51-57
Salvucci, S., Walter, E., Conley, V., Fink, S., & Saba, M. (1997). Measurement Error Studies at
the National Center for Education Statistics.
367
Sembiring, L. T. A. B. (2018). RESEARCHING STUDENTS'INTERACTION IN

COLLABORATIVE LEARNING CLASS. Journal of Applied Linguistics and Literacy,
2(2), 125-134. https://doi.org/10.25157/jall.v2i2.2197
Shuchi, I. J., & Islam, A. B. M. (2016). Teachers' and Students' Attitudes towards L1 Use in EFL
Classrooms in the Contexts of Bangladesh and Saudi Arabia. English language teaching,
9(12), 62-73. https://doi.org/10.5539/elt.v9n12p62
Smith, B. L., & Mac Gregor, J. T. (1992). What is collaborative learning?
Suchona, I. J., & Shorna, S. A. (2019). Speaking problems in English and solutions: Scrutinizing
students' perspective. International Journal of English, 8(1), 34-41.
https://doi.org/10.34293/english.v8i1.661
Taber, K. S. (2018). The use of Cronbach's alpha when developing and reporting research
instruments in science education. Research in Science Education, 48(6), 1273-1296.
https://doi.org/10.1007/s11165-016-9602-2
Taherdoost, H. (2018). A review of technology acceptance and adoption models and theories.
Procedia Manufacturing, 22, 960-967. https://doi.org/10.1016/j.promfg.2018.03.137
Tampubolon, T. C. Improving Students' Motivation in Speaking through Collaborative
Learning. International Journal of English Literature and Social Sciences, 3(2), 239289.
https://doi.org/10.22161/ijels.3.2.10
Wiener, H. S. (1986). Collaborative learning in the classroom: A guide to evaluation. College
English, 48(1), 52-61. https://doi.org/10.2307/376586
Weinberger, Y., & Shonfeld, M. (2020). Students' willingness to practice collaborative learning.
Teaching Education, 31(2), 127-143. https://doi.org/10.1080/10476210.2018.1508280
368

Vol. 21, No. 8, pp. 368-383, August 2022
Received Apr 16, 2022; Revised Jul 29, 2022; Accepted Sep 1, 2022
Navigating the New Covid-19 Normal: The

Challenges and Attitudes of Teachers and
Students during Online Learning in Qatar
Caleb Moyo
University of Central Nicaragua
Managua, Nicaragua.
Selaelo Maifala
Cape Peninsula University of Technology
Faculty of Education, Department of SP & FET, Mowbray, South Africa.
Abstract. This inquiry sought to explore the teachers’ and students’

challenges and attitudes concerning mandatory online instruction at the
beginning of the pandemic in March 2020. Using a qualitative case study,
a single international school in Doha, Qatar was sampled to participate.
The data was generated from interviews with seven teachers who
participated in one-on-one in-depth interviews and from six groups of
students who participated in focus group discussions. An inductive data
analysis approach was employed to analyse the data. The findings
revealed various technical, pedagogical, and social challenges
experienced by both teachers and students. The participants’ attitudes to
online learning were mixed as others preferred this mode of learning
while some preferred learning in physical classrooms. The implications
point toward the need for teacher professional development that targets
the use of online teaching platforms. They further point to the need for
teachers and school administrators to find strategies to make online
learning more interactive and to assign work that reduces the amount of
time spent on the screen.
Keywords: attitudes; challenges; Covid-19; online-learning; teacher

professional development
1. Introduction
The novel coronavirus of 2019 (Covid-19) first broke out in late 2019 in the city of
Wuhan in China’s Hubei province (World Health Organisation (WHO), 2020a).
By March 2020, the WHO declared the virus to be a pandemic after it had spread
to over 140 countries and territories (WHO situation report 55, 2020b). To curb the
spread of the virus, social distancing and lockdown measures were undertaken
and governments in over 110 countries resolved to temporarily close their schools.
©Authors
369
In the state of Qatar, as the number of confirmed cases rose to eighteen on the 9th
March 2020, the government announced the suspension of schooling (State of
Qatar Government Communications Office, 2020). Furthermore, because of the
turmoil caused by the virus, the government could not say when the schools
would reopen and mandated that schools go online. Other affected countries in
the region like Iran and Egypt shut down their schools with no apparent plan for
online education which can safely be attributed to a lack of financial backing and
infrastructure. Contrarily, Qatar is the richest country in the world according to
its GDP per capita (World Bank, 2022) with a population that has easy access to
technology in their homes and classrooms.
Furthermore, the government views education as a powerful tool in driving social

and economic changes in the country (Abou-El-Kheir, 2017).
In its vision for 2030, the government envisages that the education provided in
the country will be compatible with the best education systems in the world
(General Secretariat for Development Planning, 2008). Consequently, billions of
dollars have been invested in various education reform programmes such as
Education City (Khodr, 2011). Therefore, while the challenges experienced in
other countries may include a lack of infrastructure for such endeavours, we
anticipated that these would not be challenges found in Qatar.
However, while infrastructure is one thing, implementation is another. For

instance, a study by Al-Jaber and Dutta (2008) which sought to investigate the
implementation of the ICT Qatar government programme of 2004 revealed
various difficulties with implementation. Some of the difficulties included
resistance to change by parents, teachers and communities, the attitude to
technology, government censorship, and a lack of trained teachers.
In the context of the pandemic, there have not been any studies focusing on both
teacher and student challenges and attitudes to the mandatory emergency move
to online learning. Related studies include the inquiry by Chaaban, Sawalhi and
Du (2021) which sought to investigate teacher leadership during the same period.
Their findings revealed that the participants were adequately supported by their
schools and the Ministry of Education. Conversely, increased workloads were a
hindrance to professional learning in that period. Others, including the inquiries
by Newsome et al. (2022), focused on the experiences of students and lecturers in
higher education. As such, little is known about the challenges and attitudes of
learners and teachers in primary and secondary education at the beginning of
online learning. The lack of empirical evidence from Qatar could mean that Qatari
policymakers and practitioners make decisions for future planning and practices
based on the experiences and attitudes of the students and teachers in other
contexts. The objective of this study was therefore to bridge this gap in the
literature by responding to the following research questions:
• What are the challenges experienced by teachers and students during their
transition to online learning?
• What are the attitudes of teachers and students about using online
learning in the future?
370
2. Related Work
2.1 Experiences of Teachers during the Transition to Online Learning in Other
Contexts
Studies conducted in other contexts have revealed a multitude of technical and
pedagogical challenges. Notably, there are technical challenges that relate to a lack
of digital competency which was a major challenge in the literature reviewed
(Alea, Fabrea, Roldan & Farooqi, 2020; Adenoyin & Soykan, 2021). Furthermore,
the unavailability of digital devices and the lack of reliable internet connection
also posed a challenge (Ferri, Grifoni & Guzzo, 2020; Ramij & Sultana, 2020).
Another common obstacle is the heavy workload and fatigue experienced by

teachers (Ferri et al., 2020; Hermanto & Srimulyani, 2021; Adeboyin & Soykan,
2021). While this challenge was perceived negatively in many of the studies
reviewed, the inquiry by Seabra et al. (2021) showed that some teachers perceived
the heavy workload to be an opportunity for development. Moreover, it was
found in the same study that teacher ages did not influence their attitude to online
learning.
2.2 Experiences of Students during the Transition to Online Learning in Other

Contexts
Research on the experiences of students during the transition indicates that a lack
of proper digital devices and poor internet connection were one of the major
challenges (Adnan & Anwar, 2020; Famularsih, 2020). Furthermore, similarly to
the teacher challenges, a lack of digital skills when it comes to participating in
online classes was also an obstacle. For instance, the study by Ferri et al. (2020)
found that one of the reasons for low attendance in online classes could be
attributed to the students’ lack of digital skills in those between the ages of 11 and
17 years old.
In addition, students also lamented their lack of motivation to carry out the
assignments (Hermanto & Srimulyani, 2021) along with a decreased focus and
being distracted (Hussein et al., 2020) These findings are consistent with the
assertion by Zhang et al. (2020) who argued in March 2020 that working from
home could pose challenges as there would be plenty of distractions for students.
Conversely, other students blamed increased workloads rather than their
motivation to keep up with the work assigned since the transition to online
learning (Famularsih, 2020; Hussein et al., 2020). Moreover, the lack of social
interactions that come with studying online was also found to be a challenge by
students. For example, a study by Gustiani (2020) found that despite students
being motivated by wanting to pass their courses, they found online learning
uninteresting because of the reduced opportunity for interaction.
3. Methodology
3.1 Research Design
The study was underpinned by the interpretive paradigm which accepts reality
as subjective. It also relates to an understanding that what is considered true
varies from one individual to the next (Denzin & Lincoln, 2011). Thus, the
researchers had no desire to quantify the experiences of participants or generalise
371
the findings to other contexts, but to have an understanding of the phenomenon

as lived by the participants. From these assumptions, a qualitative case study was
utilised and a single international school high school in Doha, Qatar was sampled
to participate.
3.2 Research Context and Participants

Seven teachers, three male and four females, were sampled using purposive
sampling to participate in the study. The criteria for sampling the teachers was
that they had to be high school teachers working in the sampled Kindergarten to
Year 13 (K-13) school. Knowing that age can influence one's experiences with
technology (Blank & Dutton, 2012), we aimed to include teacher participants from
different age groups. Thus, the participants ranged from 26 to 55 years. Their
teaching subjects were not a criterion for selection, thus those who participated
taught across different subjects in the school.
The student participants were also selected using purposive sampling. The
criteria for selection was them being in Key Stages 3 and 4 (Year 7 to 11). We chose
not to include Key Stage 5 (Year 12 and 13) as we assumed that their challenges of
having to take their International General Certificate of Secondary Education
(IGCSE) and Advanced Level General Certificate of Education (A-Levels) in the
same academic year of this turmoil could be better understood in a different
study. Thus, a total of 94 students between the ages of 11 and 16 participated in
six focus group discussions.
3.3 Data Generation Methods and Instruments

The data was generated in the second week of April 2020, three weeks after the
start of online learning. The teacher data was generated through one-on-one,
semi-structured interviews while the student data was generated through focus
group discussions. In both instances, the data was generated virtually using the
video-conferencing application, Zoom. Focus group discussions as opposed to
one-on-one interviews for the students were selected with the knowledge of the
power of interactions facilitated by group settings when discussing a similar lived
experience. Interacting with each other allowed the participants to think, debate,
reflect, and even remember details that would otherwise be forgotten or
neglected, thus enhancing the validity of the findings (McMillian & Schumacher,
2010). Furthermore, because we sought to understand the experiences of students,
we valued hearing more student voices as opposed to sampling a smaller group
through focus group discussions, which therefore became the best data generation
method. While some of their experiences were unique to each student, we were
able to identify patterns in the responses from one focus group to the next which
led to the findings. We asked standard questions which elicited the student and
teacher experiences in terms of computer efficacy, prior preparedness, workload,
navigation issues, and whether in the future they would prefer learning online
compared to traditional classroom settings.
3.4 Data Analysis

After the completion of the documentation of the data, the inductive analysis
approach was undertaken in which all data was read, collated, coded, and sorted
372
out into clusters, patterns and categories. This sorting led to the themes that were
presented as the findings (McMillian & Schumacher, 2010).
3.5 Ethical considerations

Permission was sought from the principal to use the study in the research.
Consent was also sought from teacher participants and granted. We also
approached students in writing and verbally to explain the nature of the study
and their participation. Because the students were minors, from those who
showed an interest, we also sought permission from their parents.
4. Findings
The two modes of facilitating online instruction in the school during the initial
online learning were Zoom and Google Classroom. The teachers also revealed that
the last day of school for students was the 9th March while the first day of online
classes was the 16th of the same month. The findings in response to the first
research question revealed various challenges that the teachers and students
encountered. The similarities and differences in the challenges experienced by the
two groups are depicted in Figure 1 below.
Teachers Both
Students
Technical skills
Heavy workload
Lack of feedback
Figure 1: Teacher and student challenges at the beginning of the online

learning
4.1 Teachers’ Perspectives of their Challenges

4.1.1 Lack of Prior Training and Technical Skills
The teacher interviews indicated that apart from one participant who teaches ICT,
the teachers had either never heard of the platforms chosen by the school to
deliver the online lessons, or they had never used them before. The teachers also
indicated that between the last day of the physical classes (9th March) and online
classes (16th March), there had been no workshops to prepare them for the task.
Rather, they had been encouraged to try out the platforms to acclimatise
373
themselves with them before the first day which led to various technical
challenges in the implementation stage. One teacher who teaches Information
Communications Technology (ICT) stated that while she was one of the lucky few
who were familiar with one of the platforms, she experienced challenges still. She
stated:
I was confident in using Google classroom but not so much Zoom. It was very
disturbing and we have had to figure everything out ourselves and we’ve had some
issues with inappropriate behaviour during live lessons, like a student annotating
on the screen. I had one of my students write the word vagina on one of my slides
and I had no idea how to remove that or disable it. (Teacher 1)
Teacher 2 also concurred, indicating:

We were ambushed and there was no transition period. I had never heard of Google
Classroom. So I found that teachers like myself who had not been exposed to IT and
all the gadgets struggle more. I was not aware for example, that with Zoom the
recording is only 40 minutes long and I would be in the middle of my lesson when
I get cut off. (Teacher 2)
However, it must be stated that while it is apparent that the teachers believe that
they could have been better prepared by the school, there was an
acknowledgement of the nature of the crisis and the awareness that the mitigation
efforts were taken on without too much time to deliberate beforehand. While
responding to the question of what the school could have done better, Teacher 5
commented:
The school did the best they could. Nobody anticipated this; they just found
themselves in this situation which involves making quick decisions in a short period
of time on behalf of a very large community of people. (Teacher 5)
The lack of prior training and technical skills as experienced by the teachers in this
study led to a negative experience for them and their students. It can be concluded
that if the teachers were better prepared, they would have had a better experience
and subsequently, the students may have also had a better experience and attitude
towards online learning.
4.1.2 Classroom Management

In addition to the challenge of the students writing profanities on the shared
screen as indicated by Teacher 1, the other participants also indicated that they
had experienced classroom management challenges, especially in the first few
weeks as they were still learning how to use the platforms. For instance, two
teachers commented:
I found it difficult to manage the classroom at the beginning because I didn’t
understand the technology. Students would talk out of place and I didn’t know how
to mute them. I didn’t know that you could kick a student out of the class. (Teacher
3)
Students can act out because they know that there are no sanctions you can give
them. You are not in school and cannot give them detention. To a certain extent,
you lose authority. (Teacher 2)
374
4.1.3 Lack of Effective Assessment and Immediate Feedback

About the first challenge discussed, the teachers felt that the impersonal nature of
the online instructions meant that the students could not always access help and
the teachers could not immediately identify challenges with learning and address
them right away. For example, Teacher 7 explained:
You cannot monitor students' facial expressions to see if they are following the
lesson. It is only until you ask a question, which you cannot do with all of them all
the time. So that means you can easily lose track of students who are not engaged
in learning in a virtual classroom as opposed to a face-to-face classroom. (Teacher
7)
Similarly, Teacher 7 also stated:

What makes online teaching most difficult for me is not being able to see students’
work as they are doing it and help them right there and then. (Teacher 7).
Another assessed challenge was the originality of the students’ work which was
experienced by most of the participants. For example, Teacher 2 asserted:
Another thing I discovered is that some parents are doing the work for their
children. Actually sitting on the computer and doing the work for them. Brilliant
answers that I know for a fact could not have come from the child. (Teacher 2)
4.1.4 Heavy Workload

The teachers also found that online teaching increased their workload
significantly. In their perspective, online instructions meant more teacher
assessments and more planning. Furthermore, the teachers indicated that they
spent more time even outside of their standard working hours responding to
emails from both students and parents on matters that, during face-to-face classes,
could have been clarified to all students. In addition, the teachers found that
marking work that had been submitted online took more time. For instance,
Teacher 3 indicated:
It takes a lot longer to mark on Google Classroom than by hand, therefore increasing
the amount of time I spend working. The platform does not allow annotating on the
student’s paper or putting ticks. So I have to record them on paper so as not to lose
track of their scores. (Teacher 3)
Another aspect of online learning was where the teachers perceived that their
workload had increased due to the administrative tasks and planning they had to
carry out in addition to also trying to figure out the new mode of teaching. For
example, Teacher 2 expressed:
I struggled in the last three weeks. There has been lots of admin work and I’ve never
been a paperwork person so I really struggled. Plus, with virtual teaching, there is
a lot more planning to do and working much later hours than before.
Moreover, due to how sudden the move to online learning was, the teachers found
that the level of communication and instructions from the school administrators
had increased, thus increasing their workload. For example, Teacher 2 said:
Teachers had volumes of information coming in at the same time that they had to
process and implement without proper direction which normally is not the case so
our workload increased significantly. (Teacher 2)
375
It is clear then that navigating a new system of instruction, new communication

demands, and expectations from the management contributed to an increased
workload for the teachers.
4.2 Students’ Perspectives on their Challenges

4.2.1 Technical Problems
It is apparent from the focus group discussions that despite being commonly
known as digital natives, many of the students struggled with using the platforms.
For example, one student stated:
“I’m a slow typer so it takes me a very long time to finish work than I did when I
was writing.” (Student 1)
Connectivity to the internet was also cited by the students as one of the struggles
that they experienced in their first few weeks of online learning. For example, two
students indicated:
Sometimes the internet connection is really bad like exhibit A, right now I was just
disconnected and had to rejoin the meeting. (Student 4)
If the internet is good, then the microphone works. However, when it is bad then the
microphone does not work and I cannot participate in discussions (Student 5)
4.2.2 Heavy Workload

Similar to the teacher participants, the students also complained about the heavy
workload. For example, Student 6 stated:
I see that teachers give us much more work. When we were at school we didn’t have
as much work. I feel that online learning has brought more pressure on us. (Student
6)
Other students also indicated that when the prospect of online learning came up,
they had anticipated less work but were surprised that the teachers were
assigning even more. As such, the students expressed that they were under more
pressure in terms of schoolwork since moving online. Two students expressed:
When the online school started I thought teachers might be a bit lenient with the
work so that’s why I didn’t expect much. However, I was bombarded with so much
stuff that for two days I didn’t know what to do. (Student 3)
Everyone thought if we start online school the pressure would drop, but I feel like it
has doubled for almost every subject. Even though we are at home there are still only
24 hours. (Student 15)
4.2.3 Lack of Immediate Feedback and Clear Guidelines

Further to the two challenges alluded to above, the lack of immediate feedback
and clear guidelines proved a challenge for some students. The students felt that
reaching their teachers was often difficult as the teachers did not always respond
promptly. This finding was thought-provoking when compared to the one
determined from the data collected from the teachers stated that they found it
hard to respond to the ceaseless student questions sent outside working hours. In
addition, they felt that the teachers often assigned them work without clear
instructions.
376
The voices of two students are captured below:

In normal school we had teachers explain things to us if we didn’t understand.
Teachers could also see our facial expressions and know that we didn’t understand.
(Student 9)
Sometimes we get slides that we do not understand and when we ask some teachers
questions they write a whole paragraph about how they are disappointed in us.
(Student 4).
It is clear from these findings that in addition to the lack of immediate feedback
and clear guidelines, miscommunications that can be attributed to online learning
also occurred. This is seen in how the teachers complained about the incessant
emails adding to their workload while the students feel that the teachers were not
always available to respond to their questions.
4.2.4 Adverse Physiological Effects

The complete shift from face-to-face classes to online learning has meant a change
in activities such as group work, physical education, class presentations, and other
classroom activities to doing most tasks on the computer. This unavoidable
increase in screen time came with perceived adverse physical effects on some of
the students who complained of sore eyes, headaches, and stress. For example,
two students stated:
Sir, don't your eyes hurt? Mine have been hurting so much from sitting in front of
the computer all day long. (Student 11)
I end up spending too much time on my laptop that it makes my eyes hurt (Student
12).
As a result of the increase in workload discussed in section 4.2.2, some of the

students reportedly experienced another adverse physical effect -stress. For
example, Student 4 stated:
I get more stressed now. It is difficult to get used to the amount of homework
(Student 4).
4.3 What are the Attitudes of Teachers and Students Towards Online Learning?
Based on the challenges that we anticipated the novelty of online learning would
bring, we sought to understand what the teachers’ and students’ attitudes would
be towards online learning. The word ‘attitude’ as understood in this study comes
from the Meriam Webster Online Dictionary (2022) which defines it as “a mental
position with regard to a fact or state” (4a) or “a feeling or emotion toward a fact
or state” (4b). Therefore, by asking this question, we wanted to understand the
emotions that the teachers and students had regarding online learning based on
how they were experiencing it at that present time. Furthermore, to understand
their position about whether in the future they would prefer studying online or
face-to-face classrooms. Data from both students and teachers on their
willingness to embrace online learning revealed mixed attitudes. It was clear that
while among the two groups most found that being at home had various benefits,
many also held the perception that there are important aspects of learning face-
to-face that online learning cannot replace or address.
377
4.3.1 Teachers’ Attitudes Towards Online Learning

Most of the teachers held the view that they would prefer to incorporate aspects
of online learning into their teaching as they had acquired some relevant skills.
However, most insisted that this would only work if it was blended with face-to-
face classes rather than working completely online. Their reasons varied but some
perceived online learning to work better with high ability and/or older students,
as well as being impractical for some subjects, and there being the inability of
online teaching to replace human interactions. For example, Teacher 5 stated:
I don't mind blended learning but now that I've had the opportunity to teach
virtually, I feel like it is more suitable for older students because when you are there
you can see them learning. It is also good with high-ability students. So if I were to
do it I would do it with high-ability students. (Teacher 5)
Teacher 7 also stated his apprehension about teaching practical subjects like the
sciences through online learning, stating:
Online learning is great for some subjects but not all, I'm sure in English it's easier
to discuss works of literature with students but with mathematics, you have to show
students each step as you work, it just doesn't work. (Teacher 7)
Furthermore, the lack of interaction, as discussed earlier, influenced the

participants' attitudes towards online learning. The physical interactions meant
that teachers could build rapport g with their students which they viewed as
important to teaching and learning. Physical interactions were preferred when it
came to allowing the teachers to provide guidance they deemed more practical in
a physical environment.
In addition, it was also evident from the findings that some teachers had fears
about their privacy and online security. For example, Teacher 6 stated:
I also don’t think they considered our privacy and we were a bit naïve in not
considering these factors. Our videos could be circulated on the internet and we are
made laughing stocks. I never take a video with my face on it because I don’t want
to and I have that right. (Teacher 6)
Contrary to the doubts and concerns expressed by most teachers, Teacher 4, who
is also the youngest among the teacher respondents, preferred online teaching
over face-to-face teaching. She stated:
The time that I save on commuting to and from work is great. The systems of
submitting work are also a lot easier to manage than having students hand in work
physically. (Teacher 4)
4.3.2 Students’ Attitudes Towards Online Learning

Among the students, it was evident that attitudes were even more polarised than
those of the teacher respondents. For those who had a positive attitude towards
online learning, some felt that learning online allowed them a degree of
independence and the ability to learn new skills. For instance, one stated:
Working online makes me feel almost like a businesswoman which I want to be when
I grow up. It is just cool because I sit at my desk and have all my work laid out. So
it brings the atmosphere of growing up closer. (Student 24)
378
Other students preferred working online as they felt that it allowed them to work
at their own pace. Student 7 indicated:
In school, one is forced to stay there for 7.5 hours while here I can finish my work
in ten or thirty minutes. But also I can go to the mall when I want to. I have more
time; I’m not forced to sit in one place (Student 7)
Conversely, for those who perceived online learning negatively, it was evident
that the biggest influence was human interaction. This included the interactions
with their peers which they deemed important in collaborative learning and
friendships. Furthermore, the students deemed interactions with their teachers to
be vital to their understanding of taught content. For example, Student 16
expressed: “I know it is my house and everything but I just feel better being with my
friends”
Another student also stated:

Studying alone is not fun. We had a lesson where a teacher said we should discuss
with a partner but we were not going to do it because we can't communicate.
(Student 3)
5. Discussion
The findings of this study indicate that teachers and students experienced
numerous challenges in the sudden move to online instruction as a response to
the Covid-19 pandemic. Because schools had to move online so suddenly, it is also
clear from the findings that the lack of time and perhaps consideration when
teachers were preparing before the first day, led to some of the negative
experiences of both teachers and students. It appears that the school
administrators had confidence that because the curriculum had not changed, and
because teachers already had some knowledge of technology, asking them to go
home and try out Google Classroom and Zoom would be sufficient. However,
teachers ended up struggling with the use of both platforms. This negation of their
obligation to provide teachers with effective professional development during
this transition reveals a lack of effective instructional leadership (Leithwood,
Harris & Hopkins, 2019; Harris & Jones, 2020). It is also evident from the findings
that while the teachers did not report having connectivity problems, some of the
students struggled. These findings are in line with the findings from other studies
which indicate that particularly in developing countries, connectivity issues are
common (Ramij & Sultana, 2020; Adnan & Anwar, 2020). As such, schools need to
consider that even though the teachers may not have connectivity challenges, the
students may. As such, it is vital to work with the parents to find solutions.
It is also apparent from the findings that online learning demands new classroom
management strategies. It is also clear that the surge in online learning has
undermined the tech companies’ abilities to protect online users. For instance,
teachers struggling with students speaking out of turn and others writing
profanities on the screen is consistent with other studies and media reports. The
video conferencing platform Zoom, for example, did not initially have security
measures such as waiting rooms that would allow the meeting host to admit
participants into meetings. Studies (Khan, Brohi & Zaman, 2020; Fudge &
Williams, 2020) and media reports (Hodge, 2020; Navarro, 2020) have shown that
379
these lead to various security threats. Therefore, schools must train teachers to be
proactive in their use of these platforms and come up with new rules for classroom
management, as well as sanctions for the students during online learning.
Another important finding discovered by this study was that of a lack of effective
assessment and immediate feedback. The teachers’ perspective revealed that they
feared that because learners could not be seen, some were presenting work they
could not have done by themselves. Furthermore, some of the teachers also
lamented not being able to help students who needed it while doing work.
Similarly, findings from the student participants also indicated that they too
found the lack of immediate feedback a challenge in addition to finding that some
of the instructions sent were unclear. These findings are consistent with other
studies suggesting that the lack of immediate feedback and interaction is
perceived as a challenge by both teachers and students using online learning
platforms (Gay, 2016; Kite et al, 2020). In terms of the reliability of the online
assessments, teachers need to explore other ways of assessing that do not threaten
the integrity of assessments. Furthermore, teachers should also seek out ways of
making their online learning platforms more interactive to allow students to ask
questions and gain feedback.
Another challenge experienced by both the teachers and students in this study
was increased workloads. Teachers complained that the number of instructions
that came in requiring action were incessant in addition to the emails from parents
and students who were also evidently struggling and needing reassurance.
However, while the students stated that their workload had increased, teachers
also confirmed that they were assigning more work and having more marking to
do. It is not clear why going online resulted in an increased number of
assignments. However, these findings are consistent with other studies which
showed that both students and teachers experienced an increase in their workload
once online (Hussein et al., 2020; Hermanto & Srimulyani, 2021). As a result of the
increase in workload, in addition to also attending lessons in front of the screen,
the students spent more time online which led to adverse physiological effects
such as headaches, eye strain, and stress. It is vital that moving forward, school
stakeholders can work together to find ways of reducing the workload without
negatively affecting the quality of teaching and learning.
When it comes to the second research question which sought to understand the
attitudes to online learning, the findings showed mixed attitudes. Most teachers
felt that they would incorporate some of the online learning strategies in
conjunction with face-to-face classes. This attitude for most teachers was
influenced by the perception that online learning is more suitable for high-ability
students. This perception is consistent with the findings by Xu and Jaggers (2013)
who found that online learning was more suitable for academically-inclined
students. In addition, other teachers feared that having their videos online could
lead to an invasion of their privacy and lead to them being made a mockery of
online. This insecurity could be attributed to a lack of prior training and having
been threatened online before. Moreover, it was also clear that they appreciated
the human interactions that come with face-to-face classrooms and that this
influenced their attitude towards online learning. Only one teacher, the youngest
380
participant, preferred teaching online over face-to-face classes. She indicated that
it meant she saved time getting ready and commuting. It is not clear if age is what
is influencing her attitude as in other studies such as the one by Seabra et al. (2021)
which revealed that age does not influence the attitude towards online teaching
during the pandemic.
Referring now to the student participants, most also had a negative attitude
towards online learning because of the experiences they had. While a few
indicated that online learning provided them with the freedom to work at their
own pace and the ability to finish early and do outside-of-school activities, the
majority longed for human interactions which they believed supported their
social and academic needs. Many studies have also indicated that one of the main
challenges of online learning is lack of interaction (Gustiani, 2020; Kite et al, 2020).
Despite these negative experiences, the findings also show that the participants
are not completely against online learning, thus dealing with some of the
limitations experienced could change their attitude towards this mode of learning.
6. Conclusions
The findings of this study have revealed that the sudden migration to online
instruction undermined the school’s ability to train teachers to implement the
move successfully. This led to challenges such as a lack of technical skills, new
demands for classroom management, heavy workloads, and a lack of immediate
feedback and proper assessment. The challenges for students also mirrored those
of their teachers such as a lack of technical skills, a lack of feedback, and a heavy
workload. In addition, the students were also challenged by problems with their
internet connection and unclear instructions from the teachers. The challenges
experienced by both groups influenced their attitude towards online instruction
and their willingness to embrace it in the future. Some teachers felt that they
would rather teach through blended methods, mostly through classroom teaching
with elements of online learning such as posting assignments online. Equally, the
students mostly preferred going back to school due to the perceived impersonal
nature of online learning. These challenges and the subsequent negative attitude
to online learning are evidence that more work needs to be done before online
education is successful.
6.1 Implications of this Study

This study has resulted in multiple implications for teachers and school
administrators to use to enhance the effectiveness of online learning. First,
teachers must empower themselves and learn about the latest technologies used
in education. This will improve their general practice as well as better prepare
them to deal with the unpredictable demands of teaching in the 21st century.
Secondly, it is also imperative that both teachers and school administrators find
ways of making online learning more interactive for students to mitigate the loss
of social interaction that occurs when schools move online. In addition, solutions
must be sought regarding effective assessments in online spaces as well as finding
ways of assessment that reduce the amount of screen time for students.
381
6.2 Limitations of the Study

This was a small-scale inquiry which sampled a single school to participate. Thus,
the findings, while consistent with findings in other contexts, may not be
generalisable. Furthermore, because it was conducted at the beginning of online
learning, the attitudes of the teachers and students may have changed over time.
6.3 Suggestions for Future Research

Two years have passed since the data was generated for this study. Since that
time, various developments have occurred including blended learning and going
back to face-to-face classes. Therefore, we encourage other researchers to study
how the attitudes may have changed towards online learning once teachers and
students got over the initial hurdles reported in this inquiry.
7. References
Abou-El-Kheir, A. (2017). Qatar’s K-12 Education reform-A review of the policy
decisions and a look to the future. https://doi.org/10.17605/OSF.IO/9U8CA
Adedoyin, O. B., & Soykan, E. (2020). Covid-19 pandemic and online learning: the
challenges and opportunities. Interactive learning environments, 1-13.
https://doi.org/10.1080/10494820.2020.1813180
Adnan, M., & Anwar, K. (2020). Online learning amid the COVID-19 pandemic:
Students' perspectives. Online Submission, 2(1), 45-51.
https://doi.org/10.33902/JPSP.2020261309
Alea, L. A., Fabrea, M. F., Roldan, R. D. A., & Farooqi, A. Z. (2020). Teachers' Covid-19
awareness, distance learning education experiences and perceptions towards
institutional readiness and challenges. International Journal of Learning,
Teaching and Educational Research, 19(6), 127-144.
Al-Jaber, H., & Dutta, S. (2008). Qatar: Leveraging technology to create a knowledge-
based economy in the Middle East. In World Economic Forum (pp. 133-134).
https://www.ictqatar.gov.qa/sites/default/files/documents/Qatar_Levera
gingTechnology.pdf
Attitude. (2022). In Merriam-Webster.com. Retrieved on July 07, 2022.
https://www.merriam-webster.com/dictionary/attitude
Blank, G., & Dutton, W. H. (2012). Age and trust in the Internet: The centrality of
experience and attitudes toward technology in Britain. Social Science Computer
Review, 30(2), 135-151. https://doi.org/10.1177/0894439310396186
Chaaban, Y., Sawalhi, R., & Du, X. (2021). Exploring teacher leadership for professional
learning in response to educational disruption in Qatar. Professional
Development in Education, 1-18.
https://doi.org/10.1080/19415257.2021.1973070
Denzin, N. K., & Lincoln, Y. S. (2011). The SAGE Handbook of Qualitative Research.
Thousand Oaks, CA: Sage.
Famularsih, S. (2020). Students’ experiences in using online learning applications due
to COVID-19 in English classroom. Studies in Learning and Teaching, 1(2), 112-
121. https://doi.org/10.46627/silet.v1i2.40
Ferri, F., Grifoni, P., & Guzzo, T. (2020). Online learning and emergency remote
teaching: Opportunities and challenges in emergency
situations. Societies, 10(4), 86. https://doi.org/10.3390/soc10040086
382
Fudge, T., & Williams, L. (2020, August). Zoom in (but fasten your seatbelt).
International Conference for Media in Education 2020 (iCoME).
http://hdl.handle.net/20.500.12264/288
Gay, G. H. (2016). An assessment of online instructor e-learning readiness before,
during, and after course delivery. Journal of Computing in Higher Education,
28(2), 199–220. https://doi.org/10.1007/s12528-016-9115-z
Gustiani, S. (2020). Students’ motivation in online learning during Covid-19 pandemic
era: A case study. Holistics, 12(2).
http://jurnal.polsri.ac.id/index.php/holistic/article/view/3029
Harris, A., & Jones, M. (2020). COVID 19–school leadership in disruptive times. School
Leadership & Management, 40(4), 243-247.
https://doi.org/10.1080/13632434.2020.1811479
Hermanto, Y. B., & Srimulyani, V. A. (2021). The challenges of online learning during
the covid-19 pandemic.Jurnal Pendidikan Dan Pengajaran, 54(1), 46-57
https://doi.org/10.23887/jpp.v54i1.29703
Hodge, R. (2020, May 8). Zoom security issues: Zoom buys security company, aims for end-
to-end encryption. CNet. https://www.cnet.com/news/zoom-security-issues-
zoom-buys-securitycompany-aims-for-end-to-end-encryption
Hussein, E., Daoud, S., Alrabaiah, H., & Badawi, R. (2020). Exploring undergraduate
students’ attitudes towards emergency online learning during COVID-19: A
case from the UAE. Children and youth services review, 119, 105699.
https://doi.org/10.1016/j.childyouth.2020.105699
Khan, N. A., Brohi, S. N., & Zaman, N. (2020). Ten deadly cyber security threats amid
COVID-19 pandemic. Preprint.
https://doi.org/10.36227/techrxiv.12278792.v1
Khodr, H. (2011). The dynamics of international education in Qatar: Exploring the
policy drivers behind the development of Education City. Journal of Emerging
Trends in Educational Research and Policy Studies,2(6),514-525.
https://hdl.handle.net/10520/EJC135457
Kite, J., Schulb, T., E., Zhang, Y., Choi, S., Craske, S., & Dickson, M. (2020). Exploring
lecturer and student perceptions and use of a learning management system in
a postgraduate public health environment. E-Learning and Digital Media 17 (3)
183-198. https://doi.org/10.1177/2042753020909217
Leithwood, K., Harris, A., & Hopkins, D. (2019). Seven strong claims about successful
school leadership revisited. School leadership & management, 40(1), 5-22.
https://doi.org/10.1080/13632434.2019.1596077
McMillan, J. H., & Schumacher, S. (2010). Research in Education: Evidence-Based Inquiry,
MyEducationLab Series. Pearson.
Navarro, S (2020, March 26) Zoom security measures tighten to prevent unwanted
participants. Daily Trojan. https://dailytrojan.com/2020/03/26/zoom-
security-measures-tighten-to-prevent-unwanted-participants/
Newsome, M., Piña, A., Mollazehi, M., Al-Ali, K., & Alshaboul, Y. (2022). The effect of
learners’ sex and STEM/non-STEM majors on remote learning: A national
study of undergraduates in Qatar. Electronic Journal of e-Learning, 20(4), 360-
373. https://doi.org/10.34190/ejel.20.4.2262
Ramij, M., & Sultana, A. (2020). Preparedness of online classes in developing countries
amid COVID-19 Outbreak: A Perspective from Bangladesh. Afrin, Preparedness
of Online Classes in Developing Countries amid COVID-19 Outbreak: A Perspective
from Bangladesh (June 29, 2020). http://dx.doi.org/10.2139/ssrn.3638718
Seabra, F., Teixeira, A., Abelha, M., & Aires, L. (2021). Emergency remote teaching and
learning in Portugal: preschool to secondary school Teachers’ perceptions.
Education Sciences, 11(7), 349. https://doi.org/10.3390/educsci11070349
383
State of Qatar Government Cmmunications Office (2020, March 9). Government

Communications Office statement on the suspension of public and private schools and
universities for all students until further notice as a precautionary measure to contain
the spread of Coronavirus (COVID-19)
https://www.gco.gov.qa/en/2020/03/09/statement-on-the-suspension-of-
public-and-private-schools-and-universities-for-all-students-until-further-
notice-as-a-precautionary-measure-to-contain-the-spread-of-coronavirus/.
The World Bank (2022,). GDP per capita (Current $US)
https://data.worldbank.org/indicator/NY.GDP.PCAP.CD?end=2021&most
_recent_value_desc=true&start=1960&view=chart
World Health Organisation (2020a) Emergencies preparedness, response. Novel
Coronavirus-China. Disease outbreak news 12 January.
https://www.who.int/csr/don/12-january-2020-novel-coronavirus-
china/en/
World Health Organisation. COVID-19 Situation Report 55, (2020b).
https://www.who.int/emergencies/diseases/novel-coronavirus-
2019/situation-reports/
Xu, D., & Jaggers, S. S. (2013). Adaptability to online learning: Differences across types
of students and academic subject areas. CCRC Working Paper 54.
https://doi.org/10.7916/D82N59NB
Zhang, W., Wang, Y., Yang, L., & Wang, C. (2020). Suspending classes without
stopping learning: China’s education emergency management policy in the
COVID-19 outbreak. Journal of Risk and financial management, 13(3), 55.
https://doi.org/10.3390/jrfm13030055
384

Vol. 21, No. 8, pp. 384-406, August 2022
Received Apr 16, 2022; Revised Jul 29, 2022; Accepted Sep 1, 2022
The Classical Test or Item Response

Measurement Theory: The Status of the
Framework at the Examination Council of
Lesotho
Musa Adekunle Ayanwale* , Julia Chere-Masopha and
Malebohang C. Morena
National University of Lesotho, Roma, Maseru, Lesotho
Abstract. While the Examination Council of Lesotho (ECOL) is

burdened with a huge workload of assessment tasks, their procedures
for developing tests, analysing items, and compiling scores heavily rely
on the classical test theory (CTT) measurement framework. The CTT has
been criticised for its flaws, including being test-oriented, sample
dependent, and assuming linear relationships between latent variables
and observed scores. This article presents an overview of CTT and item
response theory (IRT) and how they were applied to standard
assessment questions in the ECOL. These theories have addressed
measurement issues associated with commonly used assessments, such
as multiple-choice, short response, and constructed response tests. Based
on three search facets (Item response theory, classical test theory, and
examination council of Lesotho), a comprehensive search was conducted
across multiple databases (such as Google Scholar, Scopus, Web of
Science, and PubMed). The paper was theoretically developed using the
electronic databases, keywords, and references identified in the articles.
Furthermore, the authors ensure that the keywords are used to identify
relevant documents in a wide variety of sources. A general remark was
made on the effective application of each model in practice with respect
to test development and psychometric activities. In conclusion, the
study recommends that ECOL switch from CTT to modern test theory
for test development and item analysis, which offers multiple benefits.
Keywords: classical test theory; item response theory; Examination

Council of Lesotho; item development; item analysis
1. Introduction
The Examinations Council of Lesotho (ECOL), the central body for all
examinations and assessments in Lesotho, is located right in the heart of Maseru,
*
Corresponding author: Musa Adekunle, Ayanwale; ma.ayanwale@nul.ls
©Authors
385
the capital. The Examinations Council Regulations were enacted in 1986. It was
then configured as a unit of the Ministry of Education and Training (MOET),
responsible for conducting public examinations at the national level. ECOL is a
non-profit organisation that undertakes various functions, including the control
and arrangement of public examinations, the issuance of certificates to all
successful applicants, and any other things necessary or incidental to the proper
administration and functioning of the Council (Exam Council of Lesotho, 2018).
In 2003, the ECOL's mandate was further expanded to include assessing the
educational system's performance and developing continuous assessments for
usage by providing school materials to facilitate the assessments. Therefore,
seeking to become the world's premier assessment institute offering high-quality
services, maintaining high standards of quality in professional education, and
ensuring that integrity is maintained in executing their core responsibility of
internationally recognised certifications to students at the pre and basic levels.
However, ECOL's responsibilities encompass a range of activities, including the
development, implementation, monitoring, and evaluation of an appropriate,
fair, and reliable education assessment system for elementary and secondary
schools, as well as awarding internationally credentialed qualifications that
become part of the education system (Exam Council of Lesotho, 2018).
ECOL also administers level evaluations at specific points in the education

system, such as in Grade 7 when students take their Primary School Leaving
Examination (PSLE), in Junior Secondary, Form C (Junior Certificate
Examination), and in Senior Secondary, Form E (O'Level/LGCSE). Furthermore,
it conducts the National Assessment Survey in partnership with the National
Curriculum Development Center at the elementary level (Grades 4 and 6). Every
two years, these surveys are conducted to evaluate the educational system's
performance throughout the country in numeracy and literacy. It is worth noting
that the Cambridge International Assessment accredits the O' Level curriculum
and examinations. As well as administering exams, the Council acts as an agent
for other international testing bodies, such as the University of London and the
Management College of Southern Africa (MANCOSA). Despite the
overwhelming assessment tasks that ECOL is saddled with, experience and
mode of operation show that their procedures for test development, item
analysis, and scoring framework are heavily reliant on the classical test theory
(CTT) method of measurement, which has been criticised for its shortcomings,
such as test-oriented rather than item-oriented, assumes linear relations between
latent variables and observed scores, hence it is impossible to estimate the true
score directly, or without making strong assumptions, item parameters such as
discrimination and difficulty of the test items depend on the sample used, and
the standard error of measurement, a function of test score reliability and
variance, is universal for all examinees. These limitations can pose several
challenges when used in high-stakes exams such as ECOL. For instance, CTT
fails to account for observed distributions of test scores that have the floor or
ceiling effects, in which a large proportion of examinees score near the low or
high end of the range (Demars, 2017; Jabrayilov et al., 2016; Rusch et al., 2017).
Due to difficulties in resolving these problems within the framework of classical
measurement theory, the measurement community and assessment
386
organisations have switched to a modern theory known as item response theory

(IRT) (Embretson & Reise, 2013).
CTT's shortcomings were addressed with the development of modern theory,

which allows for non-linear relationships, estimation of the true score
independent of the sample used, sample invariant estimation of parameter
values, and gives an assessment expert the ability to select items that are in
accordance with the desired model and applies internal consistency and
reliability concepts to derive more information about how measurements are
conducted. IRT has also been established as an essential tool for test
development, item analysis, and evaluation, which leads to precise, valid, and
relatively less burdensome instrument responses (Edelen & Reeve, 2007). The
plethora of studies have confirmed that the IRT framework offers a multitude of
advantages that have sparked the interest of educational assessment institutions,
test developers, and policymakers in the assessment industry, who have
adopted it for valid and reliable decision making (Ayanwale et al., 2019; Cai et
al., 2016; Embretson & Reise, 2013; Ewing et al., 2005; Ganglmair & Lawson,
2010; Hambleton & Swaminathan, 1985; Lang & Tay, 2021). To date, ECOL has
yet to embrace and integrate the potential of IRT in educational assessments and
testing despite its promising development. The purpose of this paper is to fill
this gap by encouraging ECOL to incorporate IRT into their existing
methodologies by providing an overview of each measurement theory, its
assumptions, its models, weaknesses, and strengths to improve the assessment
and scoring procedures currently used by ECOL, which in turn enhances the
validity of the certificate awarded.
2. Classical Test Theory

Imagine examinees are given 20 questions. Sixteen of the 20 questions are
equally hard; two are difficult while two are easy. The two examinees get 18 test
items correctly. Both get 90%. Examinee "A" has made two simple mistakes,
while examinee "B" has made two very complicated ones. How can we
determine which examinee has more ability? This scenario highlights a
significant flaw in the CTT method of testing. Historically, CTT refers to a theory
of test scores in which three elements (observe, true, and error scores) are
introduced (Hambleton & Jones, 1993; Steyer, 2001). Models of various forms
have been developed within the theoretical framework.
In the classical test model, two unobservable variables are linked to an
observable test score (X), true score (T), and error score (E), that is: X = T + E.
True score cannot be observed directly; It can only be estimated from an
examinee's responses to a set of items whose responses correspond to the actual
abilities that particular examinees possess, though there are inherent errors in
estimation. Factors such as fatigue, guessing, or stress can cause random errors
(Bovaird & Embretson, 2012). Examinees' observed scores represent their total
scores on a test. It would have been the true score if not for the error score.
Standard error of measurement (SEM) plays a major role in CTT, which are
standard deviations of measurement errors for each group of examinees. A test's
variability or spread can be determined from its measurement errors. In X = T +
E, the true score equals the average of a person's observed scores and accounts
387
for measurement error. Because measurement error cannot be determined, every

standardised test has an SEM. SEM is measured in standard deviations. In this
way, the reliability of the test is determined. Precision and reliability of
measurements are higher with a smaller SEM. The error in CTT is conceived as
random and non-systematic. Several factors, internal or external to the examinee,
may account for it. Test items created poorly or tested under poor conditions are
examples of external errors. Internal errors are those caused by the examinee,
such as fatigue, stress, and a lack of concentration (Ayanwale, 2019).
Item and test level statistics are part of CTT. Item difficulty and discrimination
are analysed at the item level. Item difficulty index is represented by ‘p’ and
indicates the proportion of correct answers. The item discrimination index is
indicated by a ‘D’; it tells us how distinct the item is between those with high
and low abilities. CTT looks at the reliability of parallel tests (Demars, 2017). A
parallel test measures the same latent ability with the examinees having the
same true score and errors on both tests. Many items are generated that
represent a single content domain for parallel tests. Ideally, this set should have
twice the number of items intended for a single test form (Brown, 2013).
2.1 Assumptions of classical test theory

In CTT, three assumptions are made. First, the correlation between the error and
true scores is zero. In this case, the variance of a true and error score is equal to
the variance of the observed score, which is true if ƔTe = 0 (Steyer, 2001). In the
equation Var(X) = Var(T) + Var(E), Var(.) is the variance, while the reliability
Rel(X) is defined as
Var (T ) Var (T )
Rel(X) = = ------------------Eqn. 1
Var ( X ) Var (T ) + Var ( E )
Consequently, correlation coefficients between two parallel measurements
determine the reliability of the CTT test. Adedoyin (2010) argues that error
variance decreases as measurement reliability increases. When the error variance
is small, the observed score of test-takers is close to the true score. However,
when error variance is large, observed scores do not always reflect true scores
(IResearchNet, 2022). The second assumption says errors have a zero mean.
Thus, these random errors are expected to cancel out over many repeated
measurements, resulting in a zero expected mean error rate. The observed score
𝑛
E
equals the true score once an error is zero, (X=T),∑ N
=0
𝑖
A third assumption is that measurements from parallel are uncorrelated. A

parallel test is defined in classical test theory as two measures of X and X1 that
have the same true score (T=T1) and the same observed variances δ2(X) = δ2(X1).
Ojerinde (2013) suggests that two tests can be considered parallel if the expected
values of X and X1 are equal (that is, E(X) = E(X1)). There is typically an equal
error variance for the two parallel scores if X ║ X1 if X1 = X2 = Ti + Ei for every
population of tests.
388
2.2 Classical test theory method of item analysis

Test items are analysed quantitatively and qualitatively to determine their
characteristics. To facilitate instrument improvement, the purpose is to revise or
discard items that do not meet minimally acceptable standards. In item
development, it is crucial to consult experts who possess a mastery of relevant
materials. Experts and review boards find it difficult to determine the quality of
"poor" items because of the test content's multidisciplinary nature and
examinees' demographics (Krishnan, 2013). Data analysis helps identify issues
that slipped experts' attention. The goal of item analysis is to select items that
maximise reliability. Matching what is taught with what is assessed is crucial.
There should be a mixture of basic and advanced knowledge in any exam.
Examinees become frustrated if items are too difficult, while overconfidence and
a decline in motivation are consequences of too easy assessments (Esmaeeli et
al., 2021). Creating item banks that are reusable is important through item
improvement. Ayanwale et al. (2019); Crocker and Algina (1986) defined item
analysis as evaluating test items for test construction and revision. This is a
technique for improving test items. In addition to identifying biased or unfair
items, item analysis can also identify poorly worded questions (Grand et al.,
2013; Khan et al., 2013). Results of item analysis are then used to refine the items
of interest. Revision is needed for items that are more difficult or too easy. In
addition, test scores can be observed to enhance item analysis by observing their
reliability, although the literature on measurement discusses item analysis
separately from reliability. To establish test scores' reliability, item difficulty and
discrimination are essential components of item analysis(Elgadal & Mariod,
2021; Toksöz & Ertunç, 2017).
2.3 Parameter estimation of Classical test theory

Item difficulty is an important concept in CTT. For DeVellis (2006), it is the
percentage of examinees who answered an item correctly. In CTT, item difficulty
is sample-based. These values are invariant only for groups of similar level
examinees. CTT often refers to item difficulty as a p-value. Divide the number of
respondents who selected a particular answer by the total number of
respondents in the sample to find the percentage of those deciding to pick that
response, and you get a p-value for each response and the correct answer. The p-
values can be expressed mathematically as:
p = number of an examinee who got the item right
total number of an examinee who attempted the items
The proportion of examinees that got the item wrong can be expressed as:
q = number of an examinee who got the item wrong
total number of an examinee who attempted the items
Hence, pq is the variance, and (SD = √pq) is the standard deviation. The item
difficulty index (p) ranges between 0 and p ≤ 1. A value of 1 is considered to be
very simple if all members of the sample correctly answered the question, while
a p-value of 0 is indicative of none of the respondents in the sample answering
the question correctly; such an item is said to be hard (Cappelleri et al., 2014;
Kline, 2014). For Courville (2005), Items with dichotomously scored items have a
389
greater item variance (that is, σi2 = piqi), indicating the importance of the item
difficulty (p) in the variance measure, while (q) indicates the significance of the
item type (difficulty). The items variance and the total variance of the result are
thus representations of item difficulty.
Further, Crocker and Algina (1986) pointed out that the item difficulty of a
norm-referenced test usually falls between 0.60 and 0.80. The reason is due to the
item format typically used on such tests. Open-ended questions have a remote
or zero likelihood of being answered correctly. The probability of guessing
correctly increases when the test format is multiple choice. As a result, p is the
proportion of respondents who know the answer (p), and 1/m is the number of
responses that reflect how many of those who didn't know the answer but
correctly guessed (m) responded. As a multiple-choice test, we do not aim to
maximise item difficulty at p=0.50; instead, we aim to maximise item score
variability. Hence, item difficulty should optimise item score variability. The
proportion of correct answers is 1/m, which is known. In addition, item
variation at 0.50 is the optimal level; the p-values of items with maximum true
score variance also vary due to examinees' random guessing. This can be written
0.5
as p1= 0.5 + , where p1 is the observed p-value, and m is the number of
m
alternatives or distracters.
However, the item difficulty index that maximises item variance is

(p1=0.5+0.125) = 0.63 for multiple-choice items with four (4) options, and
(p1=0.5+0.1) = 0.60 for five (5) options (Cohen & Swerdlik, 2009; Cohen et al.,
2013; Filgueiras et al., 2014; Hill et al., 2013). Items in a test with a difficulty level
higher or lower than 0.60 and whose difficulty level exceed or fall below 0.63
should be deemed inappropriate. In traditional norm-referenced testing, items
with a difficulty index greater than 0.70 or less than 0.30 are considered bad
items (Adegoke, 2013; Hambleton & Jones, 1993).
Item discrimination is another CTT parameter. It indicates that the examinee's

ability differs. Generally, high, average, and low scores are expected. Among the
purposes of analyzing test items is selecting items that can separate examinees
into different categories with respect to their abilities. High-ability examinees
should be able to score a test item correctly, while low-ability examinees will
score it incorrectly. Test items that have such properties are discriminatory by
nature. Criterion scores place examinees in upper or lower groups based on their
total test scores. This grouping of examinees makes the discrimination index
controversial (Algina & Swaminathan, 2015; Rusch et al., 2017). The lower
group had 50% participants, while the higher group had 50%. A criterion of
interest is easily distinguished between very high and very low scores.
For Kelley (1939), cited in Ayanwale (2019), suggested that instead of 50%-50%,
the item discrimination statistic would function correctly with a 27%-27% split
since it would omit 46% of the data. As the sample size increases, the same
statistic becomes as stable and useful when using a 27%-27% split (Crocker &
Algina, 1986). A high score on a particular item usually indicates an examinee
who has done well on the test. Hingorjo and Jaleel (2012); Vyas and Supe (2008)
390
suggest that items with negative discrimination should be revised or discarded if

they are selected by a larger percentage of the lower scoring group than the
higher scoring group. If an item is high- or low-performing, item discrimination
can be calculated as D = pu – pl, where pu is the proportion of correct answers for
the upper group, and pl is the proportion of correct answers for the lower group.
After identifying the top 27% and bottom 27% of examinees, the percentage
passing for each item is calculated for each group. The item discrimination index
is obtained by subtracting the 'p' of the lower-performing group from the top-
performing group. The index ranges from 0 to 1. A classic interpretation of item
discrimination is provided by (Ebel, 1965):
1. If D ≥ 0.40, very well-functioning item.
2. If 0.30 ≤ D ≤ 0.39, reasonably well-functioning item.
3. If 0.20 ≤ D ≤ 0.29, marginal items need to be revised.
4. If D ≤ 0.19, a poorly functioning item needs to be expunged or fully revised.
More importantly, a discrimination index provides information about how an

item differs on a certain criterion. This is problematic since it ignores a lot of
data. For example, several examinees are omitted (46% of respondents), and
information regarding examinees in the higher and lower groups (Courville,
2005). The product-moment correlation coefficient is applicable when the total
and item scores are interval scales. A point-biserial correlation between
dichotomous scored items and the total score is employed to resolve the
problem (Adegoke, 2013). It measures the direction of the linear relationship of
one factor with another that is continuous (Privitera, 2012). In point-biserial
 ( y −  x )  p
notation, pbis =   with μy is the criterion score mean for the
 x
q

proportion of respondents answering the question correctly, while μx is the
overall criterion score. A correlation coefficient between an item's performance
and an examination's performance is also used to establish item discrimination
(Brown, 2013; DeVellis, 2006). As a result, p-bis represents the correlation
between items and total scores. The correlation should be positive since it
demonstrates that correct answer holders scored higher and incorrect answer
holders scored lower. If negative, you should revise or discard the items. The
higher the value, the stronger the discrimination.
2.4 Reliability of scores in the context of classical test theory

The reliability of a test is the ability for identical scores to be achieved over a
specified period whenever the same population of test subjects is examined
(Demars, 2017). A reliability coefficient is expressed numerically, and any value
around 0.70 and above is a good estimation of the reliability coefficient for an
instrument (Preston et al., 2020). Tests with perfect reliability are seldom
available, that is, tests capable of reproducing the same scores when
administered to the same group. The observed results of a highly reliable test are
close to its true scores. Therefore, using the square of the correlation between the
observed and true score, the reliability coefficient can be calculated (Birnbaum,
391
1968). For Dent et al. (2001), true score variance is considered when determining
reliability. Reliability estimates are based on random measurement errors and
can be categorised into different types (Gay et al., 2011).
Using the test-retest method of reliability has two shortcomings. It is costly and
time-consuming to administer the instrument for the first time, let alone for a
second time. The resulting higher cost is far more concerning. If the sample
population is high in mortality, it is more difficult to assess the reliability
(Crocker & Algina, 1986). Secondly, the test-retest method can cause reactivity,
as described by (Downing, 2003). Reactivity occurs when repeated testing
eventually leads to a substantive change. In testing, memory is the main cause of
reactivity. The memory may impact performance on the second test from the
first test. Alternative reliability tests have been developed to solve these issues.
A correlation is established between two similar tests administered to the same
group (Crocker & Algina, 1986).
The test-retest method has reactivity problems which the alternative form
method solves but has its problems. This method has a significant flaw because
it is impossible to guarantee that each test samples the same content. This
happens whenever you use two tests. To solve this problem, a single test
reliability coefficient was developed. One administration of a single test is a
method of estimating reliability. As a method of assessing reliability, internal
consistency relies on the extent to which items within a single test are consistent
with each other and the test overall. Split-half reliability is appropriate for long
or hard tests, and Kuder-Richardson reliability (KR-20) is only appropriate for
items with dichotomous scores, like selection-response tests. By using the split-
half method, a test is given to all samples at once, then the test is divided into
two parts, and the parts are compared (Crocker & Algina, 1986; Jabrayilov et al.,
2016) claimed that splitting the test in many ways won't produce a unique
estimate of reliability. This caused an important issue in reliability. Spearman
(1910) developed the Spearman-Brown formula to estimate the reliability
coefficient for the scores on the whole test to correct the pitfalls associated with
split-half correlation.
For reliability estimation, item covariance methods are the most commonly used.
The Cronbach alpha coefficient is the main method used to measure the internal
consistency of a test or scale in the psychology and education fields (Demars,
2017). Alpha is merely a measure of precision and is not a measure of stability
(Crocker & Algina, 1986). Kuder Richardson 20 (KR20) is the second item
covariance analysis. Each item in the test is rated between 0 and 1. This score
indicates how items in a given test measure the same construct or concept—the
alpha coefficient increases when test items are highly correlated. Testing
reliability and alpha are not only affected by correlation, but also depends on the
length of the test. Therefore, a low value of alpha may reflect poor inter-item
correlation or a long test. Mona (2014) recommends eliminating items with poor
correlation or revising them.
392
2.5 Merits of classical test theory

CTT remains popular among educators despite new approaches to measuring
proficiency (De Champlain, 2010). Its basic concepts are straightforward. Among
its advantages, it makes relatively weak assumptions. The assumptions in CTT
enable it to be applied to a wide variety of data. Anyone with basic mathematics
skills can quickly grasp the concepts, as they aren't mathematically demanding.
Cronbach's alpha measures reliability. CTT can be used to conduct the analyses
with the common statistical packages. Psychometricians in education and
psychology find it more acceptable.
Further, CTT-based measurements of instruments easily fit into underlying

models, thus yielding desirable results. CTT is appealing because individual
items don't have to be optimal, even if they relate only partially to an underlying
construct; the concern can be alleviated by creating several items that assess the
construct. Researchers have found that reliability can be improved to any
desired level by increasing the number of items on a specific test concerning a
variable (Wells & Wollack, 2018).
2.6 Limitations of classical test theory

For Rusch et al. (2017) noted that the assessed sample of examinees influences
both item difficulty and discrimination indices. The study of Kolen (1981) found
that the difficulty index is higher for examinees with high ability. In CTT, item
difficulty has a bearing on examinee ability scores. Observed test scores are
higher if the items are easy and lower if they are difficult. Another flaw in CTT
assumes that all examinees have the same measurement error. A test's type
influences test scores and true scores. The items on the test determine what
students' scores will be. It is still possible to score lower on difficult tests and
higher on easier ones, even though one has the same ability. Depending on each
student's ability level, scores differ in error amounts. CTT also has the limitation
that the same items must be used to compare examinees' performance. Parallel
forms are difficult to achieve in CTT, further aggravating this limitation. Test
reliability depends on parallel testing, which is based on a sample provided by
the examinee.
For Traub (2015) argues reliability is a useful index of a test score's quality. Such
an indicator depends on the characteristics of the group of test-takers. It is also
test-oriented, making it difficult to predict examinees' responses to a test item
(Crocker & Algina, 1986). Test developers cannot predict a test taker's
performance on a particular item based on the CTT model. Examinee and item
dependence is the most significant limitation of CTT. They are both affected by
changes in the other's characteristics. Hence, comparing the characteristics of
different tests and items taken by different groups of students is difficult.
3. Paradigm shift from classical test to item response theory

Several new measurement methods are being developed due to the limitations
discussed above. In CTT, the group dependence, item-examinee ability
mismatch, weak assumptions, and parallel testing problems present limitations.
As an alternative, item response theory (IRT) or latent trait theory provides a
solution to CTT's shortcomings (Bovaird & Embretson, 2012). Many other
393
models focusing on measurement issues developed an alternative model. Since

IRT focuses on the item, all statistical analyses are done at the item level. It is one
of the greatest advantages over CTT. Numerous studies in the fields of
education and psychology have highlighted the same concept (Cappelleri et al.,
2014; Embretson & Reise, 2013; Tay et al., 2015). The evidence demonstrates that
IRT is widely used in these fields, and medical education is no exception (De
Champlain, 2010; Downing, 2003; Preston et al., 2020).
Moreover, IRT is widely used to develop valid and accurate data about students'
learning competencies in testing centers worldwide. The CTT assumptions were
challenging to test and apply to practical problems, leading to alternative
measurement models. The models are essentially extensions and liberalisations
of conventional test theory. In addition, IRT is a necessary tool that has to be
available in any large-scale testing center that requires a valid and reliable
instrument.
4. Item response theory

IRT is a statistical model that describes both examinee items and test
performance and further explains how the test results relate to the abilities
reflected in the items on the test (Embretson & Reise, 2013). Responses to items
may be discrete, continuous, or dichotomous. A score category may be ranked or
unranked. One or more abilities may explain test scores. A variety of models can
explore the relationship between item responses and underlying abilities. IRT
has established and applied several models to test data. Hambleton and
Swaminathan (1985) analysed the characteristics of item models as follows: the
model should describe how observed responses relate to underlying non-
observable constructs, the model should provide an estimation of the underlying
construct, the examinee's scores must help estimate the construct, and the
performance of an examinee must be completed predicted or explained by the
underlying constructs. According to item response theory, an examinee has
some unobservable, latent abilities that cannot be studied directly. IRT is used to
develop models to relate latent traits to observable characteristics of an
individual, especially their abilities to correctly answer questions in a test (Baker
& Kim, 2017; Magis, 2007).
IRT employs mathematical functions, unlike classical test theory (CTT), which
uses the model X=T+E. Based on Hambleton and Swaminathan (1985), IRTs are
characterised by a strict relationship between responses and traits. Further, IRT
is based on the assumption that one or more examinees' abilities can be
predicted from theta (θ), which constitutes one of the parameters. Additionally,
Crocker and Algina (1986) found that the observed score and the ability
parameter are related to the observed and true scores. Their study highlighted
that item difficulty and discrimination do not depend on examinee
characteristics. Additionally, the ability estimates are likewise independent of
the items and can be described as item-free, while the ability parameters are
person-free.
394
4.1 Assumptions of item response theory

IRT models are fitted to data based on assumptions about the data (Edelen &
Reeve, 2007). Assumptions in test theory state that an examinee will answer the
question correctly if they know the correct answer. Without this assumption,
testing is not justified. Other assumptions include local independence,
dimensionality, and monotonicity. The assumptions should hold regardless of
the latent trait method employed. These assumptions must be met for a test set
to be meaningful when estimating latent trait models (Bichi & Talib, 2018; Zhu &
Lu, 2017).
4.1.1. Dimensionality
A set of latent traits can explain test performance. A vector of ability scores can
describe an examinee's ability in n-dimensional (θ1, θ2, θ3….. θn). Items
responding to the test with more than one latent trait are called
multidimensional items, while items responding to the test that assumes one
latent trait are known as unidimensional items. Only one area of knowledge,
ability, or construct is measured in the items (Tay et al., 2015). The items on a
one-dimensional test reflect only one dimension. One-score tests implicitly
imply that the items share an overarching primary construct. In this model, each
examinee is assigned single theta *, and uncontrolled variables can affect item
responses as nuisance dimensions unique to the item and not shared by other
items (Adewale et al., 2017). A test or ability scale containing all its items must
measure a single latent attribute of an individual. Violating this assumption may
lead to misleading results (Immekus et al., 2019).
In their study, Ojerinde and Ifewulu (2012) identified multiple methods for
testing unidimensionality, such as the Cronbach analysis test, exploratory factor
analysis, eigenvalue test, random baseline test, biserial test, factor loading test,
congruence test, congruency or part-to-whole test, and vector frequency test, as
well as confirmatory factor analysis. Various methods exist for assessing the
unidimensionality of test data, depending on the nature of the test data.
Predictive continuous and normally distributed data are tested for
unidimensionality through parallel analysis, which VistaParan and MPLUS
implement, or confirmatory factor analysis based on Pearson's correlation matrix
(Adewale et al., 2017; Kline, 2005) implemented in AMOS or LISREL. Generally,
polychoric correlation can be used parallel (implemented in FACTOR; Vista-
Paran ) when the data is ordinal (Metibemu, 2017). In dichotomously scored
data, nonlinear factor analysis implemented in normal Ogive harmonic robust
moment (NOHARM), parallel analysis based on tetrachoric correlation matrix
(implemented in Vista-Paran), full information item factor analysis
(implemented in EQSIRT, MIRT R package, and TESTFACT), bootstrap
modified parallel analysis test (implemented in Itm R package), and stout
essential dimensionality test (implemented in DIMPACK package) can be used
(Ackerman, 2010; Finch & Monahan, 2008; Finch & French, 2015; Reckase, 2009).
The next IRT assumption is local item independence.
4.1.2 Local item independence

Local item independence means that the chance of an examinee getting an item
right is not affected by how they answered other items on the test. The fact that
395
students perform independently on different items does not mean they do not
correlate; their abilities determine their performance. An examinee's
probabilities are associated with a set of items related to the probability of a
response pattern on that set of items. An ability is constant at a particular
measurement time when it influences responses to a set of items. Therefore, the
relationship between the two items should be as close as possible to zero. The
responses may, therefore, be influenced by factors other than what the
instrument was designed to measure. Given an individual's score on the latent
trait, the observed items should be independent of each other (Debelak & Koller,
2020; Song et al., 2019). Independent means are statistically independent.
Statistically, independent items exhibit their qualities and consider examinees'
abilities to unfold their characteristic functions about them (Behavior et al., 2012;
Yen, 2006).
Several approaches assess whether local item independence is valid (Debelak &
Koller, 2020; Kim et al., 2011). These methods include the likelihood ratio G2, the
power-divergence (PD) statistic, the Q3 statistic, Fisher’s r-to-z transformed Q3,
the Wald test, the likelihood ratio test in logistic regression (LR G2), the absolute
value of mutual information difference (Tsai & Hsu, 2005), the mutual
information difference (MID), the modification index (MI) in structural equation
modelling (SEM), and the use of the residual correlation from the factor analysis
(FA). Among the methods, only the likelihood ratio G2 method is implemented
in a popular IRT computer program such as item response theory-Patience
response outcomes (IRTPRO). For Chen and Thissen (1997); Tang et al. (2020)
proposed that the local dependency (LD) χ2 statistic be computed by comparing
the observed and expected frequencies in each of the two-way cross-tabulations
between response to each item and each of the other items. Standardised χ2
values (roughly Z-scores) become large when a pair of items indicate local
dependency (Chen & Thissen, 1997). Additionally, an LD number greater than
10 signals local dependence (Adewale et al., 2017; Gay et al., 2011). The study
(Yen, 1993) suggested Yen's Q-3 statistic as an effective measure for assessing
local independence; after controlling for person location estimates, the Q3
statistic is the correlation of residuals between two items. The next IRT
assumption is monotonicity.
4.1.3 Monotonicity
A normal ogive is the item response function (IRF). Item response curves have a
mean of 0 and a standard deviation of 1. Item response functions are also known
as item characteristics curves. Items characteristic curves (ICC) relate the
probability of success on items to the ability measured by the item. In Birnbaum
(1968); Lord (2012), ICC is invariant across groups of test takers, resulting in the
invariance of item parameters that produce the item characteristic curve. This
aspect is a prominent distinguishing feature of IRT compared to CTT.
The study of Hambleton and Swaminathan (1985) argues that invariance of item
characteristics and ability parameters means that characteristics of an item do
not depend on the abilities of examinees, just as characteristics of examinees do
not depend on test items. ICC represents non-linear regressions between item
score and latent trait. Because the variable and probability are unbounded, the
396
relationship will be nonlinear. It shows the probability of answering a question

correctly as a function of ability. No matter the distribution of examinees, the
probability is constant. In this case, the ICC will take the shape of a normal ogive
since the probability remains the same no matter how many other examinees are
nearby. The ICC has three sections: the lower asymptote, the upper asymptote,
and the middle part. An ICC might require several parameters depending on the
logistic model, as shown in Figure 1.
Figure 1: ICC showing parameters
Source: (Ojerinde & Ifewulu, 2012)
ICC curves are characterised by difficulty and discrimination. The b parameter

represents item difficulty as measured on a particular axis whose values range
from -∞ to ∞; traditionally, the values are -3 to +3 when θ has a mean of 0 and a
standard deviation of 1. Nevertheless, examinee ability over ±3 isn't common.
Item difficulty value is high when the items are hard to answer. Low-ability
examinees are less likely to get the correct answer. Easy items are those with low
difficulty. Candidates with lower ability values are the potential to answer test
items correctly. As for discrimination, that is also called "a" parameter. This
information relates to whether an item may discriminate between examinees
with abilities below and above the item location. The discriminating index
parameter is calculated by tangentially connecting the curve to the difficulty
level (b) parameter (Baker, 2001; Baker & Kim, 2017). A discriminating index
parameter ranges from -∞ to ∞, with a typical value of ≤ 2.0. Hence, the steeper
the curve is, the more discriminative the item is. (Baker, 2001; Bichi et al., 2019;
Clark & Watson, 2019; Pliakos et al., 2019) indicates that low 'a' values are not
useful for discriminating between ability levels. IRT's third parameter is
guessing, called the 'c' parameter. Examinees of low ability respond correctly to
an item when this parameter is lower than the asymptote parameter. When the
three-parameter model is used, the parameter ‘c’ has the theoretical range of 0 ≤
c ≥ 1.0, however, values above 0.35 are considered unacceptable (Ayanwale,
2019). Therefore, θ ≤ c ≤ 0.35 is usually used.
397
4.2 Item response theory and its model

IRT items can be scored dichotomous or polytomous. Four models are usually
used for dichotomous items. They are classified into one, two, three, and four-
parameter logistic models (Cai et al., 2016; Cappelleri et al., 2014). However, the
three most frequently used parameters are difficulty (b), discrimination (a), and
guessing (c). Before each model can be used, it must meet all the necessary
assumptions. The simplest of the three models, the one-parameter logistic
model, is also called the Rasch model (Crocker & Algina, 1986; Hambleton &
Swaminathan, 1985; Nataranjan, 2009). A logistic function between an
examinee's ability (θ) and the difficulty of the question (b) is assumed to
determine the chance that a correct answer will be provided. This is illustrated
below.
P (
1
) = …………………Eqn. 2
b 1 + exp −1( − b )
Furthermore, the two-parameter logistic model is a dichotomous IRT, in which

the shape of the item response function is governed by two parameters,
discrimination (a) and difficulty (b). The item response function increases
monotonically when 'a' is positive or negative. However, as 'a' increases, the
slope steepens. Positive item response functions are located with the larger value
of 'b.' Examinees with the ability (θ) have the following likelihood of answering
test items correctly. This is illustrated below.
P (
1
) = ……………..Eqn. 3
a, b 1 + exp − a ( −b )
A parameter “c” calculates a lower asymptote parameter of the three-parameter
logistic model, especially useful for multiple-choice and true-false tests. As c
increases, the lower limit of the item response function also increases. The
expression is as follows:
1
P ( ) = C + (1 − C ) (0  C  1) ……………Eqn. 4
a, b, c 1 + exp− a ( −b )
Fourth parameter ‘d’ logistic models are dichotomous IRT models in which an
upper asymptote parameter is added to the three-parameter model. As ‘d’
increases, the upper limit of the item response function (IRF) increases. Even
with extreme levels of a trait, some items are so difficult that students cannot
answer them all. The item's upper asymptote doesn't equal 1. The model fit will
be improved by including a lower and an upper bound for the item response
(Reise & Waller, 2009). A common use is to assess disorders that lead to
extremely rare behavior. Hence, it is possible to expect that adding parameters
will lead to an increasingly complex and well-fitting model. A mathematical
expression for the model is:
398
d −c
P ( ) = C+ (0  c  d  1) …..Eqn. 5
a, b, c, d 1 + exp−a ( −b )
Its advantage over CTT is that only adequately scored IRT can detect the
significant differences between individuals whose scores are slightly different.
When trait scores are incredibly high or low, they are out of the normal range.
The IRT method solves this problem. Reise and Waller (2009) stipulate that items
should be "difficult" enough for the levels of the trait in question. As a result, the
four logistic parameter model, which incorporates time and slowness time
responses, has yet to be fully integrated into conventional IRT models (Zhang,
2012).
4.3 Item response theory item analysis

The process of item analysis consists of assessing an item's quality in a test and
the test as a whole based on the test results (Sim & Rasiah, 2006). In this way,
items can be improved for future use, while those that are inadequate can be
discarded. IRT analyses a scale at the item level by calculating item difficulty,
discrimination, and test information function. Further, it computes the standard
error (SE) for parameters "a" and "b" for each item and estimates the relationship
between items and the constructs. Items may be positioned around theta (θ) or
distributed uniformly from - ∞ to + ∞, depending on the purpose of the analysis.
The location parameters of the instrument should be as close to the cut-off as
possible when used to identify examinees for remedial measures or grouping
them. For IRT models to be fully effective, item parameters must be calibrated
with the right model.
The IRT model that best fits the data determines the model for item calibration
of a test under development. An analysis of model-data fit is the only way to
determine the right choice of item response theory models, as proposed by (Lee
& Ansley, 2007). The model-data fit of item response theory models is critical
when applied to real data. Estimated parameters may be compromised when a
model does not fit the data (Bovaird & Embretson, 2012; Cai et al., 2016). To
validate item response theory applications, fit tests of models need to be
performed (González & Wiberg, 2017). According to Embretson and Reise
(2013), checking item fit involves some issues. Item fit analyses can be used to
identify a test model that retains the integrity of observed data, to identify
extraneous dimensions that affect test item responses, and as a method of
identifying faulty item construction, that is, incorrect keying and item fit, that is,
those that indicate calibration errors during test development.
An item that does not fit a specific model is considered a poor fit (Hambleton &
Jones, 1993). Comparing the observed performance of individual items with the
predicted performance under the chosen model is a common way to assess
model-data fit (Lee & Ansley, 2007; Yu et al., 2007). Based on Courville (2005),
plots of observed and predicted score distributions or the chi-square test may be
used to compare observed and predicted data. In Embretson and Reise (2013),
examinees are first ranked according to their estimates (θ), then grouped into
fixed or subjective categories. According to an item response function or item
399
characteristics curve, the proportion of examinees that answer an item correctly

is calculated. A literature review on chi-square research shows that no chi-
square fit index is preferred over another (Hambleton & Swaminathan, 1985). In
Reise (1990), expressed chi-square as follows:
N j (Oij − Eij )
2
H
 2
=
Eij (1 − Eij )
……………………..Eqn. 6
j =1
'i' is the item, 'j' describes the interval based on examinees' ability estimates, 'H'
represents the number of examinees within any interval, 'Nj' indicates the
number of examinees with (θ) estimates within a given interval, and 'Eij, 'the
expected proportion of keyed responses for intervals using an item response
function evaluated at the median (θ) estimate within an interval. Chi-squares
with high estimates diagnose items that do not fit the model, that is, those items
performing differently than expected.
The likelihood ratio (G2) is a chi-square statistic representing two tests of overall
fit when items on a test are ten or less and twenty or more. (Rupp, 2003;
Tuerlinckx et al., 2004) calculate the chi-square (χ2) statistic as follows:
2n
ri
G = 2 ri log e
2
…………………Eqn.7
i =1 N − Pi i
Where 2n represents the number of possible patterns for each ‘n’ binary item
scoring, ‘ri’ is the observed frequency of pattern 'i', 'N' is the number of
respondents, and ‘Pi’ is the estimated marginal probability. The number of
degrees of freedom is 2n-Kn-1, where K is the number of parameters in the
response model. Thus, if 'G2'>a critical value, the null hypothesis is rejected, and
the ICC is expected to fit the item (Rupp, 2003).
4.4 Item and test information function

Test development and evaluation benefit from item information functions when
lCC are fitted to test data. The corresponding item statistics and item
information functions (IIF) will be incorrect if the ICCs do not fit the data well. It
may be hard to use an item in all tests even when the fit is good if the parameter
is low and the parameter is high. Additionally, an item may provide
considerable information at one end of the ability continuum but be of no use on
another end of the continuum. The information functions indicate how each item
and the test estimates ability over the scale. IRT considers the test information
function as a reliability coefficient since the variance measures the precision of
measurement (Alagoz, 2005). Asymptotic distribution of the maximum
1
likelihood estimator θˆ has mean θ and variance  = , where I ( ) is the
2
I ( )
amount of information. The ability estimate will be less precise, and the
available information about an examinee's ability will be less when the variance
of an estimator is large. The information function for the test with n items is
defined as:
400
I ( ) = 
n
Pi ( )
2
, Where Pi ( ) is obtained by evaluating the item
i =1 Pi ( )Qi ( )
Pi
characteristic curve at θ and Pi ( ) = . The item information is the

decomposition of test information into each item. It is given as:
n
I i ( ) = 
P ( )
/
i
2
, where Pi ( ) =
/ Pi
……………………………..Eqn.8
i =1 Pi ( )Qi ( ) 
Therefore, CTT's reliability coefficient and item reliability correspond to the test
and IIF (Alagoz, 2005). An important element of IRT is the test information
function (TIF). It shows how well the test estimates ability across a broad range
of scores. A test is used to assess a person's ability so that the amount of
information obtained from the test at any level can also be obtained (Birnbaum,
1968). In a test, there are items; therefore, the test information consists of the item
information at a given level of ability. Each item contains a specific amount of
information. The mathematical definition of item information may differ
depending on the item characteristic curve model employed. The test
information function therefore is:
𝐼(𝜃) = ∑𝑁
𝑖=1 𝐼𝑖 (𝜃) ………………………………………….Eqn.9
𝐼(𝜃) is the amount of information in a test at any ability level 𝜃, 𝐼𝑖 (𝜃) is the
amount of information in each item, and N is the number of items in the test.
Specifically, the TIF predicts the degree of accuracy at which we can measure
any value of latent ability. Generally, the level of information in a test will be
higher than that in a single-item test (Baker, 2001). When several items are
included in a test, the greater the amount of information is revealed. More
extended tests better measure test takers' abilities than shorter tests. A test
information function may be used to balance multiple alternate test forms for the
same exam. TIF values should be the same across all alternate forms (Song et al.,
2019).
5. Conclusion and Recommendations

The present article discusses CTT and IRT in ECOL's test development and item
analysis. Educational assessment includes the performance of tests; their results
are used to inform various educational decisions. Tests are therefore widely
regarded as an important part of education. Testing is a method of evaluating a
candidate's ability in a previously defined knowledge or skill domain. To better
understand the relationship between the observed (or actual) score on an
examination and the unobserved proficiency in the domain, we need a test
theory model. CTT and IRT are commonly used models. The CTT calculates
statistics such as correlations among items, covariance’s, difficulties,
discrimination power, reliability coefficients, variance/standard deviation of the
sample, measurement errors, etc., to improve the reliability and validity of
measurement tools. The theory deals with important measurement problems
from a constant perspective. Due to several weaknesses of CTT, the need for
another test theory emerged. These include item and test statistics that differed
401
across tests and groups; a single error estimate was produced for individuals of
all skills levels, and the weakness in test equating. A significant innovation in
educational assessment and psychometrics has been the development of IRT.
Models of IRT have been used extensively in test development and assessment
over the past several decades, attesting to their importance. The IRT models
analyse items, assemble test forms, and equate. Despite being helpful in many
situations, IRT models use strong assumptions and are mathematically more
complex than CTT models used in ECOL. In conclusion, the study strongly
recommends that ECOL shift its test development and item analysis modus
operandi from CTT to modern test theory, which has numerous benefits.
Conflict of interest
The authors declare no conflict of interest.
Acknowledgment
The authors would like to thank the reviewers for their time and efforts in
reviewing the manuscript. The manuscript has been improved by all valuable
comments and suggestions, we are deeply grateful.
6. References
Ackerman, T. A. (2010). The Theory and Practice of Item Response Theory by de Ayala,
R. J. Journal of Educational Measurement, 47(4), 471–476.
https://doi.org/10.1111/j.1745-3984.2010.00124.x
Adedoyin, O. O. (2010). Investigating the Invariance of Person Parameter Estimates
Based on Classical Test and Item Response Theories. International Journal of
Educational Sciences, 2(2), 107–113.
https://doi.org/10.31901/24566322.2010/02.02.07
Adegoke, B. A. (2013). Comparison of item statistics of physics achievement test using
Classical test theory and item response theory frameworks. Journal of Education
and Practice, 22(4), 87–96. www.iiste.org
Adewale, J.G., Adegoke, B.A., Adeleke, J.O. & Metibemu, M. A. (2017). A Training
Manual On Item Response Theory. Institute of Education, University of Ibadan in
Collaboration with National Examinations Council, Minna, Niger State.
Alagoz, C. (2005). Scoring tests with dichotomous and polytomous items.
https://getd.libs.uga.edu/pdfs/alagoz_cigdem_200505_ma.pdf
Algina, J., & Swaminathan, H. (2015). Psychometrics: Classical Test Theory. In
International Encyclopedia of the Social & Behavioral Sciences: Second Edition (pp.
423–430). Elsevier Inc. https://doi.org/10.1016/B978-0-08-097086-8.42070-2
Ayanwale, M.A. (2019). Efficacy of Item Response Theory in the Validation and Score
Ranking of Dichotomous and Polytomous Response Mathematics Achievement
Tests in Osun State, Nigeria. In Doctoral Thesis, Institute of Education, University of
Ibadan (Issue April). https://doi.org/10.13140/RG.2.2.17461.22247
Ayanwale, Musa Adekunle, Adeleke, J. O., & Mamadelo, T. I. (2019). Invariance Person
Estimate of Basic Education Certificate Examination: Classical Test Theory and
Item Response Theory Scoring Perspective. Journal of the International Society for
Teacher Education, 23(1), 18–26. https://files.eric.ed.gov/fulltext/EJ1237578.pdf
Baker, F.B. (2001). The Basics of Item Response Theory. Test Calibration. ERIC Clearinghouse
on Assessment and Evaluation.
Baker, Frank B, & Kim, S. (2017). The Basics of Item Response Theory Using R (S. E. Fienberg
(ed.)). Springer International Publishing. https://doi.org/10.1007/978-3-319-
402
54205-8_1
Behavior, S., Yen, Y. C., Chen, H., & Cheng, M. (2012). The Four-Parameter Logistic Item
Response Theory Model As a Robust Method of Estimating Ability Despite
Aberrant Responses. Social Behavior and Personality: An international
journal, 40(10), 1679-1694.https://doi.org/10.2224/sbp.2012.40.10.1679
Bichi, A. A., Embong, R., Talib, R., Salleh, S., & Bin Ibrahim, A. (2019). Comparative
Analysis of Classical Test Theory and Item Response Theory using Chemistry
Test Data. International Journal of Engineering and Advanced Technology, 8(5), 1260–
1266. https://doi.org/10.35940/ijeat.E1179.0585C19
Bichi, A. A., & Talib, R. (2018). Item Response Theory: An Introduction to Latent Trait
Models to Test and Item Development. International Journal of Evaluation and
Research in Education, 7(2), 142. https://doi.org/10.11591/ijere.v7i2.12900
Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee’s
ability. In: Lord, F.M. and Novick, M.R., Eds., Statistical Theories of Mental Test
Scores, Addison-Wesley, Reading, 397-479.
Bovaird, J. A., & Embretson, S. E. (2012). Modern Measurement in the Social Sciences. In
The SAGE Handbook of Social Research Methods (pp. 268–289). SAGE Publications
Ltd. https://doi.org/10.4135/9781446212165.n16
Brown, J. D. (2013). Classical test theory. In The Routledge Handbook of Language Testing
(pp. 323–335). Springer, Singapore. https://doi.org/10.4324/9780203181287-35
Cai, L., Choi, K., Hansen, M., & Harrell, L. (2016). Item Response Theory. In Annual
Review of Statistics and Its Application (Vol. 3, pp. 297–321). Annual Reviews Inc.
https://doi.org/10.1146/annurev-statistics-041715-033702
Cappelleri, J. C., Jason Lundy, J., & Hays, R. D. (2014). Overview of classical test theory
and item response theory for the quantitative assessment of items in developing
patient-reported outcomes measures. Clinical Therapeutics, 36(5), 648–662.
https://doi.org/10.1016/j.clinthera.2014.04.006
Chen, W. H., & Thissen, D. (1997). Local Dependence Indexes for Item Pairs Using Item
Response Theory. Journal of Educational and Behavioral Statistics, 22(3), 265–289.
https://doi.org/10.3102/10769986022003265
Clark, L. A., & Watson, D. (2019). Constructing validity: New developments in creating
objective measuring instruments. Psychological Assessment, 31(12), 1412–1427.
https://doi.org/10.1037/pas0000626
Cohen, R.J., & Swerdlik, M. E. (2009). Psychological testing and assessment: An introduction
to tests and measurement. (4th ed.). Mayfield Publishing House.
Cohen, R. . J., Swerdlik, M. E., & Sturman, E. (2013). Psychological testing and
assessment : an introduction to tests and measurement. Psychological Assessment,
53(4), 55–67. https://perpus.univpancasila.ac.id/repository/EBUPT181396.pdf
Courville, T. G. (2005). An empirical comparison of item response theory and classical
test theory item/person statistics. Dissertation Abstracts International Section A:
Humanities and Social Sciences, 65(7), 2575.
https://oaktrust.library.tamu.edu/handle/1969.1/1064
Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory. New
York: Harcourt Brace Jovanovich. https://eric.ed.gov/?id=ED312281
De Champlain, A. F. (2010). A primer on classical test theory and item response theory
for assessments in medical education. In Medical Education, 44(1), 109–117.
https://doi.org/10.1111/j.1365-2923.2009.03425.x
Debelak, R., & Koller, I. (2020). Testing the Local Independence Assumption of the Rasch
Model With Q3-Based Nonparametric Model Tests. Applied Psychological
Measurement, 44(2), 103–117. https://doi.org/10.1177/0146621619835501
Demars, C. E. (2017). Classical test theory and item response theory. In The Wiley
403
Handbook of Psychometric Testing: A Multidisciplinary Reference on Survey, Scale and

Test Development, 2(1), 49–73. https://doi.org/10.1002/9781118489772.ch2
Dent, J. A., Harden, R. M., & Hunt, D. (2001). A Practical Guide for Medical Teachers.
Journal of the Royal Society of Medicine, 94(12), 653–653.
https://doi.org/10.1177/014107680109401222
DeVellis, R. F. (2006). Classical test theory. Medical Care, 44(11), 50-59.
https://doi.org/10.1097/01.mlr.0000245426.10853.30
Downing, S. M. (2003). Item response theory: Applications of modern test theory in
medical education. Medical Education, 37(8), 739–745.
https://doi.org/10.1046/j.1365-2923.2003.01587.x
Ebel, R. L. (1965). Book Reviews : Measuring Educational Achievement. Educational and
Psychological Measurement, 25(4), 1167–1169.
https://doi.org/10.1177/001316446502500428
Edelen, M. O., & Reeve, B. B. (2007). Applying item response theory (IRT) modeling to
questionnaire development, evaluation, and refinement. Quality of Life Research,
16(1), 5–18. https://doi.org/10.1007/s11136-007-9198-0
Elgadal, A. H., & Mariod, A. A. (2021). Item Analysis of Multiple-choice Questions
(MCQs): Assessment Tool For Quality Assurance Measures. Sudan Journal of
Medical Sciences, 16(3), 334-346. https://doi.org/10.18502/sjms.v16i3.9695
Embretson, S. E., & Reise, S. P. (2013). Item Response Theory for Psychologists. Lawrence
Erlbaum Associates, Inc., Mahwah. 1–371.
https://doi.org/10.4324/9781410605269
Esmaeeli, B., Shandiz, E. E., Norooziasl, S., & Shojaei, H. (2021). The Optimal Number of
Choices in Multiple-Choice Tests : A Systematic Review. Med Edu Bull, 2(5), 253–
260. https://doi.org/10.22034/MEB.2021.311998.1031
Exam Council of, L. (2018). Establishment of ECOL.
https://www.google.com/search?q=Exam+Council+of+Lesotho%2C+2018&oq
=Exam+Council+of+Lesotho%2C+2018&aqs=chrome..69i57j33i160l2.2034j0j7&so
urceid=chrome&ie=UTF-8
Filgueiras, A., Hora, G., Fioravanti-Bastos, A. C. M., Santana, C. M. T., Pires, P., De
Oliveira Galvão, B., & Landeira-Fernandez, J. (2014). Development and
psychometric properties of a novel depression measure. Temas Em Psicologia,
22(1), 249–269. https://doi.org/10.9788/TP2014.1-19
Finch, H., & Monahan, P. (2008). A bootstrap generalization of modified parallel analysis
for IRT dimensionality assessment. Applied Measurement in Education, 21(2), 119–
140. https://doi.org/10.1080/08957340801926102
Finch, W. H., & French, B. F. (2015). Modeling of Nonrecursive Structural Equation
Models With Categorical Indicators. Structural Equation Modeling, 22(3), 416–428.
https://doi.org/10.1080/10705511.2014.937380
Ganglmair, A., & Lawson, R. (2010). Advantages of Rasch modelling for the
development of a scale to measure affective response to consumption. In E-
European Advances in Consumer Research, 6, 162–167.
https://www.acrwebsite.org/volumes/11738
Gay, L.R, Miles, G. E. & Airasian, P. (2011). Educational Research: Competencies for
Analysis and Applications. 10th Edition, Pearson Education International,
Boston.
González, J., & Wiberg, M. (2017). Applying Test Equating Methods using R. Methodology
of Educational Measurement and Assessment. https://link.springer.com/bfm:978-3-
319-51824-4/1
Hambleton, R.K. and Swaminathan, H. (1985). Item response theory: principles and
applications. p.332. https://doi.org/10.1177/014662168500900315
404
Hambleton, R. K., & Jones, R. W. (1993). Comparison of classical test theory and item
response theory and their applications to test development. Educational
Measurement: Issues and Practice, 12(3), 38–47. https://doi.org/10.1111/j.1745-
3992.1993.tb00543.x
Hill, C., Nel, J. A., van de Vijver, F. J. R., Meiring, D., Valchev, V. H., Adams, B. G., & de
Bruin, G. P. (2013). Developing and testing items for the South African
Personality Inventory. SA Journal of Industrial Psychology, 39(1), 1-13.
https://doi.org/10.4102/sajip.v39i1.1122
Hingorjo, M. R., & Jaleel, F. (2012). Analysis of one-best MCQs: The difficulty index,
discrimination index and distractor efficiency. Journal of the Pakistan Medical
Association, 62(2), 142–147. https://pubmed.ncbi.nlm.nih.gov/22755376/
Immekus, J. C., Snyder, K. E., & Ralston, P. A. (2019). Multidimensional Item Response
Theory for Factor Structure Assessment in Educational Psychology Research.
Frontiers in Education, 4. https://doi.org/10.3389/feduc.2019.00045
IResearchNet (2022). Classical Test Theory.
http://psychology.iresearchnet.com/industrial-organizational-psychology/i-o-
psychology-theories/classical-test-theory/
Jabrayilov, R., Emons, W. H. M., & Sijtsma, K. (2016). Comparison of Classical Test
Theory and Item Response Theory in Individual Change Assessment. Applied
Psychological Measurement, 40(8), 559–572.
https://doi.org/10.1177/0146621616664046
Kelley, T. L. (1939). The selection of upper and lower groups for the validation of test
items. Journal of Educational Psychology, 30(1), 17–24.
https://doi.org/10.1037/h0057123
Khan, H. F., Danish, K. F., Awan, A. S., & Anwar, M. (2013). Identification of technical
item flaws leads to improvement of the quality of single best multiple choice
questions. Pakistan Journal of Medical Sciences, 29(3), 715.
https://doi.org/10.12669/pjms.293.2993
Kim, D., de Ayala, R. J., Ferdous, A. A., & Nering, M. L. (2011). The comparative
performance of conditional independence indices. Applied Psychological
Measurement, 35(6), 447–471. https://doi.org/10.1177/0146621611407909
Kline, R. B. (2005). “Principles and practice of structural equation modelling ”. ((2nd ed.)). The
Guilford Press.
Kline, T. (2014). Classical Test Theory: Assumptions, Equations, Limitations, and Item
Analyses. In Psychological Testing: A Practical Approach to Design and Evaluation,
23(2), 91–106. https://doi.org/10.4135/9781483385693.n5
Kolen, M. J. (1981). Comparison of traditional and Item Response Theory methods for
equatingTests. Journal of Educational Measurement, 18(1), 1–11.
https://doi.org/10.1111/j.1745-3984.1981.tb00838.x
Krishnan, V. (2013). The Early Child Development Instrument ( EDI ): An item analysis
using Classical Test Theory ( CTT ) on Alberta ’ s data. Early Child Development
Mapping (ECMap) Project Community-University Partnership (CUP) Faculty of
Extension, University of Alberta.
Lang, J. W. B., & Tay, L. (2021). The Science and Practice of Item Response Theory in
Organizations. In Annual Review of Organizational Psychology and Organizational
Behavior, 8, 311–338. https://doi.org/10.1146/annurev-orgpsych-012420-061705
Lee, W., & Ansley, T. N. (2007). Assessing IRT Model-Data Fit for mixed format tests.
Journal of Applied Psychology, 92(2), 23–50. http://dx.doi.org/10.1026/apl0000636
Lord, F. M. (2012). Applications of item response theory to practical testing problems. In
Applications of Item Response Theory To Practical Testing Problems.
https://doi.org/10.4324/9780203056615
405
Magis, D. (2007). Influence, Information and Item Response Theory in Discrete Data Analysis.
Retrieved on 12 June, 2022 from
http://bictel.ulg.ac.be/ETDdb/collection/available/ULgetd-06122007-100147/.
Mona, N. (2014). Application of Classical Test Theory and Item Response Theory to
Analyze Multiple Choice Questions (Unpublished doctoral thesis). University of
Calgary, Calgary, AB. doi:10.11575/PRISM/24958
Nataranjan, V. (2009). Basic Principle of Item Response Theory and Application to Practical
Testing and Assessement. Merit Trac Services Publishing Ltd.
Ojerinde, D. & Ifewulu, B. C. (2012). Item Unidimensionality Using 2010 Unified Tertiary
Matriculation Examination Mathematics Pre-test. A Paper Presented at the 2012
International Conference of IAEA, 5–18.
Pliakos, K., Joo, S. H., Park, J. Y., Cornillie, F., Vens, C., & Van den Noortgate, W. (2019).
Integrating machine learning into item response theory for addressing the cold
start problem in adaptive learning systems. Computers and Education, 137, 91–103.
Preston, R., Gratani, M., Owens, K., Roche, P., Zimanyi, M., & Malau-Aduli, B. (2020).
Exploring the Impact of Assessment on Medical Students’ Learning. Assessment
and Evaluation in Higher Education, 45(1), 109–124.
https://doi.org/10.1080/02602938.2019.1614145
Privitera, G. J. (2012). Statistics for the behavioral sciences. Sage Publications, Inc.
https://psycnet.apa.org/record/2011-21294-000
Reckase, M. D. (2009). Multidimensional Item ResponseTheory. Springer Verlag.
Reise, S. P. (1990). A Comparison of Item- and Person-Fit Methods of Assessing Model-
Data Fit in IRT. Applied Psychological Measurement, 14(2), 127–137.
https://doi.org/10.1177/014662169001400202
Reise, S. P., & Waller, N. G. (2009). Item response theory and clinical measurement.
Annual Review of Clinical Psychology, 5, 27–48.
https://doi.org/10.1146/Annurev.Clinpsy.032408.153553
Rupp, A. A. (2003). Item Response Modeling With BILOG-MG and MULTILOG for
Windows. International Journal of Testing, 3(4), 365–384.
https://doi.org/10.1207/s15327574ijt0304_5
Rusch, T., Lowry, P. B., Mair, P., & Treiblmaier, H. (2017). Breaking free from the
limitations of classical test theory: Developing and measuring information
systems scales using item response theory. Information and Management, 54(2),
189–203. https://doi.org/10.1016/j.im.2016.06.005
Sim, S. M., & Rasiah, R. I. (2006). Relationship between item difficulty and discrimination
indices in true/false-type multiple choice questions of a para-clinical
multidisciplinary paper. Annals of the Academy of Medicine Singapore, 35(2), 67–71.
http://www.ams.edu.sg
Song, Y., Kim, H., & Park, S. Y. (2019). An Item Response Theory Analysis of the Korean
Version of the CRAFFT Scale for Alcohol Use Among Adolescents in Korea.
Asian Nursing Research, 13(4), 249–256. https://doi.org/10.1016/j.anr.2019.09.003
Steyer, R. (2001). Classical (Psychometric) Test Theory. International Encyclopedia of the
Social & Behavioral Sciences, 1955–1962. https://doi.org/10.1016/B0-08-043076-
7/00721-X
Tang, X., Karabatsos, G., & Chen, H. (2020). Detecting Local Dependence: A Threshold-
Autoregressive Item Response Theory (TAR-IRT) Approach for Polytomous
Items. Applied Measurement in Education, 280–292.
https://doi.org/10.1080/08957347.2020.1789136
Tay, L., Meade, A. W., & Cao, M. (2015). An Overview and Practical Guide to IRT
Measurement Equivalence Analysis. Organizational Research Methods, 18(1), 3–46.
406
https://doi.org/10.1177/1094428114553062
Toksöz, S., & Ertunç, A. (2017). Item Analysis of a Multiple-Choice Exam. Advances in
Language and Literary Studies, 8(6), 141.
Traub, R. E. (2015). Classical test theory in historical perspective. Journal of Educational
Measurement: Issues and Practice, 16(4), 8–14.
https://doi.org/10.1111/emip.2015.16.issue-4
Tuerlinckx, F., Rijmen, F., Molenberghs, G., Verbeke, G., Briggs, D., Van den Noortgate,
W., Meulders, M., & De Boeck, P. (2004). Estimation and software. In Explanatory
Item Response Models, 6, 343–373. https://doi.org/10.1007/978-1-4757-3990-9_12
Vyas, R., & Supe, A. (2008). Multiple choice questions: A literature review on the optimal
number of options. In National Medical Journal of India, 21(3), 130–133.
https://pubmed.ncbi.nlm.nih.gov/19004145/
Wells, C. S., & Wollack, J. A. (2018). An Instructor’s Guide to Understanding Test
Reliability. Testing and Evaluation Services, 1–7.
https://testing.wisc.edu/Reliability.pdf
Yen, W. M. (1993). Scaling Performance Assessments: Strategies for Managing Local Item
Dependence. Journal of Educational Measurement, 30(3), 187–213.
https://doi.org/10.1111/j.1745-3984.1993.tb00423.x
Yu, C. H., Popp, S. O., Digangi, S., & Jannasch-Pennell, A. (2007). Assessing
unidimensionality: A comparison of Rasch modeling, Parallel analysis, and
TETRAD. Practical Assessment, Research and Evaluation, 12(14), 1–19.
https://doi.org/https://doi.org/10.7275/q7g0-vt50
Zhang, J. (2012). Calibration of Response Data Using MIRT Models With Simple and
Mixed Structures. Applied Psychological Measurement, 36(5), 375–398.
https://doi.org/10.1177/0146621612445904
Zhu, X., & Lu, C. (2017). Re-evaluation of the New Ecological Paradigm scale using item
response theory. Journal of Environmental Psychology, 54, 79–90.
https://doi.org/10.1016/j.jenvp.2017.10.005
407

Vol. 21, No. 8, pp. 407-424, August 2022
Addressing the Issues in Democratic Civilian

Control in Ukraine through Updating the
Refresher Course for Civil Servants
Valentyna I. Bobrytska*
National Pedagogical Dragomanov University, Kyiv, Ukraine
Leonid V. Bobrytskyi
Donetsk State University of Internal Affairs, Ukraine
Andriy L. Bobrytskyi
Main Directorate of the National Police in Poltava Region, Ukraine
Svitlana M. Protska
National Pedagogical Dragomanov University, Kyiv, Ukraine
Abstract. Finding ways of implementing civilian control in the activity

of the armed forces and law enforcement agencies has been a challenge
in Ukraine. The purpose of this study was to identify the international
best practices to establish relationships between the civil institutions and
the military so that those could be adopted in Ukraine. It also sought to
update the refresher course for civil servants by introducing two
modules that are based on the shortlisted best practices. We used case
study research based on the exploratory research strategy and a pilot
study to evaluate the feasibility of the updated refresher course for civil
servants. The course was delivered offline to 358 civil servants in 2019
and 246 in 2020, and online to 119 in 2021. Forty-five randomly selected
students who had completed the course responded to a course
satisfaction survey. It was revealed that the process of building civil-
military relationships in Ukraine was in the transitional stage. In
addition, there was a need for educating and training civil servants in
developing legal, regulatory, and procedural frameworks for building
civil-military relations and exercising civilian control over the military
and law enforcement agencies. The novelty of the study is in updating
the curriculum of a refresher course for civil servants and providing
recommendations to the government and the Ministry of Defense of
Ukraine. Further research is needed in identifying how the public media
could contribute to educating citizens on civilian control of the military
and law enforcement bodies.
*
Corresponding author: Valentyna I. Bobrytska, bobrytska@ukr.net
©Authors
408
Keywords: civil-military relationships; democratic civilian control;

educational measures; military and law enforcement agencies; Ukraine
1. Introduction
The implementation of democratic civilian control in the activity of the armed
forces and law enforcement agencies has been one of the crucial challenges that
Ukraine, seen as an emerging democracy or democracy of the “third wave”, faces
on the way to transition from a military-dominated post-soviet authoritarian
regime to a democratic government (Koziy & Polyakov, 2018; Zadubinny, 2020).
The establishment of this kind of control in Ukraine is increasingly accelerated by
the international commitments related to the integration of Ukraine into the
European Union and the North Atlantic Treaty Organization (NATO) (Gusev et
al., 2018). The above implies that there is a need for developing mechanisms to
fulfill the commitments. Using educational measures to address the problems
related to establishing cooperation between civilian institutions and the military
to allow the former to control the activity of the armed forces has been proven to
be a feasible option (Kuehn, 2016; Tagarev, 1997). Although there is literature on
the evolution of civilian control and civil-military relations in post-soviet Western
European and Middle East countries, the Ukrainian case is underrepresented in
research (Demir & Bingöl, 2020; Kuehn et al., 2017).
Furthermore, there is still a gap in research and a loophole in governmental and

educational policies regarding the design and implementation of instructional
models to address the issues related to involving civilian institutions in the control
of the military. Thus, this research aims to identify the international best practices
to establish relationships between civil institutions and the military so that those
could be adopted in Ukraine. It also seeks to update the refresher course for civil
servants by introducing two modules that are based on the shortlisted best
practices. Furthermore, the research seeks to identify whether there is an
observable influence of that course on the civil students’ awareness of addressing
the inconsistencies that occur in implementing civilian control in the activity of
the organizations of the national defense sector and law enforcement system in
Ukraine. The research questions were as follows:
1. What does the relevant literature reveal about civilian control of the military
in emerging democracy and educational measures for building civilian-
military relationships?
2. How did the participants of the professional refresher course perceive the
introduced modules and a project assignment.
Literature has indicated that civilian control over the military and law
enforcement agencies involves institutional and structural perspectives (Koimaru,
2016). Koimaru (2016) referred to the institutional perspective as the interaction
between the representatives of military and civilian officers within one institution.
Contrarily, the structural perspective focuses on the objective civilian control,
which is seen as the voluntary subordination of the military and law enforcement
agencies to civilian institutions to avoid the military from taking over (usurping)
409
power and civilian order. According to Kuehn (2018), democratic civilian control
is supposed to be performed by three actors, namely government, society, and
armed forces (law enforcement agencies). The government adopts the law, shapes
security policy, and exercises law-based protection. Society influences the armed
forces and law enforcement agencies through educating the citizens, financial
supervision, and public debates. The armed forces ensure self-control to comply
with the legislation and protection of the rights of the citizens. Importantly,
educational measures are supposed to incorporate a new security culture in both
civilian and military communities (Geneva Centre for the Democratic Control of
Armed Forces [DCAF], 2008). These measures are expected to promote
democratic values, human rights, and international humanitarian law to the
society members, mainly focusing on civil-military interaction and better
integration of military and law enforcement agencies within society. Consistent
with Croissant et al. (2013), educational measures can be the causal mechanism
that is currently necessary for the authorities and society to apply civilian control
strategies, thus establishing the status quo. It will be a starting point for building
a strong civil society, consolidating the institutions, implementing democratic
reforms, and dealing with internal and external security threats.
3. Research Methods
We used the case study research design based on the exploratory research strategy
to answer the first research question. We also used a pilot study for the second
research question to evaluate the feasibility of the updated refresher course for
civil servants. Qualitative research methods were mainly used in the pre-
intervention phase of the study, whereas quantitative ones were utilized in the
intervention phase. The pre-intervention phase relied on the non-systematic
review of scientific and non-scientific texts; “gray literature” texts such as national
newspaper articles, blog posts, and media content in Ukraine; and a survey
questionnaire (Cook, 2019). Local and national media coverage were analyzed to
specify the patterns of civilian control in Ukraine. Descriptive content analysis
was also used to examine and evaluate the retrieved texts. The corpus of literature
was analyzed using software for quantitative text analysis using certain software
tools. First, Voyant Tools was employed to specify the keywords in the shortlisted
texts and correlations (Sinclair & Rockwell, 2020). Second, the Linguistic Inquiry
and Word Count (LIWC) application was utilized to identify the emotional tone
of the texts (Pennebaker Conglomerates, 2020). The qualitative method was used
to answer the research question focused on identifying the instructional models
aimed at training people in building civilian-military relationships. The same
method was utilized to analyze the feedback obtained from the participants of the
professional refresher course updated to address the questions of exercising
civilian control of the activity of the organizations of the national defense sector
and law enforcement system in Ukraine. The course satisfaction survey
questionnaire was utilized to identify how the participants of the professional
refresher course perceived the introduced modules.
The study was cooperatively conducted by the Department of Social Philosophy,

the Philosophy of Education and Educational Policy for National Pedagogical
Dragomanov University (Kyiv, Ukraine) and the Department of Criminal Law
410
and Criminology, Faculty №1 for the Donetsk State University of Internal Affairs
(Kropyvnytskyi, Ukraine). The Training Institute of the State Employment Service
of Ukraine was the host organization to run the updated refresher course for civil
servants. Its objective was to examine and collect evidence and cases to design and
pilot the refresher course that trains the civil servants in dealing with the problems
in civil society control of the army and police. The research methodology and
materials were approved at a meeting with the above departments before the
intervention stage. Five external experts in sociology, civil-military relations,
behavioral psychology, and education were involved in analyzing the results of
the survey questionnaire. The purpose of involving experts was to avoid
researcher bias (Almeida & Goulart, 2017).
3.1 Brief Description of the Procedure of the Non-Systematic Review

The procedure of the non-systematic review relied on the keyword combinations
as recommended by Bethel and Rogers (2018). This review attempted to shortlist
and evaluate relevant literature publications through their description. It focused
on what has been discussed in scientific and non-scientific texts, such as local and
national media reporting on cases or changes in participating civilian institutions
in exercising control or supervising the activity of the armed forces and the police.
The review sought to determine how the practical issues of implementing the
control or supervision were addressed. The keywords were linked to broad or
generic subjects that revealed cases or practices of exercising or training people to
perform the civil control of the armed forces and police in Ukraine. The keywords
were grouped into two domains, namely “civilian control of the military in
emerging democracy” and “educational measures for building civilian-military
relationships”. The search strings were in English, Russian, and Ukrainian. The
web search was performed using different databases, archives, and websites, such
as SCOPUS, Google Scholar, CrossRef, Supreme Council of Ukraine (Verkhovna
Rada), OSCE archive, RAZUMKOV CENTER archive, European Truth (internet
newspaper), and Deutsche Welle (DW). This also included the YouTube channel
for the “UA|TV” International Broadcasting Multimedia Platform of Ukraine and
the YouTube channel for the “Breeze TV and Radio Studio” of the Ministry of
Defence of Ukraine. The search strings used in this phase of the study are
presented in Table 1.
Table 1: Strings used in the search

Type of search Keyword-based string
Google Search TI and/or TW = (civilian control of military in emerg* democrac* OR
in Ukraine OR in post-soviet countries OR in former authoritative
regimes OR implementat* of civilian control of military in emerg*
democrac* OR building civilian-military relationships OR
educational measures OR initiatives for building civilian-military
relationships OR educational measures for building civilian-military
relationships in emerg* democrac* OR in Ukraine OR in post-soviet
countries OR in former authoritative regimes)
SCOPUS AB and/or KW and/or ID (TITLE-ABS-KEY) = (civilian control of
military in emerg* democrac* OR in Ukraine OR in post-soviet
countries OR in former authoritative regimes OR implementat* of
civilian control of military in emerg* democrac* OR building civilian-
411
military relationships OR educational measures for building civilian-

military relationships in emerg* democrac* OR in Ukraine OR in
post-soviet countries OR in former authoritative regimes)
WoS AB and/or KW and/or ID (TITLE-ABS-KEY) = (civilian control of
categories: military in emerg* democrac* OR in Ukraine OR in post-soviet
social sciences countries OR in former authoritative regimes OR implementat* of
civilian control of military in emerg* democrac* OR building civilian-
military relationships in emerg* democrac* OR in Ukraine OR in
post-soviet countries OR in former authoritative regimes)
(subcategory: Education and Educational Research) AND (Articles
OR Reviews), Indexes: Social Science Citation Index – 1988-present)
Note.: TI = title; TW = text words; AB = abstract; KW = keywords; TITLE-ABS-KEY =
title-abstract-keywords
The search and retrieval of the relevant texts lasted almost half a year, from
September 2020 to the end of January 2021. The goal in the search and retrieval
phase was not to develop an exhaustive search of all the texts that address our
research questions. It was to reveal the general trends in the current state of
scientific research, media coverage, and practice of promoting the civilian
supervision of the army and police in Ukraine through building the civil-military
relationships. The inclusion and exclusion criteria for selecting relevant texts were
as follows: 1) the period when the text was published or released (over the last 30
years); 2) regional relevance (Ukraine); 3) conceptual or analytical or propositive
narration; and 4) educational purpose of the initiative to have been revealed. The
relevance, quality, and validity of the texts were assessed using a critical appraisal
checklist (CAC) that was purposefully adopted from Bobrytska et al. (2020) and
Greenhalgh et al. (2005). The instrument relied on the five-grade scale with values
for judgments that could range from 1 = low relevance to 5 = high relevance. The
assessment of the texts was performed by two colleague volunteers. Following
this, the researchers and colleagues held a consensus meeting to settle any
disagreements through discussion. The number of text publications that were
found for each keyword-based inquiry is presented in Table 2.
The non-systematic review included four phases, as revealed by Moher et al.

(2015). These were: 1) identification, 2) screening, 3) identifying eligibility, and
4) inclusion for the review. The first phase focused on the evolution of the number
of publications in scientific and non-scientific sources and public media covering
the context of Ukrainian or post-soviet countries or former authoritative regimes.
In the screening phase, the emotional tone of narration, key ideas and concepts,
and message-framing type (gain framing and loss framing) were identified.
Following this, the texts were selected using the inclusion and exclusion criteria.
Finally, the selected texts were evaluated by experts for relevance, quality, and
validity using the CAC. In the screening phase, the randomly selected texts, from
different periods, were summarized and translated by hired student volunteers
(n = 55) who were enrolled in philology and translation studies at the National
Pedagogical Dragomanov University (NPDU). The texts shortlisted for the final
review were also translated by the NPDU student volunteers.
412
Table 2: Keyword combinations and number of texts

n of
ntotal of
Keyword combination shortlisted
texts
texts
civilian control of the military in emerg* democrac* 791 219
civilian control of military in Ukraine 391 89
civilian control of military in post-soviet countries OR in 886 137
former authoritative regimes
implementat* of civilian control of military in emerg* 338 39
democrac*
building civilian-military relationships 279 43
educational measures for building civilian-military 148 11
relationships
educational measures OR initiatives for building civilian- 175 21
military relationships in emerg* democrac* OR in Ukraine
OR in post-soviet countries OR in former authoritative
regimes
military relationships in Ukraine OR in post-soviet
countries OR in former authoritative regimes
military relationships in post-soviet countries OR in
former authoritative regimes
3.2 Description of the Modules Introduced in the Refresher Course for Civil
Servants
Two modules were introduced in a professional refresher course for the civil
servants who did the course at the Training Institute of the State Employment
Service of Ukraine in 2020 and 2021. These modules were aimed at educating civil
servants in defense and security areas to be aware of building the partnership
between civilians and the military. Furthermore, the structure of the refresher
course was updated using the curriculum for the educational program titled
“State Educational Policy”, run at NPDU, as the best practice (Vashkevich et al.,
2020). The modules were based on best practices found in the reviewed
publications.
The two modules covered the following eight topics:

• The concept of civilian control.
• The concept of democratic control of armed forces.
• International best practices and prerequisites that govern the relationship
between the society and armed forces along with the police.
• The use of best practices and prerequisites of social control as approaches
to building nationwide confidence.
• Civilian control strategies.
• Setting up the epistemic communities or expertise-based networks of
professionals or former military officers engaged in security sector reform.
• Subordination of military and police to the political institutions that are
supposed to perform supervision and whose members were chosen by
democratic election.
• Measuring the extent of democratic civilian control.
413
The course included a project assignment prompting students to develop

recommendations to the government, the Ministry of Defence of Ukraine, and
public media on how to build and exercise democratic civilian control over the
armed forces and law enforcement agencies, and how to build the relationship
between them.
We used the convenience sampling technique. The updated refresher course was
delivered offline to 358 civil servants in 2019 and 246 in 2020, and online to 119 in
2021. Forty-five randomly selected students who had completed the course
responded to the course satisfaction survey.
3.3 Ethical considerations

Since ethical inconsistencies could occur in the refresher course assignment and
survey, the participants were asked for informed consent before the intervention.
The anonymity and privacy of the data provided by the participants were given
paramount importance.
3.4 Instruments
The CAC (see Appendix A), Voyant Tools, LIWC application, and the course
satisfaction survey questionnaire were the instruments used for the study. The
numeric data were calculated using jamovi statistical software (version 2.0.0)
(Jamovi project, 2021).
The CAC was validated by Bobrytska et al. (2020). The item content validity index
(I-CVI) was < 0.85. The Fleiss’ k coefficient was < 0.83. The value for the scale-level
content validity index (SL-CVI) was 92%. The above values implied that the CAC
was a valid tool and that it could be used in the study.
The Voyant Tools software is open-source software and is designed to analyze the
text corpus in terms of patterns of words/concepts (Sinclair & Rockwell, 2015). It
is increasingly used for research purposes and has been used in research
published in peer-reviewed journals (Boyle & Hall, 2016). The LIWC software is
also used to analyze the text corpus. It uses a 100-point scale and reveals the
analytical or formal thinking in the narration style, clout taps writing, text
authenticity, and emotional tone.
The course satisfaction survey questionnaire (see Appendix B) relied on two

scales, course effectiveness and course satisfaction, and consisted of five
questions. It used a 5-point Likert scale for effectiveness and satisfaction (1 = very
ineffective or very dissatisfied up to 5 = very effective or very satisfied). The participants
had to express their judgments concerning the course design, assessment system,
and the intellectual gains that occurred in them due to participation in the course.
The questionnaire was administered online and the output data were processed
manually. Three experts were hired to assess the inter-rater reliability of the
questionnaire following the recommendation of Rodrigues et al. (2017). Before
using the questionnaire, reliability analysis was performed. The Cronbach’s 𝛼 for
the entire questionnaire was .883, which is considered a sufficient value for
internal consistency (Cortina, 1993).
414
4. Results
To address the research questions, the results of the study are discussed in two
sections, that of the non-systematic review and course satisfaction survey.
4.1 Results of the Non-Systematic Review

The review showed that the number of relevant texts has increased dramatically
since 2014, which can be associated with Russian military aggression against
Ukraine. Table 3 and Figure 1 illustrate the increase of the overall number of texts
by source in the context of Ukrainian or post-soviet countries or former
authoritative regimes.
As can be noticed in Table 3 and Figure 1, the public media in Ukraine produced
a greater volume of literature on the issues of establishing civilian control of the
military through building civilian-military relationships compared to scientific
and non-scientific military sources. This implies that society tried to influence the
government to accelerate the process of resolving these issues through legislative
and executive measures and bodies.
Table 3: Publications on the civilian control of the military in emerging democracy

and educational measures for building civilian-military relationships distributed by
year/s and source
Year/s 91- 96- 01- 06- 11-

15 16 17 18 19 20 % Mean SD
95 00 05 10 14
Source
SCOPUS 0 0 0 0 3 4 7 11 13 19 22 1.81 7.18 7.63
GS 0 0 1 4 3 6 9 17 23 30 56 3.41 13.54 16.45
CrossRef 0 0 0 0 0 0 0 28 28 117 139 7.19 28.36 48.36
SCU 1 2 6 9 5 38 51 68 91 112 146 12.2 48.09 48.16
OSCE 0 0 7 6 5 27 33 42 59 83 176 10.1 39.81 50.08
MDU 1 3 16 9 7 21 39 48 56 63 84 8 31.54 26.75
RC 0 0 3 4 8 9 17 35 41 75 94 6.59 26 30.73
ET 0 0 0 0 0 4 21 61 154 181 194 14.19 55.9 76.22
DW 0 4 8 16 7 15 66 73 79 127 296 15.95 62.81 83.59
UA|TV 0 0 0 0 0 2 38 75 118 125 147 11.66 45.9 56.44
Breeze
0 0 3 9 4 8 22 34 67 93 143 8.84 34.81 44.78
TV
Note. GS = Google Scholar; SCU = Supreme Council of Ukraine; MDU = Ministry of

Defence of Ukraine; RC = Razumkov Center; ET = European Truth; Breeze TV = “Breeze
TV and Radio Studio”
415
1497
1025
662
492
303
134
2 9 44 57 42
Figure 1: Evolution of the number of publications in scientific and non-scientific

sources and public media
In the screening phase, based on the analysis of 104 randomly selected texts from
different periods (that were summarized and translated by hired student
volunteers), the emotional tone of the texts and the key implications to be drawn
from them were identified. Table 4 presents the keywords used in the texts and
correlations of terms analysis that were identified using Voyant Tools.
Table 4: Keywords used in the texts and correlations identified using Voyant Tools
Before the Russian military
During the Russian military aggression
aggression
Keywords 𝒓 Keywords 𝒓
civilians strategies 0.984 reform Ministry of 0.971
Defence
civilians consolidation 0.951 Ministry of Parliament 0.963
Defence
analysis attempts 0.917 outside control military 0.929
society prerogatives 0.893 approach public 0.884
society powers 0.865 education restructuring 0.868
institutions new 0.838 education providing 0.827
civilians military 0.639 data public 0.797
civilian- relations 0.611 defense transparent 0.788
military
civilian- security 0.584 information media 0.757
military
As can be seen in Table 4, the keywords used in texts were different in the periods
before and during the Russian military aggression against Ukraine. In the period
before the aggression, the literature revealed this topic from the perspective of
analysis, giving power to the society and consolidating the society, which is
illustrated by high correlation values (𝑟 > 0.7). Interestingly, the correlation
between terms that implied building civil-military relationships was lower than
0.7, signifying the low importance of this matter. Additionally, it was found that
texts used tentative and hesitant language with verbs such as “seem”, “could”,
“might”, and “would be”. This proved that establishing civilian control of the
416
military and building civilian-military relationships was not a top priority task in
Ukraine in the period before the Russian aggression. The keywords found in
publications in the period between 2015 and the present illustrated the growing
importance of these tasks for the Ministry of Defence of Ukraine, public
government, and society. This was proved by the high correlation between terms
such as reform-Ministry of Defence, Ministry of Defence-Parliament, outside
control-military, and approach-public. It was important that educational
measures related to civilian control of the military were found one of the priorities
in the literature. Additionally, the language in the literature used more call-to-
action verbs, such as “must focus”, “should do”, and “need to be”.
Table 5 presents the results of analyzing the texts for the emotional tone using the
LIWC app. The texts were divided into two periods, namely before the Russian
military aggression against Ukraine and during it. The values in Table 5 suggest
that, when comparing the two periods, the narration style of publications became
less analytic (BA = 95.61; DA = 74.83) and confidence and leadership projecting
increased, as can be seen in the values for the clout variable (BA = 53.81; DA =
83.29). In addition, honesty and negativity increased (BA = 35.89; DA = 64.98). The
values for the emotional tone (BA = 53.92; DA = 43.81) suggested the growth of
pessimism in the narration style of publications.
Table 5: Results of the analysis of the emotional tone of the corpus of texts
Variable BA DA Reference value
Analytic 95.61 74.83 56.34
Clout 53.81 83.29 57.95
Authenticity 35.89 64.98 49.17
Emotional tone 53.92 43.81 54.22
Note. BA = before the Russian military aggression; DA = during the Russian military
aggression
The above values for characteristics of the narration style of publications indicated
the importance of finding a solution to address the issue. In the final review phase,
seven texts were selected by the experts using the CAC. These are outlined in
Table 6.
As can be drawn from Table 6, the key suggestions are related to educating and
training civil servants in developing legal, regulatory, and procedural
frameworks for building civil-military relations and exercising civilian control
over them. Some authors suggested reorganizing and restructuring the military
to join NATO. However, the review did not yield a precise description of any
educational measure that could be adopted.
417
Table 6: Results of the review of the selected texts

Regional Conceptual/ Mean
Author(s), Key idea/
relevance analytical/ CAC
year of text Educational measure revealed
(Ukraine) propositive score
Improvement of defense
management and planning
Koziy and
Conceptual/ through educating civil
Polyakov Relevant 3.5
analytical servants and implementing
(2018)
transparent and accountable
democratic systems
Updating and implementing
education systems and career
Gusev et al. Conceptual/
Relevant programs that comply with 3
(2018) analytical
civilian and military
democratic systems
Upgrading the education of
civil servants and encouraging
them for the continued
Bobritsky Analytical/ development of legal,
Relevant 4
(2020) propositive regulatory, and procedural
frameworks with a focus on
the roles and responsibilities of
the civilians and military
Increasing the degree of
Fluri and institutionalized civilian
Analytical/
Badrack Relevant oversight over military affairs, 3
propositive
(2016a) training civil servants to
exercise this control
Involving non-governmental
experts, activists, and
professionals from civil society
Fluri and
Analytical/ in the discussion of important
Badrack Relevant 3.5
propositive documents and concepts on
(2016b)
the development of legal,
regulatory, and procedural
frameworks
Establishing the balance
between the outside control of
Ukrainian
Analytical/ the military and their
military TV Relevant 3
propositive autonomy. Training officers in
(2016)
building relations with a civil
community.
Reforming the military to
Conceptual/ reduce its force and
Perepelitsa
Relevant analytical/ restructuring legislation and 3.5
(2002)
propositive governance with priorities on
joining NATO
The lecturers’ observations showed that the introduction of two modules that
were based on the shortlisted best practices in the professional refresher course
for the civil servants brought a complimentary change in the civil students’
awareness. This involved addressing the inconsistencies that occur in
418
implementing civilian control in the activity of the organizations of the national

defense sector and law enforcement system in Ukraine.
As a project assignment, the students were supposed to develop

recommendations to the government, the Ministry of Defence of Ukraine, and
public media on how to build and exercise democratic civilian control over the
armed forces and law enforcement agencies and how to build a relationship
between them. Below are the topics covered:
• Increase the share of civilians in the management of the armed forces.
• Non-involvement of the military in dealing with civil pickets or incidents
of civil unrest.
• Reshaping or updating legislation to comply with NATO requirements.
• Using a bottom-up approach (citizens – societies – state) to the
management of national security of Ukraine.
• In terms of its armed defense, the Constitution should be applied through
direct democracy in its entirety at the basic management levels.
• The formation and implementation of a general defense policy should be
a part of national security policy (direct use of the power of the people
[community and individual citizens]) due to the need to put up armed
resistance to the aggressor to ensure state sovereignty and territorial
integrity of the country.
4.2 Results of the Course Satisfaction Survey

The survey results suggest that participants perceived the updated course and
assignment positively. Table 7 presents the descriptive statistics of the course
satisfaction survey.
Table 7: Summary of the data drawn from the student satisfaction survey (N = 45)
Q1e Q2e Q3e Q4e Q5e Q1s Q2s Q3s Q4s Q5s
Mean 4.22 4.11 4.09 4.04 4.02 4.29 4 4.11 4.02 4.2
SD 0.85 0.775 0.848 1.07 0.988 0.757 0.769 0.859 0.753 0.786
Skewness 0.918 0.504 0.642 0.682 0.637 0.545 0.314 0.446 0.370 0.963
Std. error
0.354 0.354 0.354 0.354 0.354 0.354 0.354 0.354 0.354 0.354
skewness
Kurtosis 0.256 0.229 0.176 0.876 0.672 1.04 0.381 0.923 1.2 1.02
Std. error
0.695 0.695 0.695 0.695 0.695 0.695 0.695 0.695 0.695 0.695
kurtosis
As displayed in Table 7, the values for skewness and kurtosis were skewed right,
implying that the judgments of the participants were positive about the course’s
effectiveness (NIST/SEMATECH, 2013). The mean values suggest that the
participants were satisfied with the refresher course design, content, assessment
system, and intellectual gains it provided them.
5. Discussion
The study sought to identify the international best practices to establish
relationships between the civil institutions and the military so that those could be
adopted in Ukraine. It also sought to update the refresher course for the civil
419
servants by introducing two modules that were based on the shortlisted best
practices. Furthermore, it wanted to identify whether there is an observable
influence of the course on the civil students’ awareness of addressing the
inconsistencies that occur in implementing civilian control in the activity of the
organizations of the national defense sector and law enforcement system in
Ukraine. The above goals were addressed via answering the following two
research questions: 1) What does the relevant literature reveal about civilian
control of the military in emerging democracy and educational measures for
building civilian-military relationships? and: 2) How did the participants of the
professional refresher course perceive the introduced modules and a project
assignment? The novelty of the study is in updating the curriculum of a refresher
course for civil servants and producing recommendations to the government, the
Ministry of Defence of Ukraine, and public media on how to build and exercise
democratic civilian control over the armed forces and law enforcement agencies.
In addition, the study elicited information on how to build the relationship
between civilian government authorities and the armed forces as the outcome of
the refresher course.
The findings revealed that the number of relevant texts has increased dramatically
since 2014, which can be associated with Russian military aggression against
Ukraine. The public media in Ukraine produced a greater volume of literature on
the issues of establishing civilian control of the military through building civilian-
military relationships compared to scientific and non-scientific military sources.
This implies that society tried to influence the government to accelerate the
process of resolving these issues through legislative and executive measures and
bodies. The keywords used in texts were different in the periods before and
during the Russian military aggression against Ukraine. In the period before the
aggression, the literature revealed this topic from the perspective of analysis,
giving power to the society and consolidating the society, which is illustrated by
high correlation values (𝑟 > 0.7). Interestingly, the correlation between terms that
implied building civil-military relationships was lower than 0.7, indicating the
low importance of this matter. Additionally, it was found that publications used
tentative and hesitant language with verbs such as “seem”, “could”, “might”, and
“would be”. This proved that establishing civilian control of the military and
building civilian-military relationships was not a top priority task in Ukraine in
the period before the Russian aggression.
The keywords found in texts in the period between 2015 and the present
illustrated the growing importance of these tasks for the Ministry of Defence of
Ukraine, public government, and society. This was proved by the high correlation
between terms such as reform-Ministry of Defence, Ministry of Defence-
Parliament, outside control-military, and approach-public. It was important that
educational measures related to civilian control of the military were found one of
the priorities in the literature. Additionally, the language in the literature used
more call-to-action verbs, such as “must focus”, “should do”, and “need to be”.
The narration style of publications, when comparing the two periods (BA and
DA), became less analytic (BA = 95.61; DA = 74.83) and confidence and leadership
projecting increased, as can be seen in the values for the clout variable (BA = 53.81;
420
DA = 83.29). In addition, honesty and negativity increased (BA = 35.89; DA =

64.98). The values for the emotional tone (BA = 53.92; DA = 43.81) suggested the
growth of pessimism in the narration style of texts. The above values for
characteristics of the narration style of publications indicated the importance of
finding a solution to address the issue.
The final phase of the literature review found that the key suggestions were
related to educating and training civil servants in developing legal, regulatory,
and procedural frameworks for building civil-military relations and exercising
civilian control over them. Some authors suggested reorganizing and
restructuring the military to join NATO. However, the review did not yield a
precise description of any educational measure that could be adopted. The results
of the above literature review agree with Kenwick (2020) and Gonzalez (2022),
who concluded that military institutions should be controlled by giving access to
decision-making in the military to avoid the risk of losing government power
because of interstate disputes and in wartime. The authors emphasized the
importance of civilian leadership which can be trained in civil servants using
educational initiatives. The findings of this study align with that of Gusev et al.
(2018), who stated that democratic civilian control should rely on government-run
educational campaigns to address societal concerns and doubts. Solving this task
also requires trained people.
The results of the course satisfaction survey suggest that participants perceived
the updated course and assignment positively. The values for skewness and
kurtosis were skewed right, implying that the judgments of the participants were
positive about the course’s effectiveness. The mean values suggest that the
participants were satisfied with the refresher course design, content, assessment
system, and intellectual gains it provided them.
The findings are consistent with the relevant literature. The study agrees with
Travis (2019), who held that wartime consolidated the efforts of the military and
civilian institutions within the national security policy and this consolidation
makes both institutions equally important. The findings also concur with Pantev
et al. (2005), who explained that the professional background of civil servants,
members of parliament, and political appointees is insufficient to deal with the
increasing complexity of the tasks in the security sector and needs training.
Furthermore, the findings align with Kuehn et al. (2017), who stated that there
remains a need for the development of the mechanisms and conditions for making
this kind of civilian control efficient and there is a need for the institutions of
civilian control and educated and trained people to exercise it.
6. Conclusion
The study sought to identify best practices for establishing civilian control over
the armed forces and police in emerging democracies that could be implemented
in Ukraine. The shortlisted practices were utilized to update the refresher course
for civil servants. The output of the study was drawn from the non-systematic
literature review and feedback of the civil servants who did the updated
professional refresher course. As revealed in the literature review, the process of
421
building civil-military relationships in Ukraine is in the transitional stage. There

is a need for educating and training civil servants in developing legal, regulatory,
and procedural frameworks for building civil-military relations and exercising
civilian control over them. It was made clear that establishing civilian control over
the army and developing the strategy of building mutually trustful civilian-
military relationships was not a top priority task in Ukraine in the period before
the Russian aggression. However, this task has recently become an urgent policy
action for the Ministry of Defence of Ukraine, the public government, and society.
The above was proved by a high correlation between terms such as reform-
Ministry of Defence, Ministry of Defence-Parliament, outside control-military,
and approach-public in the corpus of shortlisted texts. Educating and training
civil servants in developing legal, regulatory, and procedural frameworks for
building civil-military relations and exercising civilian control over them were the
key findings of the literature review. This study proved that the above
instructional solution could be feasible and effective. The results of the course
satisfaction survey suggest that participants perceived the updated course and
assignment positively. The participants reported that they appreciated the
refresher course design, content, assessment system, and intellectual gains it
provided. Further research is needed in identifying how the public media could
contribute to educating citizens on civilian control of the military and law
enforcement bodies.
Acknowledgement
We express our gratitude to the experts and students for their contribution to the
study. We would also like to express our gratitude to lecturers and students for
summarizing and translating the publications and for collecting the raw data.
Conflict of Interest
No conflicts of interest related to affiliations are declared.
7. References
Almeida, C. P. B., & Goulart, B. N. G. (2017). How to avoid bias in systematic reviews of
observational studies. Revista CEFAC/CEFAC Journal, 19(4), 551–555.
https://doi.org/10.1590/1982-021620171941117
Bethel, A., & Rogers, M. (2018). Choosing the right databases and search techniques. In
P. Levay & J. Craven (Eds.), Systematic searching (pp. 73–94). Facet.
https://doi.org/10.29085/9781783303755.005
Bobritsky, L. (2020). Kontrol za sectorom bezpeky i oborony cherez hromadski
ob’ednannia [Control over the security and defense sector through community
organisations] [Conference proceeding]. In R. Voitovych (Ed.), Problematyka
protsesu detsentralizatsiyi nadannya posluh v ob`yednanykh terytorialʹnykh hromadakh
[Problems of the process of decentralization of service delivery in common
territorial communities] (pp. 41–44). TOV «Vydavnychyy dim «ArtEk».
http://ipk.edu.ua/wp-content/uploads/2020/09/Kruh-stil-15.04.20.pdf
Bobrytska, V. I., Reva, T. D., Beseda, N. A., & Filippova, L. V. (2020). Approaches to
cultivating healthy behaviours in tertiary students: Systemic review. European
Journal of Educational Research, 9(4), 1649–1661. https://doi.org/10.12973/eu-
jer.9.4.1649
422
Boyle, M., & Hall, C. (2016). Teaching Don Quixote in the digital age: Page and screen,
visual and tactile. Hispania, 99(4), 600–614.
https://doi.org/10.1353/hpn.2016.0106
Cook, D. A. (2019). Systematic and nonsystematic reviews: Choosing an approach. In
D. Nestel, J. Hui, K. Kunkler, M. Scerbo, & A. Calhoun (Eds.), Healthcare
simulation research (pp. 55–60). Springer. https://doi.org/10.1007/978-3-030-
26837-4_8
Cortina, J. M. (1993). What is coefficient alpha? An examination of theory and
applications. Journal of Applied Psychology, 78(1), 98–104.
https://doi.org/10.1037/0021-9010.78.1.98
Croissant, A., Kuehn, D., Lorenz, P., & Chambers, P. W. (2013). Democratization and civilian
control in Asia. Palgrave Macmillan.
https://www.palgrave.com/gp/book/9780230285330
Demir, S., & Bingöl, O. (2020). From military tutelage to civilian control: An analysis of the
evolution of Turkish civil-military relations. British Journal of Middle Eastern
Studies, 47(2), 172–191. https://doi.org/10.1080/13530194.2018.1491291
Fluri, F., & Badrack, V. (2016a). Demokratychmyi tsyvilny control nad sektorom bezpeky:
Aktualni dzerela [Democratic civilian control over the security sector: Current
sources]. Tsentr Doslidzhenʹ Armiyi, Konversiyi ta Rozzbroyennya.
http://bitly.ws/tcrW
Fluri, P., & Badraсk, V. (Eds.). (2016b). Democratic civilian control and military crisis in
Ukraine: Legislative aspects. Сenter for Army, Conversion and Disarmament
Studies.
https://www.dcaf.ch/sites/default/files/publications/documents/DCAF_CA
CDS_UA_Dem_Control_Legislative_Aspects_ENG_2016.pdf
Geneva Centre for the Democratic Control of Armed Forces (DCAF). (2008). Democratic
control of armed forces. DCAF Backgrounder.
https://www.files.ethz.ch/isn/55845/17_bg_dem_control_armed_forces.pdf
Gonzalez, E. (2022). Adjudicating competing theories: Does civilian control over the
military decrease conflict? Armed Forces & Society, 1–28.
https://doi.org/10.1177%2F0095327X221112026
Greenhalgh, T., Robert, G., Bate, P., Macfarlane, F., & Kyriakidou, O. (2005). Critical
appraisal checklists: Appendix 2. In T. Greenhalgh, G. Robert, P. Bate, F.
Macfarlane, & O. Kyriakidou (Eds.), Diffusion of innovations in health service
organisations: A systematic literature review (pp. 234–244). Blackwell Publishing.
https://doi.org/10.1002/9780470987407.app2
Gusev, Y, Galushko, S., & Mazyarchuk, V. (Ed.) (2018). Do pytannia demokratychnoho
tsyvilnoho kontroliu nad zbroinymy sylamy Ukrainy [On the issue of democratic civil
control over the armed forces of Ukraine]. Ofis z finansovoho ta ekonomichnoho
analizu u Verkhovniy Radi Ukrayiny. http://bitly.ws/tH2B
Jamovi project. (2021). Jamovi (Version 2.0.0) [Computer software].
https://www.jamovi.org.
Kenwick, M. R. (2020). Self-reinforcing civilian control: A measurement-based analysis of
civil-military relations. International Studies Quarterly, 64(1), 71–84.
https://doi.org/10.1093/isq/sqz092
Koimaru, T. (2016). Exploring structural factors for civilian control over the military [Master’s
thesis]. Uppsala Universitet, Sweden.
http://urn.kb.se/resolve?urn=urn:nbn:se:uu:diva-294508
Koziy, I., & Polyakov, L. (2018). Reform of democratic civil control over armed forces and other
military formations in Ukraine: Next steps. Institute for Euro-Atlantic Cooperation.
http://bitly.ws/tH2Q
423
Kuehn, D. (2016). Institutionalising civilian control of the military in new democracies: Theory
and evidence from South Korea. GIGA Working Papers, No. 282. German Institute of
Global and Area Studies (GIGA). http://bitly.ws/tcs8
Kuehn, D. (2018). Democratic control of the military. In G. Caforio, & M. Nuciari (Eds.),
Handbook of the sociology of the military (pp. 161–178). Springer.
https://doi.org/10.1007/978-3-319-71602-2_9
Kuehn, D., Croissant, A., Kamerling, J., Lueders, H., & Strecker, A. (2017). Conditions of
civilian control in new democracies: An empirical analysis of 28 ‘third wave’
democracies. European Political Science Review, 9(3), 425–447.
https://doi.org/10.1017/S1755773916000011
Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M. Shekelle, P.,
Stewart, L. A., & PRISMA-P Group (2015). Preferred reporting items for
systematic review and meta-analysis protocols (PRISMA-P) 2015 statement.
Systematic Reviews, 4(1), 1–9. https://doi.org/10.1186/2046-4053-4-1
NIST/SEMATECH. (2013). e-Handbook of Statistical Methods. ITL.NIST.GOV.
https://doi.org/10.18434/M32189
Pantev, P., Ratchev, V., Tagarev, T., & Zaprianova, V. (2005). Civil-military relations and
democratic control of the security sector: A handbook for military officers, servicemen and
servicewomen of the security and intelligence agencies, and for civilian politicians and
security experts. Procon. http://bitly.ws/tcsf
Pennebaker Conglomerates. (2020). Linguistic inquiry and word count (LIWC) application.
LIWC.App. http://liwc.wpengine.com/
Perepelitsa, G. (2002). The development of civil-military relations in post-soviet Ukraine.
In A. Cottey, T. Edmunds, & A. Forster (Eds.), Democratic control of the military in
postcommunist Europe: One Europe or several? Palgrave Macmillan.
https://doi.org/10.1057/9781403905239_13
Rodrigues, I. B., Adachi, J. D., Beattie, K. A., & MacDermid, J. C. (2017). Development and
validation of a new tool to measure the facilitators, barriers, and preferences to
exercise in people with osteoporosis. BMC Musculoskelet Disord, 18(1), 1–9.
https://doi.org/10.1186/s12891-017-1914-5
Sinclair, S., & Rockwell, G. (2015). Text analysis and visualisation: Making meaning count. In
S. Schreibman, R. Siemens, & J. Unsworth (Eds.), A new companion to digital
humanities (pp. 274–290). Wiley Blackwell.
https://doi.org/10.1002/9781118680605.ch19
Sinclair, S., & Rockwell, G. (2020). Voyant tools. GitHub. https://voyant-tools.org/
Tagarev, T. D. (1997). The role of military education in harmonizing civil-military relations (the
Bulgarian case). NATO Democratic Institutions Individual Fellowship Project, final
report. https://www.nato.int/acad/fellow/95-97/tagarev.pdf
Travis, D. S. (2019). Pursuing сivilian сontrol over the military. Armed Forces & Society,
45(3), 546–560. https://doi.org/10.1177/0095327X18781539
Ukrainian military TV (2016). Democratychny control nad sectorom bezpeky [Democratic
control over the security sector] [YouTube video].
https://www.youtube.com/watch?v=0JYpvm_NCSo
Vashkevich, V. M., Bobrytska, V. I., Terepyshchy, S. O., Protska, S. M., & Makarenko, S.
O. (2020). Osvitnio-profesiyna programa “Derzhavna Osvitnia Polityka” [“State
Educational Policy” educational program]. Natsionalʹnyy pedahohichnyy
universytet imeni Drahomanova. http://bitly.ws/r6ia
Zadubinny, A. (2020). Andrii Taran: “Ukraine hopes to receive an action plan for NATO
membership in 2021”. Government Courier/Uryadovyy Kur'yer.
http://bitly.ws/tH3G
424
Appendices
Appendix A. Critical appraisal checklist
Reviewer____________________________________ Date ________________
Author(s)____________________________________ Year ________________
Record number ___________________________
Unclea
N/A
Yes
No
# Item
r
1 Is the publication relevant to the first research question?
2 Is the rationale for the study relevant to the purpose?
3 Does the study comply with recent theory and best
practices of establishing and managing democratic civilian
control of the military and law enforcement agencies in the
country?
4 Does the study reveal any relevant project or initiative
whose purpose is to rain or inform those who are involved
in building civilian-military relationships?
5 Does the study provide a clear description of the project
or initiative?
6 Is the project or initiative output relevant to the training
of civil servants?
7 Does the study provide a doable implementation plan for
a project or initiative or intervention?
8 Does the study discuss the actions taken (the change or
the intervention or initiative) and the methods used to
evaluate them clearly?
8 Does the study use a conceptual or analytical or
propositive narration?
Overall Include ❒ Include for full-text reading ❒ Include for final review ❒
appraisal:
Exclude ❒ Comments: (reasons for exclusion): ______________________
________________________________________________________
________________________________________________________
________________________________________________________
Appendix B: Two-scales-based student satisfaction survey questionnaire
Effectiveness Satisfaction
Question
1 2 3 4 5 1 2 3 4 5
1. How do you rate the refresher course
effectiveness?
2. How do you rate the design of the
refresher course?
3. How do you rate the course content in
terms of civilian control?
4. How do you rate the course
assessment system?
5. How do you rate your intellectual
gains from the course?
425

Vol. 21, No. 8, pp. 425-445, August 2022
Supervisory Performance of Cooperative

Teachers in Improving the Professional
Preparation of Student Teachers
Ali Ahmad Al-Barakat *
University of Sharjah, Sharjah, United Arab Emirates
Rommel Mahmoud Al Ali

King Faisal University, Al-Ahsa, Kingdom of Saudi Arabia
Mu’aweya Mohammad Al-Hassan

Ministry of Education, Irbid, Jordan
Omayya M. Al-Hassan
The Hashemite University, Zarqa, Jordan
Abstract. The purpose of this study was to investigate the degree of

effectiveness of the supervisory performance of cooperative teachers in
improving the professional preparation of student teachers in the
specialization of early childhood education.The participants were 155
student teachers enrolled in the education field-training program at
Yarmouk University in Jordan. To achieve the purpose of the study, a
questionnaire was developed, after insuring its validity and reliability.
The findings showed that, according to the means and standard
deviations of the questionnaire items, cooperative teachers did not fulfill
their supervisory roles in improving the professional preparation of
student teachers during the education field-training. The findings also
showed that there were no statistically significant differences in the
effectiveness of the supervisory performance of cooperative teachers
due to gender, academic qualification and teaching experience. This
indicates that these variables do not affect the cooperative teachers'
supervisory role in improving student teachers' preparation. This study
contributes to a better comprehension of the supervisory performance of
cooperative teachers in improving the professional preparation of
student teachers. It is suggested that the teacher education program at
Yarmouk University should pay attention to improve student teachers'
preparation during the education field-training phase.
Keywords: cooperative teachers; field training; supervisory roles;

student teachers; teacher education
*
Corresponding author: Prof. Ali Ahmad Al-Barakat, aalbarakat@sharjah.ac.ae
©Authors
426
1. Introduction
Education occupies a great importance in human societies, being the main
vehicle through which these societies seek to build the personality of individuals
in all aspects of humanity (Confesor & Belmi, 2022; Ennab et al., 2020; Fraihat et
al., 2022; Zhilgildinova et al., 2022). The process of teaching and learning
represents the means on which educational goals depend in achieving that goal
(Fraihat et al., 2022; Radovan & Kristl, 2017; Rodrigues et al., 2018). Al-Kandari
(2002) pointed out that education is a right of the individual in the society and a
necessity to achieve its national security. Therefore, the teacher has a role in
achieving the goals of the educational process. According to Podgornik and
Vogrinc (2017) and Poom-Valickis et al. (2012), no educational system can
achieve its goals without the presence of the teacher, who is considered one of
the central elements in the educational system.
Reviewing educational studies showed that there is a growing interest in the
process of preparing and qualifying teachers (Ennab et al., 2020; Radovan, 2011;
Singha & Sikdar, 2018). Thus, it is essential to prepare distinguished teachers’
capabilities to perform their roles effectively, where most countries around the
world have sought to focus on student teachers' preparation programs. In
developed countries, it was found that all educational reform movements
considered preparing student teachers as the best method to achieve the
educational objectives (Al-Kandari, 2002; Podgornik & Vogrinc, 2017; Poom-
Valickis et al., 2012; Radovan, 2011; Radovan & Kristl, 2017; Rodrigues et al.,
2018).
For instance, in the Arab countries, the notion of developing teacher-preparation
programs goes along with the emergence of the dominating conception in the
public discourse, which states that “There is a weakness in the outcomes of
education at different levels of educational stages”. These results are due to the
poor level of the academic and supervisory qualifications of teachers, the low
level of professional satisfaction and the widespread sentiment that teaching is a
profession for those who have no profession (Al-Kandari, 2002).
Previous ideas have positively affected the process of reconsideration of
teachers' preparation programs in countries that already have programs and
those working on developing programs in countries that do not have ones (Al-
Barakat & Alhassan, 2009; Alhassan et al., 2012; Fraihat et al., 2022). In Jordan,
teacher-preparation programs received attention by the government and the
Jordanian educational reform movement in 2014 was a distinctive starting point,
which included all the elements of the educational processes of teacher
preparation programs (Fraihat et al., 2022). For instance, the preparation of
teachers is an essential component for the success of the educational system to
achieve its objectives.
In hindsight, Jordanian universities have started to develop special programs for
preparing teachers in line with the recommendations of the Jordanian
educational reform movement. These programs have sought to provide student
teachers with appropriate knowledge, skills and tendencies that would
contribute to the preparation of the teacher (Makovec, 2018; Fraihat et al., 2022;
Gardee & Brodie, 2022), because there are essential things considered to ensure a
high level for the teacher to participate in developing the teaching and learning
427
process (Makovec, 2018; Verde & Valero, 2021). Teacher-preparation programs

accompanying the educational development process have been characterized by
the inclusion of the so-called education field-training, which is a transitional
stage between academic and behavioral education and the practicality of the
teaching profession in order to bridge the gap between theory and practice
(Verde & Valero, 2021; Zega & Lase, 2021).
This confirms that education field-training is an important component of
teacher-preparation programs in consideration of its great role in helping
teachers understand their future profession (Jin, 2022; Nagro et al., 2020).
Subsequently, is represents one of the important criteria for the success of
teacher preparation programs (O’Doherty et al., 2020; Zhang, 2020). The
importance of education field-training stems from the fact that theoretical
knowledge which lacks practicality is valueless, because it is far from reality and
objectivity (Nagro & deBettencourt, 2019). Alhassan et al. (2012), Zanting et al.
(2003), among others reported that education field-training forms a core aspect
of pre-service in teacher education, being regarded as the main focus of the
educational reform movement.
The success of education field-training programs, which are prominent in
preparing pre-service teachers, depends on an interlocking and interactive set of
components influencing and being influences by one another (Nathans &
Brown, 2022). The cooperative teacher is considered as an important element on
which the student teacher preparation process depends during education-field
training (Anderson, 2009; Ballinger & Bishop, 2011; Maes et al., 2022). Studies by
Caires et al. (2012) and Tasdemir et al. (2007) emphasized the importance of
preparing and training cooperative teachers according to specific programs,
while adopting various training models which assist them to be prepared and
qualified as distinguished teachers. All of these require the cooperative teacher's
awareness of preparing student teachers.
Considering the significant importance of the cooperative teacher in achieving
the objectives of student teachers' preparation, modern educational perceptions
have identified a set of supervisory roles that improve the quality of student
teachers' preparation. Various educational studies, such as those conducted by
Altan & Sağlamel (2015), Clarke (2014), Harrington & Enochs (2009), Office of
Teacher Education (2022), Özdemir-Yılmazer (2021), and Sung (2007) defined a
set of supervisory roles that contribute to the success of student teachers'
preparation, which are:
- Collaboration with the supervisor in helping student teachers according to the
proper professional manner;
- Selecting the appropriate learning environment for the purposes of education
field-training;
- Informing student teachers about the objectives of the education field-training;
- Encouraging student teachers to benefit from the education field-training
- Monitoring the training process;
- Evaluating the performance of student teachers on a continuous basis;
- Providing school facilities necessary to implement the educational and
learning settings.
428
The role of the cooperative teacher should focus on training student teachers on
instructional skills in various academic subjects in accordance with
contemporary educational perceptions centered on the concept of the child-
centered learning process. However, the above mentioned assertions are based
on various studies, (Haciomeroglu, 2013; Mutlu, 2014; Zhao & Zhang, 2017;
Zhilgildinova et al., 2022) which showed that the supervisory role of the
cooperative teacher should be based on a set of standards, which are not limited
to directing student teachers, but rather focus on training them on the teaching
skills in various learning situations, based on contemporary educational
perceptions that revolve around the concept of the student-centered learning
process. In addition, the supervisory role of cooperative teachers should help
student teachers organize, plan, perform and assess the teaching and learning
process. Furthermore, cooperative teachers should monitor student teachers
with regard to the school's policy, philosophy, regulations, and facilities, as well
as observing their performance in various teaching and learning settings and
helping them solve the difficulties that they face in a logical and educational
way (Haciomeroglu, 2013; Mutlu, 2014; Office of Teacher Education, 2022).
Previous studies conducted by Haciomeroglu (2013), Hudson, et al. (2009), Zhao
& Zhang (2017), and Zhilgildinova et al. (2022) showed that the cooperative
teacher might contribute to the student teachers’ preparation by directing them
to participate in meetings with the teaching staff and participate in social
activities. Supervisory role of the cooperative teacher, Grant and Zeichnar (2002)
and Zhao & Zhang (2017) found that the supervisory role of the cooperative
teacher is mainly through observing the student teachers’ performance in the
learning environment and then holding a meeting to provide them with
feedback that can contribute to improving their instructional performance.
A set of studies conducted by Anderson, (2009), Hancock and Gallard (2004) and
O'Sullivan and Conaill (2022) revealed that the cooperative teacher has a
significant role in developing positive beliefs among student teachers towards
teaching practices based on learner-centered learning from kindergarten to high
school stage. This depends on the learning opportunities available to student
teachers to develop themselves as constructive teachers. This role requires the
cooperative teacher to assist student teachers in identifying the school's
philosophy, policy, instructions and activities, including monitoring teaching
performance with the aim to improve, develop and assist the cooperative teacher
and the university supervisor in solving problems facing student teachers.
Considering the importance of the supervisory role of the cooperative teacher in
preparing student teachers, the education Field Training Program at Yarmouk
University (2019) reported its instructions for participation of the cooperative
teacher in the process of assessing the performance of student teachers during
the semester according to a model prepared for this purpose. These included the
roles that contribute to the preparation of the teacher. These centered mainly on
the supervisory role of the cooperative teachers in training student teachers to
become the main controllers of the learner-based learning approach (Bayrakc,
2009; Hudson, et al., 2009).
As illustrated above, despite that it is apparent how important the supervisory
roles of cooperative teachers are in improving the quality of student teacher
429
preparation, rare studies have attempted to determine the nature of the

supervisory roles of cooperative teachers. Previous studies (Al-Barakat &
Alhassan, 2009; Alhassan et al., 2012; Payan & Murphy, 2012; Maes et al., 2022)
found that cooperative teachers the greatest influence on how student teachers
employ new have instructional strategies and classroom-management
techniques. However, they did not find out whether cooperative teachers have a
strong influence in performing their supervisory roles which are concerned with
assisting student teachers in their professional and personal development, thus
improving the educational process (Bayrakc, 2009; Zhilgildinova et al., 2022).
2. Problem statement and Study rationale

The cooperative teacher’s role in improving the quality of student teacher
preparation is regarded as one of the most effective components in preparing
student teachers to be effective future teachers during the instructional practices.
It is worth noting that the way in which student teachers are successfully
prepared depends on the extent of cooperative teachers’ understanding and
awareness of their supervisory roles as a part of the training program, because
the cooperative teacher who is able to perform supervisory roles efficiently will
be able to influence the quality of preparing future teachers.
Hence, the great importance of the cooperative teacher in the student teacher-
preparation process cannot be denied, as it is an essential component of
education field-training programs. However, there is a lack of information
related to the extent to which cooperative teachers perform their supervisory
roles in the preparation of student teachers. Hence, the practical experience and
field observations of the faculty members in education field-training programs
reveal shortcomings and weaknesses in improving the preparation of student
teachers. However, the study seeks to provide decision makers at Yarmouk
University with the needed feedback related to the actual performance of
cooperative teachers’ supervisory roles during field experience. As mentioned,
the study problem stemmed from the scarcity of studies in the Jordanian
environment that dealt with the effectiveness of supervisory performance of
cooperative teachers during education field-training periods. This was
supported by the study undertaken by Al-Hassan & Al-Barakat )2013(, which
recommended the necessity of conducting research on the supervisory
performance of cooperative teachers at Yarmouk University and the Hashemite
University in Jordan.
3. Objectives and research questions of the study

Based on the above, the current study aims to:
- Investigate the extents to which cooperative teachers perform their
supervisory roles in improving the quality of student teacher preparation.
- Determine whether there are any statistically significant differences in the
degree of effectiveness of supervisory performance of cooperative
teachers in improving the professional preparation of student teachers
based on gender, academic qualification and experience of cooperative
teachers.
Specifically, the study attempts to answer the following questions:
430
1. What is the degree of effectiveness of the supervisory performance of

cooperative teachers in improving the professional preparation of student
teachers in the specialization of early childhood education as assessed by
students?
2. Are there any statistically significant differences in the degree of
effectiveness of the supervisory performance of cooperative teachers in
improving the professional preparation of student teachers attributed to
the cooperative teachers’ gender, academic qualification and experience?
4. Method
4.1 Study population and sample
The population for this study consisted of all student teachers (155), who train in
the early childhood education classes in public and private schools affiliated to
the Directorate of Education in Irbid district in Jordan during the first semester
of the academic year 2021/2022. With regard to the sample of the study, it
consisted of the population itself (155 student teachers). This means that the
sample represented the total population of the study; the questionnaire was
distributed to the total population. After data collection, one hundred and forty-
six questionnaires were considered, with a return rate of 94.19%.
With regard to the study variables related to cooperating teachers, which were
obtained through official records in the Office of Education field-training
program at Yarmouk University, the results of the study data analysis, as
mentioned in the first part of the questionnaire, showed that the number of male
cooperating teachers was 56 (38.35%), while female cooperative teachers were 90
(61.65%). The reason that the number of females is more than that of males is
attributed to the tendency of the Jordanian Ministry of Education to feminize
education in early grades. With regard cooperative teachers’ experiences and
qualification, the number of those with a teaching experience of less than 10
years was 75 (51.37%), while those with a teaching experience of more than 10
years was 71 (48.63%). As for their qualification 88 (60.27%) were bachelor-
degree holders and 58 (39.73%) were postgraduate.
4.2 Study design and instrument

In this study, the quantitative research methodology was adopted to find out the
supervisory roles of cooperative teachers in improving instructional
performance of student teachers during the education field-training phase. To
achieve this, a questionnaire was developed according to Altan and Sağlamel
(2015), Clarke (2014), Kastens (2004), Harrington and Enochs (2009), Özdemir-
Yılmazer (2021) and Sung (2007). The questionnaire consisted of two parts:
4.2.1 Data related to the cooperating teacher:

This part includes three questions related to the gender of the cooperative
teacher, his/her teaching experience and his/her academic qualification. Each
student teacher answered these questions through official records in the Office
of Education field-training program at Yarmouk University.
431
4.2.2 Part two –questionnaire items

The questionnaire, in its first version, consisted of 33 items. These items be
related to the supervisory roles of the cooperative teacher, which must perform
by directing and guiding student teachers to the procedures of teaching in the
childhood education stage, according to a guide directed to cooperative teachers
by the Office of Education field-training program at Yarmouk University.
Participants were asked to rate items on a 5-point Likert scale, where (very high =
5, high = 4, moderate = 3, low = 2, very low =1).
To verify the validity of the questionnaire, eight experienced and specialized
moderated the questionnaire items, and agreed on the item measures. After the
moderators submitted their comments, which included the exclusion and
modification of some items, as well as the addition of some new items, all the
modifications and suggestions were implemented. The study instrument, in its
final version, consisted of (26) items (see appendix 1).
To assure the reliability of the instrument, Cronbach’s alpha coefficient
calculation was performed to assure the internal consistency of the
questionnaire, which amounted to 0.93, indicating that the questionnaire is
appropriate to achieve the purpose of this study.
Three weeks before commencement of the study, the questionnaire was applied
twice to an exploratory sample of (25) student teachers who were excluded from
the main sample, with on interval of two weeks between the two applications
(test-retest). Pearson's correlation coefficient of the instrument was calculated
and found to be 0.91, which is appropriate for the purpose of this study. This
gives an indication that the degree of consistency of test scores from one test
administration to another was high.
4.3 Data collection and analysis
One hundred and fifty-five questionnaires were distributed to the sample of the
study during an official meeting of student teachers. The participants’ agreement
was obtained to participate in the study. All respondents were informed about
the purpose of the study and assured of the confidentiality of the data that they
will provide in their responses. Anonymity was assured, so that respondents
could express their responses freely. The subjects of the study were given
enough time to answer the questionnaires; they were given a period of one
month to return the questionnaires by themselves.
The questionnaire was administered to all student teachers in the specialization
of childhood education. A total of 15S questionnaires were distributed, 9 invalid
questionnaires were excluded and 146 valid questionnaires were returned, with
a return rate of 94.19%.
With regard to data analysis, Statistical Package for Social Sciences (SPSS) was
used, as means and standard deviations were calculated to identify the degree of
effectiveness of cooperative teachers' supervisory performance for each item of
the questionnaire as assessed by student teachers. In addition, three-way
ANOVA was used to reveal the effect of cooperative teachers’ gender, academic
qualification and experience on their supervisory roles as evaluated by student
teachers. This test was used, because it reveals the main effect of each of the
three independent variables separately.
432
5. Results
The results of the study will be presented in two parts, based on the research
questions.
5.1 Results of the first question

This question aimed to determine the degree of effectiveness of the supervisory
performance of cooperative teachers in improving the professional preparation
of student teachers in the specialization of early childhood education from
student teachers’ point of view. Then, to answer this question, means and
standard deviations for all of the questionnaire items were calculated. Data were
presented and organized in descending order, as shown in Table 1.
Table 1: Means and standard deviations of student teachers’ responses to the

Item Mean St. Dev.
No. My assessment of the effectiveness of the performance of
cooperative teachers is as follows:
1 Assigning the student teacher for planning. 3.91 0.81
2 Assigning the student teacher to perform some teaching 3.90
0.79
tasks.
3 Providing the student teacher with feedback on his/her 2.15
0.83
teaching performance.
4 Monitoring the performance of the student teacher in the 2.15
0.73
classroom environment.
5 Providing the student teacher with opportunities to attend a 2.14
0.75
variety of teaching models.
6 Choosing practical lessons according to certain standards. 2.14 0.79
7 Supervising and monitoring the student teacher about 2.14
0.74
his/her training role.
8 Watching lessons for the teacher/student. 2.13 0.90
9 Monitoring the student teacher lesson preparation 2.12
0.90
notebook.
10 Giving the student teacher a chance to apply various 2.12
0.73
alternative assessment strategies.
11 Providing the student teacher with opportunities to 2.11
0.88
participate in the creation of educational aid means.
12 Informing the student teacher of contemporary trends in 2.11
0.69
developing teaching performance.
13 Attempting to understand the problems that the student 2.11
0.68
teacher faces in teaching.
0.71
participate in school activities.
15 Participating in supervisory meetings between the 2.10
1.01
university supervisor and the student teachers.
433
Table 1: Means and standard deviations of student teachers’ responses to the

Item Mean St. Dev.
No. My assessment of the effectiveness of the performance of
cooperative teachers is as follows:
16 Introducing the student teacher to the laws of managing the 2.09
1.11
educational environment.
17 Helping the student teacher solve his/her problems. 2.09 1.09
1.08
participate in social activities.
19 Providing the student teacher with opportunities to use 2.07
0.69
educational technologies in teaching.
20 Directing the student teacher to attend typical classes with 2.06
0.98
more than one teacher.
21 Simplifying the student teacher's task in conducting 2.05
0.91
activities outside the classroom.
22 Encouraging the student teacher to use modern teaching 2.02
1.07
methods.
23 Providing a school environment concerned with creating 2.01
0.89
human relations with the teacher/student.
24 Providing the necessary materials and tools for the student 2.00
0.90
teacher to produce educational aids.
25 Treating the student teacher as a part of the teaching 1.98
0.99
system.
26 Informing the student teacher of the importance of his/her 1.96
1.03
teaching role
Total 2.18 0.67
Table 1 facilitates the results' interpretations and determines the degree of

effectiveness of the supervisory performance of cooperative teachers for each
item included in the questionnaire. The grading was calculated according to the
following: Highest score on the response scale - Lowest score on the response scale/
3.This means 5 -1/ 3 = 1.33 (Odeh, 2018), so the grading becomes as follows:
- The mean scores below (2.33) indicate a low degree of effectiveness.
- The mean scores within (2.33-3.66) indicate a medium degree of effectiveness.
- The mean scores within (3.66 or more) indicate a high degree of effectiveness.
According to the above grading, Table (1) shows that Items (1) and (2) got high
mean scores within (3.91-3.90), which indicates a high degree of effectiveness
among cooperative teachers in improving the quality of the student teachers’
preparation in terms of directing them to perform plans for teaching and
perform teaching tasks.
Moreover, Table 1 reveals that Items (3) to (26) got low mean scores, within
(2.15-1.96), which indicates a low degree of effectiveness of the supervisory
434
performance of cooperative teachers with regard to the implementation of

teaching and assessment of learning.
Thus the results indicate that the majority of the supervisory roles of the
cooperative teachers were not effective, since 24 items got low mean scores.
Therefore, this showed an ineffectiveness of the supervisory performance in
improving the teaching performance of student teachers.
5.2 Results of the second question

The second question aimed to reveal whether there are differences in the degree
of effectiveness of the supervisory performance of cooperative teachers in
improving the professional preparation of student teachers in light of
cooperative teachers’ gender, academic qualification and experience. To achieve
this aim, three way-ANOVA was used to reveal the effect of cooperative
teachers’ gender, academic qualification and experience on performing their
supervisory roles as evaluated by student teachers. The findings of three-way-
ANOVA are shown in Table 2.
Table 2: Means and standard deviations of the effect of cooperative teachers’

gender, academic qualification and experience on their supervisory roles
Variable Category No. Mean Standard Deviation
Gender Male 56 3.95 0.61

Female 90 4.01 0.64
Experience less than 10 years 75 4.02 0.69
More than 10 years 71 4.10 0.45
Academic Bachelor 88 4.02 0.57
qualification Post graduate studies 58 4.14 0.60
Table 2 shows that there are apparent differences between the mean scores of the
participants’ responses in terms of the degree of effectiveness of cooperative
teachers' supervisory role performance in improving the professional
preparation of student teachers in light of cooperative teachers’ gender,
academic qualification and experience. It defines the significance of the
differences. Three-Way-ANOVA was calculated. Table 3 show a summary of
three-way ANOVA calculations.
Table 3: Three-way ANOVA results of the effect of cooperative teachers’
gender, academic qualification and experience on their supervisory roles
Variable Sum of Degrees of Mean of Value Significance
squares Freedom Squares of (F)
Gender 0.258 1 0.258 0.397 0.368
Experience 0.397 1 0.397 1.132 0.235
Academic
1.005 1 1.005 2.715 0.080
qualification
Error 48.922 142 0.345
Total 50.496 145
* Significant at p = 0.05
435
Table 3 shows that the value of (F) was 0.397 for the responses of the study
sample in light of the gender variable. This indicates no statistically significant
differences at (p = 0.05) in the degree of effectiveness of cooperative teachers'
supervisory role performance in improving the professional preparation of
student teachers due to cooperative teachers’ gender.
The results found that the value of (F) was 1.132 for the responses of the study
sample in light of the experience variable, which indicates no statistically
significant differences in the degree of effectiveness of cooperative teachers'
supervisory role performance in improving the professional preparation of
student teachers due to cooperative teachers’ experience (under 10 years; 10
years and more). This shows that the variations in the teaching experience
among the subjects of the study did not affect the supervisory roles of
cooperative teachers.
As per academic qualification (bachelor; postgraduate studies), Table 3 confirms
that the value of (F) was 2.715, which is less significant. This indicates that the
differences in academic qualification among the respondents of the study did
not affect their supervisory roles in improving the professional preparation of
student teachers.
6. Discussion
The discussion of the results was divided into two parts based on the study
questions, as follows:
6.1 Discussion of the results of the first question

The results of the first study question revealed that despite the existence of an
effective supervisory role of cooperative teachers regarding assigning student
teachers to plan and perform real learning situations. Contrarily, the role is not
based on prior preparation for the student teachers, because they did not receive
sufficient training and guidance on how to plan and implement teaching and
learning settings. These results reflected an ineffective supervisory role of
cooperative teachers in terms of guiding student teachers to acquire and apply
the best teaching and learning practices related to managing learning
environments, using contemporary teaching strategies and methods appropriate
to children, as well as assessing children's learning.
These findings might negatively affect the student teachers' attitudes towards
the education field-training, since the study revealed that student teachers did
not receive feedback from cooperative teachers during their co-assessment
meeting regarding planning, implementing and evaluating learning. As per
previous report by educational researchers (Al Mamun et al., 2020; Iqbal et al.,
2021; Keiler, 2018; Watson et al., 2017), this confirms lack of attention from
cooperative teachers in providing student teachers with feedback related to their
teaching performance. The respondents of the study indicated a weaker role of
cooperative teachers in providing necessary feedback, which indicates the lack
of active follow-up of the teaching performance of student teachers. These
findings contradict with the global trend that emphasizes giving student
teachers critical and conclusive feedback of all teaching and learning practices
436
(Darling-Hammond et al., 2020; Janelli & Lipnevich, 2021; Maes et al., 2022;
Panadero & Lipnevich, 2022).
Moreover, the findings of this study revealed that cooperative teachers did not
encourage student teachers to attend different models of effective learning and
teaching situations, as well as inviting them to organize extracurricular activities
in cooperating schools. This can be attributed to the existence of traditional
perceptions among cooperative teachers with regard to the teaching and
learning settings. The results emphasize the fact that cooperative teachers lack
sufficient knowledge in the field of professional preparation based on
contemporary educational perceptions, since they still apply traditional methods
in teaching. More clearly, this could be attributed to the fact that cooperative
teachers are unaware of contemporary methods and trends in improving the
student teachers’ instructional performance, since they are not regularly
following up the latest developments in the professional preparation of student
teachers.
This is inconsistent with the supervisory roles of cooperative teachers that
emphasize that student teachers should be given feedback by attending and
observing a variety of exemplary lessons in order to develop instructional
experiences (Gonzalez-Torres et al., 2022).
The current study findings concur with previous studies conducted by Altan
and Sağlamel (2015), Hamaidi et al. (2014), Izadinia, (2013), among other which
reported that cooperative teachers use traditional teaching methods, such as
providing children with information. Subsequently, the study of Miqdadi (2003)
indicated that cooperative teachers deny student teachers the opportunity to use
modern teaching methods, as perceived to differ from familiar ones, which led
to the creation of student teachers’ confusion and embarrassment in front of
students due to lack of adaption with traditional methods. Generally, the
outcomes of this study concur with the study of Lee and Walker (2000) previous
studies, which reported that cooperative teachers recognize the use of modern
teaching methods as an undesirable behaviors, since cooperative teachers prefer
the calm classroom environment instead of the active environment that depends
on the learner as the focal point of the learning process in the using modern
methods and tools to create an active-learning environment.
Furthermore, the current study showed a weakness in the supervisory roles of
cooperative teachers in providing a school environment concerned with creative
human relations with student teachers, as well as treating them as a part of the
educational system, by making them aware of the importance of their teaching
role.
The study reflected on the student teachers' sense of weakness in human
relations prevailing at the school environment, noting that weakness can never
contribute to improvement of student teachers' preparation. This result can be
attributed to the existence of misperceptions among some cooperative teachers
who believe that the student teacher is an additional burden in the school. They
do not regard the student teacher as an integral part of the teaching staff because
she/he is not an official teacher in the school.
437
Obsolete practices do not provide a school environment that enhances creating

human relations with student teachers. Thus, such practices are considered
among the most important obstacles which affect the student teachers’
preparation process. Educational studies, such as those conducted by Al-Bandari
and Al-Atoum (2002), Cascio et al. (2019) and Luis et al. (2022) pointed out the
importance of human and personal relations among the teaching staff, being the
most influential factor in the educational process, as well as their impact on
achieving adaptation among school staff. This result contradicts with
psychological and social theories that emphasize the importance of integrating
the student teacher into social and human interactions, as this effectively
contributes to the development of personality and the development in the field
of professional growth (Han & Huang, 2022; Hoppler et al., 2022; Stafford et al.,
2011; Tamir & Hughes, 2018).
Based on the results, it is worth noting that cooperative teachers did not show
sufficient interest in the development of critical supervisory practices by
focusing on the student teachers in the professional development process. The
current study recognizes the weaker roles of cooperative teachers, which
contradicts with previous studies, such as those conducted by Großschedl, et al.
(2014), Kastens (2004) and Thema et al. (2017) that the existence of an
experienced teacher with student teachers inevitably leads to the refinement of
personality to become experienced teachers, since the student teacher does not
have sufficient sources of pedagogical knowledge to apply the practices of
contemporary learning.
6.2 Discussion of the results of the second question
The findings of the second study question revealed that there are no statistical
significant differences at (p = 0.05) in the degree of effectiveness of cooperative
teachers' supervisory role performance in improving the professional
preparation of student teachers with regard to cooperative teachers’ gender,
academic qualification and teaching experience. This can be attributed to the
lack of high interest among cooperative teachers in performing their supervisory
roles to improve the quality of early childhood teachers' preparation during the
education field-training. Regardless of their gender, academic qualification and
teaching experience, they are not aware of their supervisory roles related to
considering and following up the teaching performance of the student teacher.
This result also indicates lack of cooperative teachers' awareness of the
importance of their supervisory roles in guiding the student teachers, as well as
lack of incentives and rewards for cooperative teachers to perform supervisory
roles in preparing student teachers during the education field-training phase.
Generally, these results do not conform to the role of cooperative teachers in
training and supervising student teachers. This indicates that cooperative
teachers were improperly selected based on specific criteria. Heinz (2013),
Klassen and Kim (2019) and Sawyer (2005) asserted that selecting the most
qualified cooperative teachers can perform the student teachers’ training
process, which represents a positive contribution to the education field-training
program. The reason behind these results are attributed to the lack of knowledge
of cooperative teachers about the phases of pre-service teachers’ education field-
training. Student teachers must acquire their experiences through the
438
observation of various classes and teaching processes, then through the partial
training phase for some of classroom settings, and finally through the total
practice of integrated classroom settings. Failure to provide student teachers
with the opportunity to attend exemplary lessons of more than one teacher
contradicts with the instructions of the field training as a basis of field education.
Moreover, these results are inconsistent with the actual supervisory roles of
cooperative teachers, which include organizing and directing student teachers in
order to perform their roles. They are also inconsistent with the modern
approaches, which give cooperative teachers a conclusive role in following up
the student teachers during the education field-training.
The findings of the study gave an indication that many cooperative teachers are
unaware of their supervisory roles. Miqdadi (2003) indicated that supervising
the student teacher should be a collaborative and complementary effort between
the university supervisor, the school principal and the cooperative teacher, with
the aim of increasing the effectiveness of supervision. Thus, this is anticipated to
contribute to achieving better objectives in the education field-training program,
as well as contributing to developing educational climate experiences for
student teachers, being considered as an important source of experience.
The current study has certain limitations. The specific context is a central one.
Certainly, hundred forty six respondents participated in the study from the
same training institute in Yarmouk University. Future research should expand
the target population to contain the majority of education field-training
programs in Jordan to generalize the results more widely. This study was
limited to a questionnaire to collect data from the student-teachers’ point of
view. It is therefore required that future research should focus on the
qualitative researches instruments such as semi-structured interviews and focus
groups, which will be fitting to such studies.
8. Conclusions
The period spent in the field of education training is one of the most important
components of teacher preparation programs. One of the main objectives is to
provide student teachers with the opportunity to acquire knowledge, life skills,
attitudes and beliefs. This, summarizes the success of teacher preparation related
to practical preparation. Based on the significance of preparing future teachers in
the field of childhood education, the current study was designed to find out the
effectiveness of the supervisory role performance of cooperative teachers in
improving the professional preparation of student teachers in the specialization
of early childhood education.
The study recognizes the importance of developing these practices in line with
contemporary educational trends of preparing future teachers to assist children
in building their own knowledge and experiences through partaking active roles
in learning environments. The importance of developing these practices is to
eliminate the practices of transmission-learning model that focuses on the
teacher and the academic content as being central to the learning process rather
than student-centered learning.
439
Despite the importance of the great role of cooperative teachers in improving the
teaching performance of student teachers, the current study concluded that
cooperative teachers did not impetus in directing student teachers to employ the
proper learning practices. This contradicts with the orientations of the
psychological and educational learning theories which seek to prepare teachers
who have distinctive abilities in employing various methods to develop young
children’s learning (Maes et al., 2022).
9. Recommendations
In light of the findings of the study, the following recommendations are made:
- Providing brochures to inform cooperative teachers of their duties and tasks
during the education field-training phase, noting that student teachers at
the beginning of the training process must take these duties and tasks into
account at different classroom environments. Then, they start the partial
practice phase under the supervision of the cooperative teacher and finally
comes the total practice phase where cooperative teachers and the principal
give student teachers the opportunity to be alone in the class to implement
the teaching methods.
- Selecting highly qualified cooperative teachers to perform their supervisory
roles toward student teachers.
- The necessity of selecting cooperative teachers based on the availability of
school equipment.
- The necessity of involving student teachers in school committees, which
might develop their initiatives and creativity.
- Holding training programs for cooperative teachers to inform them with
their supervisory roles.
10. References
Al Mamun, Md Abdullah, Lawrie, Gwendolyn, and Wright, Tony (2020). Instructional
design of scaffolded online learning modules for self-directed and inquiry-based
learning environments. Computers and Education, 144, 103695.
Al-Bandari, M., & Al-Atoum, A. (2002). Personal relationships among school principals
and teachers and their relation to high schools teachers’ job satisfaction in Oman
and Jordan. Journal of Educational and Psychological Sciences, 3(3), 91-12. doi:
http://dx.doi.org/10.12785/JEPS/030304
Al-Barakat, A. & Al-Hassan, A. (2009). Peer Assessment as a Learning Tool for
Enhancing Student Teachers' Preparation. Asia-Pacific Journal of Teacher Education,
37(4), 399-413 https://doi.org/10.1080/13598660903247676
Al-Hassan, O., & Al-Barakat, A. (2013). Cooperating teachers’ role in developing
constructivist learning practices of childhood education students in Jordanian
universities during field training. Journal of Educational and Psychological, 14(4), 337-
367. https://journal.uob.edu.bh/handle/123456789/2948
Al-Hassan, O., Al-Barakat, A., & Al-Hassan, Y. (2012). Pre-service teachers' reflections
during field experience. Teaching and teacher Education, Journal of Education for
Teaching, International research and pedagogy, Oxford, 38(4), 419-434.
https://doi.org/10.1080/02607476.2012.707918
Al-Kandari, J. (2002). Teacher preparation at Kuwait University: reality and hope. Journal
of Educational and Psychological Sciences, 3(3), 12-31.
http://dx.doi.org/10.12785/JEPS/030301
440
Altan, M., & Sağlamel, H. (2015). Student teaching from the perspectives of cooperating
teachers and pupils. Cogent Education, 2(1), 1-16.
https://doi.org/10.1080/2331186X.2015.108629
Anderson, D. (2009). The impact of cooperating teachers on the teaching perspectives of
student teachers. The International Journal of Learning Annual Review, 16(1), 119-133.
https://doi.org/10.18848/1447-9494/CGP/v16i01/45883
Ballinger, D., A., & Bishop, J., G. (2011). Theory into practice: mentoring student
teachers: collaboration with physical education teacher education. Strategies, 24,
30–34. https://doi.org/10.1080/08924562.2011.10590941
Bayrakc, M. (2009). In-service teacher training in Japan and Turkey: A comparative
analysis of institutions and practices. Australian Journal of Teacher Education, 34(1),
1- 22. http://dx.doi.org/10.14221/ajte.2009v34n1.2
Caires, S., Almeida, L., & Vieira, D. (2012). Becoming a teacher: student teachers’
experiences and perceptions about teaching practice. European Journal of Teacher
Education, 35(2), 163-178. https://doi.org/10.1080/02619768.2011.643395
Cascio, C., Moore, D., & McGlone, F. (2019). Social touch and human development. Dev.
Cogn. Neurosci. 35, 5–11. https://doi.org/10.1016/j.dcn.2018.04.009
Clarke, A., Triggs, V., & Nielsen, W. (2014). Cooperating teacher participation in teacher
education: A review of the literature. Review of Educational Research, 84(2),163-202.
https://doi.org/10.3102/0034654313499618
Confesor, R., & Belmi, R. (2022). Structure, activities and teacher development in the
Philippine science teachers’ community of practice. International Journal of Learning,
Teaching and Educational Research, 21(6), 71-89.
Darling-Hammond, L., Flook, L., Cook-Harvey, B., Barron, C., & Osher,
D. (2020) Implications for educational practice of the science of learning and
development. Applied Developmental Science, 24(2), 97-140.
https://doi.org/10.1080/10888691.2018.1537791
Ennab, R., Al-Shannag, M., & Al-Barakat, A. (2020).The efficiency of the PQ4R strategy
in understanding the mathematical proof among the primary school female
students. Journal of Education and Practice, 11(16), 71-79.
https://doi.org/10.7176/JEP/11-16-09
Fraihat , M., Khasawneh, A. Al-Barakat, A. (2022). The effect of situated learning
environment in enhancing mathematical reasoning and proof among tenth grade
students. EURASI, Journal of Mathematics, Science and Technology Education, 18(6),
em2120. https://doi.org/10.29333/ejmste/12088.
Gardee, A., & Brodie, K. (2022). Relationships between teachers’ interactions with learner
errors and learners’ mathematical identities. International Journal of Science and
Mathematics Education, 20, 193–214 (2022). https://doi.org/10.1007/s10763-020-
10142-1
Gonzalez-Torres, P., Cabrera-Solano, P., & Castillo-Cuesta, L. (2022). Exploring
perceptions of online feedback in teaching EFL speaking and writing skills during
the COVID-19 pandemic. International Journal of Learning, Teaching and Educational
Research, 21(7), 330-344. https://doi.org/10.26803/ijlter.21.7.17
Grant, C., & Zeichner J. (2002). Elementary Education Program Student Teaching Handbook.
University of Wisconsin. https://silo.tips/download/elementary-education-
program-student-teaching-handbook.
Großschedl, J., Mahler, D., Kleickmann, T., & Harms, T. (2014). Content-related
knowledge of biology teachers from secondary schools: Structure and learning
opportunities. International Journal of Science Education, 36(14), 1-32.
https://dx.doi.org/10.1080/09500693.2014.923949
441
Haciomeroglu, G. (2013). The field experiences of student teachers and effective

mathematics teaching in Turkey. Australian Journal of Teacher Education, 38(2), 132-
142. https://doi.org/10.14221/ ajte.2013v38n2.5
Hamaidi, D., Al-Shara, I., Arouri, J., & Abu Awwad, F. (2014). Student-teachers’
perspectives of practicum practices and challenges. European Scientific Journal, 10,
191–214. https://doi.org/10.19044/esj.2014.v10n13p%25p
Han, C., & Huang, J. (2022). Chinese college students' perceived teacher autonomy
support and engagement: a moderated mediation model. International Journal of
Learning, Teaching and Educational Research, 21(7), 269-285.
Hancock, E., & Gallard, A. (2004). Pre-service science teachers' beliefs about teaching and
learning: The influence of K-12 field experiences. Journal of Science Teacher
Education, 15(4), 281-291.
Harrington, R. & Enochs, L. (2009). Accounting for pre-service teachers' constructivist
learning environment experiences. Learning Environment Research, 12, 45-65.
https://doi.org/10.1007/s10984-008-9053-4
Heinz, M. (2013). Tomorrow’s teachers—selecting the best: an exploration of the quality
rationale behind academic and experiential selection criteria for initial teacher
education programmes. Educational Assessment Evaluation and Accountability,25(2)
93–114. https://doi.org/10.1007/s11092-013-9162
Hoppler S., S., Segerer, R., & Nikitin J. (2022) The six components of social interactions:
actor, partner, relation, activities, context, and evaluation. Frontiers Psychology, 12.
743074. https://doi.org/10.3389/fpsyg.2021.743074
Hudson, P., Nguyen, H., & Hudson, S. (2009). Mentoring EFL preservice teachers in EFL
writing. TESL Canada Journal, 27(1), 85-102.doi:
https://doi.org/10.18806/tesl.v27i1.1033
Iqbal, M., AkhterSiddiqie, S., & AbdulMazid, M. (2021). Rethinking theories of lesson
plan for effective teaching and learning. Social Sciences & Humanities Open, 4(1),
100172. https://doi.org/10.1016/j.ssaho.2021.100172
Izadinia, M. (2013). A review of research on student teachers’ professional identity.
British Educational Research Journal, 39, 694–713.
https://doi.org/10.1080/01411926.2012.679614
Janelli, M., & Lipnevich, A. (2021). Effects of pre-tests and feedback on performance
outcomes and persistence in Massive Open Online Courses. Computers &
Education, 161, 104076. https://doi.org/10.1016/j.compedu.2020.104076
Jin, M. (2022). Preservice teachers’ online teaching experiences during COVID-19. Early
Childhood Education Journal, 29, 1-11. https://doi.org/10.1007/s10643-022-01316-3
Kastens, K. (2004). Making DLESE into the source of pedagogical knowledge pertaining
to the earth and the environment. White paper. http://www.dlese.org/
swikis/quality/uploads/1/Geo_PCK_source.pdf
Keiler, L. (2018). Teachers’ roles and identities in student-centered classrooms.
International Journal of STEM Education, 5(34). https://doi.org/10.1186/s40594-018-
0131-6
Klassen, M., & Kim, E. (2019). Selecting teachers and prospective teachers: A meta-
analysis. Educational Research Review, 26, 32-51.
https://doi.org/10.1016/j.edurev.2018.12.003
Lee, J., & walker, A. (2000). Per-service primary teachers’ perceptions about principals in
Hong Kong: implications for teacher and principal education. Asia– Pacific Journal
of Teacher Education, 28 (1), 53-67. https://doi.org/10.1080/135986600109444
Luis E., Akrivou, K., Bermejo-Martins, E., Scalzo, G., & Orón J. (2022) The
Interprocessual-self Theory in Support of Human Neuroscience Studies. Front. In
Psychol. 12, 686928. https://doi.org/10.3389/fpsyg.2021.686928
442
Maes, O., Van Nieuwenhoven C., & Colognesi S. (2022) The feedback given by university
supervisors to student teachers during their co-assessment meetings. Frontiers
Education, 7. 848547. https://doi.org/10.3389/feduc.2022.848547
Makovec, D. (2018). The teacher's role and professional development. International
Journal of Cognitive Research in Science Engineering and Education, 6(2), 33-45.
https://doi.org/10.5937/ijcrsee1802033M
Miqdadi, A. (2003). Evaluation of the practical education program for mathematics
teacher preparation at the University of Jordan. Dirasat of Educational Sciences,
30(2), 314-329.
Mutlu, G. (2014). Challenges in practicum: Pre-service and cooperating teachers’ voices.
Journal of Education and Practice, 5(36), 1-7.
https://doi.org/10.15804/tner.2017.49.3.20
Nagro, S., & deBettencourt, L. (2019). Reflection activities within clinical experiences: An
important component of field-based teacher education. In T. E. Hodges & A. C.
Baum (Eds.), The handbook of research on field-based teacher education (pp. 565–586).
IGI Global. https://doi.org/10.4018/978-1-5225-6249-8.ch024
Nagro, S., Hirsch, S., & Kennedy, M. (2020). A self-led approach to improving classroom
management practices using video analysis. Teaching Exceptional Children, 53(1),
24–32. https://doi.org/10.1177/0040059920914329
Nathans, L., & Brown, A. (2022). Differences in pre-Service teacher attitude change about
family involvement across four universities. Societies, 12(65).
https://doi.org/10.3390/soc12020065
O’Doherty, D., Dromey, M., Lougheed, J., Hannigan, A., Last, J., & McGrath, D. (2018).
Barriers and solutions to online learning in medical education—an integrative
review. (Report). BMC Medical Education, 18(1), 130–
141. https://doi.org/10.1186/s12909-018-1240-0
Odeh, A. (2018). Measurement and evaluation in the teaching process. (10th ed.). Dar Al-Amal
for Publishing and Distribution.
Office of Teacher Education. (2022). Clinical supervisors and cooperating teachers:
Minnesota requirements for university of supervisors and cooperating teachers.
https://academics.cehd.umn.edu/teaching-handbook/clinical-supervisors-and-
cooperating-teachers/
O'Sullivan, M., & Conaill, A. (2022). ‘You’re kind of taking this person under your
wing…’: The experiences of cooperating primary teachers in engaging with
student teachers during school placement in the Midwest of Ireland. Student
Engagement and Partnership in Irish Higher Education, 4(1).
https://ojs.aishe.org/index.php/aishe-j/article/view/611/971
Özdemir-Yılmazer, M. (2021). Supervision beliefs in cooperating teacher-university
supervisor dyad: implications for reflective dialogue to strengthen partnership.
Bartın University Journal of Faculty of Education, 10(2), 232-245.
https://doi.org/10.14686/buefad.774178
Panadero, E., & Lipnevich, A. (2022). A review of feedback models and typologies:
Towards an integrative model of feedback elements. Educational Research Review,
35, 100416. https://doi.org/10.1016/j.edurev.2021.100416
Payant, C., & Murphy, J. (2012). Cooperating teachers’ roles and responsibilities in a
MATESOL practicum. TESL Canada Journal, 29(2).
https://www.academia.edu/9257018/Cooperating_Teachers_Roles_and_Respons
ibilities_in_a_MATESL_Practicum
Podgornik, V., Vogrinc, J. (2017). The role of headteachers, teachers, and school
counselors in the system of quality assessment and assurance of school
work. SAGE Open, 7(2), 21582.
443
Poom-Valickis, K., Oder, T., Lepik, M. (2012) Teachers’ beliefs regarding their
professional role: a gardener, lighthouse or circus director? Procedia - Social and
Behavioral Sciences, 69: 233-241. https://doi.org/10.1016/j.sbspro.2012.11.404
Radovan, M. (2011) The relation between distance students' motivation, their use of
learning strategies, and academic success. Turkish Online Journal of Educational
Technology, 10(1), 216- 222. https://files.eric.ed.gov/fulltext/EJ926571.pdf.
Radovan, M., Kristl, N. (2017) Acceptance of technology and its impact on teacher's
activities in virtual classroom: integrating UTAUT and CoI into a combined
model. Turkish Online Journal of Educational Technology, 16(3), 11-22.
http://www.tojet.net/volumes/v16i3.pdf
Rodrigues, L., de Pietri, E., Sanchez, H., Kuchah, K. (2018). The role of experienced
teachers in the development of pre-service language teachers' professional
identity: Revisiting school memories and constructing future teacher
selves. International Journal of Educational Research, 88, 146-155.
https://doi.org/10.1016/j.ijer.2018.02.002
Sawyer, G. B. (2005). A study using the star teacher selection interview to predict the successful
performance of teachers in South Carolina's program of alternative certification for
educators. Unpublished doctoral dissertation, Columbia, SC: University of South
Carolina.
Singha, S., & Sikdar, D. (2018). Professional development of teacher and professionalism
in teacher education. International Journal of Applied Social Science, 5 (8), 1320-1332.
https://www.researchgate.net/publication/357714097
Stafford, M., McMunn, A., Zaninotto, P., and Nazroo, J. (2011). Positive and negative
exchanges in social relationships as predictors of depression: evidence from the
English longitudinal study of aging. Journal of Aging Health 23, 607–628.
http://doi.org/10.1177/0898264310392992
Sung, Y. (2007). Are pre-service teachers constructivist in the constructivist teacher
education program? Korean Educational Development Institute Journal of Educational
Policy, 4(1), 9-24.
Tamir, D. I., and Hughes, B. L. (2018). Social rewards: from basic social building blocks
to complex social behavior. Perspect. Psychology Sciences, 13, 700–717.
http://doi.org/10.1177/1745691618776263
Tasdemir, M., Iqbal, M., & Asghar, M. (2020). A study of the significant factors affecting
pre-service teacher education in Turkey. https://eric.ed.gov/?id=EJ1258047
Thema, S., Rosenkränzer, F., Hörsch, C., Schuler, S., & Rieß, W. (2022). Student teachers’
pedagogical content knowledge for teaching systems thinking: effects of different
interventions. International Journal of Science Education, 39(215):1-20.
http://doi.org/10.1080/09500693.2017.136260
Verde A., & Valero J. (2021) Teaching and learning modalities in higher education
during the pandemic: responses to coronavirus disease 2019 From Spain. Frontiers
in Psychology. 12, 648592. http://doi.org/10.3389/fpsyg.2021.648592
Watson, A., Timperio, A., Brown, H., Best, K., & Hesketh, K. (2017). Effect of classroom-
based physical activity interventions on academic and physical activity outcomes:
A systematic review and meta-analysis. International Journal of Behavioral Nutrition
and Physical Activity, 14(1), 1–24. https://doi.org/10.1186/s12966-017-0569-9
Zanting, A., verloop, N., & Vermunt, J. (2003) Using interviews and concept maps to
access Mentor teachers’ practical knowledge. Higher education, 46(2), 195-214.
Zega, T., & Lase, D. (2021). How can teachers engage students in online learning? A
conceptual framework. http://doi.org/10.35542/osf.io/qj9br
Zhang, W., Wang, Y., Yang, L., & Wang, C. (2020). Suspending classes without stopping
learning: China’s education emergency management policy in the COVID-19
outbreak. Journal of Risk and Financial Management, 13(58), 1–
6. https://doi.org/10.3390/jrfm13030055
444
Zhao, H., & Zhang, X. (2017). The influence of field teaching practice on pre-service
teachers' professional identity: a mixed methods study. Frontiers in psychology, 8,
1264. https://doi.org/10.3389/fpsyg.2017.01264.
Zhilgildinova M., Abibulayeva A., Sultanova N., Yedigenova ., Seksenbayev N., Alim A.,
& Robso W., (2022). Stimulating the professional and personal self-development of
future teachers in the context of value-semantic orientation. Education Research
International. https://doi.org/10.1155/2022/8789773
Appendix 1
Item very high moderate low very

My assessment of the effectiveness of the high low
No.
performance of cooperative teachers is as
follows:
1 Assigning the student teacher to perform
some teaching tasks.
2 Assigning the student teacher for planning.
3 Supervising and monitoring the student
teacher about his/her training role.
4 Informing the student teacher of the
importance of his/her teaching role
5 Monitoring the student teacher lesson
preparation notebook.
6 Directing the student teacher to attend
typical classes with more than one teacher.
7 Watching lessons for the teacher/student.
8 Providing the student teacher with
opportunities to use educational
technologies in teaching.
opportunities to attend a variety of teaching
models.
10 Giving the student teacher a chance to
apply various alternative assessment
strategies.
11 Informing the student teacher of
contemporary trends in developing
teaching performance.
12 Choosing practical lessons according to
certain standards.
13 Encouraging the student teacher to use
modern teaching methods.
opportunities to participate in the creation
of educational aid means.
15 Monitoring the performance of the student
teacher in the classroom environment.
opportunities to participate in social
activities.
17 Simplifying the student teacher's task in
conducting activities outside the classroom.
445
Item very high moderate low very

My assessment of the effectiveness of the high low
No.
performance of cooperative teachers is as
follows:
18 Participating in supervisory meetings
between the university supervisor and the
student teachers.
19 Attempting to understand the problems
that the student teacher faces in teaching.
opportunities to participate in school
activities.
21 Helping the student teacher solve his/her
problems.
22 Introducing the student teacher to the laws
of managing the educational environment.
23 Providing a school environment concerned
with creating human relations with the
teacher/student.
24 Providing the necessary materials and tools
for the student teacher to produce
educational aids.
25 Treating the student teacher as a part of the
teaching system.
feedback on his/her teaching performance.
PUBLISHER International Journal of Learning, Teaching and
Society for Research and Knowledge Management Educational Research
Port Louis
Republic of Mauritius
www.ijlter.org The International Journal of Learning, Teaching
and Educational Research is an open-access
Chief Editor journal which has been established for the dis-
Dr. Antonio Silva Sprock, Universidad Central de semination of state-of-the-art knowledge in the
Venezuela, Venezuela, Bolivarian Republic of field of education, learning and teaching. IJLTER
welcomes research articles from academics, ed-
Editorial Board
ucators, teachers, trainers and other practition-
Dr. Fatima Zohra Belkhir-Benmostefa
ers on all aspects of education to publish high
Dr. Giorgio Poletti
quality peer-reviewed papers. Papers for publi-
Dr. Chi Man Tsui
cation in the International Journal of Learning,
Dr. Fitri Suraya Mohamad
Teaching and Educational Research are selected
Dr. Hernando Lintag Berna
through precise peer-review to ensure quality,
Dr. Charanjit Kaur Swaran Singh
originality, appropriateness, significance and
Dr. Abu Bakar
readability. Authors are solicited to contribute
Dr. Eglantina Hysa
to this journal by submitting articles that illus-
Dr. Mo'en Salman Alnasraween
trate research results, projects, original surveys
Dr. Hermayawati Hermayawati
and case studies that describe significant ad-
Dr. Selma Kara
vances in the fields of education, training, e-
Dr. Michael B. Cahapay
learning, etc. Authors are invited to submit pa-
Dr. Bunmi Isaiah Omodan
pers to this journal through the ONLINE submis-
Dr. Vassiliki Pliogou
sion system. Submissions must be original and
Dr. Meera Subramanian
should not have been published previously or
Dr. Muhammad Kristiawan
be under consideration for publication while
Dr. Wahyu Widada
being evaluated by IJLTER.
Dr. Som Pal Baliyan
Dr. Reem Khalid Abu-Shawish
Dr. Froilan Delute Mobo
Dr. Mohamed Ali Elkot
Dr. Anabelie Villa Valdez
Mr. Teody Lester Verdeflor Panela

Vol 21 No 8 August 2022

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Vol 21 No 8 August 2022

Uploaded by

Copyright:

Available Formats

IJLTER.

Society for Research and Knowledge Management

The International Journal of Learning, Teaching and Educational

Aims and Objectives

The main objective of this journal is to provide a platform for educators,

Indexing and Abstracting

The International Journal of Learning, Teaching and Educational

We are very happy to publish this issue of the International Journal of

The International Journal of Learning, Teaching and Educational

We are grateful to the editor-in-chief, members of the Editorial Board

The Editorial Board is composed of renowned people from across the

We will endeavour to ensure the reputation and quality of this journal

Editors of the August 2022 Issue

International Journal of Learning, Teaching and Educational Research

Mathematical Knowledge for Teaching in

Jogymol Kalariparampil Alex

Abstract. This paper investigates entry-level student teachers'

Keywords: Mathematical knowledge; student teachers’ entry-level;

preparation for abstract mathematics in higher education. In this regard, the

Specifically, it sought an answer to the following questions:

2. Literature review and theoretical framework

These researchers further explained that most of the in-service mathematics

Shulman makes a theoretical distinction between pedagogical content knowledge

• Knowledge of content and students (KCS) was another sub-construct that

2.2. Educational assessment

Assessment is a process of collecting, analysing and interpreting information to

and adapting their instruction to enhance students' achievement to achieve more

Baseline assessment commonly used in early childhood education gathers

assessments assist schools in understanding the students' requirements. It also aids

Table 1: The weight of content areas description of FET’s mathematics topics

3.2 Data collection

3.3. Data analysis

Figure 2: assessment items no. 9 and 16 (source: www.cami.co.za)

Figure 3: assessment items no. 20 and 25 (source: www.cami.co.za)

According to international benchmarks, 60 per cent was used as the

According to the benchmarking, "Substantial achievement" was the minimum

Table 4: One-way ANOVA single factor

According to the above findings, student teachers scored better in Grade 10

Figure 5: Frequency distribution of the student teachers’ achievements in Grades 10,

result, student teachers' entry-level mathematical content knowledge in the FET

4.2. Level of understanding of student teachers’ mathematical content

Table 5: The understanding level of student teachers’ mathematical content knowledge

for teaching Grade 10 Measurement, Grade 11 Patterns, Functions, Algebra, and

According to studies, the primary purpose of a baseline assessment in teaching

at an early stage so that a remediation program can be implemented (DBE, 2019).

According to Deacon (2016), the entrance requirements for Initial Teacher

Due to the low mathematics achievement of students entering teacher education

According to Ma (1999), teachers should possess a Profound Understanding of

Consequently, student teachers will need to improve their ability to teach

educators in HEIs, student teachers must gain a thorough understanding of the

8. Implications and contribution of the study

9. Limitations of the study

Tymms, P. (2013). Baseline Assessment and Monitoring in Primary schools: Achievement,

International Journal of Learning, Teaching and Educational Research

Exploring the Use of Chemistry-based Computer

Abstract. This study aimed at exploring the instructional activities that

Keywords: chemistry-based computer simulations; instructional

Chemistry includes abstract concepts such as chemical kinetics, equilibrium and

Chemistry-based computer simulations and animations are examples of an

Despite Tanzania's government's initiatives to integrate ICT into classrooms, little

2.2 The importance of instructional activities and development of science

students in the construction of knowledge and the acquisition of skills.

2.3 Theoretical Framework