Vol 22 No 10 October 2023

IJLTER.
ORG
International Journal
of
Learning, Teaching
And
Educational Research
p-ISSN: 1694-2493
e-ISSN: 1694-2116
Vol.22 No.10
International Journal of Learning, Teaching and Educational Research
(IJLTER)
Vol. 22, No. 10 (October 2023)
Print version: 1694-2493
Online version: 1694-2116
IJLTER
International Journal of Learning, Teaching and Educational Research (IJLTER)
Vol. 22, No. 10
This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part
of the material is concerned, specifically those of translation, reprinting, re-use of illustrations,
broadcasting, reproduction by photocopying machines or similar means, and storage in data banks.
Society for Research and Knowledge Management

The International Journal of Learning, Teaching and Educational

Research is a peer-reviewed open-access journal which has been
established for the dissemination of state-of-the-art knowledge in the
fields of learning, teaching and educational research.
Aims and Objectives
The main objective of this journal is to provide a platform for educators,

teachers, trainers, academicians, scientists and researchers from over the
world to present the results of their research activities in the following
fields: innovative methodologies in learning, teaching and assessment;
multimedia in digital learning; e-learning; m-learning; e-education;
knowledge management; infrastructure support for online learning;
virtual learning environments; open education; ICT and education;
digital classrooms; blended learning; social networks and education; e-
tutoring: learning management systems; educational portals, classroom
management issues, educational case studies, etc.
Indexing and Abstracting

Research is indexed in Scopus since 2018. The Journal is also indexed in
Google Scholar and CNKI. All articles published in IJLTER are assigned
a unique DOI number.
Foreword
We are very happy to publish this issue of the International Journal of

Learning, Teaching and Educational Research.

Research is a peer-reviewed open-access journal committed to
publishing high-quality articles in the field of education. Submissions
may include full-length articles, case studies and innovative solutions to
problems faced by students, educators and directors of educational
organisations. To learn more about this journal, please visit the website
http://www.ijlter.org.
We are grateful to the editor-in-chief, members of the Editorial Board

and the reviewers for accepting only high quality articles in this issue.
We seize this opportunity to thank them for their great collaboration.
The Editorial Board is composed of renowned people from across the

world. Each paper is reviewed by at least two blind reviewers.
We will endeavour to ensure the reputation and quality of this journal

with this issue.
Editors of the October 2023 Issue

VOLUME 22 NUMBER 10 October 2023
Table of Contents
Assessing the Effects of Flipped Classroom to the Primary Pupils’ English Learning Performance........................... 1
Thavamani Parati, Mohd Nihra Haruzuan Mohamad Said, Mohd Fadzil Abdul Hanid
Knowledge Management Using Storytelling with Infographics to Develop Creativity of Undergraduate Students
.................................................................................................................................................................................................... 18
Virawan Amnouychokanant
“I find it very difficult to go to work; it is emotionally exhausting”: Understanding the Burnout and Underlying
Emotions among Malaysian University Academics...........................................................................................................37
Fairuz A'dilah Rusdi, Ateerah Abdul Razak, Zaleha Embong
Polymer Science in Action: Transforming the Learning Experience for Undergraduates with Active Learning
Strategies................................................................................................................................................................................... 54
Chaninan Pruekpramool, Theerapong Sangpradit, Panitarn Wanakamol, Supitcha Supansomboon
Assessment of Learning about Sustainability in Students with Down Syndrome........................................................ 79

Cristina Arranz Barcenilla, Sara Gutiérrez-González, María Consuelo Saiz Manzanares, Lourdes Alameda Cuenca-Romero,
Sarah Vandekerkhof
A Systematic Review of Interventions Improving University Students’ EFL Writing Competence.......................... 93

Dandan Zhang, Joanna Tjin Ai Tan, Swagata Sinha Roy
Nature of Science: A Comparative Analysis of the High School Physics Textbooks in Indonesia and Korea........113
Hartono Bancong, Sukmawati ., Nursalam ., Danilo Jr. Tadeo
Exploring the Potential of Integrating Local Wisdom into the Development of Pocket Book Learning Media: A
Systematic Literature Review.............................................................................................................................................. 130
Sukadari ., Mahilda Dea Komalasari, Nina Widyaningsih, Gulzhaina K Kassymova4, Fang Yuqi, Lily Muliana Mustafa,
Nurudeen Babatunde Bamiro
Teacher Training’s Content and Delivery Method Related to Augmentative and Alternative Communication
(AAC): A Systematic Literature Review (SLR)..................................................................................................................152
Sook Wei Loi, Syar Meeze Mohd Rashid, Hasnah Toran
Analysis of Vocational Student Performance Criteria on Work Skills Based on Industry Needs: An Analysis for
Students' Skill Test Instruments.......................................................................................................................................... 174
Rolly Robert Oroh, Muhammad Muhdi Attaufiq, Metsi Daud, Rocky Frangky Roring
Exploring Moodle Usage in Higher Education in the Post-pandemic Era: An Activity-theoretical Investigation of
Systemic Contradictions....................................................................................................................................................... 190
Ishaq Salim Al-Naabi
Collaborative Learning in Higher Education in the Fourth Industrial Revolution: A Systematic Literature Review
and Future Research..............................................................................................................................................................209
Adi Bandono, Mukhlis Mukhlis, A. K. susilo, A. R. Prabowo, Acep Maksum
The Impact and Challenges of the Implementation of a High-Impact ePortfolio Practice on Graduate Students’
Learning Experiences............................................................................................................................................................ 231
Hawazen Alharbi
Inclusion of Metaverses in the Development of the Flipped Classroom in the University environment:
Bibliometric Analysis of Indexed Scientific Production in SCOPUS............................................................................. 247
Omar Chamorro-Atalaya, Víctor Durán-Herrera, Raul Suarez-Bazalar, Constantino Nieves-Barreto, Julio Tarazona-Padilla,
Milagros Rojas-Carbajal, Yreneo Cruz-Telada, Juan Caller-Luna, Ronald Alarcón-Anco, José Antonio Arévalo-Tuesta
Factors Contributing to Resistance in the use of Information and Communications Technology: A Snapshot on
Engineering Graphics and Design Teachers......................................................................................................................271
Philani Brian Mlambo, Mogale Simon Albert Maeko, Samuel Dumazi Khoza
Investigating the Impact of Teaching and Learning of Religious Institution [pondok] on the Society in Kelantan,
Malaysia.................................................................................................................................................................................. 290
Nik Yusri Musa, Ateerah Abdul Razak, Amanina Abdul Razak Mohamed, Asma Lailee Mohd Noor, Mohd Zain Mubarak,
Nur Azuki Yusuff, Noor Hisham Md Nawi, Marwan Ismail, Azhar Muhammad
A Comprehensive Approach to Eliminate English Second Language Learners’ Grammatical Difficulties............306

Nomasomi Hilda Matiso
Implementation of Virtual Worlds to Promote Distance Practice Teachers' Participation in the English Learning
Process.....................................................................................................................................................................................324
Ana Quinonez-Beltran, Carmen Benitez-Correa, Elsa Morocho-Cuenca
TikTok as a Source of English Language Content – Perceived Impacts on Students’ Competence: Views from
Indonesia.................................................................................................................................................................................340
Fathor Rasyid, Hidayatul Hanjariyah, Nurul Aini
Acquisition of Communicative Skills by Foreign Students in a Multicultural Learning Environment................... 359

Ran Cao, Lyaziza Sarsenbayeva
De/colonising Theoretical Literatures and the Educational Qualifications to Unpack the Grotesque Skills Gap in
South Africa............................................................................................................................................................................378
Christiana Kappo-Abidemi, Christopher Babatunde Ogunyemi
Comprehensive Academic Thesis Writing Module for English Major Undergraduates in a Public University in
China....................................................................................................................................................................................... 394
Gao Jing, Lin Siew Eng
Assessment Strategies in Outcome-Based Education: Preferences and Practices Among University Lecturers in
Vietnam................................................................................................................................................................................... 416
Phuong Hoang Yen, Nguyen Anh Thi, Le Thanh Thao, Nguyen Huong Tra, Pham Trut Thuy, Huynh Thi Anh Thu
Unlocking Classroom Potential: Exploring the Mediating Role of Teacher Mindset on Embracing Differentiated
Instruction...............................................................................................................................................................................433
Enung - Hasanah, M Ikhsan Al Ghazi, Suyatno Suyatno, Ika Maryani, Mohd Zailani Mohd Yusoff
Applying Technical Science Process in Teaching STEM Topics under Model School with Production Practice and
Social Life in Vietnam........................................................................................................................................................... 453
Dang Thi Thuan An, Dang Thi Thuan An, Huynh Van Son, Huynh Van Son, Pham Ngo Son, Pham Ngo Son, Nguyen
Mau Duc
1

Vol. 22, No. 10, pp. 1-17, October 2023
https://doi.org/10.26803/ijlter.22.10.1
Received Aug 24, 2023; Revised Oct 13, 2023; Accepted Oct 19, 2023
Assessing the Effects of Flipped Classroom to the

Primary Pupils’ English Learning Performance
Thavamani Parati , Mohd Nihra Haruzuan Mohamad Said and

Mohd Fadzil Abdul Hanid
School of Education
Faculty of Social Science and Humanities
Universiti Teknologi Malaysia, Johor Bahru, Malaysia
Abstract. In this era, implementing technology in teaching and learning

process without affecting traditional teaching method is very important
in the 21st-century education system, which can prevent pupils’ from
being outdated, as well as developing their thinking skills. Pupils in
primary schools are having difficulty acquiring the fundamental abilities
in the English language, encompassing speaking, listening, reading, and
writing proficiencies. Therefore, it is crucial to identify effective strategies
for enhancing the process of learning the English language within
Malaysia's education system. This research concentrated on assessing the
impact of Flipped Classroom learning on the English language
proficiency of primary school students, specifically in four core language
skills. In this quantitative analysis, 31 Year 3 students from a primary
school located in the Kulai district were chosen through purposive
sampling as participants in the study. Data collection involved surveys,
pre-tests, and post-tests. The results indicated a moderately positive
correlation between overall performance on these tests and a positive
influence on Year 3 students' English skills in listening, speaking, reading,
and writing. Additionally, students showed a strong positive acceptance
of the Flipped Classroom learning method for these four essential English
skills. However, it was found that the Flipped Classroom approach did
not significantly alter the students' preferred learning methods. These
findings suggest that the Flipped Classroom offers a valuable addition to
the education sector, providing an alternative learning approach to
enhance second language education and improving English language
learning.
Keywords: Flipped Classroom; Listening; Speaking; Reading; Writing;

Performance; Achievement
1. Background of the study

Education is basic human needs which can produce individuals who are
intellectually, spiritually, emotionally, physically, and socially balanced and
©Authors
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0
International License (CC BY-NC-ND 4.0).
2
harmonious. However, in today's constantly changing environment, the

traditional industrial-era educational paradigms are no longer relevant.
Educational strategies should not only adapt, but transform to align the digital
age to assist our young people to becoming the adults who are successful in the
21st century. Creating a 21st-century education system entails that all pupils are
sufficiently equipped to thrive in a competitive environment, where highly
talented individuals have many opportunities while the rest of them have few.
The focus is on harnessing the potential of technology to enhance 21st-century
competencies, facilitate innovative teaching and learning, and establish robust
educational support systems (Vockley, 2007). One of the objectives outlined in the
Ministry of Education, (2012) is to guarantee that every Malaysian has fair and
equal opportunities to receive a top-notch education that meets international
standards, as well as to use Information and Communication Technology (ICT) to
scale up quality learning across the country.
However, despite of the digital age, we cannot avoid the traditional (face-to-face)
instructional method which still prefer and familiar by the pupils and teacher
nowadays which has a practice for thousands of years. Teachers for 21st century
should be more creative and careful enough in choosing suitable and effective
methods for teaching and learning, and at the same time, reaching the pupils’
satisfaction. One of the best and effective methods, which is very popular is the
reverse learning method called “Flipped Classroom”. Flipped classroom method
is created by Jonathan Bergmann and Aaron Sams in the early days to provide
teaching and learning to pupils who do not attend school or face-to-face classes
(Bergmann & Sams, 2008). According Paez-Quinde et al., (2022), the flipped
classroom is an educational approach where students acquire new knowledge
outside of the class, as opposed to the traditional method where learning occurs
during class time. Its approach emphasizes on scheduling class time for activities,
problem-solving and other forms of instruction. On the other hand, Ruiz-Jiménez
et al., (2022) said that the flipped classroom method is a new pedagogical model
where the teachers share pre-established digital resources with pupils through a
digital platform outside of the classroom and related content taught
asynchronously.
Besides that, the flipped classroom approach helps in the creation of active
learning environment or classroom (Gustian et al., 2023; Siegle, 2014). This
learning method encourages active participation by creating a workshop-like
environment in which pupils can ask questions regarding their lessons and
engage in hands-on activities with them. It is the most approachable and simple
method of implementing technology in teaching and learning process without
affecting traditional teaching methods, which hold the direct contact between the
teachers and students, as well as between learners themselves, called the
backbone of education (Shabibi et al., 2017). Due to the COVID-19 pandemic in
2020, the Malaysian government had issued a Movement Control Order (MCO)
to manage the pandemic in an orderly manner, which led to all schools needed to
be closed. On the other hand, teachers are expected to complete the syllabus or
curriculum. As a result, the flipped classroom is strongly suggested as the best
way to resolving the problem. It helps to uplift pupils' basic understanding about
http://ijlter.org/index.php/ijlter
3
a content and let them to create new knowledge independently by doing mastery
learning with those digital materials before attending face-to-face class time after
the school reopens. Studies show that flipped classroom had proven that it helped
to motivate and increase pupils’ performance (Elian & Hamaidi, 2018; Graham
Brent Johnson, 2013).
The research gap lies in the absence of a systematic approach that integrates
flipped classroom learning and digital resources to facilitate the development of
21st-century skills, particularly the 4C's (Critical thinking, Creativity,
Collaboration, and Communication). Despite the potential of this approach to
enhance student engagement and performance (Robinson et al., 2019), challenges
such as students' limited exposure to technology and the need for tailored
materials to accommodate varying cognitive levels remain unaddressed.
Additionally, the study aims to explore the potential of flipped classroom learning
to establish stronger teacher-student communication and foster active learning,
which is crucial in a second language learning context (Wang & Yamat, 2019). By
addressing these gaps, this research endeavors to contribute valuable insights into
the effective utilization of flipped classroom learning in the Malaysian primary
education system, ultimately enhancing students' English language proficiency
and overall academic performance.
The main aim of this research is to determine the impacts of the flipped classroom
on Year 3 pupils in terms of their English skills and how it impacts their
performance and acceptance. Primary school pupils, especially in Malay schools
are facing challenges in learning English as the second language due to the
influence of their mother tongue, limitation of speaking ability, ignorance,
confusion with the language and so on. To overcome this issue, teachers should
engage pupils in learning the language constantly by using various and effective
approaches which may change the pupils’ perception positively about learning
the second language and abolish the limitation of using the language among them
by creating a quality learning process at school, as well as home. The factors that
cause pupils’ learning performance are such as, the form of questions, pupils'
perspectives toward examinations, poor learning environment, teachers' quality
and pupils' learning styles.
1.1 Conceptual Framework

This research had been implemented and chose appropriate theories and model
as a guide to identify the effectiveness of flipped classroom learning. Thus, the
framework for this research, is a revised version of Bloom’s Taxonomy and had
been mainly used as a lens for transmission of information, the Mastery learning
model, Cognitive Constructivism and Social Constructivism theories were used
to explain how learning occurs to master each level according the Taxonomy.
Figure 1 below is the conceptual framework of this study.
4
Figure 1: Synthesis of the models and theories associated with Flipped Learning
1.2 Scope of Research

This research is vital as it assesses the impact of the flipped classroom on primary
pupils' English learning performance. Mastering the English language, especially
in National schools, is challenging for primary students. Teachers must be
innovative in changing the learning environment and strategies to suit students'
cognitive levels and align with Malaysia's education system. The flipped
classroom combines traditional and modern learning approaches to help students
consistently develop their cognitive abilities using technology and traditional
tools. This approach ensures students remain relevant in the 21st century.
Additionally, it assists teachers in planning well-organized learning processes,
ultimately boosting students' English performance.
1.3 Implications and Limitations of the Research

The implications of this study in the realm of language education, especially in
English language learning within primary schools, are substantial. The Flipped
Classroom method demonstrates potential in improving student engagement and
proficiency across critical language skills, including listening, speaking, reading,
and writing.
This research will involve a small sample limited to Year 3 pupils studying at a
national primary school in the Kulai District. Therefore, the research findings are
constrained to the sample group selected based on their similar background and
English learning level, solely for assessing their viewpoint regarding the Flipped
Classroom approach and how it influences their academic results
2. Literature Review
Previous findings of researchers regarding those issues and theories aligned with
flipped classroom method in learning English had been discussed and used as a
reference to lead this research more efficiently. Both educators and students need
to acquire a fresh or broader set of skills, wherein the teacher designs interactive
learning experiences to involve pupils beyond the traditional classroom setting
and pupils are responsible for independently exploring resources outside of class,
attempting to obtain basic information before class, and then actively applying
that knowledge in the classroom (Brewer & Movahedazarhouligh, 2018).
5
Flipped classroom method in teaching and learning process will lead the pupils
to do independent learning by applying project base activities. It will encourage
the pupils to search and read more resources besides text books or notes prepared
by the teachers. Through this way, they will learn how to organise knowledge,
develop presentation skills, increase their synthesize ideas, develop their
communication skills and more. Other than that, flipped classroom learning
explained four pillars of flip which makes it a more adaptable setting, conducive
learning atmosphere, purposeful curriculum, and skilled instructor (Flipped
Learning Network (FLN), 2014). Flipped classroom method bring a flexible
environment to the pupils, which make a comfortable space for them to learn at
home using various type of sources according to their flexible learning timeline.
The main principle in 21st-century learning in line with School Transformation

Programme 2025 are: critical thinking, collaborative, creativity, communication,
citizenship, character and one of the pedagogies is digital tools and resources.
Thus, the flipped classroom method contained all the requirement of MOE to
develop 21st-century education system. Flipped classroom method received
positive responses and helped to improve the English grammar (Shaari et al.,
2021; Vuong, 2023). Learning English, especially for pupils which English is not
their native language, needs more guidance and effective methods to deliver the
knowledge. Flipped learning is a versatile approach that can be effectively
employed at any educational level (Fatimah Abd Rahman et al., 2019).
According Mandasari & Wahyudin, (2021), flipped classroom brings satisfaction

to the pupils and easier to conduct learning process while providing a chance to
create an independent learning environment along with improving English
grammar knowledge. Learning English needs more practice with variety of study
materials. Hence, the flipped classroom method gives a chance to pupils to learn
independently using various resources that help them to understand better, faster
and clear. Flipped learning method also helps to improve pupils’ verbal English
communication skill (Tazijan et al., 2017). Besides that, Su Ping et al., (2020)
proved that flipped classroom improves pupils’ English writing performance by
fully engaging with them and increase their motivation level.
According to a study carried by Teo Woon Chun & Ramesh Sathappan, (2018), it
is proved that flipped classroom learning method shows a positive perception
among pupils, which provides an effective environment for them to communicate
and collaborate with their peers and masters in particular skills taught by the
teachers. Furthermore, pupils shared their perception about flipped classroom
learning method, which is by creating a situation that the teachers direct to them
as well as motivate, engage and structure their learning process (Cueva & Inga,
2022; Haghighi et al., 2019). According to Pavanelli, (2018), the flipped classroom
approach is seen by students as a valuable educational resource and this method
has enhanced their writing abilities within an engaging, cooperative environment.
Those interactive learning resources prepared give an opportunity to them to use
and explore the technology-based tools and learning materials. Moreover, those
resources and classroom activities also bring the pupils to share knowledge
6
between their peers, help one another and turn it into a collaborative learning
environment. The pupils’ communication skills also improve tremendously here.
The flipped classroom learning approach proves to be a valuable tool for

enhancing students' foundational English skills, including listening, speaking,
reading, and writing. Yousufi (2020) emphasizes the remarkable impact of the
flipped classroom method on improving students' listening abilities, suggesting
that practicing listening activities and maintaining consistent engagement are
essential for skill enhancement. Sudarmaji (2021) also affirms the effectiveness of
the Flipped Classroom in bolstering students' English speaking skills by
promoting increased communication between teachers and peers, fostering
fluency and interactivity.
Ozturk (2021) further supports the use of the flipped classroom method, noting
its ability to motivate and involve students in reading activities. The tasks and
activities in this approach require students to independently read and follow
provided instructions, thereby encouraging more reading and vocabulary
acquisition. Additionally, Siswanto (2021) underscores that writing skills benefit
significantly from the flipped classroom, as it enhances students' participation,
confidence, and enthusiasm for learning. By providing diverse materials and
approaches and boosting students' interest and confidence in writing tasks, the
flipped classroom effectively nurtures and develops their English writing
abilities.
3. Methodology
The methodology employed by the researcher was discussed in detail, beginning
with the research design, the study's participants, research instruments, data
collection, and data analysis. The researcher chose a quantitative approach for this
research because the primary aim was to examine the impact of the flipped
classroom on English language learning by assessing pupils' performance based
on their test scores and gauging their perceptions of the learning method through
questionnaire responses.
3.1 Research Design

The study's research methodology heavily emphasizes a quantitative approach,
employing both test scores and survey responses as primary data sources. The
quantitative aspect of the research involves the administration of tests
meticulously crafted to assess pupils' performance across the four core language
competencies: listening, speaking, reading, and writing. Aligned with the Year 3
English syllabus, these tests consist of a total of 40 marks, predominantly
comprising multiple-choice questions. By utilizing these test scores, the research
aims to quantitatively measure the influence of the flipped classroom method on
pupils' English language learning outcomes, providing concrete data to evaluate
its effectiveness. Additionally, this study adopts a quasi-experimental design,
utilizing a single group of participants to assess the impact of the intervention.
In tandem with the quantitative assessment, the research incorporates a survey

instrument, designed in the form of questionnaires, to collect invaluable insights
7
into pupils' perceptions following their engagement with the flipped classroom
approach. These questionnaires are thoughtfully structured to capture pupils'
feedback and perspectives regarding the flipped classroom method in English
language learning. By analysing the quantitative data from the tests alongside the
qualitative data gathered from the surveys, the study provides an extensive
assessment of the influence and acceptance of the flipped classroom model within
the realm of language education in primary schools.
In this study, the instructional design model implemented by the researcher was
ADDIE model, which was developed by (Rossett, 1987) and used as a systematic
work guide for the purpose of developing learning activities of flipped classroom
learning method. The utilization of the ADDIE model process for product creation
continues to stand out as one of the successful and efficient approaches because
the process serves as a guiding framework for complicated circumstances plus
suitable for developing educational products and other learning resources
(Branch, 2010). The ADDIE model comprises five sequential stages in the work
process: Analysis, Design, Development, Implementation, and Evaluation.
3.2 The participants of the study

The group samples were selected using a purposive method. A total of 31 third-
grade students from a primary school located in the Kulai district were selected
using purposive sampling to participate in this quantitative research. They were
randomly selected from Year 3 pupils in the primary school who demonstrated
similar levels of proficiency, standards, and ICT skills.
3.3 Research Instruments

The test questions were designed based on four fundamental language
competencies: listening, speaking, reading, and writing, in accordance with the
Year 3 English syllabus. This section of the test comprised a total of 40 marks and
included multiple-choice questions. Additionally, a survey instrument was
created by the researcher in the form of questionnaires to gather data on the
pupils' perceptions after they had used the flipped classroom method in their
English learning.
The questionnaires were generated through Google Forms and distributed to the
sample group. The questionnaire consisted of two parts. Part A collected
demographic information and details about the participants' experiences with the
flipped classroom method in English learning. Part B focused on the participants'
perceptions of their acceptance level of the flipped classroom method in English
learning, specifically in relation to the four basic language skills. Responses were
recorded on a 4-point Likert scale, with scores ranging from 1 to 4. This instrument
facilitated the participants in expressing their opinions about the learning method
and allowed the researcher to gather data for analysis.
Before commencing the actual research, twelve sets of questionnaires were

administered as a pilot test to a sample group of 12 students. Cronbach's alpha
was used to measure reliability, and the resulting Cronbach's alpha value in the
pilot test stood at 0.88, indicating a strong level of internal consistency for the
scale. Consequently, the questionnaire was deemed suitable for use within the
8
scope of this research. Both before and after the study, the content and structure
of the questionnaire underwent rigorous examination and validation under the
supervision of the head of the English Panel at a school in the Kulai district.
Additionally, the researcher prepared a transcript and a marking guide to
enhance marking standards and standardize the grading system for both tests.
3.4 Data Collection

Instruments played a crucial role in data collection for this study. If the
instruments were unrelated or too complex for the participants to understand, it
could potentially jeopardize the entire research, leading to invalid data. In this
research, three distinct types of instruments were utilized. The pre-test and post-
test assessments were aligned with the English Common European Framework of
Reference (CEFR) learning content. Initially, the pre-test was administered before
the researcher implemented the treatment, while the post-test occurred after the
pupils had undergone the treatment, which involved the implementation of the
flipped classroom learning method. The interval between these two tests spanned
three weeks. Subsequently, a survey questionnaire was conducted after the pupils
had completed the post-test to gauge their perceptions of the treatment. This
questionnaire comprised two distinct sections, denoted as Part A and Part B.
The process of data collection was meticulously planned within a specific

timeframe and involved the following activities:
• 1 week: Introduction of the topic (Year 3 English: Topic 2 Every Day)
• 1 day: Briefing the pupils/samples about the intervention and creating a
WhatsApp group
• 1 day: Pre-Test
• 4 weeks: Intervention, involving the implementation of the Flipped
Classroom learning method to enhance Year 3 pupils' English skills
(Listening, Speaking, Reading, Writing)
• 1 day: Post-Test
• 1 day: Questionnaire
The research commenced with obtaining approval from the school administration
to conduct the study among their selected Year 3 pupils. Ethical considerations
were adhered to, as it is essential for researchers to seek administrative approval
before conducting research within an educational setting. Following this, the
research proceeded with the selection of the sample group, which was done using
purposive sampling, as previously described. Subsequently, the research
involved the administration of both the pre-test and post-test to the sample group.
The post-test was conducted immediately after the pre-test, and it occurred
subsequent to the implementation of the flipped classroom method, as per the
researcher's planned timeline. Both tests were completed within approximately
one hour, with clear instructions provided to the sample group beforehand.
Following the test phase, the survey method was employed, utilizing a
questionnaire distributed via Google Forms to gather information about the
pupils' perceptions after experiencing the flipped classroom approach. The
researcher provided a concise briefing and explanation to the sample group prior
9
to their completion of the survey, a process that typically took around 30 minutes
to finish. Subsequent data analysis of the pre-test, post-test, and survey responses
will be conducted using SPSS, with the final results discussed and summarized
by the researcher.
3.5 Data Analysis

In this research the collected data were analysed, computed, summarised and
transcribed using Statical Package for Social Science (SPSS) software.
3.5.1 Analysis of Primary Pupils’ Perception

The questionnaire form uses of 4-point of Likert scale and the scoring of items
used ordinal measurements. To analyse the primary pupils’ perception towards
flipped classroom learning method, the researcher used descriptive analysis to
find out the mean and standard deviation.
3.5.2 Analysis of Primary Pupils’ English Learning Performance

Both Pre-test and Post-test final results completed by samples of the research were
analysed by generalising the data to compare the significance of correlation level.
Firstly, Shapiro-Wilk normality test was conducted to find out the normality of
data. Subsequently, the researcher runs parametric test which is the Wilcoxon
signed-rank test since the data was normally distributed.
3.5.3 Analysis on the Relationship Between Primary Pupils’ English Overall

Performance Before and After Using Flipped Classroom
Researchers analyse the relationship between data from the sample before and
after applying the learning method. Therefore, firstly the Shapiro-Wilk normality
test was conducted to detect the normality of the data before run the analysis.
Next researcher followed by using either Spearman's rank-order correlation
coefficient (data are not regularly distributed) or Pearson's correlation coefficient
(data are normally distributed) to determine the relationship between primary
pupils' overall English performance on both exams.
3.5.4 Analysis of the Influence of Flipped classroom on Year 3 Pupils’ English

Overall Performance
For this analysis, the researcher conducted simple linear regression process to
analyse the data of overall score of both Pre-test and Post-test. This process started
with summary of the model in simple linear between the overall score of post-test
and pre-test to find out how strong the relationship between both tests. Next
researcher carried out the ANOVA test. The ANOVA test is to show the statical
differences between the means of post-test and pre-test. At the ending the
coefficients analysis was used to find out the strength of the linear relationship
between both tests. In addition, to present a clear and descriptive data, the
scatterplot chart was included in this analysis.
10
4. Results
4.1 Analysis of Primary Pupils’ Perception
Table 1 present the pupils perceptions and acceptance of flipped classroom
learning method in English learning in terms of listening, speaking, reading, and
writing skills.
Table 1. Mean value and standard deviation based on English listening, speaking,
reading and writing skills
No English basic skills Overall Mean Std.Deviation
1 Listening Skills 3.81 .342
2 Speaking Skills 3.79 .409
3 Reading Skills 3.75 .414
4 Writing Skills 3.79 .410
The table above shows the respondents’ acceptance of flipped classroom learning
method in English learning based on four basic skills. The overall mean for all the
skills is 3.81, 3.79, 3.75 and 3.79. The standard deviation, for the all skills are 0.342,
0.409, 0.414 and 0.410. Based on these results, it shows that the pupils’ has positive
response towards implementing flipped classroom learning method in learning
four English basic skills.
4.2 Effect of flipped classroom learning method on pupils’ English performance

based on Listening, Speaking, Reading and Writing Skills
Table 2. Negative Ranks and Positive Ranks data of listening, speaking, reading,
writing skills post-test and pre-test
Ranks Listening Speaking Reading Writing
Skill skill skill skill
Post test – Negative .00 3.00 .00 .00
Pre test mean
Ranks
Positive 15.50 15.93 16.00 16.00
mean
Ranks
Sum of 465.00 462.00 496.00 496.00
Ranks
Table 3. Result of Wilcoxon signed-ranks test between listening, speaking, reading,

writing skills pre-test and post-test
Test Listening Speaking Reading Writing
Statisticsª Skill skill skill skill
Pre-test – Z
Pro-test -4.871 -4.735 -4.891 -4.909
Asymp.Sig. .000 .000 .000 .000
(2-tailed)
Based on the table 2 and 3 above, the positive mean ranks of all the skills are 15.50,
15.93 and 16.00 which the mean are greater than negative mean ranks. Sum value
of positive ranks are 465.00, 462.00 and 496.00, exposing that there is improvement
in pupils’ scores from pre-test to post-test. According to the tests results, it can be
11
shown that there was a statistically significant difference between the listening,
speaking, reading and writing skills pre-test did generate a statistically significant
different with the score of post-tests (Z= -4.871, p=0.000), (Z= -4.735, p=0.000), (Z
= -4.891, p=0.000), and (Z=-4.909, p=0.000). These results revealed the
effectiveness of the flipped classroom learning method and enhanced the reading,
writing, speaking, and listening skills of Year 3 pupils.
4.3 Relationship between the English overall performance of pupils before and
after using flipped classroom learning method
Table 4. Result of Spearman’s rank-order correlation coefficient between score of Pre-

tests and Post-tests
Correlations Pre-test Post-test
Spearman’s rho Pre-test Correlation 1.000 .445*
Coefficient
Sig. (2-tailed) . .012
N 31 31
Post-test Correlation .445* 1.000
Coefficient
Sig. (2-tailed) .012 .
N 31 31
There exists a relationship between the total scores in both assessments. The
findings above indicate that there is a statistically significant moderate positive
correlation between the overall score on the pre-test and the overall score on the
post-test, or r=0.445, which is between 0.40 and 0.59. The impact of the flipped
classroom learning method on the overall English performance of Year 3 pupils
before and after its implementation was statistically significant (r=0.445, n=31,
p=0.012).
4.4 Influence of flipped classroom learning method on Year 3 Pupils’ English

Overall Performance
Table 5. Summary of the Model in a Simple Linear Regression Analysis between the
Overall Score of Post-test and the Overall Score of Pre-test
Model Summary
Model R R Square Adjusted R Std. Error of Durbin-Watson
Square the Estimate
1 .353ª .124 .094 6.108 1.999
Table 6. Analysis of Variance (ANOVA) Table for a Simple Linear Regression

Examining the Overall Score of the Post-test in Relation to the Overall Score of the
Pre-test ANOVAª
Model Sum of df Mean F Sig.
Square Square
1 Regression 153.518 1 153.518 4.115 .052
Residual 1081.966 29 37.309
Total 1235.484 30
12
Table 7. Table of Coefficients in a Simple Linear Regression Analysis for the Overall
Score of the Post-test and the Overall Score of the Pre-test
Coefficientsª
Standardized
Unstandardized Coefficients
Model Coefficients t Sig.
B Std. Error
Beta
1 (Constant) 77.120 2.586 29.821 .000
Pre-test .180 .089 .353 2.028 .052
According to tables 5, 6 and 7 above, there is a moderate positive correlation is

evident between the overall scores of the pre-test and the post-test, as indicated
by the model summary in the simple linear regression analysis for the overall
score of the post-test and the overall score of the pre-test, with an R-value of 0.353.
While R square value is equal to 0.124, which means independent variable is the
overall score of pre-test less influence the dependent variable (overall score of
Post-test), which is only 12.4%. P value is 0.052, which is more than 0.001. Thus,
based on the linear regression analysis, it can be concluded that there is no linear
relationship between the overall score of the post-test and the overall score of the
pre-test. Hence, this outcome can also be supported by referencing Figure 2, a
scatterplot generated using SPSS. The horizontal line positioned at 0 indicates the
absence of a linear relationship between the overall scores of the post-test and pre-
test when subjected to linear regression analysis.
Figure 2: Displays a scatterplot illustrating a simple linear regression analysis

between the combined scores of the post-test and those of the pre-test
5. Discussion
In the initial section of the study, the results indicated a strong endorsement of the
Flipped Classroom learning approach, particularly in the context of Year 3
students' English listening skills. The data suggested that the participants
acknowledged the effectiveness of the Flipped Classroom method in enhancing
their English listening abilities. The statement was similar with the findings of
13
(Yousufi, 2020) who stated that flipped classroom learning method shows a
remarkable effect on improving learners' listening skill. Furthermore, the second
part of the survey shows the Year 3 pupils’ acceptance of flipped classroom
learning method based on English speaking skills. This section also received
positive response from the items stated about the flipped classroom learning
method on English speaking skills of the respondents. This discovery shared a
comparable scenario with the research conducted on Abdullah et al., (2019) who
stated that technology tools used in flipped classroom learning method are
helpful, encourage and engaging learning environment to build up learners
abilities along with being the solution of learning and teaching of speaking skills
problem. Next, it is followed by the section which the items related with the Year
3 pupils’ acceptance of flipped classroom learning method based on English
reading skills, which the pupils gave positive feedbacks too. This finding can be
supported by the study of Öztürk & Çakıroğlu, (2021), find that flipped classroom
learning method encourage and engage pupils’ in reading activities and (Gok et
al., 2023) which conclude that flipped classroom learning method improve pupils’
English reading skills. The last section of the survey is about the Year 3 pupils’
acceptance of flipped classroom learning method based on English writing skill
and overall respondents gave positive feedback. The finding also similar with a
study carried by Siswanto, (2021) stated that writing skills has improve pupils'
participation, confidence and happiness to learn which lead them to develop their
English writing skills.
The Wilcoxon signed-ranks test was used to evaluate the effect of the flipped
classroom learning method on Year 3 pupils’ based on English listening skills.
Referring to the data, there is a significant difference between English listening
pre-test and English listening post-test. Therefore, this finding meant that flipped
classroom learning method is effective on Year 3 pupils’ English listening
performance. It can be justified by referring the mean value in listening post-test,
which is bigger than listening pre-test and it showed that there is a positive effect
on the pupils’ English listening performance after going through the flipped
classroom learning method. This finding is similar with the study of Yousufi,
(2020) who said that flipped classroom learning method brought a incredible
effect on improving learners' listening skill. Furthermore, the Wilcoxon signed-
ranks test was used to examine the effect of the flipped classroom learning
strategy on the English performance based on year 3 pupils’ speaking skills. The
data presented demonstrates a significant difference between the students'
performance in the speaking pre-test and their performance in the speaking post-
test. Hence, we can conclude that flipped classroom learning method is effective
by improve on Year 3 pupils’ English speaking skill. It can be proved by referring
to the mean value of speaking post-test, which is greater than speaking pre-test.
This finding justified that there is a positive outcome by using flipped classroom
learning method on Year 3 pupils’ English speaking skill. This statement also
aligned with the study of Sudarmaji et al., (2021) who stated that Flipped
Classroom is effective in increasing pupil’s performance on English speaking
skills.
14
Next, using the Wilcoxon signed-ranks test, the effect of the flipped classroom
learning method of year 3 pupils’ on their English performance of reading skills
was examined. Next, the effect of flipped classroom learning method on Year 3
pupils’ English performance according to their reading skills was analysed by
using Wilcoxon signed-ranks test. The finding displayed that there is a significant
difference between Year 3 pupils’ performance on English reading pre-test and
English reading post-test. Refer to the mean value stated, there is a positive rising
in English reading post-test compared with the mean value of English reading
pre-test. Therefore, the findings show that there is a positive effect on pupils’
reading skills after using flipped classroom learning method and it’s similarly
aligned with the study of (Reflianto et al., 2021) said that flipped learning
improving students engagement and reading comprehension skills. Lastly, the
effect of flipped classroom learning method on Year 3 pupils’ English writing skill
performance had been examined by using Wilcoxon signed-ranks test. Based on
the result, it showed that there is a significant difference between the pupils’
performance in English writing pre-test and English writing post-test. These
findings clearly indicate that there is an effect on flipped classroom learning
method on pupils’ English writing performance. The positive increase in mean
value of English writing post-test proved that there is a positive effect of flipped
classroom learning method on Year 3 pupils’ English writing skills. This research
is substantiated by the study on Indayani et al., (2022) said that pupils' writing
skills improved crucially when conducting teaching and learning process using
flipped classroom through a WhatsApp group.
According to Year 3 pupils’ English overall performance before and after applying
flipped classroom learning method had been analysed by using Spearman’s rank-
order correlation coefficient analysis. The result showed there is moderate
positive correlation between the overall pre-test score and overall post-test score.
That means the increase or decrease of overall students’ performance of post-test
are significantly related to the overall students’ performance of pre-test. The
overall students’ performance, slightly shows that the interrelation between both
tests are strong. The relationship between both tests reveal that the flipped
classroom learning method is an effective method of improving pupils’
performance in all English skills (Listening, Speaking, Reading and Writing). The
findings are totally align with a study of Mooneeb Ali et al., (2021) said that
learners who went through the learning process by utilizing the Flipped
classroom method got higher score in the test carried out and shows a good
performance.
After Spearman’s rank-order correlation coefficient analysis had been completed,

the linear regression analysis had been carried out by using both sets of overall
score pre-test and post-test data for the intent whether there is an influence of
flipped classroom learning method after used on Year 3 pupils’ English overall
performance. The discovery revealed that there is no influence of flipped
classroom learning method on Year 3 pupils’ overall performance in English and
this statement also supported from a study conducted by (Wagner, 2020),
suggested that the Flipped Classroom demonstrates more pronounced effects on
subjects in the STEM field (science, technology, engineering, mathematics)
15
compared to languages and humanities. In conclusion, this research reveals that

the objective of the study has been accomplished. It can be proved by referring to
the pupils’ results in English achievement test, which has increased drastically
after utilising the flipped classroom learning method. Moreover, pupils also give
a positive perception towards the learning method after experienced the learning
method.
6. Conclusion
In light of the research problem, where English serves as a second language for
Malaysian pupils, with limited opportunities for practice and varying cognitive
learning levels, the findings emphasize the appropriateness and effectiveness of
implementing the flipped classroom learning approach in primary school English
language instruction. Year 3 pupils, facing challenges in language retention and
retrieval due to their linguistic backgrounds, showcased a notable level of
acceptance and satisfaction when engaging with the flipped classroom approach,
encompassing all four essential language skills: speaking, listening, reading, and
writing. Furthermore, the study illuminated a discernible positive impact on Year
3 pupils' English language performance following the adoption of the flipped
classroom learning method. Recognizing the individual differences in learning
pace and the need for additional support, this approach allows for tailored
resource selection, catering to students' diverse levels and educational
requirements. This alignment with the Malaysian educational framework
positions the flipped classroom as a compelling pedagogical tool, advocating for
its broader adoption in primary school language instruction. Notably, the flipped
classroom model extends its benefits beyond students, empowering educators to
curate dynamic teaching and learning experiences that align with 21st-century
learning objectives, integral to the Malaysian education system. This model's
capacity to foster an active and enriching learning environment ultimately
enhances the overall educational journey of primary school students. In summary,
this research underscores the suitability and efficacy of the flipped classroom
learning approach in addressing the challenges posed by English language
instruction for Year 3 pupils in Malaysia. By accommodating individual learning
needs, promoting active communication, and aligning with educational
objectives, the flipped classroom emerges as a valuable tool in the realm of
primary school language education.
Acknowledgment
The authors would like to acknowledge the financial support from Ministry of
Higher Education and Universiti Teknologi Malaysia under UTM Encouragement
Research Grant Scheme (UTMER) (Q.J130000.3853.31J32).
7. References
Bergmann, J., & Sams, A. (2008). Remixing Chemistry Class: Two Colorado Teachers
Make Vodcasts of Their Lectures to Free Up Class Time for Hands-On Activities.
Learning & Leading with Technology, 36(4), 22–27.
http://files.eric.ed.gov/fulltext/EJ904290.pdf
Branch, R. M. (2010). Instructional design: The ADDIE approach. In Instructional Design:
The ADDIE Approach. Springer US. https://doi.org/10.1007/978-0-387-09506-6
16
Brewer, R., & Movahedazarhouligh, S. (2018). Successful stories and conflicts: A

literature review on the effectiveness of flipped learning in higher education. In
Journal of Computer Assisted Learning (Vol. 34, Issue 4, pp. 409–416). Blackwell
Publishing Ltd. https://doi.org/10.1111/jcal.12250
Collado-Valero, J., Rodríguez-Infante, G., Romero-González, M., Gamboa-Ternero, S.,
Navarro-Soria, I., & Lavigne-Cerván, R. (2021). Flipped classroom: Active
methodology for sustainable learning in higher education during social distancing
due to COVID-19. Sustainability (Switzerland), 13(10).
https://doi.org/10.3390/su13105336
Cueva, A., & Inga, E. (2022). Information and Communication Technologies for
Education Considering the Flipped Learning Model. Education Sciences, 12(3).
https://doi.org/10.3390/educsci12030207
Davies, R. S., Dean, D. L., & Ball, N. (2013). Flipping the classroom and instructional
technology integration in a college-level information systems spreadsheet course.
Educational Technology Research and Development, 61(4), 563–580.
https://doi.org/10.1007/s11423-013-9305-6
Elian, S. A., & Hamaidi, D. A. (2018). The effect of using flipped classroom strategy on
the academic achievement of fourth grade students in Jordan. International Journal
of Emerging Technologies in Learning, 13(2), 110–125.
https://doi.org/10.3991/ijet.v13i02.7816
Fatimah Abd Rahman, S., Md Yunus, M., & Hashim, H. (2019). An Overview of Flipped
Learning Studies in Malaysia. Arab World English Journal, 10(4), 194–203.
https://doi.org/10.24093/awej/vol10no4.15
Flipped Learning Network (FLN). (2014). The four pillars of FLI-PTM.
https://flippedlearning.org/wp-
content/uploads/2016/07/FLIP_handout_FNL_Web.pdf
Graham Brent Johnson. (2013). Student Perceptions Of The Flipped Classroom [The
University of British. ]. https://assets.techsmith.com/Docs/pdf-
landingpages/Student_Perceptions_of_the_flipped_Classroom-
_Graham_Johnson.pdf
Gustian, K., Rusmawaty, D., Mulawarman, U., Kuaro, J., Kelua, G., Samarinda Ulu, K.,
Samarinda, K., & Timur, K. (2023). The Benefits of Flipped Classroom Model for Efl
Learners. Journal on Education, 05(04).
https://doi.org/https://doi.org/10.31004/joe.v5i4.2411
Haghighi, H., Jafarigohar, M., Khoshsima, H., & Vahdany, F. (2019). Impact of flipped
classroom on EFL learners’ appropriate use of refusal: achievement, participation,
perception. Computer Assisted Language Learning, 32(3), 261–293.
https://doi.org/10.1080/09588221.2018.1504083
Mandasari, B., & Wahyudin, A. Y. (2021). Flipped Classroom Learning Model:
Implementation and Its Impact on EFL Learners’ Satisfaction on Grammar Class.
Ethical Lingua, 8(1), 150–158. https://ethicallingua.org/25409190/article/view/234
Ministry of Education. (2012). Malaysia Education Blueprint 2013-2025. www.moe.gov.my
Paez-Quinde, C., Chasipanta-Nieves, A., Hernandez-Davila, C. A., & Arevalo-Peralta, J.
(2022). Flipped classroom in the meaningful learning of the students of the Basic
Education Career: Case study Technical University of Ambato. IEEE Global
Engineering Education Conference, EDUCON, 2022-March, 785–789.
https://doi.org/10.1109/EDUCON52537.2022.9766792
Pavanelli, R. (2018). The Flipped Classroom: A Mixed Methods Study of Academic
Performance and Student Perception in EAP Writing Context. International Journal
of Language and Linguistics, 5(2). https://doi.org/10.30845/ijll.v5n2p3
17
Robinson, C. D., Scott, W., & Gottfried, M. A. (2019). Taking It to the Next Level: A Field
Experiment to Improve Instructor-Student Relationships in College. AERA Open,
5(1). https://doi.org/10.1177/2332858419839707
Rossett, A. (1987). Training Needs Assessment. Educational Technology.
https://books.google.com.my/books?hl=en&lr=&id=IWBppwNMC-
QC&oi=fnd&pg=PR7&dq=Rossett+
Ruiz-Jiménez, M. C., Martínez-Jiménez, R., Licerán-Gutiérrez, A., & García-Martí, E.
(2022). Students’ attitude: Key to understanding the improvement of their
academic RESULTS in a flipped classroom environment. International Journal of
Management Education, 20(2). https://doi.org/10.1016/j.ijme.2022.100635
Shaari, N. D., Shaari, A. H., & Abdullah, M. R. (2021). Investigating the impact of flipped
classroom on dual language learners’ perceptions and grammatical performance.
Studies in English Language and Education, 8(2), 690–709.
https://doi.org/10.24815/siele.v8i2.18872
Shabibi, J., Thurya, S., Al-Shabibi, M., Al-Ayasra, A.-K., & Al-Shabibi, J. M. (2017). The
Impact of Using Flipped Classroom in the Learning Process: A scoping Review.
https://www.researchgate.net/publication/352799143
Siegle, D. (2014). Technology: Differentiating instruction by flipping the classroom.
Gifted Child Today, 37(1), 51–55. https://doi.org/10.1177/1076217513497579
Struyven, K., Dochy, F., & Janssens, S. (2005). Students’ perceptions about evaluation
and assessment in higher education: A review. In Assessment and Evaluation in
Higher Education (Vol. 30, Issue 4, pp. 325–341).
https://doi.org/10.1080/02602930500099102
Su Ping, R. L., Verezub, E., Adi Badiozaman, I. F. bt, & Chen, W. S. (2020). Tracing EFL
students’ flipped classroom journey in a writing class: Lessons from Malaysia.
Innovations in Education and Teaching International, 57(3), 305–316.
https://doi.org/10.1080/14703297.2019.1574597
Tazijan, F. N., Abdullah, C. H., Zainol, N., Noor, S. M., & Johari, N. R. (2017). Building
Communication Skills through Flipped Classroom Article Information. International
Academic Research Journal of Social Science, 3(1), 142–147.
http://www.iarjournal.com/wp-content/uploads/IARJSS2017_1_142-147.pdf
Teo Woon Chun, & Ramesh Sathappan. (2018). The effectiveness of using Flipped
Classroom Approach to teach adjectives to Malaysian Year 4 Chinese ESL learners.
English Teacher, 47(2), 53–63.
https://melta.org.my/journals/TET/downloads/tet47_02_03.pdf
Vockley, M. (2007). Maximizing the Impact: The Pivotal Role of Technology in a 21st Century
Education System.
https://www.immagic.com/eLibrary/ARCHIVES/GENERAL/SETDA_US/S0712
11S.pdf
Vuong, N. H. A. (2023). Moodle As a Potential Online Platform Used In The Flipped
Classrooms In The Vietnamese EFL Teaching Context. European Journal of Education
Studies, 10(3). https://doi.org/10.46827/ejes.v10i3.4710
Wang, F., & Yamat, H. (2019). Identifying English Vocabulary Levels of Malaysia Year 5
Primary School Students. International Journal of Academic Research in Business and
Social Sciences, 9(12). https://doi.org/10.6007/IJARBSS/v9-i12/6669
Yousufi, U. (2020). An Integrative Review of Flipped Classroom Model. American Journal
of Educational Research, 8(2), 90–97. https://doi.org/10.12691/education-8-2-4
18

Vol. 22, No. 10, pp. 18-36, October 2023
Knowledge Management Using Storytelling with

Infographics to Develop Creativity of
Undergraduate Students
Virawan Amnouychokanant
Silpakorn University
Nakhon Pathom, Thailand
Abstract. Creativity is one of the most marketable skills in the digital

age. The main purpose of this study is to explore and enhance the
creativity of undergraduate students through knowledge management
using storytelling with infographics. In this study, an experiment was
conducted with 40 third- and fourth-year students from various majors
of the Faculty of Arts at Silpakorn University. All participants had
enrolled in the Multimedia Design and Production course. A one-group
quasi-experiment with a pretest and posttest design was used. Several
instruments were employed, including a creativity assessment form, a
self-creativity assessment form, an infographic assessment form, and an
infographic design attitudes questionnaire. Initially, participants were
asked to complete the self-creativity assessment form and the creativity
assessment form. During the sessions, all participants were taught using
storytelling with infographics. After they finished designing the
infographics, the instructor assessed the students’ work using a rubric
for infographic assessment. Additionally, students evaluated their own
work. At the end of the course, the students were asked to retake the
self-creativity assessment form and creativity assessment form. The
results showed that students’ creativity scores were significantly higher
in all components (e.g., originality, fluency, flexibility, and elaboration).
Moreover, most students received average scores in infographic design
at the advanced level. However, students were found to lack confidence
and courage in expressing diverging ideas. This highlights the
challenges for instructors and learning designers in researching and
developing suitable and effective methods to boost students’ confidence
and encourage thinking outside the box.
Keywords: creativity; infographics; knowledge management;

storytelling
1. Introduction
Innovation and technology have numerous benefits that impact various aspects
of our lives, society, and the economy. Conversely, various technologies have
©Authors
19
rendered many professions obsolete or will replace more professions in the

future (Verhoef et al., 2021; Vial, 2021). Consequently, undergraduate students
without an advantage over the inexorably improving technology may find it
difficult to survive the fierce competition in the labor market. Relying solely on
academic knowledge from the university is no longer sufficient; undergraduate
students must possess other skills in demand in the digital age.
Creativity stands as one of the most marketable skills in the labor market, as
creative individuals exhibit initiative, flexible imagination, and the ability to
generate new concepts or methods for problem-solving and improvement,
directly or indirectly benefiting organizations (Di Battista et al., 2023). Therefore,
a challenge in the digital age, where technology and creativity play pivotal roles
in development and competition, lies in developing preparatory programs for
undergraduate students to equip them for the labor market, aid self-discovery,
and enhance their skills and knowledge.
Currently, the information on the Internet is massive and dispersed. True and
false information seem to be blending on the Internet. With more people
consuming news online, a concerning consequence is the inadvertent spread of
fake news about health, international events, and political issues. Furthermore,
fake news on Twitter and LINE is more likely to be retweeted or shared (Rocha
et al., 2021; Pennycook & Rand, 2021; Pierri & Ceri, 2019). Communication
through LINE or Facebook groups can have more adverse effects on society due
to a high level of trust compared to interactions with strangers. Prolonged and
repeated interactions with like-minded individuals and exposure to the same
data set lead to a one-sided information reception or selective information
intake, commonly referred to as an “Echo Chamber.” This phenomenon entails
the selective reception of news and media, primarily from like-minded sources
and may lead individuals to perceive their ideas as the absolute and irrefutable
truth, leaving no room for dissenting voices (Terren & Borge-Bravo, 2021; Cinelli
et al., 2021). Another issue arises in close-group communication among senior
citizens, as they may struggle with search engine usage compared to other age
groups, leading to closed groups becoming sources of fake news. Additionally,
sharing misleading content might lead to cyberbullying and hate speech
(Giumetti & Kowalski, 2022; Craig et al., 2020; Matamoros-Fernández & Farkas,
2021; Paz et al., 2020; MacAvaney et al., 2019). Sharing incorrect or distorted
information immeasurably affects the world community. Therefore, students in
the digital age must possess knowledge and critical thinking skills to discern and
exchange factual and reasonable information in society.
Given the aforementioned reasons, the researcher emphasizes the importance of

classroom knowledge management, creating an environment where students
can search, screen, and share knowledge in the digital age, where information
volume is massive. The knowledge management process entails knowledge
identification, creation, acquisition, organization, codification, refinement,
sharing, and application (Ammirato et al., 2021; De Bem Machado et al., 2022). In
this study, the researcher integrated the design process with learning activities
and instructed students to summarize information or knowledge in infographic
20
form. This is because infographics allow audiences to understand the content

quickly and clearly. Furthermore, human brains process and respond to pictures
faster than text, making infographics more likely to be read than plain text
information (Apriyanti et al., 2020; Dorneles et al., 2020; Habeeb, 2020).
Silpakorn University’s Faculty of Arts covers diverse fields of science, including

linguistics, library science, history, geography, social sciences, philosophy, and
drama and music. One of the core competencies of the institute is integrated
teaching and learning across sciences to develop the skills needed for 21st-
century learners. Faculty members of the Faculty of Arts focus on developing
students’ creative thinking as it is one of the most important skills for workers in
the digital age. This aligns with the World Economic Forum’s Future of Jobs
2023 report that creativity is the top job market skill (Di Battista et al., 2023).
In this study, I attempt to examine and develop the creativity of undergraduate

students of the Faculty of Arts, Silpakorn University through knowledge
management using storytelling with infographics. I aim to address the following
research questions:
1. What is the level of undergraduate students’ creativity after learning
activities?
2. What are the undergraduate students’ attitudes toward their own creativity
after learning activities?
3. How are the undergraduate students’ infographic design skills?
4. What are the undergraduate students’ attitudes toward their own infographic
design?
2.1 Creativity in the Digital Age
Many people tie the word “creativity” with “artistic skills,” such as drawing and
playing musical instruments, to the point they associate the usefulness of
creativity only with artistically capable people. However, creativity is accessible
to everyone and is among the most marketable skills today. The World
Economic Forum’s Future of Jobs 2023 report finds analytical thinking, creative
thinking, artificial intelligence, and big data will be top in-demand skills by 2027.
In 2023, companies consider analytical thinking to be the strongest core skill,
while ranking creative thinking second. However, by 2027, creative thinking is
projected to grow in importance slightly more rapidly than analytical thinking
(Di Battista et al., 2023).
Creativity is not limited to people in the art community, even doctors need
creativity because they have to face diverse situations about patient care.
Doctors often face complex, urgent problems that need immediate solutions. In
addition to the rapidly changing healthcare system, they need more creativity,
especially if the doctor has the job to create or develop an effective patient care
system. Such development requires a specific innovation for the context the
doctor is operating in (Ten Haven et al., 2022). In today’s rapidly changing
world, scientists need to leverage creativity to advance science and technology.
Scientists need to use their creativity to produce new knowledge or develop
21
innovative solutions to address increasingly complex problems (Shen, 2023).

Creativity is key for many professions and not just for professionals in art and
design communities.
Advancements in the world community from the past to the digital age—full of
news, information, media, and technologies—could be attributed to the
creativity of persons that developed innovations, leading to more flexibility and
national development. Many countries have included creativity development in
international policy. This is covered in the educational policy, national education
plan, and curriculum and activities. The government provides support for
projects improving citizen creativity. Hence, it can be concluded that the more
creativity among the citizenry, the more development and prosperity for a
country. This is especially true in the 21st century, the era of innovation and
technology that has advanced by leaps and bounds. The students in this era
should receive creativity development support to live effectively in a world full
of competition and technological advancement (Gube & Lajoie, 2020; Puccio,
2017).
The researcher examined the components of creativity (Hendrik, 2022;

Handayani et al., 2021; Almeida et al., 2008) and found four major parts: 1)
originality: a new idea that born of imagination and thinking outside the box or
combination and adaptation of old ideas to a new one; 2) fluency: an ability to
think of things in quantity within a limited time; 3) flexibility: an ability to think
diversely and avoid repetition and redundancy, which is highly important; 4)
elaboration: an ability to think in details to fill up the gaps in the main concept.
Elaboration is related to attention to detail skills, which will be expressed in
finely crafted work. According to four components of creativity, the researcher is
interested in measuring the students’ creativity skills through these four
components.
2.2 The Knowledge Management in Classroom Context

“Knowledge management” is perceived to be a process designed to create,
organize, manage, and utilize knowledge within large organizations. In reality,
knowledge management is not location dependent. Knowledge management
could be done within small classrooms (Thongkoo et al., 2019). There are two
types of knowledge: 1) tacit knowledge gained through experience, gift, or
intuition of a person in understanding things. This is knowledge that cannot be
easily translated into words or text (e.g., working skills, crafting skills, or critical
thinking). 2) explicit knowledge can be collected and passed on through text,
theories, and manuals. Knowledge management requires a process and
knowledge management tool. Much research work in the past proposed
knowledge management through various processes and tools such as storage of
knowledge in a database, big data-based knowledge management (Sumbal et al.,
2021), ontology-based knowledge management (Osman et al., 2022), after action
reviews (Vukanović-Dumanović et al., 2022; Keiser & Arthur Jr, 2021),
mentoring system (Papadopoulou et al., 2016; Bencsik et al., 2014; Srivichai et al.,
2012), cross-functional team (Mohamed et al., 2004; Majchrzak et al., 2012),
workshop and brainstorming (White et al., 2022; Evans, 2012), and communities
22
of practice (Arthur, 2016; Reaburn & McDonald, 2017; Lewis & Rush, 2013;
Annala & Mäkinen, 2017; McDonald & Cater-Steel, 2016). Very few works use
storytelling for knowledge management. In this study, the researcher is
interested in using storytelling with infographics as a knowledge management
tool to provide empirical evidence that can help clarify its efficiency in
classroom context.
2.3 Storytelling with Infographics

Storytelling with infographics has a significant impact on students’ creativity in
several ways. Infographics combine text, images, and visual elements to convey
information. This multimedia approach engages both the left and right brain
hemispheres, stimulating creativity. Visual cues can spark creative thinking by
making complex data or concepts more accessible and memorable (Aldalalah,
2021). Effective infographics distill complex information into concise, easy-to-
understand visuals. This simplification allows individuals to focus on creative
problem-solving rather than grappling with data overload (Barlow et al., 2021).
Infographics often include data and statistics, requiring students to analyze and
interpret information. This analytical thinking can spur creativity as students
explore patterns, draw conclusions, and generate new ideas based on the data
they encounter (Chicca & Chunta, 2020). Infographics follow a narrative
structure, presenting information in a logical sequence. This storytelling aspect
can help students organize their ideas creatively, enabling them to convey
complex concepts in a coherent and compelling way (Tsai et al., 2020).
Infographics frequently use visual metaphors to represent abstract concepts or
relationships. Encountering these metaphors can encourage students to think
metaphorically and make imaginative connections between different elements,
fostering creative thinking (Ocobock, 2020). Creating infographics often involves
gathering information from various sources and condensing it into a concise and
visually appealing format. This process requires students to synthesize
information creatively, highlighting key points and leaving out nonessential
details (Jones et al., 2019). Designing an infographic involves making decisions
about layout, color schemes, typography, and visual elements. This design
process encourages students to think creatively about how to present
information in a visually appealing and accessible way (Aldalalah, 2021).
Infographics require students to communicate information clearly and concisely.
Developing these communication skills is essential for creative thinking, as it
helps students express their ideas and insights effectively. Students may need to
overcome challenges when creating infographics, such as finding the best way to
represent complex data or choosing the most suitable visual elements. These
problem-solving activities can stimulate creativity by encouraging students to
find innovative solutions (Jones et al., 2019).
In summary, incorporating infographics into learning activities can be a valuable

tool for improving students' creativity by combining visual learning, data
analysis, narrative structure, and problem-solving. Infographics also draw from
multiple disciplines, combining data analysis, graphic design, and storytelling.
Engaging with infographics can expose students to diverse perspectives and
ideas, fostering creativity through cross-disciplinary learning.
23
3. Methodology
3.1 Participants
My sample group comprised 40 undergraduate students enrolled in the
Multimedia Design and Production course. All participants were third- and
fourth-year students of the Faculty of Arts at Silpakorn University. Participants
comprised 10 men and 30 women, and they were taking different majors—
information and library science (16), linguistics (13), social sciences and history
(7), and performing arts (4).

This research used the following measuring tools: 1) creativity assessment form
2) self-creativity assessment form 3) infographic assessment form, and 4)
infographic design attitudes questionnaire.
1) Creativity assessment form: The duration of the test was 30 minutes,

consisting of three parts—picture construction (10 mins), picture completion (10
mins), and circles (10 mins). Each part was detailed as follows. Part 1: Draw
additional pictures from the given geometric shapes. Try to be as unique as
possible. Next, name the picture. Part 2: Draw the pictures from the given lines
and name the pictures. Part 3: Draw 30 pictures from the given circle. Each
picture must be different. Next, name the pictures. Pictures needed to be named
in every activity of the creativity test to encourage students to practice their
creativity in using words and language in addition to expressing their ideas
through drawing. Table 1 shows a rubric for creativity assessment.
2) Self-creativity assessment form: this assessment was designed to measure

students’ attitudes toward their own creativity. Items in the assessment form
were designed using a 5-point Likert scale (5 = “strongly agree,” 4 = “agree,” 3 =
“neutral,” 2 = “disagree,” and 1 = “strongly disagree”). The reliability of the self-
creativity assessment form was 0.84. The reliabilities of each dimension were
0.89 (creativity dimension), 0.80 (emotional dimension), and 0.84 (personality
dimension). Table 2 shows the self-creativity assessment form.
3) Infographic assessment form: The evaluation of students’ infographics

included six aspects: design, composition, color, font, presentation, and source
reliability. Table 3 shows the rubric for infographic assessment.
4) Infographic design attitudes questionnaire: I designed this questionnaire to

measure students’ attitudes toward their own infographic design. Items were
designed using a 5-point Likert scale (5 = “strongly agree,” 4 = “agree,” 3 =
“neutral,” 2 = “disagree,” and 1 = “strongly disagree”). The assessment form
contains 13 items. The infographic design attitudes questionnaire had reliability
of 0.79, indicating acceptable internal consistency.
24
Table 1: Rubric for creativity assessment

Creativity Proficient Developing Basic
components (3 points) (2 points) (1 point)
The work presents The work presents The work is the
new and unique new ideas. same as most
concepts. However, there are students in the
Originality
some parts of the class.
work duplicated
with other students.
Complete all Complete more Complete less
assignments within than half of the total than half of the
Fluency the time limit. creativity test total creativity test
within the time when the time
limit. runs out.
Create work in Create many styles Not a wide
many ways. of work, but some variety of
styles are repeated. creations and
Flexibility
most of them have
duplicate
concepts.
The work has Some parts of the Most of the work
Elaboration exhaustive details. work lack lacks details and
important details. refinement.
Table 2: Survey items of the self-creativity assessment form

Items
1. Creativity dimension
1.1 Originality: I have great imagination and dare to think and do unconventional and
unique things.
1.2 Flexibility: I accept new ideas and do not stick to original concepts.
1.3 Fluency: I can develop many solutions within a limited time.
1.4 Elaboration: I work with refinement, thoroughness, and meticulousness.
2. Emotional dimension
2.1 Curiosity: I am observant and curious. I never give up looking for answers as long
as my curiosity remains.
2.2 Enthusiasm: I like seeking knowledge, and I am eager to learn.
2.3 Sensitivity: I often perceive changes in the surrounding environment quickly.
2.4 Humor: I am fun, friendly, and not stressed, and I like making others laugh.
3. Personality dimension
3.1 Self-confidence: I have confidence in my ideas.
3.2 Courage: I dare to decide even if my decisions differ from most people’s.
3.3 Commitment: I work hard and can endure difficult and time-consuming tasks.
3.4 Independence: I can express my opinions openly without any concern.
Table 3: Rubric for infographic assessment
25
Advanced Proficient Developing Basic

Items
(4 points) (3 points) (2 points) (1 point)
Interesting Interesting No theme, but No theme and
theme and theme, but it aligns with inconsistent
consistent with inconsistent objectives and with objectives
Design objectives and with objectives target groups. and target
target groups. and target groups.
groups at some
points.
The visual The The layout of The layout of
weight is composition is the the
distributed not too composition is composition is
evenly across cluttered, but not as balanced excessively
Composition
the one side seems as it should be, disorganized
composition heavier than but it is not too and cluttered.
and not too the other. cluttered.
cluttered.
Use colors that Use colors that Use too many Use too many
match the match the colors but colors that are
Color content. content in remain not suitable for
some parts. consistent with the content.
the content.
The font styles The font styles Font sizes vary Use the same
are appropriate are suitable for according to font size for the
for the work. the work, but content priority whole work
Font Font sizes vary the font is the but use too and use too
according to same size for many font many font
content the whole styles. styles.
significance. work.
The The The Lack of
presentation of presentation of presentation of information
information is information is information is summary and
concise and concise and concise and organization.
Presentation accurate. accurate but is accurate but is There are many
misleading in misleading in misspellings.
some parts. some parts.
There are some
misspellings.
Source Sources are Some sources Many sources No references.
reliability highly reliable. are not reliable. are not reliable.
3.3 Procedure
I conducted experimental research for 2.5 hours per week over 4 months.
Initially, participants were asked to complete the self-creativity assessment form
and creativity assessment form. The teaching and learning processes for this
course were synthesized into six key steps, as follows: 1) problem analysis, 2)
knowledge creation and acquisition, 3) knowledge codification and refinement,
4) design, 5) knowledge sharing, and 6) evaluation.
26
1) Problem analysis is the step wherein students analyzed the current social
issues and turned them into infographics for storytelling. In this step, the
instructor allowed the students to freely select their topics to open their minds,
without limiting their ideas to a certain scope. However, the chosen topic for the
infographics must be informative and different from that of their classmates; 2)
Knowledge creation and acquisition is the step wherein students searched for
information from reputable sources. The instructor recommended that students
selected sources with clear author names and reputable organizational websites
that aligned with the chosen topic. For example, if the topic is about health and
disease, students were encouraged to refer to hospital websites, the Ministry of
Health’s website, or reputable health organizations. 3) Knowledge codification
and refinement involved students screening and compiling information
obtained in the previous step. They were required to understand the content
through reading and analysis. The instructor emphasizes that students should
not copy and paste to avoid plagiarism. 4) Design is the step wherein students
transformed the gathered information into infographics. The instructor then
recommended that students used free and opensource software to design
infographics. Using free and open-source software allowed students to access
resources for design without worrying about copyright issues. 5) Knowledge
sharing involves students presenting their work and exchanging knowledge
with each other and with the instructor. 6) Evaluation is the step where the
instructor assesses the students’ work using the rubric for infographic
assessment. Additionally, students evaluate their own work. At the end of the
course, the students were asked to retake the self-creativity assessment form and
creativity assessment form to compare the pretest and posttest results.
4. Results
4.1 Results of the Creativity Assessment
The researcher administered pretest and posttest creativity assessment forms to
the students for comparison, with the aim of assessing their originality, fluency,
flexibility, and elaboration scores. Table 1 presents the rubric used to assess
creativity. The creativity test obtained from 1 to 4 points is evaluated as basic
level. Creativity test scores ranging from 5 to 8 points are evaluated at the
developing level, while scores above 9 points are considered proficient level.
Table 4 presents the scores obtained in each creativity component. An average
score for each component ranging from 2.50 to 3.00 points is categorized as
proficient, scores from 1.50 to 2.49 points as developing, and scores below 1.50
points as basic. The results showed that students’ creativity scores were
significantly higher in all components. Moreover, they had average posttest
scores in all creativity components at the proficient level, except in fluency.
However, the difference between pretest and posttest scores proved that fluency
had the highest development among students. Before the learning activities,
most students could not finish the creativity assessment form within the time
limit. However, after engaging in learning through knowledge management
using storytelling with infographics, most students had improved their
creativity proficiency, enabling them to complete the creativity assessment form
within the time limit. This resulted in a higher fluency score compared to before
the learning activities began.
27
Table 4: Results from the creativity assessment form

Creativity
M SD Interpretation t p
components
Originality (pre) 1.95 0.316 Developing
10.817 <0.001**
Originality (post) 2.70 0.464 Proficient
Fluency (pre) 1.08 0.350 Basic
14.552 <0.001**
Fluency (post) 2.35 0.483 Developing
Flexibility (pre) 1.88 0.404 Developing
12.854 <0.001**
Flexibility (post) 2.85 0.362 Proficient
Elaboration (pre) 1.78 0.480 Developing
9.067 <0.001**
Elaboration (post) 2.50 0.555 Proficient
Total (pre) 6.68 0.917 Developing
21.241 <0.001**
Total (post) 10.40 1.215 Proficient
** p ≤ 0.01
4.2 Results of the Students’ Attitudes toward Their Own Creativity

To gage the students' attitudes toward their creativity, the instructor had them
complete the self-creativity assessment form both before and after the learning
activities. The results showed that after the knowledge management learning
approach, the students exhibited a positive outlook on their creativity (see
details in Table 5). They possessed greater imagination and dare to think and do
unconventional and unique things. They developed more solutions within a
limited time. They believed they could work with more refinement,
thoroughness, and meticulousness. They liked seeking knowledge and are eager
to learn. They believed they worked hard and could endure difficult and time-
consuming tasks. They could also express their opinions openly without any
concern. However, the pretest and posttest scores of self-confidence and courage
had no statistically significant difference. This indicates that the instructor
should focus on enhancing the students’ confidence in expressing their creativity
as prolonged low confidence may limit their ability to think creatively.
Table 5: Results of the self-creativity assessment

Items M SD t p
Originality (pre) 3.48 0.847
5.019 <0.001**
Originality (post) 3.95 0.714
Fluency (pre) 3.33 0.797
3.219 0.003**
Fluency (post) 3.80 0.791
Flexibility (pre) 4.35 0.580
0.467 0.643
Flexibility (post) 4.40 0.672
Elaboration (pre) 3.55 0.846
4.309 <0.001**
Elaboration (post) 4.13 0.791
Curiosity (pre) 4.00 0.716
2.223 0.032
Curiosity (post) 4.30 0.723
28
Enthusiasm (pre) 4.05 0.714

4.201 <0.001**
Enthusiasm (post) 4.50 0.555
Sensitivity (pre) 3.85 0.802
1.599 0.118
Sensitivity (post) 4.05 0.749
Humor (pre) 3.95 0.815
2.223 0.032
Humor (post) 4.25 0.981
Self-confidence (pre) 3.70 0.723
1.778 0.083
Self-confidence (post) 3.93 0.797
Courage (pre) 3.65 0.802
1.030 0.309
Courage (post) 3.80 0.823
Commitment (pre) 3.68 0.764
3.667 <0.001**
Commitment (post) 4.20 0.823
Independence (pre) 3.75 0.809
2.726 0.010**
Independence (post) 4.15 0.700
** p ≤ 0.01
4.3 Results of the Infographic Assessment

Table 3 presents the rubric used to assess the infographic. Works wherein the
obtained total score is lower than 7 points were considered basic. Works scored
7–12 points were evaluated as developing. Works scored 13–18 points were
evaluated as proficient level. Those scored more than 18 points were evaluated
as advanced. Table 6 shows the scores obtained in the sessions. Criteria for the
assessment infographic were divided into six categories: design, composition,
color, font, presentation, and source reliability. An average score in each item
ranging from 3.50 to 4.00 points was categorized as advanced, scores from 2.50
to 3.49 points as proficient, scores from 1.50 to 2.49 points as developing, and
scores below 1.50 points as basic. Figure 1 shows some examples of the students’
infographics. The title of infographics translated into English was shown under
each work. The analysis of the infographics produced by the students reveals
that most students received average scores in source reliability at the advanced
level, while average scores in other categories were at the proficient level. The
average total scores were at the advanced level. Infographic work assessment
showed that the students had good design skills and could select reputable
information sources. Nevertheless, the students had the lowest scores in
composition. Thus, the instructor should be more mindful of improving this skill
of the students.
Table 6: Results of the infographic assessment by the instructor

Items M SD Interpretation
Design 3.10 0.71 Proficient
Composition 3.05 0.71 Proficient
Color 3.28 0.75 Proficient
Font 3.10 0.67 Proficient
Presentation 3.15 0.83 Proficient
29
Source reliability 3.70 0.79 Advanced

Total 19.58 3.45 Advanced
4.4 Attitudes toward Infographic Design

After designing the infographics, the researcher had the students evaluate their
infographic design. Table 7 presents their attitudes toward infographic design.
Average scores can then be interpreted as follows. Average scores of 1.00–1.49
points were considered as strongly disagree. Average scores of 1.50–2.49 points
were considered disagree. Average scores of 2.50–3.49 points are considered
neutral. Average scores of 3.50–4.49 points are considered agree. Average scores
of 4.50 to 5.00 points are considered strongly agree. The results showed that
more students had a positive attitude toward their work, believing that their
work was useful for others and acknowledging that with more practice in
designing, they could produce even better work. Furthermore, most of the
students demonstrated proficiency in using the design program and devoted
considerable time to designing and creating their work because they aspired for
their work to be the best.
Table 7: Attitudes toward infographic design

Items M SD Interpretation
I have more confidence in designing. 4.20 0.608 agree
I am proud of my work. 4.42 0.747 agree
I am satisfied with my work. 4.35 0.662 agree
Practicing design more makes my work
4.65 0.580 strongly agree
better.
The infographic came out exactly as I
4.10 0.744 agree
expected.
I am fluent in designing. 3.70 0.939 agree
It took me a long time to create a great de-
4.15 0.700 agree
sign.
It took me a long time to design because of
3.40 1.172 neutral
a lack of expertise in using the program.
I can solve problems on my own without
3.70 0.823 agree
asking for help from the instructor.
My work is creative and unique. 3.62 0.740 agree
My work is elaborate. 3.80 0.791 agree
My work is interesting. 4.03 0.660 agree
My work is helpful to others. 4.52 0.751 strongly agree
30
Figure 1: Students’ infographics examples
5. Discussion
The findings suggest that students experienced increased creativity after
learning through knowledge management using storytelling with infographics.
This could be attributed to the learning process, which allowed the students to
freely express their creativity without limitations on the topic. The students
could express themselves freely relative to the knowledge that they had
researched and synthesized in infographic storytelling.
31
Furthermore, students' higher levels of creativity may come from practicing

telling stories with infographics. Designing an infographic involves making
decisions about layout, color schemes, typography, and visual elements. This
design process encourages students to think creatively about how to present
information in a visually appealing and accessible way. Infographics often draw
from multiple disciplines. Engaging with infographics can expose students to
diverse perspectives and ideas, fostering creativity through cross-disciplinary
learning. This is consistent with the research of Yang et al. (2022), which
examined the effectiveness of digital storytelling on foreign language learners’
English speaking and creative thinking. In their study, digital storytelling was
realized in the form of an interdisciplinary project integrated into a partnership
between an English course and a computer course, with the class time of the
former devoted to the content design and that of the latter to the multimedia
design of learner-generated digital stories. The participants were required to
work in small groups to create their digital stories in the target language,
English, under an eight-week interdisciplinary curriculum. A two-group quasi-
experiment with a pretest and posttest design was then conducted to compare
the participants’ learning outcomes. The findings revealed the meaningful
learning opportunities that digital storytelling fostered in the students’
development to become proficient English speakers and creative thinkers.
Another notable observation was that the students demonstrated more fluent
thinking skills, enabling them to generate many ideas in a limited time. This
finding aligns with Handayani et al. (2021), who also noted that practicing
thinking skills through learning by doing fosters creativity, particularly fluency
in thinking. Students produce many ideas and various answers within a limited
time.
Moreover, in addition to their developed creativity, the students excelled in

obtaining information from reputable sources. This outcome could be attributed
to the knowledge creation and acquisition step, during which the teacher
provided guidance on obtaining information from reputable sources. As a result,
the students were able to disseminate accurate information, contributing to the
creation of a society where people share information and express their opinions
based on facts and reasons (Ammirato et al., 2021; De Bem Machado et al., 2022).
However, when the students evaluated their creativity before and after the
learning activities, it became evident that they still lacked the confidence and
courage to express differing ideas. This indicates the need for the instructor to
work on instilling more confidence in the students, as prolonged low confidence
might hinder their ability to think creatively and outside the box. This result
corresponds to the findings of Ten Haven et al. (2022), which emphasized the
importance of the courage to make mistakes and learn from trial and error as a
factor in promoting creativity. To foster creativity effectively, students must be
open-minded, confident in their own ideas, and willing to think
unconventionally. Cultivating such confidence in students is not an easy task,
and it presents challenges for instructors and learning designers in researching
32
and developing suitable and effective methods to increase students' confidence

and courage to think outside the box.
6. Conclusion
Innovations and technologies have currently replaced many professions.
Undergraduate students must possess various skills to survive the competition
in the labor market. Creativity is one of the most in-demand skills in the digital
age. Graduates with a high level of creativity are likely to be more desirable to
employers than those who are not. In this study, knowledge management using
storytelling with infographics was used as an intervention to improve the
creativity of undergraduate students enrolled in the Multimedia Design and
Production course. In this study, we classified creativity into four components:
originality, fluency, flexibility, and elaboration. The results show that students’
creativity scores were significantly higher in all components. The students had
average posttest scores in all creativity components at the proficient level, except
in fluency. However, the difference between pretest and posttest scores showed
that fluency had the highest development. Before the learning activities, most
students could not finish the creativity assessment form within the time limit.
However, after engaging in learning through knowledge management using
storytelling with infographics, most students had improved their creativity
proficiency, enabling them to complete the creativity assessment form within the
time limit. Furthermore, most students received average scores in infographic
design at the advanced level. Therefore, it can be concluded that knowledge
management using storytelling with infographics could raise the creativity of
undergraduate students. However, the pretest and posttest scores of self-
confidence in this study had no statistically significant difference. This indicates
that the instructor should focus on enhancing the students’ confidence in
expressing their creativity as prolonged low confidence may limit their ability to
think creatively.
7. Limitations and Suggestions for Further Research

One limitation of this study is its sample, which only includes students from the
Faculty of Arts at Silpakorn University enrolled in the Multimedia Design and
Production course. As such, it may not be representative of all undergraduate
students in the Faculty of Arts in Thailand. Future research could address this
limitation by examining undergraduates from multiple universities in Thailand.
Additionally, future studies may use qualitative data collection and analysis to
get more insights. Conducting semi-structured interviews with open-ended
questions allows participants to provide data that is richer and more nuanced
than what can be captured through a survey using a 5-point Likert scale.
Ethical Consideration
This research is certified by the human research committee at Silpakorn
University under the Declaration of Helsinki. This research work complies with
international ethical principles and the laws, regulations, and requirements of
Thailand. This research work is subjected to an exemption review with
Certification No. 66.0103–001
33
8. References
Aldalalah, O. M. A. (2021). The Effectiveness of Infographic via Interactive Smart Board
on Enhancing Creative Thinking: A Cognitive Load Perspective. International
Journal of Instruction, 14(1), 345-364. https://doi.org/10.29333/iji.2021.14120a
Almeida, L. S., Prieto, L. P., Ferrando, M., Oliveira, E., & Ferrándiz, C. (2008). Torrance
test of creative thinking: the question of its construct validity. Thinking Skills and
Creativity, 3(1), 53–58. https://doi.org/10.1016/j.tsc.2008.03.003
Ammirato, S., Linzalone, R., & Felicetti, A. M. (2021). Knowledge management in
pandemics. A critical literature review. Knowledge Management Research &
Practice, 19(4), 415-426. https://doi.org/10.1016/j.copsyc.2022.101314
Annala, J., & Mäkinen, M. (2017). Communities of practice in higher education:
contradictory narratives of a university-wide curriculum reform. Studies in
Higher Education, 42(11), 1941-1957.
https://doi.org/10.1080/03075079.2015.1125877
Apriyanti, N., Razak, R. A., Rahim, S. S. A., Shaharom, M. S. N., & Baharuldin, Z. (2020).
Infographic instructional media as a solution and innovation in physics learning
for senior high school students in Indonesia. International Journal of Information
and Education Technology, 10(10), 773-780.
https://doi.org/10.18178/ijiet.2020.10.10.1457
Arthur, L. (2016). Communities of practice in higher education: Professional learning in
an academic career. International Journal for Academic Development, 21(3), 230-241.
https://doi.org/10.1080/1360144X.2015.1127813
Barlow, B., Webb, A., & Barlow, A. (2021). Maximizing the visual translation of medical
information: A narrative review of the role of infographics in clinical pharmacy
practice, education, and research. Journal of the American College of Clinical
Pharmacy, 4(2), 257-266. https://doi.org/10.1002/jac5.1386
Bencsik, A., Juhász, T., & Machova, R. (2014). Mentoring practice on behalf of knowledge
sharing in the light of education. Acta Polytechnica Hungarica, 11(9), 95-114.
https://doi.org/10.12700/aph.11.09.2014.09.6
Chicca, J., & Chunta, K. (2020). Engaging students with visual stories: Using infographics
in nursing education. Teaching and Learning in Nursing, 15(1), 32-36.
https://doi.org/10.1016/j.teln.2019.09.003
Cinelli, M., De Francisci Morales, G., Galeazzi, A., Quattrociocchi, W., & Starnini, M.
(2021). The echo chamber effect on social media. Proceedings of the National
Academy of Sciences, 118(9), e2023301118.
https://doi.org/10.1073/pnas.2023301118
Craig, W., Boniel-Nissim, M., King, N., Walsh, S. D., Boer, M., Donnelly, P. D., ... &
Pickett, W. (2020). Social media use and cyber-bullying: A cross-national analysis
of young people in 42 countries. Journal of Adolescent Health, 66(6), S100-S108.
https://doi.org/10.1016/j.jadohealth.2020.03.006
De Bem Machado, A., Secinaro, S., Calandra, D., & Lanzalonga, F. (2022). Knowledge
management and digital transformation for Industry 4.0: a structured literature
review. Knowledge Management Research & Practice, 20(2), 320-338.
https://doi.org/10.1080/14778238.2021.2015261
Di Battista, A., Grayling, S., & Hasselaar, E. (2023). Future of jobs report 2023. World
Economic Forum, Geneva, Switzerland.
https://www.weforum.org/reports/the-future-of-jobs-report-2023
Dorneles, L. L., Martins, V. D. P., Morelato, C. S., Goes, F. D. S. N. D., Fonseca, L. M. M.,
& Camargo, R. A. A. D. (2020). Development of an animated infographic on
Permanent Health Education. Revista Latino-Americana de Enfermagem, 28, e3311.
https://doi.org/10.1590/1518-8345.3536.3311
34
Evans, N. (2012). Destroying collaboration and knowledge sharing in the workplace: a

reverse brainstorming approach. Knowledge Management Research & Practice, 10,
175-187. https://doi.org/10.1057/kmrp.2011.43
Giumetti, G. W., & Kowalski, R. M. (2022). Cyberbullying via social media and well-
being. Current Opinion in Psychology, 45, 101314.
https://doi.org/10.1016/j.copsyc.2022.101314
Gube, M., & Lajoie, S. (2020). Adaptive expertise and creative thinking: A synthetic
review and implications for practice. Thinking Skills and Creativity, 35, 100630.
https://doi.org/10.1016/j.tsc.2020.100630
Habeeb, M. J. (2020). Effectiveness of Teaching Social Studies Subject using Infographic
Techniques and Its Effect on Achievement and Visual Intelligence. PalArch's
Journal of Archaeology of Egypt/Egyptology, 17(3), 727-745.
https://doi.org/10.48080/jae.v17i3.166
Handayani, S. A., Rahayu, Y. S., & Agustini, R. (2021). Students’ creative thinking skills
in biology learning: fluency, flexibility, originality, and elaboration. Journal of
Physics: Conference Series, 1747(1), 012040. https://doi.org/10.1088/1742-
6596/1747/1/012040
Hendrik, B., Ali, M. N., Nayan, N. M., Mat Isa, N. A., & Masril, M. (2022). A New
Robotic Learning Activity Design to Increase the Figural Creativity: Originality,
Elaboration, Flexibility, and Fluency. International Journal on Advanced Science
Enginering Information Technology, 12(1), 114-120.
https://doi.org/10.18517/ijaseit.12.1.14109
Jones, N. P., Sage, M., & Hitchcock, L. (2019). Infographics as an assignment to build
digital skills in the social work classroom. Journal of Technology in Human
Services, 37(2-3), 203-225. https://doi.org/10.1080/15228835.2018.1552904
Keiser, N. L., & Arthur Jr, W. (2021). A meta-analysis of the effectiveness of the after-
action review (or debrief) and factors that influence its effectiveness. Journal of
Applied Psychology, 106(7), 1007. https://doi.org/10.1037/apl0000821
Lewis, B., & Rush, D. (2013). Experience of developing Twitter-based communities of
practice in higher education. Research in Learning Technology, 21.
http://dx.doi.org/10.3402/rlt.v21i0.18598
MacAvaney, S., Yao, H. R., Yang, E., Russell, K., Goharian, N., & Frieder, O. (2019). Hate
speech detection: Challenges and solutions. PloS one, 14(8), e0221152.
https://doi.org/10.1371/journal.pone.0221152
Majchrzak, A., More, P. H., & Faraj, S. (2012). Transcending knowledge differences in
cross-functional teams. Organization science, 23(4), 951-970.
https://doi.org/10.1287/orsc.1110.0677
Matamoros-Fernández, A., & Farkas, J. (2021). Racism, hate speech, and social media: A
systematic review and critique. Television & New Media, 22(2), 205-224.
https://doi.org/10.1177/1527476420982230
McDonald, J., & Cater-Steel, A. (2016). Implementing communities of practice in higher
education: Dreamers and schemers. Springer.
Mohamed, M., Stankosky, M., & Murray, A. (2004). Applying knowledge management
principles to enhance cross‐functional team performance. Journal of knowledge
management, 8(3), 127-142. https://doi.org/10.1108/13673270410541097
Ocobock, C. (2020). Infographics, Podcasts, and Videos: Promoting Creativity and
Building Transferable Skills among Undergraduate Students. Teaching and
Learning Anthropology, 3(2), 61-71. https://doi.org/10.5070/T33247050
Osman, M. A., Noah, S. A. M., & Saad, S. (2022). Ontology-based knowledge
management tools for knowledge sharing in organization—a review. IEEE
Access, 10, 43267-43283. https://doi.org/10.1109/ACCESS.2022.3163758
Papadopoulou, P., Lytras, M., & Marouli, C. (2016). Capstone projects in STEM
education: Novel teaching approaches, mentoring and knowledge management
35
for empowering students. In EDULEARN16 Proceedings (pp. 5675-5685). IATED.

https://doi.org/10.21125/edulearn.2016.2358
Paz, M. A., Montero-Díaz, J., & Moreno-Delgado, A. (2020). Hate speech: A systematized
review. Sage Open, 10(4). https://doi.org/10.1177/2158244020973022
Pennycook, G., & Rand, D. G. (2021). The psychology of fake news. Trends in cognitive
sciences, 25(5), 388-402. https://doi.org/10.1016/j.tics.2021.02.007
Pierri, F., & Ceri, S. (2019). False news on social media: a data-driven survey. ACM
Sigmod Record, 48(2), 18-27. https://doi.org/10.1145/3377330.3377334
Puccio, G. J. (2017). From the dawn of humanity to the 21st century: Creativity as an
enduring survival skill. The Journal of Creative Behavior, 51(4), 330-334.
https://doi.org/10.1002/jocb.203
Reaburn, P., & McDonald, J. (2017). Creating and facilitating communities of practice in
higher education: Theory to practice in a regional Australian university.
Communities of practice: Facilitating social learning in higher education, 121-150.
https://doi.org/10.1007/978-981-10-2879-3_6
Rocha, Y. M., de Moura, G. A., Desidério, G. A., de Oliveira, C. H., Lourenço, F. D., & de
Figueiredo Nicolete, L. D. (2021). The impact of fake news on social media and
its influence on health during the COVID-19 pandemic: A systematic review.
Journal of Public Health, 1-10. https://doi.org/10.1007/s10389-021-01658-z
Shen, X. (2023). Play and Scientific Creativity: A Critical Review and an Integrative
Theoretical Framework. The Journal of Creative Behavior.
https://doi.org/10.1002/jocb.596
Srivichai, P., Meksamoot, K., Jengjalern, A., & Chakpitak, N. (2012). Integrated
knowledge management system based mentoring for new university staff
development. 2011 Ninth International Conference on ICT and Knowledge
Engineering, 150-155. https://doi.org/10.1109/ICTKE.2012.6152397
Sumbal, M. S., Ali, M., Sahibzada, U. F., Mir, F. N., Tariq, A., & Munir, H. (2021). Big
Data Based Knowledge Management vs. Traditional Knowledge Management:
A People, Process and Technology Perspective. Journal of Information Science &
Engineering, 37(5), 1053-1065. https://doi.org/10.6688/JISE.202109_37(5).0005
Ten Haven, A., Pragt, E., Luijk, S. J. V., Dolmans, D. H., & van Mook, W. N. (2022).
Creativity: A viable and valuable competency in medicine? A qualitative
exploratory study. Medical Teacher, 44(10), 1158-1164.
https://doi.org/10.1080/0142159X.2022.2072278
Terren, L., & Borge-Bravo, R. (2021). Echo chambers on social media: A systematic
review of the literature. Review of Communication Research, 9, 99-118.
https://doi.org/10.12840/ISSN.2255-4165.028
Thongkoo, K., Panjaburee, P., & Daungcharone, K. (2019). Integrating Inquiry Learning
and Knowledge Management into a Flipped Classroom to Improve Students'
Web Programming Performance in Higher Education. Knowledge Management &
E-Learning, 11(3), 304-324. https://doi.org/10.34105/j.kmel.2019.11.016
Tsai, S. T., Huang, H. Y., & Chang, T. W. (2020). Developing a motion infographic-based
learning system for effective learning. Education sciences, 10(9), 247.
Verhoef, P. C., Broekhuizen, T., Bart, Y., Bhattacharya, A., Dong, J. Q., Fabian, N., &
Haenlein, M. (2021). Digital transformation: A multidisciplinary reflection and
research agenda. Journal of business research, 122, 889-901.
https://doi.org/10.1016/j.jbusres.2019.09.022
Vial, G. (2021). Understanding digital transformation: A review and a research agenda.
Managing Digital Transformation, 13-66. https://doi.org/10.4324/9781003008637-
4
36
Vukanović-Dumanović, V., Avlijaš, G., & Jokić, S. (2022). After Action Review as a tool
for implementation of the knowledge management program. Ekonomika, 68(4),
29-40. https://doi.org/10.5937/ekonomika2204029V
White, B., Quehl, K., Costello, J. W., & Costello, J. W. (2022). A Workshop on
Brainstorming Workshops Successful brainstorming with diverse creators in a
hybrid work environment. Proceedings of the 14th Conference on Creativity and
Cognition, 494-497. https://doi.org/10.1145/3527927.3531204
Yang, Y. T. C., Chen, Y. C., & Hung, H. T. (2022). Digital storytelling as an
interdisciplinary project to improve students’ English speaking and creative
thinking. Computer Assisted Language Learning, 35(4), 840-862.
https://doi.org/10.1080/09588221.2020.1750431
37

Vol. 22, No. 10, pp. 37-53, October 2023
“I find it very difficult to go to work; it is

emotionally exhausting”: Understanding the
Burnout and Underlying Emotions among
Malaysian University Academics
Fairuz A'dilah Rusdi
University of Malaysia Kelantan, Malaysia
Ateerah Abdul Razak

Zaleha Embong
Abstract. This study investigated the academic burnout experiences of

staff at various universities in Malaysia. Qualitative content analysis was
used to deconstruct the interview texts. The study involved n = 12
academics (ages 28–46) from several Malaysian higher education
institutions. The key findings of this study centre on resilience,
engagement and burnout. Eight themes were found: warning indicators
of burnout, academic burnout triggers, coping strategies, work-life
balance, love for the career, resilience and surviving in academia, the
impact of burnout phases on oneself, and stress-inducing variables. The
themes were connected to the academics’ unique experiences with the
burnout phenomenon and problems in their current profession. Data
from interviews revealed that most academics considered their work
taxing and blamed issues like unmanageable workloads, excessive
university requirements and a lack of resources for their burnout
episodes. The results also demonstrate that academics were aware of the
difficulties and institutional circumstances that add to the complexity of
their day-to-day burnout experiences, offering a rich picture of their
individual perspectives. Furthermore, individuals with specific
personality types, such as perfectionistic traits or standards, tend to be
more vulnerable to burnout. The results imply that for academics who
experience burnout, relevant interventions and emotional support are
essential and that the most resilient are those who maintain a positive
attitude and have the capacity to cope with the challenges and
responsibilities of an academic profession.
Keywords: burnout; academics; resilience; workload; job engagement
©Authors
38
1. Introduction
Academic expectations on university faculty members include an excessive
workload, strict deadlines for research and publications, an unbalanced work-life
balance and a lack of adequate support (Parmar et al., 2022). These directives
could subject the faculty to long-term stress and exhaustion (Garcia-Rivera et al.,
2022; Henny et al., 2014; Panatik et al., 2012; Vesty et al., 2015). Although it is
known that burnout impacts faculty members’ social, psychological and physical
health, it is yet unknown how it affects their work-life. We set out to evaluate the
effects of burnout on the daily work-life of academics from various disciplines.
Depersonalisation, a lack of personal accomplishment, and long-term exhaustion

are the three main components of the phenomenon known as burnout, which is a
response to high stress levels in jobs that require communicating with and
working with others (Maslach & Jackson, 1981). People’s mental and physical
health is impacted by how effectively they can adapt to their settings’
psychosocial, contextual and physical demands (Sterling, 2012). While
physiological reactions to stress are typical and occasionally helpful, burnout may
develop from persistent exposure to high levels of professional stress (Psyhalto et
al., 2011).
According to previous studies, academics in higher education institutions may

experience everyday burnout due to the rising demands and constantly shifting
circumstances in areas like economic, political and social life (Henny et al., 2014;
Panatik et al., 2012). The increased number of academics quitting their jobs at
universities worldwide is one of the issues that university administration is
dealing with (Ologunde et al., 2012). Evidence suggests that severe job burnout
and other undesirable circumstances lead academics to leave (Ologunde et al.,
2012). Malaysia’s Ministry of Education, particularly at private universities, faces
tremendous problems retaining valuable academic staff.
As part of the nation’s goals to become a global hub for higher education by 2020,
five of Malaysia’s 20 public institutions have already been granted research
university status (Ministry of Higher Education Malaysia, 2015). These research
universities have the luxury of handling administrative, financial, student
admissions, human resources and academic issues separately (ahzah et al., 2016).
Even more public and private institutions of higher learning have been founded
due to this development and revisions to the existing colleges and universities.
Performance has become a significant indicator in Malaysian academia due to all
these changes, with academics now experiencing more stress and pressure due to
the limited resources provided by institutions and the growing competitive
pressure (Henny et al., 2014). Owing to the current intense workload placed on
these academics, it is not surprising that many academics are experiencing rising
levels of stress and burnout due to the growing number of responsibilities
demanded of them.
For this study, because the qualitative approach focuses on people’s subjective
lived experiences, the researcher chose phenomenology within qualitative
research to learn about the participants’ experiences of a phenomenon.
39
Phenomenology was chosen for this study because it focuses on how people
experience reality, even though constructivist grounded theory was also
considered as a research approach. The researcher used the phenomenological
study to get insights into how participants perceived their circumstances and
analysed people who had experienced burnout and how it affected their jobs,
among other things. A few participants were interviewed for the
phenomenological investigation to obtain a wider diversity of viewpoints and a
deeper understanding of the phenomenon.
The JD-R model, which reflects an organisational psychological approach, was the
foundation for the study’s framework. Several workplace factors can be put into
a reasonably simple model to explain how burnout and work engagement may be
influenced. Although exposure to job demands predicts burnout, job resources
are essential in determining work engagement and a decline in cynicism (Bakker
et al., 2007). This aligns with the social cognitive theory (SCT) of Albert Bandura
(1977, 2001), a psychological theory explaining behaviour. Self-efficacy is
predicted to be related to positive affective-cognitive outcomes like engagement
at work by SCT. The existence of a descending “loss spiral” where high job
demands lead to exhaustion and simultaneously contribute to an increase in job
demands over time has also been confirmed. These reciprocal relationships are
consistent with the idea of “gain spirals” in the conservation of resources theory
(Demerouti et al., 2004).
Since higher education institutions increasingly identify academics as being under

a lot of stress, this study aims to understand better the factors influencing burnout
among those working in these higher learning institutions. Studies have also
outlined how job demands affect the well-being of persons working in the higher
education sector, which is commonly acknowledged as a setting for high-stress
work. This study also sheds light on academics’ coping strategies and coping
processes for dealing with challenges at work from the standpoint of the academic
environment.
2. Materials and Methods

In-depth semi-structured interviews with academics were conducted in this
study’s longitudinal qualitative research methodology to elicit their accounts of
burnout and “gain insights into a person’s subjective experiences, opinions, and
motivations” (Busetto et al., 2020, p. 3). In-depth interviews were chosen since
they allow for a more thorough examination of a subject’s experiences and a more
comprehensive data collection.
Since in-depth interviews enable a more extensive examination of participant

experiences than other methods, they were chosen for the qualitative approach.
To track changes over time in the academics’ three quadrants (resilience,
engagement and burnout levels) and because the researcher is interested in each
academic’s unique experience over the course of a semester, a longitudinal
qualitative research design (LQR) was chosen. Incorporating phenomenology into
this study and LQR was done to understand better the meanings that university
academics give to their experiences with burnout at work and how resilience and
40
engagement have changed their day-to-day activities. According to Ashworth

(2003), phenomenology is an approach for delving into a particular event or
person’s lived experience; in other words, it examines the lived events occurring
or phenomenon as it appears to the individual. They focus on first-person stories
of burnout appearing in people’s experiences and paint a vivid picture of their
reality (Creswell, 1998).
This study comprised in-depth interviews with those individuals who were
burned out and engaged to acquire detailed narratives on the experience of
burnout in Malaysian university academics. Each person’s unique burnout and
engaged experiences were assessed, and content analysis was employed to find
trends and changes shared by the academics. Additionally, the goal was to
accurately and ultimately capture the involvement and resilience of respondents
and describe how these events occur and how each academic feels about them.
3. Participants
Most phenomenological studies employ small participant groups (Smith &
Osborn, 2003). Standard sample sizes for phenomenology research are one to ten
people (Starks & Brown, 2007). However, for the researcher to deliberately select
interview subjects contributing to a thorough understanding of the phenomenon,
Creswell (1998) recommends interviewing five to 25 persons.
As a result, twelve academics were chosen to participate in the study’s in-depth

interviews. The participants were purposely chosen to demonstrate diversity.
These 12 academics were selected from a group (n = 681) who consented to
participate in follow-up interviews after completing the Maslach Burnout
Inventory: Educator’s Survey (MBI-ES). Four of the 12 participants were chosen
because their burnout subscale scores were relatively low, two because their
burnout subscale scores were average, and six were chosen because their burnout
subscale scores were high, indicating they had significant burnout (Creswell &
Eklund, 2006).
A greater proportion of individuals in the high burnout cohort than in the average
and low cohorts was chosen to concentrate more on academics who had
experienced several burnout experiences during their careers rather than those
who were less burned out or in the medium average type of burnout.
Twelve university academics between the ages of 28 and 46 made up the

participants (M = 35.5, SD = 5.8); nine were women (75%), and three were men
(25%). Their years of academic experience they were ranged from 2 to 19 (M = 7.4
years, SD = 5.5). These participants worked as academics in public (n = 8, 66.7%)
and private (n = 4, 33.3%) universities. Academic positions comprise 50%
Lecturers, 33.3% Senior Lecturers, 8.3% Assistant Professors, and 8.3% University
Teachers. All participants, n = 12, were taken from universities under the control
of the Ministry of Higher Education of Malaysia.
41
4. Procedures
After making the necessary arrangements to meet at a particular time, date and
location, the twelve selected academics were invited to offer comprehensive
narrations of their experience with the burnout phenomenon, mainly connected
to their path as academics within the university settings. These academics were
free to discuss the specifics of their academic backgrounds and day-to-day duties.
After that, they were thoroughly questioned about their encounters with the
burnout phenomenon, including how incidents influenced their levels of job
engagement, their work environments, their stories of resilience in handling
incidents, and their feelings, among others.
As a starting point for the investigation into burnout, the sample (n = 12) was
tracked over time for any changes. Over one semester, three in-depth interviews
were done (T1, T2 and T3). A thorough account of their stories from the three
different time points needed to be developed to comprehend the occurrences and
trajectories of burnout and engagement at 4-6 weeks after the start of the semester
(T1), at week 9 of the semester (T2), and once more at 12-14 weeks before the end
of the semester (T3) (with a one-month gap between each time point).
Each interview lasted 30 to 90 minutes. Interviews were recorded after getting

consent in writing. The option to leave the study at any time was made clear to
participants, and confidentiality was ensured. The subjects’ identifying details
were removed from the interview transcripts to maintain data confidentiality.
5. Measures
A semi-structured interview script was developed to examine factors related to
how academics described their own work experiences, the academic environment
they worked in, and how burnout was perceived in this particular period. The
script’s foundation was the previous research, theoretical framework and
available literature gaps. The first general inquiry in the interviews was, “Describe
a specific situation(s) or event(s) reflective of your experiences with burnout”.
Following this were open-ended questions (core/essential questions) about

personal, organisational and job-related demands, such as “Can you offer some
examples of the daily experiences of burnout you have as an academic at this
university?” “How did you handle these episodes of burnout?” and “How has
burnout affected your perception of your career as a university academic?” The
interviewer could better understand the subject by letting participants react freely
and unrestrictedly to the scripted questions, whose order was not strictly adhered
to.
6. Data Analysis
ATLAS.ti version 7 was used to manage the study’s data since it can organise and
make the process of analysing qualitative data in large amounts of text more
accessible (Muhr, 1991). As a result, 36 interview transcripts (n = 12 x three-time
points) were entered into ATLAS.ti. However, the transcribed content was coded
and analysed using manual methods.
42
After interviews were transcribed using the phenomenological-hermeneutic

technique, they were submitted to the three-step content analysis process: 1) naive
reading, 2) structural analysis, and 3) critical interpretation, as recommended by
Lindseth and Norberg (2004). The interviews produced raw data, which was
audio-recorded. Verbatim transcriptions of the initial discussions were made. The
audio files and the texts were cross-referenced. Following the transcripts’
completion and the interviews’ finalisation, the data were examined more
thoroughly to comprehend the context of the participants’ experiences. Initial
codes were obtained in the naive reading stage (familiarisation). A text was
created using fragments pertinent to the studied subject (unit of analysis).
Preliminary codes were analysed to tie any new codes to the research concerns.
As seen in Table 1, the interview transcript was initially broken down into
meaning-conveying units (words, phrases, or text fragments) during the
structural analysis stage. This led to grouping meaning units with related content.
“Condensed Meaning Units” were created. Meaningful links were established
after the meaning units had been separated and organised. These relationships
were further theorised, divided into subthemes and organised into themes.
Table 1. An illustration of how meaning units, condensed meaning units, subthemes,

and themes were coded during the interpretation process (n = 12)
Interpretation of the
Meaning Unit/
Participant meaning (Condensed Subtheme Theme
Code text
Meaning Unit)
Respondents clearly stated

You are setting up the significance of training
high KPI (Key and having enough
Performance resources to carry out the
Triggers
Indicators) for the job properly; some
Lack of of
lecturers but you don’t respondents even
H1 appropriate burnout
provide the necessary expressed their
resources among
resources and you displeasure over the
academics
don’t even send them university
out for any training or administration’s failure to
whatsoever send staff members to
necessary training.
The objective of the critical interpretation stage is to comprehend the subjects

discussed in the interview transcript more thoroughly by building on the prior
knowledge and conclusions from the naive reading and structural analysis stages
and how they connect to the study’s context, the research questions, and the
relevant literature (Gustafsson et al., 2008; Lindseth & Norberg, 2004). Simply put,
a full grasp of the significance of the lived experience is established after
considering the text, the initial understanding, themes and relevant literature.
Here, the clarifying process follows a spiral pattern in which the full interview
text is absorbed by considering some text elements, which are included to explain
the whole (Ricoeur, 1976).
43
7. Results
Summary of Findings
The first stage of data collecting (the beginning of the semester), summarised in
Table 2, produced eight core themes (and 33 subthemes) due to the content
analysis. Four themes and eight subthemes were produced in the second stage
(halfway through the semester), as shown in Table 3, and three themes and eight
subthemes were developed in the third stage, summarised in Table 4. The first
time point offered more themes than the subsequent ones since the “core
questions” or “essential questions” were asked in the opening first stage rather
than the next stages.
Table 2. Themes and subthemes during Timepoint 1 (Start of the semester) (n = 12)
Timepoint Themes Definition Subthemes
• Feelings of emotional drain.
Depiction of The illustration of the • Emotional exhaustion upon
academics tell- indicative symptoms of thinking about the workplace.
tale signs of burnout experienced • Feeling exhausted and lacking
burnout among academics energy due to having trouble
getting along with bosses.
• Reference to the absence of
institutional resources.
• Poorly handled promotions or
assessments.
• Failure to receive social support
from superiors.
• An unbalanced workload and
excessive workplace demands.
Triggers of Factors leading to • Conflicting employment demands
burnout among academics exhaustion and and task overload.
academics burnout • Unreasonable expectations from
Timepoint 1 university administration and
(T1) superiors.
(Start of the • High standards for oneself and
semester) irrational ideas.
• Student demands and issues.
• Incompatibility of values
Eight
(incongruity between personal
Themes: (33
and university values)
Subthemes)
• Effective communication, social
support from colleagues, and a
positive, encouraging work
atmosphere.
• Close family relationships at
home, actively-involved spouses
acting as a protective shield.
Coping techniques, • Sports as a burnout coping
Coping
strategies and skills in mechanism.
mechanisms
preventing burnout • Adopting a positive outlook on
against burnout
among academics burnout and letting go of
negativity (anxiety and
pessimism).
• Taking a vacation from work,
rejuvenating or distancing
themselves.
• Religion and spirituality as
buffers to counter burnout.
44
• Incorporating self-care practices

Thoughts of and work-life balance.
The ideals of bringing • Prioritising home life after work
bringing work
work home or otherwise. hours with family and children.
home
• Love of knowledge, research and

Academics’ sense of publication.
The joy of being fulfilment in academia • A predisposition for students.
an academic and fondness towards the • Passion for teaching and serving
career. others/larger community.
• Friendly working-hours.
• The necessity of resiliency due to
no other options.
• Developing resilience through
On resiliency and adaptation and adjustments.
Resiliency
overcoming burnout and • Resilience as part of academics’
factors
other difficulties emotional intelligence.
• The capacity to endure adversity
as a result of surviving hardship
in the past.
• Psychological detachment from
relationships and the environment
Effect of burnout Cynicism and
as well as depersonalisation.
phases on depersonalisation
• Growth of cynicism and a
participants brought on by burnout.
pessimistic, uncaring outlook on
life.
• Grading assignments and the
university curriculum.
The description of stress- • Stress brought on by university
Factors management, policy or both.
triggering factors leading
triggering stress • The concept of the Research
to burnout by academics.
University (RU) was introduced,
along with the demanding KPI.
Table 3. Themes and subthemes during Timepoint 2 (Middle of the semester) (n = 12)
• Inability to handle increasing
workload.
• Decreasing levels of engagement,
Illustration of Academics give examples demotivation and pessimism.
the more of the less favourable • Expressions of less idealistic
negative aspects features of their ideas on the career.
Timepoint 2 of the career profession. • Visible emotional distress and
(T2) signs depicting burnout levels
(Middle of the not decreasing.
semester)
Descriptions of going on • Holidays not relieving workload.
Four Themes: Taking time off
vacation to get away from • Vacation drawbacks - pending
(10 to deal with
the stresses of work assignments, pileup.
Subthemes) burnout
momentarily.
Academics observed their • Learning-based adaptation or
Indication of the
newly acquired resilient- unintentional assimilation.
presence of an
promoting traits where • Adaptation due to an enhanced
adaptation
they adjust and adapt environment.
process among
favourably under
participants
pressure.
45
Academics consistently • Constant eagerness,

Relentless showed a desire to learn determination and resilience in
attitude and and a willingness to their routine work.
continual grit persevere (resilience) in • Job engagement, prioritisation of
their daily tasks. work duties and students.
Table 4. Themes and subthemes during Timepoint 3 (End of the semester) (n = 12)
• Emotional weariness due to
work expectations, workload
Ongoing Continued display of imbalances, workplace
evidence of burnout and conflicts and other external
exhaustion and exhaustion in this final causes.
burnout not stage of data collection • Burnout meltdowns caused by
decreasing from academics’ severe workloads and work
Timepoint 3 unrelated to the main academic
T3 (end of profession.
semester) Evidence of • Learning-based adaptation or
participants’ Academics had further unintentional assimilation.
Three adaptive adapted to and • Adaptation due to an
Themes: (6 processes assimilated with the improving environment and
Subthemes) (reappearing demands of the job. support from others.
theme)
The • Keeping a constant positive
demonstration of Academics display attitude.
positive perseverance through • Being immersed and engaged
enthusiasm their levels of resilience in work.
despite and robustness.
difficulties
8. Summary of the Findings

The first, second and third stages of data collection were divided into three
primary parts for the study’s qualitative component. Each segment contained
themes and subthemes unrelated to the current moment. The responses from
academics at T1 focused mainly on how they perceived their professional
positions, burnout and their degree of job engagement at the beginning of the
semester. At T2, they stated their current levels of burnout and reported any
significant changes from T1. The disparities in their involvement and fatigue
towards the conclusion of the semester are finally revealed at T3.
In conclusion, the data analysis identified eight key fundamental themes from the
first data collection stage (T1). The second round of interviews (T2) revealed four
key themes, while the third round (T3) revealed three themes.
1) Individual-level facilitators/inhibitors of burnout

Some academics may naturally possess qualities that prevent burnout; these are
referred to as individual-level burnout facilitators. Others, however, can be
predisposed to burnout due to certain traits. Some people strive for excellence in
all they do and frequently set impossible standards for their daily routines. This
makes them experience burnout much more rapidly and frequently than those
who give themselves the opportunity for error. Due to this, burnout is more prone
to occur in academics with Perfectionist Personality tendencies than those lacking
in these traits.
46
The sample also revealed that those classified as having “grit” were more likely
to succeed than those without it. Grit is the constant desire to complete what has
been begun, the capacity to persevere through difficulties and failures, the desire
to grow and succeed, the inability to give up easily under pressure, and the
willingness to put forth the consistent, occasionally uncomfortable effort to
complete a task (Areepattamannil & Khine, 2018; Duckworth et al., 2007; Yang et
al., 2023). Unquestionably, grit is crucial to the drive and persistence needed to
pursue extremely long-term goals. By adopting grit, one may be able to prevent
burnout.
Additionally, longitudinal data demonstrated that a crucial method to prevent

burnout among academics is to recognise and know when to rest, get away or put
a gap between themselves and all their obligations. Therefore, avoiding burnout
meltdowns requires mastering the ability to know when to stop or take a break.
One factor contributing to some people’s inability to disconnect is the need for
knowledge about when to do so. Knowing your breaking point is essential when
approaching or reaching a burnout juncture. Burnout could be controlled with
stability by juggling obligations at home and work. Respondents also raised
concerns about setting time limits and understanding when to bring work home
to avoid losing concentration while spending valuable time with their children
and families. Physical activity was another personal-level burnout inhibitor.
Academics who were mentally exhausted from balancing numerous obligations
frequently turned to exercise to revive themselves.
2) Interpersonal-level facilitators/inhibitors of burnout

It has been acknowledged that solid interpersonal connections and how they
change over time play a key role in whether academic burnout is facilitated or
inhibited. Academics recognised particular interpersonal facilitators and
inhibitors that influenced burnout reduction by creating trusting relationships
with co-workers they regularly seek advice from and people in similar
circumstances. These connections helped participants get through complex or
unpredictable burnout phases and difficult events they encountered daily. For
many academics in the sample, having close peers and family to confide in and
vent to was an essential interpersonal factor that prevented burnout.
Support from superiors and managers was also crucial, with participants citing
the advantages of openly addressing their concerns at work with their reassuring
bosses whenever necessary. This was an excellent chance to discuss problems and
look for solutions in a welcoming environment, preventing burnout. For those
who did not have the luxury of discussing these matters with their supervisors,
on the other hand, burnout tended to advance because of the absence of social
support, which resulted in supervisory confrontations. The type of assistance
academics received from their spouses at home was also noted. The opportunity
to share their uncertainties with their spouses was one that they sincerely valued.
Instead, those who were not married confided in a close friend or family member
to help them deal with this.
47
3) Organisational-level facilitators/inhibitors
Other topics concerned the academics’ place of employment, such as the faculty
or institution. Academics indicated specific organisational-level facilitators and
inhibitors that either increased or decreased their levels of burnout. These
included particular policies, working practices and behaviours of senior
management. The majority expressed dissatisfaction with their schools’ lack of
essential resources and amenities, and they saw this as a significant barrier to
success. They believed that finding materials themselves made burnout worse
and sped it up. Another organisational-level contributor to academic burnout was
the university administration’s shortcomings and the need for more attention to
staff training. Respondents also mentioned “unseen” behaviours that senior
management members engaged in, like cronyism and improper promotion
practices. This study’s findings were similar to those of other studies where
emotional, cognitive, and physical demands significantly impact burnout (Alloh
et al., 2019; Graizi et al., 2021)
4) Systemic-level facilitators/inhibitors
Structural problems also influence academic burnout. It is possible that these
systemic enablers or impediments – like the tendency to focus on educational
measures or performance indicators – have different effects on different people
and, to some extent, on other nations.
The systemic level comprises the local systems and university policies that
participants interact with, which defines consequences at a systemic level.
Younger academics in the sample disagreed on this point, but they both agreed
that the paradigm shift was to blame for when their burnout peaked. The
implementation of the RU model, however, was viewed as a beneficial local
system by senior and known academics. Some people criticised the need for
adequate planning for the launch of the RU status and that employees with less
experience were thrust into this change without being informed of their tasks or
what to anticipate in the first few months of the new environment.
Due to these conditions, junior academics in the sample were more likely to
experience burnout. The participants emphasised the importance of guidance,
especially while adjusting to the novel idea of being a part of the RU context.
Participants identified the KPI’s requirements as their primary cause of burnout
at this level, which was related to the RU idea. Participants described the system’s
never-ending conflicting demands and difficulty meeting KPI standards. Before
being granted tenure, having specific competencies assessed on junior staff was
quite taxing and contributed to burnout.
9. Discussion
This study sought to identify the causes of burnout experienced by Malaysian
university academics from various higher education institutions throughout the
academic year.
Overall, the findings confirm past studies on academic burnout by showing that
stress, demanding institutional elements, workload expectations, the need to
48
publish and highly competitive environments were the main drivers of burnout
for these academics (Court & Kiman 2008; Khamisa et al., 2017; Vesty et al., 2015).
Change management and work-family conflict are additional essential factors.
The diversity of categories and subcategories emerging supports the notion that
the professional setting of universities offers a variety of situations that can lead
to academic burnout. Our findings provide crucial information on various factors
influencing burnout among academics.
Evidence also shows that academic burnout in Malaysia is caused by a

combination of stressors unique to academia and stressors unique to institutions,
which can be dangerous for people working in higher education. The growing
expectations of these academics and the institutional pressures they must face due
to the demanding nature of today’s academia must be considered. In this study,
frustrated academics’ reactions ranged from contemplating quitting academia,
specifically among younger academics who desire to quit their current university,
to continuously fostering strong and durable ties with colleagues, close friends
and members of the family to counteract any unfavourable or tell-tale indicators
of burnout at work. Additionally, they use a variety of tactics to do this, including
individuals leaving the system by zealously preserving individual autonomy
(e.g., by refusing to participate in activities beyond what is legally needed) and
more.
Summary of the Burnout Dimensions

High levels of work-related stress have been shown to occur throughout career
stages, and different university types are linked to intensified burnout phases
(Khamisa et al., 2017). Academics may be more vulnerable to burnout due to
individual-specific pressures, which can also cause anxiety, demotivation, lower
productivity at work and intentions to quit.
Burnout is also more common among academics with particular personality

types, such as the Perfectionist Personality Type, who pride themselves on being
exact, thorough, organised, responsible and hardworking. They also have a strong
sense of purpose and lofty standards. Perfectionism, described as the desire for
perfection or the propensity to hold oneself to unreasonable standards, is a trait
variable that can be either adaptive or maladaptive. It has been linked to stress,
anxiety and depression (Bieling et al., 2004; Childs & Stoeber, 2012). Furthermore,
according to earlier research, perfectionism predicts burnout even in the
healthcare industry.
Interviews also indicate that academics attribute institutional or systemic

elements to their episodes of burnout, i.e. academics’ well-being can be
significantly harmed by performance pressure and insufficient institutional
support.
Burnout in academia tends to arise while working in a rigid, traditional and

conservative institutional environment (characterised by a lack of open-
mindedness, freedom to work and frequent evaluation of everything through
accomplishment and performance indicators). The problem will likely worsen
49
due to a lack of resources, budget cuts, unmanageable workloads, growing

demands from parents and students investing more in education, and academics
who must perform up to the required standards are in even greater danger of
developing burnout.
From the sample, none of the respondents sought professional assistance; instead,
they confided with close friends and family members about their burnout
experiences. Emotional dissonance can occur when tension or incongruity exists
between the emotions experienced and those that must be displayed by
established display norms (Zapf & Holz, 2006).
Evidence suggests that various risk variables and academic-specific
stressors combine to put vulnerable academics at risk for developing burnout
states. In contrast, institutional-specific stressors that cause burnout also surfaced
during the interviews and affected most people working in higher education.
10. Recommendations
We advise academics to constantly be vigilant and aware of any potential burnout
signs that could impair their performance at work. Understanding burnout is
essential since academics work in environments where the nature and
organisation of work are frequently changing, and resources are typically scarce.
It is crucial to be aware of the dangers of burnout if they want to lessen its effects.
To do this, they must be able to differentiate between the different kinds of
burnout ranges (high, mild or severe) and between demands that severely burn
them out. Self-care should be prioritised and practised.
Despite the fact that academics have a personal obligation to take care of
themselves, organisations also have a crucial role to play in the issue of burnout
prevention. As a result, it is critical that institutions identify faculty members who
are showing signs of burnout due to its negative impact on their academic
performance and their students’ well-being (Chen et al., 2014).
So, as part of their future professional development, it is necessary to develop and

adopt appropriate intervention strategies that place an emphasis on improving
their academic abilities as they must acknowledge the stressors they face as
typical, realise the dangers of burnout and its crucial indications, and reflect on
why they first chose the profession. The Ministry of Education may create uniform
criteria to gauge academic burnout among university academics and improve the
methods used to track the wellbeing of 250 academics. While metrics for assessing
job satisfaction and work engagement are well-known and widely utilised,
measurements for burnout have been less prevalent and less sought-after in the
Malaysian working setting than those for job satisfaction.
Academic burnout ratings ought to be given the same weight as other

performance indicators like workplace engagement and satisfaction. The
Malaysian context should also be considered when creating regulatory bodies or
panels to oversee academics and create assessment techniques based on these
findings.
50
11. Conclusion
This study’s key finding was that academics were aware of the challenges and
external influences contributing to the complexity of their daily burnout
experiences. This was thoroughly researched, and themes that came out during
the writing process provided a vivid portrait of the opinions of academics.
According to interviews, academics’ periods of burnout were caused by

institutional problems or the university system. Academics have raised concerns
that significantly impact their well-being and mental health. Nevertheless, many
seemed to thrive while performing and working in highly conservative and
traditional formal contexts. Academics were never given autonomy in a position
where everything is constantly assessed and scrutinised using absurd standards
and methods, and these needed to be more tolerant and open-minded. Many
academics indicated that this was what was fuelling their degrees of burnout.
The study also described in detail the current situation and difficulties facing
institutions of higher learning in Malaysia, including issues like a lack of funding,
funding reductions, impossibly high amount of work, and growing demands,
which are all expected to get worse in the coming future. Academics were very
frustrated with the lack of resources that hindered them from producing the
publication and research level required by the university. Many also expressed
dissatisfaction with being pressured to undertake excellent research but needed
to be given sufficient resources to do so.
Findings from the study showed that academics are aware of the early
interventions and coping mechanisms they will need to get through a burnout
episode. Respondents reported using various coping mechanisms, from asking
for help from co-workers and working in a positive environment to being
completely independent in resisting and surviving stressful situations by
exercising total control in their academic role.
According to several academics, the Ministry of Education should make beneficial

adjustments to help academics overcome their enormous barriers and their
current workload, performance and objectives. The findings may raise awareness
among academics, managers, university officials and other higher education
stakeholders of the need to investigate the burnout issue in Malaysian academia.
Because it is one of the few studies on job engagement and burnout among
university academics done in Malaysia from a comprehensive viewpoint on their
experiences, when considered as a whole, the findings add to the body of
knowledge on burnout. Notably, from the beginning to the middle of the
semester, most academics’ degrees of burnout declined. Still, from midway to the
conclusion of the semester, they remained constant (displaying little or no
change). Over the course of the 14-week study, patterns emerged that suggested
a balanced mix of academics would either lead to a beneficial or unfavourable
progression in terms of their levels of burnout.
51
Ethical Compliance Section

Funding
No funding was received for conducting this study.
Compliance with Ethical Standards

All procedures performed in this study were by the ethical standards of the
University’s Institutional Research Committee.
Conflicts of Interest
The authors declare no conflicts of interest.
Informed Consent
Informed consent was obtained from all individual adult participants included in
the study.
Data Availability Statement

The data supporting this study’s findings are available on request from the
corresponding author. The data are not publicly available due to containing
information that could compromise the privacy of research participants.
12. References
Alloh, M. M., Hasan, M. A., Du, X. & Romanowsk, M. H (2019). Burnout of primary
teachers in Qatar government schools during an era of educational reform.
International Journal of Learning, Teaching and Educational Research, 18(10), 1–19.
Areepattamannil S., & Khine, M. S (2018). Evaluating the psychometric properties of the
original grit scale using Rasch analysis in an Arab adolescent sample. Journal of
Psychoeducational Assessment, 36(8), 856–862.
https://doi.org/10.1177/0734282917719976
Ashworth, P (2003). The origins of qualitative psychology. In J. A. Smith (Ed.), Qualitative
Psychology: A practical guide to research methods (pp. 4–24). SAGE.
Bakker, A. B., Westman, M., & Schaufeli, W. B. (2007). Crossover of burnout: An
experimental design. European Journal of Work and Organizational
Psychology, 16(2), 220-239.
Bandura, A. (1977). Self-efficacy: toward a unifying theory of behavioral
change. Psychological Review, 84(2), 191.
Bandura, A. (2001). Social cognitive theory: An agentic perspective. Annual Review of
Psychology, 52(1), 1-26.
Bieling, P. J., Israeli, A. L., & Antony, M. M (2004). Is perfectionism good, bad, or both?
Examining models of the perfectionism construct. Personality and Individual
Differences, 36(6), 1373–1385. https://doi.org/10.1016/S0191-8869(03)00235-6
Busetto, L., Wick, W., & Gumbinger, C (2020). How to use and assess qualitative research
methods. Neurological Research and Practice, 2, 14.
https://doi.org/10.1186/s42466-020-00059-z
Chen, W. S., Haniff, J., Siau, C. S., Seet, W., Loh, S. F., & Abd, M. H (2014). Burnout in
academics: An empirical study in private universities in Malaysia. The
International Journal of Social Sciences and Humanities Invention, 1(2), 62–72.
52
Childs, J. H., & Stoeber, J (2012). Do you want me to be perfect? Two longitudinal studies
on socially prescribed perfectionism, stress and burnout in the workplace. Work
& Stress, 26(4), 347–364. https://doi.org/10.1080/02678373.2012.737547
Court, S., & Kinman, G (2008, December). Tackling stress in higher education. University
and College Union. Retrieved from
http://www.ucu.org.uk/media/pdf/8/a/ucu_hestress_dec08.pdf
Creswell, J. W (1998). Qualitative inquiry and research design: Choosing among five traditions.
SAGE.
Creswell, S. L., & Eklund, R. C (2006). The nature of player burnout in rugby: Key
characteristics and attributions. Journal of Applied Sport Psychology, 18, 219–239.
https://doi.org/10.1080/10413200600830299
Demerouti E., Geurts S. A., Kompier M. (2004). Positive and negative work-home
interaction: Prevalence and correlates. Equal Opportunities International, 24(1–2),
1–35.
Duckworth, A. L., Peterson C., Matthews, M. D., & Kelly, D. R (2007). Grit: perseverance
and passion for long-term goals. Journal of Personality and Social Psychology, 92,
1087–1101. https://doi.org/10.1037/0022-3514.92.6.1087
Garcia-Rivera, B. R., Mendoza-Martinez, I. A., Garcia-Alcaraz, J. L., Olguin-Tiznado, J.
E., Camargo Wilson, C., Aranibar, M. F., & Garcia-Alcaraz, P (2022).
Influence of resilience on burnout syndrome of faculty professors. International
Journal of Environmental Research and Public Health, 19, 910.
https://doi.org/10.3390/ijerph19020910
Graizi, M., Cheah, K. S. L., & Hoque, K. E (2021) The effects of physical, emotional, and
cognitive demands on academic leaders’ performance in Malaysian research
universities. International Journal of Learning, Teaching and Educational Research,
20(3), 282–303. https://doi.org/10.26803/ijlter.20.3.17
Gustafsson, G., Norberg, A., & Strandberg, G (2008). Meanings of becoming and being
burnout – phenomenological hermeneutic interpretation of female healthcare
personnel’s narratives. Scandinavian Journal of Caring Sciences, 22, 520–528.
https://doi.org/10.1111/j.1471-6712.2007.00559.x
Hamzah, S. R., Hamzah, A., Othman, J., & Devi, S. (2016). Impact of Islamic values on
the leadership style of Muslim women academics in Malaysia. Advances in
Developing Human Resources, 18(2), 187–203.
https://doi.org/10.1177/1523422316641402
Henny, J., Anita, A. R., Hayati, K. S., & Rampal, L (2014). Prevalence of burnout and its
associated factors among faculty academicians. Malaysian Journal of Medicine and
Health Sciences, 10(1), 51–59.
https://www.semanticscholar.org/paper/Prevalence-of-burnout-and-its-
associated-factors-Jaswantlal-
Rahman/b357039d30860449e733d9f1c99fa27e4534a8d0
Khamisa, N., Peltzer, K., Ilic, D., & Oldenburg, B (2017). Effect of personal and work
stress on burnout, job satisfaction and general health of hospital nurses in South
Africa. Health SA Gesondheid, 22, 252–258.
https://doi.org/10.1016/j.hsag.2016.10.001
Lindseth, A., & Norberg A. A (2004). Phenomenological hermeneutical method for
researching lived experience. Scandinavian Journal of Caring Sciences, 18, 145–153.
https://doi.org/10.1111/j.1471-6712.2004.00258.x
Maslach, C., & Jackson, S. E (1981). The measurement of experienced burnout. Journal of
Occupational Behavior, 2, 99–113. https://doi.org/10.1002/job.4030020205
Ministry of Higher Education Malaysia (2015). Macro-higher education institutions.
Retrieved from URL: https://www.mohe.gov.my/en/
53
Muhr, T (1991). ATLAS/ti – A prototype for the support of text interpretation.

Qualitative Sociology, 14(4), 349–371. https://doi.org/10.1007/BF00989645
Ologunde, A. O., Asaolu, T. O., & Elumilade, D. O (2012). Labour turnover among
university teachers in Southwestern Nigeria: Issue, solution and lessons.
International Journal of Humanities and Social Science, 2(14), 1–21.
Parmar, V., Channar, Z. A., Ahmed, R. R., Streimikiene, D., Pahi, M. H., & Streimikis, J
(2022). Assessing the organisational commitment, subjective vitality and burnout
effects on turnover intention in private universities. Oeconomia Copernicana,
13(1), 251–286. https://doi.org/10.24136/oc.2022.008
Panatik, S., Rajab, A., Shaari, R., Shah, I. M., Rahman, H. A., & Zainal Badri, S. K. (2012).
Impact of work-related stress on well-being among academician in Malaysian Research
University. Meeting of International Conference on Education and Management
Innovation. IACSIT Press.
Psyhalto, K.; Pietarinen, J.; Salmela-Aro, K (2011). Teacher-working-environment fit as a
framework for burnout experienced by Finnish teachers. Teaching and Teacher
Education, 27(7), 1101–1110. https://doi.org/10.1016/j.tate.2011.05.006
Ricoeur, P (1976). Interpretation theory. Discourse and the surplus of meaning. Texas
Christian University Press.
Smith, J. A., & Osborn, M (2003). Interpretative phenomenological analysis. In J.A. Smith
(Ed.), Qualitative psychology – a practical guide to research methods (pp. 51‐80).
SAGE.
Starks, H., & Brown, S. B (2007). Choose your method: A comparison of phenomenology,
discourse analysis, and grounded theory. Qualitative Health Research, 17(10),
1372–1380. https://doi.org/10.1177/1049732307307
Sterling, P. A (2012). Allostasis: A model of predictive regulation. Physiology & Behavior,
106, 5–15. https://doi.org/10.1016/j.physbeh.2011.06.004
Vesty, G., Sridharan, V. G., Northcott, D., & Dellaportas, S (2018). Burnout among
university accounting educators in Australia and New Zealand: Determinants
and Implications. Accounting & Finance, 58(1), 255–277.
https://doi.org/10.1111/acfi.12203
Yang, C., Yang, L., & Wu, D (2023). The influence of grit on nurse job satisfaction:
Mediating effects of perceived stress and moderating effects of optimism.
Frontiers in Psychology, 13, 1094031. https://doi.org/10.3389/fpsyg.2022.1094031
Zapf, D., & Holz, M (2006). On the positive and negative effects of emotion work in
organisations. European Journal of Work and Organizational Psychology, 15(1), 1–28.
https://doi.org/10.1080/13594320500412199
54

Vol. 22, No. 10, pp. 54-78, October 2023
Received Jul 31, 2023; Revised Oct 11, 2023; Accepted Oct 17, 2023
Polymer Science in Action: Transforming the

Learning Experience for Undergraduates with
Active Learning Strategies
Chaninan Pruekpramool* , Theerapong Sangpradit ,
Panitarn Wanakamol and Supitcha Supansomboon
Faculty of Science, Srinakharinwirot University,
Bangkok, Thailand
Abstract. Active learning is a powerful teaching and learning approach

that enhances students' capacity to construct understanding and apply
their knowledge in real-world contexts. However, in university science
classes, lectures remain the predominant method employed by
instructors, primarily due to time constraints and limited familiarity with
active learning strategies. This study aimed to explore practical
guidelines for implementing active learning in undergraduate polymer
science classrooms, as well as the perspectives of instructors and students
on this approach and student satisfaction with active learning. The
research involved two instructors and 34 second-year students from the
Faculty of Science at a university based in Bangkok, Thailand. Data were
collected through classroom observation, a student satisfaction
questionnaire, and instructor and student interviews. Data were analyzed
quantitatively, using means and standard deviations, as well as
qualitatively, using content analysis. The study identified four practical
guidelines for implementing active learning in the undergraduate
polymer science classroom, which are: using a variety of active learning
strategies and techniques that match the nature of students; providing
helpful learning media and technology; using higher-level questioning to
promote thinking processes; and linking authentic assessment and
constructive feedback with real-life situations. These four guidelines
proved effective for both on-site and online forms of learning.
Furthermore, the students expressed very high satisfaction levels (M =
4.64, SD = 0.49) and positive opinions regarding active learning.
Keywords: active learning; undergraduate students; action research;

practical guidelines
1. Introduction
Through its diverse range of academic programs and courses spanning various
fields, higher education equips students with the specialized knowledge, skills,
*
Corresponding author: Chaninan Pruekpramool, chaninan@g.swu.ac.th
©Authors
55
and experiences necessary for success in their chosen careers. Universities have a
multifaceted role in shaping learners for work, life, and lifelong learning by
transforming their perspectives and fostering conceptual change through the
adoption of student-centered teaching approaches instead of mere knowledge
transmission (Healy et al., 2020; Yates & Hirsh, 2022). Therefore, universities must
emphasize effective teaching and learning methods to develop quality learners
who possess both the knowledge and skills needed for work and life and who are
committed to lifelong learning (Vereijken & Rijst, 2021). To achieve this goal,
teaching and learning methods should cater to learners' needs and encourage
their holistic development. However, a review of the literature on teaching and
learning in higher education illustrates that lecturing remains the most commonly
used method for delivering information to students (Bi et al., 2019; Stecula &
Wolniak, 2022). Instructors have the responsibility of determining the content
structure and selecting appropriate teaching approaches, based on their expertise.
In terms of content structure, instructors are involved in designing or developing
the curriculum for their courses, including selecting appropriate content,
organizing it in a logical sequence, conducting appropriate assessment, and
ensuring alignment with learning objectives and standards. Furthermore,
instructors also have to select and employ appropriate teaching methods based
on the subject matter, course goals, and the needs of their students (Odebiyi, 2022;
Tatto, 2021). This highlights the need to explore and adopt innovative and
effective teaching and learning methods to enhance student learning outcomes
and prepare them for the demands of the 21st century.
The traditional lecture-based teaching method has long been the go-to approach
at the university level, resulting in a passive learning experience for students (Bi
et al., 2019; Kim et al., 2019). Unfortunately, this approach can lead to
disengagement and disinterest among students, which may hinder their ability to
fully develop their potential. Instructors may be hesitant to adopt new teaching
strategies due to time constraints and lack of knowledge, and the classroom
environment may not always support active learning (Kim et al., 2019). Al-Rawi
(2013) has pointed out the limitations of the lecture method, which mainly focuses
on content delivery through one-way communication. In particular, this approach
is commonly used in science courses at the university level, further perpetuating
the passive learning experience. Students often have limited opportunities to
actively engage in discussions or ask questions. Lectures can sometimes overload
students with a large amount of knowledge while providing limited opportunities
for feedback (Kim et al., 2019). At the university level, science instructors are
expected to possess strong knowledge of scientific content and provide students
with practical laboratory experiences and materials (Cho & Baek, 2019; Waldrop,
2015). Commonly used methods for assessing students' knowledge include tests,
quizzes, research projects, report writing, and presentations. However, despite
their expertise in scientific content, many instructors lack knowledge of effective
teaching strategies that promote active learning, resulting in fewer opportunities
for students to engage actively in the learning process (Heck et al., 2023; Kim et
al., 2019). This can have a direct negative impact on students' potential for learning
and academic success.
56
Active learning is a powerful teaching and learning approach that promotes

students' ability to construct their own understanding and apply knowledge in
real-world settings (Cooper et al., 2018; Hao et al., 2021). Unlike passive learning,
which relies on rote memorization, active learning encourages students to engage
with the material and think critically about the concepts being taught, leading to
higher levels of cognitive development (Waldrop, 2015). Compared to traditional
teaching methods, active learning allows students to engage in classroom
discussions and learn in a more interactive manner, leading to the development
of long-term memory and better application of knowledge (Waldrop, 2015).
Forms of active learning in the classroom can vary, depending on factors such as
the nature of students, the problems being addressed, the learning activities, and
the feedback provided by instructors (Schmidt et al., 2015). As a result, active
learning has been shown to improve learning outcomes and promote a deeper,
more accurate understanding of the material. Moreover, students find active
learning activities to be enjoyable and engaging, as they have the opportunity to
develop their higher-order thinking skills through hands-on experiences (Kim et
al., 2019). Active learning is a versatile teaching approach that can be applied both
inside and outside the classroom, benefiting students at all levels in individual,
small-group, and large group settings (Sandrone et al., 2021; Silberman, 1996).
Furthermore, it has been effectively employed to enhance undergraduate science
education across a wide range of scientific concepts (Clark, 2023; Hao et al., 2021;
Nardo et al., 2022; Perasso & Dominguez, 2023). However, despite the recognized
benefits of active learning in improving students' scientific achievement and
academic performance in various aspects, several limitations and barriers to its
implementation in undergraduate science classrooms have been revealed.
When considering the teaching and learning of science at the undergraduate level,
prior research has investigated the impact of active learning. Nardo et al. (2022)
found that, in an introductory chemistry course, while active learning methods
engaged students with varying preparation levels, some found worksheets
unengaging and group work stressful due to peer concerns. Students
recommended dedicating more time to group work for better engagement and
collaboration. However, most undergraduate students have a positive attitude
towards active learning and appreciate the way in which it helps them to engage
more in class and improve their learning processes through various methods and
approaches (Downing et al., 2020). It is crucial to focus on students when
implementing active learning, and there are strategies to assist university
instructors in changing their teaching style. Studies by Kim et al. (2019) and Heck
et al. (2023) explored the barriers to incorporating active learning into science
lecture classes and strategies for overcoming them. A key barrier was revealed to
be a lack of time for developing learning materials and insufficient class time.
Instructors need more flexible schedules and recognition for their teaching efforts.
Furthermore, successful implementation hinges on educator support, with science
instructors requiring training in effective teaching methods. Thus, collaborating
with science educators to enable them to implement active learning in their
courses can bridge this gap. Additionally, extracting practical guidelines for active
learning in undergraduate science classrooms would greatly benefit the
implementation of this research.
57
In 2019, the Covid-19 pandemic caused major lifestyle, economic, and societal
changes worldwide, leading to a shift from on-site to online learning at all levels,
including universities. Successfully adapting teaching and learning strategies to
this new format requires a commitment to the principles of active learning. This
research, conducted in collaboration between polymer science instructors and
science educators in Bangkok, Thailand, aimed to provide guidelines for using
active learning in an undergraduate polymer science classroom for students
during the pandemic and beyond. It reports on the obstacles encountered by both
teachers and students. The polymer science course is a two-credit, lecture-based
class for second-year Materials Science undergraduates. Traditionally, teaching
focused on concepts, and evaluations were mainly conducted through midterm
and final examinations. However, this research was conducted during the
transition from normalcy to the Covid-19 pandemic situation. The course was
initially face-to-face, but the second half shifted to online learning, with active
learning approaches being used in both settings.
2. Literature review
2.1 Active learning
Active learning is an instructional approach that has gained significant attention
and recognition in the field of education over the past few decades. This
pedagogical method emphasizes engaging students in the learning process
through various activities and exercises that go beyond traditional passive
learning techniques, such as lectures and readings (Kim et al., 2019). Specifically,
active learning is designed to promote critical thinking, problem-solving skills,
and deeper understanding of the subject matter by requiring students to actively
participate in their own learning (Hao et al., 2021; Sekwena, 2023). Furthermore,
active learning is a versatile teaching approach that is effective for learners of all
levels, including individual, small group, and large class settings (Silberman,
1996).
One of the key principles of active learning is student involvement in meaningful

and relevant activities. These activities can include group discussions, problem-
solving exercises, hands-on experiments, case studies, peer teaching, and
technology-enhanced learning tools (Silberman, 1996; Strubbe et al., 2019). Active
learning environments are designed to foster collaboration, communication, and
interaction among students, creating a dynamic and engaging classroom
atmosphere (Hodges, 2020) and helping students to identify areas for
improvement through prompt feedback (Hodges, 2020; Strubbe et al., 2019).
Research overwhelmingly affirms the advantages of active learning. Students in
active learning environments consistently outperform their counterparts in
traditional lecture-based classes, resulting in improved academic performance
and a deeper grasp of the subject matter (Strubbe et al., 2019).
Moreover, previous studies have revealed that active learning can enhance and
promote students’ long-term retention (Minick et al., 2022), promoting learning
motivation (Owens et al., 2020) and the ability to apply knowledge to real-word
situations (Hodges, 2020; Silberman, 1996). In addition, active learning can be
58
applied to students through diverse learning styles (Sandrone et al., 2021;

Silberman, 1996). It empowers students to become active participants in their own
learning journeys, preparing them for the challenges of an ever-evolving world.
2.2 Active learning strategies and media

Numerous active learning strategies are employed in diverse classroom settings.
One of the main active learning strategies commonly used in classrooms is group
discussion, in which students collaborate in small groups to exchange their ideas,
encouraging peer interaction and the development of communication skills
(Sandrone et al., 2021; Silberman, 1996; Strubbe et al., 2019). Another key strategy
is think-pair-share, which involves posing a question or prompt to students,
allowing them time to think individually, then pairing them up to discuss their
thoughts before sharing them with the whole class (Cooper et al., 2021). The
jigsaw method is also employed, whereby students work in groups to become
experts on specific topics and then share their knowledge with their peers (Baken
et al., 2022). Role-playing is another active learning approach, in which students
take on specific roles or characters related to the concepts (Effendi, 2021).
Interactive technology can enhance active learning as well; online quizzes,
interactive simulations, discussion boards, and virtual labs provide opportunities
for students to actively engage with course material (Cole et al., 2021).
Combined with diverse media resources, active learning strategies create a

dynamic and engaging educational environment. Active learning strategies often
involve interactive elements such as discussions, debates, group projects, and
problem-solving activities. These strategies are greatly enhanced through the use
of multimedia presentations, virtual simulations, and online discussion platforms.
Interactive media tools allow students to collaborate, explore, and apply their
knowledge in a dynamic and engaging manner (Jesionkowska et al., 2020). This
helps students better understand complex concepts through various sensory
channels.
2.3 Active learning in the undergraduate science classroom

Active learning has been employed to enhance students’ attainment at all levels
of education. In particular, in undergraduate science classrooms, active learning
can be applied across various scientific concepts. Previous research has examined
various teaching methods related to active learning in undergraduate science
classrooms, in both online and on-site learning settings. These methods include
group work activities with worksheets, videos, and textbook reading tasks with
targeted questions (Clark, 2023; Hao et al., 2021; Nardo et al., 2022; Perasso &
Dominguez, 2023), the flipped-classroom approach, and discussions (Clark, 2023).
Additionally, Interactive Lecture Demonstrations involving experiments have
been explored (Perasso & Dominguez, 2023).
Several recent studies have explored the effectiveness and challenges of

implementing active learning in diverse educational contexts. Nardo et al. (2022)
conducted research in an introductory chemistry course and found that, while
active learning methods such as worksheets and group work were utilized, they
posed significant equity challenges. Students with varying levels of prior
59
preparation found worksheets unengaging and group work stressful due to

concerns about fitting in with their peers. Moreover, students suggested that more
time should be dedicated to group work to improve engagement and
collaboration. Hao et al. (2021) focused on the impact of active learning
environments in computer science education and noted that active teaching
methods positively affected outcomes, irrespective of variations in the learning
environment.
In another study, Perasso and Dominguez (2023) examined the influence of an

adapted Interactive Lecture Demonstration (ILD) in an Acoustic Physics course
for first-year university students. The results indicated that students responded
positively to this modified ILD, which encouraged them to take a more active role
in their learning. Furthermore, Clark (2023) investigated student performance in
general chemistry classes, comparing active in-person learning with traditional
methods used during emergency remote teaching. The study found that
underrepresented minority students in the active learning classes, regardless of
the teaching environment, had significantly smaller achievement gaps than those
in traditionally taught classes. This highlights the importance of effective course
structure and teaching approaches, extending beyond in-class active learning, in
promoting student success.
Implementing opportunities for active learning in undergraduate science

classrooms presents a challenge for college and university lecturers. According to
the study by Kim et al. (2019), which unveiled the obstacles to adopting active
learning in lecture classrooms based on the opinions of academic staff in
biomedical science, the primary reason educators persist with traditional lecture-
based teaching methods is the class sizes. Most of the respondents reported that
they aspire to change their teaching styles to incorporate active learning, aiming
to enhance student engagement and foster active participation among students.
However, the principal barrier they face in making this transition is a lack of time.
Another significant hurdle is the lack of recognition for their teaching efforts.
Lecturers often prioritize research grants and publications, considering them key
indicators for career advancement. A study conducted by Heck et al. (2023)
yielded similar findings, highlighting personal barriers such as a shortage of time
for developing learning materials and inadequate class time. More notably, a
dearth of training in active learning strategies was identified as another significant
obstacle faced by science university lecturers.
3. Objectives
The research aims to 1) establish practical guidelines for active learning in a
polymer science classroom; 2) study instructors' and students' views; and 3)
explore student satisfaction with learning.
4. Methods
4.1 Research design
The research design of this study was classroom action research, which followed
the PAOR cycle (Kemmis & McTaggart, 1988), including eight cycles. Each cycle
consisted of four continuous steps, which are described as follows.
60
Plan: This step involved exploring problems in developing teaching and learning
in the polymer science classroom, with an emphasis on active learning for
undergraduate students. The researchers trained the instructors in the features of
active learning, explained how to apply active learning in the science classroom,
and provided examples of active learning classes, strategies, and techniques.
Instructors and researchers worked together to plan, discuss problems, design
learning activities, and choose appropriate teaching strategies and techniques for
each lesson plan.
Action: Next, the developed lesson plans were implemented in the classroom.
Instructors taught their students based on the lesson plans they had created in the
initial step.
Observe: This step ran parallel to the action step. The researcher acted as a
classroom observer, monitoring the instructors’ teaching behavior, as well as
undergraduate students’ learning behavior, using a classroom observation form.
Reflect: In this step, instructors reflected on what had happened in the class.
Instructor interviews were used to collect information based on their opinions and
perceptions. Then, the researcher and instructor discussed these reflections to
identify ways of improving the next lesson.
Figure 1: The PAOR cycle (Kemmis & McTaggart, 1988)
4.2 Participants
The study involved 34 sophomore students and two instructors from the Faculty
of Science at a university located in Bangkok, Thailand. The instructors were
motivated to enhance their teaching skills by incorporating active learning
strategies. Instructors were included if they met the following criteria: 1) Teaching
polymer science courses; and 2) Willingness to voluntarily adopt the active
learning approach in their polymer science class. Student participants were
required to meet the following inclusion criteria: 1) Enrollment in the course
during the first semester of the 2020 academic year; and 2) Consent to participate
in the data collection process for this research.
61
4.3 Instruments
The instruments used in this research were developed based on Behaviorist and
Constructivist theories. The Behaviorist theory emphasizes and identifies the
observable behaviors exhibited by teachers and students in relation to active
learning. The Constructivist theory plays a central role in understanding the ways
in which students engage in learning and in capturing evidence of students
constructing their own understanding. To achieve the objectives of this research,
the following research instruments were utilized.
1) Classroom observation form. To record the instructors' teaching

competency and behavior, as well as undergraduate students' learning behavior
in the active learning polymer science classroom, the researchers utilized a
classroom observation form. The form consisted of descriptive texts in four
aspects: i) implementation of various and appropriate active learning strategies
and techniques; ii) use of media and technology for teaching and learning; iii)
construction and organization of learning activities that emphasize the promotion
of higher-order thinking; and iv) assessment of students in compliance with the
course learning outcomes based on authentic assessment.
2) Instructor interview protocol. The semi-structured interview protocol was

used to gather data from the instructors after each weekly lesson. The following
questions were included:
1. How did you organize the learning in this class? Please identify the
active learning strategies that you used.
2. How do you provide feedback to students?
3. How do students respond to the learning activities?
4. What challenges or obstacles did you encounter during the class?
3) Student interview protocol. The semi-structured student interview

protocol was employed to gather data from students after the completion of the
first and second halves of the semester. The protocol consisted of the following
questions:
1. What do you think about learning in the polymer science course in
terms of instructors, learning activities, and assessment methods in the
first/second half of the semester?
2. Which learning activity did you enjoy the most? Please elaborate.
3. How did the instructors provide feedback to you during the course?
4. What difficulties or obstacles did you encounter during the
first/second half of the semester, and what suggestions would you like
to make to improve the course?
4) Student satisfaction questionnaire. The student satisfaction questionnaire

was developed by the researchers and consisted of 26 items rated on a five-point
scale, focusing on three aspects: 1) teaching and learning process (14 items); 2)
learning media (four items); and 3) measurement and evaluation (eight items).
The interpretation criteria for satisfaction levels are based on the mean and
divided into five categories: very high (4.51-5.00), high (3.51-4.50), moderate (2.51-
3.50), low (1.51-2.50), and very low (1.00-1.50).
62
All the research instruments were evaluated by three experts in the field of science
education to ensure their validity. The Index of Consistency (IOC) was found to
be within an acceptable range of 0.67-1.00.
4.4 Data collection

Before the start of the first semester of the 2020 academic year, the researcher and
instructors collaborated to discuss the features and characteristics of active
learning. In the role of a science educator, the researcher provided relevant
information on how to incorporate active learning in the science classroom. This
included offering the instructors examples of active learning classes and sharing
teaching strategies and techniques, along with guidelines for constructing
effective learning activities. The instructors then planned and created learning
activities, selecting appropriate teaching strategies and techniques for the
polymer science course, a lecture-based course with two credits for second-year
undergraduate students majoring in Materials Science. This planning was based
on consultations with the science educator. Starting from the first loop of action
research, the instructors designed the initial lesson plans for week one, choosing
active learning strategies alongside technology and media integration. They
subsequently implemented these lesson plans, observed student behaviors, and
provided feedback. Along with the researcher, the instructors then reflected upon
the results of the first lesson plan, leading to the planning of the second lesson
plan, and this iterative process continued for eight cycles. The two instructors
conducted the course over 16 weeks, including midterm and final examinations,
with two-hour sessions each week. For further details on the course content,
please refer to Table 1.
Table 1: Content of the polymer science course

Week Topics PAOR cycle
1 Introduction to polymer science 1st cycle
2 Polymer synthesis and chemical 2nd cycle
structure
3 The molecular mass of polymers 3rd cycle
4 Morphology and crystal structure of 4th cycle
polymers
5 First quiz
6 Crystallization and melting 5th cycle
7 Vitrification 6th cycle
8 Midterm examination
9-10 Polymer properties and 7th cycle
characterization
11 Second quiz
12 Application of polymers 8th cycle
13-15 Group work and presentation
(independent study related to
polymers)
16 Final examination
Throughout the course, the researcher closely observed each class using the
classroom observation form. In addition, after every session, the instructor was
63
interviewed to reflect on the progress and discuss areas for improvement. These
insightful discussions served as a valuable tool for fine-tuning the course and
creating an environment that was more conducive to learning. Furthermore, the
feedback gathered from the students was also crucial in evaluating the
effectiveness of the course. At the end of the semester, students were invited to
complete a student satisfaction questionnaire, which included questions on the
teaching and learning process, learning media, and measurement and evaluation.
Some students volunteered to share their experiences and insights on the course
through a student interview protocol, with 10 students being interviewed after
the midterm examination and a further 10 students being interviewed after the
final examination. This feedback was invaluable in shaping the course and
ensuring that it was tailored to meet the needs and expectations of the students.
4.5 Data analysis

The data collected from the study while conducting action research were analyzed
using both qualitative and quantitative methods. The qualitative data analysis
involved evaluating the instructors' teaching competency, identifying the
students' learning behaviors, and eliciting their opinions regarding teaching and
learning through active learning techniques. The content analysis method was
employed to identify practical guidelines for the implementation of active
learning in the undergraduate polymer science classroom. To ensure the
reliability of the research results, the researcher organized the data, identified the
data coding, generated temporary conclusions, and summarized the guidelines.
The summarized data were then verified by three other researchers to confirm the
reproducibility, stability, and accuracy of the results (Krippendorff, 1980). The
students' satisfaction towards their learning experience was analyzed
quantitatively using mean and standard deviation. The mean scores were then
interpreted to determine the overall level of satisfaction as well as satisfaction
levels for each item. This comprehensive analysis allowed for a thorough
understanding of the effectiveness of active learning in the polymer science
classroom and provided valuable insights for future implementation.
5. Results
The research findings were divided into three parts as follows.
Part 1: Practical guidelines for using active learning in the undergraduate
polymer science classroom
To address the first aim of this study, the focus was placed on identifying practical
guidelines for implementing active learning in undergraduate polymer science
classrooms. The data from weekly classroom observations (based on lesson
planning and action research procedures), student feedback, and insightful
instructor interviews were analyzed and synthesized into four practical
guidelines, which are described as follows.
1) Use a variety of active learning strategies and techniques that match the nature
of students.
This guideline was identified during the second cycle of action research. In the
first cycle, the instructor planned the lesson, comprising an introduction, a
teaching step, and a summary step. During the teaching step, the instructor
delivered lectures and utilized questioning techniques to engage students and
64
encourage their participation. However, upon reflection, the instructor observed

that students were still not responding to the questions. This hindered the
instructor from assessing students' comprehension and resulted in a lack of
classroom participation during teaching. In the second cycle, the teacher enhanced
students' participation in learning by introducing such techniques as think-pair-
share. Instead of being completed individually, activities were now carried out in
pairs, accompanied by questioning to encourage students to answer questions
collectively. It was observed that students generally preferred not to work alone;
instead, each pair made more effort to provide answers. This approach increased
student engagement in learning, but there was still room for improvement in
terms of summarizing knowledge. Therefore, in the third cycle, the instructor
reorganized the learning process according to the aforementioned guideline. By
introducing the use of mind mapping to summarize knowledge, the instructor not
only enhanced student participation in learning but also facilitated a more
accurate assessment of students' understanding.
Through classroom observations and interviews, it was evident that both

instructors had used a range of techniques to promote student engagement and
participation. During the first half of the semester, the first instructor employed
think-pair-share, group work activities, questioning, and role-playing techniques
to enhance learning outcomes. Meanwhile, the second instructor had to transition
to online active learning due to the impact of the Covid-19 pandemic in Thailand
during the second half of the semester. To facilitate student learning, the
instructor used techniques such as mind mapping, group work, and jigsaw
techniques. For more comprehensive information on the examples of active
learning techniques used in the undergraduate polymer science classroom, please
refer to Table 2 below.
Table 2: Examples of active learning techniques used in the polymer science

classroom
Instructor Active learning Explanations
techniques
1 Think-pair-share Week 2: To gauge prior knowledge, the instructor used
Kahoot to administer a 5-question multiple-choice pre-
test at the beginning of class. During lectures, she
engaged students with questions to facilitate
discussions on polymer concepts. At the end of each
class, students worked in pairs on a worksheet to
consolidate their learning. To encourage participation,
students volunteered to present their answers to the
class.
1 Questioning and Week 3: Students were asked to summarize the two
role-playing main types of polymerization and make comparisons
between them. The instructor prompted students to
think before answering questions related to the
concepts they had learned the previous week. Some
questions required students to apply their knowledge
to explain concepts such as the speed of
polymerization. One such question was, "Which types
of polymerization result in a greater molecular weight,
65
and how do you know?" This type of question

encouraged students to elaborate on their knowledge.
In an attempt to help students better understand
polymerization, the instructor provided a simulation
activity in which the students acted as monomers
trying to bond with each other. However, the
instructor felt that the activity was not entirely suitable
for the class since the two types of polymerization
were different. Nonetheless, the activity helped
students to grasp the basic idea of polymerization.
1 Group work Week 4: In groups of 4-5, students worked on a
worksheet that focused on repeating units of different
polymers. The instructor encouraged students to think
of their own questions and verify their answers with
their group members. The instructor also posed
questions such as, "Can you explain which type of
polymerization this repeating unit comes from?" After
the group work activity, the instructor explained the
concept of polydispersity and asked students to
confirm their understanding.
2 Mind mapping Week 10: The instructor frequently asked questions
about key terms and used graphs to aid in explaining
processes. To reinforce understanding, the instructor
repeated explanations a number of times and assigned
students to create mind maps summarizing the
concepts.
2 Group work Week 11: The instructor asked students to work in
pairs or small groups to research assigned polymers
across 4-5 topics and provide task-specific examples.
To facilitate collaboration, the instructor provided
shared folders and Google documents for students to
share their information.
2 Jigsaw Week 12: The instructor began by asking students
about polymer additives and presented a picture of an
airplane to prompt their thinking about potential uses.
The instructor then used the jigsaw technique to divide
students into 5 groups of 7, to learn about 7 different
polymer additives. Once all additives had been
learned, students regrouped to summarize their
findings on paper.
2) Provide learning media and technology that are helpful for learning.
This guideline was identified during the first cycle of action research. In this cycle,
the instructor planned the teaching process by posing questions before class and
assessing knowledge to prevent test scores from negatively impacting academic
performance. Various applications were employed, with the initial plan involving
the use of Kahoot and a Bingo game to engage students in answering questions
and reviewing their knowledge before class. Teaching included the use of
PowerPoint presentations in conjunction with active learning strategies. Upon
reflection, the instructor noted that students enjoyed participating in activities
that involved answering questions through applications, playing games,
competing, and having the opportunity to win prizes. In subsequent cycles, the
66
instructor continued to implement this teaching approach, making adjustments to

the applications to maintain student engagement and introducing other teaching
tools to enhance the teaching process. Classroom observations revealed that both
instructors effectively utilized media and technology in two primary ways.
Firstly, they used Kahoot, Quizzes, Mentimeter applications, and Google Forms
to assess students' knowledge before, during, and after the lessons. Quizzes was
the application used most frequently by both instructors. Secondly, they
employed Microsoft Excel, Microsoft PowerPoint, Google Classroom, Google
Docs, and games as teaching tools to enhance students' learning. Microsoft
PowerPoint was used by the instructors every week.
3) Use higher-level questions to promote students’ thinking processes.

This guideline emerged during the third cycle of action research, based on the
instructor's reflections from the first two teaching cycles. In the initial teaching
phases, the instructor primarily employed questions to assess students' acquired
knowledge. These questions often prompted a 'Yes/No' answer, limiting
students' ability to provide explanations or engage in higher-order thinking.
However, in the third cycle, the instructor revised the question format,
encouraging students to think critically and explain their answers. This change
led to students engaging in analytical and critical thinking, demonstrating greater
discretion in their responses, and showcasing their ability to apply their
knowledge to explain phenomena or predict relationships. In subsequent cycles,
the teacher continued to implement this guideline, noticing that learners became
increasingly proficient at expressing their knowledge and ideas.
The instructors utilized various forms of questions to achieve three purposes.
Firstly, to identify differences through the use of comparative questions; examples
of the questions are as follows:
“How is a monomer different from a repeating unit?” (Week 3)
“From what you learned, what are the differences between the two types
of polymerization?” (Week 3)
“From your observation, how do different fluids have different

viscosities?” (Week 10)
Secondly, to check learners' concepts and understanding, the following examples
of questions were asked:
“How does a polymer change the phase?” (Week 5)
“If tested at a higher speed, how would the original graph change at
normal temperature?” (Week 7)
“What techniques can be used to analyze this polymer? Please explain.”

(Week 14)
Finally, to encourage further investigation through inquiry-based questions,
instructors used questions such as the following:
“If all the polymers produced have the same molecular weight, what will
be the value of the polydispersity index?“ (Week 4)
67
“If you have identified the properties of modulus, strength, and breaking
point strain from a table, how would you create a graph?” (Week 7)
“What are the main properties of polymers? How can you analyze those
properties?” (Week 14)
4) Use authentic assessment and constructive feedback linked with real-life

situations to reinforce students’ learning and maintain good study habits.
This guideline was discovered during the second cycle of action research. Upon
reflecting on the first cycle of teaching, the instructor noticed that students were
primarily assessed through pre- and post-class questioning. There was no clear
assessment of students' understanding during the learning process, and feedback
was lacking. This reflection revealed a problem: instructors could not gauge
students' comprehension during lessons, and learners were unaware of the areas
they excelled in or those that needed improvement. In the second cycle,
instructors adjusted their approach by evaluating students through pre- and post-
class questions as well as introducing further questions during class activities and
through worksheets. They also provided constructive feedback to students
through collaborative work in class and the examination of worksheets. This
adjustment allowed instructors to assess students' understanding, and learners
gained insight into the areas they needed to develop in their learning.
Consequently, this approach continued to be employed in subsequent cycles.
The results of the observations showed that both instructors used a variety of
assessment methods to evaluate students' learning performance, as follows.
4.1) Asking questions before, during, and after learning. Examples of the
observations are presented below.
“The instructor sought learners' opinions and prior knowledge, gave
feedback during class without judgment, and used the Kahoot application
to assess students' understanding afterward.” (Instructor 1, week 2
observation)
“The instructor asked questions such as 'Do you know any polymer
additives?', 'What are the main polymer additives used on airplanes?',
and 'What is the purpose of using polymer additives?' During class,
students discussed and investigated to find answers. A posttest with
seven questions was administered using the Quizzes app after class.”
(Instructor 2, week 12 observation)
4.2) Providing activity worksheets and assigning tasks to summarize key

concepts. Examples of the observations are provided below.
“The instructor gave a group activity worksheet on the mechanical
properties of polymers, provided guidance on calculations and unit
conversions, and reviewed the work for necessary adjustments. Students
then presented their findings to the class.” (Instructor 1, week 6
observation)
68
“The instructor assigned a mind-mapping task which was well-executed

by the students. The instructor gave individual feedback and suggestions
for improvement.” (Instructor 2, week 12 observation)
Part 2: The views of instructors and students on active learning

The results of the instructors’ interviews are summarized as follows.
1) The implementation of active learning techniques in the polymer science
classroom is highly beneficial for students.
“Active learning is beneficial in terms of promoting student participation.

In this generation, traditional lectures often fail to engage students, who
may disengage and become disinterested in the class.” (Instructor 1,
interview)
2) The instructors acknowledged certain limitations to the use of active learning

techniques in the polymer science classroom .
• The limitation of providing feedback to students.
“It is impossible to provide complete feedback for all students individually;

it must be given as a group.” (Instructor 1, interview)
“I feel that I'm not yet able to fully manage an active learning class,
especially when it comes to providing feedback to students. Within the
limited timeframe of 1-2 hours, it's impossible to give feedback to every
student in every group.” (Instructor 2, interview)
• The difficulty of organizing students for group work.
“Grouping students with different abilities together actually takes time, is

rarely done, and difficult to control. Most students prefer to work with
friends [who] sit close to each other and choose the members of their group
themselves. However, this way of grouping has its advantages, as it allows
students to start learning and engaging in activities more quickly.”
(Instructor 1, interview)
3) This research project contributed to an improvement in the instructors' teaching
competency.
“At first, I didn't understand what active learning was. Later, I realized
that I had used active learning in my past experiences, but I didn't have
much knowledge about it. This research project helped me improve my
knowledge and enabled me to think more about teaching activities and
feedback methods.” (Instructor 2, interview)
Interviews were conducted with 10 students after the midterm examination and
10 students after the final examination. The students’ interview results are
shown as follows.
1) The students found active learning to be enjoyable and effective in motivating
them to learn science and acquire knowledge in the polymer science classroom.
69
“Learning this way was fun and helped us to gain more knowledge.”
(Student 2, interview in the first half of the semester)
“The instructor allowed all students to work in groups so that they could
debate, discuss, and understand the content. When I did the group
activities with friends, I presented our work and the instructor pointed out
where I had made mistakes and where I was still lacking.” (Student 4,
interview in the first half of the semester)
“The instructor provided us with various tasks and activities, including

individual tasks, paired tasks, group work, games, and the jigsaw method.
These activities motivated me to learn happily.” (Student 7, interview in
the second half of the semester)
2) The students enjoyed using learning media and technology such as Kahoot,
Quizzes, and games in the class, which helped them review their prior knowledge
and better understand the content.
“I like Kahoot where the whole room plays simultaneously, gets excited,
has time limits, and knows the order of winners. When the instructor gives
rewards, it makes students active.” (Student 6, interview in the second
half of the semester)
"The instructor played a Bingo game in class where students filled out
their cards with different types of polymers. The instructor provided
polymer characteristics, and the students checked their cards. The first to
complete a line won a prize, helping them recall their studies and making
learning enjoyable." (Student 7, interview in th e second half of the
semester)
3) The students recognize the importance of feedback in active learning in the

polymer science classroom.
“In the mind map activity, the instructor collected all student work to
provide feedback on the following week, including suggestions such as
adding units to certain parts or correcting graphs.” (Student 8, interview
in the first half of the semester)
“The instructor provided feedback on all student assignments and tasks,

pointing out any errors and offering additional resources if needed.”
(Student 1, interview in the second half of the semester)
4) The instructor informed students about assessment criteria and scores, and
allowed them to give feedback on adjusting the ratio of scores or criteria.
“In the first period of class, the instructor informed students about the
midterm and final exams, their percentage in the course, and collected
scores. Students were also given the opportunity to provide their opinions
70
on adjusting the exam-to-score ratio.” (Student 8, interview in the first

“The instructor discussed the assessment criteria with students during

Covid-19 and allowed for adjustments to be made in order to reach a
mutual agreement.” (Student 2, interview in th e second half of the
semester)
5) Obstacles and limitations of using active learning in the polymer science

classroom, according to student feedback, are as follows.
• Students struggle to follow the instructor's explanations and take notes
on all the concepts. Adding more detailed explanations to the slides
could help students to follow polymer science concepts more easily.
“Slides had made us understand a lot, but if the slides had too much
content for us to write, it made us unable to follow the instructor's
explanations.” (Student 9, interview in the first half of the semester)
• Students find it more difficult to concentrate when learning online

compared to learning in the classroom.
“The obstacle was a lack of concentration while studying online. Online

learning requires more concentration compared to in-person learning in
the classroom. Live teaching may be more effective in this regard.”
(Student 1, interview in the second half of the semester)
“I had no concentration when studying at home. I would rather go back to

university. At first, it was ok, but then I started not understanding the
lessons. I think that studying in the real classroom context, the instructor
could explain the concepts better.” (Student 6, interview in the second
• Communication during online active learning is not as effective as

learning in the classroom.
“In the online platform, the instructor was unable to see the students' faces
and thus did not know when they were confused. Even though the
instructor explained the concepts in detail, learning in the classroom was
better. This is because we could raise our hands for the instructor to see
and explain again.” (Student 3, interview in th e second half of the
semester)
“Without Covid, the instructor would have more learning activities.

Learning online made communication difficult.” (Student 4, interview
in the second half of the semester)
“Kahoot and Quizzes were usually given by the instructor. In normal

classroom lessons, it was more fun to play and talk with friends. However,
during online learning, the instructor sent us the links. It wasn't as fun as
71
before and we didn't talk as much.” (Student 5, interview in the second

• Students faced persistently unreliable internet connections while

learning online.
“During Covid, there was a problem with unstable internet. I could not
complete the question. Typing words and messages may not be completed.
I always forgot to ask the instructor when I had questions.” (Student 3,
interview in the second half of the semester)
“Internet problems, noise, sometimes studying at home alone would be

more stressful, not seeing friends.” (Student 8, interview in the second
“The problem was students did not have internet access at home, only
telephone internet, which is unstable.” (Student 9, interview in th e
second half of the semester)
Part 3: Students’ satisfaction toward learning

According to the results, students were highly satisfied with their learning
experience in general, with the assessment and evaluation aspect receiving the
highest mean score (M = 4.71, S.D. = 0.76). Similarly, the teaching and learning
process aspect also received a high mean score (M = 4.61, S.D. = 0.52), followed
closely by the learning media aspect (M = 4.60, S.D. = 0.49). These findings are
presented in Table 3.
Table 3: Students’ satisfaction toward learning

Level of
Items M SD
satisfaction
Aspect 1: Teaching and learning process 4.61 0.52 Very high
1.1 The instructors clarified the scope of the 4.76 0.44 Very high
content and the learning objectives.
1.2 The instructors arranged the steps for 4.65 0.49 Very high
teaching and learning in the classroom.
1.3 Students practiced analytical thinking 4.53 0.51 Very high
processes to solve problems in real-life
situations or case studies.
1.4 Students linked their prior and new 4.41 0.62 High
knowledge through their own thinking
processes and practice.
1.5 The instructors used a variety of teaching 4.76 0.44 Very high
and learning techniques and activities.
1.6 The instructors encouraged students to 4.65 0.49 Very high
participate and be a part of the learning
activities.
1.7 The instructors gave students the 4.53 0.51 Very high
opportunity to present or comment on
learning activities.
72
Level of
Items M SD
satisfaction
1.8 The learning activities encouraged 4.53 0.51 Very high
students to discuss and exchange their
knowledge and opinions.
1.9 The learning activities fostered 4.65 0.49 Very high
interaction between students and
instructors.
1.10 After learning, students had 4.53 0.51 Very high
opportunities to summarize and discuss
what they had learned.
1.11 The instructors regularly provided 4.47 0.72 High
feedback to students.
1.12 The instructors provided positive 4.76 0.44 Very high
reinforcement such as rewards and extra
points to stimulate students’ learning.
1.13 Technology was integrated into learning. 4.59 0.62 Very high
opportunity to ask questions through
various channels.
Aspect 2: Learning media 4.60 0.49 Very high
2.1 The learning media used in the activities Very high
are diverse. 4.65 0.49
were consistent with the content. 4.53 0.51
were consistent learning activities. 4.59 0.51
can promote or stimulate students'
learning. 4.65 0.49
Aspect 3: Assessment and evaluation 4.71 0.46 Very high
3.1 The guidelines and methods for 4.71 0.47 Very high
measuring and evaluating were clearly
explained to the students.
3.2 The criteria for evaluation were 4.65 0.49 Very high
consistent with the learning objectives.
3.3 The instructor gave students the 4.65 0.49 Very high
opportunity to participate in adjusting
the criteria or assessment scores.
3.4 The students were informed of their 4.71 0.47 Very high
assessment and evaluation results in
order to improve the students’ learning
process.
3.5 The instructors gave learners the 4.71 0.47 Very high
opportunity to participate in self-
assessments or peer assessments.
opportunity to reflect on what they had
learned.
3.7 The instructors applied a variety of 4.76 0.44 Very high
methods for assessing and evaluating
learners.
73
Level of
Items M SD
satisfaction
3.8 The instructors used technology to assess 4.71 0.47 Very high
and evaluate students’ learning
outcomes.
Overall 4.64 0.49 Very high
6. Discussion
The study's findings offer practical guidelines for implementing active learning in
undergraduate polymer science classrooms in Thailand. To promote student
participation and cooperation in both online and on-site active learning
environments, instructors should use a variety of active learning strategies and
techniques that align with students' learning styles. This study's instructors used
a range of techniques, such as think-pair-share, group work, role-playing,
discussion activities, questioning, and jigsaw techniques, to engage students and
facilitate their understanding of polymer concepts. According to research, active
learning activities can encourage student participation and enhance teaching and
learning (Al-Rawi, 2013; Clark et al., 2023). The benefits of group work activities
in promoting students’ academic and social abilities are well established. Mixed-
ability groups are particularly effective in facilitating learning from peers, with
previous research indicating that the most successful groups are those with
diverse members and skill levels. Additionally, a clear division of responsibilities
within groups is crucial for success (Al-Rawi, 2013; Sandrone et al., 2021;
Silberman, 1996). However, this study faced a limitation in that students were not
given the opportunity to select their own group members, which can be a time-
consuming process. One instructor noted that students preferred to work with
familiar peers and may not want to work with less familiar peers. Interestingly,
groups comprised of close friends were found to facilitate quicker and more in-
depth learning. However, the study by Nardo et al. (2022) suggests that group
work can increase students' stress, particularly for those who are less prepared
and may feel left out of the group. Therefore, when organizing students into
groups, instructors should allocate more time for students to communicate with
each other before performing the group work activity. Guiding students on how
to work effectively as a team is another way to help them learn collaboratively. In
this study, questioning and discussion activities were applied in every class. The
instructors always asked questions and allowed the students to discuss and
express their thoughts and understanding. Discussions can increase students'
confidence and encourage participation in learning. Additionally, discussion
activities can help instructors to assess their students' understanding. However,
instructors must have the skills to create effective discussion questions (Al-Rawi,
2013; Waldrop, 2015).
Secondly, incorporating learning media and technology is an effective way to

enhance the learning experience in the polymer science classroom. In this study,
applications such as Kahoot, Quizzes, Mentimeter, and Google Forms proved to
be valuable tools for assessing students' knowledge before, during, and after class.
In addition to assessment, these tools also fostered student engagement and
participation. The interview results revealed that students enjoy learning with the
aid of technology and media, as they are growing up in a technology-driven era.
74
Research suggests that technology and hands-on activities can improve students'
learning outcomes and promote critical thinking skills (Giray, 2022; Mosca et al.,
2019). Thus, teaching and learning styles in higher education need to evolve to
meet the needs of today's students and provide them with meaningful and
engaging learning experiences (Kim et al., 2019; Mosca et al., 2019). However,
students reported struggling with concentration during online learning, and
found the applications used in the online classroom – such as Kahoot and Quizzes
– less enjoyable compared to learning in the polymer science classroom.
Additionally, unstable internet connectivity hindered their ability to learn
efficiently, and communication via online platforms was deemed less effective
than in-person learning. These problems had a significant impact on students’
ability to learn. One of the instructors also reflected on the challenges of managing
an active learning class in an online environment and emphasized the importance
of instructors being well-prepared to use learning media and communication
effectively.
Thirdly, incorporating high-level questioning can enhance students' critical

thinking processes. The instructors employed comparative and conceptual
questions to assess students' comprehension. Furthermore, they used questions to
introduce the activity and investigation. Such questions motivated students to
contemplate what they had learned and solve problems. Although questioning
was used frequently in class, questions that challenged students to apply their
knowledge to novel situations were seldom used in this study. Questioning is a
crucial aspect of learning, not only in active learning classrooms, but in any
setting. Questions should be challenging, requiring students to apply their
knowledge to explain phenomena or solve problems (Waldrop, 2015). Moreover,
the questions or problems should allow students to apply what they have learned
to novel situations (Schmidt et al., 2015). However, the difficulty of generating
effective questions is a limitation of implementing active learning in the polymer
science classroom. Science instructors should collaborate with educators to hone
their skills in constructing good questions (Al-Rawi, 2013). Additionally,
instructors should avoid calling on students randomly to answer questions or
perform activities, particularly in larger science classrooms, since this can cause
students anxiety and impede their learning (Cooper et al., 2018).
The final guideline identified from this study is the use of authentic assessment
and the provision of constructive feedback, linked with real-life situations, which
can reinforce students’ learning and help them to maintain good study habits. In
the polymer science classroom, the instructors used a variety of assessment
methods, including asking questions, summarizing the knowledge with mind
maps, using worksheets, observing students’ learning behaviors, and
examinations. These various assessments allowed students to be assessed
according to their actual learning outcomes. Authentic assessment focuses on
giving feedback to students, letting them solve problems, and providing
opportunities for students to develop and improve their own learning
(Henderson et al., 2019; Sokhanvar et al., 2021). Previous research has revealed
that science students fear negative evaluations, specifically from their classmates
(Cooper et al., 2018). Thus, instructors need to create a good learning atmosphere,
75
promote learning for students, and act as role models in providing constructive
feedback. Feedback needs to be well-prepared (Johannes & Haase, 2022; Schmidt
et al., 2015), as it affects many dimensions for improving students’ learning,
including cognitive, self-assessment skills, and motivation (Henderson et al., 2019;
Johannes & Haase, 2022). In this study, the instructors always gave feedback to
students individually and in each group. As indicated in the interviews, the
students have also realized that feedback is important when learning through
active learning techniques in the polymer science classroom.
The survey of students’ satisfaction with their learning experience yielded high
mean scores in all aspects, including the teaching and learning process, learning
media, and assessment and evaluation. These findings correspond with the results
of students' interviews, in which the participants expressed that active learning
was enjoyable and effective in motivating them to learn science. This can be
attributed to the well-designed polymer science course, which was created
through a collaborative effort between science instructors and educators, with a
strong emphasis on active learning principles. The course incorporates a diverse
range of activities that encourage student engagement and class discussions.
Furthermore, the integration of media and technology serves as a motivational
tool for students. Instructors actively provide feedback to help students improve
their learning and class participation, creating a conducive learning environment.
These characteristics align with the principles of student-centered learning, which
prioritize active student engagement and participation in the learning process.
7. Conclusions
This research offers a comprehensive understanding of active learning
implementation in undergraduate polymer science classrooms in Thailand,
providing practical guidelines for educators. In addition, the study underscores
the importance of employing a diverse range of active learning strategies that
cater to students' varying learning styles. Techniques such as think-pair-share,
group work, role-playing, discussion activities, questioning, and jigsaw methods
have proven effective in engaging students and enhancing their comprehension
of complex polymer concepts. Furthermore, the integration of learning media and
technology has emerged as a powerful tool for enriching the learning experience
in the polymer science classroom. However, the challenges associated with online
learning, including issues related to concentration and technological hurdles,
highlight the vital role of well-prepared instructors who are capable of effectively
navigating virtual classrooms. Additionally, the research accentuates the value of
high-level questioning techniques to stimulate critical thinking processes. Lastly,
the study demonstrates the significance of authentic assessment and constructive
feedback linked to real-life situations in reinforcing student learning and study
habits. In summary, this research not only offers practical guidelines but also
emphasizes the transformative potential of active learning in undergraduate
polymer science classrooms. By embracing these guidelines and continuously
refining their teaching practice, educators can be empowered to create engaging,
student-centered learning environments, preparing students for success in their
academic and professional journeys. This study contributes to the ongoing
76
evolution of teaching and learning practices in higher education, particularly in

today's dynamic, technology-driven era.
8. Implications and recommendation for further studies

The implications of this research emphasized applying active learning strategies
and activities to science courses at the higher education level. Instructors and
others who are interested in implementing active learning techniques in their
science classrooms can use the four guidelines effectively for both on-site and
online forms of learning. In addition, lesson content and time management are
equally important when considering active learning. Higher-order questioning
skills need to be practiced in order to promote students’ critical thinking. Effective
questioning and real-life challenges allow students to apply their knowledge and
skills to explain new situations and solve a range of problems. Additionally, it is
important to create a friendly learning atmosphere, so that students can express
their opinions and contribute to effective discussions. This research did not focus
directly on instructors’ teaching competency and pedagogy when applying active
learning in the polymer science classroom. In further studies, the level of teaching
competency should be studied and the teaching competency criteria also need to
be developed. Moreover, the effects of using active learning techniques on science
students’ higher-order thinking and life-long learning skills offer further
interesting opportunities for future research.
9. References
Al-Rawi, I. (2013). Teaching methodology and its effects on quality learning. Journal of
Education and Practice, 4(6),100-105.
Baken, E. K., Adams, D. C., & Rentz, M. S. (2022). Jigsaw method improves learning and
retention for observation-based undergraduate biology laboratory activities.
Journal of Biological Education, 56(3), 317-322.
https://doi.org/10.1080/00219266.2020.1796757
Bi, M., Zhao, Z., Yang, J., & Wang, Y. (2019). Comparison of case-based learning and
traditional method in teaching postgraduate students of medical oncology.
Medical Teacher, 4(10), 1124-1128.
https://doi.org/10.1080/0142159X.2019.1617414
Cho, J., & Baek, W. (2019). Identifying factors affecting the quality of teaching in basic
science education: Physics, biological sciences, mathematics, and chemistry.
Sustainability, 2019(11), 3958. https://doi.org/10.3390/su11143958
Clark, T. M. (2023). Narrowing achievement gaps in general chemistry courses with and
without in-class active learning. Journal of Chemical Education, 100, 1494-1504.
https://doi.org/10.1021/acs.jchemed.2c00973
Cole, A. W., Lennon, L., & Weber, N. L. (2021). Student perceptions of online active
learning practices and online learning climate predict online course engagement.
Interactive Learning Environments, 29(5), 866-880.
https://doi.org/10.1080/10494820.2019.1619593
Cooper, K. M., Jeffrey, N. S., & Tanner, K. D. (2021). Reconsidering the share of a think–
pair–share: Emerging limitations, alternatives, and opportunities for research.
CBE Life Sciences Education, 20(1), 1-10. https://doi.org/10.1187/cbe.20-08-0200
Cooper, K. M., Downing, V. R., & Brownell, S. E. (2018). The influence of active learning
practices on student anxiety in large-enrollment college science classrooms.
International Journal of STEM Education, 5(23), 1-28.
https://doi.org/10.1186/s40594-018-0123-6
77
Downing, V. R., Cooper, K. M., Cala, J. M., Gin, L. E., & Brownell, S.E. (2020). Fear of
negative evaluation and student anxiety in community college active-learning
science courses. CBE Life Sciences Education, 19(2), 1-16.
https://doi.org/10.1187/cbe.19-09-0186
Effendi, D. (2021). Implementation of role-playing active learning to increase student
learning outcomes on the digestive system materials. International Journal of
English Education and Research, 1(2), 116-126.
Giray, L. (2022). Meet the centennials: Understanding the generation Z students.
International Journal of Sociologies and Anthropologies Science Reviews, 2(4), 9-18.
https://doi.org/10.14456/jsasr.2022.26
Hao, Q., Barnes, B., & Jing, M. (2021). Quantifying the effects of active learning
environments: separating physical learning classrooms from pedagogical
approaches. Learning Environments Research, 24, 109-122.
https://doi.org/10.1007/s10984-020-09320-3
Henderson, M., Ajjawi, R., Boud, D., & Molloy, E. (2019). Identifying feedback that has impact.
In: Henderson, M., Ajjawi, R., Boud, D., & Molley, E. (eds). The Impact of Feedback
in Higher Education. Cham: Springer International Publishing, pp.15–34.
Healy, M., Hammer, S., & McIlveen, P. (2020). Mapping graduate employability and career
development in higher education research: A citation network analysis. Studies in
Higher Education, 47(4), 799-811. https://doi.org/10.1080/03075079.2020.1804851
Heck, A. J., Cross, C. E., Tatum, V. Y., & Chase, A.J. (2023). Active learning among health
professions’ educators: Perceptions, barriers, and use. Medical Science Educator, 33,
719-727. https://doi.org/10.1007/s40670-023-01793-0
Hodges, L.C. (2020). Student engagement in active learning classes. In: Mintzes, J. J., Walter,
E. M. (eds) Active Learning in College Science. Springer, Cham.
https://doi.org/10.1007/978-3-030-33600-4_3
Jesionkowska, J., Wild, F., & Deval, Y. (2020). Active learning augmented reality for
STEAM education-A case study. Education Sciences, 10(8), 198.
Johannes, C., & Haase, A. (2022). The impact of feedback mode on learning gain and self-
efficacy: A quasi-experimental study. Active Learning in Higher Education, 1-17.
https://doi.org/10.1177/14697874221131970
Kemmis, S., & McTaggart, R. (1988). The action research planner (3rd ed.). Waurn Ponds:
Deakin University Press.
Kim, A. M., Speed, C. J., & Macualay, J.O. (2019). Barriers and strategies: Implementing
active learning in biomedical science lectures. Biochemistry and Molecular Biology
Education, 47(1), 29-40. https://doi.org/10.1002/bmb.21190
Krippendorff, K. (2013). Content analysis: An introduction to its methodology (3rd ed).
California, CA: Sage Publications.
Minick, W., Cekada, T., Marin, L., Zreiqat, M., Seal, B., & Mulroy, J. (2022). The impact of
active learning strategies on retention and outcomes in safety training. Creative
Education, 13(2), 526-536. https://doi.org/10.4236/ce.2022.132031
Mosca, J. B., Curtis, K. P., & Savoth, P. G. (2019). New approaches to learning for
generation Z. Journal of Business Diversity, 19(3), 66-74.
https://doi.org/10.33423/jbd.v19i3.2214
Nardo, J. E., Chapman, N. C., Shi, E. Y., Wieman, C., & Salehi, S. (2022). Perspectives on
active learning: challenges for equitable active learning implementation. Journal of
Chemical Education, 99, 1691-1699. https://doi.org/10.1021/acs.jchemed.1c01233
Odebiyi, O. M. (2022). Mobilizing teacher candidates’ pedagogical design capacity for
implementing inquiry-oriented lessons. Social Sciences & Humanities Open, 2022.
http://dx.doi.org/10.2139/ssrn.4257902
Owens, D. C., Sadler, T. D., Barlow, A. T., & Smith-Walters, C. (2020). Student motivation
from and resistance to active learning rooted in essential science practices.
78
Research in Science Education, 20, 253-277. https://doi.org/10.1007/s11165-017-

9688-1
Perasso, J. A., & Dominguez, A. (2023). Students’ perception of active learning in the acoustic
physics course. 2023 Annual Conference & Exposition, Baltimore Convention
Center June 25-28, 2023. American Society for Engineering Education.
Sandrone, S., Scott, G., Andersen, W. J., & Musunuru, K. (2021). Active learning-based
STEM education for in-person and online learning. Cell Leading Edge, 184, 1409-
1414. https://doi.org/10.1016/j.cell.2021.01.045
Schmidt, H.G., Wagener, S.L., Smeets, G. A. C. M., Keemink, L.M., & Van der Molen, H.T.
(2015). On the use and misuse of lectures in higher education. Health Professions
Education, 1, 12-18. https://doi.org/10.1016/j.hpe.2015.11.010
Sekwena, G. L. (2023). Active learning pedagogyfor enriching economics students’higher
orderthinking skills. International Journal of Learning, Teaching and Educational
Research, 22(3), 241-255. https://doi.org/10.26803/ijlter.22.3.15
Silberman, M. (1996). Active learning. Boston: Allyn and Bacon.
Sokhanvar, Z., Salehi, K., & Sokhanvar, F. (2021). Advantages of authentic assessment for
improving the learning experience and employability skills of higher education
students: A systematic literature review. Studies in Educational Evaluation, 70,
101030. https://doi.org/10.1016/j.stueduc.2021.101030
Stecula, K., & Wolniak, R. (2022). Advantages and disadvantages of E-Learning
innovations during COVID-19 pandemic in higher education in Poland. Journal of
Open Innovation: Technology, Market, and Complexity, 8(3), 159.
https://doi.org/10.3390/joitmc8030159
Strubbe, L. E., Stang, J., Holland, T., Sherman, S. B., & Code, W. J. (2019). Faculty adoption
of active learning strategies via paired teaching: Conclusions from two science
departments. Journal of College Science Teaching, 49(1), 31–47.
https://www.jstor.org/stable/26901347
Tatto, M. T., (2021). Professionalism in teaching and the role of teacher
education. European Journal of Teacher Education, 44(1), 20-44.
https://doi.org/10.1080/02619768.2020.1849130
Vereijken, M. W. C., & Rijst, R. M. (2021). Subject matter pedagogy in university teaching:
how lecturers use relations between theory and practice. Teaching in Higher
Education, 28(4), 880-893. https://doi.org/10.1080/13562517.2020.1863352
Waldrop, M. M. (2015). The science of teaching science. Nature, 523, 272-274.
Yates, J., & Hirsh, W. (2022). One-to-one career conversations in UK higher education:
practical approaches and professional challenges. Journal of Further and Higher
Education, 46(9), 1304-1317. https://doi.org/10.1080/0309877X.2022.2072195
Ethical considerations
Ethical approval was obtained from Srinakharinwirot University, Thailand
( Research ethics number: SWUEC-453/2563E) . Informed consent was obtained
from the second-year undergraduate students who were the participants in this
research.
Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the
research, authorship, and/or publication of this article.
Funding
This work was supported by the faculty of science, Srinakharinwirot University,
Thailand under Grant [number 201/2563]
79

Vol. 22, No. 10, pp. 79-92, October 2023
Assessment of Learning about Sustainability in

Students with Down Syndrome
Cristina Arranz Barcenilla , Sara Gutiérrez González* ,
María Consuelo Saiz Manzanares and Lourdes Alameda Cuenca-Romero
Universidad de Burgos, Spain
Sarah Vandekerkhof
Hogeschool UCLL, Belgium
Abstract. Assessment is an essential resource for guaranteeing quality

education. In order for it to be effective, it must be continuous, formative
and adapted to the characteristics of the individual student, so that it
shows their abilities. Students with Down Syndrome benefit from these
aspects, as they can demonstrate their achievements and difficulties in
different ways. This article analyses these approaches and brings together
the suggestions of 14 experts, in the field of education for students with
Down Syndrome, from three different countries, 5 members of Belgium
and Spain, and 4 members of Ireland. Through the instrumental case
study and focus discussions of the experts, using the Delphi method, a
series of basic aspects are established on the most appropriate methods
and evaluation tools for the target group. In the analysis the experts
agreed on three key aspects (with scores of three out of three):
maintaining high expectations of students' potential, encouraging self-
evaluation, and enabling students controlling their own learning process.
The group of experts proposes the use of a learning register or portfolio
as a method of assessment for students with Down Syndrome.
Keywords: assessment; down syndrome; sustainability; virtual learning

environment
1. Introduction
Achieve inclusive education requires providing equal opportunities for all
students to develop their full potential through a response tailored to their
individuality (Gamonales Puerto et al., 2018). Attending to their learning style will
facilitate and promote the teaching-learning process (Kabashi & Kaczmarek, 2019;
Kurth et al., 2020; Lindner & Schwab, 2020). To achieve this, the whole
educational process, objectives, methodology, resources, assessment, etc., must be
interrelated.
*
Corresponding author: Sara Gutiérrez González, sggonzalez@ubu.es
©Authors
80
Assessment is a vital part of pedagogical practice, a fundamental part of achieving

quality education. An instrument that provides us significant information and
data for educational improvement (Ibáñez, 2020; Jurado-de-los-Santos et al.,
2021). Assessment takes on great importance not only as an element of validation
of the knowledge acquired at a specific moment but also as a tool for control and
reflection on the educational quality. The interpretation of the results obtained
allow the student's learning to be ascertained and at the same time direct and
adapt the teaching-learning process in an appropriate way in order to achieve the
proposed objectives and (Filderman & Toste, 2018; Kurth et al., 2020; Nonato &
Almeida, 2019; Yanez, 2016), an interrelated system, not limited to student
assessment, but to the assessment of all those involved in the teaching-learning
process, whose conceptions and actions have a direct impact on the system
(Deneen & Brown, 2016; Santos Guerra, 2017), as well as on the process itself.
Although, it is necessary to recognise assessment as a complex term, in this article
we will focus only on student assessment.
Assessment goes beyond checking a result or the acquisition of knowledge; it

should be aimed at understanding the development of skills, abilities, procedures,
attitudes, and values in students. Thus, for assessment to be effective, it is
necessary to consider both the context and the resources designed to carry it out,
considering the characteristics of the students and their approach to learning
(Ibáñez, 2020; Yanez, 2016) to favour the correct demonstration of knowledge
(Lormendez Trujillo & Cano Ruiz, 2020). To this end, care must be taken in the
choice of assessment methods. Teachers, as the agents involved, must know, and
be prepared to design and develop assessment. Using methodologies that are
appropriate to the students so that they can show the skills developed (Deneen &
Brown, 2016), because different approaches to learning call for different ways of
presenting assessment (Anijovich & Cappelletti, 2017). A varied evaluation will
provide a more holistic and realistic view of both the knowledge acquired and the
learning process itself. Evaluation doesn’t necessarily result in a score, certainly
not in the light of assessing students with learning disabilities., since the whole
learning process must be subject to monitoring, to facilitate the necessary
adaptation and to achieve the established objectives. For this, it must be
continuous and practical, collecting diverse evidence, in different contexts and
moments; furthermore, the learner must be an active participant, reflecting on
their process (Araújo Chagas & Mauch Palmeira, 2019) and it must evaluate small
goals, offering adequate support to stimulate achievement.
On the other side, we can’t forget that school has a first responsibility, and that is
to offer quality education, developing each person's potential to the maximum.
To do so, it must adapt to this changing society and, in turn, to the needs of each
individual, enabling their full personal development (Pazos González et al., 2015).
In this changing world, the Information and Communication Technologies (ICTs)
have brought about a revolution, integrating and promoting transformations in
all areas of our society. These techniques are being used in all fields, including
education, where they complement traditional teaching methods (Ortiz Ruiz &
Manzano Villagra, 2013; Sosa Díaz & Valverde Berrocoso, 2020).
81
ICTs are presented as a driver of educational change and innovation. These

transformations are allowing for new possibilities that favour comprehensible
and flexible environments in order to develop the capabilities of all students to
their fullest potential (Tangarife Chalarca et al., 2016; Morilla Mazuecos, 2012).
They constitute a source of resources that facilitate the teaching-learning process,
from the access and acquisition of knowledge to the development of skills and
abilities or the promotion of attitudes and values, as well as their evolution and
assessment. These new resources and their alternatives are the ones that provide
people with intellectual disabilities with different ways to face their learning,
allowing them to adapt better to their characteristics and educational needs
(Hernández Sánchez et al., 2020). This and the lack of specific studies and
materials on sustainability for students with Down Syndrome gave rise to the idea
of developing a virtual learning platform (VLE) for them.
The platform aims to help this group to understand the need to care for our
environment, making them participants in the concept of sustainability, both in
the domestic and industrial spheres. It also seeks to facilitate awareness and the
acquisition of sustainable habits in the field of recycling.
This VLE has been developed in the project "SUSKIDS - Empowering

professionals and families to transfer knowledge and skills on sustainability to
people with Down Syndrome" co-funded by the European Erasmus+ instrument
(SUSKIDS, n.d.). This new technological tool contributes to addressing
environmental sustainability through an active and participatory methodology
focused on students.
This article collects the guidelines of three groups of experts in the field of
education of students with functional diversity and education of students with
Down Syndrome, from Ireland, Belgium, and Spain. Their aim was to discuss
various aspects of assessment, ranging from standards and methods of
assessment to appropriate forms of assessment in the VLE with regard to its
adaptation to students with Down Syndrome.
The learning portfolio is a learning and assessment system, which allows to collect
a set of evidences of the learning process and of what has been learnt as a result
of different activities carried out by the student, showing the skills and knowledge
that he/she has. It can take a variety of formats, such as photographs, videos,
designs, written texts, audios, etc.
These guidelines have been defined in the context of the development of the VLE
of the SUSKIDS project, and aim to answer the following research questions:
• What are the compiled aspects to be considered regarding the assessment of
students with Down Syndrome?
• What are the general guidelines on assessment that the experts from the three
countries agree on? How do they rate each of them?
• What are expert views on the different aspects of assessment of students with
Down Syndrome in the field of education?
82
• What assessment method do they propose to use for sustainability materials

and content designed for students with Down Syndrome?
• What opinion do the experts have about the learning portfolio as an
assessment method?
2. Methodology
In order to establish appropriate assessment methods, the following actions have
been followed. The first methodological action, through instrumental case studies
and initial input from experts, assessment methods have been explored from
multiple perspectives to establish key principles. Assessment in education
involves collecting, interpreting, and using information about learning processes
and sources. Given its complexity, a broad perspective has been adopted that
considers assessment methods for students with Down Syndrome, assessment
methods on the topic of sustainability and inclusive assessment methods. In order
to carry out this instrumental case review, around twenty-five inputs were
consulted, such as scientific articles, books scientific articles, books and official
websites, among others. The research by Lormendez and Cano (2020) collects
opinions from specialized teachers who suggest a qualitative assessment based on
evidence such as photographs or videos and rubrics, where the teacher establishes
the objectives to be achieved by their students. The selection of techniques and
instruments must be chosen according to the characteristics of said students. In
the case of students with Down Syndrome, they recommend the use of the
portfolio and observation.
In the study by Bizama and Martínez (2021), the assessment practices, for students
with intellectual disabilities, used by specialists follow traditional procedures to
which they apply access adaptations and adjustments to the objectives to be
achieved. The incorporation of the Universal Learning Design (UDL) promotes
curricular flexibility, which facilitates adaptation to the characteristics of the
students. Some of the adaptations they make are increasing the font of the texts,
presenting examples, incorporating images to improve understanding, dividing
an activity into steps, oral tasks, etc.
Other authors value the use of ICT as support for the teaching-learning process
and as an assessment tool for students with Down Syndrome. Da Cruz et al.,
(2020) analyse how the data recorded in a virtual learning platform (sequence of
actions, reaction time, decision making) provide information that allows
evaluating the learning progress of students and adopting timely measures. to
improve their performance. Toffalini et al., (2018) create virtual environments to
promote the learning and assessment of visuospatial skills; They use automatic
records of the time spent, the number and type of breaks and the errors made to
know the student's progress and evolution.
Secondly, the established key principles and a presentation of the SUSKIDS

project with its objectives, contents, methodology and tools to achieve the defined
goals is given to the focus groups from the three countries. The contributions of
the groups are submitted to a Delphi study. A systematic forecasting technique
for gathering information from subject matter experts and reaching consensus
(Cruz Ramírez & Rúa Vásquez, 2018; López-Gómez, 2017; Rowe & Wright, 1999).
83
The Delphi study was conducted to validate the assessment guidelines. Each
country gathered its own expert group on education, with 5 members in the case
of Belgium and Spain, and 4 members in Ireland. In the Ireland case the working
group consisted of Special School teachers. The Spanish Working Group had more
variety in the profiles of those attending the group: higher education teachers,
specialist in Therapeutic Pedagogy, Psychologists, Psychopedagogues and
Special Education teachers. And in Belgium the group consisted of experts in
inclusive (higher) education and diversity and special education teachers.
Following the criteria of Rodas & Pacheco (2020), the meetings were held online
and lasted approximately 1 hour. A moderator from each group was in charge of
leading the discussion on assessment methods and techniques and, in turn, of
collecting the main ideas.
The first round of the Delphi study consisted of the focus group discussions to
explore expert opinions and generate ideas on assessment methods. The
information provided by the experts from the three participating countries was
used to create draft assessment guidelines. The second round consisted of two
online surveys with the aim to reach consensus on the guidelines proposed in the
first round. For this purpose, experts received the draft assessment guidelines.
The surveys consisted of Likert-scale and open-ended questions on general
guidelines, complemented by focus group input on the proposed assessment
guidelines, to provide feedback and reach expert consensus. General comments
on the guidelines and the use of a learning log as an assessment method were
solicited, eliciting both quantitative and qualitative feedback.
3. Results
The first methodological action established (instrumental case study) provided an
overview of current evaluation methods in this field. This enabled us to define the
following key principles for assessment:
To assess students with Down syndrome:

Assess the student according to their strengths and needs.
Establish small steps in the learning and assessment process.
Use visual methods.
Combine assessment methods such as questioning, observing and collecting work
samples. Promote learning and social skills, both inside and outside the
classroom.
To assess sustainability:
Assess attitudes and behaviour as indicators.
Align assessment with learning objectives.
Use diverse assessment methods, especially reflective and performance-based
methods.
Empower students in their own learning process, through feedback from teachers,
peers and themselves, and actions of care and respect for the environment.
84
For inclusive assessment:

Assessment policies and procedures should support and enhance the successful
participation and inclusion of all students.
Align assessment accommodations with learning accommodations.
Consider the self-esteem and well-being of all students.
Adopt a growth mindset with a focus on positive achievement.
Maintain high expectations for each student and provide appropriate challenge.
Secondly, the guideline document obtained in the first round of the Delphi study
was used to form the surveys items for the second round of the study. The two
surveys were completed by 8 of the 14 experts, both with closed responses on a 3-
point Likert-type scale. The first one consists of 18 items and the second one of 3
items. The results from the different countries have been taken together. Firstly,
Figure 1 shows the aspects taken into consideration by the experts with regard to
the assessment of students with Down Syndrome and the degree of importance
of these proposed guidelines. All experts rated three key principles as the most
important with a maximum rating of 3: "Maintain high expectations for every
student and provide adequate challenge", "All assessment policies and
procedures should support and enhance the participation and inclusion of all
learners" and "Enable the student to monitor their own learning process through
feedback from teachers, peers and themselves". The experts gave lower scores and
pointed out that providing evidence of learning in a variety of ways can be an
option, but should not be an obligation for the learner and that the teacher should
have autonomy to decide on the best method of assessment, as well as the
knowledge, skills and abilities required by each learner based on their individual
characteristics. The remaining items receive high scores, ranging from 2.25 to 2.88
out of 3.
High expectations and chalenge for every student

Inclusive assessment policies
Empower self-monitoring
Beyond marks and grades, focus on future learning
Social learning and skills
Combined methods
Aligned with learning goals
Attitudes and behavior as key indicators of sustainability
Mixed range of methods, also reflective and performance-based
Growth mindset
Visual methods, adapted to visual impairments
Individual approach to strenghts and needs
Aligned with learning arrangements
Consider self-esteem and wellbeing
Systematic and predictable
Limited steps and repetition
Formative assessment and classroom-based assessment
Evidence of learning in a variety of ways
0 0,5 1 1,5 2 2,5 3
Figure 1: Expert Consensus on Evidence-Based Evaluation Guidelines
85
The second one, the proposed guidelines are presented based on the experts' input
and the degree of importance (Figure 2). The majority of the experts considered
the assessment guidelines to be important to very important, with scores above
2.5 out of 3. The greatest consensus obtained among the experts was for a varied
and diverse assessment with a score of 2.75 out of 3. One of the experts pointed
out that, when working towards equal educational opportunities, the universal
design of assessment is very important. In this way, even students with more
learning difficulties can participate in class with the same opportunities as others
(Lindner & Schwab, 2020). Another expert advised to include a practical
assessment, to verify the actual understanding and internalisation of knowledge,
as well as to check the development and acquisition of skills and abilities.
Evaluate broadly to make the students' learning

visible
The evaluation technique should be universally

designed
Evaluate in the context of the broader

curriculum/learning line to ensure a sustainable
learning process (in contrast to a stand-alone
course with temporary learning goals)
0 0,5 1 1,5 2 2,5 3
Figure 2: Expert Consensus on Expert-Based Evaluation Guidelines
The experts also gave their general opinion on the assessment guidelines. Several
of them considered that the guidelines are clearly defined, timely and useful in
this area of functional diversity. However, one of the experts stated that the
guidelines could be more goal-oriented, as it is important to know the goals in
order to be able to adapt the assessment to them. Another confirmed the clarity of
the guidelines and noted that they are inviting to put them into practice. They are
evidence-based and recognisable for practitioners who also work with scientific
knowledge, and that they have a research approach to classroom practice.
Finally, the experts were asked for their opinion on the learning portfolio as an
assessment method. It is considered by most experts as an appropriate and
interesting form of assessment, as well as a timely resource to enable students to
monitor and reflect on their own learning process. The experts pointed out that
when using this type of learning recording method, attention must be paid to the
definition of the objective to ensure that it is used according to the interests
identified and, as we have already mentioned above in the description of the
portfolio, it is necessary to take into account the format of presentation, as it
should be adjusted to the needs and possibilities of the recipient, e.g. only in
writing or incorporating digital work, the extent to which a template and structure
is provided as a model or not. According to them, the portfolio can be indicative
86
and informative of the student's learning process and can show personal
competences in the form of difficulties and strengths that will help the teacher to
adjust the support more effectively. One of the experts considers that it is
necessary to use a variety of assessment methods to check the acquisition of
knowledge and skills. As a final result of the above guidelines, they indicate an
evaluation method for the learning materials and contents on sustainability
developed in the SUSKIDS project: the portfolio.
4. Discussion
From the contributions and opinions gathered from the expert groups, based on
the case reviews conducted and their own extensive experience in the area of
assessment, sustainability and inclusive education for students with Down
Syndrome, various considerations on appropriate assessment guidelines are
established.
On the one hand, people with Down Syndrome show a more or less characteristic
learning style, apart from individual peculiarities (Balasong, 2022; Faragher et
al., 2020; Van Hooste & Maes, 2008), which provides guidance on how to approach
their educational process in order to provide a meaningful and quality education
(Faragher & Clarke, 2013; Grieco et al., 2015). Therefore, assessment, as an integral
part of the teaching-learning process, must be geared to this learning style of
students, in order to capture and enable students to demonstrate the knowledge
and skills acquired and developed. They are visual individuals and require
explicit learning (Basten et al., 2018; Ruiz Rodríguez, 2012), therefore, assessment
can benefit from the use of sufficiently large and clear pictures, drawings or visual
cues. They need more time for learning and will need more time for assessment,
so learning can be accompanied by continuous assessment and sequenced in small
steps or objectives (Ruiz Rodríguez, 2012). In general, teachers approach
assessment by combining different methods: questioning, observing and
collecting work samples (Faragher et al., 2020). It is advisable to try to apply and
transfer learning to their immediate environment in order to consolidate it outside
and encourage generalisation (Moreira, 2017). Another point to take into account
is their ability to learn by imitation and therefore to value role models in learning,
both inside and outside the classroom (Almendra & Elvas, 2020; Angulo
Domínguez et al., 2008).
On the other hand, Education for Sustainable Development (ESD) or

environmental education aims to raise awareness, knowledge, skills and positive
sustainable attitudes. Opportunities should be created for this purpose and
models and activities should be provided to increase sustainability competences
(Chawla & Cushing, 2007; Díaz-Salazar, 2020; Mínguez Vallejos, 2021; UNESCO,
2017). The United Nations Educational, Scientific and Cultural Organisation
(UNESCO) proposes, alongside learning objectives and activities, the assessment
of ESD learning outcomes, which can act as a guide and starting point for
assessing sustainability. UNESCO recommends aligning the method of
assessment with teaching-learning objectives and practices and using a variety of
methods (UNESCO, 2017). These methods should go beyond traditional
assessment to encompass more reflective and performance-based methods (Klein
87
et al., 2021), for example, teacher and peer feedback and self-assessment, to enable
the learner to monitor their own learning and identify possibilities for
improvement (Anijovich & Cappelletti, 2017; Santos Guerra, 2017).
Finally, assessment should facilitate inclusion; inclusive assessment is an

approach to assessment in general settings designed to promote the learning,
participation, and inclusion of all students as far as possible. Just as all other
elements of the teaching and learning process are adapted, assessment will be
open to appropriate adaptations to respond to the functional diversities of
students, which is not the same as lowering standards but adapting them to the
level of individual attainment (Lormendez Trujillo & Cano Ruiz, 2020).
Assessment content should address the self-esteem and general well-being of
students in order to achieve an inclusive approach (National Council for
Curriculum and Assessment [NCCA], 2017). Assessment should be used to
improve learning and progress, so it should show progress and ways in which the
learner can improve. Seeking and generating examples of positive achievements
builds self-esteem and motivates students to meet the challenges presented
(Anijovich & Cappelletti, 2017).
The key message of the expert group was to pay attention to learning objectives
and evaluation methods tailored to the pupil in front of us. The most important
thing is to adapt and create learning opportunities for a certain pupil. It is
therefore important to see and read the results as guidelines or advice that can be
used to test the learning objectives or to gain insight into the learning process of
the pupil. The simultaneous use of a multitude of different elements (visual, oral,
etc.) maintains interest in learning and ensures interactivity during learning and
assessment. The experts seem to agree that the assessment guidelines are easy to
understand and are conducive to collaboration and reflection.
The method of assessment should always come from the learner's abilities in the
first place, and be subject to the freedom of choice and action of the supervising
teacher or practitioner (Yepes Villa & Gutiérrez Avendaño, 2022). A learning log
or portfolio is proposed as a formative, planned and intentional monitoring
assessment tool of the students' learning experience (NCCA, 2017). It contains
information about what the learner is learning and how he/she is learning it. The
log is a means for the learner to reflect on his or her learning and should allow for
a dialogue between the learner and the teacher (NCCA, 2017; Sartor-Harada et al.,
2022).
The content of the log depends on the learner completing it, but teachers can
provide a structure or template for what the log will look like and what it should
contain, specific prompts or guidelines, or reflective questions. In providing this
kind of support, it should not be forgotten that too many guidelines may hinder
the learner's creativity. An alternative is to provide a limited number of questions
from which the learner can choose to answer and/or develop. An initial message
such as "Work I am proud of" can be included to motivate the learner to collect
evidence of their successes. Teachers may choose to define examples of what work
the learner should or can include according to the learning objectives, e.g., take a
88
photo of an object or describe the material used in making a design, describe what
he/she recycles, etc.
An assessment method for the portfolio can be to compare the evidence gathered
in the learning log with the initial learning objectives. The assessment should be
about the process of growth and reflect the extent to which the learning objectives
are achieved (Romero et al., 2018).
5. Conclusion
Three key aspects were identified by the experts as priorities for the assessment:
maintaining high expectations of students' potential, encouraging self-evaluation,
and enabling students controlling their own learning process. But which method
of assessment is more appropriate for sustainability materials and content
designed for students with Down Syndrome? Most of the experts in the study
conclude that a good option could be the learning portfolio that shows what and
how they learn (Lormendez Trujillo & Cano Ruiz, 2020). This procedure allows
for a variety of formats according to personal preferences and the needs and
possibilities of the student, collects both concepts and practical skills and
transversal competences and encourage self-assessment, reflection and
continuous dialogue with the teacher to adjust, redirect and/or guide educational
practice (Espinoza Freire, 2021).
The format of the portfolio depends on the choice of the learner and the teacher,
but it must always follow Universal Design Requirements. It is suggested that it
should be sustainable, e.g., avoid printing too much paper and thus be in line with
the sustainability principles of the project. It is also encouraged to combine
multiple formats, photos, videos, written reports, drawings, etc., in order to
achieve a broad picture of the learning process and the students' skills from
various perspectives and to be able to assess them in other transversal aspects,
such as digital competence.
At this point, it would be interesting to discuss extending the scope of assessment

to non-school contexts in order to assess the generalisation of learning and
autonomous development, managing this learning in a systematic way and
validating daily tasks and personal experiences.
6. References
Almendra, R. A., & Elvas, M. (2020). Inclusion of Children with Down Syndrome Through
the Creation and Use of a “Learning Object”. In: Di Bucchianico, G. (eds)
Advances in Design for Inclusion. AHFE 2019. Advances in Intelligent Systems
and Computing, vol 954 (pp. 292–300). https://doi.org/10.1007/978-3-030-20444-
0_28
Angulo Domínguez, M. del C., Gijón Sánchez, A., Luna Reche, M., & Prieto Díaz, I. (2008).
Manual de Atención al Alumnado con Necesidades Específicas de Apoyo Educativo
derivadas de Síndrome Down [Manual of Attention to Students with Specific
Educational Support Needs derived from Down Syndrome]. Junta de Andalucía.
Consejería de Educación. http://hdl.handle.net/11162/3175
89
Anijovich, R., & Cappelletti, G. (2017). La evaluación como oportunidad [Evaluation as an

opportunity]. Paidos Argentina, 85–100. http://fediap.com.ar/wp-
content/uploads/2020/07/La-evaluacion-como-oportunidad-Anijovich-y-
Cappelletti.pdf
Araújo Chagas, M. R., & Mauch Palmeira, E. (2019). Como avaliar crianças com síndrome
de Down [How to assess children with Down síndrome]. Revista Atlante:
Cuadernos de Educación y Desarrollo.
https://www.eumed.net/rev/atlante/2019/11/criancas-sindrome-down.html
Balasong, A. N. F. (2022). Memahami individu dengan sindrom Down di tengah
masyarakat dan agama [Understanding individuals with Down syndrome in
society and religion]. Mimikri, 8(2), 286–310.
Basten, I., Boada, R., Taylor, H., Koenig, K., Barrionuevo, V., Brandão, A., & Costa, A.
(2018). On the Design of Broad-Based Neuropsychological Test Batteries to Assess
the Cognitive Abilities of Individuals with Down Syndrome in the Context of
Clinical Trials. Brain Sciences, 8(12), 205. https://doi.org/10.3390/brainsci8120205
Bizama, M., & Martínez, J. (2021). Evaluative Practices of Teachers of Special Education
for Students with Intellectual Disabilities integrated into Regular Schools. Revista
Electrónica Educare, 25(3), 1-15. https://doi.org/10.15359/ree.25-3.22
Chawla, L., & Cushing, D. F. (2007). Education for strategic environmental behavior.
Environmental Education Research, 13(4), 437–452.
https://doi.org/10.1080/13504620701581539
Cruz Ramírez, M., & Rúa Vásquez, J. A. (2018). Surgimiento y desarrollo del método
Delphi: una perspectiva cienciométrica [Emergence and development of the
Delphi method: a scientometric perspective]. Biblios: Journal of Librarianship and
Information Science, 71, 90–107. https://doi.org/10.5195/BIBLIOS.2018.470
Da Cruz Netto, O. L., Rodrigues, S. C. M., de Castro, M. V., da Silva, D. P., da Silva, R. R.,
de Souza, R. R. B, Ferreria de Souza, A. A., & Bissaco, M. A.
S. (2020). Memorization of daily routines by children with Down syndrome
assisted by a playful virtual environment. Scientific Reports, 10 (1), 1-17.
https://doi.org/10.1038/s41598-020-60014-5
Deneen, C. C., & Brown, G. T. L. (2016). The impact of conceptions of assessment on
assessment literacy in a teacher education program. Cogent Education, 3(1),
1225380. https://doi.org/10.1080/2331186X.2016.1225380
Díaz-Salazar, R. (Coord.). (2020). Ciudadanía global. Una visión plural y transformadora de la
sociedad y de la escuela [Global citizenship. A plural and transformative vision of
society and school]. (SM). https://www.kaidara.org/wp-
content/uploads/2020/10/CIUDADAN%C3%8DA-GLOBAL.-Una-
visi%C3%B3n-plural-y-transformadora-de-la-sociedad-y-de-la-escuela..pdf
Espinoza Freire, E. E. E. (2021). Importancia de la retroalimentación formativa en el
proceso de enseñanza-aprendizaje [Importance of formative feedback in the
teaching-learning process]. Revista Universidad y Sociedad, 13(4), 389–397.
https://rus.ucf.edu.cu/index.php/rus/article/download/2178/2158
Faragher, R., & Clarke, B. (2013). Educating Learners with Down Syndrome. Research, theory,
and practice with children and adolescents (R. Faragher & B. Clarke, Eds.). Routledge.
https://doi.org/10.4324/9781315883588
Faragher, R., Robertson, P., & Bird, G. (2020). International guidelines for the education of
learners with Down syndrome. https://www.unsa.ba/sites/default/files/inline-
files/International_Guidelines_for_the_Education_of_Learners_with_Down_Sy
ndrome_-_DSi_-_July_2020.pdf
90
Filderman, M. J., & Toste, J. R. (2018). Decisions, Decisions, Decisions: Using Data to Make
Instructional Decisions for Struggling Readers. TEACHING Exceptional Children,
50(3), 130–140. https://doi.org/10.1177/0040059917740701
Gamonales Puerto, J. M., Porro Cerrato, C., Gil Sánchez, O., Mancha Triguero, D., Gómez
Carmona, C. D., & Gamonales Puerto, F. J. (2018). Inclusión de los alumnos con
Síndrome de Down en el Aula de Educación Infantil [Inclusion of pupils with
Down's Syndrome in the Early Childhood Classroom]. Revista Profesional de
Investigación, Docencia y Recursos Didácticos, 100, 339–350.
https://core.ac.uk/download/pdf/235851516.pdf
Grieco, J., Pulsifer, M., Seligsohn, K., Skotko, B., & Schwartz, A. (2015). Down syndrome:
Cognitive and behavioral functioning across the lifespan. American Journal of
Medical Genetics Part C: Seminars in Medical Genetics, 169(2), 135–149.
https://doi.org/10.1002/ajmg.c.31439
Hernández Sánchez, B., Vargas Morua, G., González Cedeño, G., & Sánchez García, J. C.
(2020). Discapacidad intelectual y el uso de las tecnologías de la información y
comunicación: revisión sistemática [Intellectual disability and the use of
information and communication technologies: a systematic review]. Revista
INFAD de Psicología. International Journal of Developmental and Educational
Psychology., 2(1), 177–188. https://doi.org/10.17060/ijodaep.2020.n1.v2.1830
Ibáñez, R. (2020). Los procesos de evaluación y toma de decisiones en el desarrollo y
aprendizaje de los párvulos en jardines infantiles de la Junta Nacional de Jardines
Infantiles de la Región Metropolitana, Chile [The processes of evaluation and
decision-making in the development and learning of infants in kindergartens of
the National Kindergarten Board of the Metropolitan Region, Chile.]. Pensamiento
Educativo: Revista de Investigación Educacional Latinoamericana, 57(1), 1–20.
https://doi.org/10.7764/PEL.57.1.2020.3
Jurado-de-los-Santos, P., Colmenero-Ruiz, M. J., Valle-Flórez, R. E., Castellary-López, M.,
& Figueredo-Canosa, V. (2021). The Teacher’s Perspective on Inclusion in
Education: An Analysis of Curriculum Design. Sustainability, 13(9), 4766.
https://doi.org/10.3390/su13094766
Kabashi, L., & Kaczmarek, L. (2019). Educating a Child with Down Syndrome in an
Inclusive Kindergarten Classroom. Journal of Childhood & Developmental Disorders,
05(02). https://doi.org/10.36648/2472-1786.5.2.81
Klein, S. G., Pereira, D. N., & Muenchen, C. (2021). Avaliação da aprendizagem na
abordagem tematica: um olhar para os três momentos pedagógicos [Learning
assessment in the thematic approach: a look at the three pedagogical moments].
Investigações Em Ensino de Ciências, 26(1), 375. https://doi.org/10.22600/1518-
8795.ienci2021v26n1p375
Kurth, J. A., Miller, A. L., & Toews, S. G. (2020). Preparing for and Implementing Effective
Inclusive Education With Participation Plans. TEACHING Exceptional Children,
53(2), 140–149. https://doi.org/10.1177/0040059920927433
Lindner, K.-T., & Schwab, S. (2020). Differentiation and individualisation in inclusive
education: a systematic review and narrative synthesis. International Journal of
Inclusive Education, 1–21. https://doi.org/10.1080/13603116.2020.1813450
López-Gómez, E. (2017). El método Delphi en la investigación actual en educación: una
revisión teórica y metodológica [The Delphi method in current educational
research: a theoretical and methodological review]. Educación XX1, 21(1).
https://doi.org/10.5944/educxx1.20169
Lormendez Trujillo, N., & Cano Ruiz, A. (2020). Educación inclusiva de preescolares con
síndrome de Down: la mirada de un grupo de educadoras de Veracruz, México
[Inclusive education of preschoolers with Down syndrome: the perspective of a
91
group of educators from Veracruz, México]. Actualidades Investigativas En

Educación, 20(2), 27. https://doi.org/10.15517/aie.v20i2.41665
Mínguez Vallejos, R. (2021). Una mirada educativa del desarrollo sostenible o no te andes
por las ramas [An educational look at sustainable development or don't beat
around the Bush]. III Congreso Internacional RIPEME 2021. Pedagogía y Construcción
de Ámbitos de Educación. La Función de Educar.
https://digitum.um.es/digitum/handle/10201/119467
Moreira, M. A. (2017). Aprendizaje significativo como un referente para la organización
de la enseñanza [Meaningful learning as a reference for the organisation of
education]. Archivos de Ciencias de La Educación, 11(12), 29.
https://doi.org/10.24215/23468866e029
Morilla Mazuecos, R. (2012). Las TICs en alumnos y alumnas con síndrome de Down [ICTs
for pupils with Down Syndrome]. Revista Internacional de Educación, Tecnologías de
La Información y Comunicación Aplicadas a La Educación Inclusiva, Logopedia y
Multiculturalidad, 1(2), 20–26.
https://tecnoeducalem.weebly.com/uploads/4/4/7/9/44791473/02_tecnoedu
calem_marzo_2012.pdf
National Council for Curriculum and Assessment (NCCA). (2017). Ongoing reporting for
effective teaching and learning. https://ncca.ie/en/resources/ongoing-reporting-
for-effective-teaching-and-learning/
Nonato, E. M. N., & Almeida, E. S. P. de. (2019). Breves considerações acerca da
historicidade da avaliação pedagógica [Brief considerations about the historicity
of the pedagogical assessment]. Revista Eletrônica de Educação, 13(1), 291.
https://doi.org/10.14244/198271992502
Ortiz Ruiz, Y., & Manzano Villagra, N. (2013). Uso de material audiovisual en actividades
académicas de carreras del Departamento de Educación de la Universidad de Los
Lagos [Use of audiovisual material in academic activities in courses of the
Department of Education of the Universidad de Los Lagos]. Virtualidad, Educación
y Ciencia, 4(4), 8–19. 1853-6530
Pazos González, M., Raposo-Rivas, M., & Martínez-Figueira, M. E. (2015). Las TIC en la
educación de las personas con Síndrome de Down: un estudio bibliométrico [ICT
in the education of people with Down Syndrome: a bibliometric study].
Virtualidad, Educación y Ciencia, 6(11), 20–39.
https://revistas.unc.edu.ar/index.php/vesc/article/view/12767
Rodas Pacheco, F. D., & Pacheco Salazar, V. G. (2020). Grupos Focales: Marco de
Referencia para su Implementación [Focus Groups: Implementation
Framework]. INNOVA Research Journal, 5(3), 182–195.
https://doi.org/10.33890/innova.v5.n3.2020.1401
Romero, L., Gutierrez, M., & Caliusco, Ma. L. (2018). Conceptual modeling of learning
paths based on portfolios: Strategies for selecting educational resources. 2018 13th
Iberian Conference on Information Systems and Technologies (CISTI), 1–6.
https://doi.org/10.23919/CISTI.2018.8399270
Rowe, G., & Wright, G. (1999). The Delphi technique as a forecasting tool: issues and
analysis. International Journal of Forecasting, 15(4), 353–375.
https://doi.org/10.1016/S0169-2070(99)00018-7
Ruiz Rodríguez, E. (2012). Programación educativa para escolares síndrome de Down
[Educational programming for Down Syndrome schoolchildren]. Fundación
Iberoamericana Down21. https://www.down21.org/libros-
online/libroEmilioRuiz/libroemilioruiz.pdf
Santos Guerra, M. Á. (2017). Evaluar con el corazón. De los ríos de las teorías al mar de la
práctica [Evaluating with the heart. From the rivers of theories to the sea of
92
practice]. Praxis Educativa, 21(3), 79–80.

https://doi.org/10.19137/praxiseducativa-2017-210310
Sartor-Harada, A., Ulloa-Guerra, O., Deroncele-Acosta, A., & Pérez-Ochoa, M. E. (2022).
Pedagogical Opportunities of the Reflective Learning Portfolio. Revista de Filosofía.
Centro de Estudios Filosóficos “Adolfo García Díaz” Universidad Del Zulia, 39(102),
530–551. https://doi.org/10.5281/ZENODO.7050873
Sosa Díaz, M. J., & Valverde Berrocoso, J. (2020). Perfiles docentes en el contexto de la
transformación digital de la escuela [Teacher profiles in the context of the digital
transformation of schools]. Bordón. Revista de Pedagogía, 72(1), 151–173.
https://doi.org/10.13042/Bordon.2020.72965
SUSKIDS. (s.f). European project ERASMUS+KA2. Strategic Partnerships for School
Education. Sustainable Knowledge and Skills to Down syndrome individuals
2018-2021. España: Universidad de Burgos. http://suskids.eu/?lang=es
Tangarife Chalarca, D., Blanco Palencia, S. M., & Diaz Cabrera, G. M. (2016). Tecnologías
y metodologías aplicadas en la enseñanza de la lectoescritura a personas con
síndrome de Down [Technologies and methodologies applied in the teaching of
literacy to people with Down Syndrome]. Digital Education Review, 29, 265–283.
https://dialnet.unirioja.es/servlet/articulo?codigo=5580043
Toffalini, E., Meneghetti, C., Carretti, B., & Lanfranchi, S. (2018). Environment learning
from virtual exploration in individuals with down syndrome: the role of
perspective and sketch maps. Journal of Intellectual Disability Research, 62(1), 30-40.
https://doi.org/10.1111/jir.12445
UNESCO - United nations Educational, Scientific and Cultural Organization (2017).
Education for Sustainable Development Goals: learning objectives.
https://unesdoc.unesco.org/ark:/48223/pf0000247444.locale=es
Van Hooste, A., & Maes, B. (2008). Kinderen Met Down. Een Kind Met Een Verstandelijke
Handicap In Je Gezin. Lanoo.
Yanez, P. (2016). El proceso de aprendizaje: fases y elementos fundamentales [The learning
process: phases and key elements]. Revista San Gregorio, 1, 70–81.
https://revista.sangregorio.edu.ec/index.php/REVISTASANGREGORIO/articl
e/view/19
Yepes Villa, E. E., & Gutiérrez Avendaño, J. (2022). Evaluación formativa como proceso
mentor en la enseñanza y aprendizaje hacia la calidad educativa [Formative
assessment as a mentoring process in teaching and learning towards quality
education]. Revista de Ciencias Sociales, 28, 255–269.
https://doi.org/10.31876/rcs.v28i.38844
93

Vol. 22, No. 10, pp. 93-112, October 2023
A Systematic Review of Interventions

Improving University Students’ EFL Writing
Competence
Dandan Zhang*
Universiti Tunku Abdul Rahman (UTAR), Malaysia
Joanna Tjin Ai Tan†

Swagata Sinha Roy

Abstract. Writing competence in English as a Foreign Language (EFL) is

a critical skill for university students, as it affects their academic and
career success. However, many EFL university students are not capable
of writing English compositions and face considerable challenges in
developing their writing skills. To address these challenges, various
interventions have been proposed, yet few studies have integrated them
in recent years. This study employed PRISMA guidelines to analyze
forty-two empirical studies on interventions improving university
students’ EFL writing competence based on peer-reviewed journals
published between 2012 and 2022. The review identified the types of
interventions, evaluated their effectiveness in improving students'
writing competence, and provided recommendations for teachers and
future researchers. Findings showed that teaching instruction, feedback,
cooperative learning and modern technology were applied into teaching
activities, which improved university students’ EFL writing competence
to various degrees. Teachers are required to select suitable interventions
or combine different interventions to help university students to
effectively solve problems in English writing. Furthermore, the findings
will make a contribution to understanding recent interventions in EFL
writing instruction in university settings, providing a reference for
educators to develop evidence-based practices to enhance university
students’ EFL writing competence.
Keywords: Interventions; University students; English as a Foreign

Language; writing competence
*
Corresponding author: Joanna Tjin Ai Tan; tanta@utar.edu.my
©Authors
94
1. Introduction
Writing is considered to be one of the most essential inventions and possessing a
good proficiency in writing techniques is important in every aspect of life
(Gilbert, 2010). With the ever-increasing interconnectedness between countries
and international exchanges becoming more frequent, English, as the most
widely spoken language, has occupied an increasingly important position in
teaching and learning around the world; thus, English writing ability has been
receiving more attention. Besides cultivating an ability to generate and organize
ideas, English writing practice can foster students’ analytical and critical
thinking, reinforcing their learning and reflecting their ability to study (Khalil,
2018). “Writing well in English is not only an art rather the academic and social
success of students is based on their proficiency and competency in writing
skills” (Akbar et al., 2018, p. 7).
Despite its importance, English writing has always represented a weak link for
college students in their English learning process. Research has shown that
English writing remains a challenging task that causes problems for EFL
students (Astrid et al., 2019; De Silva, 2015; Hanh & Tinh, 2022; Wang, 2015;
Zaghlool, 2020). A large number of EFL college students encounter difficulties in
making their language work in a paragraph, such as constructing a
grammatically correct sentence, or choosing appropriate vocabulary to link and
convey their ideas (Mukminin et al., 2015; Makmur et al., 2016).
To improve EFL students’ writing competence, various approaches have been

studied and applied to teaching activities. As the most acceptable method in EFL
classes (Tang, 2012; Torto, 2014), the product approach is based on linguistic
knowledge, placing great emphasis on the finished essay and requiring students
to avoid errors in terms of writing, grammar, tense and spelling (Pincas, 1982, as
cited in Hashemnezhad & Hashemnezhad, 2012; Tang, 2012). Due to the
deficiency of the traditional product approach in fostering students’ cognitive
processes and expression (Silva, 1997), the process approach shifted the
emphasis from the final written product to the writing process (Silva, 1997),
which was regarded as a complicated process to deal with problems
(Hashemnezhad & Hashemnezhad, 2012). Through brainstorming, group
discussion and re-writing, teachers encouraged their students to participate in
various classroom activities, paying attention to the whole language rather than
individual sentences (Klimova, 2013). However, realizing the failure of the
process approach in offering cultural and linguistic resources to students
(Hyland, 2003; Johns, 2002), researchers introduced genre pedagogy, which
focused on the social context where writing activities was conducted. Genres
were employed with specific social purposes (Gibbons, 2002), such as stories,
research reports, resumés and other forms, to help students succeed in particular
settings (Paltridge, 2004). Meanwhile, to compensate for the deficiency of the
above approach, the process-genre approach was introduced. It emphasized the
advantages of both the process and genre when implemented in the classroom
(Torto, 2014), combining the language, context, purpose, writing skills and the
processes in the writing. Furthermore, modern technology also significantly
©Authors
promotes the teaching of writing, through the use of online writing tutorials and
computer assisted language learning (CALL) programs (Zaini & Mazdayasna,
2014, 2015; Azari, 2017; Ismail et al., 2012; Suthiwartnarueput & Ratanakul,
2018). These interventions provide students with immediate feedback and allow
them to practice writing in a supportive environment, which is effective to
cultivate students' writing skills. Also, explicit instructions on teaching students
specific writing skills, such as how to write an effective topic sentence or how to
use transitions, were also used to improve students’ writing competence
(Alawerdy & Alalwi, 2022; Baghbadorani & Roohanik, 2014; De Silva, 2015;
Zhang & Zhang, 2021).
In studying the plethora of interventions intended to improve EFL students’

writing competence, researchers have grouped them into different categories,
such as interventions that focus on summarizing (Selvaraj & Aziz, 2019). Since
few studies have integrated interventions implemented to develop university
students’ EFL writing competence in the last decade, through a comprehensive
analysis of empirical studies, this review sought to identify the types of
interventions that have been used, summarize their effectiveness in improving
university students’ EFL writing competence, and provide recommendations for
teachers and future researchers. Specifically, this paper was dedicated to
answering the following questions:
(1) What are the interventions that have been implemented to improve
university students’ EFL writing competence?
(2) What impact did the interventions have on university students’ EFL writing
competence?
This study contains four sections. The first section provides a general
introduction. Next, the second section describes the details of the review process
before the third section presents the research results. Finally, the conclusion is
presented in the fourth section.
2. Methodology
2.1 Search strategy
To synthesize and analyze literature on the interventions implemented to
improve university students’ EFL writing competence, the researcher performed
a systematic review, which followed the PRISMA guidelines (Moher et al., 2009).
Studies related to the relevant topic were selected in this paper.
The search was applied to electronic databases as follows: Scopus,

ScienceDirect, Web of Science Core Collection as well as EBSCOhost Research
Databases that covered Academic Search Ultimate, Art & Architecture Source,
Business Source Complete, Communication & Mass Media Complete,
Computers & Applied Sciences Complete, Humanities International Complete,
MEDLINE, MEDLINE Complete, Psychology and Behavioral Sciences Collection
and SocINDEX with Full Text, during the period from 2012 to 2022. Only journal
article publications conducted worldwide and written in English were included.
Keywords related to the topic of interventions implemented to improve
university students’ EFL writing competence were used for the search. Various
related keywords were included, such as university students, pedagogy,
teaching, intervention, EFL writing, and ESL writing; these were combined with
“OR” within each group of keywords, and with “AND” between those groups.
2.2 Inclusion and exclusion criteria

a. Included articles were those published between 2012 and 2022; those
published before 2012 or after 2022 were not included.
b. The articles had to contain longitudinal or empirical investigation and have a
sample size that was statistically valid; studies analyzing the results of a
single intervention or focusing on a few particular objects did not meet the
inclusion criteria.
c. The participants in the studies had to be undergraduates of EFL/ ESL
learners in colleges or universities; graduate and postgraduate students,
primary and middle school students, and native English speakers were
excluded.
d. The assessment approaches had to be objective, or both objective and
subjective, evidenced by the writing score or the reduction of grammatical
errors; studies with results from the students’ (or teachers’) subjective
cognizance or perception only were not included.
2.3 Data extraction

The research was limited to peer-reviewed publications in English. By following
the guidelines provided by the PRISMA flow chart, four important processes
including identification, screening, eligibility and inclusion aided the selection of
the required articles (Moher et al., 2009). During the identification process, 12741
articles were found in the databases using the search keywords mentioned
above, which included 8794 from EBSCOhost Research Databases, 3162 from
ScienceDirect, 401 from Scopus and 384 from Web of Science Core Collection.
The identified studies were exported in ENDNOT(X9) (Hupe, 2019), with 288
duplications being removed prior to the screening process. Then, titles and
abstracts of the 12453 identified studies in the electronic databases were
screened for eligibility and any uncertainty over the inclusion criteria at this
stage triggered a full-text review. Following the review, 135 articles remained
and were carefully read and assessed against the eligibility criteria. As a result,
93 articles were removed as the participants were not undergraduate students in
universities, or no control group was used, or the results were not directly
related to writing competence. Finally, 42 articles remained, including 14 articles
from EBSCOhost, 5 articles from ScienceDirect, 7 articles from Scopus and 16
articles from Web of Science Core Collection. The process of selecting the
reviewed articles is shown in Figure 1. Data integration and analysis were
managed in Microsoft Excel.
Figure 1: Diagram of article selection for the systematic review
3. Results and discussion

3.1 Research design
Data relating to the included articles (n=42) are stated in Table 1. All forty-two
studies included were designed as experiments and twenty-five of these [5, 8-10,
12-15, 17, 20-21, 23-24, 26-30, 32, 34, 36-38, 40-41] stated explicitly that the
designs were quasi-experimental, with participants being segregated into one or
two experimental groups (i.e. students receiving a particular intervention) to
verify the influence of the target intervention on students’ writing competence in
various aspects, through comparison with a control group (i.e. students
receiving no intervention). Furthermore, all of the studies reviewed included a
pretest, treatment and posttest(s). Seven studies [10, 14, 23, 27, 30, 37, 40]
employed both an immediate posttest and a delayed posttest for evaluating the
development of students’ writing competence after an extended period.
Of the selected studies, nineteen employed a mixed methods approach while the
remaining twenty-three adopted a quantitative method. In addition to the
writing test, which was used in all the studies, other instruments included
(open-ended) questionnaires [1, 3-4, 6, 8-12, 14, 18, 21, 24, 27, 32, 35, 37, 40],
interviews [4, 17, 19, 25, 26, 31- 33] students’ task performances, logs, language
samples from learning platform [5, 19, 21] as well as teachers’ journals, notes and
observation [21] to provide necessary data for analysis and discussion.
Though not all the studies reported the source of their test paper and most of the
studies under review selected different test papers and rubrics for measurement,
some of them shared the same. For instance, six studies [2, 6-7, 13, 20-21] used
the Oxford Quick Placement Test to measure students’ English proficiency level
before the experiment and six studies chose IELTS writing tasks [8, 12, 20-21, 26,
35, 37] during the treatment. In addition, six studies [4, 6, 18, 23, 26, 41]
employed the rubric from Jacobs et al. (1981) for measurement.
3.2 Participants
The forty-two studies in this review were all conducted in Asian countries stated
in Figure 2, specifically China (n=10) [10, 15, 23, 25, 32, 36-37, 40-42], Iran (n=10)
[6-7, 12-13, 20-21, 26, 28, 38, 39], Saudi Arabia (n=5) [2-4, 9, 34], Vietnam (n=3)
[17, 31, 35], Taiwan China (n=2) [14, 24], Spain (n=2) [5, 16], Iraq (n=2) [1, 22],
Yemen (n=1) [29], Thailand (n=1) [33], Sri Lanka (n=1) [11], Philippine (n=1) [8],
Pakistan (n=1) [30], Malaysia (n=1) [18], Korea (n=1) [19], and India (n=1) [27].
Additionally, the participants in the included studies were mainly from the
abovementioned countries.
Figure 2: Diagram of the quantity of study conducted in each country
While four of the studies [11, 17, 38, 42] did not report any information related to
participants’ grades, and only nine studies [3, 9, 12-14, 21, 24, 30, 34] mentioned
the participants’ ages, which was approximately between 17 and 23, ten studies
[2, 6, 8, 15-16, 19, 25, 28, 33, 36] involved participants in their first year of
undergraduate study, eleven studies [10, 20, 23, 26, 27, 31, 32, 35, 37, 39, 40]
focused on second-year students and seven studies [1, 5, 7, 18, 22, 29, 41] had
participants who were senior students in their third year of study. Indeed,
participants in Alharthi’s (2021) [4] study were from various grades, including
freshmen and sophomores as well as advanced EFL learners.
In terms of the participants’ majors, most of them were English-related,

including English majors (n=10) [4, 10, 15, 20-21, 31, 35, 40-42], English language
and literature majors (n=4) [6, 23, 27, 39], English translation majors (n=2) [26,
28], Business English majors (n=2) [32, 37], teaching English as a foreign
language (n=2) [13, 38], literature and applied linguistics majors [1] and students
who were from the English department but whose major was not clearly defined
(n=4) [1, 16, 22, 29], while non-English majors were only included in ten studies
(with [25, 36] not specifically reporting), including engineering [14, 19, 33],
management [14], materials [14], architecture [14], design [14], science[11, 19, 33],
public administration [19], art [8, 19], nursing [33], and pharmacy [33],
information technology [17], Vietnamese studies [17], primary education [17],
Math pedagogy [17], and chemistry pedagogy [17], Psychology [30], Economic
[17], History [17], Islamic studies [17], and business [9]. The remaining studies
(n=7) [2-3, 5, 7, 18, 24, 34] did not report the major subjects of the participants.
Figure 3: Diagram of the quantity of study conducted in each major
3.3 Intervention categories

Based on the focus of the implemented interventions, four categories were
identified among the selected studies, which were teaching instruction
intervention (n=13), feedback-based intervention (n=8), cooperative learning
intervention (n=4), and modern technology-based intervention (n=17). A
detailed analysis of the interventions is presented as follows.
Table 1: Literature matrix of relevant studies
Sampling
Authors Country Duration Sample size Intervention Design
method
Abbas and Al-bakri (2018) simple random IG: 40;
Iraq 15 weeks Paired writing technique quantitative research
[1] sampling CG: 38
IG: 21; Explicit instruction of
Alawerdy and Alalwi simple random
Saudi Arabia over 5 weeks conjunctions as cohesive mixed research
(2022) [2] sampling CG: 22
devices
simple random IG: 23;
Aldossary (2021) [3] Saudi Arabia 8 weeks Collaborative writing quantitative research
sampling CG: 23
stratified IG: 35;
Alharthi (2021) [4] Saudi Arabia 13 weeks random Free writing mixed research
CG: 45
sampling
purposive IG: 40;
Andujar (2016) [5] Spain 6 months Mobile instant messaging mixed research
sampling CG: 40
convenience IG: 24; Weblog-based process
Azari (2017) [6] Iran not mentioned mixed research
sampling CG: 19 approach
stratified IG: 30;
Baghbadorani and Self-regulated strategy-
Iran not mentioned Random quantitative research
Roohani (2014) [7] CG: 30 based instruction
Sampling
convenience IG: 48;
Barrot (2020) [8] Philippine one semester Facebook-based e-portfolio mixed research
sampling CG: 41
IG: 31; Flipped classroom
Chatta and Haque (2020) cluster
Saudi Arabia not mentioned instruction through mixed research
[9] sampling CG: 32
Blackboard
Cheng and Zhang (2021) convenience IG: 36; Provision of comprehensive
China 9 weeks quantitative research
[10] sampling CG: 36 written corrective feedback
stratified IG: 36;
De Silva (2015) [11] Sri Lanka 6 months random Writing strategy instruction quantitative research
CG: 36
sampling
one- IG: 30;
Emotional intelligence
Ebrahimi et al. (2018) [12] Iran educational- not mentioned quantitative research
CG: 13 enhancement
year
one semester IG: 27; quantitative research
Fathi and Rahimi (2022) convenience
Iran (about 16 Flipped classroom
[13] sampling CG: 24
weeks)
Fu et al. (2019) [14] Taiwan, China 18 weeks convenience IG: 38; Mind mapping-based mixed research
sampling CG: 36 contextual gaming approach
IG1: 43;
purposive Concept-based approach to
Fu and Liu (2022) [15] China 16 weeks IG2: 21; quantitative research
sampling teaching genre
CG: 24
Ghouali and Cecilia (2021) probability IG: 21;
Spain 4 months Moodle-based assessment quantitative research
[16] sampling CG: 21
purposive IG: 29;
Hanh and Tinh (2022) [17] Vietnam 10 weeks Peer-review checklist mixed research
sampling CG: 29
cluster IG: 30;
Ismail et al. (2012) [18] Malaysia over 8 weeks IQ-Write program mixed research
sampling CG: 30
IG: 11; Collocation learning
purposive
Kang (2019) [19] Korea 16 weeks through a Web- mixed research
sampling CG: 13
Concordancer
Approx. 4 convenience IG: 67;
Karami et al. (2018) [20] Iran Electronic portfolio quantitative research
months sampling CG: 67
convenience IG: 84;
Karami et al. (2019) [21] Iran 16 weeks Electronic portfolio mixed research
sampling CG: 67
cluster IG: 45; Cognitive prospective
Khalil (2018) [22] Iraqi not mentioned quantitative research
sampling CG: 45 FLOW teaching strategy
convenience IG: 24; Structured small-group
Li and Zhang (2021) [23] China 16 weeks quantitative research
sampling CG: 24 student discussion
convenience IG: 35; A flipped contextual game-
Lin et al. (2018) [24] Taiwan, China 11 weeks mixed research
sampling CG: 33 based learning approach
IG: 26; Data-driven learning
Luo (2016) [25] China not mentioned not mentioned activities with the assistance mixed research
CG: 22
of BNCweb
Mazloomi and Khabiri purposive IG: 30;
Iran 3 months Self-assessment mixed research
(2018) [26] sampling CG: 30
Mekala and Ponmani purposive IG: 58; Direct written corrective
India 14 weeks quantitative research
(2017) [27] sampling CG: 58 feedback
Naghdipour and Koc purposive IG: 33; Writing instruction within
Iran 32 weeks mixed research
(2015) [28] sampling CG: 35 the process genre approach
Nassar and Al Tameemy simple random IG: 18;
Yemen not mentioned Written peer feedback quantitative research
(2021) [29] sampling CG: 18
stratified IG: 25;
Nusrat et al. (2019) [30] Pakistan 12 weeks random Indirect written feedback quantitative research
CG: 25
sampling
convenience IG: 35; Collaborative writing based
Pham (2021) [31] Vietnam not mentioned mixed research
sampling CG: 27 on writing process approach
convenience IG: 39; An AWE-aided assessment
Sun and Fan (2022) [32] China 16 weeks mixed research
sampling CG: 34 approach
Suthiwartnarueput and simple random IG: 80; Using line as an alternative
Thailand 1 month mixed research
Ratanakul (2018) [33] sampling CG: 80 channel
cluster IG: 42; De Bono’s six thinking hats
Swamy et al. (2019) [34] Saudi Arabia not mentioned quantitative research
sampling CG: 38 activity
convenience IG: 19;
Truong (2022) [35] Vietnam 9 weeks Process-genre approach quantitative research
sampling CG: 19
purposive IG: 66; Digital multimodal
Xu (2021) [36] China 18 weeks quantitative research
sampling CG: 30 composing (DMC)
purposive IG: 35; Self-regulated learning-
Yang et al. (2022) [37] China over 15 weeks quantitative research
sampling CG: 35 based teacher feedback
Zaini and Mazdayasna IG: 20; Computer-assisted language
Iran 1 semester not mentioned quantitative research
(2014) [38] CG: 24 learning
Zaini and Mazdayasna IG: 24;
Iran 1 semester not mentioned Computer-based instruction quantitative research
(2015) [39] CG: 20
Zhang and Cheng (2021) convenience IG: 36; Comprehensive written
China 16 weeks mixed research
[40] sampling CG: 36 corrective feedback
Zhang and Zhang (2021) convenience IG: 24; Fostering stance-taking as a
China 8 weeks quantitative research
[41] sampling CG: 22 sustainable goal
IG: 32; Connectivism model in
convenience online course platform
Zhou (2017) [42] China 16 weeks quantitative research
sampling CG: 32 within blended learning
mode
IG=Intervention Group; CG=Control Group
3.4 Research outcomes
3.4.1 Teaching instruction intervention
Thirteen studies explored the influence of interventions related to teaching
instruction; these can be divided into the macro instruction intervention group (n=7)
and the micro instruction intervention group (n=6).
3.4.1.1 The macro instruction
The macro instruction intervention referred to the traditional or improved teaching
approach during the teaching process.
Truong (2022) [35] revealed that the process-genre approach helped students achieve
better writing performance, especially in the dimensions of “coherence and
cohesion” and “grammatical range and accuracy”; furthermore, it strengthened their
general writing self-efficacy from the perspective of conventions, writing ideation as
well as self-regulation, enhancing both awareness and behaviors of their writing
autonomy.
Fu et al. (2019) [14] used a contextual gaming approach on the basis of mind
mapping, which helped students generate diverse ideas and produce positive
thoughts and feelings. It also led to improved writing performance in fluency and
elaboration although this had limited effect. On the whole, it was considered to be
beneficial and interesting.
Highlighting the necessity of engaging students to share their interpretation and

application of concepts in class, the full concept-based instruction (CBI) intervention
was employed by Fu and Liu (2022) [15] to teach genre in their study. Their findings
revealed that the intervention generated significant improvements in learners’
grasping of genre, theoretical thinking and writing achievement; it also enhanced
learners’ conceptual development. With coherent and pedagogically adequate
explanations of genre, learners were prevented from focusing excessively on forms
and considered their language selection on the basis of different social contexts.
Khalil (2018) [22] taught using the FLOW strategy, which helped students make
considerable progress in using writing skills to modify their compositions, and other
skills acquired through creative thinking abilities, such as fluency and originality,
thereby strengthening learners’ ability to write essays.
For enhancing students' initiative in classroom activities and helping create an active
atmosphere, six thinking hats activity was employed and commended in the study of
Swamy et al. (2019) [34]. Moreover, the strategy assisted students to comprehend a
topic from different perspectives, employ creative thinking skills and draw logical
conclusions, improving their writing, speaking and cognitive skills.
When integrating digital multimodal composing(DMC) in the activities of EFL

teaching, Xu (2021) [36] found that students showed marvelous progress in the
following five aspects when learning L2 writing— “text length, mean length per T-
unit, task requirement, content, and comprehensibility”(p. 1). However, few
significant improvements were discovered in terms of coherence and cohesion.
Baghbadorani and Roohani (2014) [7] followed the model of self-regulated strategy
development (SRSD) and found it significantly effective to cultivate EFL learners’
persuasive writing ability, such as “the format and content, organization and
coherence, sentence construction and vocabulary in writing” (p. 235). In particular, it
helped them understand writing strategies, use metacognitive knowledge and self-
regulation skills to monitor, and obtain an optimistic sense of self-efficacy in writing.
3.4.1.2 The micro instruction

The micro instruction group mainly focused on small or specific aspects, such as the
strategy [11], conjunctions as cohesive devices [2], stance-taking [41] and others [4,
12, 28].
After conducting writing strategy instruction, De Silva (2015) [11] concluded that
students were trainable to efficiently use writing strategies, as their ability to use
these strategies as well as their writing performance (such as the overall organization
and cohesion) increased significantly after being taught the strategy.
Following input on the explicit instruction centered on the usage of conjunctions in

the study of Alawerdy and Alalwi (2022), particularly focusing on the guidelines for
cohesion [2], students improved significantly in understanding and applying
conjunctions that belonged to the cohesiveness concept, and achieved huge progress
in paragraph writing.
Emphasizing the skill of stance-taking as a sustainable goal, Zhang and Zhang (2021)
[41] stated that explicit stance instruction enhanced students’ comprehension and
selection of stance in their writing, boosting their academic writing performance.
Students reportedly exhibited progress in “challenges concerning stance-taking, such
as single-voiced, subjective, and underuse of expansive stance” (p. 16), but this was
limited in terms of the regulation of external voices.
Naghdipour and Koc (2015) [28] incorporated instruction on genre-based strategies

by stressing the combination of reading activities with writing classes, which
contained targeted guidance, sufficient opportunities for practice and periodic
feedback. Their study demonstrated significant development in learners' writing
abilities in the dimensions of fluency, accuracy and quality, reducing their reliance
on or unnecessary transfer from their L1.
Ebrahimi et al. (2018) [12] improved students’ writing through enhancing their
emotional intelligence, stating that writing skill is dependent upon the writer’s
emotions throughout the whole writing process. Writing about their emotions and
incorporating frequent use of emotional words via consciousness-raising regarding
their feelings helped students understand their own feelings and consequently
increased their EQ, significantly improving their writing skills.
Finally, Alharthi (2021) [4] explored free writing, which enabled EFL learners to
make connections between their ideas by choosing their own topics, while improving
students’ writing performance in grammatical aspects, such as subject-verb
agreement, vocabulary selection, spelling and punctuation. The strategy helped them
write effortlessly while focusing only on the writing conventions and improving
their writing skills.
3.4.2 Feedback-based intervention
Four studies contained peer feedback [19, 30] or teacher feedback [26, 38] during the
intervention while the other groups contained direct feedback [27], indirect feedback
[30] and comprehensive feedback [10, 40].
3.4.2.1 Peer feedback and teacher feedback

With regard to peer feedback, Nassar and Al Tameemy (2021) [29] concluded in their
study that when focusing on the writing process, written peer feedback not only
improved students' writing skills in terms of unity and coherence, but also enhanced
students' critical analysis skills. Hanh and Tinh (2022) [17] found that a peer-review
checklist helped students make excellent progress in accomplishing tasks and using
vocabulary, reducing spelling mistakes. Besides, it offered students opportunities to
get feedback and recommendations about word use, which was well-received by
students.
Furthermore, Mazloomi and Khabiri (2018) [26] combined teacher feedback with self-
assessment (SA). With proper feedback and instructions from the teachers, SA
greatly assisted students to improve their writing ability and language proficiency,
helping them optimize learning process and establish goals for future study.
Similarly, Yang et al. (2022) [37] turned to self-regulated learning-based (SRL-based)
teacher feedback, which was found to positively promote EFL students’ use of SRL
writing strategies “with goal-oriented monitoring, knowledge rehearsal, feedback
handling, interest enhancement, text processing, idea planning, motivational self-
talk, and emotional control” (p. 1). Also, it seemed to play a more effective role in the
content, language and vocabulary than in language use.
3.4.2.2 Direct feedback, indirect feedback and comprehensive feedback

Direct written feedback is preferred by both teachers and students (especially low
proficiency learners), as it enables students to recognize the errors in their
compositions and master the accurate target language structures, improving their
writing proficiency (Mekala & Ponmani, 2017) [27]. On the other hand, indirect
written feedback is not explicit enough, and only helps learners notice their errors
rather than understanding them. Furthermore, it improves their accuracy only in the
short term and is insufficient to develop their implicit knowledge, especially for low
proficiency learners. Nusrat et al. (2019) [30] added that working in pairs, teams or
groups to manage indirect written feedback would ultimately teach students to be
independent and self-reliant.
Two of the selected studies [10, 40] concurred that comprehensive feedback helps
students to perform better in writing accuracy and fluency, with a statistically
significant correlation over time, but does not promote their sentence complexity.
Additionally, Cheng and Zhang (2021) [10] reported that WCF also has a limited
effect on improving content and organization in students’ compositions.
3.4.3 Cooperative learning intervention

Four studies included methods of cooperative learning, exploring the function of the
paired writing technique [1], collaborative writing framework [3, 31] and structured
small-group student discussion [23] for improving students’ writing competence.
According to Abbas and Al-bakri (2018) [1], the paired writing technique could
effectively improve the quality and quantity of EFL students' compositions, as well
as speaking and communication, and lessened their writing anxiety to some extent,
as it provided students with additional opportunities to share and learn from peer
review and feedback.
Conducting research on collaborative writing, Li and Zhang (2021) [23] found that
the effect of cooperative prewriting discussions was statistically crucial in promoting
students’ progress in EFL writing in terms of “the content, organization, vocabulary
and language use” (p. 1), but not in the mechanics. Similarly, Pham (2021) [31]
proposed that collaborative writing facilitated students’ writing fluency by
motivating them to write more words collaboratively compared to individual
writing. Moreover, students expressed positive attitudes towards the approach and
asserted that they could learn more good ideas and writing styles by contributing to
the shared essays, while also enjoying a more motivating classroom environment.
In addition, Aldossary (2021) [3] focused on the function of structured small-group

student discussion, reporting that small groups could have a more considerable
knowledge base and greater language resources to learn, which provided students
with beneficial and enjoyable activities that were conducive to their writing
improvement.
3.4.4 Modern technology-based intervention

Modern technology has been used in various aspects of society and plays an essential
role in everyday life. Among the selected studies, seventeen of them targeted modern
technology. In particular, two studies [5, 33] focused on the use of apps on a mobile
phone as a channel of communication, two studies [16, 32] focused on assessment,
three studies [9, 24, 13] used a flipped classroom, four studies [6, 8, 20, 21] applied an
electronic portfolio, and six studies [18, 25,19, 38, 39, 42] were based on the assistance
of computers or computer programs in writing.
3.4.4.1 Mobile message intervention

Andujar (2016) [5] studied mobile instant messaging and emphasized syntactic
complexity and lexical diversity in the teaching process, concluding that it improved
students’ accuracy in writing in a second language in terms of grammar, lexicon and
mechanics. Additionally, language-related episodes (LREs) in the application
provided feedback and helped in negotiating meaning, constituting a fundamental
source of knowledge for second language development.
Instead of mobile instant messaging, Suthiwartnarueput and Ratanakul (2018) [33]

used Line as an alternative channel in teaching. With Line, students were able to
share knowledge and feedback to use appropriate words and correct sentence
structure and mechanics to create well-organized paragraphs. They reported that this
resulted in a significant improvement in students’ proficiency in learning vowels,
phonemes, and syllables, which also increased their learning motivation and
confidence.
3.4.4.2 Electronic portfolio intervention

According to Karami et al. (2018) [20], applying electronic portfolio in teaching
activities left a magnificent effect on promoting learners’ writing proficiency, with no
difference between genders. Again, Karami et al. (2019) [21] verified the magnificent
effect of the e-portfolio on students’ writing proficiency and described the self-
regulated strategies they used in teaching. Barrot (2020) [8] used Facebook as the
basis of an electronic portfolio and described its positive effects on L2 writers’
writing performance “in terms of coherence and cohesion, lexical resource,
grammatical range and accuracy” (p. 11). As a result of its interactive characteristic,
students were exposed to social pressure when using the Facebook-based e-portfolio,
which also strengthened “students’ awareness of the purpose, target audience, tone
and level of formality, persona as a writer, and writing process” (p. 13), improving
students’ motivation and interest in writing.
3.4.4.3 Computer-based writing intervention

Ismail et al. (2012) [18] utilized the IQ-Write Program Online Composing Guide (IQ-
WCG), combining the online learning tools and resources into language teaching
instruction, which boosted the students’ interest in writing and made them
inquisitive as well as critical in the writing process, improving their overall writing
performance.
Luo (2016) [25] adopted data-driven learning activities with assistance of BNCweb,
helping students to perform significantly better in writing fluency and accuracy but
not in writing complexity, toward which learners showed a positive attitude.
Kang (2019) [19] explored collocation learning through a Web-Concordancer, with

which students were exposed to diverse contexts and demonstrated full
comprehension of these contexts, which promoted self-regulated learning.
Zaini and Mazdayasna (2014) [38] explored computer-assisted language learning

(CALL) instruction, which helped students improve their writing competence with
appropriate language use, producing paragraphs of higher quality. Later in their
study in 2015, besides reconfirming the function of the CALL instruction in
improving students’ writing skills, they revealed that the automatic feedback
through Microsoft Word, along with the instructor’s evaluations and
recommendations on students’ writing performance, activated their common sense,
language abilities, and logic to correct their writing errors, helping improve their
writing competence.
Azari (2017) [6] applied a weblog to the process approach and demonstrated its
positive effect on the content and the organization, although it was less successful in
improving language use, vocabulary, and the mechanics of writing. It also helped
students more easily find their learning paths but did not transform them into fully
autonomous learners. More research was recommended to confirm the relation
between students’ autonomous learning and writing performance development.
Zhou (2017) [42] studied the use of the connectivism model on an online course
platform within blended learning mode, which helped to promote a self-learning
environment and constant interaction, significantly improving students’ English
writing competence “in terms of content relevance, content sufficiency, organization
structure and language expression” (p. 1060) as well as strengthening students'
critical thinking.
3.4.4.4 Flipped classroom intervention

Lin et al. (2018) [24] employed the flipped contextual game-based learning approach,
which enhanced students’ EFL writing achievement in terms of learner performance,
autonomy, and community, helping students reduce writing errors. Similarly, Chatta
and Haque (2020) [9] took flipped classroom instruction (FCI) through Blackboard
(an E-Learning portal) and found that FCI enabled student-centered classes, in which
learners cultivated an interest in language acquisition, becoming active, enthusiastic,
motivated and autonomous, and writing better paragraphs. Moreover, Fathi and
Rahimi (2022) [13] revealed that the flipped classroom approach significantly assisted
EFL students to improve their overall writing achievement and writing fluency, yet
its efficiency on language complexity and accuracy in EFL writing was not
statistically significant.
3.4.4.5 Assessment intervention

As a flexible and interactive platform, Moodle-Based Assessment was used by
Ghouali and Cecilia (2021) [16] to boost students’ motivation to learn. The provision
of feedback and exposure to extra online materials through Moodle improved
students’ writing abilities in terms of coherence, cohesion, punctuation, spelling,
capitalization, and grammar, enabling them to assess themselves. By frequently
reflecting on their mistakes, students gradually took more control over their learning.
Similarly, Sun and Fan (2022) [32] chose AWE-aided assessment and found that it
could improve students’ writing ability in terms of organization and linguistic
accuracy and organization in the long term. Despite the decrease of students’
avoidance behaviour due to the method, there was little mediation effect of writing
anxiety discovered between the AWE-aided approach and writing achievement.
3.5 Summary and Recommendations

According to the analysis of the research outcomes, all four types of interventions
improved university students’ EFL writing competence to various extents, from
language use to learning motivation and confidence, though limited effects were also
observed with certain tools or instructing mediums in researches. When choosing
approaches to instruct EFL university students develop their writing competence,
educators should take into account the particular situation of their students,
choosing suitable approaches and applying them flexibly. On the other hand,
combining the approaches may also make them more effective. Furthermore, it
would be useful for researchers to conduct more studies to identify more effective
interventions or to further explore how to use or optimize the current interventions
to increase their positive impact on students. It is expected that researchers and
educators, especially university instructors, can benefit from this review, which not
only offers guidance and insight to help university teachers take effective measures
to improve their students’ writing competence but also recommends areas for further
related and in-depth research to be conducted in the future.
4. Conclusion
Traditional approaches to writing instruction were identified as not meeting the
needs of learners and teachers (Grabe & Kaplan, 1996), so researchers have been
exploring various ways to help improve learners’ writing competence. This
systematic review identified research articles from the last decade that have studied
interventions aimed at improving university students’ EFL writing competence and
found that the traditional teaching methods are no longer simply followed. Based on
the review, four different types of interventions were classified, most of which have
been reported as having a positive effect on students’ writing in terms of content and
format, organization and coherence, vocabulary and sentence construction.
Furthermore, such interventions have strengthened students’ autonomous learning,
cooperative ability and critical thinking, though some have shown limited effects on
certain aspects. Knowledge of the interventions applied in a range of studies over the
last decade, and particularly their impact, could provide valuable insights to
educators, especially higher education English teachers, enabling them to select
appropriate and effective measures to help their students to improve their writing
competence. Furthermore, it is hoped that this review will provide inspiration to
educators or researchers to conduct further related and in-depth experiments to
explore more effective measures to help students improve their EFL writing.
5. References
Abbas, S. H., & Al-bakri, S. A. (2018). The effect of pair writing technique on Iraqi EFL
university students' writing performance and anxiety. Arab World English Journal,
9(2), 23-37. http://doi.org/10.24093/awej/vol9no2.2
Akbar, M., Pathan, H., & Ali Shah, S. W. (2018). Problems affecting L2 learners’ English
writing skills: A study of public sector colleges Hyderabad city, Sindh, Pakistan.
Language in India, 18(5), 7–26.
Alawerdy, A. S., & Alalwi, F. S. (2022). Enhancing English as a foreign language university
students' writing through explicit instruction of conjunctions as cohesive devices: An
experimental study. Frontiers in Psychology, 13.
http://doi.org/10.3389/fpsyg.2022.1053310
Aldossary, K. S. (2021). The impact of collaborative writing on EFL learners' writing
development: A longitudinal classroom-based study in Saudi Arabia. Arab World
English Journal, 12(3), 174-185. http://doi.org/10.24093/awej/vol12no3.12
Alharthi, S. (2021). From instructed writing to free-writing: A study of EFL learners. SAGE
Open, 11(1). doi:10.1177/21582440211007112
Andujar, A. (2016). Benefits of mobile instant messaging to develop ESL writing. System, 62,
63-76. http://doi.org/10.1016/j.system.2016.07.004
Astrid, A., Marzulina, L., Erlina, D., Harto, K., Habibi, A., & Mukminin, A. (2019). Teaching
writing to EFL student teachers: Teachers' intervention and no teachers' intervention
through peer feedback writing techniques. Universal Journal of Educational Research,
7(9), 1901-1908. http://doi.org/10.13189/ujer.2019.070908
Azari, M. H. (2017). Effect of weblog-based process approach on EFL learners’ writing
performance and autonomy. Computer Assisted Language Learning, 30(6), 529-551.
http://doi.org/10.1080/09588221.2017.1329214
Baghbadorani, E. A., & Roohani, A. (2014). The impact of strategy-based instruction on L2
learners’ persuasive writing. Procedia - Social and Behavioral Sciences, 98, 235-241.
http://doi.org/10.1016/j.sbspro.2014.03.412
Barrot, J. S. (2020). Effects of Facebook-based e-portfolio on ESL learners' writing
performance. Language, Culture & Curriculum, 34(1), 95-111.
http://doi.org/10.1080/07908318.2020.1745822
Chatta, B. S., & Haque, M. I. (2020). Improving paragraph writing skills of Saudi EFL
university students using flipped classroom instruction. Arab World English Journal
(AWEJ) Special Issue on CALL (6). 228- 247. http://doi.org/10.24093/awej/call6.15
Cheng, X., & Zhang, L. J. (2021). Sustaining university English as a foreign language learners'
writing performance through provision of comprehensive written corrective
feedback. Sustainability, 13(15). http://doi.org/10.3390/su13158192
De Silva, R. (2015). Writing strategy instruction: Its impact on writing in a second language
for academic purposes. LANGUAGE TEACHING RESEARCH, 19(3), 301-323.
http://doi.org/10.1177/1362168814541738
Ebrahimi, M. R., Khoshsima, H., & Zare-Behtash, E. (2018). The impacts of emotional
intelligence enhancement on Iranian intermediate EFL learners writing skill.
International Journal of Instruction, 11(1), 437-452.
http://doi.org/10.12973/iji.2018.11130a
Fathi, J., & Rahimi, M. (2022). Examining the impact of flipped classroom on writing
complexity, accuracy, and fluency: A case of EFL students. Computer Assisted
Language Learning, 35(7), 1668-1706. http://doi.org/10.1080/09588221.2020.1825097
Fu, Q.-K., Lin, C.-J., Hwang, G.-J., & Zhang, L. (2019). Impacts of a mind mapping-based
contextual gaming approach on EFL students’ writing performance, learning
perceptions and generative uses in an English course. Computers and Education, 137,
59-77. http://doi.org/10.1016/j.compedu.2019.04.005
Fu, Z., & Liu, Y. (2022). Promoting L2 writing development via a concept-based approach to
teaching genre: A sociocultural intervention study in Chinese EFL writing
classrooms. Language Teaching Research. http://doi.org/10.1177/13621688221114363
Gibbons, P. (2002). Scaffolding language, scaffolding learning. Portsmouth, N. H: Heinemann.
Gilbert, J., & Graham, S. (2010). Teaching writing to elementary students in Grades 4–6: A
national survey. Chicago Journals, 110(4), 494–518. http://doi.org/10.1086/651193
Ghouali, K., & Cecilia, R. R. (2021). Towards a Moodle-based assessment of Algerian EFL
students' writing performance. Porta Linguarum, (36), 231-248.
http://doi.org/10.30827/portalin.v0i36.17866
Grabe, W., & Kaplan, R.B. (1996). Theory and practice of writing. New York, NY: Longman.
Hashemnezhad, H. & Hashemnezhad, N. (2012). A Comparative Study of Product, Process,
and Post-process Approaches in Iranian EFL Students' Writing Skill. Journal of
Language Teaching and Research, 3(4), 722-729. http://doi.org/10.4304/jltr.3.4.722-72
Hanh, L. T. T., & Tinh, B. T. (2022). Applying peer-review checklist to improve Vietnamese
EFL university students’ writing skills. International Journal of Learning, Teaching and
Educational Research, 21(5), 166-181. http://doi.org/10.26803/ijlter.21.5.9
Hyland, K. (2003). Genre-based pedagogies: A social response to process. Journal of Second
Language Writing, 12 (1), 17-29. http://doi.org/10.1016/S1060-3743(02)00124-8
Ismail, N., Hussin, S., & Darus, S. (2012). The Effects of IQ-write program online composing
guide towards ESL tertiary students' writing performance and interest. International
Journal of Technology, Knowledge & Society, 8(4), 107-117.
Jacobs, H. L., Zingraf, S. A., Wormuth, D. R., & Hartfiel, V. F. (1981). Testing ESL
composition: A practical approach. Rowley: Newbury House.
Johns, A. M. (Ed.). (2002). Genre in the classroom: Multiple perspectives. Mahwah: Lawrence
Erlbaum
Kang, N. (2019). Effectiveness of collocation learning through a web-concordancer in a
Korean EFL university course. Multimedia-Assisted Language Learning, 22(1), 41-84.
http://doi.org/10.15702/mall.2019.22.1.41
Karami, S., Sadighi, F., Bagheri, M. S., & Riasati, M. J. (2018). The potential impact of the
application of electronic portfolio on Iranian EFL learners' writing performance
seeking their gender role. Cogent Social Sciences, 4(1), 1-1.
http://doi.org/10.1080/23311886.2018.1562509
Karami, S., Sadighi, F., Bagheri, M. S., & Riasati, M. J. (2019). The impact of application of
electronic portfolio on undergraduate English majors' writing proficiency and their
self-regulated learning. International Journal of Instruction, 12(1), 1319-1334.
http://doi.org/10.29333/iji.2019.12184a
Khalil, E. R. (2018). Investigating the effect of FLOW teaching strategy on EFL learners’
writing achievements. Annals of the Faculty of Arts, 46(Part A), 73–133. https://search-
ebscohost-
com.libezp2.utar.edu.my/login.aspx?direct=true&db=asn&AN=137964172&site=eho
st-live&scope=site.
Klimova, B. F. (2013). Approaches to the teaching of writing skills. Procedia - Social and
Behavioral Sciences, 112, 147-151. doi: 10.1016/j.sbspro.2014.01.1149
Hupe, M. (2019). EndNote X9. Journal of Electronic Resources in Medical Libraries, 16(3–4), 117–
119. https://doi.org/10.1080/15424065.2019.1691963
Li, H. H., & Zhang, L. J. (2021). Effects of structured small-group student talk as collaborative
prewriting discussions on Chinese university EFL students' individual writing: A
quasi-experimental study. PLoS ONE, 16(5): e0251569.
http://doi.org/10.1371/journal.pone.0251569
Lin, C.-J., Huang, G.-J., Fu, Q.-K., & Chen, J.-F. (2018). A flipped contextual game-based
learning approach to enhancing EFL students' English business writing performance
and reflective behaviors. Journal of Educational Technology & Society, 21(3), 117-131.
Luo, Q. (2016). The effects of data-driven learning activities on EFL learners' writing
development. SpringerPlus, 5(1), 1-13. http://doi.org/10.1186/s40064-016-2935-5
Makmur., Mukminin, A., Ismiyati, Y., & Verawati. (2016). In search of good student teachers
in writing skill: The impact of different task variance of EFL writing proficiency.
International journal of academic research in education, 2(1).
http://doi.org/10.17985/ijare.45901
Mazloomi, S., & Khabiri, M. (2018). The impact of self-assessment on language learners’
writing skill. Innovations in Education & Teaching International, 55(1), 91-100.
http://doi.org/10.1080/14703297.2016.1214078
Mekala, S., & Ponmani, M. (2017). The impact of direct written corrective feedback on low
proficiency ESL learners' writing ability. IUP Journal of Soft Skills, 11(4), 23-54.
https://search-ebscohost-
com.libezp2.utar.edu.my/login.aspx?direct=true&db=bth&AN=127283084&site=eho
st-live&scope=site.
Moher, D., Liberati, A., Tetzlaf, J., Altman, D. G., & The PRISMA Group. (2009). Preferred
reporting items for systematic reviews and meta-analyses: The PRISMA statement.
Annals of Internal Medicine, 151(4), 264-269. http://doi.org/10.7326/0003-4819-
151-4-200908180-00135
Mukminin, A., Ali, R. M., & Ashari, M. J. F. (2015). Voices from within: Student teachers’
experiences in English academic writing socialization at one Indonesian teacher
training program. The Qualitative Report, 20(9), 1394-1407.
http://nsuworks.nova.edu/tqr/vol20/iss9/2
Naghdipour, B., & Koc, S. (2015). The evaluation of a teaching intervention in Iranian EFL
writing. Asia-Pacific Education Researcher, 24(2), 389-398. doi:10.1007/s40299-014-0191-
4
Nassar, H. M., & Al Tameemy, F. A. (2021). The impact of written peer feedback on the
writing skills of EFL university students in Yemen. Asian ESP Journal, 17(2), 290-311.
https://www.scopus.com/record/display.uri?eid=2-s2.0-
85101933327&origin=resultslist&sort=plf-
f&src=s&st1=The+Impact+of+Written+Peer+Feedback+on+the+Writing+Skills+of+E
FL+University+Students+in+Yemen&sid=ea3e92d9c12e96d61e5b6931d3362921&sot=
b&sdt=b&sl=108&s=TITLE-ABS-
KEY%28The+Impact+of+Written+Peer+Feedback+on+the+Writing+Skills+of+EFL+
University+Students+in+Yemen%29&relpos=0&citeCnt=0&searchTerm=
Nordin, S., & Mohammad, N. (2006). The best of two approaches: Process/genre-based
approach to teaching writing. Universiti Teknologi, Petronas
Nusrat, A., Ashraf, F., Khan, S., Aziz, S., & Jabeen, R. (2019). Is indirect written feedback
valuable? A study targeting ESL university students in Pakistan. International Journal
of English Linguistics, 9(5), 340-350. http://doi.org/10.5539/ijel.v9n5p340
Paltridge, B. (2004). Approaches to teaching second language writing. University of Sydney,
Adelaide.
Pham, V. P. H. (2021). The effects of collaborative writing on students' writing fluency: An
efficient framework for collaborative writing. SAGE Open, 11(1), 1-11.
http://doi.org/10.1177/2158244021998363
Selvaraj, M., & Aziz, A. A. (2019). Systematic review: Approaches in teaching writing skill in
ESL classrooms. International Journal of Academic Research in Progressive Education and
Development, 8(4), 450-473. doi:10.6007/IJARPED/v8-i4/6564
Silva, T. (1997). On the ethical treatment of ESL writers. TESOL Quarterly, 31(2), 359-363.
http://doi.org/10.2307/3588052
Sun, B., & Fan, T. (2022). The effects of an AWE-aided assessment approach on business
English writing performance and writing anxiety: A contextual consideration. Studies
in Educational Evaluation, 72, 101123. http://doi.org/10.1016/j.stueduc.2021.101123
Suthiwartnarueput, T., & Ratanakul, S. (2018). Using Line as an alternative channel for
improving English writing ability of Thai EFL students. Journal of Institutional
Research South East Asia, 16(1), 46-69.
Swamy, B. C., Haque, M. I., Koppada, V., & Kumar, N. S. (2019). The effect of conducting De
Bono's six thinking hats activity on developing paragraph writing skills of university
students in the Kingdom of Saudi Arabia. International Journal of English Linguistics,
9(6), 186-197. http://doi.org/10.5539/ijel.v9n6p186
Tang X. (2012). Discussion on the improvement of college English writing classroom teaching.
Journal of Changchun Normal University, 31(11), 170-171.
Torto, G. A. (2014). An examination of the approaches adopted by English language teachers
in teaching composition writing in cape coast basic schools, Ghana. Language in
India, 14(1), 117-138. http://languageinindia.com/jan2014/examinationghana.pdf
Truong, H. M. (2022). Impacts of process-genre approach on EFL sophomores' writing
performance, writing self-efficacy, writing autonomy. Journal of Language and
Education, 8(1), 181-195. http://doi.org/10.17323/jle.2022.12165
Wang, Y. C. (2015). Promoting collaborative writing through wikis: A new approach for
advancing innovative and active learning in an ESP context. Computer Assisted
Language Learning, 28(6), 499–512. http://doi.org/10.1080/09588221. 2014.881386
Xu, Y. (2021). Investigating the effects of digital multimodal composing on Chinese EFL
learners’ writing performance: A quasi-experimental study. Computer Assisted
Language Learning. http://doi.org/10.1080/09588221.2021.1945635
Yang, L. F., Liu, Y., & Xu, Z. (2022). Examining the effects of self-regulated learning-based
teacher feedback on English-as-a-foreign-language learners' self-regulated writing
strategies and writing performance. Frontiers in Psychology, 13, 1027266.
http://doi.org/10.3389/fpsyg.2022.1027266
Zaghlool, Z. D. (2020). The impact of using CALL online writing activities on EFL university
students’ writing achievement. Theory and Practice in Language Studies, 10(2), 141-148.
doi:10.17507/tpls.1002.01
Zaini, A., & Mazdayasna, G. (2014). The effect of computer assisted language learning on the
development of EFL learners’ writing skills. Procedia - Social and Behavioral Sciences,
98, 1975-1982. http://doi.org/10.1016/j.sbspro.2014.03.631
Zaini, A., & Mazdayasna, G. (2015). The impact of computer-based instruction on the
development of EFL learners' writing skills. Journal of Computer Assisted Learning,
31(6), 516-528. doi:10.1111/jcal.12100
Zhang, L. J., & Cheng, X. (2021). Examining the effects of comprehensive written corrective
feedback on L2 EAP students’ linguistic performance: A mixed-methods study.
Journal of English for Academic Purposes, 54, 101043.
http://doi.org/10.1016/j.jeap.2021.101043
Zhang, L., & Zhang, L. J. (2021). Fostering stance-taking as a sustainable goal in developing
EFL students' academic writing skills: Exploring the effects of explicit instruction on
academic writing skills and stance deployment. Sustainability, 13(8).
http://doi.org/10.3390/su13084270
Zhou, C. (2018). Empirical study on the effectiveness of teaching model of college English
writing within blended learning mode. Educational Sciences: Theory & Practice, 18(5),
1060-1076. http://doi.org/10.12738/estp.2018.5.009
113

Vol. 22, No. 10, pp. 113-129, October 2023
Nature of Science: A Comparative Analysis of

the High School Physics Textbooks in Indonesia
and Korea
Hartono Bancong* , Sukmawati , Nursalam
Universitas Muhammadiyah Makassar
Makassar, Indonesia
Danilo Jr. Tadeo

Seoul National University
Seoul, South Korea
Abstract. Over the past two decades, the goal of supporting students
and teachers in developing views on the Nature of Science (NoS) has been
increasingly central to the vision and discourse goals for global physics
education reform. Understanding the Nos is a critical and essential
component of scientific literacy. The main objective of this study is to
present a comprehensive picture of the NoS in physics textbooks in
Indonesia and South Korea. This research is a descriptive study, and the
data source consisted of 10 high school physics textbooks (five textbooks
from each country). The textbooks were chosen based on the results of a
Google Forms survey about the most common use of physics textbooks
in schools. The results show that the total number of NoS elements
presented in Indonesian physics textbooks is 71, of which 47 are on the
cognitive-epistemic aspect, and 24 relate to the social-institutional aspect.
In contrast, the number of NoS items presented in Korean physics
textbooks is 84, with 54 on the cognitive-epistemic aspect and 30 on the
social-institutional aspect. This study also revealed that 59% of the NoS
in Indonesian physics textbooks were located in the main text, 35% were
found in secondary texts, while 6% were presented in both. Similarly, 48%
of NoS items in Korean physics textbooks were located in the main text,
44% in secondary texts, and 8% were presented together. Therefore, this
study concludes that Korean high school physics textbooks contain more
NoS than Indonesian high school physics textbooks.
Keywords: Nature of science; Indonesian physics textbooks; Korean

physics textbooks
*
Corresponding author: Hartono Bancong, hartono.b.b@unismuh.ac.id
©Authors
114

This study attempts to present a comprehensive picture of the NoS representation
in Indonesian and South Korean physics textbooks. The NoS is an essential
element of scientific literacy that enriches students' comprehension of science
concepts. The promotion of scientific literacy is the primary goal of physics
education in almost all schools globally today (Abd-El-Khalick et al., 2017;
Australian Curriculum Assessment and Reporting Authority [ACARA], 2016;
McDonald & Abd-El-Khalick, 2018; Next Generation Science Standards [NGSS
Lead States], 2013) and one of the foundations for achieving this goal is the growth
of students' awareness of NoS (Abd-El-Khalick et al., 2017; Ayık & Coştu, 2020;
Bugingo et al., 2022; Liang et al., 2009). Therefore, over the past two decades, the
goal of assisting students and teachers in developing perspectives on the NoS has
increasingly become the core of visions and document discourse on global science
education reform (Abd-El-Khalick et al., 2017; Brunner & Abd-El-Khalick, 2020;
McDonald & Abd-El-Khalick, 2018; Park et al., 2019; So & Kang, 2014).
Understanding the NoS is now regarded as a significant outcome of school-level
physics education and a critical component of scientific literacy (Cofré et al., 2014;
McComas & Clough, 2020). It is necessary for teachers to elaborate on parts of the
NoS in physics education since it can assist students in understanding their
environment (Bancong & Song, 2020; Karampelas, 2018; Upahi et al., 2020; Zhang
et al., 2022). Furthermore, an understanding of the scientific enterprise, including
how scientific knowledge is created, as well as its strengths and limitations, will
be aided by having an informed view of NoS (Abd-El-Khalick et al., 2008; Ayık &
Coştu, 2020).
South Korea is known universally for offering high-quality education. According

to the Program for International Student Assessment (PISA), around 86% of
Korean students attained level 2 or better in science in 2018 (Organisation for
Economic Co-operation and Development [OECD], 2019). This means that 86% of
Korean students can recognize and identify valid explanations for common
scientific events, in basic cases, for example, to determine whether a conclusion is
valid based on the data provided. Additionally, 12% of Korean students
are excellent in science, indicating that they can creatively and independently use
their understanding and application of science to a range of situations, including
those that are unfamiliar (OECD, 2019).
In contrast, in Indonesia, only about 40% of students reach Level 2 or higher in

science, with the OECD average of 78% (OECD, 2019). This means that in
Indonesia, only around 40% of students have the ability to recognize the right
explanations for common scientific phenomena and can use that knowledge to
identify them, in simple cases, for example, to determine whether a conclusion is
valid based on the data provided. In addition, although 7% of OECD students are
considered to be at Level 5 or 6 proficiency in science (OECD, 2019), only 1% of
Indonesian students achieve this level.
Ardwiyanti et al. (2021) and Dhamayanti et al. (2019) have examined NoS in
Indonesian high school physics textbooks. Both of them claimed that NoS is still
not properly and comprehensively stressed in high school physics textbooks.
115
Only theories and laws indicators receive more attention than other indicators.
Furthermore, there has been no cross-country comparative research on this topic
in Indonesia. According to Liang et al. (2009), teachers in China, America, and
Turkey have different understandings of NoS owing to cultural differences.
Similarly, Zhang et al. (2022) reported that pre-service teachers in Canada have a
higher comprehension of NoS than pre-service teachers in China. Therefore, in
order to learn more about NoS as well as enhancing the related literature, this
study was designed to compare NoS in physics textbooks between Indonesia and
Korea.
This study enhances the existing body of worldwide literature by broadening the
understanding of the NoS throughout diverse educational contexts. The outcomes
of this investigation will contribute to the advancement of research on the NoS as
a subject of inquiry within the Indonesian context. Moreover, these findings will
have practical implications for physics educators, specifically in Indonesia and
Korea, as well as in countries with similar educational contexts, on how to prepare
textbooks rich in NoS aspects.
Therefore, this study aims to investigate the comparison of NoS aspects in physics
textbooks in Indonesia and Korea. The research questions in this study are as
follows:
1) Which aspects of NoS are addressed in high school physics textbooks in
Indonesia and Korea?
2) How is NoS presented in Indonesian and Korean high school physics
textbooks?
2. Literature reviews
2.1. Nature of science
NoS is not a description of how science works but rather how a scientific
enterprise works (Abd-El-Khalick et al., 2017; Upahi et al., 2020). Similar to
scientists, persons interested in the NoS seek an understanding of how scientists
work and engage with one another, how science answers questions, and how
science generates knowledge about nature. What science is, how it functions
(including epistemological and ontological concerns), how science influences and
is influenced by society, and how scientists conduct their professional and
personal lives are all topics that are explored in the NoS (Ayık & Coştu, 2020; Kaya
& Erduran, 2016; McComas & Clough, 2020).
According to Brunner and Abd-El-Khalick (2020), there are three aspects of NoS,
namely empirical, inferential, and creative. Scientific claims that rely on evidence
gathered through the senses or the extension of the senses are referred to be
empirical NoS. This is connected to the distinction between observation and
inference. Observations are descriptions of natural happenings that are generally
simple to achieve. In contrast, an inferential conclusion is one that is consistent
with observations that cannot be witnessed directly. The term "creative NoS"
refers to the requirement for scientists to employ their imagination and creativity
in order to generate ideas, explanations, and hypotheses.
116
The framework of NoS, as created by Kaya and Erduran (2016), offers a holistic
depiction of the scientific enterprise, encompassing its goals and values to
methods and scientific knowledge and also considering its social impacts. This
framework conceptualizes science as a cognitive-epistemic system that includes
inquiry processes, goals and values, methods and methodological rules, and
scientific knowledge. Additionally, it recognizes science as a socio-institutional
system comprising professional activities, scientific ethos, social certification and
dissemination of scientific knowledge, and social values. This form of
representation has the potential to aid teachers in gaining an understanding of the
elements that are absent in the teaching of NoS, as well as in making decisions
about which aspects should be prioritized and why this should be so in teaching
science in schools. Furthermore, the incorporation of socio-institutional elements
within the framework of NoS holds the promise of engaging a wider spectrum of
students, including those who exhibit aversion towards the predominantly
cognitive features emphasized in the classroom. Figure 1 shows the family
resemblance approach (FRA) of the NoS:
Figure 1: FRA framework of NoS
2.2. Previous studies on NoS

A number of studies have been reported that specifically examine the
representation of NoS in physics textbooks in schools. This analysis aims to
examine and evaluate whether the textbooks are in accordance with the
curriculum and learning objectives of physics globally. Abd-El-Khalick et al.
(2017) analyzed NoS in 18 physics textbooks commonly used in the United States
(US) over the past decade. The results showed that the representation of the NoS
in every physics textbook in the US does not differ based on aspects of the NoS. It
has simply not increased substantially over the past few years. This trend is not
comparable to the curriculum in the US, which places more emphasis on high
school students’ developing the concept of NoS.
117
McDonald and Abd-El-Khalick (2018) also examined four scientific textbooks

often used by physics teachers in Australia. According to the findings of their
investigation, there were four textbooks that exclusively and implicitly offered
NoS in various organizational forms and formats. Furthermore, Park et al. (2019)
analyzed aspects of the NoS in five physics textbooks based on the 2015
curriculum in South Korea. The results of their study revealed that the indicators
of NoS, such as scientific knowledge, scientific practice, scientific methods, and
professional activities of scientists, have been well described in the five physics
textbooks. The feature of science as a socio-institutional system, on the other hand,
was not adequately depicted in the five textbooks.
Several comparative studies have also been conducted on aspects of the NoS
across countries. Liang et al. (2009) investigated the perspectives of pre-service
teachers in the US, China, and Turkey on the six elements of NoS: observations
and inferences, tentativeness, scientific theories and laws, social and cultural
embeddedness, creativity and imagination, and scientific methods. A total of 640
pre-service teachers from the US, 212 from China, and 219 from Turkey took part
in their research survey. The results show that Chinese pre-service teachers
received the highest scores of the three countries, while Turkish pre-service
teachers received the lowest scores.
Park et al. (2014) also compared students’ understanding of the NoS in Canada
and Korea. The findings show that country differences have an effect on
subjectivity variance, empirical testability, and methods but are not significant for
tentative concepts. Students’ perspectives on aspects of the NoS are influenced by
the national situation and curriculum content. According to Zhang et al. (2022),
the differences in understanding of NoS between students in Canada and China
may be related to relevant teacher education policies, curricula, and pedagogy.
Recently, Bugingo et al. (2022) reported on an analysis of the representation of
NoS aspects in science curricula, particularly in the physics syllabus in four East
African countriies, namely Burundi, Rwanda, Tanzania, and Uganda. Their
findings revealed that aspects of the NoS were not explicitly represented in the
four physics curricula studied in these four East African countries. They also claim
that in the four physics syllabuses reviewed, much attention has been paid to
competencies without overt links to the works of scientists.
Ardwiyanti et al. (2021) and Dhamayanti et al. (2019) have examined NoS in high
school physics textbooks in Indonesia. However, these two studies only looked at
class X high school physics textbooks. Furthermore, Dhamayanti et al. (2019)
limited their study to one chapter in two types of high school physics textbooks
for class X. Then, another study only looked at three chapters for three different
types of high school physics textbooks for class X (Ardwiyanti et al., 2021). The
findings of this study provided an overview of the NoS in Indonesian high school
physics textbooks. However, there have been no cross-country comparative
studies on this topic despite the fact that different cultures will provide different
understandings of NoS (Liang et al., 2009; Zhang et al., 2022). Thus, to determine
the level and advance the relevant literature on NoS, this study was designed
118
comparatively to investigate NoS in physics textbooks between Indonesia and

Korea.
3. Methods
This study was a descriptive in nature and aimed to describe the various aspects
of the NoS as a whole as well as the actual situation in high school physics
textbooks in Indonesia and South Korea. Ten physics textbooks were analyzed in
this study, namely five physics textbooks for each country. The textbooks were
chosen based on the results of a Google Forms survey on the most widely used
physics textbooks in schools. Table 1 shows the profile of the physics textbooks
used as the source of research data, both commonly used by students and teachers
in Indonesia and South Korea:
Table 1: Profile of Indonesian and Korean physics textbooks

N Indonesian physics textbooks Korean physics textbooks
o.
Authors Publisher Publica Authors Publisher Public
tion ation
year year
1 Marthen Erlangga 2018 N. Kang, H. Choi, Chunjae 2018
Kanginan W. Choi, S. Lim, Education
T. Kang et al. Inc
2 Hari Bumi 2018 Y. Kim, I. Kim, S. Kyohak 2011
Subagya Aksara Kim, B. Park, B. Publishing
Cheong et al.
3 Bambang Yudhistira 2017 S. Kwak, S. Ryu, Chunjae 2011
Ruwanto D. Kim, J. Ahn, Education
O. Lee et al. Inc
4 Sunardi, Yrama 2016 Y. Kwak, S. Ryu, YBM books 2018
Retno, P. Widya E. Kim and J. publishing
and Park
Darmawan
, A.
5 Widodo, Departmen 2009 J. Son, B. Lee, H. Visang 2018
T., and Pendidikan Moon, S. Park, S. Education
Suparmo Nasional Lee, et al. Inc
The data collection procedure began with a thorough reading of the chapter that
serves as the research sample. The relativity chapter was chosen for this study
because it was thought to contain many aspects of the NoS. This material included
narrative text, pictures, diagrams, and other representations that explain the
concept of relativity. Each author marked every word, sentence, and other
representation that is an aspect of the NoS while reading the material chapter. The
results of each author's identification of NoS data were then discussed in order to
obtain real data.
The data analysis technique employed the content analysis method because the
object of this research was a document in the form of physics textbooks that
represented the NoS without any modifications. Furthermore, an analytical
framework is required in content analysis, which is used as a lens in the
investigation to obtain data that is consistent with the research objectives. The
119
analytical framework used was FRA as the NoS category, which consisted of two
main aspects and 11 indicators. This FRA was adapted from an instrument that
had been developed by Kaya and Erduran (2016). This instrument had been used
by several researchers who analyzed the NoS in textbooks (Chen et al., 2022; Park
et al., 2019; Yeh et al., 2022). Table 2 shows the aspects and indicators of the NoS:
Table 2: The aspects and indicators of NoS

No Indicator Description
Science as a cognitive-epistemic system
1 Aims and values Science's primary cognitive and epistemic aims, such as
precision and objectivity
2 Methods Both manipulative and non-manipulative strategies that
facilitate the process of scientific investigation
3 Scientific practices A prominent collection of epistemic and cognitive
methodologies that facilitate the acquisition and validation
of scientific knowledge through a process of social
certification
4 Scientific A range of concepts, including theories, rules, and
knowledge explanations, which serve to substantiate the outcomes
derived from scientific inquiries
Science as a socio-institutional system
5 Social certification The processes by which scientific information is reviewed,
and dissemination evaluated, and validated by scientists. This is commonly
achieved through methods such as journal peer-review
6 Scientific ethos The norms that scientists adhere to in their research
endeavors and in their professional dealings with fellow
peers
7 Social values A range of principles that are highly regarded within a given
society. These values may include but are not limited to
concepts such as individual freedom, the preservation, and
respect for the environment, as well as the promotion of
social usefulness
8 Professional Involvement of scientists in professional activities, including
activities active participation in conferences and doing critical
evaluations of scholarly publications
9 Social organizations How science is organized in institutional settings such as
and interactions universities and research institutes
10 Financial systems The financial aspect that supports knowledge, including
financing systems
11 Political power The power dynamics that are present among scientists and
structures within the scientific community
Member checking was used to ensure the reliability of the analysis in this study
(Miles et al., 2014). Two professionals in the field of science education were invited
to join the discussion. The discussion amongst researchers and experts led
to inter-rater reliability for NoS representation of 90%. During the discussion, the
observation notes were referred to in order to deepen the understanding of the
discourse. Finally, triangulation of the data was performed, and reliable results
were gathered.
120
4. Research results
The comparison of NoS provided in high school physics textbooks in Indonesia
and Korea was based on two main questions: (a) the most stressed aspect of NoS,
and (b) the manner in which NoS is presented. The following is the result of the
data analysis of the NoS comparison between Indonesian physics textbooks and
Korean physics textbooks:
4.1. The most stressed NoS aspect in high school physics textbooks in Indonesia
and Korea
Based on data analysis, there are 71 NoS items presented in Indonesian high
school physics textbooks, with 47 NoS on the cognitive-epistemic aspect and only
24 on the social-institutional aspect. When the two aspects of NoS were compared,
it was found that 66.20% of cognitive-epistemic aspects appear in physics
textbooks, while only 33.80% appear in socio-institutional questions. This means
that the NoS presented in Indonesian high school physics textbooks emphasizes
the cognitive-epistemic aspect rather than the social-institutional. In contrast,
there were 84 representations of the NoS in Korean physics textbooks, with 54
elements of the NoS on the cognitive-epistemic aspect and 30 on the social-
institutional aspect. When the two aspects of NoS were compared, the cognitive-
epistemic aspect had a percentage of 64.29%, while the socio-institutional aspect
had a percentage of 35.71%. In general, both Indonesian and Korean physics
textbooks emphasize cognitive-epistemic aspects rather than social-institutional
aspects.
As shown in Figure 2, of the four NoS indicators on the cognitive-epistemic aspect,

Indonesian physics textbooks perform well only on the scientific knowledge
indicator. The results of the data analysis show that there are 26 items that discuss
scientific knowledge in five Indonesian physics textbooks. Meanwhile, the
indicator of aims and values is minimally presented in Indonesian high school
physics textbooks (five items). Similarly, the methods and scientific practice
indicators were only mentioned eight times. Unlike in Korean high school physics
textbooks, the frequency of appearance of indicators for scientific knowledge,
methods, and scientific practice is almost balanced. In Korea there are 18 items
presented in high school physics textbooks related to scientific knowledge, 16
related to scientific practices, and 14 related to methods. The aims and values
indicator appears only six times in five high school physics textbooks in Korea.
Figure 2 shows a comparison of the NoS presented in high school physics
textbooks between Indonesia and Korea on the cognitive-epistemic aspect.
121
30
26
25
20 18
Frequency
16
15 14
Indonesian Physics
10 8 8 Textbooks
6
5
5 Korean Physics
Textbooks
0
Aims and values Methods Scientific Scientific
practices knowledge
Science as a cognitive-epistemic system
Figure 2: Comparison of NoS in Indonesian and Korean physics textbooks on the

aspect of science as a cognitive-epistemic system
Furthermore, Figure 3 shows a comparison of the NoS in terms of the socio-

institutional aspect. The Indonesian high school physics textbook performs well
on only one of the seven NoS indicators on the socio-institutional aspect, namely
scientific ethos. In total, 24 items in Indonesian high school physics textbooks
discuss the NoS regarding socio-institutional aspects. Social certification and
dissemination indicators are presented in four of the 24 items. Each of the
scientific ethos and social values indicators has five items. In contrast to Korean
high school physics textbooks, there are 30 items that discuss NoS from social-
institutional aspects. The Korean high school physics textbook excels in indicators
of social certification and dissemination (5), social values (6), and professional
activities (8). Similar to Indonesian high school physics textbooks, social
organization and interaction indicators are presented three times in Korean high
school physics textbooks.
Furthermore, indicators of the financial system and political power structures are
indicators of the NoS that are least discussed in the two countries’ high school
physics textbooks. The results of the data analysis show that there are only two
items each related to the financial system and political power structures indicators
in high school physics textbooks in the two countries. Figure 3 depicts the
comparison of the NoS presented in Indonesian and Korean high school physics
textbooks on the socio-institutional aspect:
122
10
9 8
8
Frequency 7 6
6 5 5 5
5 4 4
4 3 33
3 22 22
2
1
Indonesian Physics
0
Textbooks
Korean Physics
Textbooks
Science as a social-institutional system
Figure 3: Comparison of NoS in Indonesian and Korean physics textbooks on the

aspect of science as a social-institutional system
4.2. The way NoS is presented in Indonesian and Korean high school physics
textbooks
Based on data analysis, 59% of the 71 NoS items presented in Indonesian high
school physics textbooks were found in the main text, while 35% were found in
secondary texts (boxes, graphs, sidebars, experimental activities, student
assignments, science stories, and physics in everyday life). The remaining 6%
were presented as a combined text of the main text and secondary text. Similar to
Korean high school physics textbooks, 48% of the 84 NoS representations were
found in the main text, 44% in secondary texts (boxes, graphs, sidebars,
experimental activities, student assignments, science stories, physics in everyday
life) and 8% were presented in both the main text and secondary elements. The
text combination means that the NoS indicator is presented in the main text,
followed by a description in the secondary text. For example, the main text
describes the method used by Michelson and Morley in observing interference
patterns using the equipment developed. Then, in the box, there is a picture of the
experimental equipment and the method used by Michelson and Morley in
observing the interference pattern. Figure 4 depicts a comparison of the way NoS
is presented in Indonesian and Korean physics textbooks:
123
100
90
80
70
59
Percentages
60
48
50 44
Indonesian Physics
40 35 Textbooks
30 Korean Physics
Textbooks
20
6 8
10
0
Main Text Supplementary text Combined text
The way of presenting NoS
Figure 4: Comparison of how NoS is presented in Indonesian and Korean physics

textbooks
There are several examples of how the NoS is presented in high school physics
textbooks in Indonesia and South Korea. The following is an example of an aims
and value indicator that discusses the accuracy of the instrument used by
Michelson and Morley in detecting the presence of ether. The following quote is
taken from a Korean physics textbook (Kwak et al., 2011, p. 54):
After Maxwell discovered that light was an electromagnetic wave, scientists
thought that light also needed a medium. People called this imaginary medium
ether, and Michelson and Molly repeated the complex experiment several
times but could not see changes in the speed of light. That is, as a result of the
experiment, it was found that there is no ether.
Another example that discusses the method indicators used by Michelson and
Morley in observing interference patterns on the equipment they are developing
can be seen in the excerpt below. This quote is taken from an Indonesian physics
textbook (Sunardi et al., 2016, p. 192):
The Michelson-Morley experimental apparatus is rotated 900 about a vertical
axis. In this case, the direction of the ether velocity does not change, but the
rays coming from the M1 and M2 mirrors change places. This position
exchange is expected to cause a shift in the location of the light-dark bands in
the interference pattern. However, even though the experiment was repeated
several times, there was never a shift in the interference pattern.
Scientific practices are a series of epistemic and cognitive practices that lead to
scientific knowledge through social certification. Keywords that are usually used
124
for scientific practice indicators are observation, experimentation, data,

explanation, model, argumentation, classification, and prediction. In this study,
there are 24 items that discuss indicators of scientific practices in the ten physics
textbooks analyzed. The following is an example of scientific practices presented
in Indonesian physics textbooks (Kanginan, 2018, p. 279):
The understanding of the second postulate is illustrated as follows. For
example, on a train moving at a speed of 66 km/hour, there are two passengers
traveling in the carriage in the direction of the train's motion at a speed of 4
km/hour. In general, we would say that the speed of the person relative to the
ground (the person at rest about the edge of the rail) is 70 km/h. Does this
addition apply if the person on the train is replaced by a flash of light on a
train whose speed is 3 x 108 m/s. The 2nd postulate categorically says no.
Why? According to the 2nd postulate, the speed of light is the same in all
directions, and this is true throughout my universe regardless of the motion
of the light source or the observer.
Furthermore, the financial systems indicator is a NoS indicator, which is discussed

a little in high school physics textbooks in Indonesia and Korea. Only two of the
71 NoS indicators discussed in the five Indonesian physics textbooks discuss the
financial systems indicator. The following is an excerpt relating to financial
system indicators taken from Indonesian physics textbooks (Ruwanto, 2017, p.
154):
From 1939 to 1945, the US government has spent about 2 billion dollars for
the Manhattan Project. At 5:29:45 local time, July 16, 1945, a blinding white
flash flashed across the horizon from the Jemez desert valley in northern New
Mexico.
Similar to financial systems, an indicator of political power structure is also rarely

discussed in high school physics textbooks in both Indonesia and Korea. The
following is an excerpt relating to indicators of political power structure taken
from Korean physics textbooks (Kim et al., 2011, p. 67):
On August 2, 1939, just before the outbreak of the second world war, Albert
Einstein wrote a letter addressed to President Franklin D. Roosevelt. In the
letter, it was announced that Nazi-Germany was actively purifying U-235
and that this material might be prepared for the manufacture of atomic bombs.
Not long after, the American government held a secret project called the
Manhattan Project.
5. Discussion
The results of this study showed that the representation of NoS in Korean high
school physics textbooks was higher than in Indonesian physics textbooks. On the
cognitive-epistemic aspect, 47 NoS items were found in Indonesian high school
physics textbooks, while 54 items were found in Korean high school physics
textbooks. The main distinction is in the scientific knowledge indicator, where this
indicator is more emphasized in Indonesian high school physics textbooks. This
indicator relates to theories, laws, and explanations that support the results of
scientific investigations. Other indicators, such as aims and values, methods, and
scientific practice, are undervalued in Indonesian physics textbooks. Unlike in
Korean high school physics textbooks, the indicators of methods and scientific
125
practice are also emphasized in a balanced way. The methods indicator is a

manipulative and non-manipulative technique that supports scientific inquiry,
while the scientific practice indicator is a series of epistemic and cognitive
practices that lead to scientific knowledge. This provides students with a wealth
of experience and knowledge regarding a range of epistemic practices and
manipulative and non-manipulative techniques that support scientific theories
and laws. Therefore, the difference in scientific literacy between Indonesian and
Korean students could be attributed to the NoS aspects emphasized in different
physics textbooks.
On the socio-institutional aspect, physics textbooks for Indonesian high schools

only excel on the scientific ethos indicator. This indicator is concerned with the
norms that scientists employ in their work and interactions with colleagues. The
indicator of professional activities is the most emphasized of the 30 items in the
Korean high school physics textbook. A total of eight items were found
representing this indicator, whereas the Indonesian high school physics textbook
contained only three items. This indicator relates to how scientists engage in
professional settings such as attending conferences and doing publications. In
Indonesia and Korea, the indicators of the financial system and political power
structures are the least discussed in high school physics textbooks. According to
data analysis, there are only two items in each country's high school physics
textbooks related to the financial system and political power structures indicators.
This finding aligns with the research findings of Park et al. (2019), namely that
these two indicators receive less attention in Korean high school physics
textbooks.
Furthermore, the findings of this study are in line with previous research by
Dhamayanti et al. (2019) that theNoS is still not emphasized in Indonesian high
school physics textbooks. The indicators emphasized among those investigated
are those of scientific theory and laws. More than half of the NoS indicators in the
Indonesian high school physics textbook for class X are indicators of theory and
law. Ardwiyanti et al. (2021) also revealed that the Indonesian high school physics
textbook for class X did not yet explicitly, correctly, or completely include aspects
of the NoS. The aspects of scientific laws and the application of science in social
and cultural contexts were the most emphasized in the three high school physics
textbooks that they examined. Because of this, there needs to be an effort from
authors of textbooks, the government, and stakeholders on how physics textbooks
in Indonesia describe aspects of the NoS explicitly, correctly, and completely.
Furthermore, most science teachers use textbooks as the primary source of
information when teaching science in the classroom (Bancong & Song, 2018). The
textbooks that are rich in the NoS will have a positive impact on students'
scientific literacy skills (Abd-El-Khalick et al., 2017; Bugingo et al., 2022; Thao-Do
& Yuenyoung, 2013; Ha, 2018; McDonald & Abd-El-Khalick, 2018).
These findings are also consistent with those of Zhang et al. (2022), who found
that cultural and curriculum differences influence students' and teachers’
perceptions of the NoS. Teachers in China tend to rely more on what is written in
textbooks. Whenever they come across a point of view that has never been studied
126
in textbooks, they cannot provide examples to support that view. Conversely,

teachers in Canada can explain this view by giving concrete examples that are not
in the textbooks. Park et al. (2014) also revealed that country differences can have
an effect on subjectivity variance, empirical testability, and the methods used to
explain scientific theories or laws. Nonetheless, textbooks as the primary guide
for teachers in school teaching must be enhanced with NoS aspects. According to
Liang et al. (2009), Turkish pre-service teachers scored the lowest in all aspects of
the NoS because Turkish textbooks included misleading assumptions about NoS.
As a result, a large number of Turkish participants displayed a naïve view of
several aspects of the NoS. Therefore, the future physics curriculum, especially in
Indonesia, must include the currently known NoS concept and place it in a social
and cultural context.
6. Conclusion
Understanding NoS is now widely recognized as an important component of
studying physics at the high school level. The purpose of this study was to learn
more about the NoS and contribute to related literature by comparing aspects of
the NoS in physics textbooks used in Indonesia and Korea. This study found that
Indonesian high school physics textbooks (71) have fewer NoS elements than
Korean physics textbooks (84). The NoS in Indonesian high school physics
textbooks emphasizes cognitive-epistemic aspects (66.20%) over socio-
institutional aspects (33.80%). Similarly, Korean high school physics textbooks
place more emphasis on cognitive-epistemic aspects (64.29%) than on socio-
institutional aspects (35.71%).
The main distinction is in the scientific knowledge indicator, which has a higher
priority in Indonesian high school physics textbooks. This indicator is concerned
with theories, laws, and explanations that support the findings of scientific
investigations. In contrast to Korean high school physics textbooks, the indicators
of scientific knowledge, methods, and scientific practice are all given the same
significance. On the socio-institutional aspect, physics textbooks for Indonesian
high schools only excel on the scientific ethos indicator. Furthermore, 59% of the
71 NoS elements presented in Indonesian high school physics textbooks were
found in the main text, 35% in the secondary text, and 6% were presented in both.
Similar to Korean high school physics textbooks, 48% of the 84 NoS presented
were found in the main text, 44% in secondary texts, and 8% were presented
together in both the main text and secondary texts.
This study has several limitations, including the fact that only five high school
physics textbooks were examined for each country, and there was no qualitative
data to back up the quantitative data. Therefore, future research is required to
support and strengthen this study by using a larger sample of physics textbooks
and also collecting qualitative data, such as interviews with high school physics
teachers and authors of the textbooks, to supplement the results of this study.
Acknowledgments
This research was funded by Universitas Muhammadiyah Makassar through an
internal research scheme managed by the Research, Development, and
127
Community Service Institute with contract number 004/Kont-

Penl/Pengabd/IV/1442/2021.
7. References
Abd-El-Khalick, F., Myers, J. Y., Summers, R., Brunner, J., Waight, N., Wahbeh, N.,
Zeineddin, A. A., & Belarmino, J. (2017). A longitudinal analysis of the extent and
manner of representations of nature of science in U.S. high school biology and
physics textbooks. Journal of Research in Science Teaching, 54(1), 82–120.
https://doi.org/10.1002/tea.21339
Abd-El-Khalick, F., Waters, M., & Le, A. P. (2008). Representations of nature of science in
high school chemistry textbooks over the past four decades. Journal of Research in
Science Teaching, 45(7), 835–855. https://doi.org/10.1002/tea.20226
Ardwiyanti, D., Jumadi, J., Puspitasari, H., & Rahayu, P. (2021). Representations of nature
of science in high school physics textbooks. Jurnal Pendidikan Fisika Indonesia, 17(1),
22–30. https://doi.org/10.15294/jpfi.v17i1.24805
Australian Curriculum Assessment and Reporting Authority. (2016). The Australian
curriculum: Science version 8.3. Retrieved from
https://www.australiancurriculum.edu.au/umbraco/Surface/Download/Pdf?
subject=Science&type=F10
Ayık, Z., & Coştu, B. (2020). A study on demonstration of the nature of science in science
textbooks: History and philosophy of science perspectives. Jurnal Pendidikan IPA
Indonesia, 9(3), 451–464. https://doi.org/10.15294/jpii.v9i3.26009
Bancong, H., & Song, J. (2018). Do physics textbooks present the ideas of thought
experiments? A case in Indonesia. Jurnal Pendidikan IPA Indonesia, 7(1), 25–33.
https://doi.org/10.15294/jpii.v7i1.12257
Bancong, H., & Song, J. (2020). Factors triggering thought experiments in small group
physics problem-solving activities. New Physics: Sae Mulli, 70(5), 466–480.
https://doi.org/10.3938/NPSM.70.466
Brunner, J. L., & Abd-El-Khalick, F. (2020). Improving nature of science instruction in
elementary classes with modified science trade books and educative curriculum
materials. Journal of Research in Science Teaching, 57(2), 154–183.
https://doi.org/10.1002/tea.21588
Bugingo, J. B., Yadav, L. L., & Mashood, K. K. (2022). Representation of nature of science
aspects in secondary school physics curricula in East African community
countries. International Journal of Learning, Teaching and Educational Research, 21(8),
175–201. https://doi.org/10.26803/ijlter.21.8.11
Chen, B., Chen, S., Liu, H., & Meng, X. (2022). Examining the changes in representations
of nature of science in Chinese senior high school chemistry textbooks. Science and
Education. https://doi.org/10.1007/s11191-022-00383-7
Cofré, H., Vergara, C., Lederman, N. G., Lederman, J. S., Santibáñez, D., Jiménez, J., &
Yancovic, M. (2014). Improving Chilean in-service elementary teachers’
understanding of nature of science using self-contained NoS and content-
embedded mini-courses. Journal of Science Teacher Education, 25(7), 759–783.
https://doi.org/10.1007/s10972-014-9399-7
Dhamayanti, L. F., Putra, N. M., & Handayani, L. (2019). Analisis struktur penyajian dan
konten nature of science (nos) pada buku teks fisika SMA pokok bahasan getaran
harmonis. UPEJ: Unnes Physics Education Journal, 8(1), 16–21.
https://doi.org/10.15294/upej.v8i1.29498
Ha, S. (2018). Analysis of special relativity theory in high school physics textbooks under
the 2009 and the 2015 revised curricula. New Physics: Sae Mulli, 68(10), 1069–1080.
https://doi.org/10.3938/NPSM.68.1069
128
Kanginan, M. (2018). Fisika 3: Untuk SMA/MA kelas XII [Physics 3: For high schools class
XII]. Erlangga.
Karampelas, K. (2018). Identifying potential to promote the nos in elementary science
teaching packages. International Journal of Learning, Teaching and Educational
Research, 17(6), 1–18. https://doi.org/10.26803/ijlter.17.6.1
Kaya, E., & Erduran, S. (2016). From FRA to RFN, or how the family resemblance approach
can be transformed for science curriculum analysis on nature of science. Science
and Education, 25(9–10), 1115–1133. https://doi.org/10.1007/s11191-016-9861-3
Kim, Y., Kim, I., Kim, S., Park, B., & Cheong, B. (2011). High shool physics I. Kyohak
Publishing.
Kwak, S., Ryu, S., Kim, D., Ahn, J., & Lee, O. (2011). High school physics I. Chunjae
Education.
Liang, L. L., Chen, S., Chen, X., Kaya, O. N., Adams, A. D., Macklin, M., & Ebenezer, J.
(2009). Preservice teachers’ views about nature of scientific knowledge
development: An international collaborative study. International Journal of Science
and Mathematics Education, 7(5), 987–1012. https://doi.org/10.1007/s10763-008-
9140-0
McComas, W. F., & Clough, M. P. (2020). Nature of science in science instruction: Meaning,
advocacy, rationales, and recommendations. In W. F. McComas (Ed.), Nature of
science in science instruction: Rationales and strategies (pp. 3-22). Springer.
https://doi.org/10.1007/978-3-030-57239-6_1
McDonald, C. V., & Abd-El-Khalick, F. (2018). Representations of nature of science in
school science textbooks. In C. V. McDonald & F. Abd-El-Khalick (Eds.),
Representations of nature of science in school science textbooks: A global perspective (pp.
1-19). Routledge. https://doi.org/10.4324/9781315650524-1
Miles, M. B., Huberman, A. M., & Saldana, J. (2014). Qualitative data analysis: A methods
sourcebook (3rd ed.). Sage Publications.
NGSS Lead States. (2013). Next generation science standards: For states, by states. National
Academy Press. https://www.nextgenscience.org/
Organisation for Economic Co-operation and Development (OECD). (2019). PISA 2018
assessment and analytical framework. OECD Publishing.
https://www.oecd.org/education/pisa-2018-assessment-and-analytical-
framework-b25efab8-en.htm
Park, W., Yang, S., & Song, J. (2019). When modern physics meets nature of science: The
representation of nature of science in general relativity in new korean physics
textbooks. Science and Education, 28(9–10), 1055–1083.
https://doi.org/10.1007/s11191-019-00075-9
Park, H., Nielsen, W., & Woodruff, E. (2014). Students' conceptions of the nature of science:
Perspectives from Canadian and Korean middle school students. Science and
Education, 23 , 1169-1196. https://doi.org/10.1007/s11191-013-9613-6
Ruwanto, B. (2017). Fisika: SMA Kelas XII [Physics: High school class XII]. Yudhistira
So, K., & Kang, J. (2014). Curriculum reform in Korea: Issues and challenges for twenty-
first century learning. Asia-Pacific Education Researcher, 23(4), 795–803.
https://doi.org/10.1007/s40299-013-0161-2
Sunardi, S., Retno, P., & Darmawan, A. (2016). Fisika untuk SMA/MA kelas XII kelompok
peminatan matematika dan ilmu-ilmu alam [Physics high school class XII
mathematics and natural sciences specialization groups]. Yrama Widya
Thao-Do, T. P., & Yuenyoung, C. (2013). Nature of science presented through the history
of heat in Vietnamese physics textbooks. Some suggestions for teachers. Journal of
Applied Sciences Research, 9(4), 2575–2584.
129
Upahi, J. E., Ramnarain, U., & Ishola, I. S. (2020). The nature of science as represented in
chemistry textbooks used in Nigeria. Research in Science Education, 50(4), 1321–
1339. https://doi.org/10.1007/s11165-018-9734-7
Yeh, Y.-F., Dhurumraj, T., & Ramnarain, U. (2022). Representations of the nature of science
in South African physical sciences textbooks on electricity and magnetism. Science
and Education. https://doi.org/10.1007/s11191-022-00370-y
Zhang, G., Li, Y., Zhou, G., & Ho, S. W. Y. (2022). Exploring pre-service science teachers’
perspectives on the nature of science: A comparative study between China and
Canada. ECNU Review of Education, 5(3), 520–536.
https://doi.org/10.1177/2096531120966782
130

Vol. 22, No. 10, pp. 130-151, October 2023
Received Sep 2, 2023; Revised Oct 19, 2023; Accepted Oct 29, 2023
Exploring the Potential of Integrating Local

Wisdom into the Development of Pocket Book
Learning Media: A Systematic Literature Review
Sukadari* , Mahilda Dea Komalasari , Nina Widyaningsih

Faculty of Teacher Training and Education,
Universitas PGRI Yogyakarta (UPY), Indonesia
Gulzhaina K. Kassymova , Fang Yuqi

Institute of Pedagogy and Psychology, Abai Kazakh National Pedagogical
University, 050010, Almaty, Dostyk Ave., 13, Kazakhstan
Lily Muliana Mustafa

Management and Science University, Shah Alam, Selangor, Malaysia
Nurudeen Babatunde Bamiro

Department of Language, Arts & Social Science
Lagos State University, Ojo, Nigeria
Abstract. The incorporation of local knowledge into pocket book learning

media provides for a multidimensional strategy for enhancing academic
integrity. To create a learning atmosphere that prioritizes both academic
excellence and ethical conduct, this incorporation mixes contemporary
educational techniques with traditional cultural views. This paper aims
to systematically review and synthesize literature on the creation of
pocket book learning media based on local knowledge, with a focus on its
potential to enhance academic integrity. To investigate the research topic,
a systematic literature review was conducted using the preferred
reporting items for systematic reviews and meta-analyses (PRISMA)
methodology to address three main research objectives. The quality of
studies reviewed between 2013 and 2023 was analyzed using the Mixed
Methods Appraisal Tool (MMAT). Three (3) articles were rated as
average, eleven (11) articles as above-average quality, and the remaining
six (6) were rated high quality. The central discovery of the research
highlights a shared foundation between local wisdom and academic
integrity, which are moral and social ethics, honesty, trust, and respect,
forming a basis upon which pocket book learning media can be crafted.
*
Corresponding author: Sukadari; sukadariupy@gmail.com
©Authors
131
By infusing the principles of local wisdom into pocket book learning

resources, the potential to enhance academic integrity arises due to this
common ground. The study has further established a versatile framework
that, in conjunction with various research models, can guide the creation
of learning materials rooted in local wisdom, thereby fostering and
promoting academic integrity.
Keywords: academic integrity; local wisdom; pocket book learning

media; PRISMA; systematic literature review
1. Introduction
Two of the seventeen Sustainable Development Goals (SDGs) for 2030 outlined by
the United Nations feature implicit directives for higher education institutions to
adopt inclusive and culturally responsive approaches. The first of these objectives
seeks to guarantee that everyone, regardless of gender, age, race, or ethnicity, has
access to high-quality education. The second objective aims to foster intercultural
understanding, tolerance, and respect among people as well as a sense of global
citizenship and group responsibility (Bygrave & Aşık, 2019). Maintaining
academic integrity is a problem that affects all educational institutions, as
transgressions of academic ethics frequently result in problems. These
transgressions include cases of cheating, plagiarism, credential fraud, and
improper behavior in relation to policy changes (Denisova-Schmidt, 2016). The
survey carried out by Dr Donald McCabe in association with the International
Center for Academic Integrity (ICAI) stands out as the most thorough in relation
to identifying cheating habits. The poll, which was conducted over a 12-year
period (2002–2015) in 24 American high schools, involved over 70,000
participants, including graduate and undergraduate students. The results of this
study are astounding, with a startling 95% of respondents admitting to
plagiarizing as well as cheating on tests and homework (Unicheck, 2019). The
global progress of academic integrity is greatly aided by education. It is essential
to develop a deeper understanding of academic integrity and to carry out effective
educational activities targeted at regulating and discouraging academic
misconduct (Anayat, 2019).
The act of learning involves the dynamic interplay between students and
educational materials (Balasubramanian et al., 2014; Kamelia, 2019; Yusuf et al.,
2022). An effective learning process can be defined as one that empowers students
to achieve greater independence in their learning journey, enabling them to
construct knowledge through active engagement (Adewale & Potokri, 2023;
Awaludin et al., 2020; Putranta & Supahar, 2019). The teaching and learning
process involves reciprocal knowledge exchange between teachers and students.
However, challenges in understanding instructional material can arise due to the
verbal nature of learning, leading to reduced engagement. Educators must grasp
the learning process to guide students effectively and create a conducive
environment. There is need for innovative approaches to enhance student
motivation, foster critical thinking, and develop values. An example of such
approach is pocket book media (Harun et al., 2021; Shuhratovich, 2020).
132
Print media is one of the many forms of learning-related media currently being
developed. Materials created on paper for instruction and information are
considered print media. It serves as an alternative to textbooks (Karmintoro et al.,
2021). For both teachers and students, textbooks continue to be the main source of
knowledge during the learning process. These textbooks can be transformed into
pocket-sized textbooks, or simply called pocket books. Small books called pocket
books can fit in pockets and are convenient for carrying around (Samala et al.,
2021). Pocket books have images that complement the fabric and color to create a
pleasing appearance. Students enjoy readings that are interesting but without
many details, pictures, or colors (Dita et al., 2021; Kaniawati et al., 2021). A pocket
book is a compact book containing easily portable information, available in both
physical and digital formats. Digital pocket book learning materials offer several
benefits, including facilitated learning, on-the-go accessibility, durability, and
space efficiency. These advantages are particularly prominent in the digital realm
(Anita et al., 2021), which validates the strengths of this digital pocket book
medium.
Local wisdom refers to the diversity of the local area and includes local beliefs on
policies or ways of living. It serves the purpose of molding people into wiser, more
experienced individuals. Local knowledge is defined as local concepts that are
well-considered, wise, valuable, ingrained, and followed by the people in a
culture. Local knowledge is a component of culture. It is a profoundly ingrained
component of human life and society that is connected to human resources,
sources of culture, economics, security, and laws. Local wisdom is a body of
information, a way of thinking, an understanding, and customs specific to a given
group of people or culture. It includes all the accumulated and passed-down local
information, practices, beliefs, and abilities that are considered to be part of the
local culture. To promote national education goals, it is essential to include local
expertise into teaching and learning activities (Irwan et al., 2020).
Educational resources rooted in local wisdom were created to address the

challenges faced within schools. Specifically, this initiative aims to tackle the
insufficiency of teaching materials aligned with the local environment (Alba et al.,
2019; Tinja et al., 2017). Incorporating cultural elements into the educational
process will benefit students as well as the survival of culture as a whole. Utilizing
educational tools based on indigenous knowledge is an intentional effort to
sustain cultural norms (Hest et al., 2021). Teaching resources based on local
wisdom have the potential to improve educational outcomes. This is due to the
notion that more appealing materials encourage more student involvement in the
learning process (Faisal & Sulkipani, 2016). According to several studies on the
development of teaching materials, materials based on local knowledge or content
can increase participants’ eagerness to learn at both the primary and higher
education levels (Hutama, 2016; Kormasela et al., 2020). The creation of teaching
materials using local scripts can enhance students’ language proficiency, writing
abilities, and motivation to engage in active learning (Lestariningsih &
Suardiman, 2017). Therefore, it can be claimed that creating instructional
materials based on traditional knowledge will enhance student learning.
133
Integrity is defined as the quality of being honest and having strong morality
(Ramdani, 2018). The concept refers to the extent to which individuals think and
act honestly and have strong morality. Academic integrity has evolved
significantly since the inception of the concept in the 1800s, when it was typically
viewed as a moral code of conduct that all members of academia were supposed
to uphold (Gallant, 2008). Throughout its evolution, the commitment to academic
integrity has not been upheld consistently in numerous educational institutions
worldwide. A range of transgressions, collectively falling within the realm of
academic integrity, including dishonesty, misconduct, and fraud, has been and
continue to be documented across all tiers of academia and broader society. In the
contemporary context, even the most extensively examined of these
transgressions, namely plagiarism—defined as the act or habit of appropriating
the work, ideas, etc. of another person and presenting them as one’s own—
remains a subject of concern. Furthermore, there are those who posit that a notion
such as plagiarism might require re-evaluation and adjustment to encompass
cross-cultural variations (Click, 2012). Academic integrity is a major concern on a
global scale and is essential to both academia and society at large. According to
research, maintaining academic integrity involves more than just adhering to
rules and regulations. It cannot be handled in a vacuum and must be incorporated
into all debates (Ahmed, 2020).
The creation of pocket book learning material based on local knowledge is a

proactive move towards fostering an academic-integrity culture that is consistent
with the values of the community at large. The credibility of knowledge
production and distribution is ensured by academic integrity, a key tenet of the
educational system. Plagiarism and other forms of misconduct in the classroom,
however, continue to put the credibility of educational institutions in every
country at risk (Bretag, 2013; Perkins et al., 2020). Innovative approaches that align
with the values and beliefs of the local community are needed to address these
difficulties (Blau et al., 2021; Christensen & Eaton, 2022). Few scholarly
investigations have been conducted with a dedicated emphasis on the
formulation of pocket book learning media grounded on local wisdom to improve
academic integrity (Parhan & Dwiputra, 2023). As a result, the present research
introduces an innovative dimension to the scope and structure of incorporating
local wisdom within pocket book learning media, aimed at enhancing academic
integrity. This study assumes the role of a valuable repository of insights for
stakeholders engaged in the conception and execution of character education
programs characterized by excellence. Anticipated outcomes of such inquiries are
poised to yield enhancements not solely within the cognitive dimensions of
educational provision, but also in the preservation of indigenous knowledge and
the fortification of students’ moral attributes.
The recognition of the aforementioned gap in existing research prompted the

formulation of three research questions:
1. What is the impact of local wisdom on academic integrity?
2. What is the impact of local-wisdom integration into pocket book
media?
134
3. How is academic integrity enhanced through the synergy of local

wisdom and pocket book media?
2. Methodology
The aim of this systematic literature review was to systematically locate, collect,
assess, and combine existing research relevant to pocket book learning media,
local wisdom, and academic integrity (Parhan & Dwiputra 2023). Adherence to
the preferred reporting items for systematic reviews and meta-analyses (PRISMA)
2020 recommendations was upheld to meticulously carry out this research (Page
et al., 2021). Reviewing the current literature allowed the understanding of the
breadth and depth of existing articles as well as identifying potential gaps to be
explored. This was done by summarizing, analyzing, and synthesizing literature
related to the research topic (Xiao & Watson, 2019). The phases utilized in
selecting the reviewed articles are discussed below.
2.1 Identification
The process of identification is utilized to increase the importance of the keywords
that are utilized in the search. This is important since the identification process
increases the possibility of yielding more articles that are pertinent to the review
(Shaffril et. al., 2021). The well-known databases Google Scholar and Scopus were
employed for this systematic review. We broadened our selection of keywords
and formulated an extensive search query through the utilization of Boolean
operators, truncation, wildcard entries, and phrase searches across the two
primary databases, namely Scopus and Google Scholar. The choice of these
databases was grounded in their extensive collections and robust search
functionalities, as highlighted in prior studies (Gusenbauer & Haddaway, 2020).
Scopus offers both basic and advanced search options, empowering users to
narrow down results by factors such as document type, date, subject, author, and
recent publications, all while employing Boolean operators. In parallel, Google
Scholar enhanced our research by applying Boolean operators AND and OR in
conjunction with keywords such as “pocket book media”, “portable book media”,
“local wisdom”, “local knowledge”, “indigenous knowledge”, “academic
integrity”, “academic honesty”, and “character education” to supplement article
references. Additionally, our approach included a manual Google search,
following the methodology outlined by Jensenius et al. (2018). Google Scholar was
chosen for its user-friendly interface, accessibility, consistent search evaluation,
and rapid delivery of results. The main keywords used to search for articles are
presented in Table 1.
Table 1: Keywords used for this study

Google Scholar Scopus
TITLE (“Pocket Book Media” AND TITLE-ABS-KEY (“Pocket Book Media”
“academic integrity”), (“Pocket Book AND “Academic integrity” OR
Media” AND “local wisdom”), (“Local “Academic Honesty”), (“Pocket Media”
wisdom” AND “Academic integrity”) AND “Local Wisdom” OR “Local
Knowledge”)
135
2.2 Screening
Using the particular keywords shown above, 85 articles were identified in this
study using Scopus and Google Scholar. At this point, all the articles downloaded
from the database were screened to identify and eliminate any papers that were
undesired. Eight articles that were duplicate were eliminated at this stage.
Furthermore, 18 articles that have different titles were also eliminated. This left 59
papers that advanced to the next stage.
2.3 Eligibility
The objective of this phase was to make sure that all the journal articles that had
passed the initial screening stage complied with the requirements. At this step,
the journal articles’ titles and abstracts were examined to see if they still qualified
for review. In this phase, only articles that satisfied all requirements were selected
to be included in the review. Books, book series, book chapters, conference
proceedings, non-English-language articles, and those published before 2012
were eliminated. Journal articles published after 2013 were also included in order
to obtain correct and pertinent materials. Table 2 presents the inclusion and
exclusion criteria used in the screening process.
Table 2: Inclusion and exclusion criteria applied in selecting articles

Criterion Eligibility/inclusion Exclusion
Types of literature Research articles Chapter in a book, books,
conference proceedings
Language choice English language Non-English languages
Time range 2013–2023 2012 and earlier
Country Global –
2.4 Data Extraction

To completely evaluate numerous research studies using multiple methodologies,
we performed an integrative review in this study. We carefully reviewed the titles,
abstracts, findings, arguments, and conclusions of 20 articles that were selected
for the review. Using thematic analysis, we categorized the results of our analysis
according to the author/s, methodology, author country of affiliation, journal,
and research approach. This method finds trends by identifying patterns in
results. We concentrated on creating categories from the articles, identifying
topics such as pocket book media, local wisdom, and academic integrity.
2.5 Quality Assessment Appraisal

The Mixed Methods Appraisal Tool (MMAT) was used to assess the selected
articles. The quality of the papers that were chosen was evaluated as part of this
systematic review (Pluye & Hong, 2014). The accuracy of the statistical analyses
carried out to achieve the research objective was evaluated based on a number of
aspects, including the clarity of the research questions, the confidence in
evaluating these questions, the sampling strategies, the methods utilized for data
collection, and the suitability of the statistical analyses used (Renzi et al., 2015).
The criteria listed in the MMAT guidelines were used to evaluate the quality pf
the articles. The ratings for the quality of articles were assigned as follows: 25%
136
for lower quality, 50% for medium quality, 75% for above-average quality, and
100% for high quality. Mixed techniques were examined qualitatively for the
analysis. According to the assessment of the evaluator, Three (3) articles were
rated as average, eleven (11) articles as above-average quality, and the remaining
six (6) were rated high quality six of the articles were high-quality articles. Figure
1 illustrates the PRISMA flowchart depicting the articles that underwent review.
Figure 1: PRISMA flow diagram
137
3. Results
3.1 Research Attributes
The data extracted from the analyzed articles are presented clearly in Table 3.
Table 3: Data collection table

S/N Author/s Methodology Author Journal Research
employed country of model
affiliation
1 Eliezanatalie Qualitative Indonesia International The research
and Deta Journal of used a
(2023) Research and qualitative
Community descriptive
Empowerment research
method,
namely by
conducting a
needs
analysis
2 Gularso et al. Quantitative Indonesia Pegem Journal This research
(2023) of Education includes
and Instruction research and
development
(R & D)
research by
adopting the
ADDIE
research
method
3 Parhan and Qualitative Indonesia Journal of Systematic
Dwiputra Innovation in literature
(2023) Educational review
and Cultural
Research
4 Qurrota’aini Quantitative Indonesia Jurnal R&D
and Sukirno Pendidikan
(2013) Akuntansi
Indonesia
5 Zukmadini Quantitative Indonesia Biosfer: Jurnal R&D
et al. (2020) Pendidikan
Biologi
6 Raibowo Qualitative Indonesia Kinestetik: R&D
et al. (2023) Jurnal Ilmiah
Pendidikan
Jasmani
7 Laila et al. Quantitative Indonesia International Quasi-
(2021) Journal of experimental
Evaluation and method
Research in
Education
8 Ariyanto Mixed Indonesia Uniglobal R&D
et al. (2022) Journal of
Social Sciences
138
S/N Author/s Methodology Author Journal Research

employed country of model
affiliation
and
Humanities
9 Chaer et al. Qualitative Indonesia JIE (Journal of Literature
(2021) Islamic review
Education)
10 Yusuf (2023) Qualitative Indonesia International Meta-analysis
Journal of
Educational
Methodology
11 Dewi and Qualitative Indonesia Journal of R&D
Ramadan Education
(2021) Technology
12 Suud et al. Mixed Indonesia Journal of No specific
(2022) Pharmaceutical research
Negative approach
Results
13 Lyesmaya Qualitative Indonesia Journal of R&D
et al. (2020) Physics:
Conference
Series
14 Thalhah Qualitative Indonesia International R&D
(2020) Journal of (ADDIE)
Scientific
Engineering
and Research
(IJSER)
15 Rustan Qualitative Indonesia Malaysian Case study
Effendi Online Journal
(2020) of Educational
Management
(MOJEM)
16 Hidayati Qualitative Indonesia International R&D
et al. (2020) Journal of
Instruction
17 Raman & Qualitative Trinad & International Routine
Ramlogan Tobago Journal for interaction
(2020) Educational with students
Integrity
18 Anggraini Qualitative Indonesia Journal of Needs
and Kusniarti Education and analysis
(2017) Practice
19 Macfarlane Qualitative Hong Kong Studies in Literature
et al. (2014) & China Higher review
Education
20 Bretag et al. Qualitative Australia Studies in Literature
(2019) Higher review
Education
The tabulated information distinctly illustrates that in relation to the authors’

affiliations, a predominant 80.95% of the examined articles were authored by
139
individuals affiliated with institutions in Indonesia. This observation underscores

the significance of studies pertaining to pocket book media and local wisdom
within the Indonesian research community, thereby highlighting the concurrent
emphasis on academic integrity and honesty.
3.2 Research Methodology

Figure 2 depicts the research methodology employed in the examined articles.
Among the articles, 70% (14) employed a qualitative research method, 20% (4)
utilized a quantitative research method, and 10% (2) adopted a mixed-methods
research strategy.
Mixed
10%
Quantitative
15%
Qulaitative
75%
Figure 2: Research methodology employed in the examined articles
3.3 Research Model

Table 4 presents the diverse research models utilized in the examined articles in
the context of crafting pocket book learning materials.
Table 4: Research models utilized in the selected articles

Research model Number of articles
Research and design 7
Analysis, design, development,
implementation, and evaluation 2
(ADDIE)
Needs analysis 2
Literature review 4
Meta-analysis 1
Routine interaction 1
Case study 1
Quasi-experimental method 1
A notably prevalent research approach in the selected articles is the research and
design model. In this model, the research and development process serves to
140
generate innovative products or refine existing ones, thereby facilitating traceable

accountability, as articulated by Qurrota’aini and Sukirno (2013).
Furthermore, three of the analyzed research articles adopted the ADDIE

approach, a widely recognized instructional design methodology within
educational research. This approach guides the planning and execution of
investigations through five key phases, namely analysis, design, development,
implementation, and evaluation, as outlined by Zahra et al. (2023).
The assessed articles also encompass a range of alternative research models. These
include needs analysis, systematic literature review, meta-analysis, routine
interaction, and case study methodologies. These models collectively contribute
to a comprehensive understanding of the subject matter: the development of
pocket book learning media rooted in local wisdom to enhance academic
integrity.
3.4 Publication by Year

Figure 3 illustrates the distribution of the examined articles according to year of
publication. While the topics of pedagogical resources in the form of pocket books,
utilization of local knowledge, and the upholding of academic integrity have
constituted longstanding concerns, their recent prominence is discernible from
the graphical representation in Figure 3. Notably, the years 2023 and 2020 exhibit
the most substantial volumes of articles (n = 4 each). The year 2021 follows closely
with a total of three publications, whereas 2022 features two publications.
Furthermore, it is noteworthy that 2013, 2014, 2017, and 2019 each register a single
publication.
4
Article number
0
2013 2014 2017 2019 2020 2021 2022 2023
Year
Figure 3: Publication of selected articles by year
141
4. Discussion
4.1 The Impact of Local Wisdom on Academic Integrity
The impact of local knowledge on academic integrity has far-reaching effects in
educational situations. Local wisdom can have a significant impact on the
development of academic integrity among students since it is frequently
ingrained in traditional values, cultural norms, and indigenous knowledge
systems. This assertion can be supported by Suud et al. (2022), who looked at
students in Indonesia’s Aceh region. They concluded that Aceh’s local culture has
a big impact on students’ honesty and offers valuable insights for promoting
academic integrity. Honesty, which has its roots in religious values, influences
both personal behavior and academic ethics. In addition to reflecting
national-character goals, academic environments promote responsible citizens.
Additionally, attributes such as responsibility, discipline, and diligence that are
linked to honesty directly promote academic integrity. By inspiring students to
pursue their studies honestly and uphold the principles of fairness and integrity,
incorporating Aceh’s cultural values into the classroom can effectively promote
moral behavior.
Furthermore, Lyesmaya et al. (2020) claimed that adding local wisdom to value-
based learning models can have a big impact on academic integrity. Through peer
and teacher discussion, these models build a good learning atmosphere, develop
learner responsibility through moral behavior, and improve literacy learning
possibilities. These models promote moral behavior and a sense of belonging by
helping students connect to their cultural identities. In the end, including such
models encourages a comprehensive strategy for upholding academic integrity
by focusing on positive classroom dynamics, individual accountability, and
sincere interest in learning. Local knowledge may serve as the foundation for
character education that encourages academic honesty. For instance, a study by
Hidayati et al. (2020) indicated that by incorporating culturally specific beliefs and
behaviors, character education might be taught in higher education based on local
wisdom. With the aid of this strategy, students can better comprehend the
significance of academic honesty within the framework of their own culture and
community.
A strong framework for fostering academic integrity is produced by combining

local knowledge with character education. Teachers can encourage students to
uphold moral conduct in their academic endeavors by teaching them cultural
values and virtues. This fosters a feeling of identity, responsibility, and respect
within the learning community. This aligns with the findings of Parhan and
Dwiputra (2023) that character education that is grounded in local wisdom helps
people develop culturally ingrained values, which increases its efficacy and
promotes academic integrity. This method encourages contextual learning, a wide
range of real-world applications, and thorough integration outside of the
classroom. Because of this, Indonesian youth acquire culturally appropriate
values, which improve academic integrity in their environment. Moreover, Chaer
et al. (2021) argued that character education is a vital aspect of a nation’s education
which can be intricately linked to academic integrity. It calls for comprehensive
development within a cultural context, involving government, communities,
142
schools, and parents. A collective effort is needed to instill cultural values and
ethics, fostering unity in education.
Continuing the discussion on character education, an inseparable component of

upholding academic integrity, Yusuf (2023) reached the conclusion that
incorporating culturally pertinent learning resources influences the character
development of Indonesian students. This, in turn, cultivates a dedication to
maintaining academic honesty. Finally, Dewi and Ramadan (2021) highlighted
that local-wisdom-based curriculum materials developed in Riau province is very
doable in terms of language, content, and design. The solid information,
comprehensible language, and appealing layout of these resources successfully
increase students’ knowledge. They are advised as effective resources for
fostering student learning while upholding academic integrity.
4.2 Impact of Local-Wisdom Integration on Pocket Book Learning Media

Local knowledge infused into pocket book media can enrich the material and
make it more approachable to learners and relevant to their culture. Because they
can relate the ideas to their own experiences and traditions, students are more
engaged and have a deeper knowledge of the material. Learning results are
improved when local knowledge is incorporated into pocket book media. By
adding context, aligning with scientific learning, and utilizing local resources, the
infusion makes the material more relatable and improves comprehension. As a
result, this strategy enhances academic success and learning opportunities
(Eliezanatalie & Deta, 2023). Research by Gularso et al. (2023) on the feasibility of
the use of local-wisdom-based pocket book media has found that it is practical
and efficient for students to use a pocket book based on local knowledge as a
companion book for the thematic books that the Indonesian Government has
created. Their study further showed that pocket books based on local knowledge
are deemed viable due to their validation by media experts, who gave it an 86%
rating in the very-good category, and by teaching-materials experts, who gave it
an 84% rating under good criteria. Additionally, the student reaction of 89% and
the teacher’s assessment/response of 85% both fall into the very-good category.
The conclusions reached are in line with those of Zukmadini et al. (2020), who
found that the pocket book media they created, grounded in local context, led to
a noticeable improvement in students’ tested knowledge.
Data from the study of Thalhah (2020) confirm that the local-knowledge-infused
pocket book learning media holds substantial value as an effective educational
tool. It serves as a facilitator in attaining educational objectives. Moreover, this
media fosters independent student learning by virtue of its compact size. This
ensures accessibility anywhere and anytime, thus aligning seamlessly with the
integration of local wisdom. Furthermore, the research findings by Raibowo et al.
(2023) demonstrate that the developed pencak silat learning media in the form of
pocket books for junior high schools are not only suitable as alternative teaching
materials for PJOK (physical education, sport, and health) teachers but are also
highly practical for students as a learning medium. This is substantiated by the
readability test results, where the combined percentage of the three aspects
reached 81.11%, indicating a classification of “very practical”. These outcomes
143
highlight the successful integration of local wisdom into pocket book media,
affirming their efficacy and user-friendliness within the educational context.
4.3 How Academic Integrity can be Enhanced through the Synergy of Local
Wisdom and Pocket Book Media
Local knowledge and pocket book media work well together to promote academic
integrity. Several important advantages result from incorporating local
knowledge into educational resources using pocket book media. Nevertheless, no
existing research has directly contributed to the advancement of pocket book
learning media employing local wisdom to actively foster academic integrity. As
a result, the present section will concentrate on formulating frameworks and
models that establish a cohesive connection between pocket book media, local
wisdom, and the promotion of academic integrity.
4.3.1 The fundamental value of academic integrity

ICAI (2021) identified five core values as the basis for its definition of academic
integrity – honesty, trust, fairness, respect, and responsibility (Figure 4). The sixth
value of courage has been added as a quality element of adherence to the five core
values even in the face of difficulty (Raman & Ramlogan, 2020). Macfarlane (2014)
described academic integrity as the values, behavior, and conduct of academics in
all aspects of their practice. According to Bretag et al. (2013), academic integrity
should engage all parties involved in teaching and learning, including both
academics (teachers) and students.
Honesty
Courage Trust
Academic
Integrity
Respons
Fairness
ibility
Respect
Figure 4: Fundamental values of academic integrity
Integrity cannot exist without honesty, which is a requirement for the full
realization of trust, fairness, respect, and accountability. Honesty is the
foundation of trust, which is crucial for academic communities to operate
efficiently. Fairness relates to treating people equally and objectively, whereas
respect entails appreciating other people’s viewpoints and treating them with
decency and courtesy. In contrast to courage, which consists of withstanding
harmful peer pressure, and setting a good example, responsibility is where the
144
individual accepts ownership of their acts and holds themself accountable for
them (ICAI, 2021).
4.3.2 Pocket book learning media, local wisdom, and academic integrity framework
The core principles of academic integrity and local wisdom share overlapping
values and ideals that underscore ethical conduct, respect, and the quest for
knowledge (Chaer et al., 2021; Raman & Ramlogan, 2020). Integrating the
principles of local wisdom into pocket book media development will contribute
to creating educational materials that enhance academic integrity, as these
principles harmoniously align and reinforce one another.
The reading and writing abilities of students are greatly enhanced when local
knowledge is incorporated into education. This strategy outperforms traditional
approaches by encouraging quicker comprehension, increased passion, and
autonomous reading engagement (Laila et al., 2021). In order to sustain national
culture and improve academic integrity using pocket book media, educators are
recommended to use local wisdom. The study by Ariyanto et al. (2022) highlights
how combining local wisdom and pocket book media enhances academic
integrity. Teachers can adapt these findings to create effective learning resources
that promote learning outcomes and character education. The research serves as
a guide for developing similar materials across grade levels, fostering ethical
conduct alongside academic excellence. Figure 5 presents the conceptual
framework formulated for pocket media, local wisdom, and academic integrity.
Figure 5: Pocket book media, local wisdom, and academic integrity framework
145
The presented conceptual framework, depicted in Figure 5, offers a visual

representation of the intricate connections between the central concepts explored
in the study. These concepts are the principle of local wisdom, which encompasses
moral and social ethics, and the fundamental values of academic integrity, which
include honesty, trust, fairness, respect, responsibility, and courage. This
framework illustrates the profound interconnection between these two core
concepts, highlighting the symbiotic relationship between academic integrity and
local wisdom. It suggests that aligning pocket book learning media with this
synergy has the potential to promote academic integrity, providing educators
with a guiding framework to create resources that foster a stronger commitment
to genuine learning and ethical behavior. Ultimately, this framework underscores
the importance of integrating cultural and ethical considerations into educational
resource development, offering a foundation for further research and practical
applications in this area.
The findings of Qurrota’aini and Sukirno (2013) support the notion that the pocket
book is an effective teaching tool that can also help students become more
motivated to learn about accounting. This highlights the opportunity to enhance
academic integrity by incorporating local knowledge and pocket book media.
Students are more likely to engage in their learning honestly and develop a feeling
of personal responsibility when cultural insights and educational materials are
combined. A higher commitment to sincere intellectual inquiry and moral
conduct is encouraged by this combination.
The synergy of local wisdom and pocket book media enhances academic integrity
by fostering personal relevance, guiding effective instructional strategies,
facilitating holistic understanding, and promoting positive character
development. This integration encourages students to engage authentically,
discourages unethical practices, nurtures meaningful comprehension, and
cultivates ethical behavior, ultimately contributing to a stronger foundation of
academic integrity (Anggraini & Kusniarti, 2017).
5. Conclusion
This research explored the potential of incorporating local wisdom in the
development of pocket book learning materials, with the overarching objective of
improving academic integrity. Through a qualitative approach involving
documentation, the study revealed a compelling correlation between the
principles underpinning local wisdom and the core tenets of academic integrity.
By seamlessly integrating the essence of local wisdom into pocket book learning
resources, the study identified a pathway to enhance students’ commitment to
upholding academic integrity. Although the study’s scope is limited to articles
published between 2013 and 2023, its contribution is significant, providing
valuable insights into the potential synergies among local knowledge, academic
integrity, and pocket book learning media. The insights gleaned from this research
provide educators with a tangible avenue to infuse integrity-focused concepts into
learning materials spanning diverse educational levels.
146
It is important to realize that this study mainly relied on reviewing information

from existing studies to draw conclusions. This limitation stems from the paucity
of existing research on the topic, necessitating future empirical investigations to
validate our findings and broaden our understanding of the research topic.
Ultimately, this study underscores the potential of harmonizing local wisdom
with educational resources as a promising route towards fostering academic
integrity, thus recognizing and honoring the importance of local traditions within
the academic context. While the approach of integrating local wisdom into
educational materials to enhance academic integrity holds promise, its successful
replication in different educational contexts and regions requires careful
consideration of cultural, ethical, logistical, and practical factors. Adapting the
approach to suit local needs and values is essential for its effectiveness and
sustainability.
6. Recommendations
Educators are urged to adopt a thorough strategy for incorporating local wisdom
into instructional materials by carefully comprehending local beliefs, figuring out
shared moral principles, working with local authorities, and upholding cultural
sensitivity and context relevance. In order to effectively promote academic
integrity while respecting the significance of regional traditions within the
academic context, it is essential to stay up to date on new research, foster an
integrity-oriented culture across diverse educational levels, and acknowledge the
need for ongoing empirical investigations.
Future studies should investigate the effects of local-knowledge integration in

educational materials and how this affects academic integrity across a range of
subject areas, age groups, and educational contexts. They should also compare
strategies, take into account cultural contexts, and develop assessment tools.
7. Suggestions for Future Research

A great idea for future investigation would be to conduct comprehensive studies
to thoroughly assess the long-term impacts of applying pocket book learning
materials enhanced with local wisdom on students’ commitment to academic
integrity over a prolonged period. Researchers would be able to investigate
whether the beneficial effects on academic integrity persist after the first
implementation phase and continue to influence students’ ethical behavior
throughout their academic careers using this longitudinal methodology, which
would provide in-depth insights. Such research could provide insightful
information on the long-term effectiveness of this strategy, illuminating its
potential as a robust method for fostering academic integrity in a range of
educational contexts.
8. Reference
Adewale, S., & Potokri, O. C. (2023). Integrating African women academics in leadership:
A systematic review. Journal of Higher Education Policy and Leadership Studies, 4(3),
53–73. https://doi.org/10.61186/johepal.4.3.53
Ahmed, K. (2020). Academic integrity: Challenges and strategies for Asia and the Middle
East. Accountability in Research, 27(5), 256–270.
https://doi.org/10.1080/08989621.2019.1646646
147
Alba, A. P., Akbar, S., & Nurchasanah, N. (2019). Bahan Ajar Tema Daerah Tempat
Tinggalku Berbasis Kearifan Lokal [Teaching materials for the theme The Area
Where I Live based on local wisdom]. Journal of Education: Theory, Research and
Development, 4(4), 421–426. https://doi.org/10.17977/jptpp.v4i4.12228
Anayat, D. (2019, August 9). What international students should know about academic
integrity. U.S.News & World Report. https://www.usnews.com/education/best-
global-universities/articles/what-international-students-should-know-about-
academic-integrity
Anggraini, P., & Kusniarti, T. (2017). Character and local wisdom-based instructional
model of Bahasa Indonesia in vocational high schools. Journal of Education and
Practice, 8(5), 23–29.
https://www.iiste.org/Journals/index.php/JEP/article/view/35461
Anita, Y., Thahir, A., Komarudin, K., Suherman, S., & Rahmawati, N. D. (2021). Buku saku
digital berbasis STEM: Pengembangan media pembelajaran terhadap
kemampuan pemecahan masalah [STEM-based digital pocketbook: Learning
media development for problem-solving skills]. Mosharafa: Journal of Mathematics
Education, 10(3), 401–412.
https://journal.institutpendidikan.ac.id/index.php/mosharafa/article/view/m
v10n3_06
Ariyanto, H., Utami, S., & Ismaya, E. A. (2022). The development of digital pocketbook
media based on inquiry on plant growth materials elementary school students.
Uniglobal Journal of Social Sciences and Humanities, 1, 9–16.
https://doi.org/10.53797/ujssh.v1sp.2.2022
Awaludin, A., Wibawa, B., & Winarsih, M. (2020). Integral calculus learning using
problem based learning model assisted by hypermedia-based e-book. Jurnal
Pendidikan Indonesia, 9(2), 224–235.
https://doi.org/10.23887/jpi-undiksha.v9i2.23106
Balasubramanian, K., Jaykumar, V., & Fukey, L. N. (2014). A study on “Student preference
towards the use of Edmodo as a learning platform to create responsible learning
environment”. Procedia – Social and Behavioral Sciences, 144, 416–422.
https://doi.org/10.1016/j.sbspro.2014.07.311
Blau, I., Goldberg, S., Friedman, A., & Eshet-Alkalai, Y. (2021). Violation of digital and
analog academic integrity through the eyes of faculty members and students: Do
institutional role and technology change ethical perspectives? Journal of Computing
in Higher Education, 33, 157–187.
https://link.springer.com/article/10.1007/s12528-020-09260-0
Bretag, T., Harper, R., Burton, M., Ellis, C., Newton, P., Rozenberg, P., Saddiqui, S., & van
Haeringen, K. (2019). Contract cheating: A survey of Australian university
students. Studies in Higher Education, 44(11), 1837–1856.
https://doi.org/10.1080/03075079.2018.1462788
Bretag, T. (2013). Challenges in addressing plagiarism in education. PLoS Medicine, 10(12),
e1001574. https://doi.org/10.1371%2Fjournal.pmed.1001574
Bygrave, C., & Aşık, Ö. (2019). Global perspectives on academic integrity. In J. Hoffman,
P. Blessinger, & M. Makhanya (Eds.), Strategies for fostering inclusive classrooms in
higher education: International perspectives on equity and inclusion (Vol. 16; pp. 19–33).
Emerald Publishing Limited.
http://dx.doi.org/10.1108/S2055-364120190000016003
Chaer, M. T., Rochmah, E. Y., & Sukatin, S. (2021). Education based on local wisdom. JIE
(Journal of Islamic Education), 6(2), 145–157. https://doi.org/10.52615/jie.v6i2.216
Christensen H. J., & Eaton, S. E. (2022). Academic misconduct in higher education: Beyond
student cheating. In S. E. Eaton (Ed.), Academic integrity in Canada: An enduring and
148
essential challenge (pp. 81–102). Cham: Springer International Publishing.

https://library.oapen.org/bitstream/handle/20.500.12657/53333/1/978-3-030-
83255-1.pdf#page=99
Click, A. (2012). Issues of plagiarism and academic integrity for second-language students.
MELA Notes, 85, 44–53. https://www.jstor.org/stable/23392491
Denisova-Schmidt, E. (2016). The global challenge of academic integrity. International
Higher Education, (87), 4–6. https://doi.org/10.6017/ihe.2016.87.9494
Dewi, N. A., & Ramadan, Z. H. (2021). Local wisdom-based thematic teaching
materials. Journal of Education Technology, 5(3), 443–451.
https://doi.org/10.23887/jet.v5i3.37439
Dita, P. P. S., Utomo, S., & Sekar, D. A. (2021). Implementation of problem based learning
(PBL) on interactive learning media. Journal of Technology and Humanities, 2(2),
24−30. https://doi.org/10.53797/jthkkss.v2i2.4.2021
Eliezanatalie, S., & Deta, U. A. (2023). Needs analysis of physics learning media integrated
local wisdom. International Journal of Research and Community Empowerment, 1(2),
39–45. https://doi.org/10.58706/ijorce.v1n2.p39-45
El-Faisal, E., & Sulkipani, S. (2016). Pengembangan bahan ajar berbasis muatan lokal pada
mata kuliah Pendidikan Kewarganegaraan. Jurnal Civics: Media Kajian
Kewarganegaraan, 13(2), 113-126. [Development of local content-based teaching
materials in civic education courses]. Civics Journal: Media for Civic Studies, 13(2),
113–126. https://doi.org/10.21831/civics.v13i2.12721
Gallant, T. (2008). Revisiting the past: The historical context of academic integrity. In
T. Gallant (Ed.), Academic integrity in the twenty-first century (pp. 13–31).
Jossey-Bass.
Gularso, D., Purwoko, R. Y., Sujatmiko, W. I. P., & Ingtias, F. A. N. (2023). Developing a
local genius based pocket book for character strengthening elementary school
students. Pegem Journal of Education and Instruction, 13(3), 304–313.
https://doi.org/10.47750/pegegog.13.03.31
Gusenbauer, M., & Haddaway, N. R. (2020). Which academic search systems are suitable
for systematic reviews or meta-analyses? Evaluating retrieval qualities of Google
Scholar, PubMed, and 26 other resources. Research Synthesis Methods, 11(2),
181−217. https://doi.org/10.1002/jrsm.1378
Harun, F., Hairun, Y., Machmud, T., & Alhaddad, I. (2021). Improving students’
mathematical communication skills through interactive online learning media
design. Journal of Technology and Humanities, 2(2), 17–23.
https://doi.org/10.53797/jthkkss.v2i2.3.2021
Hest, Y. A. L., Riyadi, Kamsiyati, S., & Purnamasari, V. (2021). Pengembangan bahan ajar
berbasis muatan lokal keanekaragaman Motif Batik Ngawi sebagai sumber belajar
di Kelas V sekolah dasar [Development of teaching materials based on local
content diversity of Ngawi Batik Motifs as a learning resource in Grade V
elementary school]. Jurnal Basicedu, 5(2), 1060–1066.
https://doi.org/10.31004/basicedu.v5i1.721.
Hidayati, N. A., Waluyo, H. J., & Winarni, R. (2020). Exploring the implementation of local
wisdom-based character education among Indonesian higher education students.
International Journal of Instruction, 13(2), 179–198.
http://dx.doi.org/10.29333/iji.2020.13213a
Hutama, F. S. (2016). Pengembangan bahan ajar IPS berbasis nilai budaya using untuk
siswa sekolah dasar [Development of social studies teaching materials based on
cultural values using Individual Plan of Study (IPS) for elementary school
students]. JPI (Indonesian Education Journal), 5(2), 113–124.
https://doi.org/10.23887/jpi-undiksha.v5i2.8359
149
International Center for Academic Integrity (ICAI). (2021). The fundamental values of
academic integrity (3rd ed.). https://academicintegrity.org/resources/blog/103-
2022/september-2022/385-focusing-on-the-positive-in-academic-integrity-
education
Irwan, I., Taufiq, M. A., & Fernando, R. (2020). A review of the integration of local wisdom
in English language teaching in 5.0 Society Era. Proceeding IAIN Batusangkar, 1(3),
143–148.
https://ojs.iainbatusangkar.ac.id/ojs/index.php/proceedings/article/view/21
71
Jensenius, F. R., Htun, M., Samuels, D. J., Singer, D. A., Lawrence, A., & Chwe, M. (2018).
The benefits and pitfalls of Google Scholar. PS – Political Science and Politics, 51(4),
820–824. https://doi.org/10.1017/S104909651800094X
Kamelia, K. (2019). Using video as media of teaching in English language classroom:
Expressing congratulation and hopes. Utamax: Journal of Ultimate Research and
Trends in Education, 1(1), 34–38. https://doi.org/10.31849/utamax.v1i1.2742
Kaniawati, M., Abidin, Y., & Rakhmayanti, F. (2021). Pengembangan media pembelajaran
augmented reality pocket book dalam pengenalan huruf siswa Kelas I sekolah
dasar: Pengembangan media AR [Development of augmented reality pocket book
learning media in letter recognition for Grade I elementary school students: AR
media development]. Journal of Multiliteracies, 1(2), 77–87.
Karmintoro, Utomo, S., Suad, & Paul, D. C. (2021). The necessity analysis of mind mapping
based pocket book with materials of diversities in Indonesia. Asian Pendidikan,
1(2), 23–30. https://journalarsvot.com/index.php/aspen/article/view/127
Kormasela, D. A., Dawud, D., & Rofi’uddin, A. H. (2020). Pemanfaatan kearifan lokal
Maluku dalam pengembangan bahan ajar menulis teks prosedur untuk siswa
Kelas VII [Utilization of local wisdom of Maluku in the development of teaching
materials for writing procedural texts for Class VII students]. Jurnal Pendidikan:
Teori, Penelitian, dan Pengembangan, 5(8), 1056–1065.
https://doi.org/10.17977/jptpp.v5i8.13872
Laila, A., Budiningsih, C. A., & Syamsi, K. (2021). Textbooks based on local wisdom to
improve reading and writing skills of elementary school students. International
Journal of Evaluation and Research in Education, 10(3), 886–892.
Lestariningsih, N., & Suardiman, S. P. (2017). Pengembangan bahan ajar tematik-integratif
berbasis kearifan lokal untuk meningkatkan karakter peduli dan tanggung jawab
[Development of thematic-integrative teaching materials based on local wisdom
to improve a caring and responsible character]. Jurnal Pendidikan Karakter, 7(1).
https://doi.org/10.21831/jpk.v7i1.15503
Lyesmaya, D., Musthafa, B., & Sunendar, D. (2020, February). Local wisdom value’s-based
literacy education learning model in elementary school. Journal of Physics:
Conference Series, 1470(1), 012030.
https://doi.org/10.1088/1742-6596/1470/1/012030
Macfarlane, B., Zhang, J., & Pun, A. (2014). Academic integrity: A review of the
literature. Studies in Higher Education, 39(2), 339–358.
https://doi.org/10.1080/03075079.2012.709495
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D.,
Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J.,
Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-
Wilson, E., McDonald, S., … Moher, D. (2021). The PRISMA 2020 statement: An
updated guideline for reporting systematic reviews. Systematic Reviews, 10(1), 89.
https://doi.org/10.1186/s13643-021-01626-4
150
Parhan, M., & Dwiputra, D. F. K. (2023). A systematic literature review on local wisdom
actualization in character education to face the disruption era. Journal of Innovation
in Educational and Cultural Research, 4(3), 371–379.
https://doi.org/10.46843/jiecr.v4i3.675
Perkins, M., Gezgin, U. B., & Roe, J. (2020). Reducing plagiarism through academic
misconduct education. International Journal for Educational Integrity, 16(1), 1–15.
https://edintegrity.biomedcentral.com/articles/10.1007/s40979-020-00052-8
Pluye, P., & Hong, Q. N. (2014). Combining the power of stories and the power of
numbers: Mixed methods research and mixed studies reviews. Annual Review of
Public Health, 35(2014), 29–45. https://doi.org/10.1146/annurev-publhealth-
032013-182440
Putranta, H., & Supahar. (2019). Synthesis of the cognitive aspects’ science literacy and
higher order thinking skills (HOTS) in chapter momentum and impulse. Journal of
Physics: Conference Series, 1397(1). https://doi.org/10.1088/1742-
6596/1397/1/012014
Qurrota’aini, S. S., & Sukirno, S. (2013). Pocketbook as media of learning to improve
students’ learning motivation. Jurnal Pendidikan Akuntansi Indonesia, 11(2), 68–75.
https://doi.org/10.21831/jpai.v11i2.1692
Raibowo, S., Prabowo, A., & Permadi, A. (2023). Development of Pencak Silat learning
media in the form of pocket books for junior high schools. Kinestetik: Jurnal Ilmiah
Pendidikan Jasmani, 7(1), 228–233.
https://ejournal.unib.ac.id/kinestetik/article/view/26029
Raman, V., & Ramlogan, S. (2020). Academic integrity and the implementation of the
honour code in the clinical training of undergraduate dental students. International
Journal for Educational Integrity, 16(9), 1–20.
Ramdani, Z. (2018). Construction of academic integrity scale. International Journal of
Research Studies in Psychology, 7(1), 87-97.
https://www.researchgate.net/profile/Zulmi-
Ramdani/publication/323906170_Construction_of_academic_integrity_scale/lin
ks/5ab2700aa6fdcc1bc0c1c37f/Construction-of-academic-integrity-scale.pdf
Renzi, C., Whitaker, K. L., & Wardle, J. (2015). Over-reassurance and undersupport after
a ‘false alarm’: A systematic review of the impact on subsequent cancer symptom
attribution and help seeking. BMJ Open, 5(2), e007002.
http://dx.doi.org/10.1136/bmjopen-2014-007002
Rustan Effendi, Y. (2020). The principal’s transformational leadership approach based on
local wisdom in strengthening the character of students. Malaysian Online Journal
of Educational Management (MOJEM).
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3718268
Samala, A. D., Giatman, M., Simatupang, W., & Ranuharja, F. (2021). E-jobsheet based on
mobile pocket book as digital learning resources (DLRs). Jurnal Teknologi Informasi
dan Pendidikan, 14(2), 117–123. https://doi.org/10.24036/tip.v14i2.488
Shaffril, H. A. M., Samah, A. A., & Samsuddin, S. F. (2021). Guidelines for developing a
systematic literature review for studies related to climate change adaptation.
Environmental Science and Pollution Research, 28, 22265–22277.
Shuhratovich, I. U. (2020). Application of innovation in teaching process. European Journal
of Research and Reflection in Educational Sciences, 8(5), 4–8.
http://www.idpublications.org/wp-content/uploads/2020/05/Full-Paper-
APPLICATION-OF-INNOVATION-IN-TEACHING-PROCESS.pdf
151
Suud, F. M., Sudirman, S. A., Kibtiyah, M., & Putra, R. P. (2022). Local wisdom as the
fundamental for honest behavior of students in Aceh: A social psychology study.
Journal of Pharmaceutical Negative Results, 13(4), 863–869.
https://pnrjournal.com/index.php/home/article/view/2693
Thalhah, S. Z. (2020). Pocket book: Local wisdom-based mathematics learning.
International Journal of Scientific Engineering and Research (IJSER), 11(6).
https://doi.org/10.24252/lp.2020v23n1i12
Tinja, Y., Towaf, S. M., & Hariyono. (2017). Pengembangan bahan ajar tematik berbasis
kearifan lokal sebagai upaya melestarikan nilai budaya pada siswa sekolah dasar
[Development of thematic teaching materials based on local wisdom as an effort
to preserve cultural values in elementary school students]. Jurnal Pendidikan: Teori,
Penelitian, dan Pengembangan, 2(9), 1257–1261.
https://doi.org/10.17977/jptpp.v2i9.9990
Unicheck. (2019, September 25). Academic cheating statistics say there’s lots of work to do.
https://unicheck.com/blog/academic-cheating-statistics
Xiao, Y., & Watson, M. (2019). Guidance on conducting a systematic literature review.
Journal of Planning Education and Research, 39(1), 93–112.
https://journals.sagepub.com/doi/full/10.1177/0739456X17723971
Yusuf, F. A. (2023). Meta-analysis: The influence of local wisdom-based learning media on
the character of students in Indonesia. International Journal of Educational
Methodology, 9(1), 237–248. https://doi.org/10.12973/ijem.9.1.237
Yusuf, M., Mai, M. Y. M., Al Maki, S. H., Shaffeei, K., Folashade, A. T., Bamiro, N. B., &
Shittu, M. A. (2022). A systematic review of pedagogical practices in the Education
4.0. Hong Kong Journal of Social Sciences, (60), 504–514.
https://doi.org/10.55463/hkjss.issn.1021-3619.60.49
Zahra, G., Yulianti, D., & Perdana, R. (2023). Making pocket books based on local wisdom to
increase students’ interest in learning [Conference session]. 4th International
Conference on Progressive Education 2022 (ICOPE 2022) (pp. 544–549). Atlantis
Press. https://doi.org/10.2991/978-2-38476-060-2_50
Zukmadini, A. Y., Kasrina, K., Jumiarni, D., & Rochman, S. (2020). Pocketbook based on
local wisdom and its effectivity in improving students’ knowledge on the
utilization of traditional medicine plants. Biosfer: Jurnal Pendidikan Biologi, 13(1),
59–74. https://doi.org/10.21009/biosferjpb.v13n1.59-74
152

Vol. 22, No. 10, pp. 152-173, October 2023
Teacher Training’s Content and Delivery Method

Related to Augmentative and Alternative
Communication (AAC): A Systematic Literature
Review (SLR)
Sook Wei Loi , Syar Meeze Mohd Rashid* and Hasnah Toran
National University of Malaysia
Selangor, Malaysia
Abstract. Augmentative and Alternative Communication (AAC) is one

of the methods used to assist students with communication problems,
particularly those in special education who suffer from speech delay
and total loss of speech, for example, those with processed autism,
cerebral palsy, or intellectual disabilities. The implementation of AAC in
Malaysia’s schools falls primarily on the shoulders of special education
teachers. However, past research has demonstrated that most special
education teachers do not undergo enough teacher training to equip
themselves with skills and knowledge in the field of AAC. There is scant
literature about teacher training related to AAC as there has been a lack
of guidelines in terms of AAC training for educators. Therefore, a
Systematic Literature Review (SLR) was carried out to identify teacher
training in AAC in terms of content and delivery method so that it can
be used as a reference for an upcoming teacher training course. By using
PRISMA, as a guideline, the Google Scholar and Scopus databases
yielded a total of 18 articles. Findings reveal that previous teacher
preparation programmes did not go into detail about delivery methods,
particularly when it comes to how theory and practical experience are
combined to maximise learning for collaborative skills and AAC-related
information that was becoming more technologically advanced and
online-based. Future studies should examine how each delivery
modality impacts teacher learning in AAC and how to enhance in-field
training to reduce the gap between theory and practice concerns in
teacher preparation.
Keywords: Augmentative and Alternative Communication; teacher

training; special education; course content; course delivery
*
Corresponding author: Syar Meeze Mohd Rashid; cikgumeeze@ukm.edu.my
©Authors
153
1. Introduction
Every child, including those with special needs, deserves a quality education.
However, the learning of students in special education is greatly hindered by
their communication impairments. With the help of Augmentative and
Alternative Communication (AAC), this effect can be reduced. Therefore, for
AAC to be used in the classroom, teachers must be knowledgeable in its
application. One the seventeen Sustainable Development Goals (SDG) of the
United Nations (2015) is targeted to achieve quality education that emphasises
education equality for every child by 2030. These children include not only
typical-developing children but also children with special needs, which are
enacted in the Malaysian Persons with Disabilities Act 2008 (PWD Act, 2014) in
Malaysia. It is clearly stated in the PWD Act 2008 (section 28(1)) that special
needs for children’s right to education should not be exempted. Instead, the way
we teach and communicate should be adjusted accordingly to meet special
needs requirements. Furthermore, the PWD Act also provided in Section 30(3)
that the Malaysian government should prepare special needs children with
infrastructure, such as the use of Augmentative and Alternative Communication
(AAC) for those who have communication problems in order to ease their
learning.
According to the American Speech-Language-Hearing Association (ASHA)
(2016), AAC is a method to support users with limited speech or total loss of
speech. It also involves using one’s body or tools to replace communication via
speech. Generally, AAC can be grouped into two categories: aided AAC and
unaided AAC (Beukelman & Light, 2020). An unaided AAC does not require
tools as it involves the use of body language, sign language, facial expression
and eye gazing to communicate with each other. An aided AAC, on the other
hand, requires the use of tools to support communication with some of the tools
commonly used being tablets, iPads, picture boards, board communication, and
speech-generative devices (SGD).
Previous studies have shown the benefits of using AAC to support the
communication of children with disabilities, particularly those diagnosed with
autism, emotional disturbance, cognitive impairment, cerebral palsy, and more,
as stated in Brock et al. (2017). Overall, the use of AAC has been found to be able
to increase a child’s functional communication and decrease their challenging
behaviours (Ganz & Simpson, 2018; Walker & Snell, 2013). However, AAC can
prove problematic for the service provider, family members, therapists, teachers,
peers, and people surrounding AAC users (Da Fonte et al., 2016) due to the
complexity of its implementation, which involves several processes, such as
carrying out users’ assessments (Andzik et al., 2017; Chung & Stoner, 2016),
choosing the right device (Andzik et al., 2017), monitoring the AAC users’
progress (Andzik et al., 2017; Greene & Esposito, 2023), creating opportunities to
communicate (Greene & Esposito, 2023; Tonsing & Dada, 2016), and occasionally
solving problems related to the device (Tonsing & Dada, 2016). As a result,
previous research has emphasised the importance of supporting AAC users in a
team (Greene & Esposito, 2023). Without team support, AAC users might not
obtain the optimal development of communication even with the aid of AAC.
154
At school, the role of supervising and supporting AAC users greatly falls on the
shoulders of special education. Prior research has stated that some of the
teacher’s roles, are to: a) provide the information needed by speech language
therapists that would affect the decision to choose a suitable AAC for users; b)
create communication opportunities in school; c) help AAC users communicate
with peers; d) make sure AAC users can reach their AAC in the classroom; e)
make sure AAC users can access existing curriculum by using tools or symbols
that are familiar to them; and f) writing goals for an Individualised Education
Plan (IEP) related to AAC (Da Fonte et al., 2016; Leatherman & Wegner, 2022;
Syar Meeze & Wong, 2023; Tonsing & Dada, 2016).
As such, special education teachers are greatly encouraged to undergo training

in AAC to support AAC users at school (Da Fonte et al., 2022). Generally,
teacher training is found to be able to increase teachers’ knowledge, skills, self-
efficacy and self-confidence in supporting users with different communication
disabilities (Mollie & Jeeva, 2022; Syar Meeze & Norfatimah, 2023). Nonetheless,
prior studies have reported that special education teachers are not competent in
providing AAC services (Fields, 2015). This issue is not only mentioned in
relation to Malaysia (Mohd Hanafi et al., 2020) but also reported in Brazil
(Nunes & Walter, 2020), Saudi Arabia (Rashed Aldabas, 2019), and Sri Lanka
(Ketheeswaran, 2019). One of the most common factors reported as leading to
this phenomenon is inadequate training for preservice and in-service teachers; it
was found that special education teachers have minimum access to teacher
professionalism related to AAC (Senner & Baud, 2017; Tonsing & Dada, 2016).
Regarding past teacher training, some researchers found that AAC content was
not being focused on (Eliada et al., 2018). Penington et al. (2020) added that focus
is especially blurry in terms of content in the aspect of communication. Past
studies also showed that, even if a lesson in AAC is conducted, topics in AAC
are always taught together with other subjects without a single subject
specialised for AAC. On the other hand, Penington et al. (2020) surveyed 51
faculty members in teacher training programmes and reported another issue:
that most of the AAC lessons emphasise theory more than practice. As a result,
there is a need to study the curriculum in teacher training programmes about
AAC for the sake of future professional training (Oihana & Maria, 2020).
Douglas et al. (2020) also suggested that AAC content and its delivery format
should be focused on AAC training.
This empirical article is considered necessary to fill the knowledge gap about
AAC content and its method of delivery in teacher preparation. An article is
chosen to be included in an SLR because it enables a systematic review of prior
literature. SLR also enables researchers to consistently evaluate the most recent
data at the same time (Hayrol et al., 2020). The objective of this SLR is to identify
teacher training content and delivery method in AAC. Concurrently, this
research sought answers as to what are the content and delivery methods of
teacher training in AAC.
2. Methodology
This SLR is guided by the Preferred Reporting Items for Systematic Reviews and
Meta-Analyses (PRISMA). PRISMA has three unique characteristics: a) it can
155
define research questions clearly for systematic reviews; b) it can identify

exclusive and inclusive criteria; and c) it can enable research access from various
databases in each timeline (Sierra-Correa & Cantera Kintz, 2015). Therefore,
PRISMA is seen as a good guideline for this SLR to identify the content and
delivery method of teacher training in AAC for data analysis.
2.1 Article Search Strategy
Two databases were used to search for related articles regarding teacher training
in AAC: Scopus and Generic Web Searches (Google Scholar). Scopus was chosen
as one of the databases as it has good quality control on its articles. Furthermore,
Scopus also has more than 13 million articles from 5000 publishers globally,
which enables a wide range of articles to be included in the database whereas
Google Scholar was chosen to be a supportive database. According to
Haddaway et al. (2015), Google Scholar can be used to support systematic
review research. In addition to that, Google Scholar also has as many as 165
million articles, which enables researchers to search through related articles
from different journal sites.
Joklitschke et al. (2018) stated that keywords used during the process of
searching for articles are very important. The keywords in this article were
selected based on a) referring to earlier research about AAC and teacher
training; b) similar words that bring out the same meaning via thesaurus; and c)
suggested keywords by databases used. The chosen keywords for this SLR were
augmentative and alternative communication, teacher training, special
education, course content, and course delivery. During the article search process,
the researchers also used full-string techniques, such as Boolean operators and
phrase searching, that are available in Scopus and Google Databases. Boolean
operator means the use of ‘OR’, ‘AND’, and ‘NOT’ to limit articles found, while
phrase searching means finding articles that have the same phrase. Details of the
search string are shown in Table 1. Apart from using keywords, researchers also
used the ‘reference tracking’ technique, which is to select related articles found
through article references.
Table 1: Search string for the articles
Database Keyword
Scopus TITLE-ABS-KEY (“augmentative and alternative communication" AND
"teacher" OR "special education" OR "Preservice teacher" OR "in-service
teacher" AND "training" OR "course" OR "content" OR "delivery”)
Google "Augmentative and alternative teacher training" OR "Augmentative and
Scholar alternative communication course" OR "Augmentative and alternative
communication content" OR "Augmentative and alternative
communication delivery"
2.2 Article Selection Criteria
The process of choosing suitable articles was guided by inclusive and exclusive
criteria set beforehand. For this SLR, only articles within the last ten years (2013-
2023) were included. In addition, the articles chosen were limited to English
papers and journal articles only. However, it is worth mentioning that articles
were acceptable if training involved teachers as one of the targeted populations.
Articles that included literature, findings, or analysis regarding the content and
delivery method of teacher training qualified for inclusion. On the other hand,
156
exclusion criteria were reference materials other than journal articles that were
not in English and were published before 2013. Furthermore, articles were also
excluded if they mentioned training in AAC but did not include special
education teachers as one of the targeted populations. Details of inclusive and
exclusive criteria are shown in Table 2.
Table 2: Inclusive and Exclusive Criteria
Criterion Inclusive Exclusive
Year of publication Publication within the last ten years Publication before 2013
(2013-2023)
Language English Language other than English
Types of reference Journal articles Theses, conference papers,
materials books, and proceedings
Focus Articles that included literature, Articles that do not discuss
findings, or analyses regarding the teacher training for special
content and delivery method of education teachers
teacher training
2.3 Article Selection Process
The article selection process began in July 2023. The article selection process is
adapted from Tawfik et al. (2019) and is shown in Figure 1.
Identification
Records identified through

database searches:
Scopus = 118; Records excluded based on
Google Scholar=41 exclusion criteria:
(n=159) 1. Published before the year 2013;
2. Not in an article journal;
3. Not in English
Scopus=60; Google Scholar=22
Screening
Records screened
Scopus= 58; (n=82)
Google Scholar=19
(n=77)
1. Article that includes training but
does not focus on special education
Articles that meet all four teachers
Eligibility
eligibility criteria: Scopus=12; Google Scholar=4

Scopus=9; 2. Article that is related but not
Google Scholar=1 accessible
(n=10) Scopus=4; Google Scholar=2
4. Articles that do not meet the
criteria
Articles used for the analysis Scopus=33; Google Scholar=12
of the study: (n=67)
Included
Scopus=9;
Google Scholar=1;
reference tracking=8
(n=18)
Figure 1: Flowchart of the Article Selection Process
157
Based on PRISMA, the collection of articles is divided into four levels: The level
of identification of articles, followed by screening articles using exclusion and
inclusion criteria set beforehand. Articles that passed the screening process
were reviewed in more detail to determine its eligibility for the SLR. Lastly,
articles that fulfilled all the criteria were included in the analysis. For this SLR,
159 articles were identified by searching the Scopus and Google Scholar
databases. However, after initial screening, the number of remaining articles was
77. Upon checking the title, abstract, and whole text, 67 articles were eliminated
as most of the articles did not provide teacher training content and delivery
methods as expected. However, after going through reference tracking, the
researchers were able to extract a further eight articles that fulfilled the inclusion
criteria from different databases, such as the Journal of Augmentative and
Alternative Communication, Sage Journal, ASHA, and Journal of International
Special Needs Education. Overall, there were 18 articles eligible to be analysed
in this SLR. Table 3 shows the selected article titles, author, year, country,
journal, and purpose of the study.
2.4 Quality Assessment

A quality assessment was done by two experts upon the selection of articles. If
there was disagreement, a third expert was involved until a consensus was
agreed upon. This method is according to Wu et al. (2018).
2.5 Data Analysis

In total, 18 articles were selected using the PRISMA methods related to teacher
training content and delivery methods in AAC. In this SLR, the researcher chose
the integrative review method to analyse the articles collected. An integrative
review is one of the methods used to analyse research from different
methodologies. This is important as the researcher did not include methodology
as one of the inclusion or exclusion criteria during the article selection process.
Thus, the articles collected are from different methodologies. According to
Whitemore and Knafl (2005), an integrative review can be done by using
qualitative quantitative data or quantifying qualitative data. In this case,
researchers chose to analyse all data qualitatively by using thematic analysis.
Flemming et al. (2018) stated that the use of thematic analysis is fit for
integrative-based research. The process of thematic analysis was done by
detecting similarities among the articles, identifying themes and patterns,
creating categories, and identifying relationships that exist between the data
(Braun & Clarke, 2006).
158
Table 3: List of Selective Articles

No Author(s) and Country Study Title Journal Name Study Purpose
year of
publication
1 Muttiah et al. United Providing instructional support for AAC International Journal To investigate the professionals’
(2015) States service delivery in low- and middle- of Speech-Language experiences in augmentative and
income (LAMI) countries Pathology alternative communication (AAC)
2 Da Fonte et al. United Recommended Augmentative and Journal of To outline key training areas and
(2016) States Alternative Communication Competencies International Special implications regarding the need for future
for Special Education Teachers Needs Education research in AAC
3 Chung & United A meta-synthesis of team members’ voices: Augmentative and To review qualitative studies about the
Stoner (2016) States what we need and what we do to support Alternative perspectives of team members who support
students who use AAC Communication students in using AAC
4 Tonsing & Africa Teachers’ perceptions of the Augmentative and to explore how aided AAC was
Dada (2016) implementation of aided AAC to support Alternative implemented to increase students’
expressive communication in South Communication expressive communication in preschool.
African special schools: a pilot
investigation
5 Andzik et al. United AAC services in schools: a special International Journal To investigate the perspectives of special
(2017) States educator’s perspective of Developmental education teachers on supporting students
Disabilities with AAC
6 Wallis et al. United Augmentative and alternative Journal of Enabling To analyse training of augmentative and
(2017) Kingdom communication (AAC) training provision Technologies alternative communication (AAC) by
for professionals in England clinical services in England.
7 Andzik et al. United Exploring Relationships Between Teacher Journal of 1. To determine factors affecting the use of
(2018a) States Training and Support Strategies for International Special AAC from the aspects of intervention and
Students Utilizing Augmentative and Needs Education supports
Alternate Communication 2. To examine relationships between
teacher training and support given in the
classroom.
159

year of
publication
3. To examine relationships between
proficient communication, teacher training,
and the quantity of support.
8 Andzik et al. United A national survey describing and Developmental To study how special education teachers
(2018b) States quantifying students with communication Neurorehabilitation support students with communication
needs needs in the classroom
9 Rashed Saudi Barriers and facilitators of using International Journal To investigate barriers and facilitators in
Aldabas (2019) Arabia augmentative and alternative of Inclusive Education using AAC from the perspectives of special
communication with students with education teachers
multiple disabilities in inclusive education:
special education teachers’ perspectives
10 Douglas et al. United The Training Experiences of Augmentative American Speech- To explore the experiences of AAC
(2020) States and Alternative Communication Language-Hearing practitioners in AAC
Practitioners in One Midwestern State Association
11 Oihana & Spain The use of augmentative and alternative The European Journal To gather information about the service of
Maria (2020) communication in educational settings in of Special Needs AAC in Spain
the Basque Autonomous Community Education
(Spain)
12 Pennington et United Teacher Preparation in Communication Sage Journal To investigate the teacher’s training in
al. (2020) States Instruction for Students with Extensive communication for students with extensive
Support Needs needs
13 McCoy & United Effects of Online Training on Educators’ Journal of Behavioral To evaluate online training on teacher
McNaughton States Knowledge and Use of System of Least Education knowledge and use of the system of least
(2021) Prompts to Support Augmentative prompts (SLP) for people with autism who
and Alternative Communication use augmentative and alternative
communication (AAC)
160

year of
publication
14 Da Fonte et al. United Current preparation status in AAC: Augmentative and to examine special education teachers'
(2022) States perspectives of special education teachers Alternative knowledge and skills in AAC
in the United States Communication
15 Walker et al. United Preservice Teachers’ Preparation in Research and Practice To explore the readiness of special
(2022) States Communication Instruction for Students for Persons with education teachers to provide
with Extensive Support Needs Severe Disabilities communication instruction for students
with extensive needs
16 Leatherman & United Augmentative and Alternative American Speech- To explore teacher experience in
Wegner (2022) States Communication in the Classroom: Teacher Language-Hearing supporting students who use speech-
Practices and Experiences Association generating devices (SGDs)
17 Caron et al. United Online Training to Improve Service American Speech- To teach service providers how to use
(2022) States Provider Implementation of Letter–Sound Language-Hearing letter-sound correspondence (LSC)
Correspondence Instruction for Association instruction, those who use AAC
Individuals Who Use Augmentative and
Alternative Communication
18 Greene & United Interprofessional Practice Aligns with New The Journal of Special To share a collaborative project between a
Esposito States California Teacher Performance Education teacher training program and the speech-
(2023) Expectations for Students with Complex Apprenticeship language pathology division
Communication Needs
161
3. Findings
The analysis of the articles produced three main themes and 16 subthemes related to teacher
training in AAC. Specifically, the main themes created are 1) teacher training course delivery
with four sub-themes), 2) teacher training content with 12 sub-themes), and 3) training
activities. Details of the theme and sub-themes are shown below.
3.1 Teacher Training Delivery Method
Teacher training delivery method refers to how AAC information is distributed to the
participants of AAC training (Douglas et al., 2020). This theme produced four sub-themes:
university coursework, self-training, training by SLPs, and other training.
3.1.1 University Coursework
Of 18 articles, five mention training at the university, namely: Andzik et al. (2018a 2018b),
Oihana and Maria (2020), Da Fonte et al. (2022), and Greene and Esposito (2023). Most of the
participants reported owning a degree from a university, but only a few of them reported
taking courses related to AAC (Oihana & Maria, 2020). The same finding was yielded in Da
Fonte et al.’s (2022) research, in which participants who undergo teacher preparation courses
process the lowest level of knowledge and skills in AAC. However, Andzik et al. (2018a), in
their finding on the relationship between teacher training and support strategies, found that
AAC course delivery at the university level is associated with the use of sign language,
Voice Output Devices (VOD), and picture-based systems.
3.1.2 Self-Training
Self-training is mentioned in four articles: Andzik et al. (2017, 2018b), Douglas et al. (2020),
and Leatherman and Wegner (2022). Participants in the research mostly chose self-training
due to the lack of training provided in the workplace and SLP (Andzik et al., 2017); some
expressed their interest in AAC and, therefore, took the initiative of searching for more
information about AAC themselves (Leatherman & Wegner, 2022). Commonly, self-training
methods reported are using online platforms, which are Facebook, Google, Pinterest, Blogs,
manufacturer websites and research papers.
3.1.3 Training by SLP
Training by SLPs is mentioned in four articles: Chung and Stoner (2016) and Andzik et al.
(2017,2018a, 2018b). Internal training that involves training from SLPs towards teachers is
the most frequent type of training reported by participants in Chung and Stoner (2016). The
content of training given by SLPs is reported to be AAC systems, such the eye gaze system,
the use of PowerPoint, and Pragmatic Organisation Dynamic Display (PODD), as reported
by Andzik et al. (2017). Subsequently, Andzik et al. (2018b) drew out the relationship
between learning sign language, VOD, and picture-based systems with SLR training. From
the limited literature, it seems that training by SLPs is usually more focused on educating
teachers about the use of an AAC system.
3.1.4 Other Training
Other trainings mentioned are a) professional development courses (Andzik et al., 2018a,
2018b); b) conferences (Douglas et al., 2020; Oihana & Maria, 2020; Leatherman & Wegner,
2022); c) seminars (Oihana & Maria, 2020); d) workshops (Chung & Stoner, 2016;
Leatherman & Wegner, 2022; Muttiah et al., 2015; Oihana & Maria, 2020); e) academic
congresses (Oihana & Maria, 2020); f) training by an AAC special consultant (Andzik et al.,
2018a,2018b; Chung & Stoner, 2016); and g) training by a service provider or parents
(Andzik et al., 2018a 2018b; Chung & Stoner, 2016; Douglas et al., 2020). All these course
deliveries are under the section ‘other training’, as little information can be found via articles
to further explain them.
162
3.2 Teacher Training Content

Teacher training content refers to the topics that are relevant to AAC and should be included
in the training. This theme produced twelve sub-themes: aided AAC, AAC assessment,
collaboration in AAC, arranging the environment to promote communication or facilitate
AAC, core or fringe vocabulary within AAC, assistive technology, implementation of AAC,
cultural aspects of AAC communication, language development, unaided AAC, and other
content. The list of AAC content that can be included in teacher training is shown in Table 4
(infra), whereby the list of content is arranged in descending order according to the
frequency of articles mentioned about the same topics.
3.2.1 Aided AAC
Aided AAC was mentioned in 12 articles: namely: Muttiah et al. (2015), Chung and Stoner
(2016), Da Fonte et al. (2016), Tonsing & Dada (2016), Andzik et al. (2017), Wallis et al. (2017),
Rashed Aldabas (2019), Douglas et al. (2020), Pennington et al. (2020), Walker et al. (2020);
Da Fonte et al. (2022),and Leatherman and Esposito (2023). Research has emphasised
training that includes AAC from low technology to high technology (Da Fonte et al., 2016;
Greene & Esposito, 2023; Rashed Aldabas, 2019). According to Da Fonte et al. (2016),
examples of low-tech systems include visual support tools, communication boards,
behaviour support boards, token reward systems, schedules, picture boards, and so on.
While high tech often refers to devices that have sound, which are speech-generating
devices (Douglas et al., 2020), earlier research has shown a preference towards aided AAC
over unaided AAC (Tonsing & Dada, 2016). This is due to the benefits of using aided AAC,
which is simpler to use (Tonsing & Dada, 2016), offers speech output (Tonsing & Dada,
2016), is versatile, and is customisable (Chung & Stoner, 2016).
3.2.2 AAC Assessment
There are eleven articles mentioning AAC assessment, namely: Muttiah et al. (2015), Da
Fonte et al. (2016), Andzik et al. (2017), Wallis et al. (2017), Rashed Aldabas (2019), Douglas
et al. (2020), Pennington et al. (2020), Da Fonte et al. (2022), Leatherman and Wegner (2022),
Walker et al. (2022), and Greene and Esposito (2023). Pennington et al. (2020) and Walker et
al. (2022) whose research discussed the assessment of communication skills and needs, while
the other researchers focused on AAC assessment. Previously, participants were found to
have the least knowledge and skills in AAC assessment compared to other skills (Da Fonte,
2022). Generally, AAC assessment identifies users who need AAC and finds a suitable
device or intervention by observing their physical, cognitive, and sensory needs.
Additionally, assessment can also be used to monitor users’ progress (Andzik et al., 2017).
Research finds the need for teachers to know about AAC assessment as SLPs are often not
available (Andzik et al., 2017). Furthermore, teachers should learn how to do
communication assessments and choose response forms to carry out the assessment
(Peckham-Hardin et al., 2018).
3.2.3 Collaboration in AAC
Eight articles mention the collaboration in AAC, namely: Chung and Stoner (2016), Da Fonte
et al. (2016), Andzik et al. (2017), Wallis et al. (2017), Rashed Aldabas (2019), Douglas et al.
(2020), Leatherman and Wegner (2022), and Greene and Esposito (2023). Da Fonte et al.
(2016) and Douglas et al. (2020) emphasised the importance of working in a team to support
AAC users in all aspects. However, prior research has reported a lack of training in
collaborative work (Andzik et al., 2017). Past studies stated that teacher training about
collaboration should focus on helping teachers gain a) the value of collaboration (Chung &
Stoner, 2016); b) respecting all team members (Chung & Stoner, 2016); c) collaboration skills;
and d) different team models like multidisciplinary, interdisciplinary, and transdisciplinary
163
(Da Fonte et al., 2016; Douglas et al., 2020) and writing goals for AAC users (Da Fonte et al.,
2016; Douglas et al., 2020).
3.2.4 Arranging an Environment to Promote Communication or Facilitate AAC
There are eight articles which mentioned about arranging an environment to promote
communication, or, in other words, to create opportunities for AAC users to communicate:
Muttiah et al. (2015), Da Fonte et al. (2016), Tonsing and Dada (2016), Wallis et al. (2017),
Rashed Aldabas (2019), Pennington et al. (2020), Walker et al. (2022), and Greene and
Esposito (2023). It is important to create opportunities for communication via users’ daily life
activities so that students can become familiar with their devices and know how and when
to communicate (Greene & Esposito, 2023). However, prior study has shown that teachers
lack skill and training in creating an environment for communication (McCoy &
McNaughton, 2021; Tonsing & Dada, 2016; Wallis et al., 2017). According to Da Fonte (2016),
teacher training should focus on how to: a) identify activities in which AAC users may
participate; b) prepare tools to communicate; c) show AAC users how to communicate using
the system; d) create chances for communication to occur; e) encourage AAC users to
communicate without a communication partner; and f) search for opportunities to increase
participation. Throughout the process of creating communication opportunities, teachers
should also consider applying AAC instructional strategies such as prompting, naturalistic
teaching strategies, wait times, and so on to create more communication opportunities (Da
Fonte et al., 2016; McCoy & McNaughton et al., 2022).
3.2.5 Core and Fringe Vocabulary Within AAC
Seven articles mentioned about vocabulary in AAC, namely: Muttiah et al. (2015), Da Fonte
et al. (2016), Tonsing and Dada (2016), Wallis et al. (2017), Douglas et al. (2020), Walker et al.
(2022), and Greene and Esposito (2023). It is important to learn about the core and functional
words, as they are needed to display on the device (Greene & Esposito, 2023). Moreover,
choosing suitable and correct vocabularies allows AAC users to use them in different
contexts and be able to communicate smoothly (Da Fonte et al., 2016). Tonsing and Dada
(2016) added that the involvement of the teacher in selecting vocabulary gives the teacher a
sense of ownership in not only implementation but also customisation. Thus, teacher
training should focus on teaching how to assess users’ vocabulary needs and how to select
appropriate vocabulary for each different event and context (Da Fonte et al., 2016).
3.2.6 Assistive Technology
Six articles mentioned assistive technology: Chung and Stoner (2016), Da Fonte et al. (2016),
Tonsing and Dada (2016), Andzik et al. (2017), Wallis et al. (2017), and McCoy and
McNaughton (2021). Two countries have mandated the use of assistive technology, also
called aided AAC, for children with disabilities: Africa, as reported by Tonsing and Dada
(2016), and the United States, as reported by Andzik et al. (2017). There are a few
suggestions for what to include in assistive technology training, among them a) learning the
differences between assistive technology and AAC (Da Fonte et al., 2016); b) roles and
functions of each system (Da Fonte et al., 2016); c) AAC technology preparation, adaptation
and implementation (Wallis et al., 2017); and d) management of resources (Wallis et al.,
2017). It is worth mentioning that assistive technology is a larger collection of tools, systems,
or equipment that is used to improve disabilities in every aspect, including communication.
On the contrary, AAC refers to a system used specifically to improve communication. Often,
it can be observed that assistive technology and AAC can be combined to make better
progress for AAC users. Therefore, teachers need to know every system that exists to better
suit students’ requirements. At the same time, teachers must gain knowledge on how to
operate and solve technical issues such as device programming, troubleshooting, and
backing up the AAC system (Douglas et al., 2020; Leatherman & Wegner, 2022).
164
3.2.7 Implementation of AAC

AAC implementation is mentioned in articles by Da Fonte et al. (2016), Tonsing and Dada
(2016), Andzik et al. (2017), Wallis et al. (2017), Douglas et al. (2020), and Leatherman and
Wegner (2022). Research found common barriers to AAC implementation, which are lack of
time to prepare AAC; low team support and collaboration; lack of support from SLPs and
paraeducators; and lack of teacher training in this area (Andzik et al., 2017; Tonsing & Dada,
2016). Furthermore, Douglas et al. (2020) stated in their research that funding and district
policy were two of the reasons that impeded AAC implementation. The role of a special
education teacher in the implementation of AAC is to help the AAC user with the system
and monitor and evaluate students’ progress from time to time (Da Fonte et al., 2016). The
literature strongly expresses the need for teachers to be able to cooperate and receive
training from SLPs so that AAC can be implemented successfully in schools (Andzik et al.,
2019; Leatherman & Wegner, 2022).
3.2.8 Cultural
Past literature has mentioned the need for teachers to learn about cultural competencies
(Greene & Esposito, 2023; Muttiah et al., 2015; Pennington et al., 2020; Tonsing & Dada, 2016;
Walker et al., 2022; Wallis et al., 2017). Generally, most students are from different
backgrounds and may process different cultures; therefore, special education teachers must
take into consideration the student's cultural backgrounds before making any decisions
regarding AAC (Greene & Esposito, 2023; Pennington, 2020). To do that, teachers must first
understand the issue of having different cultures and languages used (Mollie & Jeeva, 2022);
second, they need to be taught how culture can affect the choice of intervention and
preferences (Bauce, 2014). Later, teachers must be taught how to provide instruction by
considering user gender, language, background and age (Muttiah et al., 2015).
3.2.9 Language
Six articles mentioned language to be included in AAC training, namely: Wallis et al. (2017),
Douglas et al. (2020), Pennington et al. (2020), Caron et al. (2022), Walker et al. (2022), and
Greene and Esposito (2023). Specific topics mentioned regarding language are a) aided
language stimulation (Pennington, 2020; Walker et al., 2022); b) pre-linguistic features such
as joint attention, turn-taking and gesture (Greene & Esposito, 2023); c) forms of non-verbal
communication like receptive and expressive language (Greene & Esposito, 2023); language
development (Wallis et al., 2017); d) early language development and semantic features
(Douglas et al., 2020; Greene & Esposito, 2023) and e) letter sounds corresponding
instruction (Caron et al. 2022). Douglas et al. (2020) mentioned that language development is
important in doing assessments for AAC users and, at the same time, to understand the
usage of a device. After all, it was said that the goal of using a device is to teach language for
communication.
3.2.10 Communication
Five articles mentioned the need to include communication content in AAC training,
namely: Muttiah et al. (2015), Da Fonte et al. (2016), Wallis et al. (2017), Pennington et al.
(2020), and Walker et al. (2022). Topics that have been suggested regarding communication
are foundation knowledge of communication (Muttiah et al., 2015); communication skill
development and milestones (Da Fonte et al., 2016; Greene & Esposito, 2023); identifying
appropriate communication systems (Pennington et al., 2020; Walker et al., 2022); functional
communication training like teaching how to comment, label, request, start conversation,
protest, reject and responding to others (Pennington et al., 2020; Walker et al., 2022; Wallis et
al., 2017); communication strategies (Wallis et al., 2017); communication competencies that
are divided into linguistic, operational, social and strategic (Da Fonte et al., 2016); and
teaching how to communicate with peers (Pennington et al., 2020; Walker et al., 2022). The
165
teacher needs to be able to recognise a child’s communication before introducing a suitable

AAC for them. Therefore, communication is listed as the first competency that teachers need
to master according to the teacher competency in AAC by Da Fonte et al. (2016).
3.2.11 Unaided AAC
Unaided AAC is mentioned by Tonsing and Dada (2016) and Oihana and Maria (2020).
Previous investigations showed that the usage of unaided systems is much lower than that
of aided communication systems (Oihana & Maria, 2020). Among aided systems, sign
language was reported as being used most frequently by AAC users compared to other
systems like scanning and eye gazing. Even though unaided AAC is not being focused on in
training, it is still knowledge that needs to be known by every teacher so that they can
differentiate between aided and unaided AAC. In certain conditions, aided and unaided
AAC can be combined to assist in communication (Beukelman & Pat, 2013).
3.2.12 Other Content
Other AAC content mentioned is a) system and symbol (Da Fonte et al., 2016; Wallis et al.,
2017); b) problem-solving (Andzik et al., 2017; Wallis et al., 2017); and c) research and
practice (Wallis et al., 2017). These three topics of AAC are included under ‘other content’ as
there is little elaboration in the articles; however, the topics are mentioned by researchers as
having to be considered in teacher training courses.
3.3 AAC Training Activities
Six articles mentioned about activities for teacher training. Those activities are: a) reading
(Greene & Esposito, 2023; Walker et al., 2022); b) in-class practice, like discussion (Greene &
Esposito, 2023; Muttiah et al., 2015; Walker et al., 2022); c) role play (McCoy & McNaughton,
2021; Muttiah et al., 2015; Pennington et al., 2020); d) field practice (Da Fonte et al., 2022;
Walker et al., 2022); e) engaging in instructional activities (Greene & Esposito, 2023; Muttiah
et al., 2015); f) case studies (Greene & Esposito, 2023; Muttiah et al., 2015; Pennington et al.,
2020); g) assignment (Greene & Esposito, 2023); h) video (Pennington et al., 2020; Walker et
al., 2022); and i) lectures (Walker et al., 2022). It is suggested that students read articles
regarding evidence-based research about AAC (Greene & Esposito, 2023). in addition, it is
useful to discuss in groups to share experiences, concerns, and plans. (Greene & Esposito,
2023).
Table 4: List of Contents
No Suggested content Suggested Subtopic Number of
articles
1 Aided AAC Low tech 12
High tech
2 AAC Assessment Recognise people who benefited from AAC 11
Communication assessment
Selecting response forms
3 Collaboration in AAC Value of collaboration 8
Respecting all team members
Collaboration skills
Different team models, like
multidisciplinary, interdisciplinary, and
transdisciplinary.
Writing goals for AAC users
4 Arranging an environment to Identify activities in which AAC users may 8
promote communication or participate
facilitate AAC Prepare tools to communicate
Show AAC users how to communicate using
166
No Suggested content Suggested Subtopic Number of

articles
the systems
Create chances for communication to occur
and increase participation
Encourage AAC users to communicate
without a communication partner
AAC instructional strategies
5 Core and fringe vocabulary Assessing vocabulary needs 7
within AAC displays Selection of vocabularies
6 Assistive Technology Learning the differences between AT and 6
AAC
Roles and functions of each system
AAC technology preparation, adaptation,
and implementation
Management of resources
Technical skills within AAC
7 AAC implementation Operational knowledge of AAC 6
Monitor student progress
Evaluate student progress
8 Cultural Understand the issue of multicultural and 6
multilingual
How can culture affect the choice of
intervention and preferences?
Provide instruction by considering the user’s
cultural diversity
9 Language development Aided language stimulation 6
Pre-linguistic features such as joint attention,
turn-taking, and gesture
Forms of non-verbal communication like
receptive and expressive language
Language development
Early language development and semantic
features
10 Communication Foundation knowledge of communication 5

Communication skill development and
milestones
Identify appropriate communication systems
Functional communication training
Communication strategies
Communication competencies are divided
into linguistic, operational, social, and
strategic
Communication with peers
11 Unaided AAC Sign language 2
12 Others System and symbol 2
Problem-solving 2
Research and practice 1
167
4. Discussion
Past research has reported a lack of teacher training in AAC (Andzik et al., 2017; Douglas et
al., 2020). There is a struggle for special education teachers, as most of the student
instruction and AAC support at school is dependent on the special education teacher
(Andzik et al., 2018a; Barker et al., 2013). Besides lack of training, another main issue
reported is the quality of the AAC course, both in terms of teacher preparation and
professional development. Some of the issues associated with course quality mentioned by
Pennington et al. (2020) are: a) the AAC course offered is on a surface level; b) topics of
communication are not being focused on in the course; c) the AAC courses are not connected;
and d) theory learning is more focused than practice. The drawback of teacher training
reported greatly impacted teachers' service towards children with disabilities who use AAC
in school.
A special education teacher greatly needs proper training in AAC to be equipped with the
necessary knowledge and skills to tackle a variety of communication issues at school. While
looking back to previous studies, it was found that there is a lack of analysis of teacher
training components in AAC and the delivery mode of AAC for special education teachers
(Douglas et al. 2020; Wallis et al., 2017). These recommendations from the above inputs are
crucial for future teacher training advice.
Overall, there are 12 topics successfully covered in this SLR. These include aided AAC, AAC
assessment, collaboration in AAC, arranging the environment to promote communication,
core, or fringe vocabulary with AAC display, assistive technology, AAC implementation,
cultural, communication, language development, unaided AAC, and others. It was found
that the content of AAC is about the same as suggested in previous teacher training. When
comparing the previous nature of teacher training by McConachie and Pennington (1997),
Lebel et al. (2005), and Patel and Khamis-Dakwar (2005), there are similarities in the content
suggested, such as implementation of AAC, creating communication opportunities,
understanding communication, selecting vocabularies, and others. However, researchers
have noticed several matters that led to changes in the content of AAC’s focus on teacher
training in recent years.
Firstly, it is found that the content focus is switching to technology-based. Aided AAC is
highlighted in almost every article, whereas unaided AAC is not. It is also worth mentioning
that aided AAC preferences are more towards high-tech tools such as speech-generative
devices, communication apps, and software (Oihana & Maria, 2020). This is because a wider
variety of mainstream technology is being employed to offer AAC options (Wallis et al.,
2017). Therefore, it is not surprising to see teacher training on AAC content pay more
attention to assistive technology (Andzik et al., 2017; Chung & Stoner, 2016; Da Fonte et al.,
2016; McCoy & McNaughton, 2021; Tonsing & Dada, 2016; Wallis et al., 2017). However,
with the rise of high-tech AAC tools, AAC content should also cover the subject of operating
tools so that teachers are familiar with the products. Several articles also emphasise that
teachers should always be up to date with the newest product and receive specific product
training from the service provider to fully understand the product (Wallis et al., 2017).
Moreover, special education teachers should also be equipped with technical skills to solve
problems that might arise from time to time (Douglas et al., 2020; Leatherman & Wegner,
2022).
Secondly, current teacher training is more focused on collaborative skills, as the words
collaboration or team are mentioned many times in the articles. Collaboration, or working
in teams, is one of the factors that determine the success of AAC implementation (Da Fonte
et al., 2016). Collaboration can happen among a lot of people in a school setting; it can be a
168
collaboration between teachers and students, administrators, therapists, family members, or

any other community member that is related to their service. Upon analysis, it was found
that collaboration does not occur as expected due to a lack of skills, a lack of training, and a
lack of willingness to collaborate (Tonsing & Dada, 2016). For example, a lack of
collaboration between SLP and family was reported by Rashed Aldabas in 2019, and a lack
of collaboration between teacher and students was reported by Stoner et al. (2010), Chung
and Stoner (2016), and Radici et al. (2018). Collaboration has long been mentioned in the
literature, but it has not been fully implemented in the service yet due to a lack of training
(Andzik et al., 2017). Previous research has urged teacher training to include more
collaborative skills by increasing in-field practice so that teachers’ abilities in collaboration
can be increased (Greene & Esposito, 2023; Walker et al., 2022). However, there is a need for
more research suggesting how teacher training can prepare special education teachers to be
ready in this area.
Third, the content of AAC might vary between countries because of differences in culture,
environment, education, individual needs and finances (Rashed Aldabas, 2019; Subihi,
2013). Taking a low-income country as an example, aided AAC with high technology is not
applicable when resources are scarce (Muttiah et al., 2015). Therefore, the focus on teacher
training in those countries will be different compared to other developing countries.
Furthermore, for those countries that have multicultural and multilingual issues, such as
Malaysia (Joginder Singh et al., 2020), the usage of AAC tools and implementation might
vary too. In sum, consideration of courses should be made based on learner needs and
wants before planning for teacher training (Muttiah et al. 2015).
As for the teacher training delivery format, it is interesting to note that there is some kind of
relationship between the amount of teacher training and the knowledge and skill processes
of the teacher. According to Rashed Adabas (2019), participants who have ever attended
teacher training show knowledge and skills that are better than those who have not. This is
supported by Andzik et al. (2018a), who stated that teachers with a great deal of training use
AAC support strategies more than those who received less training. By further analysing the
relationship between types of teacher training and modes of communication, it is interesting
to note that proficiency in using sign language and voice output devices (VOD) is correlated
with SLP and AAC specialist training, while the use of PECS is associated with parental and
self-training. Lastly, sign language, VOC and PECS are related to university training.
Therefore, it can be said that AAC teacher training delivery modes indirectly affect teacher
acquisition, depending on the content and time allocated for each training. In reviewing all
18 articles in this SLR, little elaboration on training has been yielded except for basic
information like the type of training ever attended without explaining in detail what is
included in each training. Future research should aim at analysing how each training affects
teacher learning in terms of content and conduct.
However, there is a trend observed regarding the teacher delivery method: teacher training
is becoming more online with AAC training in online mode mentioned in articles by Wallis
et al. (2017), Douglas et al. (2020), McCoy and McNaughton (2021), and Walker et al. (2022).
Past learners have shown a preference for online delivery methods as being more
convenient compared to traditional face-to-face methods (McCoy & McNaughton, 2021)
given the progress of technology and the very many self-learning reports at the teacher
training level (Andzik et al., 2017, 2018a, 2018b; Douglas et al., 2020; Leatherman & Wegner,
2022). Future teacher training should consider adding in more online elements so that
content can be accessible and reachable by more learners (Lebel et al., 2005). However, the
flexibility of course delivery should be ready to fulfil the teacher’s requirements (Douglas et
al., 2020).
169
Besides getting information about teacher training format, delivery, and content, the
researcher also found a few articles that mentioned activities that had been done during
teacher training. It is interesting to know that the majority of activities reported are role-play
(McCoy & McNaughton, 2021; Muttiah et al., 2015; Pennington et al., 2020), in-class
discussions (Greene & Esposito, 2023; Muttiah et al., 2015; Walker et al., 2022), case studies
(Greene & Esposito, 2023; Muttiah et al., 2015; Pennington et al., 2020), and reading (Greene
& Esposito, 2023; Walker et al., 2022). On the contrary, in-field training is only mentioned in
two articles. This finding is correlated with previous research saying that teacher training is
more theory-based than practice (Pennington et al., 2020; Walker et al., 2022). This is
alarming, as AAC is a complex process that requires a lot of practical time, particularly for
collaboration skills, assessment and implementation, which require a lot of practice.
Therefore, future teacher training should investigate how to improve in-field training to
minimise the gap between theory and practice.
5. Limitations of the Study

There are a few limitations considered in this study. First, these SLR-only study articles
ranged from 2013 to 2023. The reason behind this selection of a timeline is that the researcher
wanted the latest updated information regarding teacher training. However, it is undeniable
that there are many more suitable articles that fulfil the inclusive criteria stated in this SLR
but could not be selected due to the timeframe. Future research might include a longer
timeframe to be able to extract more related information regarding AAC training content
and course delivery.
A second limitation is that some opinions in the articles are voiced by professionals, which
include not only special education teachers but also speech-language pathologists,
occupational therapists, and many more. Therefore, some of the suggestions for content are
being seen for all AAC practitioners but not specifically for the teacher only. Future research
should explore more teacher opinions, specifically when more research regarding AAC
content and delivery is conducted.
The third limitation is that the articles in this SLR are taken from only two databases, so
there is a high probability that a few articles that are relatable to this research were
overlooked. However, the researcher scanned through every possible article through
reference tracking, trying to reduce the risk of missing articles.
6. Conclusion
This SLR aimed to identify teacher training content and delivery methods in AAC based on
earlier research. Eighteen 18 articles in total were identified via careful article searches in
two databases, namely Scopus and Google Scholar. Based on the analysis of these 18 articles,
the results show that most of the participants in the articles reported having taken an AAC
course in university, followed by training by an SLP, self-training, and so on. However, it is
interesting to know that AAC teacher training is gradually switching to online mode as
more research shows training in web-based courses, webinars, online modules, and so on. In
addition, the findings also yielded a total of 12 topics related to AAC that were suggested to
be included in teacher training. However, there were certain issues with the training system,
including a disconnect between theory and practice, a lack of adequate field experience, and
the omission of some courses. To reduce the gap between theory and practice in AAC,
future research should focus on analysing the teaching training modality in depth and ways
to improve teacher training quality.
170
Acknowledgement
We thank the FPEND Futuristic Learning Special Research Fund GG-2021-010 for the
support.
7. Reference
American Speech-Language-Hearing Association. (2016). Augmentative and alternative communication
(AAC). https://www.asha.org/njc/aac/
Andzik, N. R., Chung, Y. C., Doneski-Nicol, J., & Dollarhide, C. T. (2017). AAC services in schools: A
special educator’s perspective. International Journal of Developmental Disabilities, 65(2), 89–97.
http://dx.doi.org/10.1080/20473869.2017.1368909
Andzik, N. R., Schaefer, J. M., Nichols, R. T., & Cannella-Malone, H. I. (2018a). Exploring relationships
between teacher training and support strategies for students utilizing augmentative and
alternate communication. Journal of International Special Needs Education, 22, 25-34.
https://doi.org/10.9782/16-00044
Andzik, N. R., Schaefer, J. M., Nichols, R. T., & Chung, Y. C. (2018b). National survey describing and
quantifying students with communication needs. Developmental Neurorehabilitation, 21(1), 40–
47. https://doi.org/10.1080/17518423.2017.1339133
Barker, R. M., Akaba, S., Brady, N. C., & Theimann-Bourque, K. (2013). Support for AAC use in
preschool, and growth in language skills, for young children with developmental disabilities.
Augmentative and Alternative Communication, 29(4), 334–346.
https://doi.org/10.3109/07434618.2013.848933
Bauce, K. (2014). Cultural competence model. In J.J Fitzpatrick, & G. McCarthy (Eds.), Theories guiding
nursing research and practice (pp. 159–171). Springer Publishing. doi: 10.1891/9780826164056
Beukelman, D. R., & Light, J. C. (2020). Augmentative and alternative communication: Supporting children
and adults with complex communication needs. (5th ed.). Brookes publishing.
Beukelman, D. R., & Pat, M. (2013). Augmentative and alternative communication: Supporting children and
adults with complex communication needs. (4th ed). Brookes publishing.
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in
Psychology, 3(2). https://doi.org/10.1191/1478088706qp063oa
Brock, M. E., Cannella-Malone, H. I., Seaman, R. L., Schaefer, J. M., Page, E. J., Barczak, M. A., &
Dueker, S. A. (2017). Findings across practitioner training studies in special education: A
comprehensive review and meta-analysis. Exceptional Children, 84(1).
https://doi.org/10.1177/0014402917698008
Caron, J. G., O’Brien, M., & Weintraub, R. (2022). Online training to improve service providers
implementation of letter–sound correspondence instruction for individuals who use
augmentative and alternative communication. American Speech-Language-Hearing Association,
31(3), 1114-1132. https://doi.org/10.1044/2022_ajslp-21-00142
Chung, Y. C., & Stoner, J. B. (2016). A meta-synthesis of team members’ voices: What we need and
what we do to support students who use AAC. Augmentative and Alternative Communication,
32(3), 175-186. https://doi.org/10.1080/07434618.2016.1213766
Da Fonte, M. A., & Boesch, M. C. (2016). Recommended augmentative and alternative communication
competencies for special education teachers. Journal of International Special Needs Education, 19,
47-58. https://doi.org/10.9782/2159-4341-19.2.47
Da Fonte, M. A., Boesch, M. C., DeLuca, E. R., Papp, S. K., Mohler, A. E., Holmes, E. E., Clouse, K.A.,
Young, R. D., & Urbano, R. (2022). Current preparation status in AAC: Perspectives of special
education teachers in the United States. Augmentative and Alternative Communication, 38(1), 29-
40. https://doi.org/10.1080/07434618.2022.2046851
Douglas, S. N., West, P., & Kammes, R. (2020). The training experiences of augmentative and
alternative communication practitioners in one midwestern state. Perspective of the ASHA
Interest Groups, 5, 219-230. https://doi.org/10.1044/2019_PERS-19-00053
Eliada, P., Eleni, T., & Kakia, P. (2018). The use of augmentative and alternative communication in
Cyprus: Findings from a preliminary survey. Child Language Teaching and Therapy, 34 (1), 5–21.
http://dx.doi.org/10.1177/0265659018755523
171
Fields, A. R. (2015). Examining barriers to implementing augmentative and alternative communication in a

Midwest school (Doctoral Dissertation, Walden University).
http://scholarworks.waldenu.edu/dissertations
Flemming, K., Booth, A., Garside, R., Tunçalp, O., & Noyes, J. (2018). Qualitative evidence synthesis
for complex interventions and guideline development: Clarification of the purpose, designs,
and relevant methods. BMJ Global Health, 25(4). https://doi.org/10.1136/bmjgh-2018-000882
Ganz, J., & Simpson, R. (2018). Interventions for individuals with autism spectrum disorder and complex
communication needs. Brookes Publishing.
Greene, K. J., & Esposito, M. C. K. (2023). Interprofessional practice aligns with new California teacher
performance expectations for students with complex communication needs. The Journal of
Special Education Apprenticeship, 12(2), 14-27.
https://scholarworks.lib.csusb.edu/josea/vol12/iss2/4/
Haddaway, N. R., Collins, A. M., Coughlin, D., & Kirk, S. (2015). The role of Google Scholar in
evidence reviews and their applicability to grey literature searching. PloS One, 10(9).
https://doi.org/10.1371/journal.pone.0138237.
Hayrol Azril Mohamed Shaffril, Nobaya Ahmad, Samsul Farid Samsuddin, Asnarulkhadi Abu
Samah & Mas Ernawati Hamdan. (2020). Systematic literature review on adaption towards
climate change impacts among indigenous people in the Asia Pacific Regions. Journal of
Cleaner Production, 258. https://doi.org/10.1016/j.jclepro.2020.120595
Joklitschke, J., Rott, B., & Schindler, M. (2018). Theories about mathematical creativity in
contemporary research: A literature review. In Bergqvist, E., Österholm, M., Granberg, C., &
Sumpter, L (Eds.) Proceedings of the 42nd Conference of the International Group for the Psychology
of Mathematics Education (pp. 171–178). PME.
Joginder Singh, Diong, Z. Z., & Rahayu Mustaffa Kamal (2020). Malaysian teachers’ experience
using augmentative and alternative communication with students. Augmentative and
Alternative Communication, 36, 107-117. https://doi.org/10.1080/07434618.2020.1785547
Ketheeswaran, K. (2019). Sri Lankan special education professional’s competencies about
augmentative and alternative communication (AAC)-a study based on northern province of
Sri Lanka. European Journal of Special Education Research, 2(5).
https://doi.org/10.5281/zenodo.3553082
Leatherman, E. M., & Wegner, J. (2022). Augmentative and alternative communication in the
classroom: Teacher practices and experiences. American Speech-Language-Hearing Association,
53, 874-893. https://doi.org/10.1044/2022_LSHSS-21-00125
Lebel, T., Olshtain, E., & Weiss, P. L. (2005). Teaching teachers about augmentative and alternative
communication: Opportunities and challenges of a web-based course. Augmentative and
Alternative Communication, 21(4), 264-277. https://doi.org/10.1080/07434610500140311
McConachie, H., & Pennington, L. (1997). In-service training for schools on augmentative and
alternative communication. International Journal of Language & Communication Disorders, 32(1),
277-288. https://doi.org/10.1080/13682829709177101
McCoy, A., & McNaughton, D. (2021). Effects of online training on educators’ knowledge and use of
system of least prompts to support augmentative and alternative communication. Journal of
Behavioral Education, 30,319-349. https://doi.org/10.1007/s10864-020-09374-6
Mohd Hanafi Mohd Yasin, Shuhada Abdul Ghani, Suziyani Mohamed & Mohd Mokhtar Tahar.
(2020). Needs analysis of augmentative and alternative communication knowledge and skills
among special education teachers for Malaysian primary schools. Universal Journal of
Educational Research, 8(11), 110-116. doi: 10.13189/ujer.2020.082114.
Mollie, G., & Jeeva, J. (2022). AAC for all culturally and Linguistically Responsive Practise. Plural
Publishing.
Muttiah, N. A., McNaughton, D., & Kathryn, D. R. (2015). Providing instructional support for AAC
service delivery in low- and middle-income (LAMI) countries. International Journal of Speech-
Language Pathology. https://doi.org/10.3109/17549507.2015.1101154
172
Nunes, D., & Walter, C. (2020). AAC and autism in Brazil: A descriptive review. International Journal
of Disability, Development and Education, 67(3), 263-279.
https://doi.org/10.1080/1034912X.2018.1515424
Oihana, L., & María, O. V. (2020). The use of augmentative and alternative communication in
educational settings in the Basque Autonomous Community (Spain). European Journal of
Special Needs Education, 642-656. https://doi.org/10.1080/08856257.2020.1779981
Patel, R., & Khamis-Dakwar, R. (2005). An AAC training program for special education teachers: A
case study of Palestinian Arab teachers in Israel. Augmentative and Alternative Communication,
21(3),205-217. https://doi.org/10.1080/07434610400011638
Peckham-Hardin, K. D., Hanreddy, A., & Ogletree, B. T. (2018). Preparing teachers to facilitate
communication skills in students with severe disabilities.
http://ceedar.education.ufl.edu/tools/innovation-configurations/
Pennington, R. C., Walker, V. L., & Tapp, M. C. (2020). Teacher preparation in communication
instruction for students with extensive support needs. Teacher Education and Special Education,
1-16. https://doi.org/10.1177/0888406420978606
Person with Disabilities Act 2008. (2014, July).
https://www.ilo.org/dyn/natlex/natlex4.detail?p_isn=86297&p_lang=en
Radici, E., Heboyan, V., Mantovani, F., & Leo, G. D. (2018). Teachers’ attitudes towards children who
use AAC in Italian primary schools. International Journal of Disability, Development and
Education, 1–14. http://dx.doi.org/10.1080/1034912X.2018.1495321
Rashed Aldabas. (2019). Barriers and facilitators of using augmentative and alternative
communication with students with multiple disabilities in inclusive education: Special
education teachers’ perspectives. International Journal of Inclusive Education, 1010-1026.
https://doi.org/10.1080/13603116.2019.1597185
Senner, J. E., & Baud, M. R. (2017). The use of an eight-step instructional model to train school staff in
partner-augmented input. Communication Disorder Quarterly, 38(2), 89–95. https://
doi.org/10.1177/1525740116651251
Sierra-Correa, P.C., & Cantera Kintz, J. R. (2015). Ecosystem-based adaptation for improving coastal
planning for sea-level rise: A systematic review for mangrove coasts. Marine Policy 51, 385–
393. https://doi.org/10.1016/j.marpol.2014.09.013.
Subihi, A. S. (2013). Saudi special education student teachers’ knowledge of augmentative and
alternative communication (AAC). International Journal of Special Education, 28, 93–103.
https://eric.ed.gov/?id=EJ1024412.
Sustainable Development Goals of United Nations. (2023, July 16). Ensure inclusive and equitable quality
education and promote lifelong learning opportunities for all. https://www.un.org/en/.
Syar Meeze Mohd Rashid & Norfatimah A. Ghani. (2023). Special education teacher’s application of
entrepreneurial elements in teaching and facilitation. International Journal of Learning, Teaching
and Educational Research, 22(1), 55-71. https://doi.org/10.26803/ijlter.22.1.4
Syar Meeze Mohd Rashid & Wong, M. T. (2023). Challenges of implementing the individualized
education plan (IEP) for special needs children with learning disabilities: systematic literature
review (SLR). International Journal of Learning, Teaching and Educational Research, 22(1), 15-34.
Tawfik, G. M., Dila, K. A. S., Mohamed, M. Y. F., Tam, D. N. H., Kien, N. D., Ahmed, A.M., & Huy, N.
T. (2019). A step-by-step guide for conducting a systematic review and meta-analysis with
simulation data. Tropical Medicine and Health, 47. https://doi.org/10.1186/s41182-019-0165-6
Tönsing, K. M., & Dada, S. (2016). Teachers’ perceptions of the implementation of aided AAC to
support expressive communication in South African special schools: A pilot investigation.
Augmentative and Alternative Communication, 32(4), 282–304.
https://doi.org/10.1080/07434618.2016.1246609
Walker, V. L., & Snell, M. E. (2013). Effects of augmentative and alternative communication on
challenging behaviour: A meta-analysis. Augmentative and Alternative Communication, 29, 117–
131. https://doi.org/10.3109/07434618.2013.785020
173
Walker, V. L., Pennington, R. C., Andzik, N. R., Tapp, M. C., & Masud-Werner, A. (2022). Preservice
teachers’ preparation in communication instruction for students with extensive support
needs. Research and Practice for Persons with Severe Disabilities. 47(1), 57-64.
https://doi.org/10.1177/15407969221074720
Wallis, S., Bloch, S., & Clarke, M. (2017). Augmentative and alternative communication (AAC)
training provision for professionals in England. Journal of Enabling Technologies, 11(3), 101-112.
https:// doi.org/10.1108/JET-10-2016-0023
Whitemore, R., & Knafl, K. (2005). The integrative review: Updated methodology. Journal of Advance
Nursing, 52(5), 546-553. https://doi.org/10.1111/J.1365-2648.2005.03621.X
Wu, X. V., Chan, Y. S., Tan, K. H. S., & Wang, W. (2018). A systematic review of online learning
programs for nurse preceptors. Nurse Education Today, 60, 11-22.
https://doi.org/10.1016/j.nedt.2017.09.010
174

Vol. 22, No. 10, pp. 174-189, October 2023
Analysis of Vocational Student Performance

Criteria on Work Skills Based on Industry Needs:
An Analysis for Students' Skill Test Instruments
Rolly Robert Oroh* , Muhammad Muhdi Attaufiq ,
Metsi Daud and Rocky Frangky Roring
Universitas Negeri Manado, Indonesia
Abstract. A vocational high school (VHS) is intended to equip students

with job skills. The work skills of VHS students must be relevant to the
industry needs. This research was to determine the student performance
criteria on work skills based on industry needs. This research uses a
quantitative descriptive research approach. Research data was obtained
through instruments distributed to participants. The research
instrument was designed according to a job description of reinforced
concrete beam structures. There were 112 VHS student participants
involved in this research. This study found that the student performance
criteria on the work skills of concrete structure work include several
types of work, namely casting work, formwork, reinforcement work and
scaffolding work. The student performance criteria obtained in this
research can be a supporting component of the VHS student work skills
test instrument because they are in accordance with industry needs. The
VHS student work skills test instrument is intended to determine
whether students are competent. The work skills of VHS students can be
assessed from the 'show how' and 'do' competency levels as the basis for
assessing student performance. Student work skills will be relevant if
students are given real learning experiences through student practical
work directly in the industry as a form of theoretical and practical
collaboration.
Keywords: Vocational education; construction services; students’ work

skills
1. Introduction
Vocational high schools (VHS) have a function to prepare graduates with the
requisite skills to be able to work in certain jobs according to their fields. The
education system in Indonesia places VHS at the secondary education level. The
implementation of VHS in Indonesia is an attempt to fulfil the needs of the
industrial world, such as the field of construction services. Therefore, vocational
education or VHS is a bridge between education and the world of work.
175
The learning pattern of VHS in Indonesia shows the fact that after students have
completed their learning process, they are required to take skills competency
examinations to gain recognition through certification. These are carried out by
external parties (such as by construction service providers). The construction
services party plays the role of assessor and verifier in the implementation of the
skills test. Before students take the skills examination, the school measures
students’ competency according to the demands of the curriculum, and then the
construction services test students' work skills in an examination set by the
school. This form of external examination has an impact on the incompatibility
with the needs of the construction services sector because the format has been
prepared by the school in advance and does not involve input from external
parties. This can be one of the reasons why many VHS graduates are not
absorbed by construction services because the relevance of student work
competencies to job market needs tends to be low (BPS, 2022). Therefore,
collaboration in the implementation of vocational education between VHSs and
construction service providers is important (Hiim, 2017) to guarantee the
compatibility of competencies achieved by VHS students with employment
needs.
VHSs in Indonesia are educational units that can produce a ready-to-use

workforce at the operator level. However, data for 2022 shows that the open
unemployment rate by education level is more dominated by VHS graduates,
namely 10.38% of the total number of open unemployed, compared to other
graduates (BPS, 2022). The data shows that VHS graduates are still among the
number of unemployed people of productive age. Data for 2020 shows that
10.24% of the national gross domestic product (GDP) is a significant contribution
from the construction services sector where the number of construction workers
are dominated by VHS graduates, namely 70% (Kementerian PUPR, 2021).
Therefore, it is an important task for VHSs whose main objective is to produce
human resources with work skills that they consistently improve the quality and
relevance of graduates' work skills competencies to meet the needs of the
business world and the industrial world.
Vocational education through the VHSs is meant to equip students with work
skills competencies so they are able to compete in the world of work. Therefore,
vocational education must be relevant to the real needs of workers (Khampirat
et al., 2019) in preparing students to enter the workforce (Amilda et al., 2023;
Hansen et al., 2022; Rohr‑Mentele & Forster‑Heinzer, 2021). Vocational
education with competency-based learning focuses on achieving student work
competencies (Misbah et al., 2020; Oroh et al., 2018; Ralf et al., 2020). Skills are
not only learned in a formal school context, but also in authentic work settings
(Littke & Thang, 2015). Vocational education will be efficient if a learning
environment is provided that is in accordance with the demands of the work
environment where they will later be employed (De Vos et al., 2022; Ferm, 2020).
Because learning is always workplace based there should be no gap between
education and employment (Sylte, 2020). The learning achievements of
vocational education students can be assessed when they demonstrate their
ability (Ewing, 2017) to solve problems in the workplace (Pearce, 2015).
176
Having skills enables every individual to work effectively in the workplace

according to their field of expertise. Skills are the ability to act in real work
(Hadiyanto et al., 2021). They are defined as students' abilities to think and act
effectively and creatively in abstract and concrete fields as a development of
what they understand at school (Permendikbudristek RI, 2022). Students who
are skilled will succeed in real jobs (Chen et al., 2021). The skill aspect refers to
the ability to do work under various conditions and in diverse situations. The
skill aspect is one of the standards needed to demonstrate one's ability (Gulikers
et al., 2017). Thus, it can be stated that every skilled member of the workforce is
assured of success at work (Khampirat et al., 2019) such as when carrying out
concrete structural work.
The lack of relevance between the needs of the world of work and the quality of
VHS graduates is one of the factors driving the low absorption of VHS graduates
into the world of work (BPS, 2022). To realize an ideal VHS student skill profile,
it is necessary to determine the skills that are relevant to the needs of
construction services. Concrete structure work is a type of work required by
construction services. It requires workers who are VHS graduates. However, it
must first be confirmed whether VHS graduates have the level of concrete
structure work skills that meet the demands of construction services. Thus, to be
able to correlate the suitability of students' work skills with the needs of
construction services, this study aims to determine the vocational student
performance criteria on work skills based on industry needs.
1.1. Skills Competency of VHS Student

Vocational education learning through VHS is always directed at the process of
equipping students with work skills competencies. The VHS learning process in
Indonesia consists of normative, adaptive and productive learning packages.
The learning package that focuses on the process of providing student work
skills competence is the productive learning package (Oroh et al., 2020).
Productive learning programs consist of vocational subject matter that enables
students to be trained and develop work skills, including concrete structure
work skills. The competency of students' work skills in concrete structure work
consists of several work indicator components, such as scaffolding, formwork,
reinforcement and casting work (Kepmenaker RI, 2021).
Vocational education must be relevant to the real needs of workers and the
curriculum must meet the requirements of business (Billett, 2020). Vocational
education enables students to enter the workforce with the work skills they have
acquired. However, work skills are not fully learned only in formal school
learning, but in authentic work settings as well (Littke & Thang, 2015). Students
with the skills they have will be able to adapt these to workplace situations
(Billett, 2020). Therefore, vocational education will be relevant if it provides a
learning environment such as a real work environment.
Vocational education through VHS comprises competency-based learning.

Competency-based learning must be supported by a work-based curriculum
because the main objective of a workplace-based curriculum is to develop
177
knowledge for expert performance (Billet, 2020). One's work skills must be
supported by procedural knowledge because procedural knowledge comprises a
technique, skill and ability to secure work goals (Billet, 2020). Therefore,
students' work skills need to be supported with procedural knowledge in order
to succeed in achieving high performance goals.
The work skills provided to students in schools are still not considered
appropriate or relevant to the needs of the construction service industry. Some
of the skill competencies provided through subjects at school, in fact, have not
fully accommodated all the competency needs for construction service work,
therefore VHS teachers need to have vocational material specialization
(Estriyanto et al., 2017; Grosch, 2017). It is an empirical fact that the teaching
materials packaged in VHSs through the process of providing skill competencies
to students are not developed or based on the requirements of construction
services. Therefore, it is important to determine student performance criteria
that suit the needs of construction services in carrying out concrete structure
work.
1.2. Construction Services Requirements

The implementation of construction services in Indonesia continues to grow and
supplies the provision of construction infrastructure. The development of the
construction services business in North Sulawesi, Indonesia continues to
increase along with the construction of infrastructure facilities in various fields.
The reality in meeting the needs of the workforce in implementing construction
services is that it appears that that more skilled workers are still being brought
in from outside the region (Oroh et al., 2020). The need for labour in construction
projects continues to increase and is an important factor in supporting the
successful implementation of the construction business.
Construction service providers in Indonesia continue to increase in number in

line with the increasing need for infrastructure development. Every year there is
an increase in the number of construction service providers in small, medium
and large classifications. For example, in the province of North Sulawesi,
Indonesia, from the 2020 data recorded at the Central Statistics Agency, the
number of business entities providing construction services from small to large
classifications totalled 1,995. These consisted of 1,750 small business entities, 207
medium-sized business entities, eight large business entities, and others
numbering as many as 30 business entities (BPS, 2021). The number of
construction service actors will continue to increase every year (Isnandar et al.,
2015).
The role of the construction service industry in the development of an area is

important (Mtshali & Pillay, 2023); therefore, it needs to be supported by the
availability of a competent workforce in the field of construction work. The
VHSs are one of the contributing sources of workers who have construction
work competencies. Therefore, VHSs need to prepare their students efficiently to
have qualified and certified work skills competencies because the Law of the
Republic of Indonesia No. 2 of 2017 already requires that all construction
178
workers who work on construction projects must have certificates, especially for
the positions of experts, technicians/analysts and operators (Kementerian PUPR,
2021).
This research aims to determine criteria for student performance in work skills
in carrying out concrete structure work. This type of research has not been
carried out by other previous researchers, but was a further development of
research on the level of relevance of students' productive skills to the
requirements of construction services (Oroh et al., 2020). The contribution of this
research was to identify data about student performance criteria for work skills
that meet the needs of construction services.
2. Method
2.1. Research design
This research was conducted to determine criteria for student performance in
work skills in carrying out concrete structure work. Therefore, a quantitative
descriptive research approach was used. A quantitative descriptive design
collects quantitative data at a certain time to determine and describe trends in
research data (Creswell & Creswell, 2023; Roni et al., 2020). The quantitative data
was obtained through instruments distributed to participants. This research is
limited to the type of reinforced concrete beam structure work. Furthermore, to
determine the student performance criteria on work skills, a performance
analysis was conducted on the most dominant type of work as indicated from
student responses through the given instrument. Therefore, the data analysis
used was factor analysis.
2.2. Research participants

This study involved VHS students in the field of construction technology and
property expertise as participants. Using VHS students as participants was
intended to obtain data on student work skills competence achievements
through the learning process at VHS. There were 112 student participants
involved in this study who were randomly selected as representatives of eight
VHSs from seven districts and cities in North Sulawesi Province, Indonesia. The
demographics of the participants are given in Table 1, while an overview of the
participants’ expertise is depicted in Table 2.
Table 1: Demographic characteristics of research participants

Category Characteristics Number (n) Percentage
Gender Males 77 68,75%
Females 35 31,25%
Age 17 years old 44 39,29%
18 years old 68 60,71%
Table 2: Participants' expertise

Expertise Number (n) Percentage
Modelling design and building information 98 87,5%
Construction and property business 14 12,5%
179
2.3. Data collection

The research data is in the form of responses from research participants about
performance in reinforced concrete structure work, which in this study refers to
the type of reinforced concrete beam structure work. Data collection instruments
were distributed directly to the participants. Participants responded by giving
an oral description of the work of concrete structures. Their responses were then
assessed as to whether they were right or wrong according to existing
performance criteria. Subsequently, the results of the assessment were collated
and tabulated for analysis.
2.4. Research Instruments

Data about the description of student performance criteria in reinforced concrete
structure work were obtained through responses from research participants by
means of the research instruments provided. The research instrument comprised
a job description of reinforced concrete beam structures (Mehta et al., 2013).
Table 3 depicts an example of an overview of performance criteria for concrete
structural work. Allocating codes B1 to B30 in Table 3 for each performance
criterion was intended to facilitate analysis. This research instrument was
compiled according to the Indonesian National Work Competency Standards
No. 193/2021 (Kepmenaker RI, 2021) and the Indonesian National Qualifications
Framework (Perpres RI, 2012). The description of the 24 items in Table 3 is a
performance criterion that only focuses on reinforced concrete structure work.
Table 3: Description of performance criteria

Code Components of student skills performance criteria
Reinforcement for the main structure of concrete beams must be in
B1 accordance with the shop drawings
B2 The process of concrete reinforcement according to the work method
Selection of reinforcement forms of concrete beam structures according to
B3 technical specifications
The process of assembling the stirrup reinforcement must be good so that it
B4 does not shift when castings
B5 Binding the cross bars must be according to the working method
B6 The ends of the reinforcement are bent according to the working method
Requirements for installing concrete reinforcement that has been assembled
B7 according to the shop drawings
Model tie reinforcement according to technical specifications so as not to
B8 shift when casting
Installation of assembled concrete reinforcement must not easily shift when
B9 castings
B10 How to make formwork according to the shape of the concrete to be cast
Materials for the manufacture of formwork according to technical
B11 specifications
How to make formwork according to the working method so that it is
B12 strong
B13 Equipment for making formwork according to working method
B14 Formwork must not sway when casting
B15 How to install the formwork according to the working method
B16 Manufacture of scaffolding according to the working method
B17 Requirements for installing scaffolding according to shop drawings
180
B18 Choosing a scaffolding material must comply with technical specifications

The requirements for pouring concrete when casting must be in accordance
B19 with the work method
B20 The method of casting is carried out according to the work method
The thickness of the casting layer is made according to technical
B21 specifications
B22 Levelling of concrete after casting is made according to shop drawings
The method of compacting concrete when casting is made according to the
B23 work method
The method of mixing concrete for casting must be in accordance with the
B24 work method
2.5. Research Data Analysis

This research was conducted to determine the student performance criteria on
concrete structure work, therefore so the data analysis of this study used factor
analysis. The procedures carried out in factor analysis are calculating indicator
correlations, factor extraction, and factor rotation (Hair et al., 2019; Widarjono,
2015). Data adequacy requirements for factor analysis use the Kaiser-Meyer-
Olkin measure of sampling (KMO) values > 0.5. The requirement for instrument
components to use the measure of sampling adequacy (MSA) value for each
instrument component was that it had to had to have a value of more than 0.5 so
that it could be analysed further. Factor extraction used principal component
analysis, while factor rotation used the varimax method. Furthermore, for data
processing, communality and eigenvalues are used to determine how the
components of performance criteria in the student work skills instrument
explain the dominant factors that are formed. Finally, the SPSS sStatistics 23
application for Windows was employed for data analysis.
3. Results and Discussion

Based on the research data obtained, factor analysis was then carried out with
the help of the SPSS 23 for Windows program.
Table 4: Communalities value

Components of student skills Communalities
performance criteria
(Code)
B1 0.596
B2 0.741
B4 0.685
B5 0.675
B7 0.775
B8 0.707
B9 0.797
B10 0.582
B11 0.749
B12 0.677
B13 0.708
181
B14 0.621
B15 0.715
B16 0.610
B18 0.806
B20 0.676
B21 0.660
B22 0.781
B23 0.760
B24 0.610
After testing the first condition in the factor analysis, the KMO value was 0.616 >
0.5. Furthermore, considering the MSA values for each component of the
performance criteria on the instrument, several MSA values <0.5 were obtained.
The first stage of the analysis produced a B3 item value of 0.325 which means
<0.5. Then the second stage of analysis was continued on condition that B3 items
were removed from the analysis process for the following stage.
After going through several stages of analysis, the fifth stage, the final stage, was
reached with a value of KMO = 0.680. All the components of the performance
criteria had an MSA value of > 0.5; therefore, it can be stated that the existing
data is suitable for analysis through factor analysis. In the analysis of the second
stage, items B3, B6, B17, B19 and B12 were excluded for each stage because they
had an MSA value of <0.5. After the factor extraction process with the main
component analysis method, communalities values were obtained as shown in
Table 4.
Based on the communality values in Table 4, factor rotation was carried out so
that the eigenvalue was obtained, followed by component rotation so that the
instrument component with the appropriate criteria was obtained as shown in
Table 5. The description in Table 5 is modified from the results of SPPS statistical
23 analysis, which are ordered from the largest eigenvalue to the smallest value.
Table 5: Eigenvalues
Components of student Eigenvalue
skills performance criteria
(Code) Total % of Variance Cumulative %
B8 5,124
B22 5,124 25.621 25.621
B23 5,124
B12 2,047
B13 2,047 10.235 35.857
B15 2,047
B4 1,873
B14 1,873 9.367 45.224
B20 1,873
182
B24 1,873
B2 1,439
B5 1,439 7.196 52.420
B7 1,439
B11 1,289
B16 1,289 6.446 58.866
B21 1,289
B9 1,129
5.643 64.509
B10 1,129
B1 1,028
5.142 69.651
B18 1,028
Based on the eigenvalues and the description of the components in Table 5

which were obtained after rotating the factors using the varimax method, seven
(7) work type factors were formed (Figure 1), with details of performance criteria
as follows: Factor 1 or the casting work factor must comply with the work
method, shop drawings and technical specifications, with the performance
criteria, namely model tie reinforcement according to technical specifications so
as not to shift when casting, levelling of concrete after casting is made according
to shop drawings, and the method of compacting concrete when casting is made
according to the work method.
Factor 2 or the formwork factor must be in accordance with the work method,
with the performance criteria how to make formwork according to the working
method so that it is strong, equipment for making formwork according to
working method, and how to install the formwork according to the working
method. Furthermore, Factor 3 or the casting work factor must be in accordance
with the shop drawings and work methods, with the following performance
criteria, namely the process of assembling the stirrup reinforcement must be
good so that it does not shift when castings, formwork must not sway when
casting, the method of casting is carried out according to the work method, and
the method of mixing concrete for casting must be in accordance with the work
method.
In addition, Factor 4 or the reinforcement work factor must be in accordance

with the work method and shop drawing, with the following performance
criteria: the process of concrete reinforcement according to the work method,
binding the cross bars must be according to the working method, and
requirements for installing concrete reinforcement that has been assembled
according to the shop drawings.
Factor 5 or the material selection factor for formwork and scaffolding must
comply with the technical specifications for casting, with the performance
criteria, namely: materials for the manufacture of formwork according to
technical specifications, manufacture of scaffolding according to the working
method, and the thickness of the casting layer is made according to technical
specifications while Factor 6 or the factor of reinforcement and formwork work
183
must be in accordance with work methods, and shop drawings, with the
performance criteria, namely installation of assembled concrete reinforcement
must not easily shift when castings, and how to make formwork according to the
shape of the concrete to be cast. Finally, Factor 7 or the reinforcement and
scaffold work factor with the following performance criteria, namely
reinforcement for the main structure of concrete beams must be in accordance
with the shop drawings, and choosing a scaffolding material must comply with
technical specifications.
Figure 1: Variance values of the 7s factors of work type findings
Based on the results of the analysis of the dominant factors for the performance
criteria of concrete structures, the overall factors formed can generally explain
work indicators in concrete structure work. In addition, the results of the
analysis also found that the accumulated factors could explain the variance of
69.651% as shown in Table 5 and in Figure 1. It can be said that the work skills
shown by VHS students in general can explain the performance criteria in
concrete structure work.
The factors formed can explain several concrete structure works with their
performance criteria, namely casting work, formwork work, reinforcement
work, and scaffolding work. Casting work is mentioned by Factors 1a and 3.
Formwork work is mentioned by Factors 2, 5 and 6. Reinforcement work is
mentioned by Factors 4, 6 and 7, while scaffolding work is mentioned by Factors
5 and 7. In general, all the work that is formed is the work required in concrete
structure work. If based on the variance value of each factor in Table 3, it can be
said that casting work is the most dominant skill mastered by students with a
total variance value of 34.988% which is the sum of the variant values of Factors
1 and 2. The work of formwork, reinforcement and scaffolding together only
gives a variant value of 34.663%.
This proves that VHS students in their learning process tend to be given more
material about casting work, followed by formwork and reinforcement and
scaffolding work. Learning patterns like this are supported by learning models
outside of school through industrial work practice activities as a form of
collaboration between schools and companies (Hiim, 2023), so that students
acquire skills relevant to the job market (Nkwanyane, 2023). A practical learning
pattern like that can provide a learning experience for students (Hiim, 2017)
184
because it is important to have theoretical knowledge as well as relevant

practical work experience (Calero López & Rodríguez‑López, 2020; Sylte, 2020).
The jobs that are related to the dominant factors that appear in the analysis show
that the VHS students’ responses to the performance criteria for concrete
structure work already reveal a level of work skills that are relevant to carrying
out concrete structure work. However, an examination of the work factors
formed, namely casting, formwork, reinforcement, and scaffolding, with their
performance criteria, reveals that there are still jobs that have not fully provided
the performance criteria that meet the needs of construction services. The
performance criteria that match the needs of construction services must at least
describe performance, which is always adjusted to three work criteria, namely it
must comply with shop drawings, technical specifications and work methods
(Kepmenaker RI, 2021). The results of the analysis reveal that only the foundry
work provides criteria that are in accordance with the three performance criteria
based on the need for construction services. Meanwhile, the one that provided
the least conformity to the criteria was the scaffolding work because it only
provided technical specification criteria.
These results are in accordance with the author's observations about the learning
patterns carried out at VHS, which are introduced and explained relating to
casting, formwork and reinforcement. The learning pattern at VHS is also
supported by student work practice activities carried out at construction service
companies as a form of theoretical and practical collaboration (Sylte, 2020;
Hussain et al., 2021; Hiim, 2023). Vocational student work skills with
performance criteria prove that students are always directed to learning patterns
according to the real needs of the world of work. This proves that learning in
vocational education or VHSs always focuses on the achievement of student
work skills. Therefore, skills are important so that students are competent in
their field of work.
It is also important that vocational students master the competence of work

skills because having good work skills will become the basic capital in carrying
out work assignments according to competency skills. With good skills, students
become competent and experts in their fields and having expertise will lead to
ways of working that comply with job procedures (Billett, 2020). VHS students
in Indonesia are prepared to become skilled workers at the executive level
(Isnandar et al., 2015); therefore, students who are prepared to become skilled
workers or to manage staff must equip themselves with skills according to their
area of expertise. Work skills competence for students will be able to stand them
in good stead in their jobs so that a construction project will be successful if it is
carried out by a skilled and trained workforce (Elfaki & Alatawi, 2015). Because
VHS students in their learning have been trained in real skills such as those
needed in the workplace, students will always be able to adapt to workplace
expectations. Thus, it can be stated that any skilled workforce will always be
successful in work such as construction projects.
185
The findings of this study determining the performance criteria of VHS students
in carrying out concrete structure work can be a supporting component for
determining the achievement of student work skill competencies. This is
supported by the findings of student performance criteria in this study, which
are in accordance with the needs of the industrial world. The way that can be
done is to prepare an instrument that can assess the achievement of student
work skills. Because competency-based education needs to be assessed to
determine whether students are competent or not (Wesselink et al., 2017; Misbah
et al., 2020), it must be structured properly so that the data obtained later is not
distorted by measurement errors (Blömeke, 2017). The Indonesian National
Standards Agency issued a regulation that vocational schools must assess the
competency achievements of students' work skills by involving the industry to
ensure student competence meets the needs of the world of work (Misbah et al.,
2020).
Assessment of student work skills competence can be adjusted to the standards

required by industry or construction services. Competence can be described in
four levels, namely 'know', 'know how', 'show how' and 'do', where the two
lower levels address students' knowledge or cognitive capacity, while the two
higher levels are directed at the performance of students (Wesselink et al., 2017).
This research, through the responses of students as participants, has provided an
overview of their competence at the 'know how' and 'show how' levels. Even
though the findings of the performance criteria of students relatively do not fully
describe the performance criteria needed in concrete structural work, in general
they have described work indicators through the required performance criteria
according to the needs of construction services.
Assessment is an important factor for vocational education to determine student

learning outcomes. Assessment is defined as a series of activities to collect and
analyse information to measure student learning outcomes. The form of
assessment of student learning outcomes at VHSs in Indonesia often uses
formative and summative assessment forms with an assessment approach in the
form of authentic assessment, performance assessment, and portfolio assessment
(Misbah et al., 2020). The form of assessment that can be carried out to assess the
achievement of VHS students' work skills is a summative assessment to assess
student performance in concrete structure work. Student performance can be
assessed through a skills test instrument with performance criteria on Factors 1
to 7 from the results of this study. The VHS student work skills test instrument is
intended to determine whether students are competent or not (Wesselink et al.,
2017). To determine whether the competencies achieved by students are
compatible with the needs of the industrial world, it is necessary to be equipped
with performance criteria that are in line with the needs of the industrial world
(Misbah et al., 2020; Preston, 2017). Therefore, the performance criteria obtained
in this study can be a supporting component of the VHS student work skills test
instrument.
The discussion of this research does not include the characteristics of VHS and
students as participants, specifically the content of the learning material that
186
students have received from each VHS. Therefore, the limitations of this research
can be a recommendation for researchers who are interested in conducting
similar research to consider the content of the teaching materials given to
students.
4. Conclusion
This study determined the student performance criteria on work skills in
concrete structure work. The factors formed can explain several concrete
structure works with their performance criteria, namely casting work, formwork
work, reinforcement work, and scaffolding work. Casting work is the most
dominant work mastered by students, followed by formwork, reinforcement
and scaffolding work. The performance criteria obtained in this study through
the student performance criteria can be a supporting component of the VHS
student work skills test instrument. The VHS student work skills test instrument
is intended to determine whether students are competent or not. To determine
whether the competencies achieved by students are compatible with the needs of
the industrial world, it is necessary to be equipped with performance criteria
that are in accordance with the needs of the industrial world. The work skills of
VHS students can be seen from the 'show how' and 'do' competency levels as the
basis for assessing student performance. Student work skills will be guaranteed
to be relevant if students are given learning experiences such as real learning
patterns through student work practices carried out directly in the industry as a
form of theoretical and practical collaboration. Such a pattern of real learning is a
pattern of vocational education which is always focused on the achievement of
student work skill competencies through industry-based performance criteria.
Work skill competencies achieved by students through real learning patterns
will guarantee more quality and relevance to the needs of the business and the
industrial world.
5. References
Amilda, A., Bujuri, D. A., Uyun, M., Nasrudin, D., & Junaidah, J. (2023). Patterns of
character education for vocational school students through non-academic
programs: paradigm and implementation. International Journal of Learning,
Teaching and Educational Research, 22(4), 459-477.
Billett, S. (2020). Learning in the workplace, strategies for effective practice. Routledge.
Blömeke, S. (2017). Assuring quality in competence assessments: The value added of
applying different assessment approaches to professional education. In M.
Mulder (Ed.), Competence-based Vocational and Professional Education, Bridging the
Worlds of Work and Education (Ch. 29). Springer.
BPS. (2021). Keadaan ketenagakerjaan Indonesia Agustus 2021 [Indonesian labor situation,
August 2021]. Central Bureau of Statistics of the Republic of Indonesia (BPS RI),
Jakarta, Indonesia.
BPS. (2022). Keadaan ketenagakerjaan Indonesia per Februari 2022 [Indonesian labor situation
as of February 2022]. BPS RI, Jakarta, Indonesia.
Calero-López, I., & Rodríguez‑López, B. (2020). The relevance of transversal
competences in vocational education and training: A bibliometric analysis.
Empirical Research in Vocational Education and Training, 12(12).
https://doi.org/10.1186/s40461-020-00100-0
187
Chen, P., Goncharova, A., Pilz, M., Frommberger, D., Li, J., Romanova, O., & Lin, Y.
(2021). International curriculum comparison in vocational education and
training: A collaborative development of an analysis instrument. International
Journal for Research in Vocational Education and Training, 8(4), 16–43.
https://doi.org/10.13152/IJRVET.8.4.2
Creswell, J. W., & Creswell, J. D. (2023). Research design: Qualitative, quantitative, and
mixed-methods approaches. Sage Publishing.
De Vos, M. E., Baartman, L. K. J., Van der Vleuten, C. P. M., & De Bruijn, E. (2022).
Unravelling workplace educators’ judgment processes when assessing students’
performance at the workplace. Journal of Vocational Education & Training.
https://doi.org/10.1080/13636820.2022.2042722
Elfaki, A. O., & Alatawi. (2015). Representing the knowledge of public construction
project cost estimator by using rule-based method. Journal of Building
Construction and Planning Research, 3(4), 189-195.
http://dx.doi.org/10.4236/jbcpr.2015.34019
Estriyanto, Y., Kersten, S., Pardjono, P., & Sofyan, H. (2017). The missing productive
vocational high school teacher competency standard in the Indonesian education
system. Journal of Technical Education and Training, 9(1), 26-44.
https://publisher.uthm.edu.my/ojs/index.php/JTET/article/view/1499
Ewing, B. (2017). An exploration of assessment approaches in a vocational and education
training courses in Australia. Empirical Research in Vocational Education and
Training, 9(14). https://doi.org/10.1186/s40461-017-0058-z
Ferm, L. (2020). Vocational students’ ways of handling the academic/vocational divide.
International Journal for Research in Vocational Education and Training, 8(1), 1–20.
Grosch, M. (2017). Developing a competency standard for TVET teacher education in
Asian countries. Jurnal Pendidikan Teknologi dan Kejuruan, 23(3), 280-287.
https://doi.org/10.21831/jptk.v23i3.13418
Gulikers, J. T. M., Runhaar, P., & Mulder, M. (2017). An assessment innovation as
flywheel for changing teaching and learning. Journal of Vocational Education &
Training, 70(2), 212–231. https:// doi.org/10.1080/13636820.2017.1394353.
Hadiyanto, H., Nofer, N., Syamsurizal, S., Muhaimin, M., & Krisantia, I. (2021).
Students’ soft skills, hard skills, and competitiveness (SHC): A suggested model
for Indonesian higher education curriculum. International Journal of Learning,
Teaching and Educational Research, 20(2), 218-234.
Hair, J. F. Jr., Black, W. C., Babin, B. J., & Anderson, R. E. (2019). Multivariate data
analyisis. Pearson Prentice Hall.
Hansen, S., Rostiyanti, S. F., Setiawan, A. F., & Koesalamwardi, A. B. (2022). Developing
a work-integrated learning model adjusting to construction 4.0 concepts.
International Journal of Work-Integrated Learning, 23(1), 65-80.
https://www.ijwil.org/files/IJWIL_23_1_65_80.pdf
Hiim, H. (2017). Ensuring curriculum relevance in vocational education and training:
Epistemological perspectives in a curriculum research project. International
Journal for Research in Vocational Education and Training, 4(1), 1-19.
Hiim, H. (2023). How can collaboration between schools and workplaces contribute to
relevant vocational education? Results of an action research project in the
school-based part of Norwegian vocational education and training. Vocations and
Learning, 16, 1–21. https://doi.org/10.1007/s12186-022-09300-z
Hussain, M. A. M., Zulkifli, R. M., Kamis, A., Threeton, M. D., & Omar, K. (2021).
Industrial engagement in the technical and vocational training (TVET) system.
188
International Journal of Learning, Teaching and Educational Research, 20(12), 19-34.

Isnandar, D., A., Wena, M., & Suwarno, E. (2015). Revitalisasi sekolah menengah kejuruan
program keahlian teknik bangunan [Revitalization of vocational high schools for
building engineering skills programs]. Universitas Negeri Malang.
Kementerian PUPR. (2021). Era baru konstruksi berkarya menuju Indonesia maju [The new
era of construction is working towards a developed Indonesia]. Minister for
public works and human settlements (Kementerian PUPR). Jakarta, Indonesia.
Kepmenaker RI. (2021). Standar Kompetensi Kerja Nasional Indonesia (SKKNI) pada jabatan
kerja pelaksana lapangan pekerjaan gedung [Indonesian National Work Competency
Standards (SKKNI) for work positions implementing building work]. Regulation
of the Minister of Manpower of the Republic of Indonesia (Kepmenaker RI).
Jakarta, Indonesia.
Khampirat, B., Pop, C., & Bandaranaike, S. (2019). The effectiveness of work-integrated
learning in developing student work skills: A case study of Thailand.
International Journal of Work-Integrated Learning, 20(2), 127-146.
https://www.ijwil.org/files/IJWIL_20_2_127_146.pdf
Littke, K., & Thang, P. (2015). Learning at work in higher vocational education. Nordic
Journal of Vocational Education and Training, 5(1), 1-17.
https://doi.org/10.3384/njvet.2242-458X.15v5i1a3
Mehta, M., Armpriest, D., & Scarborough, W. (2013). Building construction, principles,
materials, and systems. Pearson Education.
Misbah, Z., Gulikers, J., Dharma, S., & Mulder, M. (2020). Evaluating competence-based
vocational education in Indonesia. Journal of Vocational Education & Training,
72(4), 488-515. https://doi.org/10.1080/13636820.2019.1635634
Mtshali, Th. I., & Pillay, A. S. (2023). Pedagogical capital strategies for civil technology
skills-based activities. International Journal of Learning, Teaching and Educational
Nkwanyane, T. P. (2023). Understanding the demand for industrial skills through the
national certificate (vocational) building and civil engineering programme.
International Journal of Learning, Teaching and Educational Research, 22(5), 674-687.
Oroh, R. R., Haris, A. S., Sugandi, R. M., & Isnandar. (2018). What are the dominant
factors of students’ productive skills in construction services? IOP Conference
Series: Materials Science and Engineering. Volume 306, 2nd International
Conference on Innovation in Engineering and Vocational Education 25–26
October 2017, Manado, Indonesia.
https://iopscience.iop.org/article/10.1088/1757-899X/306/1/012111
Oroh, R. R., Runtuwene, J. P. A., Tumanduk, M. S. S., & Palilingan, V. R. (2020).
Relevance of student productive skills competencies based on construction
services’ needs. International Journal of Innovation and Learning, 28(1), 4-11.
https://doi.org/10.1504/IJIL.2020.108461
Pearce, J. (2015). Assessing vocational competencies in civil engineering: Lessons from
AHELO for future practice. Empirical Research in Vocational Education and
Training, 7(1). https://doi.org/10.1186/s40461-015-0016-6
Permendikbudristek RI. (2022). Standar kompetensi lulusan pendidikan menengah
[Competency standards for secondary education graduates]. Regulation of the
Minister of Education, Culture, Research and Technology of the Republic of
Indonesia [Permendikbudristek RI]. Jakarta, Indonesia.
Perpres RI. (2012). Kerangka Kualifikasi Nasional Indonesia [Indonesian National
Qualifications Framework]. Regulation of the President of the Republic of
Indonesia [Perpres RI]. Jakarta, Indonesia.
189
Preston, J. (2017). Competence-based education and training (CBET) and the end of human
learning. Springer Nature.
Rohr‑Mentele, S., & Forster‑Heinzer, S. (2021). Practical validation framework for
competence measurement in VET: A validation study of an instrument for
measuring basic commercial knowledge and skills in Switzerland. Empirical
Research in Vocational Education and Training, 13(18).
https://doi.org/10.1186/s40461-021-00122-2
Roni, S. M., Merga, M. K., & Morris, J. E. (2020). Conducting quantitative research in
education. Springer.
Sylte, A. L. (2020). Predicting the future competence needs in working life: didactical
implications for VET in Norway. International Journal for Research in Vocational
Education and Training, 7(2), 167–192. https://doi.org/10.13152/IJRVET.7.2.3
Van Griethuijsen, R. A. L. F., Kunst, E. M., Van Woerkom, M., Wesselink, R., & Poell,
R.F. (2020). Does implementation of competence-based education mediate the
impact of team learning on student satisfaction? Journal of Vocational Education &
Training, 72(4), 516-535. https://doi.org/10.1080/13636820.2019.1644364
Wesselink, R., Biemans, H., Gulikers, J., & Mulder, M. (2017). Models and principles for
designing competence-based curricula, teaching, learning and assessment. In M.
Mulder (Ed.), Competence-based vocational and professional education, bridging the
worlds of work and education. Springer.
Widarjono, A. (2015). Analisis multivariat terapan [Applied multivariate analysis.].
Yogyakarta, Indonesia.
190

Vol. 22, No. 10, pp. 190-208, October 2023
Exploring Moodle Usage in Higher Education in

the Post-pandemic Era: An Activity-theoretical
Investigation of Systemic Contradictions
Ishaq Al-Naabi*
University of Technology and Applied Sciences
Nizwa, Oman
Abstract. Despite Moodle's initial popularity as a learning management

system, recent research has revealed a decline in its usage within higher
education during the post-pandemic pedagogical landscape. Drawing
upon activity theory as a theoretical framework, this qualitative study
aims to explore teachers' perceptions regarding the discontinuation of
Moodle use once the pandemic subsided. Data for this investigation was
collected through a focus group discussion involving six university
teachers. Using a thematic analysis, the findings shed light on several
factors contributing to the decline, including ineffective rules and policies
that impact teachers, as well as confusing task allocation. Apart from
contributing to the existing literature on post-pandemic pedagogy, the
study's outcomes indicated the need for well-defined polices addressing
roles and responsibilities of people involved in Moodle use and
addressing teachers’ workload in higher education.
Keywords: activity theory; COVID-19; higher education; Moodle; post-

pandemic era; teachers’ perspectives
1. Background
In the wake of the COVID-19 pandemic, higher education institutions across the
world have had to pivot rapidly to online learning, leading to a surge in the use
of learning management systems (LMS), such as Moodle, Blackboard and Canvas
(Crawford et al., 2020; Hodges et al., 2020). Learning management systems are
considered effective web-based learning systems for administering and managing
online courses, sharing study materials, tracking student activities, monitoring
students’ learning and participation, and evaluating their performance (Kant et
al., 2021). Compared to other learning platforms, Moodle has become the most
popular free, open-source and accessible learning platform among higher
education institutions (Alqahtani, 2020; Mpungose, 2020). However, as soon as
social distancing restrictions were lifted and as the world started moving towards
*
Corresponding author: Ishaq Al-Naabi, aladeeb001@gmail.com
©Authors
191
a post-pandemic pedagogy, there was a decline in the usage of Moodle and other
LMSs, despite their proven efficacy in supporting student learning (Chen et al.,
2021; Lau, 2021).
Research concerning teachers' acceptance of using Moodle and their attitudes

towards its use indicated that their prior experience, the ease of use of the platform
and its usefulness were influential factors in their use of Moodle. However, some
teachers were not satisfied with Moodle because of the potential for cheating and
academic dishonesty among students (Taamneh et al., 2022). To further enhance
teachers' practices and use of Moodle after the pandemic, additional training and
support is required (Lockee, 2021). Furthermore, there is a need to develop
effective online pedagogy, enhance learning platform capabilities, and explore the
potential of new technologies such as virtual and augmented reality to enhance
online learning experiences (Lockee, 2021; Taamneh et al., 2022). Thus, it is
essential to address these challenges and develop strategies to improve the
effectiveness and quality of online learning using Moodle.
While the adoption of Moodle was initially driven by factors such as ease of use,
flexibility, and affordability (Smith et al., 2019), studies have reported declines in
student engagement and satisfaction with online courses (McElroy, 2021, 2020;
Mihai & Dragoș, 2021), disparities in access to digital infrastructure and
technology (McElroy, 2021), and challenges faced by teachers in using Moodle.
The challenges include technical difficulties, lack of support and training,
concerns about the quality of online learning compared to traditional face-to-face
instruction, overwhelming workload, and potential for cheating and academic
dishonesty (Mohammadi et al., 2021; Pătrașcu & Grosseck, 2021; Tsai et al., 2020).
It is essential to acknowledge the significant role played by the COVID-19
pandemic, which has presented a unique opportunity for studying the adoption
process (Johnson et al., 2020). By examining the adoption process and its
underlying factors, this research aims to contribute to a deeper understanding of
technology adoption in higher education and its implications for post-pandemic
pedagogy.
The post-pandemic era has witnessed a significant shift in pedagogy, with a

heightened reliance on technology-enabled learning environments. Learning
management systems have played a pivotal role in facilitating this transition, with
research highlighting the importance of LMS adoption in post-pandemic
pedagogy (Betts et al., 2020; Mihai & Dragoș, 2021). LMS platforms offer diverse
features such as content management, collaboration tools, and assessment
capabilities, which have supported the delivery of online and hybrid learning
modalities. The adoption of LMS has not only provided educators with the means
to deliver instruction remotely but has also opened opportunities for innovative
teaching methods and personalized learning experiences. However, research
indicated the declining usage of LMSs in certain educational contexts for several
reasons. In addition to technical challenges and limited accessibility to
technology, the desire to restore a sense of normalcy and return to traditional face-
to-face instruction, as educators and students seek to regain the interpersonal and
social elements of in-person learning experiences are some factors leading to this
192
decline (Chen et al., 2021; Lau, 2021). Hence, as institutions continue to navigate
the post-pandemic landscape, understanding the factors that influence the
successful adoption and effective utilization of LMS becomes crucial.
Local practices in Omani higher education endorse the adoption of learning

management systems (LMS) which resulted in widespread utilisation of
platforms such as Moodle and Blackboard. In acknowledging the imperative to
uphold quality and efficacy in teaching and learning practices, the Oman
Academic Accreditation Authority (OAAA) has established specific criteria for
evaluating information and learning technology services within higher education
institutions (Oman Academic Accreditation Authority, 2016). In the context of the
present study, the higher education institution under investigation employs
Moodle as its official LMS, regulated and supervised by the E-learning
Implementation Policy and Moodle guide. Despite the growing utilisation of
Moodle within the institution, it is noteworthy that the COVID-19 pandemic has
played a significant role in further augmenting its adoption. Nevertheless,
anecdotal evidence and personal experiences have revealed certain challenges
associated with this increased reliance on Moodle. These challenges encompass
an augmented workload for teachers, inadequacies in technological
infrastructure, and the presence of chaotic course design and layout, all of which
have collectively hindered the effective utilisation of Moodle and led to a decline
in its use.
Therefore, this qualitative case study seeks to explore the reasons behind the
decline in Moodle usage in higher education institutions from the viewpoint of
teachers. To investigate this issue, it draws on activity theory (Engeström, 1999),
a theoretical framework that provides a holistic understanding of human activity
and its relationship to the broader social and cultural context (additional discussion
of the theory is provided in the Methodology section of this paper). By examining the
perceptions of teachers using this theoretical framework, this paper seeks to shed
light on the complex interplay of factors that contribute to the decline in Moodle
usage in a higher education institution in Oman. It sought to answer this research
question: What are the contradictions within the activity system of Moodle usage that
have hindered teachers in a higher education institution in Oman from sustaining their
use of Moodle in the post-pandemic pedagogy?
The primary objective of this research is to conduct an in-depth examination of

the adoption process of Moodle within higher education. It seeks to explore the
underlying factors that influence the implementation of this learning
management system and make significant contributions to advancing the current
understanding of technology adoption in the higher education context.
Additionally, this study aims to enrich the existing literature on the adoption of
learning management systems in the realm of post-pandemic pedagogy.
The findings from this investigation hold noteworthy implications for the design
and implementation of digital learning environments in higher education
institutions. By identifying and analysing the inherent contradictions within the
activity system associated with Moodle usage, this research intends to offer
valuable insights that can guide the development of more effective and engaging
193
digital learning environments. These insights are crucial for fostering optimal
student learning experiences in the post-pandemic era.
This literature review aims to address the challenges associated with post-
pandemic pedagogy and provide a comprehensive summary and categorisation
of recent studies examining teachers' perceptions of Moodle in higher education.
The scope of this review is limited to research conducted during the COVID-19
pandemic (from 2020 onwards). This limitation was imposed to ensure alignment
with the research question and the utilisation of activity theory to identify the
barriers encountered by teachers in utilising Moodle within the context of post-
pandemic pedagogy. The search for relevant research was conducted on the
SCOPUS database. The search was performed using specific keywords, including
"Moodle," "Blackboard," "LMS," "Learning Management System," "higher
education," "perceptions," "challenges," "post-pandemic pedagogy," and "COVID-
19." Screening of titles and abstracts was conducted to identify empirical and
theoretical research that aligned with the focus of this study. It is important to
note that certain studies were excluded due to their lack of relevance to teachers
and higher education, while others were inaccessible as full-text papers.
2.1 Teachers' Perspectives in Using Moodle During the Pandemic

Several studies explored teachers' perceptions of the Moodle platform in different
contexts. One recurring positive theme was the flexibility and convenience that
Moodle provides for delivering online courses and facilitating communication
and collaboration among students. Toquero (2020) found that Moodle's features
for posting announcements, sharing resources, and grading assignments helped
teachers to monitor student progress and provide feedback. Similarly, Alqurashi
and Alhashmi (2021) reported that teachers in Saudi universities perceived
Moodle as an effective tool for promoting interaction and engagement among
students.
Moodle has also been found to be helpful in managing course content, facilitating
student engagement and interaction, and providing feedback to students. For
instance, Wang (2021) reported that university teachers in a Japanese university
had a generally positive perception of Moodle's impact on their teaching practices.
Additionally, Zamora-Antuñano et al. (2022) found that the majority of teachers
in four public universities in Mexico had positive perceptions of Moodle and its
features, including communication with students and assessment of learning
outcomes.
Prior experience with Moodle was found to influence teachers' perceptions of the
platform. Salas-Rueda et al. (2020) reported that teachers' perceptions of Moodle
varied depending on their level of experience using the platform, with more
experienced teachers having a more positive perception of Moodle's impact. The
same finding regarding teachers’ prior experience with using Moodle was
reported by Uzunboylu et al. (2021) in Turkish universities. Finally, research
during the pandemic showed that teachers perceived Moodle positively because
of its ease of use for both teachers and students. Antuñano et al. (2022), Salas-
194
Rueda et al. (2020), and Taamneh et al. (2022) reported that teachers found Moodle
helpful during the pandemic for its communication features and assessment tools.
2.2 Teachers’ Challenges in Using Moodle During the Pandemic

However, despite its popularity as a learning management system, Moodle is not
without challenges and concerns, as reported by teachers in the literature. One
major challenge faced by teachers is technical issues. These technical issues
include poor connectivity, slow loading times, and difficulties in uploading and
downloading files. For example, a study by Kabakci Yurdakul and Inan (2020)
found that teachers encountered technical difficulties with Moodle during the
pandemic, which negatively affected their teaching and students' learning. Wang
(2021) and Kant et al. (2021) also highlighted technical difficulties and limitations
in customization of Moodle layout and navigating the platform settings. Likewise,
Alqurashi (2020), Zamora-Antuñano et al. (2022) and Uzunboylu et al. (2021)
reported teachers’ facing technical difficulties, limited access to technology and
limited access to digital resources. In their attempt to explore the challenges and
opportunities in using Moodle in China, Gao et al. (2020) found several technical
issues and difficulties in using the platform, such as poor connectivity, slow
Internet speed, and the lack of training on how to use the platform effectively.
Another issue was the lack of devices and poor technical infrastructure to use
Moodle in higher education institutions (Khan & Ghani, 2020), and lack of
integration with other educational technologies and platforms (Kant et al., 2021).
Another occurring theme in the literature is the increased workload and reduced
interaction with students resulting from the sudden shift to online teaching, which
led to higher levels of stress and anxiety among university teachers. Teachers have
reported feeling overwhelmed by the amount of time required to develop and
manage course content in Moodle, which has led to higher levels of stress and
anxiety among university teachers (Alqurashi, 2020; Gao et al., 2020; Khan &
Ghani, 2020; Wang, 2021). Additionally, research has identified challenges in
Moodle adoption during the pandemic regarding student motivation and
engagement. Despite teachers’ efforts to implement student-centered pedagogies
and collaborative learning, students’ motivation and engagement remained
minimal. The lack of interactivity and personalization in Moodle can lead to a
passive learning experience for students (Kabakci Yurdakul & Inan, 2020;
Alqurashi, 2020; Gao et al., 2020).
Scholars noted the limited ability of Moodle to support complex learning

activities, such as project-based learning and group work. For instance, a study by
Çalışkan and Yalın (2021) reported that teachers faced challenges in designing and
implementing group activities on Moodle due to its limited functionality.
Likewise, Khan and Ghani (2020) reported teachers’ difficulties in ensuring that
students participate actively and contribute to discussions when adopting
student-centred teaching approaches. In a similar investigation, Kant et al. (2021)
reported that teachers faced challenges in adapting to the new teaching and
learning paradigm that the platform introduced, such as the need for greater self-
directed learning and increased student autonomy. Finally, teachers also reported
that they faced challenges in monitoring and assessing students' learning progress
195
and providing timely feedback to students in online learning environments (Kant

et al., 2021). Teachers’ challenges in providing timely and effective feedback to
students was attributed to students’ lack of face-to-face interaction (Khan &
Ghani, 2020) and to teachers’ increased workload (Alqurashi, 2020; Wang, 2021).
2.3 Factors Contributing to the Decline in Moodle Usage in Post-Pandemic

Pedagogy
Post-pandemic pedagogy has witnessed significant changes and innovations,
accelerated or brought about by the pandemic itself. These include the increased
utilization of technology, hybrid and blended learning models, student-centered
approaches, and a renewed focus on mental health and well-being
(Ashfaquzzaman, 2020; Murphy, 2020). The present paper specifically focuses on
the use of Moodle in the post-pandemic pedagogy, an area that remains relatively
under-researched. Despite its popularity as an online teaching and learning tool
during the pandemic, studies have reported a decline in its use and raised
questions regarding its efficacy in the post-pandemic context.
Several factors have been identified as potential contributors to the declining use
of Moodle. One explanation is the desire among educators and students to restore
a sense of normalcy and return to traditional face-to-face instruction, as they seek
to regain the interpersonal and social aspects inherent in in-person learning
experiences (Bond et al., 2021; Lau, 2021). Additionally, concerns have been
expressed regarding the potential negative impact of excessive reliance on
technology on student well-being, such as increased screen time and reduced
opportunities for social interaction (Chen et al., 2021; Lau, 2021).
Moreover, technical challenges and limited access to reliable internet connectivity

and digital devices, particularly in underserved communities, pose obstacles to
the effective implementation of learning management systems (LMS) like Moodle
(Chen et al., 2021; Lau, 2021). The digital divide has become increasingly
prominent during the pandemic, underscoring disparities in technology access
and internet reliability among students, especially those from disadvantaged
backgrounds (Chen et al., 2021).
Faculty training and development also emerge as significant challenges, as the

rapid shift to remote and hybrid learning necessitated quick adaptation to new
technologies and pedagogical approaches, including the effective use of Moodle
and the professional design and layout of Moodle course pages (Aldemir &
Doğan, 2021). The pandemic has disrupted traditional modes of student
engagement, such as face-to-face interactions, extracurricular activities, and
campus events, prompting higher education institutions to explore alternative
avenues for fostering engagement and a sense of community through LMS
(Ashfaquzzaman, 2020; Murphy, 2020). Furthermore, questions have been raised
about the effectiveness of traditional assessment methods, such as exams and
grades, leading institutions to consider alternative approaches like formative
assessment, authentic assessment, and project-based assessment.
These aforementioned factors, combined with the fatigue experienced during

prolonged online learning, have contributed to the declining use of LMS in the
196
post-pandemic pedagogical landscape. It is imperative for researchers and

educators to address these challenges and explore strategies to enhance the
effective integration of LMS like Moodle in the evolving educational landscape.
The literature elucidated the experiences and challenges faced by teachers in
adopting Moodle both during the pandemic and in the post-pandemic
pedagogical landscape. Drawing upon this existing body of literature, the present
study seeks to contribute by comprehensively understanding the adoption of
Moodle in the post-pandemic context among teachers in higher education.
Specifically, the study aims to identify the contradictions in the activity system of
Moodle use that impede teachers from effectively utilising Moodle in their
instructional practices. By utilising activity theoretical underpinnings, the study
builds upon the foundation provided by the literature, guiding its focus towards
addressing the identified limitations and offering practical recommendations for
policy to enhance the utilisation of Moodle in the post-pandemic educational
milieu.
3. Theoretical Framework
This investigation employed Activity Theory (Engeström, 1999) as a theoretical
framework because it is a useful approach for understanding complex
relationships between various elements in a collective system and individual
actions (Kaptelinin & Nardi, 2006). It is applicable in higher education research
because it enables the identification of an individual's practices within a
sophisticated systemic context that involves multiple related actions and
operations and highlights different contradictions within the broader system
(Bligh & Flood, 2017). It offers a contextual method for understanding human
interactions using tools in a complex system (Hashim & Jones, 2007). In this paper,
Moodle use is regarded as a complex activity in which different teachers are
involved in practicing various operations within the main activity.
Engeström's Activity Systems Model (ASM) (Engeström, 2014) is a triangular

representation of an activity that visually depicts the elements of the activity and
their interrelationships, including subject, artifact, object, outcome(s), rules,
community, and division of labour (see Figure 1). The ASM serves as the focus of
analysis in research contexts that use the ASM. This model enables the
identification of the elements within the activity under investigation, as well as
the recognition of contradictions that arise either within or between these
elements (Al-Ali, 2020; Murphy, 2022). There are four types of contradictions that
can be observed within an activity system. Primary contradictions emerge within
the elements of the activity system itself, while secondary contradictions arise
between different elements of the same activity system. Tertiary contradictions
occur between various versions of the same activity system, and quaternary
contradictions exist between neighbouring activity systems (Engeström &
Sannino, 2010).
197
Figure 1: Engeström’s Activity Systems Model (Engeström, 2014, p. 63)
In this study, the Activity Theory was utilised as a framework to guide the
selection of case study methodology, which was deemed suitable for the research
(Al-Ali, 2020). The Activity Theory provided a comprehensive perspective that
captured the multidimensional nature of the activity and the contradictions
within the system (Engeström, 2000). Moreover, the Activity Theory, along with
the Activity System Model, informed the formulation of focus group discussion
questions for data collection. These questions were designed to identify the
elements of the system under investigation and uncover any contradictions. For
data analysis, the study employed Activity Systems Analysis, which is a
methodology that uses the Activity System Model as a guide to analyse complex
learning environments (Engeström, 2000; Yamagata-Lynch, 2010). This approach
facilitated the identification and description of the elements within the activity
system of Moodle usage. The pre-defined vocabulary within the Activity System
Model was used to accurately depict each element. Subsequently, the Activity
Theory and the Activity System Model played a crucial role in reporting the
findings from the focus group discussions and discussing their implications (Al-
Ali, 2020; Yamagata-Lynch, 2010).
4. Methodology and Methods

4.1 Research design
To investigate complex and embedded educational phenomena (Moodle usage in
higher education), a case study research methodology was adopted based on an
interpretive epistemological paradigm (Creswell, 2018). It facilitated a thorough
exploration of key aspects of the case and allowed for plausible interpretations
within the specific contextual and localized bounded space and time (Tight, 2017).
The case was bounded to a single higher education institution in Oman and
focused on online teaching and learning practices in the post-pandemic era. By
establishing specific boundaries of space and time around the case, it was possible
to delve into the phenomenon within its authentic and real-life context. These
boundaries allowed for a focused exploration of the case, ensuring that the
investigation stayed within the natural parameters of the phenomenon being
studied (Algozzine & Hancock, 2017). The teachers involved possessed
198
experience in using Moodle, having taught online for four academic semesters
during the pandemic, with each semester lasting 15 weeks. This provided a rich
and authentic backdrop for examining the case.
The case study methodology enabled an in-depth analysis of this specific case
(Berg, 2001), while incorporating Activity Theory and Activity System Analysis
provided a complementary approach for gaining a deeper understanding of the
systemic elements and contradictions within the case. Activity Theory served as a
valuable lens for analysing the activities and interactions constituting the
phenomenon (Engeström, 2000, Bligh & Flood, 2017; Yamagata-Lynch, 2010).
4.2 Research site

The research was conducted in a higher education institution in Oman in which
Moodle was the official learning platform for all students and teachers. Each
course had a dedicated Moodle page. The Moodle administrator managed course
pages, enrolment, technical issues, and provided support to teachers.
Departments had e-learning coordinators overseeing Moodle use. The E-learning
Implementation Policy and Moodle Guide (a guide for teachers developed by the
institution) supported academic staff. The administrator held an annual Moodle
workshop for teachers and provided training sessions.
The use of activity theory as a theoretical framework influenced the establishment

of boundaries within the research site, restricting participation to teachers who
have used Moodle and are willing to use Moodle. Additionally, it informed the
time boundary of the study, encompassing the period of post-pandemic.
4.3 Participants
The population consisted of teachers with masters' and PhD degrees and a
minimum of five years of teaching experience, all of whom had undergone
training in Moodle. The teachers were from diverse cultural backgrounds.
Convenient sampling techniques were employed to recruit the study sample
(Creswell, 2018), based on the availability of teachers from the population. The
final participants comprised six teachers who possessed over five years of
teaching experience, demonstrating extensive familiarity with Moodle.
Furthermore, they had actively engaged in online teaching during the pandemic
and made efforts to continue using Moodle post-pandemic (see Table 1). The
participants were teaching different specialisations in a higher education context.
Table 1: Participants demographic information

Participant Specialisation Teaching
experience
Teacher A Business studies 11 years
Teacher B English Language Teaching 6 years
Teacher C English Language Teaching 7 years
Teacher D Information Technology 13 years
Teacher E Mathematics 8 years
Teacher F English Language Teaching 12 years
199
4.4 Data collection and analysis

A focus group discussion was conducted with six teachers who had experience
teaching online during the COVID-19 pandemic and attempted to continue to use
Moodle after the pandemic. This method was appropriate for retrieving personal
attitudes, views, and beliefs (O. Nyumba et al., 2018) and allowing teachers to
share and reflect on their experiences with each other (Dilshad & Latif, 2013). It
ensured eliciting rich data through group members’ interactions (Rabiee, 2004),
which was sufficient to be exploited using ASM (Activity System Model).
Additionally, it allowed participants to offer their insights and views concurrently
(Hennink, 2013). The activity theory was used to develop the discussion questions
to elicit information about the activity and its components. It also aided to
formulate questions to identify any contradictions within the activity system.
The study adopted a deductive thematic analysis approach based on Braun and
Clarke’s (2006) six-step process: data familiarization, coding data, searching for
themes, reviewing themes, defining and naming themes, and writing up. The
ASM guided the iterative reading of the data and sketching of the activity system.
A set of contradictions were identified in the model, which was used to code the
data and later to name the themes. Four themes (contradictions in the activity
system) were identified. The names of the themes have been constructed in
relation to the ASM keywords.
4.5 Ethical considerations and issues of trustworthiness

Ethical approval was obtained from the Doctoral Programme in e-Research and
Technology Enhanced Learning via the Module Convenor. Participants were
informed about the study using an information sheet that outlined the research's
purpose, questions, and rationale for their inclusion. They were informed that
their participation was voluntary, and they were asked to sign a consent form to
participate. To facilitate the focus group discussion, a guide was created
containing some questions. To ensure the validity of the questions, they were
reviewed by three academics.
To maintain the trustworthiness, respondent validation (Tracy, 2010) was

employed. This involved sharing a draft of the main findings with the participants
to check for accuracy and completeness. Additionally, to ensure the quality of the
research, peer debriefing (Stahl & King, 2020; Tracy, 2010) was conducted by a
colleague in the TEL programme (PhD programme in Technology Enhanced
Learning) with knowledge of the context and methodology of the investigation.
This individual provided feedback on the methodological considerations and
results interpretations. Furthermore, a critical friend methodology was used to
review and enhance the quality of this research.
5. Findings
5.1 The Elements Moodle Implementation Activity System
To study the implementation of online teaching and learning with the use of
Moodle, the Activity System Model (Figure 2) was used to sketch the data, identify
and outline the different components of the activity. The findings demonstrated
that the participating teachers (the subjects) were actively engaged in the
200
implementation of online teaching and blended learning (the object). When

questioned about their goals, all teachers reported utilising Moodle as the
platform for implementing online teaching and blended learning, considering it
as an appropriate tool. Teacher B stated, "Moodle is the appropriate tool for managing
online as well as blended learning." The dataset indicated that the activity was
governed by two key documents: the Moodle Guide and the E-learning
Implementation Policy (the rules). According to the teachers, these documents
provided the guidelines and regulations for conducting the activity and specified
the roles and responsibilities of various stakeholders, including the course
coordinator, e-learning coordinator, Moodle administrator, and the ETC:
Educational Technologies Centre (the division of labour). Furthermore, in
addition to these individuals, the activity encompassed a broader community that
consisted of students, other teachers, parents, and policy makers. Most teachers
identify their involvement in the activity as significant and contributed to its
overall dynamics and outcomes.
Figure 2: Moodle usage activity system
The findings indicated two types of contradictions: primary contradictions

(contradictions within the elements of the system) and secondary contradictions
(contradictions between elements of the system). Figure 3 represents the identified
contradictions in the activity system. Following is a detailed discussion of the
primary and secondary contradictions.
5.2 Primary Contradictions in the Activity System

5.2.1. Ineffective Rules and Policies
Based on the dataset, it became evident that there were some significant
discrepancies within the rules in the system. All teachers agreed that the Moodle
guide only provided general instructions on how to use the system from a
technical perspective. Teacher B stated that the guide was very basic and only
demonstrated how to create activities in Moodle. Teacher C said, “the guide was
text heavy”. She mentioned that watching a YouTube video would be more helpful
than reading the guidelines. Similarly, Teacher E recommended that the guide be
updated to include pedagogical tips on how to use each feature in the system.
201
Teacher E said, “the guide can include a list of Moodle activities and a pedagogical
explanation on how each activity can be used to enhance teaching and learning.” Even
though the E-learning Implementation Policy outlined the roles and
responsibilities of concerned people, it did not clearly state how each person
should conduct his role. This caused a contradiction in the division of labour
element in the activity system.
5.2.2. Confusing Tasks Allocation

The findings revealed that the responsibilities of implementing Moodle were not
well-defined among the different parties involved, particularly between the
course coordinator and the e-learning coordinator. This caused a primary
contradiction within the rules element of the activity system (see Figure 3). The
course coordinator was tasked with creating course materials and updating the
Moodle page with assistance from the course teachers. However, Teacher A and
Teacher F reported that the coordinator was also involved in enrolling and
registering students on Moodle, a responsibility that falls under the jurisdiction
of the e-learning coordinator and the Moodle administrator. Teacher A said,
“coordinators were overloaded with admin tasks related to Moodle such as
registering/enrolling students which left them with less time for course development and
redesign”. It is possible that the e-learning coordinator's involvement in other
duties, in addition to their teaching load, could have contributed to this confusion.
Furthermore, the large number of students in each department may have required
the course teachers and coordinator to perform some of the e-learning
coordinator's tasks. Teacher C said, “sometimes if the course coordinator and teachers
help in registering and enrolling students in Moodle, then teachers won’t be able to use
the system from the first week of classes.”
Artifacts
Moodle
Outcome
Enhance teaching and learning
Subject Object
Teachers Implement online
teaching
Rules Division of Labour

Moodle guide Community Course coordinator
E-Learning Students e-learning coordinator
Implementation Policy Colleagues
Parents
Policy makers
Legend:
Primary contradiction
Secondary contradiction
Figure 3: Identified contradictions in the Moodle usage activity
202
5.3. Secondary Contradictions in the Activity System

5.3.1. Challenges Faced by Teachers Due to Ambiguous Division of Labour
The results revealed that teachers faced several challenges due to an ambiguous
division of labour. This posed a secondary contradiction in the activity system
between the division of labour and the subject (see Figure 3). One of the major
challenges was an increased workload because of unclear roles and
responsibilities. They were also taking the roles of the course coordinator
regarding the design and update of the materials, and the roles of the e-learning
coordinator in enrolling students in Moodle. Teacher C said, “In addition to
designing content for teaching in Moodle, we had a lot of other administrative tasks caused
by the unsodden shift to online teaching and learning.” Teachers found it difficult to
manage their workload effectively, as they were unsure of what tasks they were
responsible for and what tasks were assigned to other members (course
coordinator and e-learning coordinator). Additionally, the lack of clear division
of labour made it difficult for teachers to keep updating their courses in a timely
manner. Teacher A said, “it wasn’t clear who should be responsible for redesigning the
materials in Moodle and who should be responsible for grouping students in Moodle.”
Teachers were unsure of what materials they were responsible for updating,
which led to inconsistencies in course content. This was caused because the course
coordinator had teaching load and was busy with his/her own students. Another
challenge that teachers faced was an inability to track student enrolment
effectively. Without a clear understanding of their roles and responsibilities,
teachers found it difficult to monitor student enrolment accurately, leading to
confusion and errors in student records. Also, the teachers reported that it was
difficult to keep the practice consistent because of lack of communication between
the people involved in the system.
5.3.2. Consequences of Ineffective Rules and Policies for Teachers

The results of the study indicated that the unclarity of rules and policies placed
pressure on teachers to seek out online resources to learn how to create activities
in Moodle. This indicated another secondary contradiction in the activity system
between the rules and the subject (see Figure 3). Many teachers reported spending
a significant amount of time trying out different activities in Moodle to determine
which ones would be most effective for their students. Teacher C said, “I spent the
first weeks of the semester exploring different activities in Moodle. I reflected on my
experience and then found two activities, ‘quiz’ and ‘lesson’ to be very appropriate for my
class.” Additionally, teachers reported struggling with some tasks in Moodle that
were deemed necessary but did not have the necessary training or knowledge to
use them properly. Teacher B said, “I had to sit with the Moodle administrator and
some of my colleagues to train me on how to perform some tasks in Moodle, as they were
not highlighted clearly in the training sessions.”
6. Discussion
The findings indicated a primary contradiction in the rules of the Moodle activity
system as the Moodle guide and E-learning Implementation Policy did not
provide enough information and tips for teachers to effectively use Moodle. As a
result, many teachers stopped using Moodle altogether. This aligns with previous
research that policies related to digital literacy are crucial to ensure that teachers
and students have the necessary skills to use online learning platforms effectively
203
(Al Neyadi et al., 2021; Muirhead & Juwah, 2020). During the pandemic,
institutions had to adapt/establish polices on digital literacy which explained
their successful implementation of learning platforms. For example, the United
Arab Emirates implemented policies aimed at enhancing digital literacy skills
among teachers to support the adoption of online learning (Al Neyadi et al., 2021).
However, these polices may also pose challenges, such as concerns regarding
academic integrity and student privacy (Kebritchi et al., 2017). Therefore, policies
aimed at addressing these concerns are essential for teachers to continue their use
of these online platforms for online assessment. They need to be adaptable and
flexible to meet the need of teachers in the post-pandemic era (Dabbagh &
Kitsantas, 2012; Poellhuber et al., 2017). Dabbagh and Kitsantas (2012) believe that
traditional policies (the existing one in the context of this investigation), which are
often rigid and inflexible, can be a barrier to innovation and change in higher
education. Also, they should be developed in collaboration with faculty, staff, and
students to ensure that they are relevant and effective (Muirhead & Juwah, 2020).
The issue of unclear roles and responsibilities between the course coordinator and
e-learning coordinator was identified as another primary contradiction, which
resulted in confusion and hindered the smooth functioning of Moodle
implementation. Previous investigations have reported that such clashes of
responsibilities can have negative impacts on teachers and students, causing
stress and burden for all parties involved (Al-Fraihat et al., 2021; Kizilcec et al.,
2021; Oliveira et al., 2022). This aligns with previous investigations that Moodle
use declined due to teachers’ overload (Bond et al., 2021). It is important to note
that task allocation to staff responsible for Moodle and online teaching and
learning needs to be systematic and clear to avoid burden and stress that may
discourage teachers from using Moodle.
The findings revealed a secondary contradiction between the division of labour

and the teachers. When individuals responsible for specific tasks did not carry
them out effectively, teachers had to take on additional responsibilities beyond
their teaching and administrative duties. This led to higher stress and burnout
levels and teachers felt frustrated and overwhelmed. Previous research indicated
that this might disrupt online teaching and learning (Aydin & Tasci, 2020; Kizilcec
et al., 2021; Oliveira et al., 2022) The second secondary contradiction was between
the rules and the teachers. Due to unclear guidelines, teachers had to invest time
and effort to develop effective strategies to use Moodle in their teaching. This also
aligned with Chen et al., (2021) and Lau (2021) who associated the decline of
Moodle use with teachers’ overload.
7. Conclusion
The results revealed two primary contradictions in the rules and the division of
labour of the system and secondary contradictions between division of labour and
subjects and the rules and the subjects that led to the decline in Moodle usage.
These issues included ineffective rules and policies and confusing task allocation.
This research endeavour makes significant contributions to the literature on post-

pandemic pedagogy and the utilization of Moodle. It underscores the importance
204
of teacher training and support as well as adaptable policies. Moreover, it reveals

contradictions within the activity system of Moodle usage that impact local
policies and practices. Consequently, there is a pressing need for well-defined
policies that clearly outline the roles and responsibilities of those involved in the
implementation of Moodle, ensuring a comprehensive understanding of
expectations. Additionally, addressing teachers' workload is crucial to enable
effective course preparation and administration within Moodle. Furthermore, this
investigation contributes to theoretical advancements by highlighting the
significance of activity theory in identifying contradictions within systems,
thereby fostering organizational change and development. Understanding and
resolving these contradictions can aid organisations in adapting and enhancing
their utilisation of educational technology.
It is important to acknowledge the limitations of this study, including the small

sample size and potential sampling bias. The study was also constrained by time
limitations and restricted data collection methods, preventing an examination of
neighbouring activity systems related to Moodle use. Future research efforts may
benefit from employing interventionist research methodologies, such as Design-
Based Research (Brown & Campione, 1996) or the Change Laboratory (Virkkunen
& Newnham, 2013), to address the decline in Moodle usage in the post-pandemic
era. Additionally, it is recommended that future investigations utilizing the same
theoretical framework incorporate a variety of data collection methods, such as
stakeholder interviews and analysis of policy documents, to gather
comprehensive and nuanced data, enabling a deeper understanding of the
phenomenon under study.
8. References
Adeyemo, D. A., Olawumi, T. O., & Oyeyinka, I. A. (2021). University lecturers’
acceptance of Moodle platform in the context of the COVID-19 pandemic.
Education and Information Technologies, 1-18. http://doi.org/10.1007/s10639-021-
10522-9
Al Neyadi, S., Kamal, M. M., & Alkhaja, S. (2021). UAE teachers' readiness and perceptions
towards e-learning during COVID-19. International Journal of Emerging
Technologies in Learning (iJET), 16(2), 120-136.
http://doi.org/10.3991/ijet.v16i02.12609
Al-Ali, S. (2020). Activity systems analysis: A maze worth exploring. Studies in Technology
Enhanced Learning, 1(1). doi: 10.21428/8c225f6e.dc494046
Aldemir, Ş. & Doğan, M. (2021). Pedagogical approaches during the COVID-19 pandemic:
A review of literature. Education and Information Technologies, 26(4), 5407-5424.
http://doi.org/10.5772/intechopen.104921
Al-Fraihat, D., Joy, M., & Masa'deh, R. (2021). Stress and burnout among faculty members
during COVID-19 pandemic: An empirical study. Education and Information
Technologies, 26(1), 237-248. http://doi.org/10.1007/s10639-020-10443-6
Alqahtani, M. (2020). The role of learning management systems in facilitating distance
learning during the COVID-19 pandemic: A case study from Saudi Arabia.
Education Sciences, 10(9), 234. http://doi.org/10.3390/educsci10090234
Alqurashi, E. & Alhashmi, H. (2021). E-learning in higher education amid COVID-19
pandemic: Leadership perspectives. International Journal of Higher Education, 10(3),
92-107. http://doi.org/10.5430/ijhe.v10n3p92
205
Alqurashi, E. (2020). Challenges and Opportunities of E-Learning in the Kingdom of Saudi

Arabia during the COVID-19 Pandemic. Journal of Education and e-Learning
Research, 7(3), 229-236. http://doi.org/10.20448/journal.509.2020.73.229.236
Ashfaquzzaman, M. (2020). Pandemic pedagogy in post-COVID age. Communication
Education, 69(4), 534–535. http://doi.org/10.1080/03634523.2020.1804130
Berényi, L. (2022). Use of Moodle among public service students: pandemic effects. In
Proceedings of the Central and Eastern European eDem and eGov Days (pp. 18-24).
http://doi.org/10.1145/3551504.3551544
Berg, B. (2001). Qualitative Research Methods for the Social Sciences.
Betts, K., Keeler, A., & Kennedy, K. (2020). The transition to online teaching in response to
the COVID-19 pandemic: A qualitative exploration of faculty experiences. Online
Learning, 24(2), 207-225. http://doi.org/10.24059/olj.v24i2.2287
Bligh, B., & Flood, M. (2017). Activity theory in empirical higher education research:
choices, uses and values. Tertiary Education and Management, 23, 125-152.
http://doi.org/10.1080/13583883.2017.1284258
Bond, M., Bedenlier, S., Marín, V. I., & Händel, M. (2021). Emergency remote teaching in
higher education: Mapping the first global online semester. International Journal of
Educational Technology in Higher Education, 18(1), 1-24.
http://doi.org/10.1186/s41239-021-00282-x
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research
in Psychology, 3(2), 77–101. doi.org/10.1191/1478088706qp063oa
Brown, A. L., & Campione, J. C. (1996). Psychological theory and the design of innovative
learning environments: On procedures, principles, and systems. In L. Schauble &
R. Glaser (Eds.), Innovations in learning: New environments for education (pp. 289-
325). Mahwah, NJ: Lawrence Erlbaum Associates.
Çalışkan, E. & Yalın, H. I. (2021). Designing group activities on Moodle: Challenges and
solutions. International Journal of Emerging Technologies in Learning, 16(7), 156-168.
http://doi.org/10.3991/ijet.v16i07.13476
Chen, Y., Chen, N. S., & Ko, H. C. (2021). Teacher perspectives on adopting online learning
after COVID-19: A case study in Taiwan. Education Sciences, 11(4), 159.
http://doi.org/10.3390/educsci11040159
Crawford, J., Butler-henderson, K., Rudolph, J., Malkawi, B., Glowatz, M., Magni, P., &
Lam, S. (2020). COVID-19: 20 countries’ higher education intra-period digital
pedagogy responses. Journal of Applied Learning & Teaching, 3(1), 1–20.
http://doi.org/10.37074/jalt.2020.3.1.7
Creswell, J. W. (2018). Educational Research: Planning, Conducting and Evaluating
Quantitative and Qualitative Research (6th edition). Pearson.
Dabbagh, N., & Kitsantas, A. (2012). Personal learning environments, social media, and
self-regulated learning: A natural formula for connecting formal and informal
learning. Internet and Higher Education, 15(1), 3-8.
http://doi.org/10.1016/j.iheduc.2011.06.002
de Oliveira, E. A., Santos, V. S., Pinheiro, S. V. B., & de Souza, E. L. (2022). Clashes of
responsibilities in online teaching: A challenge to faculty workload management.
Education and Information Technologies, 27(1), 61-77.
http://doi.org/10.1007/s10639-021-10586-6
Dilshad, R. M., & Latif, M. I. (2013). Focus group interview as a tool for qualitative
research: An analysis. Pakistan Journal of Social Sciences (PJSS), 33(1).
http://doi.org/10.4135/9781506335179.n5
Engeström, Y. (2000). Activity theory as a framework for analyzing and redesigning work.
Ergonomics, 43(7), 960-974. http://doi.org/10.1080/001401300409143
Engeström, Y. (2015). Learning by expanding: An activity-theoretical approach to developmental
research. Cambridge University Press.
206
Engeström, Y. 1999. Activity theory and individual and social transformation. In

Perspectives on activity theory, ed. Y. Engeström, R. Miettinen, and R.-L. Punamäki,
19–38. Cambridge: Cambridge University Press.
http://doi.org/10.1017/cbo9780511812774
Engeström, Y., & Sannino, A. (2010). Studies of expansive learning: Foundations, findings
and future challenges. Educational research review, 5(1), 1-24.
http://doi.org/10.1016/j.edurev.2009.12.002
Gao, X., Zhang, Y., & Chen, X. (2020). Exploring the challenges and opportunities of online
learning amid the COVID-19 pandemic. Journal of Educational Technology
Development and Exchange, 13(1), 1-14. http://doi.org/10.18785/jetde.1301.01
Hennink, M. M. (2013). Focus Group Discussions. Oxford University Press.
Hodges, C., Moore, S., Lockee, B., Trust, T., & Bond, A. (2020). The difference between
emergency remote teaching and online learning. Educause Review, 7.
https://er.educause.edu/articles/2020/3/the-difference-between-emergency-
remote-teaching-and-online-learning
Kabakci Yurdakul, I. & Inan, F. A. (2020). Adaptation of a course management system
during COVID-19: A phenomenological study. Education and Information
Technologies, 25(6), 5555-5570. doi: 10.1007/s10639-020-10295-6
Kant, N., Prasad, K. D., & Anjali, K. (2021). Selecting an appropriate learning management
system in open and distance learning: A strategic approach. Asian Association of
Open Universities Journal, 16(1), 79-97. http://doi.org/10.1108/AAOUJ-09-2020-
0075
Kaptelinin, V., & Nardi, B. A. (2006). Acting with Technology: Activity Theory and Interaction
Design. MIT Press. http://doi.org/10.5210/fm.v12i4.1772
Kebritchi, M., Lipschuetz, A., & Santiague, L. (2017). Issues and challenges for teaching
successful online courses in higher education: A literature review. Journal of
Educational Technology Development and Exchange, 10(1), 1-14.
http://doi.org/10.18785/jetde.1001.01
Khan, B. H., & Ghani, U. (2020). Teachers' perception and challenges of E-Learning during
COVID-19 pandemic. Journal of Education and Educational Development, 7(2), 187-
200. http://doi.org/10.22555/joeed.v7i2.3585
Kizilcec, R. F., Cortes, E. D., & Hancock, J. T. (2021). Masking interruptions: Multitasking
online classes with mobile messaging reduces engagement and learning.
Computers & Education, 170, 104274.
http://doi.org/10.1016/j.compedu.2021.104274
Lau, K. H. (2021). Learning in the pandemic and post-pandemic era: The implications of
COVID-19 on education. Postdigital Science and Education, 3(1), 233-238.
http://doi.org/10.1007/s42438-020-00101-8
Lockee, B. B. (2021). Online education in the post-COVID era. Nature Electronics, 4(1), 5-6.
http://doi.org/10.1038/s41928-020-00534-0
McElroy, D. (2021). What have we learned and where are we headed? Journal of Digital
Learning in Teacher Education, 37(2), 82-83.
http://doi.org/10.1080/21532974.2021.1893112
Mihai, F. A., & Dragoș, D. (2021). Shaping the future of learning in post-pandemic
education: An exploratory study on students' perceptions of online learning.
Education Sciences, 11(1), 30. http://doi.org/10.3390/educsci11010030
Mohammadi, M. K., Mohibbi, A. A., & Hedayati, M. H. (2021). Investigating the challenges
and factors influencing the use of the learning management system during the
Covid-19 pandemic in Afghanistan. Education and Information Technologies, 26,
5165-5198. http://doi.org/10.1007/s10639-021-10517-z
Mpungose, C.B. (2020). Is Moodle or WhatsApp the preferred e-learning platform at a
South African university? First-year students’ experiences. Education and
207
Information Technologies, 25(2). 927-941. http://doi.org/10.1007/s10639-019-

10005-5
Muirhead, B., & Juwah, C. (2020). Developing digital literacy policies in higher education.
Journal of Learning, Teaching and Technology, 5(1), 40-53.
http://doi.org/10.22316/jltt.v5i1.114
Murphy, M. P. (2020). COVID-19 and emergency eLearning: Consequences of the
securitization of higher education for post-pandemic pedagogy. Contemporary
Security Policy, 41(3), 492-505. http://doi.org/10.1080/13523260.2020.1761749
Murphy, T. (2022). Exposing the uncomfortable: Activity theory and the limitations of the
academic in the world of TEL and programme development. Studies in Technology
Enhanced Learning, 3(1). http://doi.org/10.21428/8c225f6e.94adb00d
O. Nyumba, T., Wilson, K., Derrick, C. J., & Mukherjee, N. (2018). The use of focus group
discussion methodology: Insights from two decades of application in
conservation. Methods in Ecology and Evolution, 9(1), 20–32.
http://doi.org/10.1111/2041-210x.12860
Oman Academic Accreditation Authority. (2016). Institutional Standards Assessment
Manual. https://oaaaqa.gov.om/getattachment/b9b9ef59-ab80-4883-9b28-
3e202f37f64f/Institutional%20Standards%20%20Assessment%20Manual.aspx?b
=0
Pătrașcu, G., & Grosseck, G. (2021). A view from the inside: Teachers’ perspective on using
a learning management system during the COVID-19 pandemic. Computers &
Education, 164, 104127. http://doi.org/10.1016/j.compedu.2020.104127
Poellhuber, B., Chomienne, M.-H., Karsenti, T., & Gagnon, R. (2017). Policy for blended
learning in higher education: A systematic review. The Internet and Higher
Education, 34, 1-13. http://doi.org/10.1016/j.iheduc.2017.05.001
Rabiee, F. (2004). Focus-group interview and data analysis. Proceedings of the Nutrition
Society, 63(4), 655–660. http://doi.org/10.1079/pns2004399
Salas-Rueda, R. A., Eslava-Cervantes, A. L., & Prieto-Larios, E. (2020). Teachers’
perceptions about the impact of Moodle in the educational field considering data
science. Online Journal of Communication and Media Technologies, 10(4), e202023.
http://doi.org/10.30935/ojcmt/8498
Sarnou, H., & Dallel, S. (2021). Investigating the EFL Courses Shift into Moodle during the
Pandemic of COVID–19: The case of MA Language and Communication at
Mostaganem University. Arab World English Journal (AWEJ) Special Issue on Covid,
19. http://doi.org/10.24093/awej/covid.26
Stahl, N. A., & King, J. R. (2020). Expanding approaches for research: Understanding and
using trustworthiness in qualitative research. Journal of Developmental Education,
44(1), 26-28. https://files.eric.ed.gov/fulltext/EJ1320570.pdf
Taamneh, A., Alsaad, A., Elrehail, H., Al-Okaily, M., Lutfi, A., & Sergio, R. P. (2022).
University lecturers acceptance of moodle platform in the context of the COVID-
19 pandemic. Global Knowledge, Memory and Communication.
http://doi.org/10.1108/gkmc-05-2021-0087
Toquero, C. M. (2020). Challenges and opportunities for higher education amid the
COVID-19 pandemic: The Philippine context. Pedagogical Research, 5(4), 1-11.
http://doi.org/10.29333/pr/7937
Tracy, S. J. (2010). Qualitative quality: Eight a"big-tent" criteria for excellent qualitative
research. Qualitative Inquiry, 16(10), 837–851.
http://doi.org/10.1177/1077800410383121
Tsai, C. H., Rodriguez, G. R., Li, N., Robert, J., Serpi, A., & Carroll, J. M. (2020).
Experiencing the transition to remote teaching and learning during the COVID-
19 pandemic. Interaction Design & Architecture (s), 46, 70.
http://doi.org/10.55612/s-5002-046-004
208
Virkkunen, J., & Newnham, D. (2013). The Change Laboratory: A Tool for Collaborative
Development of Work and Education. SensePublishers. http://doi.org/10.1007/978-
94-6209-326-3
Wang, Y. (2021). In-Service Teachers' Perceptions of Technology Integration and Practices
in a Japanese University Context. JALT CALL Journal, 17(1), 45-71.
http://doi.org/10.29140/jaltcall.v17n1.377
Yamagata-Lynch, L. C. (2010). Activity Systems Analysis Methods: Understanding Complex
Learning Environments. Springer Science & Business Media.
Yang, M., & Chen, W. (2021). Analyzing the challenges of learning management system
adoption during the COVID-19 pandemic: Perspectives from teachers. Interactive
Learning Environments, 29(5), 673-686.
Zabolotniaia, M., Cheng, Z., Dorozhkin, E., & Lyzhin, A. (2020). Use of the LMS Moodle
for an effective implementation of an innovative policy in higher educational
institutions. International Journal of Emerging Technologies in Learning (iJET), 15(13),
172-189.
Zamora-Antuñano, M. A., Rodríguez-Reséndiz, J., Cruz-Pérez, M. A., Rodríguez
Reséndíz, H., Paredes-García, W. J., & Díaz, J. A. G. (2022). Teachers’ perception
in selecting virtual learning platforms: A case of Mexican higher education during
the COVID-19 crisis. Sustainability, 14(1), 195.
http://doi.org/10.3390/su14010195
209

Vol. 22, No. 10, pp. 209-230, October 2023
Collaborative Learning in Higher Education in

the Fourth Industrial Revolution: A Systematic
Literature Review and Future Research
Adi Bandono , Mukhlis Mukhlis , A. K. Susilo* ,
A. R. Prabowo and Acep Maksum
Indonesia Navy Technology College
Bumi Moro, Morokrembangan, Surabaya, Indonesia
Abstract. The goal of this study is to determine the variables and

difficulties that contribute to collaborative learning at tertiary institutions
during the fourth industrial revolution and future research gaps. This
study employs a systematic literature review to summarise and provide
a comprehensive understanding of collaborative learning. The systematic
literature review process used the Preferred Reporting Items for
Systematic Review and Meta-Analysis Protocols (PRISMA-P) technique
between 2008 and 2022, supported by NVivo and Microsoft Excel. A
systematic literature review, which is an appropriate method for
strengthening research issues and gaps, was conducted with 61 papers. It
should be noted that this manuscript only presents work from the Scopus
database with journal criteria. This study has identified six factors related
to collaborative learning in tertiary institutions in the fourth industrial
revolution era including Task & Context, Environment, Social,
Technology, Teacher, and Learner. This study presents a comprehensive
literature assessment of collaborative learning in tertiary institutions
during the fourth industrial revolution in order to acquire a clear grasp of
its importance and application in higher education. Furthermore, Several
higher education models have adopted the online-based collaborative
learning paradigm, according to the study's findings.
Keywords: Collaborative learning (CL); Higher Education; Fourth

Industrial Revolution (4IR); Preferred Reporting Items for Systematic
Review and Meta-Analysis Protocols (PRISMA-P)
1. Introduction
The 21st century is characterized by its quick technical development and the
effects of the so-called fourth industrial revolution (West & Malatji, 2021). The
educational innovation landscape has changed as a result of the fourth industrial
*
Corresponding author: A.K. Susilo; akukuhsusilo@sttal.ac.id
©Authors
210
revolution (4IR), requiring people to engage in innovative thought on

manufacturing processes, value chains, and customer service procedures
(Scepanovic, 2019). Colleges and universities offering continuing and tertiary
education have become an integral part of the government's attempts to increase
access to historically underrepresented groups and to promote collaboration
between institutions of further education and universities (Campbell et al., 2012).
In the current online learning environment, particularly during the COVID-19
epidemic, collaborative learning as a teaching and learning technique encourages
students to become active participants and engage with one another (Matee et al.,
2022). Collaborative learning refers to an instructional paradigm in which
students acquire knowledge via collaborative projects. In the setting of
collaborative learning, students cooperate towards a common objective in a
cooperative attitude, putting aside rivalry (Isaías, 2018).
According to Ming et al. (2021), the perceptions of collaborative learning and the
usage of technology can promote autonomous learning among students. In
addition, significant consideration must be given to the design of collaborative
learning activities to ensure that they are sufficiently difficult to require a high
cognitive load tied to students' past knowledge (Goedhart et al., 2019).
Opportunities for collaborative learning experiences can increase learning
effectiveness (Gamage et al., 2020). However, there is a need to develop and
deliver collaborative learning and evaluation methodologies that may be tailored
to the talents and capabilities of individual students (Barberà et al., 2022).
According to Su and Zou (2020), analysing studies from various dimensions of
collaborative learning can become further literature. Furthermore, it is necessary
to evaluate the adoption of learning analysis literature by conducting a systematic
literature review on collaborative learning (Gasevic, 2019). Consequently, The aim
of this study is to identify, through a literature review, the components of
collaborative learning in tertiary institutions during the fourth industrial
revolution. This article is an attempt to answer three main questions:
- What are the key elements of collaborative learning in higher education in the
era of 4IR?
- What are the challenges faced by institutions in Collaborative Learning in
Higher Education in the era of 4IR?
- What research gaps can guide future research regarding collaborative learning
in higher education in the era of 4IR?
Collaborative learning refers to an instructional paradigm in which student

learning results from group effort. In the framework of collaborative learning,
students work together rather than competitively towards a common objective
(Isaías, 2018) Students can improve their academic performance by engaging in
active collaborative learning, chatting with supervisors or lecturers, speaking with
group members or peers, and being involved (Alismaiel et al., 2022). It's also
crucial to remember that involving children in group activities in the classroom
can help them gain the conceptual understanding they need to sharpen their
critical thinking abilities (Al et al., 2020). Therefore, exploring additional trends
identified from a social perspective, and other dimensions as a mechanism for
validating findings and perfecting technological capability sets (Castro, 2019)
211
complements the existing literature on collaborative learning theory throughout

the fourth industrial revolution.
This study uses a systematic review to collect publications on the topic at hand,
summarise them, and provide a comprehensive overview of collaborative
learning as according to Gupta et al. (2021). This study uses the systematic
literature review process suggested by Bodolica and Spraggon (2018) using
modified Preferred Reporting Items for Systematic Review and Meta-Analysis
Protocols (PRISMA-P) technique, NVivo, and Microsoft Excel. The data used are
from the Scopus database between 2008 and 2022.
This research provides several contributions. This study expands our

understanding of the factors that influence collaborative learning in universities
during the RI 4.0 era according to Gasevic (2019). Second, this study can develop
a roadmap for future research and highlight the possibility of developing
literature on collaborative learning factors for academics and practitioners by
enriching collaborative learning theory through review articles. Third, it poses
new research questions, including original discussions about collaborative
learning and background knowledge and encouraging discussion of new
dimensions of collaborative learning in universities in the RI4 era.
This article consists of several sections. Section two describes the methodology
which consists of a systematic literature review, search strategy, selection criteria,
study selection, and analysis method. Section three explains the Results &
Analysis which consists of descriptive statistics, factors, challenges, gaps, and
future research. Section four consists of a conclusion, limitations, and
recommendations for future research.
2.1. Collaborative Learning
Although collaborative learning is a popular teaching strategy, it is frequently
underutilised in actual classroom settings (Scager et al., 2016). Students are more
likely to externalise their ideas while participating in peer exchanges and
reflecting on those interactions during the collaborative learning process (Hong et
al., 2011). Collaborative learning is defined by Rowe et al. (2010) as employed by
a group of students to accomplish a common objective as a teaching strategy. A
three-dimensional model might be used to implement this kind of learning. The
axes in this model are as follows: (1) a group of people, whether they are alone or
in more than twos; (2) a reliable source of learning material (such as course
material, an activity, and lifelong job experience); and (3) a strategy for learning
that involves group members collaborating together. When students work in
groups, they engage in collaborative learning (Roberson & Kleynhans, 2019).
Students learn from one another through interaction through collaborative
learning, which is also described as “an instructional strategy in which students
at varying skill levels work together in small groups toward a common goal”
(Westbrook, 2012).
Collaborative learning has the potential to enhance learning and achievement as

well as motivate students to complete tasks, which raises their self-esteem and
212
fosters the growth of collaboration skills. Students can learn by exchanging

knowledge, helping one another out, and settling disagreements between their
own and other people’s perspectives (Webb & Mastergeorge, 2003). Students who
participate in collaborative learning must communicate and conceptualise their
ideas through interaction with other students and learning resources (John-
Steiner & Mahn, 1996), To choose the best answer to a problem, people should
converse, consider different viewpoints, and draw on prior knowledge
(Dewiyanti et al., 2007).
Higher education students who participated in interactive blogs were shown to

have a more optimistic outlook on social interaction and academic
accomplishment (Davidson & Major, 2014). Students can connect, work together,
and participate in a social atmosphere via social media (Alismaiel et al., 2022).
The use of social media curricula for teaching and learning in higher education is
of interest to the academic community (Pérez-López et al., 2020). Cognitive
abilities, motivation for active collaborative learning in higher education,
reflection, and metacognition are among the fundamental components of social
media (Alismaiel et al., 2022). According to numerous studies, using social media
for student tasks promotes higher levels of learning (Roberson & Kleynhans,
2019).
3. Methodology
3.1. Systematic Literature Review
The SLR process suggested by Bodolica and Spraggon (2018) was implemented in
this study with several adjustments as proposed by Elmashhara et al. (2022)
related to the analysis phase and answering research questions from Usman et al.
(2021). First, the topic and research objectives were determined. Secondly, the
search through the database was performed based on the predetermined
keywords and selection criteria. Third, articles were filtered and sorted out to
include only articles that met the inclusion and exclusion criteria. Fourth, relevant
materials were extracted to provide quality results. Systematic reviews differ from
traditional literature reviews in several significant ways (Riebe et al., 2016). This
approach takes the characteristics of research projects that are robustly designed,
and replicable and facilitate research interpretation based on research questions
(Zhao et al., 2021).
213
Figure 1. PRISMA diagram on a systematic literature review adapted from

Gupta et al. (2021).
To achieve the research objectives and identify relevant research papers, the
online database from Scopus was used. Scopus database has been curated by
experts and offers higher trust (Bodolica & Spraggon, 2018). In each database,
these keywords are analysed individually to broaden the scope of collaborative
learning research. As a criterion for study selection, we favour only peer-reviewed
publications, as they are a more trustworthy source of scientific knowledge (Salam
et al., 2019). This study looks into the elements that affect group work in higher
education during the fourth industrial revolution using a modified Preferred
Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P)
technique from Männistö et al. (2019), NVivo and Microsoft Excel. This study
consisted of articles that were searched using the following queries for the title,
abstract and keywords: “collaborative learning”; “higher education”; “online
collaborative learning”; “collaborative e-learning”; “collaborative learning
process”.
One of the most crucial and significant phases of a systematic literature review is
the selection of the studies to be used (Salam et al., 2019). Regarding the rationale
for quality assessment (QA), this article selects 61 studies to go through the next
step. Any duplicate work was removed using the Mendeley application. To
214
guarantee that the study met QA standards, irrelevant papers were also excluded.
The data extraction process yielded information from each review as described in
Table 1. Each study was summarised according to its strategy, methods and
conclusions, as well as its scope or area of application. Initially, the Scopus
database yielded the identification of 2,906 articles. Following the removal of
duplicate entries, 534 unique articles remained. Following a screening of titles and
abstracts, 181 papers were retained for full-text examination, of which only 61
matched the inclusion criteria. Figure 1 provides a summary of our search strategy
for a systematic literature review, which is modified from Gupta et al. (2021).
3.2. Selection Criteria

To achieve the research objectives, due to the importance of the selection stage in
determining the overall validity of the literature review, several inclusion and
exclusion criteria were applied (Mikalef et al., 2018). Systematic selection criteria
were determined to designate the most relevant studies in collaborative learning.
During the initial search process of the Scopus database, we confined the scope of
our literature review to the years 2008 to 2022 based on a variety of factors, the
language was 'English' and the type of study was 'a peer-reviewed article'. The
reason it started in 2008 is that there was a trend towards the development of e-
learning in educational research (Valverde-Berrocoso et al., 2020) and the
development of collaborative learning.
Table 1. Inclusion and exclusion criteria
3.3. Analysis Method

The analysis and synthesis of the current evidence is a vital part in any systematic
review, depending on the number of studies to be included in the review
(Linnenluecke et al., 2020). In this study, unique content data analysis methods
are employed for each cluster. For the first cluster, content and topic analysis are
conducted to synthesise research that focuses on collaborative learning in higher
education as well as the resulting challenges. A comparative evaluation of
collaborative learning in higher education was carried out as part of the study to
answer the third research question. The remaining 61 publications were inspected
thoroughly in line with the coding method, and the necessary material was then
extracted, analysed, and synthesised.
4. Result and Analysis

4.1. Descriptive Statistics
The literature study, encompassing academic journal articles published between
2008 and 2022, began in 2008. During a 14-year period, data were collected from
a selection of scholarly papers on the research topic. The final sample is comprised
215
of 61 articles from 45 distinct publications. Table 2 displays the periodicals in

which papers on the determinants of collaborative learning in higher education
throughout the industrial revolution were published. rated as 4.0 according to the
Scopus rating index as recommended by Leijon et al. (2022).
Table 2. List of journal outlets (select) publishing collaborative learning in higher

education
Quartile Journal SJR Total
Q1 American Journal of Distance Education 0.898 1 52
British Journal of Educational Technology 1.87 1 (85%)
CBE Life Sciences Education 1.297 1
Computer Assisted Language Learning 1.839 1
Computers & Education 3.676 2
Computers & Education journal 0.143 1
Computers in Human Behavior 2.174 1
Education and Information Technologies 1.055 2
Education and Training 0.614 3
Educational Psychologist 3.537 1
Educational Psychology Review 3.255 1
Educational Research Review 3.067 1
Educational Researcher 3.374 1
Higher Education 1.729 1
Innovations in Education and Teaching 0.76 1
International
International Journal of Educational Research 0.923 1
International Journal of Educational 2.102 1
Technology in Higher Education
International Journal of Emerging Technologies 0.632 1
in Learning
International Journal of Technology and Design 0.753 1
Education
Journal of Computer Assisted Learning 1.491 1
Journal of Computers in Education 1.039 1
Journal of Computing in Higher Education 1.387 1
Journal of Educational Computing Research 1.279 1
Journal of Information Technology Education: 0.628 1
Research
Journal of Information, Communication and 0.36 1
Ethics in Society
Journal of Music, Technology and Education 0.238 1
Journal of Network and Computer Applications 2.193 1
Language Teaching Research 1.64 1
Learning and Instruction 2.484 3
Learning Environments Research 0.95 1
Learning, Culture and Social Interaction 0.685 1
Malaysian Journal of Learning and Instruction 0.286 1
Smart Learning Environments 0.9 1
216
Quartile Journal SJR Total

Sustainability 0.664 4
Teaching and Teacher Education 1.945 1
Technology, Knowledge and Learning 1.138 2
Technology, Pedagogy and Education 1.162 1
Q2 Education Sciences 0.518 2 6
European Journal of Education 0.532 1 (10%)
Frontiers in Education 0.579 1
International Journal of Educational 0.462 1
Management
Turkish Online Journal of Distance Education 0.412 1
Q3 Higher education, skills and work-based 0.33 1 2
learning
Journal of e-Learning and Knowledge Society 0.251 1 (3%)
Q4 International Journal of Learning 0.106 1 (2%)
These articles are scattered in several research journals with varying numbers
(Usman et al., 2021). Concerning journals, Sustainability stands out with four
publications in total. There are three articles published in the Education and
Training, and Learning and Instruction. Computers & Education, Education and
Information Technologies, Technology, Knowledge and Learning, Education
Sciences all consist of two articles each. Furthermore, each consists of one article
from 37 publishers. The distribution of journals according to the name of the
journal is presented in Table 2.
Figure 2. Distribution of selected article quartiles.
In assessing journals, Scopus classifies the quality of journals with the term
‘quartiles’, consisting of the quartiles Q1, Q2, Q3, and Q4. In this analysis, Q1 is
the highest or most significant cluster in terms of journal quality which consists
of 52 articles, followed by Q2 (6 articles), Q3 (2 articles), and Q4 (1 article). Overall,
85% of publications appeared in top Q1 journals, 10% in Q2 journals, 3% in Q3
journals, and 2% in Q4 journals as presented in Figure 2. Related research from
year to year from 2008 to 2022 showed the highest number of articles in 2019,
namely 13 articles. Second, in 2020 there was a total of 11 articles. With 61 articles,
217
this demonstrates the continued high level of research interest in the area of
collaborative learning in higher education during the IR4 era. The distribution of
articles included in the study by year is presented in Figure 3.
Figure 3. Distribution of articles by year.
4.2. Factors Influencing Collaborative Learning in Higher Education Institutions in the

Era of 4RI
This study's primary purpose is to determine the characteristics that promote
collaborative learning at universities during the fourth industrial revolution. The
61 researches presented in this paper are multiple pertinent studies given by
various writers to discover the characteristics that impact higher education
towards collaborative learning. Following completion of the flow of identification,
screening, and eligibility to the stages, the information gathered can be analyzed
based on methods classified by Sensuse (2019). The analysis is obtained based on
the following focus areas: Task and Context, Environment, Social, Technology,
Teacher, and Learner.
Table 3. Factors in collaborative learning in higher education in the 4IR era.

Factors Dimension References
Task & Context − Online learning (Paulsen & McCormick, 2020)
− Knowledge and expertise (Okolie et al., 2019)
− Ownership of the task (Brindley et al., 2023)
− Control of the task (Su & Beaumont, 2010)
− Learning context (Ming et al., 2021; Popov et al.,
− Terms of educational 2014) )
context (Jeong et al., 2019)
− Encourages collaborative (Herrera-Pavo, 2021)
− Collaborative activities (ChanLin, 2012; Phuthong, 2021)
− Educational designers (Barberà et al., 2022)
− Educational approach (Gress et al., 2010)
(Näykki et al., 2014)
− Educational process
(Kirschner et al., 2011)
− Institutional assessment
(Nicolau et al., 2017)
systems
(Levin & Avidov-Ungar, 2018)
(San-Martín et al., 2020)
(Kirschner et al., 2009)
218

(Hong et al., 2011)
Environment − Online learning (Chatterjee & Correia, 2020)
environments (Brindley et al., 2023)
− Space environment (Herrera-Pavo, 2021)
− Resources environment (Ku et al., 2013; Wang et al., 2020)
− organisational (Levin & Avidov-Ungar, 2018)
environment (Maqtary et al., 2019)
− Learning environment (Ansari & Khan, 2020)
− Computerised (Kolyvas, 2020)
environment
Social − Social interactive (Järvelä et al., 2010)
− Social engagement (Sarwar et al., 2019)
− Social media (ChanLin, 2012; El Massah, 2018)
− Social environment (Al-Rahmi et al., 2015)
− Social skills (Scager et al., 2016)
− Motivation (Goslin et al., 2016)
− Social networking (Gašević et al., 2018)
− Social presence (So & Brush, 2008)
− Socio-emotional (Näykki et al., 2014)
− Social constructivism (Tolmie et al., 2010)
(Ansari & Khan, 2020)
Technology − 3d virtual worlds (Ibáñez et al., 2013)
− Multimedia technology (Paulsen & McCormick, 2020)
− Network technology (Su & Zou, 2020)
− Collaborative learning (Phuthong, 2021)
technology (Gress et al., 2010; Ku et al., 2013)
− Educational technology (Gašević et al., 2018)
− Computer-supported (Zappatore, 2022)
− Software architecture (Hong et al., 2011)
− Computer workstation (Bjelobaba et al., 2022)
Teacher − Interact in multiple, (Sarwar et al., 2019)
meaningful ways, (Awang-Hashim et al., 2019)
− Develop critical thinking, (Pangestu, 2019)
− Communication (Oonk et al., 2020)
− High creativity (Ansari & Khan, 2020)
− Course and the structure (Alismaiel et al., 2022)
− Competence and
stimulate
− Attitude
− Interactivity
Learner − Student engagement (Paulsen & McCormick, 2020)
− Encourage learners (Chatterjee & Correia, 2020)
− Learner skill (Awang-Hashim et al., 2019)
− Students interact (Al-Rahmi et al., 2015)
− Student perceptions (Scager et al., 2016)
− Student reluctance (Barberà et al., 2022)
− Student competences, (So & Brush, 2008)
− Student work evaluation (Wang et al., 2020)
− Participating students (Levin & Avidov-Ungar, 2018)
(Zappatore, 2022)
(Bjelobaba et al., 2022)
219

(Alismaiel et al., 2022)
Task & Context
Technological advances are an effective way to connect the online learning
environment and provide the benefits of collaborative learning (Paulsen &
McCormick, 2020), which focuses on students' knowledge and skills in
encouraging collaborative learning (Okolie et al., 2019). Collaborative learning, in
an educational context, is a strategy that focuses on groups of students with
various levels of performance working together to achieve common goals
(Phuthong, 2021). Students work together towards a common goal, in a spirit of
cooperation and not competition (Isaías, 2018). Students incorporate group-
managed tactics, such as initiating engagement, offering recommendations to
improve group interactions, learning by assisting others in their learning, praising
group members for their efforts, and reminding others of time and progress
(ChanLin, 2012). Social networking sites and social media can be utilised as
dynamic tools to help create a learning environment by encouraging student
participation and articulation in cooperative learning (Sarwar et al., 2019). Due to
technological synchronization, instructors may manage and track their students'
participation in online classes and record their exchanges with them (Camilleri &
Camilleri, 2022). In online collaborative learning, research has investigated how
people collaborate to produce social knowledge in the context of specific learning
tasks (Wang et al., 2020). Methods that provide the same detailed information as
content analysis in real-time are required to convert the knowledge learned
through debate and dialogue. This will allow us to give students immediate
feedback on their individual and group learning processes as they take place
during group projects (Gress et al., 2010).
Environment
Although there have been advances in technology and efforts to design effective
ways in online learning environments, the impact on collaborative learning
(Chatterjee & Correia, 2020) with the use of computerised systems to enable or
facilitate the learning process can shape various techniques in collaborative
learning (Maqtary et al., 2019). Systems learning and transformation methods
enhance organisational research and entrepreneurial skills by strengthening
important network cooperation and resource-sharing mechanisms (Kumari et al.,
2020). Actors are free to customise the workspace, while the online workspace is
limited and determined by the resource platform and environment, which refers
to the use of cooperative learning resources, forming a collaborative learning
concept to expand the reach of resources (Al-Samarraie & Saeed, 2018).
Establishing a dynamic and cooperative learning environment allows for active
participation, honest dialogue, and the unrestricted exchange of ideas and
perspectives, all of which promote learning (Su & Beaumont, 2010).
Social
Collaborative learning (CL) develops social skills necessary for future scientific
careers (Scager et al., 2016). CL is used to promote social presence and foster an
atmosphere where students feel free to share their thoughts. Students with low
levels of social presence are unable to communicate diverse ideas and are unable
to receive help from peers and teachers (Phuthong, 2021). Collaboration in the
220
classroom is critical to social constructivism (Goslin et al., 2016) which emphasises

continuous learning through contact, cooperation and group work (Matee et al.,
2022). Dealing with technological developments in the field of education can be
achieved through the use of collaborative learning and online social engagement
strategies (ChanLin, 2012). Active learning literature also supports the influence
of positive social interaction approaches on student performance when using
collaborative learning strategies (El Massah, 2018). Social networking platforms
play an important role in encouraging online collaborative learning by providing
more media for students to communicate with their classmates (Ming et al., 2021).
In other words, the degree to which students comprehend how to use social media
platforms to improve their collaborative learning experiences and the potential of
these platforms to make it easier for electronic information sharing and resource
sharing (Phuthong, 2021).
Technology
Computer-supported collaborative learning has been used for many years in
educational programming and shows that group processes and computer-
supported collaboration scripts can facilitate the informed design of meaningful
collaboration for learning and teaching (Popov et al., 2014; Silva et al., 2020). The
global education industry is being drastically altered by COVID-19. After this
epidemic, collaborative and distance learning technologies are anticipated to
establish themselves as the "new normal" in education (Phuthong, 2021). Online
collaborative learning provides an instructional scaffolding design lens that can
encourage active and productive online conversations and is a possible research
tool for the future (Wang et al., 2020). Networked technological capabilities can
create opportunities for interaction between groups and within entire classes and
change the way collaborative learning takes place (Mercier & Higgins, 2013).
Future research should focus on how the use of educational technology might
enhance students' collaborative learning, particularly when it comes to problem-
solving, communication, and finishing assignments (Bond et al., 2020). Therefore,
collaborative learning models use client-server tools to build, enabling the
specification of learning workflows and setting up collaborative interactions as
theatre simulations in a 3D virtual world (Ibáñez et al., 2013) and software
architecture-based can be developed to support collaborative learning modules
(Ibáñez et al., 2013).
Teacher
Collaborative learning can encourage students to interact in diverse and
meaningful ways and develop critical, communication and social skills. The
learning methods used by the courses, lecturer plans, and collaborative learning
structures are important parts of a Collaborative Learning Design (Pangestu,
2019). Teachers may foster students' creativity by encouraging them to investigate
subject matter in more inventive ways (Awang-Hashim et al., 2019). Teachers are
required to play a crucial role in the critical interaction of higher education
institutions with society by fostering collaborative learning attitudes and
mastering cross-border competencies (Oonk et al., 2020). Therefore, interaction
with classmates and teachers as well as collaborative learning have a big impact
on students' academic success (Ansari & Khan, 2020).
221
Learner
When students are required to take an active role in learning, encouraging
collaboration and discussion with classmates through achieving educational goals
in a personalised or self-directed way, is a powerful way to increase student
engagement and motivation (Nakajima & Goode, 2019). Appropriate assessment
strategies are needed when new educational approaches are introduced, to ensure
their effectiveness and feasibility and highlight the importance of addressing
student competencies, educational needs, and collaborative learning
requirements (Zappatore, 2022). On the one hand, course structure influences
students' perceptions of collaboration, social interaction and satisfaction. Utilising
social media for educational reasons might enhance students' perceptions of their
academic success (Al-Adwan et al., 2020; So & Brush, 2008). Conversely, students'
unwillingness to engage in collaborative learning is a factor that influences the
design of individual activities rather than collaborative activities, and comparable
results were seen with graduate students in online environments (Barberà et al.,
2014). The requirement to engage in collaborative learning as part of the
educational process encourages students to acquire knowledge in the subject area.
Adoption of information about the technology and structures employed,
analytical skills, critical thinking and enthusiasm to work on projects, as well as
an improvement in the competency of students engaging in collaborative learning
can be formed (Bjelobaba et al., 2022).
4.3. Institutional Challenges in Collaborative Learning at Higher Education

Institutions
Several challenges are related to students' perceptions of the complexity of
economic interactions, and challenges faced as students perceive economic
interactions as more complex after participating in simulations in collaborative
learning (Sierra & Suárez-Collado, 2021). Several challenges to consider relating
to collaborative learning (CL) in higher education in the 4IR era: (a) Experienced
educators whose customary practices have been demonstrated to produce
positive learning results may not choose to adopt this strategy (Hernández et al.,
2019); (b) Time is required to develop activities, and university lecturers with
several duties may lack this resource (time) to do something new, and (c)
Resistance to change from professors and even from students.
Increasing student diversity and the use of technology in the application of CL are
two difficulties that higher education must address (Goedhart et al., 2019). There
are difficulties in adopting machine learning (ML) and artificial intelligence (AI)
in collaborative learning in universities in middle-income nations (Kuleto et al.,
2021). Through more active discussion, sharing, and modification of learning
resources, the use of these technologies has unquestionably led to a major increase
in student cooperation. Researchers, practitioners, and academics will be able to
understand the issues with these technologies from a wider perspective by
mapping cloud computing tools in blended learning environments (Al-Samarraie
& Saeed, 2018).
In collaborative learning, the majority of faculty members believe that
conventional methods of instruction are more useful than online-only ones. The
222
majority of instructors view student participation as the most difficult aspect of

online instruction. When adapting traditional instruction to the online
environment, teachers believe it is crucial to modify instructional content to meet
students' requirements. Other important aspects that must be changed include
teaching delivery methods and classroom policies (Pandit & Agrawal, 2022). A
recurring challenge to collaborative learning is to promote student involvement
in group activities (Silva et al., 2020).
The idea of a mission-driven institution of higher learning that adopts the
University Industrial Park model might be able to handle the challenges brought
on by the fourth industrial revolution (Alam et al., 2020). CL's low pass rate
impacts university funding; therefore, there is a tremendous amount of pressure
to boost pass rates (Roberson & Kleynhans, 2019). Universities throughout the
world have turned to online learning as a technique for addressing teaching and
learning issues as a result of the globalisation of higher education and economic
constraints. However, educating academics and students with the skills necessary
for efficient online learning remains a significant difficulty (Thomas et al., 2018).
Although integrating online learning presents a number of challenges, it does
allow teaching and learning to continue, lowers the impact on students' academic
progress, and enables distance learning for international students who cannot
travel abroad to attend classes (Ming et al., 2021). The challenge for teachers is to
remain a central figure in supporting collaborative learning, without controlling
the moments in which learning opportunities arise for students (van Leeuwen et
al., 2019).
4.4. Gaps and Future Research.

Trends in the area of collaborative learning in higher education in the 4IR era are
seen based on linked literature, and it is evident that the work studied includes
the process of building groups. It is crucial to attempt to re-implement this
approach in other situations when individuals must operate in groups. These
settings include task and context, environment, social, technological, teacher, and
learner. However, there are still a lot of concerns in this area that haven't been
fully examined and resolved. These problems are holes and flaws in the literature
that are examined and used as a foundation for determining the course of further
research, as follows:
(1) Various interactions related to student perceptions (Sierra & Suárez-
Collado, 2021) when participating in giving challenges in CL affect the graduation
rate (Roberson & Kleynhans, 2019). Therefore, by providing empirical studies
related to collaborative learning in higher education, students' perceptions of
student graduation rates can become a research gap in the future using a
quantitative method approach.
(2) The application of CL in the 4IR era is certainly loaded with the use of
technology (Goedhart et al., 2019) including the application of artificial
intelligence (AI) and machine learning (ML) (Kuleto et al., 2021). Future research
should be geared towards proposing platforms and frameworks for concrete AI
and ML projects for higher education, especially in low and middle-income
countries. This research is important to bridge this skills gap and provide
opportunities for students and professionals to gain hands-on experience.
223
(3) Various strategies include encouraging students to work together,

assessing student participation on online platforms, and using instruction and
consulting, which can be used to address challenges faced in using Virtual
Collaborative Learning in higher education institutions. To better understand the
issues associated with the use of Virtual Collaborative Learning by students and
lecturers, future research needs to identify these by including respondents
representing all higher education institutions in middle-income countries (Matee
et al., 2022).
(4) Some university faculty consider that traditional methods are more
profitable than online methods, and changing teaching adaptation modelling, and
changing the content as needed is very important (Pandit & Agrawal, 2022). For
this reason, a comparative study is needed on how the outcomes of teaching
implementation along with its content and methods are from the traditional
model and the collaborative online model. Future research can raise this aspect.
(5) Collaborative learning online still encourages the university as a learning
strategy. However, good provision is also needed to improve student learning
skills online (Thomas et al., 2018). Future research can discuss the paradox in
empirical studies between universities, learning models, and improving students'
skills. This research is important to bridge the gap between research and practice
and improve student learning outcomes.
5. Conclusion
Collaborative learning, used as a teaching and learning method in the present
online learning environment, enables students to participate actively and interact
with one another. This study seeks to identify future research gaps as well as the
elements and constraints of collaborative learning in higher education institutions
throughout the fourth industrial revolution. A systematic literature review, which
is an appropriate method for strengthening research issues and gaps, was
conducted with 61 papers. It should be noted that this manuscript only presents
work from the Scopus database with journal criteria. This study has identified six
factors related to collaborative learning in tertiary institutions in the fourth
industrial revolution era including Task & Context, Environment, Social,
Technology, Teacher, and Learner.
This research has made a substantial addition to learning education by bringing
together the numerous and different strands of collaborative learning literature.
It presents a comprehensive analysis of the literature on collaborative learning in
higher education institutions during the fourth industrial revolution to generate
a clear grasp of its relevance and use. Furthermore, the findings of this study have
revealed that the online-based collaborative learning model has been widely
adopted in several models in higher education. In addition, Comparing the
current collaborative learning features reveals major gaps in the technological
integration needed to achieve collaborative learning in higher education
disciplines throughout the fourth industrial revolution.
However, there are still some limitations to this research, including that it only
presents works from the Scopus database with journal criteria, future SLR studies
may present from other databases such as the Web of Science, etc. Second, it can
also present manuscripts from books, processes, etc. To accelerate the
224
implementation of Industry 4.0 in collaborative learning, further study on the

validation of recommended ideas and concepts utilising empirical research
methods such as simulations, prototypes, experiments, and case studies is
necessary. Lack of research on sustainability, artificial intelligence, and machine
learning indicates an undiscovered area of study that should be pursued in the
future.
Acknowledgement
This paper is supported by Indonesia Navy Technology College.
6. References
Al-Adwan, A. S., Albelbisi, N. A., Aladwan, S. H., Al Horani, O. M., Al-Madadha, A., &
Al Khasawneh, M. H. (2020). Investigating the Impact of Social Media Use on
Student’s Perception of Academic Performance in Higher Education: Evidence
From Jordan. Journal of Information Technology Education: Research, 19, 953–975.
https://doi.org/10.28945/4661
Al-Rahmi, W. M., Othman, M. S., Yusof, L. M., & Musa, M. A. (2015). Using social media
as a tool for improving academic performance through collaborative learning in
Malaysian higher education. Review of European Studies, 7(3), 265–275.
https://doi.org/10.5539/res.v7n3p265
Al-Samarraie, H., & Saeed, N. (2018). A systematic review of cloud computing tools for
collaborative learning: Opportunities and challenges to the blended-learning
environment. Computers and Education, 124(May), 77–91.
https://doi.org/10.1016/j.compedu.2018.05.016
Al, H., Aizat, S., Ahmed, S., & Alzahrani, I. (2020). A flipped classroom model in higher
education : a review of the evidence across disciplines. Educational Technology
Research and Development, 68(3), 1017–1051. https://doi.org/10.1007/s11423-019-
09718-8
Alam, G. M., Al-Amin, A. Q., Forhad, A. R., & Mubarak, M. S. (2020). Does the private
university sector exploit sustainable residential life in the name of supporting the
fourth industrial revolution? Technological Forecasting and Social Change,
159(May), 120200. https://doi.org/10.1016/j.techfore.2020.120200
Alismaiel, O A, Cifuentes-Faura, J., & Al-Rahmi, W. M. (2022). Online Learning, Mobile
Learning, and Social Media Technologies: An Empirical Study on Constructivism
Theory during the COVID-19 Pandemic. Sustainability (Switzerland), 14(18).
https://doi.org/10.3390/su141811134
Alismaiel, Omar A., Cifuentes-Faura, J., & Al-Rahmi, W. M. (2022). Social Media
Technologies Used for Education: An Empirical Study on TAM Model During the
COVID-19 Pandemic. Frontiers in Education, 7.
https://doi.org/10.3389/feduc.2022.882831
Ansari, J. A. N., & Khan, N. A. (2020). Exploring the role of social media in collaborative
learning the new domain of learning. Smart Learning Environments, 7(1).
https://doi.org/10.1186/s40561-020-00118-7
Awang-Hashim, R., Kaur, A., & Valdez, N. P. (2019). Strategizing inclusivity in teaching
diverse learners in higher education. Malaysian Journal of Learning and Instruction,
16(1), 105–128. https://doi.org/10.32890/mjli2019.16.1.5
Barberà, E, Layne, L., & Gunawardena, C. N. (2014). Designing online interaction to
address disciplinary competencies: A cross-country comparison of faculty
perspectives. International Review of Research in Open and Distance Learning, 15(2),
142–169. https://doi.org/10.19173/irrodl.v15i2.1418
Barberà, Elena, Layne, L., & Gunawardena, C. N. (2022). Designing Online Interaction to
225
Address Disciplinary Competencies : A Cross-Country Comparison of Faculty

Perspectives Designing Online Interaction to Address Disciplinary
Competencies : A. International Review of Research in Open and Distributed
Learning, 15(2), 142–169.
Bjelobaba, G., Paunovic, M., Savic, A., Stefanovic, H., Doganjic, J., & Bogavac, Z. M. (2022).
Blockchain Technologies and Digitalization in Function of Student Work
Evaluation. Sustainability (Switzerland), 14(9).
https://doi.org/10.3390/su14095333
Bodolica, V., & Spraggon, M. (2018). An end-to-end process of writing and publishing
influential literature review articles: Do’s and don’ts. Management Decision, 56(11),
2472–2486. https://doi.org/10.1108/MD-03-2018-0253
Bond, M., Buntins, K., Bedenlier, S., Zawacki-richter, O., & Kerres, M. (2020). Mapping
research in student engagement and educational technology in higher education :
a systematic evidence map. International Journal of Educational Technology in Higher
Education, 17(2), 1–30.
Brindley, J. E., Blaschke, L. M., & Walti, C. (2023). Creating Effective Collaborative
Learning Groups in an Online Environment. International Review of Research in
Open and Distributed Learning, 10(3), 1–19.
Camilleri, M. A., & Camilleri, A. C. (2022). The Acceptance of Learning Management
Systems and Video Conferencing Technologies : Lessons Learned from COVID ‑
19. Technology, Knowledge and Learning, 27(4), 1311–1333.
https://doi.org/10.1007/s10758-021-09561-y
Campbell, A., McCall, S., Eagleson, H., & McGinnis, E. (2012). An Evaluation of the
Effectiveness of an FE/HE Partnership in the Delivery of a Preparation for Practice
Module within an Undergraduate Social Work Degree Programme. Social Work
Education, 31(5), 663–677. https://doi.org/10.1080/02615479.2011.584181
Castro, R. (2019). Blended learning in higher education : Trends and capabilities.
Education and Information Technologies, 24, 2523–2546.
ChanLin, L. J. L.-J. (2012). Learning strategies in web-supported collaborative project.
https://doi.org/10.1080/14703297.2012.703016
Chatterjee, R., & Correia, A. (2020). Online Students ’ Attitudes Toward Collaborative
Learning and Sense of Community. American Journal of Distance Education, 34(01),
1–16. https://doi.org/10.1080/08923647.2020.1703479
Davidson, N., & Major, C. H. (2014). Boundary Crossings: Cooperative Learning,
Collaborative Learning, and Problem-Based Learning. Journal on Excellence in
College Teaching, 25(3&4), 7–55.
Dewiyanti, S., Brand-Gruwel, S., Jochems, W., & Broers, N. J. (2007). Students’ experiences
with collaborative learning in asynchronous Computer-Supported Collaborative
Learning environments. Computers in Human Behavior, 23(1), 496–514.
https://doi.org/10.1016/j.chb.2004.10.021
El Massah, S. S. (2018). Addressing free riders in collaborative group work: The use of
mobile application in higher education. International Journal of Educational
Management, 32(7), 1223–1244. https://doi.org/10.1108/IJEM-01-2017-0012
Elmashhara, M. G., Silva, J., Sá, E., Carvalho, A., & Rezazadeh, A. (2022). Factors
influencing user behaviour in micromobility sharing systems: A systematic
literature review and research directions. Travel Behaviour and Society, 27(March
2021), 1–25. https://doi.org/10.1016/j.tbs.2021.10.001
Gamage, K. A. A., Wijesuriya, D. I., Ekanayake, S. Y., Rennie, A. E. W., Lambert, C. G., &
Gunawardhana, N. (2020). Online Delivery of Teaching and Laboratory Practices :
Continuity of University Programmes during COVID-19 Pandemic. Education
226
Sciences, 10(291), 1–9.

Gasevic, D. (2019). How do we start ? An approach to learning analytics adoption in higher
education. The International Journal of Information and Learning Technology, 36(4),
342–353. https://doi.org/10.1108/IJILT-02-2019-0024
Gašević, D., Joksimović, S., Eagan, B. R., & Shaffer, D. W. (2018). SENS: Network analytics
to combine social and cognitive perspectives of collaborative learning. Computers
in Human Behavior. https://doi.org/10.1016/j.chb.2018.07.003
Goedhart, N. S., Westrhenen, N. B., Moser, C., & Zweekhorst, M. B. M. (2019). The flipped
classroom : supporting a diverse group of students in their learning. Learning
Environments Research, 22(2), 297–310. https://doi.org/10.1007/s10984-019-09281-
2
Goslin, A., van der Klashorst, E., Kluka, D. A., & van Wyk, J. G. U. (2016). Formative
reflections of university recreation science students in South Africa as catalyst for
an adapted service-learning program. Australian Journal of Adult Learning, 56(1),
111–130. https://www.scopus.com/inward/record.uri?eid=2-s2.0-
85018709745&partnerID=40&md5=fe60de9aefe82fdc450cf086e98e3bd1
Gress, C. L. Z., Fior, M., Hadwin, A. F., & Winne, P. H. (2010). Measurement and
assessment in computer-supported collaborative learning. Computers in Human
Behavior, 26(5), 806–814. https://doi.org/10.1016/j.chb.2007.05.012
Gupta, Y., Khan, F. M., & Agarwal, S. (2021). Exploring Factors Influencing Mobile
Learning in Higher Education – A Systematic Review. International Journal of
Interactive Mobile Technologies, 15(2), 140–157.
https://doi.org/10.3991/ijim.v15i12.22503
Hernández, M., Antonio, D. M., Guevara, V., Carlos, J., Martínez, T., Hernández, D.,
Ruben, A., & Menendez, M. (2019). Active learning in engineering education . A
review of fundamentals , best practices and experiences. International Journal on
Interactive Design and Manufacturing (IJIDeM), 13(3), 909–922.
https://doi.org/10.1007/s12008-019-00557-8
Herrera-Pavo, M. Á. (2021). Collaborative learning for virtual higher education. Learning,
Culture and Social Interaction, 28(June 2020).
https://doi.org/10.1016/j.lcsi.2020.100437
Hong, J. C., Yu, K. C., & Chen, M. Y. (2011). Collaborative learning in technological project
design. International Journal of Technology and Design Education, 21(3), 335–347.
https://doi.org/10.1007/s10798-010-9123-7
Ibáñez, M. B., García Rueda, J. J., Maroto, D., & Delgado Kloos, C. (2013). Collaborative
learning in multi-user virtual environments. Journal of Network and Computer
Applications, 36(6), 1566–1576. https://doi.org/10.1016/j.jnca.2012.12.027
Isaías, P. (2018). Model for the enhancement of learning in higher education through the
deployment of emerging technologies. Journal of Information, Communication and
Ethics in Society, 16(4), 401–412. https://doi.org/10.1108/JICES-04-2018-0036
Järvelä, S., Volet, S., & Järvenoja, H. (2010). Research on motivation in collaborative
learning: Moving beyond the cognitive-situative divide and combining individual
and social processes. Educational Psychologist, 45(1), 15–27.
https://doi.org/10.1080/00461520903433539
Jeong, H., Hmelo-silver, C. E., & Jo, K. (2019). Ten years of Computer-Supported
Collaborative Learning : A meta- analysis of CSCL in STEM education during 2005
– 2014. Educational Research Review, 28(June), 100284.
https://doi.org/10.1016/j.edurev.2019.100284
John-Steiner, V., & Mahn, H. (1996). Sociocultural approaches to learning and
development: A Vygotskian framework. Educational Psychologist, 31(3), 191–206.
https://doi.org/10.1207/s15326985ep3103&4_4
227
Kirschner, F., Paas, F., & Kirschner, P. A. (2009). A Cognitive Load Approach to
Collaborative Learning : United Brains for Complex Tasks. Educ Psychol Rev, 21,
31–42. https://doi.org/10.1007/s10648-008-9095-2
Kirschner, F., Paas, F., Kirschner, P. A., & Janssen, J. (2011). Differential effects of problem-
solving demands on individual and collaborative learning outcomes. Learning and
Instruction, 21(4), 587–599. https://doi.org/10.1016/j.learninstruc.2011.01.001
Kolyvas, S. (2020). Innovative and Collaborative Learning in Visual Arts with the Use of
Modern Educational Software. Education Quarterly Reviews, 3(2), 194–200.
https://doi.org/10.31014/aior.1993.03.02.131
Ku, H. Y., Tseng, H. W., & Akarasriworn, C. (2013). Collaboration factors, teamwork
satisfaction, and student attitudes toward online collaborative learning. Computers
in Human Behavior, 29(3), 922–929. https://doi.org/10.1016/j.chb.2012.12.019
Kuleto, V., Ilić, M., Dumangiu, M., Ranković, M., Martins, O. M. D., Păun, D., &
Mihoreanu, L. (2021). Exploring opportunities and challenges of artificial
intelligence and machine learning in higher education institutions. Sustainability
(Switzerland), 13(18). https://doi.org/10.3390/su131810424
Kumari, R., Kwon, K.-S., Lee, B.-H., & Choi, K. (2020). Co-creation for social innovation in
the ecosystem context: The role of higher educational institutions. Sustainability
(Switzerland), 12(1), 1–21. https://doi.org/10.3390/su12010307
Leijon, M., Gudmundsson, P., Staaf, P., Christersson, C., Gudmundsson, P., Staaf, P., &
Christersson, C. (2022). Challenge based learning in higher education – A
systematic literature review literature review. Innovations in Education and
Teaching International, 59(5), 609–618.
https://doi.org/10.1080/14703297.2021.1892503
Levin, O., & Avidov-Ungar, O. (2018). Students’ views on the use of ClassBoost in a
teachers’ education college. Journal of Computer Assisted Learning, 34(6), 816–827.
https://doi.org/10.1111/jcal.12290
Linnenluecke, M. K., Marrone, M., & Singh, A. K. (2020). Conducting systematic literature
reviews and bibliometric analyses. Australian Journal of Management, 45(2), 175–
194. https://doi.org/10.1177/0312896219877678
Männistö, M., Mikkonen, K., Kuivila, H. M., Virtanen, M., Kyngäs, H., & Kääriäinen, M.
(2019). Digital collaborative learning in nursing education : a systematic review.
Scandinavian Journal of Caring Sciences, 34(2), 280–292.
https://doi.org/10.1111/scs.12743
Maqtary, N., Mohsen, A., & Bechkoum, K. (2019). Group Formation Techniques in
Computer-Supported Collaborative Learning: A Systematic Literature Review.
Technology, Knowledge and Learning, 24(2), 169–190.
https://doi.org/10.1007/s10758-017-9332-1
Matee, G. L., Motlohi, N., & Nkiwane, P. (2022). Emerging perspectives and challenges for
virtual collaborative learning in an institution of higher education: a case of
Lesotho. Interactive Technology and Smart Education.
https://doi.org/10.1108/ITSE-06-2021-0110
Mercier, E. M., & Higgins, S. E. (2013). Collaborative learning with multi-touch
technology: Developing adaptive expertise. Learning and Instruction, 25, 13–23.
https://doi.org/10.1016/j.learninstruc.2012.10.004
Mikalef, P., Pappas, I. O., Krogstie, J., & Giannakos, M. (2018). Big data analytics
capabilities: a systematic literature review and research agenda. Information
Systems and E-Business Management, 16(3), 547–578.
https://doi.org/10.1007/s10257-017-0362-y
Ming, Y., Chung, P., Law, K. M. Y., Wu, C. H., Lau, Y., Guan, J., He, D., & Ho, G. T. S.
(2021). Comparative analysis of Student’s live online learning readiness during
228
the coronavirus (COVID-19) pandemic in the higher education sector. Computers

& Education Journal, 168(April), 1–17.
https://doi.org/10.1016/j.compedu.2021.104211
Nakajima, T. M., & Goode, J. (2019). Transformative learning for computer science
teachers: Examining how educators learn e-textiles in professional development.
Teaching and Teacher Education, 85, 148–159.
https://doi.org/10.1016/j.tate.2019.05.004
Näykki, P., Järvelä, S., Kirschner, P. A., & Järvenoja, H. (2014). Socio-emotional conflict in
collaborative learning-A process-oriented case study in a higher education
context. International Journal of Educational Research, 68, 1–14.
https://doi.org/10.1016/j.ijer.2014.07.001
Nicolau, B., Castonguay, G., Levine, A., Hong, Q. N., Pluye, P., Afrashtehfar, K. I., Al-
Sahan, M., Amin, M., Benbow, P., De Oliveira, A. C. B., Brondani, M., De Oliveira,
R. C., Caty, M. È., Chang, A., Dawson, A., Emami, E., Lage, C. F., Hachey, S.,
Lawrence, H. P., … Tanwir, F. (2017). Applied mixed methods in oral health
research: Importance and example of a training program. JDR Clinical and
Translational Research, 2(3), 206–210. https://doi.org/10.1177/2380084417705823
Okolie, U. C., Nwosu, H. E., & Mlanga, S. (2019). Graduate employability How the higher
education institutions can meet. Higher Education, Skills and Work-Based Learning,
9(4), 620–636. https://doi.org/10.1108/HESWBL-09-2018-0089
Oonk, C., Gulikers, J. T. M., den Brok, P. J., Wesselink, R., Beers, P.-J., & Mulder, M. (2020).
Teachers as brokers: adding a university-society perspective to higher education
teacher competence profiles. Higher Education, 80(4), 701–718.
https://doi.org/10.1007/s10734-020-00510-9
Pandit, D., & Agrawal, S. (2022). Exploring Challenges of Online Education in COVID
Times. FIIB Business Review, 11(3), 263–270.
https://doi.org/10.1177/2319714520986254
Pangestu, H. (2019). Evaluation of Collaborative Learning Tools Implementation to
Improve Learning Quality in Higher Education. 2019 International Conference on
Information Management and Technology (ICIMTech), 1(August), 477–481.
Paulsen, J., & McCormick, A. C. (2020). Reassessing Disparities in Online Learner Student
Engagement in Higher Education. Educational Researcher, 49(1), 20–29.
https://doi.org/10.3102/0013189X19898690
Pérez-López, R., Gurrea-Sarasa, R., Herrando, C., Martín-De Hoyos, M. J., Bordonaba-
Juste, V., & Acerete, A. U. (2020). The generation of student engagement as a
cognition-affect-behaviour process in a Twitter learning experience. Australasian
Journal of Educational Technology, 36(3), 132–146.
https://doi.org/10.14742/AJET.5751
Phuthong, T. (2021). Antecedents Influencing the Adoption of Collaborative Learning
Social-Media Platforms Among Thai University Students During the Covid-19
‘New Normal’ Era. International Journal of Emerging Technologies in Learning, 16(13),
108–127. https://doi.org/10.3991/ijet.v16i13.18083
Popov, V., Biemans, H. J. A., Kuznetsov, A. N., & Mulder, M. (2014). Use of an
interculturally enriched collaboration script in computer-supported collaborative
learning in higher education. Technology, Pedagogy and Education, 23(3), 349–374.
https://doi.org/10.1080/1475939X.2014.945474
Riebe, L., Girardi, A., & Whitsed, C. (2016). A Systematic Literature Review of Teamwork
Pedagogy in Higher Education. Small Group Research, 47(6), 619 –664.
https://doi.org/10.1177/1046496416665221
Roberson, J., & Kleynhans, C. (2019). First Generation Learners’ Experience of
Collaborative Learning. International Journal of Learning, 5(2), 150–154.
229
https://doi.org/10.18178/IJLT.5.2.150-154
Rowe, J. P., Shores, L. R., Mott, B. W., & Lester, J. C. (2010). Integrating learning and
engagement in narrative-centered learning environments Intelligent Tutoring
Systems. Intelligent Tutoring Systems, 6095, 166–177.
http://dl.acm.org/citation.cfm?id=2143722.2143745
Salam, M., Nurfatimah, D., Iskandar, A., Hanani, D., & Ibrahim, A. (2019). Service learning
in higher education : a systematic literature review. Asia Pacific Education Review,
20(4), 573–593. https://doi.org/10.1007/s12564-019-09580-6
San-Martín, S., Jiménez, N., Rodríguez-Torrico, P., & Piñeiro-Ibarra, I. (2020). The
determinants of teachers’ continuance commitment to e-learning in higher
education. Education and Information Technologies, 25(4), 3205–3225.
https://doi.org/10.1007/s10639-020-10117-3
Sarwar, B., Zulfiqar, S., Aziz, S., & Chandia, K. E. (2019). Usage of Social Media Tools for
Collaborative Learning : The Effect on Learning Success With the Moderating Role
of Cyberbullying. Journal of Educational Computing Research, 57(1), 246–279.
https://doi.org/10.1177/0735633117748415
Scager, K., Boonstra, J., Peeters, T., Vulperhorst, J., & Wiegant, F. (2016). Collaborative
learning in higher education: Evoking positive interdependence. CBE Life Sciences
Education, 15(4), 1–9. https://doi.org/10.1187/cbe.16-07-0219
Scepanovic, S. (2019). The Fourth Industrial Revolution and Education. 2019 8th
Mediterranean Conference on Embedded Computing, MECO 2019 - Proceedings,
0(June), 8–11. https://doi.org/10.1109/MECO.2019.8760114
Sensuse, D. I. (2019). Collaborative Learning Strategies for Online Knowledge Sharing Tool
Within Organizations : A Systematic Literature Review. 2019 International Conference
on Information Management and Technology (ICIMTech), 1(August), 652–657.
Sierra, J., & Suárez-Collado, Á. (2021). Understanding economic, social, and
environmental sustainability challenges in the global south. Sustainability
(Switzerland), 13(13). https://doi.org/10.3390/su13137201
Silva, L., Mendes, A. J., & Gomes, A. (2020). Computer-supported collaborative learning in
programming education: A systematic literature review. IEEE Global Engineering
Education Conference, EDUCON, 2020-April, 1086–1095.
https://doi.org/10.1109/EDUCON45650.2020.9125237
So, H., & Brush, T. A. (2008). Student perceptions of collaborative learning , social presence
and satisfaction in a blended learning environment : Relationships and critical
factors. Computers & Education, 51, 318–336.
Su, Fan, & Zou, D. (2020). Technology-enhanced collaborative language learning :
theoretical foundations , technologies , and implications. Computer Assisted
Language Learning, 0(0), 1–35. https://doi.org/10.1080/09588221.2020.1831545
Su, Feng, & Beaumont, C. (2010). Evaluating the use of a wiki for collaborative learning.
https://doi.org/10.1080/14703297.2010.518428
Thomas, G., Thorpe, S., Thomas, G., & Thorpe, S. (2018). Enhancing the facilitation of
online groups in higher education : a review of the literature on face-to-face and
online group-facilitation. Interactive Learning Environments, 0(0), 1–10.
https://doi.org/10.1080/10494820.2018.1451897
Tolmie, A. K., Topping, K. J., Christie, D., Donaldson, C., Howe, C., Jessiman, E.,
Livingston, K., & Thurston, A. (2010). Social effects of collaborative learning in
primary schools. Learning and Instruction, 20(3), 177–191.
https://doi.org/10.1016/j.learninstruc.2009.01.005
Usman, A., Azis, Y., Harsanto, B., & Azis, A. M. (2021). Airport service quality dimension
230
and measurement: a systematic literature review and future research agenda.

International Journal of Quality and Reliability Management.
https://doi.org/10.1108/IJQRM-07-2021-0198
Valverde-Berrocoso, J., del Carmen Garrido-Arroyo, M., Burgos-Videla, C., & Morales-
Cevallos, M. B. (2020). Trends in educational research about e-Learning: A
systematic literature review (2009-2018). Sustainability (Switzerland), 12(12).
https://doi.org/10.3390/su12125153
van Leeuwen, A., Janssen, J., Leeuwen, A. Van, & Janssen, J. (2019). A systematic review
of teacher guidance during collaborative learning in primary and secondary
education. Educational Research Review, 27(July 2018), 71–89.
https://doi.org/10.1016/j.edurev.2019.02.001
Wang, C., Fang, T., & Gu, Y. (2020). Computers & Education Learning performance and
behavioral patterns of online collaborative learning : Impact of cognitive load and
affordances of different multimedia. Computers & Education, 143(5), 103683.
Webb, N. M., & Mastergeorge, A. (2003). Promoting effective helping behavior in peer-
directed groups. International Journal of Educational Research, 39(1–2), 73–97.
https://doi.org/10.1016/S0883-0355(03)00074-0
West, J., & Malatji, M. J. (2021). Technology Integration in Higher Education: The use of
Website Design Pedagogy to Promote Quality Teaching and Learning. Electronic
Journal of E-Learning, 19(6), 629–641. https://doi.org/10.34190/ejel.19.6.2557
Westbrook, C. (2012). Online Collaborative Learning in Health Care Education. European
Journal of Open, Distance and E-Learning, 1–6.
Zappatore, M. (2022). A Design Methodology for a Computer-Supported Collaborative
Skills Lab in Technical Translation Teaching. Journal of Information Technology
Education: Research, 21(4), 137–167. https://doi.org/10.28945/4935
Zhao, Y., Llorente, A. M. P., & Gómez, M. C. S. (2021). Digital competence in higher
education research : A systematic literature review. Computers & Education Journal,
168, 1–14. https://doi.org/10.1016/j.compedu.2021.104212
231

Vol. 22, No. 10, pp. 231-246, October 2023
The Impact and Challenges of the

Implementation of a High-Impact ePortfolio
Practice on Graduate Students’ Learning
Experiences
Hawazen Alharbi
King Abdulaziz University
Jeddah, Saudi Arabia
Abstract. This study examined the impact of implementing a high-impact

ePortfolio practice on graduate students’ learning experiences and the
challenges they faced during the first year of an e-learning graduate
program. The study employed a phenomenological approach to explore
the participants' experiences in this innovative learning environment.
Qualitative data were collected from 25 participants and analyzed using
thematic analysis. Two main themes were identified: the impact and the
challenges, and seven sub-themes emerged. This study found that the
implementation of ePortfolios as high-impact practice (HIP) has a positive
impact on students. The impact includes the development of students'
skills, their sense of achievement, the ability to showcase their work, and
making sense of their learning. However, the study also revealed the
challenges students faced during the implementation process, which
included the need for additional support, issues related to workload, and
difficulties in writing up their reflections. The findings suggest that
implementing ePortfolios as HIP has more positive impact on the
students’ sense of achievement and pride and enhances different skills.
While the integration of ePortfolios has a significant impact for students,
additional support and resources are necessary to address the associated
challenges. Overall, this study has important implications for educators
and institutions seeking to implement high-impact ePortfolio practices
and highlights the need for continued research in this area.
Keywords: High-impact practices; ePortfolio; implementation; impact

and challenges; graduate students’ learning experiences
1. Introduction
There is an increased focus on applying high-impact practices (HIPs) in higher
education (HE) to improve students’ learning experiences and learning
outcomes. HIPs have been proven to be very important tools for assessing and
enhancing learning (Light et al., 2012). Additionally, there is a large emphasis in
©Authors
232
Saudi Arabia on improving students’ learning experiences to encourage

creativity and innovation, along with preparing the students for the demands of
the labor market (Vision 2030, 2016).
ePortfolios are one of the eleven HIPs named by the Association of American
Colleges and Universities (AAC&U) and they have been used in HE to deepen
student learning, motivate students to success and offer a high-quality learning
experience (Finley & McConnell, 2022; Finley & McNair, 2013; Watson et al., 2016;
Zilvinskis et al., 2023). Research has shown that creating excellent ePortfolios can
advance a student’s grades and performance, leading them to become self-
directed learners, changing their perception of themselves as professionals and
helping them to attain the necessary skills for the workplace (Eynon, Gambino &
Török, 2014a; Hakel & Smith, 2009; Kahn, Freeman & Powell, 2021; Nino & Hicks,
2021; Ring et al., 2017). Weber and Myrick (2018) reported increases in student
motivation, knowledge, and skills when introducing ePortfolios as an HIIP.
Despite research efforts and great emphasis on the application of HIPs in HE, the
implementation of ePortfolios in institutions is not as common as it ought to be.
According to the findings of a report by the AAC&U (2022, p. 23), “ePortfolios lag
in use by campuses compared with other high-impact practices, despite their utility for
assessment, career development, and integration of learning”. Locally, there have been
many attempts to study and examine the portfolio design and to apply
ePortfolios in classrooms (Alajmi, 2019; Alharthi &Woollard, 2015; Alshahrani et
al., 2019; El-Senousy, 2020). However, research on the use of ePortfolios as an HIP
is limited (Alanazi & Bensalem, 2022).
In the E-Learning master’s program offered at the Faculty of Education at a public

university in Saudi Arabia, only a few instructors have unified their approach to
using ePortfolios as one of the assessment and reflective tools in their courses.
The implementation of ePortfolios has not been made mandatory, and each
instructor has implemented the practice differently. In this research, the
instructor/researcher implemented ePortfolios as an HIP in two different
courses. These courses were offered over two semesters during the first year of
the e-learning program for the same cohort of students. This research focuses on
investigating the impact of the implementation of ePortfolios on the students’
learning experiences. Furthermore, the research uncovers the challenges that
students are likely to encounter when compiling ePortfolios. In line with the aim
of the research, the following research questions were answered: 1) What is the
impact of implementing a high-impact ePortfolio practice on students’ learning
experiences? and 2) What are the challenges that students encounter when
designing ePortfolios?
2. Background
HIPs are “an umbrella term referring to certain educationally effective activities
associated with unusually positive benefits for the undergraduate students who participate
in one or more of them” (Kuh et al., 2018, p. 8). The concept first appeared in the
literature a decade ago and it covers a range of practices including first-year
seminars, intellectual experiences, learning communities, service learning,
233
writing-intensive courses, collaborative assignments and projects, undergraduate

research, diversity/global learning, internships, and capstone courses and
projects (Kuh, 2008). Later, ePortfolios were recognized as the 11th HIP (Watson
et al., 2016).
In a survey conducted by the National Institute for Learning Outcomes

Assessment (NILOA), 50 faculty members from different universities reported
that their institutions are already implementing HIPs. Institutions understand the
need to equip their students with the necessary skills to succeed in their studies
and in the workplace. They also recognize that HIPs are evidence-based practices
that stimulate student success and provide high-quality learning experiences
(Zilvinskis et al., 2023). Many studies have indicated that using HIPs contributes
to improving students’ personal and academic growth (Kuh, 2008; Kinzie, 2012;
Finley & McNair, 2013). Furthermore, HIPs are linked to positive and meaningful
learning outcomes as they usually require hands-on and applied practices over a
long time. When students participate in HIPs during their studies, they usually
become very motivated to perform better and use proactively use their knowledge
(Finley, 2011; Kuh et al., 2018; Valentine et al., 2021).
Many elements should be implemented when designing a learning experience as

an HIP: 1) the expectations of students’ performance should be set high; 2)
students have to engage in a focused effort for a very long period; 3) the learning
experience is expected to allow interaction between students, the faculty and other
peers about the area being studied; 4) students are expected to be actively
involved with diverse experiences and situations other than the ones they are
usually familiar with; 5) students need to receive constructive feedback promptly;
6) students should be involved in real-world experiences; 7) students are expected
to showcase their abilities and skills; and 8) reflective learning opportunities are
expected (Kuh et al., 2018).
ePortfolios are more than just a display of artifacts. They are combinations of
successful educational practices and processes that connect “Reflective, integrative
and social pedagogy” (Kuh et al., 2018, p. 16). These practices produce an advanced
collection of artifacts using multimedia, reflections, and digital storylines that
express the students’ journey during their studies (Kuh et al., 2018). Reflection in
an ePortfolio can help students to manage their learning, assess their learning
growth and make their learning visible (Flynn, 2022; Eynon & Gambino, 2023).
Furthermore, reflections can stimulate the growth of students’ intellectual and
personal skills (Buyarski, 2014; Weber & Myrick, 2018).
The characteristics of high-impact ePortfolio practice are given below:

1. ePortfolios are central to the curricular design of the course, program, and
university experience. To achieve the highest impact, ePortfolio
integration should be apparent across the curriculum for the entire
learning experience of the program. Students need to be fully involved in
the process and they should understand the purpose of the ePortfolio from
the beginning. Instructors should provide a clear outline of the process,
234
including the accepted artifacts to be included, as well as the rubric and

timeline (Mueller & Bair, 2018; Richards-Schuster et al., 2014).
2. ePortfolio pedagogies support integrative, self-directed learning and the
development of student identity. Students are encouraged to assess their
learning and to look for points of connection between the courses and
within each one of them so that they can recognize their learning and
growth (Kahn et al., 2021).
3. ePortfolio-making skills are explicitly taught to students: students need to
enhance their technical and intellectual skills when creating ePortfolios.
They need extensive guidance and support on how to design and develop
their ePortfolios and on writing the reflection as well (Slepcevic-Zach &
Stock, 2018; Landis et al., 2015; Kahn et al., 2021). The ePortfolio
assessment is holistic and aligned with learning processes and outcomes.
Kahn et al. (2021, p. 13) explained that the “ePortfolio assessment has all of
the advantages of authentic assessment, with the additional advantages conferred
by digital media”.
In order for students to create effective ePortfolios, researchers suggest that they
should follow these steps: 1) collect and organize the artifacts to be included; 2)
select the artifacts that represent the students’ growth and development; 3) reflect
on how these artifacts show their learning progress; and then 4) connect ideas and
interpret the learning experience (Mueller & Bair, 2018; Parkes et al., 2013;
Richards-Schuster et al., 2014).
3. Context and Design

The researcher/instructor in this study implemented the ePortfolio as an HIP in
two graduate courses for the same cohort of students during their first year of the
program. The first course was offered during the first semester, while the second
course was in the second semester of the 2021/2022 academic year. The students
were asked to design a learning ePortfolio where they combined practices and
processes. Elements of high-impact ePortfolio practices were employed to
enhance each student's success and learning experiences in the following ways
(Kuh et al., 2018):
1. Ownership: since the university does not have a platform for creating
ePortfolios, the instructor provided instructions to the students on how to
use different websites to design their ePortfolios. This step was
challenging since there was no technical support available to the students
but it allowed them to customize their portfolios in a way that they
believed represented their identities. Students were allowed to add media
and artifacts and to be as creative as they wanted, which created a sense
of ownership among the students.
235
2. Making learning visible: the ePortfolio encouraged students to share their

work with others as it displayed their artifacts, projects, and best work.
This can make students feel proud of themselves. In this study, it was
suggested that students should design their ePortfolios with a view to
thinking how they were going to use it after it had been graded. Whether
they wanted to use it as part of their CV, when applying for a doctoral
degree, or as a reference for work, they had to design it with the result in
mind.
3. Longitudinal capacity: since the ePortfolios were developed over two
semesters, students’ skills gradually developed. The ePortfolio required
students to write up reflections which helped them to make connections
between their previous experiences, new ideas, and the topics that they
had studied during the program. Receiving frequent feedback from their
instructor was one of the main elements in the integration of the ePortfolio.
The students received feedback on each learning task and assignment and
again when they implemented it into the ePortfolio. The instructor
provided feedback on the design of the technology as well.
4. Identity development: many educational practices were employed during
the writing reflections, group projects, assignments, and class
presentations while the course was running; these enhanced students’
abilities and helped them to develop their identities.
4. Methodology
4.1 Method
This research used a phenomenological method to understand the experience of
the participants. According to Merriam (2002, p. 7), “This form of inquiry is an
attempt to deal with inner experiences unprobed in everyday life”. Patton (1990, p. 70)
explained that phenomenology is centered on, “the assumption that there is an
essence or essences to shared experience…the experiences of different people are bracketed,
analyzed, and compared to identify the essences of the phenomena”. Researchers who
follow a phenomenological approach adopt a thoroughly descriptive method and
focus on investigating the relationship between participants in the research and
the situation they are in (Finlay, 2009). This research focuses on the lived
experience of the participants and is underpinned by a thoroughly descriptive
analysis to answer the research questions based on the collection of qualitative
data.
4.2 Participants
Participants in this research were graduate students enrolled on the eLearning
master’s program in their first year of study. A total of 38 students were enrolled
in the two courses which were the focus of this research These two courses were
taught by the same instructor/researcher and involved the design and
development of ePortfolios. A questionnaire was sent to the 38 graduate students
and (n=25) responses were received from students who agreed to participate in
the research.
The participants had various backgrounds and employment statuses. Out of the
25 respondents to the questionnaire, 10 respondents were K-12 teachers, 5 were
236
working in school administrative positions, 1 was a school principal and there

were 9 graduate students with no previous work experience, as shown in Table 1.
Table 1: Participants’ Employment Status

Participants’ Employment No. of Participants Percentage
Status
K-12 Teachers 10 40%
School Administrators 5 20%
School Principal 1 4%
Full-Time Graduate Students 9 36%
Total 25 100%
4.3 Data Collection

The data were collected using an electronic questionnaire that allowed the
respondents to provide consent anonymously. The questionnaire collected
qualitative data and it consisted of twelve open-ended structured questions an
example of which is given in Appendix 1. The questionnaire captured in depth
the impact of using the ePortfolio as an HIP and how it influenced the students’
learning experiences. It also aimed to capture students’ perceptions and the
challenges they encountered as they designed and developed their ePortfolios. To
ensure the reliability and validity of the instrument (questionnaire), a pilot test
was undertaken. The questionnaire was first sent to two faculty members, who
are experts in the field, for feedback. Certain changes were then made due to their
feedback. Then, the questionnaire was sent to a sample of 10 graduate students in
the same eLearning program, but in a different cohort to the participants in this
research. Ten responses were received and analyzed and minor corrections were
made to the questionnaire (Creswell & Poth, 2017).
4.4 Analysis
Since the focus of phenomenological research is on the narrative evolving from
the data, the researcher analyzed the data qualitatively using thematic analysis to
identify emerging themes as this method is useful for investigating the
perspectives of participants (King, 2004; Finlay, 2009). Nowell et al. (2017, p. 2)
explained that “Thematic analysis provides a highly flexible approach that can be
modified for the needs of many studies, providing a rich and detailed, yet complex account
of data”. In this research, thematic analysis was conducted via three levels of
coding. The researcher used a selective method for the first level of coding to
highlight phrases. Then, the researcher color-coded the phrases to compare and
categorize them easily and to reduce the phrases (Creswell, 2012). The second
level of coding was undertaken in order to categorize the themes and to count
their frequency and calculate the distribution of perspectives in percentages
(Cohen, Manion & Morrison, 2011). The third level of coding was completed to
finalize the codes, report the findings, and insert direct quotes where applicable
as this was an essential component of the final report (King, 2004). Finally, the
researcher designed a table to explain the relationship between the themes and
sub-themes. To ensure the internal and external reliability of the research, peer
examination was used to review the analysis. The researcher also documented the
research process (Creswell & Poth, 2017).
237
5. Results
In response to the research questions, the analysis revealed two themes and
several sub-themes which related to the impact of the implementation and the
challenges that the students faced during the design and development process for
the high impact ePortfolio practice. The themes and sub-themes were organized
as shown in Table 2:
Table 2: Themes and Sub-Themes of the Impact and Challenges of ePortfolio

Implementation as an HIP
Topic Themes Sub-Theme Frequency Percentage
Impact Skills Digital Skills 7 28%

Design Skills 5 20%
Time 8 32%
Management and
Organizational
Skills
Research Skills 4 16%
Sense of Achievement 18 72%
and Pride
Showcasing Work 23 92%
Making Sense of 11 44%
Learning
Challenges Technical Challenges 10 40%
Workload 8 32%
Reflective Writing 6 24%
5.1. Impact of the Implementation of the HIP

Four themes emerged regarding the students’ perceptions of the impact of
integrating ePortfolios as an HIP in their learning experience: 1) skills, 2) sense of
achievement, 3) showcasing work, and 4) making sense of learning. There were
sub-themes related to each of these four themes.
5.1.1 Skills
Under this theme, 96% of participants reported that there was an impact on their
skills in different ways. Four sub-themes appeared in the analysis; participants
reported an impact on: their digital skills, design skills, time management and
organizational skills, and research skills. Table 2 reports the most frequently
reported themes referred to by participants regarding the impact of ePortfolio
implementation on their skills.
Participants reported that there was an obvious enhancement of their digital skills
in general. They had to solve technical issues, look for solutions, navigate
contents, pick the right technology and learn new technical skills. One participant
noted that the end product (ePortfolio) offers “Evidence of the competencies and
digital skills acquired”.
Participants stated that implementing the ePortfolio as an HIP advanced their

design skills. They developed their ePortfolios from scratch, created webpages
and added different types of digital content. One participant stated, “It helped a lot
238
in developing my design and coordination skills and in searching for solutions to any
obstacle I encountered while designing it”. Many participants reported that collecting
and organizing the artifacts and writing the reflections took them longer than
designing the ePortfolio. Another stated, “In my point of view, it has developed my
designing skills and organization and also helped to increase my motivation in making the
ePortfolio to reflect on my efforts”. Furthermore, many participants agreed that
designing and creating their ePortfolio was not as hard as they expected. This
could be a result of the students’ previous knowledge of creating ePortfolios, their
knowledge of designing websites, or due to the user-friendly websites that they
used for the creation of their ePortfolios, as described.
The most reported effect of the ePortfolios in the data was the impact on students’
organizational and time management skills as 32% of the participants agreed that
creating ePortfolios requires excellent organizational and time management skills,
especially as they have to choose and develop high-quality work to include in
their ePortfolios, while meeting the deadline for each task. One participant stated,
“First thing it helped me in organizing my work, thoughts, and ideas and I don’t deny
that it enhanced my skills in general”. Another participant asserted that, “Designing
the ePortfolio helped me to manage my time effectively and speed up tasks’ completion
time”.
The participants also reported an enhancement in their research skills, as

explained by one participant who stated, “It developed definitely my research and
technical skills”.
5.1.2 Sense of Achievement and Pride

Under this theme, 72% of the participants reported that the creation of the
ePortfolio gave them a sense of achievement and contributed to them feeling
proud of these achievements as well as motivating them. They described how
their ability to observe their own progress during the year and the advancement
in their skills was eye opening. Sometimes it is hard to be aware of a small
enhancement in one’s abilities until it is documented. One participant explained,
“I think it is an excellent idea, as it was interesting for me to design and coordinate my
ePortfolio and to see the achievements of a whole year displayed in it”. Another
participant emphasized:
It helped a lot and the benefit lies in seeing the achievements that were
accomplished each course during the year…it also shows the extent of the
development of my abilities and skills during the semesters through
what has been accomplished. It is also considered an evaluation for
student’s work each semester.
The analysis demonstrated that many of the participants mentioned that the
creation of the ePortfolios boosted their sense of pride and motivation. One
participant explained: “The design of the ePortfolio enables us to add our creative
touches and to feel proud as we can have quick publishing of our work”. Another
participant stated, “It was a creative and motivating experience to learn about
developing an ePortfolio and developing creative design ideas”.
239
5.1.3 Showcasing Work

There was evidence of an impact on the participants’ ability to display their work.
Overall, 92% of the participants reported that designing the ePortfolio had helped
them to showcase their work and present it. One participant explained that the
assignments, research and projects undertaken as part of their courses are usually
forgotten about after they are finished and that the ePortfolio helped them with
the collection and in displaying these artifacts. Additionally, 28% of the
participants reported that they plan to use the ePortfolio as part of their CV
because it represents their best work during their graduate classes. One
participant continued:
It assists in presenting all the digital productions that took place during the master's
program in an organized and arranged ePortfolio that shows the person's skills,
competencies, and capabilities through what work a person wants to include – and
I can also benefit from it by using it as a reference within the CV.
Another participant explained, “It is useful as a reference and repository to remember
what you have accomplished”.
5.1.4 Making Sense of Learning

The participants explained how the process of designing the ePortfolios and
writing reflections affected how they viewed their learning and their identity as
learners. One participant reported, “It made me change my thinking about my learning
positively”. Additionally, 44% of participants stated that the process of designing
the portfolio, choosing the artifacts to include, thinking of the connections
between courses, and writing the reflections made them think deeply about their
learning. One participant stated, “It gave me a clear vision of what I am taking (topics)
and how I am going to use it”. Another participant emphasized that the ePortfolio
“Enhanced my critical thinking and made me more focus[ed] and aware of what my
learning outcomes are”. Specifically, 20% of the participants described how they
understood the topics they had studied more fully when they wrote their
reflection and that this helped them make cross-course connections. One
participant explained, “Reflections helped me to know whether I achieved the objectives
of the course and its required outputs. It also contributed to linking my previous
knowledge and experiences with the new knowledge and skills that were acquired during
courses”. In this regard, one participant explained that she designed infographics
to demonstrate connections between the topics studied during the year and then
added them to her ePortfolio, even though it was not one of the requirements.
Additionally, 12% of participants reported that writing reflections helped them to
recognize their weaknesses and strengths. One participant stated, “Reflection is one
of the most important factors in self-development, intellectually and professionally because
when we recognize our deficiencies and weaknesses, we try to fix them. It helps also in
identifying our areas of creativity so we reinforce them”.
5.2 Challenges
Many participants reported varying challenges that they faced when designing
their ePortfolios. However, 8 participants did not report any challenges. There
were three main challenges reported: technical challenges, challenges related to
workload, and challenges related to writing up the reflections.
240
5.2.1 Technical Challenges

The participants noted the need for some technical support, especially at the
beginning of the design process. One stated, “I needed training in using the website,
knowledge of using design tools, page layout, and more”. Although the instructor
provided the students with supporting materials on how to use different design
websites, many noted the need for further support in terms of choosing the right
website for designing the ePortfolio and the need for training. One participant
stated the need for advice in choosing a platform that supports the Arabic
language. She further explained her need for support, “on adding and coordinating
the elements in the chosen platform”. Another participant stated: “I wish there were
more workshops on how to design ePortfolios”. She continued by explaining:
At first, I found it difficult to design, especially since it was my first time
designing an ePortfolio. Then, after that, it became easy, especially after
understanding the design instructions in the proposed site, the method of
adding new sections, and uploading the required files.
5.2.2 Workload
Furthermore, the participants reported that the workload was high and it was
time consuming, especially as they needed to keep adding elements and artifacts
and writing up their reflections in the ePortoflios for the whole year. One
participant who work full-time explained:
Time, as the program’s demands and requirements along with my job, my
children, and the number of achievements that I wanted to add to the
ePortfolio in a certain format were taking lots of time…It is really
important and worth the time, but we wish there was more room.
5.2.3 Reflective Writing

Finally, the participants reported that they had faced certain challenges when
writing the reflection piece for the ePortfolio. Some stated that writing the
reflections required them to dig deeper in their learning and make connections
between topics which is difficult, and some stated that it was time-consuming.
One explained, “Writing the reflection is time-consuming and needs mental effort and I
don’t find time for it to enjoy the process of writing”. Another participant noted that
there was a correlation between the time taken to write up the reflections and the
quality of the work:
Writing the reflection in the second semester was exhausting as the
workload was heavier and it felt like a duty and it was written for the sake of
the grades. Contrary to the first semester when I wrote the reflections… it
varied in the writing styles, in citing and recalling positions, and in linking them
to the topics of the course in a beautiful way.
6. Discussion
This study offers real evidence of the impact of ePortfolios on participants’
learning experiences. The findings suggest that the implementation of ePortfolios
as an HIP can have a significant impact on students in various areas, in terms of
their skills, their sense of achievement and pride, showcasing their work and in
making sense of their learning. This study revealed some challenges that the
students faced during the implementation of this HIP, in relation to the workload,
technical issues, and in writing up the reflections.
241
The majority of participants reported a positive impact on their digital skills,

design skills, organizational and time management skills, and research skills.
These findings are consistent with those of other research studies that have shown
that ePortfolios can enhance students' skills (Al-Hidabi et al., 2020).
The creation of ePortfolios also had a positive impact on the participants' sense of
achievement. The participants reported an increase in self-esteem and a sense of
pride as they saw how far they had progressed in comparison to when they
started the program. Some students reported that even though the ePortfolio was
required for only two courses per semester, they uploaded all their course artifacts
and projects to their ePortfolios during that year as they felt motivated to do so.
This finding is consistent with other research studies that have shown the positive
impact of ePortfolios on student motivation and sense of achievement (González-
Mujico, 2020).
The findings indicate that the process of designing ePortfolios can be a useful tool
for showcasing work and presenting it to potential employers. The results
demonstrate that the participants reported that designing the ePortfolio helped
them showcase their work as it required them to collect and display their artifacts.
In agreement, one participant explained that the assignments, research, and
projects completed as part of their courses are usually forgotten about after they
are finished and that ePortfolios assist with the collection and display of these
artifacts. Many participants reported that they will use their ePortfolio as part of
their CV, as it represents their best work during their graduate program. This is
consistent with the literature that indicates that ePortfolios can be used as
evidence of qualifications and achievements for future employers (Kelly & Le
Rossignol, 2022).
When participants reported that the implementation of ePortfolios as an HIP had

helped them to make sense of their learning, this proved how effective this
implementation was. This is an impact that institutions and faculty members hope
that their students will achieve and it is aligned with Eynon and Gambino’s (2023)
explanation that students can regard the creation of ePortfolios as an inquiry into
their own learning.
As has been proven in this study, there were certain challenges that students faced
in the HIP implementation. Many reported technical challenges and a need for
support in the design and development of their ePortfolio. This observation was
present in the literature as it has been found that students and teachers both
struggle with ePortfolio technology (Wijayaratne et al., 2023).
Students reported challenges in writing up their reflections. Some reported the

need for enough time to work on their reflections as they had written a very in-
depth piece during the first semester when the workload was less heavy but they
couldn’t make the same effort during the second semester as the workload was
higher. Another student pointed out the need for guidance on how to start the
writing and what to write. This issue was also discussed in the literature as many
studies found that students needed some guidance on what to write in their
reflections and how to reflect upon their work (Copland & Donaghue, 2019; Hall,
2023). An important implication of the current research is that students should be
242
provided with clear instruction and offered support on the technical aspects of the
design and development of the ePortfolio. Furthermore, instructors should
provide further guidance for students on writing the reflection.
One of the limitations of this research was related to the sample of students.
Clearly, this research could be expanded to include different populations of
graduate students and undergraduate students. Another limitation is that the
research collected only qualitative data to understand the students’ perspectives
and the lack of quantitative data could be a disadvantage.
7. Conclusion
In conclusion, this study sheds light on the perceptions of graduate students
regarding the implementation of a high-impact ePortfolio practice during their
first year of study. The findings provide evidence that the integration of
ePortfolios as an HIP has a positive impact on the students in various ways
including the development of skills, a sense of achievement, the ability to
showcase their work, and an enhanced understanding of their learning
experience. These results are in line with the previous research that has indicated
the benefits of ePortfolios as a high-impact educational practice. However, this
study also highlights some of the challenges that the students faced during the
implementation of the high-impact ePortfolio practice. These challenges include
the need for additional support, issues related to workload, and difficulties when
writing up reflections. These findings suggest that while the integration of
ePortfolios can have significant benefits for students, it is important to address the
challenges that may arise during their implementation. Overall, the findings of
this study offer important implications for educators and institutions seeking to
implement high-impact ePortfolio practices. By providing support and resources
to students, such as training on how to create effective reflections and managing
their workload, educators can help students overcome any challenges and fully
realize the benefits of ePortfolios as a high-impact educational practice.
Furthermore, our study highlights the need for continued research on the
implementation and impact of ePortfolios as a high-impact educational practice
to further enhance students’ learning experiences.
8. References
Alajmi, M. A. 2019. The impact of E-portfolio use on the development of professional
standards and life skills of students in the faculty of Education at Princess Noura
Bint Abdul Rahman University, Entrepreneurship and Sustainability Issues 6(4)
1714-1735. https://doi.org/10.9770/jesi.2019.6.4(12)
Alanazi, S. M., & Bensalem, E. (2022). Undergraduate research EFL students’ experiences
in building ePortfolios. Multidisciplinary Journal for Education, Social and
Technological Sciences, 9(2), 1–22. https://doi.org/10.4995/muse.2022.17167
AlHarthi, E., & Woollard, J. (2015). Teacher evaluation in Saudi schools: The use of e-
Portfolio. INTED2015 Proceedings (pp.1782-1788).
https://library.iated.org/view/ALHARTHI2015TEA
Al-Hidabi, D. A., Kabilan, M. K., & Shakfa, M. D. (2020). Benefits and challenges in
utilizing e-Portfolio among UCAS students’ EFL classes in Palestine. BirLEJournal,
3(1), 523-530. https://bircu-journal.com/index.php/birle/article/view/681
Alshahrani, S. M., Mohamed, H., Mukhtar, M., Mokhtar, U. A. (2020). Factors influence
the intention to use e-Portfolio in Saudi Technical and Vocational Training
243
Corporation (TVTC) sector: Pilot review. In: Saeed, F., Mohammed, F., Gazem, N.
(eds) Emerging trends in intelligent computing and informatics. IRICT 2019.
Advances in Intelligent Systems and Computing, vol 1073. Springer, Cham.
https://doi.org/10.1007/978-3-030-33582-3_95
Buyarski, C. (2014). Reflection in the first year: A foundation for identity and meaning
making. Catalyst for Learning: ePortfolio Resources and Research.
http://iupui.mcnrc.org/ref-practice/
Cohen, L., Manion. L., & Morrison, K. (2011). Research Methods in Education. (7th ed.).
Routledge. https://doi.org/10.4324/9781315456539
Copland, F., & Donaghue, H. (2019). Post observation feedback. In The Routledge Handbook
of English Language Teacher Education, edited by S. Walsh, and S. Mann, 402–416.
Routledge. https://doi.org/10.4324/9781315659824-32
Creswell, J. W. (2012). Educational Research: Planning, Conducting and Evaluating
Quantitative and Qualitative Research. (6th ed.). Pearson Education.
Creswell, J.W., & Poth, C.N. (2017). Qualitative inquiry and research design: Choosing among
five approaches (4th ed.). SAGE Publications.
El-Senousy, H. (2020). E-portfolio to assess the 21 st century skills of students in smart e-
learning environment. International Journal for Quality Assurance, 3(1).
https://ijqa.zu.edu.jo/eng/images/stories/year2019/07/Vol3_no1/133.pdf
Eynon, B., & Gambino, L. M. (2023). High-Impact ePortfolio Practice: A Catalyst for Student,
Faculty, and Institutional Learning. Taylor & Francis.
https://doi.org/10.4324/9781003445098-3
Eynon, B., Gambino, L. M., & Török, J. (2014a). Reflection, integration, and ePortfolio
pedagogy. SUNY Academic Work.
https://academicworks.cuny.edu/cgi/viewcontent.cgi?article=1026&context=n
c_pubs
Eynon, B., Gambino, L. M., & Török, J. (2014b). What difference can ePortfolio make?
A field report from the connect to learning project. International Journal of
ePortfolio, 4(1), 95-114. https://files.eric.ed.gov/fulltext/EJ1107844.pdf
Finley, A. (2011). Assessment of high-impact practices: using findings to drive change
in the compass project. Peer Review, 13(2), 29.
https://link.gale.com/apps/doc/A264095054/AONE?u=anon~c28a9df3&sid=g
oogleScholar&xid=a2c2c37f
Finley, A., & McConnell, K. D. (2022). On the same page? Administrator and faculty
views on what shapes college learning and student success. American
Association of Colleges and Universities.
https://files.eric.ed.gov/fulltext/ED625904.pdf
Finley, A., & McNair, T. (2013). Assessing underserved students' engagement in high
impact practices. Washington, DC: Association of American Colleges and Universities.
http://hdl.handle.net/10919/87004
Finlay, L. (2012). Debating phenomenological methods. In Hermeneutic Phenomenology
in Education (pp. 15-37). Brill.
Flynn, C. O. (2022). Reflections on an e-portfolio assessment in a first-year physiology
course. Biomed Eng Education, 2(1), 61–67. https://doi.org/10.1007/s43683-021-
00058-9
Gonzalez-Mujico, F. (2020). The impact of ePortfolio implementation on motivation,
self-regulation and academic language development: The learners’ and the
teachers’ perspectives. The Asian ESP Journal, 16(3), 209-42. https://
researchmap.jp/Bolstad/published_papers/30562037/attachment_file.pdf#pag
e= 209
244
Hakel, M. D., & Smith, E. N. (2009). Documenting the outcomes of learning. In Electronic
Portfolios 2.0 (pp. 133-135). Routledge. https://doi.org/10.4324/9781003444428-
20
Hall, J. M. (2023). Eportfolio post tagging for student-teacher development. Innovation in
Language Learning and Teaching, 1-13.
https://doi.org/10.1080/17501229.2022.2096034
Kahn, S., Freeman, T., & Powell, A. A. (2021, November). IUPUI’s HIP taxonomy for
ePortfolio: A tool for development, implementation, and scaling (Occasional Paper
No. 57). Urbana, IL: University of Illinois and Indiana University, National
Institute for Learning Outcomes Assessment
(NILOA). https://hdl.handle.net/1805/27849
Kelly, M., & Le Rossignol, K. (2022). Professional transitions: The eportfolio as a curated
digital narrative. Industry and Higher Education, 36(6), 784–
794. https://doi.org/10.1177/09504222221127565
King, N. (2004). Using templates in the thematic analysis of text. In C. Cassell & G. Symon
(Eds.), Essential Guide to Qualitative Methods in Organizational Research (pp. 257–
270). Sage. https://doi.org/10.4135/9781446280119.n21
Kinzie, J. (2012). High-impact practices: Promoting participation for all students. Diversity
& Democracy, 15(3). https://dgmg81phhvh63.cloudfront.net/content/user-
photos/Publications/Archives/Diversity-Democracy/DD_15- 3_FA12.pdf
Kuh, G. D. (2008). High-impact educational practices: What they are, who has access
to them, and why they matter. Association of American Colleges and
Universities, 14(3), 28-29. https://www.aacu.org/publication/high-impact-
educational-practices-what-they-are-who-has-access-to-them-and-why-they-
matter
Kuh, G. D., Gambino, L. M., Bresciani Ludvik, M., & O’Donnell, K. (2018, February).
Using ePortfolio to document and deepen the impact of HIPs on learning dispositions
(Occasional Paper No. 32). Urbana, IL: University of Illinois and Indiana
University, National Institute for Learning Outcomes Assessment (NILOA).
http://learningoutcomesassessment.org/occasionalpaperthirtytwo.html
Light, T. P., Chen, H. L., & Ittelson, J. C. (2011). Documenting Learning with ePortfolios: A
Guide for College Instructors. John Wiley & Sons.
https://doi.org/10.1353/rhe.2013.0071
Merriam, S. B. (2002). Introduction to Qualitative Research. Qualitative Research in Practice:
Examples for Discussion and Analysis, 1(1), 1-17. Jossey-Bass.
https://doi.org/10.1353/csd.2003.0041
Mueller, R. A., & Bair, H. (2018). Deconstructing the notion of eportfolio as a high impact
practice: A self-study and comparative analysis. The Canadian Journal
for the Scholarship of Teaching and Learning, 9(3). https://doi.org/10.5206/cjsotl-
rcacea.2018.3.6
Nino, M. M., & Hicks, S. (2021). The 6A eportfolio model: Professionalizing learning in
higher education. International Journal of ePortfolio, 11(2), 95-107.
https://files.eric.ed.gov/fulltext/EJ1339609.pdf
Parkes, K. A., Dredger, K. S., & Hicks, D. (2013). ePortfolio as a measure of reflective
practice. International Journal of ePortfolio, 3(2), 99-115.
http://www.theijep.com/pdf/ijep110.pdf
Patton, M. Q. (1990). Qualitative Evaluation and Research Methods. SAGE Publications, inc.
Richards-Schuster, K., Ruffolo, M. C., Nicoll, K. L., Distelrath, C., & Galura, J. A. (2014).
Using e-portfolios to assess program goals, integrative learning, and civic
engagement: A case example. International Journal of ePortfolio, 4(2), 133-141.
245
Ring, G. L., Waugaman, C., & Brackett, B. (2017). The value of career eportfolios on job
applicant performance: Using data to determine effectiveness. International
Journal of ePortfolio, 7(2), 225-236. https://files.eric.ed.gov/fulltext/EJ1159904.pdf
Segaran, M. K., & Hasim, Z. (2021). Self-regulated learning through eportfolio: A meta
analysis. Malaysian Journal of Learning and Instruction, 18(1), 131-156.
https://doi.org/10.32890/mjli2021.18.1.6
Slepcevic-Zach, P., & Stock, M. (2018). ePortfolio as a tool for reflection and self-
reflection. Reflective Practice, 19(3), 291-307.htt
ps://doi.org/10.1080/14623943.2018.1437399
Valentine, J., Price, D., & Yang, H. (2021). High-impact practices and gains in student
learning: Evidence from Georgia, Montana, and Wisconsin. Lumina Issue Paper,
Lumina Foundation. https://eric.ed.gov/?id=ED611259
Vision 2030. (2016). The Vision Realization Programs. https://www.vision2030.gov.sa/
Watson, C. E., Kuh, G. D., Rhodes, T., Penny Light, T., & Chen, H. (2016). Editorial:
eportfolios – The eleventh high impact practice. International Journal of
ePortfolio, 6(2), 65–69. http://www.theijep.com
Weber, K., & Myrick, K. (2018). Reflecting on reflecting: Summer undergraduate research
students' experiences in developing electronic portfolios, a meta-high impact
practice. International Journal of ePortfolio, 8(1), 13-25.
Wijayaratne, K., Fernando, P. R. M. K., Ragel, R., & Mudiyanse, R. (2023). Readiness to
expand smart learning via eportfolio; Perception of teachers and students in a
higher education institute. In international conference on smart learning
environments (pp. 148-151). Singapore. https://doi.org/10.1007/978-981-99-
5961-7_16
Zilvinskis, J., Kinzie, J., Daday, J., O'Donnell, K., & Zande, C. V. (Eds.). (2023). Delivering
on the Promise of High-Impact Practices: Research and Models for Achieving
Equity, Fidelity, Impact, and Scale. Taylor & Francis.
https://doi.org/10.4324/9781003444022
246
Appendix 1
1. What is the best way to describe your job and position?

2. Have you ever designed ePortfolios before this time?
3. How would you describe your experience in creating ePortfolios
(technical-wise)?
4. What were the challenges facing you in creating the ePortfolios?
5. How long did it take to create your ePortfolio at the beginning of the year
and how long did it take you to edit it the second time at the end of the
year?
6. How do you think that the implementation of the ePortfolios as a HIP
helped/didn’t help to enhance your skills?
7. How do you think that your ePortfolio could benefit you in the future
and after you graduate from this program?
8. Do you think that you will keep developing your ePortfolio even after
you graduate from this program? explain
9. How do you think that writing reflections in the courses changed your
way of thinking about your education?
10. What is the best element in creating an ePortfolio?
11. What is the hardest element in creating an ePortfolio?
12. Would you implement this method to assess your students’ progress?
247

Vol. 22, No. 10, pp. 247-270, October 2023
Inclusion of Metaverses in the Development of the

Flipped Classroom in the University environment:
Bibliometric Analysis of Indexed Scientific
Production in SCOPUS
Omar Chamorro-Atalaya
Facultad de Ingeniería y Gestión,
Universidad Nacional Tecnológica de Lima Sur, Perú*
Víctor Durán-Herrera , Raul Suarez-Bazalar ,

Constantino Nieves-Barreto , Julio Tarazona-Padilla ,
Facultad de Ciencias Administrativas,
Universidad Nacional del Callao, Perú
Milagros Rojas-Carbajal , Yreneo Cruz-Telada

Facultad de Ciencias de la Salud,
Universidad Autónoma de Ica, Perú
Juan Caller-Luna
Facultad Ciencias de la Salud,
Universidad Privada del Norte, Perú
Ronald Alarcón-Anco
Facultad de Derecho y Ciencia Política,
Universidad Alas Peruanas, Perú
José Antonio Arévalo-Tuesta

Facultad de Ciencias Económicas,
Universidad Nacional Federico Villarreal, Perú
Abstract. The flipped classroom as a teaching strategy implies a change

in the traditional class dynamics; however, in recent years the use of
metaverses has been incorporated in the development of flipped
classrooms, seeking to provide a more immersive and interactive
environment for the student. Therefore, it is necessary to analyze and
*
Corresponding author: Omar Chamorro-Atalaya, ochamorro@untels.edu.pe
©Authors
248
identify the existing gaps in the scientific production that investigate the
inclusion of metaverses in university education. In this sense, the
purpose of this study is to examine and present currents of study related
to the integration of metaverses in the development of the flipped
classroom, in university environments, using bibliometric analysis
regarding academic works indexed in the Scopus database, between
2013 and 2023. The results obtained show a constant growth in scientific
production. This increase is influenced by the context of the COVID-19
pandemic and the search for more immersive and effective solutions in
virtual education. The predominant thematic areas focus specifically on
the use of virtual reality and augmented reality as key elements for the
design and construction of metaverses, which, combined with
gamification techniques, lead to the development of flipped and
immersive classrooms. In this sense, based on the identified results, it is
concluded that there is a gap in the research corpus that needs to be
addressed in terms of guidelines and good practices for the effective
integration of metaverses in flipped classrooms.
Keywords: flipped classroom; metaverse; augmented reality; virtual

reality; higher education
1. Introduction
Although distance education has existed for many years, the pandemic
accelerated the immersion into the virtual world of the teaching and learning
processes, making the implementation of virtual platforms vitally important
(Roller, 2022). Although educational trends are leaning more towards
digitisation, higher education systems seem to have a lot to offer a very diverse
audience that has just faced a pandemic (Villamarín, 2023). We can clearly see
that COVID-19 contributed to the resurgence of various technologies, including
the metaverse, giving rise to the development of immersive and interactive
environments (Chavez, 2022; González et al., 2022). A metaverse is a virtual
environment capable of simulating both a real physical environment and a
digital world (Chediak, 2023; Vallejos, 2023) whose application to the
educational field focuses on offering virtual education solutions generating new
collaborative learning models, and thereby encouraging interactive student
participation (Ruiz-Campo et al., 2022). Currently, the metaverse is considered
as a universal, unified and interoperable 3D space with the capacity for
immersion, corporeality and persistence (Mendiola, 2022; Quiroz et al., 2023). In
this way, immersiveness means that users are immersed in the virtual world and
can feel the experience, on the other hand, corporeality allows users to be
represented by avatars that personify the user experience and persistence means
that continues execution, even without being connected (Barráez-Herrera, 2022).
Both virtual reality and augmented reality are means for the development of
metaverses with great potential to represent knowledge and display interaction
processes with a greater degree of immersion and interactivity (Carrizo, 2021;
Silva-Díaz et al., 2021). Virtual reality is defined as the creation of a synthetic
virtual environment, whose initial purpose is to physically deceive each of the
senses of the human body, changing its perception of being in reality to being in
249
the developed synthetic environment (Montoya & León, 2018). In contrast,

augmented reality offers a direct or indirect perspective of a physical
environment in the real world, where real elements are mixed with virtual
elements to generate a combined reality (Lechtaler et al., 2015; Padilla et al.,
2019). Thus, the main difference between virtual reality and augmented reality is
that the first approach creates a completely virtual world without the need to
insert virtual elements into real environments, as is the case with augmented
reality (Aznar-Díaz et al., 2018). However, with the arrival of the metaverse in
the classroom, the teacher must have a holistic and technological perspective
since entering this world implies knowing, managing and designing classrooms
in the virtual world (Mujica-Sequera, 2022; Suco et al., 2021). This implies that
the student is incorporated into a simulated or artificial environment, in which
the virtual environment is examined in the same way as it is presented in the
real world (Ramos et al., 2019); stimulating their senses and motivating them to
live an experience different from traditional education, encouraging their
curiosity and creativity, favouring the understanding and assimilation of content
in each subject (Olmos, 2022). However, the incorporation of these technological
tools in educational institutions has generated difficulties for teachers facing
considerable obstacles when trying to implement them due to their limited
training and knowledge about the possibilities and effectiveness of these tools
(Miguélez-Juan et al., 2019).
In accordance with what was stated above, it is necessary to modify the teacher
must configure his traditional teaching scheme, since, in many cases,
technological innovation requires this change (Araya-Moya et al., 2022); and so it
is that, currently, the teacher is in need of integrating emerging technologies in
learning since he offers more content options for redefining class time (Calle &
Martínez, 2018; Gaviria-Rodríguez et al., 2019). In this reality of urgent change,
more active didactic strategies and collaborative spaces arise that innovate the
type of teacher-student interaction, such as the pedagogical model termed
flipped classroom (Benites et al., 2021). The flipped classroom is presented as a
valuable technique available, which requires teaching commitment, so its
education, training and self-learning will set the tone for entering new learning
models (Cedeño-Escobar & Vigueras-Moreno, 2020). The flipped classroom
requires integrating different digital applications so that students can access
information favourably (Díaz & Díaz, 2021). In this way, it is implemented in a
simple way and processed in a cooperative way that contributes to obtaining
relevant and useful knowledge (Sánchez-Cruzado et al., 2019).
In this sense, the purpose of this study is to examine and present currents of
study related to the integration of metaverses in the development of the flipped
classroom, in university environments, using bibliometric analysis regarding
academic works indexed in the Scopus database, between 2013 and 2023.
Therefore, this study seeks to contribute to generating updated knowledge
regarding the integration of the metaverse and flipped classrooms in university
education, at the same time, which will allow the identification of gaps in the
literature that can be addressed in future research. This will help inform the
academic and educational community about the possibilities and challenges that
250
this combination offers. In this sense, the research questions (RQ) defined for
this study are:
• RQ1: What is the scientific production and what types of manuscripts
have been published regarding the inclusion of metaverses in the
development of the flipped classroom in the university environment?
• RQ2: What are the most cited manuscripts regarding the inclusion of
metaverses in the development of the flipped classroom in the university
environment?
• RQ3: What are the sources with the largest number of publications
regarding the inclusion of metaverses in the development of the flipped
classroom in the university environment?
• RQ4: What words have the highest occurrence in the titles of the
manuscripts regarding the inclusion of metaverses in the development of
the flipped classroom in the university setting?
• RQ5: What words have the highest occurrence in the abstracts of the
manuscripts regarding the inclusion of metaverses in the development of
the flipped classroom in the university setting?
• RQ6: What are the thematic areas with the highest incidence that have
been published regarding the inclusion of metaverses in the
development of the flipped classroom in the university environment?
Metaverses, also called MDV3D (3-Dimensional Virtual Digital Worlds), are
three-dimensional virtual spaces mediated by computers and composed of real
or proper symbols and icons (Bonales-Daimiel et al., 2022). They represent a
virtual and online world in which different avatars interact, but which, unlike
massive multiplayer games, do not have a competitive purpose based
exclusively on a level system (Nieto, 2022). These avatars are configurable digital
bodies that represent the materialisation of the users (Galea, 2023).
Communication within the metaverse is considered as a socialisation network
that allows interaction between users from anywhere in the world, with greater
technological innovations, tools available to any user, and a greater variety of
content (Menéndez-Aponte et al., 2023). However, for the construction of the
metaverse, a series of base technologies are needed, these being virtual reality or
augmented reality (Erazo & Sulbarán, 2022; Espinoza, 2022). It can be said that
virtual reality is a technology that enables the user, through the use of viewers
and other gadgets, to immerse themselves in three-dimensional scenarios, in the
first person and in 360º (Maldonado et al., 2020); with the feeling of being
immersed inside them and even being able to interact with the elements that
compose them (Barragán-Perea, 2023; Sánchez et al., 2022). Augmented reality,
on the other hand, refers to that technology that allows information from the
virtual world, or virtual reality, to be incorporated into the real world (Bohmer,
2021).
The flipped classroom is an educational strategy that contributes to developing,

among other things, of self-management skills, teamwork, and informational
skills of students (Fernández & Gaytán, 2019). Its objective is to provide the
student with an active role in relation to their own learning, leaving aside the
251
passive postures of reception and memorisation of information (Ocampo et al.,

2021). The flipped classroom does not imply that a physical blended attendance
is necessary with respect to classical teaching, which is sometimes
misinterpreted as reducing contact hours by half; what is blended is the activity
that the student develops outside and within the classroom (Jiménez-Serranía,
2022). Although the student is the main actor in their learning in the flipped
classroom, the teacher is the guide, facilitator and above all planner of all the
activities that are going to be used for the development of their skills (Espinoza
& Robalino, 2022; Quinto-Román, 2022). In other words, the teacher must be
flexible in his teaching strategies, be responsible for the material to promote the
previous study of the students and their reflection about it, as well as promote a
change in the role of the student towards a more active role (Arias & Mon, 2022;
Ros & Laguna, 2021). To develop the flipped classroom, technological tools are
used so that students, before class, can incorporate the topics that will later be
developed in a more practical way in the face-to-face class (Verón et al., 2021);
thus leaving more time for consultations, exchanges of opinions and learning by
doing (Prieto et al. 2021).
3. Manuscript extraction method

The manuscript extraction method used in this bibliometric review study
regarding the inclusion of metaverses in the development of flipped classrooms
in the university setting is based on the method used and validated in
Chamorro-Atalaya et al. (2023). This method used is an adaptation of the steps
established in the PRISMA declaration (Preferred Reporting Items for Systematic
reviews and Meta-Analyses). This extraction method encompasses the
development of three phases. In the first phase, the subject of study is defined, as
well as the scope with which the "identified manuscripts" will be determined. In
the second phase, the inclusion and exclusion criteria will be applied with which
the "manuscripts in projection" will be determined. Finally, in the third phase,
the abstracts and content of each manuscript will be thoroughly reviewed, thus
determining the "included manuscripts" for the analysis phase.
3.1 Phase 1: Definition of the subject of study, scope and identification of

manuscripts
In relation to this first phase, we proceeded to define the subject under study
and with it the keywords that were used to build the search equation for the
manuscripts, in the Scopus database. Scopus was used due to the composition of
the data it provides regarding the extracted manuscripts. In other words, it
segments the bibliometric fields of each manuscript, facilitating the analysis.
Other aspects that justify the selection of the Scopus database is that it stores
different types of publications such as conference papers, review articles or
scientific articles (Vaicondam et al., 2022); it also provides publications from a
large number of scientific journals indexed to this database (Prahani et al., 2022,
2023). Table 1 shows the search equation used to extract the so-called “identified
manuscripts”. As a result of the implementation of this first phase, 127
manuscripts were identified.
252
Table 1: Search equation through Boolean indicators

Database Search equation
(TITLE-ABS-KEY ( flipped AND classroom ) ) AND ( ( TITLE-ABS-
Scopus KEY (metaverse) OR TITLE-ABS-KEY (virtual AND reality) OR TITLE-
ABS-KEY (augmented AND reality ) ) )
3.2 Phase 2: Manuscripts in projection

In relation to this second phase, the inclusion (IC) and exclusion criteria (EC)
were applied to the 127 manuscripts identified in the previous phase. These
criteria were defined with the purpose of reducing the bias present when using
only the search equation. In other words, aspects such as whether the studies
were carried out at the level of university education or at the primary or
secondary level are relevant in order to have precise results regarding the
subject under study. Another consideration that was taken into account for the
construction of the IC and EC was whether the manuscript was open access,
and, if the manuscript had not been accessed, the bias in the results would have
been high. Having the possibility of downloading the entire content of the
manuscript allowed a more in-depth review of its content and thus determining
if there is a close relationship with the topic under study. Table 2 shows the
inclusion and exclusion criteria in order to determine the manuscripts in
projection. As a result of the implementation of this second phase, 88
manuscripts were identified.
Table 2: Criteria used to identify manuscripts

IC EC
Manuscripts whose results are linked to
Manuscripts whose results are linked to
the primary or secondary level of study
the university environment
of a school
Manuscripts that do not have access to
Open access manuscripts
their content
Manuscripts published in the period of Manuscripts published before the year
years between 2013 and 2023 2013
3.3 Phase 3: Manuscripts included for the analysis stage

Finally, in the third phase, a review of the abstract as well as the content of the
manuscript was carried out, thus establishing which manuscripts are closely
related to the subject of study. Through this phase, it was possible to remove 32
manuscripts of the 88 manuscripts determined in the previous phase. Figure 1
shows the flowchart of the manuscript extraction method used, which specifies
that there are 56 manuscripts finally included in the bibliometric review phase.
253
Figure 1: Flowchart of the method used for the extraction of manuscripts
4. Results
4.1 Scientific production and types of published manuscripts regarding the
inclusion of metaverses in the development of the flipped classroom in
the university environment
In relation to the scientific production of published manuscripts on the
application of metaverses in the implementation of flipped classrooms in
university education, it was identified that, in the period between 2013 and 2023,
56 manuscripts were published according to the Scopus database. Figure 2
shows that, between the years 2017 and 2023, the scientific production of
manuscripts has had a permanent growth, that is, of the total number of
manuscripts identified and included for this study, only six manuscripts were
published between the years 2013 and 2016, representing 10.71% of the total
scientific production. However, the manuscripts published from 2017 to date
represent 89.29%. An important indicator to take into account is the average
number of manuscripts published in the last ten years; in this study it was
identified that the average is five manuscripts per year. Based on this average, it
was possible to identify that the annual scientific production from 2020 to date
exceeds the average. Here it is relevant to link these results with the context of
the COVID-19 pandemic, in which the implementation of virtual class sessions
was widespread, supported by the use of teaching strategies such as the flipped
classroom. Likewise, in recent years, with the purpose of making these class
sessions more immersive, artificial intelligence tools such as metaverses have
been applied, which allow students to be taken to virtual learning environments
in which they can have greater interaction with equipment, tools and supplies,
consistent with the thematic content developed in a flipped class session.
254
Figure 2: Production of manuscripts by year of publication
Likewise, in relation to the type of manuscripts published on the inclusion of

metaverses in the development of the flipped classroom in university education,
it was possible to identify four types, these being: "Review article", "Conference
paper", "Book chapter ” and “Scientific article”. Of the 56 manuscripts chosen for
the bibliometric review study three are review articles, 25 are conference papers,
two are book chapters, and 26 are scientific articles. Figure 3 shows the
percentage distribution of the types of manuscripts published in the Scopus
database, during the period between 2013 and 2023, in which it can be seen that
the types of manuscripts that have been published on the topic under study are,
to a greater extent, scientific articles and conference papers.
Figure 3: Percentage distribution by type of manuscript
4.2 Most cited manuscripts regarding the inclusion of metaverses in the

development of the flipped classroom in the university environment
In relation to the most cited manuscripts, it is important to take into account the
findings by Repiso and Moreno-Delgado (2020) who established that the citation
is one of the main elements when establishing the repercussion of a research
and, together with the scientific production, allows the study at an aggregate
level of the knowledge generated by authors, journals and institutions. In this
sense, and in accordance with what is shown in Table 3, of the 56 manuscripts
included for this bibliometric review study, 23 were identified that were cited at
least four times. The study with the highest number of citations is that
developed by Ibañez and Delgado-Kloos (2018), titled “Augmented reality for
255
STEM learning: A systematic review” with 432 citations and an average annual
citations per year of publication of 72. Also, in second place is identified the
study developed by Guze (2015) entitled “Using technology to meet the
challenges of medical education” with 158 citations and an average annual
citation per year of publication of 17.56. Both mentioned manuscripts exceed one
hundred citations. Likewise, it was identified that 19.64% of the total
manuscripts analysed exceeded 10 citations. Table 3 shows all the manuscripts
with more than four citations. As can be seen, the average number of citations
per year is specified, with the highest average number of citations being 72 years
while the manuscript with the lowest average of citations per year is that by
Noor (2013), whose value is 0.36.
Table 3: Manuscripts with the highest number of citations

Total Average citations per
Reference
citations (TC) year
Ibáñez and Delgado-Kloos (2015) 432 72.00
Guze (2015) 158 17.56
Ferrer-Torregrosa et al. (2016) 86 10.75
Hsu (2018) 69 11.50
Liou (2016) 49 6.13
Sotgiu et al. (2020) 34 8.50
Teo et al. (2022) 33 16.50
Xiao-Dong and Hong-Hui (2020) 24 6.00
Shin et al. (2022) 18 9.00
Proskura and Lytvynova (2020) 14 3.50
Ji & Han (2019) 13 2.60
Mikułowski and Brzostek-
9 2.25
Pawłowska (2020)
Huang et al. (2019) 8 1.60
Campos-Mesa et al. (2022) 7 3.50
Pozo-Sánchez et al. (2021) 7 2.33
Kalavally et al. (2014) 6 0.60
Im et al. (2019) 6 1.20
Hsu et al. (2017) 6 0.86
Lin et al. (2023) 5 5.00
Jiao et al. (2020) 5 1.67
Fidan et al. (2021) 4 1.33
Tang et al. (2021) 4 1.33
Noor (2013) 4 0.36
4.3 Sources with the largest number of publications regarding the inclusion
of metaverses in the development of the flipped classroom in the
university environment
In relation to the most relevant journals or sources linked to the largest number
of publications regarding the inclusion of metaverses in the development of the
flipped classroom in the university environment, it was identified that these
sources are not strictly associated with scientific journals but also have records
of conferences. Table 4 shows the first 10 sources with the highest number of
publications indexed to Scopus, in which the journals "Lecture Notes in
256
Computer Science", "BMC Medical Education", "Computers and Education" and

"IEEE Access" have two publications. Of these journals, the one with the highest
H index is "Lecture Notes in Computer Science" whose H index is equal to 446,
and, according to the Scimago 2022 ranking, it is located in quartile Q3.
Likewise, the source with the largest number of conference papers is "ASEE
Annual Conference and Exposition, Conference Proceedings" with two
publications and whose H index is equal to 26. Although the H indexes of
journals are relatively high, the number of publications on the subject of study in
particular is quite low.
Table 4: Most relevant sources linked to the largest number of publications

Number of Scimago SJR -
Sources H-index
publications Quartile 2022
Lecture Notes in Computer Science 2 446 Q3
ASEE Annual Conference and No
2 26
Exposition, Conference Proceedings quartile
BMC Medical Education 2 87 Q1
Computers and Education 2 215 Q1
IEEE Access 2 204 Q1
2023 11th International Conference
No No
on Information and Education 1
H-index quartile
Technology, ICIET 2023
29th International Conference on
Computers in Education No No
1
Conference, ICCE 2021 - H-index quartile
Proceedings
Advances in Intelligent Systems and No
1 58
Computing quartile
American Society of Mechanical
No
Engineers, Fluids Engineering 1 27
quartile
Division (publication) FEDSM
Anatomical Sciences Education 1 64 Q1
4.4 Words with the highest occurrence in the titles of the manuscripts on the
inclusion of metaverses in the implementation of flipped classrooms in
university education
In order to identify the words with the highest rate of occurrence in the titles of
the 56 manuscripts included in this bibliometric review study, the Bibliometrix
application was used. By this it was possible to identify that the words "flipped",
"reality" and "learning" appear at least 20 times in the titles of the manuscripts.
Likewise, when analysing the words that appear at least 10 times in the titles of
the manuscripts, it was determined that these are "classroom", "education",
"teaching", "virtual", "augmented" and "technology". Table 5 shows the words
with the highest occurrence identified in the titles of the manuscripts;
considering that the total number of these nine words add up to 146 occurrences,
it was possible to obtain the percentage of frequency with which each of these
words appears in the titles of the analysed manuscripts, thus determining that
the word "flipped" presents a percentage of occurrence of 16.44%.
257
Table 5: Words with the highest occurrence identified in the titles of the manuscripts
Words Occurrences Percentage
Flipped 24 16.44%
Reality 22 15.07%
Learning 21 14.38%
Classroom 16 10.96%
Education 14 9.59%
Teaching 13 8.90%
Virtual 13 8.90%
Augmented 13 8.90%
Technology 10 6.85%
However, in order to achieve greater precision on the joint occurrence between

one and another word shown in Table 5, the Bibliometrix application was
configured to provide the bigrams or set formed by two words that frequently
appear in the titles of the 56 manuscripts analysed. In this way, it was identified
that these bigrams are "augmented reality", "flipped classroom", and "virtual
reality". Thus, it was also determined that there are other words conceptually
linked to those mentioned above but with a lower level of occurrence such as
"flipped learning" and "VR technology". Figure 4 shows the occurrences of the
bigrams, in which "flipped classroom" and "augmented reality" have a frequency
of occurrence of 12, followed by the "virtual reality" bigram with 10 occurrences.
These results help to understand the terms used by the authors to refer to the
same Metaverse as is the case of “VR technology” and “virtual reality”.
Figure 4: Bigrams with the highest occurrence identified in the titles of the
manuscripts
Although the results shown above give an idea of how independently the words
or grouping of two words that occur simultaneously (bigrams) are presented in
the titles of the manuscripts, it is also important to know the relationship that
exists between all the words that are present in the titles. Based on what is
indicated in Figure 5, the co-occurrence network of the words contained in the
manuscript titles is shown. Through this co-occurrence network, the words with
the highest occurrence or repetition frequency are identified, the same ones that
are represented by nodes or points, with the largest nodes being those with the
258
highest frequency. On the other hand, the connections represented by lines

represent the bond strength or number of times with which the words are
related. The thicker this line is, it means that there is a greater link strength or
the greater number of times these words have been related in the titles of the
manuscripts. In this case, the largest nodes and those with the greatest link
strength are linked with the words “flipped”, “reality”, “classroom”, “learning”
and “augmented”, thus corroborating the results obtained previously.
Figure 5: Network co-occurrence of the most relevant words in manuscript titles
4.5 Words with the highest occurrence in the abstracts of the manuscripts
classroom in the university setting
Another aspect that was taken into account in this bibliometric review is the
occurrence of words that appear in the abstracts of the manuscripts. That is, in
the analysis of the words with the highest occurrence identified in the titles of
the manuscripts, they show a first vision or basic idea on the most relevant and
studied topics in the field of research on the inclusion of metaverses in the
development of the flipped classroom in the university environment. However,
itis relevant to delve more specifically into the abstracts of the manuscripts, since
these can lead to the discovery of more subtle connections and relationships
between concepts that may not be evident in the titles. Based on what has been
indicated and using the Bibliometrix application, it was possible to identify that
the words "learning", "students", "teaching" and "flipped" appear at least 119
times in the abstracts of the manuscripts. Likewise, when analysing the words
that appear at least 52 times in the abstracts of the manuscripts, it was
determined that these are "classroom", "education", "reality", "virtual",
"technology", "online", "augmented" and “study”. Table 6 shows the words with
the highest occurrence identified in the abstracts of the manuscripts; and,
considering that the total number of these twelve words add up to 1,167
259
occurrences, it was possible to obtain the percentage of frequency with which

each of these words appears in the abstracts, thus determining that the word
"learning" presents a percentage of occurrence of 17.74%.
Table 6: Words with the highest occurrence identified in the abstracts of the
manuscripts
Words Occurrences Percentage
Learning 207 17.74%
Students 160 13.71%
Teaching 132 11.31%
Flipped 119 10.20%
Classroom 95 8.14%
Education 87 7.46%
Reality 85 7.28%
Virtual 66 5.66%
Technology 57 4.88%
Online 55 4.71%
Augmented 52 4.46%
Study 52 4.46%
In addition, the occurrence of the search was analysed to identify which

grouping formed by two words are presented simultaneously (bigrams) in the
abstracts of the manuscripts, for which, through the Bibliometrix application, it
was identified that the bigram with the highest occurrence is "flipped classroom"
with 62 occurrences, followed by "augmented reality" with 46 occurrences and
"virtual reality" with 36 occurrences. To a lesser extent, the bigrams "flipped
classrooms" and "flipped learning" were identified, both with 18 occurrences.
Figure 6 shows the bigrams with the highest occurrence identified in the
abstracts of the 56 manuscripts included in this bibliometric review.
Figure 6: Bigrams with the highest occurrence identified in the abstracts of the
manuscripts
Figure 7 shows the network of co-occurrences of the most relevant words of the
"manuscript abstracts", which shows that of the 56 manuscripts analysed, the
260
words with the greatest link strength are made between the words "Flipped
classroom”, “Virtual reality” and “Augmented reality”. This shows that, in the
period in which this study is developed, there are publications that include
metaverses such as virtual reality and augmented reality related to the
development of flipped classroom learning strategies at the university level.
Figure 7: Network of co-occurrence of the most relevant words of the abstracts of the
manuscripts
4.6 Thematic areas with the highest incidence that have been published
classroom in the university environment
For this specific point, the "thematic evolution" was previously analysed taking
as reference the 56 manuscripts extracted from the Scopus database, from 2013 to
2023. Using the Bibliometrix application four quadrants were obtained made up
of: "Emerging or declining topics", "Niche topics", "Basic topics" and “Motor
Themes”. In relation to the category of "Emerging or declining topics", it was
identified that the topic associated with web-based technology is a topic that is
not very developed and is emerging recently. On the other hand, in relation to
the category "Niche topics", it was identified that topics such as active
participation, learning tools and video conferences are topics with a low
presence in terms of frequency among the reviewed manuscripts. In addition, in
the category "Basic topics" topics such as augmented reality and flipped
learning, flipped classroom, virtual reality and reality technology were
identified with greater relevance and whose degree of development is high in
the existing literature. Finally, in relation to the "Motor Themes" category, they
are made up of the themes: Online flipped classrooms, educational technology
and VR (virtual reality) technology. These last themes are those that have a high
frequency of appearance and have received increasing attention in the literature
over time.
261
Figure 8: Categorisation of the thematic evolution regarding the inclusion of

metaverses in the development of the flipped classroom
Based on what was indicated and reviewing the titles and abstracts of the
manuscripts, it was possible to identify the fields of study with the greatest
number of publications. In this way, it was determined which of the various
types of technology are those by which metaverses can be developed to a greater
extent; the reviewed publications make use of virtual reality, followed by
augmented reality technology. In this way, when quantifying the number of
manuscripts that make use of the various technologies for the design and
construction of metaverses, it is found that of the 56 manuscripts reviewed, 30
manuscripts link in their studies "environments based on virtual reality in the
development of the flipped classroom", while 21 manuscripts link in their
studies “environments based on augmented reality in the development of the
flipped classroom”, and only five manuscripts make use of both technologies
(augmented reality and virtual reality) in the development of the flipped
classroom. Figure 9 shows the percentage distribution of the specific themes
identified in the manuscripts analysed.
262
Figure 9: Percentage distribution of specific thematic areas developed in the

manuscripts analysed
5. Discussion
In relation to the scientific production regarding the inclusion of metaverses in
the development of the flipped classroom in the university environment, it was
determined that, in recent years, particularly since the pandemic period, there
has been a significant increase on this topic under study. The massification of
virtual class sessions supported by simulation tools on account of social
distancing due to COVID-19. has led us to represent a scenario in which many
researchers seek to evaluate the contributions and difficulties that this type of
modality of teaching generated in the learning of university students. Although
working with simulators on a computer helped to give continuity to the teaching
and learning process, they did not offer the student immersion. That is why,
with the rise of virtual reality and augmented reality, it is intended that students
be inserted into a world of experience very close to a real environment and
thereby increase the ability of interactivity and student participation in the
construction of their own learning. In this regard George-Reyes et al. (2023)
indicate that learning processes must adapt to the new challenges of evolving
educational environments, as has been evidenced in the context of the pandemic,
where such environments may undergo transformations. Therefore, it is
imperative to take advantage of the opportunities offered by technological
trends, not only in emergency situations, but also to achieve a transformation
that makes use of technologies such as metaverses as an essential component to
achieve disruptive learning. In this way, with the massification of tools for the
implementation of the metaverse, together with the possibility of incorporating
them into the educational context, it raises the urgency of examining the
accessible scientific research and publications in the educational field, in order to
build a comprehensive analysis of the state of the currently available art (López-
Belmonte et al. 2023).
Thus, it was also identified that the most cited manuscript regarding the
inclusion of metaverses in the development of the flipped classroom in the
university environment is the study developed by Ibañez and Delgado-Kloos
(2018) entitled "Augmented reality for STEM learning: A systematic review"
with 432 citations and with an average of 72 annual citations per year of
263
publication and having the sources with the largest number of publications,
"Lecture Notes in Computer Science", "BMC Medical Education", "Computers
and Education" and "IEEE Access" contain" and "ASEE Annual Conference and
Exposition, Conference Proceedings" all of them with two posts. This result is
evidence that not many research papers have been published regarding the
inclusion of metaverses in the development of the flipped classroom, which is
why it represents an area in which researchers can explore, develop, investigate
and contribute to its implementation in the different university educational
institutions establishing the necessary conditions for its implementation. In this
regard, Avila and Tapia’ (2020) systematic review study concluded that,
according to the analysis of a selected sample, the flipped classroom
methodology currently has a low rate of scientific contributions, likewise, it
presents few publications in databases’ scientific data. On the other hand Gea
(2021) specified that virtual reality is currently understood as a gamified
educational resource in the field of educational technology, and whose
application at different levels of education confirms the growing interest of the
academic community to analyse in depth the specific impact of virtual reality in
education.
In addition, in relation to which are the words with the highest occurrence in the
titles of the manuscripts analysed regarding the inclusion of metaverses in the
development of the flipped classroom in the university environment, it was
possible to identify that these words are "flipped", "reality" and " learning” with
a frequency of occurrence of 20 times. While, when carrying out the same
analysis, seeking to identify the words with the highest occurrence in the
abstracts of the manuscripts, it was possible to identify that these words are
"learning", "students", "teaching" and "flipped" which appear at least 119 times in
the abstracts of the manuscripts. Here it is denoted that of these 56 manuscripts
analysed, the authors focus their main attention on linking flipped learning with
virtual reality and augmented reality technology, the purpose of which is to
improve the indicators present in the teaching and learning process. This is
evidenced in the subsequent analysis that was carried out regarding the
exploration of the frequency of occurrence of bigrams present in the titles and
abstracts of the manuscripts, identifying that these are "flipped classroom",
"virtual reality" and "augmented reality". In this regard, Anacona et al. (2019)
established that the emphasis on virtual reality applied to teaching is adapting
technologies such as the flipped classroom where it has a high quality of
pedagogical potential, strengthening the way of education of educators towards
their students. Likewise, Larreategui et al. (2021) point out that most of the
research concludes that the implementation of the flipped classroom improves
student learning, strengthens autonomy, encourages collaborative work, self-
regulates learning and improves academic performance, so teachers must place
their bets on new methodologies.
Finally, related to the most prominent thematic fields that have been addressed
in research regarding the inclusion of metaverses in the development of the
flipped classroom in the university environment, it was determined that, to a
greater extent, these focus on the inclusion of virtual reality in the flipped
264
classroom. In addition, another specific thematic area identified is the inclusion

of augmented reality in the development of the flipped classroom. While to a
lesser extent, manuscripts that mention both technologies were identified. In
relation to those indicated by Chamorro-Atalaya et al. (2023), different
technologies used in the design of metaverses for their use in university
education were identified, in which it was determined that there is a
considerable inclination towards research focused on the application of virtual
reality, unlike other technologies such as augmented reality and lifelogging,
which are used to a lesser extent. Highlighting the importance of these
technologies in the implementation of metaverses, Rodríguez (2020) concluded
that the increased steps of technology are managing to maximise motivation in
different fields of education by creating digital environments based on virtual
reality and augmented reality.
6. Conclusion
In relation to the results obtained from the research questions defined in this
bibliometric review study, several important conclusions can be drawn that
show the evolution of studies linked to the integration of metaverses in the
development of the flipped classroom in the university setting. First of all, a
constant growth is observed in the production of scientific manuscripts in this
field of study from 2017 to date. This increase becomes even more evident when
compared to previous years, where production was significantly lower. This
suggests that the use of metaverses in the context of flipped teaching has gained
relevance and attention in recent years, the increase in which seems to be
influenced by the pandemic scenario that accelerated the adoption of virtual
class sessions and innovative practices of teaching strategies. Therefore, the
implementation of artificial intelligence tools, such as metaverses, has allowed
the creation of more immersive and participatory educational experiences,
taking students to virtual environments that enrich their learning and
participation; however, it is important to highlight that, despite the increase in
scientific production, research in this field of study is still at an early stage.
Finally, it is concluded that the predominant thematic areas focus specifically on
the use of virtual reality and augmented reality as key elements for the design
and construction of metaverses, combining them with gamification techniques,
which, in their integration, give rise to the development of the classrooms as
inverted and immersive. At the same time, a lack in scientific research has been
determined regarding the following thematic areas: "Success factors that show
the contribution of the metaverse in the development of flipped classrooms",
and "Guidelines and good practices for the effective integration of metaverses in
the flipped classroom".
7. Limitations
This study focused on bibliometric analysis on publications indexed only in the
Scopus database, excluding other relevant databases such as ERIC, SAGE or
IEEE Xplore. Furthermore, metaverse applications in the flipped classroom in
the field of primary or secondary education were not taken into account.
Therefore, future studies could delve deeper into these fields of research in order
to continue contributing to closing the gaps on the state of the art in the entire
265
educational field without limiting themselves only to the university field, and
considering databases other than Scopus.
8. References
Anacona, J. D., Millán, E. E., & Gómez, C. A. (2019). Application of Metaverses and the
virtual reality in teaching. Electronic Journal of Educational Technology, 13(25), 59-
67. http://dx.doi.org/10.31908/19098367.4015
Araya-Moya, S. M., Gutiérrez, A. L. R., Cárdenas, N. F. B., & Moreno, K. C. M. (2022).
The flipped classroom as a teaching resource in the Costa Rican context: case
study on its implementation in a secondary school. Education Journal, 46(1), 1-17.
https://doi.org/10.15517/revedu.v46i1.44333
Arias, J. C., & Mon, F. E. (2022). Gamified flipped classroom as a pedagogical strategy in
higher education: A systematic review. EDUTEC Electronic Journal of Educational
Technology, 80, 84-98. https://doi.org/10.21556/edutec.2022.80.2435
Ávila, A. A. C., & Tapia, C. E. C. (2020). Systematic review of the flipped classroom in
Ecuador: approach to the state of the art. Pedagogical Studies, 46(3), 45-58.
http://dx.doi.org/10.4067/S0718-07052020000300045
Aznar-Díaz, I., Romero-Rodríguez, J. M., & Rodríguez-García, A. M. (2018). Mobile
Virtual Reality technology in education: a review of the state of the scientific
literature in Spain. EDMETIC Journal of Media Education and TIC, 7(1), 256-274.
https://doi.org/10.21071/edmetic.v7i1.10139
Barráez-Herrera, D. (2022). Metaverses in the Context of Virtual Education. RTED, 13(1),
11-19, 2022. https://doi.org/10.37843/rted.v13i1.300
Barragán-Perea, E. (2023). The Metaverse and its application in the democratization of
education. Information Studies Journal, 1(1), 90-101.
https://doi.org/10.54167/rei.v1i1.1214
Benites, L. E. H., Villalba-Condori, K. O., Arias-Chávez, D., Berrios-Espezua, M., & Cano,
S. (2021). Classroom flipped in a virtual platform for the development of
competences. Case study: applied research course. Campus Virtuales, 10(2), 185-
193. http://uajournals.com/ojs/index.php/campusvirtuales/article/view/889
Bohmer, C. A. P. (2021). The experience of the image in virtual reality augmented reality and
the Metaverse. [Thesis]. University of Chile, Santiago de Chile, Chile.
https://repositorio.uchile.cl/handle/2250/185428
Bonales-Daimiel, G., Martínez-Estrella, E. C., & Ormaechea, S. L. (2022). Analysis of the
use of advergaming and Metaverse in Spain and Mexico. Revista Latina de
Comunicación Social, 80, 155-178. https://www.doi.org/10.4185/RLCS-2022-1802
Calle, F. M. R., & Martínez, A. G. (2018). Inverted Classroom with Emerging
Technologies in Virtual Environments in the Universidad Politécnica Salesiana
del Ecuador. Cuban Journal of Higher Education, 1(1), 108-123.
http://scielo.sld.cu/scielo.php?script=sci_arttext&pid=S025743142018000100008
Campos-Mesa, M. C., Castañeda-Vázquez, C., González-Campos, G., & Delcastillo-
Andrés, Ó. (2022). Augmented Reality and the Flipped Classroom-A
Comparative Analysis of University Student Motivation in Semi-Presence-Based
Education Due to COVID-19: A Pilot Study. Sustainability (Switzerland), 4(4), 1-9.
https://doi.org/10.3390/su14042319
Carrizo, N. (2021). How can immersive technologies contribute to environmental
education? EJS Journal, 20(2), 84-97.
http://sedici.unlp.edu.ar/handle/10915/135623
Cedeño-Escobar, M. R., & Vigueras-Moreno, J. A. (2020). Inverted classroom a
motivating teaching strategy for basic general education students. Dominio de las
Ciencias, 6(3), 878-897. http://dx.doi.org/10.23857/dc.v6i3.1323
266
Chamorro-Atalaya, O., Durán-Herrera, V., Suarez-Bazalar, R., Gonzáles-Pacheco, A.,

Quipuscoa-Silvestre, M., Hernández-Hernández, F., Huamán-Flores, E.,
Chaccara-Contreras, V., Palacios-Huaraca, C., & Guía-Altamirano, T. (2023). The
Metaverse in University Education during COVID-19: A Systematic Review of
Success Factors. International Journal of Learning, Teaching and Educational
Chamorro-Atalaya, O., Olivares-Zegarra, S., Sobrino-Chunga, L., Guerrero-Carranza, R.,
Vargas-Díaz, A., Huarcaya-Godoy, M., Rasilla-Rovegno, J., Suarez-Bazalar, R.,
Poma-Garcia, J., & Cruz-Telada, Y. (2023). Application of the Chatbot in
University Education: A Bibliometric Analysis of Indexed Scientific Production
in SCOPUS, 2013-2023. International Journal of Learning, Teaching and Educational
Chavez, A. C. C. (2022). Literature review on the Metaverse approach in higher
education. [Thesis]. Salesian Polytechnic University, Guayaquil, Ecuador.
https://dspace.ups.edu.ec/handle/123456789/24393
Chediak, A. N. D. (2023). Analysis of the Metaverse and interactions in Sandbox video games:
case study: Grand theft auto V online. [Thesis]. Salesian Polytechnic University,
Guayaquil, Ecuador. https://dspace.ups.edu.ec/handle/123456789/24584
Díaz, D. S. A., & Díaz, O. A. (2021). The flipped classroom as a learning strategy. Revista
Conrado, 17(80), 152-157.
http://scielo.sld.cu/scielo.php?script=sci_arttext&pid=S199086442021000300152
Erazo, J., & Sulbarán, P. (2022). Metaverse: beyond immersive reality. The metaverse in
higher education in Mexico, 13(25), 129-140.
https://convite.cenditel.gob.ve/revistaclic/index.php/revistaclic/article/view
/1132
Espinoza, L. I., & Robalino, I. G. (2022). Inverted class as an innovative method in the
law course. Journal of Research in Legal Sciences, 5(17), 385-394.
https://doi.org/10.33996/revistalex.v5i17.134
Espinoza, J. J. V. (2022). Multiverses and Metaverses in the present global reality.
Multiverso Journal, 2(2), 4-6. https://doi.org/10.46502/issn.2792-3681/2022.2.0
Fernández, M. O. G., & Gaytán, P. H. (2019). Flipped classroom experience to promote
prosumer students of the upper level. RIED Revista Iberoamericana de Educación
a Distancia, 22(2), 245-263. http://dx.doi.org/10.5944/ried.22.2.23065
Ferrer-Torregrosa, J., Jiménez-Rodríguez, M. A., Torralba-Estelles, J., Garzón-Farinós, F.,
Pérez-Bermejo, M., & Fernández-Ehrling, N. (2016). Distance learning ects and
flipped classroom in the anatomy learning: comparative study of the use of
augmented reality, video and notes BMC Medical Education, 16(230), 1-9.
https://doi.org/10.1186/s12909-016-0757-3
Fidan, M., Debbag, M., & Cukurbasi, B. (2021). Metaphoric perceptions of pre-service
teachers about ‘LEGO Robotic Instructional Practices,’ ‘Augmented Reality’ and
‘Flipped Classroom’ concepts. Research in Comparative and International Education,
16(1), 83–99. https://doi.org/10.1177/1745499920982761
Galea, A. L. F. (2023). The Metaverse in education: The immersive school. ACTA digital
Journal, 147, 1-24.
https://www.acta.es/medios/articulos/formacion_y_educacion/147001.pdf
Gaviria-Rodríguez, D., Arango-Arango, J., Valencia-Arias, A., & Bran-Piedrahita, L.
(2019). Perception of the inverted classroom strategy in university scenarios.
Mexican Journal of Scientific Research, 24(81), 593-614.
https://dialnet.unirioja.es/servlet/articulo?codigo=7135277
Gea, E. V. (2021). Bibliometric studies on the impact of ICT. EDMETIC Journal of Medical
Education and ICT, 10(2), 1-3. https://doi.org/10.21071/edmetic.v10i2
267
George-Reyes, C. E., Ramírez-Montoya, M. S., & López-Caudana, E. O. (2023).

Imbrication of the Metaverse in the complexity of education 4.0: Approach from
an analysis of the literature. Pixel-Bit. Media and Education Journal, 66, 199–237.
https://doi.org/10.12795/pixelbit.97337
González, M. C. O., Garzón, J. F. P., Carrillo, Y. Q., & Alarcon, J. C. O. (2022). The
metaverse in higher education in Mexico. Digital Journal of Information Technology
and Systems, 6(2), 25-31. https://redtis.org/index.php/Redtis/article/view/112
Guze, P. A. (2015). Using Technology to Meet the Challenges of Medical Education.
Transactions of the American Clinical and Climatological Association, 126, 260-270.
https://pubmed.ncbi.nlm.nih.gov/26330687/
Hsu, C. H.C. (2018). Tourism education on and beyond the horizon Tourism Management
Perspectives, 25, 181-183. https://doi.org/10.1016/j.tmp.2017.11.022
Hsu, W.-C., Lin, H.-C.K., & Lin, Y.-H. (2017). The research of applying Mobile Virtual
Reality to Martial Arts learning system with flipped classroom. In: Proceedings of
the 2017 IEEE International Conference on Applied System Innovation: Applied System
Innovation for Modern Technology, ICASI 2017, pp. 1568–1571.
https://doi.org/10.1109/ICASI.2017.7988228
Huang, R., Spector, J. M., & Yang, J. (2019). Introduction to Educational Technology.
Lecture Notes in Educational Technology, 3–31. https://doi.org/10.1007/978-981-
13-6643-7_1
Ibáñez, M.-B., & Delgado-Kloos, C. (2018). Augmented reality for STEM learning: A
systematic review. Computers and Education, 123, 109-123.
Im, S.W.T, Chiu, P.H.P., Shek, C. H., Ng, M., & Li, L. (2019). Using Virtual Reality to
Enhance Learning in a Chinese Architectures Course: A Flipped Classroom
Approach. In: Proceedings of 2018 IEEE International Conference on Teaching,
Assessment, and Learning for Engineering, TALE 2018, pp. 624–629.
https://doi.org/10.1109/TALE.2018.8615369
Ji, Y., & Han, Y. (2019). Monitoring indicators of the flipped classroom learning process
based on data mining - taking the course of "virtual reality technology as an
example. International Journal of Emerging Technologies in Learning, 14(3), 166–176.
Jiao, C., Qian, K., & Zhu, D. (2020). Application of Flipped Classroom Teaching Method
Based on VR Technology in Physical Education and Health Care Teaching. IEEE
Access, 20, 1-9. https://doi.org/10.1109/ACCESS.2020.3019317
Jiménez-Serranía, M. I. (2022). Implementation and evaluation of the Flipped Classroom
in Pharmacology. In K. Konstantinidi (ed.), Metodologías de enseñanza-aprendizaje
para entornos virtuales. (pp. 12-23). Madrid, Spain: Adaya Press.
https://doi.org/10.58909/ad22804532
Kalavally, V., Chan, C. L., & Khoo, B. H. (2014). Technology in learning and teaching:
Getting the right blend for first year engineering. In Proceedings of 2014
International Conference on Interactive Collaborative Learning, ICL 2014, pp. 565–
570. https://doi.org/10.1109/ICL.2014.7017834
Larreategui, S. Y. C., Yalta, E. M. R., Torres, D. M., & Regalado, O. L. (2021). The flipped
classroom in student learning: systematic review. EDUTEC Electronic Journal of
Educational Technology, 77, 152-168.
https://doi.org/10.21556/edutec.2021.77.1967
Lechtaler, A. C., Blanc, C. F., Carden, M. L., Köhler, A., Polak, A. G., & Señoriño, J. M.
(2015). Immersive Simulation with Augmented Reality. In XVII Workshop of
Researchers in Computer Science, Salta.
http://sedici.unlp.edu.ar/handle/10915/46462
268
Lin, H. C., Hwang, G. J., Chou, K. R., & Tsai, C. K. (2023). Fostering complex professional
skills with interactive simulation technology: A virtual reality-based flipped
learning approach. British Journal of Educational Technology, 54(2), 622–641.
https://doi.org/10.1111/bjet.13268
Liou, W. K., Bhagat, K. K., & Chang, C. Y. (2016). Beyond the Flipped Classroom: A
Highly Interactive Cloud-Classroom (HIC) Embedded into Basic Materials
Science Courses. Journal of Science Education and Technology, 25(3), 460-473.
https://doi.org/10.1007/s10956-016-9606-8
López-Belmonte, J., Pozo-Sánchez, S., Moreno-Guerrero, A.-J., & Lampropoulos, G.
(2023). Metaverse in Education: a systematic Review. Distance Education Journal
(RED), 23(73), 1-25. https://doi.org/10.6018/red.511421
Maldonado, F. J., Ramírez, J. L., & Andrade, M. I. B. (2020). Immersive virtual reality
routes as a technological alternative in the educational process. Revista
Metropolitana de Ciencias Aplicadas, 3(1), 48-56.
https://remca.umet.edu.ec/index.php/REMCA/article/view/230
Mendiola, M. S. (2022). The metaverse: the door to a new era of digital education.
Research in Medical Education, 11(42), 5-8.
https://doi.org/10.22201/fm.20075057e.2022.42.22436
Menéndez-Aponte, S. M., Gil, D. R., Ramírez, E. L., & Martínez, R. R. (2023). The
Metaverse and Anesthesiology Education. Chilean Journal of Anesthesia, 52(3),
262-266. http://dx.doi.org/10.25237/revchilanestv522003154
Miguélez-Juan, B., Gómez, P. N., & Mañas-Viniegra, L. (2019). Immersive Virtual Reality
as an Educational tool for social transformation: An exploratory study on the
perception of students in Post-Compulsory Secondary Education. Aula Abierta,
48(2), 157-166. https://doi.org/10.17811/rifie.48.2.2019.157-166
Mikułowski, D., & Brzostek-Pawłowska, J. (2020). Multi-sensual augmented reality in
interactive accessible math tutoring system for flipped classroom. In Lecture
Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence
and Lecture Notes in Bioinformatics), 16th International Conference on Intelligent
Tutoring Systems, ITS 2020, pp. 1-10. https://doi.org/10.1007/978-3-030-49663-
0_1
Montoya, M. H., & León, C. A. D. (2018). Using immersive technologies in education: virtual
and augmented reality and smartroom. American University Corporation.
Mujica-Sequera, R. (2022). The Metaverse as a Transcomplex Scenario of
Technoeducation. RTED, 13(1), 20-28. https://doi.org/10.37843/rted.v13i1.268
Nieto, J. A. (2022). An introduction to the Metaverse: conceptualization and scope of a
new online universe. Scientific Journal of Strategies, Trends and Innovation in
Communication, 24, 41-56. http://dx.doi.org/10.6035/adcomunica.6544
Noor, A. K. (2013). Envisioning engineering education and practice in the coming
intelligence convergence era - A complex adaptive systems approach. Central
European Journal of Engineering, 3(4), 606–619. https://doi.org/10.2478/s13531-
013-0122-9
Ocampo, B. P. O., Romero, M. E. O., Alvarado, J. L. E., González, J. L. L., & Freire, E. E. E.
(2021). Considerations on flipped classroom and gamification in the area of
social sciences. University and Society Journal, 13(3), 497-504.
http://scielo.sld.cu/scielo.php?script=sci_arttext&pid=S2218-
36202021000300497
Olmos, M. V. M. (2022). Augmented reality: an undetectable technological tool for
immersive learning in virtual environments. Revista Honoris Causa, 14(2), 227-
238.
269
Padilla, D. B., Vázquez-Cano, E., Cevallos, M. B M., & Meneses, E. L. (2019). Use of
augmented reality apps in university classrooms. Campus Virtuales, 8(1), 37-48.
http://uajournals.com/ojs/index.php/campusvirtuales/article/view/379
Pozo-Sánchez, S., López-Belmonte, J., Moreno-Guerrero, A. J., & Fuentes-Cabrera, A.
(2021). Effectiveness of flipped learning and augmented reality in the new
educational normality of the Covid-19 era. Texto Livre, 14(2), 1-17.
https://doi.org/10.35699/1983-3652.2021.34260
Prahani, B. K., Dawana, I. R., Jatmiko, B., Amelia, T. (2023). Research Trend of Big Data
in Education during the Last 10 Years. International Journal of Emerging
Technologies in Learning, 18(3), 39-64. https://doi.org/10.3991/ijet.v18i10.38453
Prahani, B. K., Rizki, I. A., Jatmiko, B., Suprapto, N., & Amelia, T. (2022). Artificial
Intelligence in Education Research during the last ten Years: A Review and
Bibliometric Study. International Journal of Emerging Technologies in Learning,
17(8), 169-188. https://doi.org/10.3991/ijet.v17i08.29833
Prieto, A., Barbarroja, J., Álvarez, S., & Corell, A. (2021). Effectiveness of the flipped
classroom model in university education: a synthesis of the best evidence.
Education Journal, 391, 149-177. https://doi.org/10.4438/1988-592X-RE-2021-391-
476
Proskura, S. L., & Lytvynova, S. H. (2020). The approaches to Web-based education of
computer science bachelors in higher education institutions. In CEUR Workshop
Proceedings, 2643, pp, 609 – 625. https://doi.org/10.55056/cte.416
Quinto-Román, M. C. (2022). Influence of the flipped classroom on pedagogical
management during distance education. Valdizana Research, 16(1), 33-41.
https://doi.org/10.33554/riv.16.1.1296
Quiroz, M. Q. G., Alava, V. B., & Sarango, Y. L. (2023). Educational update: A literature
review on emerging methodologies in the metaverse. Podium, 43, 73-92.
https://doi.org/10.31095/podium.2023.43.5
Ramos, S. H., Rodríguez, A. L. D. T., Garay, G. O. R., Chávez, M. P. A., & Trejo, T. E. M.
(2019). Immersive and interactive virtual reality in the teaching of design: an
enriched environment for interdisciplinary teaching-learning. Revista Electrónica
sobre Ciencia, Tecnología y Sociedad, 6(11), 1-24.
https://www.ctes.org.mx/index.php/ctes/article/view/698
Repiso, R., Moreno-Delgado, A., Aguaded, I. (2020). Factors that influence the frequency
of citation of an article. Iberoamerican Journal of Science Measurement and
Communication, 1(1), 1-7.
http://eprints.rclis.org/40292/1/Opinion%20Paper%202_Spanish.pdf
Rodríguez, B. (2020). Augmented reality, virtual reality and learning in the international
higher education context. [Thesis]. Professional Academic School of
Communication Sciences and Technologies, Continental University, Huancayo,
Peru. https://repositorio.continental.edu.pe/handle/20.500.12394/10718
Roller, H. I. (2022). Prospects for Post-Pandemic Higher Education. En Líneas Generales, 8,
21-32. https://doi.org/10.26439/en.lineas.generales2022.n008.5993
Ros, G., & Laguna, M.T. R. (2021). Influence of the flipped classroom on the initial
scientific training of teachers: benefits in the teaching-learning process, attitudes
and expectations towards science. Revista de Investigación Educativa, 39(2), 463-
482. http://dx.doi.org/10.6018/rie.434131
Ruiz-Campo, S., Matías-Batalla, D. D., Boronat-Clavijo, B., & Acevedo-Duque, A. (2022).
Metaverses as teaching tool in higher education instructors training. Latin
American Journal of Educational Technology, 22(1), 135-153.
https://doi.org/10.17398/1695-288X.22.1.135
270
Sánchez, C. M. C., León, L. A. G., Yanes, R. C. A., & Oloriz, M. A. G. (2022). Metaverse:
the future of medicine in a virtual world. Metaverse Basic Applied Research, 1(4), 1-
8. https://doi.org/10.56294/mr20224
Sánchez-Cruzado, C., Sánchez-Compaña, M. T., & Ruiz, J. (2019). Real experiences of
flipped classroom as a Methodological strategy in Spanish University Education.
PUBLICACIONES 49(2),39-58.
http://dx.doi.org/10.30827/publicaciones.v49i2.8270
Shin, M., Prasad, A., Sabo, G., Macnow, A. S. R., Sheth, N. P., Cross, M. B., &
Premkumar, A. (2022). Anatomy education in US Medical Schools: before,
during, and beyond COVID-19. BMC Medical Education, 22(1), 1-8.
https://doi.org/10.1186/s12909-022-03177-1
Silva-Díaz, F., Carrillo-Rosúa, J., & Fernández-Plaza, J. A. (2021). Use of immersive
technologies and their impact on the scientific-mathematical attitudes of
Secondary Education students, compulsory in a context at risk of social
exclusion. Educar, 57(1), 119-138. https://doi.org/10.5565/rev/educar.1136
Sotgiu, M. A., Mazzarello, V., Bandiera, P., Madeddu, R., Montella, A., & Moxham, B.
(2020). Neuroanatomy, the Achilles’ Heel of Medical Students. A Systematic
Analysis of Educational Strategies for the Teaching of Neuroanatomy.
Anatomical Sciences Education, 13(1), 107-116. https://doi.org/10.1002/ase.1866
Suco, L. M. Q., Loja, C. M. L., & Suquilanda, S. A. P. (2021). The flipped classroom as a
strategy for educational innovation: Proposal for teacher training. Ciencia Latina
Multidisciplinary Scientific Journal, 5(5), 7882-7908.
https://doi.org/10.37811/cl_rcm.v5i5.881 p.7882
Tang, C., Gill, A. Pike, M., & Towey, D. (2021). Creating a virtual reality OER application
to teach web accessibility. In Proceedings-2021 IEEE 45th Annual Computers,
Software, and Applications Conference, COMPSAC 2021, pp. 1137–1142.
https://doi.org/10.1109/COMPSAC51774.2021.00156
Teo, T., Khazaie, S., & Derakhshan, A. (2022). Exploring teacher immediacy-(non)
dependency in the tutored augmented reality game-assisted flipped classrooms
of English for medical purposes comprehension among the Asian students.
Computers & Education, 179, 1-20.
Vaicondam, Y., Sikandar, H., Irum, S., Khan, N., & Qureshi, M. I. (2022). Research
Landscape of Digital Learning over the Past 20 Years: A Bibliometric and
Visualisation Analysis. International Journal of Online and Biomedical
Engineering, 18(8), 4–22. https://doi.org/10.3991/ijoe.v18i08.31963
Vallejos, C. F. V. (2023). The Metaverse's entry into the information age. [Thesis]. University
of Chile, Santiago de Chile, Chile.
https://repositorio.uchile.cl/handle/2250/191851
Verón, V. C. S., Marín, M. B., & Barrios, T. H. (2021). The flipped classroom as a didactic
strategy for the generation of competences: a systematic review. RIED Ibero-
American Journal of Distance Education, 24(2), 284-301.
https://doi.org/10.5944/ried.24.2.29027
Villamarín, J. N. C. (2023). Trends in the configuration and future of higher education.
Scientific research journal TSE' DE, 6(2), 1-25.
https://doi.org/10.60100/tsede.v6i2.146
Xiao-Dong, L., & Hong-Hui, C. (2020). Research on VR-supported flipped classroom
based on blended learning- a case study in learning English through news.
International Journal of Information and Education Technology, 10(2), 104-109.
https://doi.org/10.18178/ijiet.2020.10.2.1347
271

Vol. 22, No. 10, pp. 271-289, October 2023
Received Sep 3, 2023; Revised Oct 13, 2023; Accepted Nov 1, 2023
Factors Contributing to Resistance in the use of

Information and Communications Technology: A
Snapshot on Engineering Graphics and Design
Teachers
Philani Brian Mlambo* and Mogale Simon Albert Maeko
Faculty of Arts and Design, Durban University of Technology,
Pietermaritzburg, South Africa
Samuel Dumazi Khoza

Faculty of Humanities, Tshwane University of Technology,
Soshanguve, South Africa
Abstract. The prevalent adoption of ICT in education has compelled even

EGD teachers to integrate ICT in teaching and learning of EGD. However,
not every teacher understands the importance of using technology in
EGD lessons, even though it offers a lot of benefits for both teachers and
learners. Many scholars have attributed this resistance of teachers to
many factors, such as gender, age, and attitudes as well as the fact that
teachers do not perceive technology as important as they believe that
traditional methods are still bearing fruit. In an attempt to understand the
actual reasons behind teachers’ resistance to ICT integration in EGD
lessons, this study followed a qualitative approach, and semi-structured
interviews were used to gather data. Purposive sampling was used to
identify 11 EGD teachers to take part in this study. Furthermore, the study
employed the Technology Acceptance Model (TAM) framework, which
puts emphasis on the acceptance and adoption of technology. This
framework assisted in pinpointing factors contributing to ICT resistance
and how teachers can accept technology for the purpose of teaching and
learning. The findings show that factors such as age, lack of proper skills
to use technology, lack of exposure to technology at the tertiary level,
attitude, and shortage of resources in schools are contributing to teachers’
resistance to using technology. The study recommends that the
department of education should train teachers and provide schools with
relevant ICT resources.
Keywords: Resistance; technology; ICT training; Engineering Graphics

and Design; Information and Communication Technology; AutoCAD
*
Corresponding author: Philani Brian Mlambo; brianmdineka@gmail.com
©Authors
1. Introduction
The adoption and usage of ICT in teaching and learning is increasing at a very
high rate, and this calls for all the stakeholders in education to embrace technology
for the benefit of learners. To further emphasise the rise in the adoption of ICT
across the globe, scholars articulate that ICT has brought about changes in the
style of teaching and learning. Current technologies have achieved a level of
maturity that qualifies them to be considered enablers of teaching and learning
excellence. These technologies include, but are not limited to, the Internet,
AutoCAD, overhead projectors and even WhatsApp, which was mentioned by
Mlambo (2023) as one of the technologies that EGD teachers relied on during the
COVID-19 lockdown. In the uMgungundlovu District where this study was
conducted, it was observed that most schools have technologies, but they do not
use them, which compelled the need to investigate factors that contribute to not
using technologies. These technologies do not only assist learners but teachers as
well to deliver content in an easy manner that is of benefit to learners, as Mlambo
(2023) contends that using ICT in EGD lessons assists learners to grasp some
concepts better, resulting in higher performance in sections that learners are
finding difficult to understand. However, in spite of the benefit associated with
the integration of ICT in teaching, the literature shows that teachers are often
reluctant to adopt technologies for the enhancement of teaching and learning
(Cullen, 2018; Johnson et al., 2016), and this is not unique to the South African
context where this study was conducted. The resistance to ICT adoption among
high school teachers has been on the rise in recent years, and it was brought to
light during the COVID-19 pandemic. During the COVID-19 lockdown, most
teachers in many South African schools and the world over could not conduct
online classes, and this was attributed to many reasons. A study by Hyndman
(2018) alludes to the scarcity of teachers who are aware of the benefits associated
with the use of technology in EGD lessons as one of the reasons. Pure resistance
to using ICT has also been observed to be one of the reasons teachers are not aware
of the benefits of ICT in teaching and learning. This level of resistance to the
adoption by ICT in EGD teachers can be further attributed to many factors, such
as gender, age, and attitude among others. To meet the objectives of this study, it
aimed at investigating factors contributing to teachers’ resistance to the adoption
of ICT in EGD classrooms. This was necessitated by the current experience on the
ground where the study was conducted (uMgungundlovu District), where most
schools do have access to technologies but are not using them in EGD lessons.
Therefore, the researchers embarked on this journey of investigating factors
contributing to teachers not using these technologies.
2. Research Questions
This study was guided by the following research questions:
1. What are factors that contribute to teachers’ resistance to the use of ICT in
EGD lessons?
2. What strategies can be put in place to curb EGD teacher resistance to ICT
adoption?
3.1 Information and Communications Technology
To understand the concept of ICT that has become integral all over the world, a
definition must be attached to it. According to Education (2004), ICT is the
processing, management, and sharing of data, information, and knowledge that
are made possible through networks, technology, software, and methods of
communication, cooperation, and engagement. The above definition was given
by the DoE more than 18 years ago, which is an indication that ICT has always
been there and has slowly gained momentum. In recent years, ICT has gained
prominence; hence, Bornman (2016) postulates that ICT includes all technological
tools such as computers and tablets that enable individuals to access, create, and
modify information, which many South African schools possess. In the same vein,
Cardullo et al. (2018) describe ICT as network devices, computers, and all other
wide ranges of telecommunication technologies that can assist in interactive
communication and are also capable of performing data communication. Another
definition of ICT is given by Linden (2013) as devices used for communication,
such as cell phones, computers, and other applications used to pass information.
In EGD, such could be used to draw on a computer using an AutoCAD
programme and save them for later usage. In EGD, AutoCAD is a useful ICT tool
for design and technical documentation that replaces manual drafting with an
automated process. The concept of ICT simply means that all the means used by
teachers with the purpose of infusing technology are said to be ICT. The concept
of ICT revolves around the usage of technological tools such as computers, among
other things. This means that educators need to be fully equipped with ICT skills
so that they can infuse ICT effectively. Such could be beneficial for teaching
concepts such as solid geometry and isometric drawing, to name a few, in which
EGD can be effectively taught through ICT. In South Africa, many schools have
ICT tools and some teachers have been gotten training in many ICTs. This can be
accompanied by the fact that many learners do own a device or two, and that
means ICT is not foreign to us in the teaching and learning situation. Regardless
of the above, some EGD teachers are still resisting using technology in their EGD
lessons.
3.2 Factors influencing teachers’ ICT resistance
In spite of the transformation brought about in all aspects of society, the infusion
of ICT into education is still in the early stages of development (Mashile, 2017).
This shows that the integration of ICT into education by teachers is happening
more slowly than expected. The slow integration of ICT might be owing to many
factors, such as the lack of training of teachers and the unwillingness of teachers,
which can be associated with attitude and a lack of confidence in using ICT tools.
Mashile (2017) indicated that the slow integration of ICT is because South Africa
has only 26% of teachers who are equipped with ICT skills, which is a shocking
statistic as technology keeps evolving. However, a lot has happened six years
down the line since Mashile’s study in 2017. Many schools have wi-fi,
smartboards, and laptops, and they often use a platform called the D6
Communicator, which teachers, learners, and parents engage in. Mahdi and Al-
Dera (2013) contest that age is one of the factors that influence teachers' ICT
capabilities and willingness to infuse ICT in education. In support of Mahdi and
Al-Dera’s findings, Msila (2015) posits that older teachers tend to avoid using
computers in their lessons. One of the people interviewed by Msila said, “I feel less
professional when I do not know what seems to be the basic computer knowledge,” and
another teacher said, “I have been teaching for years without these computers." Based
on Msila’s (2015) findings, it can be seen that older teachers are resistant to using
technology, which is one of the factors that this section is trying to look at. But
also, it does not mean that younger teachers are willing to infuse ICT. There is a
wide range of factors that contribute to teachers’ resistance to infusing ICT.
Another is teachers’ shortage of skills to use technology. Raman and Yamat (2014)
further revealed that there are many factors that hinder teachers from integrating
ICT, and one of those factors is the age and experience of the teachers. In support
of this notion, a participant in a study conducted in Malaysia by Raman and
Yamat (2014) said, “I have 10 years of teaching experience. So, I am very convenient
using my manual ways, and I did not grow up in a technological environment. I prefer
reality experiences rather than looking at monitors.” Another participant said, “My age
is 53, and I have been teaching for more than 25 years." Also, “I believe in the traditional
method, which lets the learners touch, feel, and learn. I could make interactive lessons
without using the ICT tools.” and "I think this is not the time for ICT integration in my
teaching.” This resistance to using ICT may be due to many reasons, like not being
trained or a lack of facilities at schools. These forms of resistance from teachers
were also evident in Zimbabwe, as a study done by Matongo (2022) revealed that
teachers are not infusing ICT into teaching because they lack ICT skills, which can
be acquired through training. Matongo (2022) further states that not being trained
results in teachers not being confident in using technology in classes full of ICT-
capable learners and often suffering from technology phobia. Another factor that
influences the resistance of teachers is that they were not trained or taught how to
use ICT while they were at university. To validate the above, Matongo (2022) and
Quaye et al. (2015) assert that teachers are not infusing ICT in their lessons because
they were not trained in the colleges where they obtained their teaching
qualifications, which influences their attitudes when turning professional.
In addition, literature has also shown that teachers' insufficient skills in using the
internet discourage them from integrating ICT in their lessons (Yunus & Wekke,
2009). On the other hand, Jegede (2009) argues that age is not the contributing
factor, but attitude is. This notion by Jegede (2014) shows that age doesn’t
influence the usage of ICT, which is in contrast to claims made by Mahdi and Al-
Dera (2013) and Msila (2015). Scholars such as Badri et al. (2013) and Copriady
(2014) concur, as they claim that a teacher’s decision to use or not use ICT is merely
associated with their attitude. Copriady (2014) also mentioned that teachers have
a nonchalant attitude towards the implementation of ICT; hence, their level of
willingness and readiness is extremely low. In support of the above, the literature
has shown that teachers do not want to integrate ICT into their EGD lessons, as
they feel that the traditional way is still bearing fruit and EGD might not be an
exception. Interestingly, Radović-Marković (2010) indicates that traditional
teaching allows teachers to have more personal interaction with the learners,
which is in contrast to what online learning can offer. Because EGD needs lots of
practice, the teaching of the subject has always been synonymous with personal
interaction with learners through showing of teaching methods.
Another factor that can contribute to teachers’ resistance to infusing ICT is a lack
of confidence. Rastogi and Malhotra (2013) reported that some teachers had the
basic knowledge of using computers but lacked the confidence to use them in the
teaching and learning process. Rastogi and Malhotra (2013) further mentioned
that they knew the basics of many software programmes but were too sceptical to
use them for teaching and learning purposes. For example, in EGD, a teacher
might feel confident when using AutoCAD without the presence of learners, but
it becomes a challenge for them to operate it in front of them. This comes down to
a question of teachers' attitudes towards ICT, as Rastogi and Malhotra cite in the
study they conducted. In a study, they also reported that the success of
implementing the new curriculum with ICT in education depends greatly upon
the attitudes of the teachers and their willingness to embrace such technology and
ICT knowledge and skills (Rastogi & Malhotra, 2013). Rastogi and Malhotra (2013)
further explained that teachers should not only have ICT skills but also the right
attitude toward ICT use. In a study by Mustafina (2016), findings show that
teachers display a positive attitude toward ICT integration. The study further
revealed that gender, age, knowledge, and confidence play a crucial role in
shaping teachers’ attitudes toward ICT. On the contrary, Raman and Yamat (2014)
postulate that age, gender, and attitude are some factors influencing teachers'
resistance to infusing ICT. Raman and Yamat (2014) further revealed that one of
the interviewed teachers said: “My age is 53 and I have been teaching for more than 25
years. I believe in traditional method which let the learners touch, feel, and learn. I could
make interactive lesson without using the ICT tools. This is not the time for ICT
integration in my teaching.” Another teacher said: “I prefer reality experiences rather
than looking at monitors. So, applying ICT tools in my classrooms is not effective.
However, traditional forms are more effective.” The issue of attitude towards the
integration of technology seems to be a frequent problem among teachers,
translating to a poor attitude towards the usage of technology in teaching and
learning. This assertion is further expressed by Erişti et al. (2012), who cited that
one of the barriers that affects teachers' ability to infuse ICT is their attitude.
Attitude is one of the factors that influences teachers’ resistance to infusing ICT
into EGD classes. Mustafina (2016) produced factors other than attitude that
contribute to teachers’ resistance to the integration of ICT into teaching and
learning. Figure 1 below shows some other factors that influence teachers’
attitudes toward technology integration.
Figure 1: Factors that influence teachers’ attitudes toward technology integration
(Mustafina, 2016)
Figure 1 above shows distinct factors that contribute to teachers’ attitudes towards
the infusion of ICT in teaching and learning. In addition, Al-Zaidiyeen et al. (2010)
state that a teacher who possesses a positive attitude toward ICT is more inclined
to infuse it into his teaching and learning. This is also evident in a study done by
Raman and Yamat (2014), “which found that teachers' attitudes toward the use of
technology in teaching and learning is one of the key factors” in the meaningful
use of computer technology in education. If this attitude continues unabated in
practical subjects like EGD, learners will be put at a disadvantage when getting
into institutions of higher education and also when competing for jobs.
Figure 1 above shows that gender was mentioned as one of the factors that
contributes to teachers’ resistance to infusing ICT. Gebhardt et al. (2019) posit that
female teachers are seen as being less likely to use technology as opposed to male
teachers. In the same vein, as reported by the International Computer and
Information Literacy Study (ICILS) 2013, a study done by Fraillon et al. (2019)
revealed that when asked about using technology in the classroom, female
teachers responded negatively. This is an indication that female teachers are
mostly shy or not inclined to infuse technology into teaching and learning as
compared to male teachers. In support of a study done by Mukhari (2016), one of
the interviewed people's responses was, “Women made excuses when asked to go for
computer training, and if they did, they left the course early.” And “In comparison, their
male counterparts were keen to be trained on various computer programmes.” It was
further revealed that female teachers exhibit low levels of ICT usage, and this is
due to a lack of ICT skills (Du Toit, 2015). Du Toit (2015) further mentions that
females do not want to use technology despite having all technologies available
to them in schools. In contrast to Du Toit’s (2015) assertion, Aslan and Zhu (2016)
found that gender is not a contributing factor, as both male and female teachers
were infusing ICT into their lessons. Aslan and Zhu (2016) further mentioned that
“teachers are integrating technology into their practises regardless of their gender." In the
uMgungundlovu District, there is a balance in terms of gender that was evident
in the EGD workshops attended, and EGD teachers included both old and young,
but the majority are younger teachers. Even though EGD is dominated by younger
educators, the level of ICT integration is extremely low in comparison to the
preference given to the traditional method of teaching. EGD is a subject that deals
with abstract concepts, and the usage of ICT tools can help teachers deliver
content to the learners in a better way. It is presumed that this could also help
learners make sense of the abstract concepts that are being taught in EGD. This
assertion was echoed by Khoza (2013) and Makgato (2016) who found that
learners are usually poor at spatial visualisation. Integrating technology into the
classroom can be seen as the main reason for improving spatial skills (Akkuş &
Arslan, 2022). It is contended that ICT integration in the teaching and learning of
EGD is essential, and for that to be possible, teachers must be ready to infuse ICT.
Consequently, this study is looking at the factors contributing to ICT resistance
and how they can be addressed.
3.3 Remedies for teachers who are ICT resistant

The above section has shown that most teachers are resistant to using ICT in
teaching and learning, citing age, gender, and a lack of experience as other reasons
for their resistance. Not integrating ICT as part of their pedagogical strategies
somehow puts learners at a disadvantage, as technology in EGD, among other
subjects, has been proven to be an integral part. It is no secret that we are living in
a fast-paced world in terms of technology, thus it is vital that teachers use
technology when teaching so that learners can benefit immensely. The importance
of ICT integration in EGD lessons is noted by Mlambo (2023), who notes that
AutoCAD is one of the ICT tools EGD teachers could use to integrate ICT into
teaching and learning. Mlambo (2023) further argues that AutoCAD has the
ability to increase learners’ spatial awareness, which in turn improves their
performance in chapters such as isometric drawing and assembly drawing. As
much as ICT has all these kinds of benefits attached to it, the above sections have
shown that teachers are still resistant to using technology. In an attempt to assist
teachers who are resistant to the use of technology, scholars have indicated the
importance of subjecting teachers to ICT training. According to Adams (2020),
teachers without ICT are doomed. Furthermore, Barbour (2014) cited that teachers
should be subjected to development programmes and workshops where they will
be equipped with ICT skills. The aforementioned advances are an indication that
training is particularly important in ensuring that teachers are using technology
when teaching. To further emphasise the importance of ICT skills, Alazam et al.
(2013) postulate that ICT can prove to be a very crucial component in a classroom
if used wisely by a well-trained teacher. This simply means that teachers need to
be thoroughly trained so that they can infuse ICT into teaching and learning
effectively. Alazam et al. (2013) discovered that the level of teachers' ICT skills and
use was moderate in a study that examined the levels of ICT skills and ICT use in
classrooms. In addition, teachers who possess ICT skills are found to be more
useful than those who do not (Rastogi & Malhotra, 2013). This shows how
important it is for teachers to be thoroughly trained so that they can have relevant
skills for infusing ICT. The importance of training teachers is seen as a particularly
crucial step in ensuring that teachers are in a better position to infuse ICT.
Matongo (2022) further mentioned that without training, teachers develop a
phobia of technology as they are less motivated to use it due to a lack of ICT skills.
This simply means that ICT training plays a significant role in ensuring that
teachers integrate ICT into teaching and learning.
This study revolves around the adoption and usage of technology, and there are
many frameworks that revolve around the adoption and usage of technology.
According to Pultoo (2020), the Theory of Reasoned Action (TRA), the Theory of
Planned Behaviour (TPB), and the Technology Acceptance Model (TAM) are some
of the eight frameworks that revolve around the adoption and usage of
technology. In the context of this study, TAM, developed by Davis (1989), speaks
about the acceptance of using technology, and as this study investigates the
factors contributing to technophobia, the acceptance and usefulness of using
technology cannot be overlooked. It has been established that one way teachers
can adopt technology is through technology acceptance (Davis, 1989). The
aforementioned statement alludes to the fact that in order to manage teachers who
resist using technology, they must accept the adoption of technology into teaching
and learning. Consequently, this study also adopted the TAM framework as
shown in Figure 2 below.
Figure 2: Technology Acceptance Model (Davis, 1989)
TAM has six interrelated constructs made up of internal and external variables,
but for the purpose of this study, only three constructs were deemed relevant,
which are: perceived usefulness, perceived ease of use, and attitude. According to
Davis (1989), below are the definitions of these constructs:
Perceived ease of use: This is defined as one’s belief that using a certain system
will require a little effort, and this construct was used to investigate whether
teachers perceive the use of technologies in enhancing teaching and learning to be
easy.
Perceived usefulness: This is defined as the degree to which a person believes that
using a particular technology will enhance the job, and this construct was used to
investigate if teachers feel that infusing ICT in teaching and learning will bring
about improvements in learners’ ability to understand the subject matter.
Attitude: This is defined as a certain feeling towards technology usage. It is further
mentioned that showing a positive attitude towards technology will motivate a
user to use it.. This construct was used to measure attitudes in teachers towards
technology, as some authors perceive attitude as one of the factors that contributes
to technophobia.
5. Research Methodology
5.1 Research approach
In research studies, there are three commonly used research approaches:
qualitative, quantitative, and mixed-method approaches, and they are all unique
in terms of the qualities they bring. Consequently, this study deemed it fit to
employ a qualitative research approach. Roberts (2010) defines qualitative
research as a form of study that is used to gather rich descriptive data from the
participants. Put simply, it means you get the information from the horse’s mouth,
which can be referred to as a primary source of data. The researcher opted for this
approach because of its ability to gather an in-depth understanding of the
phenomena under investigation, which is the investigation of factors that
contribute to EGD teachers’ resistance to using ICT.
5.2 Population and Sampling
The study used 11 EGD teachers from uMgungundlovu District. These teachers
were selected because of their common characteristics that the researcher deemed
necessary to assist with the objectives of the study. The participants were chosen
because they are teaching EGD, which is the phenomenon that is being
investigated in this study. Consequently, the researcher employed a purposive
sampling technique, which is also termed judgmental sampling. According to
Etikan et al. (2016), purposive sampling is a non-probability sampling technique
that is normally used in qualitative studies. Taherdoost (2016) argues that
purposive sampling is used because it is less time-consuming and low- cost,
among other advantages. The aforementioned advantages are some of the reasons
the researcher employed purposive sampling. Another reason is that the
researcher wanted to understand why some EGD teachers are resistant to using
ICT in the teaching and learning of EGD; hence, the answers could only be
provided by the EGD teachers. Therefore, the participants of this study included
11 teachers who are teaching EGD, which consisted of 4 female and 7 male
teachers.
5.3 Data Collection Instruments
To gather the viewpoints of the participants as advanced by Roberts (2012), data
was collected through semi-structured interviews. Interviews are one of the data
collection techniques that are normally used in qualitative studies. Collecting data
through interviews allows a researcher to engage with the participants verbally.
Cresswell (2007), cited by Raman and Yamat (2014), posits that interviews are
used because of their ability to elicit insights from the participants. The researcher
used semi-structured interviews to get an in-depth understanding of the factors
contributing to resistance to the adoption of ICT by EGD teachers.
5.4 Data analysis
Interviews can be conducted telephonically or face-to-face; in the context of this
study, the participants were interviewed face-to-face. The interviews were
conducted in EGD teachers’ labs, and they were tape recorded. The resulting data
was subjected to thematic analysis, which followed six steps developed by Braun
and Clarke (2006): which are (1) data familiarisation; (2) coding; (3) generating
themes; (4) reviewing themes; (5) defining and naming themes; and (6) writing
up.
5.5 Ethical considerations
Pseudonyms were used as a way of concealing the participants’ actual names.
Hence, they were referred to as teacher 1, teacher 2, etc. Using pseudonyms in
research is a common occurrence that plays a significant role in research ethics
and protecting the true identity of the participants. To ensure that the ethics were
upheld, a gatekeeper’s letter was sought from the DoE to conduct the study in
KwaZulu Natal schools that offer EGD. Upon receiving the letter, permission was
further sought from the principals of the selected. Once permission was granted,
informed consent forms were issued to teachers to consent to their participation
in the study.
6. Results
Below are the findings from the semi-structured interviews conducted with 11
EGD teachers. The interviews were done in EGD teachers’ classrooms during their
free time, and English was used as the medium of instruction. Each interview
lasted for about 15 minutes. The interview schedule consisted of two sets of
questions, one speaking to the teacher’s biography. Below are the responses of the
participants based on the questions asked.
The first question asked was about the software they used at the university for the
purpose of teaching EGD. This question was asked to ascertain whether or not
resistance stems from no previous exposure during training. From the teachers’
responses below, only one theme emerged.
Theme 1: Adequate ICT exposure
Based on responses from the interviewed teachers, it appears that they were
exposed to AutoCAD as an EGD technology that assisted them in teaching EGD
with ease. This is the case in teachers from the uMgungundlovu district, as the
findings above show that they were exposed to technology at the university, and
the chances of using technology in EGD lessons are high. When the teachers were
interviewed, below is how they responded:
“Yes I did. AutoCAD was the software we were exposed to while I was still at the
university”
“Yes, the software we used was AutoCAD” (teacher 6)
“Yes we did AutoCAD” (teacher 11)
“Yes. It was AutoCAD” (teacher 2)
“Yes. We were taught on how to use AutoCAD, the was also another module where we
were taught basic computer skills” (teacher 5)
“Yes we did AutoCAD during my time at the university” (teacher 1)
“Yes, while I was at the university I did AutoCAD” (teacher 10)
“At the university I did AutoCAD and a module which was called computer
technology”(teacher 8)
The second question asked was whether or not age does contribute to resistance
to the use of ICT in EGD lessons. From the responses, only one theme emerged,
which is discussed below:
Theme 2: Age Related
Based on the above responses, it is clear that resistance to using technology is
prevalent among female teachers. They highlighted that EGD teachers who have
plenty of experience do not want to engage in the use of technology, as they claim
that they have been using the traditional approach for a very long time without
encountering problems. They went as far as saying, “Why change when it is not
broken?” Teachers’ views about age are articulated below:
“Yes, it is a contributing factor. I have observed that older teachers are still using the
traditional approach of teaching” (teacher 9)
” es aging could be a contributing factor for teachers resisting to Interact with ICT or
digital lessons” (teacher 3)
“Yes is a contributing factor, old teachers who are over 40 years are not interesting in
using technology compared to their younger counterparts who are very excited about
using technology” (teacher 11)
“Yes, in my opinion old teachers prefer the use of traditional methods of chalkboard and
set squares” (teacher 5)
“Yes, teachers who are over the age of 35 don’t want to use technology as they are firm
believers of using traditional teaching” (teacher 3)
“Yes age is the contributing factor more especially for those teachers who have over 20
years of experience they feel too old to try something new” (teacher 2)
Another question asked was whether gender does contribute to teachers’
resistance to using technology in EGD lessons. From the responses below, only
one theme emerged.
Theme 3: Gender
Teachers were asked to ascertain whether gender is a contributing factor to
teachers’ resistance to using technology in the teaching and learning of EGD. A
number of teachers were of the opinion that gender has no relationship with
resisting using technology. The following questions asked were about factors they
think contribute to EGD teachers not using technology in their lessons. The
teacher’s views are articulated below:
“No. I don’t think so. I know females who are using technology as much as males”
(teacher 1)
“No, I believe every teacher want to use technology regardless of gender” (teacher 3)
“No gender has nothing to do with the use of ICT, both males and females can actively
engage in the use of ICT, in my school every teacher uses technology regardless of
gender” (teacher 10)
“No gender is not a contributing factor; teachers use technology regardless of gender”
(teacher 11)
“I don’t think gender is the contributing factor, teachers decide to use technology
regardless of gender” (teacher 9)
“Gender as well is not a factor because I know most female teachers who are using
technology in teaching and learning of EGD” (teacher 8)
Another question asked was about factors they think contribute to EGD teachers
not using technology in their lessons. And from their responses, only one theme
emerged:
Theme 4: Factors associated with ICT resistance
Apart from age, that has already been identified as a factor in resisting the
integration of ICT. EGD teachers also attributed the resistance to using technology
to many factors, such as a lack of proper skills to use technology, a lack of
exposure to technology at the tertiary level, an attitude, and a shortage of
resources in schools. These factors were mentioned by teachers below:
“Lack of ICT skills. Teachers need to be trained so that they use technology” (teacher 2)
“Some factors are that technology is confusing, it is difficult to use hence teachers are not
using it for that reason” (teacher 9)
“Some of the factors are lack of support system at school, shortage of technologies, lack of
software’s and electricity outages (load-shedding)” (teacher 10)
“Shortage of information/knowledge to use technology” (teacher 3)
“No resources or Inadequate resources provided, some teachers are incapacitated with the
use of ICT therefore they lack knowledge, they are not motivated to engage with, in my
case I’m capacitated with the use of these online platforms but I don’t have access to it
since I’m not assisted with regards to teacher profiling for ICT” (teacher 8)
“Lack of resources and ICT skills are some of the factors contributing to resistance of
using technology” (teacher 11)
“Lack of information about the usefulness of technology, background as well as does
contribute. As most teachers from disadvantaged schools are not using technology.
Another reason is the bad attitude of teachers towards using technology” (teacher 1)
“Not being exposed to technology at the university dictates whether they are going to use
technology or not” (teacher 5)
“Another is th teachers’ attitude towards technology” (teacher 7)
“Lack of proper development, lack of resources such as whiteboard, projectors and
computers, and training are some of the factors” (teacher 6)
The last question asked was about what could be done to ensure that EGD
teachers are not shying away from the idea of using technology in their EGD
lessons. Based on the findings below, only one theme emerged:
Theme 5: Teacher Training
The above responses from EGD teachers indicate that those teachers who are not
using technology in lessons due to a number of factors can be assisted through the
department of education subjecting those teachers to ICT training so that they can
be equipped with relevant ICT skills. It has been noted that mostly EGD teachers
understand the importance of using technology in EGD lessons but are not using
it because they do not know how to. As a result, the need for ICT training was
deemed necessary. Below are the views of the teachers captured when asked
about what can be done to assist teachers who are not using technology.
“I think there must be programmes set by the Department of Education to capacitate
those teachers who are resistant to technology” (teacher 1)
“We need to get workshops where it can accommodate every EGD teacher to familiarise
themselves with such, in that way teachers will change their perspective about
technology” (teacher 5)
“Those teachers should be trained and also be taught the importance of using technology
in EGD lessons” (teacher 10)
“Training is the best way to assist those teachers who are showing signs of resistance”
(teacher 11)
“I think the Department of Education should conduct workshops to capacitate those
teachers on how to use technology in their lessons and also the department should the
schools as well that offer EGD with relevant resources to aid in the infusion of
technology” (teacher 9)
“The Department of Education should make it mandatory for all teachers to use
technology and they should also conduct developmental programmes so that they can be
capacitated on the importance of using technology” (teacher 6)
“Equip schools with necessary resources and the Department of Education should train
them so that they can be able to use these technologies in their classrooms” (teacher 2)
7. Discussions
From the responses above, it was evident that most EGD teachers had exposure
to technology while they were in university, which has been seen as an
influencing factor in using technology when they get to school. The above is
echoed by Matongo (2022) and Quaye et al. (2015) who found that teachers’
exposure to technology during university time plays a major role when they turn
professional.
Furthermore, the participants indicated that age does contribute to resistance to
using technologies. In line with the participant views, studies (Msila, 2015; Raman
& Yamat, 2014) indicate that age is associated with resistance to using technology.
The same assertion is echoed in a study by Mahdi and Al-Dera (2013), which
found that age is one of the reasons teachers are not using technology.
Furthermore, findings of a study done in Kenya by Langat (2020) revealed that
the age of teachers influences ICT adoption, as 84.7% of teachers strongly agreed
that age does contribute to ICT usage. Consequently, it is clear that older teachers
tend to resist using technology in their EGD lessons. However, when teachers
were asked about gender being a contributing factor, they responded negatively.
They went as far as saying teachers use technology regardless of their gender. The
above findings were collaborated by other authors (Aslan & Zhu, 2016; Yusop et
al., 2021). “Teachers are Integrating technology in their practises regardless of their
gender” (Aslan & Zhu, 2016). In the same vein, a study conducted by Gebhardt et
al. (2019) revealed that there is no gender difference in using technology.
Furthermore, findings in a study done in Kenya by Mwei (2020) show that gender
has a marginal or no significant influence on teachers’ perceptions of factors
affecting the integration of ICT for instructional purposes. This is an indication
that gender is not a contributing factor in using technology in EGD lessons.
However, this is in direct contrast to other claims (Du Toit, 2015; Mukhari, 2016)
that females were less inclined to use technology in teaching and learning as
compared to their male counterparts.
The findings further revealed a host of factors that contribute to teachers’
resistance. All of the above-mentioned factors are in line with what many scholars
have discovered. It was outlined that factors are categorised into internal and
external factors (Alharbi, 2021; Xuan et al., 2023). According to Ismaili (2022),
external factors are infrastructure, facilities, support resources, and training
programmes and internal factors consist of factors such as perception, proficiency,
desire, and readiness of teachers. All of the above factors contribute one way or
another to teachers’ resistance in using technology. For example, both Matongo
(2022) and Quaye et al. (2015) argue that a lack of ICT skills plays a major role in
teachers not using technology in their lessons. Furthermore, a South African study
done in the Eastern Cape has revealed that insufficient ICT skills hinder teachers
from using technology (Chisango et al., 2020). In other studies, (Badri et al., 2013;
Copriady, 2014; Jegede, 2009), attitudes have been cited as the contributing factor
in using technology. In the same vein, a study by Chisango et al. (2020) held that
s technology is disruptive in class. To support this, one interviewed teacher said,
“I think using computers is a waste of my time, I have been teaching for the past 20 years
and getting good matric results without the aid of a computer so I can just as well continue
like that.” The above signals that teachers have a negative attitude towards using
technology. A negative attitude by teachers was also expressed in a study by
Manjawira (2022), which revealed that teachers in Malawi have a negative
perception of the integration of ICT in teaching and learning.
In support of the lack of availability of resources in schools as a reason teachers
are not using technology, Mathevula and Uwizeyimana (2014) posit that there is
a shortage of resources in schools that prevents the successful integration of
technology, which is further echoed by Addandani (2011) and (Alharbi (2021). The
lack of ICT resources is echoed by Xuan et al. (2023), who argue that a lack of ICT
infrastructure has been observed to be an issue in Vietnamese schools, which
shows that not only schools in South Africa are experiencing a lack of
infrastructure. In the same vein, schools in Morocco are experiencing a shortage
of ICT equipment, as only 10% of teachers responded ‘yes’ when asked if their
school has enough ICT equipment. (Ismaili, 2022).
Teachers outlined that developmental programmes such as workshops must be
conducted by the DoE to assist those teachers who are not using technology in
their EGD lessons. They also mentioned training as very important. The necessity
of training is outlined in the literature above by many authors, such as Langat
(2020), who posits that teachers should be subjected to developmental
programmes such as training workshops so that they are equipped to use
technology. In the same vein, it has been noted that ICT integration in teaching is
very easy when teachers are well-trained (Alazam et al., 2013; Ramnarain et al.,
2023). Furthermore, Matongo (2022) asserts that teachers without proper training
develop a phobia of using technology in class. The above assertion is an indication
that training is a very important part of ensuring that teachers are integrating ICT
into teaching and learning. The importance of training is further alluded to by
Chisango et al. (2020), who assert that teachers in the Eastern Cape province can
benefit from being subjected to ICT training as they showed a lack of ICT skills.
In support of the aforementioned, one participant said, “No, I do not use computers
in teaching and learning because they are few and I do not know how to use them to teach.”
This necessitates the need for ICT training workshops.
8. Conclusion
In response to the above objectives of the study, the study discovered that age,
lack of proper skills to use technology, lack of exposure to technology at the
tertiary level, attitude, and shortage of resources are factors that contribute to
teachers’ resistance to using technology in EGD lessons. The aforementioned
factors have been outlined by other scholars as well as shown in the literature
review of the study. This means that the resistance to using ICT can be curbed if,
during in service training at universities, teachers are taught how to use
technology so that when they turn professional, they are well equipped.
Attitude has been observed to be a contributing factor based on the findings
above, which means that teachers should work on their attitude, as a positive
attitude translates to willingness to use technology in EGD lessons as opposed to
a negative attitude, which translates to resistance. The findings above indicated
that the shortage of resources in schools was voiced by teachers, which can be
curbed by the Department of Education by providing schools with the relevant
ICT resources so that they can use technology.
The findings outlined the importance of ICT training as a tool that can be used to
assist those teachers. Teachers mentioned that developmental workshops should
be conducted by the Department of Education to train teachers on how to use
technology. This is because teachers said most teachers are not using technology
because they do not know how to use it. It is worth noting that addressing these
factors contributing to resistance to ICT integration by EGD teachers will improve
the results obtained in EGD, and teachers will finish the syllabus early to have
enough time for revision.
9. Recommendations
Based on the findings above, the study recommends that the department of
education conduct developmental workshops where they will train teachers on
how to use ICT in teaching and learning. The study discovered that there is a
shortage of ICT resources in schools, which hinders the integration of ICT. The
study further recommends that the department of education should also provide
schools with ICT resources, as most teachers indicated that they are able and
willing to use technology as they understand that the nature of EGD does require
ICT integration.
These resources can be in the form of an overhead projector, a white board, or
laptops, as these are considered basic ICT tools needed to kickstart the process.
The department of education can arrange for workshops to be held at least once a
month so that teachers can be consistently subjected to training.
10. Limitations of the study
The qualitative nature of the study means that the results cannot be generalised
to a wider population, and they are contextually bound. The study had 11 EGD
teachers as participants, which is a small sample, which may affect the
generalisation of the findings. It is worth noting that the researcher intended to
use 15 EGD teachers, but only 11 responded positively; hence, the sample had 11
teachers. Another limitation of this study was that only one method (semi
structured interviews) was used to gather data, which raised concerns about the
duplication of the results if the study were to be repeated.
11. Future research

This study focused on EGD teachers only. For future research, studies must be
conducted on learners to gather their perspective on integrating ICT into the
teaching and learning of EGD. One of the factors was the lack of training for
teachers; therefore, future research must be conducted on specific training
programmes EGD teachers should be subject to.
Acknowledgement
No specific grant or funding was provided for this study from any institution.
Declaration of interests
The researchers declare no conflict of interests.
12. References
Adams, C. (2020). Teachers need lots of training to do online learning well. Coronavirus closures
gave many just days. Retrieved 25 April from
https://hechingerreport.org/teachers-need-lots-of-training-to-do-online-
learning-well-coronavirus-closures-gave-many-just-days/
Addandani, K. (2011). The lack of the ability of Saudi teachers PhD thesis, University of King
Saud, Saudi Arabia].
Akkuş, İ., & Arslan, P. Y. (2022). The Effects of Augmented Reality in the Technical
Drawing Course on Engineering Students' Spatial Ability and Academic
Achievement. Journal of Learning and Teaching in Digital Age, 7(2), 160-174.
https://doi.org/10.53850/joltida.1020075
Al-Zaidiyeen, N. J., Mei, L. L., & Fook, F. S. (2010). Teachers' Attitudes and Levels of
Technology Use in Classrooms: The Case of Jordan Schools. International education
studies, 3(2), 211-218. https://doi.org/10.5539/ies.v3n2p211
Alazam, A.-O., Bakar, A. R., Hamzah, R., & Asmiran, S. (2013). Teachers’ ICT skills and
ICT integration in the classroom: The case of vocational and technical teachers in
Malaysia. Creative Education, 3(08), 70. https://doi.org/10.4236/ce.2012.38B016
Alharbi, A. (2021). A review of the internal and external factors affecting teachers’ ICT use
in Classroom. International Journal of Education and Research, 9(12), 105-116.
Aslan, A., & Zhu, C. (2016). Influencing Factors and Integration of ICT into Teaching
Practices of Pre-Service and Starting Teachers. International Journal of Research in
Education and Science, 2(2), 359-370. https://doi.org/10.21890/ijres.81048
Badri, M., Al Rashedi, A., & Mohaidat, J. (2013). School teachers’ technology readiness–
An empirical study applying readiness factors and teacher type categorization.
Proceedings of the 2013 International Conference on Information, Business and
Education Technology (ICIBET 2013). https://doi.org/10.2991/icibet.2013.124
Barbour, M. K. (2014). Training teachers for a virtual school system: A call to action. In K-
12 Education: Concepts, Methodologies, Tools, and Applications (pp. 1398-1415). IGI
Global. https://doi.org/10.4018/978-1-4666-4502-8.ch081
Bornman, E. (2016). Information society and digital divide in South Africa: results of
longitudinal surveys. Information, Communication & Society, 19(2), 264-278.
https://doi.org/10.1080/1369118X.2015.1065285
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative research
in psychology, 3(2), 77-101. https://doi.org/10.1191/1478088706qp063oa
Cardullo, V. M., Wilson, N. S., & Zygouris-Coe, V. I. (2018). Enhanced student engagement
through active learning and emerging technologies. In Student engagement and
participation: Concepts, methodologies, tools, and applications (pp. 399-417). IGI
Chisango, G., Marongwe, N., Mtsi, N., & Matyedi, T. E. (2020). Teachers’ perceptions of
adopting information and communication technologies in teaching and learning
at rural secondary schools in eastern cape, South Africa. Africa Education Review,
17(2), 1-19. https://doi.org/10.1080/18146627.2018.1491317
Copriady, J. (2014). Self-Motivation as a Mediator for Teachers' Readiness in Applying ICT
in Teaching and Learning. Turkish Online Journal of Educational Technology-TOJET,
13(4), 115-123. https://doi.org/10.1016/j.sbspro.2015.01.529
Cullen, E. (2018). What is Technology Enhanced Learning, and why is it important.
Mentimeter, Mentimeter. https://www.mentimeter.com/blog/interactive-
classrooms/what-is-technology-enhanced-learning-and-why-is-it-important
Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of
information technology. MIS quarterly, 319-340. https://doi.org/10.2307/249008
Du Toit, J. (2015). Teacher training and usage of ICT in education. New directions for the
UIS global data collection in the post-2015 context. UNESCO Institute for Statistics,
Retrieved, 16.
Education, D. O. (2004). White Paper on E-education: Transforming Learning and Teaching
Through Information and Communication Technologies (ICTs). DoE.
https://www.gov.za/documents/white-paper-e-education-transforming-
learning-and-teaching-through-information-and
Erişti, S. D. B., Kurt, A. A., & Dindar, M. (2012). Teachers’ views about effective use of
technology in classrooms. Turkish online journal of qualitative inquiry, 3(2), 30-41.
https://dergipark.org.tr/en/pub/tojqi/issue/21395/229369?publisher=tojqi
Etikan, I., Musa, S. A., & Alkassim, R. S. (2016). Comparison of convenience sampling and
purposive sampling. American journal of theoretical and applied statistics, 5(1), 1-4.
https://doi.org/10.11648/j.ajtas.20160501.11
Fraillon, J., Ainley, J., Schulz, W., Duckworth, D., & Friedman, T. (2019). IEA international
computer and information literacy study 2018 assessment framework. Springer Nature.
https://doi.org/10.1007/978-3-030-19389-8
Gebhardt, E., Thomson, S., Ainley, J., & Hillman, K. (2019). Teacher gender and ICT. In
Gender differences in computer and information literacy (pp. 53-68). Springer.
https://doi.org/10.1007/978-3-030-26203-7_1
Hyndman, B. (2018). Ten reasons why teachers can struggle to use technology in the
classroom. Science Education News, 67(4), 41-42.
Ismaili, A. A. (2022). ICT Use in the EFL Classroom in Morocco: EFL Teachers' Personal
and Technology-Related Variables. International Journal of Information and
Communication Technology Education (IJICTE), 18(1), 1-13.
https://doi.org/10.4018/ijicte.286759
Jegede, P. O. (2009). Age and ICT-related behaviours of higher education teachers in
Nigeria. Issues in Informing Science & Information Technology, 6.
https://doi.org/10.28945/3384
Johnson, A. M., Jacovina, M. E., Russell, D. G., & Soto, C. M. (2016). Challenges and solutions
when using technologies in the classroom. Routledge.
https://doi.org/10.4324/9781315647500-2
Khoza, S. D. (2013). Difficulties in sectional drawing: a case of student teachers at a
university based in the Eastern Cape.
https://doi.org/10.12973/eurasia.2016.1220a
Langat, W. K. (2020). Teacher Related Factors Influencing Integration of Information and
Communication Technology in Public Secondary Schools in Narok North Sub-county,
Kenya, University of Nairobi.
Linden, A. (2013). The importance of technology management in the ICT requirements
definition process. In 2013 Proceedings of PICMET'13: Technology Management
in the IT-Driven Services (PICMET) (pp. 2283-2295). IEEE.
Mahdi, H. S., & Al-Dera, A. S. A. (2013). The Impact of Teachers' Age, Gender and
Experience on the Use of Information and Communication Technology in EFL
Teaching. English Language Teaching, 6(6), 57-67.
https://doi.org/10.5539/elt.v6n6p57
Makgato, M. (2016). Difficulties of student teachers in the engineering graphics and design
course at a South African university: Snapshot on sectional drawing. EURASIA
Journal of Mathematics, Science and Technology Education, 12(4), 703-715.
https://doi.org/10.12973/eurasia.2016.1220a
Manjawira, M. T. (2022). Integrating Information and Communications Technology in
Teaching: Perceptions fromTeachers in Malawi. Journal of Education & Social
Sciences,, 2(10), 10-22. https://doi.org/10.20547/jess1022210202
Mashile, T. (2017). Technology integration and the digital divide: understanding factors that
impact on educators' ability to integrate technology in South African classrooms
University of Pretoria].
Mathevula, M. D., & Uwizeyimana, D. E. (2014). The challenges facing the integration of
ICT in teaching and learning activities in South African rural secondary schools.
Mediterranean Journal of Social Sciences, 5(20), 1087.
https://doi.org/10.5901/mjss.2014.v5n20p1087
Matongo, M. (2022). Teacher readiness to teach using ICT in classroom pedagogy in
Zimbabwean primary schools. Journal of African Education, 3(2), 45-66.
https://doi.org/10.31920/2633-2930/2022/v3n2a2
Mlambo, P. B. (2023). Information and Communications Technology in Engineering
Graphics and Design classrooms: A post COVID-19 era. Jurnal Penelitian dan
Pengkajian Ilmu Pendidikan: e-Saintika, 7(2).
https://doi.org/10.36312/esaintika.v7i2.1321
Msila, V. (2015). Teacher Readiness and Information and Communications Technology
(ICT) Use in Classrooms: A South African Case Study. Creative Education, 06(18),
1973-1981. https://doi.org/10.4236/ce.2015.618202
Mukhari, S. (2016). Teachers’ experience of information and communication technology
use for teaching and learning in urban schools (DEd thesis). Pretoria: University of
South Africa.
Mustafina, A. (2016). Teachers' Attitudes toward Technology Integration in a Kazakhstani
Secondary School. International Journal of Research in Education and Science, 2(2),
322-332. https://doi.org/10.21890/ijres.67928
Mwei, P. K. (2020). Teachers’ Perception of Factors Affecting Integration of Information
and Communication Technology for Instructional Purposes in Secondary Schools
in Kenya. International Journal of Educational Technology and Learning, 8(1), 25-37.
https://doi.org/10.20448/2003.81.25.37
Pultoo, A. B., Anoop & Meunier, Jean & Sheoraj, Kaviraj & Panchoo, Shabneez &
Naseeven, Parveen & Ujoodha, Mohun & Roocha, Vikramsing & Rajcoomar,
Hrishant & Oojorah, Avinash. (2020). Classe21. Educators' Acceptance of
Technology-Enhanced Classroom Using the UTAUT Model. (14), 39-48
Quaye, F., Ametepe, W., & Annan, N. K. (2015). The Impact of ICT on Teaching and
Learning in Tertiary Institutions: A Case Study of Wisconsin International
University College, Ghana. Journal of Information Engineering and Applications, 5(5),
8-14.
Radović-Marković, M. (2010). Advantages and disadvantages of e-learning in comparison
to traditional forms of learning. Annals of the University of Petroşani, Economics,
10(2), 289-298.
Raman, K., & Yamat, H. (2014). Barriers teachers face in integrating ICT during English
lessons: A case study. Malaysian Online journal of educational technology, 2(3), 11-19.
Ramnarain, U., Ncube, R., & Teo, T. (2023). South African Life Sciences Teachers'
Pedagogical Beliefs and their Influence on Information Communication and
Technology Integration. Frontiers in Education.
Rastogi, A., & Malhotra, S. (2013). ICT skills and attitude as determinants of ICT pedagogy
integration. European Academic Research, 1(3), 301-318.
Roberts, C. M. (2010). The dissertation journey: A practical and comprehensive guide to planning,
writing, and defending your dissertation. Corwin Press.
https://doi.org/10.4135/9781452219219
Taherdoost, H. (2016). Sampling methods in research methodology; how to choose a
sampling technique for research. How to Choose a Sampling Technique for Research
(April 10, 2016). https://doi.org/10.2139/ssrn.3205035
Xuan, T. T. H., Thanh, N. B. H., & Nhi, N. T. (2023). Factors affecting the flipped classroom
in the educational context of Vietnam. Journal of Education and e-Learning Research,
10(2), 99-110. https://doi.org/10.20448/jeelr.v10i2.4441
Yunus, M. M., & Wekke, I. S. (2009). The application of multicultural education and
applying ICT on Pesantren in South Sulawesi, Indonesia.
Yusop, F. D., Habibi, A., & Razak, R. A. (2021). Factors affecting Indonesian preservice
teachers’ use of ICT during teaching practices through theory of planned
behavior. SAGE Open, 11(2), 21582440211027572.
https://doi.org/10.1177/21582440211027572
290

Vol. 22, No. 10, pp. 290-305, October 2023
Investigating the Impact of Teaching and

Learning of Religious Institution [pondok] on the
Society in Kelantan, Malaysia
Nik Yusri Musa , Ateerah Abdul Razak*
Pusat Pemikiran Keamanan dan Kesejahteraan insan (PEMIKIR), Faculty of
Language Studies and Human Development
Amanina Abdul Razak@Mohamed

Akademi Pengajian Bahasa UiTM, Cawangan Selangor Kampus Dengkil,
Malaysia
Asma Lailee Mohd Noor , Mohd Zain Mubarak , Nur Azuki Yusuff
and Burhan Che Daud , Noor Hisham Md Nawi , Marwan Ismail and
Azhar Muhammad
Pusat Pemikiran Keamanan dan Kesejahteraan insan (PEMIKIR), Faculty of
Language Studies and Human Development
Abstract. The religious learning institution [pondok] is a common form of

traditional Islamic educational institution in the archipelagic areas of
Malaysia, Indonesia, and southern Thailand. It has been widely known
and established, especially in Kelantan, Malaysia, for many years. It is
regarded as a significant learning institution since it focuses on lifelong
learning concepts. However, there are some crises faced by learners who
study at a pondok. These need to be addressed as the learning institution
receives negative perceptions from society in general. This study aimed
to analyse the perception of the role of religious learning institutions and
how this contributes to spreading knowledge so that society can get a
better understanding of the concept of learning in a pondok. This study
uses both quantitative and qualitative methods; questionnaires were
distributed and focus group discussions were conducted among selected
research participants. There were 121 research participants for the
quantitative studies and five groups for the qualitative study. This study
used a purposive sampling method and the participants were selected
according to whether they were willing to support the development of
the pondok and whether they had experience living near a religious
institution. The study revealed that the pondok requires a lot of
improvements, especially in spreading religious knowledge and having
*
Corresponding author: Ateerah Abdul Razak; ateerah@umk.edu.my
©Authors
291
a good relationship with society. It is also suggested that a proper

guideline be provided for every religious learning institution in Kelantan
so that it becomes more systematic. It was revealed that the potential of
the pondok is to ensure that society gains benefits from these religious
institutions. The significance of the current study lies in its analysis of the
quality of teaching and learning in religious institutions so that they meet
international educational qualification standards.
Keywords: knowledge; learning institution; lifelong learning; perception;

society
1. Introduction
Since the 19th century, Kuala Krai, Kelantan, Malaysia, has been home to religious
learning institutions known as pondok. Pondok is derived from the Arabic term for
a place where people can study and reside. The pondok was created in order to
disseminate Islamic knowledge and cultivate moral character. It has a specific
structure and is known for the method of circle learning [halaqah]. The students sit
in a circle as they listen to a lecture (Masyhurah et al., 2015).
Islamic theology or belief system [tauhid], Islamic law [fiqh], and sufism [tasawwuf]
comprise the majority of the curriculum and syllabus in the pondok. Initially,
learners are introduced to Malay manuscripts, and once they have a solid grasp
of the material, they will refer to Arabic manuscripts throughout the learning
process. At the conclusion of learning at the pondok, it is anticipated that students
will acquire moral values and become Muslim scholars who can contribute to
society.
The Kelantan Islamic Religious Affairs Department (JAHEAIK) currently

oversees the development of the pondok, which is well-known, particularly in
Kelantan. However, there are an issues with the pondok development, and some
of the pondok organisers do not adhere to the standard operating procedure (SOP)
for the pondok establishment. For instance, some pondoks employ unqualified
teachers and principals to instruct students. In addition, the diverse orientations
of thought among teachers in a some pondoks result in learners with discordant
religious beliefs.
2.1 Previous Research Pertaining to Pondok Institution Difficulties
Previous analysis of the difficulties of pondok in Malaysia include the curriculum
development, administration, and qualification of pondok learning (Jaafar et al.,
2017). The majority of religious learning institutions rely on donors [waqaf] to
sustain their operations (Fazial and Bahari, 2018). Consequently, pondok
organisations face challenges of insufficient funding and expertise, as well as the
issue of old buildings. This impedes the growth and long-term viability of pondok
management (Muhammad et al., 2021).
Prior studies indicate that the religious institution has encountered issues such as
inadequate funding allocation (Ramli and Bakar, 2013; Hashim et al., 2011). Since
the pondok system has existed since the 19th century — pondok in Kelantan and
Terengganu originated in Pattani, whereas pondok in Kedah originated in Kedah.
292
Thus, some buildings are old and deteriorating, which impedes the effective
implementation of pondok management (Mydin et al., 2014). Inadequate expertise
and personnel, undefined maintenance goals, and inconsistent policy also prevent
the pondok institution from achieving its vision and mission for the development
of religious institutions (Sylvia et al., 2019). According to Ramli and Bakar (2013),
the pondok institution has limited space for educational activities.
The problems that pondok religious learning institutions have been experiencing
have led to other issues, effecting the learning of students. Unquestionably,
improvements have been made to some pondok institutions in order to enhance
their facilities and personnel. To ensure that religious institutions have a
significant impact on the society and the nation, however, it is necessary to
emphasise the current issues that create a negative impression among society and
to implement more rigorous reforms.
2.2 The Concept of Lifelong Learning in Religious Institutions and Its Influence
on Society
The pondok brought traditional thought to Malaysia. It defends the older
generation’s traditional practice. For instance, at the pondok, tutors refer to the
ideas of al Syafie (one of the four great scholars of Sunni schools of thought). The
new reformation of thought derives from the salaf, who advocate strict adherence
to their comprehension of Islamic practices as enjoined by the Prophet
Muhammad and subsequently practiced by the early pious predecessors, known
as the salaf al-salih (Mohamed, 2019). In addition, they focus on eradicating any
additional religious practice that contradicts the Quran and Sunnah. These ideas
inform the educational philosophy of all the religious institutions, even though
each organisation has its own methodology and philosophy, which influence the
formulation of religious practices among students.
The numerous schools of thought can be harmonised if students and instructors

can tolerate divergent viewpoints. However, if a student is overly committed to a
single school of thought, it will negatively impact both the students and the
surrounding community. This is due to the fact that the concept of perpetual
learning is continuous learning for human development, so it must be applicable
and relevant to others (Hasan, 2017). In addition, Islam promotes the concept of
lifelong learning, which can be advantageous for self-preservation, families, and
society (Huda, 2019).
2.3 Theoretical Framework

Theoretically, al-Ghazali (a prominent Muslim scholar) divided the curriculum
learning structure into the knowledge of sharia (religion) and aqliyah
(reason/intellectual). It incorporates both the theoretical and practical concepts to
which all Muslims should adhere. In addition, this element is necessary to ensure
that an individual who acquires knowledge can implement the fardhu ain
(required of every Muslim) and fardhu kifayah (required of the Muslim
community) (Abdul Karim et al., 2021). This framework is crucial because it
depicts a Muslim who derives benefit from his education and contributes to
society. Similarly, the true purpose of education in pondok is to develop an
individual who can benefit others by setting a positive example and assisting the
disadvantaged. It has been demonstrated that the classification of this theory can
293
comprehensively address current social conditions. It can help individuals

comprehend the learning process from an epistemological standpoint (Zaini,
2017).
The framework was utilised in the present study to comprehend how learners in
religious institutions implement their knowledge specifically for the benefit of
others. The purpose of this study has been to determine how society views both
the students and the pondok institution’s administration. This study also sought to
ascertain the function and efficiency of contemporary pondok institutions. It is
anticipated that this study will assist the pondok organisation in becoming more
effective and in sharing its benefits with others.
3. Methodology
This study employed a mixed method approach. A quantitative survey was
conducted with a select group of research subjects. The instruments were created
by the researchers, and the Cronbach alpha value was used to assess their
reliability and validity. It was also used to evaluate the dependability of social
science research (Bonett and Wright, 2014). For the qualitative techniques, a focus
group discussion was organised to determine the learners’ and pondok
management’s perceptions. This technique aided the researchers in involving
participants in data collection discussions (Daheri et al., 2023).
3.1 Sampling Method

This research employed a method of purposive sampling. This was to ensure that
the data represented a wide geographic distribution, to garner rich data, and a
focus that closely matched the synthesis objective (Ames et al., 2019). There were
multiple selection criteria for participants. First, the participants had to be
individuals with an adequate educational background. Second, they had to be
among those who were willing to contribute to the expansion of pondok
organisations. Third, their opinions regarding particular religious strands could
not be extreme. Fourth, the participants had to be willing to participate in this
study voluntarily, and, fifth, they had to reside close to the religious institution
district in Kelantan, Malaysia.
3.2 Data Collection

A total of 121 research subjects participated in the quantitative investigation. They
were given the questionnaire, which required approximately 15 minutes to
complete.
Participating in the focus group discussion (FGD) were approximately 15

individuals who were divided into five groups. After obtaining official approval
from the ethical committee, the FGD began with the development of protocols and
consent forms. The interviews took approximately two weeks to complete, and
the participants were given ample time to engage in an in-depth discussion of the
topic, facilitated by the researchers who were also trained moderators. They were
situated in a secure and comfortable interview room and the conversations were
recorded (Van Eeuwijk & Angehrn, 2017).
294
During the data collection, the researchers analysed participants’ attitudes,

perceptions, knowledge, and experiences in the context of interactions with
various individuals.
3.3 Data Analysis

The quantitative study employed a questionnaire, which was analysed using SPSS
version 23.0 and a significance level of p<0.05. In the current analysis, descriptive
and inferential statistics were employed (Sim-sim et al., 2020). For the qualitative
investigation, the conversation of the focus group was videotaped. The
researchers then rendered the Malay transcription into English. The data were
analysed thematically by assigning preliminary labels to the themes in the text
(Abu Hassan Shaari & Waller, 2022).
After identifying common themes, the perceptions of students in pondok were

coded and re-coded to ascertain common themes and sub-themes. The verification
of members was then conducted via Google Meet. Participants were emailed the
results, preliminary conclusions, and video recording, but no additional feedback
or adjustments were requested.
3.4 Triangulation Technique

The researchers have utilised researcher triangulation for the triangulation
technique. For the current study, the research team was divided into four groups:
observation, interviewing, analysis, and report writing. This was in order to
reduce any biases that originated from gathering, reporting, and analysing the
data, thereby enhancing the reliability and validity of the research (Donkoh et al.,
2023).
3.5 Ethical Principles

This study was authorised by the university’s institutional review board.
4. Findings
This section discusses the quantitative and qualitative research findings. Table 1
of the quantitative analysis depicts the participants’ educational background.
Table 1: Demographic profile of participants
Valid Cumulative
Level of Education Frequency Percent
Percent Percent
Primary 3 2.5 2.5 2.5
Secondary 10 8.3 8.3 10.7
Diploma 18 14.9 14.9 25.6

Valid
Bachelor’s degree 87 71.9 71.9 97.5
Master’s degree 3 2.5 2.5 100.0
Total 121 100.0 100.0
295
Table 1 reveals that the majority of the research participants hold a bachelor’s
degree, while only about 3% are elementary school graduates. The table depicts
the variation of the participants, with more than 50% of the research participants
holding a valid educational certificate.
Table 2: The value of mean and level of agreement among the participants
Strongly Strongly Std.

Items Disagree Agree Mean
disagree Agree Deviation
Item 1. Pondok is an important 5.0 9.1 35.5 50.4 4.41 1.046

education institution in Kelantan.
Item 2. Pondok has significantly affected 7.4 5.8 41.3 45.5 4.12 1.163
the community’s way of thinking.
Item 3. Pondok provides the community 23.1 41.3 19.0 16.5 3.36 1.443
with an opportunity to be more open-
minded.
Item 4. Pondok helps the community to 28.1 57.9 8.3 5.8 4.21 1.233
understand Islam more clearly.
Item 5. 39.7 36.4 14.0 9.9 3.82 1.354

Pondok offers the most accurate Islamic
doctrine.
Item 6. The institution of pondok 10.7 6.6 26.4 56.2 1.89 1.340
influences the community to have an
extreme point of view.
Item 7. Only the pondok institution 25.6 28.9 17.4 28.1 3.18 1.533
defends and fights for the true Ahli
Sunnah Wal Jama’ah movement.
Item 8. The institution of pondok 9.9 40.5 3.3 46.3 4.10 1.221
provides a community which truly
practices religion.
Item 9. Pondok promotes the dhikr 12.4 43.0 11.6 33.1 3.73 1.360
practice which is in line with the
sunnah.
Item 10. Pondok has strong social ties 9.9 46.3 35.5 8.3 4.00 1.304
with the local community.
Item 11. Pondok emphasises the 20.7 40.5 24.8 14.0 3.35 1.504
cleanliness aspect and manages natural
resources well.
Item 12. Pondok resolves many issues in 22.3 38.0 12.4 27.3 3.16 1.576
society.
Item 13. Pondok becomes the 14.9 38.8 11.6 34.7 3.67 1.434
community’s primary resource for
religious matters.
Item 14. Pondok provides a variety of 22.3 39.7 13.2 24.8 3.31 1.528
distinctive religious practices for
everyday life.
Item 15. Pondok serves as a unifying 42.1 36.4 8.3 13.2 3.80 1.364
institution for Muslims.
296
Item 16. Education in pondok enhances 47.9 35.5 10.7 5.8 4.04 1.300
the community’s comprehension of al-
Quran and al-Sunnah.
Item 17. Pondok is able to tolerate 31.4 34.7 21.5 12.4 2.75 1.507
different Islamic schools of thought.
Item 1 reveals that more than half of the research participants concurred that
religious institutions are vital organisations in the state of Kelantan. For item 2,
approximately half of the participants concurred that pondok have a significant
impact on the way people think, while 14% disagreed. For item 3, more than half
of research subjects disagreed that pondok offers the opportunity to be an open-
minded member of society. In contrast, item 4 reveals that 86% of the participants
disagreed that pondok helps them comprehend religion more clearly. Regarding
item 5, approximately 76% of participants disagreed that pondok provides them
with an accurate comprehension. Item 6 reveals that approximately 83% of
participants concurred that pondok contributes to extreme thought in society. In
response to question 7, more than half of the participants indicated that it is not
only pondok which defends a sound Islamic worldview.
Item 8 demonstrates that approximately half of the participants did not believe
pondok is capable of practising sound religious law. For item 9, approximately
55.4% of the participants disagreed that pondok learners can practise zikr
(remembrance of God) in accordance with the prophet’s teaching. Item 10
indicates that more than half of participants concurred that pondok learners did
not create positive social relationships with society, while item 9 indicates that
pondok learners did create positive social relationships with society. This study
reveals that 61.2% of the participants cited the inability of religious learning
institutions to maintain a clean and wholesome environment in terms of
cleanliness.
Regarding item 12, approximately 60% of the participants indicated that religious
institutions in Kelantan are incapable of resolving social problems that occur
around them. Item 13 demonstrates that 53.7% of participants believe that pondok
institutions are incapable of serving as the society’s primary resource for resolving
religious issues. Regarding point 14, approximately 62% disagree that pondok
offers a particular religious practice in social life. Item 15 demonstrates that 78.5%
of the participants believe that the pondok cannot serve as institutions that unite
the Muslim community, whereas the remaining 21.5% participants said that they
can. Item 16 demonstrates that approximately 83% of the participants disagree
that the pondok helps society better comprehend the Quran and Sunnah.
Approximately 66% of the participants disagreed with the statement that the
pondok institutions celebrate differing opinions from their own.
297
Table 3: The value of Cronbach alpha for each item

Reliability
Case Processing Summary
Statistics
Cronbach’s N of
N % Alpha Items
Cases Valid 121 100.0 .921 17
Excludeda 0 .0
Total 121 100.0
Table 3 displays the Cronbach alpha value for each item. A Cronbach’s alpha
value between 0.6 and 0.8 is acceptable (Said, 2017). The items utilised in this
investigation are reliable and valid.
There were five groups of participants who participated in focus group

discussions for the qualitative results. The research participants were named
informants, followed by sequential numbers, for example, Informant 1, Informant
2, etc.
Table 4 presents the themes that emerged from the interview data regarding
efforts to enhance the quality of learning and teaching in pondok organisations.
Multiple themes emerged from their discussions and responses to the semi-
structured question “What components of the pondok learning system should be
improved?”
Table 4: List of themes, sub-themes, and related quotes from the participants
Themes Sub-theme Representative quotes
A further concern that was highlighted is that students
in religious institutions should be engaged in
Active community service. They could organise social
participation activities between local and pondok residents. This can
Concern with
between foster a harmonious relationship and members of the
society
learners and community can also pose questions and share their
society concerns. The participants are able to contribute to the
community. Each group may feel a sense of belonging
as a result of this activity.
The learning management of pondoks must be more
Teaching and New reform systematic. The development of learning qualifications
learning for the quality and soft skills should be fostered in both students and
quality of teaching teachers. It is essential to guarantee that teachers
enhancement and learning empower students with knowledge, particularly in the
field of Islamic foundations.
Not all pondok institutions adhere precisely to the
Being teachings left by the Prophet SAW and his early
receptive to companions. In addition, there are numerous
Honouring and misunderstandings that confound Muslims, and the
distinctions appreciative attitude of pondok members does not represent
of diverse authentic Islam. Islam encompasses various schools of
points of view thought. Learners of pondok should recognise the
opposing school of thought and cease being obsessive.
298
Consideration must be given to safety control in pondok

The pondok institutions. To create a healthy environment for the
Improvement institution’s learning and teaching process, it is also essential to
of facilities enhance the pondok infrastructure and facilities. Some
infrastructure should be pondok institutions with sufficient funding could
upgraded. maintain a solid infrastructure for their users. The
government should participate in this matter.
The existence of religious learning institutions should
Transformati be preserved because they are a defining characteristic
on of of the Islamic traditional education system in
religious archipelagic nations. However, pondoks should be
Collaboration
institutions’ enhanced in terms of qualifications, which should be
with
learning comparable to other Malaysian educational certificates,
government
systems into a such as the secondary religious high certificate
educational
venue for (STAM), secondary religious high certificate (STPM),
system
higher and other certificates in public universities. Aside from
education that, it is desirable for the government to collaborate
institutions. with other local and international universities on
religious education.
Table 4 reveals that five themes emerged from the participants’ discussions. First,
the pondok students must care about society and, second, teaching and learning
quality must be enhanced. Third, students must respect those with opposing
viewpoints. Fourth, the infrastructure and facilities of pondok institutions should
be improved in order to create a more comfortable environment. Fifth, it is
essential to collaborate with the government so that the value of the pondok
educational system is more apparent.
4.1 Concern with Society

Based on the interviews, the participants indicated that pondok learners should be
aware of everything occurring around them and develop an appropriate activity
in which pondok residents and society members participate.
“Residents of pondok appear to exist in their own realm. They would
rather not interact with society. Sometimes we are hesitant to approach
them, but we are more inclined to participate in their activities. This can
strengthen our bonds, and we can also learn more from them.”
(Informant 2).
Based on the response, it is evident that community activities should be organised

to teach pondok students to care more about other societies. This could also help
them comprehend different perspectives on people, rather than simply
befriending pondok learners.
4.2 Teaching and Learning Quality Enhancement

According to Informant 1, the technique of teaching and learning can affect the
quality of the learning process, and learners and educators should develop
additional skills.
“Teachers should also receive training in the application of the essential skill.
This could enable educators to acquire knowledge beyond the scope of
textbooks. This procedure will produce students who are more receptive to new
ideas and better prepared to contribute to society in the future. Occasionally,
they are hesitant to use modern technologies as a teaching and learning
assistance, preferring to rely on outdated methods. This will prevent them
299
from achieving more impactful learning objectives, given that everything is at

our fingertips. If we do not adopt the most recent technological advancements,
we will fall far behind.” (Informant 1).
According to the informant, pondok institutions should enhance their teaching and
learning. The implementation of modern technology will encourage outsiders to
learn about religious issues, and if they use social media effectively, they can share
their knowledge with those who were unable to attend the pondok organisation’s
learning programme.
4.3 Honouring Distinctions

The third theme was that learners should be aware that the information they
acquire that may conflict with other information. This emerged from the interview
with Informant 4.
“As we are aware, Islam contains a great deal of academic thought, so it is
normal for our knowledge to differ from that of our friend. In the case of pondok
learners, however, they tend to form negative opinions of those who disregard
what they have learned. This will increase societal confusion and tarnish the
reputation of the pondok organisation, as many conflicts between pondok
students are revealed on TikTok and other social media platforms.”
(Informant 4).
The interviews highlighted that different schools of thought should not be a major
source of conflict so long as all people can tolerate and respect the views of others.
The students should also be aware that every action sets an example for others.
Due to the rapid spread of global ideas and beliefs made possible by modern
technology, it is essential to acknowledge differences with a broader mindset.
4.4 Improvement of Infrastructure

The fourth theme revealed the importance of having a good environment. This
also included the issue of cleanliness and good infrastructure as discussed by
Informant 3:
“Garbage strewn along the roadside is a common sight in Kelantan. Similar
situation when we discovered that pondok residents could not set a positive
example for maintaining hygiene. It is not consistent [with Islamic] doctrine.
In addition, if the government can enhance the infrastructure, particularly the
drainage system, a more organised and clean environment will develop around
the pondok area.” (Informant 3).
The interviews disclosed the issue of developing a proper and systematic

infrastructure specifically for pondok residents. It should also be noted that
cleanliness-related self-awareness is most effective when employed prior to more
structured physical development planning. Thus, awareness among pondok
residents must come first in order to create a better, cleaner, and healthier
environment.
4.5 Collaboration with Government Educational System

Informants 5 and 3 provided a constructive response to the question of
collaborating with government institutions to expand the quality of education in
pondok.
300
“Pondok is also a site where parents send their children who are involved in
drug abuse, so some parties view the pondok institution in Kelantan
negatively if there is a problem. Therefore, the collaboration with the
government will reduce the problem of selecting students to attend pondok.
The government can also provide rehabilitation and special education to
students who are affected by the problem.” (Informant 5).
“The Malaysian government has implemented a new educational system. We
must emphasise the importance of a contemporary educational system.
Collaboration with the government is essential for the intended curriculum
structure in pondok education to be advantageous at the university level. In
addition, it can be commercialised for the world’s population.” (Informant 3)
It is evident from the interviews that collaboration between government

institutions is essential for improving the integrity of the pondok educational
system. In addition, if pondoks are cognisant of the changes brought about by
government policy, the pondok educational organisation will achieve improved
educational outcomes.
5. Discussion
Both quantitative and qualitative results are sufficient to demonstrate that the
pondok educational system faces problems. It is proven that the quality of teaching
and learning should be revised so that the learners’ knowledge can be imparted
to society. The results demonstrate that the educational system of pondoks ought
to be more adaptable and faster to respond to the socioeconomic challenges of the
present. (Chirimbu & Ionescu, 2017).
The results also indicate that the participants are capable of evaluating the impact
of the pondok system, which appears to require improvement. The societal impact
of a decent education should be realised. If there is no positive effect on other
individuals, the syllabus system must be revised. This can aid the educational
system in identifying difficulties, low levels of engagement, and insufficient
support from certain parties (Gaikwad et al., 2023). It is well known that the pondok
is an established educational institution, particularly in the archipelago. Thus,
organisations must be cognisant of social needs, necessitating adequate guidance
in religious matters in particular. The learners should also be receptive to criticism
from others, as this is the best way for them to improve their curriculum, learning,
and teaching methods to ensure that their education is the most valuable in society
and that they are well equipped for the future (Goudard et al., 2020).
Any extreme attitude should be avoided, particularly among students, and they
should be able to cope with controversial issues and diverse points of view.
Teachers may employ a distinct teaching style when addressing issues. They may
introduce students to democratic frameworks and conflict resolution strategies
that will assist them in understanding and practising nonviolent methods for
engaging in discussions and resolving conflicts (Kerr & Huddleston, 2015). In the
current study, it is beneficial for students to recognise other schools of thought so
that any misunderstandings regarding Islam can be resolved amicably.
In addition, students should practise what they have learned. In accordance with
the theoretical framework of this study, which includes both theoretical and
301
practical elements, it has been demonstrated that practical experience has a

positive influence on theoretical knowledge (Leask et al., 2020). Learners of pondok
should apply their knowledge and assist society in gaining a greater
understanding of Islamic law.
For the practise of God’s remembrance (zikr), it is essential that students practise
zikr properly, which will then be emulated by others. Numerous modes of zikr
have been introduced by the self-proclaimed Muslim scholar. However, there are
also improper zikr practices that can result in heretical doctrines.
Various recommended ways to mitigate incorrect religious practices are

suggested based on three key components: knowledge, ethics, and methodology.
It is imperative to approach the understanding of divergence and sects, as well as
the culture of discourse, from a scientific standpoint (Anas & Aminuddin, 2019).
Moreover, pondok students and teachers should recognise that strengthening
social ties with others is necessary. This is consistent with the government’s
implementation of service learning. Maharam et al. (2019) demonstrated that
learning elements that include community service can have an impact on
students, educators, institutions, and society as a whole.
The practice of hygiene has become a serious concern. The level of hygiene in
Kelantan is a problem due to the fact that some of the residents are less concerned
with sanitation, which is also related to their attitude and religion (Nik Yusri &
Mohd Izzat, 2018). If pondok students engage in the same unhygienic behaviour,
they will be judged more harshly than those who do not observe sanitation in their
daily lives. This is because hygiene is a component of religious law; if the adherent
cannot set a good example, then society will not have a good example from those
who receive religious education on a daily basis.
Furthermore, it is suggested that the infrastructure and facilities of the pondok

institution be enhanced. This also involves improving safe and healthy buildings,
optimising learning spaces, and maximising the benefits of pedagogy and the
school-community relationship (Barrett et al., 2019). These elements are
indispensable for fostering a more conducive environment at pondok institutions.
Similarly, it will entice international students to study and discover new cultures
in religious institutions.
In addition, this study suggests that the pondok organisation should improve the
quality of learning and instruction. The process can educate both students and
teachers to become more self-assured, adaptable, and motivated. Improving
learning methods necessitates a focus on content, active learning, and
collaborative support (Confesor & Belmi, 2022). These elements must be
introduced to pondok organisations in order for them to be equal to other
educational institutions. It will also encourage other parties to collaborate with
the pondok educational system so that a hybrid method of learning can be
developed and the learners can experience an effective learning process.
6. Conclusion
Based on the findings, it is apparent that pondok organisations have room for
improvement. The pondok administration must maintain an open mind in order to
302
receive constructive feedback from other parties. This study is important because
it will contribute to their growth and development as educational organisations.
They should minimise the issue of disputes among Muslim scholars due to
divergent viewpoints. A knowledge of Islam should be disseminated to the
broader society, and it is a leader’s responsibility to ensure that the populace is
educated in the correct religious philosophy. This will prevent the deviation
between teaching and practice that has a negative effect on certain educational
systems.
This research is limited by the fact that it only examines a subset of pondok
organisation in Kelantan, Malaysia. It is proposed that further research be
conducted in other states, particularly in the archipelagic regions, such as
Indonesia and southern Thailand. This will aid the pondok institution in
identifying issues that may be corrected in order to achieve a high level of
educational impact.
In order for pondok institutions to attract others to follow a positive example in

their daily lives, it is essential that the correct practices be implemented. In
addition, a quality product of education is closely related to societal benefits. This
is because the purpose of education is to shape individuals into better persons.
This study suggests that a more comprehensive reformation of pondok institutions
should be planned in order to enhance the quality and visibility of pondok
curricula on an international scale. This will encourage pondok students to interact
with society and learn about other cultures. It will also increase the visibility and
applicability of the pondok educational system globally and improve the
educational values of pondok students.
7. Acknowledgement
This study was funded by R/JAIK/A0400/00501A/003/2021/00945),
Department of Islamic Religious Affairs in Kelantan (JAHEAIK) and PEMIKIR,
University of Malaysia Kelantan research grant. The researchers would like to
thank the department for the financial support it has provided.
Contribution of Authors: All authors made substantial contributions to the
study’s conception and initial composition, data analysis and interpretation, and
manuscript review and finalisation. All authors assume public responsibility for
the manuscript’s contents.
8. References
Abdul Karim, S. R., Ismail, H., & Mohd Burhan, N. (2021). Al-ghazali’s education theory
towards islamic lifelong learning. Jurnal Sultan Alauddin Sulaiman Shah, 8(1), 69–
83. https://jsass.kuis.edu.my/index.php/jsass/issue/view/13
Abu Hassan Shaari, A., & Waller, B. (2022). Self-help group experiences among members
recovering from substance use disorder in Kuantan, Malaysia. Social Work With
Groups, 46(1), 51–67. https://doi.org/10.1080/01609513.2022.2057393
Ames, H., Glenton, C. & Lewin, S. (2019). Purposive sampling in a qualitative evidence
synthesis: A worked example from a synthesis on parental perceptions of
vaccination communication. BMC Medical Research Methodology, 19(26), 1–9.
https://doi.org/10.1186/s12874-019-0665-4
303
Anas, M. Y., & Aminuddin, R. (2019). Polemik amalan ibadah harian di malaysia [Polemic on
daily worship practise in Malaysia]. E-Journal of Islamic Thought and Understanding,
2, 38–63. https://ir.uitm.edu.my/id/eprint/26716
Barrett, P., Treves, A., Shmis, T., Ambasz, D., & Ustinova, M. (2019). The impact of school
infrastructure on learning: A synthesis of the evidence. International Bank for
Reconstruction and Development/The World Bank.
https://doi.org/10.1596/978-1-4648-1378-8
Bonett, D. G., & Wright, T. A. (2014). Cronbach’s alpha reliability: Interval estimation,
hypothesis testing, and sample size planning. Journal of Organizational Behavior,
36(1). https://doi.org/10.1002/job.1960
Chirimbu, C., & Ionescu, E. (2017). Education in the contemporary economy and society.
Risks and challenges. In S. Hugues, & N. Cristache (eds.), Risk in contemporary
economy. 18th edition of the conference Risk in contemporary economy, RCE2017, Galati,
Romania. https://doi.org/10.18662/lumproc.rce2017.1.1
Confesor, R. F., & Belmi, R. M. (2022). Structure, activities and teacher development in the
philippine science teachers’ community of practice. International Journal of
Learning, Teaching and Educational Research, 21(6), 71–89.
Daheri, M., Warsah, I., Morganna, R., Putri, O. A., & Adelia, P. (2023). Strengthening
religious moderation: Learning from the harmony of multireligious people in
Indonesia. Journal of Population and Social Studies, 31, 571–586.
http://doi.org/10.25133/JPSSv312023.032
Donkoh, S., & Mensah, J. (2023). Application of triangulation in qualitative research.
Journal of Applied Biotechnology and Bioengineering, 10(1), 6–9.
http://doi.org/10.15406/jabb.2023.10.003
Fazial, F., & Bahari, Z. (2018). Problems faced by the pondok institutions in funding
development. International Journal of Academic Research in Business and Social
Sciences, 8(6), 1313–1320. http://doi.org/10.6007/IJARBSS/v8-i6/4536
Gaikwad, S., Wadegaonkar, A., Mitra, G., & Chakravarty, D. (2023). Assessment literacy,
current assessment practices and future training: Reflections of teachers in Higher
Education. International Journal of Learning, Teaching and Educational Research, 22(7),
1–29. https://doi.org/10.26803/ijlter.22.7.1
Goudard, P., Pont, B., & Pinhsuan Huang, S.H. (2020). Curriculum reform: A literature
review to support effective implementation. Working Paper No. 239. Organisation
for Economic Co-operation and Development. https://doi.org/10.1787/19939019
Hasan, M.M. (2017). The concept of lifelong education in islam. Ar-Raniry, International
Journal of Islamic Studies, 4(2), 257-272. https://doi.org/10.20859/JAR.V4I2.139
Hashim, R., Rufai, S. A., & Mohd Nor, M. R. (2011). Traditional islamic education in asia
and africa: A comparative study of malaysia’s pondok, indonesia’s pesantren and
nigeria’s traditional madrasah. World Journal of Islamic History and Civilization, 1(2),
94–107. http://idosi.org/wjihc/wjihc1(2)11/3.pdf
Huda, M. (2019). Lifelong education in islamic perspective. International Journal of
Nusantara Islam, 7(1), 40-48. https://doi.org/10.15575/ijni.v7i1.4006
Jaafar, N., Abd Ghani, Z., Tibek, S. R., Mohd Nor, A. H, Gunardi, S., Ismail, N. A., Darmi,
R., Mat Saad, N. S., Idrus, M. M., Abdullah, H., Ishak, M., Ismail, H., & Sulaiman,
A. (2017). Cabaran institusi pendidikan pondok serta impak terhadap
pembentukan diri pelajar [Challenges of Pondok education institutions and the
effect on the self-development of students]. Journal of Islamic, Social, Economics and
Development, 2(6), 223-235. http://www.jised.com/PDF/JISED-2017-06-12-17.pdf
Kerr, D., & Huddleston, T. (Eds.). (2015). Teaching controversial issues. Council of europe.
https://doi.org/10.4135/9781473910195.n14
Leask, R., Nee Cronje, B. T., Holm, D. E., & Ryneveld, L. V. (2020). The impact of practical
experience on theoretical knowledge at different cognitive levels. Journal of the
304
South African Veterinary Association, 9(1), 1–7.

https://doi.org/10.4102/jsava.v91i0.2042
Maharam, M., Najah Nadiah, A., Zainab, I., & Hamdi, I. (2019). Service-learning in
malaysia: Practice and implementation in four public universties. International
Journal of Civil Engineering and Technology, 10(3), 1632–1639.
http://iaeme.com/Home/issue/IJCIET?Volume=10&Issue=3
Masyhurah, M. R., Harun, B., Maimun, A. L., & Siti Aisyah, R. (2015). Institusi pondok
dalam sistem pendidikan islam di malaysia. Proceeding the 7th international
workshop and conference of asean studies on islamic and arabic education and civilization
(poltan-ukm-polimed). Fakulti Pendidikan Universiti Kebangsaan Malaysia.
https://www.researchgate.net/publication/301788262_INSTITUSI_PONDOK_
DALAM_SISTEM_PENDIDIKAN_ISLAM_DI_MALAYSIA
Mohamed, A. (2019). Understanding salafis, salafism and modern salafism. Islamiyyat
41(1), 125–136. https://doi.org/10.17576/islamiyyat-2019-4001-15
Muhammad, S. H., Yuhainis, A. T., & Nor Aini, S. (2021). The challenges of facilities
management for pondok institution in Malaysia. Virtual go green: Conference and
publication (v-gogreen 2021). Research, Industrial Linkages, Community & Alumni
Network.
https://www.researchgate.net/publication/353846903_Facilities_Management_
Practices_in_Malaysia_A_Literature_Review
Mydin, M. A & Sufiza, N. (2014). The system of islamic studies at madrasah [sekolah
pondok]. Journal of Mechanical Manufacturing, 1.
Nik Yusri, M., & Mohamad Izzat, M. G. (2018). Kajian tingkah laku pengguna tandas masjid
di negeri kelantan [A study of the behaviours of mosque toilet users in kelantan].
Malim Jurnal Pengajian Umum Asia Tenggara (SEA Journal Of General Studies), 19, 1–
11. https://doi.org/10.17576/malim-2018-1901-01
Ramli, M. I., & Abu Bakar, M.Z. (2013). Kearifan tempatan dalam institusi sekolah pondok:
Malaysia research case [Local wisdom in pondok educational institutions: a
Malaysian study]. Paperwork presented at Persidangan Serantau Kearifan Tempatan
2013. Terengganu, Malaysia. http://dx.doi.org/10.6007/IJARBSS/v12-i1/12108
Sim-sim, M., Bule, M., Bico, I., Vang, V., Phomvilay, P., Maypaokha, G., & Khamlun, D.
(2020). Measuring attitudes towards motorcycle helmet use in Laos. Journal of
Population and Social Studies, 28(2), 143–155.
http://doi.org/10.25133/JPSSv28n2.010
Sylvia, G. M., Sarajul, F. M., & Saidin, M. M. (2019). Current issues and barriers of
maintenance management practices for public facilities in malaysia. International
Journal of Engineering and Advance Technology, 8(5C), 119–125.
http://doi.org/10.35940/ijeat.E1017.0585C19
Said, T. (2017). Statistical analysis: Internal-consistency reliability and construct validity.
International Journal of Quantitative and Qualitative Research Methods, 6(1), 46–57.
https://api.semanticscholar.org/CorpusID:212534910
Van Eeuwijk, P., & Angehrn, Z. (2017). How to conduct a focus group discussion (FGD).
Methodological Manual. University of Basel.
https://www.researchgate.net/publication/324075132_How_to_Conduct_a_Fo
cus_Group_Discussion_FGD_Methodological_Manual
Zaini, A. (2017). Pemikiran tasawuf imam al-ghazali. Esoterik, 2(1), 146–159.
https://doi.org/10.21043/esoterik.v2i1.1902
305
Appendix 1
Questionnaire on the impact of religious learning institutions

Items
1. Pondok is an important education institution in Kelantan/ Pondok merupakan

institusi pendidikan agama yang penting di Kelantan.
2. Pondok has significantly affected the community’s way of thinking/Pondok

memberi kesan yang besar kepada cara berfikir masyarakat.
3. Pondok provides the community with an opportunity to be more open-

minded/Pondok membuka peluang kepada masayarakat untuk berfikiran lebih terbuka.
4. Pondok helps the community to understand Islam more clearly/Pondok membantu

masyarakat untuk memahami Islam dengan lebih jelas.
5. Pondok offers the most accurate Islamic doctrine/Pondok menawarkan pemikiran

Islam yang paling tepat.
6. The institution of Pondok influences the community to have an extreme point of

view/Pondok mempengaruhi masyarakat untuk berpegang dengan aliran yang ekstrem.
7. Only the Pondok institution defends and fights for the true Ahli Sunnah Wal
Jama’ah movement/Hanya pondok yang mempertahankan dan memperjuangkan aliran
Ahli Sunnah Wal Jama’ah yang tulen.
8. The institution of Pondok provides a community which truly practices religion

/Pondok melahirkan masyarakat yang mengamalkan ajaran agama.
9. Pondok promotes the dhikr practice which is in line with the sunnah/Pondok
menganjurkan amalan zikir yang bertepatan dengan sunnah.
10. Pondok has strong social ties with the local community/Pondok mempunyai
hubungan sosial yang baik dengan masyarakat.
11. Pondok emphasises the aspect of cleanliness and manages natural resources
well/Pondok mementingkan aspek kebersihan dan pengurusan sumber alam yang baik.
12. Pondok resolves many issues in society/Pondok menyelesaikan banyak

permasalahan masyarakat.
13. Pondok becomes the community’s primary resource for religious matters/
Pondok menjadi tempat rujukan utama masyarakat dalam isu-isu agama.
14. Pondok provides a variety of distinctive religious practices for everyday

life/Pondok menawarkan amalan ibadat khusus tertentu dalam kehidupan masyarakat.
15. Pondok serves as a unifying institution for Muslims/Pondok berperanan sebagai

sebuah institusi yang menyatukan umat Islam.
16. Education in Pondok enhances the community’s comprehension of al-Quran and

al-Sunnah/Pendidikan pondok membantu masyarakat untuk lebih memahami al-Quran
dan al-Sunnah.
17. Pondok is able to tolerate Islamic schools of thought that are different from
them/ Pondok boleh menerima aliran pemikiran Islam yang berbeza dengan mereka.
306

Vol. 22, No. 10, pp. 306-323, October 2023
A Comprehensive Approach to Eliminate

English Second Language Learners’ Grammatical
Difficulties
Nomasomi Hilda Matiso
Walter Sisulu University
Faculty of Educational Sciences, Mthatha, South Africa
Abstract. In order to produce academically competent learners, a

comprehensive approach addressing English Second Language (ESL)
learners’ grammatical difficulties needs to be developed. Research has
shown that learners’ essay writing skills are spoiled by incompetent
writing abilities, in particular, grammatical difficulties. This enquiry,
whose main objective was to unravel grammatical difficulties
experienced by ESL learners and to propose a comprehensive approach
addressing these, was guided by a theoretical framework that emphasises
cognitive, socio-cultural, and communicative perspectives. An
interpretive paradigm, embedded in a qualitative approach, was used to
gather data from 16 conveniently nominated participants. A case study
design was deemed suitable for this enquiry because of its ability to focus
on a specified group of participants. The data were generated through
semi-structured interviews and thematically analysed. The findings were
that error analysis, followed by explicit instruction of language structures;
learner engagement in corrective feedback; multiple exposure to and
practice of a language structure; and immersion into rich target language
environments could be beneficial. The learners would then take
responsibility for their own learning. It is recommended that professional
development for language educators is essential to employ these
approaches to combat the widespread incompetence in learners’ writing
abilities, which negatively affect their academic performance.
Keywords: academic performance; comprehensive approach; corrective

feedback; English second language; grammatical difficulties
1. Introduction
This paper explores comprehensive approaches that can be used to eliminate
English second language (ESL) learners’ grammatical difficulties that negatively
impact their writing skills. Numerous studies have shown that ESL learners are
faced with severe challenges when mastering grammar and grammatical cohesion
(Emvula, 2020; Ndlovu; 2019; Widdowson, 2016). Because writing necessitates the
use of lexicon, syntax, parts of speech, tenses, word order, paragraphing, subject
phrases, supporting phrases, and closing phrases, writing is regarded as one of
©Authors
International License (CC BY-NC-ND 4.0). http://ijlter.org/index.php/ijlter
307
the most difficult skills to master (Alsied et al., 2018). Moreover, in his research on
academic writing of ESL learners, Opara (2016) further asserted that many
learners understand the English language but have difficulty expressing their
ideas accurately in writing.
Many studies indicate that ineffective teaching methods, incorrect use of teaching
philosophies, and the predominance of the mother tongue (Alhaysony, 2017;
Nzerem & Bob, 2021) are the major causes of grammatical difficulties. English
second language learners are more likely to make grammatical and lexical
mistakes when using English (Iipinge, 2018; Manan, 2017). Furthermore,
inadequate instructional materials and ill-prepared English teachers also
contribute to grammatical inefficiencies (Khatter, 2019). Learners’ poor
performance has been attributed primarily to teachers’ weak abilities, lack of high
levels of knowledge and poor teaching skills (Hoadley, 2012; Department of Basic
Education, 2013, 2018; Spaull, 2013). According to Ayliff (2010) and Hassan (2018),
language inadequacy is a problem that affects students’ academic performance.
As a result, their chances of succeeding in higher education is limited.
Grammar is described by (Eunson, 2020) as a system of rules, with exceptions to

those rules, that constructs meaning in a language. Farooq et al. (2012) argued that
because students must build sentences, paragraphs, and ensure that their material
is coherent, ESL learners view grammar as the most difficult part of the writing
process. In alignment with this, Ummah (2018) noted that the capacity to apply
pertinent rules to effectively communicate with an audience is known as
grammar. According to Moloney and Saltmarsh (2016), second language learners
frequently make errors with grammar rules pertaining to parts of speech, tenses,
and inflections.
English first additional language (EFAL) students struggle in several ways to

effectively master English tenses. Widianingsih (2016) asserted that EFAL learners
in Northern Ireland experience problems with tenses. Time is what determines
‘when’ an action occurs, and tense relates to time. It conveys the interval between
the moment of speaking and the time when an action takes place. According to
Cowan (2008), tense has three dimensions: present, past, and future.
Determiners are among the factors contributing to grammatical difficulties.

English second language learners sometimes have difficulty understanding when
and how to use articles in English. Determining whether to use ‘a’ or ‘an’ with
singular countable nouns or knowing when to omit articles in specific contexts
can be confusing. Jones (2016) explains determiners as modifying words that
determine the kind of reference a noun or noun group has, such as the, a, my, this,
some, twenty, each, any, which are used before nouns. Some determiners can also
be used as pronouns (without a noun following).
Shin and Yoo (2019) highlighted the challenges faced by ESL learners trying to
acquire English as a foreign language when employing determiners. The learners’
ability to refer to a fact, an object, a concept, or a person who has already been
introduced in the discourse, and to introduce a new one, is one of their major
©Authors
308
challenges (Russia, 2018). The following sentence illustrates how ESL learners
might use an incorrect determiner:
Yesterday I saw the man driving a car
In the sentence above, the noun ‘man’ has not been mentioned previously by the
speaker. Therefore, the referent is not known by the addressed. If the referent is
not known, an appropriate article to use here is ‘a’, while in cases where the
referent has been introduced earlier by the speaker, the article ‘the’ would be more
appropriate.
Worldwide, grammatical difficulties pose challenges to ESL and English foreign

language learners, particularly because of the differences between the
grammatical constructions of the mother-tongue languages and that of the
English language. In a study by Shiu (2011) in Toronto, exploring EFAL learners’
judgments of grammatical challenges in connection to ESL performance,
knowledge, and proficiency, it was discovered that learners’ perceptions of
grammatical difficulty vary according to their implicit/explicit knowledge of the
features in question.
Leki (2017) purported that grammar is more than just a series of rules; it is a
dynamic language structure that is challenging to use. Ndlovu (2019) argued that
learners in KwaZulu-Natal (KZN), South Africa, face issues every day as they
struggle to use the language structures and norms correctly that influence their
writing.
Exposing students to a lot of reading, both within and outside of the classroom, is
the best approach to teach grammar and expand their vocabulary, according to
the South African Curriculum and Assessment Policy Statement (CAPS) report.
In addition, both grammar and lexicon must be taught through the use of texts
(DBE, 2018). Given what is in the learners’ texts, it is evident that learners ignore
grammar rules (Ndlovu, 2019).
Grammar knowledge is also important for ESL learners to establish error-free

phrases in essay writing, but grammar correction and grammar feedback do not
lead to progress in writing for four specific reasons, as stated by Sjolie (2016). First,
grammar correction merely addresses grammar superficially, not the way it is
used. Second, the production of the acquisition of language and grammar appears
hierarchically, which means that offering learners grammar correction that is well
beyond their level is ineffective. Third, students may misinterpret the grammar
statements of teachers, while teachers may misinterpret the grammar of learners
too. Fourth, learners do not include the corrections they do not understand when
they execute corrections (Sjolie, 2016).
English second language learners may encounter various grammatical difficulties

in their language journey. Maintaining agreement between a subject and a verb is
a common issue. Students might forget to match the verb with a singular or plural
subject, leading to sentences that sound awkward or incorrect (Widianingsih,
2016).
©Authors
309
In English, the word order is relatively fixed in standard sentence structures

(subject-verb object). English second language learners may find it challenging to
rearrange words in sentences correctly (Saddler, 2012), particularly if their native
language follows a different word order.
Properly using prepositions (in, on, at, etc.) to express location, or other
relationships can be problematic for ESL students. Prepositions often do not
directly translate between languages, causing errors in usage (Widdowson, 1990).
Using pronouns correctly and consistently can be a struggle. English second
language learners may have trouble distinguishing between subject pronouns (I,
you, he, she, etc.) and object pronouns (me, you, him, her) (Matiso and Tyantsi,
2023).
Modal verbs such as can, could, should, may, might and so forth, have various
functions and meanings in English. English second language students might have
trouble understanding when and how to use them appropriately.
Studies have been conducted on the rectification of errors made by learners. Most
of these studies focus on the teacher as an identifier of errors, and the learners as
executors of these corrections. A gap identified by this paper is the role played by
both teachers and learners in eliminating grammatical errors. This gap could
originate from the teachers’ failure to provide effective feedback to learners,
which will enable them (learners) to provide efficient and meaningful corrections.
In this paper, the author has argued that learners should take responsibility of
their own learning by taking initiatives aimed at reducing the occurrence of
grammatical errors, which derail the efforts intended to produce error-free essays.
In this regard, both the teachers and learners should actively participate in the
reduction of errors.
Data were collected from 36 conveniently selected ESL grade 10 teachers in the
Eastern Cape, in South Africa. The research questions used to collect data were:
• How can teachers eliminate the occurrence of grammatical errors
committed by ESL learners?
• What strategies can be taken by learners to eliminate grammatical errors?
Numerous studies have been conducted on the analysis of errors caused by ESL
learners while learning the language. Factors that have been identified as sources
of linguistic errors are attributed to interlingual, and intralingual interference
(Hassan & Munandar, 2018; Zafar, 2016; Sermsook et al., 2017; Shakir et al., 2020).
Interlingual transfer occurs when learners negatively transfer vocabulary from
their primary languages to the target language (Ozkayran & Yilmaz, 2020;
Richards, 2004; Suhono, 2016). Intralingual interference results from an incorrect
application of language rules due to, among other factors, syntactic
overgeneralisation, and grammatical simplification. In ESL classrooms,
grammatical errors are inevitable and, therefore, approaches aimed at avoiding
their occurrence must be devised.
©Authors
310
The communicative language teaching approach has gained prominence in ESL

pedagogy. This approach promotes that learners must be subjected to a significant
amount of the target language when learning the language, and be allowed to
practice or use it in real-world circumstances. Richards and Rodgers (2014)
advocated for a focus on communication rather than rote grammar drills.
Incorporating interactive activities, such as role-play, debate, and collaborative
projects (Matiso & Makena, 2022), encourages learners to apply grammar in real-
life contexts. By doing so, learners not only enhance their grammatical accuracy
but also develop practical language skills.
Ndlovu (2019) reported that the communicative language approach helps to

improve the communicative dimensions of learners but fails to equip learners
with the correct usage of grammar structures as errors are ignored to avoid
creating a threatening environment for learners trying to communicate.
Consequently, essay writing skills are ignored because the goal is to express
messages rather than to use language correctly. Another negative consequence of
the communicative language approach is that when writing essays, learners
cannot detect mistakes (Toro et al., 2019).
Matiso’s (2022) study focuses on how a text-based approach (TBA) can be utilised
to eliminate EFAL learners’ grammatical difficulties. Although the TBA was
recommended by the Department of Education to be used to eliminate discrete
instruction of language structures, numerous studies show that teachers,
worldwide, still struggle to implement it due to a number of factors, which,
among others, include teachers’ varied views about it that it is time-consuming, a
lack of a structured and specified approach to implement it, and lack of support
and guidance from the policy and curriculum developers.
In this paper, the author argues that learners’ writing difficulties emanate from
the instruction of grammatical structures as discrete units. Matiso (2022) used a
poem titled ‘The Dry Grass Sings’ from the grade 10 prescribed poems, to
illustrate how the TBA can be used effectively to eliminate grammatical errors. In
texts, words appear in relation to other words, and in the contexts in which they
are used. The poem shows how concord can be taught using a text.
The poem shows how a text can be effectively used to eliminate grammatical
errors committed by learners. It provides effective and efficient use of present
tense form.
©Authors
311
“The Dry Grass Sings
When the sky’s as hot as a baking pot

And the whole earth shrinks like a skin
The mopane leaves clap and the eagle cries
The turtles snap and the thorn trees scratch
And the dry grass sings and sings
On the crackling mud the crocodile smiles
And the kudu snort in the dust
And the zebras grunt and the jackals howl
And the elephants trudge for another mile
And the dry grass sings and sings
But when lightning strikes and the clouds drop down
And the whole earth drinks from the sky
The wood dove purrs and the fever trees sigh
And the rivers roll as heavy as blood
And the dry grass sings no more” (Gray, 2010)
Committing concord errors, such as subject-verb agreement and pronoun-

antecedent, are among the challenges faced by learners when learning the English
language (Widangsih, 2016). This poem contains phrases that show explicitly how
the subject and verb agree. The poet shows how inflectional morphemes marking
grammatical contrast, specifically, the third person singular marker ‘-s’ and ‘-es’,
and plural markers ‘-s’ can be used to mark grammatical cohesion. These are
indicated by the use of the third person singular in the following colloquial
phrases:
• the earth shrinks, the eagle cries, the dry grass sings (3rd person singular);
• while the 3rd person plural is shown by the phrases that follow — the
leaves clap, the turtles sap, the thorn trees scratch, rivers roll.
When learners have mastered these structures, they can construct their own
phrases using these structures. Continuous use of these structures by learners can
yield good results if they employ these structures to other texts as well.
Pardede (2011) opined that literary texts provide real examples of grammatical
structures and vocabulary items. Literary texts raise awareness of the range of the
target language and advance their competence in all language skills (Pardede,
2011). The prescribed books for language use fail to provide real contexts in which
language structures are used. Consequently, literary texts are far richer and
consist of more language forms than language textbooks. In literary texts, students
encounter real aspects of written language, such the way sentences are put
together, ranging from statements, interrogatives, commands, and simple,
compound and complex sentences. Furthermore, learners are also exposed to the
different ways of connecting ideas, which might not be explored in real contexts
in language textbooks.
Tomlinson (2013) promoted the use of authentic materials in ESL instruction.

Incorporating real-world resources, such as newspapers, podcasts, and videos,
expose learners to natural language usage. Through immersion in authentic
©Authors
312
contexts, learners develop an intuitive understanding of grammatical structures

and nuances.
However, using literary texts to teach grammar is a debatable issue. Smit (2009)
asserted that “since one of the main aims of ESL teachers is to teach the grammar
of the language, literature, due to its structural complexity and unique use of
language, does little to contribute to this goal”. Some academics contend that
misinterpreting literature might result from using it to teach something else, like
syntax.
Effective feedback is crucial in rectifying grammatical errors. Ferris (2003) stressed

the importance of balanced error correction techniques that guide learners toward
understanding and self-correction without stifling their confidence. Written
feedback, peer editing, and one-on-one discussions contribute to a comprehensive
strategy for addressing grammatical difficulties. Modern technology tools play a
significant role in ESL education. Chapelle (2001) discussed the integration of
computer applications and online grammar-checking tools. While these tools
provide immediate feedback, they must be complemented by human guidance.
Incorporating technology can create a blended learning environment that caters
to various learning styles and speeds.
Language teaching can either be inductive or deductive. The fundamental factor

that decides whether inductive or deductive instructional practices are employed
in grammar teaching is linked to the goal of effective language teaching (Sadat,
2017). Some educators favour the inductive method, whilst others prefer the
deductive method.
According to the inductive method, language rules must be learned implicitly

through exposure to meaningful language use in a natural setting (Larsen-
Freeman, 2015). Throughout the learning process, learners derive grammatical
patterns from numerous communicative settings.
In the deductive, or rule-driven method of teaching grammar, each rule is

introduced, then instances of its application are given. According to the deductive
method, learners should be explicitly taught grammar rules through a formal
presentation of those rules (Emvula, 2020). The teacher first presents and explains
the rule, then gives examples of how to apply the rule in practice activities. Finally,
the grammatical rules are provided in a pre-made format. The deductive method
embeds in the learners’ minds the idea that learning a language depends on
remembering grammar rules. When a teacher elicits a rule, students are passive
recipients; they can memorise the rule but cannot use it to further their language
proficiency (Ellis, 2009).
One widely recognised approach involves providing explicit grammar

instruction. Celce-Murcia and Larsen-Freeman (1999) emphasised the significance
of breaking down complex grammatical rules into digestible components,
incorporating clear explanations, examples, and interactive exercises. This
©Authors
313
approach helps ESL learners grasp the fundamental structure of English sentences
and subsequently minimises grammatical errors.
The integration of various approaches to eliminate grammatical difficulties faced
by English as a second language learners can be guided by a theoretical
framework that emphasises cognitive, socio-cultural, and communicative
perspectives. The combination of these perspectives creates a comprehensive
approach that addresses the diverse aspects of language acquisition and usage.
From a cognitive standpoint, learning grammar involves internalising rules,
patterns, and structures. The cognitive perspective draws from the theories such
as cognitive load theory and information processing theory. These theories
suggest that breaking down complex grammar rules into smaller, manageable
chunks supports better comprehension and retention.
Providing explicit grammar instruction, as suggested by Celce-Murcia and

Larsen-Freeman (1999), aligns with this perspective. By presenting grammar
concepts in a structured manner, learners can process and practice each
component before integrating them into their language production. Socio-cultural
perspective, rooted in Vygotyky’s (1978) socio-cultural theory, highlights the
importance of social interaction and cultural context in language learning.
Communicative language teaching aligns with this perspective, as it encourages

learners to engage in authentic communication through activities like role-plays
and discussions (Richards and Rodgers, 2014). Such interactions facilitate the
negotiation of meaning and allow learners to apply grammar in context, reflecting
the socio-cultural nature of language development. The communicative
perspective focuses on language as a tool for communication rather than a set of
isolated rules. This perspective underpins communicative language teaching and
emphasises the practical use of language in real-life situations. Authentic
materials and exposure to natural language usage, as recommended by
Tomlinson (2013), contribute to this aspect. Learners observe grammar in action,
enabling them to grasp nuances and apply grammatical rules more effectively.
A constructive perspective, drawing from the works of Piaget and Bruner, posits
that learners actively construct knowledge through experiences. Constructivist
approaches advocate for hands-on learning and engagement with the material.
Constructive feedback strategies, as outlined by Ferris (2003), align this
perspective by guiding learners to self-correct their errors and understand the
underlying reasons behind them.
An integrative perspective combines the socio-cultural, communicative, and

constructive aspects. Technology integration, as discussed by Chapelle (2001)
exemplifies this approach. Online-grammar checking tools provide immediate
cognitive feedback while promoting communicative engagement through
practical application. This integration bridges theoretical perspectives, enhancing
both learning efficiency and efficacy.
©Authors
314
An effective theoretical framework for addressing ESL grammatical difficulties

involves a holistic integration of cognitive, socio-cultural, communicative, and
constructivist perspectives. By incorporating explicit grammar instruction,
communicative activities, authentic materials, constructive feedback, and
technology tools, educators create a dynamic and comprehensive learning
environment that caters to diverse learning styles and facilitates the elimination
of grammatical difficulties.
4. Methodological Design
Research methodology outlines the methods used by the researchers to give
comprehensive and convincing findings about the phenomenon under
examination. Carefully selected methods include an appropriate paradigm,
research approaches, design, and relevant data collection procedures related to a
particular study.
Research Paradigm
An interpretive paradigm, based on qualitative approaches, formulated the basis
for the discussion of the participants’ ways used to eliminate grammatical
difficulties. The foundation of interpretivism is built on universal traits like the
comprehension and interpretation of routine events and social structures, as well
as the interpretations that people give to phenomena. Kivunja and Kuyini (2017)
argued that what is perceived by people to be true is more important than an
objective reality. They further denoted that positions that are interpretivist are
based on the theory that reality is socially created.
Research Approach
Creswell and Creswell (2018) highlighted that qualitative research is the method
of understanding by which a comprehensive, holistic picture is developed; words
are analysed, informants’ perspectives are reported in detail; and the study is
carried out in a natural environment. Mack et al. (2022) noted that qualitative
techniques are usually more adaptable. They enable more spontaneity and
flexibility in the way the researcher and study’s participants interact; for instance,
qualitative approaches frequently ask questions that are not always phrased the
same way with each participant.
Research Design
A case study research design was employed to understand the approaches used
by ESL teachers to eliminate grammatical difficulties that hinder learners’ writing.
Freitas et al. (2017) noted that conducting research using a case study allows
researchers to immerse themselves in the context and gain intensive knowledge
of a phenomenon, which in turn demands suitable methodological principles.
Participant Selection
The population for this study was a total of 16 grade 10 ESL teachers from six
conveniently selected schools in the province of the Eastern Cape, South Africa.
From four of the schools, a total of 12 participants were drawn, that is, three
teachers from each. The other four participants were from two schools, that is, two
teachers from each. The number of teachers in each school depends on the class
sizes, and that accounts for this varied selection.
©Authors
315
Data Collection Instruments

Semi-structured interviews, which were directed to the 16 conveniently
nominated ESL educators, formulated the data collection instruments for this
enquiry. Semi-structured interviews have the potential to allow the researcher to
gain insight into the phenomenon under inquiry because the researcher has the
opportunity to probe deeper and further to get rich data that will yield good
effects (Creswell, 2014).
The interviews, which were conducted over a period of two weeks, were audio-
taped with the authorisation of the participants. During data analysis, the author
carefully studied the participants’ responses, read them several times, and
grouped similar responses for ease of interpretation. From this exercise, the
themes that formulated discussion of this study emerged.
5. Discussion and Findings

The findings of this study are discussed with specific reference to the themes that
emerged from the data analysis. The themes were discussed based on the
objectives of this enquiry, which were to determine teachers’ ways to eliminate
the occurrence of grammatical errors committed by ESL learners, and the
strategies that can be adopted by learners to eliminate grammatical errors.
The themes that emerged from the first objective were error analysis, followed by
explicit instruction of the language structure, and learner engagement in
corrective feedback. From the latter, further themes that emerged included
multiple exposure to a language structure through practice activities, and
immersion into rich target language environments.
Table 5.1 shows a summary of the objectives and themes of this paper.
Table 5.1: Objectives and themes

Objective Themes
Teachers’ ways to eliminate the Error analysis followed by explicit instruction of the
occurrence of grammatical language structure.
errors. Learner engagement in corrective feedback.
Strategies that can be adopted Multiple exposure to a language structure through
by learners to eliminate practice activities.
grammatical errors. Immersion into rich target language environments.
Objective 1: Teachers’ ways to eliminate the occurrence of grammatical errors

committed by ESL learners
Theme 1: Error analysis followed by explicit instruction of the language structure
The participants indicated that error analysis is one method used by teachers to
eliminate the occurrence of grammatical errors. Error analysis involves the
identification of errors made by learners and they are grouped according to the
frequency of their occurrence (Ozkayran & Yilmaz, 2020). However, for the
effectiveness of error analysis, identified errors should be followed by explicit
instruction of the language structure.
©Authors
316
One participant revealed that she looks at common errors made by learners after
administering a test. After that, exercises that deal with the identified errors are
given. The participant also indicated that she teaches that particular structure
explicitly.
“When I administer a test, I usually give a test that includes comprehension skills,
cartoons, summary, and grammar. After the test I look at common errors like the
application of the apostrophe, parts of speech, that is, learners usually struggle
with formulating nouns from verbs and adjectives from other word classes. After
that I give them exercises and teach that particular structure explicitly.”
(Participant K)
Participant K’s response shows that language structures that usually pose
challenges to learners are the use of the apostrophe, and parts of speech, in
particular, the derivation of new words from different word classes. In this regard,
these learners’ errors are morphological, and, thus, inform the teacher to focus on
the emphasis of both the use of inflectional morphemes and derivational
morphemes. Such errors are intralingual errors, usually caused by incorrect
application of language rules.
Brown (2023) purported that learners’ errors are also informative to the teacher as
they indicate which language structures have not been grasped by the learners.
Furthermore, the teacher can also scrutinise their instructional practices and adopt
innovative ways that will facilitate the comprehension of a particular language
structure. Error analysis has been found to be a beneficial exercise by Zafar (2016)
who conducted a study among Business Studies students to establish errors that
were frequently committed by the learners. The outcomes of the analysis
indicated that verb tenses were the most challenging area, and, the students had
a remarkable improvement after a two months’ period of explicit instruction in
these structures.
Thus, error analysis has remarkable effects which include valuable information
about how learners acquire and learn a language. Teachers also identify areas that
pose challenges to learners (Ozkayran & Yilmaz, 2020).
Theme 2: Learner engagement in corrective feedback

The teachers have different beliefs about how grammatical difficulties should be
addressed. Evidence of feedback to learners varies from teacher to teacher.
Research has shown a positive effect of corrective feedback when learners are
engaged in the feedback process. However, there is very limited evidence that
teachers provide comprehensive feedback to learners. The participants’ responses
indicate that teachers struggle to give feedback for all the errors made by the
learners.
“What I do is to underline the mistakes and write the correct word. But it is very
difficult to give correct answers for whole sentences. Learners struggle to
construct sentences and if I try to correct the sentence myself that consumes time.”
(Participant C)
The claim shows that teachers struggle to provide sufficient feedback on errors
related to sentence construction. However, teachers use various methods,
©Authors
317
including indirect feedback (Schenck, 2021), which can be more appropriate in

cases where a correct word could not be given. In this regard, teachers should
engage learners in the correction of syntactically related errors.
Hasan and Munandar (2018) contended that ignoring such errors might lead to
the construction of ambiguous and anomalous sentences, and, thus, deprive
learners of the opportunity to communicate their ideas.
Lira-Gonzales and Valeo (2023) articulated that studies that explain how learner
engagement as a factor that can have positive gains in eliminating grammatical
errors have not yet been widely researched. Learner engagement takes various
forms and, as such, the manner in which learners react to written corrective
feedback should be considered. Some learners might feel threatened, while some
respond positively and seize the opportunity as a corrective measure of their
linguistic errors.
The participants’ responses indicate that in cases where the students struggle with
sentence construction, some teachers are unable to give a corrective feedback.
There is very little evidence that teachers monitor the correction of errors.
Although the participants indicated that they mark the learners’ scripts and
identify errors, they do not monitor that the corrections are done.
“Honestly, I do not monitor if corrections have been done although I insist that
they should be done.” (Participant H)
“I do not engage learners in doing corrections. I write the corrections in the
learners’ scripts, but I never discuss the learners’ corrections with individual
learners.” (Participant E)
Corrective feedback aids in error correction. When learners receive feedback that
highlights grammatical, syntactical, or lexical errors, they become aware of their
mistakes. This awareness is a crucial step towards improvement, as students are
more likely to avoid making the same errors in future writings. Without this
guidance, learners may persistently repeat errors, hindering their progress.
Different types of corrective feedback, such as direct, indirect, and metalinguistic

cues, offer varied benefits to learners (Schenck, 2021). Direct feedback that
includes clear error correction promotes quick error correction. In contrast, giving
clues or pointing out mistakes as indirect feedback motivates students to self-
reflect and self-correct.
Metalinguistic cues encourage students to examine the underlying grammatical

or structural problems on their own, leading to a deeper comprehension of
linguistic norms (Ellis, 2009). Metalinguistic cues involve the teacher indicating
that a grammatical error has been committed and allows the learner to identify
the type of error. Lee (2019) contended that problems with giving written criticism
include a propensity to fix all mistakes, which prevents attention from being paid
to certain grammatical aspects. However, in essay writing, teachers attend to all
the errors, and, for this reason, essay writing might not be beneficial when the
teacher targets a specific language feature. Classroom activities, usually found in
©Authors
318
grammar books, are ideal for concentrating in a language structure that has been
identified by the teacher as posing challenges to learners.
While some studies claim that providing explicit information about a written error
(direct corrective feedback) leads to higher gains, others claim that providing
unambiguous facts about a grammar error (metalinguistic corrective feedback),
when combined with direct corrective feedback, leads to developed achievements
(Bitchner & Knoch, 2009; Scheck, 2021). Corrective feedback depends on the
learner’s preferences in processing learning material. Moreover, timing,
frequency, and format contribute to the effectiveness of the feedback.
The value of timely feedback is highlighted by Bitchener and Knoch (2009), since
delayed input may make it more difficult for students to relate the comments to
their work. The preferences and learning styles of each student should also be
taken into account, since some may benefit more from direct discussion, while
others may prefer more subtly worded instructions.
Lundgren (2022, p. 21) reiterated that “corrective feedback needs to provide

constructive explanations so that the students are not left confused by the
feedback.” The effectiveness of corrective feedback lies in the understanding of
the errors identified by the teacher. Then, learners will be able to complete the
corrections without effort. Furthermore, monitoring of the corrected errors by the
teacher is also essential. Identification of errors without monitoring the corrected
structures is a futile activity. Without proper guidance, learners are likely to
commit the same errors. In this regard, written corrective feedback will not be
beneficial to learners.
Objective 2: Strategies that can be adopted by learners to eliminate grammatical

errors
Theme 1: Multi exposure and practice to a language structure through practice
activities
The participants indicated that learners need multi exposure to texts and practice
to a language structure. Learners need many opportunities to practise what they
have learned. The participants indicated that when grammatical errors committed
by learners have been identified, regular exercises that focus on the language in
question should be administered. It was also indicated that common grammatical
errors committed by learners are concord, that is, agreement of the subject and the
verb.
“Extensive reading, which familiarises them with grammatical structures,
especially concord. Regular exercises as well, with specific focus on concord and
punctuation, until they get it right. Another important aspect is tense alongside
concord.” (Participant B)
Another participant indicated that learners’ grammar books provide

opportunities to practise exercises dealing with tenses, an aspect that was
identified by the participant as posing problems to learners. Grammar books, in
general, are tailored to address language structures, including comprehension
exercises.
©Authors
319
“They usually struggle with tenses so we do a lot of sentence classification and I

give them activities with tenses from our everyday texts.” (Participant C)
The participants’ responses showed that learners struggle with grammatical

structures, specifically tenses and concord. Teachers resort to error identification
and explicit instruction of the language structures that pose difficulties to learners.
All the participants indicated that the best option is to engage learners in multiple
language exercises found in grammar books. Adrianova (2023) contended that
there have been various approaches proposed by authors to eliminate the
occurrence of grammatical errors yet there are continued reports about learners’
writing difficulties. Adrianova (2023) suggested that even while involving
students in grammar-related activities does not provide positive outcomes,
students should routinely participate in speaking and writing tasks that call for
the use of grammar principles. This active output helps the student internalise
grammar structures and solidify their comprehension, which improves accuracy
and fluency.
Theme 2: Immersion in rich target language environments

Adrianova (2023) posited that to embrace immersion, language learners should
surround themselves with English language media, such as books, movies, TV
series, podcasts, and music. They should also practice with a specific goal in mind,
using tools such as grammar workbooks, online resources, and language learning
applications, to strengthen their knowledge and application of grammar rules,
rather than just mindlessly doing exercises. Students must be accountable for their
own learning, record their errors, and make an effort to employ the same grammar
structures each time they speak or write in English. “A proactive approach that
combines immersive experiences, purposeful practice, meaningful interactions,
active output, personalised guidance, and the power of technology” (Adrianova,
2023) is the key to enhancing English grammar.
Participant G stated that one method that can be used by learners is to take
advantage of their devices and utilise them to increase their vocabulary:
“Nowadays learners have smart phones where they can watch You-Tube videos.
Exposure to these can assist them eliminate grammatical errors.” (Participant G)
The response reveals that various opportunities, which include the use of devices
readily available to learners, could be used for vocabulary development. The
virtual space, therefore creates an enabling environment for language
development. However, teachers remain at the centre of the teaching and learning
environment. Therefore, the teacher, as the facilitator of interactions and needs
analyst, co-ordinates the learning environment by facilitating access to various
text types ranging from written, oral, and multi-media texts.
6. Conclusion
This paper explored approaches used by ESL teachers to eliminate learners’
grammatical errors. The findings of this study revealed that a comprehensive
approach that could be used to eliminate grammatical difficulties incorporates the
adoption of error analysis. Error analysis will yield good results if followed by
explicit instruction of the language structures identified during the error analysis
©Authors
320
process. Furthermore, learner engagement in the execution of errors is crucial but

error correction will be delayed if feedback is vague and not properly
communicated.
Moreover, multiple exposure to a language structure and immersion in rich target

language contexts facilitate the ability to identify and correct errors. Thus,
combining explicit grammar instruction, communicative teaching strategies,
constructive feedback mechanisms, technology integration, and exposure to
authentic materials offer a comprehensive approach. Educators can tailor these
strategies to create dynamic and effective learning environments, ultimately
aiding ESL learners in conquering grammatical challenges and achieving
linguistic proficiency.
Learners also need to identify their areas of weaknesses, and focus on those areas
by engaging in a wide range of texts. They should voluntarily practise utilising
structures they have identified through speaking and writing activities. A joint
effort between teachers and learners will produce good outcomes, and this will be
possible when learners realise their roles as active participants in the learning
environment. The author recommends that teachers should be trained to equip
leaners with error detection skills, and effective ways of engaging with
corrections.
7. References
Adrianova, E. (2023). Unleashing the Power of English Grammar: A Holistic Approach to
Boosting Your Skills. https://www.treeducation.net>post.how-to-boost-your-
skills.
Alhasony, M. (2017). EFL Teachers’ and Learners’ Perceptions of Grammatical Difficulties:
Advances in Language and Literary Studies, 8(1), 188-199.
https://doi.org/10.757/aiac.alls.v.Ip
Alsied, S. M., Ibrahim, N. W., & Pathan, M. M. (2018). Errors Analysis of Libyan EFL
Learners’ Written Essays at Sebha University. International Journal of Language and
Applied Linguistics, 132.
Ayliff, D. 2010. ‘Why can’t Johnny write? He sounds okay! Attending to form in English
Language Teaching. Perspectives in Education, 28(2), 1-8.
Bitchener, J., & Knoch, U. (2009). The Contribution of Written Corrective Feedback to
Language Development: A Ten Month Investigation. Applied Linguistics, 31, 193-
214. https://doi.org/10.1093/applin/amp016
Celce-Murcia, M., & Larsen, D. (1999). The grammar book: An ESL/EFL teacher’s course.
Heinle & Heinle Publishers.
Chapelle, C. A. (2001). Computer applications in second language acquisition: Foundations for
teaching, testing, and research. Cambridge University Press.
Cowan, R. (2008). The Teacher’s Grammar of English: A Course Book and Reference Guide (1st
ed.). Bukupedia Publishers.
Creswell, J. W., & Creswell, J. D. (2018). Research design (5th ed.). SAGE Publications.
Department of Education. (2013). Curriculum News. Improving the quality of learning and
teaching: Strengthening curriculum implementation from 2010 and beyond. Department
of Basic Education. https://equaleducation.org.za/2010/11/23/curriculum-
news-improving-the-quality-of-learning-and-teaching-planning-for-2010-and-
beyond/
©Authors
321
Department of Basic Education (DBE). (2018). National Curriculum Statement (NCS):

Curriculum and Assessment Policy Statement (CAPS): Home Language: Intermediate
Phase. Government Printing Works.
https://www.education.gov.za/Curriculum/CurriculumAssessmentPolicyStat
ements(CAPS)/CAPSIntermediate.aspx
Ellis, R. (2009). Task-based language teaching: sorting out the misunderstandings.
International Journal of Applied Linguistics, 19 (3), 221–246.
https://doi.org/10.1111/j.1473-4192.2009.00231.x
Emvula, H. (2020). Common English Grammatical Writing Errors Among Namibian Grade 7
Learners [Master of Education thesis, University of the Free State].
https://scholar.ufs.ac.za/handle/11660/11211
Eunson, B. (2020). English Grammar- A Critical Approach. Retrieved from:
https://www.researchgate.net/publication/341370223_English_Grammar-
_A_Critical_Approach
Farooqui, A. S. (2016). A corpus-based study of academic collocation use and patterns in
postgraduate Computer Science students writing. [Doctoral Thesis, University of
Essex].
https://repository.essex.ac.uk/16426/1/FINAL%20AFNAN%20PHD%20THES
IS-%20submit.pdf
Ferris, D. R. (2003). Response to student writing: Implications for second language students.
Routledge. https://doi.org/10.4324/9781410607201
Freitas, J. S., Ferreira, J. C. A., Campos, A. A. R., Mello, J. C. F., Cheng, L. C.., & Goncalves,
C. A. (2017). Methodological roadmapping: A study of centering resonance
analysis. RAUSP Management Journal, 53, 1465–1474.
https://doi.org/10.1108/RAUSP-04-2018-005
Gray, S. (2010). The Dry Grass Sings.
https://waverleyenglish.files.wordpress.com/2017/04/the-dry-grass-sings.pdf
Hassan, K. (2018). Difficulties Facing English Teachers in Teaching Literary Texts at
Higher Secondary Level in Bangladesh. English Language and Literature Studies,
8(3), 15. https://doi.org/10.5539/ells.v8n3p15
Hassan, I., & Munandar, A. (2018). Grammatical errors produced by VGM department
students. Lexicon, 5(2), 107-114. http://dx.doi.org/10.22146/lexicon.v5i2.41305
Hoadley, U. (2012). What do we know about teaching and learning in South African
primary schools? Education as change, 16(2), 187-202.
http://dx.doi.org/10.1080/16823206.2012.745725
Iipinge, K. (2018). Consequences of ideology and policy in the English second language classroom:
The case of Oshiwambo-speaking students in Namibia. [Doctorate thesis, University of
Western Cape]. http://hdl.handle.net/11394/6234
Kivunja, C., & Kuyini, A. B. (2017). Understanding and Applying Research Paradigms in
Educational Contexts. International Journal of Higher Education, 6(5), 26–41
https://doi’org/10.5430/ijhe.v6n5p26
Khatter, S. (2019). An analysis of the common Essay Writing Errors among EFL Saudi
learners. Arab World English Journal, 10(3), 364–381.
https://dx.doi.org/10.24093/awej/vol10no3.26
Larsen-Freeman, D. (2015). Research into practice: Grammar learning and teaching.
Language Teaching, 48(2), 262–280. http://dx.doi.org/10.1017/S0261444814000408
Lee, I. (2019). Teacher written corrective feedback: Less is More. Language Teaching, 52(4),
524–536. https://doi.org/10.1017/S0261444819000247
Leki, I. (2017). Undergraduates in a second language: Challenges and complexities of academic
literacy development. Routledge. https://doi.org/10.4324/9781315084442
©Authors
322
Lira-Gonzales, M., & Valeo, A. (2023). Written Corrective Feedback and Learner
Engagement: A case study of a French as a second language program. Journal of
Response to Writing, 9(1), Article 2. Available at:
https://scholarsarchive.byu.edu/journalrw/vol9/iss1/2
Lundgren, L. (2022). The Correctness of Corrective Feedback: A Qualitative Content Analysis of
Four Support Materials from the Swedish National Agency for Education. Orebro
University. https://www.diva-
portal.org/smash/get/diva2:1684636/FULLTEXT01.pdf
Mack, B. M. (2022). Addressing Social Workers’ Stress, Burnout, and Resiliency: A
Qualitative Study with Supervisors. Social Work Research, 46(1), 17–28.
https://doi.org/10.1093/swr/svab032
Manan, N. A., Zamari, A. M., Pillay, I. A. S., Adnan, A. H. M., Yusof, Y., & Raslee, N. N.
(2017). Mother Tongue Interference in the Writing of English as a Second Language
(ESL) Malay learners. https://doi.org/10.6007/IJARBSS/v7-i11/3566
Matiso, N. H., & Makena, B. (2022). Effective implementation of the Text-based Approach
to English Language learning and Teaching. E-Journal of Humanities, Arts and Social
Sciences, 3(10), 488-498. https://doi.org/10.38159/ehass.20223106
Matiso, N. H. (2022). The Implementation of the Text-based Approach to English Second
Language Teaching and Learning. E-Journal of Humanities, Arts and Social Sciences,
3(10), 488-498. http://dx.doi.org/10.38159/ehass.20223106
Matiso, N.H., & Tyantsi, O. (2023). Discourse Analysis as an Essential Variable in
Developing Grade 11 English First Additional Language Learners’ Writing Skills.
World Journal of English Language, 13(7), 421-429. DOI:
https://doi.org/10.5430/wjel.v13n7p421
Moloney, R., & Saltmarsh, D. (2016). Knowing your students in the Culturally and
Linguistically Diverse Classroom. Australian Journal of Teacher Education, 41(4).
https://doi.org/10./14221/ajte.2016v41n4.5
Ndlovu, P. E. (2019). Challenges when writing English: A case study of grade 8 second
language learners in a rural school in KwaZulu-Natal [Master’s Degree Education
Studies, UKZN]. https://researchspace.ukzn.ac.za/handle/10413/18209
Nzerem, J.K., & Bob, P.O. (2021). The role and significance of error linguistics in a second
language environment. ANSU Journal of Language and Literacy Studies, 1(5), 1-9.
https://www.google.co.za/url?as=t&rct=j&q=es.rc
Opara, I. (2016, February 14). Poor reading culture and readiness for e-library system. The
Guardian. https://guardian.ng/art/poor-reading-culture-and-readiness-for-e-
library-system/
Ozkayran, A., & Yilmaz, E. (2020). Analysis of Higher Education Students’ Errors in
English Writing Tasks. Advances in Language Studies, 11(2), 48-58.
www.alla.aiac.org.au
Pardede, P. (2011). Using Short Stories to Teach Language Skills. Journal of English
Teaching, 1(1). https://doi.org/10.33541/jet.v1i1.49
Richards, J. C. (2004). Error Analysis. Perceptions on second language acquisition (3rd ed.).
Longman. https://doi.org/10.4324/9781315836003
Richards, J. C., & Rodgers, T. S. (2014). Approaches and methods in language teaching.
Cambridge University Press.
http://ebooks.cambridge.org/ebook.jsf?bid=CBO9780511667305
Sadat, M. (2017). Revisiting the debate of Grammar Teaching: A Young Scholar’s
Perspective. Sino-US English Teaching, 14(1), 1-7. https://doi.org/10.17265/1539-
8072/2017.01.001
©Authors
323
Schenck, A. (2021). Defining Grammatical Difficulty to Make Better Choices about

Corrective Feedback: A Meta-Analysis of Persian SFL Learners. SAGE Open, 11(3).
http://doi.org/10.1177/21582440211047558
Sermsook, K., Liamnimni, J., & Pochakorn, R. (2017). An Analysis of Errors in Written
English Sentences: A Case Study of Thai EFL Students. English Language Teaching,
10(3), 101-110. https://dx.doi.org/10.5539/elt.v10n3p.101
Shakir, M., Rasood, A., & Khan, M. (2020). Error Analysis in English as a Second Language
Students’ Writing. International Journal of Innovation, Creativity, and Change, 14(8),
812-835. Available from www.ijicc.net
Shintani, N., Ellis, R., & Suzuki, W. (2014). Effects of written feedback and revision on
learners’ accuracy in using two English grammatical structures. Language
Learning, 64(1), 103-131. https://doi.org/10.1111/lang.12029
Shiu, L. J. (2011). EFL Learners’ perceptions of Grammatical Difficulty in relation to Second
Language Proficiency, Performance and Knowledge [Doctoral thesis, University of
Toronto]. https://hdl.handle.net/1807/29869
Shin, Y., & Yoo, I. W. (2019). Determiner Use in English Quantificational Expressions.
TESOL Quarterly, 54(2). https://doi.org/10.1002/tesq.539
Sjolie, M. (2016). Language attitudes, motivation, and standards. Cambridge University Press.
https://doi.org/10.1515/ausp-2016-0022
Smit, T. 2009. The role of African literature in enhancing critical literacy in first generation
entrants at the University of Namibia. Unpublished Doctoral dissertation.
Stellenbosch University, Stellenbosch.
Spaull, N. (2013). South Africa’s education Crisis: the quality of education in South Africa, 1994-
2011. Johannesburg Centre for Development and Enterprise.
https://section27.org.za->Spaull-2013-CDE
Suhono, S. (2016). Surface strategy taxonomy on the efl students’ composition: A study of
error analysis. Iqra, 1(2), 1. https://doi.org/10.25217/ji.vli2.128
Tomlinson, B. (2013). Developing materials for language teaching. Routledge.
https://www.academia.edu/43760213/Developing_Materials_for_Language_T
eaching
Toro, V., Camacho-Minuche, Pinza-Tapia, E., & Parades, F. (2019). The use of the
Communicative Language Teaching Approach to Improve Students’ Oral Skills.
English Language Teaching, 12(1). https://doi.org/10.5539/elt.v12n1p110
Ummah, D. (2018). Chaos/complexity science and second language acquisition. Applied
Linguistics, 18(2), 141-165. https://doi.org/10.1093/applin/18.2.141
Vygotsky, L. S. (1978). Interaction between learning and development. In M. Cole, V. John-
Steyner, S. Scriber & E. Souberman (Eds.), Mind and Society: The development of
higher psychological processes. The Harvard University Press.
https://doi.org/10.2307/j.ctvjf9vz4
Widdowson, H. G. (1990). Grammar and nonsense and learning. In H. G. Widdowson
(Ed.), Aspects of Language Teaching (pp. 79–98). Oxford University Press.
https://courses.aiu.edu/Human%20Growth%20and%20Development/8/08.H
G%20Aspects%20of%20Language.pdf
Widianingsih, N. A. (2016). Grammatical difficulties encountered by Second Language learners
of English. Proceedings of the Fourth International Seminar on English Language
and Teaching (ISELT). Book Guild Publishing Ltd.
https://www.researchgate.net/publication/303083847_Grammatical_Difficultie
s_Encountered_by_Second_Language_Learners
Zafar, A. (2016). Error analysis: a tool to improve English skills of undergraduate students’
Future Academy’s Multidisciplinary Conference. Procedia-Social and
Behavioural Sciences 217, 697-705.
©Authors
324

Vol. 22, No. 10, pp. 324-339, October 2023
Implementation of Virtual Worlds to Promote

Distance Practice Teachers' Participation in the
English Learning Process
Ana Quinonez-Beltran , Carmen Benitez-Correa
and Elsa Morocho-Cuenca
Universidad Técnica Particular de Loja, Ecuador
Abstract. Distance education has adopted innovative technology-

assisted approaches to improve teacher engagement, with virtual worlds
emerging as a potentially promising tool for enhancing the participation
of distance practice teachers in the learning process. In this context, the
purpose of this study is to investigate the incidence of using virtual
worlds as a resource to increase distance practice teachers’ participation
in the learning process. An action research design that incorporated a
combination of both quantitative and qualitative research was used to
carry out the study. The techniques applied to collect the information
were a survey, an interview, and an observation checklist. The sample
consisted of 116 participants (44 male and 72 female) whose ages ranged
from 25 to 35 years. They were enrolled in the English major of a distance
program at a private university in a southern city in Ecuador. They
participated actively in 16 synchronic activities during the intervention
period (one per week). The results showed that the use of virtual worlds
allowed pre-service teachers to increase the frequency of their
participation during synchronic classes. Besides, practice English teachers
had positive perceptions regarding the use of virtual worlds as a resource
for learning because the sense of presence provided by the platform
motivated them to participate in the activities. From the findings of the
study, it is recommended that further research consider the use of virtual
worlds for fostering participation in on-site learning environments to
further triangulate the results of the study and for comparative analysis.
Keywords: distance education; participation; practice teachers; virtual

worlds

The use of technology to enhance the learning of English as a foreign language
(EFL) has greatly increased due to globalization. This study explores the
implementation of virtual worlds to promote distance practice teachers'
participation in the learning process. Indeed, learners are favoured with a variety
©Authors
325
of tools that can be used to study and reinforce the contents of subjects, especially
those university students who have registered in a distance program, whose
limitations do not allow them to have the opportunity to ask and answer questions
any time neither receive feedback nor participate during classes (Oliveira et.,
2018).
In this context, virtual worlds (VW) are one of those technological resources that
contribute to overcome the above-mentioned difficulties found in a distance
program. Virtual worlds are combined systems of networked online devices and
sources, which allow learners who study in a distance program, to be more active
since it provides a similar environment to a real one using avatars (Zhang as cited
in Shafieiyoun & Safaei, 2013; Krassmann et al., 2020). Additionally, virtual
worlds improve students’ intrinsic motivation (Francke & Alexander, 2018) and
their learning experience (Nøgaard et al., 2018), provide more opportunities to
work collaboratively, interact, and participate through virtual field trips, virtual
city tours, role-playing, and creative construction work (Chen, 2016).
As any other resource, virtual worlds have their own characteristics and features.
Regarding their features, Martínez (2012) mentions that the main ones are
persistence, interactivity, physical presence, chat, gestures, and voice. According
to this author, persistence deals with the simulation of a 3D space; interactivity
involves the interaction among participants; and physical presence is concerned
with the avatar that represents the user. Regarding avatars, Girvan and Savage
(2019) argue that they create several perceived possibilities for education, offering
opportunities for learning process, as learners co-exist and share a virtual space.
Some studies have been carried out regarding the use of virtual worlds in the
educational context. Mørch et al. (2018) researched three contexts (social,
pedagogical, and emotional) created by teachers to foster collaborative learning
in virtual worlds. The results indicate that using avatars promotes students´ social
interaction and collaborative skills. Besides, virtual worlds help teachers and
students to express their emotions in many situations.
The study carried out by Díaz et al. (2020) investigated the use of virtual worlds
to motivate students when learning as well as to determine the learners’
perceptions of the usefulness and functionality of virtual worlds. Findings
demonstrate that participants showed high interest in navigating and interacting
with the virtual world. Moreover, virtual world environments facilitate learning
using computers or mobile devices in both synchronous and asynchronous
classes.
Contreras et al. (2018) researched a methodology for implementing VW with open

simulator in two subjects of a distance program. The results revealed that most
students perceive virtual worlds as an easy resource to interact as well as to move
within due to its functionality. Additionally, teachers considered VW as a very
entertaining tool to enhance learning through real practices.
326
Although there has been research in the field of virtual learning worlds, it has not
been investigated how virtual environments influence on distance students’
participation. In our context, in which the present study has been carried out,
distance EFL practice teachers learn by themselves and must complete different
graded asynchronous activities (forum, quizzes, and assignments) as parts of their
learning process. Additionally, they must participate in some synchronous
activities such as chats or video collaboration sessions. However, students´
participation is low during the aforementioned sessions because they do not feel
motivated to interact or complete the activities. Besides, the technological
characteristics of the platform, through which this teaching-learning process
occurs, do not promote active participation.
To overcome this limitation, virtual worlds were posed as an alternative to

increase our EFL practice teachers´ participation since this platform has some
engaging elements such as avatars that motivate them to participate dynamically
during the synchronic activities. Therefore, the purpose of the present study is to
provide insights into the use of virtual worlds as a resource to enhance distance
practice teachers´ participation in the different synchronic activities, which might
be beneficial for future teachers. Specifically, it sought answers to the following:
1) How do virtual worlds influence English practice teachers’ participation during
synchronic activities? 2) What are English practice teachers’ perceptions of the use
of virtual worlds as a resource for learning?
2.1 Distance Education
Distance education started in the 19th century; since then, it has had a notable
impact on the educational field (Saykılı, 2018). This modality emerged for some
reasons such as the geographical space between people and educational
institutions and socio-economical aspects; the willingness to learn; and the rapid
growth of technology (Casey, 2008). Along time, distance education has been
defined in different ways. According to Bagrıacık (2019) it is a type of instruction
in which students learn individually or in groups, but they neither have physical
contact with the instructor nor attend an educational institution. Besides,
Schlosser and Simonson (2009) state that distance education is a program where
students learn separately through telecommunications systems, which allows
them to interconnect all learners by using different technological resources.
Holmberg (1989) complements these definitions by adding that distance
education allows students to learn at different times and in different places; it
means that they decide when and where to study and access to the provided
material as well.
Additionally, for Keegan (1996) five elements need to be considered to define

distance education. The first one lies in the quasi-permanent separation between
the teacher and the learner, setting it apart from traditional face-to-face education.
The second one involves the active engagement of an educational institution in
creating learning materials and offering student support services. The third
element pertains to the near-constant lack of a learning community during the
entire learning period. In this setup, individuals are typically educated on an
327
individual basis rather than in collective settings. Periodic gatherings may occur
for educational and social interactions. The fourth one fosters the interactive
communication, allowing students to engage in dialogue with teachers—a feature
that differentiates it from other educational technology applications. The fifth
element implies the use of various technical media such as print, audio, video, or
computer technologies to connect teachers and learners, delivering the course
content.
In the context of technology, Berge (1995) states that it is a very important tool in
distance education since, besides promoting interaction, it allows learners to find
and experiment some ways that fit their own styles of learning. This author also
mentions that one of the benefits of technology is that it makes students construct
their own knowledge, which is created through engagement, dialogue and
interactivity (Rajesh, 2015). Certainly, according to Makarenya, et al. (2020)
distance education requires the utilization of telecommunications and electronic
devices that allow students to receive instruction from a remote location.
Currently, computers and telecommunications enable the development of

interactive and integrated environments in distance education, surpassing
conventional methods like pen-and-paper correspondence. Vásquez et al. (2006)
assert that technological advances allow distance learners not only sharing
knowledge but learning in a meaningful way since their teachers make use of
technological tools (Internet, videoconference, chat sessions, among others) for
promoting their interaction in a synchronic and asynchronic way (Martínez, 2008).
2.2 Virtual Worlds

The term virtual worlds is defined by some authors in different ways at different
times. One of the important definitions is the one stated by Girvan (2018) who
affirms that a virtual world is an environment and what sets it apart from the real
world is the various experiences it provides for users, mainly through the use of
avatars and other technical features. Similarly, Bartle (2003) states that VW is a
self-contained environment that its users perceive as complete in itself. Koster
(2004) complements this definition by explaining that a virtual world can be used
at the same time by many users, who are represented by avatars.
Regarding the characteristics of virtual worlds, Kahai, et al. (2023) assert that VW
provides a level of realism and immersion that is not found in other online
learning settings. When learning in a virtual world, individuals often engage in
spatial navigation, reasoning, object manipulation, and memorization of virtual
spatial locations and objects. Shafieiyoun and Safaei (2013) mention that social
presence is another important characteristic of virtual environments because it
makes students feel present as in an on-site class. Based on it, Zhang (2009) and
Krassmann et al. (2021) mention that virtual environments allow distance
education students to work in similar situations as in real environments because
of the interactions between avatars and virtual tools. Besides, virtual worlds offer
a stimulating and new environment to improve learning for students to generate
further interactive learning experiences. Furthermore, virtual learning worlds
have been provided as an adaptable and independent learning environment for
328
students (Richardson & Swan, 2003). All these features make virtual worlds a
good opportunity for distance education students to acquire knowledge and
develop their competencies.
In this context, Gregory et al. (2019) state that VW plays a crucial role and is
considered as a valuable tool in the educational field because it provides new and
innovative forms for supporting the learning process in general. Shafieiyoun and
Safaei (2013) consider the use of virtual environments as significant and important
tools in the educational area, especially for online and distance settings. They also
highlight that virtual environments give the opportunity to create communities,
foster trust, increase the sense of presence in learning, play with roles and identity,
and undertake activities not normally physically possible. Indeed, the educational
potential of virtual worlds has been recognized by educators, who consider them
as an available and efficient resource to support the teaching process (Jacka, 2018).
Virtual worlds are also recognized as a pedagogical opportunity and immersive
space that students can take advantage of; all of these, make virtual environments
an authentic context in which students can create and share virtual assets and
develop their scope of learning (Savin-Baden, 2010).
In this regard, Peterson (2011) states that the advantages of virtual worlds for
educational purposes are great because they constitute very useful spaces for
language learning tasks in which students can interact in the target language; they
also provide socially interactive learning opportunities such as virtual field trips,
virtual city tours, role-playing, and creative construction work. In addition,
Richardson and Swan (2003) state that collaborative learning is another advantage
of virtual environments because they permit students to work in groups in which
every student must participate and collaborate effectively to reach the goals of the
group (Hedberg & Brudvik, 2008).
Other authors have researched the advantages of VW and they have come to the
conclusion that most of the participants showed a positive attitude toward the use
of virtual worlds in a collaborative learning setting (Alshumaimeri et al., 2019).
Yu et al. (2020) demonstrated that VW contributed to reduce learners’ foreign-
language anxiety and improve their speaking proficiency. Krassmann et al. (2020)
found that the sense of presence through VW was perceived as a positive factor
in the students´ learning. Demirbilek and Koç (2021) evidenced that VW is used
as another form of education for meeting students' social needs using today's
advanced technology.
2.3 Students´ Participation in EFL Classroom

In the educational field, students´ participation refers to actively taking part in
classroom discussions, debates, questioning, inquiry, and explanations, which
allows students to construct and acquire in-depth knowledge (Simovska, 2007). In
this context, participation plays a vital role in achieving learning outcomes
(Sparapani et al., 2023; Kim, 2013) since it promotes knowledge acquisition and
critical thinking, and enhances the overall quality of discussions in the learning
environment (Davies & Graff, 2005).
329
Despite the significance of promoting active student participation in the

classroom, certain factors constrain it. Some of them are low levels of motivation,
high levels of anxiety, and fear of making mistakes, among others, which cause
students to prefer working independently and sitting at the back of the classroom
(Hanh, (2020). Therefore, it is crucial teachers use some strategies to encourage
students to participate actively in the classroom because it is a fundamental
element of learning a foreign language (Badem-Korkmaz & Balaman, 2022). These
authors also assert that making efforts to stimulate student engagement becomes
significant in language educational environments, including synchronous remote
classrooms.
3. Method
3.1 Research Design and Participants
In the present study, action research design was used using mixed methods of
quantitative and qualitative approaches. Creswell (2015) asserts that this design
approach enables educators to collect data for the purpose of tackling and
resolving specific, real-world issues within educational settings.
The sample consisted of 116 pre-service teachers, 72 female and 44 male, whose
ages ranged from 25 to 35. They were enrolled in the English major of the distance
program of a private university in Ecuador. This sample was intentionally chosen,
following the principle of selecting participant groups that happen to be available
(Mertler & Charles, 2008).
3.2 Instruments
Table 1. Presents the list of research instruments and their purpose.
Table 1: Research Instruments
Instrument Purpose Number of items
Checklist (Observation) It helped us determine the Six items related to

incidence of virtual students´
worlds to promote participation
students’ participation in
this type of setting.
Perception’s questionnaire (survey) It allowed us to know the Nine items for

participants’ perceptions eliciting answers on
regarding the use of a four-point Likert
virtual environments as a scale.
tool for enhancing their
learning.
Questionnaire (interview) It was applied to Six open questions

triangulate information
regarding how the use of
virtual worlds impacted
students´ participation.
330
3.3 Procedure and Analysis

The present study lasted five months within the academic period. It was carried
out through a virtual world platform and consisted of three stages. In the first
stage, pre-service teachers received technological training about how the
aforementioned platform works, and how to create, move and rotate the avatars
around virtual worlds. This training lasted for two weeks and was done through
virtual sessions. Participants were also provided with a video training tutorial for
them to watch whenever they needed to refresh the information given.
Additionally, the pre-service teachers received guidelines and instructions on
how the synchronic activities were going to be carried out through the platform.
In the second stage, six planned synchronic activities (video collaboration sessions
- one per week) were carried out by means of the virtual world platform. Each
one of them lasted two hours, time during which the instructors taught different
topics such as how to write paragraphs and essays, main ideas and supporting
details, organization of ideas, punctuation, and mechanics. These topics were
covered through examples that were analyzed by students to identify and correct
errors. Before starting each session, the participants chose and customized their
avatars, and went to the virtual classroom. During the sessions, students
participated actively and teachers provided feedback to reinforce students’
knowledge. In each virtual session, the student’s participation was registered in a
checklist sheet to be analyzed quantitatively later.
The third and final stage, was an interview which was conducted to support and
triangulate the information gathered from the checklist. Additionally, a survey
was administered to the students to know their perceptions regarding the use of
virtual worlds to enhance their learning process. The results from the survey were
analyzed quantitatively using the SPSS software, and the qualitative data from the
interview were used as excerpts in the discussions.
4. Results
This section includes descriptive statistical analysis of the findings of the present
study, which are related to the research questions.
4.1 Influence of virtual worlds on English practice teachers´participation during
synchronic activities
Table 2. English practice teachers’ participation in virtual worlds during synchronic

activities
ITEMS 1st 2nd 3th 4th 5th 6th 7th 8th
class class class class class class class class
1.Students participate 15% 21% 29% 45% 50% 62% 70% 87%
actively
2.The student shows 17% 24% 35% 40% 51% 68% 82% 90%
interest in participating
3.The student interacts 19% 33% 40% 49% 58% 72% 80% 87%
with the teacher
331
4.The student interacts 21% 35% 44% 50% 59% 77% 79% 82%
with his/her
classmates
5.The comments and 12% 18% 24% 29% 38% 58% 60% 64%
ideas are relevant to
the examples being
analyzed
6.The ideas and 6% 11% 18% 23% 25.% 33% 39% 41%
comments regarding
the examples analyzed
are supported
Table 2 provides a comprehensive overview of the statistical values derived from

the checklist used during the observation. The findings reveal a significant
positive shift in various aspects related to students' participation, namely their
interest, interaction, and contribution. This notable increase can be attributed to
the substantial rise in the number of participants in each class, particularly in the
final session, where the participation rate reached its highest level.
Taking as a reference the first and last class, in the first statement (students
participate actively) it can be observed an increment of 72%. Regarding the second
statement (the student shows interest in participating) the difference is 73%.
Referring to the third statement (the student interacts with the teacher) the
progress is shown in 68%. Concerning the fourth statement (the student interacts
with his/her classmates) it is observed a development of 61%. As for the fifth
statement (the comments and ideas are relevant to the examples being analyzed)
it is shown an advance of 52%. Finally, concerning the sixth statement (the ideas
and comments regarding the examples analyzed are supported) there is a
difference of 35%.
Comparing the first and last class, significant improvements can be observed
across various statements related to student participation. In the first statement,
which measures active student participation, there has been an impressive
increase of 72%. Similarly, in terms of the second statement measuring student
interest in participating, the difference reflects a substantial growth of 73%. The
third statement, assessing student interaction with the teacher, exhibits notable
progress with a positive change of 68%. Additionally, the fourth statement
measuring student interaction with classmates displays a commendable
development of 61%. When it comes to the fifth statement, which determines the
relevance of comments and ideas to the examples being analyzed, there is a
significant advancement of 52%. Finally, concerning the sixth statement
evaluating the support for ideas and comments regarding the analyzed examples,
a noticeable difference of 35% is evident, indicating an encouraging improvement.
332
4.2. English practice teachers’ perceptions of the use of virtual worlds as a

resource for learning
Table 3. Perceptions of the use of virtual worlds as a resource for learning
N. Item Totally Agree Disagree Totally

agree disagree
1 The use of virtual environments 84% 10% 6% 0%

motivated me to learn.
2 My interest towards learning 22% 74% 4% 0%

increased due to the use of
virtual environments.
3 The virtual environments 42% 55% 2% 0%

allowed me to understand the
topic in a better way.
4 Students-students and teacher- 69% 28% 3% 0%

students interaction improved
through the use of virtual
environments.
5 The class development was 48% 52% 0% 0%

more dynamic through the use
of virtual environments.
6 The sense of presence provided 85% 15% 0% 0%

by the platform motivated me to
participate in the activities.
7 The use of virtual worlds is an 60% 35% 5% 0%

innovated practice for my
learning.
8 Access to virtual worlds was 60% 30% 10% 0%

easy and intuitive.
9 The activities developed in 45% 53% 2% 0%

virtual worlds
contributed to the learning of
the subject.
The survey results obtained from the participants demonstrate a high level of
agreement regarding the positive impact of virtual worlds on various aspects of
their learning experience. Notably, 94% of participants (84% totally agreed and
10% agreed) acknowledged that virtual worlds significantly influenced their
motivation to study the subject matter. Furthermore, in terms of practice teachers'
interest in the subject, 22% of them totally agreed and 74% agreed that their
interest increased when utilizing virtual worlds. When considering the
participants' understanding of the covered topics, 42% of students totally agreed
and 55% agreed that virtual environments had a notable impact on their
333
comprehension. Regarding interaction, 69% of participants totally agreed and

28% agreed that the use of virtual worlds improved interaction between teachers
and students as well as among students themselves. The majority of pre-service
teachers (100%) agreed that virtual worlds made classes more dynamic and
interesting, where 48% totally agreed and 52% agreed. All practice teachers (100%)
expressed eagerness to participate in the various proposed activities due to the
motivation provided by real-time simulated classes, with 85% totally agreeing
and 15% agreeing. In relation to the perception of virtual worlds as an innovative
learning method, 60% of participants totally agreed and 35% agreed. Similarly,
90% of pre-service teachers found the access and use of the platform to be easy
and intuitive, with 60% totally agreeing and 30% agreeing. Lastly, 98% of
participants (45% totally agreed and 53% agreed) believed that virtual worlds
helped increase their knowledge, emphasizing the positive impact of these
platforms on their learning outcomes.
5. Discussion
In this section, we analyze qualitatively the previous results to delve and examine
the complex dynamics that shape the experiences of English practice teachers´
participation during synchronic activities as well as their perceptions regarding
the use of virtual worlds as a resource for learning.
5.1 Influence of virtual worlds on English practice teachers’ participation

during synchronic activities.
According to the results, it is evident that virtual worlds impacted positively the
participation of English pre-service teachers in the synchronous activities since
they showed a progressive improvement in both quantity and quality aspects.
Concerning quantity, observations revealed that students increased their
participation frequency, primarily due to two factors: the physical presence
offered by the platform and the use of avatars to represent themselves, which are
key features of the platform. The table below showcases chosen quotes from the
interviews, discussing students' views on how physical presence impacts their
participation levels.
Table 4. Student’s perceptions on how physical presence impacts their participation

levels
Question Teacher’s excerpt
How you “… In fact, the physical presence of avatars has significantly
think the enhanced my engagement during synchronous activities…”
physical
presence and “... Having a visual representation through avatars creates a
the use of sense of presence, making me feel more connected and
avatars has involved in the virtual environment…”
helped you to
increase your “…This immersion has boosted my active participation and
participation overall enthusiasm in the activities.”
in the
synchronous
activities?
334
In fact, these features helped students feel as if they were in a physical

environment, enabling them to actively participate and interact with their teacher
and classmates by asking questions, expressing their opinions, and providing
support among them. In this context, Occhioni and Paris (2021) suggest that
virtual worlds can serve as an effective platform for educational institutions to
involve students in exploratory and collaborative activities, providing them with
a sense of "presence" and togetherness. Regarding avatars, Linganisa, et al. (2018)
state that students found value in using avatars to represent their identities in
virtual worlds. Gadille et al. (2023) highlight that in virtual educational
environments, the avatar serves as an integral component in the framework of
empathy and detachment to facilitate situated learning. Besides, students,
represented through their avatars, have the freedom to explore and interact with
objects from various angles, leading to increased active participation (Krassmann
et al., 2021).
With respect to the quality of practice English teachers´ participation, it was

observed a notable improvement because of the platform's facilities that enable
participants to enhance synchronous communication among them. This
innovative feature fosters active engagement, collaboration, and meaningful
interactions, ultimately leading to a more dynamic and enriching learning
environment. This characteristic (synchronous communication) allowed teachers
to provide immediate and progressive feedback, which helped students gain
confidence in their participation. Consequently, the quality of their contributions
improved progressively, becoming more relevant and well-supported. In this
regard, Girvan and Savage (2019) mention that VW offers learners the chance to
receive feedback from their peers and teachers which enables them to contribute
to the creation of their projects.
5.2 English practice teachers’ perceptions of the use of virtual worlds as

a resource for learning
The results from the survey clearly demonstrated that most participants held a
positive perception regarding the use of virtual worlds, as it significantly
increased their interest and motivation to study the subject content. This finding
is consistent with the research by Kamińska et al. (2019) acknowledge that virtual
worlds used for educational purposes have the potential to promote motivation,
engagement, and active learning in an evolving educational environment.
Additionally, the participants reported that the activities conducted through
virtual worlds greatly enhanced their understanding of the topics covered in
synchronous activities. These factors collectively contributed to the improvement
of the participants' learning and the development of competencies related to the
subject. In this respect, Ghanbarzadeh and Ghapanchi (2021) found that utilizing
virtual worlds improves learning outcomes compared to traditional methods,
resulting in enhanced educational achievements.
Findings regarding the simulated classes conducted in real-time through video

collaboration sessions, a prominent feature of virtual worlds, revealed their
significant impact on students. These dynamic sessions effectively motivated
335
student participation and facilitated increased interaction between teachers and

students, as well as among students themselves. Ghanbarzadeh and Ghapanchi
(2020) state that VW have the potential to be used for education, enabling practical
problem-solving activities between students and teachers, and making a positive
societal impact. Furthermore, Ferreira and Ribeiro (2022) emphasize that body
movement in immersive environments enhances student learning by providing
an immersive multimodal experience that allows them to engage with virtual
worlds dynamically, interactively, and autonomously. The table below features
excerpts from interviews with students discussing their perspectives on virtual
worlds as didactic resources and innovative educational practices.
Table 5. Student’s perceptions about virtual worlds on virtual worlds as didactic

resources and innovative educational practices.
Question Teacher’s excerpt
“… Virtual worlds captivated me as a learner, fostering active
engagement through immersive experiences.”
“Virtual worlds facilitate interactive learning, encouraging

collaboration and hands-on exploration…”
What do you think of
the use of virtual “Virtual worlds enhanced my engagement to the class
worlds as a didactic through interactive and immersive experiences…”
resource for learning
“…Virtual worlds have enabled students from different cities
to collaborate in real-time, which fosters global
communication and cooperation, breaking down geographical
barriers…”
Do you think that the “…Virtual worlds has been an innovative practice because
use of virtual worlds they offer immersive and interactive experiences, capturing
constitutes an students' attention and keeping us engaged in the learning
innovative practice? process…”
“I think that Virtual worlds facilitate experiential learning,

students can actively participate and learn by doing. This
hands-on approach can enhance comprehension and critical
thinking skills.”
Regarding the use of virtual worlds as a didactic resource in synchronous classes,

a high percentage of participants perceived it as an innovative practice that
significantly enhanced their learning experience. In this regard, Ghanbarzadeh
and Ghapanchi (2021) assert that the use of VW in teaching is an innovative and
creative way to positively impact student learning. According to the survey
results, students expressed that the synchronous activities conducted in the
distance program on a regular basis did not allow them to use avatars. However,
virtual worlds provided distance students with the ability to utilize avatars, which
instilled a sense of virtual presence and increased their confidence to actively
participate in these activities. These results are aligned with the study conducted
by Irwin et al. (2019) in which, learners who embodied idealized avatars showed
more enjoyment and engagement with course materials. In terms of access and
usability, participants expressed a consistently positive opinion, asserting that the
336
virtual worlds tool was easy to use and intuitive. Moreover, they did not
encounter any technical difficulties when accessing the platform, which can be
attributed to the training provided at the beginning of the intervention.
6. Conclusion
The use of virtual worlds has been a fundamental factor in enhancing learner
participation during synchronous activities, leading to both increased quantity
and improved quality of engagement over time. This has positively impacted
cultivating students' enthusiasm for active involvement. The success of this
platform can be attributed to its features and the comprehensive training sessions
provided to students at the outset of the intervention. Physical presence and the
use of avatars have emerged as critical factors in fostering student participation,
enhancing motivation, and sustaining complete engagement. Importantly, our
study found that English pre-service teachers held positive perceptions of virtual
worlds, demonstrating heightened interest and motivation to learn the contents.
Their experience with VW yielded rewarding results, enabling both academic and
technological growth. Furthermore, the use of virtual environments significantly
improved students' comprehension of the presented content during synchronous
activities, resulting in enhanced subject knowledge and competencies.
Despite these positive results, the present research was limited by the low-quality
Internet access that some students experienced during the synchronic activities,
which influenced their experiences.
The pedagogical implication of the present research relies on the integration of
immersive and interactive technology into teacher training and professional
development. This could be achieved through the creation of virtual classrooms
where pre-service and in-service teachers can practice their teaching skills, engage
with students, and receive feedback in a safe and controlled digital space.
Acknowledgments
We would like to thank Universidad Técnica Particular de Loja authorities for
supporting research through the EFL Learning, Teaching and Technology
Research Group. We also thank all the participants who kindly responded to the
questionnaires.
7. References
Alshumaimeri, Y., Gashan, A., & Bamanger, E. (2019). Virtual worlds for collaborative
learning: Arab EFL learners' attitudes toward Second Life. World Journal on
Educational Technology: Current Issues, 11(3), 198-204.
https://doi.org/10.18844/wjet.v11i3.4235
Badem-Korkmaz, F., & Balaman, U. (2022). Eliciting student participation in video-
mediated EFL classroom interactions: Focus on teacher response-pursuit
practices. Computer Assisted Language Learning, 1-23.
https://doi.org/10.1080/09588221.2022.2127772
Bagrıacık Yılmaz, A. (2019). Distance and face-to-face students’ perceptions towards
distance education: A comparative metaphorical study. Turkish Online Journal of
Distance Education, 20 (1), 191-207. https://doi.org/10.17718/tojde.522705
Bartle, R. (2003). Designing virtual worlds. New Riders Publishing
Berge, Z. (1995). Facilitating computer conferencing: Recommendations from the field.
Educational Technology, 35(1). 22-30. http://www.jstor.org/stable/44428247
337
Bojanova, I. (2011). Physical Presence through Virtual Worlds. IT Professional, 13(4).

https://doi.org/10.1109/MITP.2011.66
Casey, D. (2008). A Journey to legitimacy: The historical development of distance
education through Technology. TechTrends, 52(2), 45-46.
http://dx.doi.org/10.1007/s11528-008-0135-z
Chen, Y., L. (2016). The effects of virtual reality learning environment on student cognitive
and linguistic development. The Asia-Pacific Education Researcher, 25, 637-646.
https://doi.org/10.1007/s40299-016-0293-2
Contreras, J., Sarango, P., Jara, I., Agila, M., Guaman, J. & Samaniego, J. (2018).
Implementation of virtual worlds in distance studies. In 13th Iberian Conference on
Information Systems and Technologies. CISTI. 1-6.
https://doi.org/10.23919/CISTI.2018.8399146.
Creswell, J. (2015). Educational Research Planning, Conducting and Evaluating Quantitative
and Qualitative research. Pearson.
Demirbilek, M., & Koç, D. (2021). In Their Own Words: Teachers Experiences and
Activities in 3D Virtual Worlds for Foreign Language Teaching. In ICTERI 217-
225. https://ceur-ws.org/Vol-3013/20210217.pdf
Davies, J., & Graff, M. (2005). Performance in e-learning: online participation and student
grades. British Journal of Educational Technology, 36(4), 657–663.
https://doi.org/10.1111/j.1467-8535.2005.00542.x
Díaz, J., Saldaña, C. & Ávila, C. (2020). Virtual World as a Resource for Hybrid Education.
International Journal of Emerging Technology in Learning. 15(15), 94-109.
https://www.learntechlib.org/p/217986/
Ferreira, M. C., & Ribeiro, P. N. (2022). The body movement in ESL/EFL vocabulary
learning in immersive environments. Ilha do Desterro, 74, 369-391.
https://doi.org/10.5007/2175-8026.2021.e80715
Francke, E., & Alexander, B. (2018, October). Educating the Millennial Learner: A case of
Collaborative Learning with augmented reality. In ECGBL 2018 12th European
Conference on Game-Based Learning. Academic Conferences and publishing limited.
Gadille, M., Corvasce, C., & Impedovo, M. (2023). Material and Socio-Cognitive Effects of
Immersive Virtual Reality in a French Secondary School: Conditions for
Innovation. Education Sciences, 13(3), 251.
Girvan, C. & Savage, T. (2019). Virtual worlds: A new environment for constructionist
learning. Computers in Human Behavior, 99, 396–414.
https://doi.org/10.1016/j.chb.2019.03.017
Ghanbarzadeh, R., & Ghapanchi, A. (2020). Antecedents and consequences of user
acceptance of three-dimensional virtual worlds in higher education. Journal of
Information Technology Education: Research, 19, 855- 889.
https://doi.org/10.28945/4660
Ghanbarzadeh, R. & Ghapanchi, A. (2021). Uncovering educational outcomes deriving
from students’ acceptance and involvement with 3D virtual worlds. Educ Inf
Technol 26, 311–337. https://doi.org/10.1007/s10639-020-10272-7
Girvan, C. (2018). What is a virtual world? Definition and classification. Education Tech
Research Dev, 66 (5), 1087–1100. https://doi.org/10.1007/s11423-018-9577-y
Gregory, S., Gregory, B., Jacka, L., Lierse, S., McDonald, M., Ellis, D., ... & Gaukrodger, B.
(2019). Vision-A space for digital learning and exploring pedagogies: Virtual
world education. In Proceedings of the 36th International Conference on Innovation,
Practice and Research in the Use of Educational Technologies in Tertiary Education
(ASCILITE 2019). University of Southern Queensland.
338
Hanh, N. T. (2020). Silence is gold?: A Study on Students' Silence in EFL Classrooms.

International Journal of Higher Education, 9(4), 153-160.
https://doi.org/10.5430/ijhe.v9n4p153
Hedberg, J. & Brudvik, O. (2008). Supporting dialogic literacy through mashing and
modding of places and spaces. Theory into Practice, 47(2), 138–149.
https://doi.org/10.1080/00405840801992363
Holmberg, B. (1989). Theory and practice of distance education. Routledge.
Irwin, P., Coutts, R., & Graham, I. (2019). Looking good sister! The use of a virtual world
to develop nursing skills. In Intersections in Simulation and Gaming: Disruption
and Balance: Third Australasian Simulation Congress, ASC 2019, Gold Coast,
Australia, September 2–5, 2019, Proceedings 3 (pp. 33-45). Springer Singapore.
Jacka, L. (2018). Using virtual worlds in educational settings: Making learning real. Routledge.
Keegan, D. (1996). The foundations of distance education (3rd ed.). Croom Helm.
Kahai, S., Jestice, R., & Huang, R. (2023). Gender Effects in Directed versus Incidental
Learning in a 3D Virtual World Simulation. AIS Transactions on Human-Computer
Interactions, 15(1), 55-82. https://doi.org/10.17705/1thci.00183
Kamińska, D., Sapiński, T., Wiak, S., Tikk, T., Haamer, R. E., Avots, E., ... & Anbarjafari,
G. (2019). Virtual reality and its applications in education: Survey. Information,
10(10), 318. https://dx.doi.org/10.3390/info10100318
Kim, J. (2013). Influence of group size on students' participation in online discussion
forums. Computers & Education, 62, 123-129.
Koster, R. (2004, January 07). A virtual world by any other name? [Blog post]. Retrieved from
http://terranova.blogs.com/terra_nova/2004/06/a_virtual_world.htm
Krassman, A., Mazzuco, A., Melo, M., Bessa, M. & Bercht, M. (2020). Usability and Sense
of Presence in Virtual Worlds for Distance Education: A Case Study with Virtual
Reality Experts. In Proceedings of the 12th International Conference on Computer
Supported Education (CSEDU 2020) 1,155-162.
https://doi.org/10.5220/0009350401550162
Krassmann, A. L., Nunes, F. B., Flach, J. M., Tarouco, L. M. R., & Bercht, M. (2021). A
framework to prepare the application of virtual worlds in distance education in
developing countries. Universal Access in the Information Society, 20, 451-463.
https://doi.org/10.1007/s10209-020-00771-2
Linganisa, A., Ako-Nai, A., & Ajayi, N. (2018). The Potential of Second Life as a Platform
for Learning: Student’s Perspective. 2018 International Conference on Intelligent
and Innovative Computing Applications (ICONIC).
https://doi.org/10.1109/iconic.2018.8601201
Makarenya, T., Stash, S. & Nikashina, P. (2020) Modern educational technologies in the
context of distance learning. Journal of Physics: Conference Series, 1691(1), 1-5. Doi
10.1088/1742-6596/1691/1/012117
Martínez, C. (2008). La educación a distancia: Sus características y necesidad en la
educación actual. Educación, 33(17), 7-27.
https://doi.org/10.18800/educacion.200802.001
Martínez, R. (2012). Mundos virtuales 3D: Una guía para padres y formadores. Editorial UOC.
Mertler, C. & Charles C. (2008). Introduction to educational research. Pearson Education.
Mørch, A. I., Caruso, V., Hartley, M. D., & Ludlow, B. L. (2018). Creating Contexts for
Collaborative Learning in a 3D Virtual World for Distance Education. Integrating
Multi-User Virtual Environments in Modern Classrooms. 137-164.
https://doi.org/10.4018/978-1-5225-3719-9.ch007
339
Nørgaard, C., O'Neill, L. D., Nielsen, K. G., Juul, S., & Chemnitz, J. (2018). Learning
Anatomy with Augmented Reality, In Proceedings of the 10th Annual International
Conference on Education and New Learning Technologies.
Occhioni, M., & Paris, E. (2021). Virtual Worlds to Teach Sustainability Topics in Distance
Learning. Academic Conferences International Limited.
https://doi.org/10.34190/EEL.21.133
Oliveira, M., Penedo, A., & Pereira, V. (2018). Distance education: advantages and
disadvantages of the point of view of education and society. Dialogia, 29, 139-
152. https://doi.org/10.5585/dialogia.N29.7661
Peterson, M. (2011). Towards a research agenda for the use of three-dimensional virtual
worlds in language learning. Calico Journal, 29(1), 67 – 80.
http://www.jstor.org/stable/calicojournal.29.1.67
Rajesh, M. (2015). Revolution in communication technologies: impact on distance
education. Turkish Online Journal of Distance Education, 16(1), 62-88.
Richardson, J., & Swan, K. (2003). Examining social presence in online courses in relation
to students' perceived learning and satisfaction. Journal of Asynchronous Learning
Networks, 7(1), 68-88. https://doi.org/10.24059/olj.v7i1.1864
Savin-Baden, M. (2010). A practical guide to using Second Life in higher education. McGraw-
Hill Education.
Saykılı, A. (2018). Distance education: Definitions, generations, key concepts and future
directions. International Journal of Contemporary Educational Research, 5(1), 2-
17. https://files.eric.ed.gov/fulltext/EJ1207516.pdf
Schlosser, L. A., & Simonson, M. (2009). Distance education: Definition and glossary of terms.
(3rd ed.). Charlotte, NC: Information Age.
Shafieiyoun, S., & Safaei, A. (2013). Enhancing learning within the 3-D virtual learning
environment. Journal of Knowledge. Management, Economics and Information
Technology 3(3), 1-6.
https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=aaaac8bece
14c9963c40e3e153595c24772d0f77
Simovska, V. (2007). The changing meanings of participation in school-based health
education and health promotion: the participants' voices. Health Education
Research, 22(6), 864-878. https://doi.org/10.1093/her/cym023
Sparapani, N., Tseng, N., Towers, L., Birkeneder, S., Karimi, S., Alexander, C. J., ... &
Nunnally, A. D. (2023). Factors associated with classroom participation in
preschool through third grade learners on the autism spectrum. Research in Autism
Spectrum Disorders, 105, 102186. https://doi.org/10.1016/j.rasd.2023.102186
Vásquez, R., Bongianino, C., & Sosisky, L. (2006). La tecnología educativa y la educación a
distancia. Educación a distancia de posgrado y pregrado. Universidad Nacional de
entre Ríos. Argentina
Yu, L. T., Jayoung, S., & Fu-Yuan, Ch. (2020). Using a three-dimension virtual world to
reduce language anxiety and enhance English-speaking performance of EFL
university learners: A collaborative project. Taiwan Journal of TESOL, 17, 65-89.
https://doi.org/10.30397/TJTESOL.202010_17(2).0003
Zhang, Ch. (2009). Understanding the impact of virtual world environments on social and
cognitive processes in learning. University of Nebraska at Omaha.
Vol. 22, No. 10, pp. 340-358, October 2023
Received Aug 21, 2023; Revised Oct 24, 2023; Accepted Nov 1, 2023
TikTok as a Source of English Language Content

– Perceived Impacts on Students’ Competence:
Views from Indonesia
Fathor Rasyid* , Hidayatul Hanjariyah and Nurul Aini
State Islamic Institute of Kediri
East Java, Indonesia
Abstract. This study aimed to investigate the students' perceptions of the

impacts of English content shared via TikTok on their English
competence and examine the reasons for their perceptions. Two hundred
and seven English students consisting of 117 females and 90 males in East
Java, Indonesia with prior high experience of watching English content
shared via TikTok participated in the study. The participants included 52
high school students, 97 university English students, and 58 English
course students with diverse backgrounds. A wide range of perspectives
was obtained from the research instrument, namely a four-point Likert-
scale questionnaire. This was used to elicit quantitative data on students'
perceived impacts of English content shared via TikTok on their English
competence as well as the underlying reasons for their perceptions. The
questionnaire consisted of 20 statements and was developed from four
subscales of the perception construct. Quantitative data were analyzed
through frequency and percentage, while qualitative data were analyzed
based on themes. The findings revealed that students have positively
competence. This positive perception ranged from 60.1% to 94.7%,
implying that research participants responded positively to all
questionnaire items. This suggests that English content shared via TikTok
contributes to students' English competence. In addition, four themes
emerged in the qualitative analysis, namely attractiveness, effectiveness,
relevance, and the motivation aspect. The data indicated that students
have positive reasons for their perceptions. Therefore, the results of both
the qualitative and quantitative data analysis suggest that English
content shared via TikTok is beneficial for improving students’ English
competence.
Keywords: English content; competence; perceived impact; views;

TikTok
* Corresponding author: Fathor Rasyid. Email: frasyid88@gmail.com
©Authors
341
1. Background of the Study

The emergence of TikTok in the modern era and social media causes significant
shifts in society in general and in education in particular. Nowadays the use of
social media has become a lifestyle and is thus more widely and increasingly used.
Originally, online learning systems making use of internet technology were used
to deliver learning content and to facilitate teacher-learner interactions. Students
learn by reading and completing tasks, while teachers use online learning systems
to check and evaluate the quality of students' assignments. Synchronous or
asynchronous learning then became more popular across space and time,
subsequently emerging as a popular trend (Bizel et al., 2022).
A plethora of research studies (e.g., Farahat, 2012; Faizi, et al., 2013; Kemp, 2017;
Jaffar et al., 2019; Kolhar et al., 2021; Bizel et al., 2022; Guru & Sambandam, 2022;
Kılınç, 2022) have been conducted on how and why TikTok has generated massive
interest among people, and how it affects other aspects of life, including
education. It is evident that TikTok users have increased significantly and it is
estimated that 50% of TikTok users globally are under the age of 34 with 32.5%
aged between 10 and 19 (Aslam, 2022). Therefore, TikTok receives special
attention from educational researchers. For example, Jaffar et al. (2019) examined
how TikTok could be used in classrooms as a tool to illustrate a concept or
example. Its use has become more widely spread as it has many features that other
social media platforms have not been able to provide. Hence, it has become a
serious consideration among social media users. Moreover, TikTok could be used
as an instructional medium in classrooms (Dewanta & Bahasa, 2020; Guru, 2022).
In Indonesia, internet technology has grown rapidly, becoming a lifestyle for its
citizens. It was estimated that 50% of Indonesians already use internet (Kemp,
2017) and the number is steadily increasing. It is widely used in all forms of social
media; therefore, social media users, including those using TikTok, are increasing
continuously. Kemp (2018) highlighted that Indonesia ranked third among social
media users globally. Indonesians use it for various purposes: communication,
entertainment, and the like. Moreover, it is evident now that TikTok has become
a more popular means of communication and interaction; recently it is also used
more in educational settings. This implies that students and teachers have
adopted and integrated social media into their teaching and learning. This is
undoubtedly due to the assorted features and functions of social media (Nichita
et al., 2021; Puspitasari, 2021). Hence, making use of social media has become an
educational trend.
There are four main purposes why people make use of social media applications.
They are entertainment, socialization, information, and academics (Smith & Short,
2022; Yang, 2020). From an educational point of view, for instance, social media
or TikTok, in particular, could be used as an educational tool for content-sharing
among users as it offers various contents, sharing media content, and facilitating
interaction and communication among users (Bernadette, 2020; Marta & William,
2016; Saputra et al., 2021). Content sharing among users takes place with a single
click while maintaining the speed of sharing information (Kolhar et al., 2021; Lai
& Tai, 2021).
342
As a video application, TikTok has several functions through entertainment

videos, making it easier for users to watch. However, although there is a large
number of users from different fields such as product promotion, from the
viewpoint of education, it has some limitations since it was not created specifically
for educational purposes (Basch et al., 2022; Scherr & Wang, 2021). However, it
still makes a potential contribution, and should be used as an alternative
educational tool. A large number of students are now using it. Students believe
that using TikTok is convenient, simple, and accessible from any place. Nuari
(2022) argued that the TikTok application satisfies students' learning needs. Based
on the relevant advantage of TikTok on students’ learning, this study aimed to
investigate the perceived impacts of English content shared via TikTok on
students’ English competence in Indonesia by explicitly answering the following
questions:
1) What are students’ perceived impacts of TikTok English-content-sharing on
their English competence?
2) What are the students’ views towards their perceived impacts of TikTok?
2.1. Social Media Interaction Theory
Collaborative learning, as stated by Zheng et al. (2014), is a teaching pedagogy by
means of which learning occurs through social interaction using internet
technology among students and teachers. Research studies report that there have
been increasing numbers of technology use and pedagogies to improve the
quality of teaching and learning to facilitate better collaborative learning, to
exchange information and knowledge, to gain better access to education and
teaching, and to ensure life-long learning and learning autonomy (Farahat, 2012;
Park, 2009). These all are intended to facilitate learning and to improve the cost-
effectiveness of educational services. Furthermore, studies conducted by Bertea
(2009) and Shen et al. (2006) highlight that students learn learning content digitally
partly because of social influences which later influence their attitude. This is
called behavioral intention which becomes a significant construct for using
technology along with the experience of prior usage of technology (Šumak et al.,
2011). The underlying determinant of acceptance of online learning among
students is the usefulness and ease of technology, as attitudes and social influence
factors. In addition, available literature reviews indicate that TikTok has many
features, enabling content creators to create abundant impactful messages and
content. TikTok contents vary in degrees of content, seriousness, humor, and
ambiguity (Villa-Ruiz et al., 2021; Zihan, 2022; Wengel et al., 2022). Recently,
TikTok has gained much attention and has become a means of learning and
teaching, making it more popular among children and teenagers as well as
students. Therefore, educators and language teachers need to make use of and
integrate this new lifestyle into their teaching-learning process.
2.2. Perceived Usefulness and Ease of Use
Venkatesh and Davis (2000), Alharbi and Drew (2014) proposed the technology
acceptance model (TAM) which is generated from the theory of reasoned action
(TRA). For them, there are two external factors affecting users’ acceptance of
technology in general and TikTok in particular, namely perceived usefulness and
perceived ease of use. These two contribute to users' behavioral intentions.
343
According to Fishbein and Ajzen (2009), TRA becomes the influencing process of
people's behavior, and this in turn becomes the underlying theoretical basis of
TAM. Moreover, studies conducted by Lee (2010) and Cheon et al. (2012)
proposed that the greater the intention of users to use a new technology, the
greater the frequency as well as the number of instances of actual use of the
intended technology.
Thus, perceived usefulness is simply the perception of someone who believes the
new product or technology could effectively improve the completion of a specific
task using the new technology (Huang, 2021). In other words, they contend that
the new technology is believed to be beneficial to accomplish certain tasks by
implementing that technology. The theory of perceived usefulness is thus the
extent to which a technology is expected to meet users’ performance expectations.
It indicates that the more beneficial the user perceives a certain technology to be,
the more positively they view that technology. On the other hand, the perceived
ease of use is simply the extent to which a person believes that the use of a product
or new technology will be effort free. To verify this theory, the new technology
should be easy to use, and easy to learn (Shen & Huang, 2020). Moreover, the
easier the technology is to use, the more willing users will be to use it and the
more positive an attitude users will have toward the new technology.
2.3. Continuous Usage Intention
Another underlying theoretical framework of the study is expectation

confirmation theory. This theorizes that user satisfaction will affect their
willingness to reuse a product in the future (Oliver, 1980). That is to say, when
users use a certain product – in this case new technology – and it meets their
expectations, they will be willing to continue using it. In terms of the theory of
expectation confirmation theory, Bhattacherjee (2001) contended that the decision
of users to continue using a certain product or new technology application is more
or less the same as consumers repurchasing a certain product since it satisfies
them. The continuous use of information applications depends very much on
users' satisfaction as well as whether it meets users’ expectations. This normally
results in positive perceived usefulness of the application, eventually resulting in
a stronger intention to make use of the application in the future. When users have
an intention, they tend to act; therefore, users' behavior could be predicted from
their intention (Huang, 2021). Therefore, continuous usage intention becomes an
important factor for the continuous use of the application and its success.
Therefore, the strength of their continuous use intention determines every user's
decision to continue using the application or not (Hong & Tam, 2006; Thong et al.,
2006).
3. Methodology
3.1. Research Design
This study used a survey research design to explore the perceived impacts of
English content shared via TikTok on students’ English competence in Indonesia.
Survey design is appropriate for investigating a large number of people in order
to describe their attitudes, opinions, behaviors, or characteristics (Ary et al., 2010;
Creswell, 2012). A survey is used to provide useful information, describe trends,
344
and reveal people's perceptions or opinions, as well as assess people's views

regarding their perceptions (Creswell, 2014; Fraenkel et al., 2012). Hence, the
study employed mixed methods since it utilized and elicited both quantitative
and qualitative data.
3.2. Participants
Two hundred and seven English students in East Java, Indonesia participated in
the study. They comprised one hundred seventeen females, and ninety males with
prior experience of watching English content shared via TikTok. They had varied
backgrounds as English learners to ensure a wide range of perspectives. They
were ninety seven English department students, fifty eight following English
course programs, and fifty two senior high school students making up the
composition of the 207 participants as depicted in Table 1:
Table 1: Composition of participants
No. Participants Total
1 Senior High Students 52
2 English Course Students 58
3 University English Students 97
Total 207
3.3. Instruments
The survey questionnaire used in the study was written in both English and
Bahasa Indonesia to make it more understandable to the students, and to avoid
misunderstanding and misinterpretation. The statements on the questionnaire
cover the students' perceived impacts of English content shared via TikTok on
their English competence and their views on the perceived impacts. The
questionnaire consisted of two parts: the first part was about respondents'
demographic information while the second consisted of 20 statements using a
four-point Likert scale. These statements were related to students’ perceptions of
the impact of TikTok English content on their English competence, and their views
towards these perceived impacts of TikTok.
The 20 items were developed from two scales of the perception construct: the
positivity of using TikTok and the negativity of the students’ perceptions of using
TikTok. The 20-item questionnaire consists of ten positive statements and ten
negative statements. The former ten items were further divided into four
subscales: attractiveness, effectiveness, relevance, and motivation using TikTok
(Horton, 2003). The attractiveness aspects are adapted from Balbay and Kilis
(2017), while the remaining aspects were adapted from Abdu-Raheem (2015). The
latter ten items were further divided into five components, namely distraction,
addiction, time-consuming, bullying, and unsecured privacy.
Each statement required participants to provide their attitudes or views on the
competence. In order to assure the validity of the questionnaire, the researchers
developed the blueprint presented in Table 2 and developed questionnaire items
based on it:
345
Table 2: Blueprint of questionnaire

Construct Indicators Component No.
items
Fun English content learning 1,2
Attractiveness
Enhancing English 3,4
aspect
competency
Effectiveness Faster information 5
aspect Flexibility of learning 6
Relevance The influence of TikTok Apps 7
Perceived impacts aspect The use of TikTok apps 8
of TikTok English The ease to access 9
content on their Motivation
The ease to explore deep 10
English aspect
explanation
competence
Distraction 11, 12
Negative Addiction 13, 14
perception Time-consuming 15,16,
aspect 17
Bullying 18
Privacy not secure 19, 20
3.4. Data Collection

This research was conducted to gain the students' perceived impacts of English
content shared via TikTok on their English competence and their views of their
perceived impacts. The questionnaire was administered on a large scale to many
prospective participants. They were all TikTok users making use of English
content shared via TikTok. A large number of respondents filled in the
questionnaire, while the incomplete questionnaires were discarded. After
screening, there were 207 completed questionnaires. From these questionnaires,
the quantitative data on students' perceived impacts of English content shared via
TikTok on their English competence as well as students' reasons for their
perceptions were analyzed. Finally, triangulation was done for the
trustworthiness of the data. Two triangulation procedures were used, namely
data source triangulation and finding triangulation. The former was intended to
gain plausible data, while the latter was used to gain accurate interpretation by
collaborating with other investigators.
3.5. Data Analysis

To determine the strength level of perception for each statement (very high, high,
low, and very low), the following criteria were used to determine the strength
level of perception:
Table 3: The strength of the students’ perceived impacts

No. Rating Percentage
1 Very low 0-25
2 Low 26-50
3 High 51-75
4 Very high 76-100
346
On the other hand, qualitative data were analyzed inductively based on subscales
of the constructs used in the instrument. First, the collected qualitative data were
grouped and classified based on the theoretical framework (subscales). Second,
the data were then interpreted to determine the underlying reasons for their
perceptions. Finally, the interpretation or finding or investigator triangulation
was made for its trustworthiness.
4. Findings
4.1. Students’ Perceived Impacts of TikTok English Content on their English
Competence
Table 4: Students’ perceived impacts of TikTok English content on their English

competence
Scales
No. Statements
SA A D SD
Being interested in educational 86 111 10 0
1.
English content on TikTok (41.3%) (53.4%) (4.8%) (0%)
57 127 22 1
2. Exploring English content videos
(27.4%) (61.1%) (10.6%) (0.5%)
41 119 45 2
3. Focusing on speaking skills
(19.7%) (57.2%) (21.6%) (1%)
Prioritizing grammar 28 97 75 7
4.
explanations (13.5%) (46.6%) (36.1%) (3.4%)
Being up-to-date and having more 65 115 25 2
5.
detail on English explanations (31.3%) (55.3%) (12%) (1%)
Being flexible to use any time and 74 115 18 0
6.
everywhere (35.6%) (55.3%) (8.7%) (0%)
Keeping me motivated to learn 48 134 19 6
7.
English (23.1%) (64.4%) (9.1%) (2.9%)
Fitting in my busy schedule for 62 125 18 2
8.
language learning (29.7%) (59.8%) (8.6%) (1%)
Providing in-depth English 38 119 46 4
9.
explanations (18.2%) (56.9%) (22%) (1.9%)
Being accessible on various 92 104 10 1
10.
devices (44.2%) (50%) (4.8%) (0.5%)
Distracting me from my other 18 68 108 13
11.
learning tasks (8.6%) (32.5%) (51.7%) (6.2%)
Being difficult to focus only on 33 78 88 8
12.
English videos (15.8%) (37.3%) (42.1%) (3.8%)
Getting bored with English 4 28 140 35
13.
content on TikTok (1.9%) (13.4%) (67%) (16.7%)
Struggling to stop scrolling on 37 102 54 14
14.
TikTok (17.7%) (48.8%) (25.8%) (6.7%)
39 87 68 13
15. Losing track of time on TikTok
(18.7%) (41.6%) (32.5%) (6.2%)
16 70 107 14
16. Disrupting my productivity
(7.7%) (33.7%) (51.4) (6.7%)
12 70 107 19
17. Finding it time-consuming
(5.7%) (33.3%) (51%) (9%)
13 54 116 24
18. Leading to cyberbullying
(6.2%) (25.8%) (55.5%) (11.5)
347
Not being suited to my learning 14 54 118 21

19.
style and needs (6.7%) (25.8%) (56.5%) (10%)
Having concerns about TikTok 26 83 91 7
20.
privacy and security (12.4%) (39.7%) (43.5%) (3.3%)
The results from the questionnaire consisting of 20 statements are presented in

Table 4. It presents the students' perceived impacts of English content shared via
TikTok on their English competence. This simple tabulation consists of
frequencies and percentages. Table 4 indicates that the 20 questionnaire items
could be categorized into four groups. The first are the attractiveness aspects of
English content shared via TikTok as indicated by questionnaire items 1 to 4.
When asked whether they are interested in watching English content learning on
TikTok for educational purposes, the result shows that there are 111 students
(53.4%) who agree, 86 students (41.3%) who strongly agree, 10 students (4.8%)
who disagree and none (0%) who strongly disagree. This indicates that most
students agree that they are interested in immersing themselves in English content
on TikTok for educational purposes as it helps them improve their language skills
in a fun and engaging way. Similarly, respondents' responses towards “I like to
explore or watch English content videos on TikTok as additional materials to
study” imply that they have a positive perception towards English content shared
via TikTok. A total of 127 respondents (88.5%) perceived that watching English
content on TikTok has become one of their favorite study tools as it has provided
them with a fresh perspective on the language while helping them to learn new
vocabulary and phrases that could be incorporated into their studies.
Furthermore, the majority of respondents responded positively that they would
rather watch a particular English video for enhancing their speaking ability than
other competencies. For this questionnaire item, it is interesting to observe that
there are 119 students (57.2%) who agree, 41 (19.7%) who strongly agree, 45 (4.8%)
who disagree and 2 (1%) who strongly disagree. Hence, they believe that watching
a particular English video repeatedly can be more effective in enhancing their
speaking ability as it allows them to pick up nuances in pronunciation and
intonation. Finally, the attractiveness aspect of English content shared via TikTok
is indicated by questionnaire item no. 4, namely “I would focus on finding
grammar explanations via TikTok”. The result shows that 60.1% of respondents
tend to have a visual learning style, making it more effective to focus on finding
grammar explanations via TikTok as the short, concise videos with clear examples
and graphics help them grasp complex concepts more easily.
The second classification relates to the effectiveness aspect of students' perceived
impacts of English-content sharing TikTok on their English competence as
indicated by questionnaire items 5 and 6, namely “Whether or not TikTok apps
help me to receive information faster and more detail in understanding English”.
The results reveal that, of 207 respondents, 180 or 86.6% perceived that by its vast
selection of content creators and short-form videos, TikTok has become an
invaluable tool for students in receiving information faster and gaining a more in-
depth understanding of the English language. Only 27 respondents or 13% had a
negative perception. More strikingly, when asked whether the flexibility of
TikTok assists them to learn English whenever and wherever, the results revealed
that most of them, namely 90.9%, considered that the flexibility of TikTok as a
348
mobile app has been a game-changer as it allows them to learn English at any
time or in any place, when they have a few spare minutes waiting in line or during
their daily commute.
The third is the relevance aspect of students' perceived impacts of English content
shared via TikTok on their English competence. The findings indicate that 87.5%
of respondents mention that interacting with English-speaking TikTok creators or
users can help them to stay motivated in learning English. Only 25 students
(12.5%) do not agree that “Engaging with English-speaking TikTok creators or
users has been a great way for students to stay motivated in my English learning
journey” despite the fact that it provides them with a sense of community and the
opportunity to practice their language skills with native speakers. Similarly, the
questionnaire item that states “TikTok’s short video format makes it easy to fit
language learning into my busy schedule” had a positive response from the
majority of respondents (175 or 89.5%), while only 20 respondents or 8.7%
responded negatively. It means the study found that students agreed that, despite
their hectic schedules as students, they could still watch and engage with English
content on the app., even during short breaks throughout the day.
The fourth category deals with the motivational aspect of students' perceived
impacts of English content shared via TikTok on their English competence. This
is indicated by questionnaire items no 9 and 10. Number 9 says “English content
on TikTok is a helpful source for deep explanation”. It is interesting to observe
that 75.1% of the respondents perceived that owing to the students’ perception,
English content on TikTok is not only entertaining but also a helpful source for
deep explanations on complex topics, as many creators use visual aids and
storytelling techniques to make difficult concepts more accessible and engaging.
In addition, questionnaire item 10 says “English content on TikTok is a highly
accessible platform through any variety of devices, including smartphones,
tablets, and laptops.” A total of 94.2% or 196 respondents agree that English
content on TikTok is highly accessible through a variety of devices, including
smartphones, tablets, and laptops, making it easy for students to access language
learning materials.
The last 10 of the questionnaire items are negative statements and can be classified
into five subcategories. Firstly, distraction is indicated by item numbers 11 and 12.
Respondents' responses to “The constant notifications after learning through
TikTok distract my other learning task” show that there are 68 students (32.5%)
who agree, 18 (8.6%) who strongly agree, 108 (51.7%) who disagree and 13 (6.2%)
who strongly disagree. This means that the fact that frequent interruption from
TikTok notifications annoys students is not valid owing to the high percentage of
respondents who answered “Disagree” to this item. Moreover, responses
regarding “The wide range of entertaining content on TikTok can make it difficult
for users to focus only on exploring English videos” indicate that there are 78
students (37.3%) who agree, 33 (15.8%) who strongly agree, 88 (42.1%) who
disagree and 8 (3.8%) who strongly disagree. Hence, the wide range of fun and
engaging content on TikTok did not pose a challenge for users of English videos
as the majority of students disagreed with the statement.
349
Secondly, addiction to TikTok is indicated by items 13 and 14. In response to

“English content on TikTok is boring”, it was found that 13.4% agree, 1.9%
strongly agree, 67% disagree and 16.7% strongly disagree. This implies that
TikTok's English videos are not lacking in excitement and creativity as the
majority of students disagree with the statement as reflected in the high
percentage of disagreement. Responses to item number 14, namely “I find it
difficult to stop scrolling on TikTok” reveal that 48.8% agree, 17.7% strongly agree,
25.8% disagree and 6.7% strongly disagree. The conclusion is that students find
TikTok English content fascinating and absorbing.
Thirdly, items 15, 16, and 17 are related to the time-consuming aspect of TikTok.
The responses to “I often lost track of time while scrolling through TikTok”
reflected 41.6% who agree, 18.7% who strongly agree, 32.5% who disagree, and
6.2% who strongly disagree. This suggests that students often find themselves
scrolling through TikTok for hours without realizing it. On the other hand, “The
use of TikTok disrupts my productivity” recorded 33.5% who agree, 7.7% who
strongly agree, 51.4% who disagree, and 6.7% who strongly disagree. This implies
that students do not find TikTok counterproductive as the majority of students
disagree with the statement as reflected in the high percentage. However, for most
respondents, using TikTok for academic purposes can be a great significant time
investment as reflected in the high percentage of disagreement toward the above
statement, namely 70 students (33.2%) agree, 12 students (5.7%) strongly agree,
107 students (51.7%) disagree, and 19 students (9%) strongly disagree.
Fourthly, the majority of students disagree with the statement as reflected by the
high percentages of disagreement. There is no tendency for cyberbullying to
occur. “TikTok apps tend to lead to cyberbullying.” It is interesting to observe that
70% of the respondents disagree with this statement. Last but not least, students
agree with “Learning through TikTok aligns with my preferred learning style and
needs”: there is also a high percentage of disagreement perceptions toward
“Learning through TikTok is not suited to my learning style and needs”, namely
32.5% of students agree, and 66.5% disagree with the statement. Similarly, a high
percentage of disagreeing perceptions is also true regarding “I am concerned
about the amount of personal data that TikTok collects about me, including my
location and browsing history.” This is indicated by 52.2% of the respondents
agreeing and 46.8% disagreeing. This means that students are not worried about
the amount of personal data that TikTok has access to, such as their location and
browsing history.
4.2. Students’ Views towards Perceived Impact of TikTok

Table 5 indicates the respondents' views toward the perceived impact of TikTok.
These were obtained and summarized from the open-ended statements on the
questionnaire to which the respondents freely expressed their perceptions. From
the analysis, five themes were used to describe their views towards their
perceived impacts of TikTok, namely (a) attractive and interesting, (b) effective,
(c) motivating, (d) relevant, and (e) discouraging. These were derived from the
categories of the survey questionnaires.
350
Table 5: Students’ views towards perceived impact of TikTok
Themes Students’ Views

1. It is easy to understand, to find information and knowledge.
Attractive and 2. It is good studying school topics.
interesting aspect 3. It is very interesting to watch the fun and up-to-date English
content videos.
1. The material is extensive and user-friendly, informative and
Effective aspect interactive.
2. To a certain extent, learning from TikTok is more effective.
1. It introduces new vocabulary, pronunciation,
2. It is easy to learn with visual and audio media.
3. It provides brief and useful language explanations.
Motivating aspect
4. It offers speaking practice to improve skills.
5. It enhances my speaking practice, helps to learn topic-specific
vocabulary and expressions.
1. It is more enjoyable and relatable with homework or a subject
than YouTube.
Relevant aspect
2. Seeking additional knowledge through TikTok is now a trend.
3. It is beneficial to explore it for additional learning materials.
1. TikTok can be distracting if it is not well organized.
2. Creating a to-do list to prevent interference with my
productivity
3. It depends on awareness, attitude, and ego control.
Discouraging 4. There was a lot of cyberbullying we found on TikTok
aspect comments.
5. There was worry about our personal data on TikTok, as the app
owners have any access to the viewing history.
6. Mindful of TikTok's risks, and protecting my personal
information on privacy settings and careful sharing.
5. Discussion
This study found that students' perceptions of the impact of English videos via
TikTok are divided into five themes, namely (a) Attractive, (b) Interesting and
effective, (c) Motivating, (d) Relevant, and (e) Discouraging.
5.1. Attractive and Interesting aspect

The findings of the study reveal that students agreed that using TikTok makes
studying more enjoyable and helps students grasp their understanding of English
better with no time limits. It is evident that social media, including TikTok, is
interesting and attractive, and can therefore be used as a learning opportunity to
facilitate the teaching and learning process. It is interesting owing to the diverse
content and features of TikTok (Puspitasari, 2021; Vera et al., 2022).
The students commented as follows:

“If you want to learn to only use TikTok, that's a mistake, but if it's only as a
support so you don't get bored, it's quite interesting.
TikTok's algorithms can customize what content is interesting to us so that

similar content can appear on our homepage.
351
Yes, because in my opinion it is very interesting and the discussion used is easy
to understand.
In my opinion, the most interesting videos about English that I can find only in
TikTok. It's short, interesting, and important to watch.
Every content creator on TikTok always tries to provide interesting and fun
content. Every content in English that I find is rarely or even nothing that feels
boring, especially the short duration of the video.”
Furthermore, when seen from an educational viewpoint, TikTok is a user-friendly

educational tool, provides assorted types of media content including English-
related content, offers a wide range of functions, and facilitates communication
and collaboration among users (Bernadette, 2020; Marta & William, 2016; Saputra
et al., 2021). Content-sharing among users is possible and simple by means of a
few clicks while at the same time, maintaining a constant speed (Kolhar et al.,
2021; Lai & Tai, 2021).
Moreover, the students believed that using TikTok is convenient, simple, and
accessible from any place. Nuari (2022) revealed similar findings, indicating that
at this point the TikTok application satisfies students' learning needs. Secondly,
TikTok applications attract learners owing to their novelty and the range of
features that can be utilized during their learning process. Finally, the TikTok
application represents the maturation and experience of students, as well as the
qualities of the millennial generation, who have grown addicted to and embrace
the digital age, particularly gadgets.
5.2. Effective aspect

The results of the study revealed a positive view that using TikTok in English
learning could help students gain much knowledge for additional information on
up-to-date language usage in English, especially slang words. Furthermore,
Swathi and Devakumar (2020) highlighted that TikTok is a social networking
platform that offers effective learning possibilities to aid in the teaching and
learning process. According to Yang (2020), the primary reasons for utilizing
social media applications are amusement, sociability, information, and academic
objectives. Furthermore, the findings revealed that students agreed that using
TikTok made it simpler for them to learn English since they are used to scrolling.
They contend it assists them to grasp the subject no matter where they are and
provides insightful English lessons. They also mentioned that they feel more
comfortable utilizing TikTok to learn English. Although it has the potential to
deplete their air time more quickly, they still enjoy using it because there are short-
animated videos as well as the teacher's voice to encourage them to remain
motivated (Yang et al., 2019).
In addition, TikTok can be used as a free learning resource. Consequently, users
or students can search for any interesting and attractive content that meets their
needs. This becomes additional material for their school assignments, over and
above what is taught in the classroom (Literat, 2021). Therefore, in formal
education, social media in general and TikTok in particular are effective learning
resources and encourage students to be more engaged (Faizi et al., 2013). TikTok
provides effective communication among students as the features of TikTok could
352
make the learning process more interesting. In addition, the available features of
TikTok could make students more motivated and attract students' attention and
intention to utilize learning opportunities:
The students commented as follows:
… We don't need to watch English explanation for a long duration just
to understand important English lesson, but TikTok is the best choice to
get precise lesson in a short time.
Because with its ease and various functions, TikTok can be applied in
learning activities. And learning in TikTok becomes more effective and
efficient.
…creators who specialize in language learning offer in-depth

explanations of many aspects. Those creators often break down complex
concepts into easy-to-understand explanations, using examples to make
their points clear.
5.3. Relevant aspect

The third point of discussion is the relevance aspect. In this regard, the results of
the survey revealed a positive view that the use of TikTok in English learning is
relevant to the students' needs and their capacity for learning English. Students
believed that the usage of TikTok may develop their abilities such as vocabulary
and pronunciation for better speaking, as well as accurate sentence structure for
better writing. The information sharing is simplified with a few clicks while
maintaining the speed at which information is shared with users (Kolhar et al.,
2021). TikTok can be utilized as an instructional tool, but only for short videos.
According to Jalaluddin (2016), using the medium of video allows students to
have more exposure to speaking features such as pronunciation, structure,
vocabulary, and intonation, which leads to improvements in other areas of
speaking, such as understanding and fluency. It is possible to expand students'
vocabulary merely by viewing videos with English subtitles and interpreting new
words that are not understood.
Furthermore, Medaukali (2015) highlighted the importance of social media such
as TikTok as being relevant to the ultimate goal of education—learning
autonomy—with students having access to online learning materials which
encourage them to study at their own pace (Duffy, 2008). It also provides and
enhances various skills and learning styles, and enables self-assessments for their
progress (Styati, 2016). The students commented as follows:
“The app's user-friendly interface and short video format can also make it easier
to stay engaged and focused on language learning. Moreover, it also allows me
to choose content that's relevant and interesting to me, which can always help
to motivate me and sustain my learning.
There's a selection of content that you like, so you can filter it and choose which
content appropriate to what I need.
I personally don't find it difficult because the entertaining content and the
educational content are balanced and if I want to see content in English for
learning then I can easily search for it and find a lot of suitable content. ”
353
5.4. Motivating aspect

The last topic of discussion is the motivational aspect. It is evident from the
findings that students provided positive responses regarding their enthusiasm to
utilize TikTok to study English. TikTok has many unique and innovative aspects
that can be utilized as a source of teaching and learning. It can drive students to
be more involved in the teaching and learning process. It will serve as a
motivation for students to study and enhance their English skills using the TikTok
application. Learning English using TikTok is not monotonous. In addition to the
media module book and instructor explanations, the medium that may attract
students' attention to learning is also a factor. Rachmawati and Cahyani (2021)
explained that videos can be utilized to improve English pronunciation since they
can provide recordings, moving pictures, and sound. According to the study,
students prefer video-based learning over module-based learning for topic
reference and explanations. In line with findings by Aji (2018) and Aji and Setiyadi
(2020), TikTok is a beneficial medium for teaching language since it may assist
students becoming more engaged in learning:
“Because seeing those who are good at speaking English makes me motivated to
learn more.
Interaction with the material provider will encourage and motivate me to learn
more.
I'm a very motivated person and I like to interact, so interacting with content
creators who are knowledgeable and great in English is motivating.
Interacting with English-speaking TikTok creators or users can help me to stay

motivated in learning English.
I strongly agree with that because sometimes when I feel stressed and I watch
English content on live TikTok then I can interact with the creators. It can
make me stay motivated in learning English.”
5.5. Discouraging aspect

In addition to its merits, TikTok has its disadvantages, particularly those related
to the perceived impacts of English content shared via TikTok on their English
competence. Many respondents believe that social media can only create toxicity
and harm children's brain development in this new era of social media growth.
On the other hand, many individuals are typically ignorant of the benefits of social
media for adolescents. However, there is a significant disadvantage to the TikTok
application, namely that it enables the unintentional spread of misinformation
which is a problem globally (Kılınç, 2022; Nichita et al., 2021; O'Sullivan et al.,
2012).
Even though the last ten questionnaire items of the study are negative statements,
respondents' responses are still positive in the sense that most of them stated that
TikTok does not disrupt them much, is not particularly time-consuming, and is
not counterproductive. This suggests that they perceived the positive impacts of
English content shared via TikTok on their English competence. This corroborates
the available reputable literature that concludes that social media such as TikTok
have both advantages and disadvantages, and thus have a positive as well as
negative impact on users' behaviors and students' academic performance.
However, it would be overly hasty to conclude it is the only variable affecting
354
students' performance. This is only one of the contributing factors affecting

students' competence since students nowadays are very much inclined to use new
applications (Swathi & Devakumar, 2020). Thus, it implies that English content
shared via TikTok contributes to students’ English competence in various degrees:
“I can balance my daily activities by scrolling TikTok or playing with gadgets.
TikTok is not that interesting to make me pass away from my productivity
activities.
Various types of people use TikTok, and somehow they always can turn
something normal into something negative, resulting in cyberbullying.
The sophisticated technically still has a negative side if we don't use it for a good
thing, so we just have to be careful if we use the apps or other apps.”
6. Conclusion
The study endeavored to comment on students’ perceived impacts of TikTok
English-content-sharing on their English competence, and their views towards
their perceived impacts of TikTok. The results show that perceived usefulness,
interactivity, and cost-effectiveness promote a positive attitude toward the use of
English-content sharing TikTok. Hence, they have positive perceived impacts of
English-content sharing TikTok on their English competence. In addition, it was
found that the most important factors affecting students' willingness to continue
using TikTok are the perceived usefulness, attractiveness, effectiveness, relevance,
and motivational aspects. While TikTok offers various advantages, it has negative
aspects as well that may dominate depending on the analytical skills of
individuals who use it, as well as the material they wish to see. However, it cannot
be concluded that this is the only medium which positively affects students’
performance. The study concludes that the perceived impacts of English content
shared via TikTok vary depending on the videos. There are animated input
videos, movie-maker videos, YouTube, and TikTok. They are all perceived to be
beneficial, relevant, and interesting. When used properly, TikTok could
contribute to students’ academic competence, team building, as well as
community building, which are all crucial elements in education in general and
in English language learning in particular.
7. Recommendations
The findings of the study suggest that English teachers as well as practitioners in
applied linguistics should consider the use of TikTok to enhance students’ English
competence owing to its merits. They need to maximize the potential of TikTok in
the teaching of English. On the other hand, future researchers need to conduct
research on other aspects of English content shared via TikTok, and make use of
other research designs, such as an experimental design, to be able to make a more
convincing judgment about the cause-effect relationship. In addition, for content
creators of TikTok, it is recommended that the English content needs to be more
varied, more culture-specific, and more widely accepted worldwide. Finally, it is
recommended that the full potential of TikTok be exploited to enliven the
language classroom and ultimately, to foster students’ English competence.
355
8. References
Abdu-Raheem, B. O. (2015). Teachers’ perception of the effectiveness of methods of
teaching Social Studies in Ekiti State, Nigeria. World Journal of Education, 5(2).
https://doi.org/10.5430/wje.v5n2p33
Aji, W. N. (2018). Aplikasi TikTok Sebagai Media Pembelajaran Bahasa dan Sastra Indonesia,
[TikTok app. as instructional medium for Indonesian language and literature]
Universitas Widya Dharma Klaten. Pertemuan Ilmiah Bahasa dan Sastra
Indonesia.
https://proceeding.unikal.ac.id/index.php/pibsi40/article/view/114
Aji, W. N., & Dwi, B. P. S. (2020). Aplikasi TikTok Sebagai Media Pembelajaran
Keterampilan Bersastra [TikTok app as instructional medium in literature].
Metafora, 06. https://doi.org/10.30595/mtf.v6i1.7824
Alharbi, S. & Drew, S. (2014). Using the technology acceptance model in understanding
academics’ behavioural intention to use learning management systems.
International Journal of Advanced Computer Science and Applications, 5, 143-155.
http://dx.doi.org/10.14569/IJACSA.2014.050120
Ary, D., Jacobs. L. C., Sorensen. C., & Razavieh, A. (2010). Introduction to research in
education. (8th) ed. Wadsworth
Aslam, S. (2022). TikTok by the numbers (2022): Stats, demographics & fun facts. Omnicore.
https://www.omnicoreagency.com/tiktok-statistics/
Balbay, S., & Kilis, S. (2017). Students’ perceptions of the use of a YouTube channel
specifically designed for an academic speaking skills course. Eurasian Journal of
Applied Linguistics, 3(2), 235–251. https://doi.org/10.32601/ejal.461003
Basch, C. H., Fera, J., Pellicane, A., & Basch, C. E. (2022). Handwashing videos on TikTok
during the COVID-19 pandemic: Potential for disease prevention and health
promotion. Infection, Disease and Health, 27(1), 31–37.
https://doi.org/10.1016/j.idh.2021.09.039
Bernadetha, N. (2020). The effectiveness of distance learning using social media during
the pandemic period of COVID-19: A case in Universitas Kristen Indonesia.
International Journal of Advanced Science and Technology, 29(7), 1764–1772.
http://sersc.org/journals/index.php/ijast
Bertea, P. (2009), Measuring students’ attitude towards e-learning: A case study,
Proceedings of 5th International Scientific Conference on e-Learning and Software for
Education, Bucharest, pp. 1-8.
Bhattacherjee, A. (2001). Understanding information systems continuance: An
expectation-confirmation model. Management Information Systems Quarterly, 25,
351–370. https://doi.org/10.2307/3250921
Bizel, G., Amoroso, A., & Sharma, S. (2022). A social media trend: How does TikTok
influence adults? IEDSR Association, 7(17), 192–206.
https://doi.org/10.46872/pj.468
Cheon, J., Lee, S., Crooks, S.M. & Song, J. (2012). An investigation of mobile learning
readiness in higher education based on the theory of planned behavior. Computers
and Education, 59, 1054–1064. https://doi.org/10.1016/j.compedu.2012.04.015
Creswell, J. W. (2012). Educational research: Planning, conducting, and evaluating quantitative
and qualitative research. Pearson Education.
Creswell, J. W. (2014). Research design: Qualitative, quantitative, and mixed-methods
approaches (4th ed.). Sage Publishing. http://doi.org/10.1007/s11162-004-4139-z
Dewanta, A., & Bahasa, P. (2020). Pemanfaatan Aplikasi TikTok sebagai Media
Pembelajaran Bahasa Indonesia AANBJ. Jurnal Pendidikan dan Pembelajaran Bahasa
Indonesia, 9(2). https://doi.org/10.21831/ep.v3i1.40990
356
Duffy, P. (2008). Engaging the YouTube Google-eyed generation: Strategies for using Web
2.0 in teaching and learning. Electronic Journal of E-Learning, 6(2), 119-150.
Faizi, R., Afia, A. E., & Chiheb, R. (2013). Exploring the potential benefits of using social
media in education. International Journal of Engineering & Technology, 3(4).
https://doi.org/10.3991/ijep.v3i4.2836.
Farahat, T. (2012). Applying the technology acceptance model to online learning in the
Egyptian universities. Procedia - Social and Behavioral Sciences, 64(5), 95-104.
https://doi.org/10.1016/j.sbspro.2012.11.012
Fishbein, M., & Ajzen, I. (2009). Predicting and changing behavior: The reasoned action
approach. Psychology Press. https://doi.org/10.4324/9780203838020
Fraenkel, J. R., Wallen, N. E., & Hyun, H. H. (2012). How to design and evaluate research in
education (8th ed.). McGraw-Hill.
Guru, D. & Sambandam, R. (2022). A study on impact of social media on student
education system. International Journal of Early Childhood Special Education (INT-
JECS), 14(03 2022), 25–29. https://doi.org/10.9756/INT-JECSE/V14I3.5
Hong, S.J., & Tam, K.Y. (2006). Understanding the adoption of multipurpose information
appliances: The case of mobile data services. Information Systems Research, 17,
162–179. https://www.jstor.org/stable/23015773
Horton, R. S. (2003). Similarity and attractiveness in social perception: Differentiating
between biases for the self and the beautiful. Self and Identity, 2(2), 137–152.
https://doi.org/10.1080/15298860309033
Huang, C. H. (2021). Exploring the continuous usage intention of online learning
platforms from the perspective of social capital. Information, 12, 141.
https://doi.org/10.3390/info12040141
Jaffar, B. A., Riaz, S., & Mushtaq, A. (2019). Living in a moment: Impact of TicTok on
influencing younger generation into micro-fame. Journal of Content, Community
and Communication, 10(5), 187–194. https://doi.org/10.31620/JCCC.12.19/19
Jalaluddin, M. (2016). Using YouTube to enhance speaking skills in ESL classroom English for
Specific Purposes World, 17(50).
https://www.youtube.com/user/TEDtalksDirector
Kemp, S. (2017). Digital in Southeast Asia in 2017. https://wearesosial.com/special-
reports/digital-southeast-asia-2017.
Kemp, S. (2018). Digital in 2018: World’s internet users pass the 4 billion mark.
https://wearesosial.com/blog/2018/01/global-digital-report-2018.
Kılınç, D. D. (2022). Is the information about orthodontics on YouTube and TikTok
reliable for the oral health of the public? A cross sectional comparative study.
Journal of Stomatology, Oral and Maxillofacial Surgery.
https://doi.org/10.1016/j.jormas.2022.04.009
Kolhar, M., Kazi, R.N.A., & Alameen, A. (2021). Effect of social media use on learning,
social interactions, and sleep duration among university students. Saudi Journal of
Biological Sciences, 28(4). https://doi.org/10.1016/j.sjbs.2021.01.010
Lai, C., & Tai, C.P. (2021). Types of social media activities and Hong Kong South and
Southeast Asians youths’ Chinese language learning motivation. System, 97.
https://doi.org/10.1016/j.system.2020.102432
Lee, M.C. (2010). Explaining and predicting users’ continuance intention toward e-
learning: An extension of the expectation-confirmation model. Computers and
Education, 54, 506–516.
Literat, I. (2021). Teachers act like we’re robots: TikTok as a window into youth
experiences of online learning during COVID-19. AERA Open, 7(1–15).
https://doi.org/10.1177/2332858421995537
357
Marta, R. F., & William, D. M. (2016). Studi Terpaan Media Pemasaran Melalui Posting
Instagram Terhadap Ekuitas Merek Pelanggan Sumoboo! (Analisis Eksplanatif
pada Komunitas Food Blogger #WTFoodies). Jurnal Komunikasi, 8(1).
https://doi.org/10.24912/jk.v8i1.50
Medaukali, F. (2015). Developing EFL learners’ listening comprehension through YouTube
videos: A case study of second-year students at Mohamad Khaider University of Biskra.
(Unpublished master’s thesis) Mohamed Kheidar University, Biskra.
Nichita, A., Enache, D., & Andreescu, C. V. (2021). TikTok: The influence of school
performance and social life of adolescents. Pro Edu. International Journal of
Educational Sciences, 3(4). https://doi.org/10.26520/peijes.2021.4.3.62-70
Nuari, P. L. (2022). The influence of TikTok video on students’ pronunciation in SMP Negeri 1
Purwanegara. (Unpublished thesis). UIN KH. Syaifuddin Zuhri, Purwokerto.
O’Sullivan, N. G., Manecksha, R. P., & O’Kelly, F. (2012). The unintentional spread of
misinformation on ‘TikTok’: A pediatric urological perspective. Journal of Pediatric
Urology. https://doi.org/10.1016/j.jpurol.2022.03.001
Oliver, R.L. (1980). A cognitive model of the antecedents and consequences of satisfaction
decisions. Journal of Marketing Research, 17, 460–469.
Park, S.Y. (2009), An analysis of the technology acceptance model in understanding
university students’ behavioral intention to use e-learning. Educational Technology
and Society, 12(3), 150-162.
Puspitasari, A. C. D. D. (2021). Aplikasi Tiktok Sebagai Media Pembelajaran Jarak Jauh
Pada Mahasiswa Universitas Indraprasta PGRI [TikTok as Instructional Media
in Distance Learning among University Students of PGRI Indraprasta]. Jurnal
Educatio FKIP UNMA, 7(3). https://doi.org/10.31949/educatio.v7i3.1317
Rachmawati, R., & Cahyani, F. (2021). Pengaruh Penggunaan Video Youtube terhadap
Pronunciation Skill Mahasiswa MKU Bahasa Inggris Iat Al-Khoziny Sidoarjo [The
influence of Youtube towards pronunciation skill of English students at Iat Al-
Khoziny, Sidoarjo]. Jurnal Widyyaoka Ikip Widya Darma, 8.
Saputra, N., Hikmah, N., Yustitia, V., Saputra, M., Wahab, A., & Junaedi, J. (2021).
Implementation of online learning using online media during the COVID-19
pandemic. Budapest International Research and Critics Institute (BIRCI-Journal):
Humanities and Social Sciences, 4(2), 1802–1808.
https://doi.org/10.33258/birci.v4i2.1857
Scherr, S., & Wang, K. (2021). Explaining the success of social media with gratification
niches: Motivations behind daytime, nighttime, and active use of TikTok in
China. Computers in Human Behavior, 124.
https://doi.org/10.1016/j.chb.2021.106893
Shen, D., Laffey, J. & Huang, X. (2006). Social influence for perceived usefulness and ease-
of-use of course delivery systems, Journal of Interactive Online Learning, 5(3), 270-
282.
Smith, T., & Short, A. (2022). Needs affordance as a key factor in likelihood of problematic
social media use: Validation, latent Profile analysis and comparison of TikTok
and Facebook problematic use measures. Addictive Behaviors, 129.
https://doi.org/10.1016/j.addbeh.2022.107259
Styati, E. W. (2016) Effect of YouTube videos and pictures on EFL students’ writing
performance. Dinamika Ilmu, 16(2) 507-517.
Šumak, B., Hericko, M., Pusnik, M., & Polancic, G. (2011), Factors affecting acceptance
and use of Moodle: An empirical study based on TAM. Informatica (Ljubljana),
35(1), 91-100.
358
Swathi, G.K; & Devakumar, C. (2020). A study to analyze the impact of TikTok app. on
students’ academics and psychology. Journal of Emerging Technologies and
Innovative Research (JETIR), 7(3). www.jetir.org.
Thong, J.Y.L.; Hong, S.J.; Tam, K.Y. (2006). The effects of post-adoption beliefs on the
expectation-confirmation model for information technology continuance.
International Journal of Human-Computer Studies, 64, 799–810.
https://doi.org/10.1016/j.ijhcs.2006.05.001
Venkatesh, V., & Davis, F. D. (2000). A theoretical extension of the technology acceptance
model: Four longitudinal field studies. Management Science, 46(2), 186-204.
https://doi.org/10.1287/mnsc.46.2.186.11926
Vera, A. Z., Bruce, A., Garris, J., Tochen, L., & Bhatia, P. (2022). The phenomenology of
tics and tic-like Behavior in TikTok. Pediatric Neurology, 130.
https://doi.org/10.1016/j.pediatrneurol.2022.02.003
Villa-Ruiz, C., Kassamali, B., Mazori, D. R., Min, M., & Cobos, G. (2021). Overview of
TikTok’s most viewed dermatologic content and assessment of its reliability.
Journal of the American Academy of Dermatology, 85(1).
https://doi.org/10.1016/j.jaad.2020.12.028
Wengel, Y., Ma, L., Ma, Y., A., & Maciuk, K. (2022). The TikTok effect on destination
development: Famous overnight, now what? Journal of Outdoor Recreation and
Tourism, 37. https://doi.org/10.1016/j.jort.2021.100458
Yang, H. (2020). Secondary-school students’ perspectives of utilizing TikTok for English
learning in and beyond the EFL classroom. Education, Science, Technology,
Innovation and Life. https://doi.org/10.23977/ETSS2020030
Yang, S., Zhao, Y., & Ma, Y. (2019). Analysis of the reasons and development of short
video application taking TikTok as an example. 9th International Conference on
Information and Social Science. https://doi.org/10.25236/iciss.2019.062
Zheng, L., Yang, J., & Huang, R. (2014). Emerging approaches for supporting easy,
engaged and effective collaborative learning. Journal of King Saud University –
Computer and Information Sciences, 26(1), 11-16.
https://doi.org/10.1016/j.jksuci.2013.10.002
Zihan, W. (2022). The influence of TikTok on tourism destination decision in the post-pandemic
era (Unpublished master’s thesis). ISCTE-IUL Business School.
359

Vol. 22, No. 10, pp. 359-377, October 2023
Acquisition of Communicative Skills by Foreign

Students in a Multicultural Learning
Environment
Ran Cao* and Lyaziza Sarsenbayeva
Abai Kazakh National Pedagogical University
Almaty, the Republic of Kazakhstan
Abstract. The study's relevance was determined by the lack of attention

towards specific aspects of foreign students' acquisition of
communicative skills in a multicultural setting and the need to record the
psychological and pedagogical circumstances of academic activities that
facilitate successful intercultural communication skills acquisition. This
study aimed to provide evidence for theoretical claims and describe the
experiment that aimed to validate the psychological and pedagogical
conditions under which foreign students can acquire communicative
skills within a multicultural environment at Kazakh University. To
investigate the cognitive, emotional, and personality-based behavioural
components of communicative skills, a range of valid methods were
selected. The study was conducted using a mixed method. The research
comprised 22 students from China, 14 students from the Republic of
Korea, 19 students from Afghanistan, 20 students from the Republic of
Iran, and 9 students from Vietnam, all pursuing their education in
Kazakhstan at two universities. The research results showed that high
levels of empathy and ethnic tolerance were obtained by 26.4% of the
experimental group and 28.2% of the control group students respectively.
It was found that conflict situations were more likely to arise as a result
of competition, which was confirmed by 36.8% of the experimental group
and 34.8% of the control group. The results obtained have significantly
broadened the parameters of foreign students' learning and work
activity's communicative and personality aspects.
Keywords: communicative competencies; communicative control;

empathy; ethnic tolerance; foreign students
1. Introduction
Global developments impact education across all sectors, resulting in expansive
international education initiatives. Favorable conditions for academic mobility of
both students and educators have stimulated substantial influx of trainees within
*
Corresponding author: Ran Cao, caoran351@gmail.com
©Authors
360
the global education network. The admittance of international students has

contributed to the university's international reach and educational proficiency
(Gestanti et al., 2018). Representatives from non-linguistic fields exhibit subdued
enthusiasm towards cultivating intercultural proficiency (Litvinova et al., 2021).
Studying abroad positively benefits students' overall cultural development, but

concurrent subjective and objective challenges must also be handled. Students
must acquire good communicative skills, beyond linguistic competency, to adapt
to new sociocultural conditions. Such concerns extend to the acquisition of
communicative skills in students studying at foreign educational institutions.
Language intercultural competence studies in foreign language teaching carry
more significance than psychological and pedagogical issues when it comes to
acquiring communicative skills of foreign students in a multidisciplinary
university. Munezane (2021) argues that students' readiness to communicate is a
significant factor influencing their intercultural communication. Some studies
indicate that intercultural interaction in study groups promotes the development
of intercultural communication (Some-Guiebre, 2020; Tleubay et al., 2020). Carlos
and Portugal (2021) identified issues with inadequate teacher training, insufficient
educational resources, and ineffective methods of assessing students' intercultural
communication abilities. Litvinova et al. (2021) explored the potential of online
learning materials to enhance intercultural competence in their study. Tleubay et
al. (2020) noted that learning foreign languages with an emphasis on cultural
linguistics helps to develop intercultural communicative competence. Gutiérrez-
Santiuste and Ritacco-Real (2023) define intercultural competence as an
individual's capacity to communicate and act suitably in an intercultural setting,
identifying the following elements of this competence: behavioural, affective and
cognitive. Therefore, it can be inferred that the communicative abilities of
international students are composed of emotional and personality-based,
cognitive, and behavioural factors that contribute to successful communication in
a diverse university setting.
The study's significance is established by investigating the characteristics of

communicative skills acquisition by foreign students in the multicultural setting
of a Kazakh Pedagogical University, ascertaining the psychological and
pedagogical conditions necessary for successful acquisition of intercultural
communication skills within the academic environment.
The research aims to provide theoretical grounding and experimental verification

of the psychological and pedagogical conditions of foreign students’
communicative skills’ acquisition in a multicultural environment at Kazakh
University.
Research Questions:
1) What are the foreign students' communicative skills components in a
multicultural environment?
2) Are there any interrelationships between the components of international
students' communication skills?
3) What conditions will contribute to the development of foreign students'
communication skills in a Kazakh university?
361
Academic discussion actively takes into account students' communication skills
in a new socio-cultural environment. As part of globalised higher education,
students are increasingly able to acquire knowledge from a broader range of
academic subjects. Moreover, there has been a growing presence of teachers with
international qualifications who adopt global perspectives. (Ilyashenko et al.,
2019a; Shmeleva, 2019a). Foreign students represent a significant aspect of
academic activities at universities (Shmeleva, 2019b). Intercultural
communications are viewed as a framework for vocational mobility of specialists
under global labour market conditions, as well as an important factor of self-
development aimed at achieving trans-individual goals and integration of
national education in the global education area (Gudkova et al., 2019; Onishchuk
et al., 2020). Zubkov (2020) identifies crucial aspects of intercultural competency,
comprising linguistic proficiency (language knowledge), sociolinguistic skills
(ability to communicate), sociocultural competence (comprehension of cultural
interaction), strategic planning (intentional measures for knowledge
enhancement), and cognitive learning (wide-ranging personality development).
Intercultural communication competencies entail managing communication skills
and styles while participating in the social framing of stakeholders' intercultural
identities (Yang, 2018). Meanwhile, a communicative environment is crucial for
personality growth and building relationships with others (Ibatova, 2019).
Developing intercultural competence involves integrating sociolinguistic
innovations to establish a flexible sociolinguistic setting that incorporates
elements of psychological support and adaptation (Malyuga et al., 2018;
Rubtsova, 2019). Cruz (2023) highlights the significance of developing
communication skills and understanding people from different backgrounds in
the acquisition of intercultural competence. Such a skill cannot be acquired in a
brief span of time and must be developed throughout life. Luchaninova et al.
(2019) defines communicative competence as the ability to self-assess and develop
one's interpersonal skills to successfully pursue a professional career. Klimova et
al. (2019) suggests that the best way to develop communicative competence is in
a multicultural environment.
The enhancement of students' communicative skills necessitates the provision of

appropriate conditions and the evaluation of their ability to employ modern e-
learning technologies (Ilyashenko et al., 2019b). In addition, Qin et al. (2022)
discovered that learners pursuing linguistic and non-linguistic disciplines possess
contrasting perceptions when acquiring international expertise through
MOODLE. This variance may arise from distinct motives for obtaining
competence and its relevance to respective career fields. Khajieva et al. (2020)
suggested implementing a comprehensive model for the acquisition of foreign
language communicative competencies, Shafazhinskaya et al. (2020) and Jabbarov
(2020) argued that it is needed to initiate the active use of innovative education
technologies aimed at strengthening self-study skills, free-thinking and creativity,
facilitating communicative skills’ acquisition in a multicultural learning
environment. This will also contribute to the effectiveness of students’ general
cultural competencies acquisition, which depends on a mature theoretical basis,
pedagogical conditions, new approaches and e-learning technologies (Salpykova
et al., 2022).
362
The comprehensive investigation into the social and psychological assimilation of

international students indicated a requirement for developing an efficient
adaptive atmosphere (Biserova & Shagivaleeva, 2019; Zoyirova, 2018). Students'
diverse and traditional cultural knowledge helps to avoid cultural discrepancies,
which are usually caused by psychological and pedagogical factors (Almazova et
al., 2019). To address this issue, Rerke et al. (2018) propose the use of mediation
technologies, which aim to prevent radical ideologies.
Smakova and Paulsrud (2020) conducted studies on intercultural competence

usage in Kazakhstan's educational space and found that teachers recognize the
effectiveness of its development for enhancing language skills and vice versa.
Unfortunately, such methods are not used in practice. Yelubayeva and Mustafina
(2020) demonstrated that the enhancement of intercultural proficiency yields
positive outcomes in terms of language acquisition and comprehension of social
and cultural matters, as well as in intercultural communication.
The literature data analysis allows us to infer a persistent and ongoing concern
regarding the development of communication skills among foreign students.
Nonetheless, the quest for enhancing these students' intercultural competence
across varied environments, while considering their specialisation and
educational institution's specificities, remains relevant.
3. Materials and methods

The study employed a mixed methodology, consisting of modelling and
questionnaires, to identify personality indicators. The research hypothesis of the
study assumes that the communicative skills of foreign students are a stepwise
and multifaceted system dependent on specific psychological and pedagogical
conditions, such as:
• acquisition communication skills at the level of interpersonal and intercultural
communication;
• students’ involvement in diverse forms of collaborative activities guided by
cultural dialogue;
• integrated use of interactive teaching methods and forms, designed to enhance
communication skills;
• organisation of a virtual multimedia communicative environment.
To verify the hypothesis, two groups of respondents participants were arranged.

The groups were comprised of students studying at different higher education
institutions. Specifically, the experimental group comprised 38 foreign second-
year students (22 male and 16 female, with an average age of 18.4 years) from Abai
KazNPU, while the control group comprised 46 second-year students (25 male
and 21 female, with an average age of 18.3 years) from Al-Farabi KazNPU, who
were enrolled in various academic programs. The experimental group students
actively engaged in enhancing their communication skills by utilising the authors'
recommendations, while the control group students were evaluated using the
conventional system.
The participants comprised of students from China (n=22), the Republic of Korea
(n=14), Afghanistan (n=19), Republic of Iran (n=20), and Vietnam (n=9) who had
363
an intermediate level of Russian language. This was evaluated through standard

tests during admission to the university.
The respondents then completed the questionnaires via a Google form sent to
them by email.
To explore the cognitive, emotional, and personality components of

communicative skills, a set of questionnaires was chosen. To investigate the
emotional and personality components specifically, the Megrabian and Epstein
(1972) empathy diagnosis was employed. This diagnosis includes 36 statements
to determine the levels of emotional response (empathy): very high (scores 82-90);
high (scores 63-81); normal (scores 37-62); low (scores 12-36); and very low (scores
1-11). This technique was based on: 1) the capability to detect other individuals'
emotions, coupled with the capacity to envision another's worries; 2) attaining
social adjustment as a result of interpersonal communication; 3) utilizing forward
and backward questions to confirm or deny the assertions made. Excessive
empathy levels may cause emotional reliance on others. Normal levels of empathy
facilitate appropriate responses to the actions of others and emotional regulation.
Individuals with low or very low levels of empathy experience difficulty in
communication and tend to work in isolation (Poole et al., 2022). The validity of
Lawrence et al.'s (2004) study has confirmed the reliability of this test.
The test for assessing tolerance in international relations developed by Sobkin and
Adamchuk (2015) comprises 12 question sets that establish the degree of tolerance
or intolerance (scoring from -90 to -45 signify a high level of intolerance
development; from -45 to 0 - low level of intolerance development; from 0 to 45 -
low level of tolerance development; from 45 to 90 - high level of tolerance
development). The test aims to measure objective attitudes regarding tolerance
and intolerance towards international relations and is targeted at different
context-specific scenarios: registration of emotional attitudes towards one's own
national identity; the social barriers to representatives of other nationalities; the
manifestation of tolerant or intolerant attitudes in the military, education and
cultural traditions; attitudes towards ethnic conflicts; emotional acceptance or
non-acceptance of representatives of other nationalities. The methodology used in
this study yields data on various aspects of international relations, including
national self-identification and emotional attitude towards one's nationality
(Jaramillo, 2021). The validity of this test has been proven by its authors Sobkin
and Adamchuk (2015).
The study examined the cognitive component by employing a diagnostic

approach to comprehend the significance of non-verbal behaviour. The
methodology, presented by Zimina (2015) and endorsed by the National
Commission, involves 48 images of non-verbal communication scenarios to
measure levels of comprehension. Scoring high (33-48), medium (16-32), or low
(1-15) determines the grasp of non-verbal communication meanings. Participants
were presented with images depicting non-verbal behaviour and asked to identify
its meaning. Results were compared to the correct responses and the total number
of accurate answers was tallied (Yasar Yuzlu & Dikilitas, 2022).
364
For investigating the behavioural component, the Thomas-Kilmann test was

applied. This assessment comprises 30 statement pairs designed to reveal a
respondent's conflict resolution style. This test suggests two scales:
“assertiveness” and ”cooperativeness”, with the help of which five different
conflict modes: competing (assertive, not collaborating), avoiding (unassertive,
not collaborating), accommodating (unassertive, collaborating), collaborating
(assertive, collaborating), and compromising (interim assertiveness, and
readiness to collaboration), are registered (Duong & Pham, 2022). The validity of
this test has been proven by its authors Thomas and Kilmann (1978).
In the final phase of the research, the authors investigated the correlation between
the language skills of international students and their peers' abilities during
learning and interaction. The statistical methods of Student's t-test and Spearman
correlation test were utilised with the aid of Microsoft Excel and SPSS Statistics 28
to analyse the data and determine the link between the communicative
capabilities of foreign students.
The ethical considerations outlined in the Guidelines for Research Ethics in

Science and Technology (The Norwegian National Committee for Research Ethics
in Science and Technology, 2016) have been adhered to in this article. The ethical
issues during the research were addressed by strictly adhering to bioethics norms
including obtaining informed consent from the participants, ensuring anonymity
and confidentiality of the information obtained, treating all participants fairly and
equitably and upholding the principle of integrity. No specific funding was
allocated for the research, and no conflict of interest was detected.
4. Results
Teachers' readiness to teach students from different countries is an important
aspect of their professional competence; it is crucial to be aware of learners from
different nationalities and not to emphasise their foreignness. This study involved
teachers who had more than 5 years of teaching experience in intercultural
student groups and had undergone appropriate training (advanced training
courses) on teaching foreign students. According to the “foreign students’
communicative skills” construct, the conceptual model of foreign students’
communicative skills was developed (Figure 1).
365
Figure 1: The conceptual model of foreign students’ communicative skills in the

context of the author's vision of the problem.
Сommunicative abilities in this model include personal emotional, cognitive and

behavioural components which affect the communication abilities of international
students.
Table 1 presents comparative results of research on empathy as a reflection of

emotional and personality-based components of communicative skills of students
in the experimental and control groups of respondents.
Table 1: The results of students’ empathy degree diagnosis by Megrabian and Epstein
(pre-experimental data)
The
The control
experimental Student’s
Empathy level Scores group p
group t-test
n % n %
Very high 82-90 2 5.3 3 6.5
High 63-81 8 21.1 10 21.7
Normal 37-62 16 42.1 21 45.6 1.6330 >0.05
Low 12-36 7 18.4 8 17.4
Very low 1-11 5 13.2 4 8.7
Total 38 100 46 100
Source: Calculate by Author
Following the data analysis, the empirical evidence indicates that the level of
Student's T-Test is 1.6330 (p>0.05), suggesting the students' display of empathy is
dynamic, particularly towards exceptional circumstances. Nevertheless, the
control group of respondents showed a slightly superior empathy level as
compared to the experimental group. At the same time, 18.4% of students in the
experimental group and 17.4% of those in the control group exhibited a low level
366
of empathy, while 13.2% of students in the experimental group and 8.7% of those
in the control group displayed a very low level of empathy. This necessitates close
attention from teaching staff to support such students, particularly those studying
in their home country, to prevent potential conflicts.
Table 2 presents the results of the international tolerance assessment conducted

by Sobkin and Adamchuk (2015). After analysing the results (Table 2), it was
found that the members of the experimental group showed significantly lower
tolerance (р<0.05) and higher intolerance towards people of different nationalities
compared to the respondents of the control group.
Table 2: The comparative indicators of international tolerance in experimental and

control group respondents
Experimental Control
Level group group Student’s
Indicator p
(scores) t-test
n % n %
High level of
tolerance (45- 13 34.2 19 41.3
90)
The registration
Low level of
of the
tolerance (0- 25 65.8 27 58.7
peculiarities of
45)
emotional 1.1882 >0.05
High level of
attitudes to
tolerance (45- 22 57.9 18 39.1
one’s own
90)
national identity
Low level of
intolerance 16 42.1 28 60.9
(0-45)
High level of
tolerance (45- 24 63.2 29 63.0
90)
The Low level of
characteristics of tolerance (0- 14 36.8 17 37.0
social distances 45)
6.9282 <0.05
(barriers) to High level of
other nationality intolerance 12 31.6 15 32.6
representatives (45-90)
Low level of
(0-45)
High level of
Manifestation of tolerance (45- 18 47.4 22 47.8
tolerant 90)
/intolerant Low level of
attitudes at the tolerance (0- 20 52.6 24 52.2
ideological level 45) 2.4495 <0.05
(army, High level of
education, intolerance 21 55.3 21 45.7
cultural (45-90)
tradition) Low level of
17 44.7 25 54.3
intolerance
367
(0-45)
High level of
tolerance (45- 10 26.3 12 26.1
90)
Low level of
tolerance (0- 28 73.7 34 73.9
Attitude to 45)
4.899 <0.05
ethnic conflicts High level of
(45-90)
Low level of
(0-45)
High level of
tolerance (45- 26 68.4 33 71.7
90)
Low level of
Emotional
tolerance (0- 12 31.6 13 28.3
acceptance/non
45)
-acceptance of 2.3094 <0.05
High level of
other nationality
representatives
(45-90)
Low level of
(0-45)
Total 38 100 46 100
Source: Calculated by Author
Statistically significant differences were only in the segment "The registration of

the peculiarities of emotional attitudes towards one's own national identity". The
rest of the clusters demonstrate a result of р<0.05. The most noticeable were the
differences occurring in the attitudes toward their own national identity and
social barriers to other nationality representatives (Table 2). The hindrance of
foreign students' acquisition and communication aptitudes within a diverse
academic milieu is precisely associated with specific factors. Table 3 presents the
findings of the probe into the behavioural facet of the learners' communicative
competencies, based on the Thomas-Kilmann test (1974).
Table 3: The styles of conflict modes of experimental and control group students
The prevailing mode of Experimental Control

group Student’s
behaviour in a conflict group p
t-test
situation n % n %
Competition 14 36.8 16 34.8
Collaboration 8 21.1 10 21.7
Avoidance 7 18.4 9 19.6 4.000 <0.05
Compromise 4 10.5 6 13.0
Adaptation 5 13.2 5 10.9
Total 38 100 46 100
Source: Calculated by Author
368
In both respondent groups, the dominant behavioural style was competition (refer
to Table 3). A higher percentage of students in the control group chose
compromise (13.0% versus 10.5%), indicating greater adaptability. However, the
opposite was true for the experimental group, where 13.2% of students chose
adaptation compared to 10.9% of students in the control group.
To investigate the specificities of the connection among the emotional, personal,

cognitive, and behavioural components' indicators, the existence of
interrelationships between the test indicators using Spearman’s rank correlation
coefficient were calculated (Table 4).
Table 4: Interrelations of foreign students’ communicative skills (Spearman’s rank

correlation coefficient)
Knowledge
Understanding
Ethnic ethnic cultural Communicative
Empathy non-verbal
tolerance communication control
behaviour
differences
0.699** 0.608** 0.832** 0.776**
Empathy
0.000 0.000 0.000 0.000
0.699** 0.398* 0.649** 0.800**
Ethnic tolerance
0.000 0.029 0.000 0.000
Knowledge of
ethnic and
0.608** 0.398* 0.516* 0.533*
cultural
0.000 0.029 0.003 0.002
communication
differences
Understanding
0.832** 0.649** 0.516* 0.730**
non-verbal
0.000 0.000 0.003 0.000
behaviour
Communicative 0.776** 0.800** 0.533** 0.730**
control 0.000 0.000 0.002 0.000
Note: *Correlation is significant at the 0.05 level (2-tailed); **Correlation is significant at
the 0.001 level (2-tailed)
The positive correlation at the high significance level (r=0.699) was revealed
between empathy and ethnic tolerance, empathy and knowledge of ethnic and
cultural communication differences. Also, empathy correlates (r=0.832) with
understanding non-verbal behaviour and with communicative self-control
r=0.776. Ethnic tolerance, apart from empathy (r=0.398), correlated with the
knowledge of ethnic cultural peculiarities (р=0.029, р≤0.05); understanding non-
verbal behaviour r=0.649 (р=0.000, р≤0,001); communicative control r=0.800
(р=0.000, р≤0.001). Students with higher levels of empathy exhibit greater levels
of tolerance towards other ethnic groups, better understanding of ethnic cultural
nuances, improved communication skills and better self-control during
communication. The comprehension of ethnic and cultural communication
disparities was associated with a correlation between understanding non-verbal
behaviour (r=0.516) and communicative control (r=0.533) (p=0.003, p≤0.05 and
p=0.002, p≤0.05, respectively). The correlation between understanding non-verbal
behaviour (r=0.730) and communicative control (р=0.000, р≤0.001) was evident.
Knowledge of ethnic and cultural communication differences was influenced by
this understanding, ultimately resulting in greater control over communication
among students. Effective communication requires an understanding of various
369
factors, including awareness of cultural diversity, the ability to tolerate different

perspectives, and the capacity to control communication behaviors. These
findings support the developed model of communication skills and emphasise the
importance of strengthening all aspects of this model.
. Based on the results, recommendations for the acquisition of communicative

skills by foreign students in a multicultural learning environment were proposed
and implemented in the educational process of the experimental group:
1) Adaptation: acquaintance with a group, encouragement of communicative
motivation.
This stage promotes motivation for group support and adaptive capacity, and
awareness of the importance of communication skills.
2) Raising awareness: acquisition of communicative skills; immersion in a
multicultural environment.
Applying this point would help foreign students to self-perception as
communicative actor, self-identification as an international interaction group
member.
3) Implementation: acquisition of communicative skills, and their application in
various interactive formats within a group.
This stage contributes to initiative communication, motivation of collaboration,
active use of communicative skills.
The application of the recommendations presented led to the following results.
Further analysis revealed that the behaviour modes of students in conflict

situations in the experimental group underwent positive changes. The number of
students employing a productive style increased significantly by 13.3% for
collaboration and 10.0% for compromise (as depicted in Figure 2). In contrast, the
control group experienced an increase of only 3.3% in the collaboration behaviour
mode, and the compromise mode remained unchanged.
Figure 2: The distribution of foreign students by behaviour modes in conflict

situations according to summative and formative assessment results of the
experimental group (according to the Thomas-Kilmann test) (posttest data)
The experimental group students demonstrated development in all aspects of

communicative skills, supported by statistical analysis using Spearman's rank
correlation coefficient. A significant shift was occurred in terms of empathy:
р=0.026, р≤0.05; ethnic tolerance: р=0.045, р≤0.05; communicative control: р=0.002,
370
р≤0.05; the knowledge of ethnic and cultural communication differences: р=0.000,

р≤0.001. The shift in the indicators of understanding the non-verbal behaviour has
occurred at a very high level р=0.000, р≤0.001. Within the control group, only the
indicator of communicative control demonstrated the shift: р=0.004, р≥0.05. The
peculiarities, regularities, and interrelationships that have been established will
enhance effective acquisition of communicative skills by foreign students in a
multicultural learning environment at a university.
5. Discussion
The research highlights the significance of enhancing the emotional, cognitive,
and behavioural components in the communication of international students. This
study is consistent with Bingzhuan's (2021) research on the assessment of
intercultural competence among university students using a questionnaire.
Additionally, this study builds upon and supplements the research presented in
Munezane's (2021) model of intercultural communication, which identifies eight
factors of individual differences that impact intercultural communication
proficiency. This study expands on the research conducted by Gutiérrez-Santiuste
and Ritacco-Real (2023), which explored the behavioural, affective, and cognitive
aspects of intercultural communication online. It is important to consider these
unique aspects in cross-cultural communication. Our study examines how
international students' communication skills incorporate emotional and personal
factors, such as tolerance and empathy. The cognitive component encompasses
familiarity with ethical communication distinctions, comprehension of
information communicated through nonverbal methods, and the behavioural
component, incorporating tactics for managing conflicts and regulating
communication.
Numerous studies have focused on examining global internationalisation

processes at universities, as well as the unique aspects of such processes across
different countries. According to Doucette et al. (2021), modern multicultural
education serves as a reflection of the cultural interactions that are an inevitable
part of contemporary society. These interactions encourage students to gain
valuable experience with socialising across cultures and develop their cultural
awareness. Jandevi and Zareen (2020), Kusumawati et al. (2020), and Siregar et al.
(2021) have identified the challenges in internationalising education in certain
countries, particularly in Indonesia, where universities are predominantly located
in towns with a stronger sense of tradition. The education systems of different
countries have notable variations in their cultural and religious values. Warshaw
et al. (2020) highlight the significance of broadening students' communicative
skills in a cross-cultural educational setting. White and Saqipi (2021)
distinguished the demand for financial and human resources, considering the
conflict of interest of parties, provision of infrastructure, and understanding of
short-term and long-term advantages of internationalisation. This study analyses
the idiosyncrasies of students' behavioural patterns in situations of conflict. This
analysis assists in identifying the required measures to anticipate and resolve such
conflicts effectively. Shaping soft skills has been recognised as an important trend
in education. Tsalikova and Pakhotina (2021) note that professionals in any field
require soft skills and competencies for a successful career, therefore
contemporary academic programmes must focus on developing these skills. Such
371
requirements extend to educational systems and advocate for the acquisition of

complementary competencies alongside professional skills. In this study, a
comparable approach was employed, involving a thorough cultivation of
intercultural proficiency through supplementary classes within instructional
hours. Bybee et al. (2021), Handayani and Wienanda (2020) highlighted the
necessity of combining hard and soft skills for ensuring productive activity.
Bessarab (2021) stating that the most significant soft skills are: adaptivity and
openness (82.9%), cultural understanding (74.3%), language and communicative
skills (71.4%), the co-working abilities (65.7%) and interpersonal communication
skills (54.3%). The study reveals the possible manifestations of soft skills in an
intercultural context. Effective communication requires initiating a discussion,
accepting criticism, collaborating and summarising (Warshaw et al., 2020).
Communication between students of different languages and between students
from different countries in an academic environment contributes to the
development of language and communication skills that are used in real-life
situations and is also of great importance.
Seeking academic process resources for communicative skills’ development is one

of the aspects discussed in such studies. An efficacious means for accomplishing
successful cross-cultural education involves the collaboration of students and
instructors in non-conventional lessons to establish a foundation for learning
before tackling the course's primary content (Bybee et al., 2021). During university
classes, both students and instructors are acquainted with each other and able to
amend their plans (Aksoy & Ceylan, 2021). Warshaw et al. (2020) highlighted the
significance of broadening students' communicative skills in a culturally diverse
educational setting. Prasetiani and Diner (2018) demonstrated a successful
partnership between students proficient in sign language and services supporting
Spanish-speaking families with deaf children, which enhances comprehension of
multicultural and multilingual identities. The interpersonal interaction with
instructors also influences the success level of teaching (Schembri, 2021). Teaching
teamwork communication skills is a critical condition for the development of
students’ communicative skills, which is a key to the success of training specialists
(Haas, 2019; Rautakoski et al., 2021). Aksoy and Ceylan's (2021) research identified
possibilities for enhancing classroom interaction in the social sciences course
through utilising the communicative approach. Educators should be conscious of
the communicative approach whilst interacting and communicating, as this
supports discourse style diversity (Haas, 2019). The study examined the
communication skills of international students, identifying their components and
levels. The results revealed significant positive relationships between the
communication ability indicators of the respondents at high and medium levels
of significance.
Communicative skills and language acquisition are a prerequisite for the

successful adaptation of foreign students to the new multicultural environment,
based on the case of Indonesian students in China (Jandevi & Zareen, 2020). It is
necessary to apply more flexible communicative strategies for effective adaptation
of foreign students, involving communication between foreign students and
locals, to ensure private space for foreign students. The research by Kiessling and
Fabry (2021) showed that Indonesian students adapted to the living conditions in
372
China quite successfully, but their main difficulties refer to speaking the Chinese
language. The capacities for the development of linguistic and cultural
competencies at the regional level (Malta) related to one’s professional activity
were highlighted Schembri (2021) as well. Communicative competence is
understood as a complex entity implying the ability to perform speech activity
through the language learned meeting the requirements of the communicative
aims and situations in a certain sphere of activity (Rautakoski et al., 2021). This
study enhances the available information on efficacious methods for cultivating
intercultural proficiency, accounting for language acquisition, fostering soft skills
within intercultural interaction, and elevating students' empathetic capacity. The
communicative and personal dimensions of educational and vocational activities
among foreign students are further explored. The structured features, patterns
and associations will guarantee superior learning outcomes for international
students in the diverse academic surroundings of the university.
6. Conclusions
In summary, the results verify the effectiveness of developing intercultural skills
utilizing the conceptual model developed. In today's society, there is an increasing
need for graduates to possess intercultural communication abilities and highly
advanced intercultural competencies as a communication skill set. Expanding
methods for acquiring intercultural communication skills in a multicultural
setting is vital to modern higher education and continues to intrigue researchers.
During the research, emphasis was placed on teamwork. Interactive methods
including online conferences, training, guides, quizzes, and other events were
utilized. The intense use of informational and communicative mediums enabled
success. The term "acquisition" refers to the process that leads to the structural
transformation of students' personal qualities and activity. Organised based on
the principles of communicativeness, interactivity, meaningfulness, activeness,
and creativity, the work described the qualitative changes and interrelationships
between the components of the acquired quality step by step.
The development of foreign students' communicative skills is a complex, multi-

step process that relies on specific psychological and pedagogical conditions.
These include the shaping of interpersonal and intercultural communication, the
engagement of students in various types of joint activities through cultural
dialogues, and the application of interactive and innovative learning methods.
The majority of students showed a medium level of development (43.3%-46.6%)
and insufficiently mature behaviour modes in conflict situations (3.3%), which
gave rise to the development of the program on communicative skills’ acquisition.
This necessitates the further development of programmes and utilization of
approaches that enhance students' intercultural competence. Augmenting the
curricula and approaches will escalate the degree of intercultural communication,
cultivate empathy, intercultural sensitivity and conflict resolution abilities,
thereby fostering the production of the requisite soft skills required for top-tier
professionals.
Using the conceptual model of intercultural communication skills development,

this study presents surveys that explore diverse aspects of foreign students'
personalities to prevent conflicts. Considering these peculiarities would aid in the
373
effective adaptation and future professional success of foreign students. This will
enhance the benchmarks of successful international collaboration among higher
education institutions and augment the competitiveness of the educational
establishment.
Recommendations
The prospective areas of further research are the development of psychological
and pedagogical conditions for students’ acquisition of communicative
competencies in various areas, and the development of the typology of students
based on communicative style characteristics. This will enable foreign students to
adapt to the learning environment and develop intercultural competence,
avoiding conflict situations. The implementation of the developed conceptual
model can overcome the constraints of the present study, as it enables testing on
larger student samples, assessing the levels of intercultural proficiency and
acceptance among students from diverse nations.
Limitations
The study was limited by the use of a relatively small sample of students from
two Kazakh universities during the survey, which affected the results of the study
and made it a pilot study. However, as the experimental and control groups were
randomly selected from a multicultural student environment, the sample
remained relevant. There is a strong likelihood that under different social,
cultural, and learning circumstances, comparable studies would produce
somewhat distinct outcomes. As a result, we aim to pursue further research in this
field. The results of the study are influenced by the specific cultural context in
which students study in Kazakhstan. The countries represented in the study,
including China, the Republic of Korea, Afghanistan, Iran, and Vietnam also have
an impact on the results.
Considering the suggestions in the creation of curricula and programmes can
broaden students' intercultural competence, regardless of their field of study. By
implementing the conceptual model's recommendations in the education of
students with different ethnic backgrounds, studying in various specialities,
outcomes of the current research might grow, and its limitations will be
diminished.
7. References
Aksoy, Mutlu & Ceylan, Tuncay. (2021). An Action Research on Improving Classroom
Communication and Interaction in Social Studies Teaching. Education Research
International. 2021. 1-19. https://doi.org/10.1155/2021/9943194
Almazova, N., Baranova, T., & Khalyapina, L. (2019). Development of students’
polycultural and ethnocultural competencies in the system of language education
as a demand of the globalizing world. In Z. Anikina (Ed.), GGSSH 2019: Going
global through social sciences and humanities: A systems and ICT perspective (pp. 145-
156). Springer. https://doi.org/10.1007/978-3-030-11473-2_17
Araujo Portugal, J. C. (2021). Intercultural communicative competence in foreign language
learning. Global Journal of Foreign Language Teaching, 11(4), 243–256.
https://doi.org/10.18844/gjflt.v11i4.6006
Bessarab, N. A. (2021). Improving professional competence of teachers as a condition for
educating intercultural tolerance of primary school students. Pedagogy and
Psychology, 47(2), 40-47. https://doi.org/10.51889/2021-2.2077-6861.04
374
Bingzhuan, P. (2021). Intercultural communicative competence teaching and assessment

based on modern information technology. International Journal of Emerging
Technologies in Learning (IJET), 16(07), 175-190.
Biserova, G. K., & Shagivaleeva, G. R. (2019). Socio-Psychological Adaptation of
International Students to Learning and Professional Activities. Space and Culture,
India, 6(5), 99–114. https://doi.org/10.20896/saci.v6i5.411
Bybee, E. R., Whiting, E. F., & Cutri, R. M. (2021). “I think I’m the bridge”: Exploring
mentored undergraduate research experiences in critical multicultural education.
International Journal of Multicultural Education, 23(1), 22–46.
https://doi.org/10.18251/ijme.v23i1.2263
Cruz, J. (2023). Promoting learner autonomy through tandem learning in a Japanese ESL
context. International Journal on Social & Education Sciences (IJonSES), 5(1), 34-50.
https://doi.org/10.46328/ijonses.424
Doucette, B., Sanabria, A., Sheplak, A., & Aydin, H. (2021). The perceptions of culturally
diverse graduate students on multicultural education: Implication for inclusion
and diversity awareness in higher education. European Journal of Educational
Research, 10(3), 1259-1273. https://doi.org/10.12973/eu-jer.10.3.1259
Duong, Q. P., & Pham, T. N. (2022). Moving beyond four walls and forming a learning
community for speaking practice under the auspices of Facebook. E-Learning and
Digital Media, 19(1), 1-18. https://doi.org/10.1177/20427530211028067
Gestanti, R. A., Nimasari, E. P., & Mufanti, R. (2018). Multicultural classroom: teacher’s
reflection in teaching english for international students from non-english
speaking country. Exposure : Jurnal Pendidikan Bahasa Inggris, 7(2), 179–191.
https://journal.unismuh.ac.id/index.php/exposure/article/view/1511/pdf
Gudkova, S. A., Yakusheva, T. S., Sherstobitova, A. A., & Burenina, V. I. (2019). Modeling
of scientific intercultural communication of the teaching staff at smart university.
In V. L. Uskov, R. J. Howlett, & L. C. Jain (Eds.), Smart education and e-learning 2019
(pp. 551-560). Springer. https://doi.org/10.1007/978-981-13-8260-4_48
Gutiérrez-Santiuste, E., & Ritacco-Real, M. (2023). Intercultural communicative
competence in higher education through telecollaboration: typology and
development. Education and Information Technologies, 1-28.
https://doi.org/10.1007/s10639-023-11751-3
Haas, B. W. (2019). Enhancing the intercultural competence of college students: A
consideration of applied teaching techniques. International Journal of Multicultural
Education, 21(2), 81–96. https://doi.org/10.18251/ijme.v21i2.1696
Handayani, A., & Wienanda, W. K. (2020). International mobility programs to improve
soft skills of Vocational College students and alumni. Journal of Education and
Learning (EduLearn), 14(3), 377-384.
https://doi.org/10.11591/edulearn.v14i3.14538
Ibatova, A. Z. (2019). The content and structure of future engineers’ communicative
culture. Humanities & Social Sciences Reviews, 7(4), 762-766.
https://doi.org/10.18510/hssr.2019.7498
Ilyashenko, L., Gladkova, M., Kutepov, M., Vaganova, O., & Smirnova, Z. (2019a).
Development of communicative competencies of students in the context of
blended learning. Amazonia Investiga, 8(18), 313-322.
https://amazoniainvestiga.info/index.php/amazonia/article/view/313
Ilyashenko, L. K., Markova, S. M., Mironov, A. G., Vaganova, O. I., & Smirnova, Z. V.
(2019b). Educational environment as a development resource for the learning
process. Amazonia Investiga, 8(18), 303-312.
https://amazoniainvestiga.info/index.php/amazonia/article/view/312
375
Jabbarov, U. (2020). The role of trainings in the formation and development of future
foreign language teachers. Journal of Foreign Languages and Linguistics, 1(1), 34-38.
https://journal.jdpu.uz/index.php/fll/article/view/68
Jandevi, U., & Zareen, M. (2020). Cross-cultural communication of the Indonesian
students in China. International Journal of Communication and Society, 2(2), 79-85.
https://doi.org/10.31763/ijcs.v2i2.96
Jaramillo, N. (2021). Leveraging asynchronous speaking tasks to promote willingness and
confidence to speak in Spanish: A qualitative study. Australasian Journal of
Educational Technology, 38(2), 1-14. https://doi.org/10.14742/ajet.6657
Khajieva, I., Khujaniyazova, G., Kenjaeva, K., & Jumaniyozov, F. (2020). Foreign language
competenceformation of the future teacher of vocational education in the
information and educational environment. European Journal of Molecular & Clinical
Medicine, 7(2), 360-365.
https://ejmcm.com/uploads/paper/dd0c00316648de011249c96d791f5645.pdf
Kiessling, C., & Fabry, G. (2021). What is communicative competence and how can it be
acquired? GMS Journal for Medical Education, 38(3), 49.
https://doi.org/10.3205/zma001445
Klimova, I. I., Klimova, G. V., & Dubinko, S. A. (2019). Students’ communicative
competence in the context of intercultural business communication. XLinguae,
12(1), 207-218. https://doi.org/10.18355/XL.2019.12.01.16
Kusumawati, N. S., Nurhaeni, I., & Nugroho R. (2020). The content of higher education
internationalization policy: Stakeholders’ insight of internationalization of higher
education. Journal of Education and Learning (EduLearn), 14(2), 255-262.
Lawrence, E. J., Shaw, P., Baker, D., Baron-Cohen, S. & David, A. S. (2004). Measuring
empathy: Reliability and validity of the Empathy Quotient. Psychological medicine,
34(5), 911-920. http://dx.doi.org/10.1017/S0033291703001624
Litvinova, T., Andriutsa, N., & Movchun, V. (2021). Developing students’ intercultural
communicative competence through online learning. International Journal of
Emerging Technologies in Learning (iJET), 16(8), 162-172.
Luchaninova, O., Koval, V., Deforzh, H., Nakonechna, L., & Golovnia, O. (2019).
Formation of communicative competence of future specialists by means of group
work. Revista Espacios, 40(41), 11.
http://revistaespacios.com/a19v40n41/19404111.html
Malyuga, E. N., Krouglov, A., & Tomalin, B. (2018). Linguo-cultural competence as a
cornerstone of translators’ performance in the domain of intercultural business
communication. XLinguae, 11(2), 566-582.
https://doi.org/10.18355/XL.2018.11.02.46
Medynska, A., Vasylenko, O., Lapshyna, O., Krasnopera, T., Necheporuk, Y., &
Bondarenko, O. (2022). The problem of interactive english language learning in
distance mode. Postmodern Openings, 13(3), 267–283.
https://doi.org/10.18662/po/13.3/489
Mehrabian, A., & Epstein, N. (1972). A measure of emotional empathy. Journal of
personality, 40(4), 525-543. https://doi.org/10.1111/j.1467-6494.1972.tb00078.x
Munezane Y. (2021). A new model of intercultural communicative competence: bridging
language classrooms and intercultural communicative contexts. Studies in Higher
Education, 46(8), 1664-1681. https://doi.org/10.1080/03075079.2019.1698537
Onishchuk, I., Ikonnikova, M., Antonenko, T., Kharchenko, I., Shestakova, S., Kuzmenko,
N., & Maksymchuk, B. (2020). Characteristics of foreign language education in
foreign countries and ways of applying foreign experience in pedagogical
universities of Ukraine. Revista Romaneasca Pentru Educatie Multidimensionala,
12(3), 44-65. https://doi.org/10.18662/rrem/12.3/308
376
Poole, F., Clarke-Midura, J., & Ji, S. (2022). Exploring the affordances and effectiveness of
a digital game in the Chinese dual language immersion classroom. Journal of
Technology and Chinese Language Teaching, 13(1), 46-73.
http://www.tclt.us/journal/2022v13n1/pooleclarkeji.pdf
Prasetiani, D., & Diner, L. (2018). The communication strategy used by Japanese learner at
the basic level. Journal of Education and Learning (EduLearn), 12(1), 91-96.
Qin, S., Orchakova, L., Liu, Z.-Y., Smirnova, Y., & Tokareva, E. (2022). Using the learning
management system “Modular Object-Oriented Dynamic Learning
Environment” in multilingual education. International Journal of Emerging
Technologies in Learning (IJET), 17(03), 173–191.
Rautakoski, P., af Ursin, P., Carter, A. S., Kaljonen, A., Nylund, A., & Pihlaja, P. (2021).
Communication skills predict social-emotional competencies. Journal of
Communication Disorders, 93, 106138.
https://doi.org/10.1016/j.jcomdis.2021.106138
Rerke, V. I., Sukhostavska, Y. V., Krasheninnikova, N. A., Khvataeva, N. P., Sakharova, N.
S., & Rogozhnikova, R. A. (2018). Ethno mediation in international conflicts
prevention in conditions of polyethnic educational environment. Revista Espacios,
39(20), 9. http://www.revistaespacios.com/a18v39n20/18392009.html
Rubtsova, A. (2019). Socio-linguistic innovations in education: productive implementation
of intercultural communication. In IOP Conference Series: Materials Science and
Engineering (Vol. 497, No. 1, p. 012059). IOP Publishing.
https://doi.org/10.1088/1757-899X/497/1/012059
Salpykova, I. M., Khurmatullina, R., Garifullina, A., & Shaehova, R. (2022). The
Development of General Cultural Competencies among Students by Means of E-
Learning. ARPHA Proceedings, 5, 1433-1453. doi:10.3897/ap.5.e1433
Schembri, N. (2021). Developing linguistic and cultural competence: Participants'
evaluation of a Maltese language and culture awareness program. Advances in
Social Sciences Research Journal, 8(5), 251-260.
https://doi.org/10.14738/assrj.85.10195
Shafazhinskaya, N. E., Shcherbinina, V. M., Ivanova, E. Y., Belyakova, T. E., &
Pereverzeva, M. V. (2019). Learning about world art culture as a method of
forming a universal cross-cultural communication competence. Humanities &
Social Sciences Reviews, 7(6), 1225-1229. https://doi.org/10.18510/hssr.2019.76173
Shmeleva, Z. N. (2019a). The development of cross-cultural tolerance of Bachelors by
means of studying the English language at Krasnoyarsk SAU. Baltic Humanitarian
Journal, 8(2(27)), 116-120. DOI: 10.26140/bgz3-2019-0802-0028
Shmeleva, Z. N. (2019b). The formation of cross-cultural competence of students-
managers by means of the foreign language learning at the non-linguistic
university. Azimuth of Scientific Research: Pedagogy and Psychology, 8(2(27)), 271-275.
DOI: 10.26140/anip-2019-0802-0062
Siregar, D. I., Binangkit, I. D., & Wibowo, A. P. (2021). Exploring the internationalisation
strategy of higher education: A case of Muhammadiyah Higher Institution.
Journal of Education and Learning (EduLearn), 15(1), 55-63.
Smakova, K., & Paulsrud, B. (2020). Intercultural communicative competence in English
language teaching in Kazakhstan. Issues in Educational Research, 30(2), 691–708.
https://search.informit.org/doi/10.3316/ielapa.266358342762421
Sobkin, V. S., & Adamchuk, D. V. (2015). On teacher professional development: Improving
professional qualifications and membership in professional teacher communities.
Russian Education & Society, 57(11), 991-1017,
https://doi.org/10.1080/10609393.2015.1187012
377
Some-Guiebre, E. (2020). Foreign Language Classroom Interaction: Does it Promote

Communicative Skills? International Journal of Educational Methodology, 6(3), 497–
505. doi: 10.12973/ijem.6.3.497
The Norwegian National Committee for Research Ethics in Science and Technology.
(2016). Guidelines for research ethics in science and technology. National Research
Ethics Committees. https://www.forskningsetikk.no/en/guidelines/science-
and-technology/guidelines-for-research-ethics-in-science-and-technology/
Thomas, K. W., & Kilmann, R. H. (1978). Comparison of four instruments measuring
conflict behavior. Psychological Reports, 42(3 Suppl), 1139–1145.
https://doi.org/10.2466/pr0.1978.42.3c.1139
Tleubay, S., Nurzhanova, G., Ybyshova, S., Abdigulova, S., Mankesh, A., Kerimbekov, T.,
& Ualikhanuly, A. (2020). The formation of intercultural communicative
competence of future teachers in a trilingual educational environment.
International Journal of Emerging Technologies in Learning (IJET), 15(17), 148-164.
Tsalikova, I. K., & Pakhotina, S. V. (2021). Scientific research on the issue of soft skills
development (review of the data in international databases of Scopus, web of
science). The Education and Science Journal, 21(8), 187-207.
http://dx.doi.org/10.17853/1994-5639-2019-7-187-207
Vurdien, R., & Puranen, P. (2018). Intercultural Learning via Videoconferencing. In Cross-
Cultural Perspectives on Technology-Enhanced Language Learning (pp. 264–282). IGI
Warshaw, J. S., Crume, P., & Pinzon-Perez, H. (2020). Impact of service-learning on
Hispanic college students: Building multi-cultural competence. International
Journal of Multicultural Education, 22(3), 50-75.
https://doi.org/10.18251/ijme.v22i3.2413
White, C., & Saqipi, B. (2021). Intercultural education in Kosovo: A longitudinal case
study. Journal of Education and Learning (EduLearn), 15(1), 135-143.
Yang, P. (2018). Developing TESOL teacher intercultural identity: An intercultural
communication competence approach. TESOL Journal, 9(3), 525-541.
https://doi.org/10.1002/tesj.356
Yasar Yuzlu, M., & Dikilitas, K. (2022). Translanguaging in the development of EFL
learners’ foreign language skills in Turkish context. Innovation in Language
Learning and Teaching, 16(2), 176-190.
https://doi.org/10.1080/17501229.2021.1892698
Yelubayeva, P., & Mustafina, A. (2020). Developing Kazakh students' intercultural
awareness and communication through collaborative technologies. European
Journal of Language Policy, 12(2), 235-255. https://doi.org/10.3828/ejlp.2020.12
Zimina, N.A. (2015). Psikhologicheskaia diagnostika kommunikativnogo potentsiala
lichnosti: metodicheskie rekomendatsii dlia studentov [Psychological diagnostics
of communicative potential of personality: methodological recommendations for
students]. Polygraph Center of NNGASU.
https://bibl.nngasu.ru/electronicresources/uch-metod/psychology/859788.pdf
Zoyirova, D. A. (2018). Forming discursive competence of law students. Eastern European
Scientific Journal, 6. http://journale.auris-
verlag.de/index.php/EESJ/article/viewFile/805/884
Zubkov, A. D. (2020). Professional foreign language competence of technical students:
Content, structure and formation. In Z. Anikina (Ed.), IEEHGIP 2022: Integrating
engineering education and humanities for global intercultural perspectives (pp. 503-510).
Springer. https://doi.org/10.1007/978-3-030-47415-7_53
378

Vol. 22, No. 10, pp. 378-393, October 2023
Received Aug 11, 2023; Revised Sep 18, 2023; Accepted Oct 19, 2023
De/colonising Theoretical Literatures and the

Educational Qualifications to Unpack the
Grotesque Skills Gap in South Africa
Christiana Kappo-Abidemi
(Human Resource Management) School of Development Studies
University of Mpumalanga, Mbombela, Mpumalanga
South Africa
Christopher Babatunde Ogunyemi*
(English) School of Social Sciences
University of Mpumalanga
Mbombela, Mpumalanga, South Africa
Abstract: The research uses decolonial theory to delineate the

conceptualisation of the academic qualifications provided by South
African tertiary institutions and their relevance to closing the national
skills gap. Decolonising the South African educational system represents
a progressive approach that is new to both students and educators. Both
government and employers acknowledge a skills shortage in South
Africa. However, the question of where the skills should be provided is
unanswered. Graduates are expected to have specific skills before
entering the labour market. The study uses desktop research that focuses
on secondary data to explore the phenomenon. The research has two
objectives: to present a decolonial explanation of the relationship between
academic qualifications and national skills needs and to explore ways of
integrating current academic qualifications offered by educational
institutions in South Africa to address the national skills gap. The study
explores how the content of the National Qualification Framework and
its relevance in skills development can be decolonised. The study found
that the existing educational qualification programmes in South African
tertiary institutions are unable to address the skills gap effectively due to
emerging needs highlighted in recent literature. To address this issue,
there is a need to restructure academic qualifications to better align them
with the country's skill shortages or explore alternative approaches to
aligning knowledge and skills. The study recommends examining the
national skills shortage beyond the higher education institutions’
academic structure and looking at other factors, such as workplace
discrimination in training and development. Collaboration between
stakeholders, employers and tertiary education institutions should be
*
Corresponding author: Christopher Babatunde Ogunyemi; christopher.ogunyemi@ump.ac.za
©Authors
379
encouraged for curriculum development and alternative means of skills

acquisition.
Keywords: decolonialization; educational qualifications; skills shortage;

knowledge acquisition

The South African labour market has always been perceived to lack critical skills
for nation-building and economic development (Field, Musset & Alvarez-Galvan,
2014). Over the years, the nation has relied on European and other developed
countries for expatriates to provide these skills. There is a general perception that
South Africa’s present skills development system has mostly failed to meet the
country’s skills requirements. In South Africa, there are concerns about the
economy and its potential to produce inclusive growth that will significantly
reduce unemployment and poverty. Access to education and skills is a critical
facilitator of economic progress. Therefore, there is a need to provide a framework
for the development of skills for highly competent individuals to occupy jobs that
will be beneficial to all in a diversified and dynamic society such as South Africa
(Reddy et al., 2016).
Consequently, skilled people could be able go into business for themselves and
create income-generating companies to support their families and grow to
generate job possibilities. Furthermore, in a climate of fast economic structural
change, persons with skills are more likely to be able to adapt to changing
economic requirements and therefore remain employed. A lack of education and
skills characterises the South African labour market and is reflected in several
worldwide indexes (Mouton et al., 2013) . Many people who drop out of school
do so without receiving fundamental knowledge. Acceptable qualifications are
needed by some employers before any recruitment is done, but vocational
credentials at various National Qualification Framework (NQF) levels do not fully
equip learners for the workplace. The low fundamental education basis in South
Africa is influenced by a combination of factors, including inadequate funding,
overcrowded classrooms, a lack of qualified teachers, outdated teaching methods,
socioeconomic disparities, high dropout rates, limited access to technology,
language barriers, insufficient parental involvement and cultural and societal
factors (Maisari et al., 2019). In addition, the skills system is underperforming,
often seen as failing to provide the necessary skills for both new entrants and
existing employees (Human Resource Development Council (HRDC), 2013).
South Africa has 26 universities that graduate thousands of students with various
degrees and qualifications yearly, yet the number of unemployed youths
constantly increases. The Quarterly Report (Statistics South Africa, 2021)
indicated a 34.9% unemployment rate in the first quarter of the year. The statistics
show that 2.4% of the total unemployed people were graduates. However, in 2021,
the Department of Water and Sanitation brought in 24 Cuban engineers to help
with infrastructure challenges and exchange knowledge. Likewise, in the
preceding year, about 200 doctors were brought from Cuban to South Africa to
help fight the coronavirus (Business Tech, 2021: Jika, 2020). Why were South
Africans not used when more than 50% of universities produce doctors and
380
engineers yearly? How do we justify the lack of critical skills South African
graduate doctors and engineers need to tackle the country’s economic needs with
their academic knowledge?
The common practice amongst academic institutions is to develop an educational
curriculum based on existing knowledge and practice. However, existing
knowledge may be irrelevant in certain situations, especially in this generation,
where learning and new challenges constantly evolve, and the workplace is
expected to adapt. As much as there is always a demand for new skills in the
workplace, the academic knowledge provided by educational institutions is also
expected to be updated (Modiba, n.d.). In addition, academic institutions are
expected to find common ground where knowledge can be transmitted to meet
the demands of the workplace. The skills gap will continue to widen as long as
knowledge is acquired in isolation. Likewise, unemployment will increase as
much as academic knowledge fails to be converted into workplace-needed skills.
Some people have had to devote quality time to obtain their university degrees
simply because some jobs require them to do so. While past generations may have
had only one job throughout their lives, it is now common practice to have
multiple jobs and careers. With the development of the economy, the upcoming
generations may expect to have a variety of positions and vocations throughout
their professional life (Kumar,2020). There is an emphasis on balancing what is
studied and what is required to do the job effectively. Most university graduates
are underemployed due to the mismatch between academic qualifications and the
workplace skills requirement. “Visible underemployment” refers to those forced
to work fewer hours than they would want. These people are looking for or wish
to work longer or full-time hours. Another description of underemployment is
people doing work that requires a lower qualification than the one they have
acquired (Greenwood, 1999; Mncayi & Dunga, 2016). The most disturbing aspect
is that graduates are struggling to transform their academic qualifications into
employable skills. Hence, the study explores the gaps between educational
qualifications and the workplace needs of South Africa and global work demand
and ways to close these gaps to improve the human capacity development of the
nation for economic enrichment. The study will debate how the academic
qualifications offered by South African universities and other higher academic
institutions can be used to meet the demand for skills needed for economic growth
and reduce unemployment among youths. The study will discuss different
educational structures in South Africa and their relevance to the employability of
citizens. Also, the South African national qualifications framework overseen by
the South African Qualification Authority (SAQA) will be compared to relevant
similar bodies in developed countries.
Kappo-Abidemi and Ogujiuba (2021) reckon that the corporate social
responsibility of higher education institutions (HEIs), particularly in South Africa,
should focus on the educational and skill development of historically
disadvantaged individuals and on providing employment opportunities for
graduates. The HRDC (2013) acknowledged that the present skills development
system has mostly failed to meet South Africa’s skills requirements. Likewise,
various challenges within the education system inhibit it from functioning
effectively to address the workplace knowledge, skills and capabilities of fresh
381
graduates of HEIs. HEIs in South Africa have focused on massifying education

since the onset of democracy in 1994. However, as much as the HEIs should be
accessible to all, quality cannot be compromised. Class sizes are overwhelmingly
large in the current higher education system. Lecturers struggle to control the
classes; meaningful, practically oriented engagement is almost impossible in an
overpopulated classroom environment. Adetiba (2019) and Tanga and Maphosa
(2018) acknowledged that contrary to expectations, some South African
universities may not achieve the anticipated improvement of the social order
because of the problematic implementation of policies addressing these
educational scenarios.
The study aims to critically examine the content of the current academic
qualifications offered in South Africa and their contributions to the national skills
gap. The NQF overseen by SAQA will be used as a foundation for discussing the
content of the academic qualifications offered in the country. Hence, the study has
the following objectives:
• To reconstruct decoloniality and explain the relationship between
academic qualifications and national skills needs.
• To explore various ways of integrating current academic qualifications
offered by educational institutions in South Africa into the national skills
gap.
The decolonial theory is significant to the understanding of our study. The theory
supports the contribution of new knowledge of indigenous systems and how the
South African educational system could positively benefit from this phenomenon.
According to Gyamera and Burke (2018), decolonial theory supports the
acquisition of skills by challenging traditional western-centric knowledge systems
and advocating for a more inclusive and diverse curriculum that incorporates
indigenous and non-Western perspectives, thereby enabling learners to develop
a broader and more culturally sensitive skillset. This approach encourages critical
thinking, creativity, and a deeper understanding of global perspectives,
ultimately enhancing individuals' capacity to navigate an increasingly
interconnected and diverse world. These values are encapsulated in the Critical
Cross-Field Outcomes (CCFO) of the NQF but are rarely integrated into actual
learning content of HEIs. However, the overemphasis on tests and examinations
to assess knowledge limits the acquisition of skills attributed to the knowledge
(Faller et al., 2023). Kolb (2020) acknowledged that skills need to be related to
future work-related endeavours.
The configuration of decoloniality is to ‘delink’ the Eurocentric perception of
thoughts for indigenous concepts that are thought-provoking, independent and
value-oriented. The need emerges to re-write the trajectory of oppression and
colonialism that the subjugated suffered. As an academic discourse, though it is
extremely difficult to emancipate people from traditional ways of thinking,
decoloniality challenges the ongoing economic constraints caused by harmful past
practices and strong Western influence .
According to Ogunyemi (2021), decoloniality is a systematic incorporation of
thoughts that enhance liberation in the economic and the social-cultural context.
382
It is a conscious mind-set phenomenon that aims at eliminating the monopoly of

knowledge and imperialism in the social systems. He further emphasises that
decolonising knowledge is an academic and intellectual endeavour that promotes
the use of Afrocentric perspectives to challenge Western dominance and
oppression in education. This aligns with Spivak's idea, which emphasises the
importance of rejecting the belief in white superiority to promote the dignity and
success not only of black people worldwide but also the development of Africa as
a continent.
Mignolo (2011), Vallego (2015) and Quijano (2007) unapologetically posit the
notion that European epistemology has lent credence to the social and economic
inequality of the oppressed in society. Though the origin of decolonial theory can
be clearly traced to Latin merica, it significantly reflects the cultural ethos of
contemporary deprived societies. It also reflected the depraved societies of Africa
and South-East Asia and other post-colonial settings in the world. Mignolo,
Valtego and Quijano joined forces to deconstruct the western matrix of power,
using the decolonial lens as a drastic shift from the subjugation of the weak to
constructively create a new epistemology of liberation and awareness that should
finds its roots in educational life. This new approach is a configuration of
originality underpinning the reconstruction of awareness and new knowledge
which can revolutionise education in South Africa. African countries will
obviously benefit from educational reconstruction that contributes to reflective
observation and the enhancement of learning processes in educational
development.
Reflective observation is a skill or method of learning that calls for reflection or
observation on recently experienced events understood through tangible
experience. The action allows the learner to reflect on the learning experiences that
have taken place. The learner expresses their opinion about the learning by asking
questions and discussing the experience with others. Bouw et al. (2021) posit that
reflective observation happens through integrative learning because aspects of
school and work must be purposefully blended within the learning environment.
A curriculum that focuses on vocational education affects learning environment
design. At the intersection between school and work, intentionally created
learning environments or systems include coaching by professionals from
occupational practice on genuine goal-directed work activities and the provision
of physical spaces where students can apply their academic knowledge in
developing their skills.
Furthermore, observational learning is the process of learning by observing the
actions and results of others under specific conditions. In contrast to imitation,
observational learning entails more than copying what others do. When stakes are
high, students need to be able to follow, evaluate and learn from observed
behaviours and consequences (Kang, Hernandez & Mei, 2021). Dean (2019)
explains that observational learning can be achieved through Work-Integrated
Learning (WIL) across all disciplines, qualifications and academic institutions.
The workplace or community is a place for WIL, where learners can observe and
participate in transferring theoretical knowledge into actual practice. Ma et al.
(2020) maintains that observational learning develops the understanding of a
383
learner. Reflective observation is closely related to situational learning, which

acknowledges that learning occurs best when it takes place in the context in which
it is applied (Egbert & Sanden, 2019). Socio-cultural contexts need to be
considered in applying knowledge in situated environments (Cheng et al., 2019).
In South Africa, learners’ work can be reflected in their active life experiences to
integrate them into the new knowledge that is required to address the diverse
needs of the economic environment.
Another factor to note is ‘abstract conceptualisation’, which is the notion of
developing abstract concepts that can be generalised and applied to various
situations. The conceptualisation of knowledge is a learning and teaching method
that processes knowledge through a systematic thinking process.The learners’ or
facilitators’ ability to apply the abstract concepts in practice and understand what
it means to work in an actual situation will enhance the knowledge of the learner
(McMullan & Cahoon, 1979).
3. Active Experimentation
Figure 1: A conceptual framework for the study
The national focus and concerns over the years have been the issue of skills gap
and skills development. Employers have constantly been raising issues about the
lack of skills in different sectors, but hardly anyone can define where the skills
should be acquired. Higher educational institutions’ qualifications have always
been the focus of skills acquisition. Even the recently released critical skills list by
the Immigration Act (Department of Home Affairs, 2022) quantified the identified
skills in terms of the NQF levels provided by the SAQA Act 1995. The NQF
allocates different levels, namely NQF Level 1 to NQF Level 10 to qualifications
to structure the basic requirements for progression at each level. NQF Level 1
384
means that the student has completed Grade 9 (i.e. the compulsory level of
schooling) and is in a position to advance to further education and training either
at a school to obtain the Graade 12 certificate or through a Technical & Vocational
Education & Training (TVET) college, both of which will take them to NQF Level
4. NQF Level 10 is the highest NQF Level where students obtain their doctorate
degree. The conceptual framework in Figure 1 shows that educational
qualifications needed by the labour market can be achieved at any level of
education. Likewise, the skill gaps can be bridged at all stages of learning,
provided the skills are integrated into the theoretical knowledge.
According to Hernandez-March et al. (2009), higher education policies are
increasingly included on government agendas at both the national and
supranational levels. Higher education is critical for a country’s social well-being
and economic prosperity. A country’s ability to compete in today’s knowledge-
based global economy depends on a stable supply of highly qualified workers, a
sustainable system of research and development, and an efficient mechanism for
transferring knowledge from universities and public research organisations to the
business network. This shows that HEIs are more focused on knowledge-based
education than skills acquisition. However, there is a need to find common
ground between theoretical knowledge offered by academic institutions and
practical knowledge required by the economy to bridge the skills gap.
The theoretical framework adopted for the paper encourages a practical
adaptation of theoretical knowledge, whereby the HEIs provide an avenue for
knowledge to be practised. The conceptual reality is that academic institutions
and qualifications in South Africa are structured for various purposes from which
they are not allowed to deviate. However, the skills gaps could be addressed using
multiple means rather than focusing on baseline academic qualifications provided
by tertiary institutions. Mkhonza and Letsoalo (2017) maintain that there are two
ways for organisations to fill their skill gaps, either by upskilling or hiring.
Upskilling requires training the existing workforce in the necessary skills while
hiring means appointing a new person from outside the organisation that
possesses the needed skills. The important point is that the skills have to somehow
be acquired from some source.
3.1 Academic Qualifications in South Africa
Employers in South Africa and around the world want graduates who are ready
to work. The ongoing debates about graduates from higher education institutions
lacking skills and knowledge have increased employers’ scepticism about
graduates’ ability to perform effectively in the workplace once hired.
It is expected by various stakeholders that educational institutions like schools,
colleges and HEIs should prepare individuals for life via skills development. At
the most basic level, such development, which includes teaching reading and
numeracy skills, aids in creating work prospects. The human capital paradigm
states that, contrary to popular belief, the number of years spent in education
improves one’s capacity to find work. However, it is argued that skilled worker
unemployment is complex and linked to factors such as educational quality, lack
of experience, discrimination and inflexibility (Mobarak, 2019). This view
confirms that unemployment is a socio-economic issue that is detrimental to
385
economic welfare and output and diminishes human capital (Mpendule &
Mang’unyi, 2018). There are prominent players in the area who are impacted and
influenced by the qualification system. Schools, universities, companies and
learning centres are examples of providers. Employers fund a significant portion
of adolescent and adult work-related training and education, and as gatekeepers,
they may encourage, facilitate or prevent access to learning.
A qualifications framework is a tool for developing and categorising
qualifications based on learning levels’ criteria. This collection of standards might
be hidden inside the qualifications descriptors or made apparent through a set of
level descriptors. The scope of frameworks may be broad, encompassing all
learning outcomes and routes, or narrowed to a specific sector, such as early
childhood education, adult education and training or a specific vocational field.
Some frameworks have more design features and a more rigid structure than
others; some have a legal foundation, while others represent a consensus of social
partners’ viewpoints (OECD, 2002). Similarly, Behringer and Coles (2003) describe
a qualification as a unit of recognised learning outcomes such as certificates,
diplomas, degrees and licences. Academic qualifications that will enhance skills
development and bridge the skills gap cannot be achieved without mentioning
lifelong learning relevant to organisational needs and individual development.
Lifelong learning is a personal activity. One of the factors explored by this paper
is how the qualification system influences people’s motivation, inclination and
ability to pursue more education. Lifelong learning is essential for adults, but it
also applies to school children, students and working and non-working
individuals. Even the most developed nations have begun a quest to continually
grow the educational system and improve the quality of education since
education has attained a level that cannot be restricted to specific periods in the
human lifespan (Solmaz, 2017).
The South African academic qualifications are described in terms of the NQF
levels provided by SAQA, which could be achieved through different means. The
employers assume that the required knowledge and skills for the nation and
businesses should be gained from HEIs. The ten levels of National Qualifications
Framework focuse on scope of knowledge, knowledge literacy, method and
procedure, problem-solving, ethics and professional practice, assessing,
processing and managing information, producing and communicating of
information, context and systems, managing of learning and accountability in
order to align competence in both intellectual and academic skills of knowledge
to analyse, synthesis and evaluate information (SAQA, 2012; van Huyssteen,
2002). Hence, the NQF makes provision for alignment of knowledge to skills in
principle but the process is not clear in practice. However, the universities are
structured differently for various purposes. Universities of Technology are more
aligned with workplace skills acquisitions. At the same time, traditional
universities are more knowledge-oriented, and comprehensive universities offer
programmes that could be either knowledge-based or skills-based. However,
Reddy et al. (2016) opined that basic education also plays a critical role in the
labour market’s education and training since most of the South African workforce
does not have tertiary education.
386
For this reason, discussing the difference between knowledge and skills is
essential. According to Boulet (2015) and Conway (2022), knowledge is
information gained through sensory input, such as reading, watching, listening or
touching. Knowledge is defined as familiarity with information and theoretical
concepts. Knowledge can be transferred from one person to another or gained
through observation and study. Likewise, skills refer to the practical application
of knowledge in a specific situation through sensory input and output and are
developed through practice. Social skills, for example, are developed by engaging
with people through observation, listening and speaking. Continuous practice
and engagement is probably the best way to master new skills. Alternatively,
work-related skills can be acquired through workplace training and development
programmes (Andriotis, 2017).
Racial and institutional divisions and disparities have marked South Africa’s
higher education system. Van Huyssteen (2002) acknowledges that the
universities of technology, formerly technikons, were historically planned to
award various career-focused qualifications focusing on implementing existing
knowledge, skills and procedures such as National Certificates and National
Diplomas. Before 1994, South Africa’s technikons concentrated on educating
skilled employees at the pre-professional level, primarily engineering
practitioners and technicians in various sectors, such as biotechnology, health
sciences, nature conservation, auditing, design, film and video. The distinction
between technikons and universities is that, traditionally, technikons have a
strong vocational focus, including their closeness and interaction with
workplaces, and the lower entry requirements for most technikon programmes,
resulting in a more inclusive student body than that of more elite universities.
Technikons’ industry ties were maintained through a structured and assessed
system of workplace learning in most diplomas and a regular system of industrial
advisory bodies attached to each programme (Department of Education, 2004;
Garraway & Winberg, 2019; Mentz et al., 2008). Universities of technology seems
to be better equipped to award skills-based qualifications.
The comprehensive universities, in contrast, were designed to confer degrees at
the Bachelor (3 years), Honours Bachelor (Bachelor + 1 year), Master’s and
doctoral levels. For both types of institutions, the Standard 10 (Grade 12)
certificate level, which is based on externally assessed and moderated tests on a
national basis, is the entry point into undergraduate qualifications. The standard
qualification for leaving school in South Africa is Grade 12 (sometimes called
“Matric”). Universities of technology (technikons) accepted a Grade 12 pass, but
universities demanded a “Full Matriculation Exemption certificate” requiring
particular course combinations and a higher minimum pass mark. Since the early
1990s, technikons have been permitted to provide degrees (BTech, MTech,
DTech). The ranges of qualifications supplied by the two types of institutions have
remained unique despite these developments, and there has been difficulty
articulating between the two systems (Garraway & Winberg, 2019; Mentz et al.,
2008).
According to the Department of Education (2004), changes in academic disciplines
and knowledge fields typically influence university programmes from within. It
is generally a professional curriculum that emphasises the profession and the
387
body or council regulating it. Many other university programmes will probably
lack the external contacts necessary to launch and run technikon-type
programmes successfully. In these situations, institutions must simultaneously
work on developing programmes and a network of partnerships with outside
organisations representing business, industry and the community in order to
inform curriculum development, create opportunities for cooperative and in-
service learning, and keep up with the evolving skill needs of the labour market.
The traditional universities are purely academic and offer Bachelor’s degree
programmes only for at least three years. The admission criteria for a traditional
university degree programme are slightly higher than that of a National Diploma
and National Certificate, and the focus is on acquiring workplace knowledge. The
focus of traditional universities is on general formative education with a
theoretical orientation (Department of Education, 2004).
3.2 Legal Framework for Academic Qualifications in South Africa
The National Qualification Act of 2008 by SAQA established the framework, while
the Skills Development Act 97 of 1998 governs the national skill requirements. The
ten NQF levels are sub-divided into GFETQSF (Level 1-4) and HEQSF (Level 5-
10). The GFETQSF comprises the General Certificate at level 1, a Grade 9
qualification, an Occupational Certificate at level 1, Elementary Certificate at level
2, a Grade 10 qualification and an Occupational Certificate at level 2 while the
Intermediate certificate is at level 3, a grade 11 qualification and National
Certificate at level 4, a Grade 12 qualification and the Occupational Certificate at
level 4.
The Higher Education Qualifications Sub-Frameworks (HEQSF) and General and
Further Education and Training Qualifications Sub-Frameworks (GFETQSF) are
the two sub-frameworks that make up South Africa’s ten (10) NQF levels for
academic qualifications. The Occupational Qualification Sub-Framework (OQSF),
which is comprised of eight (8) levels, is the third NQF Sub-Framework (Durham,
2021). Each framework represents the lowest rank to the highest rank of
qualifications. The qualification levels in South Africa are governed by legislation
that applies to academic and occupational certifications. According to van
Huyssteen (2002), the creation and implementation of these frameworks reflect an
audacious and daring undertaking to unite all learning under a single framework
of standards and certificates based on outcomes, with built-in quality assurance
procedures.
NQF levels give prospective employees information about education and skills.
They make it easier to select the job applicant who is most qualified and fit for an
open position. As a result, NQF certifications also assist students in making the
right decision regarding their future careers. The NQF describes the abilities
needed for a particular employment field and how to acquire those skills
(Durham, 2021). Reddy et al. (2016) indicated that 11.75 million persons in the
South African labour force have less than a Grade 12 certificate out of 15 million
employed people in South Africa, showing that more than 70% of the South
African workforce were employed at NQF level 1-3 which might be an inhibition
to career progression and skills development in the workplace.
388
The HEQSF includes a Higher Certificate at level 5 and an Occupational

Certificate at level 5, a Diploma and Advance Certificate at level 6 and an
Occupational Certificate at level 6, a Bachelor’s Degree and an Advance Diploma
at level 7, an Occupational Certificate at level 8, a Bachelors Honours Degree,
Postgraduate Degree, and a Bachelor Degree at level 9, and a Doctorate Degree.
Nevertheless, access to schools, universities and TVET programmes has improved
recently. However, quality is still difficult to come by, which leads to poor
academic advancement across all educational courses and low completion rates
at academic institutions, TVET colleges, and universities (Reddy et al., 2016).
Consequently, because of the attrition rate, the accessibility to HEIs does not
always translate to the necessary national competencies. Achievement in the
school subjects of Languages, Mathematics, and Science provides the basis for
participation and achievement in technical themes in post-secondary education
and training institutions and work. Rahmat et al. (2012) identified four generic
abilities that graduates must possess in order to succeed in the workplace:
academic, connectedness, personal management and exploration.
For holistic employees, discipline-specific talents should supplement these
abilities. According to Griesel and Parker (2009), the four main, interconnected
factors that affect employability are effective practices (communication, time
management, self-management, problem-solving, and lifelong learning);
profound understandings rooted in a disciplinary base (specialised expertise in a
field of knowledge); practical beliefs about one’s own identity and self-worth; and
metacognition (self-awareness and the capacity to reflect on, in and for action).
However, as indicated above, Reddy et al. (2016) assert that more than 70% of the
country’s present workforce has less than a Grade 12 education, which
demonstrates that the majority of South Africans in employment left school before
even acquiring the information and skills needed for the workplace. Durham
(2021) claims that the NQF system’s framework gives options and support to
those without official qualifications who want to change occupations, upskill
themselves or those who dropped out of school before acquiring their Grade 12
certificate.
The Skills Development Act provides an administrative framework for creating
and putting into effect national workplace and sector programmes to advance and
improve the skills of the South African workforce, in addition to the NQF Act .
The Skills Development Act 97 of 1998 was passed to correct a historical injustice
in which some groups of people were excluded from educational opportunities.
This Act made it possible to continue pursuing education and advancing one’s
abilities while working (Aigbavboa et al. 2016). The Act encourages employers to
provide an environment at the workplace where people may actively learn new
things and develop their knowledge. Additionally, companies were mandated to
contribute to the annual skills development plan for their workers under the Skills
Development Levies Act 1999, which is how the learning is funded. The
Employment Tax Incentive Act 26 of 2013 was also introduced to address the
unemployment problems among young people who were excluded from
economic activity due to potential employers’ regularly voiced reluctance to
recruit young job searchers, mainly because young job applicants lack the
necessary education, credentials and experience. A well-crafted incentive is
389
required, and a sizeable financial commitment is needed to provide the essential

expertise and experience (Smale, 2012).
3.3 National Qualification Frameworks
An NQF is generally developed as part of a country’s development initiatives,
showing broad consensus regarding the form that qualifications should take and
the desired outcomes of the initiative (Tuck, 2007; Young, 2005). Tuck (2007)
explained that, historically, countries such as Australia, New Zealand and South
Africa were the first countries to introduce national qualification frameworks
between 1980 and 1990, while Ireland, Malaysia, Namibia and a few other
countries followed suit as the second generation between the late 1990s and early
2000s. Countries such as Albania, Botswana, Angola, Lesotho and many others
are still considering whether to have national qualification frameworks. In
addition to NQFs, regions such as European Union, Pacific Islands and the
Southern African Development Community are considering having a regional
qualification framework (RFQs).
It is frequently asserted that NQFs will help certain countries, regions and the
global economy achieve their objectives. From a broad policy perspective, NQFs
and international frameworks provide obvious benefits for related reasons.
Employers and admissions administrators or registration officers will have a
legally recognised framework to compare various qualifications if an NQF is
available. A European Qualifications Framework will assist regional
organisations like the EU remove obstacles to unrestricted labour movement
among member states. For students, an NQF will recognise credentials on a wider
scale; for example, informal learning should theoretically also be eligible for
accreditation by means of a recognition of prior learning policy (Young, 2007).
Young (2007) notes further important distinctions that need to be considered
when analysing the possible drivers of the increasing interest in NQFs. The first
is the difference between often irrational expectations of what an NQF’s
implementation will achieve and NQFs as a practical response to real political,
economic and educational concerns like changes in labour markets and the
growth and diversification of post-compulsory education. The second distinction
is between the general idea of an agreed-upon set of qualities shared by a nation
or group of nations and the widely varying ways the concept of a framework has
been embraced in other nations. Beyond these distinctions, the NQF was created
to recognise management qualifications and informal and non-formal learning
(Behringer, 2003).
As indicated earlier, a major global trend in modernising national education and
training systems since the late 1990s has been the creation of NQFs. The
International Labour Organisation’s Recommendation No. 195 on human
resources development: education, training and lifelong learning, which was
adopted in June 2004 (Tuck, 2007; Young, 2005), recommended the adoption of an
NQF as a means of promoting the development, implementation and financing of
a transparent mechanism for the assessment, certification and recognition of
skills. Typically, a diagrammatical picture of the South African NQF is used to
display and discuss it. It shows the structure from the highest qualification (a
doctorate) to the General Certificate.
390
The study used a methodical literature review that was carefully planned and
carried out. The skills gap in South Africa has been the subject of research that
looked at elements impacting the theoretical literatures of decolonising and
educational degrees. Only articles that were released during the 10 years before
the current study were deemed to cover current trends and to be pertinent.
Additionally, studies that looked at outside influences on the creation of academic
curricula were also examined in publications.
4.1 Research design and scientific literature search
In order to seek, classify, plan and critically assess the literature in order to
develop themes, a hermeneutic outline was used (Boell & Cecez-Kecmanovic,
2014). The journal articles were located using a variety of electronic bibliographic
databases, databases from various disciplines and websites. Academic
publications, Google Scholar, company blogs, company websites, Emerald Insight
and AOSIS are all included in this database. In a search engine, the terms
“decolonisation, educational requirements, skill shortage, and knowledge
acquisition” were entered. With the use of these keywords, the study was able to
locate themes and gain a deeper understanding of the internal and external
elements affecting employee performance. To be further vetted, all the structures
were recorded in an MS Excel spreadsheet. The spreadsheet was examined more
closely in order to categorise relevant prior research that is consistent with the
goals of the current study. To ensure that only papers that could contribute to the
achievement of the goals of this study were examined, the pertinent articles were
carefully and purposefully chosen. In this essay, internal and external factors
impacting employee performance served as the unit of analysis. 95 percent of the
articles that were chosen were released during the last five years. Because of this,
the review was best based on current empirical studies that concentrated on the
variables driving the skills gap in South Africa.
5. Conclusion
The research conceptualises the teleology of decolonisation in the description of
South African educational qualifications. In an attempt to unpack the dire skills’
shortages in South Africa, the paper demonstrates various educational
qualifications and the inadequacies in meeting the skkills demand of the
economy. The history of education in South Africa has developed from the
colonial dispensation to the current system of educational qualifications in South
Africa. The article suggests how decolonising knowledge could add a new
dimension to the body politic of learners when new configurations are added that
would make room for employable skills within the socio-economic space of South
Africa. Using the decolonial lens, the research showcases how indigenous
knowledge could be incorporated into the educational domain which would
invariably develop the South African educational system and make it relevant in
the African cosmology.
Acknowledgements
Dr. Christiana Kappo-Abidemi is thankful to the National Research Foundation
(NRF), South Africa for a Thuthuka Grant (TTK210412593900) towards this
391
research work. The University of Mpumalanga Research Office is gratefully

acknowledged for support.
6. References
Adetiba, T. C. (2019, October). Massification of higher education in South Africa, the
good, the bad and the ugly. In Proceedings of International Academic
Conferences (No. 9410873). International Institute of Social and Economic
Sciences.
Aigbavboa, C. Oke, A. & Mokasha, M (2016). Implementation of Skills Development Act
in the South African construction industry. The Scientific Journal for Theory and
Practice of Socio-Economic Development 5(9): 53-64
Andriotis, N. (2017). How to identify and address skill gaps at work. EFront Learning.
https://www.efrontlearning.com/blog/2017/10/identify-skill-gaps-
workplace.html
Behringer, F & Coles, M. (2003). The role of national qualifications systems in promoting
lifelong learning. OECD Education Working Papers, No.3, OECD Publishing doi:
10.178/224841854572
Boell, S. K., & Cecez-Kecmanovic, D. (2014). A hermeneutic approach for conducting
literature reviews and literature searches. Communications of the Association for
Information Systems, 34(1), 12.
Boulet, G. (2015). Identifying the difference between knowledge and skills. ELearning
Industry. https://elearningindustry.com/difference-between-knowledge-and-
skills-knowing-not-make-skilled
Bouw, E., Zitler, I., & Bruijn, E. (2021). Designable elements of Integrative Learning
environments at the boundary of school and work. A multiple case study.
Learning Environment Research, 24, 487-517.
Business Tech. (2021). South Africa to bring in 24 Cuban engineers to help with infracture
issues. https://businesstech.co.za/news/business/484165/south-africa-to-bring-
in-24-cuban-engineers-to-help-with-infrastructure-issues/
Cheng, S. C., Hwang, G. J., & Chen, C. H. (2019). From reflective observation to active
learning: A mobile experiential learning approach for environmental science
education. British Journal of Educational Technology, 50(5), 2251-2270.
Conway, B. (2022, February 14). The difference between knowledge, skills and abilities.
Employee Connect. https://www.employeeconnect.com/blog/difference-
between-knowledge-skills-abilities-resume/
Dean, B. (2019). Observational research in work-integrated learning. International Journal
of Work-Integrated Learning. Special Issue 20(4): 375-387
Department of Education (2004). Creating comprehensive universities in South Africa:
Concept Document. http://education.prov.gov.za
Durham, S. (2021, July 1). What are the South African NQF levels? SACAP.
https://www.sacap.edu.za/blog/applied-psychology/nqf-levels/
Faller, F., Burton, S., Kaniki, A., Leitch, A., & Ntshoe, I. (2023). Achieving doctorateness:
is South African higher education succeeding with graduate attributes? South
African Journal of Higher Education, 37(2), 93-108.
Field, S., Musset, P & Alvarez-Galvan, J. (2014). A skills beyond school review of South
Africa. OECD Reviews of Vocational Education and Training. OECD Publishing.
Egbert, J., & Sanden, S. (2019). Foundations of education research: Understanding theoretical
components (2nd ed.). Routledge.
Garraway, J. & Winberg, C. (2019). Reimagining futures of universitives of technology.
Critical Studies in Teaching & Learning, 7, Special Issue.
392
Greenwood, A. (1999). International definitions and prospects of underemployment

statistics. International Labour Organization.
https://www.ilo.org/wcmsp5/groups/public/---dgreports/
stat/documents/publication/wcms_091440.pdf
Griesel, H. & Parker, B (2009). Graduate attributes. A baseline study on south african
graduates from the perspectives of employers. Higher Education South Africa & the
South African Qualifications Authority.
Gyamera, G. O., & Burke, P. J. (2018). Neoliberalism and curriculum in higher education:
A post-colonial analyses. Teaching in Higher Education, 23(4), 450-467.
Hernandez-March, J. Martin del Peso, M. & Leguey, S. (2009). Graduates skills and
higher education: The employer perspective: Tertiary Education and Management
15(1): 1-16
Human Resource Development Council. (2013). Review of the current skills development
system and recommendations towards the best model for delivering skills in the country.
http://hrdcsa.org.za/wp-content/uploads/2017/04/2.-Annex-1-Skills-System-
Review-Report-5-Dec-2013.pdf
Kang, W., Pineda Hernández, S., & Mei, J. (2021). Neural mechanisms of observational
learning: a neural working model. Frontiers in Human Neuroscience, 14, 609312.
https://doi.org/10.3389/Fnhum.2020.609312.
Department of Home Affairs. Immigration Act, 2022. Critical skills list. Government
Printers.
Jika, T. (2020). 200 Cuban medics to help SA fight Covid-19. Mail & Guardian.
https://mg.co.za/article/2020-04-25-200-cuban-medics-to-help-sa-fight-covid-
19/
Kappo-Abidemi, C., & Ogujiuba, K. (2020). Higher education institutions and corporate
social responsibility: Triple bottom-line as conceptual framework for community
development. Entrepreneurship and Sustainability, 8(2), 1103-1119.
Kurts, S., (2020). Kolb’s experiential learning theory & learning styles. Educational
Technology. https://educationaltechnology.net/kolbs-experiential-learning-
theory-learning-styles/
Ma, Q., Chan, A. H., & Teh, P. L. (2020). Bridging the digital divide for older adults via
observational training: Effects of model identity from a generational
perspective. Sustainability, 12(11), 4555.
Maisiri, W., Darwish, H., & Van Dyk, L. (2019). An investigation of industry 4.0 skills
requirements. South African Journal of Industrial Engineering, 30(3), 90-105.
McMullan, W., & Cahoon, A (1979). Integrating abstract conceptualizing with
experiential learning. The Academy of Management Review, 4(3), 453-458.
Mentz, J. & Kotze, P. & Van der Merwe, A. (2008). Searching for the technology in
universities of technology. SACJ, 42, 29-37
Mignolo, W. (2011). The darker side of western modernity: Global future, decolonial options.
Duke University Press.
Mkhonza, L. & Letsoalo, A. (2017). Understanding the skills gaps in the public service
sector. Research conducted by the Public Service Sector Education and Training
Authority. PSETA. https://pseta.org.za/wp-
content/uploads/2018/05/Understanding-the-Skills-Gaps-in-the-Public-
Service-Sector.pdf
Mncayi, P., & Dunga, S. (2016). Career choice and unemployment length: A case of
graduates from a South African university. Industry and Higher Education, 30(6),
413-423.
393
Mobarak, K. (2019). Reflections of employed graduates on the suitability of their skills

and knowledge for workplace-readiness. South African Journal of Higher
Education, 33(4), 186-202.
Mouton, N. Louw, G & Strydom, G (2013). Critical Challenges of the South African
School System. International Business a& Economics Research Journal 12(1): 31-44
Mpendulo, G. & Mang’unyi, E. (2018). Exploring Relationships between Education level
and Unemployment. Journal of Social Sciences (COES & J-JSS), 7(2): 2305-9494
Ogunyemi, C. B. (2021). Fela Kuti’s Black consciousness: African cosmology and the re-
configuration of Blackness in ‘colonial mentality’. African Identities, 19(4), 487-
501.
Organisation for Economic Co-operation and Development. (2002). Rethinking human
capital. In Education policy analysis. Paris: OECD.
Smale, N. K. (2012). An analysis of the use of tax incentives to motivate job creation (Doctoral
dissertation, University of Pretoria).
https://repository.up.ac.za/bitstream/handle/2263/26426/dissertation.pdf?se
quence=1&isAllowed=y
Quijano, A. (2007). Coloniality and modernity/rationality. Cultural Studies, 21(2-3), 168
Reddy, V., Bhorat, H., Powell, M., Visser, M., & Arends, A. (2016). Skills supply and
demand in South Africa. Human Sciences Research Council.
Rahmat, M., Ahmad, K., Idris, S., & Zainal, F. (2012). Relationship between
employability and graduate skill. Social and Behavioural Sciences, 59: 591-597.
South African Qualifications Authority. (2012). The South African Qualifications
Authority level descriptors for the South African national qualifications
framework. SAQA. https://www.saqa.org.za/level-descriptors-for-the-south-
african-national-qualifications-framework/
Solmaz, D. Y. (2017). Relationship between Lifelong Learning Levels and Information
Literacy Skills in Teacher Candidates. Universal journal of educational
research, 5(6), 939-946.
Statistics South Africa. (2021). Quarterly labour force survey. Stats SA.
https://www.statssa.gov.za/publications/P0211/P02111stQuarter2021.pdf.
Tanga, M., & Maphosa, C. (2018). Academic hurdles facing undergraduate students at
one South African University. Research in Higher Education Journal, 35, 1-15.
Tuck, R (2007). An introductory guide to national qualifications framework: Conceptual and
practical issues for policy makers. Skills and Employability Department.
International Labour Office.
Vallego, A. A. (2015). Latin-American philosophy: From identity to racial exteriority. Indiana
University Press.
Van Huyssteen, R. (2002). The introduction of the South African National Qualifications
Framework: A brief overview, with reference to higher education. IERF.
https://www.ierf.org/wp-
content/uploads/2016/01/IERFSouthAfrica2002.pdf.
Young, M. (2005). National qualifications frameworks: Their feasibility for effective
implementation in developing countries. International Labour Office- Geneva
Young, M. (2007). Qualifications frameworks: Some conceptual issues. European Journal
of Education, 42(4), 445-457
394

Vol. 22, No. 10, pp. 394-415, October 2023
Received Aug 30, 2023; Revised Oct 21, 2023; Accepted Nov 4, 2023
Comprehensive Academic Thesis Writing

Module for English Major Undergraduates in a
Public University in China
Gao Jing*
UCSI University, Education Department, Kuala Lumpur, Malaysia
Zhengzhou Normal University, Foreign Languages School, Zhengzhou, China
Lin Siew Eng

UCSI University, Education Department, Kuala Lumpur, Malaysia
Abstract. In response to the challenges in enhancing English major

undergraduates’ academic thesis writing and research capabilities,
particularly in writing empirical research-based theses, this study
addresses the current deficiencies in English academic thesis writing
instruction. The current course in Chinese universities cannot offer
sufficient information for Chinese graduates to participate in
international graduate programs and publication. This study aimed to
develop the English Academic Thesis Writing Module (EATWM) for
English major undergraduates of a public university in China and to
determine the participants’ performance after completing it. The
theoretical foundation of this study was based on Dewey’s learning by
doing theory and Kolb’s Experiential Learning Cycle. The study
employed a mixed-methods design involving three stages: module
development, validity and reliability assessment, and performance
analysis. The prototype EATWM was designed and developed using the
ADDIE model. Through the pilot study, the content validity and
reliability of the EATWM were verified, and 20 participants’ theses were
marked by 3 inter-raters to calculate the inter-rater reliability. In the
main study, 70 participants’ academic thesis writing performance and
interview responses were analyzed. The analysis of the participants’
feedback highlights benefits gained by using the module, challenges
faced in academic thesis writing, solutions to address challenges using
the module, and suggestions for module improvement. These results
indicate that the module succeeds in facilitating students to write an
English academic thesis. The EATWM benefits teachers by including
sections that are not in the current syllabus and helps students better
adapt to the international education system and publication context.
*
Corresponding author: Gao Jing; jessiegaofls@126.com
©Authors
395
Keywords: academic thesis writing; empirical research; experiential

learning; grading criteria; thematic analysis

In recent years, there has been a push of teaching reform of English major
undergraduate students in China. Under this push, one important subject that
English teaching in Chinese universities is facing is how to define explicit
teaching objectives, enrich teaching contents, improve teaching approaches, and
update teaching methods, so as to improve the English majors’ language literacy
as well as academic ability. In addition, the Syllabus for English Majors (hereafter
called the Syllabus) (National Administry Committee, 2000) stipulates that a
graduation thesis is an important method of testing undergraduates’
comprehensive ability and evaluating their academic performance. The current
requirements for awarding the bachelor of arts degree in English majors in all
Chinese universities include completing a graduation thesis in English
independently under the guidance of teachers before graduation. At present, for
undergraduates in English major, most of the Chinese public universities have
offered them an English academic writing course in which the teaching objective
is to help them master the necessary skills to complete the graduation thesis.
However, after years of teaching and after completing the course, it has been
found that a considerable number of undergraduates still lack comprehensive
academic competence to conduct scientific research and write academic papers
(Liu, 2021; Sun & Jin, 2021; Zhao, 2019).
Some educators and scholars have indicated that current undergraduate

graduation theses are generally poor in quality and lack academic, professional,
and original features (Jiang & Wu, 2021; Zhu, 2019). To overcome this situation,
it is necessary to carry out relevant research on the deficiency of current
undergraduate education in developing undergraduates’ academic writing
skills. We analyzed 202 bachelor’s degree theses of English majors who
graduated in 2020 from a Chinese university and found that the theses did not
include research problems and had unspecific research objectives, unclear
research questions, and unscientific research designs, as these are not required in
the syllabus. However, no relevant teaching contents or practices were found in
the syllabus of the English Academic Writing course in this university. The
textbook of this course does not encourage English major undergraduates to
write an empirical research-based thesis, as they lack research foundation and
require time for research and writing (Mu, 2012).
When English major undergraduates from Chinese universities participate in

master’s programs abroad, some of them encounter cognitive differences in
research methodology and process (Xue, 2021; Yang, 2017; Zhang, 2018). When
teaching a research methods course to Chinese international graduate students
studying in Canada, Beres and Woloshyn (2017) found that the Chinese
graduates’ understanding of conducting research was far from the empirical
notion. This indicates that the existing English Academic Writing courses in
Chinese public universities cannot provide sufficient information that can help
396
undergraduates, especially those who plan to participate in international

programs after graduation, to adapt to the requirements and criteria of academic
research overseas. Xu (2014) reviewed literature of international publications
and found that Chinese scholars’ journal articles lacked knowledge in research
design and scientific methodology. When discussing the difficulties that Chinese
foreign language teachers face in publishing theses in high-level international
journals, Zhang and Chen (2018) believed that the main reason is that the
teaching of research methodology is lacking in the process of teacher training.
Therefore, this study intended to develop the English Academic Thesis Writing
Module (EATWM) for English major undergraduates to improve the empirical
research-based thesis writing performance of English major undergraduates of a
public university in China.
2. Objectives of the Study

The study aimed to develop the EATWM for English major undergraduate
students in a public university in China and to determine their performance and
feedback after completing the module. More specifically, the study intended to
answer the following research questions:
1. RQ1: What are the key components of the EATWM?
2. RQ2: What is the validity and reliability of the prototype EATWM?
3. RQ3: How do the participants perform doing the EATWM?
3.1 English Academic Writing
Academic writing refers to the formal writing style utilized for the purpose of
documenting academic works (Samuels & Garbati, 2018). It is the common
writing style used by undergraduates to complete course assignments, term
papers, and graduation theses. Since Mandarin is the official teaching language
in Chinese universities, English academic writing courses are only available for
English major undergraduates in mainland China. At the undergraduate level,
thesis writing is the most difficult academic writing task that undergraduates
need to learn. From teachers’ viewpoint, thesis writing is a very effective way to
help undergraduates review and apply the knowledge and capabilities acquired
in their university life to solve practical problems. However, most
undergraduates probably have a different view. Chandrasekhar (2008)
explained that thesis writing is an obligatory task for undergraduates who wish
to obtain a degree. For them, it seems to be the most common reason for writing
a thesis. Therefore, they sometimes view thesis writing as an unpleasant, even
painful journey, ignoring the benefits it brings to their knowledge and
capabilities. Oliver (2014) claimed that thesis writing gives undergraduates an
experience of expressing their understanding of the world in a novel and
original way, in addition to serving as a means for obtaining a degree. Thesis
writing provides opportunities for undergraduates to reflect on their own
intellectual worldview in describing their research. Oliver (2014) thought it was
a pity that thesis writing was seen as simply a means to obtain an academic
degree.
397
3.2 Learning by Doing Theory

The ultimate aim of developing the module is to help undergraduates complete
their thesis writing tasks for graduation. Considering its strong practical
attributes, Dewey’s learning by doing theory (Dewey, 1915) can well direct the
design, development, and implementation of this module. It emphasizes the
integration of knowledge and action as well as the combination of learning and
doing, and believes that direct experience is the basis of all teaching approaches
(Sikandar, 2015). Based on the learning by doing theory, Dewey formed the
five-step model of inquiry. According to this model (see Figure 1), teachers place
students in a situation where their existing experience cannot help them to
directly solve the problem, discover the nature of the problem, propose a
possible plan of taking action, predict the outcomes of this plan, and put it into
action to solve the problem.
Figure 1: Dewey’s five-step model of inquiry (Morgan, 2014)
3.3 Experiential Learning Cycle

Experiential Learning Theory (ELT) emphasizes the acquisition of knowledge by
transforming experience (Kolb et al., 2014; Morris, 2020; Yusof et al., 2020). On
this basis, Kolb et al. (2014) developed the Experiential Learning Cycle, which
has become one of the most significant theoretical frameworks for experiential
instruction. Morris (2020) revised Kolb’s model by proposing a more specific
description to the four stages (see Figure 2).
Figure 2: The revised Kolb Experiential Learning Cycle (Morris, 2020)
398
The development of teaching materials is one of the important devices for

improving teaching quality. Materials that do not align with students’ expected
outcomes cannot effectively support their learning process. Among various
forms of teaching materials, modules have gained favor from instructional
designers in recent years due to their incorporation of various components to
achieve teaching objectives (Abdurrahman et al., 2018). Taking English language
education as an example, many teaching and learning modules were developed
in recent years, involving contents of English reading, writing, speaking, and so
on (Andika et al., 2023; Novitri, 2021; Parcon, 2016; Zhang, 2020). Therefore, the
module was chosen as the appropriate form of material developed in the study
to enhance the quality of teaching of English academic thesis writing.
4. Methodology
4.1 Research Design
A mixed-methods design was adopted in this study. Owing to the exploratory
nature of teaching module development study, both qualitative exploration and
quantitative verification were required. Therefore, the exploratory sequential
mixed-methods design was regarded as the suitable option for the study, for it is
frequently applied in instructional design by first collecting qualitative data to
design and then quantitative data to test (Creswell & Guetterman, 2019).
4.2 Participants of the Study

Purposive sampling was used in the study in favor of the in-depth investigation
(Creswell & Guetterman, 2019). Based on the target students of the EATWM, 70
English major undergraduates (registered in 2020 and graduating in 2024) with
academic thesis writing needs from a public university in China were selected as
participants of the study.

Four instruments were designed and developed for the study, including the
questionnaire for students’ needs, the EATWM, the Graduation Thesis Grading
Criteria (GTGC), and the interview schedule with questions.
The questionnaire for students’ needs aimed to determine the weaknesses and
gaps in students’ knowledge reserve. Ten questions were developed. The first
four questions intended to investigate students’ past experiences in relation to
conducting research and writing theses, and the remaining six questions focused
on what extent they knew about the basic concepts in empirical research.
According to Dewey’s five-step model of inquiry, the module guided students to

i) recognize the problem by informing the learning objectives of each unit; ii)
consider the nature of the problem by explaining the knowledge related to thesis
writing; iii) suggest the solution by providing authentic examples as reference;
iv) consider likely effects of the solution by introducing thesis grading criteria
before they start writing; and v) take action by assigning continuous exercises
and providing timely feedback. Inspired by the revised Kolb Experiential
Learning Cycle, the module provided students i) contextually rich concrete
399
experience by adapting examples from realistic research activities; ii) critical

reflective observation by instructing them to read examples, literature, and their
own thesis critically; iii) contextual-specific abstract conceptualization by setting
exercises to construct a specific empirical research environment; and
iv) pragmatic active experimentation by requiring them to conduct a real
research activity and write an original thesis.
The EATWM is different from the existing syllabus, in which chapters were set
following the general thesis writing steps. The units in the EATWM were
organized based on the five-chapter structure commonly found in empirical
research. Apart from the first unit, “Introduction to Research and Thesis
Writing”, units Two to Six corresponded to the five chapters of an empirical
research-based thesis, comprising “Introduction (Chapter One)”, “Literature
Review (Chapter Two)”, “Research Methodology (Chapter Three)”, “Results
(Chapter Four)”, and “Conclusion (Chapter Five)”. The sub-objectives of each
unit also place a stronger emphasis than the existing syllabus on guiding
students in mastering the specific process and methods of conducting empirical
research. In addition to the textual contents, we designed figures, charts, tables,
and examples in this module to enhance academic thesis writing as well as the
ability to apply empirical approaches and methods to analyze and address
problems.
The GTGC were developed as a complementary component to the EATWM to

address the shortcomings of the existing thesis grading criteria. The newly
developed GTGC consist of 25 grading items, encompassing the dimensions of
research plan and design, research process and results, thesis structure and
format, and English language expression. These items aim to provide a more
comprehensive evaluation framework for empirical research-based theses.
The interview questions were developed to collect feedback from participants

after completing the EATWM. Eight formal questions were developed to
investigate their perception of using the module, how they performed in writing
an empirical research-based thesis, and the challenges they faced and their
countermeasures. When conducting the interview, two more optional questions
for those performing well (A and B ratings) and two for those performing not so
well (C, D, and F ratings) were asked separately to obtain more comprehensive
feedback.
4.4. Research Procedure

The study was conducted using the ADDIE model (see Figure 3), which is a
widely accepted model that contains a set of effective standards and continuous
revisions for the implementation of teaching activities (Gustafson & Branch,
2002; Kurt, 2017). The five stages (analysis, design, development,
implementation, and evaluation) of the ADDIE model (Kurt, 2017) were
conducted to answer the research questions of the study. Stage 1 involved the
analysis of students’ needs. In Stage 2, the prototype EATWM was designed
based on the findings in Stage 1. Stage 3 comprised the development of the
400
prototype EATWM and the examination of its content validity and reliability
through a pilot study (20 participants). In Stage 4, the EATWM was
implemented (70 participants). Finally, in Stage 5, the EATWM was evaluated
through analyzing participants’ performance and feedback.
Figure 3: ADDIE model (Kurt, 2017)
4.4.1 Stage 1: Analysis of students’ needs

The bachelor’s degree theses of graduated English major students, the current
English Academic Writing Syllabus, and the needs of students were analyzed to
clarify the objectives of this study. First, analysis of 202 bachelor’s degree theses
of English majors who graduated in 2020 from the university under study
showed that only 26 (12.87%) of them could be considered as empirical
research-based theses. No obvious problem statement, research objectives,
research questions, specific research design (sampling, data collection and
analysis) could be found in the 26 theses. Moreover, weaknesses such as illogical
and uncritical literature review, outdated references, disordered and dreary
research results, and poor organized conclusions were evident. However, the
necessary guidelines for how to improve these weaknesses are not included in
the current English Academic Writing Syllabus and textbooks. In addition, a
questionnaire (verified by content experts before use) was administered to all 91
participants via WJX (an online questionnaire platform) to survey the students’
needs. Table 1 presents the frequency of participants’ questionnaire responses.
The first four questions are related to participants’ past experiences regarding
conducting research and writing theses. In this regard, 65.93% of participants
knew how to download academic literature, 53.85% have read academic
literature, 28.57% had research experience, and 45.05% had academic writing
experience. More than half of the participants had a preliminary understanding
of academic literature. Interestingly, there were more participants with academic
writing experience (n = 41) than with research experience (n = 26), indicating to
the “research-absent academic writing” phenomenon.
The remaining six questions surveyed to what extent participants knew about
the basic concepts in empirical research. Concerning this, only 8.79% knew the
meaning of empirical research, 20.88% knew the meaning of research problem,
401
30.77% knew the meaning of research objective, 24.18% knew the meaning of
research question, 8.79% knew the meaning of research design, and 24.18%
knew some research methods. Participants’ responses indicate that although
some students have some preliminary understanding of research and thesis
writing, the vast majority still need to learn systematically and scientifically.
Table 1: Frequency of participants’ questionnaire responses (N = 91)

Percent Cumulative
No. Item Categories n
(%) percent (%)
Do you know some ways to download Yes 60 65.93 65.93
1
journal articles and graduation theses? No 31 34.07 100.00
Have you ever read any academic Yes 49 53.85 53.85
2
journal articles or graduation theses? No 42 46.15 100.00
Have you ever conducted any research Yes 26 28.57 28.57
3
activities? No 65 71.43 100.00
Have you ever written any academic Yes 41 45.05 45.05
4
articles or theses? No 50 54.95 100.00
Do you know what empirical research Yes 8 8.79 8.79
5
is? No 83 91.21 100.00
Do you know what a research problem Yes 19 20.88 20.88
6
is? No 72 79.12 100.00
Do you know what a research objective Yes 28 30.77 30.77
7
is? No 63 69.23 100.00
Do you know what a research question Yes 22 24.18 24.18
8
is? No 69 75.82 100.00
Yes 8 8.79 8.79
9 Do you know what a research design is?
No 83 91.21 100.00
Do you know some specific research Yes 22 24.18 24.18
10
methods? No 69 75.82 100.00
Total 91 100.0 100.0
4.4.2 Stage 2: Design of the prototype EATWM

Based on the results of the analysis, the overall objectives and sub-objectives of
each unit of the module were first designed to identify the ultimate goal to help
students complete a five-chapter empirical research-based thesis in English.
Compared to the existing syllabus, which focuses solely on the general
procedures of academic thesis writing, the newly developed EATWM places
greater emphasis on instructing students to complete an English academic thesis
following the scientific and normative empirical research approaches and
procedures.
The prototype module was designed based on the existing English Academic
Thesis Writing Syllabus at a public university in China. Furthermore,
appropriate concepts and ideas of other modules or teaching materials were
402
adapted to complement the shortages of the existing syllabus, including

guidelines on writing research problems, research objectives, and research
questions and selecting a research design. These modules and materials
included the Research Methods Module (Kebede et al., 2015), the Research and
Thesis Writing Module (University of Wollongong, 2001), the Graduation Thesis
Writing and Defense Module (Wu et al., 2022), and a number of monographs on
the topic of conducting research and thesis writing (APA, 2020; Creswell &
Creswell, 2017; Creswell & Poth, 2017; Creswell & Guetterman, 2019; Kothari,
2019; Kumar, 2019; Swales & Feak, 2012). Similarly, content analysis of books,
theses, journal articles, reports, and related modules was used as basis and
guide in developing the module, for these materials can provide sufficient data
for the research of module development (Zahrin et al., 2022).
4.4.3 Stage 3: Development of the EATWM

The module strictly followed the instructional design to ensure the rationality,
consistency, and effectiveness of the developed teaching courseware, classroom
activities, after-class tasks (or exercises), and assessment methods. The prototype
module was developed based on the theory of learning by doing and Kolb’s
Experiential Learning Cycle. Development included focusing on the differences
of undergraduates’ knowledge and ability, controlling the difficulty of
understanding and operation reasonably, and combining the improvement of
undergraduates’ ability with the principles of module design organically. The
module was developed using six steps: i) to form the texts of teaching contents
in the module by analyzing, selecting, and adapting collected resources; ii) to
adapt suitable texts into illustrations, figures, and tables to increase the
readability; iii) to develop “lead-in questions” of every section to arouse
students’ curiosity; iv) to select and adapt examples for the teaching contents;
v) to develop step-by-step exercises to guide students to write their thesis
simultaneously with doing the module; and vi) to develop other additional
components (format, appearance, mind map, reminder, etc.) of the module.
Finally, a six-unit module was developed. The structure of the EATWM can be
seen in Figure 4.
403
Figure 4: Structure of the EATWM
After the EATWM had been developed, its content validity was verified by five
content experts. The module was amended according to the feedback of the
content experts. The reliability was measured using Cronbach’s alpha coefficient.
The result was .894 for the module, indicating that the prototype module is
reliable.
The amended module was used in the pilot study. After the 20 participants
submitted their theses, three experienced thesis raters assessed their theses
based on the grading criteria. The scores of the pilot study were analyzed using
the intraclass correlation coefficient (ICC) to assess the inter-rater reliability
among the three inter-raters. The ICC value for the average measures was 0.820,
which indicates a high level of agreement among raters.
4.4.4 Stage 4: Implementation of the EATWM

A ten-week module on learning and English academic thesis writing course was
conducted in the main study weekly. An experienced teacher spent eight weeks
(two hours per week) teaching all six units in the module. Participants submitted
their writing assignments one week after completing each unit and were given
404
two extra weeks after completing the last unit to complete and refine their
English academic thesis writing.
4.4.5 Stage 5: Evaluation of the EATWM

We assessed the 70 theses in the main study. The scores of the main study and
interview responses were analyzed to evaluate the performance of participants
in English language academic thesis writing using the module. On the one hand,
the scores representing participants’ thesis writing performance were analyzed
using the quantitative descriptive method, including calculation of the means,
percentages, and frequencies of the data, so as to reveal their general
performance after completing the module. On the other hand, the participants’
feedback collected through interviews was analyzed by using thematic analysis,
so as to further understand their performance and perceptions of completing the
module.
5. Results and Discussion

5.1 Participants’ English Academic Thesis Writing Performance
After conducting the main study, we attempted to describe the participants’
performance using the EATWM by analyzing their thesis scores. Table 2 shows
the means and percentages of the chapter scores and the thesis total score.
Table 2: Descriptive analysis of participants’ English academic thesis writing scores

(N = 70)
Chapter 1 Chapter 2 Chapter 3 Chapter 4 Chapter 5 Total
(20%) (20%) (20%) (20%) (20%) (100%)
Mean 15.36 14.57 15.30 15.60 15.73 76.56
Percentage 76.79% 72.86% 76.50% 78.00% 78.64% 76.56%
Note: Participants’ performance was divided into five rating categories according to their total
scores: A (≥ 90) = excellent, B (80–89) = good, C (70–79) = moderate, D (60–69) = pass,
F (< 60) = fail.
The mean of the total scores was 76.56, which reflects that the participants
reached a moderate performance as a whole. The interval range of the mean
score of chapters was between 14.57 (72.86%) and 15.7 (78.64%), which indicates
that although slight differences existed in participants’ performance of different
chapters, they achieved a moderate performance in writing all the chapters.
Furthermore, their performance in writing Chapter One (76.79%), Three (76.50%),
Four (78.00%), and Five (78.64%) was relatively good, although their
performance in writing Chapter Two (72.86%) was slightly weaker and needed
to be further improved.
These findings are not entirely consistent with the weaknesses pointed out by
other related studies, including the absence of research questions, improper
research methodology, non-standard thesis format, insufficient language ability,
etc. (Cao, 2020; Wang & Ming, 2018; Wu et al., 2023). It should be noted that the
findings of these related studies are based on the analysis of the thesis writing
performance of students who had never participated in any course or program
405
that taught empirical research-based thesis writing. On the contrary, the current
study analyzed the performance of those who had systematically learned
empirical research-based thesis writing by using the EATWM. The findings
reveal that compared with other chapters with certain research procedures and
writing formats, writing the literature review chapter posed a greater challenge
to the participants. Future research should provide additional support and
guidance to help students overcome difficulties and challenges in writing
Chapter Two.
Figure 5 presents participants’ thesis writing performance graphically.
Figure 5: Participants’ English academic thesis writing performance (N = 70)
Comparing the number and proportion of participants who achieved different

ratings (A, B, C, D, and F), we found that relatively few of them achieved
A (5, 7.14%) and F (3, 4.29%) ratings, while the majority achieved B (28, 40.00%),
C (20, 28.57%), and D (14, 20.00%) ratings. This indicates that more participants
were closer to the average level of performance in academic thesis writing. This
finding is similar to other previous studies (Husin & Nurbayani, 2017;
Marulanda Ángel & Martínez García, 2017), although these studies utilized
different approaches or teaching materials and their research subjects were
varied, including undergraduates and pre-service teachers. This indicates that
the newly developed EATWM can have an acceptable teaching effect of
academic thesis writing, a finding which is in line with this study’s expectations.
The descriptors of participants’ performance can be seen in Table 3.
Table 3: Descriptors of participants’ English academic thesis writing performance

Rating Descriptor
Can complete an excellent quality thesis, presenting reasonable and
A convincing problem statement, precise and realizable objectives and
(90–100) questions, scientific empirical research design and process, sufficient
Excellent results and conclusion, accurate structure and format, and proficient
academic English language.
Can complete a good quality thesis, presenting reasonable problem
B (80–89)
statement, realizable objectives and questions, clear empirical research
Good
design and process, complete results and conclusion, basically accurate
406
structure and format, and skillful academic English language, but with
minor errors.
Can complete a moderate quality thesis, presenting clear problem
statement, objectives and questions; relative suitable empirical research
C (70–79)
design, process, results and conclusion; decent structure and format; and
Moderate
relative skillful academic English language, but with few noticeable
errors or inaccuracies.
Can complete a pass quality thesis, presenting acceptable problem
statement, objectives and questions; plain empirical research design,
D (60–69)
process, results and conclusion; passable structure and format; and less
Pass
skillful academic English language, with some significant errors or
deficiencies.
Cannot complete a thesis beyond pass criteria, presenting a not workable
problem statement, objectives and questions; plain empirical research
F (0–59)
design, process, results and conclusion; disorganized structure and
Fail
format; and very limited academic English language, with many
significant errors or deficiencies.
Participants’ performance demonstrated that a minority of them were capable of

using the module very effectively to complete an excellent quality English
academic thesis. For instance, in the interview with an A-rating participant, she
claimed that,
“Through studying the module, I learned about the methods of empirical
research and how to write empirical research papers.”
Another A-rating participant articulated that,

“I learned how to find the literature and data related to my thesis more
effectively, gained a deeper understanding of the methods of empirical
research, accurately applied different methods according to my research
objectives, and developed my critical thinking ability. Learning this
module has really benefited a lot.”
These responses highlight the positive impact of the module on a minority of the
participants, leading to their excellent performance in conducting research and
writing theses. Besides, a substantial portion of them were able to use the
module to write a good or moderate quality English academic writing thesis. For
instance, a B-rating participant indicated that,
“By using the module, I not only gained professional knowledge and
research skills, but also cultivated critical thinking, logical thinking,
teamwork, and communication skills.”
Another C-rating participant thought that,

“I have understood the basic structure and requirements of English
thesis writing and am familiar with the entire writing process.”
These responses indicate that a considerable number of participants felt that

they could acquire the necessary knowledge and skills to support their research
407
and thesis writing from the module. However, a small portion of them barely
reached or were below the passing criteria. For instance, a D-rating participant
complained that,
“I do not know where to find appropriate literature materials, which are
relatively scarce.”
Another F-rating participant also felt frustrated about her thesis writing
experience. She said that,
“I feel that I have some doubts ... some problems are actually not
solved.”
These instances indicate the importance of providing additional resources,

targeted attention, and interventions to bridge participants’ understanding gap,
help them overcome the difficulties they face in using the module, and foster a
more positive learning experience.
In general, the majority of participants showed that they can
independently write an English academic thesis using the module. However,
attention should be given to those who achieved lower ratings, and additional
support and interventions should be provided to help them improve their
performance.
5.2 Participants’ Feedback

Nine participants were selected based on their ratings (2 of A, 2 of B, 2 of C, 2 of
D, and 1 of F) to take part in one-to-one interviews. In the interviews,
participants reflected on their thesis writing performance. In addition, they
reflected on the benefits they gained, the challenges they faced, and the solutions
to address challenges, and made suggestions for improving the module after
having used the EATWM. In order to gain a deeper understanding of
participants’ feedback, thematic analysis was conducted by encoding and
generating themes from the original words of the participants (Wæraas, 2022).
The first theme to be generated was “Benefits Gained”. As shown in Table 4, by

encoding participants’ responses related to this theme, 14 sub-themes were
generated from 30 codes, which cover four dimensions of English academic
thesis writing performance. Analysis indicated that the EATWM is beneficial for
cultivating participants’ comprehensive academic writing abilities. They can
gain practical insights ranging from planning, designing, researching, and
presenting results, to thesis structure, format, and language expression.
However, more sub-themes were generated from “Plan and Design” and
“Research and Results” dimensions, and fewer from “Structure and Format”
and “Language Expression” dimensions. In Liu’s (2015) study, students did not
make obvious progress in the “Arrangement” aspect, since it is difficult to
improve the language arrangement ability in a short-term training, indicating
the need for further strengthening and attention in those areas.
408
Table 4: Benefits gained using the EATWM

Interview questions:
1. What are the benefits you gained in terms of knowledge and skills of English academic thesis
writing by using the English Academic Thesis Writing Module?
2. In your opinion, what were the main factors that contributed to your good performance in English
academic thesis writing? Were there any features of the module that you found particularly
beneficial in this regard? (Only A and B ratings)
Theme Dimensions Sub-themes Codes
Learn empirical research methods
Write empirical research papers
Basic concepts
Understand the concept of thesis
Learn the steps in conducting empirical research
Topic selection Select a topic
Research objectives Determine objectives
Plan and
Research problem Clarify one’s problem statement
Design
Improve logical and critical thinking skills
Critical thinking
Develop critical thinking ability
Cultivate ability of reviewing literature
Analyze and classify literature
Literature review
Collect literature
Conduct in-depth information search and sorting
Learn empirical research approaches and
Research design
methods
Research ability Enhanced research abilities
Benefits Support from sufficient research and data
Research and
Ensure accuracy, objectivity, and credibility of
Results Data collection
data
Conducting extensive data collection
Data analysis Improve data analysis capabilities
Displaying results Design and analyze charts
Improve understanding of thesis structure
Learn how to write a paper
Develop a logical structure
Structure of thesis Organize the thesis structure
Structure and
Learn the thesis framework
Format
Understand the thesis structure
Organize the thesis structure
Format of citations
Learn the format of citations and references
and references
Language English academic Improve English academic language ability
Expression language Avoid repetitive narratives
A second theme that emerged was “Challenges Faced”. As shown in Table 5, by

encoding participants’ responses related to this theme, 9 sub-themes were
generated from 18 codes. The majority of the challenges exist within the “Plan
409
and Design” dimension, followed by the “Research and Results” and “Structure
and Format” dimensions, and a few in the “Language Expression” dimension.
This indicates that the primary challenges affecting the participants’
performance in thesis writing were in the research process, especially in the
initial plan and design phases. Fitria (2022) did a survey about the chapters
students felt were difficult to write. The result show that 50% of the participating
students chose Chapter One and Chapter Three, 47.5% chose Chapter Two, and
60% chose Chapter Four. This is consistent with our findings, suggesting
subsequent improvements to the module should concentrate on these
dimensions. The challenges identified provide valuable directions and insights
for further optimizing the module in the future. Moreover, the additional
influencing factors serve as a basis for enhancing personal ability of English
major undergraduates, improving the scheduling of graduation thesis
assignment, and refining the methods of thesis supervision.
Table 5: Challenges faced using the EATWM

Interview question:
3. What are the challenges you faced in completing English academic thesis writing?
Topic selection Topic selection
Research questions Formulation of research questions
Information gathering
Limited knowledge of existing
Plan and research
Literature review
Design Integrating references into my thesis
Effective literature review
Limited literature
Research approach and method
Research design
Challenges
Research method selection

Data collection and organization
Data collection
Research Data collection tools
and Results Data processing methods
Data analysis
Data analysis
Format of references References management
Coherent organization of ideas
Structure
Deficiencies in academic writing
and Format Structure of thesis
caused by cultural differences
Inadequate and incoherent content
Language English language
Limited English language proficiency
Expression ability
The third theme identified was “Solutions to Challenges”. As shown in Table 6,

by encoding the participants’ responses related to solutions to challenges, 8
sub-themes were generated from 16 codes, which cover the first 3 dimensions.
This reflects that effective solutions can be found within the module to address
the challenges participants face when writing a thesis. However, the participants
410
who mentioned facing challenges in the “Language Expression” dimension did

not provide corresponding solutions, indicating that relying solely on the
module was not enough to address the challenges they faced in this dimension.
This is because the improvement of English language proficiency requires
long-term training and practice (Cao, 2020). Specialized support and guidance
might be needed to enhance students’ English language proficiency.
Table 6: Solutions to address challenges using the EATWM

Interview question:
4. Can the module help you address them (the challenges)? How did you address
them by using the module?
Topic
Topic selection
selection
Research
Formulation of research questions
questions
Plan and
Classify, review, and quote literature
Design Literature
Analyze and critique literature
review
Search literature
Research Research approach and method
design Research method selection
Data Data collection and organization
Solutions Research and collection Data collection tools
Results Data Data analysis tools
analysis Data processing methods
Format of
References management
references
Writing template and detailed
Structure and explanations
Format Structure of Coherent organization of ideas
thesis Adapting to cultural differences in
academic writing
Step-by-step instruction
The last theme related to “Suggestions for Improvement”. As shown in Table 7,

by encoding participants’ responses related to this theme, three sub-themes were
generated from four codes, which include interaction and feedback, examples,
and creativity, which were also frequently pointed out in previous studies (Jiang
& Wu, 2021; Wang & Ming, 2018; Wu et al., 2023). These suggestions provide
concrete guidance for instructors to further design the teaching procedures,
activities, and learning context when using this module. This will enhance
students’ learning experience and thesis writing performance, while increasing
their engagement and motivation in learning and practicing.
411
Table 7: Suggestions for improving the EATWM

Interview question:
5. Are there any suggestions for improving the module in addressing these
challenges?
Theme Sub-themes Codes
Teacher’s interaction and feedback with
Interaction and undergraduates
feedback More interaction, exercise materials, and peer
Suggestions
evaluation
Examples More examples and analysis of examples
Creativity Creative ideas for discussion
In view of the limitations of the EATWM found in the results, we will focus on
improving Unit Three of the module in the future to help students improve their
writing performance of the literature review (Chapter Two). Meanwhile, special
attention will be paid to find solutions to the factors that lead to the relatively
lower thesis writing performance. Furthermore, future studies of students’
academic writing performance in graduate programs and English academic
publications can be conducted in other universities in China.
6. Conclusion
To improve the empirical research-based thesis writing performance of English
major undergraduate students, the study developed the EATWM, tested its
validity and reliability, and analyzed the students’ performance and feedback
using the module through a case study in a Chinese university. All three
research questions were solved in this study. For RQ1, the six-unit EATWM was
developed, including the necessary information and techniques for conducting
empirical research and writing an empirical research-based thesis in English. It
provides students with concise explanations, authentic examples, and
step-by-step exercises as effective learning scaffolding. For RQ2, results of the
content evaluation and the pilot study demonstrate that the developed module
displayed satisfactory levels of validity and reliability, indicating that it could be
effectively implemented on a larger scale with a broader participant base. For
RQ3, the general performance of participants in thesis writing achieved the
moderate level. Furthermore, the thematic analysis of participants’ feedback was
helpful to pinpoint the areas that required further attention and improvement,
providing valuable insights for refining the module in the future.
The EATWM developed in this study can contribute to enhance the English
academic thesis writing performance of English major undergraduates in China.
The study may benefit teachers teaching or supervising undergraduates’ thesis
writing, and points out the significance for further research on improving
students’ English academic thesis writing. Meanwhile, the study may also
benefit English major undergraduates who have future plans of studying abroad,
conducting empirical research, and publishing in international contexts. Finally,
we hope that this module can be implemented in more universities in China. If
both teachers and students consider this module to be beneficial, it can be
412
promoted on a larger scale to improve the thesis writing performance of English

major undergraduates in China.
7. Acknowledgments
We would like to express our gratitude to the content experts, inter-raters, and
teachers for verifying the validity and reliability of the EATWM.
8. References
Abdurrahman, A., Saregar, A., & Umam, R. (2018). The effect of feedback as soft
scaffolding on ongoing assessment toward the quantum physics concept
mastery of the prospective physics teachers. Indonesian Journal of Science
Education, 7(1), 34–40. https://doi.org/10.15294/jpii.v6i2.7239
APA (American Psychological Association). (2020). Publication manual of the American
Psychological Association (7th ed.). American Psychological Association.
https://apastyle.apa.org/products/publication-manual-7th-edition
Andika, A., Tahrun, T., & Firdaus, M. (2023). Developing English grammar instructional
materials oriented to constructivism theory. Journal of Languages and Language
Teaching, 11(2), 286–296. https://doi.org/10.33394/jollt.v%vi%i.6838
Beres, J. L., & Woloshyn, V. E. (2017). Instructional insights gained from teaching a
research methods course to Chinese international graduate students studying in
Canada. Journal of International Undergraduates, 7(3), 728–743.
Cao, L. M. (2020). Ying yu zhuan ye ben ke bi ye lun wen xian cun wen ti, cheng yin yu
dui ce: yi shanghai cai jing da xue zhejiang xue yuan wei li [The existing
problems, causes and countermeasures of graduation theses for English majors:
A case study of Zhejiang College of Shanghai University of Finance and
Economics]. Journal of Guangxi College of Education, (03), 43–45.
https://kns.cnki.net/kcms2/article/abstract?v=JhhVyKSVrEjjqDQP6QYc_6fxv8
WeOOMZEPjwcfazE6ofpLZ8iUt3l-sr3ZwldXbiaWUSaEg-g_7imhbh-3KBVHdPf
MoeAvGO4Sqngt2FJ03IDqI-aG80HJLwX_jVfYyhy1GPfeKhw_B0avcUoIRpl_gR
CdxueFtl&uniplatform=NZKPT&language=CHS
Chandrasekhar, R. (2008). How to Write a thesis: A Working Guide. The University of
Western Australia.
https://www.student.uwa.edu.au/__data/assets/pdf_file/0007/1919239/How
-to-write-a-thesis-A-working-guide.pdf
Creswell, J. W., & Poth, C. N. (2017). Qualitative inquiry and research design: Choosing
among five approaches (4th ed.). Sage.
https://us.sagepub.com/en-us/nam/qualitative-inquiry-and-research-design/
book246896
Creswell, J. W., & Creswell, J. D. (2017). Research design: Qualitative, quantitative, and mixed
methods approaches (5th ed.). Sage.
https://www.google.com.my/books/edition/Research_Design/335ZDwAAQB
AJ
Creswell, J. W., & Guetterman, T. C. (2019). Educational research: Planning, conducting and
evaluating quantitative and qualitative research (6th ed.). Pearson.
https://www.google.com.my/books/edition/Educational_Research/s7rNtAE
ACAAJ
Dewey, J. (1915). Schools of tomorrow. E. P. Dutton and Co.
https://www.google.com.my/books/edition/Schools_of_Tomorrow/mdI0nwE
ACAAJ
Fitria, T. N. (2022). Analysis of EFL students’ difficulties in writing and completing
English thesis. LLT Journal: A Journal on Language and Language Teaching, 25(1),
413
295–309. https://doi.org/10.24071/llt.v25i1.3607
Gustafson, K. L., & Branch, R. M. (2002). Survey of Instructional Development Models
(4th ed.). Syracuse University: Eric Clearinghouse on Information and
Technology.
https://www.google.com.my/books/edition/Survey_of_Instructional_Develop
ment_Mode/QTW9AAAACAAJ
Husin, M. S., & Nurbayani, E. (2017). The ability of Indonesian EFL learners in writing
academic papers. Dinamika Ilmu, 17(2), 237–250.
https://dx.doi.org/10.21.93/di.v17i2.725
Jiang, Z. Y., & Wu, Q. H. (2021). Ben ke bi ye lun wen cun yu fei yan jiu ping shu [A
discussion on the continuation or abolition of the undergraduate thesis].
Modern Business Trade Industry, 42(18), 158–160.
https://doi.org/10.19311/j.cnki.1672-3198.2021.18.075
Kebede, Y., Andargie, G., Feleku, A., & Awoke, T. (2015). Module on research methods. The
University of Gondar.
http://ndl.ethernet.edu.et/bitstream/123456789/78889/1/Full%20module%20o
n%20research%20methods.pdf
Kolb, D. A., Boyatzis, R. E., & Mainemelis, C. (2014). Experiential learning theory:
Previous research and new directions. In R. J. Sternberg and L. F. Zhang (Eds.),
Perspectives on cognitive, learning, and thinking styles (pp. 227–248). Routledge.
http://secondarycontent.pbworks.com/f/experiential-learning-theory.pdf
Kothari, C. R. (2019). Research methodology: Methods and techniques (4th ed.). New Age
International.
http://103.42.87.14:8080/jspui/bitstream/123456789/2471/1/Research-Method
ology-CR-Kothari.pdf
Kumar, R. (2019). Research methodology: A step-by-step guide for beginners (5th ed.). Sage.
http://nuir.nkumbauniversity.ac.ug/bitstream/handle/20.500.12383/924/Ranji
t_Kumar-Research_Methodology_A_Step-by-Step_G.pdf
Kurt, S. (2017). ADDIE model: Instructional design.
https://educationaltechnology.net/the-addie-model-instructional-design/
Liu, J. (2021). Ji yu duo yuan shi du li lun de ying yu xue shu lun wen xie zuo jiao xue
yan jiu [Research on the teaching of English academic writing based on the
multiliteracies pedagogy]. Journal of Tongling Vocational & Technical College,
20(01), 92–95. https://doi.org/10.16789/j.cnki.1671-752x.2021.01.020
Liu, L. (2015). A study of graduate thesis writing course for English undergraduates.
Journal of Language Teaching and Research, 6(4), 836.
http://dx.doi.org/10.17507/jltr.0604.17
Marulanda Ángel, N. L., & Martínez García, J. M. (2017). Improving English language
learners’ academic writing: A multi-strategy approach to a multi-dimensional
challenge. Gist Education and Learning Research Journal, 14, 49–67.
https://eric.ed.gov/?id=EJ1146679
Morgan, D. L. (2014). Pragmatism as a paradigm for social research. Qualitative
inquiry, 20(8), 1045-1053. https://doi.org/10.1177/1077800413513733
Morris, T. H. (2020). Experiential learning: A systematic review and revision of Kolb’s
model. Interactive Learning Environments, 28(8), 1064–1077.
https://doi.org/10.1080/10494820.2019.1570279
Mu, S. X. (2012). An introduction to thesis writing for English majors (2nd ed.).
Foreign Language Teaching and Research Press.
https://www.abebooks.com/9787513522908/English-majors-textbook-series-Pr
ofessional-7513522901/plp
National Administry Committee on Teaching English Language to Majors in Higher
Education under the Ministry of Education. (2000). Syllabus for English Majors.
Foreign Language Teaching and Research Press.
414
Novitri, S. (2021). Developing narrative text-based module for teaching reading

comprehension: Focus on practicality. J-SHMIC: Journal of English for Academic,
8(2), 132–139. https://doi.org/10.25299/jshmic.2021.vol8(2).7198
Oliver, P. (2014). Writing your thesis (3rd ed). Sage.
https://www.google.com.my/books/edition/_/o5mFnAEACAAJ
Parcon, R. E. (2016). Task-based instruction in oral English: Basis for an instructional
module enhancement. International Conference on Language, Literature, Culture and
Education (ICLLCE), 45. http://icsai.org/procarch/6icllce/6icllce-045.pdf
Samuels, B., & Garbati, J. (2018). Mastering academic writing. Sage.
https://www.google.com.my/books/edition/Mastering_Academic_Writing/lq
R5DwAAQBAJ
Sikandar, A. (2015). John Dewey and his philosophy of education. Journal of Education
and Educational Development, 2(2), 191–201.
http://jmsnew.iobmresearch.com/index.php/joeed/article/download/214/32
1
Sun, X. G., & Jin, P. (2021). Ben ke sheng xue shu lun wen xie zuo pei xun de shi jian yu
fan si [Practice and reflection of undergraduate academic writing
training]. Education and Teaching Forum, (33), 5–8.
https://kns.cnki.net/kcms2/article/abstract?v=3uoqIhG8C44YLTlOAiTRKibYl
V5Vjs7iy_Rpms2pqwbFRRUtoUImHQHb4AtRm8wJkd2lSGpwqTveJplLbO_y2
HWayh4t3cqY&uniplatform=NZKPT
Swales, J. M. & Feak, C, B. (2012). Academic Writing for Graduate Students: Essential Tasks
and Skills. (3rd ed). University of Michigan Press.
http://eli.johogo.com/Class/AWFGS.pdf
University of Wollongong. (2001). Research and thesis writing.
https://documents.uow.edu.au/content/groups/public/@web/@raid/docume
nts/doc/uow060982.pdf
Wang, G. F., & Ming, B. S. (2018). Ying yu zhuan ye ben ke bi ye lun wen xie zuo de xue
sheng fan kui shi zheng diao cha [A survey of English major students’ feedback
on graduation thesis writing]. Journal of Wuhan Business University, (05), 90–93.
https://doi.org/10.16199/j.cnki.jwbu.2018.05.023
Wæraas, A. (2022). Thematic analysis: Making values emerge from texts. In G. Espedal,
B. Jelstad Løvaas, S. Sirris, & A. Wæraas (Eds.), Researching values (pp. 153–170).
Palgrave Macmillan.
https://library.oapen.org/bitstream/handle/20.500.12657/53343/1/978-3-030-9
0769-3.pdf#page=163
Wu, H. B., Huang, X. J., & Zhao, H. B. (2023). Ren wen she ke lei ben ke bi ye lun wen zhi
gong xing wen ti ji po jie ce lue [Undergraduate thesis of humanities and social
sciences: Basic requirements, common problems and cracking strategies].
Journal of YuZhang Normal University, (04), 46–50.
https://kns.cnki.net/kcms2/article/abstract?v=JhhVyKSVrEif7Eoiyaq6YyG1Y
OON_aOp_3nktjaBm2dBeE261BPyJ1V2S_DOkgRo9MyyhBnlteHkq5zG-NW756
BOZK_sjB1v0GyWl4_GbQOjQUXzSh47dZqnKWVNy3c0_-tSiX3pglE=&uniplatf
orm=NZKPT&language=CHS
Wu, L. Z., Zeng, X. Y., Chen X. L., Li, L. J., & Zhang, W. J. (2022). Bi ye lun wen xie zuo yu
da bian [Graduation thesis writing and defense module]. South China Normal
University. https://www.icourse163.org/course/scnu-1206357801
Xue, L. (2021). Preparing Chinese international undergraduates for graduate studies in a U.S.
university through a pathway program: A case study [Master’s thesis]. California
State Polytechnic University.
https://scholarworks.calstate.edu/concern/theses/n009w724d
Xu, F. (2014). Fei ying yu guo jia xue zhe guo ji fa biao wen ti yan jiu shu ping [Review of
studies on EAL researchers’ international publishing]. Foreign
415
Language World, 1, 27–33.

https://kns.cnki.net/kcms2/article/abstract?v=3uoqIhG8C44YLTlOAiTRKgchr
J08w1e7M8Tu7YZds89dAAxe5qzazF0rOpydlNbjTo_zF3FW67hczsKNZIJ-6UEXt
cUIiRn2&uniplatform=NZKPT
Yang, C. (2017). Problems Chinese international undergraduates face during academic
adaptation in English-speaking higher institutions [Master’s thesis]. University of
Victoria.
http://dspace.library.uvic.ca/bitstream/handle/1828/8086/Yang_Chunling_M
Ed_2017.pdf
Yusof, R., Yin, K. Y., Norwani, N. M., Ismail, Z., Ahmad, A. S., & Salleh, S. (2020).
Teaching through experiential learning cycle to enhance student engagement in
principles of accounting. International Journal of Learning, Teaching and Educational
Research, 19(10), 323–337. https://doi.org/10.26803/ijlter.19.10.18
Zahrin, S. N. A., Mahmud, M. I., & Azman, N. (2022). Development of the Love for
Writing and Publishing Journal (LWPJ) Module for Higher Education.
International Journal of Learning, Teaching and Educational Research, 21(3), 434–447.
Zhang, H., & Chen S. S. (2018). Zhong shi yan jiu fang fa pei yang ti gao wai yu jiao shi
de guo ji fa biao yi shi [Value the training of research methods and increase
foreign language teachers’ awareness in international publishing]. Foreign
Languages and Their Teaching, 3, 13–21.
https://doi.org/10.13458/j.cnki.flatt.004487
Zhang, L. (2020). Gong gong ying yu ESP/EGP ke cheng mo kuai she ji yan jiu: yi
donghua da xue EAP xue shu ying yu xie zuo ke cheng wei li [ESP/EGP course
modules design: A case study of EAP-integrated writing in Donghua University].
Northeast Asia Foreign Languages Symposium (NAFLS), 9.
https://doi.org/10.26914/c.cnkihy.2020.006751
Zhang, Z. (2018). Academic writing difficulty of Chinese undergraduates: The cultural
issue behind Chinese and British academic writing styles. Studies in Literature and
Language, 17(2), 118–124. https://core.ac.uk/download/pdf/236304431.pdf
Zhao, X. (2019). Gao xiao ying yu zhuan ye xue shu lun wen xie zuo ke cheng jiao xue
gai ge tan xi [Teaching reform of academic thesis writing in English majors of
higher educational institutions]. Journal of Jiamusi Vocational Institute, (05),
119–120.
https://kns.cnki.net/kcms2/article/abstract?v=3uoqIhG8C44YLTlOAiTRKibYl
V5Vjs7iLik5jEcCI09uHa3oBxtWoK0YTmtr_7g4Ik_D5hJ1Oqh1JUeXRmxW69mso
sMmeeBE&uniplatform=NZKPT
Zhu, L. (2019). Zhuan ke he ben ke bi ye lun wen cun fei bian xi [Reservation and
termination of graduation thesis of junior college and
undergraduate]. Education Modernization, 6(95), 140–142.
https://doi.org/10.16541/j.cnki.2095-8420.2019.95.064
416

Vol. 22, No. 10, pp. 416-432, October 2023
Assessment Strategies in Outcome-Based

Education: Preferences and Practices Among
University Lecturers in Vietnam
Phuong Hoang Yen , Nguyen Anh Thi* , Le Thanh Thao

Can Tho University
Can Tho city, Vietnam
Pham Trut Thuy

Nam Can Tho University
Nguyen Huong Tra and Huynh Thi Anh Thu

Can Tho University
Abstract. Amidst the diverse landscape of modern educational

methodologies, outcome-based education (OBE) has gained prominence
as an effective approach to teaching and learning. Originating from the
desire to produce specific and measurable outcomes for learners, OBE
focuses on what students should know and be able to do by the end of
their educational experience. Motivated by the need to delve deeper into
the implementation of OBE, particularly regarding assessment
preferences, this study aims to explore the types of assessments preferred
by university lecturers for implementing OBE in Vietnam. Building on the
existing gap in literature that emphasizes the intricacies of OBE
assessment choices, utilizing qualitative research methods, in-depth
interviews were conducted with 15 university lecturers from various
disciplines. To analyze the gathered data, a thematic analysis approach
was employed. The findings reveal a strong preference for formative
assessments, aligning with educational theories such as Constructivism,
Assessment for Learning (AfL), and the Cultural-Historical Activity
Theory (CHAT). The study also highlights the growing adoption of
project-based assessments, rubrics, and reflective practices as effective
means of evaluating student outcomes. Contrary to prevailing trends, a
subset of participants also advocated for well-designed standardized
tests, providing a nuanced understanding of their role in OBE.
Implications point to a significant shift in assessment culture within
Vietnam’s higher education sector, with broader ramifications for
*
Corresponding author: Nguyen Anh Thi, nathi@ctu.edu.vn
©Authors
417
educational policy and practice. Limitations and recommendations for

further research are also discussed.
Keywords: Assessment strategies; outcome-based education;

preferences; practices; Vietnamese university lecturers
1. Introduction
The global dilemma of dwindling student performance in higher education has
led to an increased focus on outcome-based education (OBE). As nations across
the world grapple with the challenges of ensuring the quality of education, the
widespread concern over students’ performance in higher educational
institutions has driven the attention of researchers and educators. This
pedagogical approach emphasizes the importance of achieving specific learning
outcomes, and therefore necessitates debates and discussions on the most suitable
methods of assessment. In understanding these, it becomes pivotal to explore the
varied arguments educators present regarding their preferences in assessment
strategies, requiring a shift in how student performance is assessed (Davis, 2003).
Vietnam’s higher education (HE) system is also progressing toward an outcome-
based model to align with international educational standards (Thuy, 2022). This
decision resonates with the national and regional efforts to address the observed
gaps in student performance. In this transition, understanding how teachers
implement assessments tailored to measure learning outcomes is of critical
importance (Tian et al., 2023).
As Vietnamese HE institutions transition to an outcome-based framework, there

is a pressing need for understanding the assessment methods employed by
lecturers to gauge learning outcomes. Given the complexities of OBE and the
significant emphasis it places on ensuring students meet particular outcomes,
debates surrounding the right assessment techniques are inevitable. This is a
reflection of a broader global dialogue on educational practices that have emerged
due to observed performance discrepancies among students at the higher
education level, both on a local and global scale. While OBE has been explored in
diverse educational contexts, especially in the West (Harden, 2007; Davis et al.,
2007; Manzoor et al., 2017), there is limited research focusing specifically on the
types of assessments used by teachers in Vietnam’s HE institutions within an OBE
framework.
Previous studies on OBE in Vietnam have often concentrated on policy

implications, curricular reforms, or comparisons with Western models (Han et al.,
2016; Thuy, 2022; Nguyen & Lee, 2022). However, one glaring omission has been
an in-depth exploration of teachers’ preferences and strategies regarding
assessment methods tailored for OBE. This lack mirrors a larger, global research
gap on the specifics of assessment methods within the OBE paradigm, particularly
in regions where the model is newly adopted. There is a conspicuous absence of
research that solely investigates the types of assessments that teachers are using
to measure specific learning outcomes within the Vietnamese HE context. The sole
aim of this qualitative study is to investigate the types of assessments used by
teachers in Vietnamese HE institutions to measure learning outcomes. Following
418
the aim of the study, one research question is addressed as follow: “What types of
assessments do Vietnamese university lecturers use to measure their students’
learning outcomes regarding the OBE framework?”
This study seeks to fill a critical research gap by not only identifying the specific
assessments but also by comprehending the underlying reasons and motivations
that guide lecturers in their choices. By addressing both local (Vietnamese) and
more global concerns about assessment in OBE, the findings are anticipated to be
of utility to lecturers, curriculum designers, and policymakers, offering them
context-specific insights that can aid in the effective transition toward an OBE
model.
2.1. Evolution of OBE
OBE has its roots in the educational reforms of the late 20th century, particularly
in the United States and the United Kingdom (Spady, 1994). OBE aimed to
transition from traditional content-based education to a learner-centric model that
emphasizes demonstrable skills and competencies (Harden, 2002). Studies have
shown that OBE is an effective approach for ensuring educational quality and
relevance (Gurukkal, 2020). Moreover, OBE has gained global attention, with its
principles being applied in various international contexts, from Europe to Asia
(Davis & Harden, 2003).
2.2. The Significance and Types of Assessment in OBE

Assessment in OBE is an integral component that differs significantly from
traditional educational assessments. It moves beyond merely measuring what the
students know to evaluating what students can do with what they know (Biggs &
Tang, 2011). Assessments in OBE are often aligned with learning outcomes and
may include a variety of methods, such as formative assessments, summative
assessments, authentic assessments, peer and self-assessments, and e-assessments
are commonly used in an outcome-based model. Formative assessments are
ongoing assessments that inform both teaching and learning (Japee & Oza, 2021).
They may include quizzes, discussion boards, and informal observations.
Summative assessments are comprehensive evaluations often done at the end of
a course or program, such as final exams and capstone projects (Asim et al., 2021).
Authentic assessments require students to apply their skills in real-world contexts
and are particularly useful for measuring higher-order cognitive skills (Khanna &
Mehrotra, 2019). Peer and self-assessments engage students in evaluating either
their own or peers’ performances, thereby developing metacognitive skills
(Raupach et al., 2011; Thirumoorthy, 2021). With the rise of technology, online
assessments have become increasingly prevalent (Sapawi, 2021). Assessment in
OBE serves not just as a means to grade students but as a tool for ongoing
improvement in the learning process.
2.3. Assessments in OBE within Asian Contexts

Studies have also looked at OBE implementation in Asian HE settings. Research
suggests that cultural factors, educational traditions, and institutional readiness
significantly affect the types of assessments used (Coombe et al., 2020; Dorner &
419
Gorman, 2006; Kennedy, 2002). For instance, a study in Malaysia found that
although teachers appreciated the value of OBE, there were challenges in
implementing non-traditional assessment types like portfolios and presentations
(Tan et al., 2018).
Despite the existing body of knowledge on OBE and assessment methods, there
is a noticeable gap in the literature when it comes to exploring the types of
assessments used by teachers specifically in the context of Vietnamese HE. Most
of the existing studies focus either on Western educational systems or provide a
more general overview of OBE in Asian settings without diving into the particular
nuances of Vietnam’s educational landscape. The literature review highlights the
increasing importance of OBE in shaping educational practices around the world
and underscores the pivotal role of assessments in realizing the aims of OBE.
While there is a plethora of studies investigating various aspects of OBE and
assessment methods, research focusing on the types of assessments used by
teachers to measure learning outcomes in Vietnamese HE is scant. This gap in the
literature substantiates the need for the current study, aimed at exploring this
specific aspect in the Vietnamese context.
3. Methods
Given the exploratory nature of the study, a qualitative approach was deemed
most suitable to gain nuanced, context-specific insights. Qualitative research
allows for a deeper understanding of social phenomena, offering the opportunity
to capture the complexities and intricacies of educational practices, particularly in
the realm of assessments (Creswell & Poth, 2016).
There are three theoretical frameworks used in this study, including

Constructivist Paradigm (CP), Assessment for Learning (AfL), and Cultural-
Historical Activity Theory (CHAT). Firstly, the study is rooted in a CP, which
posits that knowledge is socially constructed and context-dependent (Vygotsky &
Cole, 1978). This framework aligns with the qualitative approach, providing a lens
through which to understand how lecturers construct their assessment methods
in relation to the learning outcomes they aim to measure. Secondly, AfL serves as
a secondary theoretical framework for the study. AfL argues that assessments
should not just be a tool for grading but should serve as an ongoing process that
informs teaching and learning (Black & Wiliam, 1998). This perspective will guide
the interpretation of how assessments are designed and implemented to measure
learning outcomes. Finally, given the unique sociocultural context of Vietnamese
HE, CHAT is also incorporated to understand the interplay between individual
actions (teachers’ choices in assessment methods) and the wider socio-cultural
milieu (Sannino & Engeström, 2018).
3.2. Participants
The study engaged 15 university lecturers who are currently employed in various
HE institutions in Vietnam. These lecturers are integral to the research as they
offer firsthand insights into the types and challenges of assessments used to
measure learning outcomes within an OBE framework. Chosen for their varied
420
academic backgrounds, these lecturers provide a multifaceted view of

assessments, taking into account the diverse disciplines represented in HE as well
as the complexities associated with the role of lecturers as assessors. The
utilization of university lecturers as participants is well-documented in similar
research studies, as they possess firsthand experience and a deep understanding
of the educational landscape (Zulaiha et al., 2020).
The participant group included 8 males and 7 females, offering a balanced gender
perspective that enriches the study’s comprehensiveness. The participants had an
average age of 42.8 years, coupled with an average teaching experience of 17
years. The maturity and extensive professional background of these lecturers are
expected to provide a nuanced understanding of the intricacies involved in
implementing outcome-based assessments. Researchers like Bukor (2015) have
underscored the value of capturing the experiences of seasoned educators, as they
bring a depth of understanding rooted in years of practice. Among the 15
participating lecturers, 11 hold Ph.D. degrees, while the remaining 4 are enrolled
as Ph.D. candidates. This level of academic achievement signifies not only a strong
mastery of subject matter but also a comprehensive understanding of research
methodologies, both of which are anticipated to elevate the quality of the collected
data. Academic qualifications, particularly at the doctoral level, have been
highlighted as influential in grounding participants’ responses in sound
pedagogical practices (Williams, 2011). Participants represented a broad array of
academic fields, including the sciences, social sciences, humanities, and
professional studies. Such interdisciplinary inclusion is aimed at capturing the
diverse roles and challenges of lecturers as assessors across varying academic
environments, thereby contributing to a more holistic understanding of
assessment practices in the context of Vietnam’s HE.
Participants were selected using a purposive sampling strategy, a method

recognized for its efficacy in qualitative research when researchers need to ensure
that specific characteristics or criteria are represented within the sample (Patton
et al., 2015). The primary criteria for selection were their roles as university
lecturers and assessors in Vietnamese HE institutions. Additional layers of
diversity, such as gender, age, and academic qualifications, were also considered
to ensure a comprehensive representation.
The chosen participants for this study are not only diverse in their academic
disciplines but also balanced in terms of gender, experience, and academic
qualifications. This well-rounded group is expected to provide in-depth,
multifaceted perspectives on the types of assessments used for measuring
learning outcomes in the context of Vietnamese HE. Their experiences and
insights are considered invaluable for achieving the objective of this qualitative
study. The decision to adopt such a heterogeneous sample is grounded in the
assertion by Johnson and Christensen (2012) that diversity within participant
groups often leads to richer and more comprehensive data in qualitative research.
421
3.3. Data Collection

The core data collection method employed in this study was semi-structured
interviews, chosen specifically for their ability to glean nuanced, detailed insights
from participants. These interviews were crucial for exploring the types of
assessments used by university lecturers in Vietnamese HE, particularly within
the framework of OBE.
The interview guide that steered these discussions was carefully constructed to
include a set of open-ended questions. In response to the critique regarding the
in-depth nature of questions, we ensured that none of the core questions merely
required a binary “yes” or “no” response. These questions were designed to evoke
comprehensive responses regarding the types of assessments lecturers use, the
criteria for selecting these assessments, and the challenges and benefits tied to
their implementation. Examples of questions include: “Can you describe the types
of assessments you commonly use in your courses?”, “What factors influence
your choice of assessment methods?”, and “What challenges have you
encountered in implementing these assessments within an outcome-based
framework?”
To cater to the local context, interviews were conducted both in English and
Vietnamese, depending on the participant’s comfort level with the language. This
multilingual approach aimed to remove language barriers and encourage
candidness, thereby enriching the quality of the data collected. Each interview
was designed to last between 45 to 60 minutes, and the setting for these interviews
was adapted according to the participant’s preference—either in-person or
through a secure video conferencing platform. To ensure the highest degree of
accuracy, all interviews were audio-recorded, albeit only after receiving explicit
consent from the participants. To guarantee confidentiality and foster an
atmosphere conducive to open dialogue, interviews were conducted in a private
setting.
Upon completion of each interview, the audio recordings were immediately

transcribed verbatim. For interviews conducted in Vietnamese, the transcripts
were translated into English to facilitate analysis. Once transcribed, a validation
process was initiated where a subset of participants were asked to review their
transcripts to ensure that their perspectives were accurately captured (known as
member-checking). The documents were anonymized to protect participant
identities. To further address the concerns regarding reliability, a second
researcher independently analyzed a portion of the transcripts, ensuring
consistency in theme identification and interpretation. A cursory initial analysis
followed each transcription to flag any recurring themes or interesting patterns,
thereby allowing us to make any necessary adjustments to the interview guide for
subsequent interviews. This iterative approach rendered the data collection
process dynamic, letting the research adapt to emergent themes and areas
warranting further inquiry.
Ethical considerations were stringently adhered to throughout the data collection

phase. Each participant was provided an informed consent form prior to the
422
interview. This form delineated the study’s objectives, the confidentiality

measures in place, and clarified that participation was entirely voluntary.
Participants were also assured they had the right to withdraw from the study at
any time, without any form of consequence.
3.4. Data Analysis

The data analysis phase of this study is pivotal for interpreting the rich qualitative
information collected from the 15 university lecturers. To navigate the nuances of
the gathered data, a thematic analysis was employed, grounded in the six-phase
model developed by Braun et al (2023). This model was selected due to its proven
utility in identifying, analyzing, and reporting themes within complex qualitative
data.
The initial step involved a comprehensive familiarization with the data, which
meant thoroughly reviewing all interview transcripts to grasp the underlying
tones and themes. This deep engagement with the textual data facilitated the
subsequent phase of generating initial codes. In this step, we isolated key phrases,
terms, or concepts that appeared noteworthy for the study’s research question.
Following this, the generated codes were aggregated into overarching themes that
encapsulated groups of related codes. During the theme reviewing phase, each
identified theme was rigorously evaluated for its relevance to the research
questions and the entire dataset. Some themes were refined for greater clarity and
specificity, while others that did not sufficiently contribute to the research
question were omitted. The final step in the thematic analysis process involved
defining and naming the themes to succinctly represent their core concepts.
It is important to note that the validity and reliability of this analysis were
carefully considered. Strategies such as triangulation were employed, where
multiple researchers analyzed the data to ensure consistent interpretation.
Member checking was also used to bolster the study’s validity; participants were
offered a chance to review the themes and interpretations derived from their
interviews, providing an opportunity for them to confirm or challenge the current
findings. Additionally, an audit trail documented every analytical step taken,
serving as a transparent record that can be scrutinized for the sake of validity and
to ensure that the study could be replicated by other researchers. Ethical
considerations were also strictly adhered to throughout the analysis. All efforts
were made to anonymize participants’ information to maintain confidentiality.
In summary, the data analysis approach was designed to offer a structured yet
flexible methodology for making sense of the qualitative data. By adhering to a
rigorous thematic analysis model and implementing strategies to validate the
findings, this study aims to provide robust and nuanced insights into the types of
assessments used to measure learning outcomes in the context of Vietnamese HE.
The results are expected to contribute valuable knowledge to the existing
literature and offer practical implications for the field.
423
4. Findings
In this study, one of the most significant findings is the strong preference among
university lecturers for the use of formative assessments to evaluate student
learning outcomes. Out of the 15 participants, eleven lecturers emphasized the
essential role of formative assessments in their teaching and assessment strategies.
Participant P1 noted, “One of the keys to helping students learn is continuous feedback.
Formative assessments, be it short quizzes or even simple class discussions, allow me to
immediately gauge where the students are in their learning journey.” Participant P3
highlighted, “I have found that relying solely on summative assessments like finals or
midterms does not give me a full picture of a student’s capabilities or understandings.
Formative assessments give both the student and me a chance to adjust before it is too
late.” The emphasis on formative assessments can be framed within several
theoretical paradigms. For instance, from the standpoint of the CP, this preference
makes sense. Constructivism posits that learning is an active, ongoing process,
where immediate feedback can have a transformative impact on a student’s
educational journey. Likewise, the affinity for formative assessments aligns well
with the principles of AfL, which advocates for assessments that serve as an
integral part of the instructional process rather than merely as an evaluative
endpoint. Additionally, when viewed through the lens of the CHAT, the results
reveal a potentially groundbreaking shift in the traditional Vietnamese
educational culture. CHAT points to how historical and social contexts influence
activity, and the movement from a focus on summative assessments towards a
more formative approach suggests a noteworthy transition within Vietnam’s
educational landscape.
In addition to formative assessments, the findings reveal that project-based

assessments are gaining traction as an effective means to evaluate student
outcomes. Out of 15 participants, nine (n=9) explicitly mentioned incorporating
project-based assessments into their curriculum to align with OBE principles.
Participant P6 remarked, “Project-based assessments allow me to evaluate not just the
end result but the entire process. This aligns well with the objectives of OBE where we are
interested in multiple outcomes like problem-solving and teamwork, not just factual
recall.” Participant P12 stated, “I find project-based assessments more holistic. They
require students to apply a range of skills and knowledge, reflecting real-world situations
better than traditional exams.” Project-based assessments align well with the
theoretical underpinnings of the study. The CP supports this type of assessment
as it allows for active, student-centered learning, offering an environment where
students can construct their knowledge collaboratively. Similarly, AfL
underscores the importance of assessments that inform and guide ongoing
learning, a natural fit for project-based evaluations. From the CHAT perspective,
the willingness to embrace project-based assessments may signify a broader
educational shift within Vietnam’s historically exam-focused educational culture.
Another significant element in the assessment landscape for OBE that emerged
from this current study is the use of rubrics. A majority of the participants, eleven
out of fifteen (n=11), emphasized the utility of rubrics to ensure a standardized
and transparent grading system. Participant P6 noted, “Rubrics are incredibly useful
for making the grading process transparent. Students know exactly what is expected of
them to achieve each outcome.” This sentiment was echoed by Participant P9, who
mentioned, “With a well-designed rubric, even complex projects or assignments can be
424
graded more objectively. It aligns well with OBE because it focuses on discrete skills or
competencies.” The use of rubrics seems to align well with the theoretical
underpinnings of this study. From a Constructivist standpoint, rubrics offer a
standardized yet flexible approach that can adapt to individual learning
trajectories. Additionally, they fit well within the AfL paradigm as they provide
an ongoing guide for teaching and adjustment based on student performance.
Within the CHAT framework, rubrics can be viewed as a tool that aligns
traditional Vietnamese pedagogical methods with modern OBE strategies.
Moreover, the study revealed that reflection and self-assessment techniques were
also gaining traction in the context of OBE, with seven out of fifteen participants
(n=7) mentioning their use. As Participant P1 stated, “I always ask my students to
write a reflective journal at the end of the semester. It forces them to think about what they
have learned and how they have met the outcomes we set.” Participant P10 supported
this by saying, “Self-assessment forms are a regular part of my classes. It helps students
take ownership of their learning, which is what OBE is all about.” Reflective practices
are inherently constructivist, encouraging students to actively engage with the
material and their personal learning journey, thereby fulfilling the principles of
the CP. Reflection and self-assessment also align with the principles of AfL by
making students active participants in the assessment process. Within the CHAT
framework, the use of reflective practices can be seen as a form of innovative
assessment, especially given that the Vietnamese educational context may not
typically encourage this level of student autonomy.
Also worth mentioning is the use of real-world problem-solving assessments, a

strategy noted by six out of fifteen participants (n=6). “We often assign tasks that
simulate real-world challenges. This helps students to directly apply theory to practice,
offering a tangible measure of various learning outcomes,” said Participant P11.
Participant P14 elaborated, “The purpose of education is not just to score well on tests,
but to prepare students for real-world challenges. These kinds of assessments make the
education we provide more relevant and outcome-based.” Such problem-solving
assessments align closely with the theoretical frameworks of the study. According
to the CP, learning is most effective when rooted in real-world context, which
these assessments provide. Within AfL, real-world problem-solving tasks serve as
both formative and summative assessments, guiding learning while also
measuring its efficacy. In terms of CHAT, the focus on real-world applicability
acts as a mediation tool between traditional academic learning and the skills
required in modern Vietnamese society.
An interesting observation from the interviews was the use of peer assessments
in conjunction with traditional grading methods. Five out of fifteen participants
(n=5) suggested that peer assessments were an important adjunctive tool in their
outcome-based assessment repertoire. Participant P2 elaborated, “Peer assessments
help students understand the evaluation criteria better, as they have to apply them to
evaluate their peers’ work. It also creates a sense of shared responsibility.” Participant P5
supported this notion, stating, “Peer assessments can be invaluable. They are not only
consistent with OBE objectives but also help in cultivating soft skills like critical thinking
and constructive criticism.” This particular finding aligns well with the theoretical
frameworks of this study. In the CP, peer assessments fit the profile of a dynamic
425
learning environment where students actively engage with one another to

construct knowledge. Under the AfL umbrella, peer assessments provide an
additional layer of feedback, enriching the learning experience by involving
multiple perspectives. Within the CHAT framework, the introduction of peer
assessments can be viewed as an innovative leap, given that it defies the
traditionally teacher-centric model of assessment in Vietnam.
An interesting counterpoint arose around the use of standardized tests for

outcome-based assessments. Five out of 15 participants (n=5) expressed a belief
that well-designed standardized tests could serve as effective tools for OBE. This
is especially noteworthy given the traditional reliance on such tests in Vietnamese
HE. “Standardized tests, when designed with OBE in mind, can offer a uniform measure
of outcomes across different classes or even institutions,” Participant P4 opined. This
viewpoint diverges from the CP and AfL, which typically emphasize more
learner-centered, formative approaches. However, it aligns with CHAT, as
standardized tests are deeply ingrained in the educational history and culture of
Vietnam. The advocacy for well-designed standardized tests may reflect a
transitional phase where lecturers are trying to adapt existing assessment tools to
align more closely with OBE principles.
Another surprising yet important finding was the increased integration of

technology-based assessments, as noted by four out of fifteen participants (n=4).
Participant P3 explained, “We are gradually moving towards online quizzes and
interactive assessments. These digital platforms can immediately identify which learning
outcomes the students are meeting or struggling with.” Participant P7 added, “Online
assessments are more than just convenient; they provide immediate feedback that can be
crucial for course adjustments and student self-correction.” The adoption of
technology-based assessments finds resonance in all three theoretical frameworks
that underpin this study. Within the CP, online assessments can adapt in real-time
to student responses, tailoring subsequent questions or tasks to individual
performance levels. In terms of AfL, the immediate feedback provided by digital
platforms allows for real-time adjustments by both teachers and students. From
the perspective of the CHAT, the shift toward digital assessments represents a
tool-mediated change in pedagogical practice, which is particularly noteworthy
in the context of traditional Vietnamese educational settings.
One more significant finding was the occasional use of oral assessments, as
brought up by four out of fifteen participants (n=4). Participant P15 commented,
“Oral assessments, particularly presentations and oral exams, allow us to gauge students’
understanding in a dynamic way. Students cannot hide behind memorization; they need
to show true comprehension and ability to articulate their thoughts.” Participant P2
added, “Oral assessments are particularly useful for courses that require strong verbal
communication skills. They are aligned with specific learning outcomes in disciplines like
languages, communication studies, and even law.” This approach fits well with the
CP, where the emphasis is on active, multi-faceted methods of assessment that
reveal deeper layers of student understanding. From the AfL perspective, oral
assessments can be an effective way to engage students in the learning process by
providing immediate, interactive feedback. In the context of CHAT, oral
assessments can be seen as an adaptation to fit the sociocultural aspects of
426
Vietnamese education, as they may closely align with traditional modes of

teaching and assessment that value oral proficiency and dialogue.
Furthermore, a smaller subset of participants, three out of fifteen (n=3), indicated

the use of portfolios to demonstrate a range of learning outcomes. Participant P6
stated, “Portfolios offer a broad spectrum view of student skills, from research to
communication to critical thinking.” According to Participant P11, “Portfolios not only
demonstrate the final outcome but also the process of learning, which is essential for OBE.”
Portfolio assessments could be seen through the lens of all three theoretical
frameworks as they provide a multifaceted assessment tool that encourages
continuous learning and skill development, perfectly aligning with both the CP
and AfL. They also represent a shift in traditional Vietnamese assessment culture,
highlighting the influence of more progressive, student-centric educational
philosophies, thus aligning with CHAT.
Finally, the study found that interdisciplinary assessments are beginning to gain
attention, although they are not yet widely adopted. Three out of fifteen
participants (n=3) spoke favorably about assessments that cross traditional subject
boundaries. Participant P4 noted, “Interdisciplinary assessments make the learning
more holistic. For instance, a project could require students to integrate knowledge from
both the humanities and sciences. This is very much in line with the overall goals of OBE,
which values a well-rounded skill set.” The concept of interdisciplinary assessments
intersects intriguingly with the theoretical frameworks. In terms of the CP, it adds
a layer of complexity to the learning environment, enriching it by making
connections across different domains of knowledge. Under the AfL framework,
interdisciplinary projects can be formative in nature, guiding learning as students
are required to integrate knowledge from various sources. Finally, from a CHAT
perspective, interdisciplinary assessments represent a significant departure from
traditional Vietnamese approaches, which often compartmentalize subjects,
signifying a transformative change in educational practice.
5. Discussion
The preference for formative assessments among university lecturers in this study
is consistent with previous research highlighting the growing emphasis on
formative, rather than summative, assessment in HE globally (e.g., Konopasek et
al., 2016; Dixson & Worrell, 2016). The majority of participants in this study agreed
that formative assessments provide immediate feedback, aiding in the continuous
learning process—a view that finds theoretical support in the CP and AfL. This
alignment strengthens the idea that formative assessments are effective in
promoting active learning and continuous improvement (Shen, 2022). Notably,
this study provides new insights within the context of Vietnamese HE. The shift
toward formative assessments and away from traditional summative evaluations
can be seen as a transformative change when viewed through the lens of CHAT.
Vietnamese educational culture has historically been dominated by high-stakes,
summative testing (Ngo, 2022), and the findings of this study may indicate a
broader cultural shift. Furthermore, the current study also revealed a significant
number of participants incorporating project-based and technology-based
assessments, aligning well with the principles of OBE and AfL. The emergence of
427
these types of assessments suggests a growing acceptance of diverse assessment

methods (Boey et al., 2023). Previous studies (e.g., Widiana et al., 2021; Parmiti et
al., 2021) have also supported the utility of project-based assessments but have
not highlighted the growing use of technology-based assessments, which the
current study has found to be an increasing trend.
Interestingly, this study discovered that a subset of lecturers sees a role for well-
designed standardized tests within OBE. This stands in contrast to the dominant
view in literature that champions formative, learner-centered assessment
strategies (Blyth & Davis, 2008; Konopasek et al., 2016). While the current findings
do not negate the efficacy of formative assessments, they do suggest that
standardized tests may still have a place in Vietnamese HE when designed with
OBE principles in mind, aligning with CHAT but diverging from AfL and the CP.
In alignment with modern educational philosophies, the use of rubrics (Le et al.,
2023), reflection and self-assessment techniques (Panadero et al., 2016), and peer
assessments (Langfeldt et al., 2021) were also noted in this study. While these
methods have been studied and supported in Western contexts, their emergence
in a Vietnamese setting is particularly noteworthy, possibly indicating a
harmonization of global educational practices and the adaptability of Vietnamese
HE.
6. Conclusion
The overarching aim of this research study was to explore the landscape of
assessment preferences and practices among university lecturers in Vietnamese
HE institutions. Driven by an increased focus on learner-centered pedagogies,
educational institutions worldwide are shifting their assessment strategies.
However, there has been limited investigation into how these global trends are
manifesting in the context of Vietnamese HE. To address this gap, this qualitative
study employed semi-structured interviews with university lecturers in Vietnam.
The research framework was primarily grounded in the CHAT, CP, and AfL. The
sample comprised of 15 lecturers across various disciplines, providing a cross-
sectional view of current practices and preferences in assessment methods.
The findings revealed that a majority of lecturers favor formative assessments,

citing their role in providing immediate feedback and promoting continuous
learning. This is in line with global trends and existing literature on effective
assessment in HE. Interestingly, this study also uncovered a growing inclination
towards project-based and technology-based assessments. A minority viewpoint,
but noteworthy nonetheless, suggested that standardized tests designed in
accordance with OBE principles may still have a role to play. The current study
also found an increasing usage of rubrics, self-assessment techniques, and peer
assessments, which are relatively new to the Vietnamese educational culture but
align with contemporary global educational philosophies.
In conclusion, this study provides valuable insights into the shifting trends in
assessment practices within the specific milieu of Vietnamese HE. While the
practices and preferences are generally converging towards the global shift in
learner-centered, formative assessments, there are unique adaptations and views
428
that reflect Vietnam’s specific educational culture and history. This research
contributes to a more nuanced understanding of the evolving educational
landscape in Vietnam and has implications for both policy-making and
pedagogical practice.
7. Implications
The strong preference for formative assessments among Vietnamese university
lecturers, as identified in this study, calls for a reevaluation of assessment policies
at institutional and national levels. Current systems, which often prioritize
summative assessments, might need to be adapted to encourage more formative
assessment practices. This could include teacher training programs, such as
trainning workshops or conferences in student assessment, that familiarize
lecturers with the benefits and methodologies of formative assessments.
Additionally, the growing interest in technology-based assessments highlights
the need for investment in digital infrastructure to support these initiatives.
The findings suggest that lecturers are increasingly adopting diverse assessment
methods, including project-based assessments and technology-driven
evaluations, to better align with OBE principles. This is an important shift for
pedagogical practice, suggesting that lecturers are not just focused on the ‘what’
of learning outcomes, but also the ‘how,’ investing in methodologies that foster a
broader set of skills including critical thinking, problem-solving, and teamwork.
Moreover, the adoption of rubrics, self-assessments, and peer assessments points
towards a more transparent and democratic assessment environment where
students are active participants in the learning journey. The growing preference
for these types of assessment not only signifies a shift towards transparency in
grading but also underscores a transformative move towards democratizing the
educational process. Specifically, these assessment methods empower students,
giving them agency in their own learning journey. Such an approach has the
potential to foster greater student engagement, motivation, and ownership of
their educational outcomes. By involving students directly in the assessment
process, educators can cultivate a more collaborative and constructive learning
environment that extends beyond mere content mastery to the development of
critical life skills such as self-awareness, peer feedback, and continuous self-
improvement. This shift has profound implications for curriculum development,
pedagogical practices, and institutional policies, emphasizing the need for
educators and institutions to reconsider traditional top-down teaching models
and embrace more participatory, student-centric approaches.
The interest in technology-based assessments and the usage of digital platforms

for quizzes and interactive learning activities suggest that there is an appetite for
integrating technology into the assessment paradigm. This has implications for
educational technology developers, who should consider designing platforms
and applications that can cater to the nuanced needs of a Vietnamese educational
setting.
From the perspective of CHAT, the shift towards more learner-centered

assessment methods suggests a cultural shift in the Vietnamese educational
429
landscape. Historically rooted in rote learning and high-stakes, summative

assessments, Vietnamese educational culture is showing signs of moving towards
more progressive, learner-centered approaches. This is a significant insight for
educational theorists and practitioners alike, as it indicates a broader
transformation that might be underway in Vietnam’s education system.
8. Limitations and Recommendations for Further Studies

One of the primary limitations of this study is the geographical scope; the survey
was conducted exclusively within a specific region in Vietnam, potentially
limiting the generalizability of the findings to the wider Vietnamese educational
landscape. Furthermore, the sample size consisted of university lecturers and did
not include other key stakeholders like students, administrators, or policymakers.
This narrow focus may not capture the full complexity and diversity of
perspectives on educational assessments. Another limitation lies in the study’s
cross-sectional design, which provides a snapshot of attitudes at a particular point
in time but does not capture any longitudinal changes or trends.
In light of these limitations, there are several avenues for further research. Future
studies could benefit from a larger, more diverse sample that includes various
stakeholder groups and covers multiple regions or even countries to gain a
comprehensive understanding of assessment preferences and practices.
Longitudinal studies may be valuable to capture evolving attitudes towards
educational assessments over time, particularly in the context of rapid
technological advancements and educational reforms. Moreover, experimental or
quasi-experimental designs could be employed to examine the efficacy of various
assessment methods on student learning outcomes, thus providing more concrete
evidence to inform educational practices and policies. Lastly, it is recommended
to explore the use of varied student assessment types tailored to specific
disciplines, as each field possesses unique characteristics that necessitate distinct
approaches for accurately evaluating student outcomes.
9. References
Asim, H. M., Vaz, A., Ahmed, A., & Sadiq, S. (2021). A review on outcome based education
and factors that impact student learning outcomes in tertiary education
system. International Education Studies, 14(2), 1-11.
https://doi.org/10.5539/ies.v14n2p1
Biggs, J., & Tang, C. (2011). Train-the-trainers: Implementing outcomes-based teaching
and learning in Malaysian higher education. Malaysian Journal of Learning and
Instruction, 8, 1-19. https://bit.ly/3R9RDDg
Black, P., & Wiliam, D. (1998). Inside the black box: Raising standards through classroom
assessment. Granada Learning.
Blyth, C. S., & Davis, J. N. (2008). Using formative evaluation in the development of
learner-centered materials. Calico Journal, 25(1), 48-68.
https://doi.org/10.1558/cj.v25i1.48-68
Boey, C. K., Sathish, S., & Koh, S. N. A. (2023). Impact of technology-enabled project-based
assessments on learner outcomes in higher education. International Journal of
Mobile Learning and Organisation, 17(1-2), 131-148.
https://doi.org/10.1504/IJMLO.2023.128354
430
Braun, V., Clarke, V., & Hayfield, N. (2023). Thematic analysis: A reflexive approach. SAGE
Publications.
Bukor, E. (2015). Exploring teacher identity from a holistic perspective: Reconstructing
and reconnecting personal and professional selves. Teachers and Teaching, 21(3),
305-327. https://doi.org/10.1080/13540602.2014.953818
Coombe, C., Vafadar, H., & Mohebbi, H. (2020). Language assessment literacy: What do
we need to learn, unlearn, and relearn?. Language Testing in Asia, 10, 1-16.
https://doi.org/10.1186/s40468-020-00101-6
Creswell, J. W., & Poth, C. N. (2016). Qualitative inquiry and research design: Choosing among
five approaches. Sage Publications.
Davis, M. H. (2003). Outcome-based education. Journal of Veterinary Medical
Education, 30(3), 258-263. https://doi.org/10.3138/jvme.30.3.258
Davis, M. H., & Harden, R. M. (2003). Competency-based assessment: making it a
reality. Medical Teacher, 25(6), 565-568.
https://doi.org/10.1080/0142159032000153842
Davis, M. H., Amin, Z., Grande, J. P., O’Neill, A. E., Pawlina, W., Viggiano, T. R., & Zuberi,
R. (2007). Case studies in outcome-based education. Medical Teacher, 29(7), 717-
722. https://doi.org/10.1080/01421590701691429
Dixson, D. D., & Worrell, F. C. (2016). Formative and summative assessment in the
classroom. Theory Into Practice, 55(2), 153-159.
https://doi.org/10.1080/00405841.2016.1148989
Dorner, D. G., & Gorman, G. E. (2006). Information literacy education in Asian developing
countries: cultural factors affecting curriculum development and programme
delivery. IFLA Journal, 32(4), 281-293.
https://doi.org/10.1177/0340035206074063
Gurukkal, R. (2020). Outcome-based education: an open framework. Higher Education for
the Future, 7(1), 1-4. https://doi.org/10.1177/2347631119886402
Han, M., Nguyen, D., Cohen, E., Drabble, L., Nguyen, H., Sen, S., & Tran, T. (2016).
Exploring the transferability of competency-based education model to social work
education in Vietnam. Social Work Education, 35(6), 659-671.
https://doi.org/10.1080/02615479.2016.1162782
Harden, R. M. (2002). Learning outcomes and instructional objectives: is there a
difference?. Medical Teacher, 24(2), 151-155.
https://doi.org/10.1080/0142159022020687
Harden, R. M. (2007). Outcome-based education: the future is today. Medical Teacher, 29(7),
625-629. https://doi.org/10.1080/01421590701729930
Japee, G., & Oza, P. (2021). Curriculum and evaluation in outcome-based
education. Psychology and Education Journal, 58(2), 5620-5625.
https://bit.ly/45GdCWH
Kennedy, P. (2002). Learning cultures and learning styles: Myth-understandings about
adult (Hong Kong) Chinese learners. International Journal of Lifelong
Education, 21(5), 430-445. https://doi.org/10.1080/02601370210156745
Khanna, R., & Mehrotra, D. (2019). The roadmap for quality improvement from traditional
through competency based (CBE) towards outcome based education (OBE) in
dentistry. Journal of Oral Biology and Craniofacial Research, 9(2), 139-142.
https://doi.org/10.1016%2Fj.jobcr.2019.02.004
431
Konopasek, L., Norcini, J., & Krupat, E. (2016). Focusing on the formative: building an
assessment system aimed at student growth and development. Academic
Medicine, 91(11), 1492-1497. https://doi.org/10.1097/ACM.0000000000001171
Langfeldt, L., Reymert, I., & Aksnes, D. W. (2021). The role of metrics in peer
assessments. Research Evaluation, 30(1), 112-126.
https://doi.org/10.1093/reseval/rvaa032
Le, X. M., Phuong, H. Y., Phan, Q. T., & Le, T. T. (2023). Impact of using analytic rubrics
for peer assessment on efl students’ writing performance: an experimental
study. Multicultural Education, 9(3), 41-53.
Manzoor, A., Aziz, H., Jahanzaib, M., Wasim, A., & Hussain, S. (2017). Transformational
model for engineering education from content-based to outcome-based
education. International Journal of Continuing Engineering Education and Life Long
Learning, 27(4), 266-286. https://doi.org/10.1504/IJCEELL.2017.087136
Ngo, X. M. (2022). English assessment in Vietnam: status quo, major tensions, and
underlying ideological conflicts. Asian Englishes, 25(2) 1-13.
https://doi.org/10.1080/13488678.2022.2132128
Nguyen, N., & Lee, J. C. K. (2022). Higher education curriculum reforms in Vietnam in the
era of globalization. In Discourses of Globalisation and Higher Education Reforms:
Emerging Paradigms (pp. 35-50). Springer International Publishing.
Panadero, E., Brown, G. T., & Strijbos, J. W. (2016). The future of student self-assessment:
A review of known unknowns and potential directions. Educational Psychology
Review, 28, 803-830. https://doi.org/10.1007/s10648-015-9350-2
Parmiti, D. P., Rediani, N. N., Antara, I. G. W. S., & Jayadiningrat, M. G. (2021). The
effectiveness of local culture-integrated science learning through project-based
assessment on scientific attitudes and science process skills of elementary school
students. Jurnal Pendidikan IPA Indonesia, 10(3), 439-446.
https://doi.org/10.15294/jpii.v10i3.31301
Patton, C., Sawicki, D., & Clark, J. (2015). Basic methods of policy analysis and planning--
pearson etext. Routledge.
Raupach, T., Münscher, C., Beißbarth, T., Burckhardt, G., & Pukrop, T. (2011). Towards
outcome-based programme evaluation: using student comparative self-
assessments to determine teaching effectiveness. Medical Teacher, 33(8), e446-e453.
https://doi.org/10.3109/0142159X.2011.586751
Sannino, A., & Engeström, Y. (2018). Cultural-historical activity theory: founding insights
and new challenges. Cultural-Historical Psychology, 14(3), 43-56.
https://doi.org/10.17759/chp.2018140305
Sapawi, R. (2021). Alternative and online assessment in the context of outcome based
education: a practical guide. Turkish Journal of Computer and Mathematics Education
(TURCOMAT), 12(3), 2376-2385. https://bit.ly/3Z7Oq93
Shen, J. (2022). Introduction of social media to aid active-learning in medical
teaching. Interactive Learning Environments, 30(10), 1932-1939.
https://doi.org/10.1080/10494820.2020.1766508
Spady, W. G. (1994). Outcome-Based Education: Critical Issues and Answers. American
Association of School Administrators.
Tan, K., Chong, M. C., Subramaniam, P., & Wong, L. P. (2018). The effectiveness of
outcome based education on the competencies of nursing students: A systematic
review. Nurse Education Today, 64, 180-189.
https://doi.org/10.1016/j.nedt.2017.12.030
432
Thirumoorthy, G. (2021). Outcome based education (OBE) is need of the hour. Educational
Quest, 12(1), 47-53. https://doi.org/10.30954/2230-7311.1.2021.6
Thuy, C. T. (2022). Students’ perceptions of outcome-based learning activities: a case study
of university of social sciences and humanities, Vietnam National University Ho
Chi Minh city. European Journal of Education and Pedagogy, 3(3), 69-73.
https://doi.org/10.24018/ejedu.2022.3.3.293
Tian, W., Le, H. D., & Nguyen, N. T. (2023). Vietnamese university EFL teachers’ practices
and literacy in classroom assessment: A sociocultural perspective. System, 116,
103066. https://doi.org/10.1016/j.system.2023.103066
Vygotsky, L. S., & Cole, M. (1978). Mind in society: Development of higher psychological
processes. Harvard University Press.
Widiana, I. W., Tegeh, I. M., & Artanayasa, I. W. (2021). The project-based assessment
learning model that impacts learning achievement and nationalism
attitudes. Jurnal Cakrawala Pendidikan, 40(2), 389-401.
http://doi.org/10.21831/cp.v40i2.38427
Williams, R. (2011). The contribution of gaining an academic qualification to teachers’
professional learning. Journal of Education for Teaching, 37(1), 37-49.
https://doi.org/10.1080/02607476.2011.538270
Zulaiha, S., Mulyono, H., & Ambarsari, L. (2020). An investigation into EFL teachers’
assessment literacy: Indonesian teachers’ perceptions and classroom
practice. European Journal of Contemporary Education, 9(1), 189-201.
https://doi.org/10.13187/ejced.2020.1.189
433

Vol. 22, No. 10, pp. 433-452, October 2023
Unlocking Classroom Potential: Exploring the

Mediating Role of Teacher Mindset on
Embracing Differentiated Instruction
Enung Hasanah*
Universitas Ahmad Dahlan
Yogyakarta, Indonesia
M. Ikhsan Al Ghazy
Universitas Negeri Yogyakarta
Suyatno Suyatno , Ika Maryani

Universitas Ahmad Dahlan
Mohd Zailani Mohd Yusoff

Universiti Utara Malaysia
Kedah, Malaysia
Abstract. Differentiated Digital Instruction (DDI) is a viable solution to

cater to students' personalised learning needs in 21st-century education.
Despite the numerous benefits associated with this model, not every
teacher has the capability or willingness to adopt it, primarily because of
its perceived complexity. This research examines the direct and indirect
relationship between teacher knowledge about digital differentiated
learning and the availability of digital infrastructure in schools on
implementing digital differentiated learning through the mediating role
of teacher mindset through structural equation modelling. This study was
conducted among junior high school teachers in West Sumatra,
Indonesia. A simple random sampling technique was used for data
collection. The collection and sample size consisted of 161 participants.
The data analysis process was carried out using SmartPLS 4 Structural
Equation Modelling (SEM) analysis. These findings contribute to the idea
that the teacher's mindset is a mediator between digital differentiated
learning and the availability of digital infrastructure for implementing
digital differentiated learning. This research is helpful for school
principals to prioritise developing teacher mindsets first before
instructing the implementation of digital differentiated learning in
schools. As this study highlighted the importance of this subject, it also
*
Corresponding author: Enung Hasanah; enung.hasanah@mp.uad.ac.id
©Authors
434
emphasised the need for further exploration of additional factors

regarding the implementation of the digital differentiated instruction
model.
Keywords: differentiated instruction; digital instruction; teacher mindset;

quantitative; Indonesia
1. Introduction
Digital Differentiated Instruction (DDI) is a powerful educational approach that
leverages technology to tailor learning experiences to individual students' needs,
abilities and interests. It recognises that every learner is unique, and DDI
empowers educators to provide customised content and support, fostering more
effective and engaging learning. By harnessing the potential of digital tools and
data-based insights, DDI has the potential to revolutionise education by
promoting inclusivity and personalisation within the classroom, equipping
students with values such as digital agility and knowledge of human diversity.
The importance of students equipping themselves with values like digital agility
and knowledge of human diversity cannot be overemphasised in this era of
globalisation, as emphasised by Cénat (2020), Salmela et al. (2022), Jankelová et al.
(2020) and Mishra et al. (2020). These competencies are fundamental prerequisites
for students to achieve individual well-being (Killpack & Melón, 2016), and
societal progress (Naziev, 2017) in the future. Moreover, it is equally essential for
students to master cutting-edge knowledge and technology, have control over
socio-emotional aspects, recognise their potential, develop a passion for learning
and embrace positive self-acceptance (Anderson et al., 2022), in order to attain a
complete well-being.
To realise student well-being within schools (Lindsay et al., 2023), teachers should
be capable of fostering school connectedness, joy of learning, educational purpose
and academic efficacy in the school environment (Zadworna et al., 2023). They
must also be capable of providing adaptive and inclusive educational services that
cater to the needs of each student, both individually and collectively (Aas, 2023;
Prakosha et al., 2018; Svetlichnaya, 2022). This comprehensive approach is
essential for nurturing the well-being and academic success of all students.
One form of inclusive and adaptive education concept to foster student well-being
in this digital era is DDI (Hasanah, Suyatno, et al., 2022). Digital differentiated
instruction is an educational approach that leverages technology to tailor
instruction to the individual needs, interests and abilities of students. It builds on
the principles of DI, which is an approach to teaching that recognises that students
have diverse learning styles, abilities, and needs and seeks to adapt instruction to
meet those differences. Digital tools and resources enable educators to deliver
personalised and flexible learning experiences more effectively. By employing this
model, teachers can adjust the curriculum and teaching methods to match the
individual needs and abilities of students, thereby creating an inclusive learning
environment and strengthening their overall well-being in the process. Digital
tools can collect and analyse data on student performance and engagement,
allowing teachers to tailor content, pace and assessments to each student.
Adaptive learning platforms, such as Khan Academy, can automatically adjust
435
the difficulty of problems based on student performance. It is important to

acknowledge that this learning model is highly beneficial for students, specifically
those from diverse and multicultural countries such as Indonesia (Hasanah,
Badar, et al., 2022).
In previous studies, it was found that Indonesian teachers admitted to struggling

with the implementation of DDI because the model is relatively new and requires
more effort and complex time management compared to traditional teaching
methods (Aminuriyah et al., 2022; Martanti et al., 2022). Moreover, the lack of
digital infrastructure was identified as a basic challenge hindering the successful
implementation of this teaching method in remote areas (Hasanah, Suyatno, et al.,
2022). It is crucial to address these challenges to effectively implement DDI and
achieve high-quality educational outcomes (Widyanti, 2018).
Previous studies (Bi et al., 2023; Stollman, 2018) extensively explored many factors
directly influencing the adoption of differentiated instruction (DI) strategies,
including knowledge of the model (Lavania & Mohamad Nor, 2021), social
construction of culture (Kinyanjui et al., 2015; Moulin-Stożek, 2019), teacher
competence (Hanafi & Setiyani, 2021; Ruaya et al., 2022), transformative
leadership skills of school principals (Et.al, 2021; Hoogeboom & Wilderom, 2019;
Manik, 2016), and the availability of digital learning facilities (Caldera et al., 2021;
Ghazi et al., 2021, 2022; Nosova et al., 2019; Øvrelid & Bygstad, 2019).
Accordingly, DDI implementation in Indonesia is primarily influenced by critical
factors namely teacher knowledge and the availability of digital resources, both
of which are mediated by teacher mindset. It is expedient to acknowledge that the
impact of teacher mindset in connecting various factors influencing DDI practices
has not been adequately explored. To address this empirical gap, a survey was
conducted to examine the mediating role of teachers’ mindset in DI practices by
linking teacher knowledge pertaining to the model and the availability of digital
learning resources.
In terms of DDI practices, two dominant factors have been identified to play a
dominant role in enabling teachers to adopt innovative teaching principles. These
factors include the knowledge level of teachers pertaining to the proposed
teaching method and the availability of digital learning facilities. However, these
factors alone may not successfully influence the adoption of advanced teaching
innovations if not supported by the willingness of teachers to embrace new
innovative approaches. The mindset of teachers plays a crucial role in determining
whether or not they are willing to adopt innovative and renewed work
approaches (Caniëls et al., 2023). Despite the significance, the impact of teacher
mindset in connecting various factors that influence DDI practices has not been
adequately explored. This empirical gap was examined by means of a survey on
the role of mindset in instruction practices by linking teacher knowledge on DDI
and the availability of digital learning resources.
A profound understanding of the mediating role of teacher mindset in DDI

practices may provide valuable insights for the development of inclusive and
adaptive education that prioritises the needs of students (Sewagegn, 2020).
Therefore, this study aimed to identify the possible relationships by considering
teacher mindset, teacher knowledge on DI principles, and the availability of
436
digital learning resources. It also examined the relationship between teacher

competence, transformative leadership of school principals, the availability of
digital learning facilities, and the quality of DI implementation, with the growth
of teacher mindset serving as the mediating variable.
2. Theoretical Background and Hypothesis Development

This section delves into several essential concepts pertaining to the quality of DI
practices, drawing insights from various literature sources. By exploring multiple
variables, this explanation aims to assist the formulation of indicators that are
capable of describing the latent variables and strengthen logical flow.
2.1 DDI implementation influenced by teacher knowledge about DI

The principles of DDI emphasise acknowledging individual differences among
students and providing learning experiences that align with their needs and
interests (Tomlinson, 2001; Tomlinson et al., 2003). In this situation, teachers with
a growth mindset employ diverse and flexible strategies to meet the needs of
students with different learning styles (Mesler et al., 2021). The mediating role of
a growth mindset in DDI practices extends to influencing how students perceive
their abilities and fostering an adaptive attitude toward challenges (Frondozo et
al., 2022). By reinforcing the belief of students in their capacity to grow through
effort, teachers with a growth mindset can establish a learning environment that
promotes motivation and perseverance in achieving goals (Tan & Maeda, 2021).
The importance of teacher knowledge (Zabeli et al., 2021) about DI cannot be

overestimated. This model offers an approach that allows teachers to
accommodate individual differences among students in the classrooms (Pozas et
al., 2021). Teachers with adequate knowledge on DI strategies can identify the
unique needs and learning styles of each of their students, thereby enabling the
development of appropriate lesson plans. Skilled teachers in this context can
provide relevant and meaningful learning experiences for all students, ensuring
they can reach their maximum potential (Smale-Jacobse et al., 2019). Moreover,
this approach promotes inclusion and acceptance in the classroom, leading to the
establishment of a positive and supportive learning environment (Garrote et al.,
2020). From the obtained results, it can be seen that teacher knowledge is a key
factor in creating effective and inclusive learning experiences for all students.
Based on the comprehensive literature review, Hypothesis 1 was formulated as
follows:
H1 = Teacher knowledge on DI is positively and significantly related to the

quality of DDI implementation.
2.2 DDI Implementation Influenced by the Digital Infrastructure Availability

Adequate digital infrastructure is an external factor that significantly influences
the quality of DI implementation (Mahoney & Hall, 2017). For successful
implementation, teachers require access to various digital resources, including
educational software, educational applications, online learning platforms, and
other supporting materials (Timotheou et al., 2023). Therefore, in cases where
digital infrastructure is lacking, teachers may face limitations in effectively
implementing DDI. The infrastructure required for digital learning
implementation includes internet access, learning management systems, devices,
437
and computers (Supardi & Hasanah, 2020). Previous studies have shown that
technology-oriented DI effectively motivates students to learn scientific
knowledge, leading to significant differences compared to traditional learning
methods. Based on the literature review, Hypothesis 2 was formulated as follows:
H2 = The availability of digital learning resources is positively and significantly

related to the quality of DI implementation.
2.3 The Principle of DI Influenced by Teacher Mindset.

The principle of DI is strongly influenced by the growth mindset of teachers (Ginja
& Chen, 2020). In this context, growth mindset refers to the belief that intelligence
and abilities are not fixed but can be developed and enhanced through effort and
dedication. Accordingly, individuals with a growth mindset adhere to beliefs
centred around effort and growth (Yilmaz, 2022). In an educational setting,
teachers with a growth mindset believe that the abilities and intelligence of
students can be developed through appropriate effort, practice and perseverance
(Ng, 2018). They view failure as an opportunity to learn and improve, and tend to
have an open attitude toward change, readily embracing new approaches to
teaching (Kaya & Yuksel, 2022). Based on the comprehensive literature review,
Hypothesis 3 was formulated as follows:
H3 = Teacher mindset is positively and significantly related to the quality of DI

implementation.
Based on the assumptions made in this study, a conceptual framework was

developed, which can be seen in Figure 1.
Figure 1. Conceptual framework model
3.1 Study Design
The purpose of this research is to investigate the role of the teacher's mindset as a
mediating variable that connects teacher competence and the availability of digital
media to teachers' ability to implement digital-based differentiated learning. We
used a quantitative approach to collect the empirical data required for this study.
This method provides a structured and systematic approach to data collection,
analysis and interpretation, which has various advantages in the context of this
research. Simple random sampling was chosen as the best strategy for participant
selection because it allowed the researchers to reach individuals from the easily
438
accessible target population. Junior high school teachers in West Sumatra,

Indonesia, were the target group in this study. Data from participants were
collected using standardised questionnaires. The survey was divided into two
main parts: demographic data and factors influencing the use of e-learning
technology. Participants were asked to provide information about their age,
gender, their perception of competence in DI, the availability of digital media in
their respective schools, the implementation of DDI, and teachers' paradigms
regarding learning.
3.2 Procedures and Respondents

This research is part of a multi-year project funded by the Indonesian Ministry of
Education, Culture, Research and Technology. The chosen research area was West
Sumatra. West Sumatra was selected as the research area because it is one of the
provinces in Indonesia that is currently experiencing development. However, in
some parts of the region, there are still remote areas that lack adequate
educational facilities. This is the second-year research, which aims to identify
barriers and challenges in enhancing teachers' abilities to implement digital
differentiated learning models in West Sumatra, one of which is to emphasise the
role of teachers' mindsets in improving the quality of education. In this context,
the quality of education measured is digitally based differentiated learning. In this
study, a total of 161 teachers from both public and private junior high schools in
West Sumatra, Indonesia, were selected by means of simple randomisation
(Horton, 2022). The researchers ensured that the sample size used adhered to the
minimum requirement of 100 samples, as recommended by Hair et al. (2014), for
conducting SEM analysis.
3.3 Data Collection Process

The instrument used in this study was developed by the author. During the data
collection process, instruments were developed based on the theoretical
framework prepared previously. In the initial stage, permission was obtained
from the Governor of West Sumatra to conduct a multi-year study spanning three
years (2022-2024), with a focus on DI implementation in junior high schools in the
province. After obtaining the permission, data were collected through an online
Google Form, which was distributed to teachers in these schools during the period
from June to July 2023. Subsequently, the data collection instrument presented to
teachers consisted of positive statements pertaining to all the variables observed
in this study. Teachers were asked to express their perceptions using a 4-point
Likert Scale. On this scale, the scores 1, 2, 3 and 4 represented very high
disagreement, disagreement, agreement and very high agreement, respectively
(Bacon-Shone, 2015).
3.4 Data Analysis Process

In this study, the data analysis process was conducted using the SEM method with
the Smart PLS 4 software (Hair et al., 2017). SEM is a statistical technique used to
test and model the relationships between variables in a complex system. Using
SmartPLS software version 4.0, the relationships between teacher knowledge on
DI and the readiness of digital learning tools were effectively and easily identified.
This process was mediated by Teacher Growth Mindset, by exploring significant
indirect influences. Additionally, the analysis examined direct influences to
439
gather further evidence of the impact of these various factors (Hair et al., 2017).
During the analysis, two types of mediation were considered, namely, full and
partial mediations. Full mediation occurs when there is only a statistically
significant indirect influence, while partial mediation occurs when both the direct
influence on the dependent variable and the indirect influence through the
mediator is significant (Hair et al., 2017; Timothy, 2022).
Partial mediation can be further categorised into two namely complementary

partial mediation and competitive partial mediation. Complementary partial
mediation arises when both the indirect and direct influences are significant and
move in the same direction. Competitive partial mediation, on the other hand,
occurs when both influences are significant but move in opposite directions (Kono
& Sato, 2023).
The data analysis was carried out in two main stages including (1) calculating path
coefficients using the SmartPLS algorithm and (2) evaluating the significance of
path coefficients and mediation influences through the SmartPLS bootstrap
procedure. Following this, during the bootstrap process, 5000 samples were used,
following the recommendations of (Hair et al., 2017), to calculate coefficients, as
well as direct and indirect influences.
3.4.1 Measurement Model Evaluation

The measurement model in this study consisted of both reflective and formative
measurements. The variables of teacher knowledge on DI, availability of digital
learning facilities and teacher mindset were measured reflectively, while the
quality of DI implementation was assessed in a formative manner (Hanafiah,
2020).
In this study, the cross-loading table was employed as the initial step to assess the
presence of discriminant validity among the constructs. The SmartPLS 4 data
analysis method used to assess the outer model comprised four criteria, which
include average variance extracted (AVE), composite reliability (CR),
discriminant validity and convergent validity. A model is considered acceptable
when the factor loading of each item exceeds 0.7, CR surpasses 0.7, AVE is above
0.5, and the HTMT value is below 0.90 (Hair et al., 2017). The formative model
was evaluated based on the significance of the outer weightings and the absence
of multicollinearity among the items, which was evidenced through the VIF
values below 5. A comprehensive view of these construct values is presented in
Table 1.
Table 1. Outer loading, composite reliability and average variance extracted

Measurement Outer Cronbach's
Latent Variable rho_a rho_c AVE
item Loading alpha
Speed and
reliability of
Availability of
internet
digital learning 0.877 0.923 0.925 0.942 0.764
connection on
media in schools
the school
premises
440

item Loading alpha
In my school
has integrated 0.852
smart boards
Availability of
projectors or
interactive
0.841
screens for
digital
presentations
Accessibility of
devices to all
students,
0.911
including
those with
special needs
Availability of
digital learning
applications
that support 0.888
the teaching
and learning
process
I use
technology to
0.881 0.852 0.861 0.895 0.631
differentiate
instructions
I use a study
contract in my 0.818
classroom
I make a
variety of
digital-based 0.737
teaching
DI materials
implementation I allow
students in my
class to work
0.818
from higher
grade-level
textbooks
I group
students by
ability across
0.707
classes at the
same grade
level
I understand
Teacher about the
0.778 0.832 0.842 0.881 0.598
Knowledge on DI Student
Interest Centre
441

item Loading alpha
I can make a
variety of
digital-based 0.721
teaching
materials
I understand
the Student
Choice Terms:
[about content, 0.827
process,
and/or
product]
I can do graded
assessments in 0.781
class
I can design
different
lessons based 0.757
on student
interests
There are
many ways to
help students
achieve 0.867 0.909 0.915 0.936 0.786
optimal
learning
outcomes
I am ready to
adapt and
change my
earning
0.912
methods
according to
Teacher Mindset
the needs of
students
The abilities of
everyone will
grow through 0.879
various
practices
Student-centre
learning can
produce
0.888
quality
educational
output
Based on the results presented in Table 1, it is evident that the study instrument
is valid and reliable. This validity can be observed from the cross-loading values,
which exceeded 0.7. Additionally, the obtained Cronbach's alpha values were
above 0.7 (Brown, 2002; “Confiabilidad y Coeficiente Alpha de Cronbach,” 2010),
442
indicating good reliability. The convergent validity test results based on AVE
were compared to the predetermined threshold values. AVE measures the
amount of variance explained by the construct represented by its indicators.
Higher AVE values indicate better construct reliability (Hair et al., 2014).
Accordingly, from the results presented in Table 1, it can be seen that the AVE
values for all variables were above 0.5. This indicated that the instruments used
had satisfactory convergent validity. To further evaluate the measurement model,
this study also estimated the HTMT values, as presented in Table 2 below:
Table 2. HTMT
Availability of
Teacher
Digital Learning DI Teacher
Knowledge
Media in implementation Mindset
on DI
Schools
DI Implementation 0.510
Teacher
0.451 0.605
Knowledge on DI
Teacher Mindset 0.356 0.683 0.529
The discriminant validity measure HTMT was considered more sensitive and
accurate in detecting discriminant validity. This was because the HTMT values
for all variables fell within the recommended threshold, which was below 0.90
(Hair et al., 2017), as shown in Table 2.
4. Results
4.1 Structural Model Evaluation
The evaluation of the structural model is closely tied to hypothesis testing. The
inner or structural model serves as a framework for predicting cause-and-effect
relationships between latent variables or parameters that cannot be directly
measured. This structural model delineates the interplay of causal relationships
among latent variables, based on the underlying theoretical framework. In
evaluating the structural model, bootstrapping and blindfolding procedures were
employed in SmartPLS software.
During this evaluation process, several key components were examined,

including:
(1) Coefficient of determination (R-square): This value gauges the explanatory
power of endogenous constructs within the model. R-square values of 0.67
(strong), 0.33 (moderate), and 0.19 (weak) provide insights into the strength of
the relationship within the model.
(2) Path Coefficients Estimation: Path coefficients, representing the magnitude of

relationships between latent constructs, were also estimated. This estimation
was carried out through the bootstrapping procedure, which helped to
establish the significance and reliability of these coefficients.
(3) Effect Size (F Square): This metric was employed to evaluate the goodness of
the model.
443
Table 3. R-Square
R-square R-square adjusted
DI implementation 0.484 0.474
Teacher Mindset 0.243 0.234
From the results presented in Table 3, the relationship between the variables in
this study fell under a moderate category since the obtained values were between
0.19 and 0.67. An R-Square value of 0.48 was observed, indicating that the
independent variable was capable of explaining 48% of the variance in the
dependent variable. However, the remaining 52% could not be explained by the
independent variable and may be attributed to other factors beyond its scope
(error component).
Based on the hypothesis testing results (Table 4), the following findings were
observed:
1. The first hypothesis was accepted (H1), indicating a significant influence of
the availability of digital learning media in schools on DI implementation.
The obtained path coefficient was 0.223, with a p-value of 0.000. This showed
that an increase in the availability of digital learning media in schools led to
an improvement in the quality of the implementation process. Following this,
at the 95% confidence interval, the effect of digital learning media availability
on DI implementation was between 0.106 and 0.345. This finding highlighted
the importance of digital facilities in schools to support the achievement of
high-quality implementation of the digital learning model.
2. The second hypothesis was accepted (H2), emphasising that teacher
knowledge on DI exerted a significant influence on its implementation. The
obtained path coefficient for this variable was 0.231, with a p-value of 0.000.
These values indicated that an increase in teacher knowledge undoubtedly
increased the quality of the implementation process. Furthermore, at the 95%
confidence interval, the influence of teacher knowledge on the
implementation was between 0.075 and 0.378. These values underscored the
importance of enhancing teacher knowledge on DI to achieve high-quality
digital DI implementation.
3. The third hypothesis was accepted (H3), indicating that teacher mindset
significantly influenced DI implementation. The obtained path coefficient
was 0.429, with a p-value of 0.000. This means that every change in teacher
mindset led to an increase in the quality of the learning model
implementation process. Accordingly, at the 95% confidence interval, the
influence of teacher mindset on DI implementation was found to be between
0.309 and 0.543. This underscored the importance of fostering a growth
mindset among teachers regarding DI. The study proved that mindset has
the highest direct influence compared to other variables in achieving high-
quality implementation of the digital learning model.
444
Table 4. Hypothesis Testing
95% coefficient path

Original Sample analysis P values
Path sample (O) mean (M)
coefficients lower limit upper limit
Availability of digital learning media in schools -> DI

0.223 0.227 0.106 0.345 0
implementation 0.223
Availability of digital learning media in schools -> Teacher

0.171 0.173 0.03 0.316 0.021
Mindset 0.171
Teacher Knowledge on DI -> DI implementation 0.231 0.231 0.23 0.076 0.378 0
Teacher Knowledge on DI -> Teacher Mindset 0.399 0.399 0.405 0.241 0.56 0
Teacher Mindset -> DI implementation 0.429 0.429 0.43 0.309 0.543 0
©Authors
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0).
445
Table 5. Specific indirect influence

Specific indirect
influences
Teacher Knowledge on DI -> Teacher Mindset -> DI
implementation 0.171
Availability of digital learning media in schools -> Teacher
mindset -> DI implementation 0.073
Based on the indirect influences presented in Table 5, it can be concluded that:
1. The indirect influence of teacher knowledge on the implementation quality of

DI through teacher mindset was 0.171. This means that when teacher
knowledge on DI increased by one unit, the quality of DI implementation also
increased indirectly through teacher mindset by 17.1%, indicating a positive
influence.
2. The indirect influence of the availability of digital learning media in schools
on the implementation quality of DI through teacher mindset was 0.073. This
means that for every one-unit increase in the availability of digital learning
media in schools, the quality of DI implementation potentially increased
indirectly through teacher mindset by 7.3%, also indicating a positive
influence.
4.2 Standardised Root Mean Square Residual (SRMR)
SRMR is a measure that assesses the discrepancy between the observed and
implied correlations by the model. Although the use of PLS-SEM in the structural
analysis limits reporting on traditional model fit indices, it was useful in
formulating the report on the SRMR value. The results of the structural model,
which include the variance explained (R2) of teacher knowledge on DI and the
availability of digital learning media in schools, are presented in Table 6.
Table 6. SRMR
Saturated model Estimated model
SRMR 0.067 0.067
d_ULS 0.851 0.851
d_G 0.389 0.389
Chi-square 366.583 366.583
The model fit test is essential in determining if a model accurately fits the data,
and in the case of PLS models, SRMR value is used to assess fitness. A value of
SRMR less than 0.1 indicates a good fit, while a value below 0.08 is considered
perfect. In this study, the obtained SRMR value was 0.067, which fell below the
threshold of 0.1 and met the criteria for a perfect fit. This indicates that the
proposed model fits well and aligns closely with the estimated and empirical data
correlation matrices respectively.
The results of the model fit test are visually represented in Figure 2:
©Authors
446
Figure 2. Model of the relationship between variables
6. Discussion
The obtained results showed that teacher knowledge yielded a significant positive
influence on the implementation quality. This underscores the pivotal role of
teacher knowledge as a foundational element for the effective implementation of
the digital learning model. This finding aligns with previous research (Bozena
Górska-Porȩcka, 2013; Suarta et al., 2022) that highlighted the substantial impact
of teacher knowledge concerning instructional strategies on teaching quality.
Accordingly, it is important to comprehend the critical role of teacher knowledge
in achieving teaching goals (Tatto, 2021).
Based on the outcomes of the first hypothesis, it becomes imperative for

educational institutions and policymakers to prioritise the enhancement of
teacher knowledge on DI (Tometten et al., 2021). In this situation, comprehensive
training and professional development programmes for teachers should include
key aspects of DI, such as differentiation strategies, effective classroom
management in diverse settings, and the use of learning resources that cater to the
individual needs of students. Moreover, it is crucial to integrate DI knowledge
into teacher education curricula (Gottfried et al., 2020). Prospective teachers
should also be equipped with a solid understanding of DI concepts and strategies
during their academic tenure at colleges or teacher education institutions.
Additionally, experienced teachers should be actively involved in continuous
professional development programmes in order to obtain a deeper understanding
of DI concepts and enhanced implementation proficiencies within classroom
settings.
The second hypothesis aimed to examine the influence of the availability of digital
learning media in schools on DI implementation. The results showed a significant
and positive correlation between the two observed variables under this
hypothesis. The result was consistent with several previous studies that showed
the direct influence of digital devices on the execution of digital learning (Antara
& Dewantara, 2022; Neofotistos & Karavakou, 2018). Based on the outcomes, it
447
was concluded that investing in digital infrastructure and devices in school

environments enhances the quality and effectiveness of DI. However, it is
important to acknowledge the potential limitations within this study.
The third hypothesis shed light on the noteworthy discovery that teacher
knowledge on DI and the availability of digital learning media in schools gained
enhanced positive influence when synergistically supported by growth of teacher
mindset. This observation corroborated the results of several studies, which
underscored the pivotal role played by teacher mindset as a patent mediating
factor in shaping the willingness and ability of teachers to adopt and implement
DI principle (Mesler et al., 2021). The results are expected to offer profound
insights into the importance of integrating psychological aspects and teacher
beliefs into the implementation of DI principle and the effective use of educational
technology. The deficiencies in the availability of digital infrastructure can be
effectively mitigated through the instillation of a growth-oriented perspective
among educators. However, it is important to acknowledge that the results can be
influenced by various other factors, such as the transformational leadership skills
of the school principal (Kartiwi, 2020; Sari & Rina Novianty Ariawaty, 2019), the
broader school context (Planson, 2022), administrative support (Araneta et al.,
2020; Özdemir, 2020; Wu et al., 2023), and student characteristics (Boelens et al.,
2018; Széll, 2013). Based on this finding, it can be concluded that further
comprehensive studies and in-depth analysis are essential to gain a more
profound understanding and validation of the intricate relationships between the
three aforementioned variables.
7. Conclusion
In conclusion, the significance of teacher knowledge on DI and the availability of
digital learning media within schools was heightened when underpinned by
"growth of teacher mindset." The results underscored the pivotal role played by
the beliefs exhibited by teachers, such as the ability to continuously grow and
develop, in influencing their willingness and capability to adopt and implement
DI principles.
The results also strengthened the robustness of previous studies that highlighted
the crucial role of teacher mindset as a strong mediator in shaping the extent to
which teachers were willing to and capable of implementing different teaching
approaches to meet individual student needs (DI principle). In other words,
teachers with a growth mindset experienced greater receptivity to DI, a
heightened willingness to explore and adopt various teaching methods, and were
better equipped to face challenges and difficulties that may arise during the DI
implementation process. Furthermore, this synergistic relationship bolstered the
nexus between knowledge on DI and the availability of digital learning media
within educational institutions.
8. Recommendation
Based on the results from this study, it is recommended that teachers and
stakeholders in the educational sector pay attention to the significance of
developing a growth mindset. This strategic emphasis on nurturing a growth-
oriented perspective is instrumental in bolstering the efficacy of DI
448
implementation and fostering the effective use of digital technology within the
pedagogical process.
9. References
Aas, H. K. (2023). Professional development for inclusive and adaptive education:
Lesson Study in a Norwegian context. Professional Development in Education, 49(3).
https://doi.org/10.1080/19415257.2020.1850509
Aminuriyah, S., Markhamah, & Sutama. (2022). Pembelajaran Berdifferensiasi :
Meningkatkan Kreatifitas Peserta Didik. Jurnal Mitra Swara Ganesha, 9(2).
Anderson, D. L., Graham, A. P., Simmons, C., & Thomas, N. P. (2022). Positive links
between student participation, recognition and wellbeing at school. International
Journal of Educational Research, 111. https://doi.org/10.1016/j.ijer.2021.101896
Antara, I. G. W. S., & Dewantara, K. A. K. (2022). E-scrapbook: The needs of HOTS
oriented digital learning media in elementary schools. Journal for Lesson and
Learning Studies, 5(1). https://doi.org/10.23887/jlls.v5i1.48533
Araneta, M. L. R., Catalan, R. D., & Martir, E. M. (2020). Administrative support and
teaching quality: Inputs to professional development program. Journal of Higher
Education Theory and Practice, 20(6). https://doi.org/10.33423/jhetp.v20i6.3134
Bacon-Shone, J. (2015). Introduction to quantitative research methods pres. In
Loughborough Univeristy (Issue February).
Bi, M., Struyven, K., & Zhu, C. (2023). Variables that influence teachers’ practice of
differentiated instruction in Chinese classrooms: A study from teachers’
perspectives. Frontiers in Psychology, 14.
https://doi.org/10.3389/fpsyg.2023.1124259
Boelens, R., Voet, M., & De Wever, B. (2018). The design of blended learning in response
to student diversity in higher education: Instructors’ views and use of
differentiated instruction in blended learning. Computers and Education, 120.
Bozena Górska-Porȩcka, B. (2013). The role of teacher knowledge in esp course design.
Studies in Logic, Grammar and Rhetoric, 34(47). https://doi.org/10.2478/slgr-2013-
0021
Brown, J. D. (2002). The Cronbach alpha reliability estimate. Shiken: JALT Testing &
Evaluation SIG Newsletter, 6(1).
Caldera, S., Mostafa, S., Desha, C., & Mohamed, S. (2021). Exploring the role of digital
infrastructure asset management tools for resilient linear infrastructure outcomes
in cities and towns: A systematic literature review. In Sustainability (Switzerland)
(Vol. 13, Issue 21). https://doi.org/10.3390/su132111965
Caniëls, M. C. J., Adamska, K. M., van Oortmerssen, L. A., & van Assen, M. F. (2023). Set
your mind on it: The mediating role of mindset in the relationship between a
learning-from-error climate and work-related flow. Current Psychology, 42(3), 2067–
2076. https://doi.org/10.1007/s12144-021-01572-3
Cénat, J. M. (2020). Globalization, internet and psychiatric disorders: Call for research
and action in global mental health. Neurology Psychiatry and Brain Research, 36.
https://doi.org/10.1016/j.npbr.2020.02.007
Confiabilidad y coeficiente Alpha de Cronbach. (2010). Telos. Revista de Estudios
Interdisciplinarios En Ciencias Sociales, 12(2).
Et.al, A. P. (2021). The Influence of transformational leadership, job satisfaction,
motivation and compensation on mathematics school teacherperformance. Turkish
Journal of Computer and Mathematics Education (TURCOMAT), 12(3).
https://doi.org/10.17762/turcomat.v12i3.1649
449
Frondozo, C. E., King, R. B., Nalipay, M. J. N., & Mordeno, I. G. (2022). Mindsets matter
for teachers, too: Growth mindset about teaching ability predicts teachers’
enjoyment and engagement. Current Psychology, 41(8).
https://doi.org/10.1007/s12144-020-01008-4
Garrote, A., Felder, F., Krähenmann, H., Schnepel, S., Sermier Dessemontet, R., & Moser
Opitz, E. (2020). Social acceptance in inclusive classrooms: The role of teacher
attitudes toward inclusion and classroom management. Frontiers in Education, 5.
Ghazi, M. I. Al, Paidi, P., & Wibowo, Y. (2022). Developing augmented reality of virus as
learning media. Bioedutika, 10(3), 10–18.
http://journal.uad.ac.id/index.php/BIOEDUKATIKA/article/view/24035
Ghazi, M. I. Al, Qomariyah, N., Subiantoro, A. W., & Ariyanti, N. A. (2021). The
Utilization of Rusip’s local culture as a distanced learning media in biological
lessons. Asian Journal of Advanced Research and Reports.
https://doi.org/10.9734/ajarr/2021/v15i330384
Ginja, T. G., & Chen, X. (2020). Teacher educators’ perspectives and experiences towards
differentiated instruction. International Journal of Instruction, 13(4), 781–798.
https://doi.org/10.29333/iji.2020.13448a
Gottfried, M. A., Kirksey, J. J., & Hutt, E. (2020). How can preservice teaching programs
help new teachers feel prepared to address absenteeism? Teachers College Record,
122(5). https://doi.org/10.1177/016146812012200507
Hair, J. F., Ringle, C. M., & Sarstedt, M. (2014). PLS-SEM: Indeed a silver bullet: Journal
of Marketing Theory and Practice, 19(2).
Hair, J F, M., G. T., Hult, M., C., Ringle, M., & Sarstedt. (2017). A Primer on partial least
squares structural equation modeling (PLS-SEM). T+ SAGE.
Hanafi, M. S., & Setiyani, H. (2021). The role of teachers’ competence to achieve effective
learning management in primary schools. Ilkogretim Online, 20(1).
Hanafiah, M. H. (2020). Formative vs. reflective measurement model: guidelines for
structural equation modeling research. International Journal of Analysis and
Applications. https://doi.org/10.28924/2291-8639-18-2020-876
Hasanah, E., Badar, M. I. Al, & Ghazy, M. I. Al. (2022). Factors That Drive the Choice of
Schools for Children in Middle- Class Muslim Families in Indonesia : A Qualitative Study.
27(5), 1393–1409.
Hasanah, E., Suyatno, S., Maryani, I., Badar, M. I. Al, Fitria, Y., & Patmasari, L. (2022).
Conceptual Model of differentiated-instruction (DI) Based on teachers’ experiences
in Indonesia. Education Sciences, 12(10), 650.
Hoogeboom, M. A. M. G., & Wilderom, C. P. M. (2019). Advancing the transformational–
transactional model of effective leadership: Integrating two classic leadership
models with a video-based method. Journal of Leadership Studies, 13(2).
https://doi.org/10.1002/jls.21655
Horton, M. (2022). Simple random sample: Advantages and disadvantages. Investopedia.
Jankelová, N., Joniaková, Z., Procházková, K., & Blštáková, J. (2020). Diversity
management as a tool for sustainable development of health care facilities.
Sustainability (Switzerland), 12(13). https://doi.org/10.3390/su12135226
Kartiwi, A. P. (2020). Principal transformational leadership in high school of central
kalimantan. International Journal of Educational Management and Innovation.
https://doi.org/10.12928/ijemi.v1i1.1514
Kaya, S., & Yuksel, D. (2022). Teacher mindset and grit: How do they change by teacher
training, gender, and subject taught? Participatory Educational Research, 9(6).
https://doi.org/10.17275/per.22.146.9.6
450
Killpack, T. L., & Melón, L. C. (2016). Toward inclusive STEM classrooms: What personal
role do faculty play? CBE Life Sciences Education, 15(3).
https://doi.org/10.1187/cbe.16-01-0020
Kinyanjui, M. W., Aloka, P. J. O., Mutisya, S. K., Ndeke, F. N., & Nyang’ara, N. M. (2015).
Classroom instruction reinforcement strategies and factors that influence their
implementation in Kenyan Primary Schools. Journal of Educational and Social
Research. https://doi.org/10.5901/jesr.2015.v5n3p267
Kono, S., & Sato, M. (2023). The potentials of partial least squares structural equation
modeling (PLS-SEM) in leisure research. Journal of Leisure Research, 54(3).
https://doi.org/10.1080/00222216.2022.2066492
Lavania, M., & Mohamad Nor, F. (2021). Factors Influencing the implementation of
differentiated instruction in English language instruction in rural and urban
secondary schools of Johor Bahru. Creative Education, 12(06), 1235–1246.
https://doi.org/10.4236/ce.2021.126093
Lindsay, B. L., Bernier, E., Boman, J., & Boyce, M. A. (2023). Understanding the
connection between student wellbeing and teaching and learning at a Canadian
research university: A qualitative student perspective. Pedagogy in Health Promotion,
9(1). https://doi.org/10.1177/23733799221089578
Mahoney, J., & Hall, C. (2017). Using technology to differentiate and accommodate
students with disabilities. E-Learning and Digital Media, 14(5), 291–303.
https://doi.org/10.1177/2042753017751517
Manik, E. (2016). The influence of transformational leadership on achievement
motivation and organizational climate and employee performance. International
Journal of Academic Research in Business and Social Sciences, 6(12).
https://doi.org/10.6007/ijarbss/v6-i12/2522
Martanti, F., Widodo, J., Rusdarti, R., & Priyanto, A. S. (2022). Penguatan Profil Pelajar
Pancasila Melalui Pembelajaran Diferensiasi Pada Mata Pelajaran IPS di Sekolah
Penggerak. Prosiding Seminar Nasional Pascasarjana (PROSNAMPAS), 5(1).
McCarthy, C. G. P., Mulhair, P. O., Siu-Ting, K., Creevey, C. J., & O’Connell, M. J. (2023).
Improving orthologous signal and model fit in datasets addressing the root of the
animal phylogeny. Molecular Biology and Evolution, 40(1).
https://doi.org/10.1093/molbev/msac276
Mesler, R. M. D., Corbin, C. M., & Martin, B. H. (2021). Teacher mindset is associated
with development of students’ growth mindset. Journal of Applied Developmental
Psychology, 76. https://doi.org/10.1016/j.appdev.2021.101299
Mishra, L., Gupta, T., & Shree, A. (2020). Guiding principles and practices of peace
education followed in secondary schools of mizoram. International Journal of
Evaluation and Research in Education, 9(4). https://doi.org/10.11591/ijere.v9i4.20738
Moulin-Stożek, D. (2019). The social construction of character. Journal for the Theory of
Social Behaviour, 49(1). https://doi.org/10.1111/jtsb.12188
Naziev, A. (2017). What is an education. International Conference: The Future of Education,
1(6).
Neofotistos, V., & Karavakou, V. (2018). Factors influencing the use of ICT in Greek
primary education. Open Journal for Educational Research, 2(2).
https://doi.org/10.32591/coas.ojer.0202.02073n
Ng, B. (2018). The neuroscience of growth mindset and intrinsic motivation. Brain
Sciences, 8(2). https://doi.org/10.3390/brainsci8020020
Nosova, S. S., Askerov, P. F., Rabadanov, E. R., Dubanevich, L. E., & Voronina, V. N.
(2019). The role of digital infrastructure in the digital transformation of the modern
Russian economy. International Journal of Innovative Technology and Exploring
Engineering, 8(7).
451
Øvrelid, E., & Bygstad, B. (2019). The role of discourse in transforming digital
infrastructures. Journal of Information Technology, 34(3).
https://doi.org/10.1177/0268396219831994
Özdemir, G. (2020). The mediating role of perceived administrative support for the effect
of job motivation on organizational identification. International Journal of Evaluation
and Research in Education, 9(3). https://doi.org/10.11591/ijere.v9i3.20440
Planson, S. (2022). Race, cultural capital, and school achievement in race-blind France.
Sociology of Education. https://doi.org/10.1177/00380407221139220
Pozas, M., Letzel, V., Lindner, K. T., & Schwab, S. (2021). DI (differentiated instruction)
does matter! The effects of DI on secondary school students’ well-being, social
inclusion and academic self-concept. Frontiers in Education, 6.
Prakosha, D., Kristiyanto, A., Gunarhadi, G., Salim, A., & Sunardi, S. (2018). Perception
of physical education teachers on adaptive physical education in inclusive school.
International Journal of Pedagogy and Teacher Education, 2(1).
https://doi.org/10.20961/ijpte.v2i1.17042
Ruaya, P. P., Kang, H. X., Reader, S., & Hidayat, T. (2022). Role of teacher competence to
implement the independent curriculum. International Journal of Science Education and
Cultural Studies, 1(2). https://doi.org/10.58291/ijsecs.v1i2.48
Salmela, H., Baiyere, A., Tapanainen, T., & Galliers, R. D. (2022). Digital agility:
Conceptualizing agility for the digital era. Journal of the Association for Information
Systems, 23(5). https://doi.org/10.17705/1jais.00767
Sari, N. Z. M., & Rina Novianty Ariawaty, R. (2019). The effect tranformational
leadhership to quality information systems and its impact on performance aparatur
civil asn west java indonesia. International Journal of Scientific and Technology
Research, 8(10).
Sewagegn, A. A. (2020). Learning objective and assessment linkage: Its contribution to
meaningful student learning. Universal Journal of Educational Research, 8(11).
https://doi.org/10.13189/ujer.2020.081104
Smale-Jacobse, A. E., Meijer, A., Helms-Lorenz, M., & Maulana, R. (2019). Differentiated
instruction in secondary education: A systematic review of research evidence. In
Frontiers in Psychology (Vol. 10). https://doi.org/10.3389/fpsyg.2019.02366
Stollman, S. H. M. (2018). Differentiated instruction in practice: a teacher perspective.
https://scholarlypublications.universiteitleiden.nl/handle/1887/62049%0Ahttps:/
/scholarlypublications.universiteitleiden.nl/access/item%3A2952091/view
Suarta, I. M., Noortyani, R., Yarsama, K., & Adhiti, I. A. I. (2022). The role of Teachers’
indigenous knowledge and cultural competencies in enhancing students’
engagement and learning outcomes. Journal of Ethnic and Cultural Studies, 9(1).
https://doi.org/10.29333/ejecs/1025
Supardi, S., & Hasanah, E. (2020). Junior high school students’ experiences of high
technology based learning in Indonesia. International Journal of Learning, Teaching
and Educational Research, 19(5). https://doi.org/10.26803/ijlter.19.5.9
Svetlichnaya, N. K. (2022). Development of inclusive education in adaptive physical
education of children. Tambov University Review. Series: Humanities, 3.
https://doi.org/10.20310/1810-0201-2022-27-3-705-713
Széll, K. (2013). Factors determining student achievement. Hungarian Educational
Researcch Journal. https://doi.org/10.14413/herj.2013.03.06.
Tan, D., & Maeda, Y. (2021). Perceptions of science teachers’ growth-mindset practices
and U.S. high school students’ initial science identity and its development.
International Journal of Science Education, 43(13).
https://doi.org/10.1080/09500693.2021.1958021
452
Tatto, M. T. (2021). Professionalism in teaching and the role of teacher education.

European Journal of Teacher Education, 44(1).
https://doi.org/10.1080/02619768.2020.1849130
Timotheou, S., Miliou, O., Dimitriadis, Y., Sobrino, S. V., Giannoutsou, N., Cachia, R.,
Monés, A. M., & Ioannou, A. (2023). Impacts of digital technologies on education
and factors influencing schools’ digital capacity and transformation: A literature
review. In Education and Information Technologies (Vol. 28, Issue 6). Springer US.
https://doi.org/10.1007/s10639-022-11431-8
Timothy, V. L. (2022). The effect of top managers’ human capital on SME productivity:
the mediating role of innovation. Heliyon, 8(4), e09330.
https://doi.org/10.1016/j.heliyon.2022.e09330
Tometten, L., Heyder, A., & Steinmayr, R. (2021). Links between teachers’ knowledge
about special educational needs and students’ social participation and academic
achievement in mainstream classes. Contemporary Educational Psychology, 67.
https://doi.org/10.1016/j.cedpsych.2021.102022
Tomlinson, C. A. (2001). How to differentiate instruction in mixed-ability classrooms. In
Association for Supervision and Curriculum Development (2nd ed.). Association for
Supervision and Curriculum Development Alexandria..
Tomlinson, C. A., Brighton, C., Hertberg, H., Callahan, C. M., Moon, T. R., Brimijoin, K.,
Conover, L. A., & Reynolds, T. (2003). Differentiating instruction in response to
student readiness, interest, and learning profile in academically diverse classrooms:
A review of literature. In Journal for the Education of the Gifted (Vol. 27, Issues 2–3).
https://doi.org/10.1177/016235320302700203
Widyanti, R. D. (2018). Wage inequality and return to education in Indonesia: Quantile
Regression analysis. Signifikan: Jurnal Ilmu Ekonomi, 7(1).
https://doi.org/10.15408/sjie.v7i1.6071
Wu, C. H., Kuo, P. L., Yang, C. H., Chang, Y. C., & Chen, T. L. (2023). Importance–
performance analysis (IPA) in analyzing the satisfaction of administrative support
in teaching practice research programs. Sustainability (Switzerland), 15(3).
https://doi.org/10.3390/su15031943
Yilmaz, E. (2022). Development of mindset theory scale (Growth and fixed mindset): A
validity and reliability study (Turkish Version). Research on Education and
Psychology, 6(Special Issue), 1–26. https://doi.org/10.54535/rep.1054235
Zabeli, N., Perolli Shehu, B., & Anderson, J. A. (2021). The understanding of inclusive
education in kosovo: Legal and empirical argumentation. Center for Educational
Policy Studies Journal, 11(3). https://doi.org/10.26529/cepsj.692
Zadworna, M., Kossakowska, K., & Renshaw, T. L. (2023). Measuring Subjective
wellbeing in a school context: A Polish version of the student subjective wellbeing
questionnaire. School Mental Health, 15(1). https://doi.org/10.1007/s12310-022-
09546-x
1

Vol. 22, No. 10, pp. 453-474, October 2023
Applying Technical Science Process in Teaching

STEM Topics under Model School with
Production Practice and Social Life in Vietnam
Dang Thi Thuan An
University of Education – Hue University, Vietnam
Huynh Van Son

Huynh Van Son, HCMC University of Education, Vietnam
Pham Ngoc Son

Hanoi Metropolitan University, Vietnam
Nguyen Mau Duc*

Hanoi National University of Education, Vietnam
Abstract. Currently, STEM (science, technology, engineering, and

mathematics) teaching that applies scientific and technical processes is
an approach to effectively implementing education in Vietnam. This
paper describes the application of scientific and technical processes in
teaching STEM topics in a school model associated with production,
business, and social life in Vietnam. This experimental study elicited
feedback by experts and high school chemistry teachers. Data were
collected through expert evaluation forms and questionnaires answered
with reference to a Likert scale, and the results were analyzed using
quantitative analysis. The research proposed STEM topics associated
with production practice and social life for teaching physics, chemistry,
and biology in Vietnam. Through the proposed process, we designed
and organized 33 STEM education topics associated with production
and social life in teaching physics, chemistry and biology. Research
results show that teaching STEM topics associated with production
practices and social life has clear advantages. Teachers' knowledge and
use of STEM topics was found to be at a high level. The study confirmed
the effectiveness of applying scientific processes in teaching STEM
topics and creating connections between schools and production
practices.
Keywords: STEM topic; process for teaching STEM; students;

production practice; social life
*
Corresponding author: Nguyen Mau Duc; nmduc@hnue.edu.vn
2
1. Introduction
Vietnam is currently in the process of implementing a new educational
curriculum. The application of scientific and technical processes in STEM
(science, technology, engineering, and mathematics) education has become a
focal point of concern and implementation in recent years. One promising
approach to integrating a model school system with production practice and
social life in Vietnam is through the application of the technical science process
in STEM teaching.
STEM education encompasses scientific inquiry, technology, engineering

design, mathematical analysis. It is interdisciplinary topics that align with
21st-century education standards (Johnson, 2013). Teachers understanding
these four STEM fields is essential to successfully incorporate and integrate
them into 21st-century teaching and learning (Baharin et al., 2018). The STEM
themes chosen for the 2018 General Education Curriculum in Vietnam foster
students' interdisciplinary, and they have generated interest in and support
for STEM education (Huy et al., 2023). STEM education is a multifaceted
discipline that is designed to equip students with a robust academic
foundation in science, technology, engineering, and mathematics (Bybee,
2015). Vietnam's current education strategy emphasizes the importance of
experiential learning by letting students use what they have learned in real -
world situations. Doing so promotes students' ability to think critically,
analyze facts, and solve problems. STEM education also places a strong focus
on teamwork, and encourages participation in projects that call for a variety of
skill sets (Hieu et al., 2020).
One effective approach to implementing STEM education in Vietnam is the

application of technical scientific processes at model schools; the processes
emphasize production practice (Hieu et al., 2020). This approach prioritizes
experiential learning and the application of acquired knowledge in real -world
scenarios. It involves the seamless integration of STEM education into the
curriculum, and focuses on addressing real-life challenges and enhancing
students' logical reasoning abilities across diverse domains, in line with the
2013 curriculum (Nugroho et al., 2021).
In Vietnamese high schools, STEM lessons are structured around an

engineering design process consisting of eight steps: Identifying the problem;
researching background knowledge; proposing solutions; selecting a solution;
creating a model; testing and evaluating; sharing and discussing; and
adjusting the design (Ministry of Education and Training, 2020).
Implementing STEM lessons through practical topics helps students develop
and enhance their chemical skills and improve their ability to apply
knowledge in real lifeClick or tap here to enter text. Students actively engage in
conducting experiments, such as producing acid-base indicators from purple
cabbage or creating a mixture of phosphorus and potassium from straws and
waste animal bones, which fosters practical experimentation skills (Duc et al.,
2019). Model schools that emphasize production practice and social life are
educational institutions that are designed to provide practical learning
3
experiences that are relevant to the local community. This educational model
enables students to apply their knowledge in authentic, real-world contexts,
thereby fostering critical thinking and collaborative skills. Importantly, this
approach offers a pragmatic solution to resource limitations, by encouraging
students to learn through hands-on experiences and reducing their
dependence on expensive equipment or materials (Linh, 2020).
There is a growing awareness of the significance of STEM education for

equipping students for the future. Nevertheless, Vietnam faces impediments
when it comes to the effective implementation of STEM education. These
challenges impose limitations on resources and teacher training, and indicate
the necessity for culturally relevant pedagogical approaches.
The successful integration of STEM education in Vietnam faces various

challenges, of which resource limitations emerge as a prominent concern. Th e
nation grapples with an insufficient supply of essential equipment and
materials that are essential for practical learning encounters. Additionally, a
significant portion of educators lacks the requisite qualifications to
proficiently instruct STEM subjects, which affects students’ level of
engagement and interest. Initiatives have been launched to provide teacher
training programs that are focused on STEM education (Duc et al., 2019).
Cultural relevance is another significant challenge facing the effective
implementation of STEM education in Vietnam. Developing an education
paradigm that accounts for cultural disparities and offers culturally congruent
learning experiences is crucial (Tho, 2016; Tu et al., 2022).
In response to these challenges, employing technical scientific processes in the

domain of STEM education hold promise, particularly at model schools that
engage in production practice and social life. Nevertheless, a comprehensive
understanding of how this approach can tackle these challenges proficiently
necessitates further scrutiny and elucidation. This paper delves into the realm
of STEM education and assesses the efficacy of implementing the technical
scientific process in the instruction of STEM subjects in the context of model
schools that intertwine production practice and social life in Vietnam. To this
end, the paper addresses the following research questions:
1) How effective is the application of the science process for teaching STEM
topics in a model school system characterised by production practice and
social life in Vietnam?
2) What do expert say about STEM topics related to production practice and
social life as designed by the research team?
Applying the the science process in teaching STEM topics under a model
school system characterised by production practice and social life can be
highly effective. The conceptual framework of STEM education suggests that
retaining the principles of STEM in engineering activities can improve the
practice of STEM education (Yata et al., 2020). Several studies highlight the
importance of integrating STEM education into the curriculum and providing
4
opportunities for teachers to enhance their ability to teach STEM content

effectively (Margot & Kettler, 2019; Shernoff et al., 2017). To promote lifelong
participation in STEM learning, STEM education should incorporate real-
world socialissues (Kurup et al., 2019). Research has shown that out-of-school
time science activities can significantly affect students' career interest in STEM
fields (Dabney et al., 2012). Additionally, gendered motivational processes can
affect high school students' participation, aspirations, and career plans in
mathematics and STEM (Gilchrist Watt et al., 2012). To promote equal
participation and interest in STEM subjects, addressing these motivational
factors is essential.
Integrating STEM education with sustainable development was explored by

Buturlina (Buturlina et al., 2021) who provide foundational principles for
future initiatives. Lee et al. (2014) developed an integrative STEM education
model based on the science inquiry process. Luong and Dam (2021) developed
a Grade 5 science program that included a teaching process for topics in
material and energy, oriented towards STEM education with the aim of
contributing to the improvement of science teaching in elementary schools
(Luong & Dam, 2021). Stohlmann (Stohlmann et al., 2012) suggest that using a
model is a good starting point for teachers who wish to implement and
enhance integrated STEM education. Sujarwanto (Sujarwanto et al., 2021)
applied the literature review method to develop a conceptual framework of
STEM education based on the Indonesian curriculum. Huy et al. (2023)
employed the 6E teaching model, which emphasizes technical design and
practice, to design a teaching process for STEM-oriented education. An and
Yang (2019) propose the 5E teaching process for STEM education, which
involves engaging, exploring, explaining, engineering and evaluating (An &
Yang, 2019). Khmelnikova and Maslak (2022) observed the didactic
possibilities of a STEM-oriented approach to teaching chemistry disciplines
(Khmelnikova & Maslak, 2022). They highlight its potential for developing a
holistic scientific outlook, innovative thinking, research and analytical skills
and creativity in students, the development of their STEM competence, the
implementation of innovative and experimental activities, and integration of
knowledge from natural sciences, technologies, engineering, and mathematics.
In Vietnam, there is an increasingly recognized imperative regarding the

significance of STEM education as a preparatory tool for students in
anticipation of their future roles. Nevertheless, the effective implementation of
STEM education in Vietnam faces several notable challenges, including
constraints related to limited resources, teacher training inadequacies, and a
demand for an educationally relevant approach that is rooted in cultural
considerations. A promising strategy to surmount these challenges involves
the application of the technical scientific process within a framework of a
model school that integrates production practice and social engagement (Le et
al., 2021; Nguyen et al., 2019).
This approach emphasizes experiential learning opportunities that afford

students the capacity to apply their scientific knowledge in tangible, real-
5
world contexts. By incorporating production practice and social life into the
educational model, students are exposed to authentic scenarios, thereby
cultivating their critical thinking acumen. Furthermore, this education model
actively fosters collaboration among students, thereby encouraging teamwork
and the utilization of a diverse array of skill sets in the context of project-
based learning. The model school featuring production practice and social life
endeavors to provide students with practical learning experiences that are
deeply intertwined with the fabric of their local community, thereby
facilitating a bridge between their academic pursuits and the tangible realities
of the world they inhabit.
The successful implementation of STEM education in Vietnam faces a series of

notable challenges. A central issue is the constraints imposed by limited
resources, whereby numerous educational institutions lack the essential
equipment and materials requisite for facilitating hands-on learning
experiences (Nguyen, 2019). Furthermore, a scarcity of proficient educators
who would be capable of effectively instructing STEM subjects constitutes an
additional formidable obstacle (Hieu et al., 2020; Le et al., 2021). In response to
these predicaments, initiatives have been introduced to institute teacher
training programs that are specifically tailored to the demands of STEM
education. These programs are designed to augment teachers’ pedagogical
proficiency and content knowledge and ultimately enhancing their capacity to
present STEM subjects effectively.
The application of the science process in teaching STEM topics under a model
school system with production practice and social life can be highly effective.
It is important to integrate STEM education into the curriculum, provide
professional development opportunities for teachers, and make STEM subjects
relevant and valuable to students' lives. By addressing motivational factors,
promoting equal participation, and incorporating interdisciplinary
approaches, STEM education can be enhanced and students' interest and
achievement in STEM can be improved.
Our research examines the effectiveness of applying the scientific and

technical process to teaching STEM topics in schools in Vietnam. Furthermore,
we propose some STEM education topics related to social life practices that
can apply the scientific and technical process in teaching.
3. Methodology
The research process is carried out according to the following stages:
Phase 1: Research the scientific and technical process for teaching STEM topics.
The process of teaching STEM topics associated with production practice and social life
According to the interdisciplinary approach, STEM education aims to apply

knowledge from various STEM disciplines to solving real-world problems.
Therefore, STEM education is directed to promoting active participation and
6
experiential learning, facilitating exploration and discovery, doing hands-on

experimentation, and pursuing result-oriented achievements. These
distinguishing features highlight the fundamental underpinning of STEM
educational activities, which are rooted in the STEM cycle, scientific
methodologies, and engineering design processes. The scientific method is
centered on a quest for knowledge, while engineering design is aimed at the
application of scientific knowledge to address real-world issues. Simultaneously,
the STEM cycle underscores the interrelatedness and amalgamation of STEM
disciplines.
Identify problem
Research background knowledge
Math Physics Chemistry Biology Informatics Technology

(The content of teaching according to the program is rearranged accordingly)
Propose solutions/design
Select solutions/design
Model making (prototype)
Test and evaluate
Exchange and discuss
Adjust the design
Figure 1. The process of teaching STEM
Activity 1: Identify the problem

In this activity, the teacher presents four activities for students to engage in
STEM learning. The first activity involves assigning students a problem with
specific criteria, which requires them to use new knowledge to propose and
develop solutions, and make a model. The goal is to define product criteria and
detect problems/demands, which help students understand practical situations,
7
technological processes, and the structural and operational principles of

technological equipment.
The content of this activity focuses on learning about phenomena, products, and
technology, and assessing these phenomena and products. Students are
expected to complete levels of content completion through practice, and reading
documents and reports, watching videos, engaging in discussions, and problem
detection/statement. Teachers need to anticipate the completion levels of this
product to plan appropriate solutions.
Activity 2: Study background knowledge and propose solutions

In this activity, students carry out active, self-reliant learning activities under the
guidance of teachers, and focus on developing knowledge and skills according
to high school standards. This activity includes studying textbooks, reports and
supplementary materials, doing experiments, and taking part in discussions.
Teachers operate, finalize new knowledge, and support students in proposing
solutions/prototypes.
Activity 3: Select solutions

In this activity, students present, explain, and defend their viewpoints using
new and learned knowledge. For STEM topics that require students to research
and answer scientific questions, students must propose and implement a
problem-solving solution, complete the product according to the task, and
present their chosen design.
For STEM topics that require students to complete a technical product, students
must propose solutions, choose a feasible solution, design a prototype, test and
evaluate it, and finalize their designs (publish products). The expected student
product is the solution and selected design.
Teachers have to anticipate levels of scientific hypotheses and problems the

students will solve, which are based on the experimental plans that each student
had prepared. It is important that teachers help the students identify the best
plan to solve each problem. They should organize student exchanges and
discussions to select the appropriate direction, present the final product, and
then evaluate, comment, and improve it.
Activity 4: Model making, testing and evaluation

In this activity, students create a model according to the design completed after
step 3During manufacturing, they test and adjust the original design to ensure
the prototype is feasible. The goal is to manufacture and test the model, select
experimental instruments/equipment, and support students in the
implementation process.
These activities provide students with opportunities to explore and develop new
knowledge and skills through STEM learning. By organizing tasks, providing
guidance, and fostering a collaborative environment, students will be better
equipped to tackle complex challenges and achieve their goals in STEM
education.
8
Activity 5: Exchange, discuss and adjust

In this activity, students present their research products, evaluate them, and
presente them for further adjustments. The goal is to present, exchange, discuss,
and evaluate the final product. Teachers assign tasks, while students report and
discuss their findings in appropriate forms, and teachers evaluate, conclude, and
provide feedback for further improvement.
Phase 2: Build a system of STEM teaching topics based on a school model

associated with production practices and social life in Vietnam.
The basis for building a system of STEM topics according to the school model associated
with production practice and social life
Principle 1: The content of STEM lessons is linked to practical issues of social life,
science and technology. Students are required to find solutions for problems,
acquire knowledge, and meet the requirements of the lesson.
Principle 2: STEM lessons are built around the engineering design process
comprising eight steps: problem definition; background research; proposing
solutions; selecting solution; making models (prototypes); testing and
evaluating; sharing discussion; design modification.
Principle 3: Teaching methods involve students in activities of inquiry and

discovery, and are action-oriented according to the scientific and technical
process.
Principle 4: It is crucial for STEM topics to be tailored to the cognitive level of

students and to captivate their interest and attention.
Principle 5: In Vietnam, it is essential to develop abundant resources on STEM

topics, and these subjects must be compatible with the physical infrastructure
conditions and teaching needs of schools and connected to the practical
realities of social life .This approach creates opportunities for students to explore
and utilize these resources while they generate new ideas and create meaningful
products that are connected to production practices and social life.
Phase 3: Scientific and technical processes are applied in teaching STEM topics
according to the school model associated with production practices and social
life.
Phase 4: Pedagogical experiments and experimental data are processed to

evaluate feasibility and effectiveness.
3.2. Participants
A total of 31 experts who were lecturers at three universities – Hanoi National
University of Education, Hue University of Education and Hanoi Metropolitan
University – and 218 chemistry teachers from 10 high schools across the three
regions: North, Central, and South of Vietnam,assessed the practicality,
accuracy and science, relevance and feasibility of 33 constructible topics.
9
Data analysis: After analyzing the experimental data, we concluded that

developing STEM teaching topics can be more effective if scientific and technical
processes are applied.
3.3. Instruments
Evaluate the results of STEM education topics by administering expert
evaluation sheets, teacher evaluations on STEM topics with responses on a
Likert scale, evaluation forms and product evaluation forms.
3.4. Data gathering procedure

- Collect evaluation sheets from experts.
- Collect teacher ratings (responses on a Likert scale) on STEM topics - Google
Forms.
3.5. Data analysis

Experimental data was processed and analyzed using SPSS software to
determine the effectiveness of teaching and applying scientific and technical
processes in teaching STEM topics.
4. Results
4.1. Building a system of STEM teaching topics according to the school model
associated with production practices and social life in Vietnam
Suggested STEM topics include the application of biological techniques in food
production, chemical reactions in the production of industrial products, or the
application of physics in modern technologies, such as renewable energy,
electronics and telecommunications.
These STEM topics not only help students understand the production process
and product application better, but also help them develop the skills of logical
thinking, inquiry, analysis and problem-solving, and creating skills for
generating new and innovative solutions. This will better prepare students for
the future when they access jobs related to science, technology and
manufacturing. Therefore, the proposed STEM topics that correspond with the
school model associated with production practice and social life in teaching
physics, chemistry and biology subjects is consistent with the new general
education curriculum in Vietnam.
With the STEM model, students learn physical chemistry or biology in a holistic,
integrated way, together with math, technology, engineering and other sciences.
Furthermore, students experience and interact with society and businesses,
which stimulates students’ interest, confidence and initiative in relation to
learning, forming and developing general and specific learning competencies, in
response to the new general education curriculum in Vietnam.
Through the process of conducting research and pedagogical experimentation,

our research team designed STEM topics associated with production practice
and social life in Vietnam for teaching physics, chemistry and biology to high
school students (Tables 1, 2 and 3).
10
Table 1. STEM education topics associated with real life and social life in teaching physics
Activities Topic in physics
1 Modeling of hydraulic lifters Liquid pressure
2 Crafting a flood-resistant house Floating
3 Building an economical wood-burning Heat conductivity
stove
4 Manufacturing electrical warning, Circuits connected in series
protection and control devices The circuit is connected in parallel
5 Fabricating clothes wringer Centripetal force
6 Making a simple electrochemical battery Constant current
7 Fabricating devices using solar cells Current in a semiconductor
8 Making a simple electrochemical battery Power transmission, transformers
charger
9 Making a model of a self-cooling house Infrared and ultraviolet rays
10 Making a model of a smoke machine Electric field
11 Making a model of an electrostatic sprayer Electric field
12 Fabricating alarm control circuit by sensor Electric current, electrical circuit
13 Manufacturing automatic timer switches Force balance, torque force
Table 2. STEM education topics associated with practical production and life in teaching
chemistry
Activities Topic in chemistry
1 Designing a periodic table for use by people Periodic table of chemical elements
who are visually impaired
2 Creating colorful beverage made of Chemical balance
vegetables
3 Making a mini nightlight from vegetables Electrode potential and electrochemical
source
4 Making scented candles to repel insects Alkanes
5 Preparing herbal pesticides from materials Halogen derivatives
available in nature
6 Creating a mini missile Carboxylic acid
7 Making soap from waste fat Ester – Lipid
8 Electroplating Electrolysis
9 Producing a slow-release fertilizer Fertilizer
10 Preparing glucosamine hydrochloride from Experience and practice organic chemistry
shrimp shells
11 Domestic water treatment Experience and practice inorganic chemistry
Table 3. STEM education topics associated with real life and production in teaching biology
Activities Topics in biology
1 Designing a drip irrigation system Absorption of water by mineral salts and
roots
2 Rainbow flowering Transporting substances in the body
3 Painting from leaves External features of leaves
4 Making pictures from snail shells General characteristics and roles of molluscs
11
5 Building a model of the circulatory system Circulation and lymphatic circulation

6 Designing a nutritious menu for the family Food standards. Principles of rationing.
Analysis of a given serving
7 Building a model of DNA ADN
8 Designing a dust filter for the home Environmental pollution
School-level STEM topics associated with production practices and social life in
Vietnam are becoming more popular and attracting the attention of many
educators. These topics not only help students understand the production
process better, but also help students apply the knowledge they have learned in
practice, develop logical thinking skills, practical skills, and problem-solving
abilities, motivate students to learn, develop teamwork skills, impart knowledge
and solve problems arising during practice.
STEM topics enable students to understand process impacts and apply science
and engineering knowledge to practice, benefiting students and society.
Students can use chemistry, biology, physics, and engineering to comprehend,
investigate, implement, and solve production and real societal problems in
Vietnam.
The STEM topics associated with production practices and social life proposed
in Tables 1–3 can be implemented on a small or large scale, with different
implementation times depending on the research purpose. In addition, teachers
can combine topics into broader topics, to match students' abilities.
After developing STEM topics associated with production practice and social
life, the list of topics and research issues were evaluated by experts for
practicality, accuracy and science, suitability and feasibility. The results of the
feedback provided by 31 experts on the practicality of STEM topics are listed in
Table 4.
Table 4. Feedback of experts on practicality of STEM topics

Ratio (%)
Topics Totally
Disagree Uncertain Agree
disagree
1 Modeling hydraulic lifters 9.6 6.5 16.1 29
2 Crafting a flood-resistant 0 9.6 16.1 32.2
house
3 Building an economical 3.2 9.6 19.3 38.7
wood-burning stove
4 Manufacturing electrical 6.4 9.6 12.9 32,2
warning, protection and
control devices
5 Fabricating a clothes wringer 3.2 6.5 16.1 38.7
6 Making a simple 6.4 3.2 19.3 32.3
electrochemical battery
7 Fabricating devices using 3.2 0 9.6 51.6
solar cells
8 Making a simple 0 0 9.6 48.4
electrochemical battery
12
charger
9 Making a model of a self- 0 9.6 19.3 32.3
cooling house
10 Making a model of a smoke 0 0 16.1 48.4
machine
11 Making a model of an 3.2 6.4 9.6 32.2
electrostatic sprayer
12 Fabrication of alarm control 3.2 0 9.6 51.6
circuit by sensor
13 Manufacturing automatic 0 0 16.1 54.8
timer switches
14 Designing a periodic table 0 6.5 6.4 32.3
for use by people who are
visually impaired
15 Creating colorful beverages 0 6.4 9.6 48.4
from vegetables
16 Making a mini nightlight 3.2 0 12.9 51.6
from vegetables
17 Making scented candles to 0 0 22.6 45.1
repel insects
18 Preparing herbal pesticides 0 0 12.9 41.9
from materials available in
nature
19 Mini missile 3.2 0 6.5 45.7
20 Making soap from waste fat 9.6 0 12.9 41.9
21 Electroplate 0 6.4 9.6 45.1
22 Producing slow-release 3.2 6.4 9.6 32.2
fertilizer
23 Preparing glucosamine 0 0 12.9 48.4
hydrochloride from shrimp
shells
24 Treating domestic water 9.6 0 9.6 45.1
25 Designing a drip irrigation 0 0 9.6 41.9
system
26 Rainbow flowering 6.5 0 19.3 49.4
27 Painting from leaves 0 0 16.1 48.4
28 Making pictures from snail 0 0 16.1 54.8
shells
29 Building a model of the 0 0 9.6 48.3
circulatory system
30 Designing a nutritious menu 3.2 3.2 9.6 38.7
for the family
31 Building a model of DNA 0 0 12.9 45.1
32 Designing a dust filter for the 0 0 9.6 51.6
home
Table 4 shows that experts agree and agree fully on the practicality of STEM
topics. Thus, STEM topics associated with production practices and social life
were planned in accordance with the curriculum and teaching conditions in high
schools in Vietnam.
The statistical analysis of data collected from the responses of 32 teachers

indicates that most of the proposed STEM topics received agreement. The topic
13
that received the highest level of agreement is "Crafting a flood-resistant house"

with a total of 74.3% agreement. Following that are the topics "Manufacturing
electrical warning, protection, and control devices" with 71.1% agreement and
"Making a model of an electrostatic sprayer" with 80.8% agreement. This finding
also highlights that the highest-rated topics are closely related to real-life
practical applications. The other highly rated topics also involve the application
of learned knowledge in practical scenarios, such as "Designing a periodic table
for use by people who are visually impaired" with 87.1% agreement,
"Manufacturing automatic timer switches" with 84.6% agreement, and
"Designing a drip irrigation system" with 90.4% agreement.
The topics with the highest levels of agreement could be prioritized for
curriculum development and implementation. Topics with higher percentages of
"Totally disagree" and "Disagree" responses may need further refinement or
reconsideration. These findings can help guide the selection and development of
STEM education topics, taking into consideration expert opinions and the
potential for successful implementation in teaching.
4.2. Teacher assessment of the level of understanding and application of

scientific and engineering processes applied in STEM education
We surveyed 218 teachers nationwide; their demographic traits are summarized

in Table 5.
Table 5. Statistics of survey subjects
Information about survey objects Count

Gender Female 160
Male 58
Years of experience Less than 5 years 68
From 5 to 10 years 36
From 11 to 20 years 82
More than 20 years 32
Educational qualification Bachelor 142
Master’s 70
Doctorate 6
The survey data provides key insights into the demographics of the respondents.
A significant gender imbalance is evident, with women comprising 73.39% and
men 26.61% of the respondents. This imbalance suggests a potential trend or
sampling bias that warrants further investigation. The majority of respondents
held a Bachelor degrees (65.14%), indicating that higher education qualifications,
such a Master's degree (32.11%) or Doctorate (2.75%), are not prerequisites in
this field. Experience levels are diverse, but lean towards more experienced
professionals: 37.61% had 11–20 years of experience, while newcomers with less
than 5 years accounted for 31.19%. Only a minority (14.68%) had more than 20
years of experience, which possibly indicates career transitions over time. Table
14
6 below will present the participants' level of understanding concerning the

essence of the scientific and engineering processes.
Table 6. Level of understanding of the nature of the scientific and engineering process
N Mean Std. Deviation
1. Concepts and fundamental principles of the
218 3.2936 .7596
scientific and engineering process
2. Understanding the applications of the
scientific and engineering process in STEM 218 3.2294 .7755
education
3. Knowledge about the development of
218 3.1193 .7402
science and engineering in Vietnam
Table 6 shows how well 218 respondents understood scientific and technical
procedures. Fundamental ideas, STEM education applications, and Vietnam's
scientific and engineering progress were examined. Numerically, these features
have mean scores of 3.2936, 3.2294, and 3.1193, with standard deviations ranging
from 0.7402 to 0.7755. The highest mean score of 3.2936 indicates that
respondents were most confident about their understanding of fundamental
concepts. However, the lowest mean score of 3.1193 – for knowing about local
scientific and technical advances in Vietnam – implies a knowledge deficit. The
low standard deviations imply a relative clustering of replies around the mean,
which demonstrates fair comprehension among respondents. This may indicate
that the educational system has provided a consistent foundation level of
understanding, though it also suggests a more customized approach is needed
to address a variation in learning requirements.
These findings suggest that educational and policy improvements are required.
There exists a good foundation regarding scientific and technical ideas, but the
curriculum needs additional localized context, to improve knowledge of
Vietnam-specific advancements. Table 7 below presents an analysis of the
application of scientific and engineering methodologies within the context of
STEM education, emphasizing the integration and extent of these processes in
instructional practices.
Table 7. Level of application of the scientific and engineering process in STEM topic
teaching
Std.
N Mean
Deviation
4. Implementing a STEM teaching approach based on
the scientific and engineering process. 218 3.1284 .8043
5. Applying the scientific and engineering process to

STEM lessons. 218 3.1284 .9218
6. Applying teaching methods in STEM education in

conjunction with the scientific and engineering process. 218 3.0917 .8318
7. Defining objectives when applying the scientific and

engineering process in teaching. 218 3.2752 .8355
15
8. Designing lessons and teaching activities by applying

the scientific and engineering process. 218 3.1927 .8086
9. Assessing the effectiveness of applying the scientific

and engineering process. 218 3.2661 .8439
10. Providing feedback and make timely adjustments

during the application of the scientific and engineering 218 3.1468 .7659
process.
Seven aspects of STEM education using scientific and engineering procedures

were reviewed by 218 respondents. The mean scores ranged between 3.0917 and
3.2752, with standard deviations between 0.7659 and 0.9218. The highest mean
score – 3.2752 – is for defining objectives when applying the scientific and
engineering process in teaching, which demonstrates educators are best at
creating goals. The lowest mean score, for applying teaching methods in STEM
education in conjunction with the scientific and engineering process – 3.0917 –
suggests weaker confidence, or room for development in this area of teaching.
Standard deviations lower than 1 indicate that responses cluster around the
mean. This suggests that educators apply and comprehend in a similar way, but
it may also mean that teaching methods may not be diversified enough to meet
differing educational requirements and settings.
From a policy and education perspective, these numerical insights suggest that,
while educators are confident about defining objectives and assessing
effectiveness when integrating scientific and engineering processes into STEM
teaching, they may need more professional development in relation to
implementation and application.
4.3. Evaluating the effectiveness of applying scientific processes in teaching

STEM topics according to the school model with production practices and
social life
Table 8. Practicality and application of STEM topics

Std.
N Mean
Deviation
11. Identifing issues in production process and
218 3.2202 .8408
social life that STEM can address.
12. Integrating and updating real-life situations into
STEM topics to reflect community needs and 218 3.2385 .8467
practices.
Table 8 shows mean scores of 3.2202 and 3.2385 for recognizing social and
production concerns STEM may address, and incorporating real-life scenarios
into STEM studies. The standard deviations, 0.8408 and 0.8467, indicate that
answers cluster around these means. This shows educators are moderately
confident about using STEM realistically, though there is room for growth.
These ratings indicate that educators view the real-world applicability of STEM
as closely related to its instruction, which is positive for practical application-
focused pedagogies.
16
Table 9. Linkages with the community and businesses

Std.
N Mean Deviation
13. Collaborating with businesses and social
organizations to update information and 218 2.6239 .9580
resources.
14. Organizing programs, projects, or
internships involving the participation of 218 2.6606 .9716
businesses and the community.
15. Assessing the sustainability of applying
STEM topics in practical production and 218 3.2752 .8679
social life.
16. Developing and updating development
plans for the application of STEM topics in 218 3.1835 .9124
education.
The complexity of the data in Table 9 has increased. The means drop to 2.6239
and 2.6606 for business–social organization collaboration (13) and community-
involved program or project organization (14). These lower ratings, together
with higher standard deviations of 0.9580 and 0.9716, indicate less participation
of external groups and more experience variety among respondents. Assessing
the sustainability of STEM themes in real-world applications has a higher mean
of 3.2752, showing that, while educators may not be as connected with
companies and communities, they understand the long-term viability of STEM
education.
Table 10. Awareness and consciousness

N Mean Std. Deviation
17. Awareness of the importance of connecting
218 3.4220 .8179
STEM to production practice and social life.
18. Awareness of updating and applying STEM
218 3.4037 .8159
topics related to real-world practices.
Table 10 reports data on the understanding of 218 respondents in relation to

real-world relevance and curriculum revisions of STEM education. The mean
scores are high, with 3.4220 for STEM's relevance to production practice and
social life, and 3.4037 for updating and implementing STEM ideas in real life.
With high mean scores and low standard deviations, of 0.8179 and 0.8159,
answers cluster around the mean. The closeness of these mean scores and
standard deviations suggests educators agree STEM education should be
practical and relevant.
Table 11. Linkages between schools and practical production

Std.
N Mean Deviation
19. Create opportunities for students to visit and
218 2.9908 1.0383
experience at production enterprises.
20. STEM education content accurately reflects
218 3.2477 .9021
labor market demands and trends.
17
Table 11 shows that respondents believed that student interaction with

manufacturing firms, and conformity of the STEM curriculum with labor market
demands was crucial to practical STEM education. Based on 218 replies,
providing students with chances to tour producing firms has a mean score of
2.9908 and a standard deviation of 1.0383. These results show moderate but
variable involvement, demonstrating that some schools are proactive and others
are underperforming due to budget restrictions or instructional objectives.
Aligning STEM education with job market developments had a higher mean
score, of 3.2477, and a lower standard deviation, of 0.9021. This suggests that
instructors are more consistent and optimistic about the relevance of the
curriculum to the job market. The higher mean and smaller standard deviation
imply that this STEM education feature is well executed or perceived.
5. Discussion
The goal of STEM education as it pertains to the education environment in
Vietnam is to combine academic ideas with practical application. This study
aimed to describe and validate the use of scientific and technical methods in
STEM subject instruction, particularly in Vietnamese schools that are closely
linked to the country's commercial, production, and social life.
It is clear that incorporating scientific and technology-based methods into the

teaching of STEM subjects has many benefits, especially when subjects are
connected to social and production processes. This is consistent with the ideas of
Leung (2023), who contends that STEM education should be more than the sum
of its parts; Leung calls for an integrated method of instruction that places a
strong emphasis on real-world applications. The findings of this study support
the notion that this pedagogical approach helps students to acquire the abilities
necessary for problem-solving, logical thought and invention, in addition to
facilitating a better understanding of academic ideas (National Research
Council, 2012).
The STEM topics that were suggested act as concrete links between academic
ideas and practical applications (e.g., the use of biology methods in food
production or chemical reactions in industrial products). According to Zhou et
al. (2023), this connection promotes an authentic learning environment in which
students can understand the direct relevance and ramifications of what they are
learning (Zhou et al., 2023).
The instructor feedback provided in this study offers intriguing insights. While
teachers feel confident about setting goals and evaluating student progress,
there appears to be a little difference in how these STEM subjects are actually
applied in the classroom. Numerous factors, such as a lack of funding, a lack of
professional development of teachers, or possibly the difficult task of
incorporating real-world applications into conventional curricular frameworks
may be to blame for this gap. Effective professional development, according to
Darling-Hammond (2017), is essential to closing this gap.
18
One of the main conclusions of this study is acknowledgement of the need for a
more regional curriculum that reflects developments and applications unique to
Vietnam. Localization of the curriculum can increase the relevance and
applicability of what is taught in the classroom considerably, as noted by
Darling-Hammond.
The findings also suggest that teachers may require more in-depth professional
development. Training programs could concentrate on methods to effortlessly
integrate scientific and technological concepts into lessons, as well as on how to
create strong connections between education institutions and industrial settings
(McGuigan & Hoy, 2006).
However, in order for these advantages to be fully attained, the education

system, particularly in the context of Vietnam, must address the difficulties that
teachers face and make sure that the curriculum is relevant to the local context
and in line with the country's socioeconomic structure (Rahmalia & Elida, 2022).
6. Conclusion
Vietnam is currently undergoing a significant phase of rapid advancement in the
realm of STEM education. Nevertheless, the application of the science and
technology process as an instructional framework for STEM subjects remains
relatively constrained. The primary objective of our study is to investigate the
feasibility and assess the efficacy of applying this process in an educational
context that forges meaningful connections between education institutions and
real-world scenarios, and specifically encompassing production practices. The
process of teaching STEM is underpinned by the systematic application of
scientific research procedures to address and resolve problems. This framework
encompasses a series of methodical steps, namely observation, problem
definition, hypothesis formulation, experiment design, data collection, results
analysis, and culminating in the derivation of conclusions. By adhering to this
process, a dynamic, exploratory, and creative mode of learning is engendered in
students. Research findings unequivocally demonstrate the manifold advantages
conferred by the application of the science and technology process in the
pedagogical instruction of STEM subjects.
This study illuminates the significant strides Vietnam has made in redefining
STEM education through the integration of technical science processes within a
model that aligns schooling with production practices and social life. The
development of STEM topics that resonate with real-life production and societal
functions marks a transformative approach in the educational landscape; the aim
is to equip students with the competencies required in a rapidly evolving world.
In addition to affirmation by experts that the proposed STEM topics are practical
and relevant, is evidence of potential effectiveness of the topics in the education
setting.
Meanwhile, the assessment of teachers' proficiency reveals a commendable

understanding of basic scientific concepts and a readiness to apply scientific and
engineering processes in teaching. However, there is clear room for
19
improvement regarding teachers’ grasp of Vietnam's specific scientific and

technological advances. This finding underscores the need for continuous
professional development initiatives to update educators on local innovations
and industry practices. Enhancing teachers' confidence and competence in these
areas is crucial for the successful implementation of the STEM approach.
The evaluation results also suggest a need to strengthen the connections

between schools, communities and production entities. While educators
recognize the importance of real-world applications of STEM education, there is
a discernible gap in active collaborations between education institutions,
industry and societal organizations. Bridging this gap is essential if students are
to be provided with a more immersive and practical learning experience,
thereby making education more relevant and responsive to real-world
challenges and market needs.
Furthermore, the study highlights educators' consistent acknowledgment of the

importance of aligning STEM education with market demands, and the
sustainability of educational themes. This alignment is crucial for ensuring that
students are not only academically proficient, but also equipped with skills and
knowledge that are marketable and relevant in the professional world.
In conclusion, the integration of technical science processes in teaching STEM
topics within a model that mirrors production and societal practices holds great
promise for education in Vietnam. The positive evaluations by experts and the
readiness of educators provide a solid foundation for this education model.
7. Recommendations
Teaching associated with production practice and social life helps students
realize the value of learned knowledge and skills, to practice thinking to solve
practical problems that require an investment of time, and resource materials
and technical facilities. However, many schools in remote areas cannot meet
these demands.
Students can gain a better understanding of STEM concepts and how they are
utilized in other fields if case studies and real-world examples are integrated in
the curriculum. Students can enhance their analytical and problem-solving
abilities, as well as their creativity and invention through the usage of this
methodology – two benefits that are essential for advancing Vietnam's national
development.
Students can more easily make the connection between abstract ideas and real-
world applications if case studies from real life and practical applications are
included in STEM coursework. This method will help students in Vietnam make
the connection between abstract ideas and practical applications, thereby
making their education more engaging and meaningful. Additionally, it will
enable them to get a deeper understanding of the applications of STEM
principles across a range of industries, and prepare them for potential job
options in the future.
20
This study offers some practical suggestions for Vietnamese teachers and
students in schools to use STEM teaching topics. Teaching associated with
production practice and social life can be carried out in the classroom or in
extracurricular teaching. Applying the scientific and technical process in
teaching STEM topics can be carried out in different subjects – not only
chemistry, physics, biology – it can also be effectively implemented in natural
science at the lower secondary level.
Students can develop the critical thinking, problem-solving, and cooperation

skills necessary for their future employment if they are exposed to STEM
instruction in a variety of topics. This multidisciplinary method improves
scientific understanding and encourages creativity and innovation in
technology, engineering, and mathematics.
However, for its sustainable implementation, concerted efforts are necessary to

enhance teacher training, facilitate active industry–academic partnerships, and
continuously refine the STEM topics, to ensure they remain locally relevant,
scientifically accurate, and aligned with market needs. By doing so, STEM
education in Vietnam will be positioned to cultivate a future workforce that is
innovative, skilled, and ready for the challenges of the modern world.
Acknowledgement
This study was funded by the University of Education, Hue University,
DHH2021-03-155.
8. References
An, L., & Yang, J.-W. (2019). Research on the Teaching Design and Experiment in Physics
Education at a Junior High School Based on STEAM Education and 6E Learning Process.
https://doi.org/10.2991/assehr.k.191221.145
Baharin, N., Kamarudin, N., & Manaf, U. K. A. (2018). Integrating STEM Education
Approach in Enhancing Higher Order Thinking Skills. International Journal of
Academic Research in Business and Social Sciences, 8(7).
https://doi.org/10.6007/IJARBSS/v8-i7/4421
Buturlina, O., Dovhal, S., Hryhorov, H., Lysokolenko, T., & Palahuta, V. I. (2021). STEM
Education in Ukraine in the Context of Sustainable Development. European Journal
of Sustainable Development. https://doi.org/10.14207/ejsd.2021.v10n1p323
Tho, C. C. (2016). Lessons on changes in training teachers from stem festivals and math
open days in Vietnam. Journal of Science, Educational Science, 61(8A), 196–201.
https://doi.org/10.18173/2354-1075.2016-0147
Dabney, K. P., Tai, R. H., Almarode, J., Miller-Friedmann, J., Sonnert, G., Sadler, P. M., &
Hazari, Z. (2012). Out-of-School Time Science Activities and Their Association With
Career Interest in STEM. International Journal of Science Education Part B.
https://doi.org/10.1080/21548455.2011.629455
Duc, N. M., Ninh, T. T., Toan, N. T., Hai, K. T., & Yuenyong, C. (2019). STEM education
program: manufacturing mixture of phosphate and potash fertilizer straws and
waste of animal bones. Journal of Physics: Conference Series, 1340(1), 012050.
https://doi.org/10.1088/1742-6596/1340/1/012050
Gilchrist Watt, H. M., Shapka, J. D., Morris, Z., Durik, A. M., Keating, D. P., & Eccles, J. S.
(2012). Gendered Motivational Processes Affecting High School Mathematics
Participation, Educational Aspirations, and Career Plans: A Comparison of Samples
21
From Australia, Canada, and the United States. Developmental Psychology.

https://doi.org/10.1037/a0027838
Huy, T. Q., Khanh, P. Q., Tuan, N. N., Thong, N. D., & Theu, L. T. (2023). Design and
Teaching Process of Stem Project “Smart Night-Lamp” for High School Students.
Vnu Journal of Science Education Research. https://doi.org/10.25073/2588-
1159/vnuer.4716
Johnson, C. C. (2013). Conceptualizing Integrated STEM Education. School Science and
Mathematics, 113(8), 367–368. https://doi.org/10.1111/ssm.12043
Khmelnikova, L., & Maslak, A. (2022). STEM - Education in the Chemical Training of
Future Pharmacists. Interconf. https://doi.org/10.51582/interconf.19-20.12.2022.005
Kurup, P. M., Li, X., Powell, G., & Brown, M. E. (2019). Building Future Primary
Teachers’ Capacity in STEM: Based on a Platform of Beliefs, Understandings and
Intentions. International Journal of Stem Education.
Le, L. T. B., Tran, T. T., & Tran, N. H. (2021). Challenges to STEM education in
Vietnamese high school contexts. Heliyon, 7(12).
https://doi.org/10.1016/j.heliyon.2021.e08649
Linh, H. (2020, January 13). STEM education to be applied in Vietnam’s schools.
Https://Vietnamnet.Vn/.
Luong, M. D., & Dam, T. H. (2021). Teaching Material and Energy Topics in Grade 5
Science Curriculum Based on STEM Education. Journal of Physics Conference Series.
https://doi.org/10.1088/1742-6596/1835/1/012052
Margot, K. C., & Kettler, T. (2019). Teachers’ Perception of STEM Integration and
Education: A Systematic Literature Review. International Journal of Stem Education.
McGuigan, L., & Hoy, W. K. (2006). Principal Leadership: Creating a Culture of
Academic Optimism to Improve Achievement for All Students. Leadership and Policy
in Schools, 5(3), 203–229. https://doi.org/10.1080/15700760600805816
Ministry of Education and Training. (2020). Implement STEM education in secondary
education. 3089.
National Research Council. (2012). A framework for K-12 science education: Practices,
crosscutting concepts, and core ideas. In A Framework for K-12 Science Education:
Practices, Crosscutting Concepts, and Core Ideas. https://doi.org/10.17226/13165
Hieu, N. T., lam, M. H., Huynh, M. P., Nguyen, N. P. A., & Wong Wing-Keung. (2020).
Decision Sciences in Education: The STEM tech Model to Create Stem Products at
High Schools in Vietnam. In Advances in Decision Sciences (Michael McAleer, Vol. 24,
pp. 1–50). Asia University, Taiwan.
Duc, N. M., Linh, N. Q., & Yuenyong Chokchai. (2019). Situation of organizing STEM
activities in Vietnamese Schools. Journal of Physics: Conference Series, 1340(1), 1–6.
https://doi.org/10.1088/1742-6596/1340/1/012030
Nguyen, Q. L., Nguyen, M. D., & Chokchai, Y. (2019). Developing critical thinking of
students through STEM educational orientation program in Vietnam. Journal of
Physics: Conference Series, 1340(1), 012025.
Nguyen, T. H. (2019). STEM/ STEAM education from hands-on experience to creative thinking.
Youth Publisher.
Nugroho, O. F., Permanasari, A., Firman, H., & Riandi, R. (2021). The Importance of Stem
Based Education in Indonesia Curriculum. Pedagonal : Jurnal Ilmiah Pendidikan, 5(2).
Rahmalia, S. N., & Elida, E. (2022). The Relationship Between Learning Outcomes Of
Entrepreneurial Creative Products And Entrepreneurial Readiness For Class Xii
Students Of The Catering Expertise Program At Smkn 2 Pariaman. Jurnal Pendidikan
Tata Boga Dan Teknologi, 3(2), 126. https://doi.org/10.24036/jptbt.v3i2.346
22
Rodger W. Bybee. (2015). The Case for STEM Education: Challenges and Opportunities.
In The Case for STEM Education: Challenges and Opportunities. National Science
Teachers Association. https://doi.org/10.2505/9781936959259
Shernoff, D. J., Sinha, S., Bressler, D. M., & Ginsburg, L. (2017). Assessing Teacher
Education and Professional Development Needs for the Implementation of
Integrated Approaches to STEM Education. International Journal of Stem Education.
https://doi.org/10.1186/s40594-017-0068-1
Stohlmann, M., Moore, T., & Roehrig, G. (2012). Considerations for Teaching Integrated
STEM Education. Journal of Pre-College Engineering Education Research, 2(1), 28–34.
Sujarwanto, E., Madlazim, & Sanjaya, I. G. M. (2021). A Conceptual Framework of STEM
Education Based on the Indonesian Curriculum. Journal of Physics Conference Series.
https://doi.org/10.1088/1742-6596/1760/1/012022
Tu, P. T. H., Hang, N. T., Van, L. T. T., Dung, H. Van, Nga, N. T. H., & Phan, C. X. (2022).
Experimental Teaching by Scientific Methods for Developing Students’ Natural
Finding Capacity in Teaching Natural Science in Vietnamese High Schools. World
Journal of Education, 12(6), 1. https://doi.org/10.5430/wje.v12n6p1
Yata, C., Ohtani, T., & Isobe, M. (2020). Conceptual Framework of STEM Based on
Japanese Subject Principles. International Journal of Stem Education.
https://doi.org/10.1186/s40594-020-00205-8
Zhou, S., Merzdorf, H. E., Douglas, K. A., & Moore, T. J. (2023). Development and
Validation of an Integrated STEM Teacher Classroom Observation Protocol. Journal
of Pre-College Engineering Education Research (J-PEER), 13(1).
https://doi.org/10.7771/2157-9288.1357
PUBLISHER International Journal of Learning, Teaching and
Society for Research and Knowledge Management Educational Research
Port Louis
Republic of Mauritius
www.ijlter.org The International Journal of Learning, Teaching
and Educational Research is an open-access
Chief Editor journal which has been established for the dis-
Dr. Antonio Silva Sprock, Universidad Central de semination of state-of-the-art knowledge in the
Venezuela, Venezuela, Bolivarian Republic of field of education, learning and teaching. IJLTER
welcomes research articles from academics, ed-
Editorial Board
ucators, teachers, trainers and other practition-
Dr. Fatima Zohra Belkhir-Benmostefa
ers on all aspects of education to publish high
Dr. Giorgio Poletti
quality peer-reviewed papers. Papers for publi-
Dr. Chi Man Tsui
cation in the International Journal of Learning,
Dr. Fitri Suraya Mohamad
Teaching and Educational Research are selected
Dr. Hernando Lintag Berna
through precise peer-review to ensure quality,
Dr. Charanjit Kaur Swaran Singh
originality, appropriateness, significance and
Dr. Abu Bakar
readability. Authors are solicited to contribute
Dr. Eglantina Hysa
to this journal by submitting articles that illus-
Dr. Mo'en Salman Alnasraween
trate research results, projects, original surveys
Dr. Hermayawati Hermayawati
and case studies that describe significant ad-
Dr. Selma Kara
vances in the fields of education, training, e-
Dr. Michael B. Cahapay
learning, etc. Authors are invited to submit pa-
Dr. Bunmi Isaiah Omodan
pers to this journal through the ONLINE submis-
Dr. Vassiliki Pliogou
sion system. Submissions must be original and
Dr. Meera Subramanian
should not have been published previously or
Dr. Muhammad Kristiawan
be under consideration for publication while
Dr. Wahyu Widada
being evaluated by IJLTER.
Dr. Som Pal Baliyan
Dr. Reem Khalid Abu-Shawish
Dr. Froilan Delute Mobo
Dr. Mohamed Ali Elkot
Dr. Anabelie Villa Valdez
Mr. Teody Lester Verdeflor Panela

Vol 22 No 10 October 2023

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Vol 22 No 10 October 2023

Uploaded by

Copyright:

Available Formats

IJLTER.

Society for Research and Knowledge Management

The International Journal of Learning, Teaching and Educational

Aims and Objectives

The main objective of this journal is to provide a platform for educators,

Indexing and Abstracting

The International Journal of Learning, Teaching and Educational

We are very happy to publish this issue of the International Journal of

The International Journal of Learning, Teaching and Educational

We are grateful to the editor-in-chief, members of the Editorial Board

The Editorial Board is composed of renowned people from across the

We will endeavour to ensure the reputation and quality of this journal

Editors of the October 2023 Issue

Assessment of Learning about Sustainability in Students with Down Syndrome........................................................ 79

A Systematic Review of Interventions Improving University Students’ EFL Writing Competence.......................... 93

A Comprehensive Approach to Eliminate English Second Language Learners’ Grammatical Difficulties............306

Acquisition of Communicative Skills by Foreign Students in a Multicultural Learning Environment................... 359

International Journal of Learning, Teaching and Educational Research

Assessing the Effects of Flipped Classroom to the

Thavamani Parati , Mohd Nihra Haruzuan Mohamad Said and

Abstract. In this era, implementing technology in teaching and learning

Keywords: Flipped Classroom; Listening; Speaking; Reading; Writing;

1. Background of the study

harmonious. However, in today's constantly changing environment, the

1.1 Conceptual Framework

1.2 Scope of Research

1.3 Implications and Limitations of the Research

The main principle in 21st-century learning in line with School Transformation

According Mandasari & Wahyudin, (2021), flipped classroom brings satisfaction

The flipped classroom learning approach proves to be a valuable tool for

3.1 Research Design

In tandem with the quantitative assessment, the research incorporates a survey

3.2 The participants of the study

3.3 Research Instruments

Before commencing the actual research, twelve sets of questionnaires were

3.4 Data Collection

The process of data collection was meticulously planned within a specific

3.5 Data Analysis

3.5.1 Analysis of Primary Pupils’ Perception

3.5.2 Analysis of Primary Pupils’ English Learning Performance

3.5.3 Analysis on the Relationship Between Primary Pupils’ English Overall

3.5.4 Analysis of the Influence of Flipped classroom on Year 3 Pupils’ English

4.2 Effect of flipped classroom learning method on pupils’ English performance

Table 3. Result of Wilcoxon signed-ranks test between listening, speaking, reading,

Table 4. Result of Spearman’s rank-order correlation coefficient between score of Pre-

4.4 Influence of flipped classroom learning method on Year 3 Pupils’ English

1 .353ª .124 .094 6.108 1.999

Table 6. Analysis of Variance (ANOVA) Table for a Simple Linear Regression

According to tables 5, 6 and 7 above, there is a moderate positive correlation is

Figure 2: Displays a scatterplot illustrating a simple linear regression analysis

After Spearman’s rank-order correlation coefficient analysis had been completed,

compared to languages and humanities. In conclusion, this research reveals that

Brewer, R., & Movahedazarhouligh, S. (2018). Successful stories and conflicts: A

International Journal of Learning, Teaching and Educational Research

Knowledge Management Using Storytelling with

Abstract. Creativity is one of the most marketable skills in the digital

Keywords: creativity; infographics; knowledge management;

rendered many professions obsolete or will replace more professions in the

Given the aforementioned reasons, the researcher emphasizes the importance of

form. This is because infographics allow audiences to understand the content

Silpakorn University’s Faculty of Arts covers diverse fields of science, including

In this study, I attempt to examine and develop the creativity of undergraduate

innovative solutions to address increasingly complex problems (Shen, 2023).