Journal of Baltic Science Education, Vol. 21, No. 2, 2022

Vol.21, No.
2, 2022
ISSN 1648-3898 /Print/
ISSN 2538–7138 /Online/
The International Journal of the Scientia Socialis Ltd., & SMC “Scientia Educologica”
Journal of Baltic Science Education, Vol. 21, No. 2, 2022
Editorial Board
ISSN 1648–3898 /Print/
Editor-in-Chief ISSN 2538–7138 /Online/
Prof., Dr. Vincentas Lamanauskas Vilnius University, Lithuania
Deputy Editor-in-Chief
Prof., Dr. Andris Broks University of Latvia, Latvia
Deputy Editor-in-Chief
Prof., Dr. Miia Rannikmäe University of Tartu, Estonia
Executive Secretary
Dr. Laima Railienė SMC “Scientia Educologica”, Lithuania
Editors:
Prof., Dr. Boris Aberšek University of Maribor, Slovenia
Prof., Dr. Agnaldo Arroio University of Sao Paulo, Brazil
Prof., Dr. Martin Bilek Charles University, Czech Republic
Dr. Paolo Bussotti University of Udine, Italy
Prof., Dr. Bulent Cavas Dokuz Eylul University, Turkey
Prof., Dr. Salih Cepni Uludag University, Turkey
Dr. Peter Demkanin Comenius University, Slovakia
Dr. André du Plessis Nelson Mandela University, South Africa
Dr. Markus Sebastian Feser University of Hamburg, Germany
Prof., Dr. Jack Holbrook University of Tartu, Estonia
Prof., Dr. Ryszard M. Janiuk Maria Curie Sklodowska University, Poland
Dr. Milan Kubiatko Jan Evangelista Purkyně University in Ústí nad Labem, Czech Republic
Dr. Todar Lakhvich Belarusian State Medical University, Republic of Belarus
Prof., Dr. Jari Lavonen University of Helsinki, Finland
Dr. Rita Makarskaitė-Petkevičienė Vilnius University, Lithuania
Prof., Dr. Paul Pace Malta University, Malta
Prof., Dr. Valfrids Paškevičs Daugavpils University, Latvia
Prof., Dr. Jongwon Park Chonnam National University, Korea
Prof., Dr. Raffaele Pisano University of Lille, France
Prof., Dr. Pavol Prokop Comenius University in Bratislava, Slovakia
Prof., Dr. Konstantinos Ravanis University of Patras, Greece
Dr. Alona Rauckienė-Michaelsson Klaipėda University, Lithuania
Dr. Dušica Rodić University of Novi Sad, Serbia
Prof., Dr. Heimo Saarikko Helsinki University, Finland
Prof., Dr. Hae-Ae Seo Pusan National University, Korea
Dr. Uladzimir K. Slabin Lane Community College, USA
Prof., Dr. Andrej Šorgo University of Maribor, Slovenia
Prof., Dr. Borislav V. Toshev Sofia University, Bulgaria
Prof., Dr. Georgios Tsaparlis University of Ioannina, Greece
Dr. Muhammet Usak Kazan Federal University, Republic of Tatarstan, Russian Federation
A scientific journal JBSE issued by the Scientia Socialis Ltd., in cooperation with SMC Scientia Educologica, Lithuania,
emphasizes theoretical, experimental and methodical studies in the field of science education. JBSE is an international
academic journal. In order to maintain the high standards appropriate to such a journal, all contributions received are
submitted for anonymous review by two experts, additionally to review by the Editor. The decision of the Editor on
the acceptance of articles is final and no correspondence can be entered into on reasons for rejection of a submitted
contribution.
Published since 2002 Address:

The journal is published bimonthly. Scientia Socialis, Ltd.
IF: 1.024 (2018) Donelaičio Street 29, LT-78115 Siauliai, Lithuania
H Index: 17 (2020) E-mail: jbse@scientiasocialis.lt
SJR: 0.42 (2020) Phone: +370 687 95668
ICDS: 8.8 (2020) Home page: http://www.scientiasocialis.lt/jbse/
ICV: 156.65 (2020) Skype: scientia12
ISSN 1648–3898 (Print) © Scientia Socialis Ltd., a member of the Lithuanian

Publishers Association, Lithuania, 2022
ISSN 2538–7138 (Online)
The articles appearing in this journal are indexed/abstracted in Copernicus Index, EBSCO:
174 Academic Search Premier, Web of Science Core Collection (Social Sciences Citation Index ),
SCOPUS, ProQuest, Road, Crossref, ERIC and DOI.
ISSN 1648–3898 /Print/ CONTENTS
CONTENTS
Editorial
CLIMATE EDUCATION: A GRAND CHALLENGE

Jari Lavonen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176
Articles
PRE-SERVICE SCIENCE AND COMPUTER TEACHERS’ VIEWS ON TEACHING PRACTICE

COURSES
Ayhan Bulut, Abdullatif Kaban, Ömer Bilen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
SELF-REGULATED LEARNING STRATEGIES IMPACT FOURTH-GRADE STUDENTS’ POSITIVE

OUTCOMES IN SCIENCE CLASS
Busra Cengiz-Istanbullu, Gonul Sakiz. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
AN EXAMINATION OF PRE-SERVICE TEACHERS’ EXPERIENCES IN CREATING A SCIENTIFIC

DIGITAL STORY IN THE CONTEXT OF THEIR SELF CONFIDENCE IN TECHNOLOGICAL
PEDAGOGICAL CONTENT KNOWLEDGE
Umit Izgi-Onbasili, Aysegul Avsar-Tuncay, Burcu Sezginsoy-Seker, Seyit Ahmet Kiray . . . . . . . . . . . . 207
THE EFFECT OF GAME-BASED LEARNING VIA KAHOOT AND QUIZIZZ ON THE ACADEMIC
ACHIEVEMENT OF THIRD GRADE PRIMARY SCHOOL STUDENTS
Aleksandar Janković, Dragan Lambić. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224
ALIGNMENT OF CONCEPTS OF MEIOSIS AMONG CURRICULUM, TEXTBOOKS, CLASSROOM

TEACHING AND ASSESSMENT IN UPPER SECONDARY SCHOOL IN REPUBLIC OF KOREA
Youngshin Kim, Yun-hye Lee, Hyonyong Lee, Soo-min Lim . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232
IMPLEMENTATION OF İNNOVATİVE ARTİFİCİAL İNTELLİGENCE COGNİTİONS WİTH PROBLEM-

BASED LEARNİNG GUİDED TASKS TO ENHANCE STUDENTS’ PERFORMANCE IN SCIENCE
King-Dow Su . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245
THE IMPACT OF TEACHERS’ PROFESSIONAL DEVELOPMENT IN SCIENCE PEDAGOGY ON

STUDENTS’ ACHIEVEMENT: EVIDENCE FROM TIMSS 2019
AiBin Tang, WenYe Li, Dawei Liu. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258
PRIMARY PRE-SERVICE TEACHERS’ METAPHORICAL PERCEPTIONS OF THE CONCEPT OF

ENVIRONMENTAL POLLUTION
Eser Ültay . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275
SCIENCE MAPS AND BIBLIOMETRIC ANALYSIS ON HYGIENE EDUCATION DURING 2012-2021

Muhammet Usak, Selma Sinan, Olcay Sinan. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288
THE MEASURING INSTRUMENT OF PRIMITIVE PHYSICS PROBLEM FOR UPPER-SECONDARY

SCHOOL STUDENTS: COMPILATION AND EXPLORATION
Hongjun Xing, Yanfang Zhai, Sisi Han, Yuping Zhao, Wenhui Gong, Yuting Wang, Jing Han, Qiaoyi Liu . . 305
COMPARING THE EFFECTS OF MODELLING AND ANALOGY ON HIGH SCHOOL STUDENTS’

CONTENT UNDERSTANDING AND TRANSFERABILITY: THE CASE OF ATOMIC STRUCTURE
Song Xue, Daner Sun, Liying Zhu, Hui-Wen Huang, Keith Topping . . . . . . . . . . . . . . . . . . . . . . . . 325
Information
INFORMATION FOR CONTRIBUTORS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342
PROBLEMS OF PSYCHOLOGY IN THE 21ST CENTURY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343

175
This is an open access article under the Journal of Baltic Science Education, Vol. 21, No. 2, 2022
Creative Commons Attribution 4.0
International License ISSN 1648–3898 /Print/
Jari Lavonen
University of Helsinki, Finland
Climate change and its mitigation represent a major global challenge, alongside biodiversity loss, global
pandemics, and shortages of fresh water and food. It has the power to shape the environment and future;
however, policymakers seek to prevent undesirable changes through environmental and education policies.
Hinesjeffrey et al. (2013) consider climate education to be a major challenge. For example, the mitigation of
climate change requires an understanding of climate change as a multidisciplinary phenomenon that must
be considered in addition to the natural sciences, at least from the perspective of the social sciences, engi-
neering, economics and education. This mitigation requires individuals to have a wide range of competences,
such as creative and critical thinking and socioemotional skills (Hestness, 2015).
The basic phenomenon in climate education can be described as the slow rise in average temperature
across the globe. This slow rise causes various changes in the atmosphere, in the ground, and in the seas,
which causes changes in ‘average’ weather or climate in different areas of the globe. In some areas, dry sea-
sons grow longer and more intense, while in others, rains and storms increase. This is referred to as climate
change (IPCC, 2018). The reason for the slow rise in the average temperature globally is the increase in
greenhouse gases, such as carbon dioxide and methane. This increase in greenhouse gases is strengthening
the greenhouse effect, with the result that the average global temperature is rising slowly. The greenhouse
effect itself is important, and without it, plants and animals would die. The increase in greenhouse gases
comes mainly from the burning of fossil fuels used to produce energy, as well as other human activities such
as logging, agriculture and the production of raw materials such as steel and concrete. Climate education
aims to support students in understanding the basic phenomena related to climate change, the reasons or
models behind the phenomena and approaches for mitigating climate change.
The teaching of climate change is emphasised in curriculum documents, but it is difficult to teach and
learn and not necessarily engaging for young people because of its interdisciplinary, multifaceted, com-
plex—and frankly intimidating—nature. A special challenge is based on the fact that teaching climate change
typically aims to influence students’ behaviour. Therefore, the teaching and learning of climate change and
its mitigation require reformed pedagogical approaches and improvements in teacher education (Hestness
et al., 2015).
Stratton et al. (2015) recognised four teacher-related challenges in making progress in teaching and learn-
ing climate change. First, in many countries, there is a lack of competent science teachers, especially teachers
176
https://doi.org/10.33225/jbse/22.21.176
ISSN 1648–3898 /Print/ CLIMATE EDUCATION: A GRAND CHALLENGE
(pp. 176-178)
who have qualifications in physics. Second, science teachers lack knowledge related to climate change and
pedagogical competences, especially in lab conditions and in the teaching and learning of values, ethics
and socioscientific issues in the context of teaching climate change. Third, teachers need to participate only
a few days per year in professional learning activities, and climate change has been emphasised for a short
time in initial science teacher education. Consequently, educating science teachers to teach climate change
requires a new paradigm for the preparation of science teachers. It will require a new teacher education cur-
riculum that addresses the social purposes of science education in the context of uncertainty and change.
Several researchers, such as Stratton et al. (2015), have outlined the general characteristics of climate
education. Researchers have also suggested specific models for climate education, such as Cantell et al.
(2019), who suggested a climate education ‘bicycle model’. The model emphasises the importance of the
following aspects in climate education: climate-related knowledge, thinking skills, values, identity, worldview,
action, motivation, participation, future orientation, hope and other emotions and operational barriers. I
introduce the next two climate education-related reviews, which recommend promising learning project
types as solutions to climate education.
Rousell et al. (2020) applied a systematic literature review to analyse climate education research from
1993 to 2014. The review indicated that young people’s understanding of climate change is generally limited,
erroneous and highly influenced by media. Second, pedagogical approaches to climate education appear
to be largely ineffective in affecting students’ attitudes and behaviour. The review identifies the need for
participatory, interdisciplinary, creative and affect-driven approaches to climate education, which are largely
missing from the literature. The authors call for new forms of climate education that directly involve young
people in responding to the scientific, social, ethical and political complexities of climate change. Project-
based learning could be such a new form because it guides students to ask relevant questions related to
the complex phenomenon, critically search for information, for example, on the internet according to the
questions and then analyse and present this information in a way it can be applied in different situations.
Jorgenson et al. (2019) recommend, based on their systematic review, that environmental educators
and researchers work together to develop and adopt innovations that save energy and raw materials. Here,
too, project-based learning can serve as a pedagogical frame. In this case, the starting point for working and
learning is a challenge or problem, which is addressed by finding ideas, making prototypes and evaluating
these ideas and prototypes. The end result of this work, too, is knowledge or a new way of working to solve
environmental problems.
In summary, science teachers need to have extensive subject matter and multidisciplinary knowledge
related to climate change and pedagogical content to design high-quality climate education that addresses
the complex nature of climate change and influence students’ attitudes and actions. In practice, teachers
should use versatile pedagogical approaches, such as project-based learning, in climate education.
References
Cantell, H., Tolppanen, S., Aarnio-Linnanvuori E., & Lehtonen, A. (2019). Bicycle model on climate education: Presenting and
evaluating a model. Environmental Education Research, 25(5), 717–731. https://doi.org/10.1080/13504622.2019.1570487
Hestness, E. McGinnis, J. R., & Breslyn, W. (2015). Integrating sustainability into science teacher education through a
focus on climate change. In S. K. Stratton et al. (Eds.), Educating science teachers for sustainability, ASTE series in
science education (pp. 143–162). Springer. https://doi.org/10.1007/978-3-319-16411-3_9
Hinesjeffrey, P. J., McCartneyand, M., & Wible, B. (2013). Plenty of challenges for all. Science, 340(6130), 290–291.
Incropera, F. P. (2015). Climate change: A wicked problem—Complexity and uncertainly at the intersection of science,
economics, politics and human behaviour. Cambridge University Press.
IPCC. (2018). Global warming of 1.5°C. Special repor t. The I ntergovernmental Panel on Climate Change.
https://www.ipcc.ch/sr15/
Jorgenson, S. N., Stephens, J. C., & White, B. (2019). Environmental education in transition: A critical review of recent
research on climate change and energy education. The Journal of Environmental Education, 50(3), 160–171.
https://doi.org/10.1080/00958964.2019.1604478
Rousell D., & Cutter-Mackenzie-Knowles, A. (2020). A systematic review of climate education: Giving children and
young people a ‘voice’ and a ‘hand’ in redressing climate change. Children’s Geographies, 18(2), 191–208.
https://doi.org/10.1080/14733285.2019.1614532
177
(pp. 176-178)
Stratton, S. K., Hagevik, R., Feldman, A., & Bloom M. (2015). Toward a sustainable future: The practice of science teacher
education for sustainability. In S. Stratton, R. Hagevik, A. Feldman, & M. Bloom (Eds.), Educating science teachers for
sustainability. ASTE series in science education. Springer. https://doi.org/10.1007/978-3-319-16411-3_23
Received: March 20, 2022 Revised: April 12, 2022 Accepted: April 17, 2022
Cite as: Lavonen, J. (2022). Climate education: A grand challenge. Journal of Baltic Science Education, 21(2), 176-178.
Jari Lavonen PhD, Professor, Department of Education, Faculty of Educational

Sciences, University of Helsinki, PL 9 (Siltavuorenpenger 5A),
00014 Helsinki, Finland.
E-mail: jari.lavonen@helsinki.fi
ORCID: https://orcid.org/0000-0003-2781-7953
178
This is an open access article under the
International License
PRE-SERVICE SCIENCE AND

COMPUTER TEACHERS’ VIEWS
ON TEACHING PRACTICE ISSN 1648-3898 /Print/
COURSES ISSN 2538-7138 /Online/
Abstract. The Teaching Practice course

taught in Education Faculties gives pre-
service teachers real experience before
Ayhan Bulut, starting the profession. This study aims to
examine the teaching practice courses in
Abdullatif Kaban, the undergraduate programs of educa-
Ömer Bilen tion faculties by the views of pre-service
science and computer teachers. 50 4th grade
students, 25 from the science education
department and 25 from the computer and
Introduction instructional technology education depart-
ment, participated in the study. Within the
The concept of education, which mankind is constantly trying to expand scope of the study, the data were collected
and improve, indeed includes an active and dynamic process. Countries are through online interviews of approximately
trying to expand their qualified personnel potential by making efficient 30 minutes with the participants, and con-
and effective use of all available educational resources. In addition, modern tent analysis was made. According to the
world countries have attached particular importance to the education fac- findings, pre-service teachers believe that
tor in constructing a strong future in terms of economy, social life, politics, teaching practice courses are insufficient
and health. This situation carries education in the twenty-first century, as in to ensure active participation in lessons in
every period of human history, to a position that no individual, no family, terms of learning gains and content and
and no society can give up (Topbaş, 2001). Atatürk emphasizes the value of that they lack a flexible structure that allows
education, stating, “It is education that determines whether a nation lives as them to participate in classes at various
a free, independent, glorious and sublime society or whether it is dragged grade levels. They also stated that the teach-
into captivity and misery” (Celep, 2008). While Aristotle views education as ing practice course includes result-oriented
a means for individuals to acquire moral behaviors, Checheno views it as measurement and evaluation activities.
disciplining the human mind, Descartes views it as learning to use the mind However, they underlined that the semesters
correctly, Rousseau views it as everything gained by adults who are not in- and credits allocated to undergraduate
nately human, and Kant views it as a means for humans to become human teaching practice courses are insufficient.
(Şişman, 2008). Undoubtedly, teachers are at the forefront of the most critical Furthermore, throughout the process, pre-
elements that activate, make sense and add value to the educational process. service teachers reported some communica-
Effective teachers are people who can think, ask questions, criticize, tion and coordination issues in the teaching
open to development and innovation, constantly renew themselves, and practice course. They suggested that every
love their profession (Kavcar, 1999). Teachers play a leading role in developing university with an education faculty should
countries, raising qualified manpower, and transferring the cultural values of open practice schools that include all levels
the society to future generations (Özden, 1999). The teacher’s professional of compulsory education to overcome the
responsibility in creating and influencing human behavior is quite difficult. challenges encountered during the process.
The teaching profession is seen as a form of behavioral engineering (Sönmez, Keywords: computer teaching, pre-service
2000). As can be seen, the duties, responsibilities, and expectations placed teachers, science teaching, teaching practice
on teachers have grown daily. Both the teacher education system and the course
teacher’s professional actions contribute significantly to constructing the
teacher identity (Ünal, 2011). Teachers in an information society must be
educated to a high standard and equipped to carry out their responsibilities Ayhan Bulut
Bayburt University, Turkey
(Erdem, 2013). Abdullatif Kaban, Ömer Bilen
In education faculties, the qualifications gained by pre-service teachers Ataturk Universitey, Turkey
directly affect the quality of education. A teacher candidate is expected to
graduate from the education faculty by gaining professional competence and
179
COURSES
(pp. 179-191) ISSN 2538–7138 /Online/
teaching skills. A course called “Teaching Practice”, which plays an effective and important role in gaining these
skills has been added to the curriculum of the departments in education faculties (Ermiş et al., 2010). Teaching
practice activities are related to the dimension of transforming the knowledge gained by pre-service teachers in
the pre-service program into practice (Özkan et al., 2005). Furthermore, the Teaching Practices course is critical
because it enables a teacher to recognize circumstances that may arise in the professional area and observe his/her
professional condition (Yalın Uçar, 2012). Teaching Practice course serves as a kind of internship for the students
of the faculty of education to gain experience in the teaching profession. In the Teaching Practice course, the
candidates are expected to improve themselves by developing their teaching competencies, using and evaluat-
ing textbooks and other tools and materials, developing additional teaching materials as needed, measuring and
evaluating, and finally sharing their studies with the guidance teacher and university practice course instructor
(YÖK, 1998). The teaching practice course is thought to be effective in the social-emotional development of pre-
service teachers (Caires et al., 2010). Pre-service teachers get experience through teaching practice that will help
them move from being a student to a teacher, where they will be responsible for educating others (Conderman et
al., 2005). According to pre-service teachers, teaching practices are essential in realizing professional success and
their characteristics (Singh et al., 1997).
The research topic consists of the problems that science education and computer and instructional technol-
ogy education students confront throughout their teaching practice course and their expectations for the course’s
teaching process regarding educational programs and their opinions and solution recommendations. Teaching
practice courses provide a real-world learning environment in which science education and computer and instruc-
tional technology education students can study the needs of their profession while also practicing and experienc-
ing. This study is considered very significant because it raises the possibility that education faculties can develop
and arrange teaching practice courses based on the findings of this study. Furthermore, it is anticipated that the
findings of this study will provide faculty members and practice teachers participating in the teaching practice
course with an up-to-date perspective on measurement and assessment connected to the process. It is foreseen
that it will help improve functionality by ensuring collaboration between universities with education faculties and
schools associated with the Ministry of National Education. It is expected that science education and computer
and instructional technology education students would provide a new roadmap for teaching this course to all key
education stakeholders, along with solution proposals.
Although there are different practices, in many countries, teacher training is carried out with Teaching Practice
courses that include practice and institutional preparation processes (Duman & Karagöz, 2016). This research has
focused on raising awareness among people who are interested in education in the international arena about how
these courses are conducted and giving them the opportunity to evaluate this process from a broader perspective.
In addition, it is thought that the results obtained from the research will contribute to the transfer of the Teaching
Practice course on a global scale to a platform with more universal validity and applicability. This study aimed to
examine the teaching practice courses in the undergraduate programs of education faculties by the views of pre-
service teachers who they students of science education and computer and instructional technology education
departments. For this purpose, answers to the following questions were sought.
1. What are pre-service teachers’ views about the learning outcomes and contents of the Teaching Prac-
tice Course?
2. What are pre-service teachers’ views about the semester and credit of the Teaching Practice Course?
3. What are pre-service teachers’ views about the assessment and evaluation process of the Teaching
Practice Course?
4. What are pre-service teachers’ views about the problems experienced in the Teaching Practice Course?
5. What are the suggestions of the pre-service teachers for the improvement of the Teaching Practice
Course?
Research Methodology
General Background
In this study, the case study, which is one of the qualitative research designs, was used. The case study is
defined as when the researcher offers a situation description and then collects detailed and in-depth information
about an actual situation, a current limited system, from multiple information sources (Creswell, 2013). In other
180
ISSN 1648–3898 /Print/ PRE-SERVICE SCIENCE AND COMPUTER TEACHERS’ VIEWS ON TEACHING PRACTICE
COURSES
ISSN 2538–7138 /Online/ (pp. 179-191)
words, case studies examine what is happening in the environment through systematic data collection, analysis,
and presentation of the findings. The result is a clear knowledge of why things transpired the way they did and
what needs to be studied in greater detail in future research (Davey, 1991). Therefore, considering the complexity
and diversity of educational environments and goals, case studies for various purposes provide essential data for
scholars (Cohen et al., 2000).
Participants
The criterion sampling method was used in the present study. Criterion sampling examines all instances that
satisfy a predefined set of criteria. The criteria were determined as enrolling in teaching practice courses in the fall
semester of 2021 and being a student in one of the science education and computer education departments in
Turkey. A total of 50 students, 25 from the science education department and 25 from the computer education
department participated in the study. Demographic properties of the participants were given in Table 1.
Table 1
Demographic Properties of the Participants
Science education 25
Department
Computer education 25
Female 32
Gender
Male 18
21-24 45
Age
25 and over 5
Data Collection Tool
The researcher designed a semi-structured interview form (see Appendix) for data collection. While preparing
the questions in the interview form, the relevant literature was reviewed, and four experts in the field of educational
programs and teaching, one expert in the field of educational management and planning, and one expert in the
field of measurement and evaluation were consulted. One question was removed from the interview form by
expert opinions, and the places of two others were changed. Then, as part of the pre-application study with seven
pre-service teachers in science education and computer and instructional technology education departments,
online interviews were conducted in an electronic environment. As a result of these interviews, the interview ques-
tions were rearranged, the questions deemed necessary to ask the participants during the research process were
reviewed, and the semi-structured interview form was finalized. These pre-service teachers were not included in
the following interviews.
Data Collection and Analysis

The data for this study were collected using a semi-structured questionnaire form applied to pre-service sci-
ence and computer teachers enrolled in education faculties of various universities in Turkey. Due to the Covid-19
epidemic, individual interviews lasting 30 minutes on average were conducted online with the participants who
voluntarily participated in the research. The data from the study were analyzed using the summative content
analysis method. Since summative content analysis is an inductive analytical method, it focuses on the origins of
the investigated phenomenon or event. The concepts underlying the data and the relationships between these
concepts are revealed through coding. The researcher’s objective in qualitative research is to uncover the underly-
ing themes of the problem using the descriptive and detailed data he has collected, to transform the data he has
obtained into meaningful and systematic structures, that is, to develop or verify a theory based on these facts.
Without a theory to ground the event or phenomena under investigation, inductive analysis, or content analysis
based on coding, is required (Baltacı, 2019; Şimşek & Yıldırım, 2011). Content analysis involves four stages: (1) coding
181
COURSES
(pp. 179-191) ISSN 2538–7138 /Online/
the data, (2) identifying codes, categories, and themes, (3) organizing the codes, categories, and themes, and (4)
defining and interpreting the findings (Eysenbach & Köhler, 2002; Miles & Huberman, 1994). In addition, a descrip-
tive analysis approach was also employed by including direct quotations from time to time to draw attention to
the views of pre-service science and computer teachers who participated in the study. In the descriptive analysis,
the data are classified by predetermined themes, the findings associated with the classified data are summarized,
and the summaries are evaluated using the researcher’s subjective knowledge (Baltacı, 2019). Furthermore, the
researcher establishes a cause-effect relationship between the findings and, if necessary, conducts structural dif-
ference analyses on the cases (Kitzinger, 1995; Kvale, 1994).
Research Results
The findings of the study were analyzed by considering the views of the pre-service science and computer
teachers regarding the following five themes; the sufficiency of the teaching practice course in terms of learning
gains and content, the sufficiency of the teaching practice course in terms of semesters and credits covered in the
programs, the studies conducted by practitioners to measure and evaluate the success of the teaching practice
course, the problems they encountered during the teaching practice course and activities that can be carried out
to make the teaching practice course more qualified and richer.
Pre-service Teachers’ Views about the Learning Outcomes and Contents of the Teaching Practice Course
In this subsection of the research, the findings were obtained from the views of pre-service teachers regarding
the learning outcomes and contents of the Teaching Practice course have been presented.
Table 2
The Views of Pre-service Teachers on the Adequacy of the Teaching Practice Course in Terms of Learning Outcomes and Content
Insufficient in terms of learning gains and content f
The number of gains that we can participate in class may increase. 21

Must be adaptable to practicing at various grade levels. 8
A new curriculum allowing more classes where we can teach can be developed. 7
The achievements aimed at promoting the work done out of school are insufficient. 5
Sufficient in terms of learning gains and content f
It is progressing in line with the practice process. 7

The gain expressions are arranged clearly and understandably. 7
They were planned to take us to the targeted point. 4
The process allows me to evaluate myself. In this respect, I think it is sufficient. 4
The scope of the practice was enough for me to learn the basics of the profession. 2
According to Table 2, most pre-service science and computer teachers stated that the teaching practice courses
were insufficient in terms of learning gains and content. The pre-service teachers reported that the number of
achievements in which they might participate actively in the classroom was insufficient (42%), that the teaching
practice courses did not have the acquisition and content that would provide them with the flexibility to practice
in the same term (16%). They responded that the curriculum should be constructed to provide students with
additional learning opportunities and content (14%) and that the resources available to promote extracurricular
school activities were insufficient (10%). The following are the views of some pre-service science and computer
teachers on this subject:
“The course’s learning gains and content are mostly determined by activities based on our observations. While we must
watch the practice teacher’s lessons regarding the course, I believe that we are merely spectators and listeners in this pro-
cess, which renders us passive. Such activities should be minimized, whereas activities in which we may participate actively
should be increased.” (PT30)
182
COURSES
ISSN 2538–7138 /Online/ (pp. 179-191)
“We can learn about the accomplishments of the school courses we study. However, we cannot develop experience with
the acquisitions and contents of subsequent grade levels. We should be allowed to examine different grade levels during
the practice process.” (PT41)
“I believe that the more classroom lessons I teach, the more I will improve as a result of this process.” (PT10)
Some of the pre-service science and computer teachers who participated in the study stated that the teach-
ing practice course met their learning gains and content expectations. The pre-service teachers stated that the
course’s learning gains and content were consistent with the practice process (14%), that the gains were clear and
understandable (14%), that they were organized in a way that would lead them to the desired gain (8%), and that
they would evaluate themselves throughout the process. They noted that it gave opportunities for students (8%)
and that the practice process taught basic things about the profession (4%). The following are the views of some
pre-service science and computer teachers on this subject:
“The course objectives and the subjects that comprise the course’s content overlap with the studies we conducted during
the course.” (PT33)
“In my opinion, the gain statements were clearly and comprehensibly expressed. Everything was crystal clear in that regard.”
(PT2)
“At the end of the process, I believe I have reached the point expected of me.” (PT11)
“While listening to the lecture and teaching, I had the opportunity to examine what, how, and in what way I was unable
to perform.” (PT25)
Pre-service Teachers’ Views about the Semester and Credit of the Teaching Practice Course
In this subsection of the research, the findings obtained from the views of the pre-service science and com-
puter teachers about whether the teaching practice courses included in the programs are sufficient for semesters
and credits are presented.
Table 3
The Views of Pre-service Teachers on Whether the Teaching Practice Courses are Sufficient in Terms of Semester and Credits
Covered in the Programs
Insufficient in terms of semesters and credits f
Insufficient in terms of semesters 22

Insufficient in terms of credits 15
Sufficient in terms of semesters and credits f
Sufficient in terms of semesters 8

Sufficient in terms of credits 5
According to Table 3, most pre-service science and computer teachers (74%) stated that the teaching practice
courses in the curriculum were insufficient in terms of semesters and credits. The pre-service teachers reported
that they found the teaching practice courses included in the curricula to be insufficient as the most course period
(44%) and subsequently deemed the course credits assigned to this course insufficient. The following are the views
of some pre-service science and computer teachers on this subject:
“I do not find the time allocated to this course is sufficient. Because I envision a program that is compressed into the last
two semesters of our four-year faculty career.” (PT4)
“In my opinion, it is adequate in terms of course duration, but the number of practice days and credits for course hours can
be increased.” (PT36)
183
COURSES
(pp. 179-191) ISSN 2538–7138 /Online/
Some of the pre-service science and computer teachers (26%) who participated in the study stated that
they found the current form of the teaching practice course included in the curriculum was sufficient in terms
of semesters and credits. The pre-service teachers stated that they believed it was sufficient as a period covering
most subjects (16%) and then as course credits (10%). The following are the views of some pre-service science and
computer teachers on this subject:
“I believe that the semester-long duration of this course is enough. As a result, I am unable to dedicate further time to this
class.” (PT23)
“The credit assigned to this course is adequate. We do not participate actively in class. I believe that we should conduct our
lectures rather than relying on observations during this process. However, because this is not the case, I am quite bored in
this course.” (PT9)
Pre-service Teachers’ Views about the Assessment and Evaluation Process of the Teaching Practice Course
In this subsection of the research, the findings obtained from the views of the science and computer teach-
ing technologies pre-service teachers about what kind of measurement and evaluation methods are used by the
practitioners while teaching the teaching practice courses and the adequacy of these studies conducted for the
assessment and evaluation of the teaching practice course are presented.
Table 4
The views of Pre-service Teachers on the Type and Adequacy of Measurement and Evaluation Methods Used in Teaching
Practice Courses
Insufficient measurement and evaluation activities f
Only practice files and presentations are evaluated. 18

Results-oriented evaluation is done. 11
Everyone is given an average score. 4
Adequate feedback is not provided regarding our deficiencies in the practice process 4
Sufficient measurement and evaluation activities f
The practice files are reviewed. 10

Our lectures are observed and evaluated. 10
Our absenteeism status is being evaluated. 5
According to Table 4, the majority of pre-service science and computer teachers stated that the measuring
and evaluation activities used during teaching practice courses were insufficient. The most frequently encountered
measurement and evaluation activities for pre-service teachers in this area were creating and presenting files and
delivering presentations (36%), followed by a result-oriented evaluation (22%) and assigning everyone an aver-
age score (8%). They also stated that adequate feedback was not provided in the practice process regarding their
shortcomings (8%). The following are the views of some pre-service science and computer teachers on this subject:
“Throughout this period, we continued to fill up the pertinent documents in the file. We occasionally delivered lectures. Our
practice teachers graded us based on these files we submitted and the lessons we taught.” (PT8)
“I simply believe that the process is appraised technically as result-oriented.” (PT12)
“Our practice teachers assigned the identical evaluation grade to all of our friends in my group.” (PT27)
“I am unaware of how our instructors evaluate our performance in this course at any point during the process.” (PT31)
184
COURSES
ISSN 2538–7138 /Online/ (pp. 179-191)
Some of the pre-service science and computer teachers who participated in the study stated that they found
the measurement and evaluation activities applied throughout the teaching practice courses were adequate. It
was stated that the practice files were screened for the measurement and evaluation activities that the pre-service
teachers encountered the most in this regard (20%), then lectures were observed and reviewed at the same rate
(20%), and attendance and absenteeism were evaluated at least once (10%). The following are the views of some
pre-service science and computer teachers on this subject:
“Our practice teacher evaluated us by examining our files we organized throughout the process and describing how the
process worked. This, I believe, is sufficient for the evaluation.” (PT19)
“Each of our friends gave lectures for at least four hours. We came to these classes in a planned and prepared manner and
delivered our presentations.” (PT26)
“Our instructor evaluated us based on our interest in the course and our absenteeism” (PT34)
Pre-service Teachers’ Views about the Problems Experienced in the Teaching Practice Course
In this subsection of the research, the findings obtained from the views of the pre-service science and com-
puter teachers regarding the problems they encountered during the execution of the teaching practice course
are presented.
Table 5
The Views of Pre-service Teachers on the Problems they Encountered in the Teaching Practice Course
Problems and difficulties encountered during the process f
The high number of pre-service teachers per practice teacher 30

They need more economic support during this period 17
Lack of communication and coordination 8
Practice studies should be conducted in schools with easy access 3
Heavy paperwork during the process 1
According to Table 5, the majority of pre-service science and computer teachers stated that the number of
pre-service teachers assigned to only one practice teacher for problems encountered during teaching practice
courses was high (60%). In addition, the pre-service teachers stated that they needed more economic support in
this process (34%), they had a lack of communication and coordination (16%), it is necessary to ensure that practice
studies were carried out in schools with easy transportation (6%) and that the paperwork should be reduced in
the process (2%). The following are the views of some pre-service science and computer teachers on this subject:
“First of all, I believe it is inappropriate for six people to engage in the practice process conducted in a classroom by a practice
teacher. Because the practice teacher is also having difficulty coping with large groups of us during this process.” (PT42)
“During one of the weeks that I was teaching in the classroom, I had difficulty in obtaining the materials I wanted to use in
my class due to economic reasons.” (PT7)
“I find it tough to commute to the school where I am required to practice due to the school’s opposite location and limited
transportation options.” (PT21)
Pre-service Teachers’ Suggestions for the Improvement of the Teaching Practice Course
In this subsection of the research, the findings obtained from the views of science and computer instructional
technology pre-service teachers about what kind of studies can be done to make the teaching practice course
more qualified and richer are presented.
185
COURSES
(pp. 179-191) ISSN 2538–7138 /Online/
Table 6
The Pre-service Teachers’ Suggestions for the Improvement of the Teaching Practice Course
Studies that can be done on the subject f
More active participation in the class 26

Opportunity to practice at different grade levels should be offered 12
At the compulsory education level, practice schools should be established within education faculties 8
Experienced teachers with high communication skills should be appointed as instructors in this course 7
The activities aimed at identifying school administration works should be increased 4
More time should be spent working with children outside the classroom 4
Candidates for teaching positions should also participate in studies on school-family collaboration
1
throughout the process
According to Table 6, the majority of pre-service science and computer teachers stated that they wanted to
actively participate in more courses about the types of studies that could be conducted to improve the quality
and depth of the teaching practice courses, as well as the ability to practice at various grade levels. Opportunity
should be provided (24%), practice schools at compulsory education levels should be established within education
faculties (16%), experienced teachers with strong communication skills should be appointed as instructors in this
course (14%). They also stated that the number of studies aimed at recognizing school administration affairs should
be increased (8%), that more time should be devoted to the work to be done with students outside the classroom
(8%), and those pre-service teachers should also participate in the studies on school-family cooperation during
the process (2%). The following are the views of some pre-service science and computer teachers on this subject:
“As pre-service teachers, our roles in courses should be more centered on activities in which we can actively engage and
learn. It’s like we’re a little passive in this process.” (PT1).
“As part of this process, each teacher candidate must attend at least one class and/or branch. It will not be sufficient for
him to simply attend classes at his current level.” (PT28).
“I believe that the practice teachers assigned for teaching practice courses should be selected based on a set of criteria. I
believe that instructors who lack communication skills and a basic understanding of technology will be unable to contribute
much to this process, even if they wish to.” (PT45).
“Instead of solely teaching in the classroom, more time should be allocated in the teaching practice course for out-of-class
activities in which pre-service teachers will participate.” (PT3).
“I believe that we, as pre-service teachers, should participate in activities promoting school-family cooperation throughout
the practice phase. This will provide both of us with this type of experience. Additionally, it will allow the families of our
students, with whom we spend 12 weeks, to become acquainted with us, and we will become acquainted with them.” (PT13).
Discussion
When the views of the pre-service science and computer teachers participating in the study on the teaching
practice courses in the education faculties were examined, it was determined that most pre-service teachers found
the teaching practice course inadequate in terms of learning gains and content. Additionally, they responded that
they believed the teaching practice curriculum was inadequate in terms of learning gains and content because
insufficient gains allowed them to participate actively in the program. Although the teaching practice course
demands more practice, it is worth noting that pre-service teachers expressed dissatisfaction with the limited
number of acquisitions in which they played an active part in learning gains and material in this course. They later
indicated that the teaching practice curriculum used throughout the practice phase was not flexible enough to
allow for practice at various grade levels. It is stated that science and computer teaching technologies pre-service
teachers do not wish to spend their entire twelve-week practice period attending classes and giving lectures at the
same grade level and providing them with the opportunity to practice at different grade levels during the same
term help them develop more professionally.
Furthermore, they mentioned that they desired to provide more lectures to gain more hands-on experience
and knowledge during the teaching practice course. Additionally, pre-service science and computer teachers
186
COURSES
ISSN 2538–7138 /Online/ (pp. 179-191)
stated that the activities in the teaching practice course focused on studies conducted in the classroom and that
studies conducted outside the classroom were either not implemented adequately or were overlooked by the
practice teachers. This problem can be described as pre-service teachers failing to achieve the appropriate level of
acquisitions required outside the classroom as part of the course’s curriculum during the teaching practice course.
Similar to the findings of this study, Eraslan (2009) and Aslan and Sağlam (2018) concluded that there were insuf-
ficient opportunities for practice, no opportunity to observe different classes, brief internship periods, insufficient
schools, and insufficient information on pre-service teachers at the start of the process. On the other hand, some
of the pre-service science and computer teachers who participated in the study stated that the teaching practice
course followed the practice process in terms of adequacy in terms of acquisition and content, that the course’s
accomplishments were expressed clearly and plainly, that the process was planned to achieve the desired gains,
and that the process was self-evaluating. They mentioned that it provided an opportunity for them to understand
the fundamentals of the profession.
According to another finding, most science and computer instructional technology pre-service teachers
stated that the curriculum’s teaching practice courses were insufficient in semesters and credits. This situation
demonstrates the importance of conducting a needs analysis study by curriculum development specialists to
determine whether the weight assigned to teaching practice courses in undergraduate programs is sufficient.
When the relevant literature is examined, similar research findings to those acquired in this study are also found.
In Gökçe and Demirhan’s (2005) study, practice teachers acknowledged the need to lengthen the duration of the
teaching practice course in schools for pre-service teachers. According to Işıkoğlu et al. (2007), pre-service teachers
complete their practice in a short amount of time without gaining experiences in various aspects of the school
environment, such as classroom management, material use, utilizing various teaching methods, evaluation, and
school-family cooperation.
Another finding from the research was that practitioners used various measurement and evaluation methods
when teaching practice courses. These studies for measuring and evaluating the teaching practice courses were
largely deemed inadequate by science and computer teaching technology pre-service teachers. Concerning
the subject, pre-service teachers state that they are only assessed on the files they have prepared and the pre-
sentations they delivered in the class, that the process favors result-oriented evaluations, that an average score
is assigned to everyone and the process is skipped, and that they do not receive adequate feedback about the
practice period. When the relevant literature is examined, it is concluded that similar research findings confirm
the findings of this study. Beck and Kosnik (2002) emphasized that providing complete and thoughtful feed-
back to pre-service teachers about their performance is critical to their development as teachers. Şahin (2016)
stated that pre-service teachers’ progress is not objectively evaluated and that there is an insufficient chance
for practice due to a shortage of time.
According to the research findings, the result-oriented evaluation approach, which contradicts the spirit of
constructive education, is still applied by practice teachers during the teaching practice course. Additionally, pre-
service science and computer teachers stated that the process was skipped, and they did not receive adequate
feedback on the evaluation process, indicating some issues with the measurement and evaluation component
of the teaching practice course. In general, it is vital to improve the quality of an open education system by pro-
viding feedback on each element and stage of the process. In this regard, it should be emphasized that practice
teachers either lack sufficient expertise in measurement and evaluation throughout the teaching practice course
or have shortcomings. On the other hand, science and computer education technology pre-service teachers who
stated that the measurement and evaluation activities used in the teaching practice course were sufficient stated
that their instructors conducted measurement and evaluation activities by monitoring their files throughout the
process, observing their presentations delivered in the class and attendance in the class.
According to another result obtained from the research, regarding the problems that pre-service science and
computer teachers most frequently encounter during the teaching practice course, the number of pre-service
teachers given to a practice teacher and the faculty member responsible for the practice course is higher than ever
before. In addition, the pre-service teachers stated that they needed economic support more than ever, that there
was a lack of communication and coordination during the practice process, that the schools where they would
conduct practice studies should have ease of transportation and that the paperwork load should be reduced during
the process. In line with these results, it is possible to say that reducing the number of pre-service teachers in the
teaching practice course taught by a practice teacher who carries out this job at schools, providing economic sup-
port to pre-service teachers, and addressing coordination deficiencies are all seen as critical steps toward improving
187
COURSES
(pp. 179-191) ISSN 2538–7138 /Online/
the quality of the process. Allocating specific times for each task to the teacher candidate, arranging activities that
engage the teacher candidate, and sharing all activities with all stakeholders will facilitate the practice process.
When the relevant literature is examined, it is determined that there are also research findings that support
the findings of this study. As a result, keeping the number of pre-service teachers per instructor to a minimum is
critical for maintaining a healthy process (Kırksekiz et al., 2015). Another finding from the research is that pre-service
teachers of science and computer instructional technology face financial barriers in obtaining materials. When the
relevant literature is reviewed, similar findings are found with this conclusion (Aslan & Sağlam, 2018; Demir & Çamlı,
2011; Karaca & Aral, 2011). In line with these results, during the teaching practice course, it is possible to say that
science and computer instructional technology pre-service teachers need economic support to participate more
effectively in the course, to ensure that they attend the classes in a more prepared way in terms of equipment and
materials, and to cover some extra new expenses such as transportation.
Another result from the research is a lack of communication and collaboration throughout the process. Similar
research findings parallel these findings emphasized the necessity of communication between the teacher candi-
date, practice teacher, and practice instructor (Gündoğdu et al., 2010; Karaca & Aral, 2011).
In addition, reducing the number of students per academician and teacher will increase the interest and time
allocated to each student to fully reach the desired gains in the teaching process of the teaching practice course,
to process the units more healthily, to make the educational situations and testing situations more qualified.
According to another finding obtained from the research, science and computer teaching technologies, pre-
service teachers stated that they should participate more actively in the courses to make the teaching practice
course more qualified and richer, that they should be able to practice at different class levels and that application
schools operating in compulsory education levels should be opened within the faculties of education. When the
relevant literature is examined, it has been determined that other research findings are similar to those acquired
in this study. Pre-service teachers should gain experience teaching in various educational settings (Cansaran et
al., 2006). Pre-service teachers should be provided with multiple types of lessons and opportunities to observe a
variety of different kinds of teachers (Aytaçlı, 2012).
Furthermore, they stated that experienced teachers with strong communication skills should be appointed
as instructors in this course, that studies on school administration should be increased, that more time should be
allocated to work with students outside the classroom, and that pre-service teachers should participate in studies
on school-family cooperation. Teachers-in-training who will guide future teachers should be chosen from a pool
of trained, experienced and willing teachers (Brooks & Sikes, 1997). Practice teachers should be taught to lead,
observe and assess, provide feedback, and exchange data and evaluation forms collected from short-term summer
courses with the instructor and should be given the title of “practice guidance teacher” (Brooks, 2006).
In line with these results, it may be an essential step in solving some of the problems experienced in the pro-
cess that universities with education faculties in Turkey open practice schools that provide education at primary,
secondary, and high school levels by the 4+4+4 education system to facilitate and increase the scientificness,
coordination, efficiency, productivity, controllability and accessibility of teaching practice and similar courses that
require the active participation of the students. In addition, it is possible to say that even though the teaching
practice course has a curriculum for practice and active participation of the students, practitioners did not ensure
that students participated actively in the lessons during the practice process. The following recommendations can
be made based on the research findings.
•• Studies can be conducted to eliminate the problems if the achievements in the science and computer
instructional technologies teaching practice curriculum are already aimed at ensuring the active par-
ticipation of students and if the issue arises during program implementation.
•• Flexibility in listening and lecturing can be introduced to science and computer teaching technologies
pre-service teachers at multiple grade levels in the same semester, as long as they are related to their
respective fields in the teaching practice course’s curriculum.
•• It could be suggested that practitioners involved in the teaching practice course be more sensitive to
the activities that should take place outside of the classroom in the program.
•• It may be beneficial to do a needs analysis to see whether the teaching practice course offered in un-
dergraduate programs in science and computer instructional technologies is sufficient for semesters
and credits.
188
COURSES
ISSN 2538–7138 /Online/ (pp. 179-191)
•• In-service training courses on how to include measurement and evaluation activities into the teaching
practice course can be conducted for practitioners who are already assigned or will be assigned to the
teaching practice course.
•• The number of pre-service teachers per teacher and academician can be reduced to increase the qual-
ity of the process.
•• To address some of the issues raised during the teaching practice course, establishing practice schools
covering all obligatory education levels in all institutions with education faculties can provide a differ-
ent perspective and level of involvement in the process.
Conclusions and Implications
As a result, it is thought that the number and course credits of the teaching practice course should be
increased by arranging the teaching practice courses in a way that will ensure more active participation of pre-
service science and computer teachers in terms of learning outcomes and content. In addition, it can be ensured
that pre-service teachers gain experience in this subject by bringing flexibility to practice at different grade levels
in the same term. However, it may be much more beneficial to include a result-based evaluation with the process
rather than a result-oriented evaluation of the teaching practice course by the practitioners. On the other hand, it
is clear that the number of pre-service teachers assigned to the advisor in the teaching practice course needs to
be reduced in a way that will increase the quality of the process. In addition, it would be appropriate to provide
economic support to pre-service teachers to meet their needs for transportation and educational materials during
the periods they take the teaching practice course. On the other hand, by opening practice schools from all levels
of education within the universities that train teachers, teaching practice courses in these schools will be useful in
terms of time, process and environment, and will accelerate the resolution of some problems arising in the process.
Declaration of Interest
Authors declare no competing interest.
References
Aslan, M. & Sağlam, M. (2018). Öğretmenlik uygulaması dersinin öğretmen adaylarının görüşlerine göre değerlendirilmesi
[Evaluation of teaching practice course according to opinions of student teachers]. Hacettepe Üniversitesi Eğitim Fakültesi
Dergisi (H. U. Journal of Education) 33(1), 144-162.
Aytaçlı, B. (2012). İlköğretim matematik öğretmenliği lisans programında yer alan okul deneyimi ve öğretmenlik uygulaması
derslerinin değerlendirilmesi [Evaluation of school experience and teaching practice courses in undergraduate programme
of elementary mathematics education]. (Unpublished Master Thesis). Ege Üniversitesi, Sosyal Bilimler Enstitüsü, İzmir.
Baltacı, A. (2019). Nitel araştırma süreci: Nitel bir araştırma nasıl yapılır? [Qualitative research process: How is a qualitative research
done?]. Ahi Evran Üniversitesi Sosyal Bilimler Enstitüsü Dergisi, 5(2), 368-388. https://doi.org/10.31592/aeusbed.598299
Beck, C., & Kosnik, C. (2002). Components of a good practicum placement: Student-teacher perceptions. Teacher Education
Quarterly, 29(2), 81–98. http://www.jstor.org/stable/23478294
Brooks, V., & Sikes, P. (1997). The good mentor guide: Initial teacher education in secondary schools. Open University Press.
Brooks, V. (2006). A ‘quiet revolution’? The impact of Training Schools on initial teacher training partnerships. Journal of Education
for Teaching, 32(4), 379–393. https://doi.org/10.1080/02607470600981979
Caires, S., Almeida, L. S., & Martins, C. (2009). The socioemotional experiences of student teachers during practicum: A case of
reality shock?. The Journal of Educational Research, 103(1), 17-27.
Cansaran, A., İdil, Ö., & Kalkan, M. (2006). Fen bilgisi eğitimi anabilim dallarındaki “okul deneyimi” uygulamalarının değerlendirilmesi
[The evaluation of school experience applications in the science departments]. Gazi Eğitim Fakültesi Dergisi, 26(1), 83–99.
https://dergipark.org.tr/en/download/article-file/77214
Celep, C. (2008). Öğrenme ortamı olarak okul ve sınıf [School and classroom as learning environment]. Eğitime Bakış Dergisi,
4(11), 3- 10.
Cohen, L., Manion, L. & Morrison, K. (2000). Research methods in education (5th ed.). London and Routledge Publishing.
https://doi.org/10.4324/9780203224342
Conderman, G., Morin, J., & Stephens T. J. (2005). Special education student teaching practices. Preventing School Failure, 49(3),
5-10. https://doi.org/10.3200/PSFL.49.3.5-10
Creswell, J. W. (2013). Nitel araştırma yöntemleri [Qualitative research methods]. (Translate Ed: Bütün, M. & B. Demir; S.). Siyasal
Yayın Dağıtım.
189
COURSES
(pp. 179-191) ISSN 2538–7138 /Online/
Davey, L. (1990). The application of case study evaluations. Practical Assessment, Research, and Evaluation, 2, Article 9.
https://doi.org/10.7275/02g8-bb93
Demir, Ö., & Çamlı, Ö. (2011). Öğretmenlik uygulaması dersinde uygulama okullarında karşılaşılan sorunların sınıf ve okul öncesi
öğretmenliği öğrenci görüşleri çerçevesinde incelenmesi: Nitel bir çalışma [Schools teaching practice lesson practice
problems encountered the investigation of class and opinions of pre-school students: A qualitative study]. Uludağ
Üniversitesi Eğitim Fakültesi Dergisi, 24(1), 117-139. https://dergipark.org.tr/en/download/article-file/153433
Duman, T., & Karagöz, S. (2016). An evaluation of Turkish teacher education system compared to other models in different
countries. International Journal of Educational Research Review, 1(1), 1-13. https://doi.org/10.24331/ijere.309956
Eraslan, A. (2009). İlköğretim matematik öğretmen adaylarının “Öğretmenlik Uygulaması” üzerine görüşleri [Prospective
mathematics teachers’ opinions on ‘Teaching Practice’]. Necatibey Eğitim Fakültesi Elektronik Fen ve Matematik Eğitimi Dergisi,
3(1), 207-221. https://dergipark.org.tr/en/download/article-file/39779
Erdem, A. R. (2013). Öğretmen yetiştirmenin bugünü ve geleceği: Sorunlar ve çözüm önerileri [Present and future of teacher training:
Problems and solutions]. Anı Yayıncılık.
Ermiş, H., Uygun, K., & İnel, Y. (2010, 11-13 November). Sosyal bilgiler öğretmen adaylarının Okul Deneyimi II dersi etkinliklerine
ilişkin görüşlerinin değerlendirilmesi [Evaluation of social studies teacher candidates’ views on School Experience II course
activities]. International Conference on New Trends in Education and The Implications, Antalya.
Eysenbach, G., & Köhler, C. (2002). How do consumers search for and appraise health information on the world wide
web? Qualitative study using focus groups, usability tests, and in-depth interviews. BMJ, 324(7337), 573-577.
https://doi.org/10.1136/bmj.324.7337.573
Gökçe, E., & Demirhan C. (2005). Öğretmen adaylarının ve ilköğretim okullarında görev yapan uygulama öğretmenlerinin
öğretmenlik uygulaması etkinliklerine ilişkin görüşleri [Opinions of pre-service teachers and practice teachers working in
primary schools on teaching practice activities]. Ankara Üniversitesi Eğitim Bilimleri Fakültesi Dergisi, 38(1), 43-71.
Gündoğdu, K., Coşkun, Z. S., Albez, C., & Bay, E. (2010). Sınıf öğretmeni adaylarının öğretmenlik uygulamasına ilişkin görüşleri ve
beklentileri [The opinions and expectation of teacher trainee about teaching practice]. Bayburt Üniversitesi Eğitim Fakültesi
Dergisi, 5(1), 55–67. https://dergipark.org.tr/en/download/article-file/215167
Işıkoğlu, N., İvrendi, A., & Şahin, A. (2007). Öğretmenlik uygulaması sürecine öğretmen adaylarının gözüyle derinlemesine bir
bakış [An in-depth look at the teaching practice process from the perspective of pre-service teachers]. Eğitim Araştırmaları,
26, 131-142.
Karaca, N., & H. Aral, N. (2011). Okul öncesi öğretmen adaylarının öğretmenlik uygulamalarında karşılaştıkları sorunlar [Problems
faced by pre-school teacher candidates in their teaching practices]. 2nd International Conference on New Trends in
Education and Their Implications, Antalya, Türkiye.
Kırksekiz, A. Uysal, M., İşbulan, O., Akgün, Ö., Kıyıcı, M., & Horzum, M. (2015). Okul deneyimi ve öğretmenlik uygulaması derslerine
eleştirel bir bakış: Problemler, beklentiler ve çözüm önerileri [A critical view to school experience and application of teaching
courses: Problems, expectations and solution suggestions]. Bartın University Journal of Faculty of Education, 4(2), 433-451.
https://doi.org/10.14686/buefad.v4i2.1082000250
Kitzinger, J. (1995). Qualitative research: Introducing focus groups. BMJ, 311(7000), 299-302. https://doi.org/10.1136/bmj.311.7000.299
Kvale, S. (1994). Interviews: An introduction to qualitative research interviewing. Sage Publications.
Özden, Y. (1999). Eğitimde dönüşüm eğitimde yeni değerler [Transformation in education, new values in education]. Pegem A
Yayınları.
Özkan, H., Albayrak, M., & Berber, K. (2005). Öğretmen adaylarının ilköğretim okullarında yaptıkları öğretmenlik uygulamasının
yetişmelerindeki rolü [The role of teacher candidates’ teaching practice in primary schools in their upbringing]. Milli Eğitim
Dergisi, (33), 168.
Singh, A., Doyle, C., Rose, A., & Kennedy, W. (1997). A reflective internship and the phobia of classroom management (forthcoming).
Australian Journal of Education, 41(2). https://doi.org/10.1177%2F000494419704100202
Sönmez, V. (2000). Eğitimin tarihsel temelleri öğretmenlik mesleğine giriş [Historical foundations of education introduction to the
teaching profession]. Anı Yayıncılık.
Şahin, M. (2016). Pedagojik formasyon eğitimi uygulama dersinin (staj) öğretmen adaylarının görüşleri açısından değerlendirilmesi
[The evaluation of views of prospective of teacher of practice course (internship) of pedagogical formation education].
Akademik Sosyal Araştırmalar Dergisi, 4 (22), 32-49. http://dx.doi.org/10.16992/ASOS.978
Şimşek, H., & Yıldırım, A. (2011). Sosyal bilimlerde nitel araştırma yöntemleri [Qualitative research methods in the social sciences].
Seçkin Yayıncılık.
Şişman, M. (2008). Eğitimin temel kavramları [Basic concepts of education]. Pegem Akademi.
Topbaş, E. (2001). Türkiye ve Fransa’da sınıf öğretmeni yetiştiren kurum programlarının karşılaştırılması [The comparison of curricula
of elementary education teacher training schools in Turkey and France]. (PhD thesis). Abant İzzet Baysal Üniversitesi, Bolu.
Ünal, I. (2011). Öğretmenliğe ve öğretmen yetiştirmeye ilişkin ekonomi politik bir çözümleme [A political economy analysis on
teaching and teacher training]. In S. A. Kilimci (Ed.), Türkiye’de Öğretmen Yetiştirme (pp. 3-23). Pegem Akademi.
Yalın Uçar, M. (2012). Öğretmenlik uygulamasına ilişkin durum çalışması [Case study on teaching practice]. Kuram ve Uygulamada
Eğitim Bilimleri. 12 (4), 2637-2660.
YÖK (1998). Fakülte-okul işbirliği [Faculty-school cooperation]. YÖK/Dünya Bankası Yayınları.
190
COURSES
ISSN 2538–7138 /Online/ (pp. 179-191)
Appendix:
This study is carried out to determine the views of pre-service teachers about teaching practice courses in
undergraduate programs of education faculties. If you allow, our interview will be recorded and will take approxi-
mately 30 minutes. Thank you in advance for your sincere answers.
1. Department:
Science education
Computer education
2. Gender:
Female
Male
3. Age:
21-24
25 and over
Interview Questions
1. Is the teaching practice course sufficient in terms of learning gains and content? Can you please explain?
2. Explain if the teaching practice courses are sufficient in terms of semesters and credits covered in the
programs?
3. What type of studies do practitioners conduct in order to measure and evaluate the success of the
teaching practice course? Do you think these studies are adequate?
4. Can you please specify if there are any problems you encountered during the teaching practice course?
5. What kind of studies can be done to make the teaching practice course more qualified and richer? Can
you please explain?
Received: February 19, 2022 Revised: March 16, 2022 Accepted: April 07, 2022
Cite as: Bulut, A., Kaban, A., & Bilen, Ö. (2022). Pre-service science and computer teachers’ views on teaching practice courses.
Journal of Baltic Science Education, 21(2), 179-191. https://doi.org/10.33225/jbse/22.21.179
Ayhan Bulut PhD, Assistant Professor, Bayburt University, 69000 Merkez/

(Corresponding author) Bayburt, Turkey.
E-mail: ayhanbulut44@hotmail.com
Abdullatif Kaban PhD, Assistant Professor, Ataturk University, 25240 Erzurum, Turkey.
E-mail: abdullatif.kaban@gmail.com
Ömer Bilen PhD, Assistant Professor, Ataturk University, 25240 Erzurum, Turkey.
E-mail: omerbilen76@gmail.com
191
SELF-REGULATED LEARNING
STRATEGIES IMPACT FOURTH-
GRADE STUDENTS’ POSITIVE
ISSN 2538-7138 /Online/
Abstract. For long years, researchers in the

field believed that the development of self-
regulated learning (SRL) behaviors of stu- Busra Cengiz-Istanbullu,
dents could only be initiated around middle Gonul Sakiz
school years. Nowadays, a growing number
of research around the world attempt to
challenge this belief by demonstrating that
SRL behaviors and strategies can emerge
in students at an earlier age than expected
through proper training and eventually
affect students’ positive academic outcomes Introduction
in the learning process. The purpose of this
study was to examine the impact of the In today’s world, the information flow in our lives is mesmerizingly fast.
use of self-regulated learning strategies Therefore, catching personally relevant and necessary information from the
(SRLS) on fourth-grade students’ academic
flow and eliminating the rest, staying focused and protecting our well-being
self-efficacy, science motivation, academic
are critical issues and require the adoption of the high level of self-regulatory
worry, and achievement in a primary school
science classroom. A pre-test – post-test con-
skills. Regardless of developmental differences, self-regulation exists at vari-
trol group quasi-experimental design was ous levels and qualities to attain personal goals (Zimmerman, 2001). Self-
used. The participants were 39 fourth-grade regulation involves “self-generated emotions, thoughts and actions that one
students from a private primary school in can plan and adapt to achieve the goal” (Zimmerman, 1999, p. 14).
Istanbul, Turkey. The SRLS (goal-setting, Individuals who can efficiently self-regulate their actions participate
planning, note-taking, monitoring and self- vigorously in the learning process and use the metacognitive, motivational,
evaluation) used in the study were explicitly and behavioral skills needed to accomplish their goals (Zimmerman, 1989).
taught to the students in the experimental Self-regulated individuals evaluate their internally driven conscious and/or
group. The data were collected using two unconscious behaviors and put an effort to change their reactions to the
self-report questionnaires and an achieve-
learning process (Vohs & Baumeister, 2004). They monitor their motivation
ment test. A 2x3 Split-plot ANOVA test was
and behavior to achieve their predetermined goals (Pintrich, 2000). Fa-
used for data analyses. The results revealed
that the use of SRLS significantly positively
cilitating learning affects self-development positively (Zimmerman, 1990a).
impacted students’ perceived academic self- Self-regulated learners take responsibility for their own learning, use certain
efficacy, science motivation and academic strategies, examine how those strategies affect the outcomes, and, eventually,
achievement; and, significantly negatively develop their own learning strategies (Gömleksiz & Bozpolat, 2012; Haddad,
affected their academic worry. The results 2016; Zimmerman, 1990a). Zimmerman (1990a) developed a cyclical structure
of the retention tests, however, showed of SRL which includes three basic phases: forethought, performance, and
the necessity of continuous stimulation of self-reflection. In the forethought phase, learners determine their goals and
SRLS to obtain optimal benefits for student choose the most appropriate strategies to achieve these goals; focus on the
outcomes. situations that motivate them during the process. In the performance phase,
Keywords: academic achievement,
they apply the strategies chosen and control the process through observa-
academic self-efficacy, academic worry,
tions. In the self-reflection phase, the learning process is completed, and the
science motivation, self-regulated learning
strategies (SRLS)
entire process is evaluated.
Being a self-regulated learner requires the adoption of appropriate self-
regulatory strategies, which are the certain actions and processes individuals
Busra Cengiz-Istanbullu use to succeed (Zimmerman, 1989). The utilization of self-regulated learning
Doga College, Turkey strategies (SRLS) improves individuals’ learning skills (Ader, 2014). Knowing
Gonul Sakiz
Marmara University, Turkey
the proper SRLS strategies, however, is not enough. Learners also need to
believe that the strategies are beneficial and that they can competently uti-
lize them (Cerezo et al., 2019). According to Bandura (1991) and Zimmerman
192
ISSN 1648–3898 /Print/ SELF-REGULATED LEARNING STRATEGIES IMPACT FOURTH-GRADE STUDENTS’ POSITIVE
ISSN 2538–7138 /Online/ (pp. 192-206)
(1990b), during the learning process, individuals go through many trial-and-error processes to determine the most
appropriate strategy to use. The selection of SRLS depends on person, environment, and behaviors. Students who
believe in themselves and their abilities, are more likely to use their strategic knowledge in appropriate situations,
so they set their goals and make appropriate plans.
There are a wide range of SRLS such as self-evaluation, transformation, goal-setting and planning, structuring
the environment, gathering information, taking notes and monitoring, repeating and memorizing, and seeking help
(Zimmerman & Martinez-Pons, 1986). In the present study, four major SRLS were examined: goal-setting, planning,
note-taking, monitoring and self-evaluation. These strategies are embedded in the forethought, performance, and
self-reflection phases. Limiting the use of diverse strategies with youngsters may help sustain their motivation even
under the condition of failed attempts toward successful outcomes (Cerezo et al., 2019).
Goal-setting and planning involve guessing the results of certain actions and making action plans using
determined goals and achieving the desired results (Bandura, 1999). Individuals motivated to achieve their goals
try more intensely to reach their goals by guiding their behaviors (Bandura, 1998). Students set various goals for
themselves to succeed in the learning period. The subgoals used in the goal-setting stage are like the control
points to reach the major goal (Zimmerman & Cleary, 2009). Research conducted in the field with the participation
of students from different developmental levels shows that the goal-setting strategy positively affects students’
learning outcomes (Martin & Elliot, 2016; Öztürk & Çakıroğlu, 2021; Yusuff, 2018).
Planning requires the consideration of time, environmental factors and expected behavior in relation to the
given task (Ader, 2014; Zimmerman & Cleary, 2009). Students determine the appropriate strategy to use during
planning. If the predetermined strategy is not suitable for achieving the purpose, students may change it with a
more suitable one in the process (Lavasani et al., 2011). Research shows that using the planning strategy in the
course of learning activities is positively related to students’ advanced learning outcomes (Lavasani et al., 2011;
Öztürk & Çakıroğlu, 2021; Wong et al., 2021).
Note-taking and monitoring strategies involve recording the actions and keeping track of the learning process.
When note-taking and monitoring are not used, individuals must rely on their memory to explain their achieve-
ment or failure, and memory may mislead individuals (Schunk & Zimmerman, 2003). The process of monitoring is
very effective in setting realistic goals and evaluating the progress in the process of achieving the predetermined
goals (Bandura, 1991). Without monitoring, individuals can be less certain about how well they learn. Monitoring
allows them to see their progress during learning and increases their academic self-efficacy (Schunk & Ertmer,
2000). Research shows that using the self-monitoring strategy during learning period positively affects students’
academic performance (Arslantas & Kurnaz, 2017; Guo, 2022; Hsu, 2020).
Self-evaluation involves a regular and systematic comparison of goals and obtained outcomes (Sakız & Yetkin-
Özdemir, 2014). Individuals shape their motivation and behavior by evaluating the positive and negative results
received through the actions (Bandura, 1999). In the learning process, students are in a continuous assessment
process and choose the most appropriate and easiest strategy to use. According to Zimmerman (2000), there are four
criteria used in the self-evaluation process. These include learning, previous performance, normative comparison,
and in-team function. For example, a score obtained from an achievement test, progress made compared to previ-
ous performance, performance level in comparison to others and efficiency in completing a given responsibility
in teamwork can be considered as self-evaluative actions. Studies demonstrate that the use of the self-evaluation
strategy in the learning process results in higher academic outcomes in students from different developmental
periods (Dignath et al., 2008; Lavasani et al., 2011; Raković et al., 2022).
Research Problem
Self-regulated learning strategies (SRLS) can be acquired easily when strategy instruction is placed in educa-
tion (Dignath et al., 2008; Guo, 2022; Weinstein et al., 2011). Self-regulated learning strategies (SRLS), have long
been examined in relation to a wide range of variables including academic achievement (Broadbent & Poon, 2015;
Cerezo et al., 2019; De La Fuente et al., 2020; Öztürk & Çakıroğlu, 2021; Vettori et al., 2020), motivation (Schraw et
al., 2006; Pintrich & De Groot, 1990; Pravesti et al., 2020; Uka & Uka, 2020), metacognitive skills (Bahri & Corebima,
2015; Schraw et al., 2006; Senler & Vural-Sungur, 2014), problem-solving skills (Ahghar, 2012; ElAdl & Polpol, 2020)
and gender (Kara-Ertürk & Gönen, 2015; Shoval et al., 2021). The overall results of the related studies show that
the use of SRLS contributes positively to academic achievement, motivation, metacognitive skills, learning and
problem-solving skills. Nevertheless, the research on SRLS commonly involved students in middle school (Chen &
193
(pp. 192-206) ISSN 2538–7138 /Online/
Cleary, 2009; Cheng, 2011; Daniela, 2015; Uka & Uka, 2020), high school (ElAdl & Polpol, 2020; Kartalcı & Demircioğlu,
2018; Sun & Wang, 2020; Vettori et al., 2020; Zimmerman & Kitsantas, 1997) or university settings (Aizpurua et al.,
2018; Anthonysamy et al., 2020; De La Fuente et al., 2020; Hsu, 2020; Öztürk & Çakıroğlu, 2021; Sahin et al., 2020).
Because SRL skills and strategy development have quite a long time been perceived as requiring high-level
thinking skills, and pre-school and primary school students would be insufficient in this regard, the relevant research
with young age groups remained highly limited (Veenman & Spaans, 2005; Zimmerman 1990). Contrary to com-
mon expectations, developing SRL skills in young children is crucially important (Salisch et al., 2015). Therefore, the
contemporary research studies in the field endeavor to confront the common thought related to the emergence
period of SRL behaviors in students and attempt to provide evidence supporting that students can develop SRL
skills and strategies at an earlier age than expected through the implementation of proper training programs
(Lenes et al., 2020; Pas et al., 2021; Perry & VandeKamp, 2000). Kangas et al. (2015) discussed the importance of
self-regulation in early childhood education and demonstrated the specific SRLS used by students. Dignath et al.
(2008) revealed that instructional programs for SRL at the primary school level are highly effective in the learn-
ing process and, hence, primary school students can acquire self-regulated behaviors quite efficiently. They also
emphasized that older students may possess some strategies difficult to change due to their previous experiences
affecting their current state of knowledge, and, hence, younger students may be more open to acquiring new
strategies because their existing ones are limited. In this process, students should be guided by teachers on how to
benefit most effectively from the strategies during learning (Broadbent & Poon, 2015). If teachers inform students
about how and when to use strategies and practice these procedures in class, the outcomes may potentially be
more favorable for students (Weinstein et al., 2011). Designing the classroom instruction in such a way that would
promote students’ learning and practice of SRLS may strengthen the projected positive academic outcomes for
students (Soltani & Askarizadeh, 2021).
Research Focus
In the current study, the use of SRLS was examined in relation to four critical outcomes for student success in
primary schools. These include academic self-efficacy, science motivation, academic worry, and academic achieve-
ment. Academic self-efficacy can be defined as individuals’ beliefs in what and to what extent they can achieve in
the learning process (Bandura, 1999). Learning environments should be organized to boost students’ academic
self-efficacy (Sakız, 2013). Increased self-efficacy fosters higher persistence and endurance toward difficulties
and leads to higher levels of achievement (Schunk & Meece, 2006). It also encourages a positive attitude towards
lessons linking to higher achievement (Huang, 2016; Roick & Ringeisen, 2017). Research shows that using SRLS
enhances students’ self-efficacy through increased knowledge (Cerezo et al., 2019). If SRL training is provided at
an early age, children’s participation skills, self-esteem and self-efficacy are improved notably (Kangas et al., 2015).
Motivation can be perceived as the powerful force driving the person to achieve a certain goal (Çavaş-
Huyugüzel & Çavaş, 2014). Highly motivated students have much higher energy to succeed in the learning process
(Maison et al., 2019). Negative attitudes gained at an early age are difficult to change, so developing motivation in
young children and achieving success as a result of motivation, positively affect an individual’s life (Patrick et al.,
2008). Research supports that the use of SRL in science class positively affects students’ science motivation and
achievement (Maison et al., 2019).
Worry can be defined as a cognitive process in which a person rethinks the negative consequences of an
existing condition and focuses on the part of the potential consequences that threatens the person (Vasey et al.,
1994). Worry begins in pre-school and can diversify through a lifetime (Vasey et al., 1994). Students’ thinking of
failure related to the science course causes academic worry (Kağıtçı & Kurbanoğlu, 2013; Kurbanoğlu & Nefes, 2016).
A high level of worry also negatively affects students’ problem-solving skills (Parkinson & Creswell, 2011). The use
of SRLS helps manage high levels of worry (Weinstein et al., 2011). For individuals who use SRLS effectively in the
learning process and are aware of their potential, the level of academic worry decreases, and learners develop a
positive attitude toward lessons (El-Adl & Alkharusi, 2020; Mesurado et al., 2018).
Research Purpose
From early childhood to adulthood, self-regulation of behaviors is possible and highly beneficial for achieving
independent learning. Although research shows that the instruction and the use of self-regulated learning strate-
194
ISSN 2538–7138 /Online/ (pp. 192-206)
gies in the early years of schooling is crucially important for youngsters (Dignath et al., 2008; Kangas et al., 2015),
there has not been much experimental research concerning this developmental level. The purpose of this study
was to examine the impact of using SRLS on fourth-grade students’ academic self-efficacy, science motivation,
academic worry, and achievement in a primary school science classroom. Hypothetically, it was expected that
students who were trained to use SRLS in the learning process would demonstrate higher self-efficacy beliefs and
science motivation, lower academic worry and greater academic achievement compared to those who have not
received any training on the use of SRLS.
Within the scope of this study, the science course was specifically chosen. The common purpose of science
teaching globally is to raise individuals who can solve problems, produce, manage, and control their own learning;
have critical thinking, are determined, have high communication skills, empathic ability and contribute to society
(MEB, 2018). All these expected qualifications require holding high self-regulatory skills. Besides, as a worldwide
endeavor, advancement in science achievement at all developmental levels is highly prioritized. In international
TIMSS (Trends in Mathematics Science Study) exams which include the assessment of fourth-grade students’
science proficiency levels across countries, Turkey was ranked as 36th among 50 countries in 2011, 35th of 47 in
2015 and 19th among 58 countries in 2019 (MEB, 2020) in science achievement. Even though there is an increase
in the rankings over time, the students’ level of advanced proficiency in science was 4% in TIMSS 2015 and 12%
in TIMSS 2019, and, hence, the desired outcomes have not been achieved so far. Therefore, more research in the
field is needed to enquire about the potential ways to improve students’ science proficiency in Turkey at all grade
levels. The present study would hopefully contribute to meeting this strong need.
General Background
In this study, a pre-test – post-test control group quasi-experimental design was used. In most educational
research, a random assignment of participants is not achievable due to concerns related to disturbing existing
educational conditions (Creswell, 2009). In this quasi-experimental research, students were not assigned to the
groups randomly. The equivalency in groups, however, was assured through pre-tests and acquiring similar learn-
ing conditions. The study was carried out in the fall term of the academic year 2017-2018. All permissions were
obtained from the Istanbul Provincial Directorate of National Education. The rules stated in the Higher Education
Institutions Scientific Research and Publication Ethics Directive were followed during the execution of the study.
Participants
The participants of this study were 39 fourth-grade students (nexp = 20 and ncont = 19) from two classrooms in
a private primary school located in the European side of Istanbul, Turkey. The average class size in private schools
in Istanbul is around 20, which was also the case in this study. The convenient sampling method using easily acces-
sible samples was chosen in this study. Both genders were almost equally represented in the groups. Overall, 49%
of the participants were female and the average age was 9.15 with a standard deviation of .49. The participants
were widely from middle-class families. Of all the parents, 10% held secondary or lower education, 28% held high
school degrees and 62% held undergraduate or graduate education degrees. During the study, students’ natural
classroom environments were not changed. Before the implementation, students were informed about the study,
the voluntary nature of their participation and their freedom to leave at any time, and the confidentiality of their
responses to the scale items.
Measures
Science and Technology Course Self-Efficacy Scale. Students’ self-efficacy toward science courses was measured
using Science and Technology Course Self-Efficacy Scale developed by Tatar et al. (2009). This scale included 27
items (15 positive and 12 negative items). All items were measured with a 5-point Likert type scale ranging from
1 (completely disagree) and 5 (completely agree). The sample items were as follows: “I am not very sure that, I can
understand difficult scientific and technologic concepts” and “I can accomplish science and technology projects
successfully.” The internal consistency reliability was calculated as .80.
195
(pp. 192-206) ISSN 2538–7138 /Online/
The Motivation for Science Learning Scale. Student motivation toward science learning was measured using The
Motivation for Science Learning Scale developed by Tuan et al. (2005) and adapted to Turkish by Yılmaz and Cavaş
(2007). The 5-point Likert type scale consisted of 33 items, ranging from 1 (completely disagree) to 5 (completely
agree). The sample items were as follows: “When I come across science concepts that I don’t understand, I still make
an effort to understand them” and “when learning new science concepts, I make an effort to understand them.”
The internal consistency reliability estimate for this scale was .87.
Science and Technology Lesson Worry Scale. Science and Technology Lesson Worry Scale developed by Kağıtçı
and Kurbanoğlu (2013) was used to determine students’ level of worry for the science course. The 5-point Likert type
scale consisted of 18 items, ranging from 1 (completely disagree) to 5 (completely agree). The sample items were as
follows: “Even thinking about attending the science course makes me worried” and “watching the teacher solving a
science problem at the board makes me worried”. The internal consistency reliability estimate for this scale was .85.
Science Achievement Test. Science Achievement Test developed by Üçüncü (2019) was used to determine the
achievement levels of students in the Introduction to Matter Unit. The test included 20 multiple choice questions
consisting of four options. The correct answers were scored as 1 and the wrong answers as 0. The internal consis-
tency reliability estimate for this test was calculated as .72.
Procedure
The total duration of the study, including the pre- and post-test implementations, was 10 weeks. The experi-
mental part of the study took place over three-hour class periods for eight weeks in the science unit of Introduction
to Matter. The retention test was applied four weeks after the post-test. In the control group, only pre-, post- and
retention tests were applied. Considering students’ developmental levels and the length of the scales, each scale
was applied during a 40-minute class period. The scale applications were carried out during the free activity times
in school without disrupting students’ regular course flow.
The SRLS used in the experimental group (goal-setting and planning, note-taking and monitoring and self-
evaluation) were explained during the activities. For each strategy instruction, two weeks were allocated. During the
strategy instruction of goal-setting and planning, first, a sample goal-setting map was handed out to the students
and the concept of goals as well as close and distant goals were discussed all together and detailed explanations
and several examples were provided by the teacher. The students were then asked to create close, distant, and
sub-goals for the Introduction to Matter Unit with the guidance of their teacher. In addition, the stories with heroes
that students can identify themselves with involving goal-setting and planning strategies were read by the teacher
and the stories were deepened with related videos and pictures. Finally, goal-setting maps were developed by the
students for each subject.
The students were informed about the nature of note-taking and monitoring strategies. They were explained
that the more they became aware of their actions and their results in the learning process, the fewer mistakes they
would make and the more involved they would become in their learning. The students were provided with a self-
monitoring form and explanations on how to use it during the course. Using the form, they took notes and made
markings to check their active participation in the course every 10 minutes during a 40-minute lesson. 10-minute
periods were reminded by the teacher.
During self-evaluation strategy training, the students were informed about its characteristics and benefits
for learning. The related stories were shared, and discussions were carried out for use of this strategy during the
learning process. At the end of the lesson, the students were requested to fill in a self-evaluation form in which
they assessed what and how they did in the lesson. They provided information on what they learned, how they felt,
which steps they followed when they did not understand something, how they reacted when they made mistakes
and how they used the allocated time and so on.
All essential steps for ensuring the validity and the reliability of the study were properly performed throughout
the research. Following the development of the SRL tasks, Lawshe (1975) test was conducted for the purpose of
acquiring content validity. The draft document was sent to 10 specialists. The following six measurement criteria
were submitted to them for evaluation: purpose, students’ prior knowledge, appropriateness of the allocated time
196
ISSN 2538–7138 /Online/ (pp. 192-206)
for each application, classroom organization, teacher-student roles, and difficulty level of each task. The special-
ists coded each task for each criterion as suitable, not suitable, or needs revision. Using the formula, the Content
Validity Ratio (CVR) was calculated and found to be .96, which meets the validity criteria for 10 experts (> .80, Ayre
& Scally, 2014). Using all other comments and suggestions, the final version of the SRL tasks was developed. All
scales used in the present study were selected from those with adequate and substantial information regarding
the validity and the reliability procedures.
Data Analysis
Before the experimentation, the equivalency of the groups was tested using t-test analyses. As shown in Table
1, the students in both groups were found equal in terms of examined variables [academic self-efficacy (t37 = -.62,
p = .80), science motivation (t37 = 1.94, p = .48), academic worry (t37 = 1.86, p = .07), and academic achievement
(t37 = -.82, p = .60)].
Table 1
The Comparison of the Pre-test Scores of Two Groups in Relation to Academic Self-Efficacy, Science Motivation, Academic
Worry and Academic Achievement Variables
Groups Dependent Variables N 𝑥̅ SD t df p
Experimental Gr. 20 3.08 .25 -.62 37 .80

Academic Self-Efficacy
Control Gr. 19 3.14 .26
Experimental Gr. 20 3.42 .29 1.94 37 .48
Science Motivation
Control Gr. 19 3.23 .32
Experimental Gr. 20 2.37 .40 1.86 37 .07
Academic Worry
Control Gr. 19 1.98 .82
Experimental Gr. 20 .49 .14 -.82 37 .60
Academic Achievement
Control Gr. 19 .53 .30
Following the intervention, the preliminary descriptive analyses were conducted. For further analyses, the Split-
plot ANOVA tests were applied to enable comparing and contrasting between and within groups simultaneously.
Split-plot ANOVA (mixed design) is a technique used to compare averages in repeated measurements (Chartier &
Cousineau, 2011). The data were intermittent and continuous, normally distributed and the variances showed a
homogeneous distribution. Using Cohen’s (1988) criteria, effect sizes (ƞp2) were also calculated for each test and the
cut-point values of .01, .06 and .14 were interpreted as small, medium, and large effects, respectively. The significance
level for all statistical analyses was determined as .05. Bonferroni adjustment was used for multiple comparisons.
Research Results
The Preliminary Results
The descriptive analyses of the pre-, post- and retention test scores of both groups in relation to given variables
are provided in Table 2. The preliminary results revealed that the experimental group students’ levels of academic
self-efficacy, science motivation and academic achievement were likely to increase at the post-test compared to
the pre-test, and then, slightly decrease at the retention test compared to the post-test. Students’ academic worry
levels in the same group tended to gradually decline over time.
197
(pp. 192-206) ISSN 2538–7138 /Online/
Table 2
Descriptive Analyses of Groups in Relation to Academic Self-Efficacy, Science Motivation, Academic Worry and Academic
Achievement Levels at Pre-, Post- and Retention Tests
Pre-test Post-test Retention test

Groups Dependent Variables N
𝑥̅ SD 𝑥̅ SD 𝑥̅ SD
Experimental Gr. 20 3.08 .25 3.31 .21 3.13 .28

Academic Self-efficacy
Control Gr. 19 3.14 .26 3.09 .19 2.76 .58
Experimental Gr. 20 3.42 .29 3.52 .36 3.48 .48
Science Motivation
Control Gr. 19 3.23 .32 3.21 .24 2.99 .73
Experimental Gr. 20 2.37 .40 2.04 .57 1.57 .74
Academic Worry
Control Gr. 19 1.98 .82 1.73 .71 1.85 .92
Experimental Gr. 20 .49 .14 .82 .11 .73 .11
Academic Achievement
Control Gr. 19 .53 .19 .70 .11 .62 .19
As shown in Table 2, the control group students’ levels of academic self-efficacy and science motivation were
seemingly demonstrating a slight but continuous fall throughout all testing periods. Students’ levels of academic
worry in the control group were inclined to decrease at the post-test but then slightly increase. In terms of academic
achievement outcomes, similar to the experimental group, an increase in the post-test scores was followed by a
minor decrease in the retention test scores.
Split-Plot ANOVA Results
Following the descriptive analyses, a 2x3 Split-plot ANOVA test was performed. The results are provided in
Table 3 and illustrated in Figure 1. In terms of academic self-efficacy, there was a statistically significant difference
between groups with a high effect size, in favor of the experimental group [F(1,37) = 6.46, p = .02, ƞp2 = .15]. The main
effect of time was significant and the effect size was high [F(2,74) = 7.32, p = .001, ƞp2 = .17]. The post hoc analyses
showed that the difference in time variable was caused by the difference between post- and retention test scores
(p = .003). The group and time interaction effect was significant with a medium effect size [F(2,74) = 4.98, p = .009,
ƞp2 = .12]. The post hoc analyses showed that while there was no significant difference between groups at pre-test
(p = .54), they differed significantly at post- and retention test periods (p = .003 and p = .02, respectively), in favor
of the experimental group. In the experimental group, a significant difference was only detected between pre- and
post-test scores (p = .003). In the control group, however, significant differences were observed in two time points,
between pre-and retention test scores and post- and retention test scores (p = .007 and p = .008, respectively). The
graphical representation is provided in Figure 1a.
In terms of science motivation levels, there was a statistically significant difference between groups with a
high effect size, in favor of the experimental group [F(1,37) =9.72, p = .004, ƞp2 = .21]. The post hoc analyses showed
that an obtained significant difference between groups was caused by the differences at post- and retention test
periods (p = .004 and p = .02, respectively). The main effect of time was not detected [F(2,74) = 1.42, p = .25]. The
group and time interaction effect was not significant as well [F(2,74) = 1.72, p = .19]. The graphical representation
is given in Figure 1b.
198
ISSN 2538–7138 /Online/ (pp. 192-206)
Table 3
Split-plot ANOVA Results for the Effects of the Independent Variables (Group and Time) on the Dependent Variables
Dependent Variables Source of Variance SS df MS F p ƞp2
Between Groups
Intercept 1117.26 1 1117.26 7902.05 < .001 .99
Group .91 1 .91 6.46 .02 .15
Error 5.23 37 .14
Academic
Within Groups
Self-efficacy
Time 1.29 2 .65 7.32 .001 .17
Group*Time .88 2 .44 4.98 .009 .12
Error 6.54 74 .09
Between Groups
Intercept 1281.00 1 1281.00 3885.91 < .001 .99
Group 3.20 1 3.20 9.72 .004 .21
Error 12.19 37 .33
Science Motivation Within Groups
Time .35 2 .18 1.42 .25 .04
Group*Time .43 2 .21 1.72 .19 .04
Error 9.17 74 .12
Between Groups
Intercept 434.01 1 434.01 518.52 < .001 .93
Group .56 1 .56 .67 .42 .02
Error 30.97 37 .84
Academic Worry Within Groups
Time 4.31 2 2.16 6.25 .003 .15
Group*Time 2.58 2 1.29 3.75 .03 .09
Error 25.51 74 .35
Between Groups
Intercept 50.13 1 50.13 1198.37 < .001 .97
Group .11 1 .11 2.70 .11 .07
Error 1.54 37 .04
Academic Achievement Within Groups
Time 1.26 2 .63 49.99 < .001 .58
Group*Time .17 2 .08 6.62 .002 .15
Error .93 74 .01
Academic worry levels of the students in both groups showed no statistically significant difference [F(1,37) = .67,
p = .42]. On the other hand, as presented in Table 3, a main effect of time variable was detected with a high ef-
fect size [F(2,74) = 6.25, p = .003, ƞp2 = .15]. The post hoc analyses showed that the difference in time variable was
caused by the difference between pre- and retention test scores (p = .002). The interaction effect between group
and time was also significant with a medium effect size [F(2,74) = 3.75, p = .03, ƞp2 = .09]. The post hoc analyses
revealed that the obtained difference was caused by the variations in scores between pre- and retention tests in
the experimental group (p < .001). The academic worry levels of the students in the experimental group showed
199
(pp. 192-206) ISSN 2538–7138 /Online/
a continuous fall throughout the testing periods. In the control group, however, a moderate decrease in students’
academic worry scores in the post-test was followed by a slight increase in the retention test, indicating no notice-
able change overall. The graphical representation is provided in Figure 1c.
Figure 1
Graphical Representations of the Effects of the Independent Variables on the Dependent Variables
Note: Independent Variables - Group: experimental vs. control; Time: 1- pre-test (time 1), 2- post-test (time 2), 3- retention test (time 3)
Dependent Variables - (a) academic self-efficacy, (b) science motivation, (c) academic worry, (d) academic achievement
As presented in Table 3 and illustrated in Figure 1d, academic achievement levels of the students in both
groups showed no statistically significant difference [F(1,37) = 2.70, p = .11]. On the other hand, in terms of
time variable, a statistically significant difference with a high effect size was detected [F(2,74) = 49.99, p < .001,
ƞp2 = .58]. The post hoc analyses revealed that students’ academic achievement levels differed significantly at
all three time points (p < .001). The group and time interaction effect was also significant with a high effect size
[F(2,74) = 6.62, p = .002, ƞp2= .15]. Although, there was no difference in groups at pre-test (p = .42), differences in
scores were detected at post and retention tests, in favor of the experimental group (p = .003 and p = .04). In the
experimental group, significant differences were detected between pre- and post (p < .001), pre- and retention
(p < .001), and post- and retention test scores of (p = .007). In the control group, significant differences were
found between pre- and post and post- and retention test scores (p < .001 and p = .02). No statistically signifi-
cant difference was found between pre- and retention test scores of the control group students, meaning that
students basically had not adequately remembered what they had learned earlier (p = .12). Overall, academic
achievement scores of the students in the experimental group showed a greater increase at all time intervals
compared to those in the control group.
Discussion
The purpose of this study was to examine the impact of the use of SRLS on fourth-grade students’ aca-
demic self-efficacy, science motivation, academic worry, and academic achievement outcomes in primary
school science classrooms. The major interest was to see whether the effect of SRLS on student outcomes at
200
ISSN 2538–7138 /Online/ (pp. 192-206)
the primary school level would produce similar results with the studies conducted at secondary school or
higher levels. The obtained results were in the predicted directions and coherent with the existing research.
The comparisons of the students’ scores between the experimental and the control groups in terms of self-
efficacy, science motivation, academic worry and academic achievement showed statistically significant dif-
ferences in favor of the experimental group. The obtained differences between groups were consistent with
the previous studies (Mesurado et al., 2018; Öztürk & Çakıroğlu, 2021; Schraw et al., 2006).
The use of SRLS in the learning process significantly positively impacted students’ academic self-effi-
cacy. This outcome was coherent with the existing research (Cerezo et al., 2019; Lavasani et al., 2011; Roick
& Ringeisen, 2017; Schraw et al., 2006; Zimmerman & Kistantas, 1997). The link between the use of SRLS in
the learning process and academic self-efficacy might be twofold. The present study revealed that the SRLS
positively impacts academic self-efficacy. Correspondingly, higher self-efficacy may indeed trigger one’s
motivation to learn and practice appropriate learning strategies when needed.
In the current study, similar to the self-efficacy findings, it was found that the use of SRLS in the learning
process had a positive impact on students’ science motivation. This result was also in line with the relevant
research in the field (Lavasani et al., 2011; Soltani & Askarizadeh, 2021; Schraw et al., 2006; Pintrich & De
Groot, 1990; Pravesti et al., 2020). Motivation is a key factor triggering behavioral engagement and learning
and, eventually, leading to greater academic performance in any given discipline. Therefore, searching for
ways to improve students’ motivation in learning environments attracts researchers’ attention in the field.
The present study provided supporting evidence that one way to improve students’ motivation in class is to
expose students to SRLS in learning.
The present research showed that the use of SRLS significantly negatively impacted students’ academic
worry as supported by previous research (Mesurado et al., 2018; Morosanova & Fomina, 2017). Worry and
other related negative emotions set a big barrier to students’ learning and academic performance. Students’
emotions are affected by many variables in learning contexts. In recent years, motivation research has focused
more on academic emotions (Linnenbrink-Garcia & Pekrun, 2016; Reeve, 2018). The practice of proper SRLS
may successfully inhibit the arousal of negative emotions in classroom environments, as it was the case for
worry outcomes in the current study.
One of the most critical findings in the present study was related to academic achievement. The results
of the current study provide evidence that the adoption of SRLS in the learning process significantly advances
students’ achievement outcomes. Apparently, almost all students lose or fail to remember some information
they acquired throughout the learning process, but its magnitude matters the most. Essentially, it is impor-
tant that students should not arrive at where they have already started when learning new information, as it
was almost the case for the control group students in the present study. In the experimental group, however,
students’ academic achievement scores in the retention test differed significantly positively from the pre-test,
even though a slight fall in scores over time was detected. The use of SRLS stimulates students’ active engage-
ment in their own learning process. The positive connection between SRLS and academic achievement put
forth in the relevant studies worldwide as well (Cerezo et al., 2019; Cheng, 2011; Lenes et al., 2020; Núñez et
al., 2022; Öztürk & Çakıroğlu, 2021; Vettori et al., 2020). Using SRLS in science lessons supports the enrichment
of students’ self-regulatory skills (Velayutham et al., 2011) and results in higher academic success. Moreover,
in the current study, increased self-efficacy and motivation and decreased worry outcomes, emerged as a
result of the use of SRLS, potentially stimulated students’ academic performance as well.
The overall results showed that students’ self-efficacy and academic performance decreased slightly four
weeks after the implementation of the treatment, showing the importance of continuous exposure to strategies
to preserve positive outcomes (Vassallo, 2011; Weinstein et al., 2011). Nevertheless, students’ levels of worry
continued to drop over time, suggesting that the effect of SRLS instruction had a seemingly more lasting effect
on worry outcomes. More studies are needed to understand the psychological nature of these differences.
Students’ self-regulated learning behaviors and strategies, antecedents and consequences of these pro-
cesses, and related mechanisms affecting learning have long been studied in diverse contexts, predominantly
starting from middle schools. Using an experimental design, the presented research showed that the use
201
(pp. 192-206) ISSN 2538–7138 /Online/
of SRLS had a positive impact on students’ academic self-efficacy, science motivation, academic worry, and
academic achievement outcomes in a fourth-grade science classroom. The obtained results supported the
notion that the use of SRLS, starting at an earlier age than predicted, may hold a high potential to contribute
to students’ motivation and learning. As a concerning issue for parents and teachers worldwide, many students
struggle with acquiring proficiency in science and math in schools. SRLS training in educational contexts may
help students overcome their struggles in these highly critical disciplines, which play central roles most of
the time in reaching career goals.
This study provided evidence that strategy instruction essentially works and results in positive academic
outcomes for students in primary school classrooms. That being the case, primary school teachers need to
use SRLS more often in their classrooms and spread the knowledge about the value of the use of SRLS in
the learning process. Teachers should also provide guidance for students on their individual and collabora-
tive attempts at strategy use. In order for teachers to acquire sufficient knowledge and to design effective
implications of SRLS during the learning process, they need proper training as well. Developing pre- and in-
service teacher training programs promoting the use of SRLS would certainly contribute to students’ life-long
productiveness, achievement, and well-being.
Based on the design and the outcomes of the current research, several suggestions might be provided for
future research. The present research was carried out in one private school with the participation of a limited
number of students within a single discipline. Therefore, findings may not be generalizable to students in
different contexts. This limitation leads to the need for more research in primary schools with the inclusion
of diverse disciplines. The SRLS used in this study was limited to several strategies which were goal-setting,
planning, note-taking, monitoring, and self-evaluation. The use of a wide range of SRLS may enhance the
obtained outcomes. It is expected that a gradual increase in the number of SRLS that are explicitly instructed,
modelled, and practiced in classroom settings would support students’ positive motivational, emotional, be-
havioral and academic learning outcomes. In the current study, the experimental period was limited to eight
weeks, restraining acquiring adequate information on the potential long-term effects of strategy instruction.
Therefore, more longitudinal studies focusing on diverse cognitive, affective, and behavioral variables would
contribute to the relevant literature. Moreover, in future studies, gathering diverse data using mixed methods,
including qualitative data, may help researchers better understand the nature of the relations among given
variables and the specific roles that each strategy plays on student outcomes. The use of SRLS involves an
ongoing process outside the school. Given that, the potential effect of parental involvement in this process
is also highly important, requiring attention from researchers in the field. Obviously, research factors, will
continue to grow in the future. The obtained outcomes of the current study will hopefully stimulate more
research on students’ self-regulated learning behaviors and strategies in primary school contexts.
Acknowledgments
This research was part of a master’s thesis completed by the first author under the supervision of the second
author at Marmara University, Institute of Educational Sciences, in Istanbul, Turkey. The authors would like to thank
the participants and the school administration for their voluntary involvement in this study.
References
Ader, E. (2014). Akademik özdüzenlemede strateji gelişimi [Strategy development in academic self-regulation]. In G. Sakız
(Ed.), Özdüzenleme: Öğrenmeden öğretime özdüzenleme davranışlarının gelişimi, stratejiler ve öneriler [Self-regulation:
From teaching to learning – Development of self-regulated behaviors, strategies and suggestions] (pp. 54-77). Nobel.
Ahghar, G. (2012). Effect of problem-solving skills education on auto-regulation learning of high school students in Tehran.
Procedia-Social and Behavioral Sciences, 69, 688-694. https://doi.org/10.1016/j.sbspro.2012.11.462
Aizpurua, A., Lizaso, I., & Iturbe, I. (2018). Learning strategies and reasoning skills of university students. Revista de Psicodidáctica,
23(2), 110-116. https://doi.org/10.1016/j.psicoe.2018.02.002
202
ISSN 2538–7138 /Online/ (pp. 192-206)
Anthonysamy, L., Koo, A. C., & Hew, S. H. (2020). Self-regulated learning strategies in higher education: Fostering
digital literacy for sustainable lifelong learning. Education and Information Technologies, 25, 2393–2414.
https://doi.org/10.1007/s10639-020-10201-8
Arslantas, S., & Kurnaz, A. (2017). The effect of using self-monitoring strategies in social studies course on self-monitoring,
self-regulation and academic achievement. International Journal of Research in Education and Science, 3(2), 452-463.
https://doi.org/10.21890/ijres.327905
Ayre, C., & Scally, A. J. (2014). Critical values for Lawshe’s Content Validity Ratio: Revisiting the original
methods of calculation. Measurement and Evaluation in Counseling and Development, 47(1),79-86.
https://doi.org/10.1177/0748175613513808
Bahri, A., & Corebima, A. D. (2015). The contribution of learning motivation and metacognitive skill on cognitive learning
outcome of students within different learning strategies. Journal of Baltic Science Education, 14(4), 487-500.
https://dx.doi.org/10.33225/jbse/15.14.487
Bandura, A. (1991). Social cognitive theory of self-regulation. Organizational Behavior and Human Decision Processes, 50(2),
248-287. https://doi.org/10.1016/0749-5978(91)90022-L
Bandura, A. (1998). Exercise of personal and collective efficacy in changing societies. In A. Bandura (Ed.), Self-efficacy in changing
societies (pp. 1-45). Psychology Press.
Bandura, A. (1999). Social cognitive theory: An agentic perspective. Asian Journal of Social Psychology, 2(1), 21-41.
https://doi.org/10.1111/1467-839X.00024
Broadbent, J., & Poon, W. L. (2015). Self-regulated learning strategies & academic achievement in online higher education learning
environments: A systematic review. The Internet and Higher Education, 27, 1-13. https://doi.org/10.1016/j.iheduc.2015.04.007
Cerezo, R., Fernandez, E., Amieiro, N., Valle, A., Rosario, P., & Núñez, J. C. (2019). Mediating role of self-efficacy and usefulness
between self-regulated learning strategy knowledge and its use. Revista de Psicodidáctica (English ed.), 24(1), 1-8.
https://doi.org/10.1016/j.psicoe.2018.09.001
Chartier, S., & Cousineau, D. (2011). Computing mixed-design (Split-plot) ANOVA. Mathematica Journal, 13, 13-17.
https://doi.org/10.3888/TMJ.13-17
Cheng, E. C. K. (2011). The role of self-regulated learning in enhancing learning performance. The International Journal of
Research and Review, 6(1), 1-16.
Creswell, J. W. (2009). Research design: Qualitative, quantitative, and mixed methods approaches (3rd ed.). Sage Publications.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Academic Press.
Çavaş-Huyugüzel, P., & Çavaş, B. (2014). Fen eğitiminde duyuşsal özellikler: Tutum ve motivasyon [Emotional aspects of
science education: Attitude and motivation]. In Ş. Anagün, & N. Duban (Ed.), Fen bilimleri öğretimi [Teaching science]
(pp. 115-144). Anı Yayıncılık.
Daniela, P. (2015). The relationship between self-regulation, motivation and performance at secondary school students.
Procedia-Social and Behavioral Sciences 191, 2549 – 2553. https://doi.org/10.1016/j.sbspro.2015.04.410
De La Fuente, J., Sander, P., Kauffman, D. F., & Yilmaz Soylu, M. (2020). Differential effects of self- vs. external-regulation on
learning approaches, academic achievement, and satisfaction in undergraduate students. Frontiers in Psychology, 11,
2678. https://doi.org/10.3389/fpsyg.2020.543884
Dignath, C., Buettner, G., & Langfeldt, H. P. (2008). How can primary school students learn self-regulated learning strategies
most effectively? A meta-analysis on self-regulation training programmes. Educational Research Review, 3(2), 101-129.
https://doi.org/10.1016/j.edurev.2008.02.003
El-Adl, A., & Alkharusi, H. (2020). Relationships between self-regulated learning strategies, learning motivation and mathematics
achievement. Cypriot Journal of Educational Sciences, 15(1), 104-111. https://doi.org/10.18844/cjes.v15i1.4461
ElAdl, A. M., & Polpol, Y. S. (2020). The effect of self-regulated learning strategies on developing creative problem solving and
academic self-efficacy among intellectually superior high school students. International Journal of Psycho-Educational
Sciences, 9(1), 97-106.
Gömleksiz, M. N., & Bozpolat, E. (2012). Learner autonomy in foreign language learning in elementary school. Journal of World
of Turks, 4(3), 95-114.
Guo, L. (2022). The effects of self-monitoring on strategy use and academic performance: A meta-analysis. International Journal
of Educational Research, 112. https://doi.org/10.1016/j.ijer.2022.101939
Haddad, R. H. (2016). Developing learner autonomy in vocabulary learning in classroom: How and why can it be fostered?
Procedia-Social and Behavioral Sciences, 232, 784-791. https://doi.org/10.1016/j.sbspro.2016.10.106
Huang, C. (2016). Achievement goals and self-efficacy: A meta-analysis. Educational Research Review, 19, 119-137.
https://doi.org/10.1016/j.edurev.2016.07.002
Hsu, P. Y. (2020). Self-regulated learning strategies in technology-enhanced learning: teaching self-monitoring and English oral
presentation skills in graduate program. Proceedings of the 2020 11th International Conference on e-Education, e-Business,
e-Management, and e-Learning (pp. 25-29). https://doi.org/10.1145/3377571.3377579
Kağıtçı, B., & Kurbanoğlu, N. İ. (2013). Developing an anxiety scale for science and technology class: Reliability and validity
study. Journal of Turkish Science Education, 10(3), 95-107.
Kangas, J., Ojala, M. & Venninen, T. (2015). Children’s self-regulation in the context of participatory pedagogy in early childhood
education. Early Education and Development, 26(5-6), 847-870. https://doi.org/10.1080/10409289.2015.1039434
Kara-Ertürk, G., & Gönen, M. (2015). Examination of children’s self-regulation skill in terms of different variables. Journal of
Theory and Practice in Education, 11(4), 1224-1239.
203
(pp. 192-206) ISSN 2538–7138 /Online/
Kartalcı, S., & Demircioğlu, H. (2018). Self-evaluations of high school students regarding to own metacognitive behaviours in
problem solving. MATDER Mathematics Education Journal, 3(1), 19-31.
Kurbanoğlu, N. İ., & Nefes, F. K. (2016). Context-based questions in science education: their effects on test anxiety and science
achievement in relation to the gender of secondary school students. Journal of Baltic Science Education, 15(3), 382.
https://dx.doi.org/10.33225/jbse/16.15.382
Lavasani, M. G., Mirhosseini, F. S., Hejazi, E., & Davoodi, M. (2011). The effect of self-regulation learning strategies
training on the academic motivation and self-efficacy. Procedia-Social and Behavioral Sciences, 29, 627-632.
https://doi.org/10.1016/j.sbspro.2011.11.285
Lawshe, C. H. (1975). A quantitative approach to content validit y. Personnel Psychology, 28(4), 563-
575. https://doi.org/10.1111/j.1744-6570.1975.tb01393.x
Lenes, R., McClelland, M. M., ten Braak, D., Idsøe, T., & Størksen, I. (2020). Direct and indirect pathways from children’s early
self-regulation to academic achievement in fifth grade in Norway. Early Childhood Research Quarterly, 53, 612-624.
https://doi.org/10.1016/j.ecresq.2020.07.005
Linnenbrink-Garcia, L., Patall, E. A., & Pekrun, R. (2016). Adaptive motivation and emotion in education: Research
and principles for instructional design. Policy Insights from the Behavioral and Brain Sciences, 3(2), 228-236.
https://doi.org/10.1177/2372732216644450
Maison, M., Syahria, S., Syamsurizal, S., & Tanti, T. (2019). Learning environment, students’ beliefs, and self-regulation in learning
physics: Structural equation modeling. Journal of Baltic Science Education, 18(3). https://doi.org/10.33225/jbse/19.18.389
Martin, A. J., & Elliot, A. J. (2016). The role of personal best (PB) goal setting in students’ academic achievement gains. Learning
and Individual Differences, 45, 222-227. https://doi.org/10.1016/j.lindif.2015.12.014
MEB, (2018). Fen bilimleri dersi öğretim programı (İlkokul ve ortaokul 3.-8. sınıflar). [Science curriculum (Elementary and middle
school, Grade 3-8)]. Talim Terbiye Kurulu Başkanlığı.
MEB, (2020). TIMSS 2019 Ulusal matematik ve fen bilimleri ön raporu: 4. ve 8. sınıflar [National preliminary report of TIMSS 2019
mathematics and science results- Grades 4 and 8]. Milli Eğitim Bakanlığı Ölçme, Değerlendirme ve Sınav Hizmetleri
Genel Müdürlüğü.
Mesurado, B., Vidal, E. M., & Mestre, A. L. (2018). Negative emotions and behaviour: The role of regulatory emotional self-
efficacy. Journal of Adolescence, 64, 62-71. https://doi.org/10.1016/j.adolescence.2018.01.007
Morosanova, V. I., & Fomina, T. G. (2017). Self-regulation as a mediator in the relationship between anxiety and academic examination
performance. Procedia-Social and Behavioral Sciences, 237, 1066-1070. https://doi.org/10.1016/j.sbspro.2017.02.156
Núñez, J. C., Tuero, E., Fernández, E., Añón, F. J., Manalo, E., & Rosário, P. (2022). Effect of an intervention in self-regulation
strategies on academic achievement in elementary school: A study of the mediating effect of self-regulatory activity.
Revista de Psicodidáctica (English ed.), 27(1), 9-20. https://doi.org/10.1016/j.psicoe.2021.09.001
Öztürk, M., & Çakıroğlu, Ü. (2021). Flipped learning design in EFL classrooms: Implementing self-regulated learning strategies
to develop language skills. Smart Learning Environment, 8(2). https://doi.org/10.1186/s40561-021-00146-x
Patrick, H., Mantzicopoulos, P., Samarapungavan, A., & French, B. F. (2008). Patterns of young children’s motivation for science and
teacher-child relationships. The Journal of Experimental Education, 76(2), 121-144. https://doi.org/10.3200/JEXE.76.2.121-144
Parkinson, M., & Creswell, C. (2011). Worry and problem-solving skills and beliefs in primary school children. British Journal of
Clinical Psychology, 50(1), 106-112. https://doi.org/10.1348/014466510X523887
Pas, P., Pol, H. H., Raemaekers, M., & Vink, M. (2021). Self-regulation in the pre-adolescent brain. Developmental Cognitive
Neuroscience, 51. https://doi.org/10.1016/j.dcn.2021.101012
Perry, N. E., & VandeKamp, K. J. (2000). Creating classroom contexts that support young children’s development of self-regulated
learning. International Journal of Educational Research, 33(7-8), 821-843. https://doi.org/10.1016/S0883-0355(00)00052-5
Pintrich, P. R., & De Groot, E. V. (1990). Motivational and self-regulated learning components of classroom academic performance.
Journal of Educational Psychology, 82(1), 33-40. https://doi.org/10.1037/0022-0663.82.1.33
Pintrich, P. R. (2000). The role of goal orientation in self-regulated learning. In M. Zeidner, & P.R. Pintrich (Eds.), Handbook of
self-regulation (pp. 451-502). Academic Press.
Pravesti, C. A., Wiyono, B., Dany-Moenindyah, D. M. D., Triyono, T., & Atmoko, A. (2020). Examining the effects of guidance and
counseling services to the self-regulated learning for college students. Journal for the Education of Gifted Young Scientists,
8(1), 33-45. https://doi.org/10.17478/jegys.664548
Raković, M., Bernacki, M. L., Greene, J. A., Plumley, R. D., Hogan, K. A., Gates, K. M., & Panter, A. T. (2022). Examining the
critical role of evaluation and adaptation in self-regulated learning. Contemporary Educational Psychology, 68.
https://doi.org/10.1016/j.cedpsych.2021.102027
Reeve, J. (2018). Understanding motivation and emotion (7th edition). Wiley.
Roick, J., & Ringeisen, T. (2017). Self-efficacy, test anxiety, and academic success: A longitudinal validation. International Journal
of Educational Research, 83, 84-93. https://doi.org/10.1016/j.ijer.2016.12.006
Sahin, N., Dixon, J. K., & Schoen, R. C. (2020). Investigating the association between students’ strategy use and mathematics
achievement. School Science and Mathematics, 2020, 1-8. https://doi.org/10.1111/ssm.12424
Sakız, G. (2013). Key word in success: Self-efficacy. Journal of Uludag University Faculty of Education, 26(1), 185-209.
Sakız, G., & Yetkin-Özdemir, E. İ. (2014). Özdüzenleme ve özdüzenlemeli öğrenme: Kuramsal bakış [Self-regulation and
self-regulated learning: A theoretical perspective]. In G. Sakız (Ed.), Özdüzenleme: Öğrenmeden öğretime özdüzenleme
davranışlarının gelişimi, stratejiler ve öneriler [Self-regulation: From teaching to learning – Development of self-regulated
behaviors, strategies and suggestions] (pp. 1-27). Nobel.
204
ISSN 2538–7138 /Online/ (pp. 192-206)
Salisch, M., Haenel, M., & Denham, S. (2015) Self-regulation, language skills, and emotion knowledge in young children from
Northern Germany. Early Education and Development, 26(5-6), 792-806. https://doi.org/10.1080/10409289.2015.994465
Schraw, G., Crippen, K. J., & Hartley, K. (2006). Promoting self-regulation in science education: Metacognition as part of a
broader perspective on learning. Research in Science Education, 36, 111-139. https://doi.org/10.1007/s11165-005-3917-8
Schunk, D. H., & Ertmer, P. A. (2000). Self-regulation and academic learning: Self-efficacy enhancing interventions. In M.
Boekaerts, P. Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation (pp. 631-649). Academic Press.
Schunk, D. H., & Zimmerman, B. J. (2003). Self-regulation and learning. In W. M. Reynolds, & G. E. Miller (Eds.), Handbook of
psychology: Educational psychology (Vol. 7, pp. 59-79). John Wiley & Sons.
Schunk, D. H., & Meece, J. L. (2006). Self-efficacy development in adolescence. In F. Pajares, & T. Urdan (Eds.), Adolescence and
education: self-efficacy beliefs of adolescents (pp. 71-96). Information Age Publishing.
Senler, B., & Vural-Sungur, S. (2014). Pre-service science teachers’ use of self-regulation strategies. Procedia-Social and Behavioral
Sciences, 152, 551–556. https://doi.org/10.1016/j.sbspro.2014.09.242
Shoval, E., Shachaf, M., Ramati-Dvir, O., & Shulruf, B. (2021). Gender matters when sports engagement and self-efficacy interact
with academic achievement. Social Psychology of Education, 24(1), 75-94. https://doi.org/10.1007/s11218-020-09598-4
Soltani, A., & Askarizadeh, G. (2021). How students’ conceptions of learning science are related to their motivational beliefs
and self-regulation. Learning and Motivation, 73. https://doi.org/10.1016/j.lmot.2021.101707
Sun, T., & Wang, C. (2020). College students’ writing self-efficacy and writing self-regulated learning strategies in learning
English as a foreign language. System, 90. https://doi.org/10.1016/j.system.2020.102221
Tatar, N., Yıldız, E., Akpınar, E., & Ergin, Ö. (2009). A study on developing a self efficacy scale towards science and technology.
Eurasian Journal of Educational Research, 36, 1-20.
Tuan, H.L, Chin, C.C., & Shieh, S.H. (2005). The development of a questionnaire to measure students’ motivation towards science
learning. International Journal of Science Education, 27(6), 639-654. https://doi.org/10.1080/0950069042000323737
Uka, A., & Uka, A. (2020). The effect of students’ experience with the transition from primary to secondary school on self-
regulated learning and motivation. Sustainability, 12(20), 8519. https://doi.org/10.3390/su12208519
Üçüncü, G., & Sakız, G. (2019). The effect of brain-based learning on students’ academic achievement and achievement emotions in
science course. Journal of Uludag University Faculty of Education, 32(1), 345-378. https://dx.doi.org/10.19171/uefad.533251
Vasey, M. W., Crnic, K. A., & Carter, W. G. (1994). Worry in childhood: A developmental perspective. Cognitive Therapy and
Research, 18(6), 529-549. https://doi.org/10.1007/BF02355667
Vassallo, S. (2011). Implications of institutionalizing self-regulated learning: An analysis from four sociological perspectives.
Educational Studies, 47(1), 26-49. https://doi.org/10.1080/00131946.2011.540984
Veenman, M. V., & Spaans, M. A. (2005). Relation between intellectual and metacognitive skills: Age and task differences.
Learning and Individual Differences, 15(2), 159-176. https://doi.org/10.1016/j.lindif.2004.12.001
Velayutham, S., Aldridge, J., & Fraser, B. (2011). Development and validation of an instrument to measure students’
motivation and self-regulation in science learning. International Journal of Science Education, 33(15), 2159-2179.
https://doi.org/10.1080/09500693.2010.541529
Vettori, G., Vezzani, C., Bigozzi, L., & Pinto, G. (2020). Upper secondary school students’ conceptions of learning,
learning strategies, and academic achievement. The Journal of Educational Research, 113(6), 475-485.
https://doi.org/10.1080/00220671.2020.1861583
Vohs, K. D., & Baumeister, R. F. (2004). Understanding self-regulation: An introduction. In K. D. Vohs, & R. F. Baumeister (Eds.),
Handbook of self-regulation: Research, theory, and applications (pp. 1-13). Guilford Publications.
Weinstein, C. E., Acee, T. W., & Jung, J. (2011). Self-regulation and learning strategies. New Directions for Teaching and Learning,
2011(126), 45-53. https://doi.org/10.1002/tl.443
Wong, J., Baars, M., He, M., de Koning, B. B., & Paas, F. (2021). Facilitating goal setting and planning to enhance online self-
regulation of learning. Computers in Human Behavior, 124. https://doi.org/10.1016/j.chb.2021.106913
Yılmaz, H., & Çavaş, P. H. (2007). Reliability and validity study of the Students Motivation Toward Science Learning (SMTSL)
questionnaire. Elementary Education Online, 6(3), 430-440.
Yusuff, K. B. (2018). Does personalized goal setting and study planning improve academic performance and perception
of learning experience in a developing setting? Journal of Taibah University Medical Sciences, 13(3), 232-237.
https://doi.org/10.1016/j.jtumed.2018.02.001
Zimmerman, B. J. (1989). A social cognitive view of self-regulated academic learning. Journal of Educational Psychology, 81,
329-339. https://doi.org/10.1037/0022-0663.81.3.329
Zimmerman, B. J. (1990a). Self-regulated learning and academic achievement: An overview. Educational Psychology, 25(1),
3-17. https://doi.org/10.1207/s15326985ep2501_2
Zimmerman, B. J. (1990b). Self-regulating academic learning and achievement: The emergence of a social cognitive perspective.
Educational Psychology Review, 2(2), 173-201. https://doi.org/10.1007/BF01322178
Zimmerman, B. J. (2000). Attaining self-regulation: A social cognitive perspective. In M. Boekaerts, P. R. Pintrich, & M. Zeidner
(Eds.). Handbook of self-regulation (pp.13-39). Elsevier.
Zimmerman, B. J. (2001). Theories of self-regulated learning and academic achievement: An overview and analysis. In B. J.
Zimmerman, & D. H. Schunk (Eds.), Self-regulated learning and academic achievement: Theoretical perspectives (2nd ed.,
pp. 1-37). Lawrence Erlbaum.
Zimmerman, B. J., & Cleary, T. J. (2009). Motives to self-regulate learning a social cognitive account. In K. Wentzel, & A. Wigfield
(Eds.), Handbook on motivation at school (pp. 247-264). Taylor and Francis.
205
(pp. 192-206) ISSN 2538–7138 /Online/
Zimmerman, B. J., & Kitsantas, A. (1997). Developmental phases in self-regulation: Shifting from process goals to outcome
goals. Journal of Educational Psychology, 89(1), 29-36. https://doi.org/10.1037/0022-0663.89.1.29
Zimmerman, B. J., & Martinez-Pons, M. (1986). Development of a structured interview for assessing student use of self-regulated
learning strategies. American Educational Research Journal, 23(4), 614-628. https://doi.org/10.3102/00028312023004614
Cite as: Cengiz-Istanbullu, B., & Sakiz, G. (2022). Self-regulated learning strategies impact fourth-grade students’ positive
outcomes in science class. Journal of Baltic Science Education, 21(2), 192-206. https://doi.org/10.33225/jbse/22.21.192
Busra Cengiz-Istanbullu MA, Primary School Teacher, Istanbul, Turkey.

E-mail: bsrcngz34@gmail.com
Gonul Sakiz PhD, Associate Professor, Department of Elementary Education,
(Corresponding author) Ataturk Faculty of Education, Marmara University, 34722 Istanbul,
Turkey.
E-mail: gonul.sakiz@marmara.edu.tr
206
AN EXAMINATION OF PRE-
SERVICE TEACHERS’
EXPERIENCES IN CREATING ISSN 1648-3898 /Print/
A SCIENTIFIC DIGITAL ISSN 2538-7138 /Online/
STORY IN THE CONTEXT OF

THEIR SELF CONFIDENCE IN
TECHNOLOGICAL PEDAGOGICAL
CONTENT KNOWLEDGE
Abstract. Digital stories are a form of
expression that emerges by combining
the art of storytelling with multimedia
tools such as sound, picture, and video. In
Umit Izgi-Onbasili, this research, examining the experiences
Aysegul Avsar-Tuncay, of pre-service teachers in creating digital
Burcu Sezginsoy-Seker, stories in accordance with the science
curriculum and the effects of this process
Seyit Ahmet Kiray
on the pre-service teachers’ self-confidence
in technological pedagogical content
knowledge was aimed. This research, using
a mixed model approach, was conducted
Introduction
with 24 fourth-grade pre-service teachers.
In the study, the “Technological Pedagogi-
As a result of rapid developments in technology, digital transformation
cal Content Knowledge Confidence Scale”
in the field of education has accelerated. In this period when digital technol-
(TPACK) was applied as a pre-test and post-
ogy becomes widespread and directs every aspect of life, the task of guiding
test. The quantitative findings of the study
students to use technology in a beneficial way is very important (Kocoglu,
revealed that the experiences of pre-service
2021). In order for the students called “New Millennium Learners” (Pedró,
teachers in creating a scientific digital story
2006) or “Digital Natives” (Prensky, 2001) to be successful both academically
increased statistically significantly in the
and socially, teachers should pay attention to use different methods/tech-
TK dimension of the TPACK scale and in
niques in their teaching activities. Teachers are expected to be sufficiently
the TPACK total self-confidence scores. The
equipped, especially in terms of effective use of technological tools. The
qualitative findings of the research showed
International Educational Technologies Society (ISTE; 2015) tried to draw the
that it had a positive effect on TCK, TPK,
framework of innovation in education and some standards were determined.
and TPCK dimensions, as well as on the
The framework especially emphasized (a) setting professional learning goals
TK and total score of TPACK. In the light of
by applying technology together with pedagogical approaches, (b) develop-
these findings of the research, the use of
ing themselves according to their interests by participating in national and
digital stories in educational environments
international learning networks, (c) creating a technology-enhanced teaching
can be recommended since the use of
and learning vision, (d) striving to meet the needs of students and ensure
digital stories in educational environments
equal access to educational technologies, (e) increasing students’ curiosity
is effective in developing students’ TPACK
about online resources and enable them to look critically, and (f ) creating a
self-confidence perceptions.
learning culture that improves digital literacy (ISTE, 2015).
Keywords: mixed method, pre-service
Approaches to improve teacher competence in technology integration
teachers, scientific digital story, technologi-
argue that technological knowledge alone is not sufficient and that we should
cal pedagogical content knowledge
focus on the fundamental connections between technology, pedagogy, and
content knowledge (Mishra & Koehler, 2006). Today, especially in the field of
Umit Izgi-Onbasili, Aysegul Avsar-Tuncay
science education, the integration of education and technology comes to the Mersin University 1, Turkey
fore (Turk & Seckin-Kapucu, 2021). Jacobs (2010) stated that the teaching tools Burcu Sezginsoy-Seker
to be designed for 21st century skills should have visual and organizational Balikesir University 2, Turkey
Seyit Ahmet Kiray
features that allow students to learn concretely. Wellington and Osborne Necmettin Erbakan University 3, Turkey
(2001) stated that different types of materials (visual presentations, pictures,
207
PEDAGOGICAL CONTENT KNOWLEDGE ISSN 2538–7138 /Online/
(pp. 207-223)
animated materials, etc.) can be used to communicate well with students, especially in science education. At this
point, the use of digital stories in the teaching process draws attention. Robin (2006, 2008) classified digital stories
as personal, historical, and scientific. Teachers can use digital stories to present information to their students, from
math and science, to arts, technology, and medical education (Robin, 2008). In this context, the use of “scientific
digital story” in science education emerges as one of the effective methods (Calik & Seckin-Kapucu, 2021).
Research Problem
In the 21st century, it is of great importance to use digital learning objects in educational environments and
to integrate learning materials such as interactive videos, animations, simulations, educational films, digital stories
into lessons as active learning tools (Koyunlu-Unlu & Dokme 2020; Topal et al., 2020). This research focused on digi-
tal stories from these learning tools. Digital stories can be created by combining music, video, audio, and pictures
with a story or dialogue written using Web 2.0 technologies. Storytelling is the foundation of the digital story. The
difference of digital stories from normal storytelling is that the story is supported by visual and auditory materials
in the digital environment (Seckin-Kapucu & Yurtseven-Avci, 2020). The creation phases of digital stories need to be
planned correctly. Digital story creation phases can be listed as (1) writing, (2) scripting, (3) creating storyboard, (4)
locating multimedia, (5) creating the digital story, and (6) sharing (Jakes & Brennan, 2005). The digital story creation
process requires content knowledge and pedagogy knowledge as well as technology knowledge. In this respect,
teachers’ self-confidence in TPACK is of great importance in creating a digital story.
TPACK is seen as an important approach that supports, improves, and facilitates technology integration in
education (McGraw-Hill, 2018; Santos & Castro, 2021). Mishra and Koehler (2006), mentioning the importance of
teachers’ confidence in integrating technology into teaching, proposed a new conceptual framework by adding
“technology” to Shulman’s “pedagogical content knowledge” formula. TPACK generally consists of three general
knowledge types (Technological Knowledge, Pedagogical Knowledge, Content Knowledge) and other knowledge
(Technological Pedagogical Knowledge, Technological Content Knowledge, Pedagogical Content Knowledge,
Technological Pedagogical Content Knowledge) (Kiray et al., 2018; Kartal & Dilek, 2021). This understanding forms
the basis of an effective teaching process with technology. The pedagogical techniques that are used to teach the
content knowledge and the information necessary for students to use technologies that will help them construct
new concepts are integrated by expert teachers (Koehler & Mishra, 2008). With the acceleration of digitalization,
the technology dimension of TPACK is developing day by day and its importance is increasing.
Technology literacy has become one of the main aims of education today. In order for teachers to achieve
this goal, they must first be technology literate and be able to combine their technological knowledge with their
content and pedagogical knowledge (Koehler & Mishra, 2008; Mishra & Koehler, 2006). Primary school teachers in
Turkey are obliged to teach more than one course (science, mathematics, social studies, life studies, etc.) from the
first to the fifth grade. Primary school teachers are also faced with the necessity of integrating these courses with
technology for the new generation who grow up with technology. In this context, this research aimed that the
pre-service primary school teachers create scientific digital stories for the subjects and objectives in the Ministry
of National Education (MoNE; 2018) science curriculum in Turkey, based on the “Technological Pedagogical Con-
tent Knowledge” approach, and the effects of this process on their self-confidence in technological pedagogical
content knowledge intended to be examined. For this purpose, researchers expect that pre-service teachers’ self-
confidence in TPACK has improved in the process of creating digital stories. In Table 1, the association between the
technology-related components of TPACK and the digital story creation process is presented.
Table 1
Components of TPACK and Application Examples in the Research
Components of TPACK Explanation Components of creating digital story

process
It is the knowledge about various technologies ranging from Using Web 2.0 tools such as Powtoon, Storyboard-
traditional technologies such as pen, paper, books to digital That, Pixton,
TK
technologies such as internet, digital video, smart board, and
software programs (Kiray et al., 2018; Koehler & Mishra, 2006).
208
ISSN 1648–3898 /Print/ AN EXAMINATION OF PRE-SERVICE TEACHERS’ EXPERIENCES IN CREATING A SCIENTIFIC
ISSN 2538–7138 /Online/ PEDAGOGICAL CONTENT KNOWLEDGE
(pp. 207-223)
Components of TPACK Explanation Components of creating digital story

process
It is about how teachers can use and evaluate their techno- Deciding whether the digital stories created in the
logical knowledge in a pedagogically meaningful way in the computer are suitable for the level of the student,
TPK
classroom (Koehler & Mishra, 2008; Mishra & Koehler, 2006) the principles of material development, and different
teaching strategies, methods, and techniques.
It is the knowledge that teachers change the way students Placing the scientific scenarios written according
understand and apply concepts related to a content/subject to the learning outcomes in the curriculum on the
TCK
using technology (Mishra & Koehler, 2006). storyboard by blending them with visual and auditory
elements and making use of technology.
It is the knowledge of integrating technology in a way that helps Creating a digital story by integrating content knowl-
students build their existing knowledge and new knowledge edge of science course subjects with Web 2.0 tools
TPCK by making use of pedagogical techniques to teach content such as Powtoon etc. and presenting the created
knowledge (Koehler & Mishra, 2008). story with various teaching strategies, methods, and
techniques.
Research Focus
In today’s conditions, teachers are expected to set learning goals by applying technology together with
pedagogical approaches, to create a technology-enhanced teaching vision, to make efforts for students to have
equal access to educational technologies, to increase students’ curiosity about online resources and to enable them
look critically, and to create a learning culture that improves digital literacy (ISTE, 2015). One of the main aims of
primary education is to raise technology literate individuals. Teachers’ raising individuals with this characteristic
primarily depends on their being technology literate and combining their technological knowledge with their
content and pedagogical knowledge. Approaches to improve teacher competence in technology integration
argue that technological knowledge alone is not sufficient and that it should focus on the fundamental connec-
tions between technology, pedagogy, and content knowledge (Mishra & Koehler, 2006). Literature emphasized
that teacher can use technology effectively in the technology integration process and that technology should be
made a part of the curriculum, and students’ learning is enriched in this way (Mazman & Usluel, 2011, p. 65). In sci-
ence education, it is recommended to use different materials (visual presentations, pictures, animated materials,
etc.) to communicate well with students (Wellington & Osborne, 2001). In this context, the use of “scientific digital
story” in science education emerges as one of the effective methods. Teachers can use digital stories to present
information to their students, from mathematics, science, art, technology, and medical education (Robin, 2008).
Considering that teachers gain their knowledge and experience during their education in college, it is necessary
for pre-service teachers to receive training on digital storytelling to experience this practice (Shelton et al., 2017).
It is important for the pre-service teachers to become competent in creating and presenting the digital story after
the phases of creating a digital story: developing a story idea, writing, and editing the story, creating a storyboard,
and choosing the multimedia components (Frazel, 2010). At the same time, taking the trainings that include the
software used in digital storytelling and the use of Web 2.0 tools will help pre-service teachers in creating digital
stories. In line with these considerations, in this study, researchers aimed to examine the experiences of pre-service
primary school teachers in creating scientific digital stories for the subjects and objectives in the MoNE (2018) sci-
ence curriculum in the context of the “Technological Pedagogical Content Knowledge” approach.
Research Aim and Research Questions
In this study, it became important to prepare digital stories developed by pre-service primary school teachers
for the science curriculum and to determine the effects of this process on the pre-service teachers’ self-confidence
in TPACK, especially regarding the TK, TPK, TCK, and TPCK dimensions. As researchers aimed to examine the experi-
ences of pre-service primary school teachers in creating scientific digital stories in the context of their self-confidence
in TPACK in this study, the research questions were determined as follows:
1. Does the digital story creation experience of the pre-service primary school teachers make a statistically
significant difference on their self-confidence mean scores in TPACK?
209
(pp. 207-223)
2. Does the pre-service primary school teachers’ experience of creating digital stories make a statisti-
cally significant difference between the mean scores obtained from the sub-dimensions of the TPACK
confidence scale?
3. What are the opinions of the pre-service primary school teachers about the use of digital stories in
science education?
General Background
Mixed methods research is pragmatic research that uses both quantitative and qualitative data together (Er-
dogan & Stuessy, 2022). The mixed method has different designs, one of which is the convergent parallel design.
The most distinctive feature of the convergent parallel mixed design is that quantitative and qualitative data are
collected simultaneously but analyzed separately (Creswell, 2014). Therefore, in this research, the convergent parallel
mixed method, in which quantitative and qualitative data were used together, giving the opportunity to examine
the research problem in depth, was used. In this research, the data collected by quantitative methods needed
to be supported by the data collected from qualitative methods through interviews to illuminate the subjective
meaning that could be missed during the quantitative data collection.
Participants
This research was carried out with the pre-service primary school teachers studying at a state university in
Turkey during the fall semester of 2019-2020. The research was carried out with a total of 24 pre-service teachers,
10 females (41.7%) and 14 males (58.3%) enrolled in the “Integration of Technology Based Applications in Educa-
tion” course in the undergraduate education program of Primary Education Department. Integration of Technology
Based Applications in Education course is an elective course and all of the students who chose this course were
included in the research. Until this application, the participants had only taken Computer I course (a course with
basic Office applications such as Word, Excel, PowerPoint). There is no course in the official program covering Web
2.0 applications. Before the application, the participants were informed about the application to be made. Consent
form was filled by the participants so that they could participate in the experimental application. All of the students
who chose the course volunteered to participate in the experimental application.
Instrument and Procedures
The research was carried out in the computer laboratory classroom for 2 hours a week and lasted for 14 weeks
in the fall semester of 2019-2020. All pre-service teachers who participated in the research were informed about
the research and the voluntary participation form was signed and collected. Before the applications started, the
“Technological Pedagogical Content Knowledge Confidence Scale” was applied to the pre-service teachers in the
classroom environment as a pre-test.
The association between the components of the digital story and the components of TPACK is presented in
Table 2, with reference to the digital story creation processes of the pre-service teachers during the research and
the digital story creation phases of Frazel (2010) and Ohler (2013).
Table 2
The Digital Story Creation Process of Pre-Service Teachers and the TPACK Relationship
Relevance to
Time Phases What’s done in practice
TPACK
Week 1 Pre-application The concept of “story” was explained. Group work was done in the PK (Pedagogical
classroom with scientific stories developed by Gölcük (2017). They Knowledge)
identified the elements in these stories (number of characters and their
features, setting, etc.) and wrote the appropriate dramatic questions.
They designed a scene suitable for the story and chose the music.
210
(pp. 207-223)
Relevance to
Time Phases What’s done in practice
TPACK
1. Preparation and Planning:

Week 2 a. Deciding what kind of story to Each pre-service teacher determined a subject and outcome from a CK (Content Knowl-
prepare (Frazel, 2010) unit in the MoNE (2018) science curriculum. For example: A story called edge)
“Shining Friendships” was written for the objective of “Recognizes
the circuit elements that make up a simple electrical circuit with their
functions.”
Week 3 b. Writing the dramatic question. A dramatic question was written for each story. For example: “How do
we meet the lighting needs from past to present?”
Week 4 c. Writing a screenplay (Frazel, Scientific scenarios suitable for the curriculum were written and neces- PCK (Pedagogical
2010) sary corrections were made by bilingual experts. Content Knowledge)
Week 5 d. Getting peer opinion and All stories were read in class and opinions were taken.
changing the scenario if neces-
sary (Ohler, 2013)
Week 6 e. Preparing a Story Board “The Digital Storyboard” template was shared with the participants and TCK (Technological
explanations were made. Participants prepared a digital storyboard Content Knowledge)
about the topic they determined.
Week 7 f. Determining a digital story The “Digital Story Evaluation Rubric” developed by Çıralı-Sarıca and TPK (Technological
evaluation rubric Koçak-Usluel (2016) was introduced, and information was given about Pedagogical Knowl-
the criteria. edge)
2. Before Production:
Week 8 a. Identifying the necessary All visuals, texts, and music, as well as the science subject they will use TCK (Technological
elements (visual, sound, text, in their digital stories, were determined by the pre-service teachers. Content Knowledge)
music) in the digital story (Ohler,
2013).
Week 9 b. Creating visual content to be For their stories, pre-service teachers completed the drawing of both TK (Technological
used in the story their characters and all their scenes using Web 2.0 tools Pixtoon / Knowledge)
StoryboardThat/Toondoo and Microsoft Paint.
3. Production:
Week 10 Creating the digital story They created their digital stories using Microsft Photo Story, Storyboard- TPCK
and 11 through the appropriate That, and Powtoon programs. They used stories and visuals suitable
program / software/application for the development level of the group (primary school students) to be
(Ohler, 2013). applied in the digital story creation process. They paid attention to use
scientific knowledge correctly.
4. Presentation:
Week 12 a. Presenting the prepared At the end of the semester, all digital stories were presented in the TPCK
digital stories (Frazel, 2010). classroom.
Week 13 b. Receiving feedback after Participants first evaluated their own digital stories and the application TPCK
peer review and rearranging if process, and then expressed their opinions about the use of content
necessary (Ohler, 2013). knowledge, pedagogy knowledge and technology knowledge of other
friends’ stories.
5. Sharing:
Week 14 a. Deciding where and with They recorded their stories in mp4 format and shared it with research- TK (Technological
whom the finalized digital ers on their personal YouTube channels. They signed the “Voluntary Knowledge)
stories will be shared. (Ohler, Consent Form” and submitted it to the researchers so that the digital
2013). stories they prepared could be used and published within the scope of
this research.
Data Sources

In order to collect data in the research, TPACK confidence scale and semi-structured interview form were used.
211
(pp. 207-223)
Technological Pedagogical Content Knowledge Confidence Scale. In the study, “Technological Pedagogical Content
Knowledge Confidence Scale” developed by Graham et al. (2009) and adapted to Turkish by Timur and Taşar (2011),
was used as pre-test and post-test. TPACK Confidence scale is a scale with four dimensions consisting of combinations
of technology, pedagogy, and content knowledge. The scale has 8 items in the Technological Pedagogical Content
Knowledge (TPCK) dimension (items 1-8), 7 items in the Technological Pedagogical Knowledge (TPK) dimension
(items 9-15), 5 items in the Technological Content Knowledge (TCK) dimension (items 16-20), and 11 items in the
Technological Knowledge (TK) dimension (items 21-31) with a total of 31 items. While the highest score that pre-
service teachers could get from the scale, which is adapted as a 5-point Likert type, was 186, the lowest score was
26. The internal consistency coefficients of the Cronbach’s Alpha reliability values for the sub-dimensions of the
scale range from .786 to .925, and .919 for the overall scale. These values make consistent measurements within
themselves, and the internal consistency is highly reliable (Kalaycı, 2010).
Semi-Structured Interview Form. In preparation for the interviews, a semi-structured interview form was prepared,
and this form was used as a guide by the interviewer (See Appendix 3). 4 of the 24 volunteer pre-service teachers
participating in the research were randomly chosen and their opinions regarding the digital stories they prepared
were taken. The interviews, which lasted approximately 20-25 minutes, were conducted online. Researchers and
an experienced expert working on qualitative research made separate coding on the data obtained from the
interviews, and the consistency rate was calculated by comparing the coding. The resulting codes were divided
into similar and divergent codes. According to the Miles and Huberman (1994) formula, the reliability between
the coders was calculated as 88%. The fact that the Miles-Huberman reliability formula value is more than 70%
indicates that researchers’ coding is reliable. There was also a consistency between the codes and themes deter-
mined by the researchers.
Data Analysis
The SPSS 22.0 package program was used to analyze the quantitative data obtained from the Technological
Pedagogical Content Knowledge Confidence Scale. Shapiro-Wilk test was used with skewness values to determine
whether the study group showed a normal distribution after data entry. According to the results, p value was larger
than .05 and the skewness value was between -1 and +1 (Skewness = 0.36). Accordingly, the study group had a
normal distribution, and parametric tests were used in the analysis of the data. The paired-samples t-test was used
to test whether the pretest and posttest average scores of the pre-service teachers’ digital story-making experiences
regarding the self-confidence in TPACK and the sub-dimensions of the TPACK differ.
Research Results
Quantitative Results
Self-confidence of Primary School Teacher Candidates about TPACK
The findings regarding the pretest-posttest average scores of the pre-service teachers’ scientific digital story-
making experiences regarding their self-confidence in TPACK are given in Table 3.
Table 3
Pre-test-Post-test Average Scores of the Pre-service Primary School Teachers’ Self-confidence in TPACK
Dimensions Tests N X̅ SD t df p
Pre-test 24 42.21 8.582

TK -3.005 23 .006
Post-test 24 48.79 5.485
Pre-test 24 16.46 4.836
TCK -2.029 23 .054
Post-test 24 19.21 3.945
212
(pp. 207-223)
Dimensions Tests N X̅ SD t df p
Pre-test 24 26.33 5.577

TPK -1.852 23 .077
Post-test 24 28.71 3.641
Pre-test 24 27.25 6.152
TPCK -2.050 23 .052
Post-test 24 30.54 5.013
Pre-test 24 112.25 20.484
Total Score -2.815 23 .010
Post-test 24 127.25 14.959
Note: In this research, TPCK was used as a sub-dimension of the scale, and TPACK was preferred when mentioning about the
total score obtained from the scale.
According to Table 3, there is a significant difference between the pre-test and post-test mean scores of pre-
service teachers’ scientific digital story-making experiences in only the TK sub-dimension of the TPACK Confidence
scale [t(23)=-3.005, p<.05]. There was no significant difference between the pretest-posttest mean scores in the other
sub-dimensions [t(23)=-2.050, p>.05; t(23)=-1.852, p>.05; t(23)=-2.029, p>.05]. In terms of the total score obtained
from the scale, there was a significant difference between the pre-test and post-test mean scores [t(23)=-2.815,
p<.05]. While the average of the pre-test scores before the application was X̅ = 112.25; after the application, the
average of the post-test scores increased to X̅ = 127.25. This finding shows that practice has a positive effect on
the pre-service teachers’ self-confidence in TPACK and it is due to the TK sub-dimension.
Qualitative Results
Detailed information about the digital stories prepared by the pre-service teachers is presented in Appendix
1. In addition, there are sample scenes and story texts from the digital stories in Appendix 2. Links to the scientific
digital stories that 2 pre-service teachers uploaded to their personal YouTube channel were also provided. In general,
pre-service teachers created their digital stories about the sense organs, the natural and artificial environment, the
importance of our environment being clean, the movements of the Earth, the Sun and the Moon, the shape of the Earth,
the states of matter, light and sound, force, and nutrients. At the same time, pre-service teachers made applications
in internship schools with the digital stories they prepared. 4 of the 24 pre-service teachers who participated in
the research were randomly selected and their opinions were consulted.
In terms of Technological Knowledge (TK)
In the quantitative findings of the study, there was a significant difference in the self-confidence of the pre-
service teachers, especially in terms of TK. Similarly, positive opinions especially in terms of TK draw attention in
qualitative findings. All of the pre-service teachers stated that their technology knowledge proficiency has improved
thanks to the digital story creation process. During this period of self-assessment, they especially faced difficulties
in “finding and using appropriate Web 2.0 tools”, “having language problems while using programs” (due to the fact that
programs are usually in English), “utilizing programs with different interfaces”, “limitations of programs because they
charge fees” and mentioned the contributions of this process to them. The opinions of some pre-service teachers
on this subject are as follows:
T4: When we looked at the technological knowledge while creating the story, yes, I had this. I started to use the computer
programs (Word, PowerPoint, Excel) in my primary school years. I was fast. But what I have, more precisely, what we lack
is that we were lacking in digital education... I realized that I knew very traditional programs. I am always grateful to my
teacher for this. ... In terms of technology, I have developed myself a lot in terms of programs (software used in the digital
story) and I am still developing it. … As a limitation, of course, this is technology… There may be problems with the com-
puters we use, important points of some programs may be that they charge fees, etc.
213
(pp. 207-223)
Figure 1
Pre-service Teachers Using Web 2.0 Tools and Digital Story Programs in the Computer Lab
Another pre-service teacher stated her opinion as follows:
T2: ...The process definitely improved my technology knowledge. I realized how simple and superficial I used technology. I saw
the multiple richness offered by technology and how useful they can be in teaching with the theory of multiple intelligences.
In terms of Technological Pedagogical Knowledge (TPK)
Although there was no statistically significant difference in the quantitative analysis regarding the TPK, the
qualitative findings obtained from the pre-service teachers showed that the digital story creation process also
has positive effects in the TPK dimension. Pre-service teachers stated that the process of creating digital stories
improved creativity and cooperation; individual differences were addressed with digital stories; different strategies,
methods and techniques were needed in the process of developing and using digital stories; and it had a positive
effect on classroom management and student motivation. Opinions of T2 and T4 on this subject are given below.
T2: What I can say about the digital stories is that, first, creativity was the biggest opportunity because in this process, creat-
ing a story scenario, creating visuals, using which sound and which tone, these phases are all parts that develop creativity.
Thanks to the digital story, both visual elements and sound elements are used, so it appeals to individual differences. It is
more professional than traditional methods. It also enhances collaboration. Since we use many phases at the same time
on a single screen, it accelerates the practicality of executing the process and provides coordination.
T4: Digital story is defined as the combination of traditional storytelling and multimedia technologies, and based on this,
I think it is appropriate for using digital story creation in science lessons. Since it is a very comprehensive study, the person
doing the study needs to use many methods and techniques. This has a positive effect on what we call individual difference
in the classroom. It is much easier to provide classroom management as it attracts the attention of children auditory and
visually. In general, I can say that this method (digital story) positively affects students’ motivation and creative thinking skills.
214
(pp. 207-223)
Figure 2
The Image of Pre-service Teachers Teaching Students with Digital Stories in the Schools where they Did Internship (TPK)
In Terms of Technological Content Knowledge (TCK)
Although there was no statistically significant difference in the quantitative analysis regarding the TCK, the
pre-service teachers stated that digital stories should be used in teaching some science subjects, and they stated
that digital storytellers should have science content knowledge.
T4: Frankly, since it is a somewhat arduous process, I find it appropriate to use this in some subjects, not at every stage,
where we say it would be good if we used it.
T1: Science is a very comprehensive course. I think that the person who created (the digital story) should have a good
understanding of the subject, pay attention to the concepts, and know the content well.
T3: It is very efficient for non-applied objectives such as “Explains the relationship between living life and nutrient contents.”
While explaining more abstract topics such as protein, vitamins, and carbohydrates to students, the digital stories can make
it easier for them to both watch and learn, like they watch a movie, rather than lectures or question and answer methods.
Figure 3
Screenshot of One of the Scientific Digital Stories Regarding the TCK Prepared by Pre-service Teachers Using Powtoon and
Pixton Programs
215
(pp. 207-223)
In Terms of Technological Pedagogical Content Knowledge (TPCK)
In the quantitative dimension of the study, there was no statistically significant difference in the quantitative
analysis regarding the TPCK. However, the findings obtained from pre-service teachers show that the digital story
creation process had a positive effect on the TPCK sub-dimension.
T2: The thing I like most about digital stories is that they are very educational, and I think they should be applied for every
lesson and objective. I think it is very effective especially in attracting students’ attention and giving the desired message. It
will also definitely improve students’ ICT skills. While preparing these stories, I realized that I could have more control over
technology. From a pedagogical point of view, it will be an instructive application that will easily attract the attention of
students and make the lesson more enjoyable. It will be very difficult to prepare for every lesson, as it takes a process and
effort to prepare rather than being limited. It intrigues me, and frankly, it makes me more involved and interested. It has a
very interesting effect. Lecture and memorization do not work in all lessons, especially in science lessons, more visual and
observation-based things should be used in these lessons.
Figure 4
Image of Teaching a Science Subject with a Digital Story (TPCK)
Discussion
In this research, the effect of digital stories created by pre-service primary school teachers on their self-con-
fidence in TPACK, especially for TK, TPK, TCK, and TPCK sub-dimensions, were examined. According to the results
of the research, the digital stories created according to the science curriculum objectives have a positive effect on
the TPACK confidence of the pre-service primary school teachers. The quantitative results of the research revealed
that such stories have led to the development of pre-service primary school teachers’ technological knowledge
(TK). The development of pre-service primary school teachers’ ability to use technology also led to an increase in
the total score of TPACK. The fact that digital stories increase pre-service teachers’ self-confidence in technology
knowledge has also been supported by some studies in the literature. Dogan (2012), Eguz (2020), and Seckin-Kapucu
and Yurtseven-Avci (2020) stated that digital stories increase students’ ability to use technology. Sancar-Tokmak
et al. (2014) found that digital stories they made with pre-service science teachers increased the technological
knowledge dimension of TPACK and accordingly the TPACK total score. Sancar-Tokmak and Yanpar-Yelken (2015)
stated that the use of digital stories had a positive effect on pre-service teachers’ self-confidence in TPACK, and
especially their technological pedagogical knowledge and technological knowledge scores improved significantly.
In the literature, it has been determined that the technology use skills of teachers and preservice teachers have
increased (Brown & Warschauer 2006; Ersanlı, 2016; Graham et al., 2009; Kildan & Incikabi, 2015; Sancar-Tokmak,
2015). Heo (2009) found that digital stories increase pre-service teachers’ educational technology competencies and
self-confidence. These findings support Graham et al. (2009)’s idea that self-confidence in Technology Knowledge
216
(pp. 207-223)
(TK) is the basis for improving self-confidence in the other three types of knowledge. Erdoğdu and Çakır (2020)
stated in their study that they investigated the digital story creation experiences of pre-service teachers and stated
that the pre-service teachers were confident in the use of digital story creation, and they found themselves suf-
ficient in this regard. Alabbasi (2018) emphasized that creating digital stories gave teachers a positive perception
of technology integration. In the study of Dewi (2016), primary school teachers who integrated digital stories into
the lesson plan improved their TPACK levels and teachers had positive opinions about these practices.
According to the qualitative results of this research, the use of digital stories in science education had a
positive effect on the dimensions of TK, TCK, TPK and TPCK of pre-service primary school teachers. When the
literature is examined, similar findings related to the positive contributions of the use of digital stories to the
TPACK self-confidence were found. Wang (2016) found that there was a statistically significant difference in favor
of the experimental group in terms of TK, TCK, TPK, TPCK and TPACK total score dimensions of the TPACK scale in
the experimental group in which digital stories were used. Clark (2017) revealed that developing digital stories
contributes to teachers’ improvement in all sub-dimensions of TPACK.
Both quantitative and qualitative results of this research showed that digital stories had a positive effect on TK
and TPACK total scores. For this reason, in order for teachers to realize effective technology integration, they need
to understand how to shape the teaching practices in which the components of TPACK are embedded. In this way,
their self-confidence in TPACK will also improve (Voogt & McKenney, 2017). According to the results obtained in the
TPK sub-title of the qualitative dimension, designing and applying digital stories contributes to the development of
pre-service teachers’ pedagogical knowledge such as creativity, cooperation, motivation, classroom management
and taking into account individual differences. The literature indicates that the use of digital stories improves creativ-
ity (Ohler, 2008; Wu & Yang, 2008) and collaboration (Robin, 2006; Sadik, 2008); individual differences are addressed
with digital stories (Sylvester & Greenidge, 2009; Yüksel, 2011); different strategies, methods and techniques are
needed in the process of creating and using digital stories; and it has a positive effect on classroom management
and student motivation (Morgan, 2014; Smeda et al., 2014; Turan & Sezginsoy Şeker, 2018; Ware & Warschauer, 2005;
Yang & Wu, 2012). According to the results obtained in the TCK sub-title of the qualitative dimension, pre-service
teachers stated that digital stories should be used in teaching some science subjects and they stated that digital
storytellers should have knowledge of science. This result contains similar results with the study of Pekmezci (2014),
Ulum and Ercan-Yalman (2018). Kiray et al. (2018) revealed that science teachers’ content knowledge directly af-
fects their technological content knowledge. These findings reveal that content knowledge is important in digital
story development. Since the science curriculum is designed with the wide-area design approach, the self-efficacy
perceptions of science teachers towards physics, chemistry, biology, astronomy, and earth sciences that make up
the science lesson differ. Teachers feel strong in some of these areas and weak in others. Teachers have difficulties
in teaching the areas where they feel inadequate (Sari & Kiray, 2021). For this reason, it is important for science
teachers and pre-service teachers to pay attention to the content knowledge in the use of digital stories.
In addition to the positive effect of using digital stories on the pre-service primary school teachers’ self-confi-
dence in TPACK, in the qualitative findings, they stated that they had some problems in TCK and TK dimensions (not
being able to find images due to copyright, not being able to create a scenario suitable for the outcome, difficulty
in vocalizing, long duration, difficulty in using/finding a computer/program, and not being able to find suitable
background music, etc.). The difficulties faced by pre-service primary school teachers in the digital story creation
process were paralleled by Sancar-Tokmak et al. (2014) and Kildan and Incikabi (2015).
The quantitative findings of this study revealed that developing digital stories and using digital stories in les-
sons resulted in a statistically significant increase in the TK and TPACK total scores of pre-service teachers. Although
there was no statistically significant difference in the dimensions of TCK, TPK, and TPCK in quantitative findings,
qualitative findings revealed a positive development in these dimensions. The qualitative findings of the study
also showed that digital story development and application studies contributed positively to the development of
pre-service primary school teachers’ pedagogical knowledge such as creativity, cooperation, motivation, classroom
management, and taking into account individual differences.
The qualitative findings of the study showed that along with the positive aspects of the use of digital sto-
ries, some difficulties were encountered in practice. In addition to technological difficulties such as difficulties
encountered in vocalizing digital stories, difficulties in using/finding computers/software, and difficulties in using
217
(pp. 207-223)
digital story development software in the process of creating digital stories, the pre-service teachers experienced
problems such as not being able to find images due to copyright, difficulties in creating scenarios suitable for the
objectives, the length of the process of creating digital stories, and the difficulties encountered in finding suitable
background music.
In the light of these results of the research, although there are some difficulties in developing digital stories,
such stories can be used in teacher training programs because they are effective in improving pre-service teachers’
TPACK self-confidence. Finally, it can be recommended that teachers increase the effectiveness of their lessons by
using digital stories in their lessons.
Limitations
This research is limited to the TPACK confidence scores of pre-service teachers studying at a state university
in Turkey in the 2019-2020 fall semester and their views on digital story creation.
References
Alabbasi, D. (2018). The design and application of a digital storytelling process model to enhance teachers’ understanding of
TPACK and foster positive attitudes toward teaching with technologies. International Journal of Technology Enhanced
Learning, 10(4), 309-328 https://doi.org/10.1504/IJTEL.2018.095142
Brown, D., & Warschauer, M. (2006). From the university to the elementary classroom: Students’ experiences in learning to integrate
technology in instruction. Journal of Technology and Teacher Education, 14(3), 599–621.
Calik, I., & Seckin-Kapucu, M. (2021). The effect of digital story-supported science practices on the scientific attitudes of 7th-grade
students. The Eurasia Proceedings of Educational and Social Sciences, 21, 1-14. https://doi.org/10.55549/epess.1040437
Clarke, A. M. (2017). A place for digital storytelling in teacher pedagogy. Universal Journal of Educational Research, 5(11), 2045-
2055. https://doi.org/10.13189/ujer.2017.051121
Creswell, J. W. (2014). Research design: qualitative, quantitative, and mixed methods approaches (4th ed.). SAGE Publications.
Çıralı-Sarıca, H., & Koçak-Usluel, Y. (2016). Digital storytelling in the educational context: A rubric development study.
Educational Technology Theory and Practice, 6(2), 65-84. https://doi.org/10.17943/etku.12600
Dewi, F. (2016). Digital storytelling project: An alternative to develop English teachers’ technological pedagogical and content
knowledge (TPACK). SPaCE Edu Journal, 2(1), 16–40.
Doğan, M. (2012). Prospective Turkish primary teachers’ views about the use of computers in mathematics education. Journal
of Mathematics Teacher Education, 15, 329-341. https://doi.org/10.1007/s10857-012-9214-3
Eguz, S. (2020). The use of digital storytelling in the education of social studies teacher candidates. The Eurasia Proceedings of
Educational and Social Sciences, 18, 55-58. http://www.epess.net/en/pub/issue/58018/831234
Erdogan, N., & Stuessy, C. L. (2022). Applying an ecology metaphor in a mixed methods analysis of high school science program
infrastructure. Journal of Education in Science, Environment and Health, 8(1), 86-97. https://doi.org/10.21891/jeseh.1029468
Erdoğdu, F., & Çakır, Ö. (2020, December 12-13). Digital story development: A study with pre-service mathematics teachers. Paper
presented at the 9th International Scientific Research Congress-Social and Educational Sciences, Ankara, Turkey.
Ersanlı, C. Y. (2016). Improving technological pedagogical content knowledge (TPACK) of pre-service English language teachers.
International Education Studies, 9(5), 18-27. https://doi.org/10.5539/ies.v9n5p18
Frazel, M. (2010). Digital storytelling guide for educators. Paper presented at the International Society for Technology in Education
(ISTE).
Gölcük, A. (2017). The effects of science education supported by scientific stories on students’ creativity and affective characteristics
[Unpublished masters’ thesis]. Hacettepe University.
Graham, C., Cox, S., & Velasquez, A. (2009). Teaching and measuring TPACK development in two preservice teacher preparation
programs. In I. Gibson, R. Weber, K. McFerrin, R. Carlsen & D. Willis (Eds.), Proceedings of SITE 2009--Society for Information
Technology & Teacher Education International Conference (pp. 4081-4086). Association for the Advancement of Computing
in Education (AACE). https://www.learntechlib.org/p/31297
Graham, R. C., Burgoyne, N., Cantrell, P., Smith, L., St Clair, L., & Harris, R. (2009). Measuring the TPACK confidence of in-service
science teachers. TechTrends, 53(5), 70-79. https://doi.org/10.1007/s11528-009-0328-0
Heo, M. (2009). Digital storytelling: An empirical study of the impact of digital storytelling on pre-service teachers’ self-efficacy
and dispositions towards educational technology. Journal of Educational Multimedia and Hypermedia, 18(4), 405-428.
ISTE (International Society for Technology in Education). (2015). About ISTE. https://www.iste.org/about/about-iste
Jacobs, H. H. (2010). Curriculum 21: Essential education for a changing world. ASC.
218
(pp. 207-223)
Jakes, D. S., & Brennan, J. (2005). Capturing stories, capturing lives: An introduction to digital storytelling. http://www.jakesonline.
org/dstory_ice.pdf
Kalaycı, Ş. (2010). SPSS Applied multivariate statistics techniques. Asil Yayın Dağıtım.
Kartal, T., & Dilek, I. (2021). Preservice science teachers’ TPACK development in a technology-enhanced science teaching method
course. Journal of Education in Science, Environment and Health, 7(4), 339-353. https://doi.org/10.21891/jeseh.994458
Kiray, S. A., Celik, I., & Colakoglu, M. H. (2018). TPACK Self-Efficacy perceptions of science teachers: A structural equation modeling
study. Education and Science, 43(195), 253-268. http://dx.doi.org/10.15390/EB.2018.7538
Kildan, A. O., & Incikabi, L. (2015). Effects on the technological pedagogical content knowledge of early childhood teacher candidates
using digital storytelling to teach mathematics. Education, 3-13, 43(3), 238-248. https://doi.org/10.1080/03004279.2013.804852
Kocoglu, E. (2021). The effect of digital learning environments on the learning teaching process. The Eurasia Proceedings of
Educational and Social Sciences, 20, 96-100. https://doi.org/10.55549/epess.1038715
Koehler, M., & Mishra, P. (2008). Introducing TPACK. In AACTE Committee on Innovation and Technology (Eds.), The handbook of
technological pedagogical content knowledge for teaching and teacher educators (pp. 3-29). Lawrence Erlbaum Associates,
Publishers.
Koyunlu-Unlu, Z., & Dokme, I. (2020). The effect of technology-supported inquiry-based learning in science education: Action
research. Journal of Education in Science, Environment and Health, 6(2), 120-133. https://doi.org/10.21891/jeseh.632375
Mazman, S. G., & Usluel, Y. K. (2011). ICT integration into learning-teaching process: Models and indicators. Educational Technology
Theory and Practice, 1(1), 62-79. https://dergipark.org.tr/tr/pub/etku/issue/6274/84241
McGraw-Hill Education. (2018). What is TPACK theory and how can it be used in the classroom? https://www.mheducation.ca/blog/
what-is-tpack-theory-and-how-can-it-be-used-in-the-classroom/
Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook. Sage.
Mishra, P., & Koehler, M. J. (2006). Technological pedagogical content knowledge: A framework for teacher knowledge. Teachers
College Record, 108(6), 1017-1054.
MoNE (Ministr y of National Education) (2018). Turkish Science Curriculum. https://mufredat.meb.gov.tr/
Dosyalar/201812312311937FEN%20B%C4%B0L%C4%B0MLER%C4%B0%20%C3%96%C4%9ERET%C4%B0M%20
PROGRAMI2018.pdf
Morgan, H. (2014). Using digital story projects to help students improve in reading and writing. Reading Improvement, 51(1), 20-26.
Ohler, J. (2008). Digital storytelling in the classroom: New media pathways to literacy, learning, and creativity. Corwin Press.
Ohler, J. (2013). Digital storytelling in the classroom. New media pathways to literacy, learning, and creativity (2nd ed.). Sage
Publications.
Pedró, F. (2006). The new millennium learners: Challenging our views on ICT and learning. Inter-American Development Bank.
Pekmezci, S. (2014). Effects of short stories reinforced with information technology on students’ success, their self-efficacy and their
attitudes towards science [Unpublished masters’ thesis]. Pamukkale University.
Prensky, M. (2001). Digital natives, digital immigrants. On the Horizon, 9(5), 1-6.
Robin, B. R. (2006). The educational uses of digital storytelling. Paper presented in Society for Information Technology & Teacher
Education International Conference (pp. 709-716). Association for the Advancement of Computing in Education (AACE).
Robin, B. R. (2008). Digital storytelling: A powerful technology tool for the 21st century classroom. Theory into Practice, 47(3),
220-228. https://doi.org/10.1080/00405840802153916
Sadik, A. (2008). Digital Storytelling: A meaningful technology-integrated approach for engaged student learning. Educational
Technology Research and Development, 56, 487-506. http://dx.doi.org/10.1007/s11423-008-9091-8
Sancar-Tokmak, H. (2015) Pre-service teachers’ perceptions on TPACK development after designing educational games. Asia-
Pacific Journal of Teacher Education, 43(5), 392-410, https://doi.org/10.1080/1359866X.2014.939611
Sancar-Tokmak, H., Surmeli, H., & Ozgelen, S. (2014). Preservice science teachers’ perceptions of their TPACK development
after creating digital stories. International Journal of Environmental and Science Education, 9(3), 247-264.
https://doi.org/10.12973/ijese.2014.214a
Sancar-Tokmak, H., & Yanpar-Yelken, T. (2015) Effects of creating digital stories on foreign language education pre-service teachers’
TPACK self-confidence. Educational Studies, 41(4), 444-461, https://doi.org/10.1080/03055698.2015.1043978
Santos, J. M., & Castro, R. D. (2021). Technological Pedagogical content knowledge (TPACK) in action: Application
of learning in the classroom by pre-service teachers (PST ). Social Sciences & Humanities Open, 3(1), 100-110.
https://doi.org/10.1016/j.ssaho.2021.100110
Sari, B., & Kiray, S. A. (2021). Self-efficacy perceptions of science teachers regarding content knowledge. MIER Journal of Educational
Studies Trends and Practices, 11(1), 85-102. https://doi.org/10.52634/mier/2021/v11/i1/1756
Seckin-Kapucu, M., & Yurtseven-Avci, Z. (2020). The digital story of science: Experiences of pre-service science teachers. Journal
of Education in Science, Environment and Health, 6(2), 148-168. https://doi.org/10.21891/jeseh.689444
Shelton, C., Archambault, L., & Hale, A. (2017). Bringing digital storytelling to the elementary classroom: Video production for preservice
teachers. Journal of Digital Learning in Teacher Education, 33(2), 58-68. https://doi.org/10.1080/21532974.2016.1276871
Smeda, N., Dakich, E., & Sharda, N. (2014). The effectiveness of digital storytelling in the classrooms: A comprehensive study.
Smart Learning Environments, 1(1), 1-21. https://doi.org/10.1186/s40561-014-0006-3
Sylvester, R., & Greenidge, W. L. (2009). Digital storytelling: Extending the potential for struggling writers. The Reading Teacher,
63(4), 284-295. https://doi.org/10.1598/RT.63.4.3
Timur, B., & Taşar, M. F. (2011). The adaptation of the technological pedagogical content knowledge confidence survey into
Turkish. Gaziantep University Journal of Social Sciences, 10(2), 839-856.
219
(pp. 207-223)
Topal, M., Guven-Yildirim E., & Onder, A. N. (2020). Use of educational films in environmental education as a digital learning object.
Journal of Education in Science, Environment and Health, 6(2), 134-147. https://doi.org/10.21891/jeseh.703492
Turan, T., & Sezginsoy-Şeker, B. (2018). The effect of digital stories on fifth-grade students’ motivation. Journal of Education and
Future, (13), 65-78.
Turk, H., & Seckin-Kapucu, M. (2021). Innovative technology applications in science educations: Digital holography. Journal of
Education in Science, Environment and Health, 7(2), 156-170. https://doi.org/10.21891/jeseh.748662
Ulum, E., & Ercan Yalman, F. (2018). Examining the effects of preparing digital storytelling in science and technology course on
the academically inadequate students spending much time on computers. Necatibey Faculty of Education Electronic Journal
of Science & Mathematics Education, 12(2), 306-335.
Voogt, J., & McKenney, S. (2017). TPACK in teacher education: Are we preparing teachers to use technology for early literacy?
Technology, Pedagogy and Education, 26(1), 69-83. https://doi.org/10.1080/1475939X.2016.1174730
Wang, A. Y. (2016). The impact of digital storytelling on the development of TPACK among student teachers in Taiwan. In Herring,
M.C., Koehler, M.J. & Mishra, P. (Eds). The handbook of technological pedagogical content knowledge (TPACK) for educators,
297-308. Routledge Press.
Ware, P. D., & Warschauer, M. (2005). Hybrid literacy texts and practices in technology-intensive environments. International
Journal of Educational Research, 43(7-8), 432-445. https://doi.org/10.1016/j.ijer.2006.07.008
Wellington, J., & Osborne, J. (2001). Language and literacy in science education. Open University Press.
Wu, W. C., & Yang, Y. T. (2008). The impact of digital storytelling and of thinking styles on elementary school students’ creative thinking,
learning motivation, and academic achievement. Paper presented in Society for Information Technology & Teacher Education
International Conference (pp. 975-981). Association for the Advancement of Computing in Education (AACE).
Yang, Y. T. C., & Wu, W. C. I. (2012). Digital storytelling for enhancing student academic achievement, critical
thinking, and learning motivation: A year-long experimental study. Computers & Education, 59(2), 339-352.
https://doi.org/10.1016/j.compedu.2011.12.012
Yüksel, P. (2011). Using digital storytelling in early childhood education a phenomenological study of teachers’ experiences [Unpublished
doctoral dissertation]. Middle East Technical University. https://open.metu.edu.tr/handle/11511/21182
Appendix 1
Information on Digital Stories Prepared by Pre-service Teachers
Partici- Program used/application/hand

Story Title Related science subject
pants drawing
P1 Emre’s Disease Maintaining the health of the sense organs Pixton, Microsoft photostory 3
Curious Elif and the Natural and artificial light sources Pixton, Microsoft photostory 3
P2
Fireflies
P3 Importance of the Eye Importance of eye in sense organs Pixton, Microsoft photostory 3
Wise Laws What is the importance of saving in preventing the depletion Pixton, StoryboardThat, Microsoft
P4
of resources? photostory 3
Two-Buddy Travelers Natural and artificial light sources around us StoryboardThat, Microsoft photostory 3
P5
Light and sound
Legendary Cleanliness Environment StoryboardThat, Microsoft photostory 3
Şenköy Living environment, environmental cleaning, nature, garden
P6
Dramatic question: Does environmental pollution only harm
people?
P7 Summer Burn Protecting the health of the sense organs-skin burn condition Pixton, Microsoft photostory 3
Beautiful Friendship of Events that occur as a result of the movements of the Earth Pixton, Microsoft paint, Microsoft photos-
P8
Sun and Earth tory 3
P9 Talking Foods Nutrients StoryboardThat, Microsoft photostory 3
A Clean World for Environmental cleaning, garbage, waste, nature, environmen- StoryboardThat, Microsoft photostory 3
P10 Everyone tal pollution
P11 Büşra’s One Day Change of state of matter under the influence of heat StoryboardThat, Microsoft photostory 3
220
(pp. 207-223)
Partici- Program used/application/hand

Story Title Related science subject
pants drawing
P12 Camping Adventure Environmental cleaning StoryboardThat, Microsoft photostory 3
P13 Snowman Beings around us StoryboardThat, Microsoft photostory 3
P14 Our Environment Natural and artificial environment StoryboardThat, Microsoft photostory 3
P15 Shining Friendships Simple electrical circuit Hand-drawn

Amusement Park Excite- Force Pixton, Powtoon
P16 ment Movement properties of beings; characterized as fast, slow,
rotating, swinging, and changing direction.
P17 Cherry Jam Living and inanimate objects StoryboardThat, Microsoft photostory 3
All Around the World in States of matter as solid-liquid-gas StoryboardThat, Microsoft photostory 3
P18
Every Moment
I Heard Your Voice I Sound StoryboardThat, Microsoft photostory 3
P19
Found You
P20 Jungle Adventure Sense organs StoryboardThat, Microsoft photostory 3
P21 Space Travel Shape of the Earth StoryboardThat, Microsoft photostory 3
Journey of the Water States of water Pixton, Mirsoft paint and photostory 3
P22
Drop
P23 A Warm Winter’s Tale Heating-melting of matter StoryboardThat, Microsoft photostory 3
If the electricity is cut off Lighting technologies from past to present, importance of Pixton, Microsoft photostory 3
P24
lighting tools
Appendix 2
Sample Images from Applications

Sample images and story content of P16
Story Title: Amusement Park Excitement,
The related objective in the science program: “Movement properties of beings; characterized as fast, slow, rotat-
ing, swinging, and changing direction.”
Programs used: Pixton, Powtoon
Scene 3 Scene 4
221
(pp. 207-223)
Scene 3 Scene 4
Scene Description: Emine was very happy with her mother’s offer to Scene Description: The family has arrived at the amusement park.
play. Her father also listens to Emine and her mother with pleasure. Emine excitedly comments on the carousel and is happy to think she has
won the movie award-winning game. Her mother laughs and says she
hasn’t won yet.
Narration of the Story: Emine said “Yes”. Her mother said, “Well, let’s Narration of the Story: In the meantime, they had arrived at the
play a game with you. If you know what movements the toys do when amusement park. The first thing that caught Emine’s attention was a
we go to the amusement park, we will go to the movies with you next huge carousel. She turned to her mother excitedly and said, “Mother, this
week”. Emine was very happy about this. Because she had listened to carousel is spinning. Didn’t I know? Are we going to the movies?” Her
the teacher very well. mother replied with a laugh, “Wait, you haven’t explained it all yet.”
Appendix 3
Interview form
Dear Participant,
The purpose of the interview: To learn your experiences and feedbacks about the digital stories you have prepared
for science teaching.
Introduction of the interview: You will be asked to answer various questions in order to identify the strengths and
areas of development of the Digital Story application. You don’t have to answer the questions. You can end the
interview at any time.
Risks: Your participation in this interview does not involve any risk.
Confidentiality: Interviews will be held with all students participating in the application. The information obtained
from the interviews will be separated from the personal information and brought together in the research article.
These opinions will only be used as research data and will not be used or published for any other purpose.
Consent: I understand the purpose of this interview and how it will be conducted. By participating in this meeting,
I agree to share my views on the digital story application. I know that I don’t have to answer every question and
that I can end the conversation at any time.
Semi-structured questions
1. Explain the process of creating a digital story according to the following headings.
a. creating a story idea
b. creating a storyboard
c. writing a dramatic question
d. character determination
e. writing a script
f. finding images
g. writing the story text for each scene
h. music and rhythm selection
i. determining the people who will make the voice over
j. finding Web 2.0 tools (programs)
2.
a. According to which criteria did you determine your unit or subject?
While creating your story text, did you start from a science subject that you dominate or from a subject
that you do not know or are missing?
b. How do you evaluate the story you wrote as a scientific story? -Content-Suitable for science curriculum
etc.
3. Was your technological knowledge sufficient when creating the story? Has it improved? How did this
process affect your knowledge of technology? What were the limitations and advantages of using
technology (while using Web 2.0 tools)?
222
(pp. 207-223)
4.
a) How would you evaluate the use of digital stories in science lessons from a pedagogical point of view
(in terms of using appropriate methods-techniques, measurement/evaluation, classroom management
and individual differences of students)?
b) Do you think you have succeeded in combining your pedagogical knowledge with the technology
you use in preparing digital stories? Please explain.
5.
a) How would you evaluate the use of digital stories in science lessons from a content knowledge point
of view?
b) Do you think you have succeeded in combining your content knowledge with the technology you
use while developing a digital story? Please explain.
6.
a) What are your thoughts on creating digital stories by using technology, pedagogy and content
knowledge for science subjects and teaching science subjects with these digital stories?
b) Do you think you can combine your technology, pedagogy and content knowledge while develop-
ing and applying digital stories? Please explain.
Received: March 05, 2022 Revised: March 22, 2022 Accepted: April 05, 2022
Cite as: Izgi-Onbaşılı, Ü., Avşar-Tuncay, A., Sezginsoy-Şeker, B., & Kiray, S. A. (2022). An examination of pre-service teachers’
experiences in creating a scientific digital story in the context of their self confidence in technological pedagogical content
knowledge. Journal of Baltic Science Education, 21(2), 207-223. https://doi.org/10.33225/jbse/22.21.207
Ümit Izgi-Onbaşılı PhD, Associate Professor, Faculty of Education, Mersin University, 33343,
Yenişehir / Mersin, Turkey.
E-mail: umitizgi@mersin.edu.tr
Ayşegül Avşar-Tuncay PhD, Associate Professor, Faculty of Education, Mersin University, 33343,
Yenişehir / Mersin, Turkey.
E-mail: aysegulaat@mersin.edu.tr
Burcu Sezginsoy-Şeker PhD, Associate Professor, Necatibey Educational Faculty, Balıkesir
University, Dinkçiler Mah. Soma Cad. Merkez, Balıkesir, Turkey.
E-mail: sezginsoy@balikesir.edu.tr
Seyit Ahmet Kiray PhD, Associate Professor, Ahmet Kelesoglu Education Faculty, Necmettin
(Corresponding author) Erbakan University, Konya, Turkey.
Website: https://www.erbakan.edu.tr/en/personel/1937/seyit-ahmet-kiray
223
THE EFFECT OF GAME-BASED

LEARNING VIA KAHOOT AND
QUIZIZZ ON THE ACADEMIC
ACHIEVEMENT OF THIRD GRADE
PRIMARY SCHOOL STUDENTS
Abstract. The application of student

response systems could have a lesser ef-
fect on the science education of younger Aleksandar Janković,
students compared to the effects achieved Dragan Lambić
with older students in language and social
science courses. The aim of this research
was to determine the effect of the applica-
tion of Kahoot and Quizizz on the aca-
demic achievement of third grade primary
school students (9 years old) in a natural
sciences course. A total of 113 third grade
primary school students participated in this
research. During the experimental phase
Introduction
which lasted 4 weeks, the participants
were divided into three groups. Out of two
There are various factors which influence the effects of the learning pro-
experimental groups one group used Ka-
cess on academic achievement. Besides innate abilities, students’ motivation is
hoot as a learning aid, while the other used
one of the most important factors (Lambić & Lipkovski 2012). Lack of motivation
Quizziz. The third group was the control
of students for learning (Lee & Hammer, 2011) represents a significant problem
group. At the beginning and at the end of
in contemporary education (Kiryakova et al., 2014) which can reduce learning
the experimental phase, the participants
outcomes due to the fact that it is very hard for teachers to foster student
were given an academic achievement test.
motivation over time during teaching (Wang & Tahir, 2020).
The results of this research showed that the
The problem regarding the lack of motivation for learning cannot be
experimental group which used Kahoot for
easily resolved by conventional methods (Goksün & Gürsoy, 2019). One of the
learning the content of the natural sciences
approaches which can solve this problem is the application of student response
course achieved significantly higher results
systems (SRSs) in the classroom (Wang & Tahir, 2020) which have a positive
in the post-test than the control group.
impact on learning performance (Caldwell, 2007). Another good approach is
There were no significant differences in the
the gamification of the learning process which can lead to the improvement of
academic achievement of the group which
students’ motivation and engagement (Goksün & Gürsoy, 2019). Game-based
used Quizizz and the other two groups. This
learning is very efficient because students are motivated to play a game and,
result indicated that Kahoot could be used
in this process, they are learning without even being aware of it (Gee, 2003).
effectively as a learning aid in third grade
Several applications combine these two approaches (SRS and game-based
primary education in the field of natural
learning) to resolve the concern of the lack of student motivation for learning.
sciences.
Each of these SRSs have a set of characteristics which is different from other
Keywords: game-based learning, Kahoot, software of the same type and thus it is reasonable to assume that the effects
Quizizz, primary education, pre-test-post- of the application of various software on student academic achievement can
test research design, science education be different. Unlike most of SRSs which introduce a certain type of game experi-
ence, Kahoot is the first SRS developed from the ground up as a video game
based on the game design principles which are related to intrinsic motivation
Aleksandar Janković, and game flow (Wang & Tahir, 2020). Also, Kahoot is a very popular application
Dragan Lambić used more frequently in game-based learning than other applications of that
University of Novi Sad, Republic of Serbia
type (Goksün, Gürsoy, 2019). Another popular tool is Quizizz, whose application
in education is compared with Kahoot in a number of research studies (Basuki
224
ISSN 1648–3898 /Print/ THE EFFECT OF GAME-BASED LEARNING VIA KAHOOT AND QUIZIZZ ON THE ACADEMIC
ISSN 2538–7138 /Online/ (pp. 224-231)
& Hidayati, 2019; Cadieux Bolden et al., 2017; Chaiyo & Nokham, 2017; Goksün & Gürsoy, 2019; Halim et al., 2020).
Kahoot and Quizizz differ to some extent in the presentation of the questions, feedback, progression speed and
method of questions, technical requirements, etc. Each of these differences can be considered as an advantage (Goksün
& Gürsoy, 2019). Moreover, it is important to bear in mind that the application of educational software needs to be
adapted to the learners’ age, abilities (Lambić et al., 2021) and to the subjects which are studied (Đorić et al., 2021).
Therefore, the effects of the application of different educational software on the academic achievement of students,
of a specific age, in specific subjects, need to be explored.
The efficiency of formative assessment tools is significantly influenced by the characteristics of learning content
and the way in which these tools are used (Đorić et al., 2021). Such tools have less of an effect in natural sciences
education compared with other fields (Kingston & Nash, 2011). A lesser effect can be caused by the volume of feed-
back which is often much bigger in natural scientific fields (Maier et al., 2016). The intellectual abilities of students,
required for learning science education content, depend on their working memory capacity (Gathercole et al., 2004).
The development of the working memory capacity in primary school students (grades 1-4) is closely related to their
problem-solving skills (Swanson et al., 2008) which is important for science education. For this reason, it is important
to examine the effects which can be achieved by using Kahoot and Quizizz in those aspects of primary education
related to the natural sciences.
In the previous literature, there were only a few research studies regarding the application of Kahoot or Quizizz
in science education in primary and secondary schools. In one of these studies in which 39 students participated,
the application of Kahoot in an earth science course reduced the anxiety level of 9th grade students (Lee et al., 2019).
The students in the 12th grade expressed a positive opinion regarding the use of Kahoot in the classroom for learning
mathematics and science (Curto Prieto et al., 2019). In another research, Kahoot was used to improve the motivation
and engagement in lower secondary school science course (de Sousa, 2018). None of these studies dealt with students
involved in primary education. Despite the fact that the subjects dealing exclusively with science content, such as
physics, biology and chemistry, were usually studied from the fifth grade of elementary school, science content was
also studied in the first four grades. For example, the science content in primary education in some countries was
studied in the subject “nature and society” (Diković & Gergorić, 2020).
In a literature review regarding the application of Kahoot in education (Wang & Tahir, 2020) it was determined
that only four percent, out of 93 papers, dealt with primary education. One research study regarding the application
of Kahoot, Quizizz and Socrative in social science classes included 23 second grade primary school students (Cadieux
Bolden et al., 2017), but the results of this study were invalid due to many technical problems (Wang & Tahir, 2020).
Another research dealt with the application of Kahoot, in which 23 fourth grade primary school students participated,
but Kahoot was used in social science classes (Çetin, 2018) and therefore the results of this research do not give us
an insight about the applicability of this tool in science education.
Other studies on the application of Kahoot and Quizizz in education in which primary school students par-
ticipated dealt with the learning of the English language (Tivaraju et al., 2017; Yunus & Tan, 2021). Both Kahoot and
Quizizz were used in research in which 60 primary school students participated (Halim et al., 2020), but there was no
comparison of the effects achieved by these two tools because all the items in the instrument used to measure the
students’ attitudes towards these tools were related to both Kahoot and Quizizz and therefore these tools were not
separately examined. There were also a few research studies dealing with the comparison of the effects achieved by
Kahoot and Quizizz in university and adult learning studies (Basuki & Hidayati, 2019; Chaiyo & Nokham, 2017; Goksün
& Gürsoy, 2019), but with inconsistent results.
A wider review of the literature showed that the application of Kahoot and Quizizz is beneficial in several ways
for students in secondary education and older (Curto Prieto et al., 2019; Goksün & Gürsoy, 2019; Lee et al., 2019; Wang
& Tahir, 2020). The previous research also showed that the application of Kahoot and Quizizz in social sciences and
English courses has multiple positive effects on primary school students (Cadieux Bolden et al., 2017; Halim et al.,
2020; Tivaraju et al., 2017; Yunus & Tan, 2021). However, none of these studies deals with the learning science content
in primary school. Because of the difference in intellectual abilities between students in primary school and those
older (Gathercole et al., 2004; Lambić et al., 2021; Swanson et al., 2008), plus specific characteristics of natural science
content which limits the effects of tools of this kind (Kingston & Nash, 2011; Maier et al., 2016), there is a possibility
that the application of Kahoot and Quizizz in natural science education for primary school students (grades 1-4) can
have lesser effects on the academic achievement in comparison with the results of the previous studies dealing with
the older students’ learning in language and social content. For this reason, research on the effect of the application of
these tools for learning science content in primary education is undertaken. The aim of this research is to investigate
225
(pp. 224-231) ISSN 2538–7138 /Online/
the effects of game-based learning, via Kahoot and Quizizz, on the academic achievement of primary school students
in natural science classes. Based on this aim the following research questions are formulated:
1. What is the effect of game-based learning using Kahoot on the academic achievement of primary
school students in natural science education?
2. What is the effect of game-based learning using Quizizz on the academic achievement of primary
school students in natural science education?
3. Is there a difference in the effects of game-based learning using Kahoot and Quizizz on the academic
achievement of primary school students in natural science education?
General Background
This research was conducted during the September and October of the school year 2021/2022. In this period,
for the students from the 3rd grade of primary school, the natural sciences content was taught through the subject
“nature and society”. In this study a pre-test–post-test research design was used to assess the students’ knowledge
of natural science before and after the experiment. The study began about two weeks after the beginning of the
school year.
Participants
It was planned that 118 third grade primary school students from six classes in Serbia had to fully partici-
pate in this study, but 5 students were not able to participate in all the required activities (pre-test, post-test, or
experimental phase). For this reason, one hundred thirteen students (63 boys and 50 girls; 9 years old) actually
participated. A simple random sampling technique was used to divide the participants of this research into two
experimental and one control groups. The first experimental group used Kahoot as a learning aid (n=43; 24 boys
and 19 girls), the second experimental group used Quizizz (n=41; 23 boys and 18 girls), while the control group
learned the content of natural sciences without using any tool of this type (n=29; 16 boys and 13 girls). Based
on ANOVA, no significant differences between the achievement of different groups in the pre-test were found
(F(2, 110)=3.079, p=.677, p>.05).
The academic achievement test initially consisted of 15 questions from the natural sciences content of the
“nature and society” course (related to the biology content), which were used in schools for several years to assess
the students’ knowledge of this field at the beginning of the school year. The test was reviewed by a panel of 5
teachers and experts from the field of education, and a pilot test was conducted involving 27 students. Questions
with a distinctiveness limit below the 0.20 were excluded from the final version. The final version of the test con-
sisted of 13 questions and the maximum of 42 points having a mean difficulty of .5975 percent. The reliability of
the achievement test was determined using Cronbach’s alpha. For the pre-test, Cronbach’s alpha was .75 while for
the post-test it was .77. Both values exceeded the acceptable level of .70 (Nunnally, 1978).
Before the experimental phase, all the students who participated in the research were tested using the
pre-test. The students from all three groups learned the content in the same way except for one difference. The
students from the experimental groups had the opportunity to participate in the game-based learning sessions
which were provided at the end of each class. The first experimental group used Kahoot in which the content was
presented in the form of a quiz or presentation, while the second experimental group used Quizizz in the same
manner. The game-based learning sessions in the experimental groups were used for revising the content, which
the students learned in class. In the control group, revising the content was conducted in the traditional way
without using any software. All the students participated in this research voluntarily with the signed consent from
their parents. All personal data collected in this research were treated anonymously and confidentially.
226
ISSN 2538–7138 /Online/ (pp. 224-231)
Data Analysis
After four weeks of the experimental phase, the students were tested again to determine the effects of the
applied learning strategies. A one-way independent-samples analysis of covariance (ANCOVA) was used for the
analyses of the results. The independent variables were the three strategies used in natural sciences classes, while
the dependent variables were the results of the post-test. The results of the pre-test were the covariate.
Research Results
The results of the pre-test and post-test are shown in Table 1. The value of the coefficient F(2,107) = 1.17,
p = .314305 (p > .05) confirms the hypothesis of the homogeneity of the regression coefficient.
Table 1
Means and Standard Error Values of Pre-test and Post-test Results
Pre-test Post-test
Group N
M σM M σM
Experimental group Kahoot 43 19.42 1.35 30.07 1.24

Experimental group Quizizz 41 19.41 1.19 27.83 1.18
Control group 29 18.07 1.12 24.52 1.49

In order to eliminate the influence of the pre-test (covariate) on the results of the post-test (dependent vari-
able), ANCOVA was used. The results of this test showed significant differences between the academic achievement
of the three groups after the experiment (F(2,109) = 4.43, p=.014, p < .05) (Table 2). For this reason, the students’
academic achievement was taken to be influenced by the experimental treatments. The effect size of 0.42 was
considered to be large for Cohen’s f test (Cohen 1988).
Table 2
Summary from ANCOVA Test
Source of
variation
Type III sum of squares df MS F p ηp2
Pre-test 2616.962 1 2616.962 70.432 .000 .393

Group 329.525 2 164.762 4.434 .014 .075
Error 4050.003 109 37.156
Total 78338.000 113
In order to obtain detailed results, a post hoc test was used. The results were as presented in Table 3. The
significant difference in the results of the post-test were only established between the academic achievement of
the group which used Kahoot as a learning aid and the control group (p < .05). The post hoc test analysis did not
show significant difference between the results of the groups using Kahoot and Quizizz (p > .05). Also, no signifi-
cant difference was found between the results of the group which used Quizizz and the control group (p > .05).
227
(pp. 224-231) ISSN 2538–7138 /Online/
Table 3
Results from Post Hoc Test
95% confidence interval for

differencea
(I) group (J) group MD (I-J) σM p
Lower bound Upper bound
Kahoot Quizizz 1.557 1.331 .244 -1.080 4.194

Control 4.368* 1.469 .004 1.457 7.279
Quizizz Kahoot -1.557 1.331 .244 -4.194 1.080
Control 2.811 1.483 .061 -.129 5.750
Control Kahoot -4.368* 1.469 .004 -7.279 -1.457
Quizizz -2.811 1.483 .061 -5.750 .129
*p<.05
a
Adjustment for multiple comparisons: Least significant difference (equivalent to no adjustment)
Discussion
The results of this research show that the group which used Kahoot as a learning aid has achieved signifi-
cantly better results in the post-test than the control group. This result is in line with the previous research study,
which demonstrated that game-based learning by using Kahoot increased the achievement and participation of
7th grade students in social studies classes (Turan & Meral, 2018). It is noted that other research papers, although
dealing with positive effects of Kahoot on anxiety level (Lee et al., 2019), positive opinions of students (Curto Prieto
et al., 2019; Çetin, 2018) and improved motivation and engagement (de Sousa, 2018), they did not deal specifically
with the effect of Kahoot on the academic achievement of students. Nevertheless, they reported positive effects
which could be suggested as facilitating the improvement of academic achievement after the use of this tool and
therefore implicitly seen as supporting the results of this research study.
Although some research indicates that younger students are not able to use such tools for learning to the full
extent, due to their limited intellectual abilities (Gathercole et al., 2004; Lambić et al., 2021; Swanson et al., 2008),
the results of this study show that the application of Kahoot in the 3rd grade of primary school can significantly
improve their academic achievement. Also, evidence from previous studies show positive effects for the applica-
tion of Kahoot for learning language and social sciences, but such tools have questioned or limited effects when
learning science (Kingston & Nash, 2011; Maier et al., 2016). For the above-mentioned reasons, it is to be expected
that the effects of the application of this tool on younger students is reduced to some extent. However, bearing
in mind the sample in this study and the content learned during the experiment, there is evidence that Kahoot
can be used effectively in the 3rd grade primary school for natural science education. Therefore, we can claim that
the intellectual abilities of 3rd grade primary school students and the specific characteristics of science education
content cannot significantly reduce the effects of the application of Kahoot.
The group which used Quizizz as a learning aid achieved better results than the control group, but the post
hoc test analysis does not show a significant difference. This is in line with the results of the previous research study
in which pre-service teachers participated (Goksün & Gürsoy, 2019). The lack of a significant difference between
these two groups is unexpected to some extent, due to many advantages which learning with Quizizz offers. The
reason for lesser effects of Quizizz on the academic achievement of students may lie in the limited visual feedback
capacity of the Quizizz application and the manner in which the questions progress (Goksün & Gürsoy, 2019).
The results of this research show that there is no significant difference in the academic achievement of the
groups which use Kahoot and Quizizz as a learning aid, although the group using Kahoot achieve better results
on the post-test. The absence of a significant difference can be explained by the many similar features between
Kahoot and Quizizz. On the other hand, the group using Kahoot indicate significantly better results on the post-test
compared with the control group, while the group using Quizizz do not achieve the same. This fact indicates that
Kahoot may have a greater impact on the academic achievement of students than Quizizz. The results obtained in
228
ISSN 2538–7138 /Online/ (pp. 224-231)
this research are in line with the previous study (Goksün & Gürsoy, 2019) in which no significant difference between
the effects of Kahoot and Quizizz on the academic achievement of pre-service teachers is found. Other studies show
that the application of Kahoot contributes to higher concentration, engagement, enjoyment, perceived learning,
motivation, and satisfaction than the use of Quizizz (Chaiyo & Nokham, 2017). These can be seen to support the
results of this research study indicating that to some extent, Kahoot is more effective than Quizizz.
Research by Basuki and Hidayati (2019) showed that Quizizz is more preferable than Kahoot for participants
learning the English language. Although students’ preferences can influence the efficacy of some tools, other
characteristics can have a greater effect on the learning outcomes. Also, the content presented by some tools can
have a major impact on its effectiveness (Kingston & Nash, 2011; Maier et al., 2016). Therefore, it is possible that
some features of Quizizz are more convenient for learning languages, while the features of Kahoot better support
the demands of science studies. For this reason, the effects of Kahoot and Quizizz on the academic achievement
of students from different fields of study, can be examined in future research.
Future research can also give more attention to the age of students. The results of this research study show
that students from the 3rd grade of primary school can use Kahoot effectively for learning natural sciences. However,
due to the development of the working memory capacity of primary school students, there can be differences
in the ability of students of different ages to effectively use different tools (Lambić et al., 2021). For this reason,
future research can explore the ability of students, from the first two grades of primary education, to effectively
use Kahoot in their learning.
This research study has some limitations. Although the majority of research papers dealing with the applica-
tion of Kahoot in education have a smaller number of participants than this study, there are research studies which
include a few hundred or even a thousand participants (Wang & Tahir, 2020). Also, this research is conducted only
with the participants from one country which could influence the generalizability of the obtained results to some
extent. For these reasons, research with a larger international sample is needed for higher validity.
In this research study, the effects of game-based learning by using Kahoot and Quizizz in the third grade of
primary school education were examined. Bearing in mind the importance of learners’ age when the application
of software in learning was being considered and the specific characteristics of natural science education, it was
seen as important to determine the effects which could be achieved by these tools with learners of a young age.
The intellectual abilities of 3rd grade primary school students and the specific characteristics of science education
content could be a cause for the reduction of the effects of the application of such tools.
The results of this research study showed that there were no significant differences in the academic achievement
of the group which used Quizizz and the other two groups. On the other hand, the experimental group, which used
Kahoot as a learning aid in the natural science course, achieved a significantly better result on the post-test than
the control group. This result showed that Kahoot had a significant effect on the academic achievement in the field
of natural science of the third grade primary school education. For this reason, we were able to conclude that the
intellectual abilities of 3rd grade primary school students (and older ones) and the specific characteristics of science
education content did not reduce significantly the effects of the application of Kahoot. Therefore, Kahoot could be
recommended as a learning aid in primary education starting from the 3rd grade. It is further recommended that
future research should explore the ability of students, from the first two grades of primary education, to effectively
use Kahoot in order to close the knowledge gap regarding the usability of this tool in elementary school. Another
recommendation for future research was that the effects of Kahoot and Quizizz on the academic achievement of
students from different fields of study should be examined and compared.
Based on the results of this research study it could be recommended that teachers in primary school use
Kahoot as a learning aid for revising the content which the students learn in class. Also, learners from the 3rd grade
of primary school and older can use this tool as a learning aid because of its proven positive effects on academic
achievement. Furthermore, consideration needed to be given to the professional development of teachers includ-
ing some training on how to use this tool in the classroom and how to present learning content by using Kahoot
in order to achieve the best effects on the students’ learning process.
229
(pp. 224-231) ISSN 2538–7138 /Online/
References
Basuki, Y., & Yeni, N. H. (2019). Kahoot! or Quizizz: The students’ perspectives. In D. Mulyadi, H. D., Santoso, S., Aimah, & R. Rahim
(Eds.), ELLiC Proceedings of the 3rd English Language and Literature International Conference (pp. 202-211). EAI. http://dx.doi.
org/10.4108/eai.27-4-2019.2285331
Cadieux, B. D., Hurt, J. W., & Richardson, M. K. (2017). Implementing digital tools to support student questioning abilities: A
collaborative action research report. Inquiry in Education, 9(1), Article 2. https://files.eric.ed.gov/fulltext/EJ1171738.pdf
Caldwell, J. E. (2007). Clickers in the large classroom: Current research and best-practice tips. CBE-life Sciences Education, 6(1),
9–20. https://doi.org/10.1187/cbe.06-12-0205
Çetin, H. S. (2018). Implementation of the digital assessment tool Kahoot in elementary school. International Technology and
Education Journal, 2(1), 9–20. http://itejournal.com/articles/implementation-of-the-digital-assessment-tool-kahoot-in-
elementary-school.pdf
Chaiyo, Y., & Nokham, R. (2017). The effect of Kahoot, Quizizz and Google Forms on the student’s perception in the classrooms
response system. In Institute of Electrical and Electronics Engineers (IEEE) (Ed.), 2017 International conference on Digital arts,
media and technology (ICDAMT) (pp. 178–182). IEEE. https://doi.org/10.1109/ICDAMT.2017.7904957
Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Lawrence Erlbaum Associates.
https://doi.org/10.4324/9780203771587
Curto Prieto, M., Orcos Palma, L., Blazquez Tobías, P. J., & Leon, F. J. M. (2019). Student assessment of the use of Kahoot in the
learning process of science and mathematics. Education Sciences, 9(1), Article 55. https://doi.org/10.3390/educsci9010055
Diković, M., & Gergorić, T. (2020). Teachers’ assessment of active learning in teaching nature and society. Economic Research, 33(1),
1265-1279. https://doi.org/10.1080/1331677X.2020.1728563
Đorić, B., Lambić, D., & Jovanović, Ž. (2021). The use of different simulations and different types of feedback and students’ academic
performance in physics. Research in Science Education, 51(5), 1437-1457. https://doi.org/10.1007/s11165-019-9858-4
Gathercole, S. E., Pickering, S. J., Knight, C., & Stegmann, Z. (2004). Working memory skills and educational attainment:
Evidence from national curriculum assessments at 7 and 14 years of age. Applied Cognitive Psychology, 18(1), 1–16.
https://doi.org/10.1002/acp.934
Gee, J. P. (2003). What video games have to teach us about learning and literacy. Computers in Entertainment, 1(1), 20-20.
https://doi.org/10.1145/950566.950595
Goksün, D. O., & Gürsoy, G. (2019). Comparing success and engagement in gamified learning experiences via Kahoot and Quizizz.
Computers & Education, 135, 15–29. https://doi.org/10.1016/j.compedu.2019.02.015
Halim, M. S. A. A., Hashim, H., & Yunus, M. M. (2020). Pupils’ motivation and perceptions on ESL lessons through online quiz-
games. Journal of Education and E-Learning Research, 7(3), 229- 234. https://doi.org/10.20448/journal.509.2020.73.229.234
Kingston, N., & Nash, B. (2011). Formative assessment: A meta-analysis and a call for research. Educational Measurement: Issues
and Practice, 30(4), 28–37. https://doi.org/10.1111/j.1745-3992.2011.00220.x
Kiryakova, G., Angelova, N., & Yordanova, L. (2014). Gamification in education. In Proceedings of the 9th international Balkan
education and science conference. (pp. 1-5). Trakya University. https://www.researchgate.net/publication/320234774_
GAMIFICATION_IN_EDUCATION
Lambić, D., Đorić, B., & Ivakić, S. (2021). Investigating the effect of the use of code.org on younger elementary school students’
attitudes towards programming. Behaviour & Information Technology, 40(16), 1784-1795. https://doi.org/10.1080/01449
29X.2020.1781931
Lambić, D., & Lipkovski, A. (2012). Measuring the influence of students’ attitudes on the process of acquiring knowledge in
Mathematics. Croatian Journal of Education, 14(1), 187-205. https://doi.org/10.15516/cje.v14i1.90
Lee, J. J., & Hammer, J. (2011). Gamification in education: What, how, why bother? Academic Exchange Quarterly, 15(2), 1-5. https://
www.researchgate.net/publication/258697764_Gamification_in_Education_What_How_Why_Bother
Lee, C. C., Hao, Y., Lee, K. S., Sim, S. C., & Huang, C.C. (2019). Investigation of the effects of an online instant response system on students
in a middle school of a rural area. Computers in Human Behavior, 95, 217–223. https://doi.org/10.1016/j.chb.2018.11.034
Maier, U., Wolf, N., & Randler, C. (2016). Effects of a computer-assisted formative assessment intervention based
on multiple -tier diagnostic items and different feedback types. Computers and Education, 95, 85–98.
https://doi.org/10.1016/j.compedu.2015.12.002
Nunnally, J. C. (1978). Psychometric theory. McGraw Hill.
de Sousa, B. F. P. (2018). Engaging students in the evaluation process using co-creation and technology enhanced learning. In
CEUR workshop proceedings. (pp. 1-10). CC-TEL. http://ceur-ws.org/Vol-2190/CC-TEL_2018_paper_4.pdf
Swanson, H. L., Jerman, O., & Zheng, X. (2008). Growth in working memory and mathematical problem solving in
children at risk and not at risk for serious math difficulties. Journal of Educational Psychology, 100(2), 343–379.
https://doi.org/10.1037/0022-0663.100.2.343
230
ISSN 2538–7138 /Online/ (pp. 224-231)
Tivaraju, J., Yunus, M. M., & Badusah, J. (2017). Learning English is fun via Kahoot: Students’ attitude, motivation and perceptions.
In D. Budiman (Ed.), Proceedings of seminar on transdisciplinary education (STEd 2017). (pp. 218–229). Universiti Kebangsaan
Malaysia.
Turan, Z., & Meral, E. (2018). Game-based versus to non-game-based:The impact of student response systems on students’achievements,
engagements, and test anxieties. Informatics in Education, 17(1), 105-116. https://files.eric.ed.gov/fulltext/EJ1177146.pdf
Yunus, C. C., & Tan, K. (2021). Exploring a gamified learning tool in the ESL classroom: The case of Quizizz. Journal of Education
and E-Learning Research, 8(1), 103–108. https://doi.org/10.20448/journal.509.2021.81.103.108
Wang, A. I., & Tahir, R. (2020). The effect of using Kahoot! for learning – A literature review. Computers & Education, 149, Article
103818. https://doi.org/10.1016/j.compedu.2020.103818
Received: January 03, 2022 Revised: February 22, 2022 Accepted: April 08, 2022
Cite as: Janković, A., & Lambić, D. (2022). The effect of game-based learning via Kahoot and Quizizz on the academic achievement
of third grade primary school students. Journal of Baltic Science Education, 21(2), 224-231. https://doi.org/10.33225/jbse/22.21.224
Aleksandar Janković PhD, Associate Professor, Faculty of Education, University of Novi

Sad, Podgorička 4, Sombor, Republic of Serbia.
E-mail: aleksjankovic@gmail.com
Dragan Lambić PhD, Associate Professor, Faculty of Education, University of Novi
(Corresponding author) Sad, Podgorička 4, Sombor, Republic of Serbia.
E-mail: dragan.lambic@pef.uns.ac.rs
ORCID: https://orcid.org/0000-0001-8611-353X
231
ALIGNMENT OF CONCEPTS OF
MEIOSIS AMONG CURRICULUM,
TEXTBOOKS, CLASSROOM
TEACHING AND ASSESSMENT
IN UPPER SECONDARY SCHOOL
IN REPUBLIC OF KOREA
Abstract. If the concepts used in curricu-
lum, textbooks, classroom teaching, and
assessment are not consistent, students
may have difficulty in understanding the
Youngshin Kim,
concepts of science correctly and this may
lead to the formation of misconceptions.
Yun-hye Lee,
The purpose of this study is to measure the
Hyonyong Lee,
alignment of science concepts by analyz- Soo-min Lim
ing the semantic networks of curriculum,
textbooks, classroom teaching, and student
assessment with respect to the contents
on meiosis as covered in Life Science I.
The semantic network method using the Introduction
NetMiner 4.0 program was applied. 11
concepts were extracted from the curricu- Different approaches to formal school education have been advocated
lum commentary. The textbook presented by professionals in education in Republic of Korea. Their efforts, ideas, and re-
36 concepts; classroom teaching presented search have influenced the directions of education. For instance, education is
54 concepts; a total of 23 concepts were an intended and planned activity designed to lead learners towards desirable
presented from assessment. With respect to directions (Kim & Kang, 2012). This planned activity expresses itself in the cur-
alignment of relevant concepts, 6 (gamete, riculum, learning tools (textbooks), and processes of classroom teaching and
chromosome, meiosis, gene, daughter cell assessment. Even though curriculum content is important, everything involved
and cell division) were linked to all 4 sources in the curriculum affects the success or failure of education, from textbooks,
(curriculum, textbooks, classroom teach- curriculum, and teachers’ understanding of the process, to educational imple-
ing, and assessment). These concepts are mentation (Bencze & Hodson, 1999; Loucks-Horsley et al., 1998; Parke & Coble,
mainly used to explain the process of meio- 1997; Powell & Anderson, 2002). Therefore, in order to ensure the successful
sis. It is concluded that the key concepts of implementation of a curriculum and to secure continuity, the overall process
meiosis exhibited alignment in curriculums, of curriculum implementation needs to be investigated in detail.
textbooks, classroom teaching and assess- Furthermore, a frontloading alignment is the proper method for examin-
ment. However, there are many concepts ing whether content presented in a curriculum is successfully implemented
used only in textbooks and classroom (Kang, 2006). The idea of frontloading alignment is to examine the consistency
teaching. The greater number of concepts of the content as guided in order from the overall outline of the curriculum,
in textbooks and teaching can become one as represented in textbooks, then in classroom teaching and finally in assess-
of the causes of a deficit in learning. ment. In this study, the frontloading alignment based on concepts was used
Keywords: concepts’ alignment, meiosis to measure the alignment among curriculum, textbooks, classroom teaching,
concepts, upper secondary school, seman- and assessment stages.
tic network In this view, if misalignment occurs in the flow of curriculum content
through textbooks, classroom teaching or assessment, it wastes students’
learning time, reduces intellectual curiosity, and interferes with creative and
Youngshin Kim, Yun-hye Lee, logical thinking (Blank et al., 2001; Porter & Smithson, 2001). This eventually
Hyonyong Lee, Soo-min Lim leads to a failure in achieving educational objectives (Lim & Kim, 2015). De-
Kyungpook National University,
Republic of Korea spite such urgent and pressing needs, there is a lack of consensus or aware-
ness regarding the methods currently available through the conceptual
framework of frontloading alignment.
232
ISSN 1648–3898 /Print/ ALIGNMENT OF CONCEPTS OF MEIOSIS AMONG CURRICULUM, TEXTBOOKS, CLASSROOM
ISSN 2538–7138 /Online/ (pp. 232-244)
Certain previous studies on alignment have dealt chiefly with the alignments among educational objectives,
textbooks, and assessment questions (Kim et al., 2003; Nam, 2010; Lim et al., 2011). Others have examined alignment
between curriculum and textbooks (Kim et al., 2011) or between learning objectives and educational materials
(Porter, 2002). Yet others have considered alignment between curriculum and classroom teaching on the one hand,
and assessment on the other (Kurz et al., 2010; Liu & Fulmer, 2008; Lu & Liu, 2012). Finally, some analyses have focused
on the relationship between curriculum alignment and students’ academic achievements (Squires, 2012). Studies
have often reported that there exists a misalignment between curriculum, classroom teaching and assessment.
However, most studies have analyzed the alignment between national- or state-level curriculum and text-
books, or between the national-level curriculum and national-level achievement assessment questions. There were
therefore limitations in analyzing the alignment of the curriculums that operate in the classroom; moreover, these
studies had the limitation of lacking substantial evidence because they analyzed curriculum as plan rather than as
teachers’ goals, perceptions, and realities. That is, these studies lacked analysis of actual educational situations in
the classroom and failed to analyze the alignment between educational content and assessment. There is a need
to analyze the alignments for, e.g., science content as presented in curriculum content and textbooks, classroom
teaching, and assessment. In this study, therefore, the alignment of science concepts used in curriculum materials,
textbooks, classroom teaching and assessment was analyzed.
To analyze the alignment of such concepts, a semantic network analysis (hereafter referred to as SNA) was
used as a theoretical framework. SNA enables visual text analytics by expressing hidden structures, based on an
analysis of connected forms shared between the concepts. It has an additional advantage in that the frequency of a
particular concept’s occurrence can be calculated to easily identify its role in the structural relationship of concepts
(Doerfer & Barnett, 1994; Jang & Barnett, 1994; Kim & Kwon, 2016; Lim & Kim, 2015). In this study, SNA was used to
analyze the alignment of science concepts used in curriculums, textbooks, classroom teaching and assessment.
Meanwhile, students are not accustomed to the definition of concepts related to genetics, and these concepts
are highly dependent on each other (Kim et al., 2006). Hence, it is not easy for students to accurately understand
the relation between concepts in the process of learning genetics (Kindfield, 1994; Knippels et al., 2005). In addi-
tion, it is difficult to connect the relationship between alleles and the movement of chromosomes during meiosis
(Knippels et al., 2005; Stewart & Dale, 1989; Tolman, 1982). Thus, students often do not understand the mechanisms
of chromosomal behavior at meiosis, the relationship between chromosome and gene, and the relevance to chro-
mosomes of DNA (Hafner & Stewart, 1995; Stewart, 1982; Stewart & Dale, 1989).
Looking at the current science curriculum in Korea, students learn about cellular organization and chromo-
somes, and the basic principles of inheritance and evolution, in lower secondary school. This learning is followed
in upper secondary school by study of somatic cell division and meiosis. Despite the repetitive learning process,
students have a lot of misconceptions and find it difficult to understand concepts of cell division, especially meiosis
(Hwang & Lee, 2000).
In this context, the language network of curriculum, textbooks, classroom teaching and assessment was
analyzed by using the SNA with respect to meiosis as covered in upper secondary school curriculum in Korea. The
alignment of scientific concepts across curriculum materials, textbooks, classroom teaching and assessment was
analyzed with focus on ‘Life Science I’. This study was intended to contribute to reveal causes of students’ difficulty
in learning about the biological concept of meiosis.
Theoretical Background
Curriculum and Textbooks
In Republic of Korea, “the national curriculum” refers to the documented national-level curriculum, which is
officially announced by the Ministry of Education and includes a national-level curriculum plan and curriculum
commentary. This national curriculum undergoes revision every six to seven years. When the curriculum plan is
revised, the curriculum commentary is developed to explain the curriculum content. The curriculum commen-
tary presents rationales for setting up units, learning contents, the flow of units, and notable points for each unit
(Ministry of Education, Science and Technology, 2011a). The 2009 Revised Science Curriculum has been applied in
upper secondary schools since 2015.
The subjects related to life science in upper secondary school in Republic of Korea are Life Science Ⅰ and Ⅱ.
Life Science I features a unit on meiosis, dealing with the “cell and the continuity of life.” The curriculum objective
233
(pp. 232-244) ISSN 2538–7138 /Online/
concerning meiosis is “to understand the continuity of life by relating chromosomal behaviors at meiosis to genes.”
The curriculum commentary prescribes that the behavior of chromosomes should be associated with genes to
explain meiosis. This unit is designed to further deepen the “reproduction and generation” unit included in the
ninth-grade curriculum; it is further linked to genes and genetic expression in Life Science II (Ministry of Education,
Science and Technology, 2011b).
Textbooks developed as a core part of the curriculum must obtain authorization from the Ministry of Education.
The authorization criterion for science textbooks requires that the educational content reflect what is prescribed in
the curriculum. In addition, materials should be selected according to the level of the learner and the capacity to
spark interest among students, along with topics easily connected with or applied to everyday life. It also requires
learners to perform self-directed learning. Subject areas for upper secondary are to be reviewed by members of
a committee for each course of study (Shim, 2011a). Meanwhile, there are textbooks published by six publishing
companies, based on the 2009 curriculum, for Life Science I.
Alignment
Curriculum alignment refers to the degree to which classroom teaching and assessment are in agreement with
the objectives of the curriculum. In particular, it is important to make sure that curriculum content is consistent
across textbooks, classroom teaching and assessment (Lim et al., 2014). This is because a lack of alignment causes
education to be inefficient and to lose its original purpose (Liu & Fulmer, 2008). It also acts as a deterrent to the
field application of curriculum contents (Coenders et al., 2008; Roach et al., 2008). In other words, when there is a
lack of curricular alignment, and misalignment occurs, the inefficiency of education rises, and the concept being
taught varies. Therefore, curricular alignment is very important (Lim et al., 2011)
An alignment index and a contents map have been used to analyze curricular alignment. For the alignment
index, the SEC model developed by Porter (2004) was used as a measure of two dimensions: topics, and categories
of cognitive demand. The contents map is a representation of topics and performance expectations as contours
in two or three dimensions (Porter, 2002).
Lim et al. (2011) analyzed units on digestion and circulation using the alignment index, while Kim et al. (2015)
carried out a similar analysis, also providing a contents map, regarding upper secondary school units on the cell and
the continuity of life. Lim et al. (2014) also used the contents map, to analyze “stimulation response” in Republic of
Korea’s curriculum for lower secondary school. However, these studies focused only on comparisons between cur-
riculum and lesson plan, and between educational objectives and assessments. In other words, the previous study
failed to analyze the textbooks used in the classroom, the teachers’ teachings, and the actual assessment items.
Semantic Network
A semantic network analysis examines the relationship between components so as to identify the structure of
a system. It enables semantic interpretations due to the existence of the symbolism of interrelationships between
key words (Wasserman & Faust, 1994). In addition, it analyzes and visualizes forms of connection between concepts,
making it easy to specify abstract semantic structures. Thus, it is a method in which social network analysis is ap-
plied to language. It is also an analytical method that can identify the relationship between each concept based
on the shared symbolic meaning (Lee & Ha, 2012; Shim, 2011b). Centrality approaches to the concepts’ relation-
ships, generated by representative indicators measured in the SNA, are divided into degree centrality, closeness
centrality, betweenness centrality and eigenvector centrality.
Degree centrality is used as a method to determine which concepts exist in a central position in the structure
of a concept. Closeness centrality indicates how close one concept is to another, and the distance between each
concept and the others is a core element. Betweenness centrality represents the strength of the intermediary
role of a concept in the network. Eigenvector centrality gives weight to the importance of a connected partner,
representing the degree to which a concept has relationship with other concepts at higher levels of centrality for
the entire network (Doerfel & Connaughton, 2009). Both the key concepts using eigenvector centrality and the
size of node are mainly examined in this study.
234
ISSN 2538–7138 /Online/ (pp. 232-244)
Analysis Target
The curriculum alignment was analyzed by using the semantic network for meiosis in Life Science I. The
concept related to meiosis in the 2009 Revised Science Curriculum Commentary was used to identify the network
of concepts. The concept of meiosis was selected because it is the concept that upper secondary school students
in Republic of Korea had the most difficulty and high psychological hierarchy in learning among genetic-related
concepts (Lim, 2019). Two upper secondary schools offering Life Science I were selected to analyze these concepts
as used in their classroom teaching and assessment. The classroom teaching concerning meiosis during the unit
on cells was recorded to analyze the concepts used in the classroom. The assessment questions related to the
concepts of meiosis in written examinations were selected for the analysis for assessment.
Classroom Teaching
The two upper secondary schools were located in a city with a population of 2.5 million. Both were public
schools, and the students’ academic achievement levels were in the top 30% for the Republic of Korea. The two
teachers who gave the classes were men. One had four years of classroom experience, and the other had nine.
Each meiosis lesson lasted 50 minutes, with a teacher-centered approach. Each teacher delivered the cur-
riculum contents in a manner that explained the process and significance of meiosis. Their explanation, during
“Meiosis 1,” focused on the process of meiosis, which involves the formation of bivalent chromosomes. In “Meiosis
2,” each teacher explained the generation of four daughter cells without replication of genetic materials, along
with changes in the amount of DNA. They also explained genetic diversity in the significance of meiosis. Finally,
both teachers gave explanations through a comparison of meiosis and mitosis.
Data Collection
Textbooks, recorded classes, and the teachers’ assessment questions were collected to analyze align-
ment based on the 2009 Revised Curriculum in the Republic of Korea. The recordings of the actual classes were
transcribed to collect the concepts used in the classroom. Both teachers used written questions for students’ as-
sessment. Of the 48 questions, six were applicable to meiosis. The questions were focused on the two units, and
31 out of 48 questions were about the cell and the continuity of life. The unit regarding the cell and the continuity
of life is a medium unit concerning chromosomes, mitosis, meiosis, and heredity. Four questions were based on
the chromosome unit, 11 questions on mitosis, six on meiosis, and ten on heredity. The six questions, which are
all multiple-choice questions, were analyzed in this study because this study focused on the biological concept
of meiosis. Conducting research by providing evaluation questions that have already been used is subject to ex-
emption as research conducted in relation to practice within the scope of the school’s curriculum in accordance
with Article 2 of the Elementary and Secondary Education Act and Article 2 of the Higher Education Act (KoNIBP,
2018). In addition, the provided evaluation questions were used only for research, and consent was obtained after
notifying the school, students, and parents that the students’ personal information was not collected or recorded.
Data Analysis
The sentences related to meiosis, as collected from the curriculum commentary, classroom teaching, and
assessment were transcribed to analyze the semantic network. The transcribed statements were converted into a
text file (text.txt) and then preprocessed it using NetMiner 4.0 so as to select only the relevant concepts. For this
stage, we excluded other concepts except for related concepts, and unified synonyms such as “meiosis 1” and
“heterotypic division”, and “meiosis 2” and “homotypic division”.
After thus reorganizing and selecting the concepts, we visualized each of the semantic networks as a 1-mode
network, organized for curriculum, textbooks, classroom teaching and assessment. On the other hand, each was
visualized as a 2-mode network in order to analyze the alignment across curriculum, textbooks, classroom teach-
ing and assessment (Jeong et al., 2019; Kim & Kwon, 2016). Table 1 provides the frequency and example of the
eigenvector centrality of concepts used in the curriculum network in Figure 1.
235
(pp. 232-244) ISSN 2538–7138 /Online/
Table 1
Examples of Frequency and Eigenvector Centrality: Concepts Used in the Curriculum
Eigenvector Eigenvector
Concept Frequency Concept Frequency
centrality centrality
Meiosis 3 .550 Fertilization 1 .244
DNA replication 1 .063 Chromosome 2 .443
Chromosome separa-
1 .021 Genetic material 1 .168
tion
Continuity of life 1 .097 Gene 2 .292
Gamete 2 .445 Offspring 1 .097
Cell division 1 .329
Research Results
Network
Curriculum
The eleven concepts from the curriculum commentary were extracted for the analysis of network. The frequency
was highest for “meiosis” (three times), followed by “chromosome,”“gene,” and “gamete” (twice each). Concepts with
higher eigenvector centrality were arranged in order as follows: “meiosis,” “chromosome,” “gamete” and “cell divi-
sion.” The concepts extracted from the curriculum form a network structure based on “gene” and “chromosome,”
in which “gene” is linked to “chromosome,” “fertilization,” “offspring” and “continuity of life,” while “chromosome” is
linked to “meiosis,” “cell division,” “gene,” and “genetic material.” (Figure 1)
Figure 1
Semantic Network of Concepts Used in the Curriculum
The curriculum requires teachers to introduce meiosis, and the various aspects of its process, outcomes, and
significance, based on the concept of the chromosome (Ministry of Education, Science and Technology, 2011b). In
line with this requirement, the “chromosome” was found at the center of the curriculum network. In this network,
the concepts that represent the meiosis process include “gene,” “chromosome,” “meiosis,” and “cell division.” These
236
ISSN 2538–7138 /Online/ (pp. 232-244)
exhibited higher relative importance and were emphasized over concepts representing the outcomes and signifi-
cance, such as “offspring” and “continuity of life.”
Textbooks
A total of 36 concepts related to meiosis was extracted from the textbook. The frequency was highest for
“gamete” (15 times), followed by “heterotypic division” (13 times), “daughter cell” (11 times), “meiosis” (11 times),
and “homologous chromosome” (10 times). We arranged the concepts with higher eigenvector centrality in order,
thus: “number of chromosomes,”“comparative DNA values,”“chromosome disjunction,”“mother cell,”“daughter cell,”
and “homologous chromosome.”
“Chromosome,” “number of chromosomes,” “homologous chromosome” and “homotypic division” were lo-
cated at the center of the network. “Chromosome” is linked to “mitosis,”“chromonema,”“cells and continuity of life.”
“Number of chromosomes” is linked to “chromosome disjunction,”“comparative DNA value,”“daughter cell,”“mother
cell” and “fertilized egg.” In addition, “homologous chromosome” is linked to “prophase,”“chromosome disjunction,”
“meiosis” and “heterotypic division.” “Homotypic division” is linked to “metaphase,” “anaphase,” “telophase” and
“mitosis.” (Figure 2)
Figure 2
Semantic Network of Concepts Used in the Textbook
Of the three categories identified this way (chromosome, number of chromosomes, and homologous chromo-
some), division and significance of meiosis was presented in the curriculum. The process of meiosis is explained in
connection with the concepts “chromosome,”“homologous chromosome” and “homotypic division.”The outcomes
and significance of meiosis are explained in terms of the central concept of “number of chromosomes.” That is,
although the concepts related to meiosis were not connected to each other in the curriculum, these concepts
were connected in the textbook.
Classroom Teaching
A total of 54 concepts from the recorded lectures relating to meiosis was extracted from classroom teaching.
The number of these concepts showed a marked increase in comparison with the curriculum and textbook. The
concept with the highest frequency was “chromosome” (116 times), followed by “number of chromosomes” (101
times), “gamete” (90 times), “gene” (60 times) and “prophase” (47 times). The concepts with higher eigenvector cen-
trality were arranged in order from “prophase” to “heterotypic division,” then “metaphase,”“cytokinesis,”“homotypic
division,” “telophase” and “homologous chromosome.”
237
(pp. 232-244) ISSN 2538–7138 /Online/
Figure 3
Semantic Network of Concepts Used in the Classroom Teaching
Figure 3 shows networked concepts extracted from the lectures on meiosis. The teachers’ teachings about
meiosis were focused on cells, chromosomes, and the number of chromosomes. They linked discussion of cells
to the model of a plant cell, with interphase and chromosome. The discussion of chromosomes showed links to
concepts such as heterotypic division, homotypic division, nuclear fission, chromosome condensation and ana-
phase. The process of meiosis was explained based on the chromosome concepts. The number of chromosomes
was linked to chromosomes, gametes, and mother cells. Thus, the teachers gave explanations about meiosis with
a focus on the number of chromosomes, showing that their lectures exhibited similar patterns to the conceptual
network shown in the textbook.
In class, the teachers explained the process of meiosis first and then the significance of meiosis. To explain
the process, they presented concepts related to the behavior of chromosomes, such as organelles, chromosomes,
and homologous chromosomes. As for the significance of meiosis, the explanations focused on the concepts of
chromosomes and comparative DNA values. Even in the curriculum, organelles, chromosomes, and homologous
chromosomes are repeatedly explained in relation to meiosis, as the concepts to be recycled from previous teachings.
Of the numerous concepts extracted from the lectures, the largest number of concepts was related to the
process of meiosis. This is because of the increasing number of concepts associated with organelles, as organelles
were prescribed to be presented for explanations.
There was a prominent tendency to explain the process and outcomes of meiosis in relation to chromosomes.
That is, the frequency of “chromosomes” and “number of chromosomes” was very high, and in addition to these
concepts, “gamete,” “homologous chromosome” and “bivalent chromosome” appeared in close spatial proximity
on the graph.
Even during the classes related to the process in which reproductive cells undergo meiosis, giving rise to
daughter cells, the teachers emphasized changes in number of chromosomes, nuclear phases, and comparative
DNA values. However, changes in genetic information were not detailed. This may be why students have often
failed to perceive the various possibilities of gene combinations in daughter cells. As described above, students
have often appeared relatively unaware of differences in genetic information between mitosis and meiosis.
Assessment
The number of concepts extracted from assessments for meiosis was 23. The frequency was highest for “cell” (13
times), followed by “heterotypic division” (6 times), “animal cell” (6 times), “homotypic division” (5 times), “meiosis” (4
times), “gamete” (4 times), and “metaphase” (4 times). The concepts with higher eigenvector centrality were arranged
in order from “cell” to “metaphase,” “heterotypic division,” “G1 phase,” “gamete,” “homotypic division,” and “meiosis.”
238
ISSN 2538–7138 /Online/ (pp. 232-244)
Figure 4
Semantic Network of Concepts Used in the Assessment
In the conceptual network for assessment in meiosis, the network is composed mainly around “cell” (Figure
4). This concept showed links to “G1 phase,”“gamete,”“chromosome disjunction,”“metaphase,”“homotypic division”
and “heterotypic division.” That is, the assessment questions reflected the process of meiosis.
The concepts associated with the outcomes of meiosis, such as “number of chromosomes,” “daughter cells,”
“genes,” and “comparative DNA values,” were not linked to the process of meiosis. However, in class, these concepts
had been connected to “gamete” and to “S phase.” This suggests that the process, outcomes, and significance of
meiosis were not connected to each other, but instead each stage was divided from the others in assessment. The
following Figure 5 is an example of the assessments question.
Figure 5
Example of the Assessment Question
(a) (b)
The figure (a) shows the cell cycle when the reproductive cell of an animal (2n) is formed, and (b) shows a part of
the meiosis process that occurs in the animal.
Choose a correct explanation from the following options.
A. (b) is observed in M2 phase of (a).

B. There is no change in nuclear phase as a result of cell division in (b).
C. The cell of (b) has the same number of chromosomes as that of daughter cells made from the result of
the M1 phase.
239
(pp. 232-244) ISSN 2538–7138 /Online/
In their meiosis content, the assessment questions showed preparation based on the process and significance
of meiosis. With regard to the process, the assessment focused on the stages and their characteristics. Regarding
the significance of meiosis, the assessment focused on comparative DNA values and number of chromosomes.
This suggests that the curriculum content, designed to explain meiosis in relation to the behavior of chromosomes
(reference-curriculum commentary) was faithfully implemented in the assessments.
Alignment
Figure 6 shows alignment among the concepts used in curriculum materials, textbooks, classroom teachings,
and assessment questions. There were six concepts in total (“gamete,” “chromosome,” “meiosis,” “gene,” “daughter
cell” and “cell division”) linked across all four curriculum stages. These were the most-used concepts explaining
the process of meiosis.
Four concepts appeared consistently in textbooks, classroom teaching and assessment: “gamete,” “chromo-
some,”“cell division” and “meiosis.” Seven appeared only in classroom teaching and students’ assessment, including
“cell cycle,” “nuclear fission,” “animal cell,” “S phase” and “chromosome disjunction.” Fifteen concepts appeared in
both textbooks and classroom teachings, and they included “gene,”“mother cell,”“homologous chromosome” and
“chromosome condensation.” The concepts of “gene” and “DNA replication” appeared in curriculum guidelines,
textbooks, and classroom teachings.
Meanwhile, many of the concepts appeared only in classroom teaching, and they included 16 concepts, such
as “plant cell,”“unicellular cell,”“centrosome” and “chiasma.” Although they were not used in the assessments, most
of these concepts were presented in order to explain the process.
In other words, the concepts extracted from the assessments were those that had appeared either in the text-
books or in class. In the assessment stage, teachers were found to have made the assessment questions out of the
concepts mentioned in class. However, presenting many concepts in class could contribute to enhance students’
understanding of learning. Therefore, it seems that improvements need to be made in class and evaluations based
on the concepts used in curriculums, classroom teaching and assessment.
Figure 6
Alignment between the Concepts Used in Curriculum, Textbook, Classroom Teaching, and Assessment
240
ISSN 2538–7138 /Online/ (pp. 232-244)
Discussion
The alignment of concepts in the Republic of Korea’s upper secondary school Life Science - curriculum is ex-
amined based on curriculum materials, textbooks, classroom teachings and assessments regarding meiosis, using
semantic networks. A comparison of the four sources of research data reveals that the number of concepts is much
higher in classroom teaching than in the other sources. It seems to be due to teachers’ intentions to teach as many
concepts as possible, although there is a possibility that the various related concepts are used in class to explain
the contents of the presented curriculum (Jeong et al., 2010). However, presenting a large number of concepts to
students is likely to be a distraction from learning process (Kim & Kwon, 2016). Therefore, it seems that there is a
need to establish guidelines for a desirable degree to which the concepts presented in the curriculum are taught
(Jeong et al., 2010; Kim et al., 2009).
The teachers’ classroom teachings are based on their respective textbooks. The curriculum guidelines present
key concepts in relation to meiosis, according to three categories: Process, outcomes, and significance of meiosis. On
this basis, the textbook describes the process, outcomes, and significance of meiosis, with a focus on the concept of
the chromosome. Even in the classroom, the meiosis process is explained based on the key concepts presented in
textbooks (Figures 2 and 3). Meanwhile, the assessments focus on the process and significance of meiosis, except
for outcomes of meiosis, emphasizing chromosomes.
It is found that the concepts connecting curriculum materials, textbooks, classroom teaching, and assessment
are linked by the key concepts of meiosis. That is, the four concepts of “gamete,” “chromosome,” “cell division” and
“meiosis” connect across the four stages of the curriculum. These concepts serve as a key to understanding meiosis.
In other words, the process, outcomes, and significance of meiosis can be explained by the behavior of chromo-
somes, so it appears reasonable to suggest that the content presented in the curriculum is faithfully reflected in
textbooks, classroom teaching, and student assessment.
In previous studies that analyzed alignment, a comparison was made between learning goals, textbooks,
and assessment questions (Kim et al., 2003; Nam, 2010; Lim et al., 2011), curriculum materials and textbooks (Kim
et al., 2011), learning goals and educational materials (Porter, 2002), curriculum, classroom teaching, and assess-
ment (Kurz et al., 2010; Liu & Fulmer, 2008; Lu & Liu, 2012). This is an analysis of the alignment of some fragmented
parts of the teaching-learning process, and thus the overall teaching-learning process could not be analyzed. As
shown in the study by Squires (2012) that the higher the degree of alignment, the higher the student’s achieve-
ment. Therefore, it is very necessary to make efforts to increase alignment. In particular, more efforts are needed
to increase alignment in the actual class. In this study, it is meaningful that the actual class situation was analyzed,
and the alignment was analyzed by visualizing the concepts actually used as networks in detail. As a result of this
study, the alignment on meiosis in the Republic of Korea’s upper secondary school was relatively high. Neverthe-
less, students still have difficulties in learning about meiosis and showed low achievement. Further studies are
required to analyze these causes.
In this study, the alignment is analyzed in curriculum materials, textbooks, classroom teaching, and assess-
ment, using SNA with respect to the concepts on meiosis used in the Republic of Korea’s upper secondary school
Life Science Ⅰ curriculum, examining the relationship among these concepts.
First, the key concepts of meiosis exhibited alignment across the curriculum. The concepts “meiosis,”“reproduc-
tive cells,” “chromosomes,” “cell division” and “gene” appeared at all stages of the curriculum (materials, textbooks,
classroom teaching, and assessment). Meanwhile, as the process, outcomes and significance of meiosis are stated
separately in the curriculum, textbooks and classroom teaching were found to comply fully within this curriculum.
It was also found that the curriculum commentary was faithfully implemented, as meiosis was explained based
on the behavior of chromosomes. However, the concepts that are common at all stages were not located at the
center of the network in the curriculum stages presented in textbooks and classroom teaching. This is because
the question of centrality became unclear as the number of concepts rose. In the textbooks, “heterotypic division,”
“number of chromosomes,” and “daughter cell” were all located at the center, whereas “cell,” “chromosome,” and
“number of chromosomes” were located at the center in both of the studied classrooms.
Second, the drastically increased number of concepts in textbooks and classroom teaching could be one
cause of a deficit in learning. Due to an increase in the number of concepts in textbooks and classroom teaching,
241
(pp. 232-244) ISSN 2538–7138 /Online/
the network of concepts became more complicated, possibly making the relation of concepts unclear. Although
the concepts were clearly connected to the basis on chromosomes called for in the curriculum guidelines, the
concepts appeared to be unclear due to the complicated network in textbooks and classroom teaching. Therefore,
it is likely that students have had difficulties in acquiring the concepts because they need to understand many
concepts with complicated network structures. In this regard, there is a need for measures to reduce the number
of concepts presented in textbooks and classroom teaching.
The alignment and network of concepts used for meiosis across the curriculum showed numerous concepts
that were used only in textbooks and in teachers’ teachings. This suggests that the concepts used in the textbooks
and classroom teaching should be reduced to increase students’ understanding of meiosis.
When teachers teach various scientific concepts in the unit, they need to focus on the key concepts of the
unit and reflect the networks of concepts for better classroom teaching and assessment. In particular, within each
network of concepts used in classroom teaching and assessment, the key concepts should be located at the center
and properly connected with other concepts.
The network of concepts that focus on students’ learning was not analyzed in this study. It could be necessary
to compare the conceptual status and changes between the concepts used by the teacher in the actual classroom
teaching and the concepts acquired by the students after class based on the network of concepts. The key con-
cepts and network could be expected to help us find out issues and problems related to the students’ conceptual
learning and deficit.
References
Bencze, L., & Hodson, D. (1999). Changing practice by changing practice: Toward more authentic science and
s c i e n c e c u r r i c u l u m d e v e l o p m e n t . J o u r n a l o f R e s e a r c h i n S c i e n c e Te a c h i n g , 3 6 ( 5 ) , 5 2 1 - 5 3 9 .
https://doi.org/10.1002/(SICI)1098-2736(199905)36:5<521::AID-TEA2>3.0.CO;2-6
Blank, R. K., Porter, A., & Smithson, J. (2001). New tools for analyzing teaching, curriculum and standards in mathematics and science.
Council of Chief State School Officers.
Chung, Y., & Kim, E. (2002). Students’ understanding of chromosomes, genes, and genetic information. Biology Education, 30(4),
316-324.
Coenders, F., Terlouw, C., & Dijkstra, S. (2008). Assessing teachers’ beliefs to facilitate the transition to a new chemistry curriculum:
What do the teachers want? Journal of Science Teacher Education, 19(4), 317-335. https://doi.org/10.1007/s10972-008-9096-5
Doerfel, M. L., & Barnett, G. A. (1999). A semantic network analysis of the international communication association. Human
Communication Research, 25(4), 589-603. https://doi.org/10.1111/j.1468-2958.1999.tb00463.x
Doerfel, M., & Connaughton, S. (2009). Semantic networks and competition: Election year winners and losers in U.S. televised
presidential debates, 1960–2004. Journal of the American Society for Information Science and Technology, 60(1), 201-218.
https://doi.org/10.1002/asi.20950
Hafner, R., & Stewart, J. (1995). Revising explanatory models to accommodate anomalous genetics phenomena: Problem solving
in the “context of discovery”. Science Education, 79(2), 111-146. https://doi.org/10.1002/sce.3730790202
Hale, J. A., & Dunlap, R. F. Jr. (2010). An educational leader’s guide to curriculum mapping: Creating and sustaining collaborative
cultures. Corwin Press.
Hwang, S., & Lee, J. (2000). Suggestions to improve high school students’ understanding about meiosis. Biology Education, 28(4),
342-347.
Jang, H., & Barnett, G. A. (1994). Cultural differences in organizational communication: A semantic network analysis. Bulletin of
Sociological Methodology, 44(1), 31–59. https://doi.org/10.1177/075910639404400104
Jeong, J., Lee, Y., & Kim, Y. (2010a). A survey of the background concept for genetics domain proposed by biology teacher.
Secondary Education Research, 58(2), 25-47.
Jeong, S., Kim, Y., & Lim, S. (2019). Consistency of concepts used by elementary and middle school students in the observation,
question generation, and experimental design stages. Biology Education, 47(3), 273-289.
Jeong, J., Yoon, J., Sohn, J., Lee, T., & Kim, Y. (2010b). A Study on the recognition about cell and gene domain to be taught in
elementary, high schools by high school biology teacher. Journal of the Korean Association for Science Education, 30(5),
636-646.
Kang, H. (2006). A new paradigm of subject matter education: Theory and practice of subject matter education. Academy press.
Kim, D., Lim, S., Kim, Y., & Seo, K. (2015). Analysis of curriculum alignment of ‘Cell and Continuity of Life’ unit in life science Ⅰ.
Biology Education, 43(4), 445-454.
242
ISSN 2538–7138 /Online/ (pp. 232-244)
Kim, H., Lee, D., & Lee, J. (2006). Analysis of conceptual level and organization on the unit related with continuity of the life in
lower secondary school science textbook. Biology Education, 34(3), 342-354.
Kim, J., Yoon, I., & Lee, W. (2011). An analysis of alignment between content topics of informatics curriculum standards and
textbooks. The Secondary Education Research, 59(2), 381-414.
Kim, M., & Kang, H. (2012). Analysis of instructional objectives of the elementary Korean curriculum based on Bloom’s revised
taxonomy of educational objectives. The Journal of Curriculum Studies, 30(1), 27-58.
Kim, Y. (2009). Analysis of 8th grade students’ biology concepts for before and after being taught in ‘Stimulus and Reaction’ unit.
Biology Education, 37(4), 459-472.
Kim, Y., Jeong, J., Yoon, J., & Sohn, J. (2009). The study on the recognition about evolution, biodiversity and ecology domain to
be taught in elementary, high schools by secondary biology teacher. Secondary Education Research, 57(3), 29-52.
Kim, Y., & Kwon, H. (2016). A comparative study of articulation on science textbook concepts and extracted concepts in learning
objectives using semantic network analysis: Focus on life science domain. Journal of Korean Elementary Science Education,
35(3), 377-387.
Kim, Y., Kwon, Y., Yang, I., & Chung, W. (2003). Exploring reasoning patterns of students’ scientific thinking, inquiry activities in
textbook, and examination items in Korea. Journal of the Korean Association for Research in Science Education, 23(4), 309-318.
Kindfield, A. C. H. (1994). Understanding a basic biological process: Expert and novice models of meiosis. Science Education, 78(3),
255-283. https://doi.org/10.1002/sce.3730780308
Knippels, M. C., Waarlo, A. J., & Boersma, K. T. (2005). Design criteria for learning and teaching genetics. Journal of Biological
Education, 39(3), 108-112. https://doi.org/10.1080/00219266.2005.9655976
Korea National Institute for Bioethics Policy [KoNIBP] (2018). Bioethics guidelines for researchers (pp. 20-21). National Bioethics
Committee.
Kurz, A., Elliott, S. N., Wehby, J. H., & Smithson, J. L. (2010). Alignment of the intended, planned, and enacted curriculum in
general and special education and its relation to student achievement. The Journal of Special Education, 44(3), 131-145.
https://doi.org/10.1177/0022466909341196
Lee, J., & Ha, M. (2012). Semantic network analysis of science gifted middle school students’ understanding of fact,
hypothesis, theory, law, and scientificness. Journal of the Korean Association for Science Education, 32(5), 823-840.
https://doi.org/10.14697/jkase.2012.32.5.823
Lee, S. (2014). A content analysis of journal articles using the language network analysis methods. Journal of the Korean Society
for Information Management, 31(4), 49-68. https://doi.org/10.3743/KOSIM.2014.31.4.049
Lim, S. (2019). An analysis of error types of solving the cell division problem based on semantic network analysis. Journal of
Learner-Centered Curriculum and Instruction, 19(11), 249-272. http://dx.doi.org/10.22251/jlcci.2019.19.11.249
Lim, S., Hong, H., Lee, T., Kwon, D., & Kim, Y. (2014). Analysis of curriculum alignment index and curriculum contents map of 8th
grade on ‘Digestion and Circulation’ unit. Biology Education, 42(1), 42-53.
Lim, S., Jang, B., Jeong, J., Lee, J., & Kim, Y. (2011). An analysis of alignment between instructional objectives, student’s acquired
knowledge and evaluation items in ‘Stimulus and Reaction’ unit in science class of lower secondary schools. Biology
Education, 39(1), 95-105.
Lim, S., & Kim, Y. (2015). An analysis of articulation in life science revision 2009 curriculum with semantic network analysis. Journal
of Korean Elementary Science Education, 43(1), 84-96.
Liu, X., & Fulmer, G. (2008). Alignment between the science curriculum and assessment in selected NY State regent exams. Journal
of Science Educational and Technology, 17(4), 373-383. https://doi.org/10.1007/s10956-008-9107-5
Loucks-Horsley, S., Hewson, P. W., Love, N., & Stiles, K. (1998). Designing professional development for teachers of science and
mathematics. National Institute for Science Education.
Lu, Q., & Liu, E. (2012). Alignment between high school biology curriculum standard and the standardized tests of four provinces
in China. Journal of Biological Education, 46(3), 149-164. https://doi.org/10.1080/00219266.2011.645855
Min, J., Park, J., & Kim, H. (2004). Analysis of students’ alternative conceptual frameworks on genetics and evolution. Biology
Education, 32(2), 158-172.
Ministry of Education, Science and Technology (2011a). Science curriculum. Korea Institute of Curriculum & Evaluation.
Ministry of Education, Science and Technology (2011b). Upper secondary school science curriculum instruction book. Korea Institute
of Curriculum & Evaluation.
Nam, H. (2010). A review on conformity of teacher and student’s education. Yoksa Kyoyuk Ronjip: History Education Review, 45(1),
33-64.
Parke, H. M., & Coble, C. R. (1997). Teachers designing curriculum as professional development: A model for transformational
science teaching. Journal of Research in Science Teaching, 34(8), 773-789.
Porter, A. C. (2002). Measuring the content of instruction: Uses in research and practice. Educational Researcher, 31(7), 3-14.
https://doi.org/10.3102/0013189X031007003
Porter, A. C. (2004). Curriculum assessment. In Green, J., Camilli, G., & Elmore, P. (Eds). Contemporary methods for research in
education. American Educational Research Association.
Porter, A. C., & Smithson, J. L. (2001). Chapter IV: Are content standards being implemented in the classroom? A methodology
and some tentative answers. Teachers College Record: The Voice of Scholarship in Education, 103(10), 60-80. https://doi.org
/10.1177/016146810110300804
Powell, J., & Anderson, R. D. (2002). Changing teachers’ practice: Curriculum materials and science education reform in the USA.
Studies in Science Education, 37(1), 107-135. https://doi.org/10.1080/03057260208560179
243
(pp. 232-244) ISSN 2538–7138 /Online/
Roach, A. T., Niebling, B. C., & Kurz, A. (2008). Evaluating the alignment among curriculum, instruction, and assessments: Implications
and applications for research and practice. Psychology in the Schools, 45(2), 158-176. https://doi.org/10.1002/pits.20282
Shim, J. (2011a). The difference between approved and authorized textbook. Education Square, 41, 40-43.
Shim, J. (2011b). Analysis of conflict frames using semantic network analysis. The Korea Public Administration Journal, 20(2), 183-212.
Squires, D. (2012). Curriculum alignment research suggests that alignment can improve student achievement. A Journal of
Educational Strategies, 85(4), 129-135. https://doi.org/10.1080/00098655.2012.657723
Stewart, J. (1982). Difficulties experienced by high school students when learning basic mendelian genetics. The American Biology
Teacher, 44(2), 80-89. https://doi.org/10.2307/4447413
Stewart, J., & Dale, M. (1989). High school students’ understanding of chromosome/gene behavior during meiosis. Science
Education, 73(4), 501-521. https://doi.org/10.1002/sce.3730730410
Tolman, R. R. (1982). Difficulties in genetics problem solving. The American Biology Teacher, 44(9), 525-527.
https://doi.org/10.2307/4447599
Wasserman, S., & Faust, K. (1994). Social network analysis: Methods and applications (Vol. 8). Cambridge University Press.
https://doi.org/10.1017/CBO9780511815478
Cite as: Kim, Y.-S., Lee, Y.-H., Lee, H.-Y., & Lim, S.-M. (2022). Alignment of concepts of meiosis among curriculum, textbooks,
classroom teaching and assessment in upper secondary school in Republic of Korea. Journal of Baltic Science Education, 21(2),
232-244. https://doi.org/10.33225/jbse/22.21.232
Youngshin Kim PhD, Professor, Department of Biology Education, College of Education,

Kyungpook National University, 80 Daehakro, Bukgu, Daegu, 41566,
Republic of Korea.
E-mail: kys5912@knu.ac.kr
Yun-hye Lee PhD, Professor, Department of Biology Education, College of Education,
Kyungpook National University, 80 Daehakro, Bukgu, Daegu, 41566,
Republic of Korea.
E-mail: dia1225@naver.com
Hyonyong Lee PhD, Professor, Department of Earth Science Education, College of
Education, Kyungpook National University, 80 Daehakro, Bukgu,
Daegu, 41566, Republic of Korea.
E-mail: hlee@knu.ac.kr
Soo-min Lim PhD, Research Professor, Science Education Research Institute,
(Corresponding author) Kyungpook National University, 80 Daehakro, Bukgu, Daegu, 41566,
Republic of Korea.
E-mail: bbolsar@naver.com
244
IMPLEMENTATION OF
INNOVATIVE ARTIFICIAL
INTELLIGENCE COGNITIONS ISSN 1648-3898 /Print/
WITH PROBLEM-BASED ISSN 2538-7138 /Online/
LEARNING GUIDED TASKS

TO ENHANCE STUDENTS’ Abstract. This research conceived multi-
PERFORMANCE IN SCIENCE variate learning concepts and aimed at the
implementation of integrating artificial
intelligence (AI) life learning of face recogni-
tion, self-driving, and robots’ situational
issues into problem-based learning (PBL) in
King-Dow Su natural science. This research put forward
problem-solving and group cooperation
in which students worked together to
complete learning tasks from encountering
problems to planning problem-solving. All
127 participants engaged in this research
Introduction
from the Hungkuo Delin University of
Technology in Taiwan. 7 experts helped
As we all know, the 21st century is an era of emerging technologies and
construct the content validity, construct
creativity, and new industries are rapidly emerging. Artificial Intelligence (AI),
validity, and reliability through instru-
the internet of things (IoT), and 5G are quietly entering everybody’s circle
ment developments of 5 open-ended test
of life. The application of AI in the social field is becoming more diversified,
questions and 31 learning attitude items.
such as transportation, agriculture, scientific research, health and safety,
The statistical analyses revealed that those
public domain, virtual reality and augmented reality, marketing, advertis-
students improved their scientific cognitive
ing, criminal justice, financial services, etc. With the development of global
skills of problem-solving by quantitative
AI, AI seems to have become a contemporary science and continues to light
analyzes and presented their logical rea-
up students’ interest in the AI social application. Therefore, AI acts as an
soning and activation ability by qualitative
evaluation tool for an application not only in the own field but also in cross-
ones. In addition, the descriptive statistical
disciplinary (Rihtaršič et al., 2016). However, when AI has become the core
analysis of learning attitude revealed that
tool of the fourth technological revolution in the new century, talent training
students had positive thinking attributes
has become a positive issue of discussion today (Su, 2021).
for the application of cross-disciplinary
Furthermore, AI education research still has more limitations and chal-
PBL and AI new knowledge. These findings
lenges for the knowledge construction of most students in the field of inte-
had implications for the AI cognitions with
grating natural science teaching (Benitti, 2012). Therefore, some researchers
PBL educational environment to increase
(Altin & Pedaste, 2013; Sullivan & Heffernan, 2016) pointed out that a new
innovation in the classroom. Pedagogical
pedagogy still needs to integrate more research and development, active AI
suggestions presented that students’ learn-
learning strategies, and effectiveness in natural science. Accordingly, some
ing growth in the environment requires
advanced countries get involved actively in AI educational integration and
long-term cultivation to exhibit significant
positive research (Alimisis, 2013). Integrating AI emerging technologies
learning outcomes in the future.
into natural science learning is an activating learning strategy that will help
improve their learning performance (Huan, 2018). The learning strategy is in Keywords: artificial intelligence, cross-
proportion to their interests and has substantive needs to be selected. For ex- disciplinary, problem-based learning,
ample, problem-based learning (PBL) is a student-centered teaching method, situational issues
which students learn from their relevant and interesting problem structure.
PBL enhances their learning motivation and fosters a teaching and learning
model of practical skills such as teamwork, problem-solving, self-discipline, King-Dow Su
Hungkuo Delin University of Technology &
and promoting their learning performance (Mundilarto, 2018; Syadiyah et al., Chung Yuan Christian University, Taiwan
2017). This method has been paid much attention by educational scholars in
various subjects of teaching and learning (Sakir & Kim, 2020).
245
IMPLEMENTATION OF INNOVATIVE ARTIFICIAL INTELLIGENCE COGNITIONS WITH PROBLEM-
BASED LEARNING GUIDED TASKS TO ENHANCE STUDENTS’ PERFORMANCE IN SCIENCE
(pp. 245-257) ISSN 2538–7138 /Online/
PBL is derived from the theory of constructivism and stresses the active and critical construction of knowledge
based on prior knowledge (Lapuz & Fulgencio, 2020). It was developed in the mid-1960s by Howard Barrows, a
Canadian medical educator (Torp & Sage, 2002), who proposed student-centered teaching to guide students.
Mundilarto pointed out that the method prevails in domestic and international higher education and many sub-
ject areas. PBL allows students to develop a viable problem-solving plan, and then integrate theory and practice,
apply knowledge and skills to explore the process of problem-solving, and learn how to learn (Sockalingam et
al., 2011). Students play the role of problem holders in PBL processes. The educator acts as the cognitive coach or
promoter to organize learning content and situational teaching by the ill-structured problem. Students discuss
learning topics, use resources, collect information to learn and apply new knowledge, conceive problem-solving,
and evaluate programs with the group cooperative learning.
Therefore, this research focuses on the effective integration of AI life learning of face recognition, self-driving,
and robot situational issues into PBL in natural science. Three life situation issues of the social application provide
the experience of authenticity problems, strengthen students’ narrative ability, and problem-solving skills. Students’
scientific knowledge experience is activated to enhance their cognitive understanding of learning effectiveness,
evaluate their attitude, and conduct feedback analysis in an interview.
Application of PBL
PBL was a problem-based, student-centered approach to collaborative learning, which was also the principle
of problem-based acquisition of new knowledge (Jansson et al., 2015; Syadiyah et al., 2017). What differentiates
the PBL approach from other approaches were ill-structured problems as learning contexts in which students
initiate learning by solving life problems, linking learning experiences, motivating learning, and activating learn-
ing problems to improve their knowledge. Yoon et al. (2014) and Jansson et al. (2015) proposed that PBL teaching
methods could help students enhance their problem-solving and assessment skills and deepen their understanding
of the science curriculum. Gunter and Alpat (2017) revealed that PBL had notable results for their scientific learn-
ing processes and achievements. Rillero and Chen (2019) also found that PBL could combine different curriculum
themes with meaningful experiences. In addition, Hernández-Ramos et al. (2021) emphasized that the PBL approach
offered learning potential in addressing real environmental problems in natural science education. Accordingly,
this research used the PBL teaching method and designed the life teaching situation of AI application in society
as the issue of classroom discussion.
Social Applications of AI
Lai (2016) defined AI as the effective transformation and realization of human intelligence, such as percep-
tion, learning, memory, knowledge, semantics, reasoning, language, and thinking, implemented on computers
through machine learning. Ng (2016), Ricoy and Feliz (2016) revealed that AI could provide students with possible
one-to-one teaching support to help problem-solving, learning judgment, and thinking of decisions. AI topics were
cross-disciplinary integration and experience practice of emerging technologies. The generation and development
of technological concepts were closely related to human life experience and social survival (Topcu et al., 2010).
The social applications of AI in life were becoming more and more diverse, such as face recognition, self-
driving, robot, virtual reality, augmented reality, etc. In this era, how to enrich the social connotation of students’
AI background knowledge and generate new insights from cross-disciplinary learning. However, emerging tech-
nological issues related to AI are gradually appearing in everyone’s life. For example, discussions on the learning
benefits of cloud technology education courses have received more attention from academics (Kaur & Rampersad,
2018; Rahman et al., 2017). Traveling long distances in driverless cars or buses would allow passengers to focus on
business and increase productivity (Bansal et al., 2016; Nordhoff et al., 2017).
In addition, the face recognition systems were almost indistinguishable due to the technologies of generative
adversarial networks (Hsu et al., 2020), and these problems were related to students’ learning literacy and social
science cognition. The impact of AI technology issues has become a new trend. Therefore, this research offered
authentic insights into the PBL teaching method to construct AI teaching materials and develop an AI learning
effectiveness questionnaire. All strategies were to evaluate their learning effect and attitude towards AI.
246
ISSN 1648–3898 /Print/ IMPLEMENTATION OF INNOVATIVE ARTIFICIAL INTELLIGENCE COGNITIONS WITH PROBLEM-
ISSN 2538–7138 /Online/ (pp. 245-257)
Research Purpose and Questions
This research focused on evaluating students’ science learning effectiveness and completing the teaching
site focus of issues. They would make teaching more diverse, and learning becomes more meaningful. Based on
the purpose, the responses to the following research questions:
1. How to construct AI-PBL experience teaching materials of life situations?
2. How to develop the validity and reliability of pre-test and post-test items and attitude questionnaires
for assessment tools of learning effectiveness?
3. What were the influencing factors of students’ learning attitude of cross-disciplinary experience through
one-way ANOVA?
4. What were the feedback analyses after random interviews after AI-PBL teaching?
General Background
In this research, the background included students’ problem-solving guidance, data collection of learning
effects, influencing factors of learning attitudes, feedback analysis, and presentation of AI-PBL textbooks. The
contribution of this research came not only from group discussion, preparation, and response but also from the
evaluation of teaching with more possible AI questions to suggest future research needs. The program lasted
two hours of AI-PBL instruction per week, nine weeks in two semesters, for a total of 36 hours. Therefore, there
was a limit to the impact on their scientific questions and learning attitudes. Research topics covered life science
contextual issues such as face recognition, self-driving, and robots. All research results are needed to reveal AI-PBL
research questions, teaching methods, group discussions, and learning assessments to build a positive learning
environment and improve their scientific cognition. During the 2020 academic year, the students demonstrated
the AI learning effect that followed the natural science learning objectives. Their experience learning from AI-PBL
social application life situations enhanced their scientific learning effectiveness.
Participants
This research comprised 127 participating students who attended the author’s course, screened in two
stages of qualification tests, as a total study sample of university students in Taiwan. The participants included 70
males and 57 females, ranging in age from 20 to 22. The pilot test guided 44 Second-year students to engage in
the research for the AI-PBL developments with the pre-knowledge at the first stage. Other 83 participants with
high cognitive skills had attended this research with AI-PBL strategy at the second stage as the research sample.
All 127 participants based on volunteering with suggestive results and the anonymous findings might publish in
this research. All ethical considerations met with students’ approval of experimental processes (Su, 2018) in this
research. In terms of experts, seven professors engaged in this research, consisting of three science education
scholars, two information education communication scholars, one humanities education scholar, and one AI edu-
cation scholar. They logically revised and reviewed the intelligibility of the questionnaire draft and their suitability
to the participant level to form experts’ content validity.
AI-PBL Teaching Design
This research implements teaching innovation from student roles, teachers’ vision, AI-PBL natural science
teaching goals, and AI-PBL teaching materials design. The design of this research was based on the learning objec-
tive so that students can gain a profound understanding of the practical value of AI fundamentals and scientific
knowledge, understand the new functions of AI in social applications, and the need for cross-disciplinary talents.
They got hands-on and experienced new AI knowledge to enhance understanding, make decisions, and solve
scientific problems facing around life. The PBL pedagogy was a student-centered collaborative learning method
that creates an experiential learning situation step-by-step in the science classroom, allowing students to find
learning opportunities and build self-competence. Teachers were the facilitators of their learning to facilitate and
guide the experience of designing authentic AI social contextual life issues. In addition, the goals of the natural
247
(pp. 245-257) ISSN 2538–7138 /Online/
sciences could combine experiential texts and assessment questions to stimulate science learning and to judge
right and wrong in the process. The AI-PBL teaching focus showed that students could use the PBL method to
complete problem-solving tasks, strengthen the absorption and application of AI life science knowledge, and draw
meaningful conclusions through AI-PBL teaching and group discussions.
It was more important to clarify the new knowledge they build on existing infrastructure and connect it to the
prior knowledge to enhance their learning effectiveness. And the way of student achievement assessment included
their self-assessment checklist and paper-and-pencil tests. In the grading checklist, 60% of the scales attempted
to help group students to plan problem-solving and discussion, and paper-delivered examined to account for
40%. All scales included group self-assessment form, problem-solving plan form, member mutual evaluation form,
work record form, resource record form, and learning outcome evaluation form. However, three PBL life situational
issues of AI social application tried to integrate into the course, namely face recognition system, the social issue
of self-driving, and AI robot functionality. The three situations activated the group discussion at the teaching site,
and students could further put forward their needs for the course. The study took 18 hours (nine weeks) of AI-PBL
instruction in the first and second semesters. In addition, this research used five open-ended questions to evaluate
their learning effectiveness at the life-like level of AI social applications.
Research Design
In the PBL method, this research applied AI in the real-life social application context to conduct experimental
teaching of natural science courses for general education. Students divided into six groups and discussed their
learning process, problem-solving abilities, learning attitude, and feedback analysis. Table 1 reveals a PBL method
design. The PBL teaching experimental process included an introduction, problem confirmation, data collection,
group discussion and problem-solving demonstration, group results publication, and learning contribution evalu-
ation. Figure 1 shows the PBL teaching process model (modified from Lee & Bae, 2008). There are six procedures in
the flowchart. All steps include ill-structured questions, clarifying problems, planning self-directed learning, putting
forward problem-solving, reviewing problem-solving, and presenting their final reflection feedback. Students im-
mediately conduct the open-ended posttest and learning attitude questionnaire after the teaching experiment.
Subsequently, 3-4 students are randomly selected for each group to make their perception and feedback analysis
of the AI-PBL teaching.
Table 1
The Experimental Research Processes of the PBL Method
Group Pre-test Process Post-test Feedback
E1 ~ E6 Open-ended questions PBL method Open-ended questions Attitude questionnaire and

interview
Figure 1
The PBL Teaching Process Model
248
ISSN 2538–7138 /Online/ (pp. 245-257)
Research Instrument
Through the development of surveying instruments and data collecting, the findings presented quantita-
tive and qualitative data and discussed students’ creativity in their learning in this research. The assessment tools
included open-ended test questions, PBL attitude questionnaires, and semi-structured questionnaire interviews.
This research designed five open-ended questions for their AI test items with the content validity of experts. Based
on the opinion of experts, they suggested revising the readability, accuracy, and applicability of the instrument
content to form the appropriate content validity. This research used these questions to evaluate their cognitive
responses to the social application of AI in life and try to understand learning effectiveness. The open-ended test
item scoring was modified from the criteria of Gunter and Alpat (2017), as shown in Table 2, and applies to this
research according to the scoring criteria.
Table 2
The Scoring Criteria of Students’ Responses for Open-ended Questions
Categories (Abbreviation) Score Content Description
Incomprehension (I) 0 This response is not related to the question.

Specific Misconception (SM) 1 This response is not true scientifically but is considered true by the student.
Partial Understanding with Specific Misconception PUSM) 2 The response is correct, but its explanation is wrong or inverse.
The response contains only some aspects, but not all aspects of the desired
Partial Understanding (PU) 3
response.
Clear Understanding (CU) 4 The response to the question is accurate.
In the PBL attitude questionnaire, this research developed the learning attitude questionnaire to assess their
perceptions, which included two parts: the first was about their basic background information, and the second was
the structured learning attitude questionnaire of 5-point Likert scale, with options such as strongly agree, agree,
neutral, disagree, and strongly disagree. Students’ background information provided independent variables in
the research. Besides, six subscales of the questionnaire offered dependent variables from the composition of the
learning attitude questionnaire. The first draft test items were modified and revised from Su’s (2016) questionnaire.
All seven experts conducted the substantive review, revision, and deleted test items to confirm the validity of the
content and surface of the questionnaire. A pilot test responded according to 44 students who took the science
course and performed after the mid-term examination between November and December 2020.
However, in terms of the construct validity, Bartlett’s test for Sphericity proved significant, indicating that the
factor analysis was suitable for the attitude questionnaire. All six Eigenvalues were above 1.0 with an accumulative
explained variation of 72% in the principal component analysis. Six subscales of learning attitude included: Q1,
attitude towards AI situational problem-based learning (AI-PBL) courses; Q2, attitude towards teachers; Q3, attitude
towards AI-PBL learning environment; Q4, attitude towards AI-PBL situational contexts; Q5, attitude towards self-
learning of AI-PBL situational contexts, and Q6, attitude towards learning results of the AI-PBL situational contexts.
Table 3 reveals the results of the pilot test. In Table 3, the overall questionnaire average (M) is 4.00, the standard
deviation (SD) is .69, and the Cronbach’s α value is .95. The coefficient reliability of any scale is over .90, and the
internal consistency is excellent (Salta & Tzougraki, 2004). The final questionnaire, AIPBLQ, consisted of 31 test
items in this research which were summarized as follows: Item 1, AI-PBL content, is justly the type of learning I want.
Aspect M SD Cronbach’s α
Q1 3.95 .67 .96

Q2 4.17 .58 .96
Q3 4.02 .74 .95
Q4 4.00 .86 .96
Q5 3.91 1.00 .94
249
(pp. 245-257) ISSN 2538–7138 /Online/
Aspect M SD Cronbach’s α
Q6 3.94 1.01 .96

Total 4.00 .69 .95
Students’ Feedback
This research referred to the author’s semi-structured questionnaire (Su, 2018) and designed three open-
ended interview reference items. After the logical review and fluency modification of the above seven experts,
they became three interview questions with content validity. Three interview questions, such as Q1: Using the PBL
method to teach AI situational issues, does the textbook design help you? Can you give a concrete example of
your feedback? Q2: Does the problem-solving method of PBL help you apply it in daily life? Why? Please provide
an example. Q3: What is your overall evaluation and feelings about the use of PBL for AI teaching issues?
Why? Can you give an example to describe your thinking? Accordingly, this research used three questions
to make up for the deficiency of the structural attitude questionnaires and attempts to understand students’
feedback and reflection. After the post-test and structured questionnaire, 38 students engaged in semi-structural
interviews randomly.
Data Analysis and Ethics
AI-PBL teaching pre-test and post-test management, the collected experimental data would be tested
by computer code and Arabic numerals. Statistical methods contained Cronbach’s α internal consistency of test
questionnaires, descriptive statistical analysis, and one-way ANOVA. All statistical analysis came from SPSS for
MS Windows 22.0 software. The consent of the students conducted data collection and analysis. The interview of
students was represented by English letters, such as the first interview student code of the S1 form, to meet the
requirements of academic ethics. All students volunteered and actively participated in this research. Students went
to this research with their approval.
Research Results
Based on the constructive learning theory of Ausubel (2000), this research designed three topics for students
to experience the social application context of the scientific content in the life of AI-PBL, to enhance their broader
basic knowledge, understand social implications, and need AI cross-disciplinary, with AI life issues, designing
face recognition systems, self-driving cars, and robots, guiding PBL multivariate learning environment. In face
recognition systems, the contexts consisted of the generative adversarial network, visitor control, clothing design,
interior design, solar car model design, and financial consumer payment. In self-driving cars, the contexts included
the introduction of sensors, autonomous driving levels, technology development, the internet of vehicles, cloud
computing, and other application fields. Furthermore, in robots, the contexts embodied applications in education,
tourism, finance, retail, transportation, and healthcare. In summary, students could generate innovative thinking
by cross-disciplinary to solve the problems encountered in life around.
This research attempted to develop a three-unit AI-PBL science experience textbook and to provide a profound
understanding of AI in life. The instructor was both a facilitator and a mentor to achieve their authentic conceptual
acquisition in class. Teachers used AI-PBL science experience materials to guide students in interactive learning
and acquire scientific concepts. The students thought deeply about this three-unit AI-PBL scientific experience
textbook to enhance the scientific concepts. Students practiced the problems encountered, the problem-solving
plan, self-learning, and data collection. Their group discussion established a consensus on solving and proposing
solutions to complete the problem-solving task. Appendix 1 summarizes group students’ learning processes in AI
life issues. In Appendix 1, their practiced presentations and
online discussions present the learning outcomes of the natural sciences classroom. Due to the impact of
the Covid-19, part of the experimental teaching would conduct online simultaneously. The students were very
interested in AI applications in life and actively presented the results of their discussions.
In terms of learning effect, students’ learning in AI-PBL course, this research used open-ended test questions
250
ISSN 2538–7138 /Online/ (pp. 245-257)
before and after to examine their learning effect and differences. They checked their answers to the open-ended
test questions and accorded to the scoring criteria proposed by Gunter and Alpat (2017) and shown in Table 4. In
Table 4, the statistical analysis describes the percentage of their responses. The results were as follows. The mean
0 points for the pre-test and post-test were 3.7% and 0%, respectively. Statistical results for students’ answers
showed that they had nothing to do with the questions, blanks, or incomprehensible (I) were all scored as 0 points.
The average percentage of pre-test and post-test averages of 1 point was 29.2% and 15.7%, respectively. If the
question response confirmed an incorrect scientific answer and presented the student as a specific misconcep-
tion (SM), then awarded 1 point. The results showed that students decreased by 13.5% after the PBL instruction
misconception. The average percentage of pre-test and post-test scores with a mean of 2 points was 58.2% and
67.8%, respectively. It indicated that the answer was correct but wrong interpretation, or wrong answer but correct
explanations. These all showed partial understanding and a specific misconception (PUSM), awarded 2 points. The
results showed PBL-guided teaching provided a partial answer with significant misconceptions that increased by
9.6%. The average percentage of pre-test and post-test averages of 3 points was 8.1% and 14.8%, respectively. If
the answer to the question was only partially correct, indicating that the student has partial understanding (PU),
get 3 points. The results showed that after PBL guided teaching, the number of students who answered partially
increased by 6.7%. The average percentage of pre-test and post-test scores of 4 was 0.8% and 1.7%, respectively. If
the question response showed accurate answers to the questions and showed that students clear understand (CU),
get 4 points. The results revealed that after PBL guided teaching, students with a clear understanding increased
by 0.9%, and the overall level of conceptual cognition improved.
Table 4
Students’ Learning Effectiveness Response Percentage (%) for Five Open-ended Test Questions
Response Test Item 1 Test Item 2 Test Item 3 Test Item 4 Test Item 5 Average
Level post-test/pre-test post-test/pre-test post-test/pre-test post-test/pre-test post-test/pre-test post-test/pre-test
I 0/7.4 0/5.6 0/3.7 0/1.9 0/0 0/3.7

SM 17.4/31.4 17.4/35.2 13.0/31.4 19.6/31.4 10.9/16.7 15.7/29.2
PUSM 65.3/55.6 58.7/50.0 67.4/55.6 67.4/63.0 80.4/66.6 67.8/58.2
PUSM 10.9/3.7 23.9/9.2 19.6/9.2 13.0/3.7 6.5/14.8 14.8/8.1
CU 6.5/1.9 0/0 0/0 0/0 2.2/1.9 1.7/0.8
The descriptive statistical analysis of their AI-PBL situational issues learning attitude questionnaire (AIPBLQ,
effective recovery rate 82%) showed that the overall average of each subscale was 3.71, the standard deviation
was .60, and the overall questionnaire reliability was .96. After the experimental teaching, all students participated
in the test of the AIPBLQ structured questionnaire. This questionnaire evaluated their attitudes towards AI-PBL
situational issues (Q1), attitudes towards teachers (Q2), attitudes towards multimedia learning environment (Q3),
attitudes towards AI-PBL classmates (Q4), and attitudes towards AI-PBL students (Q4), attitudes towards self-learning
AI-PBL courses (Q5), and attitudes towards AI-PBL course learning outcomes (Q6). This evaluation tool emphasized
strategic application to examine their six subscales of learning attitude.
A series of one-way ANOVA statistical analyses presented the relations between dependent and indepen-
dent variables. Six subscales acted as dependent variables in the AIPBLAQ evaluation tool. Then, independent
included background information such as gender, enrollment method, the frequency of using 3C products, the
degree of disposition toward AI courses, and AI-related knowledge they have learned. The Wilks’ Lambda variable
selection method adaptive test multivariate reached significance (p <.05). The results of one-way ANOVA are as
follows in Table 5. In Table 5, the independent variable, gender for the six dependent variables of the AIPBLAQ,
only the subscale of the attitude of AI-PBL classmates (Q4) is significant. In addition, females (M= 3.87) are better
than males (3.47), Cohen’s (1988) effect size, f is .264 above the medium (f= .25), other dependent variables have
no significant difference, and Cohen’s effect sizes are below medium.
251
(pp. 245-257) ISSN 2538–7138 /Online/
Table 5
Three Comparative Case Dispositions of Individual Learning Attitude in ANOVAs
Attitude Measure
locking Analysis of
Variable Variance
Q1 Q2 Q3 Q4 Q5 Q6
Gender F-ratio .780 1.978 2.818 4.563 1.104 .608
1. males, 24 p-value .378 .164 .098 .036 .297 .438
2. females, 44 f .110 .173 .207 .264 .128 .095
Enrollment F-ratio 5.315 2.699 3.043 1.786 2.137 2.25
1. Registration, 14 p-value .001 .038 .023 .143 .087 .074
2. Recommendation, 6 f .580 .413 .440 .337 .368 .378
3. Applying, 37 Scheffé 1>3; 5>3
4. Stars, 7
5. Other, 4
Frequency F-ratio 3.24 4.908 7.671 1.345 2.115 1.997
1. Never, 0 p-value .076 .030 .007 .250 .151 .162
2. Occasionally, 4 f .222 .272 .341 .143 .179 .173
3. often, 64
Disposition F-ratio 7.644 8.088 4.699 6.800 3.253 5.712
1. very positive, 17 p-value .001 .001 .012 .002 .045 .005
2. Positive, 39 f .484 .498 .380 .457 .335 .418
3. Neutral, 12 Scheffé 1>3 1>2; 1>3 1>3 1>2; 1>3 1>3 1>3
AI-related knowledge F-ratio 3.079 1.115 1.409 1.324 3.247 3.084
1. Basic, 27 p-value .034 .350 .248 .274 .028 .033
2. PYTHON, 13 f .380 .229 .257 .248 .390 .380
3. None, 25
4. other, 3
In Table 5, the independent variable enrollment method is significant for the six dependent variables of AIP-
BLAQ. Attitude towards Q1, Q2, and Q3, and their Cohen (1988) experimental effect size f-values are better than
.4. Furthermore, Scheffé post hoc comparisons found that only Q1 has a significant difference, and registration
distribution is better than applying for admission, and others are better than applying for admission. The dependent
variables Q4-Q6 are no significant differences. While the frequency of use of 3C products is another independent
variable, among the six dependent variables of AIPBLAQ, only Q2a and Q3 reveal significant differences, and their
effect sizes are above medium. After further Scheffé’s post hoc comparisons find no significant difference, and
other dependent variables Q1, Q4, Q5 and Q6 are no significant differences.
In Table 5, five independent variables show disposition toward AI courses in this research. The six dependent
variables are all significantly different. Their Cohen (1988) experimental effect sizes are above the medium level.
After further Scheffé’s post hoc comparisons, they present significant differences from Q1 to Q6. From subscale Q1
to Q6, all show 1>3, which means “very positive disposition” more than “neutral”, and another subscale Q2 and Q4
show 1>2, which means “very positive disposition” more than the “positive disposition”. Both Q2 and Q4 indicate
above the large Cohen’s effect sizes (f = .4).
In Table 5, in the independent variable of AI-related knowledge for the six dependent variables of the AIPBLAQ,
there are three dependent variables, Q1, Q5, and Q6 showing significant differences. Cohen’s effect sizes are above
252
ISSN 2538–7138 /Online/ (pp. 245-257)
medium. Further, Scheffé post hoc comparisons found no significant difference. In addition, there are no significant
differences in the dependent variables Q2 to Q4.
Finally, in their feedback, this research designed a semi-structured interview questionnaire to conduct tests
after the experimental teaching post-test and the learning attitude questionnaire. After that, randomly select 3-4
participating students in each group, a total of 38 students in the academic year to conduct interviews (the code
names are S1, S2, S3...) to understand their perceptions of AIPBL learning, their impressions of question teaching
after learning, and question interview. The student feedback results were summarized as follows:
Interviewed students, their response to interview question Q1 illustrated as follows:
Both S1 and S10 students believed that listening to everyone’s opinions, motivating, communicating, and learning from
each other contribute to scientific problem-solving. Two students, S2 and S3, thought that the teaching material guided
them to think and learn in AI. Let us discover problems, and after group discussions, put forward ideas and discussions on
problem-solving, which aroused my interest in AI learning. Six students, S5, S6, S8, S19, S29, and S37, found that collect-
ing information on AI-related issues could broaden AI knowledge and increase self-learning willingness. Eight students,
including S4, S7, S9, S15, S16, S17, S18, and S20, believed that PBL guided teaching and group discussions would help
us communicate about self-driving issues, realize in-depth learning, and combine the role of technology to achieve the
purpose of the problem-solving. S9 and S20 believed that the PBL teaching method could profound their understanding
through group communication. Integrating the application of technology into human life makes us more curious about
the issue of self-driving cars. Eleven students, including S11, S12, S14, S21, S22, S23, S24, S25, S26, S27, and S28, felt that
this AI-PBL situational teaching let them have a better understanding of the development, origin, and the importance of
AI, and it allowed them to collect AI-related information. Such as the application of robots in uninhabited inns, medical
treatment, and food production enriched my learning connotation and improved my learning horizons. Moreover, five
students, including S13, thought that AI was not acceptable to just a computer application with many problems, not as
good as the media was just a basic demonstration.
Most of the interviewed students were aware of the PBL method, which guided me to solve problems and
apply them in my daily life. Their response contents to interview question Q2 were as follows:
Six students, including S7, S9, S10, S22, S28, and S29, found that the way through group questions helped to inspire think-
ing, simplified complex things, and found ways to solve problems from divergence to convergence. During the group
communication, using the PBL method to find solving problems, you could also establish a personal network passbook.
Six students, consisting of S11, S16, S19, S21, S24, and S30, believed that PBL problem-solving method would help improve
the ability of logical thinking and make things more efficient. Furthermore, combined with the function of AI robots to ac-
company learning it would help find answers and solve problems. Eleven students, including S4 and others, thought that
PBL had not encountered any problem yet and had no personal experience. If they have the opportunity in the future, they
will try to experience it.
In terms of comprehensive evaluation and feelings, excerpts from students’ feedback of interview question
Q3 were as follows:
Four students, S8, S19, S29, and S37, found PBL to teach AI situational issues, which teacher gave suggestions and proposed
many vivid examples, such as the application of AI in uninhabited inns. This business model made me curious, surprised, and
apprehensive. The curious one was that the technology was so advanced, and the one who was surprised and frightened
was the future graduates of the hospitality department, the pressure to find a job will increase. Seven students, including
S9, S10, S11, S15, S16, S21, and S32, felt that the group discussion would allow everyone to participate in the PBL teaching.
A more detailed analysis of the topics discussed in the group also helped understand the rapid changes in society due to
emerging technologies. Then, it would help me change and improve my shortcoming and understand the different thinking
patterns among classmates. Five students, including S22, S23, S27, S30, and S38, pointed out that we could not underesti-
mate the contribution to the PBL cooperative learning. This teaching method let us impress and experience the new issue
of the face recognition system, self-driving car, and robot. More cross-disciplinary knowledge and more feel the previous
listening-style courses of learning differences. The decision-making model was more affirmative and more confident that
it enhanced my value and vision for science learning.
Discussion
The most responsibility of education was to promote their self-learning, critical thinking, decision-making,
and problem-solving abilities. AI social issues integrated with PBL instruction, used in this research to help students
proactively conceptualize (Shemwell et al., 2010) and enhance their reasoning skills (Sonnleitner et al., 2013). It
would reduce the memory learning of recitation and enable students to link and construct knowledge for positive
learning effectiveness (Hwang et al., 2011; Liu et al., 2009). Therefore, this research successfully designed three
253
(pp. 245-257) ISSN 2538–7138 /Online/
constructive social application situation issues for AI-PBL experience science content. According to this study of
Lopez et al. (2014), their cultivation of problem-solving skills and knowledge of emerging technology concepts
were authentic cognition applications of cross-disciplinary. The diligence and application of knowledge were sci-
ence learning essential elements for their knowledge development and accumulation.
As STEM scholars (Mohtar et al., 2019) thought that the integrated cross-disciplinary knowledge could help
students enhance their learning efficiency, ensure continuous interest, and get better creations. Therefore, the results
of this research responded to the thesis of STEM scholars and combined the content of general education courses
in natural sciences. Students who use emerging technology products, such as recognition systems, self-driving
cars, and AI robots engage in hands-on and brain-based multiple cross-disciplinary learning. Therefore, instructors
integrated emerging technologies into learned thinking and showed their brand new literacy to enhance their
vision and value in science education. The purpose of this research was to allow students to evolve the truth from
the transfer of knowledge to the skills they have learned so that they do not only accept it.
An assessment tool of five open-ended test items with validation evaluates the learning effectiveness of their
AI-PBL textbooks. To sum up, after the AI-PBL situational topic teaching conducted the post-test administration
with this evaluation tool and compared it with the pre-test. The results showed that the number of students with
nothing to do with the questions, blank and incomprehensible (I) papers decreased from 3.7% to 0%, students
with SM decreased by 13.5%, PUSM students increased by 9.6%, PU students increased by 6.7%, and CU students
increased by 0.9%. Su (2018) pointed out that an effective auxiliary tool could help students explore questions,
present problem-solving abilities, and cultivate reasoning skills. Sonnleitner et al. (2013) stressed that students’
problem-solving skills were related to their reasoning skills. Researchers (Sadler et al., 2016) had also emphasized
the importance of the PBL method of contextual issues, whose knowledge and cognitive learning helped activate
logical skills in problem-solving.
Because this research emphasized the social application of AI in daily life texts, PBL provided real situational-
based problems, allowing students to learn from the cross-disciplinary of emerging technologies. They could experi-
ence the practical value of new scientific knowledge and understand the functionality of AI in social applications.
The importance of new knowledge could improve their thinking ability and decision-making skills, generate new
insights by the agitation of emerging technologies, and solve the scientific problems in life.
All six dependent variables included the learning attitude for AI-PBL courses, the teacher, the multimedia
learning environment, attitudes towards AI-PBL classmates, self-learning AI-PBL courses, and AI-PBL course learning
results. In this research, a learning attitude from their descriptive statistical analysis presented a positive learning
attitude. The analysis results of the students’ AI situational issues AIPBLQ showed that the overall average of the
dependent variable was 3.71(>3.50). Su pointed out that this data presented positive attributes for their learning
attitudes. This finding responded to scholars’ arguments that PBL-guided learning could help students improve
their learning attitudes, deepen learning abilities (such as collaboration, synthesis, communication), and enhance
their problem-solving skills (Adesope & Nesbit, 2012; Jansson et al., 2015). In addition, the results of the one-way
ANOVA found that students’ disposition toward AI courses indicated the most impact on learning attitude. Secondly,
the effect factors were their learning attitudes towards AI-related knowledge and enrollment methods; the lastly
influencing factors were the use frequency of 3C products and gender.
Owing to the guidance of the questions, students drove their AI curiosity and interest in the interview for group
discussion. This research found that students liked to learn from the group discussion of emerging technology AI.
In the group discussion, they constructed new knowledge and used group discussion to search for information and
enhance cooperative learning. Students would be brave enough to accept the challenges of new problems. In this
research, due to the integrated limitation and higher-order thinking ability lack, spending more time and training
were required to exhibit higher learning effectiveness. Their group cooperative learning showed an atmosphere
of teamwork, which was conducive to correct thinking guidance and training. In short, the lack of communica-
tion skills and self-confidence were obstacles to effective teaching and learning. It took a long time to construct a
teaching environment, which was also a reason for the teaching practice in class.
The quantitative and qualitative results showed that integrating AI situational issues into PBL teaching helped
improve their learning effectiveness and attitude, achieved the purpose of problem-solving, and enabled students
to participate in these spaces more meaningfully (Mayer, 2011; Mundilarto, 2018). The application of semi-structured
interviews could indeed strengthen the inadequacy of structured questionnaires. From the perspective of construc-
tive learning theory, teachers were more aware of the students’ demand for teaching materials. Therefore, this study
found that the PBL approach offered great potential in supporting real-world problem solving of AI contextual
254
ISSN 2538–7138 /Online/ (pp. 245-257)
issues in science education (Hernández-Ramos et al., 2021). As mentioned above, Rillero and Chen (2019) found
that PBL could combine diverse AI subjects with meaningful experiences for authentic learning outcomes.
Conclusion and Implications
Based on the result and discussion, the fruitful results of integrating AI emerging technologies into natural
science education and the PBL teaching method in this research were encouraging and helpful. AI-PBL teaching
contexts helped most students enrich learning connotations, nurture problem-solving thinking, and enhance their
learning levels through interaction and guidance. Evaluating tools consisted of open-ended test items, attitude
questionnaires, and semi-structured interview questionnaires in this research. All had good validity and reliability.
They examined the learning effect of quantitative analysis of incomprehension and the specific misconception to
promote partial understanding and clear understanding in nature science learning.
All findings suggested that their logical reasoning and thinking skills could promote problem-solving abili-
ties. Descriptive statistical analysis of learning attitudes revealed that cooperative learning enhances their positive
thinking attributes. The One-way ANOVA indicated that the disposal of AI courses had the most influence on their
learning attitudes; AI-related knowledge and enrollment methods were the second factors. The third influencing
factors were the frequency of using 3C products and gender. In addition, the interview found that the PBL method
was helpful for students to learn and reflect, apply to simple problem-solving in daily life, apply communication in
group discussion, and apply the concept of cross-disciplinary emerging technologies to provide positive learning
effectiveness.
In fact, in addition to comparing, analyzing, and critical thinking with literature, students learned problems in
class that needed to be solved urgently and then practiced teaching objectives. The research focused on the social
application of AI in daily life is helpful to the positive learning effect of students. It could provide a pedagogical
reference for teachers of general education in the natural field; however, the limited sample size and time at the
teaching site led to the need for moderation in making inferences. Therefore, from the perspective of teaching
practice and future research on design thinking, this research put forward the following two suggestions:
1. From the perspective of teaching practice, the time constraints of the curriculum design it was impos-
sible to provide students with more discussion time, giving students a full grasp of the essence of the
problem and key decision-making skills. Furthermore, the on-site teaching found that students lack
communication skills and self-confidence. These limitations of learning growth require long-term
cultivation in the educational environment to exhibit significant learning outcomes. In the future,
teaching will focus on students’ cognition, application ability, and literacy to construct and design to
make learning more flexible.
2. In terms of research, future research will increase the number of samples and allow more students to
participate. Students are passionate about learning, brave to accept new challenges, and have team
spirit in the teaching field to improve the inference of research value and the vision. Therefore, increas-
ing the sample size will help them produce more positive thinking.
Acknowledgement
The author would like to acknowledge the Ministry of Education (under Grant No. PGE1090526) and Ministry
of Science Education (under Grant No. MOST 110-2511-H-237-001) who supported their financial sources for this
research in Taiwan.
References
Adesope, O. O., & Nesbit, J. C. (2012). Verbal redundancy in multimedia learning environments: A meta-analysis. Journal of Educational
Psychology, 104, 250-263. https://doi.org/10.1037/a0026147
Altin, H., & Pedaste, M. (2013). Learning approaches to applying robotics in science education. Journal of Baltic Science Education,
12(3), 365-377. https://dx.doi.org/10.33225/jbse/13.12.365
Ausubel, D. P. (2000). The acquisition and retention of knowledge: A cognitive view. Kluwer Academic Publishers.
Alimisis, D. (2013). Educational robotics: Open questions and new challenges. Themes in Science & Technology Education, 6(1), 63-71.
https://www.learntechlib.org/p/148617/
Bansal, P., Kockelman, K. M., & Singh, A. (2016). Assessing public opinions of and interest in new vehicle technologies: An Austin
perspective. Transportation Research Part C: Emerging Technologies, 67, 1-14. https://doi.org/10.1016/j.trc.2016.01.019
255
(pp. 245-257) ISSN 2538–7138 /Online/
Benitti, F. B. V. (2012). Exploring the educational potential of robotics in schools: A systematic review. Computers & Education, 58(3),
978-988. https://doi.org/10.1016/j.compedu.2011.10.006
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Erlbaum.
Gunter, T., & Alpat, S. K. (2017). The eﬀects of problem-based learning (PBL) on the academic achievement of students studying
‘Electrochemistry.’ Chemistry Education Research and Practice, 18 (1), 78-98. https://doi.org/10.1039/C6RP00176A
Hernández-Ramos, J., Pernaa, J., Cáceres-Jensen, L., & Rodríguez-Becerra, J. (2021).The effects of using socio-scientiﬁc issues and technology
in problem-based learning: A systematic review. Education Sciences, 11(10), 640. https://doi.org/10.3390/educsci11100640
Hsu, C.-C., Zhuang, Y.-X., & Lee, C.-Y. (2020). Deep fake image detection based on pairwise learning. Applied. Sciences, 10, 370.
https://doi.org/10.3390/app10010370
Huang, S. P. (2018). Effects of using artificial intelligence teaching system for environmental education on environmental
knowledge and attitude. EURASIA Journal of Mathematics, Science and Technology Education, 14(7), 3277-3284.
https://doi.org/10.29333/ejmste/91248
Hwang, G. J., Wu, P. H., & Ke, H. R. (2011). An interactive concept map approach to supporting mobile learning activities for natural
science courses. Computers & Education, 57, 2272-2280. https://www.learntechlib.org/p/50817/
Jansson, S., Söderström, H., Andersson, P. L., & Nording, M. L. (2015). Implementation of problem-based learning in environmental
chemistry. Journal of Chemistry Education, 92, 2080−2086. https://doi.org/10.1021/ed500970y
Kaur, K., & Rampersad, G. (2018). Trust in driverless cars: Investigating key factors influencing the adoption of driverless cars. Journal
of Engineering and Technology Management, 48, 87-96. https://doi.org/10.1016/j.jengtecman.2018.04.006
Lai, C. Y. (2016). Training nursing students’ communication skills with online video peer assessment. Computers & Education, 97,
21-30. https://doi.org/10.1016/j.compedu.2016.02.017
Lapuz, A. M., & Fulgencio, M. N. (2020). Improving the critical thinking skills of secondary school students using problem-based
learning. International Journal of Academic Multidisciplinary Research, 4(1), 1-7. https://www.ijeais.org/ijamr
Lee, H., & Bae, S. (2008). Issues in implementing a structured problem-based learning strategy in a volcano unit: A case study.
International Journal of Science and Mathematics Education, 6, 655-676.
Liu, T. C., Lin, Y. C., & Tsai, C. C. (2009). Identifying senior high school students’ misconceptions about statistical correlation and their
possible causes: An exploratory study using concept mapping with interviews. International Journal of Science and Mathematics
Education, 7, 791-820.
Lopez, E., Shavelson, R. J., Nandagopal, K., Szu, E., & Penn, J. (2014). Factors contributing to problem-solving performance in first-
semester organic chemistry. Journal of Chemical Education, 91, 976−981. http://doi.org/10.1021/ed400696c
Mayer, K. (2011). Addressing students’ misconceptions about gases, mass, and composition. Journal of Chemical Education, 88(1),
111-115. https://doi.org/10.1021/ed1005085
Mohtar, L. E., Halim, L., Rahman, N. A., Maat, S. M., Iksan, Z. H., & Osman, K. (2019). A model of interest in STEM careers among secondary
school students. Journal of Baltic Science Education, 18(3), 404-416. https://doi.org/10.33225/jbse/19.18.404
Mundilarto, H. I. (2018). Effect of problem-based learning on improvement physics achievement and critical thinking of senior high
school student. Journal of Baltic Science Education, 16(5), 761-780. https://doi.org/10.33225/jbse/17.16.761
Nordhoff, S., van Arem, B., Merat, N., Madigan, R., Ruhrort, L., Knie, A., & Happee, R. (2017). User acceptance of driverless shuttles
running in an open and mixed traffic environment. Paper presented at the Proceedings of the 12th ITS European Congress.
Ng, E. M. (2016). Fostering pre-service teachers’ self-regulated learning through self-and peer assessment of wiki projects. Computers
& Education, 98, 180-191. https://doi.org/10.1016/j.compedu.2016.03.015
Rahman, M. M., Lesch, M. F., Horrey, W. J., & Strawderman, L. (2017). Assessing the utility of TAM, TPB, and UTAUT for advanced driver
assistance systems. Accident Analysis & Prevention, 108, 361-373. https://doi.org/10.1016/j.aap.2017.09.011
Ricoy, M. C., & Feliz, T. (2016). Twitter as a learning community in higher education. Journal of Educational Technology & Society,
19(1), 237.
Rihtaršič, D., Avsec, S., & Kocijancic, S. (2016). Experiential learning of electronics subject matter in middle school robotics courses.
International Journal of Technology and Design Education, 26(2),
205-224. https://doi.org/10.1007/s10798-015-9310-7
Rillero, P., & Chen, Y. C. (2019). The use of a digital problem-based learning module in science methods courses. Journal of Problem
Based Learning in Higher Education, 7(1), 107-119. https://doi.org/10.5278/ojs.jpblhe.v7i1.2349
Sadler, T. D., Romine, W. L. & Topçu, M. T. (2016). Learning science content through socio-scientific issues-based instruction: A multi-level
assessment study. International Journal of Science Education, 38(10), 1622-1635. http://dx.doi.org/10.10 80/09500693.2016.1204481
Sakir, N. A. I., & Kim, J. G. (2020). Enhancing students’ learning activity and outcomes via implementation of problem-based learning.
Eurasia Journal of Mathematics, Science and Technology Education, 16(12), Article em1925. https://doi.org/10.29333/ejmste/9344
Salta, K., & Tzougraki, C. (2004). Attitudes toward chemistry among 11th grade students in high schools in Greece. Science Education,
88, 35-547. https://doi.org/10.1002/sce.10134
Savery, J. R. (2006). Overview of problem-based learning: Definitions and distinctions. Interdisciplinary Journal of Problem-Based
Learning, 1(1), 9-20.
Shemwell, J., Fu, A., Figueroa, M., Davis, R., & Shavelson, R. (2010). Assessment in schools’ secondary
science. In P. Peterson, E. Baker, & B. McGaw (Eds.). International encyclopedia of education (3rd
ed., pp. 300-310). The Netherlands: Academic Press.
Sockalingam, N., Rotgans, J., & Schmidt, H. G. (2011). Student and tutor perceptions on attributes of effective problems in problem-
based learning. Higher Education: The International Journal of Higher Education and Educational Planning, 62(1), 1-16. https://
doi.org/10.1007/s10734-010-9361-3
256
ISSN 2538–7138 /Online/ (pp. 245-257)
Sonnleitner, P., Keller, U., Martin, R., & Brunner, M. (2013). Students’ complex problem-solving abilities: Their structure and relations
to reasoning ability and educational success. Intelligence, 41(5), 289-305. https://doi.org/10.1016/j.intell.2013.05.002
Su, K. D. (2016). Strengthening strategic applications of problem-solving skills for Taiwan students’ chemistry understanding. Journal
of Baltic Science Education, 15(6), 662-679. https://doi.org/0.33225/jbse/16.15.662
Su, K. D. (2018). Enhancing students’ corresponding reasoning of cognitive performances by animated concept mapping in
electrochemistry. Journal of Baltic Science Education, 17(4), 662-673. https://doi.org/10.33225/jbse/18.17.662
Su, K. D. (2021). Lightening up a new AI cognitions and performances for engineering students’ problem-based learning in nature
general education programs. In: Huang, Y. M., Lai, C. F., & Rocha, T. (Eds), Innovative Technologies and Learning. ICITL 2021. Lecture
Notes in Computer Science, 13117, 30-38. Springer. https://doi.org/10.1007/978-3-030-91540-7_4
Sullivan, F. R., & Heffernan, J. (2016). Robotic construction kits as computational manipulatives for learning in the STEM disciplines.
Journal of Research on Technology in Education, 48(2), 105-128. https://doi.org/10.1080/15391523.2016.1146563
Syadiyah, A. S., Mohammad, N. N., & Azrul, A. I. M. (2017). Integration of Naqli and Aqli elements in Problem Based Learning (PBL).
Education Journal, 6(6), 164-169. https://doi.org/10.11648/j.edu.20170606.11
Topcu, M. S., Sadler,T. D., &Yilmaz-Tuzun, O. (2010). Preservice science teachers’informal reasoning about socioscientific issues:The influence
of issue context. International Journal of Science Education, 32(18), 2475-2495. https://doi.org/10.1080/09500690903524779
Torp, L., & Sage, S. (2002). Problems as possibilities: Problem-based learning for K-16 education (2nd ed.). Association for Supervision
and Curriculum Development.
Yoon, H., Woo, A. J., Treagust, D., & Chandrasegaran, A. L. (2014). The efficacy of problem-based learning in an analytical
laboratory course for pre-service chemistry teachers. International Journal of Science Education, 36(1), 79-102.
https://doi.org/10.1080/09500693.2012.727041
Appendix 1
Group cooperative learning to report on stage for a) robot issue; and b) online face recognition issue in Chinese
(Covid-19 impact)
a) b)
Cite as: Su, K.-D. (2022). Implementation of innovative artificial intelligence cognitions with problem-based learning
guided tasks to enhance students’ performance in science. Journal of Baltic Science Education, 21(2), 245-257.
King-Dow Su PhD, Professor, Department of Hospitality Management and Center for

General Education, Hungkuo Delin University of Technology; NO.1, Lane
380, Ching-Yun Road, Tu-Cheng District., New Taipei City, Taiwan 23646,
& Center for General Education, Chung Yuan Christian University, 200
Chung Pei Road, Chung Li District, Taoyuan City, Taiwan 32023.
E-mail: su-87168@mail.hdut.edu.tw
257
THE IMPACT OF TEACHERS’

PROFESSIONAL DEVELOPMENT
IN SCIENCE PEDAGOGY ON
STUDENTS’ ACHIEVEMENT:
EVIDENCE FROM TIMSS 2019
Abstract. Teachers’ professional develop- AiBin Tang,

ment (PD) is considered to be a topic of
interest in science education. This study
WenYe Li,
examines the impact of professional Dawei Liu
development in science pedagogy (PD in
PED) on students’ achievement and how
it influences teachers’ instruction based
on the Trends in International Mathemat-
ics and Science (TIMSS)2019. The sample
Introduction
comprised 2,968 fourth-grade students
and their 145 teachers and 3,265 eighth-
Teachers are the implementors of educational policies. Hence, teacher
grade students and their 150 teachers in
professional development (PD) is undoubtedly the focus of academic research
Hong Kong. Teachers’ emphasis on science
and the key to science education practice reform. There is no other area
investigation (TESI) was included as a medi-
within science education that affects academics, practice, and students as
ating factor. Multilevel mediation analyses
comprehensively as the aforementioned development does. Considering the
revealed that PD in PED was significantly
development’s ultimate goal of teaching students to learn so they themselves
positively related to students’ science
develop excellently, the impact of such development is worthy of attention.
achievement, while the link between PD
There is a recognised need for teacher professional development that has a
in PED and students’ science achievement
major impact on student outcomes (Andersson & Palm, 2017; Johnson et al.,
was positively and completely mediated by
2007a); however, research has shown that many professional development
TESI in the fourth grade. However, TESI had
initiatives appear to be ineffective in supporting changes in teacher practices
no mediating effect in the eighth grade,
(Darling-Hammond et al., 2017). The idea that PD can foster improvements in
a finding that may be attributed to the
teaching is widely accepted: In Hong Kong, PD is required by virtually every
characteristics of teacher PD programmes
teaching contract, and teachers must participate in PD every year (Gore et
in different grades. The results also show
al., 2017). Nevertheless, few studies have provided persuasive evidence of
that many teacher PD programmes are
such activities’ impact within actual school environments and on students’
not of high quality and are intellectually
academic achievements, respectively. Moreover, there is a lack of proof as
superficial from the perspective of students’
to whether different PD programmes produce a similar impact on teacher
achievement gains. Implications for teacher
practices. This is also so with regards to schools which differ in level, namely,
PD practices are discussed as well.
primary and secondary schools.
Keywords: professional development, Meanwhile, a large body of research has shown that teachers’ in-
science achievement, hierarchical linear structional approaches have important implications for student outcomes
model, TIMSS (Hubber et al., 2010; Ismail et al., 2018; Lin & Tsai, 2021). Improving teachers’
AiBin Tang pedagogy ability through PD in pedagogy (PD in PED) is the main approach.
Nanjing Normal University, People’s Examining the potential benefits of teachers’ pedagogy can reveal specific
Republic of China recommendations that could be adopted in wide-ranging contexts. The
Hangzhou Tianchang Primary School,
People’s Republic of China current study examined data from the Trends in International Mathematics
WenYe Li and Science Study 2019 (TIMSS 2019) to determine the unique influence that
Nanjing University, People’s Republic of China effective PD in science pedagogy has on student science achievement and
Dawei Liu
Nanjing Xiaozhuang University, People’s the underlying mechanism of this relation in the fourth and eighth-grade
Republic of China contexts in Hong Kong.
258
ISSN 1648–3898 /Print/ THE IMPACT OF TEACHERS’ PROFESSIONAL DEVELOPMENT IN SCIENCE PEDAGOGY ON
ISSN 2538–7138 /Online/ (pp. 258-274)
Literature Review
Impact of Professional Development on Student Achievement
Although there is no literature on the relation between PD in PED and students’ achievement, numerous stud-
ies have found that various pedagogies can change teachers’ practices and students’ learning (Adjapong & Emdin,
2015; Asamoah et al., 2020; Baker, 2013; Özdem et al., 2017), and some have discussed the features of different
pedagogy in science teaching (Pugh & Girod, 2007; Schindel, 2016).
Further, a vast amount of contemporary research has tested the relation between teachers’ PD and students’
achievement, much of which has aimed to prove that teachers’ PD can dramatically positively predict student
achievement (Didion et al., 2020; Fischer et al., 2018; Piper et al., 2018; Taylor et al., 2017). The guidance function
and impact of PD on teachers, and the effect of PD towards results of students, have all been analysed by several
extensive explanatory trials of specific PD models (Jacob et al., 2017; Llosa et al., 2016; Roth et al., 2019). Meanwhile,
the positive impacts of teachers’ PD and, to some extent, its positive impacts on student outcomes can be found
in the literature (Akiba & Liang, 2016; Fischer et al., 2020; Zakharov et al., 2020); however, some studies have failed
to find any consistent association between teachers’ PD and student achievement (Guskey & Yoon, 2009; Jacob
et al., 2017). Antoniou and Kyriakides (2011) observed that a different approach to teacher professional develop-
ment led to varying degrees of improvement in student achievement. Data from Germany in PISA 2003 showed
the positive impact of teachers’ PD programmes on students’ interest, motivation, and outcomes in science and
mathematics (Ostermeier et al., 2010).
Hundreds of experimental and quasi-experimental research studies, including both small and medium-sized,
have studied PD interventions, whose results can enable educational scientists, policymakers, and regional leaders
to understand the characteristics of effective PD. However, small studies can be difficult to interpret because they
lack the ability to determine statistically significant effects (Gore et al., 2017). Most of the supporting empirical
evidence was collected in research settings, rather than in normal school environments, and research on whether
and how pedagogy PD programmes impact student achievement is especially rare.
Influence of PD Participation on Teachers’ Instructional Practices
Desimone’s research in 2002 is the largest and most influential related study, which demonstrated teachers’
PD and its impacts on changing teaching practices in mathematics and sciences via a three-year longitudinal
investigation (Desimone et al., 2002). Thereafter, Desimone’s (2009) follow-up study found that the most direct
results of teachers’ participation in PD activities were improvements in their knowledge and skills and changes in
their classroom instruction, which can indirectly lead to improvements in students’ learning. Thus, one’s experience
as a teacher might abate PD’s efficacy towards student accomplishment.
Korthagen (2016) identified that teachers’ PD was effective at influencing their classroom behaviour. In the
same vein, Bruce and Ross (2008) found that teachers implemented key strategies for effective mathematics
teaching after peer coaching, while in their book Effective Teacher Professional Development, Darling-Hammond
et al. (2017) demonstrated that structured professional learning that results in changes in teacher practices had a
major impact in this field. A few experimental or quasi-experimental studies have also found science teachers to
show notable performance advancement in relation to their PD (Al-Balushi & Al-Abdali, 2015; Buczynski & Hansen,
2010; Roth et al., 2011).
Impact of Science Investigation on Student Achievement
According to the National Science Education Standards (NRC,1996), ‘science investigation’ forms part of what
is called ‘scientific inquiry’. Since the Next Generation Science Standards (NGSS) were promulgated in 2011, ‘science
practice’ has also become a popular term in the field of science education. Over the years, the phrases ‘science
investigation’, ‘scientific inquiry’, and ‘scientific practices’ have often been used interchangeably in the literature,
although educational practitioners are most familiar with ‘scientific inquiry’. However, though much of the extant
literature has confirmed that scientific inquiry is an important factor in science learning, findings on the effects of
scientific inquiry on student achievement have been inconsistent. Many empirical studies have observed that sci-
entific inquiry has a positive impact on student achievement (Forbes et al., 2020; Jiang & McComas, 2015; Johnson
259
(pp. 258-274) ISSN 2538–7138 /Online/
et al., 2007a; Lati et al., 2012; Wen et al., 2020), although others have found negative effects (Areepattamannil, 2012;
Cairns & Areepattamannil, 2019), and yet others have confirmed that the impact of scientific inquiry is extremely
complex (Cairns, 2019; Teig et al., 2018).
Theoretical Framework
This study attempted to validate that teachers’ PD in science pedagogy participation can increase students’
science achievement. It also examines educational practice as a potential process factor of the relation between
teacher’s PD in science pedagogy participation and students’ science achievements in the fourth and eighth grades.
The theoretical framework in this study is consistent with those of Desimone et al. (2013), which is similar in scope
and refined Desimone’s (2009) framework structure. As shown in Figure 1, PD in science pedagogy has a direct
influence on students’ achievement and teachers’ emphasis on science investigation, while PD in science pedagogy
influences students’ achievement through teachers’ emphasis on science investigation, teachers’ emphasis on sci-
ence investigation acted as a mediator. Among them, PD in science pedagogy can increase teachers’ knowledge
and skills, teachers’ emphasis on science investigation can change teachers’ practice, and students’ achievement
is the result of improving students’ learning.
Figure 1
Modes of PD in Science Pedagogy on Teachers and Students [adapted from Desimone (2009)]
Research Hypotheses
The research hypotheses were as follows:

H1: Teachers’ PD in science pedagogy participation has a positive impact on fourth-grade and eighth-grade
students’ science achievement.
H2: Teachers’ emphasis on science investigation is a mediating factor between teachers’ PD in science
pedagogy participation and students’ science achievement in the fourth and eighth grades, and it
positively impacts on students’ science achievement.
260
ISSN 2538–7138 /Online/ (pp. 258-274)
General Background
The hierarchical linear model was adopted as an analytic strategy. This study used Hong Kong 278 schools’
fourth and eighth grade data which was collected from teacher and students’ questionnaires and in TIMSS 2019.
TIMSS is an international assessment of student achievement in mathematics and science at fourth and eighth
grades, conducted every four years since 1995. When the TIMSS2019 data collection began in the first half of 2019,
it was largely unaffected by the COVID-19 pandemic.
Participants
The sampled students comprised 3,026 fourth-graders and 3,377 eighth-graders. There were 58 missing data
for the fourth-grade group; therefore, we ultimately analysed 2,968 data valid items. The sample comprised 53.5%
boys and 46.5% girls from 142 schools. This sample also included 145 fourth-grade science teachers, including
teachers who taught both maths and science. 59.2% were female, and 40.8% were male, and there were three
missing data items. There was a total of 3,377 participants in the eighth grade; with 112 missing data, the valid data
totalled 3,265 items. The sample included 54.2% boys and 45.8% girls from 136 schools. The sample also included
150 science teachers. Eight data items were missing; 41.5 % were female, and 58.5% were male. As the sponsor,
IEA (International Association for the Evaluation of Educational Achievement) ensure that all involved are well
informed about the correct methods and protocols, participate in this study that was obtained at the beginning
of data collection in accordance with educational research ethics (IEA, 2022).
Professional Development in Science Pedagogy/Instruction
The professional development in science pedagogy/instruction by both fourth and eighth-grade teachers
was measured with the question: ‘In the past two years, have you participated in professional development in any
of the following? Science pedagogy/instruction’ (Yes = 1; No = 0).
Science Achievement
According to TIMSS 2019 International Database, five estimates, also named ‘plausible values’, were sepa-
rately made on each student’s score on each achievement scale. Innate uncertainty was outlined in the variability
between the five. Under the circumstance of the said database, the plausible values made the A-list procurable
metric in measuring the students’ achievement on the specific scale, thus ought to be set as the gauge for the
outcome when researching student achievement; thus, the fourth and eighth graders’ science achievement data
were used in the present analysis.
Teachers’ Emphasis on Science Investigation
The Teachers’ Emphasis on Science Investigation scale was created to measure teachers’ instructional practices
and was based on teachers’ responses to the eight items listed below, which included, ‘When teaching science
to the students in this class, how often do you ask them to do the following? (1) Fourth grade: Observe natural
phenomena such as the weather or a plant growing and describe what they see / Eighth-grade: Observe natural
phenomena and describe what they see; (2)Watch me demonstrate an experiment or investigation ;(3) Design or
plan experiments or investigations ;(4) Conduct experiments or investigations ;(5) Present data from experiments
or investigations ;(6) Interpret data from experiments or investigations ;(7) Use evidence from experiments or in-
vestigations to support conclusions and (8) Do fieldwork outside the class’. Teachers responded on a 4-point scale
(4= Every or almost every lesson;3= About half the lessons;2= Some lessons;1=Never).
261
(pp. 258-274) ISSN 2538–7138 /Online/
Covariates
Student-level covariates included gender (0 = boy,1 = girl), and ‘home resources for learning’ in the fourth
grade or ‘home educational resources’ in the eighth grade (there were slight differences in the home resources
scale in two grades context questionnaires). ‘Home resources for learning’ items (fourth grade) included (1)
Number of books in the home; (2) Number of home study supports—which featured on the students’ context
questionnaire—and (3) Number of children’s books in the home; (4) Highest level of education of either parent;
(5) Highest level of occupation of either parent, which featured on the home context questionnaire.
In the eighth-grade questionnaire, Q3 (‘Number of children’s books in the home’) and Q5 (‘Highest level of
occupation of either parent’) were deleted, and the scale was named ‘Home educational resources’ (as opposed
to ‘Home resources for learning’ in the fourth-grade questionnaire), although both scales represented the families’
socioeconomic status. Previous studies have shown the above factors are significantly related to students’ achieve-
ments; for example, using data from TIMSS 2011, Reilly et al. (2019) found that in non-OECD nations, girls scored
higher than boys in mathematics and science achievement. In the US, Quinn and Cooc (2015) revealed that girls
outperformed boys in Grade 3, although this gap may narrow slightly by the eighth grade. Furthermore, since the
Coleman Report (1988), extensive research has been conducted on the relation between socioeconomic status
and academic achievement (Blums et al., 2017; Sarsour et al., 2011), and the results of PISA 2015 indicated that
the strength of the relationship between socio-economic status and scientific achievements was significantly
connected to the country’s income level (Liu et al., 2020). However, there is inconsistency among existing research
results, coupled with little understanding of the relationship between students’ socio-economic status and scientific
achievements in Hong Kong. Thus, in the present study, the variables mentioned above are considered as covariates.
Teacher-level covariates were reported by each teacher and included the items ‘gender’ (0 = male, 1 = female)
and ‘What is the highest level of formal education you have completed?’ A higher level of knowledge in science and
mathematics was discovered by Wayne and Youngs (2003) to have a positive correlation with student achievements,
while Chudgar and Sankar (2008) demonstrated that having a female teacher is advantageous for student’s language
learning. Using data from a five-year panel, Winters et al. (2013) also estimated a statistically significant relation
between teacher gender and student achievement in middle and high school. Moreover, a large body of literature
has demonstrated that teacher’s gender affects children’s science achievement scores, although consistent conclu-
sions have not been reached (Beilock et al., 2010; Lee et al., 2019; Muralidharan & Sheth, 2016; Tatar et al., 2016).
To control for the impact of other teacher PD programmes, whether the teacher had ever attended additional
programmes in the seven areas in the previous two years was included in the form of the following covariates: (1)
science content, (2) science curriculum, (3) integrating technology into science instruction, (4) improving students’
critical thinking or inquiry skills, (5) science assessment, (6) addressing individual students’ needs, (7) integrating
science with other subjects (1= yes, 0 = no). ‘Integrating science with other subjects’ was deleted from the eighth-
grade questionnaire.
Statistical Analyses
SPSS 23 and hierarchical linear modelling (HLM 6.08, Scientific Software International, Skokie, Il) were used
to take the nested data structure (students as the individual level units on level 1, nested within teachers as the
group-level unit on level 2) into account because HLM was able to appropriately address the hierarchically nested
design of the study (Hofmann, 1997).
At first, the intra-class correlation coefficient (ICC) was calculated by researchers for the obtainment of un-
conditional models’ outcome and mediator variables (Hofmann, 1997). The fact that group-level characteristics
can justify a responding variable’s variance is shown by ICC (Woltman et al., 2012). Besides, a notable variance of
a responding variable among groups is implicitly suggested given ICC value’s surpassing of the 0.059 standard.
Consequently, a hierarchical linear analysis is necessitated (Wen & Chiou, 2009). In this study, science achievement
was entered into the HLM analysis as a dependent variable, with no predictors in the model; the results indicated
significant variance of science achievement among the teachers. Therefore, in the analysis, teacher-level indepen-
dent variables were entered into level 2 and student-level independent variables were entered into level 1 analysis.
The continuous variables, including the dependent variables, were standardised using Z scores across all of
262
ISSN 2538–7138 /Online/ (pp. 258-274)
the teachers in the study, in a method similar to the grand-mean centre method suggested by statistical meth-
odologists (Zhang et al., 2016). Dummy variables, such as gender or whether teachers had ever attended PD
programmes, were uncentered.
Using the fourth-grade statistical analyses as an example, the independent variable (PD in PED) must be
related to the dependent variables (science achievement) after controlling for the student level (gender, home
resources for learning) and teacher level (teacher gender, formal education level, attendance of the other seven
PD programmes) covariates: coefficient c in Eq 1.
Level 1: Science Achievementij=β0j 1j
(SGENDER ij 2j
(HRLij rij
Level2: β0j 00 01
(PEDj 02
(CONj 03
(CURj 04
(TECj 05
(CRIj 06
(ASSj 07
(NEEDj 08
(INTSUB j 09
(TGENDER j 010
(TEDU j u0j
β1j 10
u1j
β2j 20
u2j
Notes: SGENDER: gender of student; HRL: home resources for learning; PED: PD in Science pedagogy/instruction;
CON: PD in science content; CUR: PD in science curriculum; TEC: PD in integrating technology into science instruction;
CRI: PD in improving students’ critical thinking or inquiry skills; ASS: PD in science assessment; NEED: PD in addressing
individual students’ needs; INTSUB: PD in integrating science with other subjects; TGENDER: gender of teacher; TEDU:
the highest level of formal education the teacher had completed.
Secondly, the independent variable (PD in PED) must correlate with the mediator (TESI) after controlling for
covariates: coefficient a in Eq 2.
TESI=β0+a/β1 (PEDj +β2(CONj +β3(CURj +β4 (TECj +β5 (CRIj +β6 (ASSj +β7 (NEEDj +β8 (INTSUB j +β9 (TGENDER j
+β10(TEDU j + ut
Thirdly, a connection must be established between the responding variable (achievement in science) and
the mediator variable (TESI) when the predictor (PD in PED) is under control: coefficient b in Eq 3. Coefficient c’
represented the relationship betwixt accomplishment in science and PD in PED.
Level 1: Science Achievementij=β0j 1j
(SGENDER ij (HRLij
2j
rij
Level 2: β0j 00 01
(PEDj 02
(CONj (CURj
03 04
(TECj 05
(CRIj 06
(ASSj 07
(NEEDj 08
(INTSUB j 09
(TGENDER j 010
(TEDU j 011
(TESI j u0j
β1j 10
u1j
β2j 20
u2j
The incidental impact of PD in PED on students’ accomplishments in science is caused when path a is multiplied
by path b. The reduction in path from PD in PED to students’ achievement in science still allows for the happening
of partial mediation provided said path was nonetheless notable with the mediator TESI in the model. However,
when said path was insignificant as the mediator’s existence, complete mediation took place.
Research Results
Descriptive and Correlation Results

Table 1
Descriptive and Correlational Statistics for Student and Teacher Variables in the Fourth Grade
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Pearson Correlation
1. Student gender
2. Home resources for learning -.024
3. Teacher gender .067*** -.012
4. Formal education .022 -.008 -.029
5. Science content .036 .009 .054** -.082***
263
(pp. 258-274) ISSN 2538–7138 /Online/
1 2 3 4 5 6 7 8 9 10 11 12 13 14
6. Science pedagogy/instruction .033 .088*** -.059** -.089*** .590***

7. Science curriculum .019 .077*** .032 .017 .667*** .584***
8. Integrating technology -.020 .020 .020 -.093*** .603*** .485*** .484***
9. Student critical thinking -.026 -.007 .044* -.084*** .435*** .450*** .423*** .504***
10. Science assessment .043* -.042* -.017 -.056** .446*** .466*** .412*** .428*** .437***
11. Student needs -.038* -.072*** -.004 -.051** .123*** .179*** .121*** .091*** .381*** .357***
12. Integrating science subjects -.018 -.010 .074*** -.092*** .328*** .300*** .265*** .482*** .361*** .359*** .240***
13. TESI -.011 .060** -.076*** -.116*** .059** .057** -.004 -.010 .048** -.003 .074*** .051**
14. Science achievement -.007 .397*** .026 -.030 .041* .075*** .071*** .022 .017 -.068*** -.059** -.008 .102***
Descriptive statistics
M 10.48 5.19 8.68 532.53
SD 1.92 0.5 2.08 70.20
MIN 3.82 4 6.59 245.65
MAX 14.88 6 16.14 744.67
Note. *p<.05; **p<.01; ***p<.001.
Table 2
Descriptive and Correlational Statistics for Student and Teacher Variables in the Eighth Grade
1 2 3 4 5 6 7 8 9 10 11 12 13
Pearson Correlation
1. Student gender
2. Home educational resources .076***
3. Teacher gender .017 -.002
4. Formal education .062*** .061*** -.002
5. Science content -.065*** -.020 -.050** .210***
6. Science pedagogy/instruction .015 .023 .072*** -.045* .277***
7. Science curriculum .002 -.015 .024 .086*** .500*** .350***
8. Integrating technology .052** .081*** .013 .216*** .371*** .364*** .545***
9. Student critical thinking .013 .081*** .148*** .050** .174*** .360*** .286*** .351***
10. Science assessment -.036* .127*** .066*** -.077*** .277*** .383*** .391*** .341*** .562***
11. Student needs -.072*** .053** .088*** .170*** .132*** .091*** .213*** .271*** .232*** .223***
12. TESI -.019 .047** -.035* .065*** .203*** .188*** .105*** .095*** .179*** .070*** .053**
13. Science achievement .007 .258*** -.012 .074*** -.024 .050** .034 .060** .070*** .085*** .073*** .086***
Descriptive statistics
M 10.28 5.47 9.41 498.79
SD 1.55 0.554 1.78 93.30
MIN 4.55 4 6.51 178.19
MAX 13.52 7 16.03 773.86
Note. *p < .05; **p < .01; ***p < .001.
264
ISSN 2538–7138 /Online/ (pp. 258-274)
Multilevel Mediation Analyses of The Fourth Grade
As Table 3 shows (Step 1), there was a significant main impact of PD in PED on students’ science achievement
after controlling for all covariates (β=0.316, t=2.058, p < .05), such that teachers’ attendance of PD in PED in the
previous two years was related to their students’ higher science scores. After establishing the main impact of PD in
PED on students’ science achievement, it was also shown that home resources for learning had a large and statisti-
cally significant positive impact (β=0.205, t=7.962, p < .001) on science achievement. However, unexpectedly, PD
in science assessment had a negative effect (β=-0.341, t=-2.247, p < .05).
The next step was to establish associations between PD in PED and the mediator TESI. Table 3 (Step 2) shows
that teachers’ attendance of science pedagogy PD in the previous two years was related to higher ratings of TESI
among them (β=0.158, t=3.063, p < .01). All the other programmes also showed significant impacts except for
PD in integrating science subjects. Notably, several programmes had a negative impact on TESI–—PD in science
curriculum (β=-0.271, t=-4.925, p < .001), PD in integrating technology (β=-0.326, t=-5.641, p < .001), PD in science
assessment (β=-0.203, t=-3.943, p < .001)—although some had a positive impact: PD in science content (β=0.292,
t=5.067, p < .001), PD in student critical thinking (β=0.269, t=5.203, p < .001), and PD in student needs (β=0.149,
t=3.485, p < .01). Furthermore, in this step, teachers’ gender and educational levels were shown to have a significant
negative impact on TESI; in other words, male teachers with lower educational levels placed more emphasis on
science investigation.
Step 3 established the impact of the mediator on the dependent variable when controlling for PD in PED
and other covariates. Table 3 (final column) shows that higher ratings of TESI were associated with higher science
scores (β=0.192, t=3.406, p =.001). Specifically, science scores increased by 0.192 unit for every unit increase in
TESI. As such, the association between PD in PED and achievement was assumed to be either nonsignificant or
of lesser magnitude when taking the mediators into account. When comparing parameter estimates of the as-
sociation between PD in PED and achievement, the final estimate was of lesser magnitude than the first (β=0.289
vs. 0.316; t=1.963, p>.05); it had declined by 0.03 and the independent variable no longer had a significant influ-
ence on the dependent variable, demonstrating complete mediation. To further test the indirect impact of PD
in PED and achievement (path ab; see Figure 2), we followed Krull and MacKinnon (1999). The specific pathway
from PD in PED to science achievement through TESI was significant (Sobel’s z=2.298, p = .02; see Figure 2).
Figure 2
Figure 2
Mediation Model for the Fourth Grade
Mediation Model for the Fourth Grade
a=0.158**
PD in PED TESI
Level 2 b=0.192*
c= 0.316*
c’=0.289
Level 1
Science achievement
265
(pp. 258-274) ISSN 2538–7138 /Online/
Table 3
Mediation Analyses: Association between PD in PED and Students’ Science Achievement through TESI in the Fourth Grade
STEP 1: Dependent STEP 2: Dependent STEP 3: Dependent

Science achievement TESI (AdjR2=0.082, F=25.753) Science achievement
β SE t β SE t β SE t
Intercept -0.170 0.644 -0.265 1.561*** 0.208 7.521 -0.476 0.604 -0.789
Level 1 covariates
Student gender -0.053 0.037 -1.421 -0.05 0.037 -1.376
Home resources for learning 0.205 ***
0.026 7.962 0.206 ***
0.026 7.999
Level 2 covariates
Teacher gender 0.073 0.112 0.651 -0.213*** 0.040 -5.356 0.110 0.103 1.067
Formal education 0.015 0.113 0.130 -0.282 ***
0.039 -7.303 0.070 0.109 0.642
Science content -0.089 0.154 -0.578 0.292*** 0.058 5.067 -0.150 0.165 -0.908
Science curriculum 0.102 0.143 0.709 -0.271*** 0.055 -4.925 0.138 0.146 0.947
Integrating technology 0.038 0.149 0.254 -0.326 ***
0.058 -5.641 0.111 0.145 0.761
Student critical thinking 0.101 0.122 0.824 0.269*** 0.052 5.203 0.058 0.117 0.500
Science assessment -0.341 *
0.152 -2.247 -0.203 ***
0.052 -3.943 -0.294 0.154 -1.906
Student needs -0.062 0.105 -0.587 0.149** 0.043 3.485 -0.091 0.109 -0.833
Integrating science subjects -0.072 0.116 -0.622 0.011 0.047 0.238 -0.082 0.110 -0.745
Level 2 independent
PD in PED 0.316* 0.154 2.058 0.158** 0.051 3.063 0.289 0.147 1.963
Level 2 mediator
TESI 0.192** 0.056 3.406
Random Effects
Deviance -2x log 6617.394 6608.849
Difference -2x log 73.975 8.545
Note. ICC1was calculated with the intercept-only model (Null Model: Science achievement=β0j+ 0j); ICC1= 2/ 2+σ2). This gives
the percentage of variance that is attributable to differences between teachers ( 2=variance between teachers; σ2=variance
between students). Null Model: ICC1=32.34%, σ2= 0.684, 2= 0.327, Deviance -2x log=6691.369; Intercept:
β0j=-0.016, SE=0.058, P=.781. *p < .05; **p < .01; ***p < .001.
Multilevel Mediation Analyses of The Eighth Grade
As evident in Table 4 (Step 1), students’ gender (β=-0.128, t=-4.262, p < .001) and home educational resources
(β=0.074, t=4.772, p < .001) and teachers’ educational levels (β=0.326, t=2.487, p < .05) and PD in science content
(β = -0.345, t=-2.371, p < .05) had significant direct impacts on science achievement: male students with more
home educational resources taught by teachers with higher levels of formal education who had not attended
PD in science content achieved higher science scores. However, PD in PED was not related to students’ science
achievement (β=0.135, t=0.765, p > .05).
According to Table 4 (Step 2), PD in PED showed a significant positive impact on TESI (β =0.640, t=7.023,
p < .001). In addition, several PD programmes had a large negative impact on TESI, including PD in integrating
technology (β =-0.282, t=-2.977, p < .01), PD in science assessment (β =-0.332, t=-3.475, p < .01), and PD in student
needs (β =-0.331, t=-4.056, p < .001). Conversely, PD in science content (β =0.579, t=6.153, p < .001) and PD in
student critical thinking (β =0.517, t=5.142, p < .001) were both significantly positively associated with TESI.
As shown in Table 4 (Step 3), PD in PED (β =0.086, t=0.501, p > .05) was unrelated to students’ science achieve-
ment, while TESI (β=0.137, t=2.252, p < .05) was positively related to such achievement, meaning that TESI was not
a significant mediator between PD in PED and students’ science achievement. Though the mediating impact of TESI
266
ISSN 2538–7138 /Online/ (pp. 258-274)
was not significant, the direct impact of TESI on students’ science achievement was still investigated. A detailed
model is presented in Figure 3.
Figure 3
Mediation Model for the Eighth Grade
a=0.640***
PD in PED TESI
Level 2
b=0.137*
c= 0.135
c’=0.086
Level 1 Science achievement
Table 4
Mediation Analyses: Association between PD in PED and Students’ Science Achievement through TESI in the Eighth Grade
STEP 1: Dependent STEP 2: Dependent STEP 3: Dependent

Science achievement TESI (AdjR2=0.057, F=16.671) Science achievement
β SE t β SE t β SE t
Intercept -1.751* 0.729 -2.401 8.820*** 0.372 23.726 -1.695* 0.727 -2.332
Level 1 covariates
Student gender -0.128*** 0.030 -4.262 -0.128*** 0.030 -4.264
Home educational resources 0.074 ***
0.016 4.772 0.074 ***
0.016 4.767
Level 2 covariates
Teacher gender 0.120 0.132 0.906 -0.084 0.078 -1.077 0.126 0.130 0.966
Formal education 0.326 *
0.131 2.487 0.008 0.067 0.117 0.324 *
0.129 2.503
Science content -0.345* 0.145 -2.371 0.579*** 0.094 6.153 -0.402** 0.145 -2.768
Science curriculum -0.041 0.196 -0.211 -0.133 0.099 -1.340 -0.026 0.193 -0.136
Integrating technology 0.017 0.191 0.087 -0.282** 0.095 -2.977 0.039 0.186 0.209
Student critical thinking 0.083 0.174 0.478 0.517 ***
0.101 5.142 0.044 0.172 0.258
Science assessment 0.215 0.167 1.288 -0.332 **
0.096 -3.475 0.238 0.163 1.457
Student needs 0.086 0.155 0.556 -0.331*** 0.082 -4.056 0.113 0.147 0.771
Level 2 independent
PD in PED 0.135 0.176 0.765 0.640*** 0.091 7.023 0.086 0.172 0.501
Level 2 mediator
TESI 0.137* 0.061 2.252
Random Effects
Deviance -2x log 6755.605 6753.152
Difference -2x log 45.572 2.453
Note. ICC1was calculated with the intercept-only model (Null Model: Science achievement=β0j+ 0j); ICC1= 2/ 2+σ2). This gives
the percentage of variance that is attributable to differences between teachers ( 2=variance between teachers; σ2=variance
between students). Null Model: ICC1=59.79%, σ2= 0.429, 2= 0.638, Deviance -2x log=6801.177 ; Intercept: β0j=-0.037, SE=0.079,
P=.636. *p < .05; **p < .01; ***p < .001.
267
(pp. 258-274) ISSN 2538–7138 /Online/
Discussion
Association between PD in PED and Students’ Science Achievement
With reference to previous studies, the researchers controlled for gender and home resources for learning
at the individual student level (Sirin, 2005; Xuan et al., 2019) and gender and formal education at the teacher
level (Toropova et al., 2019; Wayne & Youngs, 2003). The analysis of the data from the fourth grade revealed
that students achieved higher science scores when the teacher had attended PD in PED in the previous two
years. This was in stark contrast to the other types of PD programmes, which did not have any significant
impact on students’ science performance; in fact, the science assessment PD indicated significant negative
effects. None of the teachers’ PD activities were effective, and the science content PD programme indicated
negative significant effects on eighth-grade students’ performance.
The results provided evidence that many teacher PD programmes are not of high quality and are intel-
lectually superficial from the perspective of students’ achievement gains. No comparison of differences across
grades in connection to TIMSS science achievement measures has been conducted to date. The results from
the fourth-grade data are consistent with those reported by Buczynski and Hansen (2010) and Taylor et al.
(2017), which observed that students in the middle and upper grades of primary schools can improve their
science achievement when their teacher receives science pedagogical instruction, and PD can influence
teachers’ classroom practices. However, the observed association between PD in PED and science achieve-
ment in the eighth grade contrasted Johnson et al.’s (2007b) study, which revealed a positive impact of PD
programmes on students’ science achievement in the sixth to eighth grades. This contrast is likely related to
the difference in developmental stages between fourth and eighth graders leading to changes in classroom
characteristics; for example, fourth graders are more energetic, excitable, boisterous, and easily distracted
by one another, meaning teachers must expend more energy on managing classroom discipline, whereas
eighth-grade classrooms tend to be quieter. Therefore, fourth-grade teachers must use teaching methods to
control the classroom and ensure the quality of teaching.
Furthermore, the results revealed that teachers’ gender and formal education levels did not have posi-
tive impacts on student achievement in the fourth grade; however, their formal education levels had positive
significant impacts on student achievement in the eighth grade: the higher the teacher’s educational attain-
ment, the higher their students’ scores. One reason for this result is that compared with the fourth grade, the
eighth-grade science curriculum content was slightly more difficult, thus requiring teachers to have a higher
education level. Combined with the finding that only the PD in science content significantly affected student
achievement in the eighth grade, this may be another indication that the reason teacher PD in PED does not
have a significant impact on student achievement in higher grades is that teachers need other training to
improve their teaching skills, such as PD in science content. The findings indicated that teachers’ attendance
of different PD programmes in different grades predicted students’ science performance to varying degrees.
This means that teachers of different grade require different kinds of PD programmes to allow students to
profit from them, for instance, PD in PED is the most appropriate programme to improve student’s achieve-
ment in the fourth grade.
Teachers were more willing to choose or accept the PD programmes provided by their school and the
Education Bureau because teachers are often passively focused on their own PD. Appropriate PD training
content should be selected based on an understanding of teachers’ current needs, situation, and charac-
teristics and the difficulties of students and teachers at different grade levels. Long-term PD plans should
be developed to meet the diverse needs of teacher-learners, thus facilitating their more targeted par-
ticipation in training and thereby improving the quality of training and promoting students’ development.
The Mediation of TESI between PD in PED and Students’ Science Achievement
These results suggest that PD in PED led to teachers’ increased emphasis on science investigation and
TESI could significantly impact students’ science achievement in both grades, which supports the analyses of
268
ISSN 2538–7138 /Online/ (pp. 258-274)
Jiang and McComas (2015) and Forbes et al. (2020), although differs from the findings presented by Areepat-
tamannil (2012) and Cairns and Areepattamannil (2019).
Furthermore, only a few PD programmes had a significant impact on students’ science scores in the
two grades, although almost all the programmes had a significant impact on TESI (for some, this impact was
positive; for others, it was negative). This could imply that the PD programmes directly affected the teachers’
practice (Bernard & Dudek,2020), thereafter indirectly affecting the students’ achievements. As a result, instead
of being confined in the experimental and quasi-experimental studies, the researchers consider it most neces-
sary to combine the theory, Desimone’s (2009) conceptual framework for teacher PD in real-world scenarios.
Consistent with previous studies (Buczynski & Hansen, 2010; Greenleaf et al., 2011), the results suggested
that changes in teaching occur if teachers undertake appropriate professional development. What was disclosed
by Desimone (2009) and Wayne et al. (2008) in their exploring of the mechanism of PD was substantiated by
this study, namely, having experienced effective professional development, including aspects of knowledge,
skills, attitudes, and beliefs change, teachers can utilize given instruction in the advancement of their own
tutelage and pedagogical means, eventually, students’ learning can thus be facilitated under such advancement.
Based on the above findings, policymakers and school authorities should pay more attention to provid-
ing PD programmes that teachers really need and make sense for teacher practice and focus on how teachers
use the knowledge, content, or pedagogy to change student learning in the future. Hong Kong’s education
authorities should promote efficient, practical, and evidence-based teacher PD programmes; develop teacher
PD programmes that are oriented to the actual needs of teachers; and instil more practical value in the
courses. Based on the findings of previous research, a consortium of universities and primary and secondary
schools should be established that allows teachers to participate in PD programmes with a coach or a highly
knowledgeable teacher, which would result in a larger effect on improving teacher practice skills (Buczynski
& Hansen, 2010; Jacob et al., 2017; Kapanadze et al.,2015). The goal of teacher PD must be changed from
‘providing opportunities’ to ‘providing high-quality programmes’.
Beyond educational authorities, school-based or workshop-based PD programmes should also be encour-
aged. Since teacher’s work is practical, dynamically changing, and varies according to the individual, what
teachers need most is not to be centrally ‘managed’ according to uniform standards but to be encouraged,
supported, and assisted according to the specific situation of the school and class. School-based learning
plays a unique role in teachers’ professional growth because it provides optional and customised training for
teachers. With the vigorous development of online training, the personalised development of teachers has
now become possible. Education authorities should enrich online resources for teachers to give them the
opportunity to obtain personalised professional development resources.
With respect to research design, the present review introduces new questions about PD in PED and the
role of TESI in students’ science achievement. As in any secondary data analysis, some limitations should
be noted when interpreting the findings. First, the results were based on self-reported questionnaire data,
which potentially included social desirability bias. Second, the training components of PD are often combined
organically rather than separately. The enhancement of teachers through PD involves a combination of the
impacts produced by all aspects; therefore, in this study, it was difficult to separate the impacts of PD in PED
from those of other training components. Third, only dichotomous variables (yes or no) were included in the
PD programmes scale. Adding intensity, duration, or other dimensions that can represent the efficiency of
PD could have yielded more robust findings. Furthermore, among the many science pedagogies that have
been found to be effective in science teaching (Slavin et al., 2014), such as inquiry-based pedagogy, problem-
based learning, project-based learning, STEM, and so on, it is uncertain which most affects teacher practice
and student achievement. Teachers’ PD takes many forms; however, the data did not show whether the PD
programmes teachers attended were large-scale programmes or peer coaching or whether they were run
by full-time trained associates or small group facilitators who were hired specifically. Under ideal conditions,
teachers should be followed up for at least one full year after PD is completed in order to find out to what
extent they can maintain new practice after PD support disappears (Kennedy, 2016). Unfortunately, TIMSS
does not collect information on this scale; therefore, the researchers did not have enough statistical power
to investigate the impact of the above factors, which should be further explored in future research.
269
(pp. 258-274) ISSN 2538–7138 /Online/
Fourth, according to previous research (Mellom et al., 2018), attached significant importance to the con-
nection of PD and scholarly accomplishments, within the setting of Hong Kong, the inspiration and attitudes
of teachers towards PD is worth deeper research. Finally, few longitudinal studies were available that have
evaluated changes in the impact of PD in PED or other programmes on academic achievement over time. This
study also did not provide concrete benchmarks for how to change the PD currently offered in Hong Kong.
Future research on how to develop high-quality PD training, incorporate new ideas into ongoing practices,
and the selection of appropriate PD programmes at different stages could be combined.
Basing on Desimone’s conceptual framework of teachers’ PD, the research revealed direct and indirect
links between PD in PED and science achievement in the fourth grade, though the direct impact was weak to
some extent. Consequently, the results confirmed both the hypotheses in the fourth-grade context; however,
these assumptions were not confirmed in the eighth grade. It was regrettable that the study also found that
not all PD programs improved students’ achievement, and some even had a negative impact, it complements
the existing conclusions in the field. Hence, numerous unanswered questions remain with regards to the finest
method to perfect PD and its potential for teacher practice and student achievement. Given this situation, the
researchers must contemplate the most ideal method in the provision of guidance and encouragement to
promote this trend, such as improve the quality of PD programs and give teachers more independent rights
to choose targeted courses and so on.
The results also pointed to a mechanism in which PD in PED led to teachers’ increased emphasis on sci-
ence investigation in both grades. In the fourth grade, TESI completely mediated the relation between PD in
PED and science achievement after controlling for the influences of teachers’ gender, level of formal educa-
tion, and other programmes they had attended in the previous two years. In the eighth grade, though PD in
PED was unrelated to students’ science achievement, it did significantly impact teachers’ emphasis on science
investigation. Further, the results also showed that higher degrees of TESI had a significant positive impact on
students’ science performance in both grades. Although PD plays a relatively limited role in directly improv-
ing student performance, it can promote teachers’ practices largely. The importance of teacher practice has
been strongly verified again in this study and the reasons why the two grades have different results and how
teachers’ practice in the classroom can be enhanced were discussed.
The current study demonstrated the impact of PD in PED on students’ science achievement and simulta-
neously examining the mediating impact of TESI in Hong Kong, proved the theoretical model, extended the
existing research which can contribute to developing both theory and practice. The findings indicated that
PD study should focus on how different programmes impact teachers’ practices and students’ achievements
and how to provide appropriate PD programmes in the future, also led the researchers to think whether the
effectiveness of PD programs should be evaluated in terms of changing classrooms and improving students.
Last decades have witnessed the advance of new technology and the relentless propel on the consolidation
of the foundation for research on PD programmes in view of its plan, execution, and assessment, the scale of
which may probably continue to be broadened drastically in the following years.
Acknowledgements
The authors would like to express our gratitude to Prof. JiJun Yao, Shi Hu and YuanZe Zhu (School of
Educational Science, Nanjing Normal University, People’s Republic of China) for their suggestions on data
analysis and article writing. And thanks to the editor and the reviewers for their work.
270
ISSN 2538–7138 /Online/ (pp. 258-274)
References
Adjapong, E. S., & Emdin, C. (2015). Rethinking pedagogy in urban spaces: Implementing hip-hop pedagogy in the urban science
classroom. Journal of Urban Learning, Teaching, and Research, 11, 66-77. https://files.eric.ed.gov/fulltext/EJ1071416.pdf
Akiba, M., & Liang, G. (2016). Effects of teacher professional learning activities on student achievement growth. The Journal
of Educational Research, 109(1), 99-110. https://doi.org/10.1080/00220671.2014.924470
Al-Balushi, S. M., & Al-Abdali, N. S. (2015). Using a moodle-based professional development program to train science teachers
to teach for creativity and its effectiveness on their teaching practices. Journal of Science Education and Technology,
24(4), 461-475. https://doi.org/10.1007/s10956-014-9530-8
Andersson, C., & Palm, T. (2017). The impact of formative assessment on student achievement: A study of the effects
of changes to classroom practice after a comprehensive professional development programme. Learning and
Instruction, 49, 92-102. https://doi.org/10.1016/j.learninstruc.2016.12.006
Antoniou, P., & Kyriakides, L. (2011). The impact of a dynamic approach to professional development on teacher instruction
and student learning: Results from an experimental study. School Effectiveness and School Improvement, 22(3), 291-
311. https://doi.org/10.1080/09243453.2011.577078
Areepattamannil, S. (2012). Effects of inquiry-based science instruction on science achievement and interest in science:
Evidence from Qatar. The Journal of Educational Research, 105(2), 134-146. https://doi.org/10.1080/00220671.2010.533717
Asamoah, D. A., Doran, D., & Schiller, S. (2020). Interdisciplinarity in data science pedagogy: A foundational design. Journal
of Computer Information Systems, 60(4), 370-377. https://doi.org/10.1080/08874417.2018.1496803
Baker, D. (2013). What works: Using curriculum and pedagogy to increase girls’ interest and participation in science. Theory
into Practice, 52(1), 14-20. https://doi.org/10.1080/07351690.2013.743760
Beilock, S. L., Gunderson, E. A., Ramirez, G., & Levine, S. C. (2010). Female teachers’ math anxiety affects girls’ math achievement.
Proceedings of the National Academy of Sciences, 107(5), 1860-1863. https://doi.org/10.1073/pnas.0910967107
Bernard, P., & Dudek-Różycki, K. (2020). The impact of professional development in inquiry-based methods on science
teachers’ classroom practice. Journal of Baltic Science Education, 19(2), 201-219. https://doi.org/10.33225/jbse/20.19.201
Blums, A., Belsky, J., Grimm, K ., & Chen, Z. (2017). Building links between early socioeconomic status,
cognitive ability, and math and science achievement. Journal of Cognition and Development, 18(1), 16-40.
https://doi.org/10.1080/15248372.2016.1228652
Bruce, C. D., & Ross, J. A. (2008). A model for increasing reform implementation and teacher efficacy: Teacher peer coaching in
grades 3 and 6 mathematics. Canadian Journal of Education,31(2), 346-370. https://files.eric.ed.gov/fulltext/EJ797195.pdf
Buczynski, S., & Hansen, C. B. (2010). Impact of professional development on teacher practice: Uncovering connections.
Teaching and Teacher Education, 26(3), 599-607. https://doi.org/10.1016/j.tate.2009.09.006
Cairns, D. (2019). Investigating the relationship between instructional practices and science achievement in
an inquiry-based learning environment. International Journal of Science Education, 41(15), 2113-2135.
https://doi.org/10.1080/09500693.2019.1660927
Cairns, D., & Areepattamannil, S. (2019). Exploring the relations of inquiry-based teaching to science achievement and
dispositions in 54 countries. Research in Science Education, 49(1), 1-23. https://doi.org/10.1007/s11165-017-9639-x
Chudgar, A., & Sankar, V. (2008). The relationship between teacher gender and student achievement: Evidence from five
Indian states. Compare, 38(5), 627-642. https://doi.org/10.1080/03057920802351465
Coleman, J. S. (1988). Social capital in the creation of human capital. American Journal of Sociology, 94, S95-S120.
https://www.jstor.org/stable/2780243
Darling-Hammond, L., Hyler, M. E., & Gardner, M. (2017). Effective teacher professional development. Learning Policy Institute.
https://files.eric.ed.gov/fulltext/ED606741.pdf
Desimone, L. (2009). Improving impact studies of teachers’ professional development: Toward better conceptualizations
and measures. Educational Researcher, 38(3), 181-199. https://doi.org/10.3102/0013189X08331140
Desimone, L. M., Porter, A. C., Garet, M. S., Yoon, K. S., & Birman, B. F. (2002). Effects of professional development on teachers’
instruction: Results from a three-year longitudinal study. Educational Evaluation and Policy Analysis, 24(2), 81-112.
https://doi.org/10.3102/01623737024002081
Desimone, L., Smith, T. M., & Phillips, K. (2013). Linking student achievement growth to professional development
participation and changes in instruction: A longitudinal study of elementary students and teachers in title i
schools. Teachers College Record, 115(5), 1-46.
Didion, L., Toste, J. R., & Filderman, M. J. (2020). Teacher professional development and student reading achievement:
A meta-analytic review of the effects. Journal of Research on Educational Effectiveness, 13(1), 29-66.
https://doi.org/10.1080/19345747.2019.1670884
Wayne, A. J., Yoon, K. S., Zhu, P., Cronen, S., & Garet, M. S. (2008). Experimenting with teacher professional development:
Motives and methods. Educational Researcher, 37(8), 469–479. https://doi.org/10.3102/0013189X08327154
Fischer, C., Fishman, B., Dede, C., Eisenkraft, A., Frumin, K., Foster, B., Lawrenze, F., Levy, A. J., & McCoy, A. (2018).
Investigating relationships between school context, teacher professional development, teaching practices, and
student achievement in response to a nationwide science reform. Teaching and Teacher Education, 72, 107-121.
https://doi.org/10.1016/j.tate.2018.02.011
271
(pp. 258-274) ISSN 2538–7138 /Online/
Fischer, C., Foster, B., McCoy, A., Lawrenz, F., Dede, C., Eisenkraft, A., Fishman, B., Frumin, K., & Levy, A. J. (2020). Identifying levers
related to student performance on high-stakes science exams: Examining school, teaching, teacher, and professional
development characteristics. Teachers College Record, 122(2), 1-64. https://doi.org/10.1177/016146812012200202
Forbes, C. T., Neumann, K., & Schiepe -Tiska, A. (2020). Patterns of inquir y-based science instruction and
student science achievement in PISA 2015. International Journal of Science Education, 42(5), 783-806.
https://doi.org/10.1080/09500693.2020.1730017
Ford, M. J. (2015). Educational implications of choosing “practice” to describe science in the next generation science
standards. Science Education,99(6),1041-1048.https://doi.org/10.1002/sce.21188
Gore, J., Lloyd, A., Smith, M., Bowe, J., Ellis, H., & Lubans, D. (2017). Effects of professional development on the quality of
teaching: Results from a randomized controlled trial of quality teaching rounds. Teaching and Teacher Education, 68,
99-113. https://doi.org/10.1016/j.tate.2017.08.007
Greenleaf, C. L., Litman, C., Hanson, T. L., Rosen, R., Boscardin, C. K., Herman, J., Schneider, S. A., Madden, S., & Jones, B. (2011).
Integrating literacy and science in biology: Teaching and learning impacts of reading apprenticeship professional
development. American Educational Research Journal, 48(3), 647-717. https://doi.org/10.3102/0002831210384839
Guskey, T. R., & Yoon, K. S. (2009). What works in professional development? Phi Delta Kappan, 90(7), 495-500.
https://doi.org/10.1177/003172170909000709
Hofmann, D. A. (1997). An overview of the logic and rationale of hierarchical linear models. Journal of Management, 23(6),
723–744. https://doi.org/10.1016/S0149-2063(97)90026-X
Hubber, P., Tytler, R., & Haslam, F. (2010). Teaching and learning about force with a representational focus: Pedagogy and
teacher change. Research in Science Education, 40(1), 5-28. https://doi.org/10.1007/s11165-009-9154-9
I nter national Association for the Evaluation of Educational Achievement. (2022). Study Suppor t. IEA.
https://www.iea.nl/research-services/study-support#section-10
Ismail, A. B., Sawang, S., & Zolin, R. (2018). Entrepreneurship education pedagogy: Teacher-student-centred
paradox. Education+ Training,60(2),168-184. https://doi.org/10.1108/ET-07-2017-0106
Jacob, R., Hill, H., & Corey, D. (2017). The impact of a professional development program on teachers’ mathematical knowledge
for teaching, instruction, and student achievement. Journal of Research on Educational Effectiveness, 10(2), 379-407.
https://doi.org/10.1080/19345747.2016.1273411
Jiang, F., & McComas, W. F. (2015). The effects of inquiry teaching on student science achievement and attitudes:
Evidence from propensity score analysis of PISA data. International Journal of Science Education, 37(3), 554-576.
https://doi.org/10.1080/09500693.2014.1000426
Johnson, C. C., Kahle, J. B., & Fargo, J. D. (2007a). A study of the effect of sustained, whole-school professional development on
student achievement in science. Journal of Research in Science Teaching, 44(6), 775-786. https://doi.org/10.1002/tea.20149
Johnson, C. C., Kahle, J. B., & Fargo, J. D. (2007b). Effective teaching results in increased science achievement for all students.
Science Education, 91(3), 371-383. https://doi.org/10.1002/sce.20195
Kapanadze, M., Bolte, C., Schneider, V., & Slovinsky, E. (2015). Enhancing science teachers’ continuous professional
development in the field of inquiry based science education. Journal of Baltic Science Education, 14(2), 254-266.
Kennedy, M. M. (2016). How does professional development improve teaching? Review of Educational Research, 86(4), 945-
980. https://doi.org/10.3102/0034654315626800
Korthagen, F. (2016). Inconvenient truths about teacher learning: Towards professional development 3.0. Teachers and
Teaching: Theory and Practice, 23(4), 387-405. https://doi.org/10.1080/13540602.2016.1211523
Krull, J. L., & MacKinnon, D. P. (1999). Multilevel mediation modelling in group-based intervention studies. Evaluation Review,
23(4), 418–444. https://doi.org/10.1177/0193841X9902300404
Lati, W., Supasorn, S., & Promarak, V. (2012). Enhancement of learning achievement and integrated science process skills
using science inquiry learning activities of chemical reaction rates. Procedia-Social and Behavioral Sciences, 46, 4471-
4475. https://doi.org/10.1016/j.sbspro.2012.06.279
Lee, J., Rhee, D. E., & Rudolf, R. (2019). Teacher gender, student gender, and primary school achievement:
Evidence from ten Francophone African countries. The Journal of Development Studies, 55(4), 661-679.
https://doi.org/10.1080/00220388.2018.1453604
Lin, C. L., & Tsai, C. Y. (2021). The effect of a pedagogical STEAM model on students’ project competence and learning
motivation. Journal of Science Education and Technology, 30(1), 112-124. https://doi.org/10.1007/s10956-020-09885-x
Liu, J., Peng, P., & Luo, L. (2020). The relation between family socioeconomic status and academic achievement in China: A
meta-analysis. Educational Psychology Review, 32(1), 49-76. https://doi.org/10.1007/s10648-019-09494-0
Llosa, L., Lee, O., Jiang, F., Haas, A., O’Connor, C., Van Booven, C. D., & Kieffer, M. J. (2016).Impact of a large-scale science
intervention focused on English language learners. American Educational Research Journal, 53(2), 395–424.
https://doi.org/10.3102/0002831216637348
Mellom, P. J., Straubhaar, R., Balderas, C., Ariail, M., & Portes, P. R. (2018). “They come with nothing:” How professional
development in a culturally responsive pedagogy shapes teacher attitudes towards Latino/a English language
learners. Teaching and Teacher Education, 71, 98-107. https://doi.org/10.1016/j.tate.2017.12.013
272
ISSN 2538–7138 /Online/ (pp. 258-274)
Muralidharan, K., & Sheth, K. (2016). Bridging education gender gaps in developing countries: The role of female teachers.
Journal of Human Resources, 51(2), 269-297. https://doi.org/10.3368/jhr.51.2.0813-5901R1
National Research Council [NRC]. (1996). National science education standards. Washington: The National Academies Press.
Ostermeier, C., Prenzel, M., & Duit, R. (2010). Improving science and mathematics instruction: The SINUS project as an
example for reform as teacher professional development. International Journal of Science Education, 32(3), 303-327.
https://doi.org/10.1080/09500690802535942
Özdem Yilmaz, Y., Cakiroglu, J., Ertepinar, H., & Erduran, S. (2017). The pedagogy of argumentation in science
education: Science teachers’ instructional practices. International Journal of Science Education, 39(11), 1443-1464.
https://doi.org/10.1080/09500693.2017.1336807
Piper, B., Zuilkowski, S. S., Dubeck, M., Jepkemei, E., & King, S. J. (2018). Identifying the essential ingredients to literacy
and numeracy improvement: Teacher professional development and coaching, student textbooks, and structured
teachers’ guides. World Development, 106, 324-336. https://doi.org/10.1016/j.worlddev.2018.01.018
Pugh, K. J., & Girod, M. (2007). Science, art, and experience: Constructing a science pedagogy from Dewey’s aesthetics.
Journal of Science Teacher Education, 18(1), 9-27. https://doi.org/10.1007/s10972-006-9029-0
Quinn, D. M., & Cooc, N. (2015). Science achievement gaps by gender and race/ethnicity in elementary and middle school:
Trends and predictors. Educational Researcher, 44(6), 336-346. https://doi.org/10.3102/0013189X15598539
Reilly, D., Neumann, D. L., & Andrews, G. (2019). Investigating gender differences in mathematics and science:
Results from the 2011 Trends in Mathematics and Science Survey. Research in Science Education, 49(1), 25-50.
https://doi.org/10.1007/s11165-017-9630-6
Roth, K. J., Garnier, H. E., Chen, C., Lemmens, M., Schwille, K., & Wickler, N. I. (2011). Videobased lesson analysis:
Effective science PD for teacher and student learning. Journal of Research in Science Teaching, 48(2), 117-148.
https://doi.org/10.1002/tea.20408
Roth, K. J., Wilson, C. D., Taylor, J. A., Stuhlsatz, M. A., & Hvidsten, C. (2019). Comparing the effects of analysis-of-practice
and content-based professional development on teacher and student outcomes in science. American Educational
Research Journal, 56(4), 1217–1253. https://doi.org/10.3102/0002831218814759
Sarsour, K., Sheridan, M., Jutte, D., Nurujeter, A., Hinshaw, S., & Boyce, W. T. (2011). Family socioeconomic status and child
executive functions: The roles of language, home environment, and single parenthood. Journal of the International
Neuropsychological Society, 17(1), 120–132. https://doi.org/10.1017/S1355617710001335
Schindel Dimick, A. (2016). Exploring the potential and complexity of a critical pedagogy of place in urban science
education. Science Education, 100(5), 814-836. https://doi.org/10.1002/sce.21233
Sirin, S. R. (2005). Socioeconomic status and academic achievement: A meta-analytic review of research. Review of Educational
Research, 75(3), 417-453. https://doi.org/10.3102/00346543075003417
Slavin, R. E., Lake, C., Hanley, P., & Thurston, A. (2014). Experimental evaluations of elementary science programs: A best-
evidence synthesis. Journal of Research in Science Teaching, 51(7), 870-901. https://doi.org/10.1002/tea.21139
Tatar, E., Tüysüz, C., Tosun, C., & Ilhan, N. (2016). Investigation of factors affecting students’ science achievement according
to student science teachers. International Journal of Instruction, 9(2), 153-166. https://doi.org/10.12973/iji.2016.9211a
Taylor, J. A., Roth, K., Wilson, C. D., Stuhlsatz, M. A., & Tipton, E. (2017). The effect of an analysis-of-practice, videocase-based,
teacher professional development program on elementary students’ science achievement. Journal of Research on
Educational Effectiveness, 10(2), 241-271. https://doi.org/10.1080/19345747.2016.1147628
Teig, N., Scherer, R., & Nilsen, T. (2018). More isn’t always better: The curvilinear relationship between inquiry-based teaching and
student achievement in science. Learning and Instruction, 56, 20-29. https://doi.org/10.1016/j.learninstruc.2018.02.006
Toropova, A., Johansson, S., & Myrberg, E. (2019). The role of teacher characteristics for student achievement
in mathematics and student perceptions of instructional quality. Education Inquiry, 10(4), 275-299.
https://doi.org/10.1080/20004508.2019.1591844
Wayne, A. J., & Youngs, P. (2003). Teacher characteristics and student achievement gains: A review. Review of Educational
Research, 73(1), 89–122. https://doi.org/10.3102/00346543073001089
Wayne, A. J., Yoon, K. S., Zhu, P., Cronen, S., & Garet, M. S. (2008). Experimenting with teacher professional development:
Motives and methods. Educational researcher, 37(8), 469-479. https://doi.org/10.3102/0013189X08327154
Wen, C. T., Liu, C. C., Chang, H. Y., Chang, C. J., Chang, M. H., Chiang, S. H. F., Yang, C.W., & Hwang, F. K. (2020). Students’
guided inquiry with simulation and its relation to school science achievement and scientific literacy. Computers &
Education, 149, Article 103830. https://doi.org/10.1016/j.compedu.2020.103830
Wen, F. H., & Chiou, H. J. (2009). Methodology of multilevel modelling: The key issues and their solutions of hierarchical
linear modelling. NTU Management Review, 19(2), 263-293. http://140.136.247.242/~nutr2027/hawjeng09c.pdf
Winters, M. A., Haight, R. C., Swaim, T. T., & Pickering, K. A. (2013). The effect of same-gender teacher assignment on student
achievement in the elementary and secondary grades: Evidence from panel data. Economics of Education Review, 34,
69-75. https://doi.org/10.1016/j.econedurev.2013.01.007
Woltman, H., Feldstain, A., Mackay, J. C., & Rocchi, M. A. (2012). An introduction to hierarchical linear modelling. Tutorials
in Quantitative Methods for Psychology, 8(1), 52–69. http://modir3-3.ir/article-english/article-m2568.pdf
Xuan, X., Xue, Y., Zhang, C., Luo, Y., Jiang, W., Qi, M., & Wang, Y. (2019). Relationship among school socioeconomic status,
teacher-student relationship, and middle school students’ academic achievement in China: Using the multilevel
mediation model. PLOS ONE, 14(3), e0213783. https://doi.org/10.1371/journal.pone.0213783
273
(pp. 258-274) ISSN 2538–7138 /Online/
Zakharov, W., Strobel, J., & Diefes-Dux, H. A. (2020). Teacher level factors and student achievement in a cyber-enabled
engineering education professional development program. International Journal of Research in Education and Science,
6(1), 48-60. https://digitalcommons.unl.edu/biosysengfacpub/726
Zhang, X., Xuan, X., Chen, F., Zhang, C., Luo, Y., & Wang, Y. (2016). The relationship among school safety, school liking,
and students’ self-esteem: Based on a multilevel mediation model. Journal of School Health, 86(3),164–172.
https://doi.org/10.1111/josh.12364
Cite as: Tang, A. B., Li, W. Y., & Liu, D. W. (2022). The impact of teachers` professional development in science
pedagogy on students` achievement: Evidence from TIMSS 2019. Journal of Baltic Science Education, 21(2), 258-274.
AiBin Tang PhD Student, School of Education Science, Nanjing Normal University,
No.122, Ninghai Road, Gulou District, Nanjing, People’s Republic of China
& Hangzhou Tianchang Primary School, People’s Republic of China.
E-mail: tangaibin_njnu@163.com
WenYe Li PhD Student, School of Education Research, Nanjing University,
No.22, Hankou Road, Gulou District, Nanjing, People’s Republic of China.
E-mail: 841659607@qq.com
DaWei Liu PhD, Associate Professor, School of Early Childhood Education, Nanjing
(Corresponding author) Xiaozhuang University, No.41,Beiwei Road, Jianye District, Nanjing,
People’s Republic of China.
E-mail: tyb_18@foxmail.com
274
PRIMARY PRE-SERVICE
TEACHERS’ METAPHORICAL
PERCEPTIONS OF THE ISSN 1648-3898 /Print/
CONCEPT OF ENVIRONMENTAL ISSN 2538-7138 /Online/
POLLUTION
Abstract. Metaphors are expressions

that are frequently used in people’s minds
Eser Ültay to explain concepts with other unrelated
concepts and have a personal emphasis
on learning. The purpose of this study is to
determine and interpret the metaphorical
perceptions of primary school pre-service
teachers towards environmental pollution.
Introduction The “phenomenology” research design was
used in this study. The study group of this
The environment is a system that includes all living things on Earth as research consists of a total of 372 primary
well as elements such as air, soil, and water that are required for life (Bonnett, pre-service teachers. Data were collected
2007). The environment, which is a network of relationships, is critical for liv- using semi-structured forms to determine
ing things to live a healthy life and includes also all self-arising things. Thus, the metaphorical perceptions of primary
environment should be seen from a holistic view and unfortunately, majority school pre-service teachers. The pre-service
of people, love their pets and hate bacteria. But this point of view cannot be teachers were asked to complete the
seen as holistic. However, it could be argued that the rejection of holism here sentence, “Environmental pollution is like...
is a manifestation of an underlying instrumentalism - loving only what we because....”. At the conclusion of multi-
believe meets our needs and desires - which, in turn, is an expression of the ple comparisons, by the relation degree
metaphysical mastery that has resulted in an atomistic dominance of nature. between environmental pollution and its
Furthermore, and perhaps more importantly, it has limited our understanding metaphors, it was determined that signifi-
of what the world has to offer and who we are (Bonnett, 2007). cant difference was in favour of the 2nd
According to Tay and Diener (2011), while needs appear to be a relatively year between the 2nd and 4th years and in
inelastic part of people’s consumption, wants in the form of desires are more favour of the 3rd year between the 3rd and
variable as they are shaped by cultural perceptions of what is important or 4th years. As a result of the research, it was
valuable. People in general do not know which behaviours shaped by their discovered that the majority of the meta-
desires affect others’ use of available resources, and they do not receive phors produced by the pre-service teachers
specific feedback about the outcomes of behaviour changes (Gatersleben were in the “human” and “harmful/danger-
et al., 2002). As a result, when someone is aware that others are in pain, they ous/uncomfortable situations” categories.
can act altruistically by feeling obligated to alleviate that pain (Schwartz, Furthermore, the study discovered that
1977), while they can more easily engage in behaviours about which they as the year level increased, the number of
are unaware of the extent to which they affect the other person. In a similar related metaphors about environmental
perspective throughout the history of mankind, environmental changes have pollution decreased.
inevitably emerged as a result of people’s desire to meet their needs and
Keywords: environmental pollution, meta-
desires, such as physical comfort, mobility, labour savings, enjoyment, power,
phorical perceptions, primary pre-service
status, personal security, tradition and care of family, technology, etc. (Stern,
teachers
2000, Tay & Diener, 2011) without knowing that some people are causing
someone pain. Actually, many different factors contribute to the problems
encountered, including population growth, unintentional use of natural
resources, environmental degradation, and urbanization (Bonnett, 2004). Eser Ültay
People’s need for natural resources, unconscious consumption, and Giresun University, Turkey
environmental insensitivity also contribute to a gradual increase in envi-
ronmental pollution. Indeed, studies show that human factors are the root
275
(pp. 275-287) ISSN 2538–7138 /Online/
causes of environmental pollution (Gardner & Stern, 1996; Tung et al., 2002). While humans use rapidly depleting
natural resources to meet all of their wants and needs, they also pollute the environment with residues from pro-
duction and consumption, gradually breaking a link in the chain that maintains natural balance (Şanlı, 1984). As a
result, educators bear a great deal of responsibility in reducing and halting the recent increase in environmental
pollution. Individual environmental awareness, which can be gained through education, can allow the elimination
of problems that may be hazardous to our world (Davis, 1998). As a result, educators must instil environmental
awareness in students. While humans cause environmental problems consciously or unconsciously in order to meet
their needs, they also play a significant role in eradicating these problems (Bonnett, 2007).
Environmental education studies must be conducted in order to find a solution to this problem that threatens
our future. According to the studies, environmental education should begin at a young age in order to eliminate this
problem because unfortunately, many families in all over the world do not have enough awareness and equipment
to educate their children about the environment (Mandel, 2013; Payne, 2005). Environmental education, which
should be continued throughout one’s life, should begin at a young age. Because interests and attitudes are formed
at a young age allow human behaviour to be positively shaped (Davis, 1998). Although the young generation do
not cause the existing environmental problems, they will be the most affected by them (Ateş & Karatepe, 2013).
People’s environmental awareness can enable them to live a happier, healthier, and more peaceful life (İpek, 2014;
Yalçınkaya, 2012).
The concept of “environmental pollution,” the subject of our research, has been evaluated using the “metaphors”
to reveal different ways of thinking. Metaphors are expressions that are frequently used in people’s minds to explain
concepts with other unrelated concepts and have a personal emphasis on teaching (Botha, 2009). Metaphors serve
an important purpose in expressing the desired explanation in fewer words and with greater emphasis. In other
words, it is possible to determine the chance of expressing a strong expression with fewer words with metaphors.
It is possible to list the characteristics of metaphors as follows (Kalyoncu, 2012; Tompkins & Lawley, 2002):
- Defining a concept with a different concept
- Describing a concept by analogy with a different concept
- Seeing a concept in a different dimension
- Interpreting a concept with a different pattern
As the definitions show, metaphors are words that help to express what is wanted to be known or said in
the simplest, most creative, and permanent way possible, in a scope that can correspond to many words that are
wanted to be said in daily life (Koç, 2014). Metaphors generate ideas and hypotheses that can be investigated and
possibly tested. They serve as powerful cognitive models through which educators and students can comprehend
educational phenomena by relating them to something previously encountered (Botha, 2009). Metaphors also
allow you to compare two things, highlight similarities between two things, or explain one thing by substituting
it for another (Saban, 2004). The way different concepts are understood by individuals is attempted to be shown
through metaphors in metaphor studies (Ateş & Karatepe, 2013). The metaphors developed for the concept of
“environmental pollution” discussed in our study have not only revealed the perceptions of university students
about environmental pollution, but also have reflected the importance given to the environment and environ-
mental pollution.
When the literature is examined, it has been seen that many metaphor studies have been carried out in the
field of education. In some of these studies, for example Yılmaz et al. (2013) and Ekiz and Koçyiğit (2012) have ex-
plored pre-service teachers’ perceptions about “teacher” concept by metaphors. Similarly, Kalyoncu (2012) has also
explored “teaching profession” perceptions of visual arts pre-service teachers and Koç (2014) has searched primary
pre-service teachers’“teacher and teaching profession” perceptions by metaphors. In a similar manner, Navari et al.
(2009) have investigated “teacher and learner” perceptions of students, Şengül et al. (2014) have searched “math-
ematics teacher” perceptions of students by metaphors. Additionally, Furuoka and Nikitina (2008) have searched
“language teacher” perceptions. While Dönmez Usta and Ültay (2015) have explored pre-service teachers’ “chem-
istry” perceptions, Ekici (2016) has searched “microscope” perceptions, and Ergen and Yanpar-Yelken (2015) have
focused on “technology” perceptions of students by metaphors. Bessette and Paris (2020) in their studies analysed
teachers’ professional identities by eliciting textual (written) and visual (drawn) metaphors of teaching to reflect
on their teaching role(s) and circumstances.
When we look at metaphor studies on the environment, we can see that the studies are primarily concerned
with the environment and environmental problems. For example, Aydın (2011) and Doğan (2017) have focused
on “environment” concept perceptions by metaphors. A content analysis has been used by Çakmak (2018) to
276
ISSN 1648–3898 /Print/ PRIMARY PRE-SERVICE TEACHERS’ METAPHORICAL PERCEPTIONS OF THE CONCEPT OF
ISSN 2538–7138 /Online/ (pp. 275-287)
review the metaphor studies on the environment. At the end of her research, she has reviewed 27 studies on
environmental metaphors in Turkey and has discovered that only one study on “environmental pollution” has
been conducted by 2018. Apart from this study, two studies were also found about the metaphoric perceptions
related to environmental pollution. Köseoğlu (2017) has explored the concept of “water pollution” perceptions
of biology pre-service teachers through metaphors. Arık and Yılmaz (2017) have studied on science pre-service
teachers’ metaphorical perceptions of “environmental pollution.” When the literature is examined, no other study
has yet been found that examines the metaphorical perceptions of primary school pre-service teachers towards
the concept of “environmental pollution.” In this regard, it is anticipated that this study can significantly contribute
to the literature and serve as an example for future research.
Research Purpose
The purpose of this study was to determine and interpret the metaphorical perceptions of primary school
pre-service teachers studying at Giresun University, Education Faculty, Primary Education Department, towards
environmental pollution. The following questions were addressed in this study:
1. What metaphors do primary school pre-service teachers use to explain their understanding of the
concept of “environmental pollution”?
2. How can the metaphors created by primary school pre-service teachers about the concept of “envi-
ronmental pollution” be classified?
3. Is there a difference in association levels between classes?
Research Design
The “phenomenology” research design was used in this study to determine primary school pre-service teach-
ers’ metaphorical perceptions of environmental pollution. The goal of the phenomenological design is to reveal
common practices as well as define and explain the meanings created by the participants (Annells, 2006). Data
sources in phenomenological studies are people or communities who have experienced the phenomenon under
study and can express or reflect on it (Patton, 1990).
Participants
The study group of this research consisted of a total of 372 pre-service teachers from 1st, 2nd, 3rd and 4th
years studying at Giresun University, Education Faculty, Primary Education Department in the fall semester of the
2017-2018 academic year. These pre-service teachers, whose ages ranged from 18-24, were 148 boys and 224 girls.
No sample selection was made, and all of the pre-service teachers registered at that time were included in the
study. The reason for the selection of pre-service teachers studying in primary school was that, as mentioned in
the introduction, teaching environmental education and environmental awareness at an early age can yield good
consequences. For this reason, it was considered important to include these pre-service teachers, who will be the
first teachers of our children, in this study.
Data Collection Tools
Data were collected using semi-structured forms to determine the metaphorical perceptions of primary school
pre-service teachers who participated in the research on the concept of environmental pollution. The researcher
carried out the implementation of the research with the permission of the course lecturer. First and foremost, the
pre-service teachers were educated on the metaphor technique. Later, the pre-service teachers were given a form
for writing a sentence, and they were asked to complete the sentence, “Environmental pollution is like... because....”
pre-service teachers were given ten minutes to complete this task. As a result, the metaphors and explanations as
the research data were formed.
277
(pp. 275-287) ISSN 2538–7138 /Online/
Data Analysis
Before starting the data analysis, it should be decided whether to use parametric or non-parametric tests. In
order to perform parametric tests on quantitative data, the data must have a normal distribution and homogeneous
variances (Patton, 1990). Because these two conditions were not met in this study, we used the Kruskal-Wallis Test,
which is one of the non-parametric tests used to compare the averages of more than two groups, instead of Para-
metric Tests for analysis. The quantitative data from the study were analysed using IBM SPSS v22.
The descriptive content analysis method was used to analyse the qualitative data from the study. Descriptive
content analysis is defined as a systematic, repeatable technique in which some words from a text are summarized
into smaller content classes using coding based on specific rules (Ültay et al., 2021). Descriptive content analysis
is useful for summarizing, standardizing, comparing, or transforming existing data (Smith, 1975; Huberman &
Miles, 1994). In three stages, metaphors created by classroom pre-service teachers were analysed in this context
(Kalyoncu, 2012; Saban, 2008). These stages are used in data analysis;
a. Naming stage
b. Elimination stage
c. Category development stage
a. Naming Stage: At this stage, the metaphors created by the primary school pre-service teachers for the con-
cepts of “environment” and “environmental pollution” were transferred and sorted on the computer. Meaningful
metaphors and their explanations were examined following the ranking process. The sentences before and after
“because” that were thought to be related were given 2 points, sentences that were thought to be less related
were given 1 point, and unrelated sentences were given 0 (zero) points. At this stage, codes were written for the
metaphors to create a transition to the categorization stage.
b. Elimination Stage: Metaphors produced by primary school pre-service teachers at this stage, as in Saban
(2008);
- The subject of the metaphor,
- The source of the metaphor,
- The relationship between the subject of the metaphor and its source is discussed.
In this direction;
Metaphors which do not contain a source,
Those who do not give a reason for the metaphor produced
Metaphors that do not contribute to the explanation of the concepts of “environment” and “environmental
pollution” were determined and they were excluded from the scope of the study.
c. Category Development Stage: The concepts of “environment” and “environmental pollution” produced by
primary school pre-service teachers were analysed and conceptualized in terms of their characteristics. Metaphors
with similar themes were included in the same category.
Reliability and Validity
The researchers divided the metaphors produced by the primary school pre-service teachers into categories,
and validity and reliability studies were conducted. To increase the validity of the research study, the participants’
statements were taken directly, and the process was detailed. Four independent researchers classified the metaphors
created by pre-service teachers. The researchers then came together, compared their lists, and held discussions until
agreement was reached on the ideas that differed. The resulting categories were tabulated and expressed as frequen-
cies and percentages based on how frequently they were used. The data were interpreted in light of the findings.
Ethics
The consent of the primary school pre-service teachers was obtained for the sharing of the data obtained during
the data collection process. In addition, it was clearly stated that some demographic information of the participants
would be shared with the reader, and their consent was obtained, and it was stated that they would not be harmed
by this. Within the framework of research ethics, pre-service teachers of each class in the data collection process
were coded as “1P1, 1P2, ... , 1P90”, “2P1, …, 2P69”, “3P1, …, 3P83”, “4P1, …, 4P130” respectively. The number at the
beginning of the code represents the year level. The last number represents the participant’s sequence number.
278
ISSN 2538–7138 /Online/ (pp. 275-287)
Research Results
Data were collected using a semi-structured data collection form. The pre-service teachers’ metaphors were
classified as related (2), partially related (1), or unrelated (0), and then the categories were created.
Quantitative Results
After the definition of a total of 352 metaphors and the distribution of these metaphors into categories, all
data were transferred to the IBM SPSS v22 statistical program. After this process, the number of pre-service teach-
ers representing each metaphor, the mean and standard deviation of the relationship scores of the classes were
calculated and shown in Table 1.
Table 1
Descriptive Statistics
Study Year N SD
1st Year 90 1.289 0.838

2nd Year 69 1.565 0.581
3rd Year 83 1.530 0.704
4th Year 130 1.315 0.778
Total 372 1.403 0.751
The average score of the metaphors written by the 1st year pre-service teachers was 1.289, the average of
the 2nd year pre-service teachers’ scores was 1.565, the average of the 3rd year pre-service teachers’ scores was
1.530, and the 4th year pre-service teachers’ average scores were 1.315. In total, the mean of the scores of 372
classroom pre-service teachers was calculated as 1.403 and the standard deviation as .751. In order to determine
the significant difference between the correlation means of the classes, the sample must be at least 30, the data
must show normal distribution, and the variances must be homogeneous in order to perform parametric tests.
Since the sample is larger than 50, the normal distribution test, which is the second step of the parametric tests,
was performed and is shown in Table 2.
Table 2
Test of Normality
Kolmogorov-Smirnova
Study Year
Statistics df p
1st Year Related Score 0.335 90 < .05

2nd Year Related Score 0.382 69 < .05
3rd Year Related Score 0.398 83 < .05
4th Year Related Score 0.318 130 < .05
According to Table 2, since Kolmogorov-Smirnov (Test of Normality) p values are <.05, the data do not show
a normal distribution. The homogeneity test of the variances was performed and is shown in Table 3.
279
(pp. 275-287) ISSN 2538–7138 /Online/
Table 3
Homogeneity of Variances Test
Levene Statistic df1 df2 p
7.531 3 368 < .05
According to Levene’s test, the variances are not homogeneous as the p value is <.05. Since two conditions
were not met, non-parametric tests were used instead of parametric tests for analysis. The Kruskal-Wallis test, one
of the non-parametric tests, was used to compare the means of more than two groups.
Since the p value was <.05 according to the Kruskal-Wallis test results, it was concluded that there was a sig-
nificant difference between at least two of the study years [χ2(df=3)=7.70, p<.05]. In the study, the Mann-Whitney
U test was used to determine which study years were significantly different (for multiple comparisons), and the
Mann-Whitney U Tests’ Results for cross-year analyses are shown in Table 4.
Table 4
Cross-year Mann-Whitney U Tests
Study Year Mann-Whitney U Z p
1st and 2nd years 2637.000 -1.826 .068

1st and 3rd years 3188.000 -1.885 .059
1st and 4th years 5821.000 -0.068 .945
2nd and 3rd years 2835.000 -0.124 .901
2nd and 4th years 3790.500 -2.001 .045
3rd and 4th years 4581.500 -2.078 .038
According to the cross-year Mann-Whitney U tests’ results, since the p (2-tailed) value is >.05, there is no
significant difference in the mean relationship scores between the 1st and 2nd years, the 1st and 3rd years, the
1st and 4th years, and the 2nd and 3rd years. However, as can be seen from the table, since the p (2-tailed) value
is <.05, there are significant differences in the mean relationship scores between the 2nd and 4th years, and the
3rd and 4th years.
The Kruskal-Wallis test revealed a significant difference between the relationship scores of the four classes
[χ2(df=3)=7.70, p<.05]. At the conclusion of the Mann-Whitney U test multiple comparisons, it was determined
that this difference was in favour of the 2nd grade between the 2nd and 4th grades and in favour of the 3rd grade
between the 3rd and 4th grades. Table 5 summarizes these results.
Table 5
Summary of Statistical Results
Years N df χ2 p Significant Difference
1st Year 90 3 7.701 .048 2nd-4th

2nd Year 69 3rd-4th
3rd Year 83
4th Year 130
Qualitative Results
According to the study’s findings, 352 metaphors were classified based on their year level. The metaphors
elicited from participants as a result of the content analysis are listed below under the headings of 1st, 2nd, 3rd, and
4th year participants’ metaphors for the concept of “environmental pollution” and their categories. Table 6 shows
280
ISSN 2538–7138 /Online/ (pp. 275-287)
the metaphors obtained from the first year pre-service teachers, as well as frequency and percentage analyses of
their categories, as a result of the content analysis.
Table 6
Participants’ Metaphors about Environmental Pollution
Category Metaphors of 1st year Metaphors of 2nd year Metaphors of 3rd year Metaphors of 4th year
Human Stealing (2), Taking Death (5), Smoking (4), Heart full of bad thoughts Human (2), Pessimistic thinking
one’s own life (2), An evil Unnecessary waste (2), Evil (8), Apathy (2), Germ (2), person (1), Person in the dark (1),
spirit (1), Man who talks (2), Drinking alcohol (2), Cancer (1), Communication Person in danger of life (1), Uncared
unnecessarily (1), Cruelty Disease (2), End of living (1), Disrespecting oneself person (1), Man in a primitive life (1),
(1), Hostility to nature (1), Decay of life (1), Beast (1), Characterless people Gathering of bad people (1), People’s
(1), Soul pollution (1), (1), Oxygen (1), Cigarette (1), Human mind (1), fault (1), Person with a stopped heart
Insensitivity (1), Death (1), Loud sound (1), Virus Distorted psychology (1), (1), Human guilt (1), A dead person (1)
(1), Evil (1), Unhappiness (1), Weapon (1), Loud music Thought pollution (1), a Throwing garbage on the ground (1),
(1), Doing unfairly (1), (1), a diseased body (1), smoking lung (1), Window Polluting the sea (1), Bad thoughts of
Insensitive people (1), a breathless lung (1), a (1), Garbage (1), Swamp people (1), Stress (1), Disrespect (1),
A person in prison (1), healthy life (1), Faded lungs (1), Polluted world (1) Anger (1), a negative behaviour (1),
Suicide (1), the living (1), Swamp (1), Vascular Murder (1), Traffic accident (1), Evil
dead (1) occlusion (1) (1), Traffic monsters (1), Sadness of
Despair (2), Betrayal (1), the world (1), Lack (1), Exhaustion of
Human Pollution (1), Gossip life (1), Everybody saying something
(1), Carnage (1), Human mind (1), Prolonging school (1), Disorderly
(1), Human psychology (1) life (1)
Example Pollution of the Environmental pollution is like Environmental pollution Environmental pollution is
sentences environment is equivalent a monster that accelerates is like a smoked lung, like a disease. It has negative
to injustice. Because it the end of the world because because it is a deliberate consequences for humans and even
makes it impossible for it destroys the ecological harm done by people with all living things.
people to live in a beautiful balance by destroying nature their own hands.
environment. and our beauties, therefore
the world becomes unlivable.
Harmful / Cigarette (2), Smoker Germ (4), A smoking lung Disease (7), Death (4), End -
dangerous / human body (1), Smoker (1), the end of the world (1), of the world (3), Disruption
uncomfortable human lung (1), Alcohol a smoking liver (1), Failing of natural balance (2),
situations (1), Smoking (1) machine (1), a growing Human (2), Virus (2),
lake (1), Broken phone (1), Disaster (1), Killer (1),
Avalanche (1), Tortured death Experimental waste (1),
(1), Cancer (1), Problem (1), Pest (1), Infested hair
Illness (1 (1), Cancer (1), Influenza
(1), Extinction (1), Can’t
breathe (1), Darkness (1),
Visual pollution (1), Surplus
(1), World (1)
Noise (2), Messy room (1),
Truck wheel (1), Socks
(1), Cold food (1), Visual
pollution (1)
Example Environmental pollution is Environmental pollution is Environmental pollution is -
sentences like the body of a smoker like a growing lake because like death because it kills
because gradually the it continues to increase as nature.
environment becomes it increases. As it grows, an
polluted and poisoned. insignificant drop turns into a
huge lake.
281
(pp. 275-287) ISSN 2538–7138 /Online/
Nature and Life Life without oxygen (2), Nature pollution (1), Virus (1), Being unable to breathe -
Catastrophe (2), Virus (2), Natural disaster (1), Cut one’s (4), Death (1), Being deaf
Cancer (2), Crocodile (1), own throat (1), Disturbance (1), Living in a garbage
Rotten lung (1), Nature (1) dump (1), a dirty house (1),
massacre (1), Killing Hot tea (1), Marriage (1), Dirty Disruption of the natural
a human (1), Filth (1), toilet (1), Not getting ready balance (1), an axe hitting
the end of the world (1), for the exam (1), Dirtying a tree (1)
Today’s problem (1), Lung our house (1), University (1), Life without water (1),
(1), Not breathing in an Human life (1), the person Murderer (1), Self-harm (1),
oxygen environment (1), a who doesn’t take a shower A dirty house (1), Not being
life without forest (1) (1), the uncared woman able to breathe (1), Looking
at the world with astigmatic
eyes (1)
Example Environmental pollution Environmental pollution is Environmental pollution is -
sentences is like cancer because it like a dirty toilet because it like a dirty house because
slowly destroys all beauty. is nauseous, dirty, and living nature is our living space.
conditions are unfavourable.
Waste and Trash (4), Vomit (1), Not - - Garbage (16), Pollution of life (1),
garbage flogging the toilet (1), Trash can (1), Living in a dirty house
Rotten pear (1), Fruit (1), (1), Throwing garbage on the ground
Dirty our house (1), Insect (1), Rotten fruit (1), Withered flower
(1), Piggy bank (1), Living (1), Noise pollution (1), Living in
in the trash (1) garbage (1), Water-soil-sea pollution
(1), Visual pollution (1), Microbe (1)
Example Environmental pollution - - Environmental pollution is like a trash
sentences is like vomit because it can because the pollution of the
disturbs people with their environment over time is like a trash
appearance and sense can; it gets worse over time.
organs.
Health Disease (7), Garbage (3), - - Respiratory failure (4), Disease (4),
Bacteria (3), Virus (2), A contagious disease (3), Cancer
Rotting fungi (1), Microbe (2), Inability to sense (1), A black
society (1), Sick human (1), lung (1), Disease that will bring the
a huge microbe (1), Living end of the world (1), Complementary
in the trash (1) organs (1), Depleted lung (1), Breath
(1), An injured creature (1), Heart
attack (1), Antibiotic (1), Harmful
substance in the body (1), Substance
that harms human (1), Bad breath (1),
The stomach of the man who eats
everything (1), Stomach disease in
the human body (1), Nausea (1), An
ecological society (1), Turbid water
(1), Life (1)
Example Environmental pollution - - Environmental pollution is like organs
sentences is like garbage because that complement each other because
it is the source of human deterioration in one affects the other.
diseases, it contains
microbes.
282
ISSN 2538–7138 /Online/ (pp. 275-287)
Various ideas Student house (1), - - Unwashed dishes (1), a pillowless

(uncategorized) Mountain (1), Negligent mattress (1), a stinky container (1), a
woman (1), Paper (1), collapsed house (1), a black colour
Ungrateful cat (1), (1), Dishes (1), Dominoes (1), Stuck
Surveyors (1), Bleeding gum (1), Open-ended question (1),
eye (1), Bad life (1), Wet toilet slippers (1), Messy room
Raining in the desolate (1), a wilted flower (1), a broken
desert (1), Worship (1), smartphone (1), a piece of cake (1),
Killing the environment (1), a punching bag (1), Dirty socks (1),
Venus (1), Adana kebab Messy house (1), a dirty house (1),
without red pepper (1), Socks ripped (1), a very nice dress
Festival (1), Regret (1), the dirty (1), Vacuum cleaner (1), Air -
beast (1) water - soil (1), Waste dispersal (1),
Rushing river (1), a dead world (1)
Example Environmental pollution - - Environmental pollution is like wet
sentences is like a student house toilet slippers because we cannot
because it is not normal, it understand the gravity of the event
is abnormal. before it happens to us.
According to Table 6, six metaphor categories were formed. The only metaphor category identified as
common at all year levels is “human”. In this category, pre-service teachers stated a lot of different metaphors
related to “human” such as “stealing, death, evil, cruelty, smoking, etc.” 1st, 2nd, and 3rd year pre-service teachers
said some metaphors such as disease, cigarette, germ, etc. in the category of “harmful/dangerous/uncomfort-
able situations” and additionally, in the category of “nature and life.” In the categories of “waste and garbage”,
“health”, and “various ideas (uncategorized)”, 1st and 4th year pre-service teachers stated several metaphors.
Discussion
The study included 372 primary school pre-service teachers, and the total number of valid metaphors
for environmental pollution was 352, with these metaphors examined in six different categories. The reason
for producing too many metaphors could be that pre-service teachers were instructed on how to produce
metaphors at the start of the application. Bessette and Paris (2020) stated that it is recommended that an
instruction should be given to the participants about what metaphor means before exploring their metaphoric
perceptions. Another reason for this could be that the environmental issue is very broad, comprehensive,
and complex (Aydın, 2011), and the participants may not be well-versed in environmental issues. Tung et al.
(2002) stated that students’ knowledge of environment is limited and incomplete. On the other hand, it was
discovered that there was a statistically significant relationship between the pre-service teachers’ educa-
tion level and their perceptions of environmental pollution. It was discovered that as pre-service teachers’
education levels increased, they were able to make less related sentences about environmental pollution.
The reason for this could be that the 2nd and 3rd years are required to take the “Environmental Education”
course as part of their curriculum. The freshness of the information on this subject may have been reflected
in the research findings and it might have led pre-service teachers’ sensitive thoughts. On the other hand, it
is observed that as the class levels of the pre-service teachers increase, they define environmental pollution
with a more individual thought, and it might have been thought as they had internalized their knowledge
about environmental pollution. In some studies, conducted in the literature (Arık & Yılmaz, 2017), it is seen
that different results (socialist perspective) have been reached. Contrary to these studies, it has been observed
in some studies that attitudes towards environmental pollution are more individual, and social thinking is
more in the background (Erol, 2005). As Tung et al. (2002) said, it is not evident in all areas of environmental
issues yet. Despite these different results, it is a well-known fact that pre-service teachers’ knowledge and at-
titudes towards environmental pollution are not generally reflected in their behaviours (Esa, 2010). Garbage,
for example, can be discovered in the faculty where these pre-service teachers learn as well as in the faculty’s
garden. This demonstrates that there is a serious distinction between knowing something is incorrect and
putting it into practice. But still, it is expected of pre-service teachers to think and behave in a correct way
283
(pp. 275-287) ISSN 2538–7138 /Online/
and responsibly because they will be teachers and educate young individuals who do not have information
about the environment and environmental pollution.
It has been discovered that the human category contains the greatest number of metaphors across all
year levels. As Geary (2011) said, metaphor is such a wonderful thing that it mixes the known with the un-
known and creates a wonderful world.” In this sense, pre-service teachers stated metaphors from a very large
viewpoint from stress to murder, and even to gossip. This greatest number of metaphors can be interpreted as
pre-service teachers’ individualistic views on environmental pollution. However, whether one’s way of thinking
is individual or socialist, the outcome affects the entire society. Individual and socialist perspectives can be
said to be inextricably linked. That is, there are people at the bottom of society, and people’s interventions
in the environment lay the groundwork for environmental problems (Buell, 2004). Furthermore, even local
human interventions in the environment have resulted in global environmental problems over time (Cairns,
2002; Chawla, 1998). The association of the first, second, and third classes with environmental pollution with
harmful/dangerous/disturbing situations indicates that environmental pollution causes negative situations.
Similarly, when we examine the category of associating with nature and life, we see that they have a nega-
tive/hopeless outlook. As a result, pre-service teachers’ perspectives on preventing/reducing environmental
pollution are hopeless/negative. The reason for this could be that they believe it is extremely difficult to pre-
vent environmental pollution and thus environmental problems. Indeed, Arık and Yılmaz (2017) emphasized
in their study that environmental pollution of science pre-service teachers is a process that spreads quickly
and unevenly. As a result, he discovered that pre-service teachers consider environmental pollution to be
an irreversible process. All in all, pre-service teachers’ negative metaphors about environmental pollution in
several categories are understandable, and also interesting. According to Tung et al. (2002), the knowledge
of students can be increased, it is reasonably simple, but their values, beliefs and attitudes are generally dif-
ficult to change. Pre-service teachers’ negative viewpoints can arise from their pessimistic thoughts about
the world. In general, they cannot be seen as unfair due to global warming, extinction of some animals, and
also depletion of some food resources, wars, terrorism, etc.
It was discovered that participants focused on the types of damages caused by environmental pollution
and the effects of these damages in the metaphor categories established. Similarly, Çakmak (2018) concluded
in his study of metaphor studies on the environment in Turkey that the focus is primarily on environmental
pollution and its consequences. As a result of the data obtained within the scope of this research, it can be said
that primary school pre-service teachers have a high level of sensitivity towards environmental pollution, but
they do not have a strong perspective on environmental pollution. It is clear that the participants do not use
terms like global climate change, greenhouse effect, carbon emission, or acid rain to describe environmental
pollution. Based on this, it is not possible to say that the participants’ knowledge of environmental pollution
is advanced. The fact that the participants did not mention species extinction or natural disasters (avalanches,
floods, earthquakes, etc.) in their metaphors or explanations is further evidence of this. According to Arık and
Yılmaz (2017), pre-service teachers have low perceptions of environmental problems and pollution. In fact, in
the twenty-first century, the environment and environmental problems are among the most pressing concerns
around the world (Akgün et al., 2016). Specifically for this sample, pre-service teachers may be bored of talking
about this negative trend. They may not have wanted to discuss it any longer, as environmental, and global
problems, as well as their fears for the future, have pushed them to this gloomy viewpoint.
Metaphors are labels, meanings, or conceptual expressions that a concept generates in a person. It’s a
process of seeing and comprehending. It is more significant and powerful mental production than simply
explaining one concept with another because it expresses the breadth and depth of the associated concept
(Eraslan, 2011). From this vantage point, creating metaphors about environmental pollution is both difficult and
simple. It is simple because almost everyone has had some exposure to environmental pollution. It is difficult
because it requires in-depth knowledge of environmental pollution in order to generate metaphors on the
subject. Furthermore, Arık and Yılmaz (2017) discovered that pre-service teachers lacked special knowledge
(global warming, climate change, etc.) about environmental pollution. In this case, it can be stated that some
pre-service teachers lack knowledge about environmental pollution, while others, despite having knowledge,
are unable to convey or apply that knowledge.
284
ISSN 2538–7138 /Online/ (pp. 275-287)
Conclusions and Recommendations
The primary goal of this study was to identify the metaphors developed by pre-service teachers for the
subject of “Environmental Pollution” and to see if there was a difference between year levels. As a result of the
research, it was discovered that the majority of the metaphors produced by the pre-service teachers were in
the “human” and “harmful/dangerous/uncomfortable situations” categories. Furthermore, the study discovered
that as the year level increased, the number of related metaphors about environmental pollution decreased.
According to the findings of the study, the metaphors used by pre-service teachers to describe environmental
pollution are human and human movements, as well as situations that directly affect people are harmful or
disrupt their health/comfort. Also, at the conclusion of multiple comparisons, by the relation degree between
environmental pollution and its metaphor, it was determined that significant difference was in favour of the
2nd year between the 2nd and 4th years and in favour of the 3rd year between the 3rd and 4th years.
As a result of the research, it was discovered that the primary school pre-service teachers had only a
cursory understanding of environmental pollution and lacked a comprehensive understanding or perspec-
tive. As a result, it can be stated that pre-service teachers’ perceptions of environmental pollution are low. To
improve primary school pre-service teachers’ environmental knowledge and interests, it is suggested that the
“Environmental Education” course in the curriculum can be taught through the active learning model, with pre-
service teachers actively learning their knowledge through practice and interaction with the environment. As
a result, by internalizing their knowledge of the environment and environmental problems or pollution, they
can contribute to taking steps in the right direction for society. Universities can help/mediate the formation
of specific social clubs and social societies to raise environmental awareness. It is suggested that participants
participate in extracurricular activities, expand their knowledge, and become active members of school clubs.
Endnote
A part of this study was presented as a summary paper at the 27th International Conference on Educa-
tional Sciences, 18-22 April 2018, Antalya, Turkey.
References
Akgün, A., Duruk, Ü., & Güngörmez, H. G. (2016). Secondary school students’ metaphors related to environmental education.
Dicle University Journal of Ziya Gökalp Faculty of Education, 28, 215-224. https://doi.org/10.14582/DUZGEF.714
Annells, M. (2006). Triangulation of qualitative approaches: Hermeneutical phenomenology and grounded theory. Journal
of Advanced Nursing, 56(1), 55-61. https://doi.org/10.1111/j.1365-2648.2006.03979.x
Arık, S., & Yılmaz, M. (2017). Prospective science teachers’ attitude toward the environmental problems and their
metaphorical perceptions about “environmental pollution”. Kastamonu Education Journal, 25(3), 1147-1164.
https://dergipark.org.tr/tr/download/article-file/313420
Ateş, M., & Karatepe, A. (2013). The analysis of university students’ perceptions towards “environment” concept with the
help of metaphors. International Journal of Social Science, 6(2), 1327-1348. http://dx.doi.org/10.9761/jasss_630
Aydın, F. (2011). The metaphoric perceptions of university students towards environment concept. Eastern Geographical
Review, 16(26), 25-44. https://dergipark.org.tr/tr/download/article-file/26965
Bessette, H. J., & Paris, N. A. (2020). Using visual and textual metaphors to explore teachers’ professional roles and identities.
International Journal of Research & Method in Education, 43(2), 173-188. http://dx.doi.org/10.1080/1743727X.2019.1611759
Bonnett, M. (2004). Retrieving nature: Education for a post-humanist age. Blackwell.
Bonnett, M. (2007). Environmental education and the issue of nature. Journal of Curriculum Studies, 39(6), 707-721.
http://dx.doi.org/10.1080/00220270701447149
Botha, E. (2009). Why metaphor matters in education. South African Journal of Education, 29(4), 431-444.
http://dx.doi.org/10.15700/saje.v29n4a287
Buell, F. (2004). From apocalypse to a way of life: Environmental crisis in the American century. Routledge.
Cairns, J. (2002). Sustainability and sacred values. Ethics in Science and Environmental Politics, 13, 15-27.
https://doi.org/10.3354/ESEP002015
Chawla, L. (1998). Significant life experiences revisited: A review of research on sources of environmental sensitivity. Journal
of Environmental Education, 29(3), 11-21. https://doi.org/10.1080/00958969809599114
Çakmak, M. (2018). Content analyses of metaphor studies performed in terms of environment concept in Turkey. Mediterranean
Journal of Educational Research, 12(25), 179-193. https://doi.org/10.29329/mjer.2018.153.10
285
(pp. 275-287) ISSN 2538–7138 /Online/
Davis, J. (1998). Young children, environmental education, and the future. Early Childhood Education Journal, 26, 117-123.
https://doi.org/10.1023/A:1022911631454
Doğan, Y. (2017). Middle school students’ intuitive perceptions related to concept of the environment: A metaphor
analysis. Ahi Evran University Journal of Kırşehir Faculty of Education, 18(1), 721-740. https://dergipark.org.tr/tr/
download/article-file/1481514
Dönmez Usta, N., & Ültay, N. (2015). A comparative study of preschool student teachers’ “chemistry” metaphors. The Black
Sea Journal of Social Sciences, 7(02), 1-14. https://doi.org/10.12780/uusbd296
Ekici, G. (2016). Determination of the preser vice biology teachers’ perceptions of microscope: Example
for metaphor analysis. Ahi Evran University Journal of Kırşehir Faculty of Education, 17(1), 615-636.
Ekiz, D., & Koçyiğit, Z. (2013). Exploring primary school teachers’ metaphors concerning “teacher”. Kastamonu Education
Journal, 21(2), 439-458. https://arastirmax.com/tr/system/files/1066/21_2_3.pdf
Eraslan, L. (2011). Sociological metaphors. Academic Perspective Journal, 27, 1-22. http://eski.bingol.edu.tr/
media/262572/9sosyolojikdusunme-SOSYOLOJiK-METAFORLAR.pdf
Ergen, B., & Yanpar-Yelken, T. (2015). Metaphoric perceptions of primary 3rd class students about technology concept. The
Journal of Academic Social Science Studies, 39, 509-527. http://dx.doi.org/10.9761/JASSS2975
Erol, G. H. (2005). Primary school teaching department sophomore students’ attitudes toward environment and environmental
problems [Unpublished master’s thesis]. Pamukkale University.
Esa, N. (2010). Environmental knowledge, attitude, and practices of student teachers. International Research in Geographical
and Environmental Education, 19(1), 39-50. https://doi.org/10.1080/10382040903545534
Furuoka, F., & Nikitina, L. (2008). A language teacher is like…”: Examining Malaysian students’ perceptions of
language teachers through metaphor analysis. Electronic Journal of Foreign Language Teaching, 5(2), 192-205.
https://files.eric.ed.gov/fulltext/ED509070.pdf
Gardner, G. T., & Stern, P. C. (1996). Environmental problems and human behavior. Allyn & Bacon.
Gatersleben, B., Steg, L., & Vlek, C. (2002). Measurement and determinants of environmentally significant consumer behavior.
Environment and Behavior, 34(3), 335-362. https://doi.org/10.1177/0013916502034003004
Geary, J. (2011). I is an other: The secret life of metaphor and how it shapes the way we see the world. Harper Collins.
Huberman, A. M., & Miles, M. B. (1994). Qualitative data analysis. Sage.
İpek, M. (2014). On the environment-man relationship in Islamic culture. Journal of Academic Research in Religious Sciences,
14(3), 227-240. http://isamveri.org/pdfdrg/D03296/2014_3/2014_3_IPEKM.pdf
Kalyoncu, R. (2012). Visual arts pre-service teachers’ metaphors related to concept of “teaching”. Mustafa Kemal University
Journal of Social Sciences Institute, 9(20), 471-484. https://dergipark.org.tr/tr/download/article-file/183098
Koç, E. S. (2014). The metaphorical perceptions of classroom pre-service teachers regarding teacher and teaching profession
concepts. Inonu University Journal of the Faculty of Education, 15(1), 47-72. https://doi.org/10.17679/iuefd.79408
Köseoğlu, P. (2017). An analysis of university students’ perceptions of the concepts of “water” and “water pollution”
through metaphors. EURASIA Journal of Mathematics Science and Technology Education, 13(8), 4343-4350.
https://doi.org/10.12973/eurasia.2017.00930a
Mandel, P. (2013). Children as change agents: The influence of integrating environmental education into home learning projects
on families and community members. https://digitalcommons.fiu.edu/cgi/viewcontent.cgi?article=1098&context=sferc
Navari, S., Pishghadam, R., & Torghabeh, R. (2009). Metaphor analysis of teachers’ beliefs and conceptions of language
teaching and learning in iranian high schools and language institutes: A qualitative study. Iranian EFL Journal, 4(4),
6-40. https://profdoc.um.ac.ir/articles/a/1011729.pdf
Patton, M. Q. (1990). Qualitative evaluation and research methods. Sage.
Payne, P. G. (2005). Families, homes, and environmental education. Australian Journal of Environmental Education, 21, 81-95.
https://doi.org/10.1017/S0814062600000975
Saban, A. (2004). Entry level prospective classroom teachers’ metaphors about the concept of “teacher”. The Journal of
Turkish Educational Sciences, 2(2), 131-155. https://dergipark.org.tr/tr/download/article-file/256419
Saban, A. (2008). Metaphors about school. Educational Administration: Theory and Practice, 55, 459-496.
Schwartz, S. (1977). “Normative influences on altruism”. In L. Berkowitz (Ed.), Advances in experimental social psychology
(10th Ed.). Academic Press.
Smith, H. W. (1975). Strategies of social research: The methodological imagination. Prentice-Hall.
Stern, P. C. (2000). Toward a coherent theory of environmentally significant behavior. Journal of Social Issues, 56(3), 407-424.
https://doi.org/10.1111/0022-4537.00175
Şanlı, Y. (1984). Environmental problems and food contamination. Journal of Selcuk University Faculty of Veterinary Medicine,
Special Issue, 17-37. http://eurasianjvetsci.org/pdf/pdf_EJVS_957.pdf
Şengül, S., Katrancı, Y., & Gerez-Cantimer, G. (2014). Metaphor perceptions of secondary school students about “mathematics
teacher”. The Journal of Academic Social Science Studies, 25(1), 89-111. http://dx.doi.org/10.9761/JASSS2155
Tay, L., & Diener, E. (2011). Needs and subjective well-being around the world. Journal of Personality and Social
Psychology, 101(2), 354-365. http://dx.doi.org/10.1037/a0023779
286
ISSN 2538–7138 /Online/ (pp. 275-287)
Tompkins, P., & Lawley, J. (2002). The mind, metaphor and health. Positive Health, 78, 9-12. http://www.positivehealth.com/
issue/issue-78-july-2002
Tung, C. Y., Huang, C. C., & Kawata, C. (2002). The effects of different environmental education programs on the environmental
behavior of seventh-grade students and related factors. Journal of Environmental Health, 64(7), 24-29. https://
shibbolethsp.jstor.org/start?entityID=https%3A%2F%2Fidp.giresun.edu.tr%2Fidp%2Fshibboleth&dest=https://www.
jstor.org/stable/44528864&site=jstor
Ültay, E., Akyurt, H., & Ültay, N. (2021). Descriptive content analysis in social sciences. IBAD Journal of Social Sciences, 10,
188- 201. https://doi.org/10.21733/ibad.871703
Yalçınkaya, E. (2012). The levels of primary 6th grade students’ awareness of environmental issues. Marmara Journal of
Geography, 25, 137-151. https://dergipark.org.tr/tr/download/article-file/3280
Yılmaz, F., Göçen, S., & Yılmaz, F. (2013). Pre-service teachers’ perceptions of the concept of teacher: A metaphoric study. Mersin
University Journal of the Faculty of Education, 9(1), 151-164. https://dergipark.org.tr/tr/download/article-file/160870
Received: January 18, 2022 Revised: March 16, 2022 Accepted: April 04, 2022
Cite as: Ültay, E. (2022). Primary pre-service teachers’ metaphorical perceptions of the concept of environmental pollution.
Eser Ültay PhD, Associate Professor, Faculty of Education, Giresun University, Güre,
Prof. Ahmet Taner Kışlalı Cd. No. 34, 28200 Giresun Merkez/Giresun,
Turkey.
E-mail: eserultay@gmail.com
287
SCIENCE MAPS AND

BIBLIOMETRIC ANALYSIS ON
HYGIENE EDUCATION DURING
2012-2021
Muhammet Usak,
Selma Sinan,
Olcay Sinan
Abstract. Hygiene education is becoming Introduction

increasingly popular and is now addressed
in both formal and informal education Hygiene education has become a popular concept discussed in both
systems. Examining hygiene education formal and informal education systems. Examining research in the field of
research and developing a vision for the hygiene education and developing a forward view of this topic will lead to the
future will lead to creating a roadmap for development of a roadmap for future research as well as an analysis of past
future research as well as an analysis of research. Before we define the boundaries of hygiene education, we must
past research. Research on hygiene educa- define the term “hygiene.” It is in this context that the World Health Organi-
tion encompasses a variety of subtopics. zation’s concept of hygiene emerges. Hygiene refers to the conditions and
It is critical for future researchers and behaviors that help maintain health and prevent the transmission of disease
thematic studies in this area to determine (WHO, 2017). The focus of the definition is on protecting the health and not
if there is a pattern to these concerns that spreading disease. Health is a social science topic with significant difficulties
cover a wide range of topics. The purpose of and potential for the future of humanity, and it is closely related to education
this study is to examine the topic of hygiene (Arnold, 2018). Two main concepts are mentioned: The first concept is condi-
education using bibliometric analysis. From tions and the second is behavior. The fulfillment of conditions can be associated
the Scope Database, 503 records remain with many variables. When it comes to the concept of “behavior”, learning the
for bibliometric analysis. This results in an behavior, reinforcing the right behavior, and eliminating the wrong behavior
average number of 5.02 publications per are associated with “education”.
year. 1973 people contributed to the study. The definition of hygiene education in this context is hygiene education
Among the top 10, most influential sources aimed at providing the necessary training to prevent the spread of disease and
in terms of the number of articles are four the acquisition of the necessary behavior, knowledge, and attitude to protect
websites related to dental hygiene. The oneself from disease. The details of the definition will be more specific depend-
United States leads the world in both the ing on the subject area. For example, in the field of hand hygiene (HH), hygiene
number of publications and a total number education is understood as learning and applying hand hygiene standards and
of citations, followed by Canada and China. guidelines (Kelcíkova et al., 2012). Oral or dental hygiene education is about
Most of the research was related to oral learning oral health behaviors that help prevent oral health impairment (Mueller
hygiene education. et al., 2022). Hygiene education is a multidisciplinary field of study (V. Curtis et
Keywords: bibliometric analysis, hand al., 2003) that is closely related to health (Huang et al., 2019), health education
hygiene, hygiene education, dental hygiene (Jatmika & Maulana, 2018), and science education (Eley et al., 2020; Morrish &
Neesam, 2021) in education. The material in the hygiene education curriculum
is integrated into science education curricula, particularly at the elementary
Muhammet Usak
Kazan Federal University, Russia and secondary school levels (Morrish & Neesam, 2021). Even when considered
Selma Sinan, Olcay Sinan in this context, hygiene education is closely related to science education.
The University of Texas at Austin, USA Hygiene education is considered crucial in preventing the spread of vari-
ous dangerous diseases, especially among adolescents. According to Diouf
288
ISSN 1648–3898 /Print/ SCIENCE MAPS AND BIBLIOMETRIC ANALYSIS ON HYGIENE EDUCATION DURING 2012-2021
(pp. 288-304)
et al. (2014), the majority of diarrheal diseases are preventable through the implementation of water, sanitation,
and hygiene initiatives (WASH), all of which aim to stop the fecal-oral transmission routes colloquially referred to
as the five “F’s” (liquids, fields, flies, fingers, and food). Numerous small-scale measures to improve hygiene and
water treatment help reduce diarrheal disease in children (Clasen et al., 2007; Ejemot et al., 2008). Hand hygiene
may have prevented diseases such as gastroenteritis, respiratory infections, trachoma, fatal neonatal infections,
and worm infections (Biran et al., 2014). Hand hygiene education has been shown to help increase handwashing
compliance among health care workers (Kelcíkova et al., 2012). Some studies have demonstrated that individuals’
knowledge and performance of hand hygiene habits improved as a result of a handwashing activity or campaign
(Biran et al., 2014; Grover et al., 2018).
According to Curtis et al. (2009), large-scale handwashing promotion initiatives are needed that go beyond
the basic premise that teaching about germs and diseases would influence behavior. They proposed a strategy to
identify how handwashing can be integrated into children’s routines, motivations, and planned activities in their
home environment. Studies in hygiene education are important for all age groups and countries. Hygiene education
is included in the curricula of many countries (Wittberg et al., 2021). There is an urgent need for increased targeted
teaching activities on proper hand hygiene (Guinan et al., 2002; Szilágyi et al., 2013). Kenyan students in grades 1-8
received hygiene training. Results suggest that the provision of low-cost, locally accessible materials combined
with low-intensity teacher training led to increased latrine cleanliness in + HW schools and improved availability of
handwashing materials and behaviors in LC +HW and HW schools throughout the study period (Caruso et al., 2014).
For hand hygiene programs implemented to be successful, the positive contributions of all stakeholders, such
as school administrators and teachers, should be incorporated (Guinan et al., 2002). For the school environment,
hygiene training is necessary not only for students but also for other staff members. Tan et al. (2013) emphasize
the importance of food service workers being aware of hand hygiene (RTE) and knowledgeable about the pres-
ence of germs and proper handwashing. To promote better hand hygiene, the government should emphasize the
need for food service workers to wear mouth guards and wash their hands thoroughly. Another component of
hygiene education is dental health. Oral disease is a serious public health problem that affects an estimated 3.9
billion people worldwide, according to WHO (Eley et al., 2020; Marcenes et al., 2013). It is an obvious fact that oral
disease prevention is becoming increasingly important as evidence accumulates that oral health is not only about
oral health and quality of life, but can also affect overall health (Mueller et al., 2022).
Numerous studies (Hamilton et al., 2017; Kressin et al., 1999; Scheerman et al., 2016; Tadin et al., 2022) have
shown an association between oral hygiene behaviors such as frequency of tooth brushing, interdental cleanliness,
dental visits, use of fluoride oral hygiene products, and sugar reduction with personality traits and psychological
factors. Dental and oral problems do not only exist in developing countries (Jatmika & Maulana, 2018), but also in
developed countries (Eley et al., 2020). Regular and proper tooth brushing is the most effective preventive mea-
sure to maintain oral health (Tadin et al., 2022). According to Jatmika and Maulana (2018), oral hygiene instruction
improved students’ dental hygiene practices, so similar training should be continued in schools. Other points of
discussion include the hygiene education curriculum. Numerous nations are forced to make changes due to intense
curricula (Nordin & Sundberg, 2016). In this case, there is debate about whether the hygiene education curriculum
should stand alone (Birch et al., 2019) or be integrated into the science curriculum (Arnold, 2018; Zeyer, 2012).
Hygiene and health education have come to the forefront, especially after the COVID -19 epidemic (Lounis, 2020;
Čavojová et al., 2022; Morrish & Neesam, 2021).
Bibliometrics Analysis
Bibliometrics is the process of obtaining quantifiable data from published research papers through statistical
analysis, as well as the process of determining how the knowledge contained in a publication is used. A method for
methodically quantifying the number of publications within their field was developed by American psychologists
in the early twentieth century and served as the basis for subsequent groundbreaking metric research (Agarwal et
al., 2016; Thanuskodi, 2010). Authors’ keywords are used to analyze clusters formed by pairing documents. Pairing
is defined as two articles that are bibliographically linked if at least one cited source or keyword is included in the
references and keyword lists of both articles (Kessler, 1963). It was decided to use a network analysis technique
based on the co-occurrence of keywords contributed to the research by the authors. The notion of degree cen-
trality and weighted links are introduced, and these concepts can be used to evaluate co-authorship and citation
networks, respectively (Kretschmer & Kretschmer, 2007).
289
(pp. 288-304)
Callon et al. (1991) expressed that the basis of this type of analysis is the idea that the number of co-occurring
words describes the content of the documents in the file. Since it is part of a larger network, it can be considered a
cluster or node. This is because it is connected to other clusters or nodes in the larger network. It could be thought
of it as a group of words that are connected to each other. It can also be thought of as a dense network that is
more or less connected and strong. A method of analysis that examines how often each author’s words are used
together is called a “thematic map.”
Thematic networks are represented in two dimensions, with the horizontal and vertical axes representing
the centrality and density of the thematic network. The axes show the relevance of the theme in the study area
and its evolution (Noyons, 2001; Zammarchi & Conversano, 2021). Centrality is a measure of how strong the con-
nections of a cluster are to other clusters. The more connections and the stronger they are, the more that cluster is
considered a group of important research problems identified by the scientific or technical community. A cluster
is made up of many words, and density refers to the strength of the connections that link them all together. The
stronger these connections are, the more research challenges associated with the cluster form a single whole. One
could say that density is a good indicator of how likely a cluster is to stay together and grow over time in the field
under study (Callon et al., 1991; Haustein & Larivière, 2015). Based on the research conducted in the introduction,
it is clear that studies on hygiene education cover a wide range of subtopics. It is critical for future researchers, as
well as thematic studies in this area, to determine if there is a pattern in these questions that cover a wide range
of topics. This study will conduct a study on hygiene education based on bibliometric analysis.
The following research questions were asked:
1) What research, authors, contexts, and sources are the most important and most cited on the topic of
hygiene education?
2) What are the popular topics in hygiene education today?
3) What are the research priorities in hygiene education?
4) What about coincidence, co-citation, and collaboration among nations in the field of hygiene educa-
tion?
General Background
To answer the research questions, a bibliometric analysis was conducted. According to Gokhale et al. (2020),
a bibliometric analysis allows the reader to gain a comprehensive understanding of a particular field over some
time. In this study, a bibliometric strategy was used that adhered to PRISMA guidelines (McInnes et al., 2018; Welch
et al., 2016). Since the goal is to examine the most recent 10 publications in research, publications between the
years 2012 and 2021 are used as the baseline. The Scopus database provides access to journal articles on hygiene
education and the references cited in those publications, allowing the searcher to go both forward and backward in
time. The database can be used for both collection building and research. In addition, the Sopus database provides
suitable data for bibliometric analysis.
Data Sources
Science, social sciences, arts, and humanities are served by Scopus, a world-class research platform that facilitates
the discovery, analysis, and dissemination of information. The Scopus database contributes to the efficiency and ef-
fectiveness of the research process by providing access to a wealth of information. In this case, the Scopus database
was selected because it indexes the most prestigious journals in the fields of education, social sciences, and medicine,
and also provides suitable data for bibliometric studies. The search was conducted on the Scopus database website
using an online search engine. The search term “hygiene education” was chosen as the starting point. Various restric-
tions were then applied, such as language and time restrictions. The query string used for the search was:
TITLE-ABS-KEY ( “hygiene education” ) AND ( EXCLUDE ( PUBYEAR , 2022 ) ) AND ( LIMIT-
TO ( PUBYEAR , 2021 ) OR LIMIT-TO ( PUBYEAR , 2020 ) OR LIMIT-TO ( PUBYEAR , 2019 ) OR LIMIT-
TO ( PUBYEAR , 2012 ) ) AND ( LIMIT-TO ( LANGUAGE , “English” ) )
290
(pp. 288-304)
Figure 1
Flow Chart for Selecting Publication
Since the last ten years were decided as part of the research, the scans were made to cover the years 2021-
2012, which corresponded to the research period. The data on the studies obtained as a result of the search were
downloaded in both BibTex and CSV formats. Data were reviewed by opening the Excel file in CSV format. Studies
with more than one study were identified. The related papers were manually deleted in both BibTex and CSV formats
because they were a single publication. On the other hand, papers without an author were deleted. In this step, 32
records were deleted. In the next step, the title and abstract parts of each publication were examined. Studies on
“vocal hygiene education” and “sleep hygiene education” were eliminated. In addition, studies that did not relate
to hygiene education were excluded from the study. As a result of this process, 209 study data were deleted. 503
study data were remaining for the bibliometric analysis.
Data Analysis
It is possible to divide the bibliometric analysis into two categories. The first is the descriptive and performance
analysis of the data. This study contains general information about the sources and document types used. Also, the
number of studies and citations is estimated on an annual and total basis, providing statistical information. Then,
depending on the top 10 or 20 studies, a list of the most cited studies is displayed. It is then offered to readers as
the most prolific authors, sources, institutions, or nations. Second, scientific mapping and network analysis are
used to perform the analyzes. The current bibliometric study was conducted using the shiny app for Bibliometrix,
which is part of the R statistical package. It has many features that make it particularly suitable for conducting
in-depth bibliometric analyzes of scientific publications. Web-based software that provides access to the features
and functions of Bibliometrix 3.1.4 (Aria & Cuccurullo, 2017).
Research Results
The results are presented in the following order: general statistics first, followed by effective source, publica-
tion, and author information. The last section deals with the results of the scientific map.
291
(pp. 288-304)
Table 2
Descriptive Information on Publication
Description Results
Main Information About Data

Timespan 2012:2021
Sources (Journals, Books, etc) 215
Documents 503
Average years from publication 5.04
Average citations per document 6.555
Average citations per year per doc 0.997
References 13177
Document Types
article 471
review 32
Document Contents
Keywords Plus (ID) 2751
Author’s Keywords (DE) 1378
Authors
Authors 1973
Author Appearances 2307
Authors of single-authored documents 23
Authors of multi-authored documents 1950
Authors Collaboration
Single-authored documents 25
Documents per Author 0.255
Authors per Document 3.92
Co-Authors per Documents 4.59
Collaboration Index 4.08
As shown in Table 2, the statistics in the study span 2012-2021, or the last decade. In total, there are 2015 sources
and 503 publications in this collection. According to the general data, there were an average of 5.02 publications
per year. Each publication has cited an average of 6.5 times. These works cited a total of 13177 publications. Among
these publications, there are 471 articles and 32 reviews. The authors used 1378 different keywords. The studies
have 1973 authors. These authors have a total of 2307 appearances. Although there is only one author listed in
each of the 25 publications, the “authors per document” value is 3.92.
292
(pp. 288-304)
Figure 2
Top 10 Journals Based
Figure 2 on Total Citations
Top 10 Journals Based on Total Citations
103
82
24
4
Journal Of Dental Education
Environmental Research And

Public Health
Journal Of Water Sanitation
Bmc Public Health

And Hygiene For Development
Journal Of Hospital Infection
Amerıcan Journal Of Infection

International Journal Of Dental
Canadian Journal Of Dental
Control
American Journal Of Tropical

Journal Of Dental Hygıene : JD..
Hygiene
Hygiene
Medicine And Hygiene

International Journal Of
The first four places on the list of the top ten sources in terms of the number of publications are occupied by
sources on dental hygiene (Figure 2). The other six sources all relate to the topic of general health.
Figure
Figure 3 3
Effective ArticlesArticles
Effective based onbased
citationon citation
11
90
10.11
9.90 10
80 9.10 9.12
9
8.47
70 8
7.42
7
60 6.84 6.84
6.59
6.23
Normalized TC
Total Citations
5.90 6
50 5.68
5.32
5
40 4.61
4.08 4
30
3.22
3
20
2
10
8.27 7.55 7.80 6.10 5.40 6.00 8.50 6.13 4.27 11.50 10.00 9.00 7.33 2.00 1.50 1.50 1
---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ----
0 91 83 78 61 54 54 51 49 47 46 30 27 22 4 3 3 0
(Schmidlin et al., 2013)
(Diouf et al., 2014)
(Lazzarotto et al., 2020)
(Tan et al., 2013)
(Gyorkos et al., 2013)
(Darvesh et al., 2017)
(Hogan et al., 2021)
(Fritz et al., 2012)
(Cheng et al., 2015)
(Garin et al., 2012)
(Huda et al., 2012)
(Huang et al., 2019)
(Głąbska et al., 2020)
(Rashid, 2021)
(Wittberg et al., 2021)
(Montresor et al., 2020)
293
(pp. 288-304)
The effectiveness of the publications was determined by the number of citations. In Figure 3, the total
number of citations is represented by the height of the blue bars of the publications. According to the width of
the blue bars, the ratio of the number of citations per year is shown. The normalization of the total citations is
represented by the red circles. When looking at total citations, Schmidlin et al. (2013) are at the top, while Huda
et al. (2012) are at the top when years are put into proportion and normalized. Although the studies are related
to hygiene education in some way, the specifics of the research are related to different areas. Nine of the sixteen
studies (Cheng et al., 2015; Darvesh et al., 2017; Diouf et al., 2014; Garin et al., 2012; Głąbska et al., 2020; Huang et
al., 2019; Montresor et al., 2020; Schmidlin et al., 2013; Tan et al., 2013) were directly related to the use of hygiene
interventions to reduce the prevalence of a specific disease. Each study examined the impact of improvements in
water, sanitation, and hygiene (Hogan et al., 2021; Wittberg et al., 2021) and hand hygiene (Fritz et al., 2012; Huda
et al., 2012). Each study focused on a different aspect of general health education, a social responsibility initiative
related to hygiene education, and knowledge, attitudes, and behaviors related to hand hygiene (Gyorkos et al.,
2013; Lazzarotto et al., 2020; Rashid, 2021).
Table 3
Effective Authors Based on Citations
h h index g g index m m index TC

Element TC NP PY start
index rank index rank index rank Rank
Gyorkos Tw 3 5 3 12 0.30 36 113 1 3 2013
Fritz Sa 2 18 2 23 0.18 49 94 2 2 2012
Hogan Pg 2 19 2 24 0.18 50 94 3 2 2012
Gadbury-Amyot Cc 4 1 6 1 0.40 21 47 21 9 2013
Inglehart Mr 3 6 6 3 0.38 29 39 22 9 2015
Kinoshita A 4 2 6 2 0.36 34 39 23 7 2012
Bray Kk 4 3 4 9 0.40 22 34 26 4 2013
Li J 2 33 2 33 0.67 1 6 45 2 2020
Liu Y 2 34 2 34 0.67 2 4 46 2 2020
Aiemjoy K 1 47 1 47 0.50 3 3 47 1 2021

Note PY start: Year of publication by the author TC: Total citations NP: number of publications
In the analysis of the ten most effective authors in terms of effectiveness (as shown in Table 3), Gyorkos Tw,
which ranks first in terms of a total number of citations, was first published in 2013. Gyorkos Tw’s first published
in 2013, which puts it in first place overall. In terms of the h-index, Gyorkos Tw is ranked fifth, while in terms of the
g-index it is ranked 12th and in terms of the m-index, it is ranked 36th. By the h- and g-index, Gadbury-Amyot CC
is the first author of the paper, which was first published in 2013. According to the m-index, it is ranked twenty-
one. Li J. is ranked first in the m-index according to the m-index. The author’s first study was conducted in 2020.
Each index categorizes authors by a different method from the others. Based only on the total number of citations,
older publications are considered favorable. Depending on the goal of the comparison, multiple indices may be
used for comparison.
294
(pp. 288-304)
Figure 4
Top-Authors’ Production over the Time
Reviewing the effectiveness of the writers is checked over time, and we find that Kinoshita A has been suc-
cessful for nine years. Kanji Z and Partido Bb have been available for at least three years. The number of publica-
tions is indicated by the size of the circles in the illustration. At the same time, the blackness of the colored blue
indicates that the number of citations has exceeded the annual average number. For example, Bayd Ld published
four articles on this topic in 2020. Similarly, Rowe Dj published four articles in 2016. Looking at the average annual
number of citations, Kinishita A has the most on the list in 2020 with an average of 3 citations. The annual citation
rate for Rodo Ej was 2.78 in 2014, as shown in the data.
Figure 5
Corresponding Author’s Country
295
(pp. 288-304)
Looking at the distribution of corresponding authors by country, they are divided into two categories: one
country and multiple countries. Switzerland has the highest MCP ratio in terms of proportion. As MCP, the num-
ber of publications is larger in the United States. In Korea, Turkey, Hong Kong, Nigeria, Brazil, and Hong Kong, all
publications were classified as SCP.
Figure 6
Country Collaboration Network
When the cooperation between countries is examined, 2 clusters are formed. In the first cluster, the USA made
the largest contribution. The second cluster includes Norway and Uganda. The size of the circles here represents
the folding of the country, while the rest of the lines represent the cooperation between the two countries.
Figure 7
Country Contribution to Topic
50
489
500
47.00 45
450
40
400
35
350
33.00
Average Article Citations
30
300
Publication
27.00 25
250
23.50
20
200
15
150
15.00
100 10
90
9.38
73
8.28
55
7.29
5
44
50 5.50
5.22
33
15
2.67
0 0
POLAND 5
HUNGARY 4
SLOVAKIA 1
MADAGASCAR 1
USA
CANADA
CHINA
JAPAN
TURKEY
AUSTRALIA
SWITZERLAND
296
(pp. 288-304)
It is necessary to evaluate the number of publications produced by each country, as well as the number of
citations received and the average number of citations per article. The total number of citations is represented by
the height of the blue bars of the publications. The width of the blue bars shows the average citation per docu-
ment. The normalization of total citations is shown by the red circles. The United States leads the world in both
the number of publications and total citations, with Canada second and China third. In Sharma’s Clean Water and
Sanitation During the 2011-2020 study, China ranked first in total publications, while the United States ranked
second. In terms of the average number of citations per document, the United Kingdom and Australia ranked first
and second, respectively (Sharma, 2021). In terms of the average number of citations per publication, papers by
authors from Madagascar and Slovakia were the most cited overall. Publications originating from Turkey saw the
largest decreases. With a total of 44 publications, Turkey ranked 5th on the list of publications. However, the number
of citations was only as high as Slovakia, which has only one publication, and the average number of citations per
publication was the lowest of all the countries on this list.
Figure 8
Trend Topic on Hygiene Education
The calculations of the trend topics were performed under keywords. The terms “sanitation” and “access to
care” have not been used in conjunction for years. Several terms such as “dental hygiene students,” “oral hygiene,”
and “education” became more popular after 2017. While the term “dental hygiene education” was the most com-
mon, other keywords such as “dental hygiene” “allied dental education” and “hand hygiene” were also common.
297
(pp. 288-304)
Figure 9
Clusters by Hygiene Education Documents Coupling
When the publications were classified according to the keywords used, the studies that included the terms
“oral hygiene,”“hand hygiene,” and “sanitation” ranked lowest in terms of centrality but had the greatest impact. The
impact of papers containing the terms “dental hygiene education,” “dental hygiene students,” and “dental hygien-
ists” is slightly lower than normal, but the centrality is close to average. The centrality is above average, and the
impact value is below average for the keywords “dental education,” “dental hygiene education,” and “interprofes-
sional education.” There was no cluster with a high impact value or centrality in the clustering. The frequency of
keywords directly related to science education was not high enough to be associated with the themes formed.
Figure 10
Clusters by Hygiene Education Studies Themes
298
(pp. 288-304)
A cluster with low centrality and density, identified as a declining theme in the theme analysis based on the
keyword, was not formed. The cluster that can be evaluated as a niche topic area includes the topics of competen-
cies, competency-based education and professionalism, and access to care. These clusters are more isolated than
other clusters. In classifying the basic topics, 3 clusters were formed. These include the topics of public health, health
promotion, and patient education (González et al., 2020). The topics of hand hygiene, sanitation, and education
are above average in centrality and just below average in density. The other cluster includes the topics of dental
hygiene, dental hygiene, and dental education and has the highest centrality and at the same time the lowest
density. In the cluster with high centrality and density, which is called the motor theme, 1 cluster is formed and
includes the keywords oral hygiene, oral health, and dental hygienist. When viewed through the lens of science
education, hygiene education studies do not dominate the conversation about “science education.” Although hy-
giene and health education are considered closely related to the topic of science education (Arnold, 2018; Zeyer,
2012), collaborative research should be more prominent.
Figure 11
Co-occurrences network on Hygiene Education
The Co-Occurrences network is based on the use of terms in conjunction with each other. The size of the
circle indicates how frequently it is used. The thickness of the connecting lines indicates how often they are used
in conjunction with each other. As a result of the cluster analysis performed based on the results of the study, two
groups were formed. In the large group, the color red represents the epicenter of oral hygiene education. The
terms hygiene and sanitation are found in the blue category of keywords. It can be concluded that the majority of
the studies were conducted in the context of oral hygiene education. This is supported by the fact that the most
widely read journals are also included in this category.
299
(pp. 288-304)
Figure 12
Co-citations Network
The analysis of the network of co-citations is based on common publications that studies share in reference
lists. The thickness of the lines expresses the frequency of shared citations. It consists of 3 different clusters. The
first cluster “A discourse on dental hygiene education in Canada” (Kanji et al., 2011) and “Educational and Career
Pathways of Dental Hygienists: Comparing Graduates of Associate and Baccalaureate Degree Programs” (Rowe et
al., 2008) are cited. been a group. There are 7 publications in the second group. Most notably, these include the
publications “Hygiene in the Home: Relating bugs and behavior” (V. Curtis et al., 2003) and “Planned, motivated
and habitual hygiene: a review of eleven countries” (V. A. Curtis et al., 2009). In the third cluster, “The impact of
a comprehensive handwashing program on absenteeism in elementary schools” and “What factors need to be
considered to help dental hygienists help their patients quit smoking?” (Freeman et al., 2012).
Conclusions
This article contains bibliometric reviews of hygiene education articles published between 2012 and 2021.
This collection contains a total of 2015 sources and 503 publications. An average of 5.02 publications are pub-
lished annually. The studies were authored by 1973 individuals. Dental hygiene-related sources occupy the top
four spots on the list of the ten most influential sources based on the number of articles. The Journal of Dental
Education ranks first in terms of the total number of publications related to hygiene education. While the studies
are all relevant to hygiene education in some way, the details of the study cover a wide range of topics. Looking
at collaboration among countries, two groups emerge. The United States made the largest contribution to the
first group. Norway and Uganda are included in the second group. The United States leads the world in both
publication volume and total citations, followed by Canada and China.
When publications were grouped by keywords used, articles using the terms “oral hygiene,”“hand hygiene,”
and “sanitation” had the least centrality but the greatest influence. The influence of articles containing the terms
“dental hygiene education,” “dental hygiene students,” and “dental hygienists” was slightly lower than average,
but their centrality was close to average. Dental education, dental hygiene education, and interprofessional
education are all highly relevant but have a poor effect value. The frequency of keywords related to science
education was insufficient to correlate with the themes developed.
300
(pp. 288-304)
The cluster that can be analyzed in the subject area collects information on competencies, competency-
based education and professionalism, and access to care. These clusters are less interrelated than other clusters.
Three clusters were identified when categorizing the Fundamentals theme. These are public health issues, health
promotion, and patient education. The themes of hand cleanliness, hygiene, and education are more central
than normal and slightly less dense than typical. The other cluster, which includes the topics of dental hygiene,
dental hygiene, and dental education, has the highest centrality and also the lowest density. In the cluster with
the highest centrality and density, a cluster is formed, which is called a motor theme and includes the terms oral
hygiene, oral health, and dental hygienist. As a result of the cluster analysis conducted in response to the results
of the study, two categories were formed. The color red represents the epicenter of oral hygiene instruction in
the large group. The terms hygiene and sanitation are included in the blue keyword category. This indicates
that most of the studies were related to oral hygiene instruction. This is also confirmed by the fact that the
most widely read journals also fall into this category. Research on hygiene education has not dominated the
discourse on “science education” when viewed from the perspective of science education. While hygiene and
health education are considered closely related to science education, joint research should be more common.
Like previous bibliometric studies, this one provided insight into online hygiene education, previewed future
research, and suggested opportunities for collaboration through analysis of historical study data. A systematic
literature review of the relationship between hygiene education and science education can be conducted.
The main limitation of the study is that only studies that were indexed in the Scopus database were included.
There may have been studies that made a significant contribution to the topic of hygiene education but were
not indexed by Scopus; therefore, these papers are not available. On the other hand, analyzes were performed
using keywords chosen by the authors. If you substitute different terms formed in a larger context, you may
get a variety of results. When conducting bibliometric studies on this topic, researchers can conduct a more
comprehensive study by combining data from numerous databases. Future researchers are advised to consider
the above groupings when selecting study topics. By relating their study topics to other topics, authors increase
the visibility of their work in the field.
References
Agarwal, A., Durairajanayagam, D., Tatagari, S., Esteves, S. C., Harlev, A., Henkel, R., Roychoudhury, S., Homa, S., Puchalt, N. G.,
Ramasamy, R., Majzoub, A., Dao Ly, K., Tvrda, E., Assidi, M., Kesari, K., Sharma, R., Banihani, S., Ko, E., Abu-Elmagd, M., …
Bashiri, A. (2016). Bibliometrics: Tracking research impact by selecting the appropriate metrics. Asian Journal of Andrology,
18(2), 296–309. https://doi.org/10.4103/1008-682X.171582
Aria, M., & Cuccurullo, C. (2017). Bibliometrix: An R-tool for comprehensive science mapping analysis. Journal of Informetrics,
11(4), 959–975. https://doi.org/10.1016/j.joi.2017.08.007
Arnold, J. C. (2018). An integrated model of decision-making in health contexts: The role of science education in health
education. International Journal of Science Education, 40(5), 519–537. https://doi.org/10.1080/09500693.2018.1434721
Biran, A., Schmidt, W.-P., Varadharajan, K. S., Rajaraman, D., Kumar, R., Greenland, K., Gopalan, B., Aunger, R., & Curtis, V. (2014).
Effect of a behaviour-change intervention on handwashing with soap in India (SuperAmma): A cluster-randomised trial.
The Lancet Global Health, 2(3), Article e145–e154. https://doi.org/10.1016/S2214-109X(13)70160-8
Birch, D. A., Goekler, S., Auld, M. E., Lohrmann, D. K., & Lyde, A. (2019). Quality assurance in teaching K–12 health education:
Paving a new path forward. Health Promotion Practice, 20(6), 845–857. https://doi.org/10.1177/1524839919868167
Callon, M., Courtial, J. P., & Laville, F. (1991). Co-word analysis as a tool for describing the network of interactions between
basic and technological research: The case of polymer chemsitry. Scientometrics, 22(1), 155–205. https://doi.org/10.1007/
BF02019280
Caruso, B. A., Freeman, M. C., Garn, J. V., Dreibelbis, R., Saboori, S., Muga, R., & Rheingans, R. (2014). Assessing the impact of
a school-based latrine cleaning and handwashing program on pupil absence in Nyanza Province, Kenya: A cluster-
randomized trial. Tropical Medicine & International Health: TM & IH, 19(10), 1185–1197. https://doi.org/10.1111/tmi.12360
Čavojová, V., Šrol, J., & Ballová Mikušková, E. (2022). How scientific reasoning correlates with health-related beliefs and behaviors
during the COVID-19 pandemic? Journal of Health Psychology, 27(3), 534–547. https://doi.org/10.1177/1359105320962266
Cheng, V. C. C., Chau, P. H., Lee, W. M., Ho, S. K. Y., Lee, D. W. Y., So, S. Y. C., Wong, S. C. Y., Tai, J. W. M., & Yuen, K. Y. (2015). Hand-
touch contact assessment of high-touch and mutual-touch surfaces among healthcare workers, patients, and visitors.
Journal of Hospital Infection, 90(3), 220–225. https://doi.org/10.1016/j.jhin.2014.12.024
Clasen, T., Schmidt, W.-P., Rabie, T., Roberts, I., & Cairncross, S. (2007). Interventions to improve water quality for preventing
diarrhoea: systematic review and meta-analysis. BMJ, 334(7597), 782. https://doi.org/10.1136/bmj.39118.489931.BE
Curtis, V. A., Danquah, L. O., & Aunger, R. V. (2009). Planned, motivated and habitual hygiene behaviour: An eleven country
review. Health Education Research, 24(4), 655–673. https://doi.org/10.1093/her/cyp002
Curtis, V., Biran, A., Deverell, K., Hughes, C., Bellamy, K., & Drasar, B. (2003). Hygiene in the home: Relating bugs and behaviour.
Social Science and Medicine, 57(4), 657–672. https://doi.org/10.1016/S0277-9536(02)00409-4
301
(pp. 288-304)
Darvesh, N., Das, J. K., Vaivada, T., Gaffey, M. F., Rasanathan, K., & Bhutta, Z. A. (2017). Water, sanitation and hygiene interventions
for acute childhood diarrhea: A systematic review to provide estimates for the Lives Saved Tool. BMC Public Health,
17(Suppl 4). https://doi.org/10.1186/s12889-017-4746-1
Diouf, K., Tabatabai, P., Rudolph, J., & Marx, M. (2014). Diarrhoea prevalence in children under five years of age in rural
Burundi: An assessment of social and behavioural factors at the household level. Global Health Action, 7(1).
https://doi.org/10.3402/gha.v7.24895
Ejemot, R. I., Ehiri, J. E., Meremikwu, M. M., & Critchley, J. A. (2008). Hand washing for preventing diarrhoea. Cochrane Database
of Systematic Reviews, 1. https://doi.org/10.1002/14651858.CD004265.pub2
Eley, C., Weston-Price, S., Young, V., Hoekstra, B., Gadhia, T., Muirhead, V., Robinson, L., Pine, C., & McNulty, C. (2020). Using oral
hygiene education in schools to tackle child tooth decay: a mixed methods study with children and teachers in England.
Journal of Biological Education, 54(4), 381–395. https://doi.org/10.1080/00219266.2019.1585380
Freeman, T., Roche, A. M., Williamson, P., & Pidd, K. (2012). What factors need to be addressed to support dental hygienists to
assist their patients to quit smoking? Nicotine and Tobacco Research, 14(9), 1040–1047. https://doi.org/10.1093/ntr/ntr329
Fritz, S. A., Hogan, P. G., Hayek, G., Eisenstein, K. A., Rodriguez, M., Epplin, E. K., Garbutt, J., & Fraser, V. J. (2012). Household
versus individual approaches to eradication of community-associated Staphylococcus aureus in children: A randomized
trial. Clinical Infectious Diseases, 54(6), 743–751. https://doi.org/10.1093/cid/cir919
Garin, B., Gouali, M., Wouafo, M., Perchec, A. M., Thu, P. M., Ravaonindrina, N., Urbès, F., Gay, M., Diawara, A., Leclercq, A., Rocourt,
J., & Pouillot, R. (2012). Prevalence, quantification, and antimicrobial resistance of Campylobacter spp. on chicken neck-
skins at points of slaughter in 5 major cities located on 4 continents. International Journal of Food Microbiology, 157(1),
102–107. https://doi.org/10.1016/j.ijfoodmicro.2012.04.020
Głąbska, D., Skolmowska, D., & Guzek, D. (2020). Population-based study of the influence of the COVID-19 pandemic on hand
hygiene behaviors — Polish adolescents’ COVID-19 experience (PLACE-19) study. Sustainability, 12(12), 4930. https://
doi.org/10.3390/su12124930
González, A. J., Fortunato, M. S., Korol, S. E., & Gallego, A. (2020). Nitrate contamination of bottled water: Description of
a Project-based learning experience. Interdisciplinary Journal of Environmental and Science Education, 16(3), e2218.
https://doi.org/10.29333/ijese/8337
Gokhale, A., Mulay, P., Pramod, D., & Kulkarni, R. (2020). A Bibliometric analysis of digital image forensics. Science and Technology
Libraries, 00(00), 1–18. https://doi.org/10.1080/0194262X.2020.1714529
Grover, E., Hossain, M. K., Uddin, S., Venkatesh, M., Ram, P. K., & Dreibelbis, R. (2018). Comparing the behavioural impact of
a nudge-based handwashing intervention to high-intensity hygiene education: a cluster-randomised trial in rural
Bangladesh. Tropical Medicine and International Health, 23(1), 10–25. https://doi.org/10.1111/tmi.12999
Guinan, M., McGuckin, M., & Ali, Y. (2002). The effect of a comprehensive handwashing program on absenteeism in elementary
schools. American Journal of Infection Control, 30(4), 217–220. https://doi.org/10.1067/mic.2002.120366
Gyorkos, T. W., Maheu-Giroux, M., Blouin, B., & Casapia, M. (2013). Impact of health education on soil-transmitted helminth
infections in schoolchildren of the Peruvian Amazon: A cluster-randomized controlled trial. PLoS Neglected Tropical
Diseases, 7(9). https://doi.org/10.1371/journal.pntd.0002397
Hamilton, K., Bonham, M., Bishara, J., Kroon, J., & Schwarzer, R. (2017). Translating dental flossing intentions into behavior: A
longitudinal investigation of the mediating effect of planning and self-efficacy on young adults. International Journal
of Behavioral Medicine, 24(3), 420–427. https://doi.org/10.1007/s12529-016-9605-4
Haustein, S., & Larivière, V. (2015). The use of bibliometrics for assessing research: Possibilities, limitations and adverse
effects. In: Welpe, I., Wollersheim, J., Ringelhan, S., Osterloh, M. (Eds), Incentives and performance. Springer.
https://doi.org/10.1007/978-3-319-09785-5_8
Hogan, P. G., Parrish, K. L., Mork, R. L., Boyle, M. G., Muenks, C. E., Thompson, R. M., Morelli, J. J., Sullivan, M. L., Hunstad, D. A.,
Wardenburg, J. B., Rzhetsky, A., Gehlert, S. J., Burnham, C. A. D., & Fritz, S. A. (2021). HoMe2 study: Household versus
personalized decolonization in households of children with methicillin-resistant staphylococcus aureus skin and soft
tissue infection—a randomized clinical trial. Clinical Infectious Diseases, 73(11), E4568–E4577. https://doi.org/10.1093/
cid/ciaa752
Huang, S. S., Singh, R., McKinnell, J. A., Park, S., Gombosev, A., Eells, S. J., Gillen, D. L., Kim, D., Rashid, S., Macias-Gil, R., Bolaris,
M. A., Tjoa, T., Cao, C., Hong, S. S., Lequieu, J., Cui, E., Chang, J., He, J., Evans, K., … Miller, L. G. (2019). Decolonization
to reduce postdischarge infection risk among MRSA carriers. New England Journal of Medicine, 380(7), 638–650.
https://doi.org/10.1056/nejmoa1716771
Huda, T. M. N., Unicomb, L., Johnston, R. B., Halder, A. K., Yushuf Sharker, M. A., & Luby, S. P. (2012). Interim evaluation of a
large-scale sanitation, hygiene and water improvement programme on childhood diarrhea and respiratory disease in
rural Bangladesh. Social Science & Medicine, 75(4), 604–611. https://doi.org/10.1016/j.socscimed.2011.10.042
Jatmika, S. E. D., & Maulana, M. (2018). Dental and oral health education for elementary school students through patient
hygiene performance index indicator. International Journal of Evaluation and Research in Education (IJERE), 7(4), 259.
https://doi.org/10.11591/ijere.v7i4.14856
Kanji, Z., Sunell, S., Boschma, G., Imai, P., & Craig, B. J. (2011). A discourse on dental hygiene education in Canada. International
Journal of Dental Hygiene, 9(4), 242–249. https://doi.org/10.1111/j.1601-5037.2010.00495.x
Kelcíkova, S., Skodova, Z., & Straka, S. (2012). Effectiveness of hand hygiene education in a basic nursing school curricula.
Public Health Nursing, 29(2), 152–159. https://doi.org/10.1111/j.1525-1446.2011.00985.x
302
(pp. 288-304)
Kessler, M. M. (1963). Bibliographic coupling between scientific papers’ received. Journal of the Association for Information
Science and Technology, 14(1), 10–25. https://doi.org/10.1002/asi.5090140103
Kressin, N. R., Iii, A. S., Bossé, R., & Garcia, R. I. (1999). Personality traits and oral self-care behaviors: Longitudinal findings from
the normative aging study. Psychology & Health, 14(1), 71–85. https://doi.org/10.1080/08870449908407315
Kretschmer, H., & Kretschmer, T. (2007). Application of a new centrality measure for social network analysis to
bibliometric and webometric data. 2006 1st International Conference on Digital Information Management, 199–204.
https://doi.org/10.1109/ICDIM.2007.369353
Lazzarotto, T., Blázquez-Gamero, D., Delforge, M. L., Foulon, I., Luck, S., Modrow, S., & Leruez-Ville, M. (2020). Congenital
cytomegalovirus infection: A narrative review of the issues in screening and management from a panel of European
experts. Frontiers in Pediatrics, 8(January). https://doi.org/10.3389/fped.2020.00013
Lounis, M. (2020). Promoting school health education: A lesson from the COVID-19 pandemic. Contemporary Mathematics
and Science Education, 1(2), ep20009. https://doi.org/10.30935/conmaths/8579
Marcenes, W., Kassebaum, N. J., Bernabé, E., Flaxman, A., Naghavi, M., Lopez, A., & Murray, C. J. L. (2013). Global
burden of oral conditions in 1990-2010: A systematic analysis. Journal of Dental Research, 92(7), 592–597.
https://doi.org/10.1177/0022034513490168
McInnes, M. D. F., Moher, D., Thombs, B. D., McGrath, T. A., Bossuyt, P. M., Clifford, T., Cohen, J. F., Deeks, J. J., Gatsonis, C.,
Hooft, L., Hunt, H. A., Hyde, C. J., Korevaar, D. A., Leeflang, M. M. G., Macaskill, P., Reitsma, J. B., Rodin, R., Rutjes, A. W. S.,
Salameh, J. P., … Willis, B. H. (2018). Preferred reporting items for a systematic review and meta-analysis of diagnostic
test accuracy studies the PRISMA-DTA statement. JAMA - Journal of the American Medical Association, 319(4), 388–396.
https://doi.org/10.1001/jama.2017.19163
Montresor, A., Mupfasoni, D., Mikhailov, A., Mwinzi, P., Lucianez, A., Jamsheed, M., Gasimov, E., Warusavithana, S., Yajima, A.,
Bisoffi, Z., Buonfrate, D., Steinmann, P., Utzinger, J., Levecke, B., Vlaminck, J., Coolsid, P., Vercruysse, J., Cringoli, G., Rinaldi,
L., … Gyorkos, T. W. (2020). The global progress of soil-transmitted helminthiases control in 2020 and world health
organization targets for 2030. PLoS Neglected Tropical Diseases, 14(8), 1–17. https://doi.org/10.1371/journal.pntd.0008505
Morrish, D., & Neesam, M. (2021). Trends in coverage of hygiene and disease prevention topics across national
curriculum frameworks for primary science, physical education, and health. Prospects, 51(1–3), 363–381.
https://doi.org/10.1007/s11125-020-09525-7
Mueller, M., Schorle, S., Vach, K., Hartmann, A., Zeeck, A., & Schlueter, N. (2022). Relationship between dental
experiences, oral hygiene education and self-reported oral hygiene behaviour. Plos One, 17(2), e0264306.
https://doi.org/10.1371/journal.pone.0264306
Nordin, A., & Sundberg, D. (2016). Travelling concepts in national curriculum policy-making: The example of competencies.
European Educational Research Journal, 15(3), 314–328. https://doi.org/10.1177/1474904116641697
Noyons, E. (2001). Bibliometric mapping of science in a polic y contex t. Scientometrics, 50(1), 83–98.
https://doi.org/10.1023/a:1005694202977
Rashid, A. (2021). Institutional shareholding and corporate social responsibility reporting: Evidence from Bangladesh. Journal
of Asia Business Studies, 15(1), 153–173. https://doi.org/10.1108/JABS-09-2019-0285
Rowe, D. J., Massoumi, N., Hyde, S., & Weintraub, J. A. (2008). Educational and career pathways of dental hygienists:
Comparing graduates of associate and baccalaureate degree programs. Journal of Dental Education, 72(4), 397–407.
https://doi.org/10.1002/j.0022-0337.2008.72.4.tb04505.x
Scheerman, J. F. M., van Loveren, C., van Meijel, B., Dusseldorp, E., Wartewig, E., Verrips, G. H. W., Ket, J. C. F., & van Empelen, P.
(2016). Psychosocial correlates of oral hygiene behaviour in people aged 9 to 19 - a systematic review with meta-analysis.
Community Dentistry and Oral Epidemiology, 44(4), 331–341. https://doi.org/10.1111/cdoe.12224
Schmidlin, T., Hürlimann, E., Silué, K. D., Yapi, R. B., Houngbedji, C., Kouadio, B. A., Acka-Douabélé, C. A., Kouassi, D., Ouattara,
M., Zouzou, F., Bonfoh, B., N’Goran, E. K., Utzinger, J., & Raso, G. (2013). Effects of hygiene and defecation behavior on
helminths and intestinal protozoa infections in Taabo, Côte d’Ivoire. PLoS ONE, 8(6), 1–12. https://doi.org/10.1371/
journal.pone.0065722
Sharma, A. K. (2021). A scientometric analysis of Indian publication output in clean water and sanitation during 2011-2020.
Library Philosophy and Practice, 2021(April).
Szilágyi, L., Haidegger, T., Lehotsky, Á., Nagy, M., Csonka, E. A., Sun, X., Ooi, K. L., & Fisher, D. (2013). A large-scale
assessment of hand hygiene quality and the effectiveness of the “WHO 6-steps.” BMC Infectious Diseases, 13(1).
https://doi.org/10.1186/1471-2334-13-249
Tadin, A., Poljak Guberina, R., Domazet, J., & Gavic, L. (2022). Oral hygiene practices and oral health knowledge among students
in Split, Croatia. Healthcare, 10(2), 406. https://doi.org/10.3390/healthcare10020406
Tan, S. L., Bakar, F. A., Abdul Karim, M. S., Lee, H. Y., & Mahyudin, N. A. (2013). Hand hygiene knowledge, attitudes, and practices
among food handlers at primary schools in Hulu Langat district, Selangor (Malaysia). Food Control, 34(2), 428–435.
https://doi.org/10.1016/j.foodcont.2013.04.045
Thanuskodi, S. (2010). Journal of Social Sciences: A Bibliometric Study. Journal of Social Sciences, 24(2), 77–80.
https://doi.org/10.1080/09718923.2010.11892847
Welch, V., Petticrew, M., Petkovic, J., Moher, D., Waters, E., White, H., & Tugwell, P. (2016). Extending the PRISMA statement to
equity-focused systematic reviews (PRISMA-E 2012): Explanation and elaboration. Journal of Clinical Epidemiology, 70,
68–89. https://doi.org/10.1016/j.jclinepi.2015.09.001
WHO. (2017). Hygiene. https://www.afro.who.int/health-topics/hygiene
303
(pp. 288-304)
Wittberg, D. M., Aragie, S., Tadesse, W., Melo, J. S., Aiemjoy, K., Chanyalew, M., Emerson, P. M., Freeman, M. C., Nash, S.
D., Kelly Callahan, E., Tadesse, Z., Zerihun, M., Porco, T. C., Lietman, T. M., & Keenan, J. D. (2021). WASH Upgrades
for Health in Amhara (WUHA): Study protocol for a cluster-randomised trial in Ethiopia. BMJ Open, 11(2), 1–8.
https://doi.org/10.1136/bmjopen-2020-039529
Zammarchi, G., & Conversano, C. (2021). Application of eye tracking technology in medicine: A bibliometric analysis. Vision,
5(4), 56. https://doi.org/10.3390/vision5040056
Zeyer, A. (2012). A win-win situation for health and science education: Seeing through the lens of a new framework
model of health literacy. In: Zeyer, A., & Kyburz-Graber, R. (Eds), Science | Environment | Health (pp. 147-173). Springer.
https://doi.org/10.1007/978-90-481-3949-1_9
Cite as: Usak, M., Sinan, S., & Sinan, O. (2022). Science maps and bibliometric analysis on hygiene education during 2012-2021.
Muhammet Usak PhD in Science Education, Associate Professor, Leading Research

(Corresponding author) Fellow at Scientific and Educational Center of Pedagogical
Researches, Kazan (Volga region) Federal University, Kazan, Russian
Federation.
E-mail: musaktr@gmail.com
ORCID ID: https://orcid.org/0000-0002-6537-9993
Selma Sinan PhD, Researcher, Department of Molecular Biosciences, College of
Natural Sciences, The University of Texas at Austin. Austin, Texas,
78712, USA.
E-mail: selmasinan@utexas.edu
Olcay Sinan PhD, Researcher, Department of Molecular Biosciences, College of
Natural Sciences, The University of Texas at Austin. Austin, Texas,
78712, USA.
E-mail: olcaysinan@utexas.edu
304
THE MEASURING INSTRUMENT

OF PRIMITIVE PHYSICS
PROBLEM FOR UPPER- ISSN 1648-3898 /Print/
SECONDARY SCHOOL ISSN 2538-7138 /Online/
STUDENTS: COMPILATION AND

EXPLORATION
Hongjun Xing, Abstract. Problem solving is considered

Yanfang Zhai, one of the most critical competencies for
Sisi Han, life. However, many students may perform
Yuping Zhao,
well in school yet cannot transfer the skills
Wenhui Gong,
Yuting Wang, they learned in school to solve real-world
Jing Han, problems, especially in science education.

Qiaoyi Liu This study discusses the characteristics of
the problem from the physics education
perspective, based on Problem Representa-
tion Theory, and proposes a new type of
Introduction problem: The Primitive Physics Problem.
Moreover, the significance and contribution
“All life is problem solving,” as Popper (1999) said. It is widely regarded
that problem solving is the most significant cognitive activity in daily life and in promoting students’ physics problem-
career development. There are countless problems to be solved in everyone’s solving ability, both theoretical and empiri-
life, such as, how to divide one’s time into various types of work, whether to cal, are described. Then the study develops
go to the gym or read on one’s own for relaxing. All situations share a famil-
a set of instruments that was tested on 811
iar pattern: there are a target state and a current state, and one needs the
transition from the current to the desired state. Succeeding in the transition students in 12th grade from 10 upper-
is called solving a problem, which often gets meaningful rewards, such as secondary schools in China. The results
self-worth, valuable feeling. Solving real-life problems has two significances: showed that the new instrument had good
intellectual and social-cultural values (Jonassen, 2000).
reliability and validity. The promising ap-
Since the capability of problem solving is vital in a world full of uncer-
tainty, it becomes an ambitious goal for educators to strive for. As Gagne plication in the instruction of the Primitive
(1970) stated, the core goal of education is to cultivate individuals’ thinking Physics Problem and in measuring upper-
abilities, to train them to use the power of rationality, and to become better secondary school students’ problem-solv-
problem solvers. As a higher-order cognitive skill, the capability of problem-
ing ability of the instrument is discussed.
solving is considered the most critical learning outcome for life by most
psychologists and educators (Zoller et al., 1995; Zoller, 2000). Nowadays, it Keywords: primitive physics problem;
has been regarded as a central objective of education in many countries. problem solving; problem representation
For example, countries participating in PISA (the Program for International theory; assessment instrument.
Student Assessment) focus on a 15-year-old’s ability to use knowledge and
skills to meet real-life challenges and problem-solving competencies.
Hongjun Xing, Yanfang Zhai, Sisi Han,
The series of studies are helpful to clarify how to cultivate individuals’ Yuping Zhao, Wenhui Gong,
capacities to deal with challenges in life for future learning, effective partici- Yuting Wang
pation in society, and individual development (Lesh & Zawojewski, 2007). Capital Normal University, China
Jing Han, Qiaoyi Liu
Research about developing problem-solving ability started as early as 50 The Ohio State University, USA
years ago (Garrett, 1986), and it involves many topics in education, such as
305
(pp. 305-324) ISSN 2538–7138 /Online/
instruction, cooperative learning, experienced and inexperienced problem-solvers. However, it was found that
students with high achievements in school are often unable to apply what has been learned in class to daily life,
especially for science. Moreover, many higher-level students’ problem-solving skills are inadequate, and the efforts
to improve their ability to solve problems are not enough (Reif, 2010; Redish et al., 2006; Van Heuvelen, 1991).
Thus, researchers need more effort, multi-angle, and suggestions for promoting students’ problem-solving ability.
In this study, a new type of problem named primitive physics problem is introduced, which places the focus
on physical facts and phenomena, the basis of physical sciences, with little abstraction, unlike typical textbook
problems. In doing so, primitive physics problems require students to be creative in constructing the abstraction
themselves, along with combining real-life scenarios with knowledge they learned in class. Hence, primitive phys-
ics problems can be an important tool to promote students’ critical thinking and problem-solving skills. Here, the
characteristics of primitive problems are examined from the physics perspective and the problems’ characteristics
and the critical representations of problem-solving are analyzed. The significance and contribution of primitive
physics problem are explored in promoting students’ problem-solving ability, both theoretically and empirically. In
theory, the proposed problem’s unique value and essential difference from other types of problems are discussed.
Empirically, a set of instruments is developed with good reliability and validity to obtain the preliminary empirical
evidence to support the idea that this type of problem promotes students’ development of problem-solving skills.
Literature Review
Two kinds of factors influence problem solving, including external factors involving the essential characteristics
of a problem and internal factors concerning the problem solver, like personal pre-knowledge, pre-understanding,
and primary strategies. According to this framework, the reasons limiting students’ problem-solving performance
in education are: firstly, as an external factor, the types of problems students encounter in school and life are very
different; secondly, as an internal factor, students have difficulty in improving from novice to expert in different
problem contexts. As follows, the characteristics and types of the problem in physics education and then the
characteristics of the cognitive representation of physics problem solving are summarized.
Problems in problem solving
Researchers proposed many different definitions with various perspectives, such as psychology, epistemol-
ogy, methodology, logic, information theory, artificial intelligence, and so on. Some researchers define a problem
as “an obstacle or a kind of difficulty”, “a situation that people have no idea what to do to get the ideal goal when
he or she wants to get it” (Newell & Simon, 1972), some are “a gap between a given state and a goal state” (Hayes,
1989; Wheatley, 1984), some are “a situation which people do not know how to deal with” (Reys et al., 1998). From
these definitions, it can be concluded that a problem is a dynamic system consisting of expectations (goal pursuit),
givens and obstacles, and a situation in which there is no direct, obvious method to follow. It has these character-
istics: “undesirability,”“concern,”“difficulty,” and “solvability”. Based on the understanding of the characteristics of a
problem, problems are divided into different categories. Among them, well-structured and ill-structured problems
are the most common classification, which also distinguishes precisely the problem students face in school and life.
Well-structured and ill-structured problems
Between well-structured and ill-structured problems, the most significant difference is shown on the con-
tinuum of structuredness (Jonassen, 1997, 2000; Newell & Simon, 1972; Voss & Post, 1988). It is worth noting that
structuredness represents a continuum rather than a dichotomy variable (Jonassen, 2010). The problems students
encountered in formal educational contexts are mostly well-structured, typically found in textbooks and tests.
Well-constructed problems consist of “a well-defined given state, a known goal state, and a constrained set of
logical operators” (Greeno, 1980), in which there are all the pieces of information needed to solve the problem.
For well-structured problems, a limited number of rules and principles must be applied, organized in a predictable
and prescriptive way, with correct and convergent answers, and there is a preferred, prescribed solution process
(Wood, 1983). On the other hand, the problems encountered in life and career are mostly ill-structured. Different
from the well-structured problems, one or more essential elements needed to solve the problem are unknown in
ill-structured problems (Wood, 1983); there are multiple solutions, solution paths, or no solutions at all (Kitchner,
306
ISSN 1648–3898 /Print/ THE MEASURING INSTRUMENT OF PRIMITIVE PHYSICS PROBLEM FOR UPPER-SECONDARY
ISSN 2538–7138 /Online/ (pp. 305-324)
1983). Because of multiple criteria for evaluating solutions to ill-defined problems, learners are not sure what con-
cepts, rules, and principles are needed and how they are organized, and learners usually need to make judgments
and express their personal opinions or beliefs about the problem (Meacham & Emont, 1989).
Whether the strategies to solve the two types of problems are the same or not and whether they can be
transformed, the research results are inconsistent. Some researchers believe that the strategies to solve problems
are transferable between well-structured and ill-structured problems from an information-processing perspective,
which is based on the assumption that the process used to solve the two kinds of problems is the same (Simon,
1978). However, some researchers find different ways for people to solve well-defined problems and ill-defined
problems. (Cho & Jonassen, 2003; Dunkle et al., 1995; Hong et al.,2003; Jonassen & Kwon, 2001). It seems that well-
structured and ill-structured problem solving involves significantly different cognitive processes (Jonassen, 2010).
Types of problems in physics
In physics education, three types of problems have been discussed: Practice problems or exercise, Context-
rich problems, and Real problems.
Practice problems or exercise. It is suggested that practice problems are the typical textbook problems,
which are also called standard problems by Maloney (1994). The essential characteristic of the type of problem is
the idealized objects, which are usually not related to realities. In this kind of problem, the gap between goal states
and initial states is eliminated. This kind of problem refers to a thorough knowledge of facts and procedures but
excludes executing. When the textbook problems are to be solved, students are required to recall, comprehend,
and apply specific rules or principles (Henderson, 2002), make sense of the necessary process to reach the goal,
and execute the procedures like finding equations and plugging data. Besides, students do not experience the
fruitfulness of possessing problem-solving frameworks.
Context-rich problems: They are also called “case study problems.” They are defined as “stories that include
a reason for calculating specific quantities about real objects or events” by Heller and Hollabaugh (1992). They are
more realistic and related to students’ life compared with practice problems. Context-rich problems are designed
to improve students’ problem-solving skills. When students solve this kind of problem, the typical approach is no
longer feasible. The common characteristics of these problems include: (a) making reasonable assumptions may
simplify the problems; (b) not including complete information for solving a problem or containing redundant
information; (c) variables are not always clear and definite.
Real or true problems: The definition of them is the same as mentioned above, referring to the general
problem.
Based on the primary problem framework, along a continuum from well-structured to ill-structured, it is
found that practice problems or exercises are well-defined. By contrast, real problems are most ill-defined, and
the context-rich problems fall between. As many researchers state it is hard to find the well-defined problems that
contribute to fostering students’ conceptual capacity (Byun & Lee, 2014; Kim & Pak, 2002). When students learn to
solve well-defined problems, only their simple strategies are developed, which hinders students’ reflection upon
the concepts and principles, leading to inadequate conceptual development (Pulgar et al., 2021). Compared with
textbook problems, context-rich problems set the context as a significant part of a problem and an essential part
of its solution (Wood, 1983). It also creates a bridge linking the modeled and the authentic world. The outstanding
characteristic of this kind of problem is authenticity, which is rooted in the real world. However, it is not always ef-
fective in developing students’ ability to solve real-world problems characterized by complex, ill-defined, and vague
methods, and so on. For instance, when well-defined problems are embedded in specific contexts, the contexts
have little meaning to students. The problem-solving process that students experience is consistent with solving
exercises and well-structured problems. Similarly, when extremely ill-defined problems are embedded in specific
contexts, the problems are so context-dependent that the problems become no sense without the context. Thus, it
prevents students from developing their problem-solving ability and meaningful transformability from school to life.
In summary, the gap based on problem types became one reason for students’ difficulties in problem-solving
in different contexts. Textbook problems or exercises, a typical representative of well-defined problems, provide
an intermediary for training for students to solve regular problems. Nevertheless, even students who successfully
solve conventional problems are often helpless facing unconventional problems. At the same time, situational
problems lie between well-structured and poorly structured problems and tend to be well structured compared
with real problems. It provides a medium or story background for students to connect with real-world problems
307
(pp. 305-324) ISSN 2538–7138 /Online/
and provides good support and promotion for students to solve problems close to the real world, yet the problem
is not enough to cultivate students’ ability to face the real world. The external factors that affect students’ problem
solving are discussed above. Next, the internal factors influencing students’ problem solving and the characteristics
of problem solvers will be discussed.
Problem representation in problem solving
Previous studies have found that novice students have trouble attaining expert-like skills in problem solving
after traditional instruction. Substantial research has provided several reasons why experts are better problem
solvers than novices (Heller & Hollabaugh, 1992; Larkin et al., 1980; Hsu et al., 2004; Wieman, 2015). One of the
most important reasons is problem representations (Chi, 2018), which is the key to problem solving. Experts create
more rich, more integrated mental representations than novices do (Björklund, 2013).
In the cognitive psychology domain, problem representation is generally recognized as the cognitive structure
of the problem constructed by the problem solver to understand the problem and the explanation of the initial
conditions and constraints of the perceived problem. In short, problem representation as an internal representa-
tion is a way of recording information in the brain. Once the problem-solvers develop a specific representation, a
particular solution, always a good one, will emerge (Voss, 1988, 1991). Representation of a problem is established
by building a cognitive model of a problem space.
There are three main views on the nature of the representation process. The first one is that external representa-
tion is only the stimulated output of internal consciousness. So, even if many cognitive tasks need to be completed
during the interaction with the environment, all cognitive processing activities can only occur in the internal model
of external representation. Therefore, when the problem-solver is faced with a task that needs to interact with the
environment, he first needs to establish an internal model about the problem presentation and its environment
through the coding process, and then complete the psychological operation of specific content symbols, sub
symbols or other forms in this structured psychological model. Finally, through the processing procedures, the
internal processing products are externalized to the environment. It is a widely accepted view in traditional AI and
other fields of cognitive science (Newell, 1994). The second view emphasizes the role of external representation in
problem solving. The perspective argues that the information of the problem itself and the context of the problem
can be fully perceived by an individual’s perceptual system, which is conducive to completing cognitive tasks.
Because of the highly structured external representation itself, there is no need to establish any internal model.
That is, the essence of the problem can be understood only through external representation. As Gibson (2014)
said, the invariable information can be directly perceived and ensures the completion of cognitive tasks even if
there is no memory, reasoning, thinking, or other psychological processes which need internal representation. In
Gibson’s view, the information of the environment is sufficient to determine the objectives. Therefore, the final
product of perception is not an internal representation of the environment, but the constant information directly
perceived from the environment. The last view is that external representation and internal representation interact
to influence problem solving when one is completing cognitive tasks. Based on ecological psychology, Zhang and
Norman (1994) studied distributed cognitive tasks’ representational properties. They argue that external represen-
tations can be directly perceived and used without the need to be interpreted and formulated explicitly. External
representation constrains the range of cognitive activities through its physical structures to anchor cognitive be-
havior. Moreover, even if the abstract structure of tasks is the same, tasks with or without external representation
are entirely different for problem solvers.
Therefore, a consensus is found on the understanding of these three representations: (1) Problem solving
involves external and internal representation, and they are independent but correlated; (2) In general, the internal
representation of the problem is the key to solving the problem. Wrong or incomplete internal representation can
become the most significant internal obstacle to problem solving; (3) There are two mechanisms for the external
representation of problems to influence problem solving: one is to activate the internal complex cognitive process
and establish the internal psychological model to form the problem space; the other one is to detect the invariable
structures in the external representation directly through the perceptual system and complete the problem solving
without the involvement of the activation and reasoning of some complex mental models in the internal memory
system. Generally, to solve a problem it is necessary to process the information from external representations and
internal representations integrated and dynamically.
Internal representations: A significant number of researchers have focused on the construction of internal
308
ISSN 2538–7138 /Online/ (pp. 305-324)
mental problem representations. It is believed that internal representation is the beginning of problem solving by
most psychologists. Internal representations play a crucial role in many aspects, such as connecting problems to
domain understanding, and transferring problem solving. It is assumed to be the knowledge in memory, in mul-
tiple forms, including propositions, schemas, and neural networks. Briefly defined, it involves the problem solver
extracting information from the problem context, recalling information from the memory system, and encoding
information in working memory to form the psychological schema of the problem structure. The correct and
appropriate internal representation is the key to solving the problem. Establishing an internal representation of
problems depends on the perception of relevant problem information by the perceptual system and the explana-
tion of perceptual information by existing experience, especially the problem examples and theoretical categories,
making problems represented quickly. The situation of the problem is the essential condition for information
representation. The approach, environment, and structure of problem presentation determine what information
and spatiotemporal structure problem solver can be perceived. Furthermore, the knowledge, experience, and per-
sonality of the problem solver are also important factors, which are about an individual’s perceptual interpretation
of stimulus information and organizational construction.
External representations: External representation is an essential and intrinsic component of problem solv-
ing, and without external representation, the problem cannot be established. It is defined as the knowledge and
structure in the environment and includes external rules, constraints, and relations embedded in physical layouts.
The typical forms of external representation are graphs, diagrams of curves, photos, and writing symbols, often
used in cognitive tasks such as problem solving, reasoning, and decision making. Many studies have shown that
these forms make problem information more specific, interpretable, and processable (Chambers & Reisberg, 1985;
Kleinmuntz & Schkade, 1993). External representations facilitate problem solving through symbols of an object,
which represents an individual’s knowledge and the knowledge structure. In this way, complex cognitive process-
ing is promoted (Vekiri, 2002). External representation can help an individual to elaborate a problem, transform its
fuzzy state into clear conditions, ease the cognitive load, and create solutions (Scaife & Rogers, 1996).
Experts and novices in representations: Distinguished with the general aspects of problem solving that psy-
chologists focused on, educational researchers paid more attention to domain-specific problem-solving strategies.
They produced a deep insight into expert-novice studies. There are many studies on physics as a classic established
problem-solving discipline. What makes a distinction between experts and novices lies in the amount of domain-
specific knowledge and how this knowledge is used in these situations (Ericsson et al., 2018; Foster, 2000). Experts
have a large number of physics knowledge, while novices have insufficient knowledge. Experts’ knowledge is highly
interrelated and integrated, while novices’ knowledge structure is loose and incoherent. Experts’ representations
are comprehensive, whereas novices represent problems in a stretched way. Furthermore, experts are good for
retrieval from working memory, while novices are hard to recall unless cued by something (Horn & Blankson, 2012).
Besides differences in physics concepts, experts and novices significantly differ in their representation features.
Experts use in-depth features to classify problems, whereas novices resort to superficial features in order to solve
problems (Chi et al., 1981). The process of solving a problem for experts includes three steps. The first one is the
description stage and transitions of the problem statement into a more domain-specific description of the prob-
lem. Secondly, one uses applicable procedures to find solutions. The last one is evaluating the solution according
to criteria (Reif & Heller, 1982). Specifically, experts spend more time completing the description stage, which is
regarded as a qualitative description based on principles, while novices always skip this step. Qualitative analysis
is served for decision guidance of planning and evaluating (Larkin, 1979), which efficiently promotes the further
solution steps. By contrast, novices lack qualitative physical explanation (Foster, 2000) and rush into directly using
formulas (Larkin, 1979) or stinging together miscellaneous equations, and then their problem solving always ends
at this step. The gap between experts and novices has been widely studied. It is concluded that the obstacle novices
face, compared with experts, includes how to construct a problem representation generated from both internal
and external sources. Primarily, external representation refers to the transition from environmental information
structure, which directly or even decisively influences internal representation.
As reviewed above, external representations function as affordances that constrain cognitive activity, then
training students to interact with those representations, including external and internal, should afford more sig-
nificant cognitive benefits. Furthermore, if students are tuned to attend to the external properties of the problem,
they will be able to acquire the understanding of the problem successfully, which is crucial to solving the problem
and moving on to the next step. However, it depends on problem space, which provides opportunities for students
to develop their problem-solving abilities. Researchers pay attention to theoretical descriptions of problem solving
309
(pp. 305-324) ISSN 2538–7138 /Online/
primarily based on solving textbook problems, which are simplified and idealized situations with narrow problem
space for students to construct; generally, only the internal representation space is included. In many physics’
textbooks, external representations are usually expressed in direct mathematical forms, which makes students dif-
ficult to understand the underlying physical nature concepts (Huffman, 1997). For real-life problems, it is ill-defined,
open-ended, and has multiple possibilities, so students may suffer from a so high cognitive load that they quickly
fail to solve. The problem-solving ability transformation between school and life is hard to eliminate for school
education. For context-rich problems, real scenarios and embedded mathematical forms reduce unnecessary cog-
nitive load and motivate students to solve problems. The context-rich problems provide textbook problems with
a real-world background to bridge the problem-solving gap between the real world and school education. They
are essentially based on the textbook problems, with the situation as the link to the real world, to train students’
problem construction ability. From textbook problems to real-world problems, from well-defined problems to ill-
defined problems, context-rich problems are closer to the textbook problems, compared with the more obvious
difference from real-world problems, and their problem space is also limited. It is important to note that expand-
ing problem space to represent both the external representation and the problem solver’s internal representation
broads the range of distributed cognitive tasks. Moreover, external representation promotes the generation of a
psychological model, which is a crucial feature in the learning process. However, students’ understanding can only
be developed when external representation is consistent with its cognitive internal representation.
Research Aim and Research Questions
Based on the literature review, this study hypothesizes that the different essence of cognitive processes
included in different kinds of problems results in obstacles to the problem-solving ability transfer for students in
school and real life. For example, there have been several types of problems in physics education, such as exercises,
context-rich problems, and real problems, which are significantly different in the continuum of well-structured
ill-structured problems. Here, it is supposed that expanding the problem space, including sufficient external and
internal representation spaces, can benefit from improving students’ problem-solving ability, especially the gap
between problem-solving ability in the real world and school education can be further bridged.
Therefore, this study aimed to:
1. Based on the theoretical framework of ecological problem representation, propose a new problem
type, different from the existing problem types, which is named primitive physics problem.
2. Describe the characteristics of primitive physics problems theoretically and empirically. As for the theoreti-
cal description, the critical attributes of the new problem type will be examined, which is different from
the existing problem types and explain the significance of training students’ problem-solving ability.
3. Design an instrument based on primitive physics problems for upper-secondary school students.
Theoretical Framework
Based on the synergetic theory and ecological problem representation, this study proposes the self-organizing
representation theory (SORT) by combining Newell’s and Simon’s (1972) Theory of Human Problem Solving and
Representations in distributed cognitive task developed by Zhang & Norman (1994).
In the theoretical framework, problem solving is assumed to combine continuity with mutation, indepen-
dence with correlation, control with spontaneity, cooperation with competition, and necessity with contingency.
Representation-state is the state of problem representation, which is the relatively stable level of problem repre-
sentation at a specific time in problem solving. Representation-state is the state in which the internal knowledge
of the problem solver interacts with the external information in the process of problem solving. There are two
cognitive models: data-driven and concept-driven. When the problem is presented to the solvers, the initial state
of existence is the separation of external information and internal information, which is a non-representation
state; Finally, when the problem is solved, it is the interaction of internal and external information that realizes the
complete representation of the problem.
Based on SORT, it is assumed that there is a process in which the representation state changes when solving
physics problems. The whole process includes six different representation states add: abstraction representation,
assignment representation, image representation, physics representation, methodology representation, and
mathematics representation.
310
ISSN 2538–7138 /Online/ (pp. 305-324)
Abstraction Representation. Students can create mental entities for a higher level of organization of new
concepts by the process of abstraction (Ainsworth, 2006). What information needs to be discarded and retained
from the physics problem requires the students to make their own decisions. The process of abstracting ideal
objects from the real physical phenomenon is to simplify and purify the process of object movement and to sub-
limate it into an ideal state by imaginative, intuitive, and logical thinking so that the essence of the problem can
be reflected more clearly.
Assignment Representation. Assignment representation is a unique representation type, rarely involved in
the routine problem-solving process. Students should set physics quantities according to their needs and list the
known and unknown quantities. Intermediate quantities are also necessary at times to be set up, which appear
in the process of solving the problem, but they do not appear in the final equation. Indeed, the physics quantities
set in this stage are not required to be consistent because there is no need for uniform and fixed assignments to
solve problems.
Image Representation. Image representation is to sketch in the problem-solving process, which is included
in traditional exercises and physics problem-solving processes and needs students to complete independently
with the help of image thinking.
Physics Representation. A major aspect for students to solve physics problems is considering what primary
principles or concepts can be applied to problem solving (Yerushalmi et al., 2007). The process of identifying and
using physical conceptions and principles to solve problems refers to physics representation, which means students
need to relate the physics problem to the physical concepts and principles they have learned before.
Methodology Representation. In this stage, scientific methodology or specific methods should be identified
by students to solve problems, which reflects the relationship between physics knowledge and physics methods.
Mathematics Representation. It refers to a series of mathematical deduction steps, including column equa-
tion, solution equation, necessary mathematical transformation, and approximation, which is more abstract than
physics representation (McDermott & Larkin, 1978).
The changing pattern of the representation state differs for different solvers, and it shows some continuity
and discontinuity, linear and nonlinear, static, and dynamic. The characteristics of the transition of states are below.
Linear and Nonlinear. The change of these problem representations usually is nonlinear. However, in a specific
stage, the factors that affect the problem representation have relatively stable or linear change characteristics, so
the representation state also has a local linear change process. The linear and nonlinear changes of the representa-
tion state determine that the process of problem solving has the characteristics of linear and nonlinear changes.
Static and Dynamic. When solving this kind of problem, solvers are always in a cognitive field, which is in-
fluenced by many factors and shows the static and dynamic combination characteristics. When there are many
interactions between internal and external factors, the representation of the problem needs to be repeated or
even reconstructed, and the representation state is dynamic. When the interaction between internal and external
factors is less, the representation state is more static.
Assimilation and Adaptation. Assimilation means that the subject extracts and identifies the external given
problem condition information and understands the meaning and mutual relationship of the information; On
the other hand, based on information assimilation, adaptation forms some abstract and general problem repre-
sentations. There are two critical factors affecting information assimilation, and one is the extraction of external
information, the other is the activation of internal related knowledge.
General Background
To develop a measuring instrument of primitive physics problems for upper-secondary school students, this
study was divided into two parts. For Part I, based on an extended theoretical framework, qualitative research was
carried out to propose a new type of problem, which was described qualitatively and quantitatively. Next, for part
II, a measuring instrument consisting of a series of primitive physics problems was developed. To test the reliability
and validity of the instrument, data of students’ responses to the instrument was collected. The test was applied at
the beginning of the 2019 – 2020 school year. The test time was approximately 45 minutes.
311
(pp. 305-324) ISSN 2538–7138 /Online/
Part I: A new type of problem: Primitive Physics Problem
Primitive Physics Problem
A primitive Physics Problem describes a problem abstracted from the objective physics phenomena in real
life. Thus, the Primitive Physics Problem is situated between context-rich and real-life problems, far away from a
textbook problem and close to a real-life problem along the continuum from well-structured to ill-structured. For
instance, the graph in Figure 1 shows four categories of problems.
Figure 1
Typology of Problems in Physics
Problems
Practice Context-rich Primitive Real-life

problems problems Physics problems
Problems
Internal representation included only Both Internal and external representation included
Well-structured Ill-structured
The primitive physics problem refers to the typical phenomenon that has not been abstracted in nature, social
life, and production. It has the following characteristics:
1. It is a description of the phenomenon, without abstracting the phenomenon to any extent;
2. It is a text description, usually without any known conditions, and the implicit variables and constants
need to be set by students;
3. There is no schematic diagram, and students need to draw the images needed to solve the problems
themselves;
4. It is not conventional for students and cannot be solved by simple imitation;
5. From real-life situations;
6. It is exciting and charming and can arouse students’ thinking and put forward intellectual challenges
to them;
7. There is not necessarily a unique answer. Students of different levels can answer from the simple to the
deep;
8. Solving it needs to be accompanied by individual or group activities.
An example for a typical primitive physics problem is as follows: Some people think it is not safe for an adult
to hold a baby in a car. Now, please estimate how much force it takes to hold a baby in a very short crash. There are
great differences between the primitive physics problem and Exercise. Exercise is a problem that abstracts, simplifies,
and decomposes the phenomenon. It is a manual exercise. It has the following characteristics:
1. It is not a description of the phenomenon, but a high degree of abstraction of the phenomenon;
2. Although it is also a text description, all the known conditions have been given, so students do not
need to set them by themselves;
3. All the images needed to solve the problem have been drawn, and students are not required to draw
them by themselves;
4. It is routine for students, which can be solved by simple imitation;
5. A few of them come from real-life situations, while most of them do not;
6. Lack of interest and charm mainly used to train students to master knowledge;
7. There is only one answer;
8. Individual solution, no need for group activities.
312
ISSN 2538–7138 /Online/ (pp. 305-324)
An example for a typical exercise problem is as follows: A typical exercise problem: It is not safe for an adult to
hold a baby in a car. Please calculate: in a 0.1 s crash, if the vehicle speed is 60 km / h before the crash, how much
force does one need to hold a 10 kg baby.
From the perspective of problem space, both the primitive physics problems and exercises include three in-
ternal representations: physical representation, method representation, and mathematical representation. In ad-
dition, the essential distinction from exercises is that the primitive physics problems include not only three internal
representations but also external representations including abstract representation, assignment representation,
and image representation. As for abstraction representation, it refers to grasping the main factors, ignoring the
secondary factors, extracting the element and structure with physical domain characteristics from the description
of the problem to facilitate the transformation of the prototype into a preliminary physical model. As for assignment
representation, it needs to set multiple physical quantities, like constants, variables, and intermediate variables,
which makes the model numerical. In contrast, exercises or textbook problems with the characters of sufficient
information, static state, and no interaction with the real world, do not have these problem spaces.
Although the exercises are related to phenomena in form, they provide detailed data. However, they do not
provide students with real-world problem situations. Therefore, the cultivation of students’ ability to analyze and
solve problems is severely reduced. The original problem is to embed each to be known quantity in the natural
phenomenon instead of giving it directly. Based on the analysis of the situation, students need to obtain the
required variables and data through assumptions, abstractions, and means and then construct an ideal model.
After a layer of “stripping” process, the conclusion “breaks the cocoon.” The relationship between primitive physics
problems
Figureand2 exercises is shown in Figure 2.
Representational Relationship between Traditional Exercise and the Primitive Physics
Figure 2
Problems
Representational Relationship between Traditional Exercise and the Primitive Physics Problems
Abstraction Assignment Image Physics Methodology Mathematics

Representation Representation Representation Representation Representation Representation
External Representation Internal Representation
Representation process of Primitive Physics Problem Representation process of exercise problem
The difference between primitive physics problems and context-rich problems is also obvious. For context-
rich problems, what connects with the real world is the background of the problem. The space embedded in the
problem is a “story” related to the real world. Students need to construct their problem representation process in
the context of materials representing the real world. For the Primitive Physics Problems, what connects with the
natural world is the problem itself. That is, the problem itself is a description abstracted from the physical phenom-
ena in the real world. Students need to abstract and assign values to the problem description through an external
representation process to transit to the solvable domain of routine problem solving.
Part II: The development of the instrument of Primitive Physics Problem
Participants
In this study, 1000 students were randomly selected from 10 upper-secondary schools in China. The research
involved 811 valid samples, including 417 boys and 394 girls after eliminating invalid information samples. The
characteristics of the screened samples maintained a high degree of consistency with the original overall sample
distribution characteristics. All of the participants had completed the entire physics course, including junior high
school and upper-secondary school, and had an excellent academic background. This academic excellence was
measured based on the Lawson classroom test of scientific reasoning (LCTSR).
Given that the participants came from different schools, they were invited to enroll in the test representatively.
313
(pp. 305-324) ISSN 2538–7138 /Online/
Therefore, the test was administered with the same length of time and other test conditions. All participants vol-
unteered to participate in this study. All parties, including students, teachers, and parents signed a formal consent
form for their participation in the study, which described the activities in which they would be involved, and ensured
that all personal data would be treated anonymously and confidentially.
Based on an extended theoretical framework and the new type of problem proposed in this study, six primitive
physics problems were designed. Next, the six problems were reviewed by three science education experts who
worked in the Faculty of Education at Capital Normal University for the readability, scientificity, and rationality of
these problems and how well they were likely to measure students’ thinking process. Two items were excluded for
their lack of appropriateness, leaving 4-items. The instrument appropriate for upper secondary school students
used to test included four primitive physics problems based on the classic physical phenomena in real life.
Moreover, according to the six representations included in the problem, the rubric of the instrument was
developed. The example of the rating criteria was listed in Table 1. The score for each representation was a di-
chotomy scoring system, which scored 1 or 0. “1” was assigned if the representation did occur, and “0” if it never
occurred. Take assignment representation as an example, students could get 1 point in the grading process if they
ultimately set sufficient physical quantities and 0 points if any setting was incomplete or incorrect. Each problem
was resolved with six representations, so the total score for each question was six points, and the whole final score
was twenty-four. One of the measuring instruments of upper secondary school students’ primitive physics problem
was as following example. (See Appendix A for complete instrument and Grading Rubric)
Example: The Primitive Physics Problem 1—Ship Roll Reduction

Problem: On December 24, 1990, the Chinese scientific research vessel polar encountered a strong wind
at sea and was impacted by periodic waves. As we all know, if the frequency of wave impact force is close to the
natural frequency of ship swing, the ship swing will be intensified. Once the resonance state is reached, the ship
may capsize. At this time, the polar adopted the measures of changing course and reducing speed, which reduced
the swing range of the ship.
Please explain with an expression why polar could reduce the ship’s swing amplitude by changing the course
and reducing the speed.
Table 1
Rating Criteria of the Primitive Physics Problem for the Ship Roll Reduction
Representations Rating Standard
Abstraction Representation Ignore the ship's length, height, width, and mass, and regard the ship as a particle
1. Set the speed of the ship to be ;
2. Set the speed of the wave to be ;
3. Set the velocity of the wave relative to the ship to be ;
4. Set the length of the wave to be ;
Assignment
5. Set the length of the wave relative to the ship to be ;
representation
6. Set the frequency of the wave to be ;
7. Set the frequency of the wave relative to the moving ship to be ;
8. Set the angle between the speed direction of the ship and the wave propagation direction to be .
1. Draw rectangular coordinate system ;

2. Draw the velocity of the wave , along
Image 3. the -axis;
Representation 4. Draw the speed of the ship , in the
5. the direction of angle to the -axis;
6. Draw the angle between the ship’s speed and the wave’s speed.
314
ISSN 2538–7138 /Online/ (pp. 305-324)
Methodology 1. Velocity decomposition method (including establishing the coordinate system and specifying positive direction);
Representation 2. Induction (from special to general).
Physics
Representation 1. The formula among wavelength, frequency, and wave velocity .
1. Firstly, the situation that the ship’s velocity direction is the same as the wave propagation direction is studied.
The speed of the wave relative to the ship: ;
The length of the wave is constant for a moving ship: ;
So that the frequency of wave impact force ship received:
.
2. Then, the situation that the ship’s velocity direction is perpendicular to the wave propagation direction is
Mathematics
studied.
Representation
The speed of the wave relative to the ship: ;
The velocity of a wave is the velocity of its propagation. When observed perpendicular to its propagation direc-
tion, the observer’s motion does not affect the result: ;
So that the ship received the wave’s impact force frequency is:
3. The results of inductive situations 1. and 2.
Data Collection
A printed version of the measuring instrument was answered by students participating in the study. The test
was applied at the beginning of the 2019-2020 school year. The collection of these data was conducted during
physics classes. The test was administered with the same length of time and other test conditions. The teacher of
each class was responsible for the test implementation.
Data Analysis
The difficulty level of the test (P), is an important indicator to measure the quality of the test, which together
with discrimination affects and determines the quality of the test. The difficulty level of the measuring instrument
was calculated using a passing rate, with a minimum value of 0 and a maximum value of 1. A small difficulty value
indicates that the test is difficult, while a large value indicates that the test is easy.
Discrimination (D), an index of discrimination, is an indicator of the ability level of the participants. The dis-
crimination of the measuring instrument was calculated using the difficulty level of the top 27% of the whole score
group ( ) and that of the bottom 27% of the whole score group ( . Then its calculation formula is .
Discrimination is generally between 0 and 1, and 0.4 or higher indicates that the question is well discriminated.
To assess the reliability of the instrument, Cronbach’s alpha coefficient of internal consistency, the item-total
correlation by split-half reliability, and t-tests were used. SPSS 22.0 software was used for statistical analyses.
To test the construct validity of the instrument, confirmatory factor analysis (CFA) was used. Several indicators
are used to test the compatibility of a proposed model with data. χ2 goodness-of-fit statistics determines the size
of the discrepancy between the sample and fitted covariance matrices. Because the size of the data set affects the
value for p, the supplement fit indexes need to be used. In this study, Tucker-Levis Index (TLI), Comparative Fit Index
(CFI), Standardized Root Mean Square Residual (SRMR), and Root Mean Square Error of Approximation (RMSEA)
indexes were used. CFI and TLI values over 0.95 indicate perfect fit with the model, whereas values between 0.85
and 0.90 indicate acceptable fit (Marsh et al., 2004). For RMSEA and SRMR, values 0.20 and below indicate accept-
able fit. SPSS 22.0 software was used for statistical analyses. MPLUS 8.0 software was used for statistical analyses.
315
(pp. 305-324) ISSN 2538–7138 /Online/
Research Results
Difficulty Level and Discrimination Analysis
The difficulty level of the measuring instrument was calculated as P = 0.220. The result demonstrates that the
instrument of 12th grade primitive physics problem is slightly difficult. The discrimination of the measuring instru-
ment was calculated, D = 0.406, which indicates that the measuring instrument of the primitive physics problem
can distinguish students with different abilities.
Reliability and Validity Analysis
The reliability is computed. The coefficient of Cronbach’s Alpha value is 0.742. Nunnally recommended that
alpha values be at least 0.7 (Nunnally, 1978), suggesting a higher degree of consistency among the problems.
Further Confirmatory Factor Analysis (CFA) was used to testify the construct validity. Using MPLUS to construct
the first-order model, the fitting index of the model is shown in Table 2. Models (e.g., CFA) could be accepted if they
met the following requirements: (1) the ratio of c2 and df was statistically nonsignificant (p>.01); (2) Standardized
Root-Mean-Square Residual (SRMR) and Root-Mean-Square Error of approximation (RMSEM) were smaller than
0.08; and (3) Confirmatory Fit Index (CFI) and gamma hat were more significant than 0.90 (Marsh et al., 2004). The
values of CFI (0.931) and TLI (0.884) are more significant than 0.80 while that of SRMR (0.034) and RMSEA (0.191)
(see Table 2), which fit the evaluation index. Thus, the result of confirmatory factor analysis implies the fitting index
of the model is acceptable.
Table 2
Model Fitting Index of the Primitive Physics Problem Instrument
χ2 df TLI CFI AIC BIC SRMR RMSEA
275.37* 9 0.884 0.931 8655.25 8739.75 0.034 0.191
Based on the data analysis, it can be seen that the primitive physics problem instrument has moderate difficulty
and good discrimination, and the problems are closely related to the real situation. Moreover, the instrument also has
high reliability and validity, which made it possible to identify students’ ability to solve physics problems effectively.
Discussion
Representation Style of Primitive Physics Problems: from Internal Representation to External Representation
High hopes are placed on that problem solving can be taught as a technique like computing. Traditional
school training often emphasizes the construction of internal representation through practice, which leads students
to memorize formulas to find algorithm solutions without a deep understanding of concepts. According to the
mental mode theory, problem solving depends on the construction and operation of internal representation. The
construction of the psychological model is formed by the connection between the problem description elements
and the underlying knowledge base. Exercises contain elements to promote the construction of the psychological
model, which is the internal representation mentioned in this study.
Further, students have difficulty with solving real-world problems in the context of traditional school training.
Research in physics education also found that more emphasis on qualitative representation was helpful to improve
the achievement of student problem solving. So, it is critical for students to learn how to understand a problem
before solving it, which refers to interacting with appropriate representations.
The Primitive Physics Problems supply students with sufficient problem space, including external representa-
tions and internal representations, and the nature of problems they face are changed completely. When students
face primitive physics problems, they need to extract the information with physical characteristics from the problem
description, which involves transforming the problem prototype into the preliminary physical model. The represen-
tation process distinguishes the primitive physics problems from other types of problems: exercises, context-rich
316
ISSN 2538–7138 /Online/ (pp. 305-324)
problems, and real problems in the continuum from well-structured to ill-structured problems. Specifically, solv-
ing exercises and context-rich problems does not include this process, and to solve real problems is challenging
for students to carry out this process efficiently. In addition, according to the preliminary physical model, to set
the physical quantities and parameters, to draw images actively, refer to experiencing assignment representation
and image representation, which is the transition way from prototype to internal representation structure. These
three representations together constitute the external representation space of primitive physics problems, which
is also the particular space of this problem. When students succeed in the transformation of the problem statement
from it to wanted state, in many different formats, like verbal, visual, mathematical representations, it is said that
students have mastered the skills of how to transform real-world problems into patterns which are consistent with
their internal psychological structure and finally solve them.
Cognitive Driven Mode of Primitive Physics Problem: from Data-Driven to Concept-Driven
There are two ways of problem-solving cognitive driven mode: one is data-driven, that is, scientists firstly col-
lect a large amount of data, then analyze these materials to find out the patterns, and then explain these findings;
the other is concept-driven, that is, scientists first put forward hypothetical theories, then make predictions, and
last verify theories supported by facts.
Experts are distinguished from novices because of their cognitive mode for problem solving. When solving
a new problem, novices are more likely to lack knowledge structure and likely jump into quantitative expressions
quickly, including haphazard formula-seeking and solution pattern matching. So, they showed a data-driven model,
which is a process from bottom to up. It takes observation and the discovery of scientific facts as the starting point
of scientific research. Its processing takes scientific research data as the processing object and makes the process-
ing information meet the requirements of the explanation of a phenomenon. In contrast, experts show more
concept-driven patterns in solving problems. They tend to redescribe the problem quickly and analyze a problem
qualitatively based on principles and concepts at first, and then elaborate on them in greater mathematical detail.
The experts tend to work forward from given values and known quantities to the wanted quantity. It is said that
they showed a concept-driven mode, which refers to the top-down procedure. It takes the contradiction between
scientific theory and empirical facts and the contradiction of scientific theory itself as the starting point of scientific
research. Information satisfies the consistency of the theory itself.
In physics education, it is necessary to support students to become an expert equipped with problem-solving
skills. Students have been exposed to long-time textbooks problems or exercises. Textbook problems or exercises
are depicted in mathematical forms and full of data, resulting in difficulty in perceiving and grasping the underlying
concepts or explanatory principles highlighted by the theory under consideration. The guidance leads students
to focus on the surface features of problems and train superficial cognitive processing, which shows a data-driven
mode. Exercises tend to encourage means-end rather than knowledge development strategies because students
frequently focus on finding the equation that best fits the problem. Exercises encourage students to use a data-
driven cognitive mode. The “data” or “condition” of exercises is the fundamental defect of exercises. “Data” is not only
the crutch of students’ thinking but also a critical “hint.” If students are only allowed to practice physics exercises
without solving physics problems, it would be difficult for students to lose this “crutch” forever, affect their ability
development, and cause obstacles for students to develop to the expert.
The Primitive Physics Problems describe physical phenomena and keep the “original flavor” of phenomena.
Compared with exercises, the Primitive Physics Problems only describe phenomena, which do not contain any
data or an abstract model. Because it hides the “data” or “condition” in the real phenomenon, it needs students to
determine “data” or “condition” by themselves according to the phenomenon through assumptions, abstractions,
and other methods. Therefore, the solution of the Primitive Physics Problem must be concept-driven processing
instead of data-driven processing. Compared with concept-driven, data-driven often needs the “starting point” or
“fulcrum” of processing. When solving the Primitive Physics Problem, it has unknown conditions, to force students
to give up data-driven processing. In other words, when students learn how to solve Primitive Physics Problems, the
concept-driven cognitive mode is used, including description, problem analysis qualitatively based on concepts,
which are in line with the experts’ mode for solving problems.
317
(pp. 305-324) ISSN 2538–7138 /Online/
Cognitive State of Primitive Physics Problem: from Organized to Self-Organized
The cognitive state refers to the organizational state of the problem solver’s brain or the “phase” of the problem
solver’s brain. According to German theoretical physicist Hermann Haken, the evolutionary form of organization
can be divided into two categories: be-organized and self-organized. If external instructions organize a system, it
is be-organized; If there is no external instruction, the system automatically forms an orderly structure according
to some tacit rules, it is self-organized. From the perspective of system theory, “self-organized” refers to the process
that a system develops from simple to complex, from rough to detailed, and constantly improves its complexity
and fineness driven by internal mechanisms (Sherblom & Stephen, 2017).
This study believes problem solving involves the transition from certainty to diversity and uncertainty, from
linear to nonlinear, from knowledge poor to knowledge rich. The solution of primitive physics problems is a self-
organization process of problem solvers to represent the state of problems, and this process has the characteristics
of nonlinearity, mutation, and self-organization. In other words, solving primitive physics problems combines con-
tinuity with mutation, independence with correlation, cooperation with competition, control with spontaneity, and
a necessity with contingency. As Anderson’s ACT-theory (Adaptive Control of Thought) said the process of complex
cognitive skills was divided into three stages. Firstly, learners generate initial solutions by weak methods in new
problem situations. Secondly, a knowledge compilation process facilitates the movement from slow, controlled
processing to more automated processing. Thirdly, whenever cognitive rules are successfully applied, cognitive
rules will accumulate strength. Finally, after extensive practice, this reinforcement process may eventually automate
the performance of problem-solving skills.
Physics exercises usually state the events that follow one or some disciplinary principles indirectly. The cogni-
tive process of physics exercises is based on specific cognitive strategies and skills to complete the disconnection
between knowledge and cognitive chain “caused by” indirectness, to make the problem elements of physics
exercises explicit. The event with dominant characteristics can clearly show the “context” of the event. When solv-
ing this kind of problem, the process is intervened by the outside factors and has the typical characteristics of
weak self-selection and dynamic evolution, weak initiative innovation, and adaptability. Limited by the restricted
problem space, students can only carry out “cloze” problem solving, showing the characteristics of the cognitive
state of being organized.
For a long time, there has been a phenomenon of “the sea of exams tactics” in Chinese physics education,
which has not been significantly improved so far. Up to now, there is a new trend word, “Equation solver from a small
town,” to describe students skilled at exams rather than solving problems. Most of the cognitive state of Chinese
students is in the state of being organized, and only a few of them are in the state of self-organization. The primi-
tive physics problem makes students return to the phenomenon and pay attention to the “real world.” The return
is consistent with the process of students’ knowledge generation and mastery, which needs interaction in the real
and natural social environment. Through a certain amount of primitive physics problem training, when students
solve practical physics problems, a variety of problem-solving strategies can be quickly retrieved without having to
rummage to compare the types of questions they have done so that they can preview the next step in their brain
when dealing with the previous step. Even when students are carrying out creative activities, they can also rely
on physical intuition rather than experience to explore the right way to solve the problem. In this sense, primitive
physics problem training can not only make students learn physics knowledge but also promote the development
of students’ physics cognitive state. At the same time, it is precisely because the primitive physics problems apply
the strict control of exercises to physical phenomena, and can become exercises after being abstracted, which has
the function of efficiently completing the internalization of indirect empirical knowledge to a certain extent, and
better unifies the internal validity and external validity of physics education, Thus, it provides beneficial enlighten-
ment for breaking the phenomenon of “sea topic tactics” in physics education.
In this study, a new type of problem was added to the existing problem categories based on the problem
continuum. It provided a bridge between the problem-solving training in school education and the problem-solving
practice in real life. It provided more effective tools and teaching scaffolding for students to develop real-world
problem-solving abilities. Further, the primitive physics problem instrument was developed, and the instrument
also has high reliability and validity, which made it possible to identify upper-secondary students’ ability to solve
physics problems effectively.
318
ISSN 2538–7138 /Online/ (pp. 305-324)
There are two limitations to this study. First, only one type of task was used with the participants to examine
the effects of the overall response. As such, it is possible that the response collected may be confined to these four
items. Additionally, more data could be collected by adding more items. Alternatively, students could have been
provided with the answer sample as a scaffold to solve the test’s problem. These options are left for future research.
The second limitation of this study is that the students with high-level performance were sampled, which leads
to a certain limitation in the application of instruments. Furthermore, a more general sample should be chosen to
carry out the study. Finally, it is worth noting that the focus of this study is to propose a new type of problem and
preliminary exploration of measuring instruments.
Acknowledgments
This research was funded by the General Project of Education of the National Social Science Foundation of
China: “Research on the Composition and Performance of Key Competence”. (Grant number BBA190024).
References
Ainsworth, S. (2006). DeFT: A conceptual framework for considering learning with multiple representations. Learning and
Instruction, 16(3), 183-198. https://doi.org/10.1016/j.learninstruc.2006.03.001
Björklund, T. A. (2013). Initial mental representations of design problems: Differences between experts and novices. Design
Studies, 34(2), 135-160. https://doi.org/10.1016/j.destud.2012.08.005
Byun, T., & Lee, G. (2014). Why students still can’t solve physics problems after solving over 2000 problems. American Journal of
Physics, 82(9), 906-913. https://doi.org/10.1119/1.4881606
Chi, M. T. H., Feltovich, P. J., & Glaser, R. (1981). Categorizations and representation of physics problems by experts and novices.
Cognitive Science, 5, 121-152. https://www.sciencedirect.com/science/article/pii/S0364021381800298
Chambers, D., & Reisberg, D. (1985). Can mental images be ambiguous? Journal of Experimental Psychology: Human Perception
and Performance, 11(3), 317-328. https://doi.org/10.1037/0096-1523.11.3.317
Cho, K. L., & Jonassen, D. H. (2003). The effects of argumentation scaffolds on argumentation and problem solving. Educational
Technology: Research & Development, 50(3), 5–22. https://doi.org/10.1007/BF02505022
Chi, M. T. (2018). Learning from examples via self-explanations. In Knowing, learning, and instruction (pp. 251-282). Routledge.
Dunkle, M. E., Schraw, G., & Bendixen, L. D. (1995). Cognitive processes in well-defined and ill-defined problem solving. Paper
presented at the annual meeting of the American Educational Research Association, San Francisco, Calif., April.
https://doi.org/10.1002/acp.2350090605
Huffman, D. (1997). Effect of explicit problem-solving strategies on high school students’ problem- solving performance and
conceptual understanding of physics. Journal of Research in Science Teaching: The Official Journal of the National Association for
Research in Science Teaching, 34(6), 551-570. https://doi.org/10.1002/(SICI)1098-2736(199708)34:6<551::AID-TEA2>3.0.CO;2-M
Ericsson, K. A., Hoffman, R. R., Kozbelt, A., & Williams, A. M. (Eds.). (2018). Cambridge handbooks in psychology. The Cambridge
handbook of expertise and expert performance (2nd ed.). Cambridge University Press. https://doi.org/10.1017/9781316480748
Foster, T. M. (2000). The development of students’ problem-solving skill from instruction emphasizing qualitative problem-solving.
University of Minnesota.
Gagne, R. M. (1970). The conditions of learning (2nd ed.). Holt, Rinehart and Winston. http://eduq.info/xmlui/handle/11515/12803
Garrett, R. M. (1986). Problem-solving in science education. Studies in Science Education, 13(1), 70-95.
https://doi.org/10.1080/03057268608559931
Gibson, J. J. (2014). The ecological approach to visual perception: Classic edition (1st ed.). Psychology Press.
https://doi.org/10.4324/9781315740218
Greeno, J. G. (1980). Trends in the theory of knowledge for problem solving. Problem Solving and Education: Issues in Teaching
and Research, 9-23.
Hayes, J. R. (1989). The complete problem solver (2nd ed.). Routledge. https://doi.org/10.4324/9780203062715
Heller, P., & Hollabaugh, M. (1992). Teaching problem solving through cooperative grouping. Part 2: Designing problems and
structuring groups. American Journal of Physics, 60(7), 637-644. https://doi.org/10.1119/1.17118
Henderson, C. R. (2002). Faculty conceptions about the teaching and learning of problem-solving in introductory calculus-based
physics. University of Minnesota.
Hong, N. S., Jonassen, D. H., & McGee, S. (2003). Predictors of well-structured and ill-structured problem solving in an astronomy
simulation. Journal of Research in Science Teaching, 40(1), 6–33. https://doi.org/10.1002/tea.10058
Horn, J. L., & Blankson, A. N. (2012). Foundations for better understanding of cognitive abilities. In D. P. Flanagan & P. L. Harrison
(Eds.), Contemporary intellectual assessment: Theories, tests, and issues (pp. 73–98). The Guilford Press.
319
(pp. 305-324) ISSN 2538–7138 /Online/
Hsu, L., Brewe, E., Foster, T. M., & Harper, K. A. (2004). Resource Letter RPS-1: Research in problem solving. American Journal of
Physics, 72(9), 1147-1156. https://doi.org/10.1119/1.1763175
Jonassen, D. H. (1997). Instructional design model for well-structured and ill-structured problem-solving learning outcomes.
Educational Technology: Research and Development, 45(1), 65–95. https://doi.org/10.1007/BF02299613
Jonassen, D. H. (2000). Toward a design theory of problem solving. Educational Technology Research and Development, 48, 63–85.
https://doi.org/10.1007/BF02300500
Jonassen, D. H., & Kwon, H. I. (2001). Communication patterns in computer mediated vs. face-to-face group problem solving.
Educational Technology: Research and Development, 49(10), 35–52. https://doi.org/10.1007/BF02504505
Jonassen, D. H. (2010). Learning to solve problems: A handbook for designing problem-solving learning environments. Routledge.
https://doi.org/10.4324/9780203847527
Kitchner, K. S. (1983). Cognition, metacognition, and epistemic cognition: A three-level model of cognitive processing. Human
Development, 26(4), 222–232. https://doi.org/10.1159/000272885
Kim, E., & Pak, S. J. (2002). Students do not overcome conceptual difficulties after solving 1000 traditional problems. American
Journal of Physics, 70(7), 759-765. https://doi.org/10.1119/1.1484151
Kleinmuntz, D. N., & Schkade, D. A. (1993). Information displays and decision processes. Psychological Science, 4(4), 221-227.
https://doi.org/10.1111/j.1467-9280.1993.tb00265.x
Larkin, J. H., & Reif, F. (1979). Understanding and teaching problem-solving in physics. European Journal of Science Education, 1(2),
191-203. https://doi.org/10.1080/0140528790010208
Larkin, J., McDermott, J., Simon, D. P., & Simon, H. A. (1980). Expert and novice performance in solving physics
problems. Science, 208(4450), 1335-1342. https://doi.org/10.1126/science.208.4450.1335
Lesh, R., & Zawojewski, J. (2007). Problem solving and modeling. In F. K. Lester (Ed.), Second handbook of research on mathematics
teaching and learning (pp. 763-799). National Council of Teachers of Mathematics and Information Age Publishing.
Maloney, D. P. (1994). Research on problem solving: Physics. In D. L. Gabel (Ed.), Handbook of research on science teaching and
learning (pp.327-356). Macmillan Publishing Company.
Marsh, H. W., Hau, K.-T., & Wen, Z. (2004). In search of golden rules: Comment on hypothesis-testing approaches to setting cutoff
values for fit indexes and dangers in overgeneralizing Hu and Bentler’s (1999) findings. Structural Equation Modeling, 11(3),
320-341. https://doi.org/10.1207/s15328007sem1103_2
McDermott, J., & Larkin, J. H. (1978). Re-representing textbook physics problems. In National Conference of the Canadian Society
for Computational Studies of Intelligence. University of Toronto Press.
Meacham, J. A., & Emont, N. C. (1989). The interpersonal basis of everyday problem solving. In J. D. Sinnott (Ed.), Everyday problem
solving: Theory and applications (pp. 7–23). Praeger.
Newell, A., & Simon, H. A. (1972). Human problem solving. Prentice-Hall.
Newell, A. (1994). Unified theories of cognition. Harvard University Press.
Popper, K. (1999). All life is problem solving. Routledge.
Pulgar, J., Fahler, V., & Spina, A. (2021). Investigating how university students collaborate to compose physics
problems through structured tasks. Physical Review Physics Education Research, 17(1), Article 010120.
https://doi.org/10.1103/PhysRevPhysEducRes.17.010120
Reif, F., & Heller, J. I. (1982). Knowledge structure and problem solving in physics. Educational Psychologist, 17, 102-127.
https://doi.org/10.1080/00461528209529248
Reisberg, D. (1987). External representations and the advantages of externalizing one’s thoughts. In Proceedings of the 9th Annual
Conference of the Cognitive Science Society Cognitive Science Society (pp. 281-293). Psychology Press.
Reif, F. (1995). Understanding and teaching important scientific thought processes. Journal of Science Education and
Technology, 4(4), 261-282. https://doi.org/10.1007/BF02211259
Reys, R. E., Suydam, M. N., Lindquist, M. M., & Smith, N. L. (1998). Helping children learn mathematics (5th ed.). Allyn and Bacon.
Redish, E. F., Scherr, R. E., & Tuminaro, J. (2006). Reverse-engineering the solution of a simple physics problem: Why learning
physics is harder than it looks. The Physics Teacher, 44, 293-300. https://doi.org/10.1119/1.2195401
Reif, F. (2010). Applying cognitive science to education: Thinking and learning in scientific and other complex domains. MIT press.
Scaife, M., & Rogers, Y. (1996). External cognition: How do graphical representations work? International Journal of Human-computer
Studies, 45, 185-213. https://doi.org/10.1006/ijhc.1996.0048
Sherblom, & Stephen, A. (2017). Complexity-thinking and social science: self-organization involving human consciousness. New
Ideas in Psychology, 47, 10-15. https://doi.org/10.1016/j.newideapsych.2017.03.003
Simon, H. A. (1978). Information-processing theory of human problem solving. Handbook of Learning and Cognitive Processes,
5, 271-295.
Stenning, K., & Oberlander, J. (1995). A cognitive theory of graphical and linguistic reasoning: Logic and implementation. Cognitive
Science, 19(1), 97-140.
Van Heuvelen, A. (1991). Learning to think like a physicist: A review of research-based instructional strategies. American Journal
of Physics, 59(10), 891-897. https://doi.org/10.1119/1.16667
Vekiri, I. (2002). What is the value of graphical displays in learning? Educational Psychology Review, 14, 261-312.
https://doi.org/10.1023/A:1016064429161
Voss, J. F. (1988). Problem solving and reasoning in ill-structured domains (pp. 74-93). SAGE Publications.
Voss, J. F., & Post, T. A. (1988). On the solving of ill-structured problems. In M. T. H. Chi, R. Glaser, & M. J. Farr (Eds.), The nature of
expertise (pp. 261–185). Lawrence Erlbaum Associates.
320
ISSN 2538–7138 /Online/ (pp. 305-324)
Voss, J. F., Lawrence, J. A., & Engle R. A. (1991). From representation to decision: An analysis of problem solving in international
relations. In R. J. Sternberg, & P. A. Frensh (Eds.), Complex problem solving (pp. 119-157). Lawrence Erlbaum.
Wheatley, G. H. (1984). Problem solving in school mathematics. MEPS Technical Report, 84, l.
Wieman, C. (2015). Comparative cognitive task analyses of experimental science and instructional laboratory courses. The Physics
Teacher, 53(6), 349-351. https://doi.org/10.1119/1.4928349
Wood, P. K. (1983). Inquiring systems and problem structure: Implications for cognitive development. Human Development, 26(5),
249–265. https://doi.org/10.1159/000272887
Yerushalmi, E., Henderson, C., Heller, K., Heller, P., & Kuo, V. (2007). Physics faculty beliefs and values about the teaching and
learning of problem solving. I. Mapping the common core. Physical Review Special Topics-Physics Education Research, 3(2),
Article 020109. https://doi.org/10.1103/PhysRevSTPER.3.020109
Zhang, J., & Norman, D. A. (1994). Representations in distributed cognitive tasks. Cognitive Science, 18, 87-122.
https://doi.org/10.1207/s15516709cog1801_3
Zoller, U., Nakhleh, M.B., Dori, J., Lubezky, A. & Tessier, B. (1995). Success on algorithmic and LOCS vs. conceptual exam questions.
Journal of Chemical Education, 72, 987-989. https://doi.org/10.1021/ed072p987
Zoller, U. (2000). Teaching tomorrow’s college science courses - are we getting it right? Preparing students to become informed
and responsible participants in the decision-making process. Journal of College Science Teaching, 29, 409– 414.
Appendix: The Primitive Physics Problem and Grading Rubric
The Primitive Physics Problem 1—Bicycle turning

Problem: When a cyclist turns, he is always leaning at a certain angle. What is the angle value so that he turns
without overturning?
Please explain with expression to support your answer.
Table 3.
Rating Criteria of the Primitive Physics Problem 1 Bicycle turning
Abstraction Representation 1. Abstract bicycle and human as straight bar model.

1. Set the supporting force of the ground to be ;
2. Set the friction force to be ;
3. Set the mass of the bar to be ;
Assignment
4. Set the velocity of the bar to be ;
representation
5. Set the length of the bar to be ;
6. Set the turning radius to be ;
7. Set the angle between the straight bar and the ground to be .
1. Draw the mass center of the straight bar be located at
the middle point of the straight bar .
Image 2. Draw the angle between the straight bar and the
Representation ground;
mg
3. Draw the directions of supporting force, gravity, and
friction force.
Methodology 1. Model method (building a straight pole with people and bicycles);
Representation 2. Supposition method (supposing that the straight bar rotates around the mass center of the straight bar).
1. Balance between two generalized forces;
Physics
2. The formula of centripetal force;
Representation
3. The moment balance principle.
1. Supporting force and gravity balance in the vertical direction: ;
Mathematics 2. In the horizontal direction, friction acts as a centripetal force: ;

Representation 3. The moment sum of friction force and supporting force to the center of mass is zero: ;
4. , so, .
321
(pp. 305-324) ISSN 2538–7138 /Online/
The Primitive Physics Problem 2—”Vasa” capsizing

Problem: On August 10, 1682, in Stockholm, Sweden, the largest warship, Vasa, was making its maiden
voyage. When a breeze came, it capsized and sank! It’s been a mystery for hundreds of years. On April 24, 1961,
the ancient warship, which had been sleeping for hundreds of years, was finally salvaged, and the mystery of the
wreck was revealed.
Please use your physical knowledge to quantitatively explain the mystery of the “vasa” capsizing.
Table 4
Rating Criteria of the Primitive Physics Problem 2 “Vasa” capsizing
Abstraction 1. Ignore wind-force;

Representation 2. Abstract Vasa ship as a rigid body with a certain mass.
1. Set the gravity of a ship to be ;
2. Set the buoyancy of a ship to be ;
3. Set the inclination of the ship to be ;
Assignment 4. Set the intersection of the ship’s axis of symmetry and the water surface to be ;
representation 5. Set the center of buoyancy at point located on the ship’s symmetry axis and set the distance from the point
to be ;
6. Set the center of gravity at point located on the ship’s symmetry axis and set the distance from the point
to be .
1. Draw the gravity of a ship;
F
2. Draw the buoyancy of a ship;
3. Draw the inclination of the ship;
4. Draw the axis of a ship. θ
Image
5. Draw the ship balance situation: center of gravity is
Representation higher than the center of buoyancy
mg
1. Model method (building vasa into a rectangular object with symmetrical shape);
Methodology
Representation 2. Hypothetical method (assuming that the vasa rotates around the intersection of the axis of symmetry and the
water surface).
Physics 1. Principle of two forces balance;
Representation 2. Principle of moment balance.
1. The gravity and buoyancy of a ship are equal when a ship floats on the water: ;
2. The moment of gravity ship received: ;
Mathematics 3. The moment of buoyancy ship received: ;
Representation 4. The resultant moment ship received:
5. , , the center of buoyancy of the vasa was above the center of gravity, the
moment on the vasa does not provide the restoring moment, so the vasa capsized.
The Primitive Physics Problem 3—Basketball landing

Problem: A basketball falls freely from a certain height, bumps into the ground, then bounces, rises to a certain
height, and then falls freely, then bounces, falls again and again until the basketball is still.
Please deduce the expression of basketball acceleration changing with time for the whole process, and draw
the image of basketball acceleration changing with time.
322
ISSN 2538–7138 /Online/ (pp. 305-324)
Table 5.
Rating Criteria of the Primitive Physics Problem 3 Basketball landing
Abstraction 1. Ignore air resistance;

Representation 2. Abstract basketball as an elastic body with a certain mass
1. Set the quality of basketball to be ;
2. Set the spring force of basketball during touchdown to be ;
3. Set the equivalent elastic stiffness coefficient of basketball
Assignment to be ;
representation
4. Set the shape variable of basketball to be ;
5. Set the acceleration of basketball to be ;
6. Set the time to be .
1. Draw a straight line parallel to the -axis, the acceleration of basketball in the air remains constant;
Image 2. Draw the change of acceleration in the process of deformation and compression of basketball;
Representation 3. Draw the change of acceleration in the process of recovery;
4. Draw the peak of the line gradually moves up but does not exceed the horizontal axis.
Methodology 1. Hypothesis method(assuming that the elastic force changes linearly with time);
Representation 2. Model method (building a basketball into a spring model).
Physics 1. Hooke's law;
Representation 2. Newton's second law.
1. Basketball in the air only gravity, constant acceleration: g.
2. In the process of contact with the ground, the elasticity of basketball: ;
3. Then assuming that the basketball shape variable changes linearly with time, then in the compression
phase: , ;
Mathematics
In the recovery phase , ;
Representation
4. In the compression phase, the elasticity of basketball: ,
according to Newton’s second law: ; so ;
5. In the recovery phase, the elasticity of basketball: , according to Newton’s second law:
, so .
323
(pp. 305-324) ISSN 2538–7138 /Online/
Received: January 28, 2022 Revised: March 15, 2022 Accepted: April 02, 2022
Cite as: Xing, H., Zhai, Y., Han, S., Zhao, Y., Gong, W., Wang, Y., Han, J., & Liu, Q. (2022). The measuring instrument of primitive
physics problem for upper-secondary school students: Compilation and exploration. Journal of Baltic Science Education, 21(2),
305-324. https://doi.org/10.33225/jbse/22.21.305
Hongjun Xing PhD, Professor, College of Teacher Education, Capital Normal

(Corresponding author) University,105 West Third Ring Road North, Haidian District, Beijing,
100048 China.
E-mail: xhjcnu1960@sina.com
Website: https://jsjy.cnu.edu.cn/info/1071/1830.htm
Yanfang Zhai PhD Student, College of Education, Capital Normal University,105 West
(Co-corresponding author) Third Ring Road North, Haidian District, Beijing, 100048 China.
E-mail: 2200401017@cnu.edu.cn
Sisi Han PhD, Associate Professor, College of Teacher Education, Capital Normal
University,105 West Third Ring Road North, Haidian District, Beijing,
100048 China.
Website: https://jsjy.cnu.edu.cn/info/1071/2002.htm
Yuping Zhao PhD Student, College of Education, Capital Normal University,105 West
Third Ring Road North, Haidian District, Beijing, 100048 China.
Wenhui Gong PhD Student, College of Education, Capital Normal University,105 West
Yuting Wang PhD Student, College of Education, Capital Normal University,105 West
Jing Han PhD, Research Scientist, Physics Department, The Ohio State University,
USA.
Qiaoyi Liu PhD Student, Physics Department, The Ohio State University, USA.
324
COMPARING THE EFFECTS OF

MODELLING AND ANALOGY
ON HIGH SCHOOL STUDENTS’ ISSN 1648-3898 /Print/
CONTENT UNDERSTANDING ISSN 2538-7138 /Online/
AND TRANSFERABILITY: THE

CASE OF ATOMIC STRUCTURE Abstract. Analogies and modelling have
been developed and applied in learning
and teaching science to facilitate students’
understanding of abstract concepts, such
as atomic structure. Considering few
studies focus on comparing the effects of
Song Xue, two teaching strategies—analogy-based
Daner Sun, teaching (ABT) and modelling-based teach-
Liying Zhu, ing (MBT)—this study aims to compare
the effects of ABT and MBT on high school
Hui-Wen Huang, students’ content understanding and trans-
Keith Topping ferability of atomic concepts in science.
Implementing a quasi-experimental design
with pre-post-delayed tests, the study
compared learning outcomes achieved by
the MBT group (N = 68) and the ABT group
(N = 69). The results showed both MBT
Introduction
and ABT could improve students’ content
understanding and promote transferabil-
Atomic concepts have been one of the core concepts in science curri- ity. However, the MBT group significantly
cula (Muştu & Özkan, 2019). The concept of atomic structure, in particular, is outperformed the ABT group in terms of
considered fundamental for understanding the nature of matter at secondary generating initial models and overall trans-
and university levels (McKagan et al., 2008; Park & Light, 2009; Taber, 2003). ferability. Although there was no difference
However, many studies have revealed that students usually find the concept in content understanding, or near or far
of atomic structure difficult to attain, and instead, they develop alternative transferability, at post-test between the two
conceptions, as the atom is a kind of matter in the sub-microscopic world groups, the MBT group maintained more
whose structure cannot be observed directly. Moreover, students only receive extended memory of atomic structure on
the delayed post-test. Moreover, qualitative
limited opportunities to explore atomic structures at the sub-microscopic
analysis of students’ drawings of atomic
level, and few strategies have been adopted to help them learn the charac-
models revealed that both groups were able
teristics of atomic models (Sunyono & Sudjarwo, 2018). to develop and transfer their models, but
Among the existing teaching strategies, analogy-based teaching (ABT) inadequate scientific knowledge affected
and modelling-based teaching (MBT) are two effective instructional ap- the quality of the transfer product. These
proaches for facilitating students’ understanding of abstract and intangible findings have implications for designing
concepts, recommended by numerous empirical studies (e.g., Brown & and implementing instructional approach-
Salter, 2010; Khan, 2007). However, there were observations that ABT and es that leverage analogy and modelling in
MBT were sometimes inappropriately designed and implemented without the science class.
achieving the intended outcomes. For example, analogies were presented Keywords: analogy-based teaching,
statically and didactically to students as textbook or lecture content that atomic concepts, modelling-based teach-
ing, science education
failed to engage the students, or were used uncritically and thus led to a
less accurate understanding (Cin, 2007; May et al., 2006). Additionally, MBT Song Xue
was criticised as it was “too challenging” for learners and teachers in the University of Dundee, United Kingdom
classroom (Settlage, 2007). To further unfold the mechanisms behind these Daner Sun
Education University of Hong Kong, China
contradictions, further research is needed to explore the effects of ABT and Liying Zhu
MBT on students’ learning. Zhejiang Normal University, China
Since analogies and modelling can transform knowledge and provide Hui-Wen Huang
Huizhou University, China
predictive and explanatory capability for making sense of the familiar and Keith Topping
unfamiliar (Khan, 2007), it is intriguing to investigate how teaching strate- University of Dundee, United Kingdom
325

(pp. 325-341) ISSN 2538–7138 /Online/
gies can reinforce such transferability effectively. According to Dori and Sasson (2013), there is a gap between
theoretical and empirical studies, due to a lack of emphasis on the importance of near and far transfer in educa-
tional processes. Particularly, although far transfer is more associated with deep learning and long-term retention
than near transfer, it is much more difficult to achieve (Kassai et al., 2019; Sala et al., 2019). Accordingly, the main
purpose of the present study is to explore how to improve high school students’ transferability through the use of
two teaching strategies: ABT and MBT.
Analogies in the Teaching of Atomic Concepts
According to the assumption of Sarantopoulos and Tsaparlis (2004), an analogy is a system of relations between
parts of two particular domains: the analogue (the familiar domain) and the target (the unfamiliar one). An analogy
involves the identification and conversion of relational information from the analogue to the target, which consists
of finding the similarities between the two domains (Vosniadou, 1988). According to related studies (Buckley et al.,
2004; Else et al., 2008; Gobert & Buckley, 2000), ABT is a teaching approach in which teachers help students conduct
analogies for understanding the specific content by recognising the analogy’s coherence with the target concept.
ABT often assists students in comprehending perplexing concepts and visualising abstractions by representations
in different forms, such as texts, pictures, videos, verbal examples, and computer simulations (Trey & Khan, 2008).
The commonly used analogy of the atomic structure is the solar system. In this case, the spatial and dynamic
features of the sun and the surrounding planets are analogous to the atomic nucleus and electrons (Taber, 2013).
Goh et al. (1994) proposed other useful analogies of atomic structure. For example, determining the location of a
student based on his or her schedule is comparable to the identification of an atomic orbital; the way passengers
occupy seats on a bus provides a good analogy of Hund’s rule; and an analogy is drawn between one’s address
(that includes the name, unit number, house number, and street name together), and the four quantum numbers
(that include the spin quantum number, magnetic quantum number, subsidiary quantum number, and principal
quantum number). These analogies illustrate and visualise the key features of atomic concepts with reference to
real-world scenarios (Devecioglu-Kaymakci, 2016), and draw on students’ prior knowledge and experience to en-
able their understanding. Inverse analogies can be productive, too: using atomic structure as the familiar domain,
Slabin (2017) applied valence bond and molecular orbital theories to explain formation of educationally valuable
chemical eponyms (the unfamiliar domain). As confirmed by Dangur et al. (2014), engaging diverse analogies and
encouraging connections between scientific concepts and daily life experiences can improve students’ understand-
ing of complex concepts (e.g., quantum mechanical concepts).
Leveraging analogies may improve the instruction of abstract and sub-microscopic concepts. However, it
should be noted that there are no perfect analogies that could completely represent the target concepts (Taber,
2001), and stopping at a literal understanding of an analogy may result in limited comprehension. Taber (2005)
reported that students are likely to equate quantum mechanical spin to the spinning of a basketball or a planet.
Such a superficial interpretation may lead to misconceptions. The generation of mixed/hybrid atomic models (e.g.,
a localised electron included in the “surface” of the electron cloud) also indicated inadequate understanding of
the target concept (Dangur et al., 2014; Tsaparlis & Papaphotis, 2009). To eliminate these misunderstandings and
maximise the efficacy of analogies, a more precise design of ABT is needed.
Modelling-based Teaching
Modelling builds knowledge through an ensemble of processes involving building, critiquing, and modify-
ing models (Campbell et al., 2012; Gobert & Buckley, 2000; Schwarz et al., 2009). Modelling-based teaching (MBT)
provides an effective way for learning key scientific concepts and unobservable phenomena through modelling
processes (Al-Balushi & Al-Hajri, 2014; Chittleborough & Treagust, 2007; Maia & Justi, 2009; Nakiboglu, 2008). Having
students develop integrated mental models that they can employ in a dynamic and recursive process of model
development is the objective of MBT (Gilbert & Justi, 2016, p.58).
An MBT framework comprised of multiple and complex stages (involving elaboration, testing, reformulation,
and discussion of the proposed models) was proposed for advancing students’ learning about the main qualitative
aspects of chemical equilibrium in Maia’s study (Maia & Justi, 2009). In this framework, modelling was considered a
non-linear process. Khan (2007) introduced a productive modelling cycle that contains model generation, evaluation,
and modification (GRM). Translating GEM modelling cycle into the university chemistry classroom has contributed
to students’ understanding of molecular structure and intermolecular forces.
326
ISSN 1648–3898 /Print/ COMPARING THE EFFECTS OF MODELLING AND ANALOGY ON HIGH SCHOOL STUDENTS’
ISSN 2538–7138 /Online/ (pp. 325-341)
In addition, several studies revealed students’ outstanding performance on assessments in relation to their
non-modelling counterparts (Campbell et al., 2011; Dukerich, 2015; Settlage, 2007). However, teachers were uneasy
about controlling MBT in their classes as it was too time demanding (Campbell et al., 2011; 2012). Since MBT is a
complex approach that can be manifested in many disparate forms and processes, it is beneficial to establish a
diverse set of cases and clarify how MBT is implemented in chemistry classrooms to support teachers who incor-
porate modelling pedagogies in their classrooms.
Transferability in ABT and MBT Classes
To learn from an analogy, learners must first understand why the analogue (the familiar domain) behaves the
way it does, and then transfer that understanding to the unfamiliar domain (Gray & Holyoak, 2021). Most analogies
relate to empirical phenomena, including the key functional relations that involve causes and their consequences
(Gray & Holyoak, 2021). The analogy presents a new explanation of why various phenomena occur by transfer-
ring knowledge about causal relations, which enables further transfers to new situations. MBT is a constructivist
teaching approach, and learners construct mental models during their interaction with environments, artefacts,
technology, and communities (Johnson-Laird, 1983). According to mental model theory, a mental model is defined
as an internal cognitive representation or an interconnected concept that correlates in some way to the external
structure that it represents (Gentner & Stevens, 1983). Mental models are incomplete and unstable, and they
can also be analogical, partial, and fragmentary (Greca & Moreira, 2000; Norman, 1983). In a dynamic situation,
learners often construct mental models to understand the world and solve problems (Seel, 2017). Successful and
unsuccessful comprehension involves processes relevant to mental model development or transfer (Rapp, 2005).
Further, educators intend to pass on knowledge and skills, which will apply in future and unknown contexts
to the students (Sasson & Dori, 2015). Regarding this, transferability takes an essential role in ABT and MBT classes.
Transferability refers to the ability of learners to retain knowledge and skills and apply them in a new learning
environment (Dori & Sasson, 2013; Salomon & Globerson, 1987). Dori and Sasson (2013) proposed a framework
of transferability and indicated that near transfer occurs when the learning situation is comparable to the previ-
ous learning situation; far transfer, on the other hand, occurs when a student is required to perform in a new and
unfamiliar learning context. They further defined three main characteristics of a transfer task: (a) task distance
[TD], (b) interdisciplinarity [I], and (c) skill set [S] (detailed explanations of these terms can be found in Dori and
Sasson’s study in 2015). However, near and far transfers are ambiguous, and the term “closeness” is vaguely defined
and very subjective (Perkins & Salomon, 1992). Whether near or far, the assessment of transfer heavily depends
on one’s understanding of learning (Lave, 1988; Marton, 2006) and the assessment instruments used (Bransford &
Schwartz, 1999; Schwartz et al., 2005). In addition, Sasson and Dori (2015) constructed assignments and evaluated
ninth-grade students’ transfer skills, finding that boys’ near transferability was much higher than for girls. It was
suggested that effective teaching intentions should be further explored to develop learners’ transferability in the
science classroom.
Several studies have explored transferability between different external representations, the effect of hypoth-
esis-testing skills on the transfer of learning, and the impact of learning models on the development of transfer-
ability in different contexts. However, few empirical studies have optimally explored how students transfer their
mental model in the chemistry class. This study helps to fill this gap, by demonstrating the effect of two different
teaching strategies (ABT and MBT) on students’ transferability in science.
Research Purpose and Questions
This study examined the effects of two teaching strategies ABT and MBT on high school students’ content
understanding of atomic structure and transferability. The study seeks to answer the following questions:
(1) Which teaching strategy enhances students’ content understanding and transferability in the context
of the atomic structure?
(2) Are there significant differences in students’ content understanding and transferability between the
two groups after experiencing the teaching interventions?
(3) How do both groups develop and transfer their mental models after receiving different teaching strate-
gies?
327

(pp. 325-341) ISSN 2538–7138 /Online/
Context
Curricular Context: The lessons designed and implemented were from the same module of atomic structure
prescribed in the local chemistry curriculum. According to the learning objectives, the students were expected
to learn about the history of atom discovery and understand the concepts of atomic structure, spectrum, atomic
orbital, electronic configuration, the periodic table, and chemical bond formation. In the local context, educational
policymakers and practitioners advocated the development of the students’ modelling skills. The National Acad-
emy for Educational Research (Ministry of Education in Taiwan, 2018) had specific requirements for modelling at
the high school level. Taking the chemistry subject as an example, 10th graders should have the ability to “build
models based on scientific problems or through group discussions and be able to use, for example, ‘analogous or
abstract’ representations to describe a systematic scientific phenomenon, and then understand the limitations of
the model” (p.32).
The design and implementation of ABT and MBT: Both ABT and MBT groups were engaged in the same analogical
activities guided by a common set of instructional worksheets, a feature for ensuring common practice between
conditions. The analogical examples consisted of a set of forms presenting the different atomic concepts, in the
form of text, pictures, verbal examples, and computer simulations. There were two main differences between the
two teaching strategies. One was that analogies were provided by the teacher in the ABT group (teacher generates
analogy for students), while the students in the MBT group (teacher and student co-construct analogy) also had to
recall analogies based on the teachers’ enquiries. Another difference was the modelling process was integrated into
analogical activities in the MBT group, while there was no modelling process in analogical activities in the ABT group.
Referring to the principles of the analogical approach which were proposed by Gray and Holyoak (2021), the
teacher selected analogies (solar system) to describe scientific concepts (the atomic structure) based on their prior
knowledge. The students then engaged in interactive dialogue with their teacher, explaining the corresponding
relations between the analogy and the target concepts. After a comparison between analogy and target concepts,
the students realised the limitations of the previous analogies and the teacher promoted a new analogy (climbing
ladders) for better representing the target. Finally, the students summarised all of the analogies and clarified the
relationships between the analogies and the atomic concepts. The teacher encouraged the students to generate
inferences founded on basic knowledge of the atomic structure, for example, a question was asked: do you think
that two different atoms and their structure can be combined to form a molecule? The ABT was derived from the
students’ existing ideas and the teacher further developed the efficacy of analogies, but the students learned with
analogies and generated models without the process of modelling. In this way, the students did not have owner-
ship of the analogies.
In the MBT group, the students and the teacher contributed the ideas, and the analogies were generated
building on the students’ existing knowledge, which were guided by the scientific modelling GEM (Generate,
Evaluate, and Modify) approach (Khan, 2007). It is important to make the distinction that not all analogies are
models; analogies are the medium through which a model is expressed (Lee et al., 2017). In this study, the GEM
indicated that students were able to develop mental models (understanding of atomic concepts). These mental
models were represented by analogies or drawings which showed aspects of scientific objects. Below are illustra-
tions of GEM approach.
In the model generation stage (Generate), the teacher helped the students develop an initial model of atomic
structure. The basic concept that “An atom is composed of a nucleus and extra-nuclear electrons, and that electrons
run around and surround the nucleus” was first introduced. The students were then guided to recall their prior
knowledge and experiences that could relate to this concept through questioning. Then, the teacher guided the
students to select the most appropriate analogy and asked them to explain the reason for this selection. The solar
system was considered as a plausible analogy for this concept, the Rutherford Atomic Model. An explanation that
the sun represented the atomic nuclei, the encircling planets represented electrons, and their orbits represented
the atomic orbital was made after teacher-and-student discussion.
In the model evaluation stage (Evaluate), more advanced knowledge was introduced, that was critical to
understanding the concept of the atomic structure (e.g., electron movement, energy change in electronic transi-
tion), and the students were encouraged to evaluate whether the initial solar system analogy could accommodate
such additional information. They thought about how energy would change when an electron moved from a layer
328
ISSN 2538–7138 /Online/ (pp. 325-341)
of lower electronic energy to the one of higher electronic energy. With a revised understanding, students would
consider looking for a new analogy to compare with the initial model that had been deemed inadequate.
During the model modification stage (Modify), the initial model was modified and improved based on the
limitations identified in the evaluating phase. Then the students applied the elaborated model, the Bohr atomic
model (with a new analogy if possible), to accommodate new situations (e.g., energy change as electrons move
from atomic orbital K to L). The new scenario would provoke another round of model evaluation and modification,
and thus build up the students’ mental models and conceptual understanding.
Research Design
This study adopted a quasi-experimental design (Plomp, 2013), with a pre-post-delayed test design. In total,
there were eight instructional sessions for the ABT class and nine for the MBT class. To guarantee that both groups
had the same class time, the ABT group had one more session to review atomic structure concepts and related
tasks (eight teaching sessions and one session for review). The implementation of these two instructional methods
continued for three weeks. There were three 55-min lessons each week. The total time for teaching intervention
was 8 hr 25 min hr. The pre-test and post-test were administered to examine the students’ content understanding
and transferability. Two weeks after the post-test, a delayed post-test was conducted to evaluate the sustainability
of the students’ understanding of the target concepts and compare the two groups in retention of content under-
standing of atomic structure (Mauro & Furman, 2016). The time interval of two weeks was to avoid the effects of
another chapter in the local curriculum, Molecular, on the students’ content understanding of atomic structure. To
explore the students’ transferability, a qualitative analysis of student-generated drawings of atomic models was
performed, to reflect how mental models were developed and transferred based on different transfer scenarios
(Gobert & Pallant, 2004; Sun & Looi, 2013). This was intended to provide a more comprehensive, detailed descrip-
tion of the level of content understanding and model development achieved by the two groups.
Participants
Altogether, four classes of 10th graders (n = 137, age 15-16 years old) from a school in Taiwan participated
in this study (35% male and 65% female). The four classes were assigned randomly to the MBT group (N = 68, 2
classes) and the ABT group (N = 69, 2 classes). To ensure the sample size was appropriate, G*Power 3 was used
(Mayr et al., 2007). Given an effect size of 0.4 with 80% power in a one-way ANCOVA (2 groups, α = 0.05, df = 1, and
1 covariate), the suggested a total sample size of 52. Therefore, the sample size of this study was satisfactory. This
study followed the ethical considerations outlined by Taber (Taber, 2014). The administrators and teachers of the
selected classes were contacted, and consent was gained. Students were informed about the purpose of the study
before the intervention, but participation was voluntary and anonymous. By the end of the study, no students had
refused to participate.
In the study, two participating teachers discussed, designed, and implemented the lesson plans in close
collaboration with the researchers, to control interference from teacher variables such as the competence of the
teachers (Cumming & De Miranda, 2012). Specifically, a male teacher A with six years of teaching experience in
chemistry taught the MBT group, and a female teacher B with eight years of teaching experience in chemistry
implemented ABT in another group.
Instruments
The test items were developed in relation to the local chemistry curriculum for secondary schooling (Ministry
of Education of Taiwan, 2018). The pre-test and post-test were composed of two parts, including Part I true/false
questions and Part II open-ended questions. Pre and post-test had the same questions, but the post-test had a
minor difference in the description and the order of items. A total of eight items with a true or false question format
were designed for Part I, which tested the students’ content understanding of atom composition, the electronic
and structural properties of an atom, electronic cloud, and the element periodic law. In addition, the students were
requested to provide justifications for their choices. Regarding Part II, the three items of mental model transfer-
ability, based on the transfer framework (Dori & Sasson, 2013), were defined as an initial mental model, for near
transfer and far transfer. In this part, the students were required to respond to three transfer items by drawing.
329

(pp. 325-341) ISSN 2538–7138 /Online/
The students’ illustrations could make their mental models explicit and provide an assessment indicator for model
development and content understanding (Larkin & Simon, 1987). Samples of Part I and II are presented in Tables 1
and 2. A delayed post-test with twenty multiple-choice questions for evaluating the students’ retention of content
understanding of the atomic structure was administered two weeks later.
Table 1
Example Items for Evaluating Content Understanding
Test Item
As shown in the figure, the outermost electrons have higher energy than the in-
nermost electrons.
Pre-test
•• True. Please provide the reason: ____
•• False. Please provide the reason: ____
As shown in the figure, the outermost electrons have lower energy than the in-
nermost electrons.
Post-test
•• True. Please provide the reason: ____
•• False. Please provide the reason: ____
As shown in the figure, which is the most stable layer?
•• Left
Delayed post-test •• Middle
•• Right
•• Not sure
Note: These items were used for evaluating students’ content understanding of the electron orbitals energy such that: the in-
nermost electron, near the nucleus, has the lowest energy and is also the most stable. On the contrary, the higher the energy
(farther away from the nucleus) the outermost electron has, the more unstable and the easier for electronic transitions.
Table 2 demonstrates examples of transfer items. These three items dealt with how students applied their
initial model (the atomic structure of a general atom) to a near transfer question (atomic structure with specific ele-
ments) and a far transfer question about a molecular structure. Referring to the three transfer attributes framework
(Dori & Sasson, 2013), the first item was designed to assess if students could recognise the qualitative features of
the atomic structure (e.g., the composition of an atom). Regarding the near transfer item, it instructed students to
apply knowledge of atomic structure to a specific element’s atomic structure and to identify the correct connec-
tions between protons, neutrons, and electrons. It was similar to the first mental model item based on the same
scientific concept, and this question featured a low degree of all three transfer framework attributes (TD, I, and
S). The far transfer items required students to apply knowledge from the atomic structure to a different concept,
molecular structure, and to distinguish how the orbitals overlapped and how the valence electrons were shared
across the hydrogen and oxygen atoms. Hence, this item featured all three transfer framework attributes to a high
degree (TD, I, and S).
Table 2
Pre/Post-test Part 2 Model Development Items and Sample Responses
Question Learning objective Sample response
1. Please draw a schematic diagram of the atomic Students were expected to recognise the qualitative
e- e-
structure features of atomic structure (e.g., the composition of an
atom).
2. Considering the drawing you created in the Students were expected to acknowledge the quantitative
last item, please draw a schematic diagram features of the atomic structure (e.g., the exact number
of the distribution of electrons in an oxygen of electrons in each energy layer)
atom ( )
330
ISSN 2538–7138 /Online/ (pp. 325-341)
Question Learning objective Sample response
3. Considering the drawings you created in the last Students were expected to apply the model developed
two items, please draw the electron arrangement to new situations and evaluate its fitness (e.g., designing
in an H2O molecule a model for the compound of H2O based on the atomic
structures developed for the elements of H and O)
Scoring for content understanding. The students’ understanding of the target concept was reflected in their
test scores. The full score for each “true or false” question in the pre-test and post-test was 3 points, and a student’s
answer would be awarded 1, 2, or 3 points based on their selections and justification. If a student selected the cor-
rect answer and justified it well, 3 points would be awarded; If only partial explanations were provided, 2 points
would be awarded; and for incorrect or correct answers, without explanations or with wrong explanations, only
1 point would be awarded. For multiple-choice questions in the delayed post-test, the full score was 5 points. No
points would be awarded if the response was wrong or missing. The test papers were marked by two chemistry
teachers who achieved 99% consistency in scoring. The student test scores were analysed using SPSS 24.0.
Scoring for transferability. The scoring rubric stemmed from Supasorn and Promarak (2015), who used it
to classify three categories of student conceptions (good conception, alternate conception, and misconception)
in comparison to the scientific consensus. The answers to the three open-ended questions were graded on a
scale of one-to-three, where one meant “wrong answer”, two meant “partially correct answer”, and three meant
“correct answer”. If the illustration was provided correctly and displayed the relevant components (the nucleus
and electrons), relations (situationally or mathematically), and numerical features (e.g., the number of electrons)
of an atomic model, 3 points would be given. If the illustration only showed a partially accurate model, correctly
displaying the relevant components and relations, but with numerical features missing or being incorrect, 2 points
would be given. Only 1 point would be given if the illustration noted an inaccurate model, or the question was left
unanswered. For examples of student illustrations that were awarded 3 points, 2 points, and 1 point respectively,
please refer to Fig. 1. The scoring rubric of this item only focused on whether students could draw an accurate
atomic model including the essential elements of electrons and nuclei (protons or neutrons), without consideration
if it was a scientific model. This is the reason for giving 3 points in Fig. 1. Although this student did not provide a
scientific model (the Electronic Cloud Model), the drawing includes the required atomic components, and so it
was given 3 points. All of the students’ models were reviewed by a chemistry educator who coded 10% (n = 14)
of the data to establish intercoder reliability with the first author. Intercoder reliability was calculated as 95% and
reached 100% after discussion (Patton, 2002).
Figure 1
Examples of Student Illustrations of an Atom in the First Item of the Initial Model
3-point illustration (correctly displayed the 2-point illustration (Only partial components
1-point illustration (no relevant components).
relevant components). are presented. Electrons are missing.
Two researchers designed and administered the pre-and post-tests. A pilot of the pre- and post-tests was
carried out involving a class of 39 students who had learnt the topic. This was intended to potentially modify the
instrument. The test items and questions were revised based on the students’ responses. The delayed post-test
was generated collectively by three experienced chemistry teachers who had taught chemistry for 10 years in this
school. All of the test items were peer-reviewed by two chemistry teachers and two professors in science educa-
tion to ensure their validity and legitimacy. The Cronbach Alpha coefficient of the pre-test was 0.81, the post-test
331

(pp. 325-341) ISSN 2538–7138 /Online/
was 0.84, and the delayed post-test was 0.90. The results of the pilot study indicated a good degree of reliability
in the instruments (Taber, 2018).
Data Analysis
Before conducting paired sample t-tests and an ANOVA analysis, the Shapiro–Wilk test had demonstrated
normal distribution of the pre-, post-, and delayed-tests results (p > 0.05). There was no significant variance be-
tween the two groups in terms of pre-test scores (F = 1.202, p = 0.275). For exploring the differences between the
variables of students’ content understanding and transferability before and after teaching interventions, a two-way
ANOVA analysis comparing the two different methods (ABT and MBT) and the time of assessment (pre-test and
post-test) was performed. Then paired sample t-tests were carried out to examine whether the students’ content
understanding, and transferability had improved significantly between the groups. Regarding transferability, it
was categorised according to three scales: initial model, near transfer, and far transfer.
The paired sample t-test generated Cohen’s d effect size, suggesting small, medium, and large effect sizes
would be 0.20, 0.50, and 0.80 respectively. The effect size when the ANOVA analysis was conducted generated
partial eta squared, considering values of 0.01 as a small effect, 0.06 as a medium effect, and 0.14 as a large effect
(Bryman & Cramer, 2009).
The qualitative method for analysing the students’ drawings was in three transfer items, by using classical
content analysis for a priori codes of correct, partial-accurate, and inaccurate models (Patton, 2001). Data were
analysed for patterns within the two conditions, for how the students developed and transferred their models.
The qualitative analysis involved a continual process of data coding, displaying, and verification to identify themes
within the drawings and their comments that revealed the characteristics of their mental model (Patton, 2001). Two
independent ratters coded three items in the pre- and post-tests. The inter-rater reliability was 92% and Cohen’s
Kappa was 0.84. Inconsistent codes were discussed and resolved.
Research Results
Students’ Performance on Content Understanding
First, a paired-sample t-test was administered to examine whether the two groups of students had improved
their learning outcomes after the instruction. The results revealed that students, in both the MBT group and the
ABT group, performed significantly better in the post-test (Table 3). The effect sizes (Cohen’s d) ranged from 0.34 to
1.51, indicating that while the two instructional conditions showed large positive effects on the students’ content
understanding, the initial model and overall transferability had a medium effect on near transfer and had a small
effect on far transfer. This provided evidence for the effectiveness of the two instructional approaches.
Table 3
Paired Sample t-test Results: Pre-test and Post-test
MBT Group ABT Group
Test t p Cohen’s d t p Cohen’s d
Content Pre-test
-1.339 <.001 0.92 -.889 < . 001 0.84
understanding Post-test
Pre-test
Initial models -8.492 < .001 1.51 - 8.804 < . 001 0.89
Post-test
Pre-test
Near transfer -3.780 < .001 0.62 -4.331 < . 001 0.51
Post-test
Pre-test
Far transfer -2.711 < .001 0.43 -2.815 < . 001 0.34
Post-test
332
ISSN 2538–7138 /Online/ (pp. 325-341)
MBT Group ABT Group
Test t p Cohen’s d t p Cohen’s d
Pre-test
Overall -8.496 < .001 1.28 -6.682 < . 001 1.06
Post-test
ANOVA revealed no significant differences in the students’ post-test scores of content understanding between
the two groups (F (1,135) = 2.5, p = .12). However, Table 4 shows a significant statistical difference between the two
groups (F (1,135) = 3.09, p = .00 < .001, partial eta squared η2 = .022) in terms of overall transferability. Regarding
the comparison of the two groups’ performance on the model transferability test, students in the MBT group, in
general, obtained higher scores than their counterparts, but these differences were not statistically significant in
terms of near transfer and far transfer. The only significant distinction between the two groups in the post-test was
in the item about the initial model, where the MBT group obtained higher scores than the ABT group (F (1,136)
= 8.34, p = .01 < .05, η2 = .06). Even though the average scores that the MBT group achieved in other items were
slightly higher than the ABT group, these differences were not statistically different.
For further investigation of the retention impact of MBT and ABT, ANOVA was performed on the student
scores on content understanding in the follow-up test. The results further confirmed the positive effect of the MBT
approach. Students in the MBT group outperformed the ABT group F (1,136) = 1.83, p = .04. Compared to ABT, it
appeared MBT facilitated knowledge retention and long-term memory of the target concept.
Table 4
ANOVA Analysis Results for the Post-test and Delayed Post-test of the Two Groups
MAI Group AI Group

F p η2
M SD M SD
Content understanding 17.49 2.22 16.21 2.13 2.50 .12 .03

Initial model 2.41 0.82 2.02 0.93 8.34 .01 .06
Near transfer 1.80 0.74 1.78 0.69 0.06 .81 .00
Far transfer 1.28 0.67 1.23 0.46 1.83 .17 .01
Overall 5.56 1.33 5.30 1.41 3.09 .00 .02
Delayed post-test
Content understanding 71.36 12.44 66.99 15.36 1.83 .04 .03
The Students’ Performance on Transfer Tasks
The qualitative analysis identified correct models, partial-accurate models, and inaccurate models from stu-
dents’ drawings in the post-test, by one-to-three scores respectively. Since it was found that the students had very
limited prior knowledge of model transferability in the given questions, the students’ drawings in the post-test
were mainly for assessing how the students’ models had progressed. Based on the scoring method introduced
above, the scores from the two groups, calculated in percentages, are presented in Fig. 2.
333

(pp. 325-341) ISSN 2538–7138 /Online/
Figure 2
Transfer Scores in Post-test
Initial model. For the first item, the students were expected to generate an initial model of atomic structure
that included all the necessary components such as the nucleus and electrons. It was not necessary for the students
to manage to draw the scientific position between the nucleus and the electrons at this item. For example, the
students who wrote a Bohr model should get more points than the ones with the Dalton model. The quality of the
Bohr or quantum models of the atom, the most frequently accepted models, were examined from the students’
drawings. Examples of student illustrations in the post-test for generating initial models are provided in Fig. 3. As
observed, the students’ developed models at this stage were very similar, with most being of the Bohr atomic model
(n = 54 for MBT group, n = 37 for ABT group). This result showed that the students gained an enriched content
understanding of the atomic structure from the MBT.
Figure 3
Examples of the Student Illustrations in the Post-test: Generating Models
Thomson Model Rutherford Atomic Model Bohr Atomic Model Electronic Cloud Model
Thomson Model Rutherford Atomic Bohr Atomic Model Electronic Cloud Model
Model
The solar system was the most frequently adopted analogy. Overall, students in the MBT group outperformed
the ABT group. They generated more accurate models (62% vs. 43%) and more diverse types of mental models.
Besides the Bohr atomic model (the predominant model provided by the ABT group), the MBT group illustrated
the Rutherford atomic model, the Thomson atomic model, the Dalton atomic model, as well as the Electron cloud
model. The illustrations also unveiled the problems students encountered in understanding the target concept.
There were instances of alternative conceptions and misconceptions. Many students failed to grasp the idea of
electronic movement, and the positions of the electrons in the nucleus. Also, some of the students were not able
to identify the relationship between the number of electrons and protons.
Near transfer. In the second item, students were expected to transfer the model they had developed in the
334
ISSN 2538–7138 /Online/ (pp. 325-341)
first item to a new context (the specific atom), evaluating whether the model accurately depicted the quantita-
tive relationship between protons, neutrons, and electrons. In the pre-test responses, most students were unable
to describe the atomic model of a specific element (Oxygen - O2). Only 15% of the students generated a partially
correct model of O2. Most of the students had difficulties in understanding the relationships between the number
and location of protons and electrons.
In the post-test, 19% of students in the MBT group and 15% of students in the ABT group generated correct
models. Students in the MBT group preferred to apply and adapt the initial model constructed to the new situation.
Yet in the ABT group, the students preferred to drop the initial model and provide a new one in the new scenario.
Figure 4 displays two atomic models developed by Student 1 (S1) in the MBT group and S2 in the ABT group. In
the first item of the initial model, S1 developed a Bohr atomic model. When he noted the model was not suitable
for the new situation, he kept the original model but adapted the way the electrons were arranged. In contrast, S2
dropped the Bohr atomic model which was developed in the first item and proposed a Rutherford atomic model
to accommodate the situation.
Figure 4
Figure 4
Examples of Student Illustrations in the Post-test: Near Transfer
Examples of Student Illustrations in the Post-test: Near Transfer
Initial model Near transfer
S1
(MBT group)
S2
(ABT group)
In the post-test, most models provided were still partially correct. Among the different kinds of models gener-
ated, the Rutherford atomic model was the most common. This indicated the overall accessibility and acceptability
of the “solar system” analogy. Compared to the pre-test, the illustrations of the Rutherford atomic model provided
in the post-test were more accurate. Based on the student responses collected, the areas that would require further
instructional efforts were identified. Some students failed to visualise the electronic structures and Lewis structures;
some hesitated to interpret numerical symbols of an atomic sign (e.g., 16 and 8 in ), and some students had
difficulty in figuring out the correct number of electrons of an atom.
Far transfer. The third item was more challenging as the students were expected to consider different structures
of atomic models and combine them. Only 3% of the students in the MBT group generated correct models. And in
the ABT group, the percentage was even lower (1%). Students who could come up with a partially correct model
were very limited as well. Almost half of the MBT group (49%) and one-third of the ABT group (29%) illustrated a
partially correct model in the post-test. This observation echoed the frustrations the students expressed in their
written reflections. For the majority of the students, it was indeed difficult for them to develop a model as required.
A lot of them failed to distinguish between electronic structures and Lewis structures. And even though some
students could figure out the atomic structure of the elements of oxygen and hydrogen respectively, structuring
a water molecule was still beyond them.
Based on the illustrations provided at this stage, the trajectory of model transferability could be outlined.
Taking the modelling practices of Student S3 in the MBT group and Student S4 in the ABT group as an example
(see Fig. 5), S3 provided an initial model based on the analogy of the “solar system”. In this model, the components
of the nucleus, electrons, and atomic orbital were provided and correctly positioned. In the following stage of the
near transfer, S3 further shifted to a Rutherford atomic model, with each component quantitatively specified. At
the 3rd stage of the far transfer, S3 correctly illustrated the basic structure of a water molecule that included the
335

(pp. 325-341) ISSN 2538–7138 /Online/
elements of hydrogen and oxygen. She also tried to combine the two elements and structured them together,
however, as she did not include valence shell electrons, the model she generated was still only partially correct.
Similarly, S4 of the ABT group provided an initial model based on the “solar system” analogy that incorporated
most of the essential elements, with only the atomic orbital missing. In the near transfer item, S4 visualised the
main structure of an oxygen atom and labelled the components (e.g., electrons, protons, and the electronic charge).
As for far transfer, the illustration provided by S4 included the key elements but with other components missing.
The processes of model transferability depicted above were typical. Such observations reflected the challenges
and difficulties the students encountered as they tried to develop and further elaborate their mental models of
the atomic structure.
Figure 5
Examples of the Student Illustrations in the Post-test: Far Transfer
Item 1 Item 2 Item 3
S3
(MBT group)
S4
(ABT group)
Discussion
The first two research questions in this study were to examine the difference between the MBT and ABT on the
students’ learning outcomes, including content understanding and transferability. A quantitative analysis between
pre- and post-test results showed that generally, both instructional approaches could significantly facilitate a stu-
dent’s content understanding and transferability of sub-microscopic concepts, with a large effect size. This finding
revealed the value of the two teaching strategies for facilitating the understanding of abstract concepts (Çalik &
Ayas, 2005; Dilber, 2012; Orgill et al., 2015). When comparing the post-test results between the two groups, there
was a difference in performance of generating initial models and overall transferability with a small effect in favour
of the MBT group, but no noticeable difference was found in content understanding and the other two transfer
items. This might imply that MBT benefits students’ transferability more than analogical activities without modelling.
It was noted earlier that far transfer is difficult to achieve, so this study contributes significantly to the literature.
Following the GEM cycle of modelling instruction, the students in the MBT group developed an enriched un-
derstanding of the atomic structure model, and their modelling practices resulted in more accurate and complete
models, This finding is consistent with the established literature that modelling-based instruction is a more effective
way of promoting the development of scientific process skills, such as reasoning and explanation (e.g., Aktamis
& Ergin, 2008; Baze, 2017; Liguori, 2014; Wen et al., 2020). However, these results do not seem to be aligned with
previous studies that noted that scaffolded modelling conditions could support students in developing a better
conceptual understanding of science (Al-Balushi & Al-Hajri, 2014; Chittleborough & Treagust, 2007; Maia & Justi,
2009; Nakiboglu, 2008;).
Although there was no significant difference in content understanding at post-test, the delayed post-test
results affirmed the significance of integrated modelling in terms of facilitating content knowledge retention.
The MBT group remembered and recalled the atomic concepts thoroughly and did better in the delayed post-
test. Involving the modelling process in teaching sequences may enable students to store concepts in long-term
memory. This finding may add to the value of incorporating modelling-based instructional activities to further
assist the retention of scientific knowledge.
336
ISSN 2538–7138 /Online/ (pp. 325-341)
Modelling-based teaching (MBT), in essence, is processed by students making information meaningful to

them personally by using various models to enable the comprehension of contents and contexts (Windschitl et al.,
2008). In the MBT group, the drawings that students developed in the previous stage were considered, critiqued,
and leveraged upon. Student modelling processes were connected. However, in the ABT group, analogies were
proposed independently, the teacher did not provide a “bridge” between them, and students did not explore the
connections between the proposed models. Compared to the ABT, the MBT provided a more conducive environ-
ment for content understanding and long-term memory of concepts.
The results also indicated that two teaching strategies had the same effect on promoting students’ performance
of near transfer and far transfer. Modelling has often been considered as a significant factor, meaning that notable
changes occurred when compared to non-modelling conditions, but this did not always mean that learning was
occurring. Campbell et al. (2011) compared the difference between two instructional conditions, model-based
inquiry and traditional demonstration and lecture on student outcomes (science content, scientific process/reason-
ing, nature of science, and attitudes toward science) in physics classrooms. There were no significant differences
in the results between groups. This seemed to suggest that appropriate analogical activities benefit students’
transferability, as both groups implemented teaching materials with the same analogy. This finding is consistent
with a previous study by Schönborn and Bögeholz (2009), whose study focused on the nature and role of transfer
and translation in biology, which required moving across more than one external representation that delivered the
same or different biological ideas. In this study, multiple forms of analogy were used such as computer simulations,
pictures, and verbal representations, which may support the linkage and integration of information for fostering
content understanding and transferability. On the other hand, it seems to suggest that incorporating modelling
processes has no additional learning effect regarding students’ near and far transfer.
The qualitative analysis demonstrated how two groups of students developed their models over different
transfer distances. It was found that the MBT group built more correct models, which included a more accurate
element of the atomic structure such as electrons and nucleus. Even though the MBT group outperformed the ABT
group in the generation of initial models, they had a similar product in the near and far transfer items. Successful
cases in both groups showed that students had the ability and awareness to apply their initial models to another
model. Students in the MBT group were able to transform a stable mental model (such as the Bohr atomic model)
to a similar situation with a little revision by giving required quantitative information. In comparison with the ABT
group, although students were able to develop an atomic model for near transfer, they seemed to rather choose a
new model than reference their initial models. In far transfer items, both groups favoured changing their models
to find a proper way to satisfy the given question. This finding could indicate that both groups decontextualised
the model and reorganised it in a new context in such a way that it helped them make accurate decisions (Samp-
son et al., 2011; Schwarz et al., 2009). During the near transfer process, while utilising and evaluating the model in
the new context, students could decide to reject the model or restart the process from the beginning (Galbraith
& Stillman, 2006; Louca & Zacharia, 2012). Therefore, it was found that both instructional activities in this study
could foster students’ modelling practice in the model transfer process (Bamberger & Davis, 2013; Justi & Gilbert,
2002; Louca & Zacharia, 2015).
Transfer tasks are still relatively rare in educational studies (Sasson & Dori, 2015). The empirical literature has
reported difficulty in achieving transfer (De Corte, 2003). The ability to read and draw sub-micro representation
phenomena is related to one’s reasoning abilities. The ability to transfer is closely aligned to reasoning, which has a
correlation with the students’ scientific knowledge (Devetak & Glažar, 2010). The qualitative analysis of the student
models revealed the difficulties the students had in transferring understanding of the concept of atomic structure,
especially at the nanoscale, which is compatible with previous studies on learning the atomic structure (Sunyono &
Sudjarwo, 2018; Wang & Barrow, 2013). In both groups, a considerable proportion of the students failed to identify
the quantitative relationship of atomic structure. They were unable to figure out the number of electrons of each
orbital. For most students, combining atomic structures to develop a molecular model was unattainable, and it
required additional support and experience.
This study adds to the ongoing conversation about analogy and modelling in science education and con-
tributes empirical evidence to justify the use of analogies and modelling in improving content understanding and
transferability. To address the first two research questions, the quantitative analysis found that with the appropriate
337

(pp. 325-341) ISSN 2538–7138 /Online/
implementation of two instructions, MBT and ABT, both ways could significantly increase students’ content under-
standing and transferability in the context of the atomic structure. Particularly, MBT could better support students
with longer knowledge retention. The students’ ability to transfer in both groups showed the same increase in near
and far transfer items, but the MBT group could develop better initial models than the ABT group. This suggested
that the transfer changes made in the two groups are related to the use of analogy, but the use of modelling has
little or no added effect. The qualitative analysis responded to the third research question. The results indicated
that both of the groups could be competent with transfer tasks after experiencing instructions, but challenges and
difficulties in the accuracy of near and far transfer still exist because of students’ inadequate scientific knowledge.
This result emphasized the relationship between the mastery of scientific knowledge and transfer level.
These results have practical implications for science education. Modelling-based teaching (MBT) may be more
effective for enhancing and retaining students’ content understanding, but there is no evidence for increasing
short-term transferability in comparison with analogy-based instruction. Moreover, appropriate implementation
with analogy can enable a better understanding of the concepts of the atomic structure and students’ transfer-
ability. Instructors should keep in mind how and what kind of analogies are favourable for teaching specific sci-
ence knowledge, such as multiple forms of analogy, critical use of analogies, comparing the unshared features
between analogy and target concept, based on a student-centred environment, and emphasising the limitation
of analogy. This study provides an insight into how two educational interventions in the content of atomic struc-
ture are implemented for promoting students’ learning outcomes. Further research could focus on analogy with
other pedagogical support to explore which method can generate substantial learning effects when compared
to analogy activities without support.
References
Al-Balushi, S. M., & Al-Hajri, S. H. (2014). Associating animations with concrete models to enhance students’ comprehension
of different visual representations in organic chemistry. Chemistry Education Research and Practice, 15(1), 47–58.
https://doi.org/10.1039/C3RP00074E
Bamberger, Y. M., & Davis, E. A. (2013). Middle school science students’ scientific modelling performances across
content areas and within a learning progression. International Journal of Science Education, 35(2), 213–238.
https://doi.org/10.1080/09500693.2011.624133
Boo, H. K., & Toh, K. A. (1997). Use of analogy in teaching the particulate theory of matter. Teaching and Learning, 17(2), 79-85.
https://repository.nie.edu.sg/bitstream/10497/417/1/TL-17-2-79.pdf
Bransford, J. D., & Schwartz, D. L. (1999). Rethinking transfer: A simple proposal with multiple implications. Review of Research in
Education, 24(1), 61–100. https://doi.org/10.2307/1167267
Brown, S., & Salter, S. (2010). Analogies in science and science teaching. American Journal of Physiology- Advances in Physiology
Education, 34(4), 167–169. https://journals.physiology.org/doi/full/10.1152/advan.00022.2010
Bryman, A., & Cramer, D. (2009). Quantitative data analysis with SPSS 14, 15 & 16: A guide for social scientists. Routledge.
Buckley, B. C., Gobert, J. D., Kindfield, A. C. H., Horwitz, P., Tinker, R. F., Gerlits, B., Wilensky, U., Dede, C., & Willett, J. (2004). Model-
based teaching and learning with BioLogicaTM: What do they learn? How do they learn? How do we know? Journal of Science
Education and Technology, 13(1), 23–41. https://doi.org/10.1023/b:jost.0000019636.06814.e3
Çalik, M., & Ayas, A. (2005). An analogy activity for incorporating students’ conceptions of types of solutions. Asia-Pacific Forum
on Science Learning and Teaching, 6(2), 1–3.
Campbell, T., Oh, P. S., & Neilson, D. (2012). Discursive modes and their pedagogical functions in model-based inquiry (MBI)
classrooms. International Journal of Science Education, 34(15), 2393–2419. https://doi.org/10.1080/09500693.2012.704552
Campbell, T., Zhang, D., & Neilson, D. (2011). Model based inquiry in the high school physics classroom: An exploratory
study of implementation and outcomes. Journal of Science Education and Technology, 20(3), 258–269.
https://doi.org/10.1007/s10956-010-9251-6
Chittleborough, G., & Treagust, D. F. (2007). The modelling ability of non-major chemistry students and their understanding of
the sub-microscopic level. Chemistry Education Research and Practice, 8(3), 274–292. https://doi.org/10.1039/B6RP90035F
Chiu, M., & Lin, J. (2005). Promoting fourth graders’ conceptual change of their understanding of electric current via multiple
analogies. Journal of Research in Science Teaching, 42(4), 429–464. https://doi.org/10.1002/tea.20062
Cin, M. (2007). Alternative views of the solar system among Turkish students. International Review of Education, 53(1), 39–53.
https://doi.org/10.1007/s11159-006-9029-5
Cumming, J. M., & De Miranda, M. A. (2012). Reducing retroactive interference through the use of different encoding
techniques: An exploration of pre-test/post-test analyses. International Journal of Higher Education, 1(1), 22–31.
https://doi.org/10.5430/ijhe.v1n1p22
Dagher, Z. R. (1994). Does the use of analogies contribute to conceptual change? Science Education, 78(6), 601–614.
https://doi.org/10.1002/sce.3730780605
338
ISSN 2538–7138 /Online/ (pp. 325-341)
Dangur, V., Avargil, S., Peskin, U., & Dori, Y. J. (2014). Learning quantum chemistry via a visual-conceptual approach:
Students’ bidirectional textual and visual understanding. Chemistry Education Research and Practice, 15(3), 297–310.
https://doi.org/10.1039/c4rp00025k
Devecioglu-Kaymakci, Y. (2016). Embedding analogical reasoning into 5E learning model: A study of the solar system. Eurasia
Journal of Mathematics, Science and Technology Education, 12(4), 881–911. https://doi.org/10.12973/eurasia.2016.1266a
Devetak, I., & Glažar, S. A. (2010). The influence of 16-year-old students’ gender, mental abilities, and motivation on their reading
and drawing sub-microrepresentations achievements. International Journal of Science Education, 32(12), 1561–1593.
https://doi.org/10.1080/09500690903150609
Dilber, R. (2012). The effects of analogy on students’ understanding of direct current circuits and attitudes towards physics lessons.
European Journal of Educational Research, 1(3), 211–223. https://doi.org/10.12973/eu-jer.1.3.211
Dori, Y. J., & Sasson, I. (2013). A three-attribute transfer skills framework-part I: Establishing the model and its relation to chemical
education. Chemistry Education Research and Practice, 14(4), 363–375. https://doi.org/10.1039/c3rp20093k
Dukerich, L. (2015). Applying modeling instruction to high school chemistry to improve students’ conceptual understanding.
Journal of Chemical Education, 92(8), 1315–1319. https://doi.org/10.1021/ed500909w
Else, M. J., Clement, J., & Rea-Ramirez, M. A. (2008). Using analogies in science teaching and curriculum design: Some guidelines.
Model Based Learning and Instruction in Science, (pp. 215-231). Dordrecht: Springer.
Eryilmaz Muştu, Ö., & Özkan, E. B. (2019). Determining the pre-service teachers’ perceptions of atom and atomic structure through
word association test. Asia-Pacific Forum on Science Learning and Teaching, 20(1), 1–30. https://eric.ed.gov/?id=EJ1233471
Galbraith, P., & Stillman, G. (2006). A framework for identifying student blockages during transitions in the modelling process.
ZDM-International Journal on Mathematics Education, 38(2), 143–162.
Gentner, D., & Stevens, A. L. (1983). Mental models. Psychology Press.
Gilbert, J. K., & Justi, R. (2016). Modeling based teaching in Science education. Springer.
Gobert, J. D., & Buckley, C. (2000). Introduction to model-based teaching and learning in science education. International Journal
of Science Education, 22(9), 891–894. https://doi.org/10.1080/095006900416839
Gobert, J. D., & Pallant, A. (2004). Fostering students’ epistemologies of models via authentic model-based tasks. Journal of Science
Education and Technology, 13(1), 7–22. https://doi.org/10.1023/B:JOST.0000019635.70068.6f
Goh, N. K., Chia, L. S., & Tan, D. (1994). Applications and analogies: Some analogies for teaching atomic structure at the high
school level. Journal of Chemical Education, 71(9), 733–734. https://doi.org/10.1021/ed071p733
Gray, M. E., & Holyoak, K. J. (2021). Teaching by analogy: From theory to practice. Mind, Brain, and Education, 15(3), 250–263.
https://doi.org/10.1111/mbe.12288
Greca, I. M., & Moreira, M. A. (2000). Mental models, conceptual models, and modelling. International Journal of Science Education,
22(1), 1–11. https://doi.org/10.1080/095006900289976
Heng, C. K., & Karpudewan, M. (2017). Facilitating primary school students’understanding of water cycle through guided inquiry-based
learning. Overcoming Students’ Misconceptions in Science (pp. 29-49). Springer. https://doi.org/10.1007/978-981-10-3437-4_3
Johnson-Laird, P. N. (1983). Mental models: Towards a cognitive science of language, inference, and consciousness. Harvard University
Press.
Justi, R. S., & Gilbert, J. K. (2002). Modelling, teachers’ views on the nature of modelling, and implications for the education of
modellers. International Journal of Science Education, 24(4), 369–387. https://doi.org/10.1080/09500690110110142
Kassai, R., Futo, J., Demetrovics, Z., & Takacs, Z. K. (2019). A meta-analysis of the experimental evidence on the near-and far-transfer
effects among children’s executive function skills. Psychological Bulletin, 145(2), 165. http://dx.doi.org/10.1037/bul0000180
Khan, S. (2007). Model-based inquiries in chemistry. Science Education, 91(6), 877–905. http://dx.doi.org/10.1002/sce.20226
Lave, J. (1988). Cognition in practice: Mind, mathematics and culture in everyday life. Cambridge University Press.
Lee, S. W. Y., Chang, H. Y., & Wu, H. K. (2017). Students’ views of scientific models and modeling: Do representational
characteristics of models and students’ educational levels matter? Research in Science Education, 47(2), 305–328.
https://doi.org/10.1007/s11165-015-9502-x
Louca, L. T., & Zacharia, Z. C. (2012). Modeling-based learning in science education: cognitive, metacognitive, social, material
and epistemological contributions. Educational Review, 64(4), 471–492. https://doi.org/10.1080/00131911.2011.628748
Louca, L. T., & Zacharia, Z. C. (2015). Examining learning through modeling in K-6 science education. Journal of Science Education
and Technology, 24(2–3), 192–215. https://doi.org/10.1007/s10956-014-9533-5
Maia, P. F., & Justi, R. (2009). Learning of chemical equilibrium through modelling-based teaching. International Journal of Science
Education, 31(5), 603–630. https://doi.org/10.1080/09500690802538045
Marton, F. (2006). Sameness and difference in transfer. The Journal of the Learning Sciences, 15(4), 499–535.
https://doi.org/10.1207/s15327809jls1504_3
Mauro, M. F. D., & Furman, M. (2016). Impact of an inquiry unit on grade 4 students’ science learning. International Journal of
Science Education, 38(14), 2239-2258. https://doi.org/10.1080/09500693.2016.1234085
May, D. B., Hammer, D., & Roy, P. (2006). Children’s analogical reasoning in a third-grade science discussion. Science Education,
90(2), 316–330. https://doi.org/10.1002/sce.20116
Mayr, S., Erdfelder, E., Buchner, A., & Faul, F. (2007). A short tutorial of GPower. Tutorials in Quantitative Methods for Psychology,
3(2), 51–59.
McKagan, S. B., Perkins, K. K., & Wieman, C. E. (2008). Why we should teach the Bohr model and how to teach it effectively. Physical
Review Special Topics-Physics Education Research, 4(1), 10103. https://doi.org/10.1103/PhysRevSTPER.4.010103
339

(pp. 325-341) ISSN 2538–7138 /Online/
Ministry of Education in Taiwan (2018). Curriculum standards for grades 1–12. Taipei: Ministry of Education.
Nakiboglu, C. (2008). Using word associations for assessing non major science students’ knowledge structure before and after
general chemistry instruction: The case of atomic structure. Chemistry Education Research and Practice, 9(4), 309–322.
https://doi.org/10.1039/B818466F
Norman, D. A. (1983). Some observations on mental models. Mental Models. Psychology Press, pp. 15–22, Hillsdale, NJ: Lawrence
Erlbaum Associates, Publishers. https://doi.org/10.1039/b818466f
Orgill, M., Bussey, T. J., & Bodner, G. M. (2015). Biochemistry instructors’ perceptions of analogies and their classroom use. Chemistry
Education Research and Practice, 16(4), 731–746. https://doi.org/10.1039/c4rp00256c
Park, E. J., & Light, G. (2009). Identifying atomic structure as a threshold concept: Student mental models and troublesomeness.
International Journal of Science Education, 31(2), 233–258. https://doi.org/10.1080/09500690701675880
Patton, M. Q. (2002). Qualitative research & evaluation methods: Integrating theory and practice. Sage Publications.
Perkins, D. N., & Salomon, G. (1992). Transfer of learning. International Encyclopedia of Education, 2, 6452–6457.
Plomp, T. (2013). Educational design research: An introduction. Educational Design Research, 11–50. London: Routledge.
Rapp, D. N. (2005). Mental models: Theoretical issues for visualizations in science education. In Visualization in science education
(pp. 43-60). Springer. https://doi.org/10.1007/1-4020-3613-2_4
Sala, G., Aksayli, N. D., Tatlidil, K. S., Tatsumi, T., Gondo, Y., Gobet, F., Zwaan, R., & Verkoeijen, P. (2019). Near and far transfer in
cognitive training: A second-order meta-analysis. Collabra: Psychology, 5(1). https://doi.org/10.1525/collabra.203
Salomon, G., & Globerson, T. (1987). Skill may not be enough: The role of mindfulness in learning and transfer. International
Journal of Educational Research, 11(6), 623–637. https://doi.org/10.1016/0883-0355(87)90006-1
Sampson, V., Grooms, J., & Walker, J. P. (2011). Argument-driven inquiry as a way to help students learn how to participate
in scientific argumentation and craft written arguments: An exploratory study. Science Education, 95(2), 217–257.
https://doi.org/10.1002/sce.20421
Sarantopoulous, P., & Tsaparlis, G. (2004). Analogies in chemistry teaching as a means of attainment of cognitive and affective
objectives: A longitudinal study in a naturalistic setting, using analogies with a strong social content. Chemistry Education
Research and Practice, 5(1), 33–50. https://doi.org/10.1039/B3RP90029K
Sasson, I., & Dori, Y. J. (2015). A three-attribute transfer skills framework-part II: Applying and assessing the model in science
education. Chemistry Education Research and Practice, 16(1), 154–167. https://doi.org/10.1039/c4rp00120f
Schönborn, K. J., & Bögeholz, S. (2009). Knowledge transfer in biology and translation across external representations:
Experts’ views and challenges for learning. International Journal of Science and Mathematics Education, 7(5), 931–955.
https://doi.org/10.1007/s10763-009-9153-3
Schwartz, D., Bransford, J., & Sears, D. (2005). Efficiency and innovation in transfer. In J. Mestre (Ed.), Transfer of learning from a
modern multidisciplinary perspective (pp. 1-51). Information Age Publishing.
Schwarz, C. V., Reiser, B. J., Davis, E. A., Kenyon, L., Achér, A., Fortus, D., Shwartz, Y., Hug, B., & Krajcik, J. (2009). Developing a learning
progression for scientific modeling: Making scientific modeling accessible and meaningful for learners. Journal of Research
in Science Teaching, 46(6), 632–654. https://doi.org/10.1002/tea.20311
Seel, N. M. (2017). Model-based learning: A synthesis of theory and research. Educational Technology Research and
Development, 65(4), 931-966. https://doi.org/10.1007/s11423-016-9507-9
Settlage, J. (2007). Demythologizing science teacher education: Conquering the false ideal of open inquiry. Journal of Science
Teacher Education, 18(4), 461–467. https://doi.org/10.1007/s10972-007-9060-9
Slabin, U. (2017). Scientific eponym in educational universe. Journal of Baltic Science Education, 16(2), 144-147.
Sun, D., & Looi, C.-K. (2013). Designing a web-based science learning environment for model-based collaborative inquiry. Journal
of Science Education and Technology, 22(1), 73–89. https://doi.org/10.1007/s10956-012-9377-9
Sunyono, S., & Sudjarwo, S. (2018). Mental models of atomic structure concepts of 11th grade chemistry students. Asia-Pacific
Forum on Science Learning and Teaching, 19(1), 1–21.
Supasorn, S., & Promarak, V. (2015). Implementation of 5E inquiry incorporated with analogy learning approach to enhance
conceptual understanding of chemical reaction rate for grade 11 students. Chemistry Education Research and Practice,
16(1), 121–132. https://doi.org/10.1039/c0xx00000x
Taber, K. S. (2001). When the analogy breaks down: Modelling the atom on the solar system. Physics Education, 36(3), 222–226.
https://doi.org/10.1088/0031-9120/36/3/308
Taber, K. S. (2005). Learning quanta: Barriers to stimulating transitions in student understanding of orbital ideas. Science Education,
89(1), 94–116. https://doi.org/10.1002/sce.20038
Taber, K. S. (2013). Upper secondary students’ understanding of the basic physical interactions in analogous atomic and solar
systems. Research in Science Education, 43(4), 1377–1406. https://doi.org/10.1007/s11165-012-9312-3
Taber, K. S. (2014). Ethical considerations of chemistry education research involving ‘human subjects.’ Chemistry Education Research
and Practice, 15(2), https://doi.org/109–113. 10.1039/C4RP90003K
Taber, K. S. (2018). The use of Cronbach’s alpha when developing and reporting research instruments in Science Education.
Research in Science Education, 48(6), 1273–1296. https://doi.org/10.1007/s11165-016-9602-2
Trey, L., & Khan, S. (2008). How science students can learn about unobservable phenomena using computer-based analogies.
Computers and Education, 51(2), 519–529. https://doi.org/10.1016/j.compedu.2007.05.019
Tsai, C. C. (1999). Overcoming junior high school students misconceptions about microscopic views of phase change: A study
of an analogy activity. Journal of Science Education and Technology, 8(1), 83–91. https://doi.org/10.1023/A:1009485722628
340
ISSN 2538–7138 /Online/ (pp. 325-341)
Tsaparlis, G., & Papaphotis, G. (2009). High-school students’ conceptual difficulties and attempts at conceptual
change: The case of basic quantum chemical concepts. International Journal of Science Education, 31(7), 895–930.
https://doi.org/10.1080/09500690801891908
Vosniadou, S. (1988). Analogical reasoning as a mechanism in knowledge acquisition: A developmental perspective. Center
for the Study of Reading Technical Report; no. 438. https://www.ideals.illinois.edu/bitstream/handle/2142/18000/
ctrstreadtechrepv01988i00438_opt.pdf?sequence=1
Wang, C.-Y., & Barrow, L. H. (2013). Exploring conceptual frameworks of models of atomic structures and periodic variations, chemical
bonding, and molecular shape and polarity: A comparison of undergraduate general chemistry students with high and low
levels of content knowledge. Chemistry Education Research and Practice, 14(1), 130–146. https://doi.org/10.1039/C2RP20116J
Windschitl, M., Thompson, J., & Braaten, M. (2008). Beyond the scientific method: Model-based inquiry as a new paradigm of
preference for school science investigations. Science Education, 92(5), 941–967. https://doi.org/10.1002/sce.20259
Cite as: Xue, S., Sun, D., Zhu, L., Huang, H.-W., & Topping, K. (2022). Comparing the effects of modelling and analogy on high
school students’ content understanding and transferability: The case of atomic structure. Journal of Baltic Science Education,
21(2), 325-341. https://doi.org/10.33225/jbse/22.21.325
PhD candidate, University of Dundee, School of Education and Social Work,

Song Xue DD14HN, United Kingdom.
(Corresponding author) E-mail: 2393774@dundee.ac.uk
Website: https://discovery.dundee.ac.uk/en/persons/song-xue
Daner Sun PhD, Assistant Professor, Department of Mathematics and Information
Technology, The Education University of Hong Kong, Hong Kong SAR, China.
E-mail: dsun@eduhk.hk
Website: https://pappl.eduhk.hk/rich/web/person.
xhtml?pid=179988&name=SUN-Daner
Liying Zhu PhD, Lecturer, College of Teacher Education, Zhejiang Normal University,
Jinhua, China.
E-mail: zhuly2000@163.com
Hui-Wen Huang PhD, Associate Professor, School of Architecture and Civil Engineering,
Huizhou University, Huizhou, China.
E-mail: fupjp@hotmail.com
PhD, Professor, University of Dundee, School of Education and Social Work,
Keith Topping DD14HN, United Kingdom.
E-mail: k.j.topping@dundee.ac.uk
Website: https://www.dundee.ac.uk/people/keith-topping
341
INFORMATION
FOR CONTRIBUTORS
EDITORIAL AND REVIEW PROCEDURES
Journal of Baltic Science Education (JBSE) publishes original scientific research articles in the field of Natural Science Education and
related areas for all educational levels in the Baltic and other countries. It is possible to publish special (thematic) issues of JBSE. The
papers should be submitted and will be published in English. JBSE will promote to establish contacts between researchers and practical
educators both in the Baltic countries and countries around.
The authors of the manuscripts are responsible for the scientific content and novelty of the research materials. Articles, published
before in other international journals or papers’ collections will not be accepted for publication in JBSE.
As a publication that represents a variety of cross-disciplinary interests, both theoretical and practical, the JBSE invites manuscripts on
a wide range of topics, especially in the following areas:
• Didactics of natural sciences. • Philosophical, political, economical and social aspects
• Theory and practice in natural science teacher of natural science education.
education. • The supplementary natural science education.
• Integrated natural science education. • ICT in natural science education.
• Natural science and technological literacy. • The standardisation of natural science education etc.
• General and professional natural science education.
MANUSCRIPTS GUIDELINES
The structure of the research paper presented to the Journal of Baltic Science Education should be as follows: abstract - short report of the
investigation; introduction inc. aim and subject of the research; research methodologies and methods; results of the research incl. discussion;
conclusions; list of references in APA style (7th Ed.).
The papers should be submitted in English. If English is a second language for the author, please consider having the manuscript
proof read and edited before submitting.
The text must be elaborated in Word for Windows, using 12 point Times New Roman letters. An article should not exceed 7-10 A4 pages,
included figures, tables and bibliography. Publishing of longer articles should be negotiated separately. Texts margins: top and bottom 20mm,
left - 25mm, right - 20mm. The title: capital letters, 14pt, bold; space between the title and the author’s name is one line interval. Author’s name
and surname: small letters, 12pt, bold. Under the name, institution: 11 pt, italics; space between the title and the text: 1 line interval. Abstract
– about 100-150 words - precedes the text. The text: 12pt Single or Auto spacing, in one column. Key words: no more than five words. The
language must be clear and accurate. The authors have to present the results, propositions and conclusions in a form that can suit scientists
from different countries.
Titles of the tables and figures: 11 pt, small letters. Space between figures or tables and the text: 1 line interval. Introduction, titles of
chapters and subchapters: 12pt, bold, small letters. Numbers: Arabic, subchapters numbered by two figures (1.1, 1.2, etc.). Figures, tables and
captions should be inserted within the manuscript at their appropriate locations. Diagrams and graphs should be provided as finished black
and white line artwork or electronic images. When there are a number of illustrations, the author should endeavour to reduce the amount of
text to accommodate the illustrations in the limited space available for any article.
References in the text should be presented in brackets (Knox, 1988; Martin, 1995). If necessary, the page can be indicated: (Martin,
1995, p.48). The list of references should be presented after the text. The Words List of References: 11pt, bold, small letters. The references
should be listed in full at the end of the paper in the following standard form:
For books: Saxe, G.B. (1991). Cultural and Cognitive Development: Studies in Mathematical Understanding. Hillsdale, NJ: Erlbaum.
For articles: Bekerian, D.A. (1993). In Search of the Typical Eyewitness. American Psychologist, 48, 574-576.
For chapters within books: Bjork, R.A. (1989). Retrieval Inhibition as an Adaptive Mechanism in Human Memory. In: H.L. Roediger III & F.I.M.
Craik (Eds.), Varieties of Memory & Consciousness (pp. 309-330). Hillsdale, NJ: Erlbaum.
On a separate page, author - related data should be presented in English: name, surname, degree and academic title, institution, full
correspondence address in the clearest and most complete form /ordinary post and e-mail addresses /, position (to ensure anonymity in the
review process). The author (authors) should confirm in writing, that the manuscript has not been published in other journal or
handed over (transferred) to other journal for publication.
EDITORIAL AND REVIEW PROCEDURES

Manuscripts will be sent anonymously to reviewers with expertise in the appropriate area. All manuscripts will be rewieved by two
experts before JBSE’s accept them for publication. This process usually takes about two months. The journal co-editors will make minor
editorial changes; major changes will be made by the author(s) prior to publication if necessary. JBSE’s redaction will sent to author(s)
only one correcture which must be sent back within 2 weeks. JBSE will not review submissions previously published elsewhere through
print or electronic medium.
Manuscripts submitted to the JBSE cannot be returned to authors. Authors should be sure to keep a copy for themselves. Authors’
signatures should be at the end of the paper and its second checked proofs.
Manuscripts, editorial correspondence (and other correspondence for subscription and exchange), and any questions should be
sent to editor-in-chief or to regional redactors.
Journal`s requirements for the authors are available online: http://www.scientiasocialis.lt/jbse/files/JBSE_requirements_2019.pdf
342
JBSE
SMC „SCIENTIA EDUCOLOGICA“
The Associated Member of Lithuanian Scientific Society, European Society for the History of Science
(ESHS) and International Council of Association for Science Education /ICASE/,
SCIENTIA SOCIALIS Ltd
A member of the Lithuanian Publishers Association & Crossref
E-mail: scientia@scientiasocialis.lt Website: http://www.scientiasocialis.lt
Dear colleagues,
We would like to invite articles for an international blind peer-review
scientific journal (Twenty fifth CFP)
Problems of Psychology in the 21st Century

ISSN 2029-8587 (Print) ISSN 2538-7197 (Online)
http://www.scientiasocialis.lt/ppc/ 2022: Vol. 16, No. 1; No. 2
Papers submitted to PPC_25CFP_2022 should be original work and

substantively different from papers that have been previously published or are
under review in a journal or another peer-reviewed conference. We invite
submission of papers describing innovative research on all aspects of psychology
and related areas. Submitted papers will be assessed based on their novelty,
technical quality, potential impact, and clarity of writing.
Responsible
E-mail: problemsofpsychology@gmail.com Publication
Peer Reviewed
The call is open till 25 May 2022. Open Access
Global Readership
Application form + declaration + full paper: 25 May 2022
EBSCO http://www.scientiasocialis.lt/ppc/files/PPC_guidelines_2018.pdf
INDEX We advise you to use a template for paper preparation. You can download it
COPERNICUS from our website: http://www.scientiasocialis.lt/ppc/node/6 (template for paper,
ANVUR Word format). Theoretical papers usually follow an argumentative pattern and
CNKI are organised around the solution of a problem.
OAJI
Ulrich's Periodicals Last but not least
Directory
MIAR International journal „Problems of Pscyhology in the 21st Century“ is a high quality
ROAD, scientific journal which publishes original research on all areas of psychological
Europub, sciences. The Journal welcomes the submission of manuscripts that meet the general
DOI, criteria of significance and scientific excellence. Submitted papers will be assessed
Crossref, based on their novelty, scientific and technical quality, potential impact, and clarity of
Scilit, writing. All submissions will be checked by iThenticate before sending to reviewers.
Internet Archive, The publisher and journal have a policy of “Zero Tolerance on the Plagiarism”.
ScienceGate etc. PPC is an open access academic journal published semiannually.
Submission online: http://www.scientiasocialis.lt/ppc/node/136
Information related to APC can be found at:
http://www.scientiasocialis.lt/ppc/node/node/11
Sincerely yours, Editorial Board
343
ISSN 1648-3898 /Print/, ISSN 2538-7138 /Online/
Compiled by: Vincentas Lamanauskas

Linguistic Editor: Ilona Ratkevičienė
Cover design by: Jurgina Jankauskienė
Layout design by : Linas Janonis

25 April 2022. Publishing in Quires 10,5. Edition 90
Publisher Scientia Socialis Ltd.,

Donelaicio Street 29, LT-78115 Siauliai, Lithuania
E-mail: scientia@scientiasocialis.lt
Phone: +370 687 95668
http://www.scientiasocialis.lt

Printing Šiauliai printing house
9A P. Lukšio Street
LT-76207 Šiauliai, Lithuania
Phone: +370 41 500 333.
Fax: +370 41 500 336
E-mail: info@dailu.lt
https://siauliuspaustuve.lt/

Journal of Baltic Science Education, Vol. 21, No. 2, 2022

Uploaded by

Document Information

Copyright

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Journal of Baltic Science Education, Vol. 21, No. 2, 2022

Uploaded by

Copyright:

Vol.21, No.

Published since 2002 Address:

ISSN 1648–3898 (Print) © Scientia Socialis Ltd., a member of the Lithuanian

ISSN 2538–7138 /Online/

CLIMATE EDUCATION: A GRAND CHALLENGE

PRE-SERVICE SCIENCE AND COMPUTER TEACHERS’ VIEWS ON TEACHING PRACTICE

SELF-REGULATED LEARNING STRATEGIES IMPACT FOURTH-GRADE STUDENTS’ POSITIVE

AN EXAMINATION OF PRE-SERVICE TEACHERS’ EXPERIENCES IN CREATING A SCIENTIFIC

ALIGNMENT OF CONCEPTS OF MEIOSIS AMONG CURRICULUM, TEXTBOOKS, CLASSROOM

IMPLEMENTATION OF İNNOVATİVE ARTİFİCİAL İNTELLİGENCE COGNİTİONS WİTH PROBLEM-

THE IMPACT OF TEACHERS’ PROFESSIONAL DEVELOPMENT IN SCIENCE PEDAGOGY ON

PRIMARY PRE-SERVICE TEACHERS’ METAPHORICAL PERCEPTIONS OF THE CONCEPT OF

SCIENCE MAPS AND BIBLIOMETRIC ANALYSIS ON HYGIENE EDUCATION DURING 2012-2021

THE MEASURING INSTRUMENT OF PRIMITIVE PHYSICS PROBLEM FOR UPPER-SECONDARY

COMPARING THE EFFECTS OF MODELLING AND ANALOGY ON HIGH SCHOOL STUDENTS’

INFORMATION FOR CONTRIBUTORS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342

PROBLEMS OF PSYCHOLOGY IN THE 21ST CENTURY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343

CLIMATE EDUCATION: A GRAND CHALLENGE

Jari Lavonen PhD, Professor, Department of Education, Faculty of Educational

PRE-SERVICE SCIENCE AND

COURSES ISSN 2538-7138 /Online/

Abstract. The Teaching Practice course

Data Collection Tool

Data Collection and Analysis

Insufficient in terms of learning gains and content f

The number of gains that we can participate in class may increase. 21

Sufficient in terms of learning gains and content f

It is progressing in line with the practice process. 7

Insufficient in terms of semesters and credits f

Insufficient in terms of semesters 22

Sufficient in terms of semesters and credits f

Sufficient in terms of semesters 8

Insufficient measurement and evaluation activities f

Only practice files and presentations are evaluated. 18

Sufficient measurement and evaluation activities f

The practice files are reviewed. 10

Problems and difficulties encountered during the process f

The high number of pre-service teachers per practice teacher 30

Studies that can be done on the subject f

More active participation in the class 26

Conclusions and Implications

Authors declare no competing interest.

Ayhan Bulut PhD, Assistant Professor, Bayburt University, 69000 Merkez/

Abstract. For long years, researchers in the

Groups Dependent Variables N 𝑥̅ SD t df p

Experimental Gr. 20 3.08 .25 -.62 37 .80

The Preliminary Results

Pre-test Post-test Retention test

Experimental Gr. 20 3.08 .25 3.31 .21 3.13 .28

Split-Plot ANOVA Results

Dependent Variables Source of Variance SS df MS F p ƞp2

Conclusions and Implications

Authors declare no competing interest.

Busra Cengiz-Istanbullu MA, Primary School Teacher, Istanbul, Turkey.

A SCIENTIFIC DIGITAL ISSN 2538-7138 /Online/

STORY IN THE CONTEXT OF

Components of TPACK Explanation Components of creating digital story

Components of TPACK Explanation Components of creating digital story

Research Aim and Research Questions

Instrument and Procedures

1. Preparation and Planning:

Self-confidence of Primary School Teacher Candidates about TPACK

Pre-test 24 42.21 8.582

Pre-test 24 26.33 5.577

In terms of Technological Knowledge (TK)

Another pre-service teacher stated her opinion as follows:

In terms of Technological Pedagogical Knowledge (TPK)