Journal of Baltic Science Education, Vol. 21, No. 4, 2022

Vol.21, No.
4, 2022
ISSN 1648-3898 /Print/
ISSN 2538–7138 /Online/
The International Journal of the Scientia Socialis Ltd., & SMC “Scientia Educologica”
Journal of Baltic Science Education, Vol. 21, No. 4, 2022
Editorial Board
ISSN 1648–3898 /Print/
Editor-in-Chief ISSN 2538–7138 /Online/
Prof., Dr. Vincentas Lamanauskas Vilnius University, Lithuania
Deputy Editor-in-Chief
Prof., Dr. Andris Broks University of Latvia, Latvia
Deputy Editor-in-Chief
Prof., Dr. Miia Rannikmäe University of Tartu, Estonia
Executive Secretary
Dr. Laima Railienė SMC “Scientia Educologica”, Lithuania
Editors:
Prof., Dr. Boris Aberšek University of Maribor, Slovenia
Prof., Dr. Agnaldo Arroio University of Sao Paulo, Brazil
Prof., Dr. Martin Bilek Charles University, Czech Republic
Dr. Paolo Bussotti University of Udine, Italy
Prof., Dr. Bulent Cavas Dokuz Eylul University, Turkey
Prof., Dr. Salih Cepni Uludag University, Turkey
Dr. Peter Demkanin Comenius University, Slovakia
Dr. André du Plessis Nelson Mandela University, South Africa
Dr. Markus Sebastian Feser University of Hamburg, Germany
Prof., Dr. Jack Holbrook University of Tartu, Estonia
Prof., Dr. Ryszard M. Janiuk Maria Curie Sklodowska University, Poland
Dr. Milan Kubiatko Jan Evangelista Purkyně University in Ústí nad Labem, Czech Republic
Dr. Todar Lakhvich Belarusian State Medical University, Republic of Belarus
Prof., Dr. Jari Lavonen University of Helsinki, Finland
Dr. Rita Makarskaitė-Petkevičienė Vilnius University, Lithuania
Prof., Dr. Paul Pace Malta University, Malta
Prof., Dr. Valfrids Paškevičs Daugavpils University, Latvia
Prof., Dr. Jongwon Park Chonnam National University, Korea
Prof., Dr. Raffaele Pisano University of Lille, France
Prof., Dr. Pavol Prokop Comenius University in Bratislava, Slovakia
Prof., Dr. Konstantinos Ravanis University of Patras, Greece
Dr. Alona Rauckienė-Michaelsson Klaipėda University, Lithuania
Dr. Dušica Rodić University of Novi Sad, Serbia
Prof., Dr. Heimo Saarikko Helsinki University, Finland
Prof., Dr. Hae-Ae Seo Pusan National University, Korea
Dr. Uladzimir K. Slabin Lane Community College, USA
Prof., Dr. Andrej Šorgo University of Maribor, Slovenia
Prof., Dr. Borislav V. Toshev Sofia University, Bulgaria
Prof., Dr. Georgios Tsaparlis University of Ioannina, Greece
Dr. Muhammet Usak Kazan Federal University, Republic of Tatarstan, Russian Federation
A scientific journal JBSE issued by the Scientia Socialis Ltd., in cooperation with SMC Scientia Educologica, Lithuania,
emphasizes theoretical, experimental and methodical studies in the field of science education. JBSE is an international
academic journal. In order to maintain the high standards appropriate to such a journal, all contributions received are
submitted for anonymous review by two experts, additionally to review by the Editor. The decision of the Editor on
the acceptance of articles is final and no correspondence can be entered into on reasons for rejection of a submitted
contribution.
Published since 2002 Address:

The journal is published bimonthly. Scientia Socialis, Ltd.
IF: 1.182 (2021) Donelaičio Street 29, LT-78115 Siauliai, Lithuania
H Index: 20 (2021) E-mail: jbse@scientiasocialis.lt
SJR: 0.48 (2021) Phone: +370 687 95668
ICDS: 8.8 (2020) Home page: http://www.scientiasocialis.lt/jbse/
ICV: 156.65 (2020) Skype: scientia12
ISSN 1648–3898 (Print) © Scientia Socialis Ltd., a member of the Lithuanian

Publishers Association, Lithuania, 2022
ISSN 2538–7138 (Online)
The articles appearing in this journal are indexed/abstracted in Copernicus Index, EBSCO:
534 Academic Search Premier, Web of Science Core Collection (Social Sciences Citation Index ),
SCOPUS, ProQuest, Road, Crossref, ERIC and DOI.
ISSN 1648–3898 /Print/ CONTENTS
CONTENTS
Editorial
A PERSONAL REFLECTION ON COVID-19 ONLINE TEACHING, LEARNING, AND ASSESSMENT
WITH SUGGESTIONS FOR RESEARCH
André du Plessis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 536
Articles
THE JIGSAW TECHNIQUE IN LOWER SECONDARY PHYSICS EDUCATION: STUDENTS’
ACHIEVEMENT, METACOGNITION AND MOTIVATION
Branislava K. Blajvaz, Ivana Z. Bogdanović, Tamara S. Jovanović, Jelena D. Stanisavljević, Milica V. Pavkov-
Hrvojević . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 545
COMPARING PRE-SERVICE PRIMARY SCHOOL TEACHERS’ AND LOWER-SECONDARY
LEARNERS’ UNDERSTANDING THE PARTICULATE NATURE OF MATTER
Eva Hejnová . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 558
USING A MACHINE LEARNING APPROACH TO EXPLORE NON-COGNITIVE FACTORS
AFFECTING READING, MATHEMATICS, AND SCIENCE LITERACY IN CHINA AND THE UNITED
STATES
Lu Ye, Yuqing Yuan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 575
THE TYPES OF PEER SCAFFOLDING IN SCIENTIFIC EXPERIMENTAL ACTIVITIES
Seong-un Kim . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 594
PREDICTION OF THE CORRELATION BETWEEN PROBLEM-SOLVING SKILLS AND CONCEPTUAL
REASONING IN STOICHIOMETRY
Love Kotoka, Jeanne Kriek . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 615
UNDERGRADUATE STEM AND NON-STEM STUDENTS’ INTERPRETATION OF MEAN IN AN
INFOGRAPHIC
Zorana Lužanin, Marija Kaplar, Tatjana Hrubik-Vulanović . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 638
SCIENCE TEACHERS’ PERCEPTIONS OF THEIR KNOWLEDGE BASE FOR TEACHING FORCE
CONCEPTS
Maphole Marake, Loyiso C. Jita, Maria Tsakeni . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 651
EFFECTS OF INQUIRY-BASED CHEMISTRY EXPERIMENTATION ON STUDENTS’ ATTITUDES
TOWARDS THE TEACHING AND LEARNING OF CHEMISTRY
Christian Bob Nicol, Emmanuel Gakuba, Gonzague Habinshuti . . . . . . . . . . . . . . . . . . . . . . . . . 663
APPLYING FACTOR ANALYSIS FOR ASSESSING KNOWLEDGE STRUCTURE OF STUDENTS IN
GRADE 10: THE SUBJECT OF REDOX REACTION
Wenxiu Tang, Xintong Zhu, Yangyi Qian . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 680
THE EFFECTS OF SCIENCE TEACHERS’ PEDAGOGICAL CONTENT KNOWLEDGE ON STUDENTS’
ATTITUDES TOWARD SCIENCE AND THEIR ACHIEVEMENT
Muhammet Usak, Harun Uygun, Muharrem Duran . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 694
THE EFFECT OF STUDENTS’ ATTITUDE TOWARDS SCIENCE, TECHNOLOGY, ENGINEERING,
AND MATHEMATICS ON 21ST CENTURY LEARNING SKILLS: A STRUCTURAL EQUATION
MODEL
Shao-Rui Xu, Shao-Na Zhou . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 706
ASSESSMENT OF KNOWLEDGE INTEGRATION IN STUDENT LEARNING OF BUOYANT FORCE
Yi Zou, Lizhen Jin, Yanbing Li, Tao Hu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 720
Information
INFORMATION FOR CONTRIBUTORS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 739
PROBLEMS OF PSYCHOLOGY IN THE 21ST CENTURY . . . . . . . . . . . . . . . . . . . . . . . . . . . .
535
This is an open access article under the Journal of Baltic Science Education, Vol. 21, No. 4, 2022
Creative Commons Attribution 4.0
International License ISSN 1648–3898 /Print/
A PERSONAL REFLECTION ON COVID-19 ONLINE TEACHING, LEARNING, AND

ASSESSMENT WITH SUGGESTIONS FOR RESEARCH
André du Plessis
Nelson Mandela University, South Africa
Introduction and Background
COVID-19 had a severe impact on teaching and learning in schools and tertiary education institutions. Human
Rights Watch (Impact of Covid-19 on Children’s Education in Africa, 2022) posits that due to the closure of many
schools in African countries, a significant majority of learners were excluded from continuing their education as
learning and teaching halted. The pandemic has also had its effect on the higher education sector within the South
African context, however, data about non-attendance and drop-out seems not to be freely available as revealed
by searching online. Nonetheless, it is reasonable to assume that there were drop-outs and non-attendance by
students, however, not to the same extent as within schools as universities embarked on contingency plans to
minimize dropouts. Nelson Mandela University in Gqeberha South Africa where the author of this editorial resides,
implemented contingency plans within weeks of the COVID pandemic outbreak to offer students ICT devices on
loan, e.g. by debiting it against their student accounts while Wi-Fi data access was secured through collaboration
with internet providers. It appears that the university sector was able to be more proactive than schools, which can
probably be attributed to the fact that the majority of universities have their own ICT infrastructures. In addition,
many modules incorporated the use of ICT in various ways, including the use of learning management systems
such as Moodle or similar. This resulted in many universities being better prepared to embrace the opportunity to
extend online teaching, learning, and assessment in various forms during the COVID-19 pandemic.
At the same time, the challenge was not whether students had the behavioural intention to adopt online learn-
ing tools for teaching and learning, but to what extent lecturers in academia were ready and had the behavioural
intention to do so as purported by the unified theory of acceptance and use of technology (UTAUT) theoretical
framework (see Venkatesh, Morris, Davis & Davis, 2003; Venkatesh, Thong, & Xu, 2016). As such, the challenge within
higher education was embracing forms of online learning which then leads to thinking differently about teach-
ing, learning and assessment and this is in line with what Bond, Bedenlier, Marín & Händel (2021) argued, namely
that COVID has had a significant impact on how academia perceives technology, as Information Communication
Technologies (ICTs) had to be embraced to continue teaching and facilitate learning. Within the African context,
specifically, South Africa, the above implied that academics had to embrace ICTs in existing and innovative ways.
However, one of the major challenges seems to be that in many instances, many academics have not used ICT to
the extent that was required from them then, as many had minimal knowledge of the ‘how to’ and as such had to
find their way through the proverbial forest, e.g., either learning the ‘how to’ on their own through self-discovery
and experimentation, with assistance of colleagues and (or) training which was provided to them organised by
their university.
536
https://doi.org/10.33225/jbse/22.21.536
ISSN 1648–3898 /Print/ A PERSONAL REFLECTION ON COVID-19 ONLINE TEACHING, LEARNING, AND ASSESSMENT
ISSN 2538–7138 /Online/ (pp. 536-544)
The above then paved the way for the focus of this editorial, namely, to personally reflect on my online teach-
ing learning and assessment during COVID and post-COVID, as in our Faculty of Education, students are currently,
i.e. in August 2022, continuing with online learning without traditional face-to-face teaching and learning in most
of their courses and modules, while a hybrid mode is used in a few modules. Returning to campus is under review
and this will then be implemented when the new year and new first semester commences in February 2023. Over
this period of approximately four semesters, I had to redesign my courses (modules) to ensure online presence
(see Garrison, 2007; Garrison, Anderson & Archer, 1999, 2009, 2010) and assisted staff members which enabled me
to reflect on and contextualise my personal experiences.
As alluded to previously, the purpose of this editorial is to provide a personalised subjective reflection related
to my own experiences and interactions concerning online teaching and learning, while being aware of personal
bias playing a role in this reflection. My aim is not to provide a scientific editorial or paper utilising statistical tools,
but merely to provide my subjective take while I do recognise that many readers’ experiences related to COVID and
online learning might reflect similar experiences and concerns while acknowledging that others might have had
different experiences, and therefore could challenge my perceived opinions and even defeat them with grace. In
the end, the qualitative reflective piece is my position at this moment and from a critical realist (see Benton & Craib,
2001; Sayer, 2000) position, my opinions are not reified, but open to change. In addition, I posit that my reflection
can be viewed as a personal case study of online learning experiences and perceptions, as I am reflecting in depth
in a real-world context on a contemporary real-life phenomenon (Yin, 2014), the case being personal-centred/
related online learning perceptions and experiences. I do not present multiple data sources for triangulation
purposes, nevertheless, I argue by using Stake’s (1995) perspective that the reader is invited to interrogate my nar-
rative and claims, to observe generalizations based on my claims, namely to agree with some of my perspectives
and to disagree with others, or simply to disregard what I am stating in totality. In the conclusion section, I will
make a final case for the importance of assessment integrity and quality concerns associated with the complexity
of online teaching, learning and assessment while also suggesting possible research areas based on my reflection
and personal experiences that could be pursued by academics informed by the shift in education praxis caused
by the enduring COVID-19 pandemic.
Moving From a Traditionalist to an Online Presence Mode Was a Given Requiring Change
During this disruptive time, contrary to the traditional way higher education institutions (HEIs) operated
(Iglesias-Pradas et al., 2021), one of the main challenges was how academic staff in numerous instances were
not well-prepared to embrace this forced transition, as many were lacking the necessary skills to embrace online
teaching, learning and assessment (Erlam et al., 2021). Despite this situation, academics had to continue to teach
online and design online assessments. In many instances, the alternative to traditional face-to-face learning re-
sulted in a shift to emergency online teaching where traditional teaching methods were simply continued but
using ICT (Hodges et al., 2020). MsTeams or Zoom became the new delivery tools accompanied by email and
Whatsapp to ‘courier’ notes and assignments between the lecturer and student and vice versa. This mode where
Ms Teams and Zoom are utilised has been referred to as ‘Pedagogy Zoom’ an instructivist and teacher-centred
pedagogy (Du Plessis & Blignaut, 2020). This type of teaching and learning is not embracing the full opportunities
which online learning pedagogy provides, rather, it is an approach to retaining the status quo through traditional
forms of teaching and assessment methods. On the other hand, utilising an online Learning Management System
(LMS) such as Moodle, BlackBoard, CANVAS, etc. affords an online learning experience which has the potential to
extend emergency teaching, yet, the mere availability of using familiar teaching methods such as the provision
of notes, links, videos, etc. could result in the use of LMS in a repository mode which does not extend the learning
envisaged. For online learning to harness its full potential of learning, the principles of social presence, cognitive
presence and teacher presence should be explicitly integrated into the planning, made present or visible (Garrison,
2007; Garrison, Anderson & Archer, 1999, 2009, 2010) which can be daunting to implement to the extent that it
is required for many experienced and novice academics, something which I also grappled with. Invoking these
presences was something I tried to harness and this will hopefully become prevalent in the subsequent sections.
Table 1 compares my pre-COVID and in-COVID teaching and learning context.
537
(pp. 536-544) ISSN 2538–7138 /Online/
Table 1
Comparing My Pre-COVID and In-COVID Teaching and Learning Context
Pre-COVID During-COVID
Teaching •• Face-to-Face •• Screen-to-Screen

•• Eclectic: Lecturing, Group work, Discussions, •• Eclectic: Lecturing, Discussions, Cutting-and-pasting and manipulating
Cutting-and-pasting and manipulating media, media, Methodology in the context of the topic and student Presenta-
Methodology in the context of the topic and student tions [Not the same ‘quality’ of interaction]
Presentations •• WhatsApp group discussions and clarifications
•• MS Teams Chat
Traditional tools •• Chalkboard and whiteboard •• Electronic Whiteboard [e.g. Ms Whiteboard or Whiteboard in Zoom]
ICT Tools •• Email, YouTube and Moodle as a repository for •• Email, YouTube, Moodle as teaching, learning, sharing of material and
sharing of material and links links, assessment tool and feedback tool
•• Ms Whiteboard sessions with voice
•• Ms Teams Chat option during contact sessions
Material •• Comprehensive and detailed module material with •• Comprehensive and detailed module material with content, links and
content, links and homework AND study letter homework AND and study letter online available separately for each
printed as one document unit in pdf
•• No recordings of contact sessions •• Recording of each Ms Team contact session made available on Moodle
•• PowerPoints (if used), links to online material and •• PowerPoints (if used), links to online material and YouTube videos are
YouTube videos are made available on Moodle made available on Moodle
Assessment •• Uploading assignments to Moodle •• Online reflections

•• Paper-based assignment: Lesson plan OR Poster •• Online forum discussions
without using ICT tools •• Uploading assignments to Moodle: Individual and Group assignment
•• Two to three smaller semester tests in a controlled •• Paper-based assignment: Lesson plan OR Poster to be designed using
setting on several units on campus: short, medium ICT tools where possible
and longer questions [Not a test after each unit]
•• Online MCQ test after every unit or every second unit module in a
•• One final sit-down examination in a controlled non-controlled setting
setting on campus: short, medium and longer
questions •• Two to three larger MCQ tests during the semester on larger three or
more units: Mid-semester and at the end of the semester
•• Excluded from the final exam if the year mark is
less than 40% •• No exclusions, final mark of 50% to be obtained to pass – Continuous
assessment
Support •• Question time during contact sessions •• Question time during online contact sessions
•• Appointments in my office •• Appointments via Ms Teams
•• Email responses to questions •• Email responses to questions
•• Test memorandum provided on Moodle •• WhatsApp group: Students and lecturer engage with anything related
•• Questions about the test discussed during class- to the module
room time •• WhatsApp group: Share and discuss queries about the tests questions
•• Homework discussed during class time after every test
•• Posting written and drawn images and voice notes to explain answers
which are queried
•• Homework uploaded as evidence of completion and discussed during
class time
Assessment •• 20% for assignments, 30% for two to three large •• 15% for online reflections and discussions and forums, 30% for online
weighting tests reworked to 100% as the year mark (50% assignments, 25% for unit tests and 35% for larger tests adding to a
weighting) and 100% of the sit-down exam mark total of 100% continuous assessment.
(50% weighting)
538
ISSN 2538–7138 /Online/ (pp. 536-544)
Teaching Style Compromised or Not?
Using a personal perspective, I will be reflecting on my own experience of having to design and facilitate
three Intermediate Phase Mathematics Methodology modules during 2020 forming part of the ‘old’ B Ed Inter-
mediate Phase programme and the design and implementation of three new Intermediate Phase Mathematics
Methodology modules for our new B Ed Intermediate Phase programme in 2021 and 2022, modules where
most of the class sizes ranged from 100 to 160.
I have to admit from the outset that how I would have ‘preferred’ to teach, interact, and informally and
formally assess my students did not materialise due to some of my pedagogical initial limitations and my limited
knowledge about online learning, i.e. using MsTeams, Whatsapp and Moodle as Learning Management System
(LMS). My teaching style is eclectic, as I prefer to utilise pedagogical practices associated with behaviourism,
cognitivism, social constructivism and constructionism. From a behaviourist perspective, e.g., I utilise repetition,
praise and positive reinforcement while from a cognitivist viewpoint I want students to process information
through meaningful interactions such as individual problem solving, planning, execution or implementation,
self-reflection, etc. I believe that it is important to promote Bloom’s taxonomy (Bloom, Engelhart, Furst, Hill, &
Krathwohl, 1956), as well as the revised taxonomy of Anderson et al. (2001) to promote thinking on all cogni-
tive levels as presented in all six levels by the associated action verbs. The social constructivist and cooperative
learning dimension is catered for, e.g., through utilising jigsaw, problem-solving groups, discussions, group
presentations, collaborative design of and the presentation of lesson plans and its execution in a micro-teaching
context to receive on-the-spot constructive feedback from myself and their peers.
I also prefer that students experience the introduction and development of mathematics teaching-facili-
tation in a hands-on classroom space, e.g. when teaching perimeter and area of squares, rectangles, triangles
and parallelograms; I provide them with square grid 1cm2 paper and then they draw the various shapes based
on the sizes that I provide. They count, colour-in, cut-out, etc. and I use these activities to enable them to ‘dis-
cover’ the formulas for the perimeter of a square and rectangles, as well as the area for all these shapes, thus
I model how they could teach it. I am aware that the students know what the formulas are, hence they have
the procedural and formula knowledge, but when asking them why the formula is what it is, many do not
always have the conceptual understanding. The same applies when I engage with Pythagoras’ theorem, why
Pi is 3.14, why the formula for the circumference of a circle is πr 2 and why the area of a circle formula is . The
same applies again when engaging with the different types of angles, triangles, interior, and exterior angles
of regular polygons, etc. The tactile and visual dimension is thus vital from my perspective. I have provided my
students with material to engage in during my lessons when I show them the ‘why’ and the ‘how to’, as well as
providing opportunities to cement procedural fluency. The focus was thus not just on theory and academic
readings, as it is evident from the above that I also opted to focus in my online class sessions on showing the
‘how’ one could hands-on explain and (or) let learners experience how to find formulas or to make deductions
by being practically involved, while I also require that my students interact with aspects related to calculations
related to the content, i.e. how to perform the calculations procedurally and accurately. Equally important, I
also included online apps or software and simulations, e.g., Geogebra, whenever possible to model the utilisa-
tion of online learning apps and software that can be used within the school classroom context when I teach.
Another example, when I teach different methods of the four basic operations and order of operations in their
first year, I prefer that students show and share which methods and strategies they used when they were at
school, while I also provide them with several methods in printed format and then request them to individu-
ally, in pairs or threes or fours engage with these methods and to make sense of them by requesting them to
explain what these methods entail. This is then followed by individuals, pairs or larger groups presenting their
understanding(s) to the class.
Although I have tried to incorporate some of these ideas in my online class sessions, the responses of
students have not been as positive as I envisaged. What I have noted, is that it was in many instances the same
engaged students who responded consistently, and when some students were asked to provide feedback to
a question, they were ‘online’ in MsTeams, but NOT present online, as they did not respond while the system
indicated they were connected. This might have been the result of students knowing that sessions are recorded,
or they do not or cannot stay in front of the screen for all their online sessions due to possible ‘screen fatigue’
or some might just not ‘feel-like it’ responding at that time. In some instances, a student would 5 or 10 minutes
later respond to a question which might be due to not being in a position to contribute at the actual point
539
(pp. 536-544) ISSN 2538–7138 /Online/
in time, or not being in front of the screen at that time or perhaps a friend has notified the student that the
lecturer was directing a question to that person and they then subsequently responded. Although I was able
to establish some form of teaching and social presence, I believe that this was not at the same level as I would
have been able to do so within a face-to-face on-campus mode. Perhaps my perception based on the above
is based on the fact that I relied too much on my Ms Teams’ traditional lecturing interactions. However, on the
other hand, it is also important to acknowledge that there is also the possibility that a great deal of learning
has occurred by using both Ms Teams and Moodle, but due to my preference for the more traditional face-to-
face on-campus model, this could have tainted my perceptions and experiences.
My Whatsapp group and individual interactions also increased with students, as students started to post
questions to me and to the group which resulted that many students starting to assist their peers which also
promoted learning. As such, social, cognitive and teacher-student presences increased which provided learning
opportunities which were not harnessed before and this also showed me that the possibilities of alternative
tools that I have not used previously offer great potential.
Assessment Conundrum
Pre-COVID, I utilised mostly sit-down written tests and one assignment for assessment purposes while
also requesting paper-based reflections. COVID-19 made me rethink assessment as I started to map formative
activities and formal activities to my module and unit outcomes. These included students required to engage
with online reflections on module unit experiences, critical reflection on topics related to the modules, reflec-
tions based upon questionnaires related to determining personal learning philosophies and learning theory
preference, group online presentations graded individually and as a group, online lesson plan assignments and
online tests after each unit, as well as two to three larger online tests addressing multiple units. The weight-
ings for each Mathematics methodology module have been indicated in Table 1. Previously, I did not assess
to this extent.
The design and preparation of multiple-choice questions were very time-consuming as between 80 to 150
MCQs were set for each unit test counting between 20 and 40 marks and extra questions were designed for the
larger tests to curb any forms of academic dishonesty. What made it more challenging is that online tests had
to be opened for longer times due to load shedding, at times students could be without power for 2 to 3 hours
once or twice a day when power outages were experienced at a regular interval. Having many more questions
than required for the total of a test meant that no student completed the same questions in the same order
nor did they all receive the exact questions, however, this required a great deal of extra time to set this range
of questions and to set them in such a manner that students received questions on different cognitive levels.
In addition, with large student numbers, it has also become extremely difficult and time-consuming to assess
longer questions online, specifically when one wants students to draw examples to accompany their explana-
tions, or to use fractions and equations due to the non-availability of certain software plug-ins in our current
Moodle system. I followed the following process when designing my MCQ, namely the questions and answers
were typed in Ms Word, then imported to Respondus where the layout and answers were checked again, and
next the MCQ tests were imported to Moodle and all the settings were affected.
After each test was written, the test was re-opened for students to check their answers and to indicate
to me as lecturer if there were incorrect ones. I checked and corrected answers where necessary and then run
a recalculation or regrade for students on Moodle, after which the final test assessment marks were updated
automatically. This checking period normally lasted for 24 hours and students were informed via email and on
their Whatsapp group that tests were open for review. Where students needed an explanation, I provided a writ-
ten explanation and (or) drawings where appropriate which were photographed and Whatsapp to the student
and in some instances shared with students during our online presence sessions when using Ms Teams. Further
engagements, if required, followed with students on an individual level, and also on the Whatsapp group as a
whole when necessitated. I believe that these reflective and student(s)-lecturer interactions promoted learning
and created social presence, teacher presence and thinking presence, and when more students engaged in
the Whatsapp and MS Teams groups, it also promoted these presences to a larger extent (see Garrison, 2007;
Garrison, Anderson & Archer, 1999, 2009, 2010). Regarding online assignment assessment tasks, students who
struggled and did not achieve the minimum 50% for an assignment were provided with a second opportunity
to achieve a maximum of 50% for their second attempt to not disadvantage students who scored above 50%,
540
ISSN 2538–7138 /Online/ (pp. 536-544)
a practice which is in line with our Faculty of Education’s humanizing pedagogy (see Salazar, 2013; Zinn, Adam,
Kurup & Du Plessis, 2016).
Concerns Related to Online Learning from a Student Perspective
South Africa has one of the highest Gini coefficients in the world (Department of Statistics, 2020), thus it
is argued that poverty played and still plays a vital role in limiting our students’ ability to have access to all the
resources that they would have had when they stayed on campus in hostels or near the university when there
is face-to-face teaching and learning. However, due to the postponement of face-to-face tuition, students have
not had the same contact with lecturers as previously while the data that they received were capped at 10
gigabytes of daytime data and 20 gigabytes of night-time data, something I alluded to in a previous section.
Poverty, housing contextual conditions that the majority of our student population had to endure, connectivity
issues, access to laptops or tablets despite being offered options by the university, as well as not being able
to return to campus due to COVID regulations by many were some of the challenges that our students had
to endure. These ICT-related and contextual challenges were not only a major stumbling block, as internet
connectivity was also problematic since students received 10 gigs of daytime data per month which was not
enough, as many of them reported that their data was depleted after 20 days of each month and did not have
in many instances the financial resources to purchase more data. The 20 gig of night data was available from
00:00 to 05:59 which required students who had to access the internet to do so at times when they have to
recuperate after their daytime sessions, resulting in many of them showing signs of sleep deprivation. This then
also probably led to less online teaching and learning presence during online sessions, as well as less time to be
in touch with me as the lecturer at certain times, thus influencing lecturer teaching and interaction presence.
Equally important, this could also have impacted opportunities to review the recorded Ms Teams online class
session videos which were made available after each class on Moodle.
Concerns Related to Online Learning From a Personal Perspective
The quality of my assessment instruments, especially the online tests, did not cause my greatest concern,
as I have indicated in a previous section how I have tried to ensure that the assessment was fair and addressed
different cognitive levels for each student without students having to answer questions at just one level. What
was my main concern was that I started to have doubts regarding the reliability of online assessment tests, i.e.
whether I would obtain similar results if my students wrote the tests in a controlled environment as before or
when measures could be implemented where student cameras had to be switched on, including browser lock-
ing while they were completing online assessments. Despite these measures of cameras and browser locking,
I believe that there would still be challenges if online assessments are not conducted in a traditional secure
venue, either online or through paper-based assessments. These doubts arose as there is a possibility that some
students might ‘sit’ together when responding to a test – even when they do not write the same test due to
having a large question bank to select from and (or) the possibility that someone else might assist when writing
the test, and (or) someone else might be writing their tests. Furthermore, without browser locking and even
with it, there is still the possibility to search for answers online on a second device even if the questions and
online answers found are not identical, but similar. On the other hand, trying to find the correct answer online,
though not identical, might again create a learning opportunity also. Another concern was the provision of a
longer time frame window during which online tests could be conducted due to load shedding schedules as a
result of electricity issues that our country experiences. These longer time frames have the potential to create
opportunities for screen image capturing that can easily be shared via email, Messenger, Whatsapp, etc. It is
thus quite possible that due to the non-controlled assessment environment, possibilities for academic dishon-
esty can increase (see Comas-Forgas, Lancaster, Calvo-Sastre & Sureda-Negre, 2021; Janke, Rudert, Peterson,
Fritz & Daumiller, 2021; Erguvan, 2021) and that is something that I have wrestled with, although I do not have
evidence to substantiate this concern or claim.
In addition, I have also compromised my assessments, as in many of my online tests, the focus has become
more on procedural fluency, however, this resulted as I have realised from my assessment analysis that many
of my students do not have the Mathematical procedural fluency as expected on an Intermediate Phase level
grade (grade 4 to 6), including Senior Phase level (grade 7 to 8), and hence I had to make provision that they
541
(pp. 536-544) ISSN 2538–7138 /Online/
also show the ability to perform certain calculations. As an experienced Maths lecturer, I consider that both
procedural fluency, as well as conceptual understanding, is vital. Thus, if one only focuses on conceptual un-
derstanding, it does not imply that a student can engage with calculations in front of learners in a classroom
context without a calculator, for example, or even with one and procedural fluency is also vital. As such, I posit
that my cognitive and teaching presences might have been compromised in general as a result of online
learning which I believe would not have been the case when on campus during face-to-face sessions instead
of screen-to-screen sessions. Lastly, providing two opportunities to write each test could have played a part
in students obtaining a higher mark during the second opportunity, however, marks did not always increase
for all students during the second attempt. Thus, providing multiple opportunities for each test, also created
additional challenges and I would revise this approach in the future by taking the average of the two tests as
the final assessed mark, rather than the highest mark of the two test opportunities.
Summing-up
The quality of assessment practices and the advocacy for the development of different assessment practices
to the traditional sit-down examinations have been debated before the COVID-19 pandemic and have forced
academics to re-think and reflect on their assessment practices which lead to different and innovative assess-
ment approaches (Hughes & Tait, 2021; Twist, 2021). The disruption led to deeper critical personal reflection by
myself and many international colleagues about the purpose of traditional assessments, however, at the same
time, it is important to ascertain to what extent the pandemic impacted students’ learning (Twist, 2021) and
school learners’ learning as indicated in the introduction section. As such, it is thus important to reflect and
gather data to establish to what extent these ‘pandemical-changes’ have impacted positively and (or) negatively
on students’ learning in higher education, as well as on learners at the school level. I noted that my averages
for pre-COVID tests in my modules were significantly lower than those attempted during COVID while assign-
ment results were quite similar on the other hand. This became a concern, as there is the possibility that the
integrity of tests as assessments could have been compromised due to possible academic dishonesty. On the
other hand, the provision of two opportunities to write a test with the highest mark awarded as the one taken
could also have impacted this, but as Table 1 shows, a great deal of extra support in the form of recordings of
online sessions, Whatsapp group discussions of queries, etc. to name a few only, might have also attributed to
higher averages and ‘better’ learning opportunities, as well as using MCQs more than longer questions.
One suggestion to assist with establishing to what extent learning has occurred, something I alluded to
in a previous section, is to utilise formative assessment frequently. Liberman, Levin and Luna-Bazaldua (2020)
have argued that formative assessment holds the key to ascertaining to what extent learners and students are
learning, as it can be implemented in synchronous and asynchronous modes. In my context, students were
provided with opportunities to formatively assess themselves when they were engaged with online examples
during the contact session or when we discussed homework during the online session. As such, I as the lec-
turer was not able to see their progress, but the students could formatively assess their competence through
self-assessment.
This then also raises the following questions that can be pursued, how frequently does one assess during
a semester in order not to over-assess leading to possible ‘assessment fatigue’ for the learners, students, teach-
ers and lecturers? How do we ensure that online formative and summative assessments in all forms reflect
the authentic ‘work’ of the learners at schools and students at universities? How can one ensure and minimize
possible academic dishonesty? How do we ensure that or determine whether our pre-COVID teaching and
learning and in-COVID teaching and learning yielded not only ‘better’ results but also ‘better’ learning? What
measures or instruments can we use to determine this? Are we retreating to the traditional pre-COVID pedagogy
related to our teaching praxis and assessment, or not? Why or why not is this the case? Lastly, to what extent
are the behavioural intention and actual usage of online learning tools continuing in a post-COVID society? The
above are only some of the issues that could be explored through quantitative, qualitative or mixed method
research. It is hoped that such research would feature in forthcoming issues related to Science and Technol-
ogy in the Journal of Baltic Science Education and may the debate on the concerns and positive outcomes of
online learning continue.
542
ISSN 2538–7138 /Online/ (pp. 536-544)
References
Anderson, L. W., Krathwohl, D. R., Airasian, P. W., Cruishank, K. A., Mayer, R., Pintrich, P. R., Raths, J. D., & Wittrock, M. C. (2001).
Taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. Longman.
Benton, T. & Craib, I. (2001). Philosophy of Social Science: The Philosophical Foundations of Social Thought. Palgrave.
Bloom, B. S., Engelhart, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D. R. (1956). Taxonomy of educational objectives: The
classification of educational goals. Vol. Handbook I: Cognitive domain. David McKay Company.
Bond, M., Bedenlier, S., Marín, V. I., & Händel, M. (2021). Emergency remote teaching in higher education: Mapping
the first global online semester. International Journal of Educational Technology in Higher Education, 18(1), 1–24.
https://doi.org/10.1186/S41239-021-00282-X/FIGURES/6
Comas-Forgas, R., Lancaster, T., Calvo-Sastre, A., & Sureda-Negre, J. (2021). Exam cheating and academic integrity breaches
during the COVID-19 pandemic: An analysis of internet search activity in Spain. Heliyon, 7(10), Article e08233.
https://doi.org:10.1016/j.heliyon.2021.e08233
Del Carmen Salazar, M. (2013). A humanizing pedagogy: Reinventing the principles and practice of education as a journey
toward liberation. Review of Research in Education, 37(1), 121–148. https://doi.org/10.3102/0091732X12464032
Department of Statistics (2022). How unequal is South Africa? https://www.statssa.gov.za/?p=12930
Impact of Covid-19 on Children’s Education in Africa (2022, August 26). Human Rights Watch. https://www.hrw.org/
news/2020/08/26/impact-covid-19-childrens-education-africa
Du Plessis, A., & Blignaut, S. (2020). Offline - online information and communication technology (ICT) teaching and learning
strategy in the age of COVID-19 and beyond. In N. Ndimande-Hlongwa, L. Ramrathan, N. Mkhize, & J. A. Smit (Eds.),
Technology-based teaching and learning in higher education during the time of COVID-1 (pp. 92-119). Alternation African
Scholarship Book Series, Volume #02. CSSALL Publishers: https://doi.org/10.29086/978-0-9869936-1-9/2020/AASBS02
Erguvan, I. D. (2021). The rise of contract cheating during the COVID-19 pandemic: A qualitative study through the eyes
of academics in Kuwait. Lang Test Asia, 11(34). https://doi.org/10.1186/s40468-021-00149-y
Erlam, G. D., Garrett, N., Gasteiger, N., Lau, K., Hoare, K., Agarwal, S., & Haxell, A. (2021). What really matters: Experiences of
emergency remote teaching in university teaching and learning during the COVID-19 pandemic. Frontiers in Education,
6, 397. https://doi.org/10.3389/FEDUC.2021.639842/BIBTEX
Garrison, D. R. (2007). Social, cognitive, and teaching presence issues. Journal of Asynchronous Learning Networks, 11(1),
61-72. https://eric.ed.gov/?id=EJ842688
Garrison, D. R., Anderson, T., & Archer, W. (1999). Critical inquiry in a text-based environment: Computer conferencing in
higher education. The Internet and Higher Education, 2(2–3), 87–105. https://doi.org/10.1016/S1096-7516(00)00016-6
Garrison, D. R., Anderson, T., & Archer, W. (2009). Critical thinking, cognitive presence, and computer conferencing in distance
education. American Journal of Distance Education, 21(1), 7–23. https://doi.org/10.1080/08923640109527071
Garrison, D. R., Anderson, T., & Archer, W. (2010). The first decade of the community of inquiry framework: A retrospective.
The Internet and Higher Education, 13, 5–9. https://doi.org/10.1016/j.iheduc.2009.10.003
Hodges, C. B., Moore, S., & Lockee, B. B. (2020). The difference between emergency remote teaching and online learning
school educational technology planning in Georgia view project TechTrends special section on hybrid or online
doctoral programs in educational technology view project. https://er.educause.edu/articles/2020/3/the-difference-
between-emergency-remote-teaching-and-online-learning
Hughes, G., & Tait, A. (2021). Evaluating pre- and post-pandemic reforms in assessment practices in University of
London Distance Education programme. A Centre for Distance Education Teaching and Research Award Report
undertaken by Gwyneth Hughes (UCL and CDE Fellow) and Alan Tait (Open university and CDE Fellow, June 2021).
https://www.london.ac.uk/sites/default/files/assessment-reforms-tra-report.pdf
Iglesias-Pradas, S., Hernández-García, Á., Chaparro-Peláez, J., & Prieto, J. L. (2021). Emergency remote teaching and students’
academic performance in higher education during the COVID-19 pandemic: A case study. Computers in Human Behavior,
119, Article 106713. https://doi.org/10.1016/J.CHB.2021.106713
Janke, S., Rudert, S. C., Petersen, Ä., Fritz, T. M., & Daumiller, M. (2021, May 5). Cheating in the wake of COVID-19: How
dangerous is ad-hoc online testing for academic integrity? https://doi.org/10.1016/j.caeo.2021.100055
Liberman, J., Levin, V., & Luna-Bazaldua, D. (2020, April 27). Are students still learning during COVID-19? Formative assessment
can provide the answer. World Bank Blogs. https://blogs.worldbank.org/education/are-students-still-learning-during-
covid-19-formative-assessment-can-provide-answer
Sayer, A. (2000). Realism and social science. SAGE.
Stake, R. E. (1995). The Art of Case Study Research. SAGE Publications.
Twist, L. (2021). Changing times, changing assessments: International perspectives. Educational Research, 63(1), 1-8.
https://doi.org:10.1080/00131881.2021.1876590
Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User acceptance of information technology: Toward a unified
view. MIS Quarterly, 425-478. https://www.jstor.org/stable/pdf/30036540
Venkatesh, V., Thong, J. Y. L., & Xu, X. (2016). Unified theory of acceptance and use of technology: A synthesis and the road
ahead. Journal of the Association for Information Systems, 17(5), 328-376. https://aisel.aisnet.org/jais/vol17/iss5/1/
Yin, R. K. (2014). Case study research design and methods (5th ed.). Sage.
543
(pp. 536-544) ISSN 2538–7138 /Online/
Zinn, D., Adam, K., Kurup, R., & Du Plessis, A. (2016). Returning to the source: Reflexivity and transformation
in understanding a humanising pedagogy. Educational Research for Social Change, 5(1), 70-93.
http://dx.doi.org/10.17159/2221-4070/2016/v5i1a5
Received: July 24, 2022 Revised: July 30, 2022 Accepted: August 05, 2022
Cite as: Du Plessis, A. (2022). A personal reflection on Covid-19 online teaching, learning, and assessment with suggestions
for research. Journal of Baltic Science Education, 21(4), 536-544. https://doi.org/10.33225/jbse/22.21.536
André du Plessis PhD, Professor, Nelson Mandela University, Gqeberha, South Africa.
E-mail: andre.duplessis@mandela.ac.za
ORCID: https://orcid.org/0000-0002-9089-4460
544
This is an open access article under the
International License
THE JIGSAW TECHNIQUE IN

LOWER SECONDARY PHYSICS
EDUCATION: STUDENTS’ ISSN 1648-3898 /Print/
ACHIEVEMENT, METACOGNITION ISSN 2538-7138 /Online/
AND MOTIVATION
Abstract. Physics teaching should facilitate
students to acquire knowledge, increase
metacognitive awareness and motivation
for learning physics. The implementation
of cooperative learning could be useful
Branislava K. Blajvaz, for improving teaching. The aim of this
research was to examine the effect of
Ivana Z. Bogdanović, cooperative learning (specifically the jigsaw
Tamara S. Jovanović, technique) on students’ achievement in
Jelena D. Stanisavljević, physics, metacognitive awareness, and
Milica V. Pavkov-Hrvojević motivation. An experiment with parallel
groups (experimental and control)
was carried out with 92 seventh-grade
students (in lower secondary education).
Jigsaw technique was implemented in
Introduction the experimental group, while the control
group was taught using teacher-directed
A number of students think that physics content in lower secondary edu- teaching method. Students’ physics
cation is difficult, and they cannot master it successfully. To improve physics achievement, metacognitive awareness
teaching, it is important to supervise students’ achievement, characteristics, and motivation were measured using
and feelings. Successful teaching results in functional knowledge, increase knowledge tests and questionnaires
in students’ metacognitive awareness and motivation for learning. Social (before and after the experiment).
interaction within the group is essential for the educational development Statistical analysis included calculations
of an individual, as it allows the exchange of ideas, views and attitudes in of Cronbach’s Alpha coefficient,
building and shaping knowledge necessary for bringing up common solu- performance of the Shapiro-Wilk test,
tions (Cole & Stanton, 2003). Since the ability of working in team is important Mann-Whitney U test and the Wilcoxon
quality of employers nowadays, it is important to foster students’ teamwork Signed Ranks Test. The research results
skills starting from their early age (Xiang & Han, 2021). showed that the implementation of jigsaw
Cooperative learning is probably the oldest form of group learning. technique in physics classes significantly
The theoretical foundations of cooperative learning stem from the research improved students’ physics achievement,
of Deutsch (1949), who has studied the effectiveness of groups that foster metacognitive awareness, and motivation.
cooperation rather than competition with regards to the individual and group Therefore, it can be suggested that
achievement. Johnson and Johnson (2002) have spent nearly four decades this technique is beneficial in lower
studying and understanding the effects and effectiveness of cooperative secondary physics education, and it can
learning within different levels of the educational process. be recommended to implement the jigsaw
Cooperative learning is a learning approach characterized by the re- technique in everyday school practice.
sponsibility of the individual to understand their own part of the material/ Keywords: cooperative learning,
task, but also the responsibility to understand/solve the material/task of the metacognitive awareness, parallel group
whole group (Johnson & Johnson, 2002). It is an active method of learning design, physics education
in which students in small groups work together on inquiry and problem
solving (Slavin, 2014). They help each other in learning, developing commu- Branislava K. Blajvaz, Ivana
nication techniques, strengthening self-confidence, and active participation Z. Bogdanović, Tamara S. Jovanović
University of Novi Sad, Republic of Serbia
in the learning process (Akçay & Doymuş, 2014). Johnson et al. (1998) have Jelena D. Stanisavljević
defined five key elements on which cooperative learning is based: positive University of Belgrade, Republic of Serbia
interdependence, face-to-face interaction, individual and group responsibil- Milica V. Pavkov-Hrvojević
University of Novi Sad, Republic of Serbia
ity, social skills development, and group evaluation.
545
(pp. 545-557) ISSN 2538–7138 /Online/
According to Davidson and Major (2014), the following learning strategies supporting cooperative learning have
been developed: the group research technique (Sharan & Sharan, 1990), the shared learning technique (Johnson
et al., 1998), the read-write-present technique or think-match-present technique (Lyman, 1992), the timed-pair-
share technique (Kagan & Kagan, 2009), the three-step interview technique (Kagan & Kagan, 2009) and finally the
jigsaw technique (Aronson & Patnoe, 2011).
The jigsaw is a cooperative learning technique based on the work and cooperation of students within the home
and expert groups. The teacher forms the home groups of four to five students (diverse in terms of gender, ability,
and other students’ characteristics). Each member of the home group is assigned some material (content/tasks)
which then falls under the student’s responsibility. After sharing the material, the students that were assigned the
same task (one from each home group) are transferred to form the expert groups, in which all the members work
together to master the same task. While working on the task, the members of the expert group cooperate with
each other, communicate, explain to each other, convey information, and discuss, but also devise a plan on how
to transfer their part of the task to other members of the home group (Fortner, 1999). After returning to the home
group, individual experts must cooperate to reach a pre-determined solution together. In this way, cooperative
learning becomes meaningful to everyone in the learning process. Students exhibit different efforts in social rela-
tions and learning behavior, in the search of possible solutions, in the exchange of information, thus expanding
their understanding of the subject (Karacop, 2017).
Literature Review
There are several research studies that support a wide range of outcomes of cooperative learning, for in-
stance, students’ academic achievement in various subjects, social relations, group behavior, social cohesion, or a
person with disabilities inclusion into the group (Slavin, 2014). The cooperative learning, in contrast to competi-
tive and individual learning, enables greater achievement on tests in several subjects for students from primary
school to university (Johnson & Johnson, 1988). Slavin (1983) has indicated that cooperative learning resulted
in better students’ achievement than the traditional learning in more than half of the relevant research about
implementation of cooperative learning; some of the research studies have found no difference and only few
research have shown that the traditional learning resulted in better students’ achievement than the cooperative
learning. Kagan (1994) has shown that students with low achievement made the most progress when applying
the cooperative method. The decades of research into the teaching of physics using the cooperative method
have led to incorporating cooperative learning within the physics class reform around the world to increase the
students’ achievement in physics (Awoniyi & Kamanga, 2014; Eshetu et al., 2017). Howe et al. (2007) and Topping
et al. (2011) have shown significant improvement in students’ academic achievement and the improvement in
social relations within groups.
Motivation during cooperative learning may depend on an adequate structure of the task set by the teacher,
the students’ desire to solve the task within the group as successfully as possible, and the achievement of a
predicted reward for solving the problem (Slavin, 1995). Each member of the group can achieve personal goals
and be responsible for his/her part of the task, help other members of the group in solving the task to make the
group as a whole more successful, encourage other members to work and perform better. Students strengthen
their interpersonal structure, but also their social relations with other members of the group through coopera-
tion within that group. Teacher-directed teaching method in physics classes, in which teachers are active and
students are passive, affects students’ motivation negatively (Baş, 2010; Schaal, 2010). Students simply listen,
record, and become burdened with information that they merely memorize, without the opportunity to ask
questions and come to a solution by dealing with the problem itself (Cano et al., 2013). Humphreys et al. (1982)
have shown that students who studied physics in cooperative groups described their learning experience much
more positively than the other respondents, when compared to the competition groups and individual work.
Metacognition refers to metacognitive knowledge, metacognitive regulation, and metacognitive experi-
ences (Efklides, 2006). Metacognitive knowledge refers to the awareness of oneself as a learner, the knowledge
of learning strategies and the appropriacy of a particular strategy for implementation. Metacognitive regula-
tion includes planning of the learning process, managing information, monitoring, evaluating, and debugging
during the thinking/learning process. Metacognitive experiences include a variety of feelings and judgments
related to thinking/learning (e.g., feeling of knowing, judgment of learning). Studies have shown that higher
metacognition is related to a more successful cooperative learning (Bernard & Bachu, 2015; Hurme et al. 2015,
546
ISSN 1648–3898 /Print/ THE JIGSAW TECHNIQUE IN LOWER SECONDARY PHYSICS EDUCATION: STUDENTS’
ISSN 2538–7138 /Online/ (pp. 545-557)
Kramarski & Mevarech, 2003). Further, Bilgin and Geban (2006) have emphasized that the active involvement
of students in the learning process during the application of cooperative learning results in the improvement
of critical thinking, reasoning and problem solving, which is directly reflected in the change of the students’
metacognition. On the other hand, Slavin (1995) has stated that cognitive understanding is a key to the interac-
tion between group members, linking the success of the method to the students’ metacognition and vice versa.
Chang and Mao (1999) have suggested that each method of cooperative learning, if adequately used, can allow
students to increase the understanding of the learning flow in all the members of the group. The increase in the
academic achievement directly affects the change of metacognitive experience – the increase in self-esteem,
positive attitude towards learning and school, and others (Bilgin & Geban, 2006). Alternatively, Cheong (2010)
has mentioned that cognitive development during cooperative learning depends on the teachers’ approach
and interaction with the student groups.
Research Problem
The achievement in physics directly depends on the students’ interest in studying physics in schools and
universities (Smithers, 2006). Young people often believe that physics as a science is too difficult. It is unpopular
since requiring skills to think, precise and correct use of language, solve algebra problems and use arguments
(Demkanin, 2018). The reasons for the declining interest in studying one of the fundamental sciences could be
the fact that physics has a reputation of a ’difficult and boring’ science that deals with abstract concepts that the
students find hard to conceive (Sillitto & MacKinnon, 2000). Moreover, students around the world find most of the
physics curriculum boring, uninteresting, and irrelevant (Lavonen et al., 2007; Lyons, 2006) and accordingly, the
interest in studying physics declines during the course itself.
The curriculum of physics in lower secondary education in Serbia has been reformed with the aim of facilitat-
ing learning. Moreover, various methods for teaching physics are being proposed (Cvjetićanin et al., 2015). The
focus has shifted from learning tasks to learning outcomes. This includes modernizing the learning and teaching
processes, such as teachers’ training, and the implementation of the group and project-based learning that enable
the accomplishment of learning outcomes.
Considering the above characteristics of teaching physics and the method of cooperative learning, there is
a need to implement cooperative learning in the realization of physics content in lower secondary education and
measure its effects. There is a lack of comprehensive research that simultaneously examined students’ achievement,
metacognition, and motivation (looking at their components separately) when cooperative learning is imple-
mented. The importance of this research is reflected in providing a comprehensive insight into the possibilities of
implementing cooperative learning, specifically jigsaw technique, in physics classes.
Research Aim and Research Questions
The aim of this research was to explore the effect of the jigsaw technique in lower secondary education
on student’s physics achievement, metacognitive awareness, and motivation. Therefore, the following research
questions were defined: (1) Would the implementation of the jigsaw technique increase students’ achievement in
physics? (2) Would the implementation of the jigsaw technique increase students’ metacognitive awareness? (3)
Would the implementation of the jigsaw technique increase students’ motivation for learning physics? Answers to
these questions can give an important insight into the extent in which the implementation of the jigsaw technique
can improve physics classes.
Research Methodology
Research Design
To answer the above-mentioned research questions, an experiment with parallel group of students (ex-
perimental and control) was applied. The experimental factor was cooperative learning (jigsaw technique). It was
introduced in the experimental group of students (E group), while the control group (C group) was taught using
teacher-directed teaching method. During the research, both groups were taught the same content by the same
teacher. The research was carried out from the beginning of September to the end of December in 2019.
547
(pp. 545-557) ISSN 2538–7138 /Online/
Research Sample
The research was carried out in a school in the Republic of Serbia. A convenient research sample was selected.
Four classes of seventh grade (13 years old) participated in the research, i.e., 92 students (38 boys and 54 girls). This
students’ age was selected because physics as separate subject is introduced in the sixth grade of lower secondary
education in the Republic of Serbia, so the seventh-grade students already have experience of one year in learn-
ing this subject. In the seventh grade, the students are taught about basics of mechanics (four teaching topics)
and thermal phenomena (one teaching topic) in physics. The two class groups formed the E group (a total of 44
respondents, 17 boys and 27 girls) and another two class groups formed the C group (a total of 48 respondents, 21
boys and 27 girls). The E and C groups were formed based on equality: to have approximately the same number of
students, similar gender distribution, the same level of students’ achievement in physics, metacognitive awareness,
and motivation for learning physics (Table 2).
Prior to the implementation of the research, the school board gave the ethical approval for carrying out
the research. Moreover, the topic, purpose and the procedure of the research were introduced to the students’
parents and the formal parental consent on the students’ participation in the research was obtained. The research
was anonymous, and all the participants were voluntarily involved in the research and could leave it at any time.
Research Instruments
For the purpose of this research, knowledge pre-test (covering knowledge about acceleration and Newton’s
second law of motion) and post-test (covering knowledge about motion under the influence of a force) were
created. Each test consisted of five items which were scored differently (the tasks requiring the higher levels of
knowledge for the correct answers were scored with more points) and the maximum score that a student could
gain was 100 points. The tests were further verified by an expert group. The tests were evaluated as appropriate for
use. The reliability of the tests was verified by calculation of Cronbach’s alpha; the value of this coefficient indicated
that these tests could be considered reliable (Table 1).
Students’ metacognitive awareness was estimated using the Serbian version of Metacognitive Awareness
Inventory (Bogdanović et al., 2015). This is a translated and adapted version of the original questionnaire developed
by Schraw and Dennison (1994). The adapted version retained 31 items with the 5-point Likert-type scale responses
(from “strongly disagree”-1 to “strongly agree”-5). This adaptation of the questionnaire proved as applicable for
the participants in this research; also, it covered the following metacognitive components: declarative knowledge,
procedural knowledge, conditional knowledge, planning, information management, monitoring, debugging and
evaluation. The Cronbach’s alpha coefficient for each subscale indicated acceptable reliability (Table 1).
Students’ motivation toward science learning (SMTSL) questionnaire, originally developed by Tuan et al.
(2005), was also used in its’ Serbian version (Olić et al., 2016). This version consists of 29 items with the 5-point
Likert-type scale responses (from “strongly disagree”-1 to “strongly agree”-5). This questionnaire covers five aspects
of motivation: self-efficiency, active learning strategies, physics learning value, the performance goal, and the
achievement goal. For the listed subscales, the calculated values of the Cronbach’s alpha coefficient suggest their
reliability (Table 1).
Research Procedure
Before the research started, all the students took the pre-test in physics that served for the assessment of
students’ physics achievement. This testing was realized through a standard paper and pen test that lasted for 45
minutes. In another class, students filled out questionnaires on their motivation for learning physics and metacogni-
tive awareness. The completion of the questionnaires was also realized through standard testing (paper and pen)
for which students had 30 minutes. Based on the pre-test results and several aligned criteria already mentioned,
two classes were selected for the E group and other two classes for the C group. During the research, the following
teaching units were covered in the topic Motion under the influence of a force: Acceleration, Non-uniform motion,
Graph of non-uniform motion, Motion under the influence of gravity, Acceleration due to gravity, Free fall, Tossed
objects upward, and Tossed objects downward.
In the E group, teacher explained the basics of cooperative learning and jigsaw technique to the students using
video examples. After that, the E group was divided into five home groups (each home group had five members)
548
ISSN 2538–7138 /Online/ (pp. 545-557)
based on the students’ achievement, motivation, and social skills (based on the teacher’s judgment). The groups
were heterogeneous according to all the criteria, so that the students with low achievement could improve their
knowledge, the demotivated ones could develop motivation for learning physics, and the marginalized ones could
be included in social life of the class.
For the realization of the jigsaw method, for each topic, the teacher prepared worksheets with five tasks for
the home groups (the example for teaching unit Free fall is given in Appendix 1). The home groups were diverse
in terms of gender, ability, and other students’ characteristics. The tasks set by the teacher on the worksheets were
aligned with the learning outcomes. Within each home group, the students themselves discussed the topic and
they could freely choose one of the suggested tasks, after which they were separated into the expert groups (from
each home group a student that selected a particular task was transferred to the expert group dealing with that
task) (Appendix 2). Although cooperative learning stemmed from the home group, all the steps and elements of
cooperative learning are visibly realized in the expert group. The work within the expert group was based on the
problem identification, the consideration of possible approaches to the problem research, problem research plan
creation, communication with each other with the purpose of explaining the smallest details of the problem by
raising as many questions as possible. During the research realization, the students practiced communication,
encouraged each other, helped each other, and praised each other after achieving the goal. After working in the
expert groups, where they developed the sense of individual responsibility, positive interdependence, and social
skills in addition to the academic skills in physics, the students returned to the home groups. Each student had to
explain the process by which he/she realized his/her task to the rest of the home group and pass on conclusions
and new knowledge to them. While one student explained his/her part of the work, the other students filled in
their worksheets. The home group had one large piece of paper where each student in the group filled in the part
to complete the group task. During the process of transferring the acquired knowledge among the students, the
students themselves evaluated their work, improved social skills, and positive interdependence. At the very end,
each home group presented their work to the teacher or the whole class, and the evaluation of the group’s work
was carried out (Appendix 2).
During the treatment of the topic Motion under the influence of a force, within the C group, the teacher ap-
plied the teacher-directed teaching method with the predominant ‘ex cathedra’ lecturing, accompanied by some
occasional group and individual students’ work. The presentations and video materials were used while processing
the new content, and the students were solving tasks on the blackboard; the quizzes were used for the recap of
the material, either in the introductory or the final parts of the lesson.
The post-test was realized after the end of the research in the form of a standard physics test (covering the
content realized during the research) lasting for 45 minutes. At the end of the research, the students had 30 minutes
for filling questionnaires of motivation and metacognition (pen and paper).
Data Analysis
In the data analysis, students’ achievement was expressed in scores on physics knowledge tests. Metacognitive
awareness was expressed in scores on subscales on MAI (declarative knowledge, procedural knowledge, conditional
knowledge, planning, information management, monitoring, debugging, evaluation) and motivation for learning
physics was expressed in scores on subscales on SMTSL (self- efficacy, active learning strategies, physics learning
value, performance goal and achievement goal). The Cronbach’s Alpha coefficient was calculated for each subscale
to estimate the reliability of instruments used for data collection. The Shapiro-Wilk test was performed to check
normality of data and since the data were not normally distributed within groups, for further analysis nonparametric
tests were used. All variables were analyzed using Mann-Whitney U test to explore possible differences between C
group students and E group students, on both pre-test and post-test. Besides, for all variables was performed the
Wilcoxon Signed Ranks Test to compare post-test and pre-test scores within Group C, as well as Group E.
Research Results
To reflect the effects of the jigsaw technique on the students’ physics achievement, their metacognitive
awareness and their motivation in detail, the subscales measuring different components of students’ metacogni-
tive awareness and motivation were analyzed separately. Accordingly, Cronbach’s alpha coefficient was calculated
for these subscales (Table 1).
549
(pp. 545-557) ISSN 2538–7138 /Online/
Table 1
The Cronbach’s Alpha Coefficient for the Subscales of the Research Instruments
Cronbach’s Alpha
Pre-test Post-test
Physics achievement .70 .65

Metacognitive awareness Declarative knowledge .53 .77
Procedural knowledge .78 .65
Conditional knowledge .74 .81
Planning .93 .92
Information management .82 .90
Monitoring .86 .90
Debugging .61 .77
Evaluation .89 .92
Motivation Self- efficacy .90 .96
Active learning strategies .91 .96
Physics Learning Value .82 .91
Performance Goal .70 .52
Achievement Goal .78 .83
Since the p values of Mann-Whitney U test were below .05 for most variables (except for planning, evaluation,
and self- efficacy in the C group, monitoring, and performance goal in both groups), normal distribution could
not be assumed.
After equalizing the C and the E group, students in both groups showed similar physics achievement, meta-
cognitive awareness, and motivation in the pre-test (Table 2).
Table 2
Mann-Whitney U Test and Median Values for the Pre-Test and Post-Test Scores
Pre-test Post-test
Variable
Group Md U z p Md U z p
C 67.5 62.5
Physics achievement 872 -1.60 .11 575 -3.90 <.0001
E 80.0 85.0
C 4.33 4.33
Declarative knowledge 675 -3.15 .002 853 -1.77 .076
E 3.67 4.33
C 4.00 3.67
Procedural knowledge 968 -0.87 .38 428 -5.07 <.0001
E 4.00 4.33
C 4.00 3.67
Conditional knowledge 957 -0.96 .34 688 -3.05 .002
E 4.33 4.67
C 3.40 3.30
Planning 831 -1.92 .054 399 -5.25 <.0001
Metacognitive E 4.00 4.40
awareness Information manage- C 4.00 4.10
948 -1.02 .30 629 -3.51 <.0001
ment E 4.20 4.80
C 3.60 3.60
Monitoring 834 -1.90 .057 471 -4.70 <.0001
E 4.00 4.60
C 4.33 4.33
Debugging 1023 -0.45 .66 654 -3.36 .001
E 4.33 4.67
C 3.38 3.75
Evaluation 892 -1.45 .15 570 -3.95 <.0001
E 4.00 4.50
550
ISSN 2538–7138 /Online/ (pp. 545-557)
Pre-test Post-test
Variable
Group Md U z p Md U z p
C 3.71 3.57
Self- efficacy 897 -1.41 .16 528 -4.28 <.0001
E 4.14 4.71
Active learning strate- C 4.00 4.00
753 -2.53 .012 513 -4.38 <.0001
gies E 4.50 4.75
Physics Learning C 3.80 4.00
Motivation 887 -1.49 .14 440 -4.95 <.0001
Value E 4.00 4.60
C 3.00 2.80
Performance Goal 795 -2.21 .027 959 -0.94 .345
E 2.80 2.60
C 4.25 4.50
Achievement Goal 858 -1.73 .083 877 -1.60 .110
E 4.75 4.75
Regarding the students’ metacognitive awareness, it was shown that there was a significant difference in the
declarative knowledge of the C group students and the E group students, in favor of the C group students. Within
the students’ motivation for learning physics, the difference was shown to be significant in active learning strategies
(in favor of the E group students) and the performance goal (in favor of the C group students).
In the post-test scores, the results of Mann-Whitney U test showed that the statistically significant difference
existed between the groups C and E in all the variables measured, except for the following three: declarative
knowledge within metacognitive awareness, the performance goal, and the achievement goal within motivation
(Table 2). All the discovered differences were in favor of the E group students.
For the C group students, the Wilcoxon Signed Ranks Test showed that only four variables were significantly
different on the post-test with regards to the pre-test. These four variables showed an unexpected decrease from
the pre-test to the post-test (Table 3).
Table 3
The Statistics of Wilcoxon Signed Ranks Test Exploring the Differences between the Pre-Test and the Post-Test Scores in the
Group C and Group E
Group C Group E
M Z p M Z p
Pre- Post- Pre- Post-

Variable
test test test test
Physics achievement 64.5 60.8 -1.17a .24 74.7 82.0 -2.41a .016
Declarative knowledge 4.04 3.97 -0.99a .32 3.62 4.34 -5.05a <.0001
Procedural knowledge 3.76 3.44 -3.66a .000 3.96 4.31 -3.29a .001
Conditional knowledge 3.85 3.63 -2.39a .017 4.03 4.30 -2.89a .004
Metacognitive Planning 3.28 3.16 -1.40 a
.16 3.71 4.23 -5.02 a
<.0001
awareness Information management 3.86 3.90 -0.23b .82 4.11 4.55 -5.02a <.0001
Monitoring 3.46 3.46 -0.22 b
.82 3.85 4.32 -4.97 a
<.0001
Debugging 4.10 4.08 -0.27a .79 4.20 4.60 -4.43a <.0001
Evaluation 3.40 3.48 -0.57b .57 3.72 4.26 -4.65a <.0001
Self- efficacy 3.66 3.41 -2.73 a
.006 3.83 4.37 -5.27 a
<.0001
Active learning strategies 3.86 3.77 -1.32a .19 4.27 4.52 -3.74a <.0001
Motivation Physics Learning Value 3.75 3.72 -0.01 b
.99 3.98 4.44 -5.47 a
<.0001
Performance Goal 3.28 2.80 -4.69a .000 2.84 2.71 -2.07b .038
Achievement Goal 4.23 4.32 -1.81 b
.071 4.50 4.59 -2.07 a
.038
a
. Based on positive ranks.
b
. Based on negative ranks.
551
(pp. 545-557) ISSN 2538–7138 /Online/
For the E group students, the Wilcoxon Signed Ranks Test showed that all the variables were significantly
different between the post-test and the pre-test. All the variables showed an increase from the pre-test to the
post-test, except for the performance goal where the decrease was shown (Table 3).
Discussion
The results of this research showed that the changes in the physics teaching, particularly the implementation
of the jigsaw technique in physics classes, can lead to significant changes in the students’ achievement in physics,
increase motivation for learning physics and metacognitive awareness.
The analysis of the students’ achievement in the pre-test showed that there were no differences between the
pre-test scores of four classes of the sample (Table 2). These results indicate that before the research, the teaching
process in physics has been carried out in the same way in all the classes, and that the researchers adequately
divided the classes into C and E groups at the beginning of the research. After the implementation of the jigsaw
technique in the E group and the teacher-directed teaching method in the C group, a post-test of acquired physics
knowledge was administered to students. The students from the E group had significantly higher achievement
compared to the pre-test, while in the C group there was no significant difference between pre-test and post-test.
A significant difference was shown between the students’ post-test achievement in E and C group, in favor of the
students’ achievement in the E group. These results, indicating that the cooperative learning has a positive effect
on students’ achievement in physics, are in line with the results of the research carried out by Topping (2011) et
al. Ho and Boo (2007).
According to the previous research, students’ metacognitive abilities may (Bilgin & Geban, 2006, Chang &
Mao, 1999) or may not (Slavin, 1995) develop during the application of the cooperative learning, depending on
the adequacy of the method and the approach, as well as the interaction of the teachers with groups of students
(Cheong, 2010). In this research, the students’ metacognitive awareness was measured using questionnaire before
and after the pedagogical experiment. A significant difference was only shown between the declarative knowledge
in pre-test of students in the E and C group, in favor of the students in the C group. This result is in accordance
with E and C group being equalized. The post-test results showed that there was no significant difference between
the declarative knowledge of students in C and E group, that is, the E group students caught up with the C group
students. The progress of the E group during the research is in line with the previous research, confirming that
students’ metacognition develops by applying cooperative learning (Bernard & Bachu, 2015; Hurme et al., 2015;
Kramarski & Mevarech, 2003). In the post-test, the values of all other metacognitive components (procedural
knowledge, conditional knowledge, planning, information management, monitoring, debugging and evaluation)
were significantly higher in the E group than in the C group. The obtained results are in accordance with the already
cited literature (Bernard & Bachu, 2015; Hurme et al., 2015; Kramarski & Mevarech, 2003) in which it was shown that
the cooperative learning develops students’ metacognition. Bilgin and Geban (2006) showed that active student
participation in the cooperative learning leads to the development of critical thinking, reasoning and problem
solving, which directly affects the development of students’ metacognition. If each step that students apply and
each element of cooperative learning are observed in detail, including the division of tasks, planning their realiza-
tion, carrying out research and the knowledge acquisition, individual responsibilities for the task, development
of cooperation, social skills, self-evaluation, and the group evaluation, it can be suggested that students develop
their metacognitive abilities. On the other hand, the obtained results of the increase in metacognitive aware-
ness can be directly related to the increase in the students’ achievement in physics, which aligns with the results
of research carried out by Bogdanović et al. (2015). Finally, through the application of the cooperative learning,
students develop their independence in work and the increase in all the metacognitive components is expected.
Besides, research results showed that the C group students showed the decrease in the procedural and conditional
knowledge, which indicated that the students are passive, and the metacognition is not nurtured in the teacher-
directed teaching method. For the E group students, all the metacognitive components had significantly higher
values on the post-test than on the pre-test. Accordingly, it can be indicated that the implementation of the jigsaw
technique had a positive effect on the development of students’ metacognitive awareness.
In the pre-test there was no significant difference between C and E group regarding motivational aspects,
except for the aspect of active learning strategy which was in favor of the E group students while the aspect of
the performance goal was in favor of the C group students. According to the data obtained on the post-test, the
variables of self-efficiency, active learning strategies and the physics learning value were significantly higher for
552
ISSN 2538–7138 /Online/ (pp. 545-557)
the E group students than for the C group students. These results are in line with the results of the research on the
effect of cooperative learning on students’ motivation carried out by Slavin (1995). The difference in the first two
variables can be interpreted through the E group students’ desire to successfully solve the set task within the expert
group while the jigsaw technique was implemented, and to transfer their knowledge to the students within the
home group with the aim of attaining awards for being successful. Besides, the increase in self-efficiency can be
achieved through accomplishing personal goals and the development of individual responsibility for the task. The
significantly higher level of physics learning value of E group students than C group students confirmed that the
implementation of cooperative learning in physics classes can lead to the change in status of physics as a school
subject. No significant difference was obtained within the performance goal and achievement goal between the
E and C group students. By comparing the data on the student motivation in the pre-test and post-test, a decrease
in the students’ self-efficiency has been shown in the group C, which might be expected for long-term application
of teacher-directed teaching method. This result points to the need for a change in the way the teachers deal with
students. The same has been shown for the performance goal, where the cause of the decrease in value could be
the fact that students work hardest at the beginning of the school year. As the semesters go by, students count
on the good impression they left, or they simply lose enthusiasm to do better because they already have grades
that they have accepted as “theirs”. For the E group students, there is an increase in the self-efficacy, active learning
strategies, the physics learning value, and the achievement goal. The performance goal, like within the group C,
has slightly declined compared to the pre-test, which could be explained by the students’ satisfaction with their
own performance, and their focus on the realization of the cooperative learning, and not on the grading.
In this research, it was shown that the implementation of the jigsaw technique in the seventh grade of lower
secondary education increased students’ achievement in physics, metacognitive awareness, and motivation for
learning physics. However, there are some research limitations. The main limitation of the research lies in the sam-
pling procedure. The researchers used pre-formed classes instead of randomly selecting students to form C and
E groups. Also, only the seventh-grade students participated in the research. The students of different age might
give different outcomes with regards to motivation and metacognition, which might change the overall outcome
of this research. Besides, there was a limitation in the form of the width of the treated physics content since the
research was limited to a certain period. Finally, the self-assessment scales were used, which are quite demanding
for younger students and there is also a possibility that students wanted to show themselves in a ’better’ light so
they gave answers they recognized as preferable.
Conclusions and Implications
The purpose of this research was to explore the effect of jigsaw technique in the seventh grade of lower
secondary education on students’ achievement in physics, metacognitive awareness, and motivation. Students’
metacognition and motivation were recognized by educators worldwide as important students’ qualities that
should be nurtured through education. These variables, along with students’ achievement, were measured and
explored in many studies. However, there is a further need for validating various techniques that would improve
students’ physics achievement, metacognition, and motivation for learning physics since the difficulties are often
most pronounced in complex school subjects, such as physics. The jigsaw technique is proposed by a number of
researchers as useful for improving teaching process and although it has been already explored in relation to vari-
ous school subjects and different levels of education, its effects on physics achievement of 13-year-old students,
metacognition and motivation for learning physics were not previously explored. The uniqueness of this research
also lies in the fact that all components of students’ metacognition and different aspects of students’ motivation for
learning physics when cooperative learning, specifically jigsaw technique, is implemented were examined separately.
Despite the above-mentioned research limitations, all the three research questions were successfully an-
swered. It was shown that the implementation of the jigsaw technique increased the seventh-grade students’
achievement in physics, metacognitive awareness, and motivation for learning physics. In addition, this research
highlights that all the metacognitive components and all the motivational aspects were positively affected by
the implementation of the jigsaw technique. Moreover, the significance of this research reflects in the fact that,
so far, there has been no research that deals with the effect of jigsaw technique in physics classes on these three
variables simultaneously. Providing a comprehensive insight into the possibilities of implementing jigsaw tech-
nique in lower secondary education physics classes can be of great value to science teachers worldwide because
physics content is often least popular within school subject Science. Further, this research provides new insight
553
(pp. 545-557) ISSN 2538–7138 /Online/
into the potential to encourage students’ metacognition and motivation with appropriate teaching techniques.
The results of this study indicate that the training of teachers and students about the implementation of the jigsaw
technique is advisable. Also, the findings obtained in this research may be the starting point in the future research
on the application of cooperative learning. On the same note, the verification of these results can be realized by
increasing the sample and expanding the age range of students, increasing the time interval, and increasing the
volume of processed material, introducing other instruments for assessing motivation and metacognition, and
including other variables in the research.
References
Akçay, N. O., & Doymuş, K. (2014). The effect of different methods of cooperative learning model on academic achievement in
physics. Journal of Turkish Science Education 11(4), 17–30. https://doi.org/10.12973/tused.10124a
Aronson, E., & Patnoe, S. (2011). Cooperation in the classroom: The jigsaw method. Pinter & Martin.
Awoniyi, S. A., & Kamanga, J. (2014). Comparative study of effectiveness of cooperative learning strategy and traditional
instructional method in the physics classroom: A case of Chibote girls’ secondary school, Kitww District, Zambia. European
Journal of Educational Sciences, 1(1), 30–41.
Bandura, A., Freeman, W. H., & Lightsey, R. (1997). Self-efficacy: The exercise of control. Journal of Cognitive Psychotherapy, 13(2),
158–166. https://doi.org/10.1891/0889-8391.13.2.158
Baş, S. (2010). Effects of multiple intelligences instruction strategy on students’ achievement levels and attitudes towards English
lesson. Cypriot Journal of Educational Sciences, 5(3), 167–180.
Bilgin, I., & Geban, O. (2006). The effect of cooperative learning approach based on conceptual change condition on
students’ understanding of chemical equilibrium concepts. Journal of Science Education and Technology, 15(1), 31–46.
https://doi.org/10.1007/s10956-006-0354-z
Bernard, M., & Bachu, E. (2015). Enhancing the metacognitive skill of novice programmers through collaborative
learning. In: A. Peña-Ayala (Ed.), Metacognition: Fundaments, applications, and trends (pp. 277–298). Springer.
https://doi.org/10.1007/978-3-319-11062-2_11
Bogdanović, I., Obadović, D. Ž., Cvjetićanin, S., Segedinac, M., & Budić, S. (2015). Students’ metacognitive awareness and physics
learning efficiency and correlation between them. European Journal of Physics Education, 6(2), 18–30.
Chang, C-Y., & Mao, S-L. (1999). The effects on students’ cognitive achievement when using the cooperative learning method in
earth science classrooms. School Science and Mathematics, 99(7), 374–379.
Cheong, C. (2010). From group-based learning to cooperative learning: A metacognitive approach to project-based group
supervision. Informing Science: The International Journal of an Emerging Transdiscipline, 13, 73–86. https://doi.org/10.28945/1173
Cano, E. M., Ruiz, J. G., & Garcia, I. A. (2013). Integrating a learning constructionist environment and the instructional design
approach into the definition of a basic course for embedded systems design. Computer Applications in Engineering Education,
23(1), 36–53. https://doi.org/10.1002/cae.21574
Cole, H., & Stanton, D. (2003). Designing mobile technologies to support co-present collaboration. Personal and Ubiquitous
Computing, 7(6), 365–371. http://dx.doi.org/10.1007/s00779-003-0249-4
Cvjetićanin, S., Obadović, D., & Rančić, I. (2015). The efficiency of student-led and demonstration experiments in initial physics-
chemistry education in primary school. Croatian Journal of Education, 17(3), 11-39.
Davidson, N., & Major, C. H. (2014). Boundary crossings: Cooperative learning, collaborative learning, and problem-based learning.
Journal on Excellence in College Teaching, 25(3/4), 7–55.
Demkanin, P. (2018). Concept formation: Physics teacher and his know-how and know-why. Journal of Baltic Science Education,
17(1), 4–7. https://doi.org/10.33225/jbse/18.17.04
D eutsch, M. (1949). A theor y of cooperation and competition. Human Relations, 2, 129–152.
https://doi.org/10.1177/001872674900200204
Efklides, A. (2006). Metacognition and affect: What can metacognitive experiences tell us about the learning process? Educational
Research Review, 1, 3-14. https://doi.org/10.1016/j.edurev.2005.11.001
Eshetu, F., Gebeyehu, D., & Alemu, M. (2017). Effects of cooperative learning method on secondary school students’ physics
achievement. International Journal of Multidisciplinary and Current Research, 5(3), 669-676. http://ijmcr.com/wp-content/
uploads/2017/06/Paper26669-676.pdf
Fortner, R. W. (1999). Using cooperative learning to introduce undergraduates to professional literature. Journal of College Science
Teaching, 28(4), 261–265.
Hennessey, M. G. (1999). Probing the dimensions of metacognition: Implications for conceptual change teaching-learning [Meeting
presentation]. Annual meeting of the National Association for Research in Science Teaching, Boston.
Ho, F. F., & Boo, K. W. (2007). Cooperative learning: Exploring its effectiveness in the physics classroom. Asia-Pacific Forum on
Science Learning and Teaching, 8(2), 1–21.
Howe, C., Tolmie, A., Thurston, A., Topping, K., Christie, D., Livingston, K., Jessiman, E., & Donaldson, C. (2007). Group
work in elementary science: Organizational principles for classroom teaching. Learning & Instruction, 17(5), 549–63.
https://doi.org/10.1016/j.learninstruc.2007.09.004
554
ISSN 2538–7138 /Online/ (pp. 545-557)
Hurme, T.-R., Järvelä, S., Merenluoto, K., & Salonen, P. (2015). What makes metacognition as socially shared in mathematical
problem solving? In: A. Peña-Ayala (Ed.), Metacognition: Fundaments, applications, and trends (pp. 259–276). Springer.
https://doi.org/10.1007/978-3-319-11062-2_10
Humphreys, B., Johnson, R. T., & Johnson, D. W. (1982). Effects of cooperative, competitive, and individualistic learning on students’
achievement in science class. Journal of Research in Science Teaching, 19(5), 351–356. https://doi.org/10.1002/tea.3660190503
Johnson, R. T., & Johnson, D. W. (1988). Cooperative learning: Two heads learn better than one. Transforming Education: In Context,
18, 34.
Johnson, D. W., Johnson, R. T., & Smith, K. A. (1998). Active learning: Cooperation in the college classroom. Interaction Book Company.
Johnson, D. W., Johnson, R. T., & Holubec, E. J. (1998). Cooperation in the classroom. Interaction Book Company
Johnson, D. W., & Johnson, R. T. (2002). Cooperative learning and social interdependence theory. In: R. Scott Tindale et al. (Eds.),
Theory and research on small groups (pp. 9–35). Kluwer Academic Publishers.
Kagan, S. (1994). Cooperative learning, Resources for Teachers. Kagan Cooperative Learning.
Kagan, S., & Kagan, M. (2009). Kagan cooperative learning. Kagan Publishing.
Karacop, A. (2017). The effect of using jigsaw method based on cooperative learning model in the undergraduate science
laboratory practices. Universal Journal of Educational Research, 5(3), 420–434. https://doi.org/10.13189/UJER.2017.050314
Kramarski, B., & Mevarech, Z. R. (2003). Enhancing mathematical reasoning in the classroom: The effects of cooperative learning and
metacognitive training. American Educational Research Journal, 40(1), 281–310. https://doi.org/10.3102/00028312040001281
Lavonen, J., Angell, C., Byman, R., Henriksen, E. K., & Koponen, I. T. (2007). Social interaction in upper secondary physics classrooms
in Finland and Norway: A survey of students’ expectations. Scandinavian Journal of Educational Research, 51(1), 81–101.
https://doi.org/10.1080/00313830601079082
Lyman, F. (1992). Think-pair-share, thinktrix, thinklinks, and weird facts: An interactive system for cooperative thinking. In N.
Davidson & T. Worsham (Eds.), Enhancing thinking through cooperative learning (pp. 169–181). Teachers College Press.
Lyons, T. (2006). The jigsaw of falling enrolments in physics and chemistry courses: putting some pieces together. Research in
Science Education, 36(3), 285–311. https://doi.org/10.1007/s11165-005-9008-z
Olić, S., Ninković, S., & Adamov, J. (2016). Adaptation and empirical evaluation of the questionnaire on students’ motivation
towards science learning. Psihologija, 49(1), 51–66. https://doi.org/10.2298/psi1601051O
Schaal, S. (2010). Enriching traditional biology lectures: Digital concept maps and their influence on achievement and motivation.
World Journal on Educational Technology, 2(1), 42–54.
Schraw, G., & Dennison, R. S. (1994). Assessing metacognitive awareness. Contemporary Educational Psychology, 19(4), 460–475.
http://dx.doi.org/10.1006/ceps.1994.1033
Sharan, Y., & Sharan, S. (1990). Group investigation expand cooperative learning. Education leadership, 47(4), 17–21.
Sillitto, R., & MacKinnon, L. M. (2000). Going SPLAT! - Building a multimedia educational resource for physics learners. Physics
Education, 35(5), 325–331. https://doi.org/10.1088/0031-9120/35/5/302
Slavin, R. E. (1983). When does cooperative learning increase student achievement? Psychological Bulletin, 94(3), 429–445.
https://doi.org/10.1037/0033-2909.94.3.429
Slavin, R. E. (1995). Cooperative learning: Theory, research, and practice (2nd edition). Allyn & Bacon.
Slavin, R. (2014). Cooperative learning in elementary schools. Education 3-13: International Journal of Primary, Elementary & Early
Years Education, 43(1), 5–14. https://doi.org/10.1080/03004279.2015.963370
Smithers, R. (2006, August 11). Physics in downward spiral as pupils think it is too difficult. The Guardian.
https://www.theguardian.com/uk/2006/aug/11/schools.alevels
Topping, J. K., Thurston, A., Tolmie, A., Christie, D., Murray, P., & Karagiannidou, E. (2011). Cooperative learning in science:
Intervention in the secondary school. Research in Science & Technological Education 29(1), 99–106. https://doi.org/10.108
0/02635143.2010.539972
Tuan, H. L., Chin, C. C., & Shyang, S. H. (2005). The development of a questionnaire to measure student’s motivation towards
science learning. International Journal of Science Education, 27(6), 639–654. https://doi.org/10.1080/0950069042000323737
Xiang, J.-W., Han, C.-Q. (2021) Effect of teaching and learning-scrum on improvement physics achievement and team collaboration
ability of lower-secondary school student. Journal of Baltic Science Education, 20(6), 983-1000. https://doi.org/10.33225/
jbse/21.20.983
Yassin, A. A., Razak, N. A., & Maasum, N. R. M. (2018). Cooperative learning: General and theoretical background. Advances in
Social Sciences Research Journal, 5(8), 642–654. https://doi.org/10.14738/assrj.58.5116
555
(pp. 545-557) ISSN 2538–7138 /Online/
Appendix 1. Worksheets with tasks for teaching unit Free fall in E group
Task Number 1 – Motion of the freely falling object

Demonstrate (using a ball) and describe the free fall:
•• Describe the motion of the freely falling objects?
•• Determine what type of motion the free fall represents, whether the free fall has an initial velocity.
•• Which force affects the freely falling objects?
•• What kind of acceleration freely falling objects have?
•• How does the velocity change during the free fall?
•• When is the velocity of the freely falling object greatest?
•• Sketch the freely falling ball pattern and try to define free fall.
Watch a video about Felix Baumgartner’s jump.
Task Number 2 – Velocity of the freely falling object

Demonstrate free fall (using a ball) and describe the change in the velocity during the free fall:
•• How does the velocity change during the free fall?
•• Express a formula for the velocity of the freely falling object based on the formula for the velocity of
the uniformly accelerated motion.
Check your conclusion using the PhET simulation that shows the dependence between the velocity and the
time of the falling object.
Solve the computational problem: The body falls freely for 10 s and hits the ground. Determine the velocity
of the body after 2 s in motion, after 5 s and when hitting the ground.
Task Number 3 – Distance the object falls

Demonstrate free fall (using a ball) and describe the change in the distance an object falls during the free fall:
•• Describe the correlation between the distance an object falls, and the time spent freely falling.
•• Express a formula for the distance an object falls based on the formula for the distance of the uniformly
accelerated motion.
Solve the computational problem: The freely falling object hits the ground after 5 s. Determine the distance
after 3 s of falling and distance to the ground.
Task Number 4 – Correlation between the velocity and the distance

Demonstrate free fall (using a ball) and describe correlation between the velocity and the distance:
•• Find the correlation between the velocity and the distance an object falls while freely falling and express
a formula for free fall based on the formula for the uniformly accelerated motion.
Solve a computational problem: The body falls freely from a height of 20m. Determine the velocity at which
it hits the ground.
Task Number 5 – Weightlessness

Use the video materials related to the concept weightlessness (paratroopers jumping from planes, astronauts
on the International Space Station) to comprehend, describe and define the concept.
Solve a computational problem: The parachutist has a mass of 76 kg. Determine his weight before opening
the parachute – during free fall.
556
ISSN 2538–7138 /Online/ (pp. 545-557)
Appendix 2. Scheme of Jigsaw steps
Step 1. Five home groups (each student chose one of five tasks)
Step 2. Expert groups (formed by students from home groups who have chosen the same task) solve their tasks.
Step 3. Students from expert groups back to their home group and share the knowledge about the task
Received: June 27, 2022 Revised: July 07, 2022 Accepted: August 06, 2022
Cite as: Blajvaz, B. K., Bogdanović, I. Z., Jovanović, T. S., Stanisavljević, J. D., & Pavkov-Hrvojević, M. V. (2022). The JIGSAW
technique in lower secondary physics education: Students’ achievement, metacognition and motivation. Journal of Baltic
Science Education, 21(4), 545-557. https://doi.org/10.33225/jbse/22.21.545
Branislava K. Blajvaz MSc, PhD Student, University of Novi Sad, Faculty of Sciences,
(Corresponding author) Trg Dositeja Obradovića 4, 21000 Novi Sad, Republic of Serbia.
E-mail: zmaj.fizika@gmail.com
Ivana Z. Bogdanović PhD, Associate Professor, University of Novi Sad, Faculty of Sciences,
Trg Dositeja Obradovića 4, 21000 Novi Sad, Republic of Serbia.
E-mail: ivana.bogdanovic@df.uns.ac.rs
Tamara S. Jovanović PhD, Associate Professor, University of Novi Sad, Faculty of Sciences, Trg
Dositeja Obradovića 3, 21000 Novi Sad, Republic of Serbia.
E-mail: tamara.jovanovic@dgt.uns.ac.rs
Jelena D. Stanisavljević PhD, Full Professor, University of Belgrade, Faculty of Biology, Studentski
trg 16, 11000 Belgrade
E-mail: jelena.stanisavljevic@bio.bg.ac.rs
Milica V. Pavkov- PhD, Full Professor, University of Novi Sad, Faculty of Sciences,
Hrvojević Trg Dositeja Obradovića 4, 21000 Novi Sad, Republic of Serbia.
E-mail: milica@df.uns.ac.rs
557
COMPARING PRE-SERVICE
PRIMARY SCHOOL TEACHERS’
AND LOWER-SECONDARY
ISSN 2538-7138 /Online/
LEARNERS’ UNDERSTANDING
THE PARTICULATE NATURE OF
MATTER
Abstract. Cross-age studies have
suggested that although children’s
notions of scientific phenomena change
moderately with increasing age, certain
Eva Hejnová
misconceptions persist from preschool to
a higher educational level. The purpose of
this study was to diagnose the prevalence
of selected misconceptions about the
particulate nature of matter among pre-
Introduction
service primary school teachers and to
compare the findings with the results of
The atomic hypothesis, i.e., that all things are composed of atoms that
lower-secondary learners. The participants
are constantly moving and attract each other when they are a certain distance
in the study were 197 pre-service primary
from each other and repel when they are close together, is undoubtedly one of
school teachers and 170 ninth graders.
the most important scientific findings at present. Atomism therefore holds an
The multiple-choice diagnostic test was
important position in science curriculum worldwide. The particulate nature of
developed to collect data. In the analysis of
matter (PNM) is considered a threshold concept in science education (Karataş
the test results, the paired difference test and
et al., 2013; Park & Light, 2009) and the conceptual understanding of PNM
Pearson’s chi-square independence test for
plays a fundamental role in science learning, as the particle model of matter
the contingency table were used. Frequency
serves as the basis for understanding the states of matter, phase changes,
analysis was performed for individual
and properties of substances (Haidar & Abraham, 1991; Snir et al., 2003).
questions in the test and percentages
A great deal of research has been conducted on the understanding of
were calculated. The results obtained with
PNM by students (e.g., De Vos & Verdonk, 1996; Driver et al., 2003; Gabel et al.,
primary school teacher trainees demonstrate
1987; Griffiths & Preston, 1992; Harrison & Treagust, 1996; Krnel et al., 1998; Liu
the existence of a number of misconceptions
& Lesniak, 2005; Novick & Nussbaum, 1978, 1981; Özmen et al., 2002; Özmen,
similar to those observed with ninth graders;
2011; Pella & Carey, 1967; Renström et al., 1990; Stepans, 2003) and numerous
some misconceptions were observed to even
misconceptions (that is, conceptions which differ from those accepted by the
be present significantly more frequently
scientific community) have been identified at various educational levels. A
among pre-service primary school teachers.
particular misconception is considered significant if it was observed in at least
This study confirmed that age maturity
10% of the study group (Tan et al., 2002; Chu et al., 2009). One of the most
and mere knowledge of scientific concepts
widespread are, for example, animism, that is, the idea that atoms are alive
such as atoms, molecules, etc. do not have
(Griffiths & Preston, 1992; Harrison & Treagust, 1996; Palečková et al., 1997;
to be sufficient to allow students to better
Taber & Abdo, 2013); the idea of matter as continuous, that is, there is no
understand the particulate nature of matter.
empty space between particles or that the space between the atoms is filled
Keywords: lower-secondary learners,
with some matter (Boz, 2006; Harrison & Treagust, 1996; Novick & Nussbaum,
misconceptions, multiple-choice diagnostic
1981; Özmen, 2011; Tatar, 2011; Valanides, 2000); and the assignment macro-
test, particulate nature of matter, pre-service
scopic properties of substances to the atoms or molecules that compose the
primary school teachers
substance (Ben-Zvi et al., 1986; Harrison & Treagust, 1996; Lee et al., 1993).
Cross-age studies (Ayas et al., 2010; Boz, 2006; Liu & Lesniak, 2005, 2006;
Novick & Nussbaum, 1981; Westbrook & Marek, 1991) have suggested that
Eva Hejnová
University of Jan Evangelista Purkyně in although children’s notions of scientific phenomena change moderately
Ústí nad Labem, Czech Republic with increasing age, certain misconceptions persist from preschool to higher
educational level and some cognitive difficulties are not overcome by many
older subjects, for example, difficulty in conceiving a vacuum. The cross-
558
ISSN 1648–3898 /Print/ COMPARING PRE-SERVICE PRIMARY SCHOOL TEACHERS’ AND LOWER-SECONDARY
ISSN 2538–7138 /Online/ (pp. 558-574)
age study by Novick and Nussbaum (1981, p. 7) involving apart from pupils in a primary, lower-secondary, and
upper-secondary school, university non-science majors has indicated a reluctance to think of the space between
particles as completely empty (20% at the primary and lower-secondary levels and increasing to only 37% at the
upper-secondary and university levels).
Research with Pre-service Primary School Teachers
Some established studies (e.g., Aydeniz et al., 2017; De Jong et al., 2005; Kiray, 2016; Mumba et al., 2014; Unver
& Arabacioglu, 2015) have shown that pre-service science teachers have a great number of misconceptions in the
topic of PNM, which could potentially perpetuate the belief of these misconceptions in the future generations of
students taught by these teachers. The Valanides study (2000) has focused on the conceptions of primary school
teacher trainees related to aspects of dissolving and the effects of filtering or heating solutions. The results of his
study have shown that the majority of the pre-service teachers had a limited conceptual understanding of PNM,
for example, the existence of an empty space within matter (Valanides, 2000, p. 259). Tatar’s study (2011) has aimed
to determine misconceptions among pre-service primary school teachers about the differences between solid,
liquid, and gaseous states of matter. The participants of this study stated that the particles of solids cannot move,
there is no space between the particles of solids, and solids are completely made up of particles, but liquids and
gases are not completely made up of particles (contain other things).
Education Context of the Study
The education system in the Czech Republic includes pre-primary education (aged 3 to 6), primary education
(aged 6 to 11), lower-secondary education (aged 11 to 15), upper-secondary (aged 15 to 19), and tertiary educa-
tion (university). Integrated science is a compulsory subject in primary education, and when students come to the
lower-secondary school, chemistry, physics, and biology are compulsory subjects.
Pre-service primary school teachers, who teach children ages 6-11 years, are usually prepared for their profes-
sion in faculties of education in 5-year master’s degree programs. To a greater or lesser extent, they are introduced
to a knowledge of natural science, mostly within the mandatory science courses taken in their programme areas,
such as the subject of Integrated Scientific Basics. These courses are then usually followed by didactically oriented
disciplines, sometimes supplemented by various practical natural scientific lessons, in which students become
acquainted with simple experiments.
The period of early primary education can be considered very important, if not crucial, in terms of encouraging
an interest in natural sciences. At this age, children love to watch and explore things and think about the correct
explanation of various phenomena. Thus, children’s curiosity can be very well used for motivation, as well as to
build a positive attitude toward the natural sciences. Teachers play a particularly significant role in this process, as
they can provide children with suitable topics to think about and experiment with and may also lead their learners
to formulate correct ideas about how the world around them works.
According to a strategic document of the Ministry of Education, Youth and Sports (2021) entitled the Frame-
work Educational Program for Elementary Education, knowledge about matters and their properties are included
within the teaching of elementary natural science in the educational area Man and His World, specifically in the
thematic area Diversity of Nature. For primary learners, however, atoms and molecules are not yet introduced as
particulates of matter (Podroužek, 2003). Children typically encounter these concepts no earlier than 6th grade.
Pella and Carey (1967), however, demonstrated that children can already grasp the main concepts of atomic theory
(that is, matter is composed of particles that are moving) at an early school age. In addition, it can never be ruled
out that more curious learners will ask their teachers questions about the structure of matter in the early stages
of primary school.
Research Problem
From the aforementioned studies, it emerged that certain misconceptions about the topic of PNM persist from
preschool to a higher educational level. Ayas et al. (2010) have stated that understanding of the PNM increased
with educational level. Some studies (Valanides, 2000; Liu & Lesniak, 2006) have argued that the range and nature
of student conceptions support the suggestions that atomism is primarily a function of school studies. Liu and
559
(pp. 558-574) ISSN 2538–7138 /Online/
Lesniak (2006, p. 322) suggest that “children’s matter concept development is attributable to not only maturation
by age but also school context, such as curriculum and instruction”.
Although many research studies have been carried out on the understanding of PNM, only a few studies
have addressed misconceptions about the topic of PNM among pre-service primary school teachers (Tatar, 2011;
Valanides, 2000) and more recent cross-ages studies also usually did not include both pre-service primary school
teachers and lower-secondary learners. In the Czech Republic, detailed research on the conceptual understanding
PNM has not yet been carried out at any educational level.
The aim of the present study was to diagnose the prevalence of selected misconceptions in the PNM topic
among pre-service primary school teachers (TEA) and to compare the findings with the results of lower-secondary
learners ages 14-15 years (LEA).
The following research questions (RQ) were posed at the beginning of this research:
RQ1: Is there a significant difference between the frequencies of the occurrence of correct responses for
each test question between TEA and LEA?
RQ2: Is there a significant difference between the choice of individual responses for each test question
and the study group (TEA or LEA)?
RQ3: Is there a significant difference between the frequencies of the occurrence of the most frequent
misconceptions between TEA and LEA?
General Background
This study used quantitative methods using a multiple-choice diagnostic test to obtain data on the concep-
tual understanding of PNM from university and lower-secondary school students in the Czech Republic. Although
children get acquainted with the particle structure of substances and the first models of the atom usually in the
6th grade of lower-secondary school (ages 11 years), the subject of atomic and nuclear physics is included mainly
in the 8th and 9th grade (ages 13-15 years). Therefore, learners’ misconceptions were explored at the end of the
9th grade when more comprehensive knowledge can already be expected in the field of PNM. Pre-service primary
school teachers engaged in the research study were in the first, second, and third years of master’s studies, in which
the teaching of the basics of natural sciences and their didactics is usually included.
The test was administered to the LEA group in May and June 2017 and to the TEA group during the fall of
2017, using the paper-and-pencil test form. TEA and LEA completed the test in approximately twenty minutes.
The duration of the test was chosen to facilitate its administration during the lesson, especially in the LEA group.
Participants
The research sample consisted of 197 TEA, enrolled in five-year master degree programs at the Faculty of
Education of the Jan Evangelista Purkyně University in Ústí nad Labem (University A), which belongs to the younger
regional universities, and at the Faculty of Education of the Masaryk University in Brno (University B), which is one
of the largest traditional universities in the Czech Republic. The selection of the students covered all forms of their
study. The structure of TEA is shown in Table 1.
Table 1
Structure of TEA
Structure of TEA
University Total
First-year student Second-year student Third-year student
University A - 55 41 96
University B 96 5 - 101
560
ISSN 2538–7138 /Online/ (pp. 558-574)
The results of the TEA study group were compared with the results of 170 LEA ages 14-15 years from the
fourth grade of an eight-year general secondary school and from six classes in the 9th grade of lower-secondary
schools. The schools were located in the Ústí nad Labem Region, the Moravian-Silesian Region and the capital city
of Prague. The structure of LEA is shown in Table 2.
Table 2
Structure of LEA
Number of LEA
Study group Prague Total
Ústí nad Labem Region Moravian-Silesian Region
Region
Lower-secondary school 47 74 19 140
Eight-year general secondary school - 30 - 30
Comparisons were not made with respect to gender, but the number of males and females in the LEA partici-
pants turned out to be much the same. In the TEA group, however, women prevailed significantly. The size of both
samples (TEA or LEA) was adequate for α = .05 with the margin of error 3% (Barlett et al., 2001, p. 48).
This research study followed the ethical considerations outlined by Taber (2014). TEA were informed of the
purpose of the study and their participation was voluntary. Parents of LEA have given their informed consent. The
data collected were anonymous and only used for the purposes of the research; no references are made to the
results of the TEA and LEA studies.
Instruments and Procedures
The multiple-choice diagnostic test used in the study was developed in three stages. In the first stage, the
content of the test was determined. The test consisted of questions focused on the most common misconceptions
on the topic of PNM, which have been repeatedly reported in many established studies on the topic of PNM; thus,
its construct validity stems from its relation to these studies. The questions in the test were aimed at the conceptual
areas listed in Table 3.
In the second stage, the first version of the test was taken by 30 learners in the fourth grade of an eight-year
general secondary school and 14 university students. Students were interviewed to identify any problems that might
have occurred in answering the test questions. On the basis of piloting, necessary modifications of the diagnostic
instrument were made, and a 24-question test was formulated as seen in the Appendix.
Table 3
Conceptual Areas Including in the Multiple-choice Diagnostic Test
Conceptual area of the PNM Questions in the test Reference on research
Griffiths & Preston, 1992

Harrison & Treagust, 1996
Visibility of atoms Q1
Lee, 1993
Unver & Arabacioglu, 2015
Animism (atoms are alive) Q2, Q3 Harrison & Treagust, 1996
Palečková et al., 1997
Shape, size, and weight of atom Q4, Q5, Q6 Kind, 2004
561
(pp. 558-574) ISSN 2538–7138 /Online/
Conceptual area of the PNM Questions in the test Reference on research

Space between atoms Q7, Q8, Q9, Q11 Özmen, 2011
Tatar, 2011
De Vos & Verdonk, 1996
Transfer of properties of macro-objects into the microcosm Q7, Q10, Q12 Özalp & Kahveci, 2015
Stepans, 2003
Processes taking place in atomic nucleus and shell Q13, Q14 Unver & Arabacioglu, 2015
Participants could select an answer from four alternatives for each question, but they were also provided with
a space marked “Other response” with room in it to supply an open-ended response. Each question had only one
most acceptable answer from a scientific point of view, which is called the correct answer in the following text for
simplicity. As far as the evaluation of the test is concerned, a participant of the study could receive one point for
the correct answer to each question.
The reliability of the research instrument was identified using the Kuder-Richardson formula 20 (KR-20),
which is appropriate for this type of test (Kuder & Richardson, 1937, p. 158). The reliability of the test achieved the
value of .653 for the LEA group, the value of .480 for the TEA group, and the value of .569 for the entire sample of
participants (LEA and TEA together). Kehoe (1995) points to a reliability of 0.5 that can be accepted for short tests
(10-15 questions). Given the length of the test that involved 14 questions, the reliability values can therefore be
considered satisfactory.
Data Analysis
The data consist of the responses of the TEA and LEA groups to the test. Quantitative techniques were employed
to analyse the data. First, percentage analyses of the correct answers were performed in the TEA and LEA groups
on the individual questions of the test. The 14 partial null hypotheses (H01 – H014) formulated assumed that there
were no statistically significant differences between the frequencies of the occurrence of correct responses in the
TEA and LEA groups for each question in the test (that is, hypothesis H01 is related to question Q1, hypothesis H02
to question Q2, etc.).
In a further analysis, the difference between the choice of individual responses for each test question and
the study group was examined in more detail. For this purpose, Pearson’s chi-square test of independence was
used for the contingency table. The 14 partial null hypotheses (H01* – H014*) formulated assumed that there was
no statistically significant difference between the choice of individual response for each question in the test and
the TEA or LEA groups. The strength of the difference was examined using the Cuprov’s contingency coefficient
K (Chráska, 2007).
Finally, percentage analyses of the most frequent misconceptions in the TEA and LEA groups were per-
formed. 25 strongly held misconceptions that were diagnosed in at least 10% of participants in one of the TEA or
LEA groups were selected. The 25 partial null hypotheses (H0M1b, H0M2a, H0M2d, H0M3a, H0M3c, H0M3d, H0M4a,
H0M4b, H0M5c, H0M5d, H0M6b, H0M7d, H0M8b, H0M8d, H0M9b, H0M9d, H0M10c, H0M10d, H0M11a, H0M11d, H0M12b,
H0M13b, H0M13c, H0M13d and H0M14c) were formulated. They assumed that there were no statistically significant
differences between the frequencies of the occurrence of a misconception in both study groups.
The statistical software used to calculate the independence tests (paired difference test and Pearson’s
chi-square test of independence for the contingency table) was Statistica 13.3 (StatSoft, Inc., 2017). The level of
significance α = .05 was used in all tests. When the p-value obtained was below the significance threshold α = .05,
the null hypothesis was rejected.
562
ISSN 2538–7138 /Online/ (pp. 558-574)
Research Results
The basic descriptive test characteristics for both groups are presented in Table 4. From the results shown
here, it is clear that both tested groups achieved very similar values in individual statistical characteristics. The LEA
group, however, shows greater variance; therefore, we noticed greater reliability of the test in this case.
Table 4
Basic Descriptive Test Characteristics
Test characteristics TEA LEA Total
Number of respondents 197 170 367

Average score 7.74 7.69 7.71
Standard deviation 2.39 2.86 2.62
Median 8 7.5 8
Modus 8 7 8
Minimum 2 1 1
Maximum 14 14 14
The Occurrence of Correct Responses in Both Study Groups
First, the differences between the frequencies of the occurrence of correct answers between LEA and TEA
were tested. The percentage of TEA who correctly answered the individual questions ranged from 23.35 to 77.66,
and the percentage of LEA correct responses ranged from 21.18 to 74.71 (see Table 5). Since most of the questions
were correctly answered by 30% to 80% of the study participants, it is possible to consider the test as an appropri-
ate research instrument (Kehoe, 1995).
TEA and LEA were most successful in answering question Q1 ‘Can we see atoms?’ (77.66% of TEA and 74.71%
of LEA), the least successful were the TEA and LEA study groups in solving question Q3 ‘What happens to the atoms
of an animal after it dies?’ (23.35% of TEA and 21.18% of LEA). The percentage of correct responses of TEA and LEA
is clearly indicated in Figure 1.
Figure 1
Comparison of the Success Rate of Questions in the TEA and LEA Groups
90
80
70
Success rate (%)
60
50
40
30
20
10
0
Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Q12 Q13 Q14
Question number
TEA LEA group
563
(pp. 558-574) ISSN 2538–7138 /Online/
Furthermore, it was examined whether the differences in the incidence of correct answers between TEA and LEA
were statistically significant or not. The paired difference test of independence showed that there was a statistically
significant difference between the frequencies of the occurrence of correct answers in the test questions Q4, Q10,
Q12, and Q13, that is, the null hypotheses H04, H010, H012, and H013 were rejected. Other null hypotheses, that
is, there is no statistically significant difference between the frequencies of the occurrence of correct answers in
the study groups of TEA and LEA, could not be rejected at the level of significance threshold α = .05. The results
are shown in Table 5.
Table 5
Summary of the Success Rate of Individual Questions in the TEA and LEA Groups
TEA LEA
Question p
% %
Q1 77.66 74.71 .255

Q2 67.01 68.82 .356
Q3 23.35 21.18 .309
Q4 50.25 37.06 .006
Q5 58.88 54.12 .179
Q6 73.10 65.29 .053
Q7 51.78 49.41 .325
Q8 47.72 53.53 .134
Q9 62.44 58.24 .206
Q10 70.05 57.65 .007
Q11 59.90 61.76 .358
Q12* 45.69 55.88 .026
Q13* 36.55 62.94 .000
Q14 55.84 55.88 .497
Note: The statistically significant differences between the relative frequencies of the answers in the LEA and
TEA groups are marked in bold, the cases where the relative frequencies of the correct answers in the TEA group
are statistically significantly lower than in the LEA group are marked with an asterisk.
The Differences Between the Choice of Individual Responses for the Test Questions and the Study Group
The differences between the choice of correct and incorrect answers for individual questions and the study
group were also examined, that is, if the frequencies of answers to the question depended, or were independent
of the study group. The results are shown in Table 6.
More detailed analysis using Pearson’s chi-square test of independence for the contingency table showed that
there was a statistically significant difference between the choice of answers and the study group for questions
Q10, Q11 and Q13, that is, we rejected the null hypotheses H010*, H011* and H013*. Other null hypotheses, that is,
there is no significant difference between the choice of individual answers for each test question and the study
group, cannot be rejected at the level of the significance threshold α = .05.
564
ISSN 2538–7138 /Online/ (pp. 558-574)
Table 6
Pearson’s Chi-square Test for Individual Questions of the PNM test
χ2 - test
Question Cuprov’s K
χ2 df p
Q1 3.314 3 .346 ˗
Q2 3.127 3 .373 ˗
Q3 1.084 3 .781 ˗
Q4 34.483 3 .000 ˗
Q5 2.135 3 .545 ˗
Q6 4.940 3 .176 ˗
Q7 2.658 3 .447 ˗
Q8 6.452 3 .092 ˗
Q9 4.106 3 .250 ˗
Q10 9.074 3 .028 .302
Q11 10.215 3 .017 .312
Q12 3.990 3 .263 ˗
Q13 32.161 3 .000 .414
Q14 2.842 3 .417 ˗
Note: The statistically significant dependencies are marked in bold. Some rows were marked in grey (Q1, Q4,
Q8), because Pearson’s chi-square test cannot be considered sufficiently reliable in these cases due to the low
expected frequencies in some cells in the contingency table (the expected frequencies were below the number
5 in 25% of the total number of cells).
The Most Frequent Misconceptions Diagnosed in the TEA and LEA Groups
Finally, some misconceptions will be discussed that were diagnosed throughout the test and can be considered
important, particularly regarding the frequency of their occurrence and their severity. The selected misconceptions
and the percentage of TEA and LEA who held these conceptions are summarised in Table 7. A misconception is
denoted by the letter M and, furthermore, by a number and letter referring to the respective question in the test.
For every misconception, there is a link to at least one source that mentions the particular misconception.
The paired difference test of independence showed that there was a statistically significant difference be-
tween the frequencies of the appearance of misconceptions between both groups in the misconceptions M4a,
M4b, M6b, M8d, M10d, M11d, M13b, M13d, M14c, that is, we reject the null hypotheses H04a, H04b, H06b, H08d,
H010d, H011d, H013b, H013d, and H014c. Other null hypotheses, that is, there is no statistically significant difference
between the frequencies of the occurrence of misconceptions between both groups, cannot be rejected at the
level of significance threshold α = .05.
Statistical analysis showed that there was a statistically significant difference between the frequencies of
occurrence of the most frequent misconceptions in the TEA and LEA groups in nine cases of 24 selected mis-
conceptions. These nine misconceptions concern the shape, size, and weight of an atom (M4a, M4b, M6b), the
space between atoms (M8d, M11d), the transfer of properties of macro-objects into the microcosm (M10d), and
the processes taking place in the atomic nucleus and shell (M13b, M13d, M14c). It should be noted that seven of
those nine misconceptions were found to occur significantly more frequently among the TEA group than in the
LEA group. The only misconceptions M6b and M10d had a statistically more significant incidence in the LEA group
than in the TEA group.
As clearly shown in Table 7, among the most strongly held misconceptions are the M3c, M7d, and M12b
misconceptions with a frequency of occurrence greater than 30% recorded in both groups. In M4a, M4b, M8d,
565
(pp. 558-574) ISSN 2538–7138 /Online/
M13b, and M14c misconceptions, an incidence greater than 30% was observed in at least one of the study groups
(although in all cases it was the TEA group).
Table 7
Overview of Selected Misconceptions and the Relative Frequencies of Their Occurrence in the TEA and LEA Groups
No. Misconception % of TEA % of LEA p
No, atoms cannot be seen; we can only believe that they exist.
M1b 17.77 15.29 .262
(Unver & Arabacioglu, 2015)
Atoms are alive because they can grow and divide.
M2a 10.15 13.53 .158
(Harrison & Treagust, 1996)
Only the atoms of living things are alive. (Griffiths & Preston, 1992; Unver
M2d 13.20 8.82 .092
& Arabacioglu, 2015)
When an animate being dies, the atoms of which it was made stop
M3a moving. 10.15 8.82 .333
(Palečková et al., 1997)
When an animate being dies, the atoms cleave to simpler parts, and
M3c those then create new atoms. 43.65 47.65 .221
Atoms cease to exist once an animate is decomposing.
M3d 17.77 21.18 .205
Atoms may have different shapes depending on the kind of matter
M4a* they are composed of (they may be round, oval, oblong, etc.). 38.07 24.71 .003
Atoms are in the form of balls that are full inside.
M4b* (Griffiths & Preston, 1992; Harrison & Treagust, 1996; Unver & 31.76 9.64 .0001
Arabacioglu, 2015)
Atoms do not have the same size because the size of an atom is deter-
M5c mined only by the number of protons and neutrons in the atomic nucleus. 19.29 22.35 .235
All atoms are equal but produce molecules of different sizes.
M5d 16.75 20.59 .173
(Kind, 2004)
All atoms do not have the same weight, because the weight of an
M6b atom depends on how many simpler atoms it is composed of. 12.69 20.59 .021
A piece of gold consists of gold atoms and matter that fills the space
M7d between the gold atoms. 37.06 39.41 .322
Because there are no gaps in the particles of solid matter, atoms cannot
M8b move there. 19.80 22.94 .232
(Özalp & Kahveci, Tatar, 2011; Unver & Arabacioglu, 2015)
Atoms do not move in a solid; only electrons move in atomic shells.
M8d* 31.98 20.00 .005
When an iron rod is heated, electrons will release from the atomic nuclei;
M9b thus, enlarging the atoms. 16.75 18.24 .354
During heating, only the rod increases in volume; the size of the atoms
M9d and the distance between them do not change. 11.68 17.06 .070
When the coal splinters into dust, the atoms will also disintegrate.
M10c 10.66 11.76 .369
566
ISSN 2538–7138 /Online/ (pp. 558-574)
No. Misconception % of TEA % of LEA p
When a piece of coal is hammered to dust, small parts of some

M10d carbon atoms fall off, which decreases the size of these atoms. 10.15 20.59 .003
If you remove all paper sheet atoms, a small amount of paper dust
M11a remains. 14.72 19.41 .116
If you remove all paper sheet atoms, the energy remains.
M11d* 19.29 8.82 .002
Air is elastic; therefore, the air atoms are also elastic; that is why they
can be easily compressed and will not break in the event of collision with
M12b 39.09 31.18 .057
a train.
(Harrison & Treagust, 1996; Unver & Arabacioglu, 2015)
Electrons are one of the fundamental particles that an atom is com-
M13b* posed of. If the electron separates from the atom, the atom divides. 34.01 15.88 .0001
All you have to do is break the shell of the atom.
M13c 10.66 11.76 .369
(Harrison & Treagust, 1996)
Electrons cannot be torn off an atom.
M13d* 18.78 8.24 .002
Atoms cannot transform because each of the protons, neutrons, and
M14c* electrons in an atom is unique. 31.47 22.94 .034
Note: The cases of misconception, in which statistically significant differences were found between the relative
frequencies of the answers, are indicated in bold. Asterisks are used to distinguish between misconceptions in which
a statistically significant higher frequency of occurrence was observed in the TEA group than in the LEA group.
Discussion
The results of the research showed consistency with studies in the field literature, but also shed light on some
findings that are not mentioned. Based on the results stated above, it is possible to answer the research questions
as follows:
(RQ1) There is a significant difference between the frequencies of the occurrence of correct responses
between TEA and LEA in four test questions.
(RQ2) Three test questions reliably demonstrated a significant difference between the choice of individual
responses for the question and the test group.
(RQ3) There is a significant difference between the frequency of the occurrence of misconceptions
between both groups in nine cases of 25 investigated misconceptions, of which seven misconceptions
occur significantly more frequently among TEA.
Regarding RQ1, based on the findings of other studies, we assumed that the TEA group would be more success-
ful in choosing the correct answers for each question. It turned out, however, that the TEA responded statistically
significantly better than LEA for only two questions (Q4 and Q10). The question Q4 concerned the shape of an atom
(Q4). For this question, it turned out that almost a third of LEA (31.76%) had an idea of the atom as a sphere that
is full inside (misconception M4b), while in the TEA group this idea was significantly less widespread (9.64%). This
often frequented misconception shows a clear parallel to the ideas of Greek atomists, according to which atoms
are inviolable, that is, indivisible, absolutely rigid, and without emptiness (Hejnová & Hejna, 2018). School teachers
often use this idea as the simplest model of the atom as a rigid ball, which better matches the ability of younger
learners in particular to accept such an idea. This may be the reason for the significant incidence of this idea in LEA.
For the question Q10, the TEA group had the correct answers with high frequency (70.05% correct answers). Even
in this case, however, the strong influence of the transfer of what is happening with the piece of coal, that is, the
macroscopic material, at the atomic level can be inferred. For example, the misconception M10c (‘When the coal
567
(pp. 558-574) ISSN 2538–7138 /Online/
splinters into dust, the atoms will also disintegrate.’) appeared in approximately 11% of TEA, which is consistent
with the results of a study by Unver and Arabacioglu (2015, p. 74) conducted with pre-service science teachers.
The surprising finding was that for two questions (Q12 and Q13), in contrast, the LEA group was statically
significantly more successful. The question Q12 was aimed at the transfer of properties of macro-objects into the
microcosm. This question showed a very strong misconception of M12b in both groups that air is elastic and therefore
air atoms are also elastic. 39.09% of TEA and 31.18% of LEA were in favour of this idea, which is a relatively serious
finding, as other research has not shown such a high prevalence of misconception, e.g., Unver and Arabacioglu
(2015, p. 73) reports the incidence of this misconception in pre-service science teachers 15.1%.
Also, in the question Q13, LEA were significantly more successful than TEA, even twice as much (36.55% TEA,
62.94% LEA). On the contrary, a third of the TEA group showed a very strong misconception of M13b (‘Electrons
are one of the fundamental particles that an atom is composed of. If the electron separates from the atom, the
atom divides.’). Similarly, the study by Unver and Arabacioglu (2015, p. 73) showed this misconception in 20% pre-
service science teachers. The M13d misconception (‘Electrons cannot be torn off an atom.’) demonstrates that TEA,
in particular, did not differentiate well between the actions occurring in the electron shell and the nucleus and the
consequences which these actions lead to. This is a basic knowledge of atomic and nuclear physics which should
be understood by all secondary school graduates. However, the conducted research has shown that a considerable
number of TEA are not familiar with even the most basic knowledge.
As far as RQ2, only three test questions (Q10, Q11 and Q13) reliably demonstrated a significant difference
between the choice of the answers for the questions and the test group, which to some extent corresponds to the
results obtained in the previous analysis of differences between the frequencies of the occurrence of correct answers;
a statistically significant difference between the frequencies of the occurrence of correct answers between TEA and
LEA was in the test questions Q10 and Q11; for the question Q4, Pearson’s chi-square test is not sufficiently reliable.
Regarding RQ3, the prevalence of many misconceptions has been shown to be the same in the TEA group as
in the LEA group, but some misconceptions have occurred significantly more frequently in the TEA group, which is
somewhat surprising. For example, for the question Q8, two very strong misconceptions appeared in both groups
(M8b and M8d). The M8b misconception (‘Because there are no gaps in the particles of solid matter, atoms cannot
move there.’) points out that about a fifth of LEA s and TEAs consider atoms small pieces of solid bodies, between
which there is no space. In Unver and Arabacioglu (2015) the same misconception was studied in a group of pre-
service science teachers, whose incidence was comparatively high (23.3%). Similar results were shown by the study
Özalp and Kahveci (2015), where this misconception was diagnosed in 27.3% ninth graders. This idea that atoms
are ‘densely sown’ side by side (Driver et al., 2003) is probably why almost a third of TEA think that atoms do not
move, but electrons are moving in atomic shells (M8d).
Specifically, of the misconceptions concerning animism (see the misconceptions M2a, M2d, M3a, M3c, M3d in
Table 6), that is, the idea that an atom is alive (having the characteristics of a living organism) belongs among the
most common, which was also confirmed in this study. A more frequent incidence of two concepts was revealed
in either group: ‘atoms can grow and divide’ (M2a), and ‘only atoms of living organisms are alive’ (M2d). In both
cases, the incidence was approximately 10%. Furthermore, in our research a very powerful idea (M3c) appeared in
both groups, that ‘when an animate being dies, the atoms split into simpler parts, which then create new atoms’
(43.65% of TEA and 47.65% of LEA). The same question was also included in the Trends in International Mathematics
and Science Study (TIMSS) (Palečková et al., 1997) conducted in 1995 in 43 countries around the world for seventh
and eighth graders. In this study, about 22% of the students chose the correct answer in both age categories. We
recorded very similar results in this study, essentially the same for the TEA and LEA groups.
Mention should be made of one more significant misconception about M14c (‘Atoms cannot transform because
each of the protons, neutrons, and electrons in an atom is unique.’), which occurred statistically more significantly
in the TEA group. Almost a third of TEA think that the electrons of different atoms are unique. The cause of this
misconception may be poor understanding of the fact that the different properties of atoms are related to their
different structures, not to their constitution.
Participants in our study could also provide their own response. Both TEA and LEA responses, however, were
rare. For most of the questions, there were one to three own answers; only for Q14, 14,6% of the participants chose
their own answer. Among the own answers were those that could be considered correct but were not directly
related to the question; for example, in Q9 (‘Do the atoms in an iron rod expand when the rod is heated?’) where
one of the learners wrote that when they warm up, the atoms start to move faster. Some of the children’s responses
indicated that they were thinking more deeply about the question and trying to formulate their original answer,
568
ISSN 2538–7138 /Online/ (pp. 558-574)
for example, in Q11 (‘Imagine that all atoms of a sheet of paper have been removed. What would then be left?’)
where two learners replied that after removing all the atoms of a paper sheet, ‘the ink atoms would remain’ or
‘substances of a different kind contained in the paper would remain’. For Q14 (‘How can the atoms of one element
be transformed into the atoms of another element?’) 11 children gave their own answer, but most of them were
incorrect or inaccurate (for example, ‘atoms cannot change under any circumstances’; ‘they can, but it is a matter of
millions of years, for example, diamonds, previously it was ordinary carbon, then with the influence of pressure and
heat it gradually transformed’; ‘may change by the addition or removal of an electron or proton’). It is also worth
noting that about 13% of TEA and 10 % of LEA think that the cause of the transformation of the atom is sunlight
or thunder and lightning. The learners’ own answers also pointed to some other misconceptions. A student, for
example, stated in Q7 that an atom is a very small piece of gold with gold properties, which is the typical child’s
initial idea of an atom as a small piece of material substance or an ultimate piece of material substance, which we
obtain by gradual division of the material.
Influencing and eliminating misconceptions is not an easy task because of their considerable resistance.
However, there are some appropriate teaching techniques that can effectively help pre-service primary school
teachers overcome their incorrect conceptions. These include, for example, instructional approaches based on
constructivism, such as the method called Concept Cartoon (Naylor & Keogh, 2010; Pekel, 2021; Samková, 2018),
and science-based instruction methods that engage students in observation and experiments (e.g., Unver & Ara-
bacioglu, 2015) and allow them to better develop correct ideas about fundamental physical phenomena.
Limitations of the study
Undoubtedly, there are limitations to the research presented, for example, the use of an available sample of
respondents, especially with regard to their random selection. It should also be noted that the participants of the
TEA group focus more on the humanities, and the natural sciences are not their main field of study in the university.
Although there are many cross-age studies aimed at understanding of PNM, almost no research has been
performed involving lower-secondary learners and also pre-service primary school teachers, who in their university
studies are not primarily focused on natural science.
The prevalence and diversity of the observed misconceptions among TEA in this study indicate that most of
them are not familiar with the nature and constitution of matter and that the issue is not adequately addressed
during their education. The results obtained with TEA demonstrate the existence of a number of misconceptions
similar to those observed with LEA, some of them can be considered very strongly-held misconceptions with a
frequency of occurrence higher than 30% and some misconceptions were observed to even be present signifi-
cantly more frequently in the TEA group. This study confirmed that age maturity and mere knowledge of scientific
concepts such as atoms, molecules, etc. do not have to be sufficient to allow students to better understand and
correctly grasp the key concepts of PNM.
To a deeper conceptual understanding of the PNM, the content of the curriculum but also suitable teaching
activities during school studies play a crucial role because students need both a sufficiently long time and suitable
opportunities to rethink and grasp key concepts of atomism. Given the importance of primary science education, it
would be appropriate to allow pre-service primary school teachers to specialise more in learning subjects focused
on natural sciences already within the frame of their pre-graduate training. This part of university education of
primary school teacher trainees could be accentuated more in this way. Students could be offered science-based
teaching modules made up of experiments and other suitable activities, particularly argumentation-based ones,
and at least among those portions of students, who would like to study the topic in more depth.
References
Ayas, A., Özmen, H., & Çalik, M. (2010). Students’conceptions of the particulate nature of matter at secondary and tertiary level. International
Journal of Science and Mathematics Education, 8(1), 165-184. https://link.springer.com/article/10.1007/s10763-009-9167-x
Aydeniz, M., Bilican, K., & Kirbulut, Z. D. (2017). Exploring pre-service elementary science teachers’ conceptual understanding of
particulate nature of matter through three-tier diagnostic test. International Journal of Education in Mathematics, Science
and Technology, 5(3), 221-223. https://ijemst.net/index.php/ijemst/article/view/120/121
569
(pp. 558-574) ISSN 2538–7138 /Online/
Barlett, J. E., Kotrlik, J. W., & Higgins, Ch. C. (2001). Organizational research: Determining appropriate sample
size in sur vey research. Information Technology, Learning, and Per formance Journal, 19(1), 43-50.
https://www.opalco.com/wp-content/uploads/2014/10/Reading-Sample-Size1.pdf
Ben-Zvi, R., Eylon, B., & Silberstein, J. (1986). Is an atom of copper malleable? Journal of Chemical Education, 63(1), 64-66.
https://pubs.acs.org/doi/abs/10.1021/ed063p64
Boz, Y. (2006). Turkish pupils’ conceptions of the particulate nature of matter. Journal of Science Education and Technology, 15(2),
203-213. https://link.springer.com/article/10.1007/s10956-006-9003-9
De Vos, W., & Verdonk, A. H. (1996). The particulate nature of matter in science education and in science. Journal of Research in
Science Education, 33(6), 657-664. https://doi.org/10.1002/(SICI)1098-2736(199608)33:6<657::AID-TEA4>3.0.CO;2-N
De Jong, O., Van Driel, J. H., & Verloop, N. (2005). Preservice teachers’ pedagogical content knowledge of using particle models
in teaching chemistry. Journal of Research in Science Teaching, 42(8), 947-964. https://doi.org/10.1002/tea.20078
Driver, R., Squires, A., Rushworth, P., & Wood-Robinson, V. (2003). Making sense of secondary science. Routledge.
Gabel, D. L., Samuel, K. V., & Hunn, D. (1987). Understanding the particulate nature of matter. Journal of Chemical Education, 64(8),
695-698. http://dx.doi.org/10.1021/ed064p695
Griffiths, A. K., & Preston, K. R. (1992). Grade-12 students’ misconceptions relating to fundamental characteristics of atoms and
molecules. Journal of Research in Science Teaching, 29(6), 611-628. https://doi.org/10.1002/tea.3660290609
Haidar, A. H., & Abraham, M. R. (1991). A comparison of applied and theoretical knowledge of concepts based on the particulate
nature of matter. Journal of Research in Science Teaching, 28(10), 919-938. https://doi.org/10.1002/tea.3660281004
Harrison, A. G., & Treagust, D. F. (1996). Secondary students’ mental models of atoms and molecules: Implications for teaching
chemistry. Science Education, 80(5), 509-534. https://doi.org/10.1002/(SICI)1098-237X(199609)80:5<509::AID-SCE2>3.0.CO;2-F
Hejnová, E., & Hejna, D. (2018). Miskoncepce žáků o atomech v kontextu představ starověkých myslitelů o stavbě hmoty [Pupils’
misconceptions about atoms in the context of ancient thinkers’ ideas about the structure of matter]. Scientia in educatione,
9(2), 22-43. https://doi.org/10.14712/18047106.1176
Chráska, M. (2007). Metody pedagogického výzkumu [Methods of pedagogical research]. Grada.
Chu, H. E., Treagust, D. F., & Chandrasegaran, A. L. (2009). A stratified study of students’ understanding of basic optics concepts
in different contexts using two-tier multiple-choice items. Research in Science & Technological Education, 27(3), 253-265.
https://doi.org/10.1080/02635140903162553
Karataş, F., Ünal, S., Durland, G., & Bodner, G. (2013). What do we know about students’ beliefs? Changes in students’ conceptions
of the particulate nature of matter from pre-instruction to college. In G. Tsapralis, & H. Sevian (Eds.), Concepts of matter in
science education (Vol. 19, pp. 231-247). Springer. http://dx.doi.org/10.1007/978-94-007-5914-5
Kehoe, J. (1995). Basic item analysis for multiple-choice tests. Practical Assessment, Research & Evaluation, 4(10).
https://doi.org/10.7275/07zg-h235
Kind, V. (2004). Beyond appearances: Students’ misconceptions about basic chemical ideas (2nd ed.). Durham University.
Kiray, S. A. (2016). The pre-service science teachers’ mental models for concept of atoms and learning difficulties. International Journal
of Education in Mathematics, Science and Technology, 4(2), 147-162. https://www.researchgate.net/publication/301904478_
The_Pre-service_Science_Teachers%27_ Mental_Models_for_Concept_of_Atoms_and_Learning_Difficulties
Krnel, D., Watson, R., & Glažar, S. A. (1998). Survey of research related to the development of the concept of ‘matter’. International
Journal of Science Education, 20(3), 257-289. https://doi.org/10.1080/0950069980200302
Kuder, G. F., & Richardson, M. W. (1937). The theory of estimation of test reliability. Psychometrika, 2(3), 151-160.
https://doi.org/10.1007/BF02288391
Lee, O., Eichinger, D. C., Anderson, Ch. W., Berheimer, G. D., & Blakeslee, T. D. (1993). Changing middle school students’ conceptions
of matter and molecules. Journal of Research in Science Teaching, 30(3), 249-270. https://doi.org/10.1002/tea.3660300304
Liu, X., & Lesniak, K. M. (2005). Students’ progression of understanding the matter concept from elementary to high school.
Science Education, 89(3), 433-450. https://doi.org/10.1002/sce.20056
Liu, X., & Lesniak, K. (2006). Progression in children’s understanding the matter concept from elementary to high school. Journal
of Research in Science Teaching, 43(3), 320-347. https://doi.org/10.1002/tea.20114
Ministry of Education, Youth and Sports. (2021, February 11). Framework educational programme for elementary education.
http://archiv-nuv.npi.cz/t/rvp-pro-zakladni-vzdelavani.html
Mumba, F., Chabalengula, V. M., & Banda, A. (2014). Comparing male and female pre-service teachers’ understanding of the
particulate nature of matter. Journal of Baltic Science Education, 13(6), 821-827. https://dx.doi.org/10.33225/jbse/14.13.821
Naylor, S., & Keogh, B. (2010). Concept cartoons in science education. Millgate House Publishers.
Novick, S., & Nussbaum, J. (1978). Junior high school pupils’ understanding of the particulate nature of matter: An interview
study. Science Education, 62(3), 273-281. https://doi.org/10.1002/sce.3730620303
Novick, S., & Nussbaum, J. (1981). Pupils’ understanding of particulate nature matter: A cross- age study. Science Education, 65(2),
187-196. https://doi.org/10.1002/sce.3730650209
Özalp, D., & Kahveci, A. (2015). Diagnostic assessment of student misconceptions about the particulate nature of matter from
ontological perspective. Chemistry Education Research and Practice, 16(3), 619-639. https://doi.org/10.1039/c5rp00096c
Özmen, H., Ayas, A., & Coştu, B. (2002). Determination of the science student teachers’ understanding level and misunderstandings
about the particulate nature of matter. Educational Sciences: Theory & Practice, 2(2), 507-529.
Özmen, H. (2011). Turkish primary students’ conceptions about the particulate nature of matter. International Journal of
Environmental & Science Education, 6(1), 99-121. https://www.researchgate.net/publication/265438996_Turkish_primary_
students%27_conceptions_about_the_particulate_nature_of_matter
570
ISSN 2538–7138 /Online/ (pp. 558-574)
Palečková, J., Tomášek, V., & Straková, J. (1997). Třetí mezinárodní výzkum matematického a přírodovědného vzdělávání (Výsledky
žáků 7. a 8. ročníků – přírodovědné předměty) [The third international research in mathematics and science education
(Results of 7th and 8th graders - science subjects)], Ústav pro informace ve vzdělávání.
Park, E. J., & Light, G. (2009). Identifying atomic structure as a threshold concept: Student mental models and troublesomeness.
International Journal of Science Education, 31(2), 233-258. https://doi.org/10.1080/09500690701675880
Pella, M. O., & Carey, R. L. (1967). Levels of maturity and levels of understanding for selected concepts of the particle nature of
matter. Journal of Research in Science Teaching, 5(3), pp. 202-215. https://doi.org/10.1002/tea.3660050304
Pekel, F. O. (2021). The effect of concept cartoons and argumentation-based concept cartoons on students’ academic achievements.
Journal of Baltic Science Education, 20(6), 956-968. http://dx.doi.org/10.33225/jbse/21.20.956
Podroužek, L. (2003). Úvod do didaktiky prvouky a přírodovědy pro primární školu [Introduction to the didactics and science for
primary school]. Aleš Čeněk.
Renström, L., Andersson, B., & Marton, F. (1990). Students’ conceptions of matter. Journal of Educational Psychology, 82(3), 555-
569. https://doi.org/10.1037/0022-0663.82.3.555
Samková L. (2018). Assessing future teachers’ knowledge on fractions: Written tests vs concept cartoons. Journal on Efficiency
and Responsibility in Education and Science, 11(3), 45-52. http://doi.org/10.7160/eriesj.2018.110301
Snir, J., Smith, C. L., & Raz, G. (2003). Linking phenomena with competing underlying models: A software tool for introducing
students to the particulate model of matter. Science Education, 87(6), 794-830. https://doi.org/10.1002/sce.10069
StatSoft, Inc. (2013). Statistica (data analysis software system), version 13.3. http://statistica.io
Stepans, J. (2003). Targeting students’ science misconceptions. Showboard.
Taber, K. S., & Abdo, K. (2013). Developing chemical understanding in the explanatory vacuum: Swedish high school students’
use of an anthropomorphic conceptual framework to make sense of chemical phenomena. In G. Tsapralis, & H. Sevian
(Eds.), Concepts of matter in science education (Vol. 19, pp. 231-247). Springer. http://dx.doi.org/10.1007/978-94-007-5914-5
Taber, K. S. (2014). Ethical considerations of chemistry education research involving ‘human subjects.’ Chemistry Education Research
and Practice, 15(2), 109-113. https://doi.org/10.1039/C4RP90003K
Tan, K. Ch. D., Goh, N. K., Chia, L. S., & Treagust, D. F. (2002). Development and application of a two-tier multiple choice diagnostic
instrument to assess high school students’ understanding of inorganic chemistry qualitative analysis. Journal of Research
in Science Teaching, 39(4), 283-301. http://dx.doi.org/10.1002/tea.10023
Tatar, E. (2011). Prospective primary school teachers’ misconceptions about state of matter. Educational Research and Reviews,
6(2), 197-200.
https://www.researchgate.net/publication/268420412_Prospective_primary_school_teachers%27_misconceptions_
about_states_of_matter
Unver, A. O., & Arabacioglu, S. (2015). Helping pre-service science teachers to understand atomism through observations and
experiments. Journal of Baltic Science Education, 14(1), 64-84. https://dx.doi.org/10.33225/jbse/15.14.64
Valanides, N. (2000). Primary student teachers’ understanding of the particulate nature of matter and its transformations during
dissolving’. Chemistry Education: Research and Practice in Europe, 1(2), pp. 249-262. http://dx.doi.org/10.1039/A9RP90026H
Westbrook, S. L., & Marek, E. A. (1991). A cross-age study of student understanding of the concept of diffusion. Journal of Research
in Science Teaching, 28(8), 649-660. https://doi.org/10.1002/tea.3660280803
571
(pp. 558-574) ISSN 2538–7138 /Online/
Appendix - The Multiple-Choice test
1. Can we see individual atoms?

a) Yes, the atoms are large enough for us to be able to see them under a normal microscope.
b) No, atoms cannot be seen; we can only believe that they exist.
c) Yes, large atoms can be seen even with the naked eye.
d) Yes, we can see atoms, but only with a special laboratory instrument.
e) Other response:
2. Are atoms alive?

a) Yes, because atoms can grow and divide.
b) No, atoms do not have the characteristics of living organisms.
c) Yes, atoms are alive because they move.
d) Only atoms of living things are alive.
e) Other response:
3. Animate beings consist of many atoms. What will happen to these atoms after the animal dies?
a) The atoms stop moving.
b) The atoms return to the environment.
c) When an animate being dies, the atoms cleave to simpler parts, and those then create new atoms.
d) The atoms cease to exist once an animate is decomposed.
e) Other response:
4. What is the shape of the atoms?

a) Atoms may have different shapes depending on the kind of matter they compose (they may be round,
oval, oblong, etc.).
b) Atoms are in the form of balls that are full inside.
c) Atoms are flat (as when we smash a ball made of modelling clay).
d) The only thing we can say about an atom is that it has a positive nucleus and an electron shell.
e) Other response:
5. Are all atoms of the same size?

a) They are not because the size of an atom is determined by the number of protons, neutrons, and elec-
trons that the atom is composed of.
b) They are because there is only one basic type of atom in the universe from which all more complex
atoms have been created.
c) They are not because the size of an atom is only determined by the number of protons and neutrons
in the atomic nucleus.
d) All atoms are equal but produce molecules of different sizes.
e) Other response:
6. Do all atoms have the same weight?

a) No, because atoms of gases have no weight at all.
b) No, because the weight of an atom depends on how many simpler atoms it is composed of.
c) Yes, all atoms are particles of the same weight.
d) No, because atoms have a different number of protons, neutrons, and electrons.
e) Other response:
572
ISSN 2538–7138 /Online/ (pp. 558-574)
7. Do the atoms of gold have the same properties as a small piece of pure gold?
a) Yes. Gold is hard, and therefore the gold atom is also hard.
b) No, a single atom of gold does not have the same properties as a piece of gold.
c) Yes, the atom of gold is shiny, like the gleam of any item of gold.
d) No, since a piece of gold is composed of both gold atoms and matter that fills the space between
these atoms.
e) Other response:
8. Do atoms in a solid move?

a) Yes, the atoms in a solid vibrate.
b) No, they cannot move because there are no gaps between the particles in a solid.
c) No. The atoms do not move because the atoms in a solid are heavy.
d) No, the atoms do not move; only electrons move in atomic shells.
e) Other response:
9. Do the atoms in an iron rod enlarge when the rod is heated?

a) When heating an iron rod, only the nuclei of the atoms will grow.
b) When heating an iron rod, electrons will release from the atomic nuclei; thus, enlarging the atoms.
c) Heating the rod will increase the distance between the atoms and the range of their vibrations.
d) During heating, only the rod increases in volume; the size of the atoms and the distance between them
do not change.
e) Other response:
10. Coal consists of carbon atoms. What happens to the carbon atoms after a piece of coal is hammered
to dust?
a) The carbon atoms will not change.
b) The blows of the hammer will cause a change in the size of the atoms.
c) When the coal splinters into dust, the atoms will also disintegrate.
d) Small parts will fall off from some carbon atoms, and so these atoms will become smaller.
e) Other response:
11. Imagine that all atoms of a sheet of paper have been removed. What would then be left?
a) Some paper dust would remain.
b) Nothing would be left.
c) Only a tiny piece of paper of very low weight would remain.
d) Energy would remain.
e) Other response:
12. What happens to the air atoms when they collide with a train moving at high speed?
a) At the moment of collision, the atoms are protected by their rigid shells; much as a nut shell protects
the nut.
b) Air is elastic, therefore, the air atoms are elastic, too; that is why they can be easily compressed and will
not break in the event of collision with a train.
c) If the speed of the train is sufficiently high, some of the air atoms will break on impact, breaking into
small fragments.
d) In a collision with a train, the air atoms will not change in any manner whatsoever.
e) Other response:
573
(pp. 558-574) ISSN 2538–7138 /Online/
13. Does an atom need to be broken to free an electron?

a) No, the electrons can break off without breaking the atom.
b) Electrons are one of the fundamental particles that an atom is composed of. If the electron separates
from the atom, the atom divides.
c) No; you can break the shell of the atom.
d) Electrons cannot be removed from the atom.
e) Other response:
14. How can the atoms of one element be transformed into the atoms of another element?
a) The cause of the transformation of atoms into different atoms is sunlight.
b) Unstable atoms may spontaneously change to other atoms during radioactive decay.
c) Atoms cannot transform because each of the protons, neutrons, and electrons in an atom is unique.
d) Atoms may be transformed into other atoms during the occurrence of such natural phenomena as
thunder and lightning.
e) Other response:
Received: May 14, 2022 Revised: June 25, 2022 Accepted: July 20, 2022
Cite as: Hejnova, E. (2022). Comparing pre-service primary school teachers’ and lower-secondary learners’ understanding
the particulate nature of matter. Journal of Baltic Science Education, 21(4), 558-574. https://doi.org/10.33225/jbse/22.21.558
Eva Hejnová PhD, Assistant Professor, Department of Physics, Faculty of Science,

University of Jan Evangelista Purkyně in Ústí nad Labem, Pasteurova
3544/1, 400 96 Ústí nad Labem, Czech Republic.
E-mail: eva.hejnova@ujep.cz
Website: https://www.physics.ujep.cz/en/
574
USING A MACHINE LEARNING

APPROACH TO EXPLORE
NON-COGNITIVE FACTORS ISSN 1648-3898 /Print/
AFFECTING READING, ISSN 2538-7138 /Online/
MATHEMATICS, AND SCIENCE

LITERACY IN CHINA AND THE Abstract. Non-cognitive factors are
UNITED STATES considered critical aspects in shaping
students’ academic achievement. This
study aims to analyze and explore the
mechanisms of the influence of non-
cognitive factors on 15-year-old students’
Lu Ye, Yuqing Yuan
abilities in China and the United States.
Based on the Programme for International
Student Assessment (PISA) 2018 education
dataset, the Classification and Regression
Introduction
Tree (CART) model identifies and explains
the factors. The study finds that there
Personal ability is a student’s ability to sift, compare, contrast, and in-
are 11 most influential common features
tegrate information from multiple sources (Duckworth & Yeager, 2015). The
in China and 9 in the United States. The
degree of personal ability is significant in determining whether students have
two countries have 5 common features,
acquired the knowledge and skills to adapt to future society. The Programme
the meta-cognition assess credibility,
for International Student Assessment (PISA) regularly evaluates the reading,
summarizing text ability, PISA test difficulty
mathematics, and science literacy of 15-year-old students to assess their
perception, science learning time, and
personal abilities. Education reflects cultural variations, which lead to vari-
school lessons numbers per week. Family
ous educational ideals, educational views, educational systems, educational
economic status also impacts personal
techniques, and evaluation standards. As a result, education in China and the
ability. Regarding subject characteristics,
United States exhibits glaring contrasts. Based on the viewpoint of cultural
attitude towards failure is the determinant
values, Zhao et al. (2006) contend that the primary distinction between
of reading and mathematics. Cooperation
Chinese and American educational concepts is the distinction between
and competition among students help
general education and individual education, which is primarily manifested
to improve mathematics and science.
in: innovation and uniformity, independent thought and passive acceptance,
Furthermore, the comparison between
equality, looseness and strict authority, and emphasis on knowledge teach-
the two countries concludes that self-
ing and ability training. In the PISA 2018, in 4 provinces and cities in China
awareness, family economic status, and
(Beijing, Shanghai, Jiangsu, and Zhejiang), students ranked first in the three
school learning environment are critical
subjects among the participating countries. The students’ basic literacy
to personal ability. The study concludes
achievement rate is the first among the participating countries, and the
that it is necessary to foster a sense of
total number of high-level students is the highest. From the vertical point of
healthy competition among students at the
view, China has made significant progress compared with the 10th ranking
school level and provide more attention to
in PISA 2015. In a side-by-side comparison, the United States ranked 25th
students with low family socioeconomic
globally, 13th in reading, 18th in mathematics, and 37th in science, ranking
status to improve their abilities.
low among the 79 countries participating in the test. And with 6% of GDP
invested in education, the United States is at the top of the world. The annual Keywords: machine learning approach;
investment per student is more than $15,000, more than ten times that of non-cognitive factors; PISA 2018; personal
China, ranking second among the Organization for Economic Co-operation ability
and Development (OECD) countries (U. S. Department of Education, 2014).
When comparing the two countries we cannot ignore the differences in the
reading of Chinese versus English. Chinese is a logographic writing system Lu Ye, Yuqing Yuan
and is visually denser than English. English words consist of letter strings, Zhejiang Sci-Tech University, China
while Chinese words are composed of characters. They operate differently
in attention, word identification, and eye-movement control (Yu & Reichle,
575
STATES ISSN 2538–7138 /Online/
(pp. 575-593)
2017). From the results of input leading and achievement developed countries backward, it is a recognized fact
to evaluate the high investment and poor effect of primary and secondary education in the United States. There-
fore, studying the factors influencing personal ability can explore why China’s primary education is ahead of the
United States in three subjects and has great significance in promoting the overall development of the educational
environment in developing countries.
This study adopts the Classification and Regression Tree (CART), a machine learning approach to research the
complex relationships between non-cognitive factors. The CART model examines the effects of non-cognitive fac-
tors on students’ reading and mathematics science scores and whether the influencing factors vary from discipline
to discipline, which can help us understand the impact of different non-cognitive factors on personal quality and
how it affects students’ reading, mathematics, and science ability. Further, understand the differences between
the two countries and how to improve students’ personal ability.
The contributions of the paper are threefold. First, the study investigated the non-cognitive factors in the per-
formance of three subjects and enriched the research results on the factors influencing students’ personal ability.
Based on PISA 2018 Chinese and American data, all relevant non-cognitive variables were selected. To explore the
factors influencing students’ personal ability and identify the key variables that distinguish high ability students
from low ability students. By improving these variables, students’ personal abilities will be enhanced.
Second, the study applied the machine learning method to study this problem and enriched the application
of machine learning in pedagogy. Due to PISA data’s large and complex relationship, there are more non-response
errors. And the initial variables in this study cover a wide range and a large number. So, CART, which implements
a decision tree classification algorithm, was chosen to avoid the subjective bias brought by manual selection.
Third, common and different features are derived by comparing and analyzing the performance of non-cog-
nitive factors on students’ achievement in three subjects in China and the United States. Education is the nation’s
foundation. China and the United States are typical countries with different historical, cultural, and psychological
backgrounds. The United States is the representative developed country, whereas China is the largest developing
country. On the one hand, although national conditions are different, for personal ability itself, the factors that
influence personal ability performance identified in this study using machine learning methods are objective. On
the other hand, also taking into account the differences in national conditions, factors specific to both countries
are further analyzed. This study compares and analyzes the influencing factors of students’ personal abilities in
the two countries, reformulates the differences between Chinese and American education, and provides feasible
suggestions for improving students’ personal abilities.
Literature Review
Previous studies have found that many factors affect students’ performance in personal ability (Lee & Shute,
2010; Yorulmaz et al., 2009; Eriksson et al., 2021). Scholars have focused on teaching methods and activities, but
these instructional factors are likely to interact with students’ specific dispositions (Tonga et al., 2019; Bellova et
al., 2018). Too much attention given to learners’ general and specialized cognitive talents and intellectual tenden-
cies may limit the ability to profit from various types of experience (Heckman et al., 2006). However, little research
has been done on the links between personal ability and non-cognitive factors to see how they alter. Duncan and
Murnane (2014) stated that non-cognitive factors are related to the quality of teaching and the internal capacity
accepted by students. The educational experience transforming content-based knowledge into applied knowledge
may be shaped by non-cognitive factors related to students’ individual, family, and school. There are non-cognitive
factors such as seriousness and learning methods related to reading, mathematics, and science that shape personal
ability. For example, learning style (Tseng & Chu, 2018), reading frequency, self-cognition, and learning methods,
including meta-cognition and knowledge application ability, are essential predictors of readers’ good reading abil-
ity. However, other factors not explicitly related to students’ study experiences are also associated with personal
ability. These factors are collectively called motivational factors, such as mastery or learning goals (Toste & Didion,
2020), learning drive, and goal attractiveness. Hobri et al. (2018) also emphasized that critical thinking is one of
the essential competencies individuals should have in modern society and a top priority in higher education. It is
an objective and self-regulated judgment aimed at solving problems or making decisions that can be recognized
as a predictor of potential study success.
In addition to the factors related to students’ personal ability, the research also found environmental factors
affecting students’ personal ability (Chen et al., 2021; Bailey et al., 2010; Krieken et al., 2015). Environmental influ-
576
ISSN 1648–3898 /Print/ USING A MACHINE LEARNING APPROACH TO EXPLORE NON-COGNITIVE FACTORS
ISSN 2538–7138 /Online/ STATES
(pp. 575-593)
ences frequently provide resources and support for learning and growth processes connected to improved reading,
mathematics, and scientific literacy. Juan and Visser (2017) found that students’ most pertinent social contexts are
the home and school environments, involving different actors, social interactions, and resources.
In terms of the home environment, Chen et al. (2019) have found that parents’ educational attainment, em-
ployment position, and home assets are all factors that directly affect students’ achievement. Huang et al. (2021)
also claimed that these home assets, such as culture and learning resources, tend to assist pupils’ motivation and
resources for personal ability improvement. Personal ability is also influenced by parents’ expectations, educa-
tional techniques, and family environment. Tan and Hew (2017) have shown that home use of information and
communications technology (ICT) positively impacts students’ reading, mathematics, and science performance.
Hu and Gong (2018) found that students with higher ICT levels perform better than those with lower ICT levels.
Furthermore, these factors in the home environment are linked to families’ socio-economic status.
Regarding classroom teaching, with the rise in popularity of quality education in the international commu-
nity (Aditomo & Koehler, 2020), China’s quality education is rising. The essence of teaching and educating people
is gradually being highlighted, encouraging people to conduct in-depth examinations and pay attention to the
influencing factors of students’ personal ability. As a result, the evaluation of teaching levels emerges at a historical
juncture and spreads swiftly worldwide (Shao & Sun, 2019). The reform of China’s education system has intensified
in recent years, and the size of education has been continuously enlarged, increasing the impact of teaching level
on human ability. Conducting rigorous, objective, and quantitative assessments of students’ learning environments
and teaching approach is critical.
Beyond the classroom, there are also essential factors in the school environment that support student learning
and achievement (Gimenez & Ciobanu, 2021; Liu et al., 2017; Saeki & Quirk, 2015; Laftman et al., 2017). For example,
Gimenez and Ciobanu (2021) concluded that the peer effect is essential for academic performance. The class com-
position according to gender or race, students’ ability, and their socio-economic levels are the most commonly
used characteristics to measure peer effect. Furthermore, each school has different learning resources, including
basic infrastructure, materials, and teaching resources (Liu et al., 2017). These issues were frequently linked to
school financing sources and the overall quantity of resources available. Some school-level factors not related to
school financing resources have also influenced the students’ academic performance to a large extent. Although
not related to school financing resources, students’ social and interpersonal experiences are equally important in
the school environment. For example, students, social connections (Saeki & Quirk, 2015), and exposure to bullying
(Laftman et al., 2017) have also been essential predictors of personal ability.
From the above research, students have high ability, which comes from the support of personal learning, fam-
ily economic status, school resources, and other aspects. Although a general association between these influence
factors and personal ability has been found in prior research, the influences of these factors on personal ability may
vary in different subjects (reading, mathematics, and science). In addition, most research articles usually only focus
on some specific factors, and few articles systematically and comprehensively study the relationship between all
non-cognitive factors and personal ability. The development of non-cognitive factors is necessary for students to
acquire good literacy and ability, which will improve students’ adaptability to future society.
Therefore, this study aimed to explore the influence of non-cognitive factors on personal ability. The following
research questions were determined:
(a) Do students’ grades in three subjects make a statistically significant correlation? If so, what non-cognitive
factors simultaneously impact all three subjects?
(b) What are the subject characteristics of reading, mathematics, and science?
(c) What are the same and different non-cognitive factors in China and the United States?
General Background
This study was divided into three parts to explore the critical non-cognitive factors that influence students’
personal abilities. In the first part, factors influencing all three subjects of reading, mathematics, and science were
577
(pp. 575-593)
identified and described qualitatively and quantitatively based on machine learning methods and existing theoreti-
cal frameworks. Next, for the second part, the single-subject characteristics of the three subjects were explored.
In the third part, the results from both countries were comparatively analyzed to obtain constructive suggestions.
Instrument and Procedures
Based on the proposed conceptual framework for the relationship between non-cognitive factors and stu-
dents’ academic performance, this study employed variables from the PISA 2018 database. PISA 2018 mainly as-
sessed students’ literacy in reading, mathematics, and science, and all three were used as dependent variables in
the present study. The variables analyzed in this study are divided into three categories: personal variables, family
variables, and school variables. Personal variables refer to the learning motivation, learning attitude, and learning
purpose related to reading, and family variables refer to the characteristics of parents and learning resources at
home. Finally, school variables refer to the teachers’ teaching methods, classroom learning experience, the school’s
educational resources, and students’ life experiences in school.
In this study, 114 initial variables were set. First, set the missing values of variables to 0, delete the samples
with more than half of the missing value, and then use k Nearest Neighbor for interpolation. Finally, 62 variables
were considered totally in China, including 36 student-level variables, 10 family-level variables, and 16 school-level
variables. There were 73 variables in the United States, including 40 student-level variables, 13 family-level variables,
and 20 school-level variables. The differences between the Chinese and American variables were mainly in the
ICT questionnaire. Since Chinese students lacked a questionnaire on ICT use, only one variable about ICT was set.
The data from the ICT questionnaire of American students were complete and comprehensive, and nine relevant
variables were selected in this study.
Some variables in this study were based on the original indices, while some were derived variables that PISA
develops. The derived variables were calculated using the Item Response Theory (IRT), which is a general term for a
series of psychometric models (Davier et al., 2019). The purpose of IRT was to determine whether underlying char-
acteristics can be reflected by the PISA questions and the interaction between the test questions and the test taker.
Data Sources
The data for this study were from the Chinese and American samples in the OECD PISA 2018 database. PISA
examines 15-year-old students’ personal ability using a stratified sample approach. 12,058 Chinese students were
sampled from 361 schools across Beijing, Shanghai, Jiangsu, and Zhejiang. And 4,838 students were sampled from
across the United States. Moreover, we combined the ICT familiarity and students’ social background question-
naires completed by students to gather student information. After deleting the invalid questionnaire, it was finally
determined that the sample size of 11,977 in China and 4,838 in the United States.
Data Analysis
A data mining approach was often deployed for data analysis when detecting and interpreting a large da-
tabase with complex relations between numerous variables. Given many possible predictors of different types
in this study, CART is adopted. CART belonged to the supervised learning methods of data mining techniques,
for specifying the conditional distribution, given a vector of predictor values. Such a model was developed on a
random subset of the data (training sample), and then the results were validated on a separate random sample
(test sample) (Breiman & Friedman, 2015; Strobl et al., 2009).
CART did not make any assumptions about the probability distribution of the individuals sampled from it
and was not affected by the issue of multi-collinearity between predictors. According to statistical criteria, the al-
gorithm divided participants into consecutive binary groups. Each independent variable was assessed by splitting
the subjects into two groups (referred to as the child nodes) based on their capacity to reduce the contaminants of
the parent node, starting with the entire sample (referred to as the root or parent node). The independent variables
might be sorted, continuous, or nominal (Hox, 2010). Non-cognitive factors were entered into the CART: student-,
family-, and school level.
For machine learning, a comparable distribution of each group, low and high, was preferable to remove
bias in model training. Students were divided into two groups: those with low ability and those with high ability.
578
(pp. 575-593)
Low-ability students were those with limited ability at reading levels two and below, mathematics levels three
and below, and science levels three and below. High-ability students were those with reading at levels three and
better, mathematics at levels four and better, and science at levels four and better. As dependent variables, the
study had taken the values of the scores obtained by the school in the general skills evaluated in each competence
(PV1READ, PV1MATH, PV1SCIE) (OECD. PISA 2018 Data Base). At the same time, other items in the questionnaire
linked to personal ability were used as independent variables.
After data processing, first, student, family, and school-level non-cognitive factors are used as independent
variables. After being divided into high-low levels, the scores of the three subjects are used as dependent variables
to build CART models for different subjects. Second, the top 20 most relevant factors for each of the three subjects
are derived separately. Third, in the top 20 factors of the three subjects, find the same features affecting the three
subjects. Expressed them as the key influencing factors on students’ personal ability, that is, the common features.
Fourth, factors affecting only two subjects were identified in addition to the factors common to all three subjects.
These factors can reflect the subject’s characteristics. Fifth, the above steps are the same in China and the United
States, and the results in the two countries will be compared and analyzed.
Research Results
Descriptive Statistics
In PISA 2018, Chinese students scored an average of 555, 591, and 590 in reading, mathematics, and science.
American students scored an average of 500, 493, and 497 in reading, mathematics, and science. Mathematics and
science were nearly 100 points lower than Chinese students. The distributions of scores in three subjects in China
and the United States are summarized in Figure 1.
Figure 1
Distribution of Scores in Each Subject
The students (China N=11,977, the United States N=4,838) were divided into low-high groups according to
their scores in the three subjects. As a result, 2,252 (19%) and 9,725 (81%) Chinese students belonged to the low and
high reading performing groups, respectively. 3,195 (27%) and 8,782 (73%) in low-high mathematics groups, 3,839
(32%) and 8,138 (68%) in low-high science groups. The distributions of the United States showed that 2,040 (42%)
and 2798 (58%) of the students belonged to the low and high reading performing groups. Low-ability students
dominated mathematics and science. 3,720 (77%) and 3,463 (72%) students belonged to the low mathematics and
science performing groups. China’s mathematics and science scores were higher than the United States overall
and were spread over a wide range of high-level students. Figure 2 showed that China’s performance in all three
subjects was higher than that of the United States and was concentrated among low-level students.
579
(pp. 575-593)
Figure 2
Distribution of Scores in Each Subject Using Low-high Groupings
The correlation diagram was shown in Figure 3. The first picture was China, and the second picture was the
United States. Of the three subjects, they all had a strong correlation. Scores of the variables (PV1READ, PV1MATH,
PV1SCIE) presented high correlations. The results were statistically significant under the 95% confidence interval,
and the pairwise correlations were between 0.8 and 0.9. Comparing the two countries, the correlation coefficients
of mathematics and science were 0.88 and 0.89. The two countries’ correlation between mathematics and science
was consistent. Furthermore, the correlation between reading and mathematics, reading and science American
data showed a stronger correlation than in China.
Figure 3
The Correlation Diagrams of Three Subjects in China and the United States
Common Features of Three Subjects in China
Table 1
Common Features of the Three Subjects Identified in China
Value range
Rank Feature Description
(min.~max.)
-1.41~2.33
Student 1 METASPAM Meta-cognition: assess credibility (IRT scaling)
-1.72~1.36
3 METASUM Meta-cognition: summarizing (IRT scaling)
-1.64~1.5
5 UNDREM Index understanding and remembering (IRT scaling)
-2.7114~2.6574
6 JOYREAD Index enjoyment of reading (IRT scaling)
580
(pp. 575-593)
Value range
(min.~max.)
-1.272~3.0064
8 PISADIFF Perception of difficulty of the PISA test (IRT scaling)
-3.1675~2.3693
10 RESILIENCE Resilience (IRT scaling)
-5.0771~3.1015
Family 9 ESCS Index of economic social and cultural status (IRT scaling)
School 2 SMINS Instructional time: Science courses 0~2210
4 ST059Q03TA Number of class periods in science 0~40
7 ST060Q01NA number of class periods per week. 10~80
-3.2583~2.7562
11 BELONG Index sense of belonging (IRT scaling)
In China, 11 common features with high scores in three subjects were selected from 67 factors. Among these
11 factors, six were related to individual students, four were at the school level, and only one was related to families.
As a result, the 11 common features were identified and are provided in Table 1.
At the student level, the most important variable related to classifying students into low and better-performing
students in reading, mathematics, and science was METASPAM. A dependability index displayed people’s sincerity
in dealing with things and was positively connected with the target variable. Similarly, METASUM, the third most
crucial variable, was positively correlated with the target variable. METASUM is an index for the students’ meta-
cognitive awareness of strategies for summarizing texts. UNDREM is a measure of one’s capacity to comprehend and
recall information. Students with a good understanding and recall tend to do well in school. JOYREAD is an index
of enjoyment of reading, and PISADIFF is an index of subjective difficulty perception of the PISA test. RESILIENCE
is resilience to events, also known as self-healing ability. These six variables indicated that students’ self-efficacy
was closely related to personal ability. Poor readers differed from better readers because they have lower intrinsic
sensitivity to learning and weaker meta-cognitive awareness of learning strategies. These results were consistent
with the literature on the role of self-efficacy (Li et al., 2020), intrinsic enjoyment of learning (Areepattamannil et al.,
2011), and metacognitive awareness of strategies in education (Sun-Lin & Chiou, 2017) in students’ achievement.
ESCS was the only family variable among the 11 key factors that affect all three topics at the family level. The
ESCS contained a basic summary of each parent’s educational level, family property status, and job position. In
terms of family social status, both the highest educational background and family property status of parents had
positive correlations with the scores of all subjects. Increased family status meant that students had access to better
learning resources, positively impacting their reading, mathematics, and science literacy.
4 of the top 20 variables were non-cognitive school variables related to reading, mathematics, and science.
There were two variables, a single item was included in some other indexes, and one variable was a single item
measurement of a factor. These variables were discussed in more detail. SMINS is an index of students’ science
courses learning time per week, including the entire learning time of in-class and out-of-class scientific stud-
ies. This research considered that curriculum learning belonged to teaching rather than self-study. Thus, it was
classed as a school-level variable. Science learning time (SMINS) and scientific-related course numbers per week
(ST059Q03TA) demonstrated that science study benefited personal development. ST060Q01NA is an index of school
lesson numbers per week, and this variable was positively connected with the target variable. Scientific learning
may develop people’s cognitive capacity and practical ability. Students who take many school classes are likely
to receive good scores, which supports the opinion of Woods-McConney (2013, 2014). In addition to the number
of hours and sessions of classroom learning, BELONG, a variable that measures a student’s sense of belonging to
the school, was also a significant predictor of personal ability. BELONG is an index measuring the students’ sense
of belonging in their schools, showing students’ cognition, emotion, motivation, conduct in school interactions,
classroom learning time, and the number of classes.
581
(pp. 575-593)
Common Features only Two Subjects in China
Table 2
Common Features only Two Subjects Identified in China
Value range
Special Feature Description
(min.~max.)
read×math -1.8939~1.8905
GFOFAIL General fear of failure (IRT scaling)
read×science -2.1464~1.7411
EUDMO Eudaemonia: meaning in life (IRT scaling)
math×science -1.6391~2.0165
PERFEED Index perceived feedback (IRT scaling)
-4.4106~1.2099
HEDRES Index home educational resources (IRT scaling)
-2.1428~1.6762
PERCOOP Perception of cooperation at school (IRT scaling)
-1.9892~2.0378
PERCOMP Perception of competitiveness at school (IRT scaling)
More pertinent to the research questions in China, the results of the common features in every two subjects
analyzed for reading, mathematics, and science were summarized in Table 2. Both reading and mathematics and
reading and science had only one particular shared variable, GFOFAIL, and EUDMO, respectively, and both were
at the level of individual student variables. GFOFAIL is the fear of failure Index, which is how students react when
faced with failure. Lower scores indicated weaker resistance to failure and a greater tendency to be poor learners.
The index ranked 17 in reading predictors and 15 in mathematics, and more scientifically, it had a relatively large
impact on reading and mathematics performance. EUDMO is an index of how much students think about the
meaning of life. The results showed that students with clear life goals and long-term plans had higher reading
and science scores. Thinking about the meaning of life played a decisive role in the effective orientation of one’s
role. Clear goals guided and promoted one’s growth and positively impacted the improvement of one’s abilities.
As shown in Table 2, mathematics and science contained four special common features. Three belonged
to the school level. PERFEED is an index to measure the quality of teachers’ instructional practice. PERCOOP and
PERCOMP belonged to the ‘competitive and cooperative relationship of students in learning’ factor. In addition,
HERDES belonged to the family category, indicating the family education resource index. The comprehensive
literature review also revealed the consistency between the results of this study and those previously reported.
For example, teaching quality and family resources have been recognized as important factors in mathematics
and science literacy (Lau & Lam, 2017).
Common Features of Three Subjects in the United States
Table 3
Common Features of the Three Subjects Identified in the United States
Value range
(min.~max.)
-1.272~3.0064
Student 1 PISADIFF Perception of difficulty of the PISA test (IRT scaling)
-1.41~2.33
2 METASPAM Meta-cognition: assess credibility (IRT scaling)
-1.72~1.36
4 METASUM Meta-cognition: summarizing (IRT scaling)
-2.5144~2.0258
5 AUTICT Perceived autonomy related to ICT Use (IRT scaling)
582
(pp. 575-593)
Value range
(min.~max.)
-2.6033~1.9885
7 COMPICT Index ICT: Perceived ICT competence (IRT scaling)
-2.345~2.0054
9 COMPETE Competitiveness (IRT scaling)
-6.8518~4.2599
Family 6 WEALTH Family wealth PISA 2006 (IRT scaling)
School 3 ST060Q01NA number of class periods per week. 10~80
8 SMINS Instructional time: Science courses 0~2210
Among the 9 variables of these three subjects selected from 73 factors in the United States, 6 were related
to individual students, 2 were at the school level, and only 1 was related to families. As a result, the 9 common
features were identified and are provided in Table 3.
At the student level, as illustrated in Table 3, the most important variable related to classifying students into
low and better-performing students in reading, mathematics, and science was PISADIFF. PISADIFF is an index of
subjective difficulty perception of the PISA test. This indicator determined students’ ability to perceive the difficulty
of the PISA test in 3 aspects: storage of word statements, overall comprehension of the text, and logical thinking
when doing the questions. Ranking 2nd and 4th are METASPAM, an index of how seriously students take things,
and METASUM, an index of their ability to summarize texts. They were indexes of metacognitive awareness and
strongly predicted Chinese students’ academic performance, positively correlated with the target variable. AUTICT
is an index of perceived autonomy related to ICT use. The question “How often do you use digital devices for the
following activities outside of school?” reflected the autonomy of ICT use by investigating the frequency of active
use of electronic devices or information technology outside of school. COMPICT is an individual’s self-perception
of the ability to use information communication. Unlike autonomy, this index measures students’ perceived ability
to use ICT. Students with solid perceptions learned efficiently when using ICT, leading to good learning outcomes,
and thus helping improve academic performance. The last of the nine common features in the United States was
COMPETE. It ranked 17th in reading and mathematics and 18th in science among the top 20 variables extracted
for a single subject. It was clear that although students’ sense of competition did not have a decisive effect on
personal ability, it did affect all three subjects, and developing a sense of competition to improve personal ability
was an area of concern.
WEALTH was the only family variable among the nine key factors affecting all three topics at the family level.
WEALTH is an index of a family’s wealth, which provides a picture of the family’s wealth status. The family’s economic
situation influenced the learning environment and the educational resources provided to the child, and these
resources were closely linked to the child’s success. A more robust family economy brought more social capital
and human resources to invest in their children. On the other hand, families with weaker economic power will
have a disproportionate amount of resources invested in meeting the family’s basic needs and then will have an
insufficient investment in education. Families of different economic levels also have different levels of parenting
for their children, which can affect their children’s academic performance.
2 of the top 20 variables were non-cognitive school variables related to reading, mathematics, and science.
One single variable item was included as part of some other index, and one variable was a single-item measure of
a factor. School lesson numbers per week (ST060Q01NA) and science learning time (SMINS) were prescribed hours
of study. ST060Q01NA was ranked 3rd among the total variables, which showed that the higher the number of
courses of study established by the school, the more it contributes to students’ academic performance. In-school
education played a vital role in developing the personal abilities of students.
583
(pp. 575-593)
Common Features only Two Subjects in the United States
Table 4
Common Features only Two Subjects Identified in the United States
Value range
Special Feature Description
(min.~max.)
read×math TMINS Total Learning time (minutes per week) 170~3000
Self-concept of reading: -2.4403~1.8839

SCREADCOMP Perception of competence (IRT scaling)
ST059Q04HA Number of class periods per week in foreign language 0~34
ST185Q01HA Clear meaning or purpose. 1~4
-1.8939~1.8905
GFOFAIL General fear of failure (IRT scaling)
-7.8077~5.3188
read×science HOMEPOS Index home possessions (IRT scaling)
-1.3048~2.4969
ICTOUTSIDE Subject-related ICT use outside of lessons (WLE) (IRT scaling)
-4.0953~3.2545
math×science ESCS Index of economic social and cultural status (IRT scaling)
In the United States, the results of the common features in every two subjects’ analyses were summarized
in Table 4. Common features were specific to Americans in reading and mathematics, 4 from the student level
and 1 from the school level. Total time spent by students studying (TMINS) and the number of foreign language
courses (ST059Q04HA) were important indexes of high or low student achievement in reading and mathematics.
Students’ study time can be extended to homework and other extracurricular activities (Xiao & Hu, 2019). In this
study, students with high literacy tended to spend more time studying each week. In addition to study time, the
factor of metacognitive awareness was also important. Research has shown that students who read with an explicit
self-concept can grasp the focus of a text and effectively grasp the entire meaning (Salchegger, 2016). As a result,
these students achieve better academic performance. Clarity of personal planning (ST185Q01HA) was also an
important indicator of students with high literacy. GFOFAIL is the fear of failure index, the only variable affecting
Chinese students’ reading and mathematics performance.
In this study, other home possessions like a desk that students can use and a quiet place to study (included
in HOMEPOS) were associated with students’ reading and science literacy, consistent with the previous study
(Ngorosho, 2010). What’s more, ICT available out of lessons (ICTOUTSIDE), including test language lessons, science,
music, and sports, was also a strong index of students with high reading and science literacy (Hu et al., 2018). The
frequency of using electronic devices outside of the classroom was a better index of students’ ability to use ICT. As
digital reading became increasingly important, PISA 2018 utilized a new e-reading model. Students’ ability to use
ICT subjectively became particularly important.
ESCS is an index of family socioeconomic status. Numerous studies have demonstrated that socioeconomic
status significantly affects students’ academic performance (Flores-Mendoza et al., 2021; Muelle, 2020). And unlike
previous studies, the index positively affected all three subject scores in China, but only mathematics and science
were associated with it in the United States. It ranked 4th in mathematics and 3rd in science. This may be since the
United States invests 6% of its GDP in education resources, which is at the top of the world and is more prevalent
at home. Regardless of the family’s economic status, books, laptops, and other learning materials are available at
home, less impacting children’s reading motivation.
584
(pp. 575-593)
A Comparative Analysis of China and the United States
Table 5
Comparison of Common Features between China and the United States
Rank in China Feature Rank in US Feature
Student 1 METASPAM 1 PISADIFF
3 METASUM 2 METASPAM
5 UNDREM 4 METASUM
6 JOYREAD 5 AUTICT
8 PISADIFF 7 COMPICT
10 RESILIENCE 9 COMPETE
Family 9 ESCS 6 WEALTH
School 2 SMINS 3 ST060Q01NA
4 ST059Q03TA 8 SMINS
7 ST060Q01NA
11 BELONG
The section comparatively analyzed the factors between China and the United States. There were nine com-
mon features in the United States, 5 of which were non-cognitive factors identical to China’s. 3 came from the
individual student level, PISA test difficulty perception (PISADIFF), meta-cognition assess credibility (METASPAM),
and summarizing texts ability (METASUM). 2 came from the school level, science learning time (SMINS), and school
lessons numbers per week (ST060Q01NA). All 3 variables at the student level involved cognitive abilities, perceived
difficulty of the PISA test, and the student’s attention and summarizing and analyzing ability when treating things.
The 2 variables on metacognition (METASPAM, METASUM) were both ranked high. However, PISADIFF, the perceived
difficulty of the PISA tests, was ranked very differently. In China, PISADIFF ranked 9th out of 11 common features
at the bottom of the list and had a relatively small effect on performance in the three subjects. This indicated that
students’ perceptions of the difficulty of the test were highly influential in their final performance. The two variables
at the school level were the number of weekly lessons (ST060Q01NA) and science study time (SMINS). These two
variables also differed in importance for the two countries. Science learning time (SMINS) was more important
for China than for the United States, ranking 2nd in China and 8th in the United States. However, the number of
weekly lessons (ST060Q01NA) was highly relevant to American student achievement. However, it ranked 7th in
China, which showed that science learning was more effective in promoting student achievement in China than
in the United States. At the same time, the number of school courses was more effective in the United States.
2 of the 4 non-cognitive factors specific to the United States were related to ICT, which was significantly
different from China. AUTICT reflected the autonomy of ICT use by investigating the frequency of active use of
electronic devices or information technology outside school. COMPICT measured students’ perceived ability to
use ICT. The other two variables specific to the family level were WEALTH and COMPETE, a student-level measure
of student competitiveness. Similar to the Chinese family indicator ESCS, both were the educational resources or
life assistance the family can provide to the student, but the difference was that WEALTH measured the family’s
property status. At the same time, ESCS was more focused on the family’s social situation. ESCS was not a common
feature in all three subjects in the United States. However, it was at a high level in the ranking of importance in
mathematics (4th in a single subject) and science (3rd in a single subject). It was not a significant factor in reading,
as it was located after the 20th place.
585
(pp. 575-593)
Discussion
To explore the non-cognitive factors influencing students’ personal abilities, this study applied the CART al-
gorithm to analyze the reading, mathematics, and science subjects separately, classifying students as low or high
ability and identifying the top 20 variables that had the most significant impact on the model. The study found
five factors in the three subjects in China and the United States. The factors were meta-cognition assess credibility
(METASPAM), summarizing texts ability (METASUM), PISA test difficulty perception (PISADIFF), science learning
time (SMINS), and school lessons numbers per week (ST060Q01NA ). PISADIFF belonged to the student level, and
SMINS, ST060Q01NA belonged to the school level. The student-level variables had the most significant influence,
and the family-level variables had a relatively small impact.
At the student level, METASPAM, METASUM, and PISADIFF were non-cognitive factors that jointly affect the two
countries, illustrating the importance of the learner’s self-awareness to personal ability. The findings suggested that
persons who are more cautious and serious about their job have a higher personal ability level, implying that work
attitude is vital to academic progress (METASPAM). Different summary styles (METASUM) had a significant influence
on the three subjects’ outcomes. In addition to self-awareness, reading style and interest positively impact student
achievement. When reading an article, read it rapidly twice more or only look at the main lines, and then explain
it in your own words to help you grasp it better. Discussing the article’s topic with others based on a basic grasp is
beneficial to a deeper understanding and recall. It may be inferred that a student’s capacity to comprehend and
recall information (UNDREM) is an important factor in determining their grades. Reading enjoyment (JOYREAD)
is an important factor in measuring motivation to read, which is consistent with the outcomes found by Cho and
Toste (2018). Individuals who have loads of fun in reading will take the initiative to peruse, while those with a low
index think reading is an exercise in futility or just when there are rigid requirements in school. The outcomes show
that students with high records have higher scholarly execution at the student level. You can begin by developing
the interest and learning motivation to improve students’ performance.
At the family level, family socioeconomic status (ESCS) was the only family variable among the 11 key factors
that affect all three subjects in China. Family property status (WEALTH) was the only family variable in the United
States. Family socioeconomic status has long been a significant focus of educational study over the last 50 years
as a critical environmental element impacting kids’ learning. Consistent with Chiu and Ming (2017), their family’s
socioeconomic condition influences students’ academic success. According to relevant meta-analysis research,
family socioeconomic status (ESCS) was favorably connected with kids’ overall academic achievement (White et
al., 1993). Children’s socioeconomic situation offers them some crucial social impact, which will help them obtain
good academic achievements and pave the route for their success. Conger and Donnellan (2007) and Longo and
Lombardi (2017) reported that parental investment in resources (such as financial, human, and social capital) aids
children’s development. A parent’s social and economic circumstances influence their emotional relationship and
parenting conduct, substantially impacting their children’s academic performance and accomplishment level
(Chen, 2018; Kriegbaum & Spinath, 2016). In general, parents with a higher socioeconomic standing are more will-
ing and able to spend more on their children materially and emotionally. Emotional investment is quite significant.
Their academic performance will increase equally and favorably if parent-child contact within the family is more
harmonic and regular (Fan & Chen, 2001). Furthermore, if a student’s family’s general socioeconomic situation is
poor, parents will exhibit some behavioral and emotional abnormalities, preventing or reducing parents’ emotional
attention and educational investment in their children. The result is a decline in the same level of children’s academic
performance and achievement (April et al., 2017).
At the school level, the students’ school as a social environment also gave rise to specific experiences that
positively relate to students’ personal ability. 2 of these factors were included in the PISA 2018 survey as part of
student well-being assessment (BELONG BEINGBULLIED). The findings reveal that school belonging (BELONG) is
also linked to students’ academic achievement. Students who belong to a high school are more likely to have excel-
lent grades, and students who belong to a low school are more likely to have poor grades. This is in line with prior
research findings. Students’ time spent on homework and learning motivation is positively connected with their
sense of belonging at school (Hagborg, 1994). Even when ideal learning environments and other characteristics
are considered, school belonging is still the most important predictor of academic accomplishment (Gonzalez &
Padilla, 1997). School membership is positively connected with academic success (Smerdon, 2002).
The above analysis revealed that students’ beliefs, motivation, resources, and environment distinguish the
strength of their overall personal ability. Besides, the characteristics of single subjects should not be neglected.
586
(pp. 575-593)
Many factors do not affect all three subjects but have significant predictive power for the performance of two
subjects.
Reading and mathematics are more responsive to failure. When people feel the humiliation and shame gener-
ated by failure in an achievement circumstance (GFOFAIL), they tend to avoid self-expression and endeavor. This
concept captures the basic emotion of failure anxiety and its behavioral repercussions (Conroy, 2004; McGregor &
Elliot, 2005). Failure causes dread because it deprives people of their worth (Conroy, 2001). Fear of failure may have
various negative consequences (Martin & Marsh, 2011), including increased anxiety and decreased motivation,
and a detrimental influence on academic performance (Martin, 2002). The reason that science subjects are not
sensitive to failure may be because learning science encounters setbacks. Learning science subject is a process of
repeatedly experiencing failure and continually overcoming it, so the psychological tolerance in the face of failure
is large and does not have too serious consequences.
Mathematics is a rational subject that focuses more on theory and thinking, and reading and science learn-
ing focus more on practice and thinking about the meaning of life. Individuals’ understanding of the connotation
of life is referred to as their sense of the meaning of life (EUDMO). One of its purposes is to improve self-control.
Individuals with a sense of purpose in life can better think about long-term objectives based on cultural norms and
then change their emotions and actions to deal with scholastic assignments, which helps students improve their
grades. The findings showed a strong link between learning goals and academic success (Bouffard et al., 1995).
The results revealed that top performers’ mathematics and science teachers generally spent a larger pro-
portion of class time demonstrating, explaining, and extending scientific ideas, which involved interaction and
whole-class discussion with students. High achievers in mathematics and science literacy generally perceive their
teachers as providing considerable feedback or adapting lessons to students’ individual needs (PERFEED) (Han
& Xu, 2020). However, as the PISA report (OECD 2016) claimed, students typically receive teachers’ feedback as a
remedial measure to help them become academically stronger. Like Singapore, it outperformed all other partici-
pating countries in science literacy on PISA 2015. It is reported to be a country where teachers commonly provide
individual feedback to students in the class (Lau & Lam, 2017). Accordingly, a stronger association between this
form of teaching strategy and student competency might improve students’ mathematics and science scores.
The results of the study showed that the stronger the cooperative and competitive (PERCOOP, PERCOMP)
relationships, the better the students’ academic achievement. Cooperation and competition among students
helped improve individual mathematics and science achievement. Many studies have been conducted abroad
and found that cooperative learning, both in problem-solving situations and real-world knowledge, significantly
enhances problem-solving skills and academic achievement (Roseth & Johnson, 2008). Consistent with Tauer and
Harackiewicz (2004), competition and cooperation have positive aspects. They have the same effect on students’
learning by setting up two different cooperative and competitive teaching environments (Lawrence & Sherman,
2006). Cooperative learning helps cultivate students’ awareness of equal cooperation and enables students to
cooperate, encourage and complement each other as feedback information at different levels to form a multi-
directional information exchange. Finally, Clhen and Sapon-shevin (2004) confirm that “individual wisdom” affects
“collective wisdom” so that cooperative learning can improve students’ learning and social skills. It is also an assistant
to teachers in classroom management and teaching. While in today’s society, adolescents have an increasingly
prominent desire to express themselves. According to psychological studies, adolescent students gradually rec-
ognize the relationship between personal and social development under new living and educational conditions.
The ‘I’ and others, the ‘main self’ and the ‘guest self’ changed, began thinking of themselves, and really ‘discovered’
themselves. This has led to a new development of self-awareness and expressive consciousness. This desire for
expression is the psychological basis for participation in the competition, making individuals more willing to learn,
more interested in learning, more motivated to learn, and more time for education, which positively contributes
to academic achievement.
The previous discussion highlighted how specific variables seem to arise in the individual, the home, and the
school and the characteristics of the three subjects of reading, mathematics, and science. Due to the lack of some
answers in the questionnaire, the variables involved in this study may not be comprehensive. For example, there
were fewer responses on ICT in the questionnaire for Chinese students. Therefore, this study further explored the
predictor variables that influenced the personal abilities of United States students and analyzed them in comparison
with China, resulting in 3 findings worth considering.
First, competitive relationships were critical in today’s society. The two predictors of competition (COMPETE
and PERCOMP) in the United States and China were similar. Competitive relationships among students were vital
587
(pp. 575-593)
for students’ academic performance. However, in contrast to the views of some scholars, who argued that highly
competitive learning environments were detrimental to student development, Asian countries and regions tended
to “score first,” and many educational administrators prioritize competitive management models to achieve higher
scores (Hau et al., 2008). The OECD Social and Emotional Competence Assessment 2021 (OECD, 2021) also found that
when students are in highly competitive school environments, their ability to communicate with and trust others
decreases. Compared to students in OECD countries, students in China perceive higher levels of competition and
cooperation in the learning environment and hold higher achievement motivations to pursue success. With the
acceleration of the information age in our society and the prevalence of high technology and the Internet, today’s
society has gradually entered a period where it likes to argue, explore, and debate. Nowadays, students tend to
prefer to think independently to get things done. The fierce competition in the general environment of Chinese
society has a subtle impact on students. It is highlighted that it drives them to be significantly more motivated to
achieve in their pursuit of success. Thus, although a highly competitive learning environment is not conducive to
students’ psychological well-being, it can significantly motivate students to learn and better prepare them for future
social competition. At the same time, moderate “comparison” can help foster students’ achievement motivation
in pursuit of success and enhance their sense of meaning in life, which seems to be overlooked by most studies.
Second, unlike China, the United States ESCS indicator was only related to mathematics and science and does
not significantly influence reading literacy. However, the household property status indicator (WEALTH) impacted
all three subjects. Families with wealth accumulation and income levels above the social meant pay more attention
and invest more in students’ education. It has a certain utility on students’ subjective perceptions and judgments
of their learning abilities. Then it will significantly impact students’ academic performance and achievement levels.
Therefore, society and schools should pay more attention to students from low socioeconomic status families, as
they are more likely to have low self-efficacy than other students. Improving the quality of parental involvement
can compensate for the negative effects of family background disadvantages on students’ early development.
Families with low socioeconomic status should pay more attention to acquiring parent-child interaction skills and
improving the quality of parent-child interaction, which helps children improve their academic self-efficacy and
promote students’ academic achievement.
Third, Chinese students’ academic performance was less correlated with ICT use than in the United States.
Only reading literacy was correlated with ICT. The factor is called ICTRES, an indicator of communication infor-
mation technology resources, ranking 10th in the single subject of reading. The study results showed that for
the United States, the more opportunities to use information technology outside of school (AUTICT) to support
subject learning or school activities, the better the students’ academic performance. However, in China, this is
not the case. The reasons for this are as follows. First, for most areas in China, integrating information technology
with the curriculum in the school setting has not worked as well as it should. Such as insufficient attention to the
information technology courses offered, and insufficient facilities and resources to offer such courses. Second, the
design of using information technology to assist students’ learning in the out-of-school setting is not enough to
truly promote student learning autonomy and diversification.
To this end, integrating ICT with curriculum and teaching is needed to enhance Chinese students’ capabilities.
First, the quality of IT-integrated courses should be improved. Make subject knowledge relevant to the real world and
allow students to explore ICT-based real-life problems. Second, it is essential to enhance ICT use outside of school.
For example, as Hu and Gong (2018) observed, assigning relevant homework based on real-life situations and tasks
and interacting through feedback on results allow students to gain self-esteem. Finally, creating contexts further
enhances students’ interest in ICT. Students’ interest in ICT significantly predicts academic achievement. However,
students’ interest in ICT should focus on academically relevant content rather than leisure and entertainment.
This study analyzed and explored how individual, family, and school-related non-cognitive factors influence
secondary school students’ personal abilities in 4 Chinese provinces and cities, based on the OECD publicly available
PISA 2018 large education data set. It also compares and analyzes the non-cognitive factors affecting Chinese and
American students in reading, mathematics, and science, filling the research gap on the influence of non-cognitive
factors on different subjects and systematically and comprehensively exploring the importance of non-cognitive
factors. The study examined a broader range of non-cognitive factors that may help describe the experiences and
attributes of students assessed as having higher reading, mathematics, and science proficiency.
588
(pp. 575-593)
The study found that three subjects were closely associated with one another, and personal ability depended
much on their self-awareness and family income. Regarding subject characteristics, attitude towards failure was
the determinant of reading and mathematics. Mathematics and science were improved by student cooperation
and competition. The comparison of the two countries revealed the significance of rivalry in contemporary culture.
Additionally, family status had little effect on personal ability in the United States. The findings have implications
for students, families, and schools regarding the most important factors for students’ personal ability and future
studies regarding the effect of non-cognitive factors.
Acknowledgments
This study is supported by the National Natural Science Foundation of China (Reference NO.71901195), Hang-
zhou Philosophy and Social Science Planning Project (Reference NO.M22JC104).
Declaration of Interest
The authors declare no competing interest.
References
Aditomo, A., & Köhler, C. (2020). Do student ratings provide reliable and valid information about teaching quality at the school
level? Evaluating measures of science teaching in PISA 2015. Educational Assessment, Evaluation and Accountability, 32(3),
275-310. https://doi.org/10.1007/s11092-020-09328-6
Bailey, M. F., & Borooah, V. K. (2010). What enhances mathematical ability? A cross-country analysis based on test scores of
15-year-olds. Applied Economics, 42(29), 3723-3733. https://doi.org/10.1080/00036840802314598
Bellova, R., Melichercikova, D., & Tomcik, P. (2018). Possible reasons for low scientific literacy of Slovak students in some natural science
subjects. Research in Science and Technological Education, 36(2), 226-242. https://doi.org/10.1080/02635143.2017.1367656
Breiman, L., Friedman, J., Olshen, R., & Stone, C. (2015). Classification and regression trees. Belmont, ca: Wadsworth international
group. Encyclopedia of Ecology, 57(1), 582-588. https://doi.org/10.1007/978-3-642-57292-0_10
Bouffard, T., Boisvert, J., Vezeau, C., & Larouche, C. (1995). The impact of goal orientation on self-regulation and performance among
college students. British Journal of Educational Psychology, 65(3), 317-329. https://doi.org/10.1111/j.2044-8279.1995.tb01152.x
Chen, J., Zhang, Y., Wei, Y., & Hu, J. (2021). Discrimination of the contextual features of top performers in scientific literacy using
a machine learning approach. Research in Science Education, 51, 129-158. https://doi.org/10.1007/s11165-019-9835-y
Conger, R. D., & Donnellan, M. B. (2007). An interactionist perspective on the socioeconomic context of human development.
Social Science Electronic Publishing, 58(1), 175-199. https://doi.org/10.1146/annurev.psych.58.110405.085551
Conroy, D. E. (2001). Progress in the development of a multidimensional measure of fear of failure: The performance failure
appraisal inventory (PFAI). Anxiety Stress and Coping, 14(4), 431-452. https://doi.org/10.1080/10615800108248365
Conroy, David, E. (2004). The unique psychological meanings of multidimensional fears of failing. Journal of Sport and Exercise
Psychology, 26(3), 484-491. https://doi.org/10.1123/jsep.26.3.484
Davier, M. V., Yamamoto, K., Shin, H. J., Chen, H., & Kandathil, M. (2019). Evaluating item response theory linking and
model fit for data from PISA 2000-2012. Assessment in Education Principles Policy and Practice, 26(4), 466-488.
https://doi.org/10.1080/0969594X.2019.1586642
Duckworth, A. L., & Yeager, D. S. (2015). Measurement matters: Assessing personal qualities other than cognitive ability for
educational purposes. Educational Researcher, 44 (4), 237-251. https://doi.org/10.3102/0013189X15584327
Eriksson, K., Lindvall, J., Helenius, O., & Ryve, A. (2021). Socioeconomic status as a multidimensional predictor of student
achievement in 77 societies. Frontiers in Education, 6. https://doi.org/10.3389/feduc.2021.731634
Flores-Mendoza, C., Ardila, R., Gallegos, M., & Reategui-Colareta, N. (2021). General intelligence and socioeconomic status as
strong predictors of student performance in Latin American schools: Evidence from PISA items. Frontiers in Education, 6.
https://doi.org/10.3389/feduc.2021.632289
Gimenez, G., Ciobanu, D., & Barrado, B. (2021). A proposal of spatial measurement of peer effect through socioeconomic indices
and unsatisfied basic needs. Economies, 9, 72. https://doi.org/10.3390/economies9020072
Gonzalez, R., & Padilla, A., M. (1997). The academic resilience of Mexican American high school students. Hispanic Journal of
Behavioral Sciences, 19(3), 1-317. https://doi.org/10.1177/07399863970193004
Hagborg, W., J. (1994). An exploration of school membership among middle- and high-school students. Journal of Psychoeducational
Assessment, 12(4), 312-323. https://doi.org/10.1177/073428299401200401
Han, Y., Xu, Y. (2020). The development of student feedback literacy: The influences of teacher feedback on peer feedback.
Assessment and Evaluation in Higher Education, 45(5), 680-696. https://doi.org/10.1080/02602938.2019.1689545
Heckman, J. J., Stixrud, J., & Urzua, S. (2006). The effects of cognitive and non-cognitive abilities on labor market outcomes and
social behavior. NBER Working Papers, 24(3), 411-482. https://doi.org/10.1086/504455
589
(pp. 575-593)
Hobri, H., Murtikusuma, R. P., Fatahillah, A., Susanto, & Rini, S. M. (2018). The analysis on critical thinking ability in solving PISA
question, and its scaffolding. Advanced Science Letters, 24(11), 8215-8218. https://doi.org/10.1166/asl.2018.12526
Hox, J. J. (2010). Multilevel analysis: Techniques and applications (2nd ed.). New York: Routledge.
Hu, X., Gong, Y., Lai, C., & Leung, F. K. (2018). The relationship between ICT and student literacy in mathematics, reading, and science
across 44 countries: A multilevel analysis. Computers and Education, 125, 1-13. https://doi.org/10.1016/j.compedu.2018.05.021
Huang, X. H., Hua, L., Zhou, X. Y., Zhang, H., Zhang, M., Wang, S., Qin, S. R., Chen, J., & Wang, X. H. (2021). The association between
home environment and quality of life in children and adolescents in Hangzhou city, China. Journal of Child and Family
Studies, 30(6), 1416-1427. https://doi.org/10.1007/s10826-021-01951-1
Juan, A., & Visser, M. (2017). Home and school environmental determinants of science achievement of South African students.
South African Journal of Education, 37(1), 1-10. https://doi.org/10.15700/saje.v37n1a1292
Kriegbaum, K., & Spinath, B. (2016). Explaining social disparities in mathematical achievement: The role of motivation. European
Journal of Personality, 30(1), 45-63. https://doi.org/10.1002/per.2042
Krieken, K. V., Sanders, J., & Hoeken, H. (2015). Viewpoint representation in journalistic crime narratives: An analysis of grammatical
roles and referential expressions. Journal of Pragmatics, 220-230. https://doi.org/10.1016/j.pragma.2014.07.012
Laftman, S. B., Ostberg, V., & Modin, B. (2017). School climate and exposure to bullying: A multilevel study. School Effectiveness
and School Improvement, 28(1), 153-164. https://doi.org/10.1080/09243453.2016.1253591
Lau, K. C., & Lam, T. Y. P. (2017). Instructional practices and science performance of 10 top-performing regions in PISA 2015.
International Journal of Science Education, 39(15), 2128-2149. https://doi.org/10.1080/09500693.2017.1387947
Lee, J., & Shute, V. J. (2010). Personal and social-contextual factors in K-12 academic performance: An integrative perspective on
student learning. Educational Psychologist, 45(3), 185-202. https://doi.org/10.1080/00461520.2010.493471
Li, L., Peng, Z., Lu, L., Liao, H., & Li, H. (2020). Peer relationships, self-efficacy, academic motivation, and mathematics
achievement in Zhuang adolescents: A moderated mediation model. Children and Youth Services Review, 105358.
https://doi.org/10.1016/j.childyouth.2020.105358
Liu, R., Sun, L., Jing, Y., & Bradley, K. (2017). Using the 2006 PISA questionnaire to evaluate the measure of educational
resources: A Rasch measurement approach. International Journal of Assessment Tools in Education, 4(2), 211-211.
https://doi.org/10.21449/ijate.319486
Longo, F., Lombardi, M. P., & Dearing, E. (2017). Family investments in low-income children’s achievement and socioemotional
functioning. Developmental Psychology, 53(12), 2273-2289. https://doi.org/10.1037/dev0000366
Mar tin, A. J., & Marsh, H. W. (2011). Fear of failure: Friend or foe? Australian Psychologist, 38(1), 31-38.
https://doi.org/10.1080/00050060310001706997
Martin, A. J., & Dowson, M. (2009). Interpersonal relationships, motivation, engagement, and achievement: Yields for theory, current
issues, and educational practice. Review of Educational Research, 79, 327-365. https://doi.org/10.3102/0034654308325583
Martin, A. (2002). Motivation and academic resilience: Developing a model for student enhancement. Australian Journal of
Education, 46(1), 34-49. https://doi.org/10.1177/000494410204600104
Masarik, A. S., & Conger, R. D. (2017). Stress and child development: A review of the family stress model. Current Opinion in
Psychology, 13, 85-90. https://doi.org/10.1016/j.copsyc.2016.05.008
McGregor, & H., A. (2005). The shame of failure: Examining the link between fear of failure and shame. Pers Soc Psychol Bull, 31(2),
218-231. https://doi.org/10.1177/0146167204271420
Muelle, L. (2020). Socioeconomic and contextual factors associated with low academic performance of Peruvian students in PISA
2015. Apuntes-Revista De Ciencias Sociales, 47(86), 111-146. https://doi.org/10.21678/apuntes.86.943
Ngorosho, D. (2010). Reading and writing ability in relation to home environment: A study in primary education in rural Tanzania.
Child Indicators Research, 4(3), 369-388. https://doi.org//10.1007/s12187-010-9089-8
OECD. PISA 2018 DataBase. Available online: https://www. Oecd. Org/PISA/data/2018database/ (accessed on 30 June 2021)
Roseth, C. J., Johnson, D. W., & Johnson, R. T. (2008). Promoting early adolescents’ achievement and peer relationships. Psychological
Bulletin, 134(2), 223-246. https://doi.org/10.1037/0033-2909.134.2.223
Saeki, E., & Quirk, M. (2015). Getting students engaged might not be enough: The importance of psychological needs satisfaction
on social-emotional and behavioral functioning among early adolescents. Social Psychology of Education, 18(2), 355-371.
https://doi.org/10.1007/s11218-014-9283-5
Salchegger, S. (2016). Selective school systems and academic self-concept: How explicit and implicit school-level
tracking relate to the big-fish-little-pond effect across cultures. Journal of Educational Psychology, 108(3), 405-423.
https://doi.org/10.1037/edu0000063
Sherman, L. W. (2010). A comparative study of cooperative and competitive achievement in two secondary biology classrooms:
The group investigation model versus an individually competitive goal structure. Journal of Research in Science Teaching,
26(1), 55-64. https://doi.org/10.1002/tea.3660260106
Strobl, C., Malley, J., & Tutz, G. (2009). An introduction to recursive partitioning: Rationale, application, and characteristics
of classification and regression trees, bagging, and random forests. Psychological Methods, 14(4), 323-348.
https://doi.org/10.1037/a0016973
Sun-Lin, H. Z., & Chiou, G. F. (2017). Effects of comparison and game-challenge on sixth graders’ algebra variable learning
achievement, learning attitude, and meta-cognitive awareness. Eurasia Journal of Mathematics Science and Technology
Education, 13(6), 2627-644. https://doi.org/10.12973/eurasia.2017.01244a
Tauer, J. M., & Harackiewicz, J. M. (2004). The effects of cooperation and competition on intrinsic motivation and performance.
Journal of Personality and Social Psychology, 86(6), 849-861. https://doi.org/10.1037/0022-3514.86.6.849
590
(pp. 575-593)
Tseng, J. C. R., Chu, H. C., Hwang, G. H., & Tsai, C. C. (2008). Development of an adaptive learning system with two sources of
personalization information. Computers and Education, 51 (2), 776-786. https://doi.org/10.1016/j.compedu.2007.08.002
Tonga, F. E., Eryiğit, S., Yalçın, F. A., & Erden, F. T. (2019). Professional development of teachers in PISA achiever
countries: Finland, Estonia, Japan, Singapore and China. Professional Development in Education, 48(1), 88-104.
https://doi.org/10.1080/19415257.2019.1689521
Toste, J. R., Didion, L., Peng, P., Filderman, M. J., & Mcclelland, A. M. (2020). A meta-analytic review of the relations
between motivation and reading achievement for K-12 students. Review of Educational Research, 90(3), 420-456.
https://doi.org/10.3102/0034654320919352
United States Department of Education. (2014). 10 facts about K-12 education funding. Us Department of Education.
http://www.ed.gov/about/overview/fed/10facts/index.html
White, S. B., Reynolds, P. D., Thomas, M. M., & Gitzlaff, N. J. (1993). Socioeconomic status and achievement revisited. Urban Education,
28(3), 328-343. https://doi.org/10.1177/0042085993028003007
Woods-McConney, A., Oliver, M. C., McConney, A., Maor, D., & Schibeci, R. (2013). Science engagement and literacy: A retrospective
analysis for indigenous and non-indigenous students in Aotearoa New Zealand and Australia. Research in Science Education,
43(1), 233-252. https://doi.org/10.1007/s11165-011-9265-y
Woods-McConney, A., Oliver, M. C., McConney, A., Schibeci, R., & Maor, D. (2014). Science engagement and literacy: A
retrospective analysis for students in Canada and Australia. International Journal of Science Education, 36(10), 1588-1608.
https://doi.org/10.1080/09500693.2013.871658
Xiao, Y., & Hu, J. (2019). Assessment of optimal pedagogical factors for Canadian ESL learners’ reading literacy through artificial
intelligence algorithms. International Journal of English Linguistics, 9(4), 1-14. https://doi.org/10.5539/IJEL.V9N4P1
Yu, L., & Reichle, E. D. (2017). Chinese versus English: Insights on cognition during reading. Trends in Cognitive Sciences, 21(10),
721-724. https://doi.org/10.1016/j.tics.2017.06.004
Zhao, Y., Hoge, J. D., Choi, J., & Lee, S. Y. (2006). Comparison of social studies education in the United States, China, and South
Korea. International Journal of Social Education, 21(2), 91-122. http://www.bsu.edu/classes/cantu/journal.htm
Appendix
Table 6
Features with Questions Included and Value Scale
Feature Question Value scale
METASPAM
1. I write a summary. Then I check that each paragraph is covered in the summary, because the
content of each paragraph should be included.
2. I try to copy out accurately as many sentences as possible. 1=Not useful at all
METASUM 3. Before writing the summary, I read the text as many times as possible. 6=Very useful
4. I carefully check whether the most important facts in the text are represented in the summary.
5. I read through the text, underlining the most important sentences. Then I write them in my own
words as a summary.
1. I concentrate on the parts of the text that are easy to understand.
2. I quickly read through the text twice.
3. After reading the text, I discuss its content with others. 1=Not useful at all
UNDREM
4. I underline important parts of the text. 6=Very useful
5. I summarize the text in my own words.
6. I read the text aloud to another person.
1. I read only if I have to.
1=Strongly Disagree
2. Reading is one of my favorite hobbies.
2=Disagree
JOYREAD 3. I like talking about books with other people.
3=Agree
4. For me, reading is a waste of time.
4=Strongly Agree
5. I read only to get information that I need.
1=Strongly disagree
1. There were many words I could not understand.
2=Disagree
PISADIFF 2. Many texts were too difficult for me.
3=Agree
3. I was lost when I had to navigate between different pages.
4=Strongly Agree
ST059Q03TA How many class periods per week are you typically required to attend for the following subjects?
ST060Q01NA In a normal, full week at school, how many class periods are you required to attend in total?
591
(pp. 575-593)
1. I feel like an outsider (or left out of things) at school.

2. I make friends easily at school. 1=Strongly disagree
3. I feel like I belong at school. 2=Disagree
BELONG
4. I feel awkward and out of place in my school. 3=Agree
5. Other students seem to like me. 4=Strongly Agree
6. I feel lonely at school.
1. Browsing the Internet for schoolwork.
2. Browsing the Internet to follow up lessons.
3. Using email for communication with other students about schoolwork. 1=Never or hardly ever
4. Using email for communication with teachers and submission of homework or other schoolwork. 2=Once or twice a month
AUTICT 5. Using social networks for communication with other students about schoolwork or teachers. 3=Once or twice a week
6. Downloading, uploading, or browsing material from my school’s website (e.g., timetable or course 4=Almost every day
materials). 5=Every day
7. Checking the school’s website for announcements.
8. Doing homework/using learning apps/learning websites on a computer or a mobile device.
1. I forget about time when I’m using digital devices.
2. The Internet is a great resource for obtaining information. 1=Strongly disagree
3. It is very useful to have social networks on the Internet. 2=Disagree
COMPICT
4. I am really excited discovering new digital devices or applications. 3=Agree
5. I really feel bad if no Internet connection is possible. 4=Strongly Agree
6. I like using digital devices.
1=Strongly disagree
1. I enjoy working in situations involving competition with others.
2=Disagree
COMPETE 2. It is important to perform better than others on a task.
3=Agree
3. I try harder when I’m in competition with other people.
4=Strongly Agree
Table 7
Features Reflecting Subject Characteristics with Questions Included and Value Scale
1. When I am failing, I worry about what others think of me. 1=Strongly disagree
2. When I am failing, I am afraid that I might not have enough talent. 2=Disagree
GFOFAIL
3. When I am failing, this makes me doubt my plans for the future. 3=Agree
4=Strongly Agree
1. My life has clear meaning or purpose. 1=Strongly disagree
2. I have discovered a satisfactory meaning in life. 2=Disagree
EUDMO
3. I have a clear sense of what gives meaning to my life. 3=Agree
4=Strongly Agree
1. The teacher gives me feedback on my strengths in this subject. 1=Never or almost never
2. The teacher tells me in which areas I can still improve. 2=Some lessons
PERFEED 3. The teacher tells me how I can improve my performance. 3=Many lessons
4=Every lesson or almost
every lesson
1. A desk to study at 1=Yes
2. A room of your own 2=No
3. A quiet place to study
4. A computer you can use for schoolwork
5. Educational software
6. A link to the Internet
HEDRES 7. Classic literature (e.g., Shakespeare)
8. Books of poetry
9. Works of art (e.g., paintings)
10. Books to help with your schoolwork
11. Technical reference books
12. A dictionary
13. Books on art, music, or design
592
(pp. 575-593)
1. Students seem to value cooperation. 1=Not at all true

2. It seems that students are cooperating with each other. 2=Slightly true
PERCOOP
3. Students seem to share the feeling that cooperating with each other is important. 3=Very true
4. Students feel that they are encouraged to cooperate with others. 4=Extremely true
1. Students seem to value competition. 1=Not at all true
2. It seems that students are competing with each other. 2=Slightly true
PERCOMP
3. Students seem to share the feeling that competing with each other is important. 3=Very true
4. Students feel that they are being compared with others. 4=Extremely true
1. I am a good reader. 1=Strongly disagree
2. I am able to understand difficult texts. 2=Disagree
3. I read fluently. 3=Agree
SCREADCOMP
4. I have always had difficulty with reading. 4=Strongly Agree
5. I have to read a text several times before completely understanding it.
6. I find it difficult to answer questions about a text.
How many class periods per week are you typically required to attend for the following subjects?
ST059Q04HA
ST185Q01HA My life has clear meaning or purpose.

1. Televisions
2. Cars
3. Rooms with a bath or shower 1=None
4. Cell phones with Internet access (e.g., smartphones) 2=One
HOMEPOS
5. Computers (e.g., desktop computer, portable laptop, or notebook) 3=Two
6. Tablet computers (e.g., iPad, BlackBerry, PlayBookTM) 4=Three or more
7. E-book readers (e.g., KindleTM, Kobo, Bookeen)
8. Musical instruments (e.g., guitar, piano)
1. Test language lessons 1=No time
2. Mathematics 2=1-30 minutes a week
3. Science 3=31-60 minutes a week
4. Foreign language 4=More than 60 minutes
ICTOUTSIDE 5. Social sciences a week
6. Music 5=I do not study this
7. Sports subject
8. Performing arts
9. Visual arts
Cite as: Ye, L., & Yuan, Y. (2022). Using a machine learning approach to explore non-cognitive factors affecting reading,
mathematics, and science literacy in China and the United States. Journal of Baltic Science Education, 21(4), 575-593.
Lu Ye Research Center for Ecological Civilization, College of Economics and

Management, Zhejiang Sci-Tech University, Hangzhou, 310018, China.
E-mail: zjwzajyl@hotmail.com
Yuqing Yuan Research Center for Ecological Civilization, College of Economics and
(Corresponding author) Management, Zhejiang Sci-Tech University, Hangzhou, 310018, China.
E-mail: eureka346@hotmail.com
593
THE TYPES OF PEER

SCAFFOLDING IN SCIENTIFIC
EXPERIMENTAL ACTIVITIES
Abstract. This study studied the types

of peer scaffolding presented in scientific Seong-un Kim
experimental activities. The study included
14 university students. For the experimental
activity of ‘determining temperature
changes using the meridian altitude of the
Introduction
sun,’ information regarding experimental
behaviors, thinking aloud, discourse, and
‘Scientific experimental activities’ are considered essential in science
retrospective interview data were collected
classes. In the classes, students could have the opportunities to reproduce
and analyzed. A model was derived and
‘real natural phenomena’ through scientific experimental activities. Through
utilized for experimental activities to
the activities, students would have the chance to observe the phenomena
analyze peer scaffolding, wherein students
and connect the natural world with internal representations through obser-
internally structured their experiences with
vation of phenomena. Scientific experimental activity is a unique teaching
the experimental activities. The results
and learning method used in science subjects (Hofstein & Lunetta, 2004).
indicated that students utilized seven
This teaching and learning strategy is broadly used worldwide for teaching
means of peer scaffolding: ‘demonstrating’,
science in primary, lower-secondary, and upper-secondary school (Kipnis &
‘assisting’, ‘monitoring’, ‘posing’,
Hofstein, 2007). Through scientific experimental activities, students can de-
‘questioning’, ‘explaining’, and ‘suggesting’.
velop abilities considered important in science education (Hofstein & Lunetta,
Three types of peer scaffolding emerged:
2004). In particular, because students can achieve certain learning goals in
task completion-, model elaboration-, and
these scientific experimental activities, contexts where learners are exposed
learning support-oriented scaffolding. Each
to the chance for the actual experience, experience in scientific experimental
type differed in purpose, main mean, and
activities is paramount for learners (Girault et al., 2012; Tiberghien et al., 2001).
major explanation details. Additionally, this
The learning environment in an experimental activity is composed of fac-
study has observed the level of the model
tors such as the student, teacher, experiment manual, and materials. Among
for the experimental activities and the time
these factors, the interaction between students through conversations and
that had been provided to understand
behaviors is frequent and critical for learning (Roth, 2006). Scholars have
the experimental procedure influenced
argued that collaborative experimental activities in small groups are enough
the three peer scaffolding types. These
to afford more chances of interactive communication (Andersson & Enghag,
outcomes indicate that providing ample
2017), more effective than those performed in individual groups in various
time to students independently structure
aspects (Bilgin, 2006; Hofstein et al., 2005; Howe et al., 2007). Furthermore,
the experimental procedure and supporting
these activities offer more opportunities to participate in important cognitive
other students during experimental
activities such as planning for experimental activities and monitoring (Lin et
activities are essential. Moreover, providing
al., 2001; Shi, 2013). Nevertheless, studies reviewing the interaction among
assistance to students in focusing on
students during experimental activities have been relatively few (Raviv et
observable phenomena by reducing the
al., 2019; Wei et al., 2019).
cognitive load required to process the
Students actively use their own previous experiences in experimental
experimental procedure is crucial.
activities (Högström et al., 2010). In the class environment, spontaneous
Keywords: peer scaffolding, scientific
interactions manifest based on the students’ diverse knowledge and prior
experimental activities, students’
experiences. Supplemental assistance in the form of assisting students in
interaction, qualitative case study
participating in learning tasks during interaction with peers is referred to
as peer scaffolding (Kim & Hannafin, 2011). Assistance from a peer with a
similar level of knowledge is as important as that from a teacher. Through
Seong-un Kim peer scaffolding, phenomenon observation, inferencing on the observed
Korea National University of Education, phenomenon, clarifying explanation of a phenomenon, and stimulating
South Korea
thoughts that operate to connect the just-heard-explanation with the con-
cepts that they already know could be facilitated. So, students need to listen
594
ISSN 1648–3898 /Print/ THE TYPES OF PEER SCAFFOLDING IN SCIENTIFIC EXPERIMENTAL ACTIVITIES
(pp. 594-614)
to queries from others (Driver, 2012). Peer scaffolding promotes knowledge restructuring by enabling students to
recognize differences in knowledge, beliefs, and experiences between themselves and peers, as well as to experi-
ence cognitive conflicts (Palincsar et al., 1987). Moreover, students encounter diverse experiences by perceiving
differences from their peers and endeavoring to bridge these gaps (Choi et al., 2005).
However, research on peer scaffolding regarding scientific activities is relatively inadequate. Kim and Han-
nafin (2011) argued that peer scaffolding effectively facilitates scientific problem-solving by promoting a reflective
discourse between students. In addition, Shin et al. (2020) identified nine types of peer scaffolding in inquiry-based
learning. They also found that the interaction patterns depended on the level of knowledge of each group. However,
other studies on peer scaffolding have mainly focused on peer scaffolding appearing in online contexts. Research
on peer scaffolding during actual face-to-face interactions has not been spotlighted. Furthermore, there seem to
have been no studies focusing on peer scaffolding in experimental activities yet. Therefore, the characteristics and
usefulness of peer scaffolding that students demonstrate during experimental activities still remain unknown.
As such, this study has analyzed peer scaffolding shown in scientific experimental activities. In this manner, this
study provides implications regarding measures that should be implemented to facilitate scientific experimental
activities effectively. Specifically, this study presents an empirical answer to the following question: What kinds of
peer scaffolding appear in scientific experimental activities in the Korean science classroom?
Theoretical Background
Student Behaviors in Scientific Experimental Activities
Scientific experimental activities in primary, lower-secondary, and upper-secondary schools in Korea and
other countries worldwide primarily comprise a verification experiment and a discovery experiment (Domin, 2007;
Tiberghien et al., 2001; Yang et al., 2007). These types of scientific activities require an experimental manual that
describes the experimental procedure. Various kinds of empirical evidence have indicated that students focus
more on following the experimental procedure rather than learning the intended contents (Abrahams & Miller,
2008; Högström et al., 2010; Osborne, 1993; Tamir & Lunetta, 1981). Students spend most of the time on the ex-
perimental procedure’s manipulative aspects using a low skill level. Students are somewhat seemingly interested
in completing laboratory tasks instead of the concepts and functions that the teacher aims to teach the students
through scientific experimental activities (Berry et al., 1999).
The cause of such scientific experimental behavior of students is due to the difficulty of processing the
experimental procedure. A new experiment is filled with a plethora of information, including difficult terms,
materials, and scientific concepts (Gunstone & Champagne, 1990). Students should understand and manage the
procedure, which is constructed with new information (Lehman, 1990). Moreover, because of the newly exposed
and stimulating phenomena that students experience in experimental activities, they neglect the aspects they
should observe (Johnstone & Al-Shuaili, 2001). Students are also required to invest various cognitive resources to
connect the experimental activity results to abstract concepts (Hodson, 1993). Johnstone (1997) explained that the
students’ information processing during a scientific experimental activity is perceived and filtered by the bound-
ary the students know and afford to understand. Accordingly, the students would not recognize the information
outside the perceived boundary. Therefore, it cannot convey its entirety. In addition to this, Johnstone pointed
out that due to the excessive information, students may experience a cognitive load. Therefore, students cannot
express the current activity and the reason for conducting it even after its completion (Gunstone, 1991; Hodson,
1993; Hofstein & Lunetta, 2004).
Many scholars on science education possess a skeptical point of view regarding the effectiveness of scientific
experimental activities (Abrahams & Miller, 2008; Hodson, 1993; Hofstein & Lunetta, 2004; Tiberghien et al., 2001).
Not only trying to students experience various abilities of science subject intended to teach, but also trying to
teach them by providing direct experience on scientists’ activities was challenging. This is because students do not
possess the abundant experience or theoretical elaboration of scientists (Kirschner, 1992).
Student–Student Interaction in Scientific Experimental Activities
Scientific experimental activities offer opportunities for various types of student-student interactions
(Hofstein & Lunetta, 2004). In the activities, cooperative operation activities, communication of discussion and
595
(pp. 594-614)
demonstration, and various forms of problem-solving require collaborative work. In this process, students experi-
ence such as the elaboration of scientific explanation, exchange and expansion of ideas, negotiation, observa-
tion and imitation of diverse functions, use of scientific language, and other variegated experiences (Hofstein
et al., 2005; Howe et al., 2007; Olubu 2015). Such experiences of interaction have advantages in various aspects.
For example, they positively influence not only both cognitive and affective aspects but also economic aspects
(Raviv et al., 2019) of improving achievement (Keys, 1996; Lazarowitz & Karsenty, 1990), improving inquiry abil-
ity (Hofstein et al., 2005; Lazarowitz & Karsenty, 1990), understanding the collaborative character of generating
scientific knowledge (Lunetta et al., 2007), and forming a sound learning atmosphere (DeCarlo & Rubba, 1994).
Meanwhile, previous studies have reported that the activities may be less effective unless teachers proactively
intervene in the students’ interaction (Alton-Lee et al., 1993). Others also argued that group activities are more
time-consuming and require more teacher effort than individual activities (Clyde, 1998; Raviv et al., 2019).
Precedent Research on Peer Scaffolding in Scientific Learning Situations
Research on peer scaffolding concerning scientific teaching and learning has mainly focused on online learn-
ing environments. However, in studies on peer scaffolding during face-to-face classes, Kim and Hannafin (2011)
and Shin et al. (2020) analyzed the araising patterns of peer scaffolding through web-based exploratory learning.
Kim and Hannafin (2011) analyzed the scientific problem-solving process presented by students in technol-
ogy-enhanced science classrooms by observing two classes in the 6th grade. They researched the effects of peer,
teacher, and technology-enhanced scaffolds on the inquiry activity of students. As a result of the study, the types
of peers, teachers, and technology-enhanced scaffolds were derived. It was revealed that a distinct inquiry pattern
occurs in students’ problem-solving processes. They also illustrated that disparate types of scaffolds are integrated
to facilitate the inquiry activities of students.
In Shin et al. (2020)’ study, they researched inquiry-based, web-based learning materials of six classes in the
9th grade. They also observed the peer scaffolding that appeared in the classes. Based on the findings, they distin-
guished the types and patterns of peer scaffolding. In addition, according to each group’s knowledge level that was
classified as high, blended, and low levels based on already measured prior knowledge of the students; they could
confirm the newly revealed patterns of the scaffolding of the students. Peer scaffolding requires knowledge about
each area and metacognitive skills, and the quality of peer scaffolding influences the students’ prior knowledge level.
Still, peer scaffolding revealed in face-to-face scientific classroom environments remains only partially un-
derstood. These findings indicate that peer scaffolding research is useful in terms of showing various aspects of
students’ learning of science. The results also reveal the need for effective peer scaffolding to increase the effec-
tiveness of learning science.
In this study, peer scaffolding in scientific experimental activities is studied. Scientific experimental activities
play a central role in learning school science, and this has been proved through various research concerning the
activity. However, peer scaffolding occurring within the activities has not been deeply studied and has only par-
tially been understood. This study spotlights the peer scaffolding occurrence in scientific experimental activities.
By doing so, this study raises the importance of understanding peer scaffolding appearing in scientific activities
that use real objects included in the scientific experiment activities.
Background
This study used the method of qualitative case studies to analyze how peer scaffolding appears in each case
where students’ cooperative experimental activities are conducted. This study collected students’ experimental
activity experiences in various ways, such as observing experimental behavior, thinking aloud, and having a con-
versation during experimental activities. In addition, retrospective interviews, experimental behavior observation
records, and field notes were also created and collected. For the in-depth analysis of the collected data, a model of
the experimental activities was derived, and the means of peer scaffolding used by the students were identified.
Finally, each case’s types of peer scaffolding were classified and analyzed based on the means of peer scaffolding
revealed by students’ discourse, behavior, and the contents of peer scaffolding.
596
(pp. 594-614)
Participants
The research participants of this study were fourteen students (five female students and nine male students)
enrolled in a four-year national university in South Korea. The university is in the middle of South Korea and is one
of the most prominent teacher education universities. The recruitment targeted university students for seamless
communication and a smooth simultaneous verbalization of cognition during the experimental activity. In addition,
it targeted students undertaking other majors apart from the natural sciences to eliminate any professionality on
the task. Because building rapport is vital for the smooth interaction in the task performance of a team compris-
ing two members. In this study, two students already acquainted with each other were recruited as one team. The
reason for this group’s composition was to reduce the time to build rapport between them. Since they already
know each other, they could easily jump into the activity with comfortable interaction. A total of seven cases (six
same-gender cases and one mix-gender case) were created. Participants in this study were 18 to 20 years old, and
they all were in their first or second grades when the data was collected. The all-research participants were fully
informed of the anonymity of their participation. The safety of their participating task was also notified in advance.
Context
The experimental activity needs to reflect small-group activity environments in schools sufficiently. To this
end, the experiment of this study was designed by following the consultation of two experts in science education.
Two aspects were also discussed with the researcher of this study and the two experts. First, peer scaffolding is
possible under the context of natural social interaction. If the role of a tutor is assigned to a participant, then the
participant may try to meet the expectations for the assigned role (Roscoe & Chi, 2007), which would render the
experimental activity become unnatural. Hence, this study created a situation where two students performed the
experimental activity without being assigned special roles and freely cooperating.
Second, one of the two individuals should possess prior experience of the experimental activity for peer
scaffolding, as it presumes that one has more knowledge and experience in the small group (Wood et al., 1976).
Therefore, it was important to design the format of the group to be one has to experience the experimental activity
while the other one has no experience at all. In the first instance, researchers selected an experimental activity that
the two individuals in a team had never experienced before. Then, among them, only one was offered a chance to
perform the pre-task related to the selected experimental activity. This created the format that one person pos-
sesses more knowledge than the other.
Task
To answer the research question of this study, the researcher of this study decided on the following criteria in
the task selection: (1) an experiment that fully reflects the science curriculum in Korea; (2) an experiment in which
the students did not experience in their school years; (3) an experiment using various and unfamiliar materials
and tools; and (4) an experiment at the elementary and middle school levels that is simple enough so that can be
solved by students who do not study natural science. Based on the consideration of the four criteria, this study
selected the experimental activity entitled ‘determining temperature changes according to the meridian altitude
of the sun,’ which was derived from a sixth-grade science textbook published by Company I. This activity uses a
model experiment to investigate temperature differences according to the angle of the sun and the surface of
the earth. First, students produced two electric circuits, where a solar panel and a buzzer were connected with an
electric wire, then installed each lamp by differing the angle of the bulb and the solar panel. Subsequently, the
lamp was shone toward the solar panel to qualitatively observe and compare the volume of sound created by
the buzzer. This experiment prompts students to reflect on the seasonal temperature changes that happen as the
solar-radiation energy that reaches the surface of the earth varies according to solar altitude.
The experiment manual was created by reflecting the experimental procedure included in the textbook. To
aid the understanding of the students and to help them easily visualize the difference in the angles of the lamp and
the solar panel, that is, a variable that requires caution, a visual material was additionally inserted at the bottom
of the experiment manual (Appendix 1). Through a preliminary experiment conducted by one university student,
researchers confirmed that the experiment manual and the experiment materials were free of errors.
597
(pp. 594-614)
Data Collection
The experiment recruited two participants as one team, where one undertook the experimental activity in
advance. In total, the number of participants was 14 students, and they were paired into seven groups. Then, each
group started their given experimental activity together.
Figure 1
Data Collection Procedure
Figure 1 presents the data collection procedure. First, one of the two participants moved to a waiting room
separated from the laboratory, then stayed there. To analyze the experience of the participant who performed
the experimental activity alone, the process of verbalizing the cognition of the participants was required. Toward
this end, 10 to 15 min of ‘Thinking Aloud Training’ was conducted using a block assembly task. Afterward, the par-
ticipant independently conducted a preliminary experimental activity and thought aloud simultaneously. After
the experiment, an interview was conducted on aspects students found challenging and their impressions of the
activity. The instructor of the class paid attention to prevent the interview from being served as a clue about the
experimental activity.
In the experiment, the two team members were required to engage in active communication as they performed
the experimental activity. All experiments lasted for approximately 20 to 30 minutes, and the experimental activity
was recorded using a video camera. The researcher created field notes not only about the behavior but also the
discourse of the participants during the experiements. Subsequently, a one-on-one retrospective interview was
conducted with each participant. The interview was designed as semi-structured, and the recorded video was used
as the cues for retrospective thinking. Based on the field notes, the participants were asked to provide a detailed
explanation of each situation and respond to the given questions about their thought at a certain moment during
the research process with a detailed description. Several follow-up questions were also provided. The interview
lasted approximately 20 min per participant, and all interviews were recorded using a video camera. All data col-
lected in the two-session experiments and the interviews (thinking aloud, a conversation between the participants,
and retrospective interviews) were transcribed.
Data Analysis
The patterns of peer scaffolding during the experimental activity appear diversely according to the pairs and
the performance of the activity. By using a qualitative case study methodology, this study identified how and why
peer scaffolding appears in selected cases. This study also categorized the outcomes according to each type of
peer scaffolding. The collected data was analyzed, and the characteristic of the model students composed through
their participation in the experimental activity was identified. As the model about the experimental activities be-
comes the cognitive basis for students to be able to offer peer scaffolding for their colleagues, then, based on the
model, the means of peer scaffolding that the students were using were also distinguished. After that, the type of
means of peer scaffolding the students employed with his/her pair was analyzed according to each case. At the
last stage of the data analysis, the kinds of peer scaffolding the students used were diagnosed. Also, by deducing
the model for the experimental activity that becomes the basis of the scaffolding means, the characteristics of the
peer scaffolding types were described for each case (see Figure 2).
598
(pp. 594-614)
Figure 2
Data Analysis Framework
In peer scaffolding, a student with experience implicitly provides information to the student without prior
experience (Shin et al., 2020). Therefore, revealing the type of knowledge foundational to peer scaffolding is essen-
tial for the in-depth analysis of peer scaffolding in experimental activity. This study presumed that the participants
structure a certain type of model stored in their long-term memory through their experience in the experimental
activity. It then explored the type of information that comprises such a model of the experimental activity. To
understand the information factors in the model, this study focused on the differences in experimental behavior
and language before and after the experimental activity. The reason for doing so was that these differences were
caused by the variations in the derived models. The data analysis was conducted by the repeated process of writ-
ing down notes while reviewing the transcript, video, and observation records to capture the segments indicat-
ing differences in experimental behavior and language before and after the experimental activity. Through this
process, the information required of the participant and whether the participant possesses this information could
be determined. Subsequently, the types of identified information were named and categorized. Throughout the
study, the researchers repeatedly performed the process of writing the line-by-line coding, writing down notes,
and checking the collected data to theoretically saturate the information that composed the model. Appendix 2
represents the results.
The model for the experimental activity was comprised of 4 categories and 15 categories of information. It
was also task-specific. Rather than independently, the information categories were seemingly used in a complex
manner during an interaction with a peer. Meanwhile, the model structured by the participants did not comprise
accurate information. During the experimental activity, the model can be structured based on the experience of
the participants that differs from the initially intended by the experiment manual. Moreover, the model can be
organized according to one’s experience with failure. Finally, the model can be modified or elaborated through
the experience of additional experimental activity.
Among the 15 categories of information, the meaning of the procedure and how to perform the procedure
were perceived as easily obtained because they were explicitly revealed in the experiment manual. However, the
rest of the information could be acquired only through the experience of the actual experimental activity partici-
pation, and it could be accumulated by the interaction with the experiment manual and materials. Particularly,
meta information on procedure performance was difficult to attain without metacognitive processing during the
experimental activity. The perspective of the model for such experimental activity enabled the study to explain
the aspects through which students structure knowledge through their experience in the experimental activity
and apply it to peer scaffolding.
The next step of data analysis was to identify the means of peer scaffolding. Toward this end, the study con-
ducted the top-down and bottom-up processes of data analysis. The initial coding system was developed based
on the types of peer scaffolding suggested by Kim and Hannafin (2011) and Shin et al. (2020), who examined peer
scaffolding during scientific activities. The data collected in a multilateral manner were repeatedly examined to
599
(pp. 594-614)
modify the initial coding system, and the means of peer scaffolding were identified and confirmed through a review
conducted by experts outside of the research.
The last step of data analysis was distinguishing the types of peer scaffolding indicated in each case of the
seven teams. The types of peer scaffolding per case were identified based on the type of information conveyed and
the means and purposes of the respective cases. Furthermore, the researcher described the relationship between
each case with the experience in the previous experimental activity. Moreover, the results were used to establish
labels for the types of peer scaffolding of each case.
This study used several strategies to ensure the validity and reliability of the qualitative analysis. First, data
analysis related to experimental behavior, thinking aloud, conversation during the experimental activity, retrospec-
tive interview, and field notes were diversified, and comparative analysis was continually conducted. In addition,
the presumption and potential bias of the researcher were self-identified and self-recognized. The researcher of
this study had experience instructing students on scientific experimental activities for more than ten years as a
science teacher and has frequently participated in science education studies as a qualitative researcher. In addition,
throughout the process of analysis, the method was shared and discussed with an expert in science education with
more than 20 years of experience in qualitative research. The final analysis then categorized data were reviewed
through a discussion with one of the participants of this study.
Research Results
Identification of Peer Scaffolding Means during the Scientific Experimental Activity
Peer scaffolding can be diversely provided according to the means and objectives through which it is provided.
The results revealed seven means of peer scaffolding: ‘demonstrating’, ‘assisting’, ‘monitoring’, ‘posing’, ‘questioning’,
‘explaining’, and ‘suggesting’.
Demonstrating
The most frequently observed mean of peer scaffolding was ‘demonstrating,’ in which the procedural imple-
mentation was directly shown to the peer through actions. ‘Demonstrating’ emerged along with ‘explaining.’ As
the student personally illustrated the experimental procedure, s/he explained to the peer the current process and
the reason for following the procedure. Providing ‘demonstrating’ would enable the other peer to observe, imitate,
then understand the procedure. The below example was the part of the discourse of Team E on the implementation
of procedures 1 and 3. Students no.1 per every term has experience in conducting the experiment, and every no.
2 students per every team have no experience in the experimental activity.
E1: I will connect this here … (connects the solar panel and the buzzer with electric wire)
E2: Ah … (sees and imitates E1’s operational situation)
(omitted)
E1: A sound will be made when I turn on the lamp, like this. One, two, three! (personally, turns on the lamp)
E2: Oh, I can hear the sound.
E1: Now, I will check which sound is louder according to the angle difference.
<Procedures 1 and 3; Team E main experiment>
For instance, in the case of Team E, when making an electric circuit that connects the solar panel and the
buzzer with the electric wire, the one student (E1) allowed the other student(E2) to understand the procedural
implementation, then led E2 to conduct the rest through imitation by personally demonstrating the process. In
the meantime, many cases occurred in which the no. 1 student-led the procedure without allowing the peer(no.2)
to imitate. Accordingly, the objective of demonstrating mainly lies in making the peer understand the process and
not necessarily in imitation of the other.
600
(pp. 594-614)
Assisting
‘Assisting’ denoted providing aid in the procedural implementation of a peer.
F2: Now, I will place the solar altitude meter like this …. Then, let’s turn on the lamps at the same time.
Please turn on the lamp!
F1: Okay (turns on the lamp).
F2: This one makes a sound. Does that one make a sound, too?
F1: Yeah, this one makes the sound, as well.
<Procedure 3; Team F main experiment>
As demonstrated, the student(F1) assisted when the peer(F2) needed help or asked for help, such as executing
according to the instruction of the peer to turn the lamp on and off and plugging it for the peer because s/he is
closer to the outlet than the peer. ‘Assisting’ saved the time and efforts of the peer. It also enabled the peer to quickly
perform the experimental activity. Meanwhile, ‘assisting’ was in contrast with ‘demonstrating.’ Although ‘assisting’
provided supplementary support for the procedure that could be fully implemented by the peer, ‘demonstrating’
intended to make the peer understand by personally taking action to demonstrate the procedure that the peer
was unable to independently implement.
Monitoring
‘Monitoring’ refers to inspecting or evaluating the status by paying attention to the understanding or imple-
mentation of the procedure by the peer. In other words, it helps retain an adequate status to enable the peer to
reach the goal. This process typically features a metacognitive characteristic.
A2: I will do this… in this way (adjust the distance by placing the lamp far from the solar panel)
A1: Hmm … (points to the two lamps and the solar panel) I think there’s a bit of difference between here
and there …
A2: Ah — If so, how about this? (moving one lamp a bit further)
A1: 20 cm is closer than we think.
<Procedure 2; Team A main experiment >
For instance, the process included inspecting whether the distance between the two lamps and the solar panel
set by the peer was maintained consistently and verifying the difference in the angles of the two lamps and the
solar panel. Through the ‘monitoring,’ students observed the peer’s understanding of procedural understanding or
implementation of the experiment. Then one confirmed the fulfillment or non-fulfillment of each procedure of the
other colleague’s work by evaluating the results based on one’s model for the experimental activity. Such ‘monitoring’
signaled the appropriateness of relevant procedural handling and enabled one to proceed to the next step. Moreover,
‘monitoring’ included providing sufficient time for handling the procedure, such as waiting and watching the peer.
Posing
‘Posing’ facilitated as the mean one posed a question deliberately to the other peer. Introducing challenging
questions that required metacognitive processing and opposing viewpoints, this means enabled the peer to clarify
one’s thoughts and inspected the implementation.
B2: (Reading the experimental procedure) Next, attach each solar altitude meter to the lab table beside
the solar panel.
B1: How do you think you should attach them?
B2: Solar altitude meter on the lab table …. Wouldn’t it be okay to attach it … (attaching the solar altitude
meter on top of the solar panel) like this?
B1: If so, why would you need this (solar altitude meter)?
B2: Um …I am not sure…
<Procedure 3; Team B main experiment >
601
(pp. 594-614)
Similar to the aforementioned case, asking about the installation of the solar altitude meter during the ex-
perimental activity and the need for doing so or questioning how to vary the tilted degree of the two lamps were
information that cannot be gained from the experiment manual. They were the essential factors to the effective
operation of the experimental activity. Such ‘posing’ helped the accurate procedural processing of the peer and
offered a chance for in-depth thoughts by clarifying the focus of thinking. ‘Posing’ questions appeared as the means
of introducing new ideas or concepts.
Questioning
‘Questioning’ operated to asking curious concerns to the peer.
A1: Why do the manual instructions maintain the distance between the two solar panels as 30 cm?
<Procedure 1; Team A main experiment >
C1: Do you know how to use this (solar altitude meter)?

<Procedure 1; Team C main experiment >
As demonstrated by the narrative above, ‘questioning’ pertained to asking questions about aspects that
aroused curiosity during the experimental activity. ‘Questioning’ appeared in contrast with ‘posing.’ If ‘posing’ in-
tended to help the peer to conduct a more effective procedural processing or learning based on the model, then
‘questioning’ referred to asking curious concerns to structure one’s model for the experimental activity in a more
elaborate manner.
Explaining
‘Explaining’ intends to provide information to the peer by explaining or comparing an experience, idea, con-
cept, and situation in detail.
C1: No sound is made when you connect the solar panel and the buzzer by the same magnetic pole.
<Procedure 1; Team C main experiment >
D1: When you shine the light on the solar panel, the buzzer makes a sound; it makes a louder noise when
the intensity of light is stronger.
<Procedure 1; Team D main experiment >
As indicated in the narratives above, ‘explaining’ denoted offering information about the experience in terms
of the methods or results of the procedure. ‘Explaining’ was chiefly exhibited when the peer was asking a question
when one discovered a situation in which the peer required information during ‘monitoring,’ and when one was
‘demonstrating.’ In addition, ‘explaining’ provided information based on the model for the experimental activity
and explicitly conveyed it or used various formats of analogy, hints, and examples.
Suggesting
‘Suggesting’ denotes the relay of unverified ideas and hypotheses to the peer.
E1: It says here (experiment manual) to turn on the lamps at the same time. Don’t you think this means
that we have to turn on and off the lamps at the same time to control the variable?
E2: Oh, that’s right.
E1: If so, since the temperatures differed earlier, let’s take a short break and then try turning it on and off
at the same time again.
<Procedure 6; Team E main experiment >
The text suggested the conveyance of one’s opinion concerning problem-solving. The information provided
602
(pp. 594-614)
through ‘suggesting’ generated an opinion based on an inelaborate and incomplete model for the experimental
activity. Thus, it was less trusted than the information provided through ‘explaining.’
Types of Peer Scaffolding Exhibited by the Scientific Experimental Activity
Peer scaffolding in this study was significant in terms of their characteristics according to small groups. This
research classified the types of peer scaffolding based on the characteristics of each case and inferred the patterns
of peer scaffolding and the personal objectives demonstrated from each kind. By analyzing peer scaffolding per
case, the concept could be organized into three types: task completion- scaffolding, task model elaboration- scaf-
folding, and learning support-related scaffolding.
Task Completion-Oriented Scaffolding: Teams A, D, and G
The objective of ‘task completion-oriented scaffolding’ is to accurately complete a given task with a peer. The
following conversation demonstrated this type of scaffolding implemented during procedure 3.
G1: It says 20 cm, so let’s do it like this, and then what was the next instruction?
Then I need to attach each altitude meter to the lab table beside the solar panel.
G2: Is this the altitude meter?
G1: Yeah.
It is written here that it is a solar altimeter.
G2: Besides the solar panel …. It is this one (points to the solar altitude meter), right?
G1: Yeah.
G2: I need to attach each to the lab table …. Is this supposed to stick?
G1: Yeah, you can stick it with a suction plate underneath.
G2: Oh, it sticks well?
<Procedure 3; Team G main experiment>
G1 demonstrated the arrangement of the electric circuit on the lab table to G2 and explained the action,
whereas G2 performed the procedure after seeing and following G1’s action. In this manner, the main means of
peer scaffolding under these types are ‘demonstrating’ and ‘explaining.’ Students’ behaviors that mainly appeared
during the scientific experimental activities were significant in relation to the task completion, that was, processing
of the experimental procedure. Therefore, the details of the explanation were primarily manifested as information
about the procedure and materials. This was because the team focused on understanding and implementing the
experimental procedure.
Moreover, ‘monitoring’ was occasionally identified as the means of peer scaffolding because a student without
experience in the experimental activity needed additional time to understand the procedure.
A1: Should we turn it on at the count of three?

A2: Turn it on.
A1: One, two, three! (Turns on the lamp)
A2: Oh …. What happened? No, turn it off again.
Let’s try after reading this (look at the experiment manual).
<Procedure 3; Team A main experiment>
D1: First, let’s connect the solar panel and the buzzer with electric wire.
D2: And here, the solar panel and the buzzer must be connected by the same magnetic pole.
Oh! Is this the buzzer?
D1: I think I did it with two electric wires, one for each, earlier.
D2: Oh~ Okay. And then …
<Procedure 1; Team D main experiment>
603
(pp. 594-614)
In team A’s case, because A1 took the lead in implementing the experimental activity, A2 could not fully un-
derstand the experimental procedure. Therefore, A2 demanded that A1 pause the implementation and have the
time to read and understand the procedure. Thereafter, A1 conducted ‘monitoring’ by waiting and inspecting the
procedural understanding requested by A2. In the case of team D, D1 provided D2 with ample time even without
D2’s request and inspected procedural understanding by ‘monitoring’ to allow D2 to read and understand the
procedure. The introduced two cases indicate that the understanding of the peer about the procedure should
precede the performance of the experimental activity with the peer. For this objective, sufficient time for under-
standing the procedure should be provided. Without such an opportunity, the student without experience may
find it difficult to construct a model for the experimental activity. A detailed explanation does not always lead to
better learning outcomes. Rather than that, they pointed out that helping the learner independently understand
the learning content is more important than providing an explanation. Thus, allowing understanding the experi-
mental procedure of the scientific experimental activity to peers without experience with the activity would be
a crucial experience that could lead the peers to be able to construct the structures of the experimental activity
model, thereby promoting subsequent learning.
As the experimental activity progressed, the students’ models became gradually elaborated. Hence, the level
of the models for the experimental activity held by the two students in a pair slowly became similar. Accordingly, in
the latter part of the activity, the means of scaffolding changed into complex ones, such as ‘assisting,’‘questioning,’
and ‘suggesting.’ Simply put, they mutually shared the responsibility of the given task in terms of achieving the
purpose of accurately completing the task.
The following vignettes are parts from the thinking aloud material, conversation about the experiment, and the
post-interview material during the preliminary experiment, which pertains to A1’s implementation of procedure 5.
A1: Let’s compare the sound volume.

A1: If it is difficult to compare, then reduce the brightness of the lamp ….
A1: Let’s turn it on.
A1: I think this one (points to one buzzer) is a bit louder?
Let’s turn it off now and ….
<Procedure 5; A1 preliminary experiment>
A1: One, two, three!

A2: This one is louder.
A1: This one here is louder? I had opposite results earlier ….
A2: Really?
A1: This one does seem a bit louder ….
A2: But this one is louder?
A1: Then, should we go with this one being louder?
<Procedure 5; Team A main experiment>
A1: Since the result differed from the previous one, it seemed there was an error in the experiment. So, I
thought I had made a mistake. That’s why I tried to once again check if the solar panel had been moved
or tried to listen closer by moving around to see if the sound was being properly made.
<Interview with A1>
The results of the preliminary experiment and the main experiment of A1 differed from each other. For this
reason, A1 discussed with the peer the outcomes of the main experiment. A1 expressed a lack of confidence in
the outcomes through vague responses, such as “should we go with this one being louder?” After that, A1’s peer
scaffolding patterns changed from ‘demonstrating’ and ‘explaining’ to using complex means. In this sense, the
cause of the ‘task completion-oriented scaffolding’ type seems to appear due to the incomplete model used in the
experimental activity by the student with prior experience.
604
(pp. 594-614)
Model Elaboration-Oriented Scaffolding: Teams C and E
The ‘model elaboration-oriented scaffolding’ type aims to elaborate one’s model for the experimental activity.
This type of scaffolding does not focus on the learning of the peer but on accurately completing the task by resolv-
ing one’s curiosity about the experimental activity earlier and, therefore, elaborating on the incomplete model.
Team C’s conversation in implementing procedure 2 presents this aspect.
C1: These two (two solar panels) are supposed to have temperature differences.
C2: That’s right.
C1: So, we shouldn’t shed light only on one side.
C2: Ah — you’re right.
C1: To make the variables the same, we have to first ….
Should we try doing this (moving the lamp) for this one first? Like this ….
The distance between this one (solar panel) and this one (lamp) must be 20 cm.
C2: Oh, 20 cm.
C1: But while they need to be the same 20 cm, you make one angle big like this. Make it almost 90 degrees.
Make the angle small for the other one. But it still needs to be the same 20 cm!
C2: Does cm measure from the lamp to here (solar panel)?
C1: Yeah, from the lamp to here (solar panel).
<Procedure 2; Team C main experiment>
C1 explained to C2 the process by personally demonstrating the adjustment of the angle and the distance
between the lamp and the solar panel. C2 watched the way C1 showed. The main means of scaffolding used by
C1 were ‘demonstrating’ and ‘explaining.’ The details of the focus of the explanation were around the information
on the procedure and materials, which is similar to the early stage of the experiment in the group that belonged
under the ‘task completion-oriented scaffolding’ type. Specifically, ‘task completion-oriented scaffolding’ frequently
demonstrated ‘monitoring,’ in which the no. 1 students waited and inspected to help the other peer(no.2) under-
stand the procedure. Conversely, this type of scaffolding did not provide the time the peer required to understand
the procedure. As such, C1 lacked interest in the peer’s understanding of the procedure and was unaware that the
peer took the necessary time and effort to understand the procedure. Thus, C2 did not experience the experimental
activity firsthand but was left to structure the model by merely seeing and hearing the demonstration and explain-
ing that it was quickly conducted by C1. However, as C2 gradually participated passively and provided negative
responses about the experimental experience in the follow-up interview, it seems C2 found that structuring a
model was challenging to him/her.
C1: What should I do with this one (solar altitude meter)?

C1: 90 degrees? Um …. How could I use this?
C1: Um… I’m not sure how to use it, but…
C1: finding an accurate angle for this one doesn’t seem to matter much. So, first, the one on the left has
a small angle, and the one on the right has a big angle.
<Procedure 4; C1 preliminary experiment >
C1: What I thought while doing this earlier is that when you shine the light here, then a shadow on this
stick… may produce an angle.
C2: Ah … It seems okay… I think that’s right.
(omitted)
C1: Then, there’s the hypothesis I set up earlier. The solar altimeter. Should we try it roughly to see if that
was correct? If my idea was right?
(omitted)
C1: What I had thought earlier was right. The reason it didn’t work when I tried the last time must have
been because it was 90 degrees. If so, let’s turn it off again.
<Procedure 1 to 4; Team C main experiment>
605
(pp. 594-614)
C1: I said earlier that I explained to the peer after setting up a hypothesis. But it occurred to me that I
wanted to try verifying the idea since I came up with it anyway. So, since I thought that the number
that points to the end of the shadow along the length of the stick’s shadow should mean the angle
that illuminates the surface, I experimented with that briefly.
<Interview with C1>
C2: First, since I was only reading words while doing something, I didn’t understand some parts very well.
So, because I don’t know what it is in the first place, it doesn’t hit home with me, and I think they were
read as mere letters to me.
<Interview with C2>
As C1 could not understand enough how to use the solar altitude meter in the preliminary experiment, C1
performed procedure four inaccurately. Immediately after starting the main experiment, C1 suggested a new
option based on the previous experience to the other peer and implemented it according to the option. In this
manner, ‘suggesting’ was also a frequently used means of scaffolding. The pattern of elaborating the model for the
experimental activity in this type of peer scaffolding was due to the incomplete structure of the model constructed
through the experimental activity in the previous experience, which was confirmed in the discourse above. In ad-
dition, the difference from the ‘task completion-oriented scaffolding’ was that the peer did not seek the time to
understand the procedure or that the student could not monitor the peer.
Learning Support Scaffolding: Teams B and F
The objective of the ‘learning support scaffolding’ type was learning through the experience of the peer with
the experimental activity. Thus, this type of scaffolding was considered the ideal type of peer scaffolding to achieve
the purpose of the activity. The following example was the conversation by team B for implementing procedure 2.
B1: Now, we have to adjust the angle. How would you go to do that?
B2: I think I can do it with this one (solar altitude meter).
B1: That one? That one’s not yet introduced in this step.
B2: You’re right …. Hmm …. Make one large and make one small … Should I make this one large and this
one small? (pointing to the lamps on both sides)
B1: Alright. Do you want to make this one large? (pointing to the lamp on one side)
B2: I think I can have a look at this photo first (experiment photo in the experiment manual) and make
it similar.
B1: Right. Let’s try it.
B2: To make it large, first, make the distance 20cm ….
But adjusting this to 20cm is …. Should I try to make the angle large like this?
B1: This angle … does not seem… that bad.
<Procedure 2; Team B main experiment>
B2 led the implementation after reading and understanding the experimental procedure. B1 monitored B2
and participated in the experiment from an assisting position. B1 provided ample time while inspecting the peer’s
understanding and implementation of the procedure. Moreover, through ‘posing,’ B1 helped the peer, who was
experimenting for the first time, to recall an important matter that the peer had failed to consider. In this manner,
the primary means of scaffolding used by the team were ‘monitoring’ and ‘posing.’
B1: I’ll try first. What you must pay attention to is apparently connecting the same magnetic poles and
connecting the plus and minus poles with the same ones first.
(omitted)
B1: Haha, I’ll connect the code for you.
(omitted)
B1: I’ll turn on the lamp.
<Procedures 1 and 3; Team B main experiment>
606
(pp. 594-614)
Earlier, B1 encouraged B2 to directly adjust the angle of the lamp and the solar panel, then to maintain the
distance between the lamp and the solar panel at 20 cm. However, B1 did not wait for B2 to conduct unimportant
procedures, such as connecting the electric circuit, connecting the lamp’s code, and turning on the lamp. This seems
that B1 implemented these tasks on behalf of B2, and this seems to imply that B1 distinguished the important
and unimportant segments in the experimental activity and appropriately responded according to the level of
importance. ‘Assisting’ was conducted in the scope of refraining from interrupting the peer’s new learning as the
peer already understood the procedure. In addition, ‘assisting’ was used for less important aspects of learning by
determining the importance of the procedure. By contrast, ‘monitoring’ was used to allow the peer to independently
conduct the important procedures.
The details for ‘explaining’ and ‘posing’ for this peer scaffolding type relatively required more meta-information
on performance and information on the experimental activity compared with those for other cases, and the in-
formation on the procedure and materials was minimized. This would be due to the student who was aware that
the peer would naturally acquire the information by handling the experimental procedure and experiencing the
experimental activity. In this type of peer scaffolding, the student suitably used ‘monitoring’, ‘posing’, ‘assisting’, and
‘explaining’ for the experimental activity situations of the peer. By identifying the importance of each procedure,
the student monitored the other peer in independently performing the important procedure and assisted with
less experimental procedures or those that did not significantly help for learning. This type of peer scaffolding was
superficially possible because the student with experience presented a high-level model in terms of processing the
experimental activity. The peer scaffolding of a group with a high level of knowledge focused more on the parts
wherein each other required some help instead of supporting the procedures. In addition, whether the students
of this type acquired a high-level model for the experimental activity because of the preliminary experimental
activities or due to the influence of their prior knowledge and experience remains unknown. However, that aspect
is not in the scope of the study, the analysis concerning that aspect will not be dealt with in this study.
The aforementioned types of peer scaffolding observed during the experimental activity illustrated different
objectives, means, and details of scaffolding (Table 1).
Table 1
The Types of Peer Scaffolding Observed During the Experimental Activity
Type Purpose Main means of scaffolding Main details of explanation
‘Task completion-orient- Accurate completion of the experi- Early-stage: ‘demonstrating’, ‘explain- Information on the procedure and
ed scaffolding’ mental activity by collaborating with ing’, and ‘monitoring’ materials
the peer Latter stage: all means of scaffolding
‘Model elaboration- Elaboration of one’s model for the ‘Demonstrating,’ ‘explaining,’ and ‘sug- Information on the procedure and
oriented scaffolding’ experimental activity gesting’ materials
‘Learning support scaf- Peer’s learning ‘Monitoring’, ‘posing’, and ‘assisting’ Meta information on procedural
folding’ performance and information on the
experimental activity
Furthermore, while the ‘task completion-oriented scaffolding’ and ‘model elaboration-oriented scaffolding’
mainly remained at the surface level of processing the procedure, such as understanding and implementing the
experimental procedure, ‘learning support scaffolding’ seemed to have improved the outcomes of the peer com-
pared with the other types of scaffolding by relatively addressing meta-information on procedural performance
and information on the experimental activity that the peer was unable to obtain from the experiment manual.
The reason for this difference in the types of peer scaffolding was affected by the degree of the elaborate-
ness of the model for the experimental activity structured during the preliminary experiment, that is, the level of
the model for the experimental activity. Additionally, the ‘task completion-oriented scaffolding’ and the ‘model
elaboration-oriented scaffolding’ types varied depending on whether sufficient time was provided for the peer to
understand the procedure.
607
(pp. 594-614)
Discussion
Many science educators have been keeping trying to construct students’ science experiment activities out of
the form of “following cooking recipes.” This effort to increase the effectiveness of scientific experimental activities
has increased students’ interest in learning scientific reasoning and scientific inquiry methods during experimental
activities (Hofstein & Lunetta, 2004; Lunetta et al., 2007), clarifying the objectives of experimental activities (Hodson,
1990; Hart et al. 2000), promoting cooperation (Land & Zembal-Saul, 2003) and enhancing an understanding of
the nature of science (Lunetta et al., 2007). This approach sincerely focuses on what students want to gain from
scientific experimental activities.
However, the ideal approach of these existing studies is somewhat different from the actual approach of
scientific experimental activities in the school context. In many cases, students are struggling with the process of
the laboratory manual, and teachers in the context focus on the students and their interaction while they possess
different background knowledge and work together to accurately process the experimental manual (Högström
et al., 2010) to reach the goal of the class. Improving the accuracy and effectiveness of the experimental manual
processing is the prerequisite to achieving the goal of experimental activities. In this manner, this study sufficiently
reflects the context of experimental activities in the school context, and through the reflection of the actual sci-
entific experimental activities and research outcome about peer scaffolding occurrence, this study represents the
high ecological validity. Through the results of this study, it was possible to reveal what kinds of interactions were
occurring between students with different experiences in the context of actual experimental activities, and based
on this, the way of effective guidance for students’ experimental activities.
This study derived ‘demonstrating’, ‘assisting’, ‘monitoring’, ‘posing’, ‘questioning’, ‘explaining’, and ‘suggesting’ as
means of peer scaffolding. Although this outcome differs from the means and types of peer scaffolding proposed
by Van de Pol et al. (2010), Kim and Hannafin (2011), and Shin et al. (2020)’s studies, it is similar in a broad category
in terms of providing scaffolding to peers. The means for scaffolding can be mainly classified into four categories:
‘providing actions for imitation,’ ‘providing information related to the performance of the students’, or ‘providing
information necessary for performance, which encourages students to solve problems independently without
providing information on purpose’ and ‘demanding cognitive answers.’ The types of peer scaffolding in this study
could be considered to demonstrate the characteristics of scientific experimental activities.
The three types of peer scaffolding were derived, and the crucial factor in determining these types was the
level of the model for the experimental activity previously structured by the student who provided scaffolding
regarding specific tasks during the experimental activity. This model for the experimental activity consists not only
of information about the experimental procedure and materials but also of complex and multidimensional informa-
tion, such as meta information on procedural performance and the experimental activity. Among the three types
of peer scaffolding derived from this study, ‘learning support,’ which is a desirable scaffolding type for supporting
the peer’s learning, was observed in only two out of the seven cases. This result largely differed from the general
notion that obtaining prior experience with an experimental activity will enable one to help the peer’s learning
fully. The reason for this outcome is that the experience of a preliminary experimental activity is insufficient for
structuring a high-level model for the experimental activity.
If so, why did the students fail to structure the model for the experimental activity through experience easily? The
most commonly observed scene was where the students were preoccupied with handling the experimental procedure
and materials during the activity. This phenomenon was shown clearly in studies on student-student interactions
during experimental activities (Lehman, 1990; Högström et al., 2010; Wei et al., 2019). The reason underlying this
phenomenon was that the students might have no choice but to follow the instructions of the experiment manual
to complete the task due to the characteristics of experimental activities (Högström et al., 2010), and considerable
cognitive endeavor was required to process experimental procedures (Kim & Hannafin, 2004). Students faced various
information during an experimental activity. In other words, they encountered new experiment manuals, unfamiliar
terms, new materials, new concepts, and numerous information irrelevant to the experiment during an experimental
activity. Based on the given information, they had to identify and organize important ones and determine which ones
required a high level of cognitive processing. Therefore, all cognitive resources were invested into processing the pro-
cedure (Johnstone, 1997). As a result, the abilities to observe the phenomenon and solve the problem become limited.
Students concentrate on handling the procedure and focus more on completing the task instead of accomplishing a
meaningful experimental activity (Rop, 1999; Kim & Hannafin, 2011). Thus, completing an experimental activity task
is not enough to readily equip one with a high-level model for experimental activity.
608
(pp. 594-614)
In this aspect, the focus should be placed on the reason for conducting an experimental activity. For an effective
activity, the focus should be on the scientific notions provided by the experimental activity and the abilities that
could be gained from the experience, and what type of experience procedures were handled was not as important
(Högström et al., 2010). The concept that should be vitally learned from an experimental activity is a phenomenon
that requires observation and scientific concepts. In other words, support from the experienced peer should be
provided to reduce the cognitive load of the other peering in terms of handling the experimental procedure to
enhance the effectiveness of peer scaffolding. In this manner, the peer should be allowed to concentrate more on
the observable phenomenon.
Furthermore, why does a student with a low-level model for the experimental activity fail to provide effective
peer scaffolding? As mentioned previously, an experimental activity demands a substantial cognitive-processing
capacity. In addition, students need to observe the peer and monitor the current status beyond the handling of a
simple experimental procedure during peer scaffolding. In addition, students should identify an emergent issue
and select appropriate responses (Hannafin et al., 2003; Shin et al., 2020). The students must also understand the
entire situation throughout the processes (Choi et al., 2005; Liu & Tsai, 2008). In short, peer scaffolding requires
metacognitive processing that goes beyond the implementation of the experimental procedure. It disappeared
as the effective peer scaffolding was difficult because the student with an incomplete model for the experimental
activity used more cognitive capacity in handling the experimental procedure.
Based on this, this study recommends effective methods for teachers in providing instructions for scien-
tific experimental activities. For effective scaffolding, it can be determined that teachers need a model for an
experimental activity at a level that no longer distributes cognitive processing resources to the handling of the
experimental procedure. Thus, it is necessary that teachers first acquire all information related to understanding
and implementing the experimental procedure by conducting a prior experiment. The model for the activity can
then be structured. Moreover, sufficient time to understand the experimental procedure should be provided for
the students. A high-level model for the experimental activity is necessary to enable students to concentrate on
the actual observable phenomenon. When students lack sufficient time, they may only focus on handling the
experimental procedure and acquire a superficial understanding of the experimental activity. This tendency was
identical to the suggestion by Lunetta et al. (2007), that is, the teacher and the peer needed to provide adequate
time for the student to independently generate questions and structure knowledge for effective scientific activi-
ties. In sum, students should be supported to prevent them from feeling a cognitive burden to understand and
implement the procedure so that they can exert significant cognitive efforts toward observing the phenomenon
and inferring about the overall experimental activity.
This study analyzed peer scaffolding by collecting data on the behaviors and discourses between students
with the experience of the experimental activity with their peers. For the data analysis, this study firstly derived the
information about the model for the activity that the students built. From the data, the study examined per case
to categorize the types of scaffolding, then identified the information provided and used means for peer scaffold-
ing, which were. Through the research outcome, of this study, the characteristics of the types of peer scaffolding
demonstrated by each case could be described to derive, and the factors that determined each type also gained.
These findings provide implications for a deeper understanding of small-group experimental activities in school
contexts and the operation of effective experimental activity.
The results indicate that the students used seven means of peer scaffolding: ‘demonstrating’, ‘assisting’, ‘monitor-
ing’, ‘posing’, ‘questioning’, ‘explaining’, and ‘suggesting’. Three types of peer scaffolding emerged: ‘task completion-
oriented scaffolding’, ‘model elaboration-oriented scaffolding’, and ‘learning support-oriented scaffolding’. Each
type was varied in terms of objective, main mean, and details of explanation. Moreover, two factors influenced
the determination of these types. The first was the level of the already possessed model from the experienced col-
league in the scientific experimental activity, and the second was whether sufficient time was provided to enable
the other peer with no experience to understand the experimental procedure.
For effective peer scaffolding, students need to possess a high level of an experimental activity model. The
high-level experimental activity model would be implemented not only to process the procedure of the experiment
but also the support the peer in the activity. In addition, to support the non-experienced peer’s effective learning,
the peers should be given sufficient time to understand and construct the experimental procedure independently.
609
(pp. 594-614)
Furthermore, the non-experienced peers should be supported by experienced peers in terms of reducing the
cognitive load required in handling the experimental procedure and so allow them to concentrate on observation
rather than implementation. In a scientific experimental activity, new information is imported from time to time,
and, consequently, substantial cognitive resources are used in handling the experimental procedure; therefore,
building a model for a high-level experimental activity is not a simple and easy task to achieve.
In this study, 7 cases of Korean university students’ activities were introduced and studied. Because this study
is qualitative and targets a small number of participants, its possibility for generalization is insufficient. Moreover,
although the effort was made to describe the results outside the special situation of this experimental task of the
study, the results may vary according to the context created by specific experimental activities. In this study, peer
interactions in the context of scientific experimental activities in the school context were analyzed in-depth to
provide implications for what kinds of scaffolding should be provided to students in terms of operating actual
scientific experimental activities. The findings of this study demonstrate the importance of peer scaffolding to
assist the students’ effective learning and the role of teachers in scientific experimental activities. In addition to
this, the model for the experimental activity designed and conducted in this study is deemed useful for explain-
ing numerous constructs related to scientific experimental activities, such as the handling process and outcomes
of the activity and the behaviors and cognitive processes of students. The research outcome would impact those
pre-service and in-service teachers who want to design and implement their science classes to be more active and
participative. The insights suggested in this study also assist the teachers’ and researchers’ epistemological stance
on science teaching and learning as a rather active one – not the lecture-oriented subject. Additional studies are
required to further subdivide the various aspects suggested in this study. An investigation of the level of influence
of each factor also will be required.
References
Abrahams, I., & Millar, R. (2008). Does practical work really work? A study of the effectiveness of practical work as a
teaching and learning method in school science. International Journal of Science Education, 30(14), 1945-1969.
https://doi.org/10.1080/09500690701749305
Alton-Lee, A., Nuthall, G., & Patrick, J. (1993). Reframing classroom research: A lesson from the private world of children. Harvard
Educational Review, 63(1), 50-85. https://doi.org/10.17763/haer.63.1.uh00236162314763
Andersson, J., & Enghag, M. (2017). The laboratory work style’s influence on students’ communication. Journal of Baltic Science
Education, 16(6), 958-979. https://doi.org/10.33225/jbse/17.16.958
Berry, A., Mulhall, P., Gunstone, R., & Loughran, J. (1999). Helping students learn from laboratory work. Australian Science Teachers
Journal, 45(1), 27-31.
Bilgin, I. (2006). The effects of hands-on activities incorporating a cooperative learning approach on eight grade students’ science
process skills and attitudes toward science. Journal of Baltic Science Education, 5(1), 27-37.
Choi, I., Land, S. M., & Turgeon, A. J. (2005). Scaffolding peer-questioning strategies to facilitate metacognition during online
small group discussion. Instructional Science, 33(5), 483-511. https://doi.org/10.1007/s11251-005-1277-4
Clyde, F. H. (1998). Why isn’t cooperative learning used to teach science? Bioscience, 48(7), 553-559. https://doi.org/10.2307/1313317
DeCarlo, C. L., & Rubba, P. A. (1994). What happens during high school chemistry laboratory sessions? A descriptive case
study of the behaviors exhibited by three teachers and their students. Journal of Science Teacher Education, 5(2), 37-47.
https://doi.org/10.1007/bf02962856
Domin, D. S. (2007). Students’ perceptions of when conceptual development occurs during laboratory instruction. Chemistry
Education Research and Practice, 8(2), 140-152. https://doi.org/10.1039/b6rp90027e
Driver, R., (2012), Constructivist approaches to science teaching. In L.P. Steffe & J. Gale (Eds.), Constructivism in education (pp.
403-418), Routledge. https://doi.org/10.4324/9780203052600-30
Girault, I., d’Ham, C., Ney, M., Sanchez, E., & Wajeman, C. (2012). Characterizing the experimental procedure in science laboratories:
A preliminary step towards students’ experimental design. International Journal of Science Education, 34(6), 825-854.
https://doi.org/10.1080/09500693.2011.569901
Gunstone, R. F. (1991). Reconstructing theory from practical experience. In B. Woolnough (Ed.), Practical science (pp. 67-77).
Open University Press.
Gunstone, R. F., & Champagne, A. B. (1990). Promoting conceptual change in the laboratory. In E. Hegarty-Hazel (Ed.), The student
laboratory and the science curriculum (pp. 159–182). Routledge.
Hart, C., Mulhall, P., Berry, A., Loughran, J., & Gunstone, R. (2000). What is the purpose of this experiment? Or can
students learn something from doing experiments? Journal of Research in Science Teaching, 37(7), 655-675.
https://doi.org/10.1002/1098-2736(200009)37:7<655:AID-TEA3>3.0.CO;2-E
Hodson, D. (1993). Re-thinking old ways: Towards a more critical approach to practical work in school science. Studies in Science
Education, 22(1), 85–142. https://doi.org/10.1080/03057269308560022
610
(pp. 594-614)
Hofstein, A., & Lunetta, V. N. (2004). The laboratory in science education: Foundations for the twenty-first century. Science
Education, 88(1), 28-54. https://doi.org/10.1002/sce.10106
Hofstein, A., Navon, O., Kipnis, M., & Mamlok-Naaman, R. (2005). Developing students’ ability to ask more and better
questions resulting from inquiry-type chemistry laboratories. Journal of Research in Science Teaching, 42(7), 791-806.
https://doi.org/10.1002/tea.20072
Högström, P., Ottander, C., & Benckert, S. (2010). Lab work and learning in secondary school chemistry: The importance of
teacher and student interaction. Research in Science Education, 40(4), 505-523. https://doi.org/10.1007/s11165-009-9131-3
Howe, C., Tolmie, A., Thurston, A., Topping, K., Christie, D., Livingston, K., Jessiman, E., & Donaldson, C. (2007). Group work in
elementary science: Towards organizational principles for supporting pupil learning. Learning and Instruction, 17(5), 549-
563. https://doi.org/10.1016/j.learninstruc.2007.09.004
Johnstone, A. H. (1997). Chemistry teaching-science or alchemy? Journal of Chemical Education, 74(3), 262-268.
Johnstone, A. H., & Al-Shuaili, A. (2001). Learning in the laboratory: Some thoughts from the literature. University Chemistry
Education, 5(2), 42–51.
Keys, C. W. (1996). Writing collaborative laboratory reports in ninth grade science: Three case studies of social interactions. School
Science and Mathematics, 96(4), 178-186. https://doi.org/10.1111/j.1949-8594.1996.tb10222.x
Kim, M. C., & Hannafin, M. J. (2004). Designing online learning environments to support scientific inquiry. Quarterly Review of
Distance Education, 5(1), 1-10.
Kim, M. C., & Hannafin, M. J. (2011). Scaffolding problem solving in technology-enhanced learning environments (TELEs): Bridging
research and theory with practice. Computers & Education, 56(2), 403-417. https://doi.org/10.1016/j.compedu.2010.08.024
Kipnis, M., & Hofstein, A. (2007). Inquiring the inquiry laboratory in high school. In Roser, P. & Digna, C. (Eds.), Contributions from
science education research (pp. 297-306). Springer. https://doi.org/10.1007/978-1-4020-5032-9_23
Kirschner, P. A. (1992). Epistemology, practical work and academic skills in science education. Science and Education, 1(3), 273-
299. https://doi.org/10.1007/bf00430277
Land, S. M., & Zembal-Saul, C. (2003). Scaffolding reflection and articulation of scientific explanations in a data-rich, project-based
learning environment: An investigation of progress portfolio. Educational Technology Research and Development, 51(4),
65-84. https://doi.org/10.1007/BF02504544
Lazarowitz, R., & Karsenty, G. (1990). Cooperative learning and students’ academic achievement, process skills, learning
environment, and self-esteem in tenth-grade biology classrooms. In S. Sharan (Ed.). Cooperative learning: Theory and
research (pp. 123-150). Praeger.
Lehman, J. R. (1990). Students’ verbal interactions during chemistry laboratories. School Science and Mathematics, 90(2), 142-150.
https://doi.org/10.1111/j.1949-8594.1990.tb12006.x
Lin, S. S., Liu, E. Z. F., & Yuan, S. M. (2001). Web-based peer assessment: feedback for students with various thinking styles. Journal
of Computer Assisted Learning, 17(4), 420-432. https://doi.org/10.1046/j.0266-4909.2001.00198.x
Liu, C. C., & Tsai, C. C. (2008). An analysis of peer interaction patterns as discoursed by online small group problem-solving
activity. Computers & Education, 50(3), 627-639. https://doi.org/10.1016/j.compedu.2006.07.002
Lunetta, V., Hofstein, A., & Clough, M. (2007). Learning and teaching in the school science laboratory: An analysis of research,
theory, and practice. In S. Abell & N. Lederman (Eds.), Handbook of research on science education (pp 393-441). Routledge.
Olubu, O. M. (2015). Effects of laboratory learning environment on students learning outcomes in secondary school chemistry.
International Journal of Arts & Sciences, 8(2), 507–525.
Osborne, J. (1993). Alternatives to practical work. School Science Review, 75(271), 117-123.
Palincsar, A. S., Brown, A. L., & Martin, S. M. (1987). Peer interaction in reading comprehension instruction. Educational
Psychologist, 22(3-4), 231-253. https://doi.org/10.1207/s15326985ep2203&4_3
Raviv, A., Cohen, S., & Aflalo, E. (2019). How should students learn in the school science laboratory? The benefits of cooperative
learning. Research in Science Education, 49(2), 331-345. https://doi.org/10.1007/s11165-017-9618-2
Rop, C. J. (1999). Student perspectives on success in high school chemistry. Journal of Research in Science Teaching, 36(2), 221-237.
https://doi.org/10.1002/(sici)1098-2736(199902)36:2<221:aid-tea7>3.0.co;2-c
Roscoe, R. D., & Chi, M. T. (2007). Understanding tutor learning: Knowledge-building and knowledge-telling in peer tutors’
explanations and questions. Review of Educational Research, 77(4), 534-574. https://doi.org/10.3102/0034654307309920
Roth, W. M. (2006). Learning science: A singular plural perspective. Sense.
Shi, W. Z. (2013). The effect of peer Interactions on quantum physics: A study from china. Journal of Baltic Science Education, 12(2),
152-158. https://doi.org/10.33225/jbse/13.12.152
Shin, S., Brush, T. A., & Glazewski, K. D. (2020). Patterns of peer scaffolding in technology-enhanced inquiry classrooms:
Application of social network analysis. Educational Technology Research and Development, 68(5), 2321-2350.
https://doi.org/10.1007/s11423-020-09779-0
Tamir, P., & Lunetta, V. N. (1981). Inquiry-related tasks in high school science laboratory handbooks. Science Education, 65(5),
477-484. https://doi.org/10.1002/sce.3730650503
Tiberghien, A., Veillard, L., Le Maréchal, J. F., Buty, C., & Millar, R. (2001). An analysis of labwork tasks used in science teaching
at upper secondary school and university levels in several European countries. Science Education, 85(5), 483-508.
https://doi.org/10.1002/sce.1020
Van de Pol, J., Volman, M., & Beishuizen, J. (2010). Scaffolding in teacher-student interaction: A decade of research. Educational
Psychology Review, 22(3), 271-296. https://doi.org/10.1007/s10648-010-9127-6
611
(pp. 594-614)
Wei, J., Treagust, D. F., Mocerino, M., Lucey, A. D., Zadnik, M. G., & Lindsay, E. D. (2019). Understanding interactions in face-to-face
and remote undergraduate science laboratories: A literature review. Disciplinary and Interdisciplinary Science Education
Research, 1(1), 1-16. https://doi.org/10.1186/s43031-019-0015-8
Wood, D., Bruner, J. S., & Ross, G. (1976). The role of tutoring in problem solving. Child Psychology & Psychiatry & Allied Disciplines,
17(2), 89–100. https://doi.org/10.1111/j.1469-7610.1976.tb00381.x
Yang, I. H., Kim, S. M., & Cho, H. J. (2007). Analysis of the types of laboratory instruction in elementary and secondary schools’
science. Journal of the Korean Association for Science Education, 27(3), 235-241.
Appendixes
Appendix 1: The Experimental Manual Translated into English
Determining temperature changes according to the meridian altitude of the sun
Materials
2 solar panels, 2 solar altitude meters, 2 buzzers, 4 electric wires,
2 lamp, tape measure, infrared thermometer
Procedures
1. Place two solar panels approximately 30 m apart on the surface and connect the buzzer and solar panels. Both materials must be connected using
the same magnetic pole.
2. Install the lamp by placing the lamp and the solar panel at large angles, and the other one at small angles. Adjust the lamp to place 20 cm between
the lamp and the solar panel.
3. Attach each solar altitude meter to the lab table beside the solar panel and turn on the lamps at the same time.
4. Compare the angle of the lamp and the solar panel using the solar altitude meter.
5. Compare the loudness of the sound from the buzzer according to the angles of the lamp and solar panel. If comparing the loudness is difficult, then
reduce the brightness of two lamps and compare the loudness.
6. Turn off the lamp and measure and compare the temperature of the solar panel using an infrared thermometer.
7. Discuss what objects are being represented by the lamp and the angles of the lamp and solar panel.
8. Based on the results, explain why temperature changes depending on the meridian altitude of the sun.
When the angle made by the lamp and the solar panel is large When the angle made by the lamp and the solar panel is small
612
(pp. 594-614)
Appendix 2: Information Comprising the Model for the Experimental Activity
Category Information Explanation Example
“The angle of the lamp and the solar panel refers to this
Meaning of pro- Meaning of the sentence in the procedure
angle (pointing the hand toward the end of the lamp while
cedure and the terms presented in the procedure
laying the hand on the surface next to the solar panel)”
“Connecting the solar panel and buzzer using the same

Information on How to perform the Information on how to perform the
magnetic pole means we have to check the + and − of the
procedure procedure procedure
two materials and connect the same colors”
Results showed “The temperatures of the solar panels are 22.6 ℃ and 42.9
Information on the results that will be
when performing ℃. Why do they differ so much? This is completely different
produced when performing the procedure
the procedure from the previous result.”
How to efficiently Information on how to understand or

“Differing the angle of the solar panel and buzzer is much
process the pro- execute the procedure more easily and
easier if you refer to the picture.”
cedure quickly
“Shouldn’t we move (the distance between the lamp and the

solar panel) 20cm further apart?”
Information on the relative importance of
Importance of the
each procedure in the entire experimental “Um, I don’t think this is right, (the distance between the
procedure
Meta informa- manual lamp and the solar panel) falls a bit short here.”
tion procedure -> Repetitively inspects the important performance results of
performance the procedure
“Because I did not understand this part (how to use the sun
The difficulty of the Information on time and effort required to
altitude meter) and felt lost, I thought it would be hard for my
procedure understand or execute the procedure
colleague to do this alone.”
“I think the reason for setting the distance between the solar
Reason for perform- Information on why the procedure needs to panels as approximately 30 cm is because the buzzer of the
ing the procedure be performed solar panel on the other side can make sound due to the
heat of the lamp on one side.”
Name and shape of Names of the given materials and each “The square one is the solar panel. Behind it is the electric
materials part, the shapes and structure of materials wire of +, −.”
How to use the Information on how to use the given “For the tape measure, you can press this middle part and
materials materials then remove your hand and it won’t go back in again.”
Information on
materials “When the solar panel and the buzzer are properly con-
Operational forms Information on what kind of appearance
nected and you hit the light, the sound is made from here
of materials the given materials operate with
(buzzer). Let’s try until the sound comes out well.”
Information on why the given materials are “The reason you need a solar panel is that different sounds
Use of materials
needed are made depending on the amount of light.”
The overall process Overall information on pre-and post- “Not now (measuring the distance between the solar panels)
of the experimental sequence relation of all procedures of the but we need the tapeline again later for measuring the
activity experimental activity distance between the lamp and the solar panel.”
Information on what needs to be achieved

The goal of experi- “It is to find out how the temperature changes according to
or acquired through the experimental
Information on mental activity the angle of the lamp and the solar panel.”
activity
the experimental
activity Information on the types and methods of
Manipulation and “To control the variables of the experiment, you need to turn
variables that need to be manipulated or
control of variables the lamp on and off at the same time.”
controlled in the experimental activity
“The summer with a high solar meridian altitude has a

Related scientific Information on scientific concepts that are
higher temperature than the winter with a low meridian
concepts learned through the experimental activity
altitude.”
613
(pp. 594-614)
Received: May 19, 2022 Revised: July 06, 2022 Accepted: July 28, 2022
Cite as: Kim, S.-U. (2022). The types of peer scaffolding in scientific experimental activities. Journal of Baltic Science Education,
21(4), 594-614. https://doi.org/10.33225/jbse/22.21.594
Seong-un Kim PhD in Science Education, Researcher, Department of Elementary Education,

Korea National University of Education, 28173, Cheongju, Republic of Korea.
E-mail: auul@naver.com
614
PREDICTION OF THE
CORRELATION BETWEEN
PROBLEM-SOLVING SKILLS AND ISSN 1648-3898 /Print/
CONCEPTUAL REASONING IN ISSN 2538-7138 /Online/
STOICHIOMETRY
Abstract. Learners underperform in
stoichiometry as they lack conceptual
reasoning of the underlying concepts and
Love Kotoka, the ability to solve stoichiometric problems.
Jeanne Kriek Therefore, it was necessary to determine
if there is a statistical correlation between
problem-solving skills and conceptual
reasoning in stoichiometry and if so, whether
one can significantly predict the other.
The theoretical framework is the cognitive
Introduction load theory (CLT). This theory expects
teachers to know where to focus their
This study differs from previous studies because the focus is to ex- teaching and how to assess their learners’
plore if there is a statistical correlation between problem-solving skills and work to avoid unnecessary overloading
conceptual reasoning in stoichiometry and if one significantly can predict of the working memory, which might
the other. Research on learners’ misconceptions involving chemical phe- affect their performance. The explanatory
nomena has been conducted widely (Bowen & Bunce, 1997; Bridges, 2015; sequential mixed-method research design
Gabel & Bunce, 1994; Gultepe et al., 2013; Nakhleh, 1992), and some studies was employed with 410 grade 11 Physical
focused on problem-solving and conceptual understanding in Chemistry Science learners in their intact classes. The
(Carson, 2007; Chiu, 2001; Gultepe et al., 2013; Mandina & Ochonogor, 2017; participants wrote the learner achievement
Sanger, 2005). Other studies looked at the problem-solving and conceptual test (LAT) and responded to a semi-
understanding of the topic of stoichiometry (Dahsah & Coll, 2007; Hanson, structured interview. The learners’ test scores
2016; Mandina & Ochonogor, 2017; Mashamba, 2018; Schmidt & Jignéus, were then used to run a statistical test. The
2003). If the research shows that conceptual reasoning can predict learners’ Pearson correlation and regression showed
problem-solving skills in a statistically significant way, it will then presume that the justifications given by learners for
that teachers should focus more on conceptual understanding and solving choosing correct or incorrect multiple-choice
problems in a sequential manner, showing as many details as possible. This options were not due to chance, and the
is supported by (Chirinda, 2013) who stated that teachers use systematic results of the learner interviews supported
approaches to teaching problem-solving skills to create scientifically liter- the learners’ performance in the test.
ate citizens. Moreover, the findings indicated that there
Stoichiometry is a section of Chemistry as part of Physical science was a positive correlation between problem-
which cut across the Further Education Training (FET) phase which include solving skills and conceptual reasoning
grade 10 – 12 in the South African curriculum. The sections on stoichiome- where statistically, conceptual reasoning
try-related concepts are in line with the South African high school syllabus predicted learners’ problem-solving skills
known as the Curriculum and Assessment Policy Statement (CAPS). This using regression analysis.
study focused on grade 11 learners because stoichiometry is more pro- Keywords: conceptual reasoning, cognitive
nounced in this grade and in grade 12, where more complex calculations load theory, explanatory sequential research
are required than in grade 10. The grade 12 class is an examination class design, stoichiometry, problem-solving skills
hence could not be used.
Research Problem
Love Kotoka, Jeanne Kriek
According to the National Senior Certificate (NSC) diagnostic report University of South Africa, South Africa
(DoBE, 2020), learners in the final year grade (grade 12s) are not perform-
ing well in the topics that have a direct bearing on stoichiometry, namely
615
(pp. 615-637) ISSN 2538–7138 /Online/
Reaction Rate and Stoichiometry; Chemical Equilibrium, and Acids and Bases. Therefore, learners need to master
stoichiometry in the preceding grades before entering their final year, which is grade 12 (Bridges, 2015).
Table 1 shows the summary of the average learner’s performance (marks per question) expressed as a percent-
age in seven years according to the National School Certificate diagnostic reports (DoBE, 2014–2020). The sections
in Chemistry that are of concern for this paper are those highlighted in the table. In the highlighted portion of the
column for 2014, learners are not doing well – they had an average mark of 36, 43, and 48, respectively. Similarly,
in 2016 the percentages were 39, 33, and 29 (see Table 1).
Table 1
A summary of Average Marks per Question in Percentage from 2014-2020
Percentage per year

Question Title of the Section in
Number Chemistry paper two
2014 2015 2016 2017 2018 2019 2020
1 Multiple Choice Question 52 45 50 46 54 48 62

2 Organic Nomenclature 65 64 66 62 48 57 57
3 Physical Properties of Organic Compounds 47 55 48 28 53 50 50
4 Organic Reactions 61 36 41 51 49 46 50
5 Reaction Rate and Stoichiometry 36 35 39 35 48 40 52
6 Chemical Equilibrium 43 40 33 50 43 45 37
7 Acids and Bases 48 34 29 43 44 37 45
8 Galvanic Cell 45 46 47 53 46 46 56
9 Electrolytic Cell 33 35 35 35 45 39 23
10 Fertilizers 58 50 34 49 39 44 49
Research Focus
Teaching stoichiometry is aimed at developing learners’ conceptual reasoning of the underlying concepts
and their ability to solve stoichiometric problems (Kimberlin & Yezierski, 2016)”ISSN”:”19381328”,”abstract”:”Stude
nts’ inaccurate ideas about what is represented by chemical equations and concepts underlying stoichiometry are
well documented; however, there are few classroom-ready instructional solutions to help students build scientifi-
cally accurate ideas about these topics central to learning chemistry. An intervention (two inquiry-based activities.
This study looked at conceptual reasoning as the process of applying logical thinking to a situation (a question) to
achieve the correct result (reasoning and words used to describe the process). Accordingly, problem-solving will
be explored as the strategy applied to a given question; using the strategy to bring about a solution that can be
algorithmic in nature. As a result, the outcome of this study would highlight the necessity of conceptual reason-
ing and efficient and meaningful problem-solving skills for achieving success in solving problems in Chemistry.
Theoretical Framework
Problem-solving is described in the context of the cognitive load theory (CLT) because this theory views
problem-solving as a process that includes reflection, observation, and experiential development. The basic notion
of the cognitive load theory is that cognitive capacity in the working memory is limited (Paas et al., 2016) – thus, if
a learning activity requires too much capacity, learning will be hindered. There simply may not be enough working
memory capacity available for learning entirely new tasks. Therefore, teachers should design instructional activi-
ties that will optimize the use of working memory capacity and avoid cognitive overload (Sweller, 1988) – until
fundamental skills have been developed. It is worth noting that some information may impose a high working
memory load, while other information may impose a low working memory load (Sweller, 2016). Hence it is believed
that the findings from this study could influence teachers to ensure that they do not overload learners’ working
616
ISSN 1648–3898 /Print/ PREDICTION OF THE CORRELATION BETWEEN PROBLEM-SOLVING SKILLS AND CONCEPTUAL
ISSN 2538–7138 /Online/ (pp. 615-637)
memory – to enable learners to demonstrate their problem-solving skills, for example, when performing stoichio-
metric calculations. This implies that teachers must ensure that their learners are proficient in understanding and
performing stoichiometric calculations.
Research Aim and Questions
As noted by Surif et al. (2012), a gap exists between learners’ conceptual reasoning and their problem-solving
ability in Chemistry. Stoichiometry calculation capability tends to be particularly poor in the developing world, even
among teachers (Stott, 2021). Thus, the study aimed to explore if conceptual reasoning can predict the problem-
solving skills of grade 11 Physical Science learners and if one can significantly predict the other. Therefore, this
study was designed to answer the following research questions:
1. Is there a statistical correlation between problem-solving skills and conceptual reasoning in
stoichiometry?
2. Can one significantly predict the other?
Research Design
An explanatory sequential research design was used with a mixed-method approach where both quantita-
tive and qualitative methods were utilized to collect and analyze data. The mixed-method approach was adopted
because the research outcomes could not be achieved by quantitative or qualitative methods alone (Creswell,
2012). No single viewpoint can present the entire picture and there may be multiple realities. Therefore, different
approaches can be complementary and used.
Research Sample
A convenient sample technique was chosen due to the availability and willingness of the participating schools
to partake in the study (Creswell, 2012); and also because the researchers reside in the province and the schools
were easily accessible. The research population are high schools in the Tshwane North District. The researchers
focused on schools in circuit 4 of this district and the selection of the sample were 11 intact classes. The sampled
schools are found within the same geographical location. They have similar socio-economic status, where the
learners are amongst others, part of a school feeding scheme and are referred to as quintile one schools. These
schools form a group of schools catering to the poorest 20% of learners – in other words, a group representing
20% of a population with the lowest income (Nordstrum, 2012). The teachers of the sampled learners were from
similar academic backgrounds (Dilnot, 2016). The research time was approximately 4 weeks.
The participants were all grade 11 learners (n = 410) from eleven selected public schools in a District of the
Gauteng Province, in South Africa. The average age of the learners is seventeen. Gender was not considered in the
data analysis due to an unbalanced distribution of gender. The learners participated in a two-phase exercise which
comprised writing a test and taking part in a semi-structured interview.
All participants gave consent and written letters were given to assure them of full confidentiality as well as
anonymity. In addition, the learners were informed of their right to withdraw from the research at any time during
the study, if they so deem it necessary (Kotoka, 2020). Letters were also written to all other stakeholders such as
the Gauteng Department of Education, principals and teachers at the schools, and parents of the participating
learners. Ethical clearance was granted by Unisa’s research ethics review committee (2015_CGS/ISTE_009).
Instruments
Two research instruments were used namely a learner achievement test (LAT) and a semi-structured learner
interview schedule (LIS). The learner achievement test consisted of two sections – section A with five multiple-
choice items, and section B with five short-answered written questions. The mark allocations for the first section
of the test are three marks per question and that of the second section is five marks per question. The questions in
617
(pp. 615-637) ISSN 2538–7138 /Online/
the test were adapted and slightly modified from sources such as the South African matriculation past examina-
tion papers for Physical science (Chemistry) (Parent24, 2014) and commonly used South African Physical science
textbooks (de Vos, du Plessis, Nel, Spies & van Wyk, 2015).
The learner interview schedule consisted of five questions that were posed to selected learners. The interview
was done to confirm how the learners understood the learner achievement test they wrote. The questions on the
learner interview schedule are stated below.
Q1; Were the problems on the learner achievement test difficult or easy?
Q2; How did you balance the chemical equations in questions one and five in section B?
Q3; How did you relate mass, molar mass, and moles of substances?
Q4; Regarding question three, in section B, how did you apply the concept of mole ratio?
Q5; How do you understand limiting reactants?
Validity
Validity demands that an item describes what it is meant to describe (Bulsara, 2014). Therefore, the LAT and
LIS were given to three experts in Physical science in the Tshwane North District in the Gauteng province of South
Africa to read and make corrections and suggestions. This was done to ensure both face and content validity. These
experts included a cluster leader, a district subject advisor, and a seasoned senior teacher of reputable standing
in content and curriculum in the field of science education. Their criticism and comments were to ensure a high
level of validity. For example, they suggested rephrasing the questions. There was a suggestion that open-ended
questions that require learner thinking, be incorporated into the test since conceptual reasoning and problem-
solving skills are being tested.
Reliability
Reliability tells test users about the consistency of the scores produced in a test and it is therefore important
for judging the suitability of a test or measuring instruments (Golafshani, 2003). The marks obtained from the
learners who wrote the learner achievement test twice on pilot bases were used to calculate the Spearman cor-
relation coefficient using SPSS version 23 (Siegle, 2013). The Spearman correlation coefficient obtained was .924
which showed a high correlation and indicated high reliability of the test. Fourteen learners from another district
in Gauteng province were interviewed using the prepared learner interview schedule to test its reliability. The
learners’ responses were consistent, indicating that the learner interview instrument was reliable. This means that
while the absolute responses of what learners report may differ somewhat from one another, the effect is consistent
across all learners so that the effect does not appear to advantage or disadvantage one learner group compared
with another (Bostic & Sondergeld, 2015).
Procedures and Data Collection
Eleven classes of learners (n = 410) took the test in one session. Their chemistry teachers supervised the test.
The learners were required to answer all the questions (10 items in the two sections). Learners were to provide
both the answer and an argument for their answers in the first section. The test required the learners to write down
their chosen answers to the conceptual questions and to provide reasons for their answers using their conceptual
knowledge about the topic. To gain a deeper understanding of the learners’ problem-solving skills, the second
part of the test involved algorithmic questions. The question required the learners to work through a procedure
to find a numerical solution to a chemical problem. These types of questions were considered problem-solving
questions where the learners were expected to show reasonable solutions. See the appendix for the test item and
its marking memorandum.
Section A of the test aimed to obtain data on learners’ conceptual reasoning whereas section B was to gather
data on learners’ problem-solving skills. Therefore, section A contained multiple-choice (A-D) questions, and spaces
were provided for the learners to state their reasoning behind each choice they made. The learners who took part
in the interviews were selected based on their performance according to the test results. With the assistance of the
subject teachers and based on the first term test, learners were classified as good, average, or weak. There were six
618
ISSN 2538–7138 /Online/ (pp. 615-637)
learners sampled from each of the eleven schools that participated in the interview. Due to technical problems,
however, two audiotapes out of the eleven schools could not be used for analysis. Therefore, the total number
of learners was those from nine schools. The data from the interview transcripts were used to substantiate and
authenticate the results of the analysis of the test.
Data Analysis
The learner achievement test was analyzed quantitatively and qualitatively. These tests were marked, and
the scores were recorded in Excel and later exported to SPSS version 23 for statistical analysis. Pearson correlation
was used to do a statistical correlation to determine the association between learners’ problem-solving skills and
their conceptual reasoning. Furthermore, a regression analysis was conducted to see if learners’ problem-solving
skills depended on their conceptual reasoning. In other words, can learners’ conceptual reasoning predict their
problem-solving skills? Even though the study aims to explore the correlation between conceptual reasoning and
problem-solving skills, Pearson correlation and regression were conducted on the multiple-choice responses alone
and the reasoning behind the choices made. This was done to ascertain whether the learners’ choice out of the
multiple choices was merely due to chance.
Apart from the quantitative analysis, the multiple-choice responses with their accompanying learner reasoning
were also analyzed qualitatively. Both quantitative and qualitative analysis were used as the qualitative results were
used to shape the quantitative results based on the views and opinions of the participants. Conceptual reasoning
refers to the learners’ ways of thinking – these include analyzing a problem, developing new ideas, and reflecting
on them (Rahman, 2019). Problem-solving signifies how an individual uses previously acquired knowledge, skills,
and understanding to satisfy the demands of an unfamiliar situation (Carson, 2007). Conceptual reasoning per
this study means that when a learner reasons through the concepts in each problem and then applies adequate
problem-solving skills to resolve it, then it is assumed that the learner understands the problem. The multiple
items were analysed qualitatively using descriptive method (see Figure 1, Table 9 and Appendix B). The learners’
conceptual reasoning was categorised into No conceptual reasoning (NCR), Partial conceptual reasoning (PCR),
and Good conceptual reasoning (GCR) where different marks were allocated to each (see Appendix B).
In section A of the learner achievement test (conceptual reasoning questions) learners were asked to justify
the reason for choosing the correct multiple-choice answer in all five test items. This is to show the conceptual
reasoning of the learner regarding the concept being tested. The justification had been categorized into ‘no con-
ceptual reasoning’ (NCR), ‘partial conceptual reasoning’ (PCR), and ‘good conceptual reasoning’ (GCR). The con-
notation of NCR was that the learner gave an incorrect reason or no reason at all for his/her choice of an answer.
PCR means that the learner gave a vague or a partially valid reason, an incomplete reason, and/or an imprecise
answer. GCR would mean that the learner stated sound conceptual reasoning; a completely valid reason and/or
a precise answer for his/her multiple-choice. The correct multiple-choice was awarded one mark, no conceptual
reasoning was awarded a zero-mark, partial conceptual reasoning was awarded one mark, and good conceptual
reasoning was awarded two marks. In section B of the learner achievement test, the learners were tested for their
problem-solving skills. There were five questions in total, and each question was allocated five marks, which made
it easier for the researchers to award a mark to the different steps shown by the learners in their solutions. Hence,
on the memorandum of the learner achievement test, marks were awarded for each step of thinking required.
The learners’ interviews were audiotaped and transcribed word by word for analysis. Document analyses
were used to find themes in the responses. The responses were grouped under three themes that related to the
main questions that were asked during the interview sessions (Saldana, 2015). The prominent themes that arose
for each response were colour-coded and titled.
Most favourable responses
Fair (to neutral) responses
Unfavourable responses
Therefore, the transcribed interviews were classified according to the themes (Saldana, 2015).
619
(pp. 615-637) ISSN 2538–7138 /Online/
Research Results
Quantitative Results on Learner Achievement Test
Statistical tests were run for the multiple-choice questions versus their conceptual reasoning. Table 2 represents
all the Pearson correlation coefficients and shows values between 0.3 and 0.6 demonstrating a strong correlation
at p = .001. For example, objective Q1 (question 1) is significant, [Pearson r = .367, n = 410, p < .001]. From table 2,
objective Q4 (question 4) showed the strongest correlation. The Pearson correlation gave a significance p of .001
which is less than .01 and .05 and since the .001 is less than the chosen significance levels. Therefore, the correla-
tion showed statistically significant relationship between the variables (conceptual reasoning and problem-solving
skills). The results showed that the justifications given by learners for choosing correct or incorrect multiple-choice
options were not due to chance. This suggested that a learner choosing the right multiple-choice option has the
right conceptual reasoning as far as the findings of this study are concerned.
Table 2
Correlations between the Objective Question Scores and their Conceptual Reasoning
Objective Objective Objective Objective Objective
Q1 Q2 Q3 Q4 Q5
Pearson
Correlation .367** .472** .467** .615** .426**
p .001 .001 .001 .001 .001
n 410 410 410 410 410
**Correlation is significant at the .01 level (2-tailed).
* Correlation is significant at the .05 level (2-tailed).
Regression goes beyond correlation by adding prediction capabilities (Gerber, 2013; Pérez & de Los Campos,
2014). Therefore, regression analysis was done for the same objective questions versus their conceptual reasoning
to establish the strength of the significance of the two variables on each other (see Tables 3 and 4). Table 3 presents
the model summary for the five questions where most of the adjusted R square values show a modest variation.
The average adjusted R square (for the 5 questions) was .225 and the predictor (conceptual reasoning) varies from
the dependent variable (objective questions) by an average of .367. The standard error of the estimate of .367, is a
measure of how much R (the conceptual reasoning) is predicted to vary (about 37% variation).
Table 3
Model Summary for the Questions
Model R R Square Adjusted R Square SE
1 .367a .134 .132 .277

2 .472 a
.223 .221 .360
3 .467a .218 .216 .441
4 .615 a
.378 .376 .391
5 .426a .182 .180 .366
a
. Predictors: (Constant), Conceptual Reasoning Q1-Q5
The table of coefficients (Table 4) for the five questions indicates that the regression for Q1 is significant, (b =.367,
t = 7.960; p < .001). The average (all question) standardized coefficient b value of .4694 at the significant level of
.001, shows a significant difference. Considering that the standardized coefficients b values are greater than .001,
620
ISSN 2538–7138 /Online/ (pp. 615-637)
there is a statistically significant correlation between the dependent (objective questions) and the independent
(Conceptual Reasoning) variables at that significant level.
Table 4
Coefficientsa for Questions
Unstandardized Coefficients Standardized

Model Coefficients
t p
B SE Beta
1 (Constant) .791 .021 36.921 .001

Conceptual Reasoning Q1 .101 .013 .367 7.960 .001
1 (Constant) .561 .028 20.307 .001
1 (Constant) .224 .030 7.417 .001
1 (Constant) .330 .025 13.276 .001
1 (Constant) .634 .025 25.606 .001
a
. Dependent Variable: Objective Q1-Q5
Conceptual Reasoning vs. Problem-solving Skills
The Pearson correlation illustrates the correlation conducted between learners’ conceptual reasoning (CR)
and their problem-solving skills (PSS) (see Table 5). The results show the descriptive statistics indicating the means,
the standard deviation, and the number of learners. The mean score for the 410 learners’ CR was 8.81 (SD = 3.59)
and the mean score of learners’ PSS is 8.08 (SD = 5.89). The Standard Deviation of problem-solving skills of 5.89
indicates a widespread PSS attained by learners.
Table 5
Descriptive Statistics for Conceptual Reasoning and Problem-solving Skills
M SD N
CR 8.81 3.59 410

PSS 8.08 5.89 410
The correlation to show the connection between conceptual reasoning and problem-solving skills is illustrated
in table 6 below. The Pearson correlation between conceptual reasoning and problem-solving skills (see Table 6) is
[r = .48, n = 410, p < .001]. This correlation revealed a positive connection between the two variables. Thus, there is
a positive correlation between conceptual reasoning and problem-solving skills. However, this positive correlation
is a moderate correlation because is less than .5.
621
(pp. 615-637) ISSN 2538–7138 /Online/
Table 6
Correlations for Conceptual Reasoning and Problem-Solving Skills
CR PSS
CR Pearson Correlation 1 .483**

p .001
n 410 410
PSS Pearson Correlation .48** 1
p .001
n 410 410
**. Correlation is significant at the .01 level (2-tailed).
A few outliers of the data set may have resulted in the value of .483 (p at .001 ˂ 0.05). The outliers might
have caused the moderate correlation between the conceptual reasoning and problem-solving skills otherwise a
stronger correlation may have resulted.
Conceptual Reasoning Predicting Problem-solving Skills
To determine if conceptual reasoning can predict problem-solving skills, regression analysis was used. The
model summary for the conceptual reasoning and the problem-solving skills which gave an adjusted R square of
.23 is presented in table 7. The variation of .23 (23%) between the predictor and the dependent variables indicated
a modest fit between them comparing it to the rule of thumb as a guide (Muijs, 2010).
Table 7
Model summary for Conceptual Reasoning and Problem-solving Skills
Standard Error of the

Model R R Square Adjusted R Square
Estimate
1 .48a .23 .23 5.17

a
. Predictors: (Constant), CR
b
. Dependent variable: PSS
The coefficients for conceptual reasoning (CR) and Problem-solving skills (PSS) are presented in table 8. The
regression is significant (b = .483; t = 11.14, p < .001) because the b value, .483 is greater than p < .001. This indicates
that the independent variable can significantly predict the dependent variable. Therefore, there is a statistically
significant prediction between CR and PSS.
Table 8
Table of Coefficients for Conceptual Reasoning and Problem-Solving Skills
Unstandardized Coefficients Standardized Coefficients

Model
B SE Beta t p
1 (Constant) 1.08 .68 1.60 .001

CR .79 .07 .48 11.14 .001
a. Dependent Variable: PSS
622
ISSN 2538–7138 /Online/ (pp. 615-637)
Qualitative presentation of the Result of the Learner Achievement Test
Section A
The summary of the analysis of the conceptual reasoning questions (section A) of the LAT is presented in Fig-
ure 1. To make the interpretation visually clearer, the data was used to draw a multi-bar graph which showed the
Right multiple-choice, No conceptual reasoning (NCR), Partial conceptual reasoning (PCR), and Good conceptual
reasoning (GCR) in different colours. The number of learners was expressed in percentages (see Figure 1 below).
Figure 1
Learners’ Right Objective Answer and its Justification in Percentages.
Questions one, two, and five were the best answered in terms of making the best objective choices and giving
the reasons for the choices made in section A. For questions three and four the highest percentages were recorded
for no conceptual reasoning (NCR) which means learners could not give reasons for their choices. The extracts of
how the learners’ scripts were marked in terms of the different categories (NCR, PCR, GCR) is presented in appendix B.
Section B
Table 9 shows problem-solving questions (section B) where the question numbers were split into the ex-
pected demonstrated skills by the learners, the number of learners who showed right method and wrong method
expressed in percentages. In the table, learners demonstrated more wrong methods in their problem-solving
process compared to the right method. As far as problem-solving questions in section B are concerned, question
four was the best-answered question among the problem-solving questions and question three was the most
poorly answered (LAT section B).
Table 9
Problem Solving Skills Demonstrated by the Learners in each Question
Question Skills Learners are Expected to Wrong Percentage Right Percentage

number Demonstrate method (%) method (%)
Balance equation 185 45.1 225 54.9

Calculate molar mass 291 70.9 119 29.1
1 Calculate moles 320 78.0 90 22.0
Use mole ratio 228 55.6 182 44.4
Calculate mass 343 83.7 67 12.3
623
(pp. 615-637) ISSN 2538–7138 /Online/
Question Skills Learners are Expected to Wrong Percentage Right Percentage

number Demonstrate method (%) method (%)
Use a balanced equation 203 49.5 207 50.5

2 Calculate moles 323 78.9 87 21.1
Use mole ratio 190 46.3 220 53.7
Calculate moles 266 64.9 144 35.1
Determine the mole ratio 357 87.1 53 12.9
3
Compare quantities 378 92.2 32 7.8
Determine limiting reactant 369 90.0 41 10.0
Calculate volume using molar volume 151 36.8 259 63.2
4 Change subject formula/use ratios 155 37.8 255 62.2
Convert units 190 46.3 220 53.7
Balance equation 187 45.6 223 54.4
Calculate moles 288 70.2 122 29.8
5
Use mole ratio 307 74.9 103 25.1
Calculate percentage yield 320 78.0 90 22.0
Qualitative presentation of the result of the Learner Interview Schedule
Below are the tabulated results from the interview sessions expressed in percentages.
Table 10
Number of Responses given by the Learners and its Percentages
Most favourable responses Fair (to neutral) responses Unfavourable responses
Responses % Responses % Responses %
1. 18 33.3 30 53.7 6 13.0

2. 33 61.1 6 11.1 15 27.8
3. 47 87.0 7 13.0 0 -
4. 13 24.0 0 - 41 76.0
5. 33 61.1 0 - 21 38.9
Average 53.30 15.56 31.14
The learners belonging to the different themes identified earlier in this section were compared (see Table 10).
Learners belonging to the ‘most favourable responses’ theme constitute the highest percentage at 53.30%. This
group represented the learners with much insight into the concept of stoichiometry. The learners belonging to the
theme of ‘fair to neutral responses’, constitute the least percentage; and learners belonging to the ‘unfavourable
responses’ theme (31.14%) were those who seem to have no insight into stoichiometry. The above results imply
that learners have knowledge about concepts taught in stoichiometry (see Figure 1).
624
ISSN 2538–7138 /Online/ (pp. 615-637)
Discussion
With regard to the research questions, quantitatively, the data in table 2 indicates “If a learner gets a multiple-
choice question right it means the learner has the right conceptual reasoning” and vice versa. These findings are in
agreement with other studies that verified the above correlation (Chiu, 2001; Cracolice et al., 2008; Hanson, 2016).
To further respond to the research questions, the statistical test showed a positive correlation between the learn-
ers’ conceptual reasoning and their problem-solving skills (from Table 4). The above results are consistent with the
findings of other researchers in this field. For example, according to a study done at a local high school in Taiwan,
most learners were considered both good problem solvers and good conceptual thinkers (Chiu, 2001). The cur-
rent result also supports the findings by Gultepe et al. (2013), where increased conceptual problem-solving ability
improved algorithmic problem-solving skills in their study. The conceptual reasoning had statistical significance
for learners’ problem-solving skills. The current findings, which indicate a positive correlation between conceptual
reasoning and problem-solving skills, are in agreement with a study that found that there is a significantly positive
correlation between algorithmic problem-solving skills and conceptual understanding and mathematical process-
ing skills (Al-Mutawah et al., 2019).
To deal with the research questions qualitatively, looking at conceptual reasoning, question 1, when the learner
chooses the right multiple-choice answer, he/she scores one mark out of the three marks allocated – and this ap-
plies to all five multiple-choice questions in the section. If a learner justifies his/her answer by saying he/she used
the knowledge of how the mole is related to the mass and molar mass of oxygen, for example, 1 mol O = 16g/mol
= molar mass, and mass of O given = 35.2g and did the calculation shown below to obtain the moles; n = m/M
O = 35.2g/16 = 2.2 mol, the learner is awarded two marks for GCR due to the correct and complete conceptual
reasoning provided. Another example: if the learner shows the formula n = m/M and uses the formula to explain
the associations among the variables, he/she will be awarded two marks. A learner is awarded one mark, which is
PCR, when he/she gives a partially valid reason – like just writing n = m/M – with no explanation. When a wrong
formula or reason is given or noting is written for the justification, 0 is awarded, which is NCR.
For conceptual reasoning question 2, where the learners were asked to calculate the mass of a substance. They
must show or explain the formula, calculate molar mass, and use it to find the mass. It is only then they obtain the
2 marks for GCR. The molar mass of C8H9O4 = 169g/mol, and n = m/M so, m = n x M = 0.432mol x 169g/mol = 73g.
If they only provide formula and could not find the molar mass to calculate the mass, it is partial work done and
deserves 1 mark. No mark would be awarded if the mass is incorrect or no reason at all is provided.
With the third conceptual reasoning question, the justification is being able to identify the reactant and the
product and their coefficient. Also, they must know the mole ratio between O2 and CO2 produced for the two
marks, thus, O2: 2CO2 = 1:2. Therefore, if the learner mentioned mole ratio in his/her explanation but did not get
the right coefficients and in their correct order, it is given a mark – PCR. The wrong mole ratio such as 2:2, 2:1, or
1:1 is NCR (0 mark).
Conceptual reasoning question 4 was testing the knowledge of the amount of substance (Moles) from the
other terminologies such as grams, litres, and particles. Therefore, all the multiple-choice options provided were
the same in terms of the number of reactants and the number of products. Here the learners were expected to
justify their multiple-choice answers based on total moles (amount of substance) for reactants and products and
the difference between moles, mass, volume, and atoms. 1 + 8 = 5 + 6 thus, 9:11 gives a learner the two marks
for good conceptual reasoning. Any other justification will not be awarded a mark.
The final question requested the definition of Limiting reagent, which requires knowing the concept of limiting
reagent and being able to define it as the substance that is totally used up when the chemical reaction is complete.
This determines how many moles of a product should be formed for the two marks. A partial or incomplete defini-
tion was awarded one mark and no definition was 0 mark for the justification of one’s choice.
Five marks were allocated to each of the five questions in section B of the learner achievement test where
problem-solving skills were being tested. A mark is awarded for a correct step shown in the problem-solving process,
as stated previously above. For example, when a leaner was able to balance the given chemical equation, calculate
the moles of one molecule and use mole ratio to find the moles of the required molecule and, continue to calculate
mass where necessary, providing the right formula, methodology, and appropriate unit. This learner would have
demonstrated acceptable problem-solving skills. The same applies to those questions that required the learners to
calculate the molar volume and the percentage yield of a compound as well as the limiting reagent of a substance.
625
(pp. 615-637) ISSN 2538–7138 /Online/
Concerning interview question two, 61.1% of the learners knew how to balance the chemical equation. How-
ever, 55% were able to balance chemical equations for questions one and five in the test (section B). From interview
question three on mass, mole, and molar mass, 87% of the learners were able to mention the formula that connects
the three concepts (see table 10 for interview response). From marking the learners’ scripts, particularly section B,
the test results revealed that an average of 70% of the learners could not calculate moles and mass of substances
using the molar mass correctly (see table 9 for test outcome). The researchers suspect that the reason why some
of the learners were not able to calculate moles and mass of substances using the molar mass correctly could be
due to the multiple steps involved in this type of calculation. The learners had to first calculate the molar mass of
the substance to be able to calculate the moles and/or mass of the substance. In situations where the chemical
equation given is not balanced, the learners had to balance the chemical equation before commencing with the
calculations, hence they had difficulties.
Interview question four indicated that 24% (see table 10) of learners used mole ratio correctly in solving stoi-
chiometric problem 3 in section B. From the test results, however, only 12.9% (table 9) of the learners were able to
apply the mole ratio correctly to solve the given problem (question 3, in section B). Indeed, only 16 learners out
of 410 (4%) were able to solve question 3 completely right. Lastly, the fifth interview item, attested that the learn-
ers could define limiting reactants (61.1%), but the outcome of the test showed that merely 10% of the learners
could work through the calculations of limiting reactants using the right method as required in the question 3 of
section B of the test.
The cognitive load theory (CLT) provides guidelines that assist in the presentation of information in a manner
that encourages learner activities that optimize intellectual performance (Jalani & Sern, 2015). With an under-
standing of the CLT and its instructional implications, educators will be in a better position to design and develop
instructional materials that align with human cognitive architecture (Sithole, 2019). In the end, instructional
materials that employ CLT guidelines can enhance learning effectiveness and efficiency for learners in a variety of
educational contexts (Artino, 2008).
This study has found that learners’ conceptual reasoning determined their problem-solving skills. So according
to CLT, to reduce the overloading of learners’ working memory, teachers should explain complex phenomena to
learners; provide learners with relevant previous knowledge before expecting them to use new knowledge, and
encourage learners to apply available resources to advanced cognitive processes (Chang & Karpudewan, 2020).
The qualitative findings were consistent with quantitative results. It was discovered through statistics that
learners’ conceptual reasoning informed their choices of answers to the multiple-choice options. The existence of
a positive correlation between learners’ conceptual reasoning and their problem-solving skills means that learn-
ers’ problem-solving skills reflect their conceptual reasoning. This means that one variable predicted the other.
Thus, in line with the CLT standpoint, teaching methods need to provide learners with guidance and support to
maintain learner focus and avoid cognitive overload during a task that has the potential to become overwhelming
for learners due to its complexity.
Limitations
The findings cannot be generalized as the data presented was from a single district and should therefore be
regarded as illustrative rather than exhaustive. This limits the transferability of the study to other contexts since it
is not representative of other contexts (Vasileiou, et al., 2018).
Conclusion and Implications
Literature showed that stoichiometry is regarded as one of the more challenging topics to teach. We explored
how grade 11 Physical Science learners’ problem-solving skills can be predicted by conceptual reasoning in stoichi-
ometry in the South African context. It has been established that there is a positive correlation between learners’
conceptual reasoning and their problem-solving skills. The results revealed that conceptual reasoning indeed
reflects how learners approach questions and indicated that learners with good reasoning skills solve problems
much better. Therefore, it was shown that the one can predict the other.
Stoichiometry should be taught in such a way to enhance learners’ conceptual understanding, thereby leading
to good problem-solving skills. During instruction, teachers should pay attention to all concepts of stoichiometry
such as the mole concept, molar gas volume, concentration of solutions, percentage composition, empirical
626
ISSN 2538–7138 /Online/ (pp. 615-637)
and molecular formula, stoichiometric calculations, and so forth, to promote learners’ conceptual reasoning and
problem-solving skills.
The implication of this study is that teachers during instructional activities should be mindful not to overload
the working memory of learners in an attempt to enhance their problem-solving skills of the learners. Teachers need
to ensure that all aspects of curriculum needs are met and that all levels of the curriculum would align properly.
References
Al-Mutawah, M. A., Thomas, R., Eid, A., Mahmoud, E. Y., & Fateel, M. J. (2019). Conceptual understanding, procedural knowledge
and problem-solving skills in mathematics: High school graduates work analysis and standpoints. International Journal of
Education and Practice, 7(3), 258–273. https://doi.org/10.18488/journal.61.2019.73.258.273
Artino, A. R. J. (2008). Cognitive load theory and the role of learner experience: An abbreviated review for educational practitioners.
Association for the Advancement of Computing in Education, 16(4), 425–439. http://www.editlib.org/d/25229/article_25229.
pdf
Bostic, J. D., & Sondergeld, T. A. (2015). Measuring sixth-grade students’ problem solving: Validating an instrument addressing
the mathematics common core. School Science and Mathematics, 115(6), 281–291.
Bowen, C. W., & Bunce, D. M. (1997). Testing for conceptual understanding in general chemistry. The Chemical Educator, 2(2),
1–17. https://doi.org/10.1007/s00897970118a
Bridges, C. D. (2015). Experiences teaching stoichiometry to students in grades 10 and 11. In Doctoral Dissertations. Walden
University.
Bulsara, C. (2014). Using a mixed methods approach to enhance and validate your research [Notre Dame University]. http://www.
nd.edu.au/downloads/research/ihrr/using_mixed_methods_approach_to_enhance_and_validate_your_research.pdf
Carson, J. (2007). A problem with problem solving. Teaching Thinking without Teaching Knowledge, 17(2), 7–14.
Chang, F. S., & Karpudewan, M. (2020). Working memory capacity and teaching and learning of stoichiometry. Springer.
Chirinda, B. (2013). The Development of mathematical problem solving skills of grade 8 learners in a problem-centred teaching and
learning environment at a secondary school in Gauteng (Issue June). Doctoral dissertation, University of South Africa.
Chiu, M. H. (2001). Algorithmic problem solving and conceptual understanding of chemistry by students at a local high school
in Taiwan. Chemical Education International, 11(1), 20–38.
Cracolice, M. S., Deming, J. C., & Ehlert, B. (2008). Concept learning versus problem solving : A cognitive difference. Journal of
Chemical Education, 85(6), 873–878. https://doi.org/10.1021/ed085p873
Creswell, W. J. (2012). Educational research: Planning, conducting, and evaluating quantitative and qualitative research (4th ed).
Pearson Education.
Dahsah, C., & Coll, R. K. (2007). Thai grade 10 and 11 students’ conceptual understanding and ability to solve stoichiometry
problems. Research in Science & Technological Education, 25(2), 227–241. https://doi.org/10.1080/02635140701250808
de Vos, E., du Plessis, S., Nel, J., Spies, S. & van Wyk, K. (2015). DocScientia Physical Sciences Book 2 Grade 11 (Revised ed).
Department of Basic Education. (2020). National Senior Certificate Examination. In NSC School Subject Report 2019 (Vol. 3, Issue
150). https://www.education.gov.za/ExamResults2019.aspx
Dilnot, C. (2016). How does the choice of A-level subjects vary with students’ socio-economic status in English state schools?
British Educational Research Journal, 42(6), 1081–1106.
Gabel, D. L. (1995). Handbook of research on science teaching and learning. In Choice Reviews Online (Vol. 32, Issue 07). Macmillan
Publishing Company, Division of Macmillan. https://doi.org/10.5860/choice.32-4019
Gerber, H. (2013). Data Analysis with a statistical package -SPSS. HR Statistics (Pty).
Golafshani, N. (2003). Understanding reliability and validity in qualitative research. The Qualitative Report, 8(4), 597–607. http://
www.nova.edu/ssss/QR/QR8-4/golafshani.pdf
Gultepe, N., Celik, A. Y., & Kilic, Z. (2013). Exploring effects of high school students’ mathematical processing skills and conceptual
understanding of chemical concepts on algorithmic problem solving. Australian Journal of Teacher Education, 38(10),
106–122.
Hanson, R. (2016). Ghanaian teacher trainees ’ conceptual understanding of stoichiometry. Journal of Education and E-Learning
Research, 3(1), 1–8.
Jalani, N. H., & Sern, L. C. (2015). The example-problem-based learning model: Applying cognitive load theory. Procedia - Social
and Behavioral Sciences, 195, 872–880. https://doi.org/10.1016/j.sbspro.2015.06.366
Kimberlin, S., & Yezierski, E. (2016). Effectiveness of inquiry-based lessons using particulate level models to develop high
school students’ understanding of conceptual stoichiometry. Journal of Chemical Education, 93(6), 1002–1009. https://doi.
org/10.1021/acs.jchemed.5b01010
Kotoka, L. (2020). Investigating grade 11 learners’ problem-solving skills and conceptual reasoning on concepts in stoichiometry.
Doctorial dissertation, Unisa, Pretoria.
Mandina, S., & Ochonogor, E. C. (2017). Problem solving instruction for overcoming students’ difficulties in stoichiometric problems.
Journal of Educational Psychology, 10(4), 69–78.
Mashamba, N. E. (2018). Examining the relationship between science teachers Beliefs and the PCK in stoichiometry in final year pre-
service teachers. Doctoral dissertation, University of the Witwatersrand, Johannesburg.
Muijs, D. (2010). Doing quantitative research. Sage Publications.
627
(pp. 615-637) ISSN 2538–7138 /Online/
Nakhleh, M. B. (1992). Why some students don’t learn chemistry: Chemical misconceptions. Journal of Chemical Education, 69(3),
191-195.
Paas, F., Tuovinen, J. E., Tabbers, H., & Van Gerven, P. W. (2016). Cognitive load measurement as a means to advance cognitive
load theory. In In Educational Psychologist (pp. 63–71). Routledge.
Parent 24. (2014). https://www.news24.com/parent/learn/freeexamresources/matric-past-exam-papers/nsc-old-exam-papers-
physical-sciences-20161006
Pérez, P., & de los Campos, G. (2014). BGLR: A statistical package for whole genome regression and prediction. Genetics, 198(2),
483–495.
Rahman, M. (2019). 21st century skill ‘problem solving’’: Defining the concept.’ Asian Journal of Interdisciplinary Research, 2(1), 64–74.
Saldana, J. (2015). The coding manual for qualitative researchers (3rd edition). SAGE Publications. https://doi.org/10.1108/QROM-
08-2016-1408
Sanger, M. J. (2005). Evaluating students ’ conceptual understanding of balanced equations and stoichiometric ratios using a
particulate drawing. Journal of Chemical Education, 82(1), 131–134.
Schmidt, H.-J., & Jignéus, C. (2003). Students´ strategies in solving algorithmic stoichiometry problems. Chemistry Education
Research and Practice, 4(3), 305. https://doi.org/10.1039/b3rp90018e
Siegle, D. (2013). Instrument reliability. University of Connecticut Educational Research Basics website.
Sithole, S. T. (2019). Enhancing blended learning materials using cognitive load theory. Journal of Modern Accounting and Auditing,
15(1), 40–53.
Stott, A. E. (2021). South African physical sciences teachers’ use of formulae and proportion when answering reaction-based
stoichiometry calculation questions. Chemistry Education Research and Practice, Advance Ar. https://doi.org/10.1039/
D0RP00291G
Surif, J., Ibrahim, N. H., & Mokhtar, M. (2012). Conceptual and procedural knowledge in problem solving. Procedia - Social and
Behavioral Sciences, 56(Ictlhe 2012), 416–425. https://doi.org/10.1016/j.sbspro.2012.09.671
Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12(2), 257–285. https://doi.
org/10.1207/s15516709cog1202_4
Sweller, J. (2016). Working memory, long-term memory, and instructional design. Journal of Applied Research in Memory and
Cognition, 5(4), 360-367. https://doi.org/10.1016/j.jarmac.2015.12.002
Vasileiou, K., Barnett, J., Thorpe, S., & Young, T. (2018). Characterising and justifying sample size sufficiency in interview-based
studies: systematic analysis of qualitative health research over a 15-year period. BMC Medical Research Methodology, 18(1),
1–18.
628
ISSN 2538–7138 /Online/ (pp. 615-637)
Appendix A
Learner Achievement Test (LAT)
Learner Code ______________ School Code __________ Date __________ Class____

Total: 40 Marks Time: 1 Hour 30 Minutes
Instructions and information

1. This question paper consists of TWO sections.
2. Answer ALL the questions.
3. Keep the question numbers correctly as used in this question paper.
4. Non-programmable and non-graphical calculators may be used.
5. All calculations must be clearly shown.
6. Write neatly and legibly.
SECTIONS A (15 Mark)

Stoichiometry multiple choices test on Conceptual Reasoning.
Four options are provided as possible answers to the following questions. Each question has only ONE correct an-
swer. Only circle boldly the letter (A–D) next to the question number. Give reason(s) for the option you chose. Three
marks will be awarded for each question, one mark for choosing a correct option, and two marks for the justification.
1. Calculate the number of moles of oxygen atoms in 35.2 grams of oxygen.

A. 2.20 moles
B. 4.42 moles
C. 0.54 moles
D. 2.57 moles (1)
Give reason for your answer (justify) ______________________________________________________________

_____________________________________________________________________________________________
___________________________________________________________________________________________ (2)
2. What is the mass of 0.432 moles of C8H9O4?

A. 86.9g
B. 391g
C. 113.8g
D. 73.0g (1)

Give reason for your answer (justify) _______________________________________________________________
_____________________________________________________________________________________________
___________________________________________________________________________________________ (2)
3. In the reaction 2CO(g) + O2(g) → 2CO2(g), what is the ratio of moles of oxygen used to moles of CO2 pro-
duced?
A. 1:1
B. 2:1
C. 1:2
D. 2:2 (1)

Give reason for your answer (justify) _______________________________________________________________
_____________________________________________________________________________________________
___________________________________________________________________________________________ (2)
629
(pp. 615-637) ISSN 2538–7138 /Online/
4. Which of the following is true about the total number of reactants and the total number of products in
the reaction shown below?
C5H12(ℓ) + 8O2(g) → 5CO2(g) + 6H2O(g)
A. 9 moles of reactants chemically change into 11 moles of product.
B. 9 grams of reactants chemically change into 11 grams of the product.
C. 9 litres of reactants chemically change into 11 litres of product.
D. 9 atoms of reactants chemically change into 11 atoms of product. (1)
Give reason for your answer (justify) ______________________________________________________________
_____________________________________________________________________________________________
__________________________________________________________________________________________ (2)
5. When two substances react to form products, the reactant which is used up is called the ____.
A. determining reactant
B. limiting reactant
C. excess reactant
D. catalytic reactant (1)
Give reason for your answer (justify) ________________________________________________________________

_______________________________________________________________________________________________
____________________________________________________________________________________________(2)
[15]
SECTIONS B
Problem Solving Skills Questions on Stoichiometry. Five marks each, a total of 25 marks.
Instructions:
1. Show all the steps in your calculations of the following problems.
2. Don’t round off until the very last answer.
3. Do not forget to write the units. Answer the questions on the lines provided below.
4. Make sure you are working with a properly balanced equation where necessary.
Question 1
Hydrogen sulfide reacts with sulfur dioxide to give H2O and S, balancing the equation;
H2S(g) + SO2(g) → H2O(ℓ) + S(s).
If Hydrogen sulfide contains 125 g, how much S(s) is produced? (5)
Question 2
How many grams of Na are required to react completely with 75.0 grams of Chlorine using this reaction: 2 Na +
Cℓ2 → 2 NaCℓ. (5)
Question 3
If 50 cm3 of silver nitrate solution with a concentration of 0.2 mol/dm3 is added to 100 cm3 of a sodium chloride
solution with a concentration of 0.5 mol/dm3. How many moles of silver nitrate and sodium chloride were present
in the solutions? What is the limiting reactant?
AgNO3(aq) + NaCℓ(aq) → AgCℓ + NaNO3(aq) (5)
Question 4
How many liters do 3.8 moles of O2 occupy at STP (standard temperature and pressure)?
NOTE: At STP, for 1 mole of any gas, molar volume = 22.4ℓ. temperature = 273K (0 C) and pressure = 1 atm
(1.013 x 105). (5)
630
ISSN 2538–7138 /Online/ (pp. 615-637)
Question 5
When 12.8g Cu is allowed to burn in oxygen, 15.2g copper (II) oxide is produced. Balance the chemical equation
for the reaction that occurs; Cu(s) + O2 (g) → CuO(s).
Determine the percentage yield using the formula below;
Percentage yield = x 100%

(5)
[25]
Marking Guidelines for Learner Achievement Test (LAT)

Learner Code_______________ School Code_________ Date________ Class______
Total: 40 Marks Time: 1 Hour 30 Minutes
SECTIONS A (15 Mark)

Stoichiometry multiple choices test on Conceptual Reasoning.
1. A
Justification: Using the knowledge of mole relating it to the mass and molar mass of Oxygen (1mol = 16g/mol,).
n = m/M O = 35.2g/16 = 2.2 mol
2. D
Justification: Know calculation of molar mass, use relation between mass & moles n = m/M
so, m = n x M = 0.432 x 169 = 73g because C8H9O4 = 169g/mol
3. C
Justification: Know the mole ratio between O2 and CO2 produced. Also, be able to identify the reactant and the
product and their coefficient. O2 : 2CO2 = 1:2
4. A
Justification: Know total moles (amount of substance) for reactants and products and the difference between
moles, mass, volume, and atoms. 1 + 8 = 5 + 6 thus 9:11
5. B
Justification: Know the concept of limiting reagent and being able to define it as the substance that is totally used
up when the chemical reaction is complete. And that it determines how many moles of a product should be formed.
Marking criteria for SECTION A: 3 MARKS EACH

Question Concepts tested Objective choice Conceptual reasoning
1 mark NCR = 0 PCR = 1 GCR = 2
1 Mole Calculation
2 Mole Calculation
3 Mole Ratio
Distinguish between Moles, Atoms, Mass,
4
Volume, etc.
5 Limiting Reagent
NCR- No Conceptual Reasoning, PCR- Partial Conceptual Reasoning and

GCR- Good Conceptual Reasoning.
NCR PCR GCR
No reason at all Partially valid reason Sound conceptual reasoning
Incorrect reason Incomplete reason Complete valid reason
Imprecise reason Precise reason
631
(pp. 615-637) ISSN 2538–7138 /Online/
SECTION B (25 marks) Problem Solving Questions on Stoichiometry

Criateria: Formula/ Equation (1)
Substitution (1)
Method/Evaluation (2)
Correct Answer (1)
Q1. Solution:
Balanced reaction: 2 H2S + SO2 → 2 H2O + 3 S (solid). ü
n(H2S) = = x 2ü = = 1.8382mol ü
n(S) = 3 × 1.8382 = 5.5146mol ü
mass(S) = 5.5146 × 32 = 176.47g ü
OR
2 H2S + SO2 → 2 H2O + 3 S (solid). ü
n(H2S) = = = 3.676molü
mole ratio is 2:3ü
n(S) = 3/2 × 3.676 = 5.5146mol ü
mass(S) = 5.5146 ×32 = 176.47g ü
OR
2 H2S + SO2 → 2 H2O + 3 S (solid). ü
125g x
68g/mol 96g/mol
X= = 176.47g
OR
2 H2S + SO2 → 2 H2O + 3 S (solid). ü
n(H2S) = = = 3.676 molü
mole ratio is 2:3ü
n(S) = 3:2 = X:3.676
2X = 3.676 x 3
n(S) = 5.5146 mol ü
mass(S) = Nm
= 5.5146 × 32
= 176.47g ü
Q2. Solution:
Moles(Cℓ2) = = ü =1.06 molü
From mole ratio: 2 : 1ü
So, 2 x 1.06mol = 2.12 mol of Naü
2.12 mol × 22.99 g/mol = 48.7 gü
OR
2Na(g) Cℓ2 ü
46g/mol 71g/molü
X 75gü
X= ü= 48.6gü
632
ISSN 2538–7138 /Online/ (pp. 615-637)
Q3. Solution:
n = cv = 0.2(50 ÷ 1000) = 0.01 mol AgNO3 ü
n(NaCℓ) = 0.5(100 ÷ 1000) = 0.05 mol. ü
From mole ratio; 1mole of AgNO3 : 1mole NaCℓ
So, 0.01mole of AgNO3 : 0.01mole of NaCℓ
Therefore, if all the AgNO3 is used up, there is still 0.05 – 0.01 = 0.04mol NaCℓ left. ü
Hence, AgNO3 is the limiting reagent. ü
Q4. Solution:
Volume = n × Vmü
= 3.8 molü × 22.4L ü
= 85.12 litres of O2 üü
OR
pV =nRTü
1.013 x 105 x Vü = 3.8 x 8.31 x 273 ü
V= = 0.085m3 ü
0,085 x 1000 = 85Lü
At STP, 1 mol of any gas = 22.4dm3 or 22.4L, T = 273K (0°C) and 1 atm
Q5. Solution;
2Cu(s) + O2 (g) → 2CuO(s). ü
M(CuO) = 63.5 + 16 = 79.5 g/mol
M(Cu) = 63.5 g/mol
n= = = 0.2 mol Cu ü
mole ratio = 2 : 2 ü
0.2 mol Cu : 0.2 mol CuO
So, m = nM = 0.2(79.5) = 15.9g CuO should theoretically be formed. ü
= x 100% = 95.59% ü
OR
m(2Cu) = 127g/mol
n= = 12.8/127 = 0.1molü
m(2CuO) = 79.5 x 2 = 159g/molü
n(Cu) = n(CuO) ü
n= = 0.1 x 159 = 15.9ü
= x 100% = 95.59% ü
633
(pp. 615-637) ISSN 2538–7138 /Online/
Appendix B
Extracts of Marked Learner Test Script
634
ISSN 2538–7138 /Online/ (pp. 615-637)
635
(pp. 615-637) ISSN 2538–7138 /Online/
636
ISSN 2538–7138 /Online/ (pp. 615-637)
Received: March 26, 2022 Revised: June 16, 2022 Accepted: August 05, 2022
Cite as: Kotoka, L., & Kriek, J. (2022). Prediction of the correlation between problem-solving skills and conceptual reasoning
in stoichiometry. Journal of Baltic Science Education, 21(4), 615-637. https://doi.org/10.33225/jbse/22.21.615
Love Kotoka PhD Student in the Department of Physics, Eureka Building, Science
Campus, Florida, South Africa.
E-mail: kotokalove@gmail.com
Jeanne Kriek PhD, MSc, Professor, University of South Africa, Department of Physics,
(Corresponding author) Eureka Building, Science Campus, Florida, South Africa.
E-mail: kriekj@unisa.ac.za
Website: https://sites.google.com/site/jeannekriek
637
UNDERGRADUATE STEM
AND NON-STEM STUDENTS’
INTERPRETATION OF MEAN IN
AN INFOGRAPHIC
Abstract. The use of infographics for

Zorana Lužanin,
presenting data in the media and on
the Internet has become a widespread
Marija Kaplar,
phenomenon. This study examines how Tatjana Hrubik-Vulanović
well undergraduates interpret the mean
when presented in infographic, their
attitudes towards the misuse of statistics,
and their perceptions of their knowledge
and use of statistics. The infographic
was intended for the general public. The Introduction
questionnaire was developed in a pilot
study and then it was presented to a The understanding of statistical information is essential in many aspects
sample of 270 students from universities of everyday life. Individually and collectively, people make choices based on
in Serbia and the USA. The study showed the analysis of data presented in common consumer practices as well as in
that STEM undergraduates and those who social and political debates, where data analysis becomes an important tool
attended a course in statistics interpreted for active citizenship (Carvalho & Solomon, 2012; Maass et al., 2019). Public
the infographic better. However, between health and climate change are two examples of areas where the wrong
46% and 65% of those misinterpreted the data interpretations may cause huge negative consequences (Faghmous &
infographic which indicates that changes Kumar, 2014; Park et al., 2021). In many professions, the ability to interpret
are needed in statistics education to improve data is crucial and the desired competencies are recommended by profes-
statistical literacy. Concerning demographic sional organizations, governments, and research papers (AlMuraie et al.,
characteristics, somewhat better results were 2021; Dönmez et al., 2022; Erydice, 2016). Zilinski et al. (2014) summarized
achieved by the American students, while the required skills as follows: “identify problems through data collection, ap-
gender had weak significance. Students ply logical processes to analyze information and draw conclusions, identify
whose interpretations were more successful inconsistent or missing information, critically review, analyze, synthesize,
perceived their knowledge and use of compare and interpret the information” (p.2). The 21st-century skills in the
statistics as better. An extremely low number domain of mathematics include techno-mathematical literacies which consist
of students, 7.5%, disagreed that statistics of mathematical knowledge, workplace and software knowledge, multi-step
is often misused, while only 14.2% trusted calculation and estimation, the ability to interpret abstract data, and com-
the research results presented in the media. munication skills (Hoyles et al., 2002, Hoyles et al., 2013). Data literacy, as a
The results from this study can be useful category of techno-mathematical literacies, proposes that the students must
for curriculum developers, teachers, and be able to analyze and interpret technical data and graphical representations,
researchers on statistical education. draw conclusions and take action accordingly (Wal et al., 2019).
Keywords: infographics, mean The ability to critically interpret the data is being developed at all levels
interpretation, non-STEM students, of education but students in higher education as future leaders and decision-
statistical literacy, STEM students, students’ makers should be especially well prepared and confident in their knowledge.
attitudes STEM (Science, Technology, Engineering, and Math) undergraduates typically
take statistics courses (Kaplar et al., 2021; Zilinski et al., 2014), but non-STEM
(Social Sciences) undergraduates also get training in interpreting the data
Zorana Lužanin, Marija Kaplar and statistics. Standardized tests are often used to evaluate their knowledge
University of Novi Sad, Serbia of statistics gained in statistics classes (Chan & Ismail, 2014; Garfield, 2003;
Tatjana Hrubik-Vulanović
Kent State University at Stark, USA Maddens et al., 2021). In comparison, this study aims to evaluate how well
their skills transfer outside of their field. The students were asked to analyze
data about the basic statistical concepts of distribution and mean presented
638
ISSN 1648–3898 /Print/ UNDERGRADUATE STEM AND NON-STEM STUDENTS’ INTERPRETATION OF MEAN IN AN
INFOGRAPHIC
ISSN 2538–7138 /Online/ (pp. 638-650)
in the infographic. The variability is a fundamental concept in statistics so the concept must be understood well
(Franklin et al., 2007). The mean is often calculated and interpreted as a center and/or mode which, depending on
the distribution, may be wrong. The misinterpretation of the mean with the respect to typicality and distribution
happens the most in everyday life (Jacobbe & Carvalho, 2011; Leavy & Middleton, 2011). This is not surprising because
even though calculating the mean is not a very demanding task, the higher-order tasks that include understand-
ing and interpretation are challenging (Cooper & Shore, 2008). The misconceptions or superficial understanding
of measures of the center have been cited at all levels of education (Cai et al., 2000; Chatzivasileiou et al., 2011;
Cooper & Shore, 2008; Mokros & Russell, 1995).
When the data is presented visually that may introduce an additional level of difficulty. Many studies show that
even students in primary and secondary schools had difficulties in reasoning about the data when it was provided
in visual form (Friel et al., 2001; McClain, 1999; Watson et al., 2003; Watson & Moritz, 2000). Undergraduate students
also had difficulties in understanding the mean and variability when the data was presented in graphical form
(Cooper & Shore, 2008; Watson & Moritz, 2000). On the other hand, graphical representations often attract the at-
tention of the reader and are increasingly used in many professions and everyday life (Jimerson et al., 2019; Pardo,
2018). Educators emphasize the importance of teaching students to interpret visual communication well (DiStaso
et al., 2009; Kent et al., 2011; Kim & Chung, 2012; Manalo et al., 2013). Infographics are a type of visual interpreta-
tion of the data intended to present the information quickly, and in an accessible form (Delello & McWhorter, 2015;
Sudakov et al., 2016). A good infographic tells the story according to data and attracts a reader to the presented
data and information (Siricharoen, 2013). The wide usage of infographics is not questionable but the research on
an understanding of infographics is relatively new (Siricharoen, 2013). Although there are studies that examine
the student’s understanding of the mean which is given in the visual form, no studies have been found on the
interpretation of the mean presented in infographics as is the case in this study. Also, the peculiarity of this study is
that the infographics used are given in a real context, i.e., the original infographics published by official institutions,
intended for the general public. As the infographic was published for the general public, the students were also
asked how much they trusted the media, and how confident they were in their knowledge and use of statistics.
The students from Serbia and the USA were included in the study. In Serbia, there is a mandatory 8-year
primary and lower secondary education which, in the end, has a standardized national examination. The students
then pick the upper secondary school which can be three or four years long. The three-year upper secondary
schools are considered vocational. Only students who complete 4-year upper secondary school can apply to the
university. There is no examination at the end of upper secondary schools. Serbia joined Bologna Convention for
Higher Education in 2003. This means that the students at the beginning of their study choose one of the nation-
ally accredited programs and after they finish it, they get an appropriate recognition of academic qualification.
In the USA the children first complete a 5-year primary and 3-year lower secondary education. The curriculum
is based on a state and county level, so significant differences in covered topics between schools exist. Upper sec-
ondary schools last 3-4 years. Because the education levels may vary depending on the upper secondary school
attended, some colleges require their prospective students to take standardized tests such as SAT and ACT. All
students in the study completed a 4-year upper secondary school before they started their studies at the university.
Due to the differences in culture and educational systems between Serbia and the USA, in this study, the
students’ interpretations of the infographic and their attitudes about the misuse of data and perception of their
statistical knowledge were analyzed by the country. Other factors considered are STEM and non-STEM orientation,
gender, and previous training in statistics. Although both STEM and non-STEM students usually have some training
in data analysis and/or statistics, STEM students often have a greater need to understand and apply concepts related
to data processing and statistics in their profession (Kaplar et al., 2021; Zilinski et al., 2014). It is expected that STEM
students possess higher competencies in that area. However, it is uncertain how STEM students interpret data in
everyday life, and that is why students’ orientation was included in this study. Some earlier studies have found
that training in statistics can have positive effects on students’ achievements in data interpretation (DelMas et al.,
2007; Gauvrit & Morsanyi, 2014; Gigerenzer et al., 2007; Masel et al., 2015). That is why a previously taken course
in statistics was included as a factor. Although the students from different fields take different statistical courses,
they should be able to interpret the data intended for the general public.
The research questions were:
•• How well do the students interpret the statistical data presented in the infographic and are there any
differences in students’ interpretation by the factors (STEM and non-STEM orientation, country, gender,
and training in statistics)?
639
INFOGRAPHIC
(pp. 638-650) ISSN 2538–7138 /Online/
•• How much do students trust the reporting in the media, believe that statistics is misused, and how do
they perceive their knowledge and use of statistics? Are there any differences in their attitudes by the
factors (STEM and non-STEM orientation, country, gender, and training in statistics)?
General Background
In this cross-sectional exploratory study, the current ability of undergraduates to interpret statistical data has
been evaluated. Although the sample was large to include undergraduates from different study orientations and
from two countries, for more general results, the sample must be even larger and include more countries. Students
were asked to analyze the mean because the mean is a commonly used statistical measure that all undergraduates
know about as opposed to many other statistical topics that require specific training in statistics. The mean was
presented in infographics to attract the students’ attention. The infographics are also commonly used, and the
students should be able to interpret them well.
To answer the research questions, the study was conducted in two phases. In the first phase, the question-
naire was developed in a pilot study. The questionnaire consisted of demographic data, a four-question about
the statistical infographic, statements about the students’ attitudes related to the possible misinformation in the
media reporting and misuse of statistics, and statements about students’ perceptions of their knowledge and use
of statistics. In the second phase, the questionnaire was filled out in the fall term of the academic year 2019-2020
at two public universities in Serbia and USA. Participants were STEM and non-STEM students.
Pilot Study
The pilot study was conducted during the spring term in 2018 at the University of Novi Sad, Serbia. 35 students
who were in their final year of study in teaching mathematics were asked to choose one of the nine infographics
about the population in Serbia and write an essay based on the chosen data. All participants previously completed
the course in probability and statistics and were expected to be quite skilled in reading and interpreting the data.
The infographics were published by the Statistical Office of the Republic of Serbia, and they cover different aspects
of the population such as births, deaths, marriages, and divorces. The infographics can be viewed at the link1 below.
The pilot study was qualitative because it provided insight into students’ thinking (Bogdan & Biklen, 1992; Groth,
2005). The purpose was to see how these students interpreted the public data. The greatest number of students
chose the infographic about marriages and divorces. The infographic presents the number of new and divorced
marriages, the average age of men and women at first marriage, the average age of men and women at divorce, and
the average length of the divorced marriage. The data is from the year 2016. The infographic is shown in Figure 1.
Figure 1
Infographic about Marriages and Divorces for 2016.

1
https://github.com/marijakaplar/Infographics
640
INFOGRAPHIC
ISSN 2538–7138 /Online/ (pp. 638-650)
The qualitative analysis of the students’ essays showed that out of nine essays, six included unfounded con-
clusions, while five had wrong conclusions. Another misconception was missing the fact that the data is reported
for one year only (for the year 2016). The high number of wrong or unfounded conclusions indicates that even
students who are majoring in mathematics tend to conflate the average with the mode. The examples of students’
conclusions are in Table 1.
Table 1
The Examples of Students’ Conclusions
Unfounded conclusions Wrong conclusions
•• Based on the demographic statistics of Serbia, the person should marry as •• In 2016, out of 35921 marriages, 9046 got divorced.
late as possible, so that the person is mature enough to pick a partner well •• Based on the results presented, already after 13 years, divorce
and thus reduce the number of divorces. follows.
•• In the present times, the people got more freedom, so they marry more often
but they divorce often too.
•• As the average age for men is 31.2 years and for women 28.2, we can conclude
that divorces are not due to marrying too early.
The students’ essays related to the infographic were analyzed and the most frequent poor interpretations were
formulated as statements. Four of the most frequent incorrect statements are presented in Table 2.
Table 2
Students’ Statements about the Infographic
No Students’ Statements
Q1. In 2016 were 35921 marriages of which 9046 ended up in divorce.
Q2. The average age at first marriage for men is 31.2 and for women, it is 28.2. Based on that we can conclude that divorces are not
the consequence of getting married at a young age.
Q3. Most men who got married had around 31.2 years and women had around 28.2 years.
Q4. The average marriage lasts 13.3 years. Women most often get divorced at 40, while men get divorced at 43.5 years.
Q1 is about the relation and dependency between two numbers, Q2 is about the interpretation of the mean,
while Q3 and Q4 are about the difference between the mean and the mode. These four false statements were
included as True/False questions in the survey questionnaire. Since in True/False questions some answers may
be correct by chance, an open-ended paragraph was added to each question where students had to justify their
choice. This provided insight into students’ answers and their reasoning (Hubbard et al., 2017; Kaplar et al., 2021)
instructors must understand the affordances and limitations of available question formats. Here, we use a crossover
experimental design to identify differences in how multiple-true-false (MTF. The reliability of the four questions
was measured by Chronbach’s Alpha. For completed 270 surveys, Chronbach’s Alpha was .66, which is adequate
(Taber, 2018).
The four True/False questions about infographics represent the main part of the questionnaire but four 5-point
Likert questions were added to gain insight into students’ attitudes towards the research presented in the media,
possible misuse of statistics, and students’ perception of their statistical knowledge. The statements evaluated on
the Likert scale are presented in Table 3. Likert scale used is 1 (strongly disagree), 2 (disagree), 3 (neutral), 4 (agree),
and 5 (strongly agree).
641
INFOGRAPHIC
(pp. 638-650) ISSN 2538–7138 /Online/
Table 3
The Statements about Students’ Attitudes
No Statements
S1 I believe the research results presented in the media.
S2 Statistics is often misused.
S3 My statistical knowledge helps me to critically read the results of published research.
S4 I use my statistical knowledge to better interpret any data.
S1 and S2 are about the possible misuse of statistics in media while S3 and S4 are about students’ percep-
tions of their knowledge and use of statistics. The established questionnaires about the students’ attitudes toward
statistics such as the Attitudes Toward Statistics Scale (Wise, 1985) or Survey of Attitudes Toward Statistics Scale
(Schau et al., 1995) consist of various sub-scales which measure a wide range of students’ attitudes. Some of these
subscales exceed the need of this explanatory study and that is why the number of questions related to students’
attitudes and perceptions is limited to four. The general questions such as information about students’ gender,
and whether students took at least one course in statistics during their study at the university were also added to
the questionnaire.
The questionnaire was tested at the end of the pilot study for clarity and completeness. 43 students completed
the questionnaire. These students did not take part in the main research. The questionnaire was recognized as clear
and interesting so minimal changes were made before the main research in phase two.
Sample
The sample consisted of 285 students from whom 270 were included in the study. About 6% of the sample
were excluded from the study due to blank or incorrectly completed questionnaires. The sample was collected
in the academic 2019-2020 year. The sample size was determined by the procedure of the sample-variable ratio
increased by 20% due to the possible withdrawal of students. In the study, the sample-variable ratio was 60:1,
which was higher than the minimum of 20:1 (Hair et al., 2018; Memon et al., 2020). 55.6% of the participants are
from Serbia (University of Novi Sad) while 44.4% of participants are from the USA (Kent State University). The con-
venience sampling was implemented as the instructors from different fields conducted the survey in their classes
voluntarily but because STEM and non-STEM orientation of students was expected to make a difference, every
attempt was made to ensure that both orientations are represented well. The sample is presented in Table 4 by
the factors of STEM and non-STEM orientation, gender, and country.
Table 4
The Sample
State
Orientation Gender Total gender Total orientation
Serbia USA
Female 17 18 35
STEM 88
Male 33 20 53
Female 66 55 121
Non-STEM 182
Male 34 27 61
Total 150 120 270
642
INFOGRAPHIC
ISSN 2538–7138 /Online/ (pp. 638-650)
Out of 150 students from Serbia, 55.3% are female, while 44.7% are male. 33.3% are STEM students recruited
from university departments related to electrical engineering. 66.7% are non-STEM students recruited from uni-
versity departments related to economy and legal studies.
Out of 120 students from the USA, 60.8% are female, while 39.2% are male. 31.7% are STEM majors and 68.3%
are non-STEM majors.
Procedure
The research for the participants in the USA was approved by Kent State Institutional Review Board, Kent
State University, IRB log number: 19-047. In Serbia, the office of Mathematics vice-dean approved the research.
The approval was also obtained from the offices of vice-deans of the colleges where the study was conducted.
The survey in both countries included a paragraph about the purpose of the study and voluntary and anonymous
participation. Data were collected during the 2019-2020 academic year, as a voluntary activity before, after, or dur-
ing regular college classes at both universities. Anonymity was guaranteed to all students, and no rewards were
offered for the survey completion. The data were collected by researchers or by trained instructors who explained
the purpose of the research to students and who ensured that there were no instances of collusion. Most students
completed the questionnaire well before the 20-min time limit.
Data Analysis
Three researchers independently reviewed and coded the student answers. The discrepancies were found
in 3% of the questionnaires and these were reviewed and resolved. All four statements in the questionnaire were
false. To count the answer as correct, the student had to circle “False” and give a good justification for his/her choice.
Answers with “False” circled and with the wrong justifications were counted as wrong. Answers with “False” circled
and no justification were omitted. There were only a few omitted answers. If the student marked the statement
as “True”, it counted as a wrong answer regardless of the existence of justification. This was done because in the
pilot study was found that when students marked the statement as “True”, they usually gave typical comments
such as “It is obvious” or “I see it in the infographics”. For this reason, justification wasn’t required for the statements
marked “True”.
Students’ answers per question were analyzed as a function of the students’ STEM or non-STEM orientation,
country, gender, and an earlier completed course in statistics. Logistic regression was conducted for the analysis of
questions. Students’ attitudes were analyzed using a student’s t-test. In both cases, the R programming language
was used for statistical computing (Version 3.5.1).
Research Results
Students’ Interpretation of the Infographic
The total percent of correct answers for each question is 37.4% for Q1, 20% for Q2, 29.3% for Q3, and 25%
for Q4 (Table 5).
Table 5
Students’ Correct Answers with Correct Justification
Serbia Serbia USA USA Total Total

Total
non-STEM STEM non-STEM STEM non-STEM STEM
% % % % % % %
Q1/correct 21.0 50.0 40.2 57.9 29.7 53.4 37.4
Q2/correct 3.0 22.0 32.9 34.2 16.5 27.3 20.0
Q3/correct 21.0 46.0 20.7 47.4 20.9 46.6 29.3
643
INFOGRAPHIC
(pp. 638-650) ISSN 2538–7138 /Online/
Serbia Serbia USA USA Total Total

Total
non-STEM STEM non-STEM STEM non-STEM STEM
% % % % % % %
Q4/correct 19.0 34.0 20.7 36.8 21.4 35.2 25.0
Average 16.0 38.0 28.7 44.1 22.1 40.6 27.9
When students marked the correct answer and provided a wrong justification such answers were counted
as wrong. There were 8 correct answers with the wrong justification in Q1, 28 in Q2, 20 in Q3, and 28 in Q4. The
wrong justifications were caused either by the wrong mathematical conclusion or by justifying the answer based
on personal belief rather than data. Justifications based on personal beliefs occurred mostly in Q2, Q3, and Q4.
Students’ Interpretations of the Infographic with Respect to Factors
The analysis of the students’ answers per question with respect to STEM and non-STEM orientation, country,
gender, and statistics courses taken is presented in Table 6. The difference between STEM and non-STEM students
was statistically significant in all questions except Q2. STEM students did significantly better in Q1, Q3, and Q4.
The differences between countries were statistically significant in Q1 and Q2 where the students from the USA
did better. Regarding gender, the only statistically significant difference was in Q2 where males did better than
females. Students who earlier had a statistics course did better in Q1 and Q2.
Table 6
Students’ Correct Answers by the Question, with Respect to Factors
Factor Model Q1 Model Q2 Model Q3 Model Q4
B B B B
Odds Odds Odds Odds
(SE) (SE) (SE) (SE)
Orientation
non-STEM -0.928*** -0.373 -1.076*** -0.566***
0.395 0.689 0.341 0.568
(0.297) (0.363) (0.306) (0.316)
Country
Serbia -0.688*** -1.901*** -0.017 -0.041
0.503 0.150 0.983 0.960
(0.267) (0.387) (0.281) (0.286)
Gender
Male -0.220 0.967*** -0.042 0.050
0.803 2.631 0.959 0.951
(0.280) (0.360) (0.292) (0.297)
Course
No -0.575** -1.250*** -0.407 -0.435
0.563 0.287 0.666 0.647
(0.277) (0.350) (0.294) (0.301)
Const 0.886 -0.159 0.054 -0.454
(0.330) (0.396) (0.335) (0.343)
Goodness of fit Statistic p Statistic p Statistic p Statistic p
-2LL 330.7 <.001 221.1 <.001 306.2 <.001 298.6 <.001

HL 5.1 .648 6.923 .437 8.8 .265 2.2 .904
Note. B- parameter estimate, SE-Standard error, Const.-intercept, -2LL-2Log-likelihood statistic (deviance), HL- Hosmer and Leme-
show’s test, Statistic-value of chi-square statistic.
*p < .05. **p < .01. ***p < .001.
Overall, for all four questions, the average of correct answers for non-STEM students was 1.17 while for STEM
students was 2.02. A Kruskal-Wallis test showed that such a difference is statistically significant H(2)= 25.814, p < .001.
644
INFOGRAPHIC
ISSN 2538–7138 /Online/ (pp. 638-650)
Students’ Attitudes with Respect to Factors
The four statements about students’ attitudes and percentages related to them are presented in Figure 2.
In statement 1, “I believe the research results presented in the media.”, 39.9% of students disagreed or strongly
disagreed, 45.9% were neutral, while only 14.2% agreed or strongly agreed. There was no statistical difference
between students’ scores on the four-question assessment about the infographic and positive or negative student
attitudes presented in S1 (t = 0.70, p = .48).
In statement 2, “Statistics is often misused.”, 61.9% of students agreed or strongly agreed, 30.6% were neutral,
while only 7.5% disagreed or strongly disagreed. These potentially alarming results showed the prevalence of nega-
tive perceptions of the media and the use of statistics in the media. There was no statistical difference between
students’ scores on the four-question assessment about the infographic and positive or negative student attitudes
presented in S2 (t = 1.92, p = .056).
Students were mostly confident about their knowledge of statistics. In statement 3, “My statistical knowledge
helps me to critically read the results of published research.” 45.7% agreed or strongly agreed, 41.6% were neutral,
while only 12.7% disagreed or strongly disagreed. The generally positive attitude towards their knowledge of sta-
tistics was confirmed in statement 4, “I use my statistical knowledge to better interpret any data.”, where 51.9% of
students agreed or strongly agreed, 30.6% were neutral, while 17.5% disagreed or strongly disagreed. There was
a statistical difference between students’ scores on the four-question assessment about the infographic and posi-
tive or negative student attitudes presented in S3 (t = 2.04, p = .043) and S4 (t = 3.08, p = .002). Students who had
a positive perception of their knowledge also had higher scores on the four-question assessment than students
with negative attitudes.
Figure 2
Students’ Attitudes toward Misuse of Data and Students’ Perception of their Knowledge
Note. Positive – students’ answers agree (4) or strongly agree (5), neutral – students’ answers neutral (3), negative – students’
answers disagree (2) or strongly disagree (1)
The analysis of student attitudes with respect to different factors is presented in Table 7. In statement 1, “I
believe the research results presented in the media.”, there was a statistical difference between countries, and
based on gender. Students from Serbia were more suspicious of the media than students from the USA. Males
were more suspicious than females.
In statement 2, “Statistics is often misused.”, there was a statistical difference between STEM and non-STEM
orientation, gender, and previously taken courses in statistics. STEM students thought that statistics is misused
more than non-STEM students. Males thought that statistics is misused more, as well as students who previously
took a course in statistics.
In statement 3, “My statistical knowledge helps me to critically read the results in published research.”, there
was a statistical difference between countries and previously taken course in statistics. Students from the USA were
645
INFOGRAPHIC
(pp. 638-650) ISSN 2538–7138 /Online/
more confident in their knowledge of statistics than students from Serbia. Students who previously took a course
in statistics were also more confident in their knowledge of statistics.
In statement 4, “I use my statistical knowledge to better interpret data.”, there was a statistical difference
between countries and previously taken courses in statistics. Students from the USA thought that they used their
knowledge of statistics more than students from Serbia, as well as students who previously took a course in statistics.
Table 7
Students’ Attitudes on Four Statements with respect to Factors
Factor Group Statement S1 Statement S2 Statement S3 Statement S4
M M M M
p p p p
(SD) (SD) (SD) (SD)
Orientation
2.52 4.2 3.53 3.39
STEM
(0.97) (0.87) (0.83) (1.06)
.283 <.001 .235 .120
2.65 3.61 3.39 3.60
non-STEM
(0.91) (0.94) (0.94) (0.93)
Country
2.36 3.89 3.33 3.29
Serbia
(0.9) (0.95) (0.96) (1.12)
<.001 .088 .033 .001
2.92 3.69 3.57 3.68
USA
(0.88) (0.96) (0.82) (0.83)
Gender
2.75 3.65 3.34 3.37
Female
(0.84) (0.95) 0.89 (1.01)
.004 .003 .052 .075
2.41 4.02 3.57 3.59
Male
1.01 (0.93) (0.93) (1.02)
Course
2.61 4.13 3.62 3.36
Yes
(0.94) (0.83) 0.81 (1.09)
.962 <.001 .011 .033
2.61 3.6 3.32 3.61
No
(0.92) (0.98) (0.95) (0.89)
Discussion
The four-question assessment about infographics measured how well undergraduates interpret the mean when
they analyze the data published for the general public. The percent of correct answers that were well justified was
low for all four True/False questions and that is a big concern. In Q1 there were 37% correct answers that also had a
correct justification, 20% in Q2, 39% in Q3, and 25% in Q4. The low percentage of correct answers and the analysis
of students’ justifications of their choices suggest that, when reading infographics, students often misinterpret
the mean and variance and/or conflate the mean and the mode. The lack of a deep understanding of the mean is
a pervasive problem. Previous studies have shown that undergraduates are prone to mistakes when interpreting
the mean given in a visual form through histograms, stem-and-leaf plots, and bar graphs (Cooper & Shore, 2008;
Cui & Liu, 2021; Watson & Moritz, 2000). Similar to Kaplar (2022), superficiality and reasoning based on personal
beliefs rather than analyzing the presented data were observed in this study as well. Weak skills in interpreting
the mean were also reported in lower secondary and upper secondary education (Watson & Moritz, 2009). This
implies that improvement in teaching statistics is needed at all educational levels, not only in higher education.
The lack of knowledge (in this case statistical knowledge), misunderstandings, misconceptions, and a super-
ficial approach to answering questions are the barriers to critical thinking. To be able to think critically about the
statistical data, the students must be trained well. Although in this study, the students who previously took a course
646
INFOGRAPHIC
ISSN 2538–7138 /Online/ (pp. 638-650)
in statistics did better, the results were still disappointing because the scholarly knowledge was poorly transferred
to the real-life situation. Similar conclusions were made in other studies where real-life statistical context was used
(Castro Sotos et al., 2007; DelMas et al, 2007; Lavigne et al., 2008). How to ensure that students are trained well and
that the statistical course is effective? A well-designed course can have positive effects on overcoming misconcep-
tions (DelMas et al., 2007; Gauvrit & Morsanyi, 2014; Gigerenzer et al., 2007; Masel et al., 2015). The use of real-life
data, context, and examples as well as simulations and animations help students to overcome misunderstandings
of basic statistical concepts (Cui & Liu, 2021; Jamie, 2002; Kaplar, 2022; Neumann et al., 2011; Wang et al., 2011).
The infographics engaged students well, so they have a strong educational potential and should be included in
statistics courses.
Although different majors may include courses in statistics that differ in the topics covered, all undergradu-
ates should be better prepared for interpreting the statistical data intended for the public. Repeated studies like
this one should be able to detect the changes in students’ ability to analyze the data.
STEM students outperformed non-STEM students in all questions but the percentage of correct answers for
STEM students is still low. On the other hand, STEM students are increasingly expected to analyze information and
draw conclusions based on it (Hoyles et al., 2002; Hoyles et al., 2013; Kriesberg et al., 2013; Zilinski et al., 2014).
This brings up the significant gap between the students’ preparedness and the expectations of what they can do.
The differences between countries are significant in Q1 and Q2, both times in favor of the students from the USA.
Non-STEM students in Serbia are the weakest group and need the improvement most.
Infographic used in the study was effective in attracting the students’ attention so the use of infographics
should be prominent in statistics education. The infographics must leave as little room for misinterpretation as
possible. For example, infographics intended for the general population may include a text explanation along with
infographics that would dispel the most common misinterpretations.
In the age of big data, statistical literacy is crucial. Training in statistics is an integral part of many curricula
around the world and that is the reason why students’ attitudes towards statistics are of more interest as well
(Emmioǧlu & Capa-Aydin, 2012; Hilton et al., 2004; Judi et al., 2011; Nolan et al., 2012). The majority of published
studies investigated the students’ attitudes toward a certain course in statistics (Donohue & Richards, 2009; Griffith
et al., 2012; Mills, 2004; Rhoads & Hubele, 2000). In comparison, in this study, similar to Kaplar (2022), general stu-
dents’ attitudes were measured. The students generally do not trust the media and believe that statistics is often
misused. They are fairly confident in their knowledge and use of statistics when interpreting data. STEM students
believe that statistics is misused more than non-STEM students. Students from the USA are more confident about
their knowledge of statistics and claim that they use that knowledge more. Students from Serbia are less trusting
of the statistical reports in the media, and they feel that they use their knowledge of statistics less. Females trust
the media more than males. Males think that they use their knowledge of statistics more than females. Students
who previously took a statistics course at the university believe that statistics is often misused, and they believe
that they use their knowledge of statistics more.
The earlier studies have shown that the students’ attitudes toward data and statistics were often related to
student achievement (Fullerton & Umphrey, 2001; Papanastasiou, 2008; Tapia & Marsh, 2001). This was confirmed
in this study as well because students who did better in interpreting the infographic were also more confident in
their knowledge and use of statistics.
The results of this study show that undergraduates interpreted statistical data presented in the infographic
poorly. On the four-question assessment about the mean, on average, there were only 27.9% correct answers. Since
the infographic represented the data intended for the general public, the results are even more alarming. They call
for more and better training in statistics and critical thinking at all educational levels with a special emphasis on
STEM orientation where the gap between the students’ abilities and expectations of what they can do is the largest.
On some questions, STEM students performed better than non-STEM students, the USA students performed
better than students from Serbia, and students, who previously had a course in statistics, performed better than
students who did not.
With the respect to students’ attitudes, the students mostly did not trust the media and believed that statistics
is often misused. They were fairly confident in their knowledge and use of statistics when interpreting data. Students
from the USA were less suspicious of media reporting, and they were more confident in their knowledge and use of
647
INFOGRAPHIC
(pp. 638-650) ISSN 2538–7138 /Online/
statistics than students from Serbia. Females were less suspicious of media reports and misuse of statistics. Males
were more confident in their knowledge of statistics.
The general conclusion is that the skill of critically reading and interpreting statistical data needs to be de-
veloped more in both STEM and non-STEM students. The infographics that represent statistical data are effective
at attracting attention and are expected to be used even more in the future both at work and in everyday life. The
students need to be trained to read and interpret them well. As the students increase their knowledge and general
skills, it can be expected that their confidence in the knowledge and use of statistics will grow.
Limitations
This study was conducted with a limited number of students and only at two universities. For cross-cultural
differences, more universities and more countries need to be included. The students analyzed only one infographic
and that is a limitation. The number of factors taken into the consideration is limited to STEM vs non-STEM, country,
gender, and previously taken courses in statistics. Similar studies can be conducted at different educational levels.
References
AlMuraie, E., Algarni, N. A., & Alahmad, N. Sh. (2021). Upper-secondary school science teachers’ perceptions of the
integrating mechanisms and importance of stem education. Journal of Baltic Science Education, 20(4), 546-557.
Bogdan, R., & Biklen, S. (1992). Qualitative research for education: An introduction to theory and methods. Allyn and Bacon.
Cai, J. (2000). Understanding and representing the arithmetic averaging algorithm: An analysis and comparison of the US and
Chinese students’ responses. International Journal of Mathematical Education in Science and Technology, 31(6), 839–855.
https://doi.org/10.1080/00207390050203342
Carvalho, C., & Solomon, Y. (2012). Supporting statistical literacy: What do culturally relevant/realistic tasks show us about the
nature of pupil engagement with statistics? International Journal of Educational Research, 55, 57–65. https://doi.org/10.1016/j.
ijer.2012.06.006
Castro Sotos, A. E., Vanhoof, S., Van den Noortgate, W., & Onghena, P. (2007). Students’ misconceptions of statistical inference: A
review of the empirical evidence from research on statistics education. Educational Research Review, 2(2), 98–113. https://
doi.org/10.1016/j.edurev.2007.04.001
Chan, S. W., & Ismail, Z. (2014). Developing statistical reasoning assessment instrument for high school students in descriptive
statistics. Procedia-Social and Behavioral Sciences, 116, 4338-4343. https://doi.org/10.1016/j.sbspro.2014.01.943
Chatzivasileiou, E., Michalis, I., Tsaliki, C., & Sakellariou, I. (2011, August 21-26). Service elementary school teachers’ conceptions
of arithmetic mean. In 58th World Statistical Congress, Dublin. https://2011.isiproceedings.org/papers/950460.pdf
Cooper, L. L., & Shore, F. S. (2008). Students’ misconceptions in interpreting center and variability of data represented via histograms
and stem-and-leaf plots. Journal of Statistics Education, 16(2). https://doi.org/10.1080/10691898.2008.11889559
Cui, L., & Liu, Z. (2021). Synergy between research on ensemble perception, data visualization, and statistics education: A tutorial
review. Attention, Perception, & Psychophysics, 83(3), 1290-1311. https://doi.org/10.3758/s13414-020-02212-x
Delello, J. A., & McWhorter, R. R. (2015). New visual social media for the higher education classroom. Social Media and Networking:
Concepts, Methodologies, Tools, and Applications, 4–4(February), 2151–2175. https://doi.org/10.4018/978-1-4666-8614-4.
ch098
DelMas, R., Garfield, J., Ooms, A., & Chance, B. (2007). Assessing students’ conceptual understanding after a first course in statistics.
Statistics Education Research Journal, 6(2), 28–58.
DiStaso, M. W., Stacks, D. W., & Botan, C. H. (2009). State of public relations education in the United States: 2006 report on a national
survey of executives and academics. Public Relations Review, 35(3), 254–269. https://doi.org/10.1016/j.pubrev.2009.03.006
Dönmez, I., Idin, S., & Gürbüz, S. (2022). Determining lower-secondary students’ STEM motivation: A profile from Turkey. Journal
of Baltic Science Education, 21(1), 38. https://doi.org/10.33225/jbse/22.21.38
Donohue, S. K., & Richards, L. G. (2009, October). Factors affecting student attitudes toward active learning activities in a graduate
engineering statistics course. In 39th IEEE Frontiers in Education Conference (pp. 1-6). IEEE.
Emmioǧlu, E., & Capa-Aydin, Y. (2012). Attitudes and achievement in statistics: A meta-analysis study. Statistics Education Research
Journal, 11(2), 95–102. https://doi.org/10.52041/serj.v11i2.332
Erydice. (2016). Eurydice Publications. https://eacea.ec.europa.eu/national-policies/eurydice/content/2016-eurydice-publications_
en
Faghmous, J. H., & Kumar, V. (2014). A big data guide to understanding climate change: The case for theory-guided data science.
Big Data, 2(3), 155-163. http://doi.org/10.1089/big.2014.0026
Franklin, C., Kader, G. Mewborn, D., Moreno, J., Peck, R., Perry, M., and Scheaffer, R. (2007). Guidelines for Assessment and
Instruction in Statistics Education (GAISE) Report: A Pre-K-12 Curriculum Framework. American Statistical Association. URL:
http://www.amstat.org/education/GAISE/
Friel, S. N., Curcio, F. R., & Bright, G. W. (2001). Making sense of graphs: Critical factors influencing comprehension and instructional
implications. Journal for Research in Mathematics Education, 32(2), 124–158. https://doi.org/10.2307/749671
648
INFOGRAPHIC
ISSN 2538–7138 /Online/ (pp. 638-650)
Fullerton, J., & Umphrey, D. (2001). An analysis of attitudes toward statistics. Annual Meeting of the Association for Education in
Journalism and Mass Communication, 356–366. https://files.eric.ed.gov/fulltext/ED456479.pdf
Garfield, J. (2003). Assessing Statistical Reasoning. Statistics Education Research Journal, 2(1), 22-38.
Gauvrit, N., & Morsanyi, K. (2014). The equiprobability bias from a mathematical and psychological perspective. Advances in
Cognitive Psychology, 10(4), 119–130. https://doi.org/10.5709/acp-0163-9
Gigerenzer, G., Gaissmaier, W., Kurz-Milcke, E., Schwartz, L. M., & Woloshin, S. (2007). Helping doctors and patients make sense
of health statistics. Psychological Science in the Public Interest, Supplement, 8(2), 53–96. https://doi.org/10.1111/j.1539-
6053.2008.00033.x
Griffith, J. D., Adams, L. T., Gu, L. L., Hart, C. L., & Nichols-Whitehead, P. (2012). Students’ attitudes toward statistics across the
disciplines: a mixed-methods approach. Statistics Education Research Journal, 11(2), 45-56. https://10.52041/serj.v11i2.328
Groth, R. E. (2005). An investigation of statistical thinking in two different contexts: Detecting a signal in a noisy process and
determining a typical value. Journal of Mathematical Behavior, 24(2), 109–124. https://doi.org/10.1016/j.jmathb.2005.03.002
Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2018). Multivariate Data Analysis (8th ed.). United Kingdom: Cengage Learning
Hilton, S. C., Schau, C., & Olsen, J. A. (2004). Survey of attitudes toward statistics: Factor structure invariance by gender and by
administration time. Structural Equation Modeling, 11(1), 92–109. https://doi.org/10.1207/S15328007SEM1101_7
Hoyles, C., Noss, R., Kent, P., & Bakker, A. (2013). Mathematics in the Workplace: Issues and Challenges. In A. Damlamian, J. F.
Rodrigues, & R. Sträßer (Eds.), Educational interfaces between mathematics and industry (Issue April 2016, pp. 43–50). Springer.
https://doi.org/10.1007/978-3-319-02270-3
Hoyles, C., Wolf, A., Molyneux-hodgson, S., & Kent, P. (2002). Mathematical Skills in the Workplace - Final Report to the Science,
Technology and Mathematics Council.
Hubbard, J. K., Potts, M. A., & Couch, B. A. (2017). How question types reveal student thinking: An experimental comparison of
multiple-true-false and free-response formats. CBE Life Sciences Education, 16(2), 1–13. https://doi.org/10.1187/cbe.16-
12-0339
Jacobbe, T., & Carvalho, C. (2011). Teachers’ Understanding of Averages. In C. Batanero, G. Burrill, & C. Reading (Eds.), Teaching
Statistics in School Mathematics-Challenges for Teaching and Teacher Education (New ICMI S). Springer, Dordrecht. https://
doi.org/https://doi.org /10.1007/978-94-007-1131-0_21
Jamie, D. M. (2002). Using computer simulation methods to teach statistics: A review of the literature. Journal of Statistics Education,
10(1), 4. https://doi.org/10.1080/10691898.2002.11910548
Jimerson, J. B., Cho, V., Scroggins, K. A., Balial, R., & Robinson, R. R. (2019). How and why teachers engage students with data.
Educational Studies, 45(6), 667-691. https://doi.org/10.1080/03055698.2018.1509781
Judi, H. M., Ashaari, N. S., Mohamed, H., & Tengku Wook, T. M. (2011). Students profile based on attitude toward statistics. Procedia
- Social and Behavioral Sciences, 18, 266–272. https://doi.org/10.1016/j.sbspro.2011.05.038
Kaplar, M. (2022). Recognizing misconceptions in working with data as a basis to enhance mathematical literacy (Publication
No. 29111073) [Doctoral dissertation, University of Novi Sad]. ProQuest Dissertations and Theses Global.
Kaplar, M., Lužanin, Z. & Verbić, S. (2021). Evidence of probability misconception in engineering students —why even an inaccurate
explanation is better than no explanation. International Journal of STEM Education, 8, Article 18. https://doi.org/10.1186/
s40594-021-00279-y
Kent, M. L., Carr, B. J., Husted, R. A., & Pop, R. A. (2011). Learning web analytics: A tool for strategic communication. Public Relations
Review, 37(5), 536–543. http://dx.doi.org/10.1016/j.pubrev.2011.09.011
Kim, Y. S., & Chung, D. S. (2012). Exploring the current state of and future directions for visual communication curriculum in the
United States. Visual Communication Quarterly, 19(3), 134–147. https://doi.org/10.1080/15551393.2012.706569
Kriesberg, A., Frank, R. D., Faniel, I. M., & Yakel, E. (2013). The role of data reuse in the apprenticeship process. Proceedings of the
ASIST Annual Meeting, 50(1). https://doi.org/10.1002/meet.14505001051
Lavigne, N. C., Salkind, S. J., & Yan, J. (2008). Exploring college students’ mental representations of inferential statistics. The Journal
of Mathematical Behavior, 27(1), 11–32. https://doi.org/10.1016/j.jmathb.2007.10.003
Leavy, A. M., & Middleton, J. A. (2011). Elementary and middle grade students’ constructions of typicality. The Journal of
Mathematical Behavior, 30(3), 235–254. https://doi.org/10.1016/j.jmathb.2011.03.001
Maass, K., Doorman, M., Jonker, V., & Wijers, M. (2019). Promoting active citizenship in mathematics teaching. ZDM, 51(6), 991–1003.
https://doi.org/10.1007/s11858-019-01048-6
Masel, J., Humphrey, P. T., Blackburn, B., & Levine, J. A. (2015). Evidence-based medicine as a tool for undergraduate probability
and statistics education. CBE Life Sciences Education, 14(4), 1–10. https://doi.org/10.1187/cbe.15-04-0079.
Maddens, L., Depaepe, F., Janssen, R., Raes, A., & Elen, J. (2021). Research skills in upper secondary education and in first year of
university. Educational Studies, 47(4), 491-507. https://doi.org/10.1080/03055698.2020.1715204
Manalo, E., Uesaka, Y., Pérez-Kriz, S., Kato, M., & Fukaya, T. (2013). Science and engineering students’ use of diagrams during note
taking versus explanation. Educational Studies, 39(1), 118-123. https://doi.org/10.1080/03055698.2012.680577
McClain, K. (1999). Reflecting on Students’ Understanding of Data. Mathematics Teaching in the Middle School, 4(6), 374–380.
Memon, M. A., Ting, H., Cheah, J. H., Thurasamy, R., Chuah, F., & Cham, T. H. (2020). Sample size for survey research: review and
recommendations. Journal of Applied Structural Equation Modeling, 4(2), 1-20
Mills, J. D. (2004). Students’ attitudes toward statistics: Implications for the future. College Student Journal, 38(3), 349-362.
Mokros, J., & Russell, S. J. (1995). Children’s concepts of average and representativeness. Journal for Research in Mathematics
Education, 26(30), 20–39. https://doi.org/10.2307/749226.
Neumann, D. L., Neumann, M. M., & Hood, M. (2011). Evaluating computer-based simulations, multimedia and animations that
help integrate blended learning with lectures in first year statistics. Australasian Journal of Educational Technology, 27(2).
https://doi.org/10.14742/ajet.970
649
INFOGRAPHIC
(pp. 638-650) ISSN 2538–7138 /Online/
Nolan, M. M., Beran, T., & Hecker, K. G. (2012). Surveys assessing students’ attitudes toward statistics: A systematic review of validity
and reliability. Statistics Education Research Journal, 11(2), 103–123. https://doi.org/10.52041/serj.v11i2.333
Papanastasiou, C. (2008). A residual analysis of effective schools and effective teaching in mathematics. Studies in Educational
Evaluation, 34(1), 24–30. https://doi.org/10.1016/j.stueduc.2008.01.005
Pardo, A. (2018). A feedback model for data-rich learning experiences. Assessment and Evaluation in Higher Education, 43(3),
428–438. https://doi.org/10.1080/02602938.2017.1356905
Park, S., Bekemeier, B., Flaxman, A. D. (2021). Understanding data use and preference of data visualization for public health
professionals: A qualitative study. Public Health Nursing, 8(4), 531-541. https://doi.org/10.1111/phn.12863
Rhoads, T. R., & Hubele, N. F. (2000). Student attitudes toward statistics before and after a computer-integrated introductory
statistics course. IEEE Transactions on Education, 43(2), 182-187. https://10.1109/13.848071
Schau, C., Stevens, J., Dauphinee, T. L., & Vecchio, A. D. (1995). The development and validation of the survey of attitudes toward
statistics. Educational and Psychological Measurement, 55(5), 868-875. https://doi.org/10.1177/0013164495055005022
Siricharoen, W. V. (2013). Infographics: The new communication tools in digital age. The International Conference on E-Technologies
and Business on the Web (EBW2013), 169–174.
Sudakov, I., Bellsky, T., Usenyuk, S., & Polyakova, V. (2016). Infographics and mathematics: A mechanism for effective learning in
the classroom. Primus, 26(2), 158-167.
Taber, K. S. (2018). The use of Cronbach’s alpha when developing and reporting research instruments in science education.
Research in science education, 48(6), 1273-1296. https://doi.org/10.1007/s11165-016-9602-2
Tapia, M., & Marsh, G. E. (2001). Effect of gender, achievement in mathematics, and grade level on attitudes toward mathematics.
The Annual Meeting of the Mid-South Educational Research Association, 1–16.
Wal, N. J. Van Der, Bakker, A., & Drijvers, P. (2019). Teaching strategies to foster techno ‑ mathematical literacies in an innovative
mathematics course for future engineers. ZDM, 2014. https://doi.org/10.1007/s11858-019-01095-z
Wang, P.Y., Vaughn, B. K., & Liu, M. (2011). The impact of animation interactivity on novices’ learning of introductory statistics.
Computers & Education, 56(1), 300–311. https://doi.org/10.1016/j.compedu.2010.07.011
Watson, J., Kelly, B., Callingham, R., & Shaughnessy, J. (2003). The measurement of school students’ understanding
of statistical variation. International Journal of Mathematical Education in Science and Technology, 34(1), 1–29.
https://doi.org/10.1080/0020739021000018791
Watson, J. M., & Moritz, J. B. (2000). The longitudinal development of understanding of average. Mathematical Thinking and
Learning, 2(1-2), 11–50. https://doi.org/10.1207/S15327833MTL0202_2
Watson, J. M., & Moritz, J. B. (2009). The longitudinal development of understanding of average. Mathematical Thinking and
Learning, 37–41. https://doi.org/10.1207/S15327833MTL0202_2
Wise, S. L. (1985). The development and validation of a scale measuring attitudes toward statistics. Educational and psychological
measurement, 45(2), 401-405. https://doi.org/10.1177/001316448504500226
Zilinski, L. D., Nelson, M. S., & Van Epps, A. S. (2014). Developing professional skills in stem students: Data information literacy.
Issues in Science and Technology Librarianship, 77, 1-10. https://doi.org/10.5062/F42V2D2Z
Received: May 30, 2022 Revised: July 10, 2022 Accepted: July 29, 2022
Cite as: Lužanin, Z., Kaplar, M., & Hrubik-Vulanovic, T. (2022). Undergraduate STEM and non-STEM students’ interpretation of
mean in an infographic. Journal of Baltic Science Education, 21(4), 638-650. https://doi.org/10.33225/jbse/22.21.638
Zorana Lužanin Phd, Full Professor at the Faculty of Science, University of Novi Sad, Trg
Dositeja Obradovića 3, 21000 Novi Sad, Serbia.
E-mail: zorana@dmi.uns.ac.rs
Website: https://people.dmi.uns.ac.rs/~zorana
Marija Kaplar Phd, Teaching Assistant at the Faculty of Technical Sciences at the
(Corresponding author) University of Novi Sad, Trg Dositeja Obradovića 6, 21000 Novi Sad, Serbia.
E-mail: marija.radojicic@uns.ac.rs
Website: http://www.kel.ftn.uns.ac.rs/2048339075/marija-kaplar
Tatjana Hrubik-Vulanovic Phd, Associate Professor in Mathematics at Kent State University at Stark,
Ohio, 6000 Frank Avenue NW, North Canton, OH 44720, USA.
E-mail: thrubik@kent.edu
Website: https://www.kent.edu/math/tatjana-hrubik-vulanovic-0
650
SCIENCE TEACHERS’
PERCEPTIONS OF THEIR
KNOWLEDGE BASE FOR ISSN 1648-3898 /Print/
TEACHING FORCE CONCEPTS ISSN 2538-7138 /Online/
Abstract. It is important for students to

understand force concepts because they
‘Maphole Marake, are central to learning physics and other
Loyiso C. Jita, sciences; however, students find it difficult
Maria Tsakeni to understand. There are calls for teachers
to tap into their professional knowledge
and develop beliefs that help them assist
students comprehend the topic. To meet
this challenge, teachers’ perceptions of their
knowledge base for teaching force concepts
Introduction should be probed because perceptions
act as windows into teachers’ practices.
Understanding force concepts is one of the basic competences that This study, therefore, explored physics
students need to develop in order to understand most of the science topics, teachers’ perceptions of their knowledge
particularly physics. This notion is supported by Carson and Rowlands (2005) base for teaching force concepts. Science
who expounded that Mechanics, of which force concepts are constituents, is a teachers’ pedagogical content knowledge
domain of physics that is fundamental to learning science and understanding (PCK) frameworks were used to develop
natural phenomena. It provides skills needed to understand other science a questionnaire based on a 5-point Likert
topics and the natural environment (Singh & Schunn, 2016). For instance, scale administered to 100 physical science
people do not experience force rather; they feel and see its effects such as teachers who were randomly selected from
change of speed and size. Therefore, for students to understand why objects 54 schools in five districts in Lesotho. Data
of different mass and size that fall from the same height reach the ground were analysed using descriptive statistics.
at the same time, they should understand force and be able to engage in The results suggest that, even though
imaginative thinking, which is one of the competences needed to understand teachers reported positive views about their
and advance in science related fields (Carson & Rowlands, 2005). It is therefore knowledge base, there is no correlation
important for students to understand force concepts correctly as it provides between sub-components of curricular
basic tools for understanding science (Sadoglu & Durukan, 2018). knowledge. It is concluded that teachers
Teaching force concepts proved to be problematic because most stu- should build the curricular knowledge by
dents seem to have preconceptions of force that are not scientifically correct. participating in collaborative activities. It
Handhika et al. (2016) have indicated that students believe that no forces act is, therefore, recommended that studies
on stationary objects while Khandagale and Chavan (2017) explained that that probe teachers’ actual knowledge
students hold the view that a continuous force is needed for continuous of PCK constructs are executed, so that
motion to take place. Nonetheless, these are non-scientific views because, appropriate information is available
for all stationary objects, there are balanced forces acting on them and mov- when planning professional development
ing objects eventually stop because of friction (Singh & Chunn, 2016). For activities targeting teachers’ PCK.
instance, a ball rolling on the ground eventually stops because of frictional Keywords: force concepts, pedagogical
force between the ball and the ground. If there was no friction, the object content knowledge, teacher knowledge,
would continue moving. Students’ alternative conceptions arise through their teachers’ perceptions of knowledge
encounter with force in their daily experiences. They engage in imaginative
thinking and try to make sense of what they see happening in their natural
world (Carson & Rowlands, 2005; Nasri et al., 2020). Consequently, they de- ‘Maphole Marake, Loyiso C. Jita,
velop alternative conceptions that do not align with scientific knowledge. This Maria Tsakeni
in turn creates problems when learning about the concepts in school. That University of the Free State, South Africa
students have different conceptions of the force most of which contradict
scientific knowledge has implications for teaching and teacher knowledge.
651
CONCEPTS
(pp. 651-662) ISSN 2538–7138 /Online/
Teachers play a critical role of being catalysts in teaching and learning (Margot & Kettler, 2019). They are
responsible for creating the environment that is conducive for all learners to learn effectively. For this to happen,
teachers should have solid knowledge base for teaching among which pedagogical content knowledge (PCK)
is the most essential (Shulman, 1987). PCK is the knowledge base that enables teachers to plan and teach in the
manner that the subject matter is comprehensible to all students. Nonetheless, how teachers enact their teaching
depends on many factors including their perceptions.
Perceptions are people’s personal understandings about a phenomenon, such as knowing how to teach and
one’s content knowledge of force (Cheng et al., 2016). Understanding teachers’ perceptions of their knowledge
base provides windows into their classrooms, because they decide on which knowledge base to enact based their
perceptions (Liepertz & Borowski, 2018; Şen & Sarı, 2017). However, there are conflicting views regarding teachers’
perceptions and practice. On the one hand, Anagün (2018) highlighted a connection between teachers’ perceptions
of constructivist learning environment and 21st Century skills. On the other hand, Park et al. (2016) expounded that
although teachers had positive views about teaching approaches such as Science, Technology, Engineering, Arts
and Mathematics (STEAM) education, they had challenges implementing it. Nonetheless, van Schaik et al. (2018)
indicated that teachers’ perceptions and attitudes towards their knowledge base often serve as a barrier towards
its utilisation. In this study therefore, physics teachers’ perceptions of their knowledge base were measured so that
a window into their educational practice could be understood.
Research about the teaching of concepts related to force indicated that physics teaching is inclined to rely
on traditional teacher-centred strategies, in which teachers transmit most of the information (Mazibe et al., 2018;
Melo- Niño, 2017; Qhobela & Moru, 2014). Mazibe et al. (2018) compared the reported and enacted PCK of four
physics teachers in South Africa when they taught graphs of motion, while Melo-Niño (2017) examined the initial
characterization of four Colombian in-service teachers’ PCK while they taught electric field. Both studies found that
teachers were teaching using more teacher-centred strategies. In a study done in Lesotho, Qhobela and Moru (2014)
explored areas where physics teachers may need professional development, and teachers’ views about science
teaching, and found that their actual practices were based on teacher-centred strategies.
Research Problem
Research about science teachers’ perceptions generally focused on their views about particular educational
strategies (Anagün, 2018; Feyzioğlu, 2019), implementation of approaches such as Science, Technology, Engineer-
ing, Arts and Mathematics (STEAM), integrated teaching and learner-centred strategies (Du Plessis, 2020; Park et
al, 2016; Tudor, 2015) and the teaching of science in general (Qhobela & Moru, 2014). The topic of forces is very
important in the physics curriculum because it is part of the curriculum for students who will pursue STEM careers,
such as engineering (Canu et al., 2017). However, it has been found that the way it is taught is not helpful in helping
students correct their misconceptions such as “continuous force is needed for continuous motion to take place”
(Khandagale & Chavan, 2017, p. 202).
It is in this regard that it is imperative for teachers to understand that the way they perceive different aspects
of teacher knowledge have a bearing on how they enact teaching. Consequently, this study explores the percep-
tions of physics teachers about their knowledge base to teach concepts of force.
Research Focus
This study is grounded on the premise that teachers’ educational practices are dependent on their percep-
tions about their knowledge to teach. Educational practices in this study refer to the teaching and assessment
strategies that teachers use when teaching force concepts. It also relates to teachers’ knowledge of how students
learn the concepts, including how the importance of force concepts in the school curriculum and in students’ lives
can be discerned. This study seeks to understand the perceptions of science teachers about their knowledge to
teach the concepts of force to Grade 11 students. The quality of educational practices depends on the teachers’
specialised knowledge (Gul et al., 2021). However, how teachers enact their professional knowledge depends
on different factors, such as their perceptions and beliefs about their abilities. Teachers’ perceptions about their
abilities to teach are imperative as they serve as guides that determine and shape what teachers learn and can do
(Gess-Newsome, 2015).
What teachers do in the classroom is directed by what they know and what they believe they know and can do
652
ISSN 1648–3898 /Print/ SCIENCE TEACHERS’ PERCEPTIONS OF THEIR KNOWLEDGE BASE FOR TEACHING FORCE
CONCEPTS
ISSN 2538–7138 /Online/ (pp. 651-662)
(Anagün, 2018; Cavendish et al., 2019). Within the landscape of teachers’ knowledge base, Shulman (1987) pointed
to PCK as the most prominent component of teacher knowledge that is essential for teaching. PCK refers to teachers’
knowledge and skills as it relates to representing particular scientific concepts, by using powerful representations
and illustrations so that it becomes easy for students to understand (Azam, 2020; Fischer et al., 2012; Morrison &
Luttenegger, 2015). Azam (2020) referred to this conception as topic-specific pedagogical construction (TSPC).
Therefore, PCK is a collection of TSPCs that a teacher develops during teaching of a particular topic, such as force
concepts, over a long time. In summary, PCK is the knowledge to teach.
Park and Oliver (2008) referred to PCK as “both an internal and external construct” (p. 263); in particular, teachers
realise their PCK during planning and actual teaching. It is a construct that is held personally by teachers as they
think about what to teach, how to teach it, and what they do, mainly during planning. Consequently, recent debates
about PCK referred to it as personal PCK (pPCK) and enacted PCK (ePCK) (Azam, 2020). However, it is worth noting
that as teachers develop TSPCs and consequently PCK, there is a knowledge exchange that takes place between
various aspects in the teaching and learning situation (Carlson & Daehler, 2019). For instance, using a particular
analogy to represent a type of force may not be as helpful as the teacher initially thought it would be. As a result,
that particular teacher may develop a negative view about the representation, and hence reject it as part of their
PCK. Subsequently, teachers’ beliefs and perceptions are also shaped by their experiences when enacting each of
the TSPCs. PCK is therefore experiential and embedded in values and practice (Azam, 2020). The framework that
is used to delineate teacher knowledge bases that this study is concerned with is explained in Table 1. These are
the teacher knowledge components that science teachers are expected to possess in order to teach effectively.
Studies concerning teachers’ perceptions about their knowledge to teach, particularly PCK, were mostly
qualitative (Barendsen & Henze, 2017; Mazibe et al., 2018; Nilsson & Vikström, 2015). Barendsen and Henze (2017)
probed how teachers’ articulated knowledge and classroom practice shape each other, while Mazibe et al. (2018)
focused on how reported and enacted PCK relate to each other. The authors found a mismatch between reported
and enacted PCK. That is, teachers’ enacted PCK did not match the reported PCK in some of the aspects of PCK.
Table 1
Components of Science Teachers’ PCK
Knowledge base
Definition
component
Teacher orientations These refer to what teachers believe to be the reason why they teach science. Particularly, orientations develop
and are developed by what teachers know. Just like the conception of PCK as either subject- or topic-specific, so
is the conception of teacher orientations.
Knowledge of the science It refers to what teachers know about the breadth and depth of each topic they teach, such as concepts of force.
curriculum This relates to their ability to relate that topic to others and their familiarity with issues that students previously
learned and those they are learning in the present and higher grades. Moreover, it refers to knowing which topics
to teach at a particular grade and their importance. In other words, why it is important for students to learn it and
what value it has in relation to the entire curriculum that students are learning.
Knowledge of students’ under- This is the knowledge of the requirements for students’ understanding. If teachers’ purposes and beliefs regarding
standing students’ learning of force concepts is to engage them in inquiry activities, they should know the skills and knowl-
edge that students need. In this way, the students can do effective inquiry activities (Magnusson et al., 1999).
Knowledge of educational This constitutes knowledge of activities that provide evidence of targeted topics, such as force, and representa-
strategies tions that help students visualize ideas that cannot be directly observed (Smith & Banilower, 2015).
Knowledge of assessment This points to teachers’ ability to assess. It includes “any planned method or strategy used in the classroom to es-
strategies tablish the level of difficulties or understanding of a particular concept or idea with the purpose of helping students
to succeed in learning” (Rahman, 2018, p. 274).
As the literature reviewed in this study indicated, the studies that focused on teachers’ perceptions about their
knowledge to teach were qualitative. Teachers’ perceptions about their knowledge base were captured through
teacher interviews and content representations (CoRes). Nonetheless, qualitative studies are concerned with few
participants whose results cannot be generalised. For instance, CoRes were used with small number of participants,
653
CONCEPTS
(pp. 651-662) ISSN 2538–7138 /Online/
mostly pre-service teachers, with the purpose of capturing their PCK so that professional development activities,
for that particular group, can be planned and provided (Williams, 2012). Furthermore, CoRes take time to complete
(Bertram, 2012) and require teachers to willingly express themselves. To bridge this gap, this study used a quan-
titative method. This minimised the errors that might have been brought about by unwillingness of respondents
to express themselves and their lack or limited ability to use written language to express themselves. This study
therefore aspired to determine teachers’ perceptions on the five components of science teachers’ PCK for teaching
force concepts. Accordingly, the study was guided by the following questions:
1. What are science teachers’ perceptions of the five components of teacher knowledge base?
2. What is the correlation among the components of teacher knowledge base?

General Background
A quantitative survey design was used to measure Lesotho physics teachers’ perceptions of their knowledge to
teach concepts of force. Mainly, the survey measured the teachers’ perceptions of the knowledge base for science
teachers which have been described by Magnusson et al. (1999) as the components of science teachers’ PCK (Table
1). Data were gathered for four months (August to November 2018) among physics teachers who taught grade
11 classrooms. The questionnaires (100) were distributed to schools by the first researcher. The questionnaire was
suitable to collect data about the teachers’ perception and to describe the perceptions of their knowledge base
to teach force concepts (Creswell, 2012).
Sample Selection
The participants were selected from 54 upper secondary schools (offering grades 8 to 12) in the five districts
located in the lowlands of Lesotho. The five districts did not have equal numbers of upper secondary schools;
therefore, proportional clustered sampling was used. In order to make sure that the sample was spread equitably,
the ratios were worked out as follows (Bordens & Abbott, 2011; Creswell, 2012, p. 145):
𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑠𝑐ℎ𝑜𝑜𝑙𝑠 𝑝𝑒𝑟 𝑑𝑖𝑠𝑡𝑟𝑖𝑐𝑡 𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑠𝑐ℎ𝑜𝑜𝑙𝑠 𝑝𝑒𝑟

× 𝑝𝑜𝑝𝑢𝑙𝑎𝑡𝑖𝑜𝑛
𝑇𝑜𝑡𝑎𝑙 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑠𝑐ℎ𝑜𝑜𝑙𝑠 𝑖𝑛 𝑡ℎ𝑒 𝑓𝑖𝑣𝑒 𝑑𝑖𝑠𝑡𝑟𝑖𝑐𝑡𝑠 𝑇𝑜𝑡𝑎𝑙 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑠𝑐ℎ𝑜𝑜𝑙𝑠 𝑖𝑛 𝑡ℎ
��
�� instance, the district that had 51 upper secondary schools, 37 schools (teachers) (�� × 100 = 37.5) were
For
( × 100 = 37.5)
identified.
�� Since some schools had more than one physics teacher, all were asked to participate, because they all
taught physics.
This study aimed at determining the perceptions of physics teachers in the following components of teachers’
PCK: 1) orientations towards teaching, 2) science curricular knowledge, 3) knowing how students understand physics,
4) knowing educational and 5) assessment strategies. A closed-ended questionnaire was used to determine what
physics teachers perceived they know about teaching force concepts. Some of the questions in the questionnaire
were developed by the researchers, while others were adopted from the 1999 and 2015 Trends in International
Mathematics and Science Study (TIMSS, 1999, 2015). Table 2 presents a sample of question items adapted from
TIMSS while Table 3 shows sample questions in the questionnaire.
The questionnaire contained 65 question items which were all answered on a 5-point Likert scale. The ques-
tionnaire included five items that sought to determine the demographics of the respondents. It was divided into
five sections pertaining to the five knowledge components. Knowledge of the science curriculum was divided into
two parts: knowledge of the breadth and depth of force concepts, and knowledge of their importance. The scales
of the two parts were 1 = strongly disagree to 5 = strongly agree and 1 = not important to 5 = very important,
respectively. A different scale, ranging from 1 = never to 5 = always, was used for knowledge of educational and
assessment strategies.
654
CONCEPTS
ISSN 2538–7138 /Online/ (pp. 651-662)
Table 2
Sample of Original and Adapted Questionnaire Items
Wording in the source TIMSS (1999) Wording in the questionnaire
11. To be good at science at school, how important do you think it is for Rate how important the following are to students when you teach science
students to... topics such as force
a. Remember formulas and procedures 10. When learning the topic force, students should remember formulas
and procedures etc.
b. think in a sequential and procedural manner 11. Students should think in a sequential and procedural manner when
learning science concepts such as force.
c. understand science concepts, principles, and strategies 12. Students should understand science concepts, principles, and strate-
gies.
Table 3
Sample Questions from PCK Domains
Knowledge domains Sample items
Knowledge of the science curriculum I can clearly explain science concepts in the topic force
Students should think in a sequential and procedural manner when learning science concepts such as
force
Knowledge of students’ understanding Before I teach the topic force, I anticipate difficulties that students might have about the force concepts
(e.g., Moment of force)
Knowledge of educational strategies I ask students to complete challenging exercises that require them to go beyond the teaching that they
receive on force concepts
Knowledge of assessment strategies I use standardised tests produced outside the school when I assess students’ learning of force
I assess students’ ability to write definitions or other short writing assignments about force concepts
Orientations towards science teaching When I teach the topic force, I pay more focus on students’ ability to make observations
Validity and Reliability of the Questionnaire
The questionnaire was piloted with 23 physics teachers who attended a three-week workshop in March 2018.
The piloted questionnaire had 83 items from which 18 were removed. When conducting the main study, the first
researcher hand delivered the questionnaires to schools and asked to meet with the teacher respondents where
possible. The purpose of meeting with the respondents was to explain the purpose of the survey and to help build
the rapport. The effects of these meetings were evidenced by the high return rate of 92%.
Measures were also taken to check the reliability and internal consistency of the questionnaire. First, the ques-
tions were grouped so that those that sought for teachers’ views about a particular theme were given a matching
heading, such as their knowledge about students’ understanding. This helped respondents to know what information
each item was intended to elicit from them. Secondly, the researchers calculated the Cronbach’s alpha coefficients
for the separate themes and the whole questionnaire. The reliability estimates of the final questionnaire ranged
from .784 to .918, suggesting that the instrument is reliable and could be used with confidence (Taber, 2018).
Data Analysis
In order to describe the teachers’ perceptions and to answer the research questions, the following was done:
1. The mean and standard deviations for all 60 items and the five knowledge components were calculated.
2. The correlations among the components were determined using Spearman correlation coefficients.
During data analysis, responses in the range 1 to 2.9 were considered to be in disagreement with the con-
cept. Responses with a value of 3 represented neutral views, serving as a decision point. Responses that fell in the
range 3.1 to 5 were considered to be in agreement with the concept. The means and standard deviations were
also calculated. The mean signifies the respondents’ general response to a specific question and domain whereas
655
CONCEPTS
(pp. 651-662) ISSN 2538–7138 /Online/
the standard deviation designated the width of the variation of responses.
Research Results
Table 4 shows the means and standard deviations of each of the components of physics teachers’ PCK used
in this study.
Table 4
Summary of Mean and Standard Deviations of Components of Knowledge
KSC KAS
KSU KIS OT
Importance of force What to
Breadth and depth Methods
concepts assess
M 4.34 4.53 4.11 4.05 3.44 3.37 3.78

SD 0.775 0.68 0.75 1.9 1.03 1.01 0.91
Note: KSC = knowledge of science curriculum; KSU = knowledge of student understanding; KIS = knowledge of educational
strategies; KAS = knowledge of assessment strategies; OT = orientations towards teaching
In general, the respondents seemed to have positive views about the knowledge to teach force concepts.
Especially, they seemed to be confident about their knowledge of the importance of force concepts, indicated by
this topic having the highest mean 4.53 and the lowest standard deviation 0.68. However, the respondents did
not seem as confident about their knowledge of the aspects of force concepts for assessment, as highlighted by
the lowest mean score 3.37.
Teachers’ Perceptions of their Knowledge of the Science Curriculum
Teachers’ perceptions on the science curricular knowledge have been categorized into two subcomponents:
the breadth and depth of the curriculum, and the importance of force concepts in the school curriculum. Results
showed that the respondents perceived themselves to be knowledgeable about the breadth and depth of force
concepts with the mean of 4.34 (0.775) and the importance of force concepts in the school science curriculum with
mean 4.53 (0.68). The respondents believed that they knew how to clearly explain force concepts. Furthermore,
the respondents believed they were conversant with the force concepts taught in grade 11 as well as how the
concepts were organised and the goals and purposes of teaching the topic. As evidenced by the lowest standard
deviation (0.68), there was agreement among the respondents regarding why students must learn force concepts,
including the skills students should develop as they learn science.
Teachers’ Perceptions of their Knowledge of Students’ Understanding
As indicated by the mean scores 4.11 (0.75), the respondents had positive perceptions about their knowledge
of how students learn science. The respondents agreed that they knew what improves students’ understanding of
the topics of force concepts. In order to say a teacher knows how students learn, they are expected to understand
students’ prior knowledge both scientific and non-scientific including knowing the aspects of the topic that are
difficult for students to comprehend. Likewise, teachers are expected to know prerequisite skills students should
possess to comprehend concepts such as force, including what they find interesting and boring. The average mean
score also highlighted that the respondents agreed that they considered the characteristics of students that en-
hance learning in their teaching of force concepts. The respondents perceived themselves as being able to assess
what students already know before teaching force concepts.
656
CONCEPTS
ISSN 2538–7138 /Online/ (pp. 651-662)
Teachers’ Perceptions of their Knowledge of Educational Strategies
The overall mean of 4.05 (0.9) for this component indicates that the respondents believed that they knew and
varied the strategies when teaching the concepts of force. The strategies that they believed they used included
small-group and whole-class discussions, presentations, using multiple representations of force, doing exercises
about force, and ensuring that they connect what students already know to what is to be learned. The standard
deviation indicates that the respondents’ views on the use of different strategies were fairly homogeneous.
Teachers’ Perceptions of their Knowledge of Assessment Strategies
The results (Table 4) showed that respondents perceived themselves to know different assessment methods
as evidenced by the mean score 3.44 (1.03). The results indicated that the teachers believed that they used differ-
ent assessment methods in their classrooms. The standard deviation, however, shows that the responses were not
consistent. It can be said that the respondents’ perceptions about the assessment methods they used were different
and, perhaps, inconsistent. The assessment methods that the respondents believed they used were written tests,
assignments, and homework assignments.
Furthermore, Table 4 shows that the respondents held positive views about their knowledge of the facets of
the concepts of force to assess. This is highlighted by the mean score 3.37 (1.01). The respondents indicated that
they assessed the recall of facts; definitions; experimental skills; identification and uses of force concepts; and oral
communication. However, the standard deviation was greater than 1, which designates variations of the responses.
Nonetheless, the respondents perceived that they knew the facets of force concepts that students are expected
to learn, because they indicated that they assessed them, although not always.
Teachers’ Perceptions of their Orientations towards Teaching
Table 4 indicates that the respondents ranked themselves high with respect to the purpose of science teaching
and learning as shown by the mean of 3.78 (0.91). In general, respondents believed that they involved students
when teaching and they created environment that enhanced meaningful learning. In other words, they had posi-
tive perceptions that they engaged students in both knowledge construction and transmission. The respondents’
responses were homogeneous, as indicated by the standard deviation being less than 1.
Correlations between the Knowledge Components
The correlations between the knowledge components were ascertained by doing Spearman’s correlation test
(Table 5). The correlation test was applied to highlight the connection available amongst the five domains of knowl-
edge in the questionnaire. The null hypothesis was that “there is no correlation among the domains of knowledge”.
Table 5
Correlations between Knowledge Components
Spearman correlation coefficients (N = 92)

Prob ˃ IrI under H0: Rho= 0
KSC KAS
Breadth and Importance of KSU KIS What to OT

Methods
depth force concepts assess
.088 .316 .051 .150 -.002 -.015
Breadth and depth 1
.4066 .0022 .6317 .1549 .9869 .8899
Importance of force .088 .211 .14 -.014 .256 .229
1
concepts .4066 .0436 .185 .8929 .0138 .0285
.316 .211 .47 .337 .395 .384
KSU 1
.0022 .0436 < .0001 .001 < .0001 .0002
657
CONCEPTS
(pp. 651-662) ISSN 2538–7138 /Online/
Spearman correlation coefficients (N = 92)

Prob ˃ IrI under H0: Rho= 0
KSC KAS
Breadth and Importance of KSU KIS What to OT

Methods
depth force concepts assess
.051 .14 .470 .47 .658 .375
KIS 1
.6317 .185 < .0001 < .0001 < .0001 .0002
.15 -.014 .337 .47 .595 .457
Methods 1
.1549 .8929 .001 < .0001 < .0001 < .0001
-.002 .256 .395 .658 .595 .427
What to assess 1
.9869 .0138 < .0001 < .0001 < .0001 < .0001
-.015 .229 .384 .375 .457 .427
OT 1
.8899 .0285 .0002 .0002 < .0001 < .0001
Note: KSC = knowledge of science curriculum; KSU = knowledge of student understanding; KIS = knowledge of educational
strategies; KAS = knowledge of assessment strategies; OT = orientations towards teaching
Table 3 shows that there is a weak (0 < r < .4; p < .05) and moderate (.39 < r < .7; p < .05) positive correlations
among the different domains of knowledge. Knowledge of the science curriculum, constituting knowing the impor-
tance of the concepts of force and the breadth and depth of the curriculum, correlated with a few other domains.
On the one hand, the respondents’ perceptions about the knowledge of the breadth and depth of force concepts
showed a weak correlation only with the knowledge of how students understand force concepts (.316). This means
that when teachers believe they have more knowledge of science content, they tend to be more confidence that
they know how students learn science. On the other hand, knowledge of the importance of force concepts had
a weak correlation with knowledge of students’ understanding (.211), assessment of scientific literacy (.256), and
teacher orientations (.229). It was unexpected that curricular knowledge correlated only with the knowledge of how
students learn. The highest correlation was between the respondents’ perceptions of the knowledge of educational
strategies and knowledge of what to assess (r = .658; p = .0001). This is a moderate correlation.
Discussion
The study explored physics teachers’ perceptions about their knowledge base for teaching concepts of force.
Based on the premise that teachers’ PCK is topic specific (Azam, 2020; Mazibe et al., 2018), the results of this study
contribute to literature about teachers’ perceptions of PCK on the topic of force. Studying teachers’ perceptions is
important, because beliefs, attitudes, and perceptions influence teachers’ practices (Feyzioğlu, 2019; Meschede et
al., 2017). Moreover, the results add to existing literature about how teachers’ knowledge of the breadth and depth
of the science curriculum relates with its importance to students’ learning.
The results revealed that respondents believed that they had the appropriate knowledge for teaching force
concepts. In other words, respondents believed that they know how to teach force concepts. These results reaffirm
those of other scholars who studied teachers’ perceptions about their knowledge base and PCK in particular (Drits-
Esser & Stark, 2015; Gul et al., 2021). Nisperuza et al. (2019) indicated that teachers perceived that they know how
students think, including educational and assessment strategies. Similar to results in this study, they highlighted
that teachers had differing views about their orientations towards teaching force concepts. That teachers had dif-
fering views about their orientations could be because they used both teacher- and learner-centred orientations
when teaching.
It was also found that respondents were mostly confident about their science curricular knowledge, particu-
larly the importance of force concepts in the science curriculum and the breadth and depth of the curriculum.
Concerning this, the respondents believed that they know the force concepts that are core, and that they can
modify activities based on the nature of students they are teaching and the level of difficulty of the concepts.
Because this subcomponent of curricular knowledge is more linked to content knowledge, it is understandable
why respondents strongly believed that they know it. This finding is in accord with that of Catalano et al. (2019),
who indicated that pre- and in-service teachers were positive about their knowledge of content. It can be implied
658
CONCEPTS
ISSN 2538–7138 /Online/ (pp. 651-662)
that teachers believe they know the topic of force concepts because they have experience teaching it and their
students’ performance in the topic is good.
One interesting finding worth noting is that the respondents’ knowledge of the subcomponents of curricular
knowledge, breadth and depth of force concepts and their importance in the science curriculum, did not correlate
with each other. On the one hand, knowing the importance of the topic indicates understanding its significance
in relation to the entire curriculum and the goals for students’ learning of the topic. It also makes it possible for
teachers to identify main concepts, adapt activities, and target conceptual understanding. On the other hand,
knowledge of the breadth and depth of the curriculum relates to knowing what students learned in lower grades,
what they should learn in the present grade, and what they will learn in higher grades within a particular topic
(Ball et al., 2008; Park & Oliver, 2008). Having found that there is no correlation between the two subdomains of
knowledge of the science curriculum is surprising because one would expect that knowing the core concepts to
teach in a particular topic, such as force concepts, requires one to know what students learned and know in that
topic. This is because teachers can modify activities and integrate students’ prior knowledge when teaching new
content in the present grade so that they can develop conceptual understanding.
That there was no correlation between the two components could mean that knowing one component does
not have an effect on the other. That is, whether teachers know the importance of why students should learn force
concepts in upper secondary school does not necessarily mean that they know what students learn under the
topic throughout their school years. Therefore, knowledge of one subcomponent does not necessarily translate
to knowledge of the other. For one to gain knowledge of the subcomponents, that is the breadth and depth
and importance of force concepts in the school science curriculum, teachers should make an effort by studying
curriculum materials and collaborating with peers. As indicated by Drits-Esser and Stark (2015), when teachers
collaborate to develop curriculum materials, they get opportunities to learn from peers. They thus gain a deeper
understanding of some of the professional knowledge that they were not familiar with and those that did not
cross their minds as being important.
Another possible explanation for why there was no correlation between teachers’ knowledge of the impor-
tance and the breadth and depth of the concepts of force is that teachers often do not bother to find out what their
students learned in lower grades and what they will learn in the future grades. That is, teachers do not often study
the syllabi of the different grades and other curriculum materials with the purpose of identifying what might be
similar and/or different. Moreover, most of the curriculum knowledge is gained through experience and collabora-
tive activities. Teachers should know this information, because even if they had learned it during teacher training,
it keeps on changing because of the ongoing curriculum reforms in most countries. Therefore, teachers should
always update their knowledge. For instance, Kavanagh and Sneider (2007) highlighted that in elementary school,
students learn that “things just fall”. However, in middle and upper secondary school, the concept of gravitational
force is taught, and students are expected to learn that things fall due to gravitational pull. When teachers do not
make an effort to find out what was taught and accepted at lower grades, they may not know how to connect the
two and help students to change and/or improve the preconception developed in lower grades. This is therefore
an important finding because it points to the need for teachers to be part of communities of practice where they
discuss their professional knowledge and practice.
Conclusions and Recommendations
The study explored physics teachers’ perceptions of the knowledge base needed to teach the concepts of
force to grade 11 students, particularly the PCK components. The teachers had positive perceptions about the
PCK components, mainly knowledge of the science curriculum. Since curricular knowledge is linked to content
knowledge, it seems that physics teachers were confident about their knowledge of force concepts taught in school
including their importance as well as the scope at which these concepts are taught. However, of importance to
note is that, knowing one aspect of curriculum knowledge does not necessarily translate to knowing the other
component. This was highlighted by the lack of correlation between knowledge of the breadth and depth and
the importance of force concepts in school science curriculum. However, to help teachers develop integrated cur-
ricular knowledge, school- or centre-based collaborative activities among teachers who teach different levels and/
or grades should be planned. Mainly to facilitate discussions that will culminate into all groups understanding the
scope at which concepts are taught at different grades, what is accepted as correct and not correct, including the
purpose of teaching them at those different levels.
659
CONCEPTS
(pp. 651-662) ISSN 2538–7138 /Online/
This study focused on teachers’ perceptions not the actual practice. This approach has some limitations as
the actual practice may not align with what teachers believe they know and do. It is therefore recommended that
future studies i) explore the interplay between teachers’ knowledge of the importance of the concepts of force
and knowledge of the breadth and depth of the concepts when teaching force concepts, ii) design professional
development activities that help teachers deepen their understanding of curricular knowledge and relate it to
classroom practices. This study also recommends collaboration between the local university and Ministry of Edu-
cation to provide large scale professional development activities for teachers that target improvement of specific
categories of teachers’ PCK.
References
Anagün, Ş. S. (2018). Teachers’ perceptions about the relationship between 21st century skills and managing constructivist
learning environments. International Journal of Instruction, 11(4), 825-840. https://doi.org/10.12973/iji.2018.11452a
Azam, S. (2020). Locating personal pedagogical content knowledge of science teachers within stories of teaching force and
motion. EURASIA Journal of Mathematics, Science and Technology Education, 16(12), 1-20. https://doi.org/10.29333/
ejmste/8941
Ball, D. L., Thames, M. H. & Phelps, G. (2008). Content Knowledge for Teaching: What Makes It Special? Journal of Teacher
Education, 59(5), 389-407. http:// DOI: 10.1177/0022487108324554
Barendsen, K., & Henze, I. (2017). Relating teacher PCK and teacher practice using classroom observation. Research in Science
Education, 49, 1141-1175. https://doi.org/10.1007/s11165-017-9637-z
Bertram, A. (2012). Getting in touch with your PCK: A look into discovering and revealing science teachers’ hidden expert
knowledge. Teaching Science, 58(2), 18-23. http://search.info.org
Bordens, K. S., & Abbott, B. B. (2011). Research design and methods: A process approach (8th ed.). McGraw-Hill.
Canu, M., Duque, M., & De Hosson, C. (2017). Active learning session based on didactical engineering framework for conceptual
change in students’ equilibrium and stability understanding. European Journal of Engineering Education, 42(1), 32-44.
http://dx.doi.org/10.1080/03043797.2016.1190689
Carlson, J., & Daehler, K. R. (2019). The refined consensus model of pedagogical content knowledge in science education. In A.
Hume, R. Cooper, & A. Borowski (Eds.), Repositioning pedagogical content knowledge in teachers’ knowledge for teaching
science (pp. 77-92). Springer.
Carson, R., & Rowlands, S. (2005). Mechanics as the logical point of entry for the enculturation into scientific thinking. Science
Education, 14, 473-492. https://doi.org/10.1007/s11191-004-1791-9
Catalano, A., Asselta, L., & Durkin, A. (2019). Exploring the relationship between science content knowledge and science
teaching self-efficacy among elementary teachers. IAFOR Journal of Education, 7(1), 57-70. https://files.eric.ed.gov/
fulltext/EJ1217961.pdf
Cavendish, W., Morris, C. T., Chapman, L. A., Stoutenburg, L. O., & Kibler, K. (2019). Teacher perceptions of implementation
practices to support secondary students in special education. Preventing School Failure: Alternative Education for Children
and Youth, 64(1), 19-27. https://doi.org/10.1080/1045988X.2019.1628000
Cheng, P. Y., Talib, O., & Othman, A. (2016). Science teaching: Perceptions, attitudes and instructional practices. Jurnal Pendidikan
Sains & Matematik Malaysia, 6(2), 1-16. https://ejournal.upsi.edu.my/index.php/JPSMM/article/view/2163
Creswell, J. W. (2012). Educational research: Planning, conducting, and evaluating quantitative and qualitative research. Pearson.
Drits-Esser, D. & Stark, L.A. (2015). The impact of collaborative curriculum design on teacher professional learning. Electronic
Journal of Science Education, 19(8), 1-27 http://ejse.southwestern.edu
Du Plessis, E. (2020). Student teachers’ perceptions, experiences, and challenges regarding learner centred
teaching. South African Journal of Education, 40(1), 1-10. https://doi.org/10.15700/saje.v40n1a1631
Feyzioğlu, B. (2019). Examination of laboratory perceptions of pre-service science teachers with different goal orientations
on inquiry-based analytical chemistry courses: A case study. International Journal of Education in Mathematics, Science
and Technology, 7(3), 281-310. https://www.ijemst.com/index.php/ijemst/article/view/678
Fischer, H. E., Borowski, A., & Tepner, O. (2012). Professional knowledge of science teachers. In B. J. Fraser, K. G. Tobin, & C. J.
McRobbie (Eds.), Second international handbook of science education (pp. 435-448). Springer.
Gess-Newsome, J. (2015). A model of teacher professional knowledge and skill including PCK: Results of the thinking from
the PCK Summit. In A. Berry, P. Friedrichen, & J. Loughran (Eds.), Re-examining pedagogical content knowledge in science
education (pp. 28-42). Routledge.
Gul, N., Ullah, I., & Bibi, N. (2021). Perception of working and prospective teachers about the application of their professional
knowledge at secondary level. The Dialogue, 16(1), 39-50. https://www.researchgate.net/publication/350609578
Handhika, J., Cari, C., Soeparmi, A., & Sunarno, W. (2016). Student conception and perception of Newton’s Law. AIP Conference
Proceedings, 1708(1), 1-6. https://doi.org/10.1063/1.4941178
660
CONCEPTS
ISSN 2538–7138 /Online/ (pp. 651-662)
Kavanagh, C. & Sneider, C. (2007). Learning about gravity I. Free fall: A guide for teachers and curriculum developer. The
Astronomy Education Review, 2(5), 21-52. https://doi.org/10.3847/AER2006018
Khandagale, V. S., & Chavan, R. (2017). Identification of misconceptions for gravity, motion and inertia among secondary
school students. Aayushi International Interdisciplinary Research Journal, 4(6), 197-205. https://eric.ed.gov/?id=ED593127
Liepertz, S., & Borowski, A. (2018). Testing the consensus model: Relationships among physics teachers’ professional knowledge,
interconnectedness of content structure and student achievement. International Journal of Science Education, 41(7), 890-
910. https://doi.org/10.1080/09500693.2018.1478165
Magnusson, S., Krajcik, J., & Borko, H. (1999). Nature, sources, and development of pedagogical content knowledge for science
teaching. In J. Gess-Newsome, & M. G. Lederman (Eds.), Examining pedagogical content knowledge (pp. 95-132). Kluwer
Academic Publishers.
Margot, K., & Kettler, C. T. (2019). Teachers’ perception of STEM integration and education: A systematic literature review.
International Journal of STEM Education, 6(2), 1-16. https://doi.org/10.1186/s40594-018-0151-2
Mazibe, E. N., Coetzee, C., & Gaigher, E. (2018). A comparison between reported and enacted pedagogical content knowledge
(PCK) about graphs of motion. Research in Science Education, 50(3), 941-964. https://doi.org/10.1007/s11165-018-9718-7
Melo-Niño, L. V., Cañada, F., & Mellado, V. (2017). Initial characterization of Colombian high school physics teachers’ pedagogical
content knowledge on electric fields. Research in Science Education, 47, 25-48. https://doi.org/10.1007/s11165-015-9488-4
Meschede, N., Fiebranz, A., Möller, K., & Steffensky, M. (2017). Teachers’ professional vision, pedagogical content knowledge
and beliefs: On its relation and differences between pre-service and in-service teachers. Teaching and Teacher Education,
66, 158-170. https://doi.org/10.1016/j.tate.2017.04.010
Morrison, A. C., & Luttenegger, K. C. (2015). Measuring pedagogical content knowledge using multiple
points of data. The Qualitative Report, 20(6), 804-816. https://cpb-us-e1.wpmucdn.com/sites.nova.edu/dist/a/4/
files/2015/06/morrison11.pdf
Nasri, N. M., Nasri, N., & Talib, A. A. A. (2020). Physics teachers’ perceptions on sustainable physics education. Journal of Baltic
Science Education, 19(4), 569-582. https://doi.org/10.33225/jbse/20.19.569
Nisperuza, E. F., Salgado, A. G., & García, L. M. (2019). Science teachers’ perceptions of their pedagogical content knowledge
(PCK). CISETC 2019 International Congress on Educational and Technology in Sciences, Arequipa, Perú. http://ceur-ws.org/
Vol-2555/paper26.pdf
Nilsson, P., & Vikström, A. (2015). Making PCK explicit: Capturing science teachers’ pedagogical content knowledge (PCK) in
the science classroom. International Journal of Science Education, 37(17), 2836-2857. http://dx.doi.org/10.1080/09500
693.2015.1106614
Park, H., Byun, S. Y., Sim, J., Han, H., & Bae, S. Y. (2016). Teachers’ perceptions and practices of STEAM education in South Korea.
Eurasia Journal of Mathematics, Science & Technology Education, 12(7), 1739-1753. http://doi.10.12973/eurasia.2016.1531a
Park, S., & Oliver, J. S. (2008). Revisiting the conceptualisation of pedagogical content knowledge (PCK): PCK as a conceptual
tool to understand teachers as professionals. Research in Science Education, 38, 261-284. https://doi.org/10.1007/s11165-
007-9049-6
Qhobela, M., & Moru, E. (2014). Examining secondary school physics teachers’ beliefs about teaching and classroom practices
in Lesotho as a foundation for professional development. International Journal of Science and Mathematics Education,
12(6), 1367-1392. https://doi.org/10.1007/s10763-013-9445-5
Rahman, M. (2018). Exploring teachers’ practices of classroom assessment in secondary science classes in
Bangladesh. Journal of Education and Learning, 7(4), 274-283. https://doi.org/10.5539/jel.v7n4p274
Sadoglu, G., & Durukan, U. G. (2018). Determining the perceptions of teacher candidates on the concepts of science course,
science laboratory, science teacher and science student via metaphors. International Journal of Research in Education
and Science, 4(2), 436-453. https://doi.org/10.21890/ijres.428260
Şen, Ö. F., & Sarı, U. (2017). Pre-service science teachers’ beliefs about science teaching and perception of the nature of
science. The Electronic Journal for Research in Science & Mathematics Education, 21(1), 1-14. https://ejrsme.icrsme.com/
article/view/16409
Shulman, L. (1987). Knowledge and teaching: Foundations of the new reform. Harvard Educational Review, 57(1), 1-23. https://
doi.org/10.17763/haer.57.1.j463w79r56455411
Singh, C., & Schunn, C. D. (2016). Connecting three pivotal concepts in K-12 science state standards and maps of conceptual
growth to research in physics education. Journal of Physics Teacher Education Online, 5(2), 16-42. http://d-scholarship.
pitt.edu/id/eprint/22903
Smith, P. S., & Banilower, E. R. (2015). Assessing PCK: A new application of the uncertainty principle. In A. Berry, P. Friedrichen,
& J. Loughran (Eds.), Re-examining pedagogical content knowledge in science education (pp. 88-103). Routledge.
Taber, K. S. (2018). The use of Cronbach’s alpha when developing and reporting research instruments in science education.
Research in Science Education, 48, 1273-1296. https://doi.org/10.1007/s11165-016-9602-2
TIMSS ( Trends in International Mathematics and Science Study). (1999). Science teacher questionnaire:
Main survey. National Center for Education Statistics, U.S. Depar tment of Education.
https://nces.ed.gov/timss/pdf/1999_8th_grade_Science_Teacher_Questionnaire.pdf
TIMSS (Trends in International Mathematics and Science Study). (2015). Teacher questionnaire science Grade 8. National Center
for Education Statistics, U.S. Department of Education.
Tudor, L. S. (2015). Perception of teachers on curriculum integration: Integration patterns practice. Procedia – Social and
Behavioral Sciences, 127, 728-732. http://doi:10.1016/j.sbspro.2014.03.344
661
CONCEPTS
(pp. 651-662) ISSN 2538–7138 /Online/
van Schaik, P., Volman, M., Admiraal, M. & Schenke, W. (2018). Barriers and conditions for teachers’ utilisation of academic
knowledge. International Journal of Educational Research, 90, 50-63. http://doi:10.1016/j.ijer.2018.05.003
William, J. (2012). Using CoRes to develop the pedagogical content knowledge (PCK) of early career science and technology
teachers. Journal of Technology Education, 24(1), 34-53. http://scholar.lib.vt.edu/ejournals/JTE
Received: April 23, 2022 Revised: June 08, 2022 Accepted: July 29, 2022
Cite as: Marake, M., Jita, L., & Tsakeni, M. (2022). Science teachers’ perceptions of their knowledge base for teaching force
concepts. Journal of Baltic Science Education, 21(4), 651-662. https://doi.org/10.33225/jbse/22.21.651
‘Maphole Marake PhD, Research Fellow, School of Mathematics, Science and

(Corresponding author) Technology Education, University of the Free State, South
Africa.
E-mail: mmaphole7@gmail.com
Loyiso Jita PhD, Professor, Dean of the Faculty of Education and SANRAL Chair in
Science and Mathematics Education, University of the Free
State, South Africa.
E-mail: JitaLC@ufs.ac.za
Maria Tsakeni PhD, Senior Lecturer, School of Mathematics, Science and
Technology Education, University of the Free State, South
Africa.
E-mail: tsakenim@ufs.ac.za
662
EFFECTS OF INQUIRY-BASED
CHEMISTRY EXPERIMENTATION
ON STUDENTS’ ATTITUDES ISSN 1648-3898 /Print/
TOWARDS THE TEACHING AND ISSN 2538-7138 /Online/
LEARNING OF CHEMISTRY
Abstract. Students’ attitude towards
a subject is a key indicator of academic
performance, therefore its enhancement is
Christian Bob Nicol, imperative for academic success. A sample
Emmanuel Gakuba, of 328 grade eleven students was selected
Gonzague Habinshuti using cluster random sampling. Two
different study groups; the experimental
and control groups, were composed
and exposed to the guided inquiry and
demonstration methods of teaching
chemistry experiments respectively. The
Non-equivalent control group research
design was employed with instruction
Introduction
lasting six weeks. Data on pre-test and post-
test attitudes were collected using a 25-item
The attitudes of upper secondary school students towards science are
attitude towards teaching and learning
fast becoming a growing concern to science educators. Empirical evidence
chemistry through experiment survey.
indicates that there is a gradual dampening of learners’ interest in chemistry
Comparisons of mean attitude scores
due to wariness (McClary & Bretz, 2012; Potvin & Hasni, 2014). Said et al.
were made between the experimental
(2016) have explained that this wariness can be due to the more abstract
and control groups, and between the
nature of the concepts in upper secondary school curriculums compared to
pre-test and post-test for each group. The
the lower secondary schools, where objects discussed are easily seen. This
Mann-Whitney U test analyses indicated a
loss of interest is reflected in the steady decline in the number of students
higher mean attitude score for the control
opting to pursue studies in chemistry and other science disciplines (Halim
group. However, the difference in the mean
et al., 2018; UNESCO, 2015).
scores was not statistically significant.
The world may now be faced with an imminent threat of a reduction
The Wilcoxon Signed-Rank test analyses
in the supply of science and technology professionals (Galama & Hosek,
indicated statistically significant differences
2008), with grave implications for national and global scientific innovations
between the pre-test and post-test attitude
in diverse areas of development. Thus, the pursuit of developing more posi-
mean scores for both independent groups.
tive attitudes towards learning science should be a crucial goal of science
These outcomes show that both methods
education worldwide (Gonzalez, 2016). Little wonder the last decade has
of teaching chemistry through experiments
witnessed an increase in the advocacy by education reformers for a shift
enhance students’ attitudes to essentially
towards a more learner-friendly science classroom environment that will
the same extent. These findings imply that
enhance learners’ positive attitudes to the subject.
Liberian teachers can enhance students’
Attitude is a measurable attribute that can be expressed through a
attitudes through demonstrations.
continuum of answer alternatives on a survey instrument. It can be ex-
pressed through a person’s opinions as a like or dislike or as an expression Keywords: chemistry experiments,
of the extent of like or dislike about an object or a process (Cheung, 2007). demonstration, guided inquiry, non-
Research outcomes have suggested a direct correlation between learners’ equivalent control group, students’ attitude
attitudes towards a subject and the academic performance thereof (Bassey
et al., 2009). Also, innovative teaching strategies are linked to students’
Christian Bob Nicol,
enthusiasm, motivation, and general attitudinal change towards science Emmanuel Gakuba,
subjects in the research literature. If students show an inclination to or Gonzague Habinshuti
liking for a science subject in school, they tend to make an effort to learn University of Rwanda, Rwanda
and understand the meaning of concepts that they are taught (Akbas &
663
(pp. 663-679) ISSN 2538–7138 /Online/
Kan, 2006; Brandriet et al., 2011; Freedman, 1997; Salta &Tzougraki, 2004; Weinburgh, 1995). Teachers, therefore,
have a responsibility to make lesson presentations appeal to learners; this may arouse their interests and extend
their attention spans.
Notwithstanding, the research literature has revealed that students’ attitudes towards learning chemistry
largely depend on the techniques of teaching in the laboratory even though the relationship between experience
in manipulation and skills in the laboratory and learning science has not been established. Lessons loaded with
facts unconnected with their origins in science have been cited in the literature as responsible for the decline
in learners’ attitudes towards the natural sciences (De Vos et al., 2002). Attitudes, being multidimensional, seem
to be affected by very many factors, which makes it imperative to explore the construct in different settings and
contexts. Although the subject of students’ attitudes has been widely discussed in the literature, it continues to
attract more attention because it is an important indicator of learning.
A large number of studies of learners’ attitudes towards science have been conducted in the past four
decades. For instance, Zudonu and Njoku (2018) have used guided inquiry teaching, demonstration, and conven-
tional teaching methods to study their effects on learners’ attitudes towards teaching and learning chemistry in
Nigeria. They found that the guided discovery led to significantly greater positive attitudes towards the teach-
ing and learning of chemistry. This outcome is corroborated by that of Hoftein et al (2004), and Burnterm et al.
(2014), who employed guided inquiry teaching versus structured inquiry methods of teaching experiments in
Israel and Thailand respectively. However, in Cheung’s (2009) study that involved students in Hong Kong, the
students’ positive attitudes towards chemistry were only marginal.
Kuo et al (2018) examined the effects of high and low achieving eight grade students’ motivation toward
science learning in Taiwan. The authors realized a significant jump in the outcome variables for the high achiev-
ing learners compared to the low achievers. In addition, while the high achievers made significant gains in their
expectancy and learning strategies, low achievers enhanced their confidence, and value for science learning
as well as achievement goals, learning strategies, and perception of the learning environment. However, these
gains were not associated with statistical significance unlike those of the high achievers. The authors realized
that the results could have been different if the intervention lasted longer than 6 weeks. These results indicate
that inquiry-based instruction benefits learners differently, based on students’ academic levels of achievement.
Research findings have also revealed that learners not only show different levels of attitudes towards
chemistry (Hao, 2020) but that these attitudes are affected among other factors by the amount of subject matter
content and teachers’ approach to teaching (Méndez, 2006). According to Sata and Tzougraki (2004), the attitudes
of eleventh-grade students in Greece regarding the difficulty of chemistry lessons were found to be related
to concepts, symbols, and problem-solving. The students’ attitudes regarding their interest in chemistry were
neutral. The positive attitudes were traced to students’ ability to relate the knowledge of chemistry to everyday
application in real-life situations. However, only 4% of the students indicated motivation to continue studying
chemistry for a career. Science and chemistry teachers need to be sufficiently trained in teaching techniques
that will positively impact learners’ attitudes and raise their levels of motivation to study chemistry.
Under virtual and real laboratory conditions, where high school students were exposed to an expository
style of teaching in reaction kinetics, Winkelmann et al. (2014) reported that students’ scores in the real and
virtual settings were similar in attitudes. Although report writing was challenging for both groups, activity in the
physical laboratory took longer to complete. The results showed that the presence of physical materials in science
laboratories may not be necessary if virtual simulations of the experiments can be afforded. In another related
study using online laboratory resources, open, guided, and structured levels of inquiry were experimented with
in practical chemistry lessons in India (Nedungadi et al., 2015). In this study, significantly higher achievements
were made in students’ motivation and interests in science.
Nevertheless, in a systematic review of the literature that spanned between 2006 and 2016 on the effects
of various didactic interventions on students’ attitudes towards science, Aguilera and Perales-Palacios (2018)
found a strong effect size of 0.54 associated with the context-based teaching, inquiry-based teaching, model-
based teaching, project-based learning, and cooperative learning approaches. The authors found that these
alternative teaching methods enhanced students’ attitudes towards science, more greatly than the demonstra-
tion methods of teaching.
664
ISSN 1648–3898 /Print/ EFFECTS OF INQUIRY-BASED CHEMISTRY EXPERIMENTATION ON STUDENTS’ ATTITUDES
ISSN 2538–7138 /Online/ (pp. 663-679)
Research Problem

Students’ attitudes towards science teaching and learning have been identified as a determinant of their
academic performance (Najdi, 2013). Upper-secondary school students’ attitudes towards sciences also affect
their decisions for enrolment into science disciplines or the choice of a career path (Hofstein & Naaman, 2011). In
Liberia, evidence exists of unsatisfactory students’ performances in the West African Senior Secondary Certificate
Examination (WASSCE), since the end of the fifteen-year civil war, which ended in 2003 (Hinneh & Nenty, 2016; LSF,
2012; MOE, 2016). The West African Examinations Council’s (WAEC) Chief Examiners’ reports, which do not present
statistical measures of candidates’ performances, describe students’ performance in chemistry variously as “very
poor” WAEC (2017 p.65) and “below average” (WAEC, 2016 p.41).
A cumulative effect of the numerous challenges in the Liberian educational system on students’ learning out-
comes was realized in August 2013, when the Liberian Ministry of Education pronounced that all 25,000 candidates
who sat for the University of Liberia’s entrance examination failed (Chan et al., 2015; Gberie & Mosley, 2016). This
level of challenge in the Liberian system of education probably means that Liberia has the weakest system of educa-
tion in West Africa (Gberie & Mosley, 2016). The cause(s) for this decline has not yet been empirically studied (MOE,
2016), and there is no research output on upper secondary school students’ attitudes towards science teaching
and learning as a factor that could contribute to their academic achievements. It, therefore, seems imperative to
study upper secondary school students’ attitudes towards teaching and learning chemistry through experiments.
Research Focus
The aforementioned challenges in the education system have informed the decision to bridge the gap in
knowledge by studying the effects of inquiry-based teaching on students’ attitudes towards teaching and learning
chemistry through experiments in Liberia. The preference for grade eleven in this study is on the basis that, being
in the mid-secondary school, they truly represent the secondary school experience that was cardinal in this study.
Besides, the teachers in the eleventh-grade class were more receptive to manipulation of the method of teaching
than in the twelfth grade, where teachers were nervous about the completion of the WASSCE syllabus with their
students in preparation for the high school leaving regional examinations. At the time of data collection, the tenth-
grade students were just a few weeks old in their new classes.
Research Aim and Research Objectives
This research aims to determine the effect of inquiry-based chemistry laboratory teaching methods on the
attitudes of grade eleven students towards teaching and learning chemistry. The following specific objectives
guided and focused the study.
a) To compare the post-test mean scores of the experimental and control groups on the Attitudes towards
the Teaching and Learning of Chemistry through Experiments (ATLCE).
b) To compare the experimental group’s pre-test and post-test mean scores of the Attitudes towards
Teaching and Learning Chemistry through Experiments (ATLCE).
c) To compare the control group’s pre-test and post-test mean scores of the Attitudes towards Teaching
and Learning Chemistry through Experiments (ATLCE).
These objectives lead to the following research questions
1. How do the experimental and control groups compare on attitudes toward teaching and learning
chemistry?
2. How do the attitudes toward teaching and learning chemistry pre-test and post-test mean scores
compare in the experimental group?
3. How do the attitudes toward teaching and learning chemistry pre-test and post-test mean scores
compare in the control group?
665
(pp. 663-679) ISSN 2538–7138 /Online/
General Background

The Non-equivalent Control Group Design, illustrated in Table 1 was used. This design is a form of the Quasi-
experimental Design, which allows non-random assignment of subjects to study groups. A research permit to
conduct a study in Bong County was granted by the Liberian Ministry of Education. Thereafter, a preliminary
visitation of schools was made to acquaint the school’s administrators with the research team (researcher and
two research assistants) for subsequent interactions, and to make an assessment of the availability and ideality of
the schools’ facilities for the study, since it was laboratory-based. Thereafter, separate teacher training workshops
were conducted for the Inquiry and demonstration methods teachers. This was followed by a pilot test of the
instrument, and subsequent administration of the pre-test to experimental and control groups, which was done
concurrently. Teaching lasted six weeks after the pre-test and was followed by the concurrent administration of
the post-test in the seventh week.
Table 1
Non-equivalent Control Group Design
Group Pre-test Intervention (6 weeks) Post-test (7th week)
Experimental ATLCE Inquiry-based experimentation ATLCE

Control ATLCE Traditional demonstration ATLCE
Study Participants
12 schools from a population of 30 upper secondary schools in Bong County were initially selected purposively
based on the ideality and readiness for the experimentations. Therefore, the initial 12 schools had either the space
or the materials needed for these experiments. Thereafter, using the Lottery method, the names of four schools were
drawn to constitute the experimental group, followed by the second draw of a set of four schools to constitute the
control group. A sample of 328 (170 males and 158 females) eleventh-grade students was drawn from a popula-
tion of 1754. These students, whose average age was 17, represented those who are being taught using a newly
implemented competency-based curriculum that came into effect in 2018. This sample size is considered statisti-
cally valid by the Morgan and Krajcik’s (1970) Table of Required Sample Sizes calculated at a 95% confidence limit
and .05 margin of error. All the grade eleven students of the selected eight upper secondary schools constituted
the sample. Participating teachers and students signed informed consent before intervention.

The attitude towards teaching and learning chemistry through experimentation (ATLCE), adapted from Cheung
(2009), comprised 25 items. It consisted of five subscales: students’ motivation to learn chemistry by experimenta-
tion, students’ preferences in chemistry experimentation, students’ behavior in chemistry experimentation class,
and students’ anxiety in chemistry experimentation. Validities of the construct and content were determined by
testing and evaluation professionals at the Cuttington University School of Graduate and Professional Studies in
Liberia. The reliability and stability of the ATLCE were determined through the Cronbach Alpha coefficient com-
putation in SPSS version 26.0. The value of 0.84 obtained indicated acceptable reliability (Fraenkel et al., 2012).
Pre-tests and post-tests were administered to all sampled students concurrently under similar testing conditions.
The attitude surveys were administered to ascertain participants’ attitudes before the intervention. There-
after, in the intervention, the inquiry-based method of teaching for the experimental group on one hand, and
the demonstration method for the control group, on the other hand, were implemented simultaneously. Three
educational contact hours per week were maintained throughout for both groups. There were debriefing sessions
with the inquiry-based methods teachers at the end of every week to allow the teachers to share their experiences
666
ISSN 2538–7138 /Online/ (pp. 663-679)
in the intervention. Professional advice was given to the teachers on areas for improvement. This enhanced the
teachers’ experiences and the quality of lessons subsequently. At the closure of teaching, post-tests, which consti-
tuted reshuffled pre-test attitude subscales were administered to both groups of students. This partly helped to
minimize threats to internal validity due to pre-test sensitization. Consistency in the instrument, the scorers, the
test administrators, and the test administration procedures were strictly observed.
Guided Inquiry Lesson
The experimental and control groups were taught the same topic, Solutions and Solubility, from the eleventh-
grade chemistry national curriculum. The only difference in the intervention between the two groups was the
teaching methods. The inquiry lesson was characterized by the following teaching practices.
1. Students worked in small cooperative groups on the experiments

2. Students designed experiments and used a list of materials that were provided for the investigation.
3. Teachers stepped in to offer timely guidance
4. Teachers responded to questions with leading questions
5. Teachers allowed wait time, so students could process the information before voicing out the answer
to a question.
6. Teachers circulated among the working groups to offer help
Therefore, it differed from the demonstration method of teaching, in which students merely watched the
teacher do all the manipulation of the apparatus and ideas. Following are descriptions of the experiments that
were conducted. Students’ activity worksheets are found in Appendix A.
The Guided Inquiry Experimental Procedure
Every activity started with the teacher asking brainstorming questions to identify and determine learners’
knowledge gaps and misunderstandings of the solutions and solubility. Therefore, using carefully crafted questions,
the teacher elicited students’ previous knowledge and understanding of the Solutions and solubility concept. It also
served to reveal misconceptions. Students were instructed to ask scientific questions that would be answered by
experimentation, design experiments diagrammatically in small cooperative groups, and follow the designs to
experiment, analyze and discuss the results with peers in these groups, and in the larger group, with appropriate
teacher guidance. The students’ activities and cooperative group worksheets are presented in Appendices A and
B respectively.
Data Analysis
Aggregated item scores were fed into Microsoft Excel and later copied into the Statistical Package for
Social science Statistics (SPSS) version 26.0 to test for normality of the distributions of the scores. The outcome
of this test determined the non-suitability of the use of parametric statistics for analysis, since the Shapiro-Wilks
statistic is <.05. This implied that the test scores deviated from a normal distribution. Therefore, the Mann-Whitney
U test, known to be a non-parametric equivalence of the independent samples t-test was used to compare the
mean ranks of the experimental and control groups. Similarly, the Wilcoxon Signed-Rank test, the non-parametric
equivalence of related (dependent) samples t-test was used to compare the mean ranks of the pre-test and post-
test scores for each group, and to determine whether there were any significant differences between the groups’
attitude scores. Effect sizes were calculated with the aid of an online effect size calculator using the formula;
Cohen’s d = (M2 - M1) ⁄ SD (pooled)
SD (pooled) = √ ((SD12 + SD22) ⁄ 2)
Research Results
Following are the results of the statistical analysis conducted in this study, organized in tables.
667
(pp. 663-679) ISSN 2538–7138 /Online/
Table 2
Results of Descriptive Statistics of Study Participants
Groups N M SD Min Max

Post-test
Experimental 163 85.01 9.62 64.00 114.00
Control 165 87.10 12.30 52.00 115.00
Table 2 shows a difference of approximately 2 points in the mean attitudes scores between the experimental
and control groups. Also, a difference of approximately 2.5 was observed in the standard deviations between the two
groups. This implies that there is only a slight variation in the individual attitude scores for both groups. Although
the minimum scores are at a relatively larger variance, the maximum scores for both groups are also identical.
Table 3
Results of Comparison between the Experimental and Control Groups’ ATLCE Post-test Scores
Mann-Whitney U test Test Statistic
Group N Sum of Ranks Mean Ranks U t p
Experimental 163 26554.50 162.91

13188.50 26554.50 .763
Control 165 27401.50 166.07
Total 328
Considering the ATLCE scores in Table 3, the mean rank of 166.07 for the control group is higher than the
experimental group (162.91). Also, a Mann-Whitney U test for significance indicated a p > .05, implying that the
difference in the mean ranks is statistically insignificant at a .05 level of significance.
Table 4
Results of Descriptive Statistics for the Experimental Group’s Pre-test and Post-test Scores
Group N M SD Min Max
Pre-test 170 80.94 15.38 35.00 120.00

Post-test 163 86.34 11.26 60.00 115.00
Examination of Table 4 shows a higher post-test mean score than pre-test for the experimental group by a
difference of 5.51 points. However, the standard deviation is higher for the pre-test scores than for the post-test
one by 4.12 points. This means that the scores were more spread away from the mean at the pre-test than at the
post-test. Also, although there is a relatively large difference between the minimum scores, the maximum scores
were comparable. The Wilcoxon signed-rank test results shown in Table 5 indicate a higher positive mean rank than
that for the post-test. The difference between the mean ranks of pre-test and post-test scores was indicated to be
statistically significant by an indication of p <. 05.
Table 5
Wilcoxon Signed Rank Test Results for Comparing the Experimental ATLCE Group’s Pre-test and Post-test Mean Scores
N Sum of Ranks Mean Ranks p
Negative Ranks 59 4104.00 69.56

Positive Ranks 97 8142.00 83.94 .0001
Post-test-pre-test
Ties 7
Total 175
668
ISSN 2538–7138 /Online/ (pp. 663-679)
Table 5 shows a higher positive mean rank than negative rank. This means that the attitude post-test mean
score for the experimental group was higher than that for the pre-test. Also, the difference between the mean
ranks at an α = .05 level of significance is indicated to be significant, by an indication of a p < .05.
Table 6
Results of Descriptive Statistics of the Control Group’s Pre-test and Post-test Scores
Group N M SD Min Max
Pre-test 165 85.68 12.27 53.00 115.00

Post-test 165 87.10 12.30 52.00 115.00
Table 6 shows a higher post-test attitude mean score than the pre-test. Also, the standard deviations of the
pre-test and post-test scores are comparable. Besides, the minimum and maximum scores for pre-test and post-
test are similar. The scores for the variables are comparable.
Table 7
Results of Comparison between the Experimental and Control groups’ ATLCE Scores
Wilcoxon Signed-Rank Test Test Statistic
N Sum of Ranks Mean Ranks p
Negative Ranks 29 1103.00 69.56

Positive Ranks 79 4783.00 83.94 .0001
Post-test-pre-test
Ties 57
Total 165
Table 7 shows a higher positive mean rank than a negative rank, this shows that the post-test mean score was
higher than that for the pre-test. Also, the difference between the mean scores is statistically significant at α = .05.
Table 8
Effect Size of the Differences between Groups
Study group d Δ g
Experimental group/control group 0.19 0.22 0.19

Experimental group – ATLCE (Pre-test/post-test) 0.40 0.35 0.40
Control group –ATLCE (Pre-test/post-test) 0.12 0.12 0.12
Table 8 indicates the effect size of the differences between the mean scores of the experimental and control
groups, and that between the pre-test and post-test mean scores of both experimental and control groups. These
results show that the effect sizes range between 0.12 and 0.40, which means that the effect sizes are very large
(Cohen, 1988). Accordingly, these results indicate that the differences between the experimental and control groups,
and between the pre-test and post-test mean scores for the experimental and control groups have considerably
important practical relevance.
Discussion
A higher standard deviation of the pre-test than post-test scores of the experimental group indicates that
students were more greatly diverse in their opinion before instruction but tended to agree more among themselves
after the guided inquiry method of teaching. This implies students’ significant change of opinion in the end, which
669
(pp. 663-679) ISSN 2538–7138 /Online/
can reasonably be attributed to the method of instruction. The range of pre-test scores (85) compared with that of
the post-test (55) confirms that the students tended to agree more after teaching. A similar pattern of observation
in the control group shows that the two groups may be similar in the variables of interest in this study.
Also, the wider variation of the control group’s scores than that of the experimental group is a demonstration
of a greater consensus among the experimental group concerning their opinion on the teaching and learning
process. Nearly identical mean scores suggest that the two groups essentially converge in their expressed attitudes
towards teaching and learning chemistry by experiments, despite the difference in the methods of teaching. The
greater attitude demonstrated by the control group while unexpected, may not be unconnected with the chal-
lenges that are usually associated with inquiry-based teaching, where the task to construct scientific knowledge,
and cultivate students’ critical thinking is shifted more towards learners than their teachers. While experimenta-
tion is reputed for enhancing students’ motivation and general attitudes towards science, it is reasonable to think
that students’ liking for the inquiry-based method of teaching may wane due to the rigorous mental exercise in
figuring out experimental designs, and the right terminologies for explaining results. However, this effect may not
have been of great relevance, since the difference in the attitudinal mean scores between the groups is statistically
insignificant. Thus, it seems experimentation, whether conducted using an inquiry approach or not, will enhance
students’ attitudes. This outcome portrays the guided inquiry approach of science teaching as being essentially
equal in potential to influence students’ attitudes. Given that the majority of these students are not exposed to
experiments in Bong County, the similarity in outcomes of attitudes may be attributed to the fact that experiments
just excite and appeal to students, irrespective of the level of students’ engagement, owing to the long anticipation
of it. All they seem to appreciate is the experience of witnessing an experiment; either demonstrated by themselves
or seeing it being demonstrated.
A better indicator of the effectiveness of the teaching methods may be the difference between the pre-test
and post-test mean scores for each group. In this regard, a meaningful gain for both groups was found, which
supports the earlier assertion that both methods of instruction tend to improve students’ attitudes toward the
teaching and learning of chemistry through experiments. Beyond the statistical interpretation of the difference
between the pre-test and post-test mean scores, these differences also have practical relevance by indication of
their effect sizes. The significance of the effect of both teaching approaches in this study is therefore both statisti-
cal and practical, and hence can be used to inform educational reform in Liberian science education programs.
Although the outcome of this study is in agreement with many others in the research literature, which claim
the significant effect of both inquiry and traditional experimentation methods of teaching on learners’ attitudes
(Nedungadiet al, 2015; Zudonu & Njoku, 2018), it is also in disagreement with the findings of many quasi-exper-
imental studies, which claim significant differences between inquiry and traditional teaching methods (Aguilera
& Perales-Palacios, 2018; Kuo et al., 2018; Zudonu & Njoku, 2018). The results in this study imply that chemistry
teachers in Bong County need to expose their students to experiments of some sort, to arouse their students’
interest in, and liking for the subject. This assertion is predicated on the claim that students’ interest in a subject is
positively correlated with their academic performance (Kuo et al., 2018).
This study compares the effect of the guided inquiry and demonstration methods of teaching chemistry
experiments on grade eleven learners’ attitudes toward the teaching and learning of chemistry in Bong County,
Liberia. Using the Non-equivalent control group design, this study demonstrates that after six weeks of teaching,
the guided inquiry and demonstration approaches improve learners’ attitudes towards the teaching and learning of
chemistry through experiments to a comparable extent. The outcome of this study proves a point that practitioners
and researchers should not give up on the demonstration of experiments in chemistry. It may also be reasonably
inferred that the use of a blend of both approaches to instruction may enrich the practical science experience in
upper secondary school by capitalizing on the strengths of each approach. The frequent and widespread use of
experimentation in science classrooms stands to raise students’ motivation and liking for the science disciplines
and improve learning outcomes. Science teachers’ main energies should be directed towards arousing students’
interest in science through purposeful laboratory activities.
The regaining of students’ interest in science will hence reduce the imminent threat of a reduction in science
and technology professionals, whose contributions to national and global development cannot be underestimated.
The outcome of this study adds to the growing amount of evidence of the effect of teaching approaches on learn-
670
ISSN 2538–7138 /Online/ (pp. 663-679)
ers’ attitudes towards science from the Liberian context. It, therefore, contributes to the global debate around
the influence of inquiry-based instruction. These findings imply that teachers in Liberia and other countries that
have resource-challenged science programs could effectively garner students’ attitudes through traditional dem-
onstration. However, one limitation of the study is that it did not relate students’ attitudes to students’ academic
performances in chemistry. Further research needs to compare the effects of the comparison among the prevalent
expository, demonstration, and inquiry-based teaching approaches on students’ attitudes in Liberia.
Acknowledgement
This research was funded by the African Centre of Excellence for Innovative Teaching and Learning Mathemat-
ics and Science (ACEITLMS), College of Education, University of Rwanda.
References
Aguilera, H., & Perales-Palacios, F. J. (2018). What effects do didactic interventions have on students’ attitudes towards science?
A meta-analysis. Research in Science Education, 50(2), 573-597. http://doi.org/10.1007/s11165-018-9702-2
Akbas, A., & Kan, A. (2006). Affective factors that influence chemistry achievement (motivation and anxiety) and the power of
these factors to predict chemistry achievement. Journal of Turkish Science Education, 4(1), 10-19. https://www.researchgate.
net/publication/26459265
Bassey, S. W., Joshua, M. T., & Asim, A. E. (2009). (eds.). Gender Differences and mathematics achievement of rural senior
secondary students in Cross River State, Nigeria. epiSTEME 2 International Conference to Review Research on Science, Technology
and Mathematics Education, Conference Proceedings. India: Macmillan.
Bourdeau, V. D. (2003). The 4-H inquiry in action model. Oregon State University. http://oregon.4h.oregonstate.edu/sites/default/
files/inquiry_in_action_model.pdf
Brandriet, A. R., Xu, X., Bretz, S. L., & Lewis, J. E. (2011). Diagnosing changes in attitude in first-year college chemistry students
with a shortened version of Bauer’s semantic differential. Chemistry Education Research and Practice, 12(2), 271–278.
https://doi.org/10.1039/C1RP90032C
Bunten, T., Lee, K., kong, L., J, N., Srikoon, S., & Vangpoonmyai, R. J., et al. (2014). Do different levels of inquiry lead to different
learning outcomes? A comparison between guided and structured inquiry. International Journal of Science Education,
36(12), 1937-1959. https://doi.org/10.1080/09500693.2014.886347
Chan, M., Pan, A., Zhao, A., & Zhao, E. (June 18 - July 4, 2015). Enhancing science education in Liberia: A trip to Liberia, Summer
2015. I-HELP.
Cheung, D. (2009). Developing a scale to measure students’ attitudes toward chemistry lessons. International Journal of Science
Education, 31(16), 2185-2203. https://doi.org/10.1080/09500690802189799
Cohen, J. (1988). Statistical power analysis (2nd ed.). Lawrence Erlbaum Associates Publishers.
De Vos, W., Bulte, A. M. W., & Pilot, A. (2002). Chemistry curricula for general education: Analysis and elements of a design. In J. K.
Gilbert (Ed.), Chemical education: Towards research-based practice (pp. 101-124). Kluwer Academic Press.
Fraenkel, J. R., Wallen, N. E., & Hyun, H. H. (2012). How to design and evaluate educational research. McGraw-Hill.
Freedman, M. P. (1997). Relationship among laboratory instruction, attitude toward science, and achievement in science knowledge.
Journal of Research in Science Teaching, 34(4), 343– 357. https://doi.org/10.1002/(sici)1098-2736(199704)34:4<343: aid-
tea5>3.0.co;2-r
Galama, & Hosek (May 7, 2008). U.S. Competitiveness in Science and Technology. http://www.rand.org/pubs/research_briefs/
RB9347/
Gberie, L., & Mosley, J. (2016). Research and knowledge systems in Liberia. Viewpoints of INASP. https://www.inasp.info>Country
profile-Liberia
Gonzalez, B. (2016). Time for action: “vision and change” implementation in an online biology course at a community college.
Journal of College Science Teaching, 45(4), 15–21. http://www.jstor.org/stable/43748454
Halim, L., Soh, T. M. T., & Arsad, N. M. (2018). The effectiveness of STEM mentoring programs in promoting interest towards STEM.
Journal of Physics: Conference Series, Volume 1088, The 6th South East Asia Design Research International Conference (6th
SEA-DR IC)27–28 June 2018, Banda Aceh, Indonesia. Conf. Ser. 1088 012109.
Hao, Q., Barnes, B., & Jing, M. (2020). Quantifying the effects of active learning environments: separating physical learning
classrooms from pedagogical approaches. Learning Environments Research. http:// doi.org/10.1007/s10984-020-09320-3
Hinneh, J. T., & Nenty, H. J. (2016). Analysis of the predictive validity of students’ performance in core sciences on Liberian
SHSCE with WASSCE as the criterion. Journal of Research & Method in Education (IOSR-JRME), 6(4), 25-31. https://doi.
org/10.9790/7388-0604042531
671
(pp. 663-679) ISSN 2538–7138 /Online/
Kennedy, J., Quinn, F., & Taylor, N. (2016). The school science attitude survey: A new instrument for measuring
attitudes towards school science. International Journal of Research & Method in Education, 39(4), 422-445.
https://doi.org/10.1080/1743727X.2016.1160046
Kopparla, M., Bicer, A., Vela, K., Lee, Y., Bevan, D., Kwon, H., …Capraro, R. M. (2018). The effects of problem-posing intervention
types on elementary students’ problem-solving. Educational Studies, 1–18. https://doi.org/ 10.1080/03055698.2018.1509785
Kuo, Y.-R., Tuan, H.-L., & Chin, C.-C. (2018). Examining Low and Non-low Achievers’ Motivation Towards Science Learning
Under Inquiry-Based Instruction. International Journal of Science and Mathematics Education, 17(5), 845-862.
https://doi.org/10.1007/s10763-018-9908-9
Lemov, D. (2010). Teach like a champion-49 teaching techniques that put students on the path to college. Jossey- Bass.
Liberian Science Foundation. (2012). Needs assessment study of Liberia’s science education, Monrovia: Global hands.
McClary, L. M., & Bretz, S. L. (2012). Development and assessment of a diagnostic tool to identify organic chemistry
students’ alternative conceptions related to acid strength. International Journal of Science Education, 34(15), 2317–
2341. https://doi.org/10.1080/09500693.2012.684433
Méndez, G. (2006). Using students’ cultural heritage to improve academic achievement in writing. Multicultural Perspectives, 8(4),
29–38. https://doi.org/10.1207/s15327892mcp0804_6
Ministry of Education (2016, 15th December). Getting to the best education sector plan 2017-2021. https:www.moe.gov.lr
Nedungadi, P., Malini, P., & Raman, R. (2015). Inquiry-based learning pedagogy for chemistry practical experimentsusingOLabs.
In Advances in intelligent informatics (pp. 633-642). Springer, Cham
Orodho, J. A. (2009). Elements of education and social science research methods. Kanezja Publisher.
Potvin, P., & Hasni, A. (2014). Interest, motivation and attitude towards science and technology at K-12 levels: A systematic review of
12 years of educational research. Studies in Science Education, 50(1), 85–129. https://doi.org/10.1080/03057267.2014.881626
Said, Z., Summers, R., Abd-El-Khalick, F., & Wang, S. (2016). Attitudes toward science among grades 3 through 12 Arab
students in Qatar: findings from a cross-sectional national study. International Journal of Science Education,
https://doi.org/10.1080/09500693.2016.1156184
Salta, K., & Tzougraki, C. (2004). Attitudes toward chemistry among 11th-grade students in high schools in Greece. Wiley Inter-
Science, 536-547. https://doi.org/10.1002/sce.10134
UNESCO (2015). Global Education Monitoring Report. https://en.unesco.org/gem- report/taxonomy/term/199
Weinburgh, M. (1995). Gender differences in student attitudes toward science: A meta-analysis of the literature from 1970 to
1991. Journal of Research in science Teaching, 32(4), 387-398. https://doi.org/10.1002/tea.3660320407
West African Examinations Council. (2016). Chief Examiners’ Report. Monrovia: WAEC. http://www.liberiawaec.org
West African Examinations Council. (2017). Chief Examiners’ Report. Monrovia: WAEC. http://www.liberiawaec.org
Winkelmann, K., Scott, M., & Wong, D. (2014). A study of high school students’ performance of a chemistry experiment within the
virtual world of second life. Journal of Chemical Education, 91(9), 1432-1438. https://doi.org/10.1021/ed500009e
Zudonu, O. C., & Njoku, Z. C. (2018). Effect of laboratory instructional methods on students’ attitudes in some chemistry concepts
at senior secondary school level. GSJ, 6(7), 46. http://www.globalscientificjournal.com/
672
ISSN 2538–7138 /Online/ (pp. 663-679)
Appendix A: Experimental Activities
Activity 1: The effect of particle size of a solid on its solubility in a liquid

Students were provided with the following
4 cubes of sugar
6 test tubes
1 Wash bottles containing water
1 stopwatch
In this activity, students in their groups were given the following instruction.
In your respective group and using the materials provided,
1. Design an experimental procedure to determine the effect of the molecular size of sugar (solute) on its solubility
in water (solvent).
2. Carry out the investigation, make keen observations and record your observations on a sheet. Data should include
a. time taken for the sugar samples to dissolve,
b. temperatures of the water
3. Note your observation
4. What is the practical everyday application of this investigation?
Activity 2: The effect of temperature on the solubility of a solid in a liquid

Students were provided with the following
4 Sugar cubes
1 Thermometer
3 beakers holding water at 3 different temperatures
6 Test tubes
In your respective group and using the materials provided,
1. Design an experimental procedure to determine the effect of temperature on the solubility of sugar (solute) in
water (solvent).
2. Carry out the investigation, make keen observations and record your observations on a sheet. Data should include
a. the temperature of the water in the beakers just before pouring it.
a. time taken for the sugar samples to dissolve,
c. date and time, and
d. names of group members.
Activity 3: The effects of intermolecular forces on miscibility and solubility of one liquid in another
The materials provided were as follows;
6 test tubes
5 beakers, one each containing water, kerosene and vegetable oil, vinegar, green alcohol
3 test tube holders
In your respective group and using the one set of materials at a time,
1. Design an experimental procedure to determine the effect of intermolecular forces on the solubility of two liquids.
2. Carry out the investigation, make keen observations and record your observations on the activity sheet. Data
should include responses to the following
a. which two liquids are miscible?
b. why are the two liquids miscible?
c. which liquids are immiscible
673
(pp. 663-679) ISSN 2538–7138 /Online/
Appendix A: investigating the effect of intermolecular forces, and density on the solubility of liquids
Title of Activity 4: distinguishing between Solutions and Suspensions
The students were provided with the following

4 cubes of sugar
2 test tubes
1 Wash bottles containing water
Powdered chalk
Students in their groups were instructed for activity four.
In your respective group and using the materials provided, perform an experiment that shows the differences
between a solution and a suspension.
1. Carry out the investigation, make keen observations and record your observations on a sheet. Data should
include responses to the following
a. Which of the two leaves a residue on a filter paper after filtration?
b. Which one can allow particles to settle after putting them aside in a test tube for 10 minutes?
c. A test tube containing one of them can be used if inclined to read letters in a book. Which one?
3. Summarize the main differences between suspension and solution in your conclusion?
4. What is the practical everyday application of solutions and suspensions?
674
ISSN 2538–7138 /Online/ (pp. 663-679)
Appendix B: Students’ Activity worksheets
675
(pp. 663-679) ISSN 2538–7138 /Online/
676
ISSN 2538–7138 /Online/ (pp. 663-679)
677
(pp. 663-679) ISSN 2538–7138 /Online/
678
ISSN 2538–7138 /Online/ (pp. 663-679)
Received: January 09, 2022 Revised: May 12, 2022 Accepted: July 25, 2022
Cite as: Nicol, C. B., Gakuba, E., & Habinshuti, G. (2022). Effects of inquiry-based chemistry experimentation on
students’ attitudes towards the teaching and learning of chemistry. Journal of Baltic Science Education, 21(4), 663-679.
Christian Bob Nicol PhD Student in Chemistry Education at the African Center of
(Corresponding author) Excellence for Innovative Teaching and Learning Mathematics and
Science (ACEITLMS), College of Education, University of Rwanda,
Rwanda.
E-mail: cbnicol2009@gmail.com
Emmanuel Gakuba Lecturer of Chemistry at the Department of Mathematics, Science
and Physical Education, College of Education, University of Rwanda,
Rwanda.
E-mail: egakuba16@gmail.com
Gonzague Habinshuti Lecturer and Postgraduate Coordinator at the School of Inclusive and
Special Needs Education, University of Rwanda, Rwanda.
E-mail: habinshutihgo@gmail.com
679
APPLYING FACTOR ANALYSIS

FOR ASSESSING KNOWLEDGE
STRUCTURE OF STUDENTS IN
GRADE 10: THE SUBJECT OF
REDOX REACTION
Abstract. Redox reaction is an important Wenxiu Tang,

concept in chemistry, and a well-organized Xintong Zhu,
knowledge structure of redox reaction is
beneficial for concept learning. This study
Yangyi Qian
investigated the knowledge structure
regarding redox reaction from 459 Grade
10 students. The pool of 15 redox reaction
concepts was developed by content
analysis, questionnaire survey, and
interview. Six initial competing models with
Introduction
15 concepts were identified via exploratory
factor analysis (EFA) and paper-pencil
The nature of knowledge is structure (Nakiboglu, 2008). Students need
test. Confirmatory factor analysis (CFA)
to comprehend the meanings and interrelationships of the related concepts
was conducted to test and modify the
to comprehend and acquire knowledge about a topic (Qian, 2008). After
six competing models according to the
students cognitively process these interrelated concepts, these concepts
rating data of the students. As a result, six
will be stored in their minds in a certain organizational form, which is the
modified models fit the data well. However,
knowledge structure (Burrows & Mooring, 2015). Scientific, logical, and
the high inter-factor correlations indicate
explicit knowledge structures are easier to invoke, allowing students to
that the two- and three-factor models
execute relevant tasks with greater flexibility and speed. In many stud-
are the students’ knowledge structures of
ies, knowledge structure is also called conceptual structure (Kurt, 2013a,
redox reaction. The two-factor model is
2013b) or cognitive structure (Atabek-Yigit, 2015; Gercek, 2018; Tasci &
comprised of two distinct but correlated
Yurdugul, 2017).
factors: the process of redox reaction
Chemistry knowledge encompasses a vast array of topics, each cover-
and metrology. The three-factor model
ing many related concepts. Students who do not understand the signifi-
is comprised of three factors: the process
cance of and connections between related concepts can’t fully grasp a topic
of redox reaction, reaction ability, and
(Nakiboglu, 2008). Redox reaction is widely regarded as one of the most
metrology. The finding inflects the abstract
difficult chemistry subjects in the teaching and learning process (Chiang et
relationships between the concepts related
al., 2014; Qian, 2009). The topic of redox reaction contains many concepts
to redox reaction in students’ minds.
that are textually similar but have different meanings, which confuses
Keywords: redox reaction, chemistry
students and makes it difficult to correctly understand the meanings and
education, knowledge structure, factor
interrelationships of the concepts (Delisma et al., 2019).
analysis
Moreover, one reason that students struggle in advanced courses such
as electrochemistry is that their knowledge structures of redox reaction
Wenxiu Tang are inadequate and incoherent (Burrows & Mooring, 2015). Therefore, it
South China Normal University, P. R. China
Guangdong Panyu Senior High School, is critical to reveal how students comprehend and organize the concepts
P. R. China related to redox reaction, that is, the knowledge structure of redox reaction.
Xintong Zhu, Yangyi Qian The existing literature on redox reaction focuses on revealing miscon-
South China Normal University, P. R. China
ceptions of students (Brandriet & Bretz, 2014b; Ferouni et al., 2012; Masykuri
680
ISSN 1648–3898 /Print/ APPLYING FACTOR ANALYSIS FOR ASSESSING KNOWLEDGE STRUCTURE OF STUDENTS IN
ISSN 2538–7138 /Online/ (pp. 680-693)
et al., 2019), the effects of a certain teaching intervention (Basheer et al., 2017; Cole et al., 2019; Seyhan et al.,
2008; Syawal & Amanatie, 2019), and the development of inventories on examining the psychometric properties
(Jin et al., 2020), understandings and confidence of students (Brandriet & Bretz, 2014a). However, few studies
have examined the knowledge structure of redox reaction held by students (Chiang et al., 2014), especially us-
ing factor analysis.
Knowledge Structure and Factor Analysis
In the literature, a variety of research methods have been conducted on the study of knowledge structure
in science education. The methods include concept map (Adamov et al., 2009; Burrows & Mooring, 2015; Chiang
et al., 2014), word association (Bahar & Tongac, 2009; Nakiboglu, 2008), phenomenological approach (Choi & Oh,
2021; Tóth & Ludányi, 2007), think-aloud interview (Ahmadian et al., 2019; King et al., 2022), multi-dimensional
scaling (MDS) (Chiou & Anderson, 2010; Tilga et al., 2017), knowledge space theory (Segedinac et al., 2018),
pathfinder network algorithm (Casas-Garcia & Luengo-Gonzalez, 2013), factor analysis (Mai, Qian, Li et al., 2021),
and reaction time technique (Mai, Qian, Lan et al., 2021).
It is worth noting that factor analysis has long been an important statistical tool in the social sciences (Mon-
toya & Edwards, 2021), but is rarely used in science education. Recently, Mai, Qian, Li et al. (2021) has successfully
used factor analysis for uncovering students’ conceptual structure regarding chemical equilibrium. Additional
studies are still necessary, especially in chemistry, that examine factor analysis as an assessment tool (Burrows &
Mooring, 2015; Tóth & Ludányi, 2007). As such, we used factor analysis to explore students’ knowledge structure
of redox reaction.
Factor analysis is a statistical technique to extract common factors from variables, which can reveal the latent
and representative factors among various variables. Factor analysis is comprised of exploratory factor analysis
(EFA) and confirmatory factor analysis (CFA). EFA is a critical analysis to reveal the unknown relationship between
observed and latent variables (Toma, 2021), whereas CFA is the subsequent level of analysis to test, compare,
and modify the models derived from EFA or theoretical assumptions (Hair, et al., 2010; Harrington, 2009).
Researchers generally use CFA to test whether the data match the existing models (Chan et al., 2022). If no
model has been proposed, EFA can be used to explore the relationships between variables (model), and then CFA
is employed to test and modify the model (Alotaibi & Alotaibi, 2021; Pereira et al., 2021). This research adopted
the latter analytical procedure since there is no literature on using factor analysis to investigate the relationships
between the concepts related to redox reaction.
The instrument used for factor analysis is a rating scale. In the study focusing on the knowledge structure
of science education, the items in the rating scale are usually concepts highly related to a certain topic (the
concept pool). A high-quality concept pool is critical in knowledge structure research. A concept pool contain-
ing multiple and chaotic concepts may obscure vital information about the knowledge structure, while few and
incomplete concepts cannot reveal the comprehensive picture of the knowledge structure.
In previous studies, researchers have selected a few concepts as the concept pool of redox reaction based
on experience (Chiang et al., 2014; Jin et al., 2020), making it difficult to obtain a meaningful and comprehensive
structure. Moreover, the negative effects of personal subjective judgment of such a concept pool are so strong
that the knowledge structure is difficult to genuinely reflect the objective reality in students’ minds. Therefore,
it is important to obtain an objective and unbiased concept pool of redox reaction before exploring the knowl-
edge structure held by students.
Research Questions
Existing literature lacks the use of factor analysis to explore the knowledge structure of redox reaction and
lacks an objective and unbiased concept pool of redox reaction. An appropriate concept pool improves the quality
of the knowledge structure. This research aimed to utilize factor analysis to reveal tenth-grade students’ knowl-
edge structure of redox reaction. The specific research issues that this research tried to answer were as follows.
1. What is the concept pool of redox reaction?
2. What is the knowledge structure of redox reaction acquired by students in Grade 10?
681
(pp. 680-693) ISSN 2538–7138 /Online/
General Background
This research was a quantitative survey in which 333 upper-secondary school chemistry teachers, 36 postgradu-
ate students, 8 chemistry professors, and 459 tenth-grade students in China participated during the 2017-2021
academic year. This research was separated into two sections. In the first section, the concept pool of redox reac-
tion was established through content analysis, questionnaire survey, and interview. In the second section, initial
competing models were identified via EFA and paper-pencil test, and then were tested and modified by CFA with
the rating data of tenth-grade students.
Participants
This study included six samples of participants. Their demographic information is presented in Table 1. The
data from six samples were utilized to create the concept pool of redox reaction. Meanwhile, the data from Sample
6 were also analyzed by factor analysis to obtain knowledge structure of redox reaction.
Samples 1, 2, and 5 were upper-secondary school chemistry teachers from all over China. Sample 3 consisted
of postgraduate students majoring in Chemistry Teacher Education at South China Normal University, and Sample
4 was composed of chemistry professors at South China Normal University. Grade 10 students in Sample 6 from five
upper-secondary schools were involved in this study after they had learned the content of redox reaction in the
compulsory curriculum of chemistry. For the sake of convenience, the following text will refer to upper-secondary
school students in Grade 10 as students and upper-secondary school chemistry teachers as teachers.
Based on the authors’ explanation, all participants understood the purpose of the current study and volun-
teered to take part.
Table 1
Participants’ Demographic Information
Age Gender Teaching experience

Recovery rate
Sample N
(%)
M SD Male Female M SD
Sample 1 76 76.0 34.07 5.56 42 34 17.92 6.22
Sample 2 91 91.0 34.19 5.34 51 40 17.68 6.06
Sample 3 36 90.0 24.17 1.18 2 34 - -
Sample 4 8 - 47.75 4.27 6 2 19.75 3.24
Sample 5 166 66.4 38.86 7.44 102 64 17.99 6.10
Sample 6 459 76.7 15.03 .51 251 208 - -
Instruments and Procedures
The instruments in this study were an open-ended questionnaire and two 7-point Likert scales. To collect the
concepts related to redox reaction, content analysis and an open-ended questionnaire survey were conducted.
Thirty-eight concepts were collected through content analysis. The contents included three versions of Chinese
upper-secondary school chemistry compulsory textbooks (Song, 2007; Wang, 2007; Wang, 2014), chemistry cur-
riculum standard of China (MOE, 2003), and the examination syllabus (National Education Examinations Authority,
2017). Besides, 30 concepts were elicited from 76 teachers (Sample 1) by an open-ended questionnaire survey.
After sorting out the above-mentioned results and removing duplicates, 41 concepts were collected as items.
Three efforts had been done to screen concepts. First, a 7-point Likert scale comprised of 41 items was de-
veloped and distributed to 91 teachers (Sample 2) and 36 postgraduate students (Sample 3). They were asked to
rate the relatedness between 41 items and oxidation-reduction reaction, ranging from 1 (the least relevant) to 7
682
ISSN 2538–7138 /Online/ (pp. 680-693)
(the most relevant). Taking the average score > 5 as the cut-off, 32 items were obtained.
Second, 8 chemistry professors (Sample 4) and 10 teachers from Sample 1 agreed to verify the results and
identify the concepts most related to redox reaction in an interview. As a consequence, 16 concepts were sorted
out from 32 concepts.
Third, a 7-point Likert scale with 16 items was delivered to 166 teachers (Sample 5) and 459 students (Sample
6), who were asked to value the relatedness between 16 items and redox reaction, ranging from 1 (the least relevant)
to 7 (the most relevant). By taking the average score > 5 as the standard, 15 items were eventually identified and
recognized as the concept pool of redox reaction.
The knowledge structure regarding redox reaction of students was obtained in two steps. To begin, six rival
models were identified via EFA and paper-pencil test. Furthermore, CFA was conducted to test and modify the six
rival models.
Data Analysis
According to the rating data of the four samples (Sample 2, 3, 5, and 6), descriptive statistics of SPSS 21.0 was
employed to calculate the average scores of items. Researchers inspected the rating data and discovered that full
scale (from 1 to 7) had been used by both teachers and students. The items with an average score of greater than
5 were used to determine the concepts related to redox reaction.
459 students (Sample 6) were randomly divided into two groups (Sample 6a and Sample 6b). The data from
Sample 6a (N=230) were analyzed with EFA to explore the relationships between variables (model). The initial fac-
tors and their model were extracted using the principal axis factoring and Promax rotation method in SPSS 21.0.
The data from Sample 6b (N=229) were analyzed with CFA to test and modify the models acquired by EFA and
paper-pencil test (Pereira et al., 2021). The model parameters were estimated by employing the Robust Maximum
Likelihood Estimation (MLR) of Mplus 7 (Mai, Qian, Li et al., 2021).
Research Results
The Concept Pool of Redox Reaction
Through content analysis and an open-ended questionnaire, 41 concepts were collected. A total of 26 concepts
were deleted through three rounds of concept screening.
Nine items with an average score of less than 5 were deleted in the first round, and in the second round, 16
items were eliminated according to the suggestions of teachers and professors, as shown in Table 2. The items
“double-track bridge method” and “single-track bridge method” are the methods summarized by Chinese upper-
secondary school chemistry teachers to express the electron transfer in a redox reaction.
In the final round, the item “metallic corrosion” was eliminated since its average scores given by both teach-
ers and students are less than 5, as shown in Table 3. In the end, the remaining 15 concepts were regarded as the
concept pool of redox reaction. These concepts were coded in Table 3 for the convenience of subsequent research.
It is worth noting that the concept “conservation of gain and loss electrons” refers to the fact that the number
of gain electrons equals the number of loss electrons in a redox reaction. This concept is summarized by Chinese
upper-secondary school chemistry teachers. Students are familiar with this concept because it is utilized frequently
in the practice of chemistry teaching in upper-secondary schools in China.
Table 2
The Deleting Items in the First and Second Rounds of Concept Screening
Item Round Item Round
Double-track bridge method First Strong reducing agent Second

Single-track bridge method First Weak reducing agent Second
Endothermic reaction First Strong reducing ability Second
Exothermic reaction First Weak reducing ability Second
683
(pp. 680-693) ISSN 2538–7138 /Online/
Item Round Item Round
Maximum oxidation number First Combustion Second
Minimum oxidation number First Explosion Second
Energy First Metal activity Second
Smelting First Slow oxidation Second
Balancing equations First Gain of oxygen Second
Strong oxidizing agent Second Loss of oxygen Second
Weak oxidizing agent Second Oxidizing Second
Strong oxidizing ability Second Reducing Second
Weak oxidizing ability Second
Table 3
The Average Scores of Items in the Survey
Teachers Students
Code Item
M SD M SD
1 Conservation of gain and loss electrons 6.56 .87 6.12 1.33
2 Electron transfer 6.47 .96 6.04 1.31
3 Oxidizing ability 6.17 1.27 5.63 1.45
4 Oxidizing agent 6.01 1.21 5.57 1.46
5 Number of gain and loss electrons 5.99 1.23 5.86 1.40
6 Oxidation numbers 5.98 1.34 5.63 1.56
7 Oxidation state changes 5.86 1.34 5.88 1.40
8 Reducing agent 5.71 1.29 5.49 1.44
9 Oxidation 5.58 1.38 5.94 1.31
10 Reducing ability 5.51 1.46 5.57 1.41
11 Reduced 5.49 1.37 5.49 1.45
12 Reduction 5.40 1.53 5.78 1.39
13 Oxidized 5.31 1.43 5.38 1.48
14 Oxidation product 5.20 1.32 5.31 1.50
15 Reduction product 5.04 1.45 5.28 1.54
- Metallic corrosion 4.90 1.63 4.55 1.73
The Competing Models of Redox Reaction
The data of Sample 6a were analyzed with EFA, and then a model with two factors emerged. The value of
Bartlett’s test of sphericity is 2905.717 (p < .001) and the value of the Kaiser-Meyer-Olkin (KMO) measure of sample
adequacy is .926 > .70, indicating that the factor analysis is adequate (Mai, Qian, Li et al., 2021). Two factors were
retrieved based on the Eigenvalue larger than 1, which together explained 69.862% of the variance.
684
ISSN 2538–7138 /Online/ (pp. 680-693)
As illustrated in Table 4, factor 1 consists of 10 items with factor loadings ranging from .714 to .907, while
factor 2 contains 5 items with factor loadings ranging from .532 to .937, indicating that the two factors have good
explanatory power for the corresponding items (Oort, 2011).
Table 4
The Model Acquired by EFA
Number of Range of The cumulative variance

Factor Items
items loadings contribution rate
1 10 .714−.907 3, 4, 8, 9, 10, 11, 12, 13, 14, 15 58.727

2 5 .532−.937 1, 2, 5 , 6, 7 69.862
To obtain more competing models, 20 students in Sample 6 were randomly invited to classify the 15 con-
cepts according to their understanding in a paper-pencil test. As a result, six models were recognized. The name
and frequency of each model are shown in Table 5. F2 model refers to a model with two factors, which is identical
to the model derived from EFA. F3a, F3b, and F3c models are three kinds of models with three factors, which are
three different results of splitting factor 1 of the F2 model into two factors. F4a and F4b models are two distinct
models with four factors. F4a model was formed by splitting factor 1 of the F3a model into two factors, while the
F4b model was formed by splitting factor 1 of the F3b model into two factors. All in all, six initial rival models were
prepared for CFA.
Table 5
The Distributions of Concepts in Six Initial Models
Model Factor 1 Factor 2 Factor 3 Factor 4 Frequency
F2 3, 4, 8, 9, 10, 11, 12, 13, 14, 15 1, 2, 5, 6, 7 - - 6

F3a 4, 8, 9, 11, 12, 13, 14, 15 3, 10 1, 2, 5, 6, 7 - 5
F3b 3, 4, 8, 10, 14, 15 9, 11, 12, 13 1, 2, 5, 6, 7 - 4
F3c 3, 4, 9, 13, 14 8, 10, 11, 12, 15 1, 2, 5, 6, 7 - 2
F4a 4, 8, 14, 15 9, 11, 12, 13 3, 10 1, 2, 5, 6, 7 2
F4b 3, 4, 14 8, 10, 15 9, 11, 12, 13 1, 2, 5, 6, 7 1
The Knowledge Structure of Redox Reaction
Numerous fit indices are capable of evaluating the model’s reliability (Schreiber et al., 2006). However, academ-
ics have not established a consensus on which fit index is superior. To avoid misevaluating the models as much as
possible, we combined four fit indices as the evaluation criteria: Root Mean Square Error of Approximation (RMSEA)
＜ .08, Comparative Fit Index (CFI) ＞ .90, Tucker-Lewis Index (TLI) ＞ .90 (Kline, 2005), and Standardized Root Mean
Square Residual (SRMR) ＜ .08 (Hu & Bentler, 1999). As shown in Table 6, the RMSEA, CFI, and TLI of the six initial
models are not within the acceptable range of model fit, indicating that the six models do not adequately match
the data. The modification index by Mplus suggests adding error covariances between some items. The modified
suggestions of both F3a and F4a models are adding error covariances between items “oxidation product” and
“reduction product”, while the modified suggestions of F2, F3b, F3c, and F4b models are adding error covariances
between items “oxidation product” and “reduction product”, items “oxidizing ability” and “reducing ability”. The
specific modified information is shown in Table 6.
Upon further examination, we discovered that two pairs of items that need to be added error covariances
are highly correlated in terms of chemical knowledge. First, the oxidation product and reduction product must be
produced simultaneously in a redox reaction. Secondly, oxidizing ability and reducing ability refer to the ability
of a substance to gain and lose electrons, respectively. Moreover, two error covariances modifications are within-
factor. From these aspects, we accepted the suggestions of the software and used CFA to test each modified model.
685
(pp. 680-693) ISSN 2538–7138 /Online/
Each model was tested once when each corresponding error covariance was added. The fit indices of the six final
modified models fit the data well, as shown in Table 7. Chi-square difference tests were conducted between the six
initial models and the corresponding modified models (Schreiber et al., 2006), and the associated parameters are
listed in Table 8. All p values are less than .001, suggesting that the model fit of the modified models is improved
significantly compared with the matching initial models (Mai, Qian, Li et al., 2021).
Table 6
The Fit Indices of Initial Models Derived from CFA
Model S-B χ2 df RMSEA [90% CI] CFI TLI SRMR Modification suggestions
F2 298.589* 89 .101 [ .089, .114] .864 .840 .047 1, 2
F3a 267.434* 87 .095 [ .082, .108] .883 .859 .045 1
F3b 290.166* 87 .101 [ .088, .114] .868 .841 .046 1, 2
F3c 290.285* 87 .101 [ .088, .114] .868 .841 .047 1, 2
F4a 252.060* 84 .093 [ .080, .107] .891 .864 .045 1
F4b 281.787* 84 .101 [ .088, .115] .872 .840 .046 1, 2

Note: p ＜ .001, modification suggestions “1” and “2” refer to adding error covariances between items “oxidation product” and
*
“reduction product”, items “oxidizing ability” and “reducing ability”, respectively.
Table 7
The Fit Indices of Modified Models Derived from CFA
Model S-B χ2 df RMSEA [90% CI] CFI TLI SRMR
F2-1 210.827* 87 .079 [ .065, .092] .920 .903 .042
F3a-1 210.365* 86 .079 [ .066, .093] .919 .902 .042
F3b-1 203.567* 85 .078 [ .064, .092] .923 .905 .043
F3c-1 192.279* 84 .075 [ .061, .089] .930 .912 .048
F4a-1 202.158* 83 .079 [ .065, .093] .923 .902 .043
F4b-1 196.364* 82 .078 [ .064, .092] .926 .905 .043

Note: p ＜ .001
*
Table 8
The Indices of Chi-square Difference Test
Initial model Modified model cd TRd p
F2 F2-1 3.326 49.046
F3a F3a-1 5.225 21.013
F3b F3b-1 3.207 49.735

＜ .001
F3c F3c-1 3.229 56.594
F4a F4a-1 3.883 23.625
F4b F4b-1 3.353 47.071

Note: “cd” refers to difference test scaling correction, and “TRd” refers to Satorra-Bentler scaled chi-square difference test.
686
ISSN 2538–7138 /Online/ (pp. 680-693)
High correlation coefficients among factors of a model strongly support combining the high-correlation-
coefficients factors (Duffin et al., 2012). In general, inter-factor correlations need to be less than .8 to be considered
as distinct factors (Brown, 2006). However, experts recommended that we use inter-factor correlations of less than
.9 as the criterion because many concepts related to redox reaction are unity of opposites, resulting in strong cor-
relation coefficients among factors.
Table 9 shows the four modified models that include factors with high correlation coefficients. All F3b-1, F3c-1,
and F4b-1 models turned into a new model with two factors after integrating the high correlation factors of F3b-1
(factor 1 and factor 2), F3c-1 (factor 1 and factor 2), and F4b-1 models (factor 1, factor 2, and factor 3), respectively.
Meanwhile, a new model with three factors was formed after integrating factor 1 and factor 2 of the F4a-1 model.
The distributions of concepts in the new model with two factors and the new model with three factors are the
same as the F2-1 and F3a-1 models, respectively. Models with the same distribution of concepts will provide the
same model after fitting the same data and being modified. Therefore, the new model with two factors evolved
into the F2-1 model after modification according to the same rating data, while the new model with three factors
turned into the F3a-1 model. In other words, the F3b-1, F3c-1, and F4b-1 models were converted into the F2-1
model after integrating factors with correlations greater than .9 and matching the rating data, while the F4a-1
model was transformed into the F3a-1 model.
In conclusion, the six modified models in Table 7 fit the data well. It shows that these 15 concepts have six
organizational forms in students’ minds. However, the high inter-factor correlations indicate that the F2-1 and
F3a-1 models are more representative. Therefore, F2-1 and F3a-1 models were accepted as students’ knowledge
structures of redox reaction. The F2-1 model composed of two factors was called two-factor model, while the F3a-1
model comprised of three factors was labeled three-factor model. The structural diagrams of the two models are
depicted in Figure 1 and Figure 2, respectively.
Table 9
The High-Correlation-Coefficients Factors of Modified Models
New model
The high-correlation- Correlation
Modified model
coefficients factors coefficient Number of
Distribution of concepts
factors
F3b-1 Factor 1 and factor 2 .951

F3c-1 Factor 1 and factor 2 .994
Factor 1: 3, 4, 8, 9, 10, 11, 12, 13,
Factor 1 and factor 2 .967 2 14, 15
Factor 2: 1, 2, 5, 6, 7
F4b-1 Factor 1 and factor 3 .924
Factor 2 and factor 3 .958
Factor 1: 4, 8, 9, 11, 12, 13, 14, 15
F4a-1 Factor 1 and factor 2 .933 3 Factor 2: 3, 10
Factor 3: 1, 2, 5, 6, 7
Note: “New model” refers to the model formed due to the changes in the distribution of concepts after merging the highly cor-
related factors.
687
(pp. 680-693) ISSN 2538–7138 /Online/
Figure 1 Figure 2
Two-factor Model Three-factor Model
Discussion
In this study, 15 concepts were selected as the concept pool of redox reaction, and the two- and three-factor
models were regarded as the students’ knowledge structures of redox reaction by using factor analysis.
The Concept Pool of Redox Reaction
Many chemical phenomena and principles involve the knowledge of redox reaction. Therefore, there are many
concepts related to redox reaction. However, only highly relevant concepts play a key role in students’ scientific
understanding of redox reaction. Three rounds of concept screening were carried out to obtain the concepts highly
related to redox reaction (the concept pool). In the process, we adopted three research methods, including content
analysis, questionnaire survey, and interview. On the other hand, we invited a variety of participants, including 333
upper-secondary school teachers, 36 postgraduate students, 8 college professors, and 459 students in Grade 10.
Compared to previous studies on redox reaction (Chiang et al., 2014; Jin et al., 2020), the concept pool identified
in this study is more objective, credible, and representative.
In the interview with teachers and college professors, 16 concepts were eliminated for three different reasons.
To begin, eight concepts are subordinate to others, it is preferable to remove the subordinate concepts to get a
concise concept pool. For instance, both “strong oxidizing agent” and “weak oxidizing agent” are “oxidizing agent”,
but the latter is more general. Thus, we accepted the concept “oxidizing agent” and rejected the concepts “strong
oxidizing agent” and “weak oxidizing agent”. For the same reason, the concepts “strong oxidizing ability”, “weak
oxidizing ability”, “strong reducing agent”, “weak reducing agent”, “strong reducing ability”, and “weak reducing
ability” were deleted, while the concepts “oxidizing ability”, “reducing agent”, and “reducing ability” were accepted.
Secondly, although some concepts are connected to redox reaction, they are ineffective at assisting Grade
10 students in comprehending redox reaction. Therefore, six concepts (“combustion”, “explosion”, “metal activity”,
“slow oxidation”, “gain of oxygen”, and “loss of oxygen”) were rejected. It is noted that students judge redox reaction
688
ISSN 2538–7138 /Online/ (pp. 680-693)
mainly from the perspective of the gain and loss of oxygen in junior high school. However, students realize that
this method is one-sided after the in-depth study of redox reaction in upper-secondary school and rarely use this
method to analyze redox reaction. For upper-secondary school students, the concepts “gain of oxygen” and “loss
of oxygen” are relatively unimportant to understand redox reaction. Accordingly, the concepts “gain of oxygen”
and “loss of oxygen” were eliminated.
Thirdly, both concepts “oxidizing” and “oxidizing ability” relate to the matter’s ability to acquire electrons, and
both “reducing” and “reducing ability” refer to the matter’s ability to lose electrons. To avoid conceptual duplication
and comply with Chinese upper-secondary school textbooks, the concepts “oxidizing ability” and “reducing ability”
were preserved, while the concepts “oxidizing” and “reducing” were abolished.
According to the data in Table 2, the ranges of average scores given by teachers and students for these 15 con-
cepts are 5.04−6.56 and 5.28−6.12, respectively. It shows that both teachers and students consider these concepts
closely related to redox reaction. Therefore, the 15 concepts in the concept pool can apply to reveal knowledge
structure regarding redox reaction of upper-secondary school students.
The Knowledge Structure of Redox Reaction
The six initial competing models with 15 concepts were obtained by EFA and paper-pencil test, which were
tested and modified using CFA. As a result, the six modified models fit the data well. It implies that students’ un-
derstanding of the relationships between these 15 concepts is diverse. However, the high inter-factor correlations
indicate that the two- and three-factor models are more representative. Thus, the two- and three-factor models
were accepted as the knowledge structures of redox reaction held by students. The combination forms of concepts
distinguish the two models. The concepts in factor 1 of the two-factor model were recombined into two more
refined factors to generate the three-factor model.
As illustrated in Figure 1, the factor loadings of each item in the two-factor model range from .691 to .865,
indicating that these factors have good explanatory power for the items (Ximenez, 2016). Factor 1 of the two-factor
model contains ten concepts and was named “the process of redox reaction” by the authors. These concepts are
“oxidizing ability”, “oxidizing agent”, “reducing agent”, “oxidation”, “reducing ability”, “reduced”, “reduction”, “oxidized”,
“oxidation product”, and “reduction product”. The internal relationships of the ten concepts are as follows. Take the
following redox reaction (a net ionic equation) as an example:
Fe(s) + Cu2+(aq) → Fe2+(aq) + Cu(s)
Iron metal (reducing agent) is oxidized to form an iron(Ⅱ) ion (oxidation product), whereas copper ion (Ⅱ)
(oxidizing agent) is reduced to yield copper metal (reduction product). Iron metal is undergoing oxidation and
exhibiting reducing ability while copper ion (Ⅱ) is undergoing reduction and exhibiting oxidizing ability during
the reaction given previously.
Factor 2 of the two-factor model contains five concepts and was named “metrology”. These concepts are
“conservation of gain and loss electrons”, “electron transfer”, “number of gain and loss electrons”, “oxidation num-
bers”, and “oxidation state changes”. The internal relationships of the five concepts are as follows. The essence of
redox reaction is electron transfer, which includes the gain and loss of electrons. In the net ionic equation above,
electrons transfer from iron metal to copper ion (Ⅱ), the oxidizing number of iron metal increases due to the loss
of electrons, while the oxidizing number of copper ion (Ⅱ) decreases as the result of the gain of electrons. The
number of electrons lost is equal to the number of electrons gained, which comprises the meaning of the concept
“conservation of gain and loss electrons”.
As demonstrated in Figure 2, the factor loadings of each item in the three-factor model range from .692 to
.895, showing that these factors have adequately strong explanatory power for the items. The precise names and
implications of the three factors are given as follows.
The name of factor 1 was “the process of redox reaction”, including concepts “oxidizing agent”, “reducing agent”,
“oxidation”, “reduced”, “reduction”, “oxidized”, “oxidation product”, and “reduction product”. In a redox reaction, the
oxidizing agent is undergoing reduction and reduced to produce the reduction product, while the reducing agent
is undergoing oxidation and oxidized to produce oxidation product.
Factor 2 contains two concepts: “oxidizing ability” and “reducing ability”. Oxidizing ability and reducing ability
refer to the ability of a substance to gain and lose electrons, respectively. Therefore, this factor was named “reac-
tion ability”. Factor 3 was called “metrology”, and its concepts and meaning are the same as that of factor 2 in the
two-factor model.
689
(pp. 680-693) ISSN 2538–7138 /Online/
In-depth interviews reveal that although students have the two- and three-factor models, they may not
fully understand the meanings of the 15 concepts and the relationships between them. This is reflected in some
students’ confusion about the meanings of some concepts, such as oxidizing agent and reducing agent, oxidation
product and reduction product. On the other hand, some students indicated that the teachers usually guide them
to analyze the redox reaction through the model in Figure 3, so they could classify these concepts into one category.
In fact, some of them only memorize the model but do not comprehend the scientific connections between the
concepts in the model (Yan, 2011). In other words, the model in Figure 3 may be passively accepted rather than
actively constructed based on understanding by some students. Based on this, teachers can design correspond-
ing teaching tasks to help students correctly understand the meanings and interrelationships of the concepts.
Figure 3
The Model Summarized by Chinese Upper-Secondary School Chemistry Teachers
Reduced, reduction
Oxidizing agent + Reducing agent → Reduction product + Oxidation product
Oxidized, oxidation
The distributions of concepts in the two- and three-factor models are consistent with the two most frequent
models proposed by students in the paper-pencil test. This demonstrates that the models obtained from factor
analysis can reflect the knowledge structures in most students’ minds. Based on this, it can be concluded that factor
analysis can indeed be used to investigate the knowledge structures in the field of science education, particularly
in chemistry. Furthermore, the models obtained by factor analysis can provide more detailed information (Her-
win & Nurhayati, 2021). For example, according to the error covariances between concepts, we can know which
concepts are not only explained by the corresponding factor but also links to each other by unknown variables.
In this study, factor analysis was utilized to investigate the knowledge structure of redox reaction from 459
Grade 10 students. The concept pool of redox reaction contains 15 concepts. The two- and three-factor models
serve as the knowledge structures of students in Grade 10. The two-factor model consists of two components: the
process of redox reaction and metrology. The three-factor model is composed of three components: the process of
redox reaction, reaction ability, and metrology. Overall, this study further confirms that factor analysis is a proper
method to reveal knowledge structure regarding a specific learning topic, especially in chemistry.
Some implications are drawn from the preceding conclusions. First, teachers can use 15 concepts in the concept
pool to evaluate students’ understanding of redox reaction. These concepts are highly related to redox reaction and
play a key role in students’ understanding. Through in-depth interviews, we found that some students may not fully
grasp the meanings and relationships of these concepts. Therefore, teachers can design corresponding identifica-
tion tasks for these 15 concepts to assess students’ comprehension. Then, teachers can conduct interviews with
students to find out the reasons for students’ misunderstanding. Based on the results of the above two sections,
teachers can design appropriate teaching tasks to correct or deepen students’ understanding of redox reaction,
and then assist students in developing more diverse, flexible, and scientific knowledge structures of redox reaction.
Second, further research can be conducted to determine the association between different knowledge struc-
tures regarding redox reaction and the academic achievement of students. This study found that students’ knowledge
structures of redox reaction are diverse. However, the relationship between students’ academic achievement and
their knowledge structures is unknown. We predict that students with different levels of academic achievement
may have different knowledge structures. Therefore, it is necessary to explore the differences in the knowledge
structures of students with high and low academic achievement levels. The results can provide a reference for
teachers to effectively carry out hierarchical teaching designs for students with different academic performances.
Third, it is appropriate to concentrate on the development of students’ knowledge structures of redox reaction.
Students’ understanding of redox reaction will continue to develop with the learning of new knowledge, which
690
ISSN 2538–7138 /Online/ (pp. 680-693)
may have an impact on students’ knowledge structures. Therefore, subsequent researchers can investigate the
knowledge structures regarding redox reaction of upper-secondary school freshmen, sophomores, and juniors,
to provide support for teachers to adopt targeted teaching in each learning stage.
References
Adamov, J., Segedinac, M., Cvjeticanin, S., & Bakos, R. (2009). Concept maps as diagnostic tools in assessing the acquisition and
retention of knowledge in biochemistry. Odgojne Znanosti-Educational Sciences, 11(1), 53-71.
Ahmadian, M., Yazdani, H., & Mehri, E. (2019). The effectiveness of learners’ preferred and unpreferred written corrective feedback:
A think-aloud study. Journal of Asia TEFL, 16(2), 448-467. https://doi.org/10.18823/asiatefl.2019.16.2.1.448
Alotaibi, B., & Alotaibi, A. (2021). Exploratory and confirmatory factor analyses of the Arabic version of the childhood autism
rating scale. Research in Autism Spectrum Disorders, 86, Article 101827. https://doi.org/10.1016/j.rasd.2021.101827
Atabek-Yigit, E. (2015). Exploring the relationship between cognitive structure outcomes and test achievements of
preservice science teachers on chemical bonding via flow mapping. Journal of Baltic Science Education, 14(4), 524-534.
Bahar, M., & Tongac, E. (2009). The effect of teaching approaches on the pattern of pupils’ cognitive structure: Some evidence
from the field. Asia-Pacific Education Researcher, 18(1), 21-45.
Basheer, A., Hugerat, M., Kortam, N., & Hofstein, A. (2017). The effectiveness of teachers’ use of demonstrations for enhancing
students’ understanding of and attitudes to learning the oxidation-reduction concept. Eurasia Journal of Mathematics
Science and Technology Education, 13(3), 555-570. https://doi.org/10.12973/eurasia.2017.00632a
Brandriet, A. R., & Bretz, S. L. (2014a). The development of the redox concept inventory as a measure of students’
symbolic and particulate redox understandings and confidence. Journal of Chemical Education, 91(8), 1132-1144.
https://doi.org/10.1021/ed500051n
Brandriet, A. R., & Bretz, S. L. (2014b). Measuring meta-ignorance through the lens of confidence: Examining students’ redox
misconceptions about oxidation numbers, charge, and electron transfer. Chemistry Education Research and Practice, 15(4),
729-746. https://doi.org/10.1039/c4rp00129j
Brown, T. A. (2006). Confirmatory factor analysis for applied research. New York: Guilford.
Burrows, N. L., & Mooring, S. R. (2015). Using concept mapping to uncover students’ knowledge structures of chemical bonding
concepts. Chemistry Education Research and Practice, 16(1), 53-66. https://doi.org/10.1039/c4rp00180j
Casas-Garcia, L. M., & Luengo-Gonzalez, R. (2013). The study of the pupil’s cognitive structure: The concept of angle. European
Journal of Psychology of Education, 28(2), 373-398. https://doi.org/10.1007/s10212-012-0119-4
Chan, L., Liu, R. K. W., Lam, T. P., Chen, J. Y., Tipoe, G. L., & Ganotice, F. A. (2022). Validation of the world health organization well-
being index (who-5) among medical educators in Hong Kong: A confirmatory factor analysis. Medical Education Online,
27(1), Article 2044635. https://doi.org/10.1080/10872981.2022.2044635
Chiang, W. W., Chiu, M. H., Chung, S. L., & Liu, C. K. (2014). Survey of high school students’ understanding of oxidation-reduction
reaction. Journal of Baltic Science Education, 13(5), 596-607. https://dx.doi.org/10.33225/jbse/14.13.596
Chiou, G. L., & Anderson, O. R. (2010). A multi-dimensional cognitive analysis of undergraduate physics students’ understanding of
heat conduction. International Journal of Science Education, 32(16), 2113-2142. https://doi.org/10.1080/09500690903258246
Choi, K. M., & Oh, I. (2021). A phenomenological approach to understanding sexual minority college students in South Korea.
Journal of Multicultural Counseling and Development, 49(4), 225-238. https://doi.org/10.1002/jmcd.12227
Cole, M. H., Rosenthal, D. P., & Sanger, M. J. (2019). Two studies comparing students’ explanations of an oxidation-reduction
reaction after viewing a single computer animation: The effect of varying the complexity of visual images and depicting
water molecules. Chemistry Education Research and Practice, 20(4), 738-759. https://doi.org/10.1039/c9rp00065h
Delisma, D., Wiji, W., & Widhiyanti, T. (2019). Conception, threshold concept, and troublesome knowledge in redox reaction.
Journal of Physics Conference Series, 1521, 1-6. https://doi.org/10.1088/1742-6596/1521/4/042070
Duffin, L. C., French, B. F., & Patrick, H. (2012). The teachers’ sense of efficacy scale: Confirming the factor structure with beginning
pre-service teachers. Teaching and Teacher Education, 28(6), 827-834. https://doi.org/10.1016/j.tate.2012.03.004
Ferouni, A., Khyati, A., Talbi, M., El Jamali, S., & Radid, M. (2012). Identification of a few difficulties in chemistry in Moroccans high
school: Case of the redox. Procedia Social and Behavioral Sciences, 46, 101-107. https://doi.org/10.1016/j.sbspro.2012.05.075
Gercek, C. (2018). Prospective teachers’ cognitive structures concerning protein synthesis and their degree of understanding.
Journal of Baltic Science Education, 17(1), 19-30. https://doi.org/10.33225/jbse/18.17.19
Hair, Jr. J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2010). Multivariate data analysis: A global perspective (7th ed). Pearson
Education.
Harrington, D. (2009). Confirmatory factor analysis. Oxford University Press.
Herwin, & Nurhayati, R. (2021). Measuring students’ curiosity character using confirmatory factor analysis. European Journal of
Educational Research, 10(2), 773-783.
691
(pp. 680-693) ISSN 2538–7138 /Online/
Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new
alternatives. Structural Equation Modeling-a Multidisciplinary Journal, 6(1), 1-55. https://doi.org/10.1080/10705519909540118
Jin, Y., Rodriguez, C. A., Shah, L., & Rushton, G. T. (2020). Examining the psychometric properties of the redox concept inventory:
A Rasch approach. Journal of Chemical Education, 97(12), 4235-4244. https://doi.org/10.1021/acs.jchemed.0c00479
King, E. C., Benson, M., Raysor, S., Holme, T. A., Sewall, J., Koedinger, K. R., Aleven, V., & Yaron, D. J. (2022). The open-response
chemistry cognitive assistance tutor system: Development and implementation. Journal of Chemical Education, 99(2), 546-
552. https://doi.org/10.1021/acs.jchemed.1c00947
Kline, R. B. (2005). Principles and practice of structural equation modeling (2nd ed). The Guilford Press.
Kurt, H. (2013a). Determining biology teacher candidates’ conceptual structures about energy and attitudes towards energy.
Journal of Baltic Science Education, 12(4), 399-423. https://doi.org/10.33225/jbse/13.12.399
Kurt, H. (2013b). Turkish student biology teachers’ conceptual structures and semantic attitudes towards microbes. Journal of
Baltic Science Education, 12(5), 608-639. https://doi.org/10.33225/jbse/13.12.608
Mai, Y. H., Qian, Y. Y., Lan, H. H., & Li, L. S. (2021). Students’ concept organisation regarding chemical equilibrium in
upper-secondary education: Based on reaction time technique. Journal of Baltic Science Education, 20(3), 443-455.
Mai, Y. H., Qian, Y. Y., Li, L. S., & Lan, H. H. (2021). The conceptual structure of chemical equilibrium in upper-secondary school students:
Evidence from factor analysis. Journal of Baltic Science Education, 20(1), 80-92. https://doi.org/10.33225/jbse/21.20.80
Masykuri, M., Afifa, F. N., & Ashadi. (2019). Students’ misconceptions on basic concept of redox reaction. AIP Conference Proceedings,
2194, Article 020062. https://doi.org/10.1063/1.5139794
MOE (Ministry of Education, PRC). (2003). Putong gaozhong huaxue kecheng biaozhun (Shiyan) [National standard of general
upper-secondary school chemistry curriculum (Experimental version)]. People’s Education Press.
Montoya, A. K., & Edwards, M. C. (2021). The poor fit of model fit for selecting number of factors in exploratory factor
analysis for scale evaluation. Educational and Psychological Measurement, 81(3), 413-440, Article 0013164420942899.
https://doi.org/10.1177/0013164420942899
Nakiboglu, C. (2008). Using word associations for assessing non major science students’ knowledge structure before and after
general chemistry instruction: The case of atomic structure. Chemistry Education Research and Practice, 9(4), 309-322.
https://doi.org/10.1039/b818466f
National Education Examinations Authority. (2017). 2018 Nian putong gaodeng xuexiao zhaosheng quanguo tongyi kaoshi dagang
(Like) [National unified examination outline (Science) of general college entrance examination in 2019]. Higher Education
Press.
Oort, F. J. (2011). Likelihood-based confidence intervals in exploratory factor analysis. Structural Equation Modeling-a
Multidisciplinary Journal, 18(3), 383-396. https://doi.org/10.1080/10705511.2011.582009
Pereira, D. S. M., Pereira, A. M., Castanho, T. C., Silva, G. A., Falcao, F., Costa, P., & Pego, J. M. (2021). Applicability and validation
of the reaction to tests scale (RTT) in a sample of Portuguese medical students. Bmc Psychology, 9(1), Article 166.
https://doi.org/10.1186/s40359-021-00656-w
Qian, Y. (2008). Gaozhong shisheng huaxue xueke guanjianci gainian jiegou de yanjiu [A study on high school teachers and students’
concept structure of keywords in chemistry] [Unpublished doctoral dissertation]. South China Normal University.
Qian, Y., Zhang, J., Luo, X., & Huang, F. (Eds.) (2009). Huaxue gainian yu huaxue ‘xueke guanjianci’ de xuexi yu renzhi [Chemistry
concepts and keywords in chemistry (KIC): Learning and cognition]. Science Press.
Schreiber, J. B., Nora, A., Stage, F. K., Barlow, E. A., & King, J. (2006). Reporting structural equation modeling and confirmatory
factor analysis results: A review. Journal of Educational Research, 99(6), 323-337. https://doi.org/10.3200/joer.99.6.323-338
Segedinac, M. T., Horvat, S., Rodic, D. D., Roncevic, T. N., & Savic, G. (2018). Using knowledge space theory to compare expected
and real knowledge spaces in learning stoichiometry. Chemistry Education Research and Practice, 19(3), 670-680.
https://doi.org/10.1039/c8rp00052b
Seyhan, H. G., Temel, S., & Morgil, I. (2008). The effect of web-based worksheets usage on students’ academic success, logical
thinking skills and attitudes towards chemistry in the teaching of redox topic within chemistry education. Energy Education
Science and Technology, 21(1-2), 91-107.
Song, X. Q. (Ed.) (2007). Huaxue 1 [Chemistry 1] (3rd ed). People’s Education Press.
Syawal, N. M., & Amanatie. (2019). The effects of scientific approach based jigsaw model on students’ self-efficacy and achievement.
Journal of Physics: Conference Series, 1156, Article 012030. https://doi.org/10.1088/1742-6596/1156/1/012030
Tasci, G., & Yurdugul, H. (2017). Biology teaching through self-regulated learning and cognitive structure: An analysis of the effect
of learning strategies for cognitive development via latent growth model. Journal of Baltic Science Education, 16(1), 20-31.
Tilga, H., Hein, V., & Koka, A. (2017). Measuring the perception of the teachers’ autonomy-supportive behavior in physical
education: Development and initial validation of a multi-dimensional instrument. Measurement in Physical Education and
Exercise Science, 21(4), 244-255. https://doi.org/10.1080/1091367x.2017.1354296
Toma, R. B. (2021). Measuring children’s perceived cost of school science: Instrument development and psychometric evaluation.
Studies in Educational Evaluation, 70, Article 101009. https://doi.org/10.1016/j.stueduc.2021.101009
Tóth, Z., & Ludányi, L. (2007). Using phenomenography combined with knowledge space theory to study students’ thinking
patterns in describing an ion. Journal of Baltic Science Education, 6, 27-33.
Wang, L. (Ed.) (2007). Huaxue 1 [Chemistry 1] (3rd ed). Shandong Science and Technology Press.
Wang, Z. H. (Ed.) (2014). Huaxue 1 [Chemistry 1] (5th ed). Jiangsu Education Publishing House.
692
ISSN 2538–7138 /Online/ (pp. 680-693)
Ximenez, C. (2016). Recovery of weak factor loadings when adding the mean structure in confirmatory factor analysis: A simulation
study. Frontiers in Psychology, 6, Article 1943. https://doi.org/10.3389/fpsyg.2015.01943
Yan, L. X. (2011). Tianzige moxing: Yanghua huanyuan fanying de youxiao jiaoxue fenxi moxing chutan [Tianzige model:
An effective teaching analysis model of redox reaction]. Chinese Journal of Chemical Education, 32(8), 20-22.
https://doi.org/10.3969/j.issn.1003-3807.2011.08.009
Cite as: Tang, W., Zhu, X., & Qian, Y. (2022). Applying factor analysis for assessing knowledge structure of students in grade
10: The subject of redox reaction. Journal of Baltic Science Education, 21(4), 680-693. https://doi.org/10.33225/jbse/22.21.680
Wenxiu Tang Master of Pedagogy, Teacher, Guangdong Panyu Senior High School,
No.501, Longqi Road, Shiqiao Street, Panyu District, Guangzhou, P. R.
China.
E-mail: tangwenxiu@m.scnu.edu.cn
Xintong Zhu Postgraduate Student, School of chemistry, South China Normal
University, No.378, West Waihuan Street, Higher Education Mega
Center, Panyu District, Guangzhou, P. R. China.
E-mail: 390779509@qq.com
ORCID: https://orcid.org/0000-0002-0763-719X
Yangyi Qian PhD, Professor, School of chemistry, South China Normal University,
(Corresponding author) No.378, West Waihuan Street, Higher Education Mega Center, Panyu
District, Guangzhou, P. R. China.
E-mail: qianyy@scnu.edu.cn
693
THE EFFECTS OF SCIENCE

TEACHERS’ PEDAGOGICAL
CONTENT KNOWLEDGE
ON STUDENTS’ ATTITUDES
TOWARD SCIENCE AND THEIR
ACHIEVEMENT
Abstract. Pedagogical content knowledge

Muhammet Usak,
(PCK) is one of the most important research
topics in science teacher education. The
Harun Uygun,
purpose of this study was to investigate Muharrem Duran
the PCK of science teachers with different
teaching experiences and examine the
effects of teachers’ PCK on students’
achievement and attitudes toward science.
A mixed methods research design was used
for the study. Participants were science
teachers and their students from two
Introduction
middle schools. Interviews, observation
forms, achievement tests, and attitude
One of the main goals of science teaching is to increase students’
scales were used as instruments for data
interests and achievements in science. A teacher has an important role in
collection. Data were collected and
designing, planning, and implementing a lesson (Hashweh, 1987; Kinskey
analyzed using content analysis, descriptive
& Zeidler, 2021; Njiku et al., 2021; Ozden et al., 2013). Researchers have
and inferential statistics. The findings
agreed that teaching is one of the most influential factors that affect student
obtained from the quantitative and
achievement (Kim et al., 2019). Goldberg (2001) and Guskey (2003) have un-
qualitative data showed some significant
derlined that highly qualified teachers affect students’ learning. Researchers
differences between teachers’ PCKs
(Gess-Newsome & Lederman, 1999; Shulman, 1987) have emphasized that
according to their teaching experience.
the main criterion for successful teachers is to have a solid knowledge base
The results also showed that the PCK of a
that includes a mixture of content knowledge and pedagogical knowledge
teacher with the least experience was lower
or knowledge for teaching.
than that of the other two experienced
In the second half of the 1980s, Shulman (1986 & 1987) claimed that
science teachers. The results indicated
teachers transformed their knowledge to teach in the classroom context and
that experienced teachers who developed
indicated a need to understand how a teacher transformed their content
PCKs did not significantly impact student
knowledge into teaching in the classroom. Pedagogical content knowl-
achievement and attitudes. Educational
edge (PCK) is defined by Shulman (1986 & 1987) as a blend of content and
implications and recommendations for
pedagogy concerned with understanding how topics are effectively taught,
further studies are also provided.
organized, and presented in the classroom. Since Shulman’s defining the PCK,
Keywords: attitudes toward science,
many research studies have been conducted to understand teachers’ PCKs
pedagogical content knowledge, science and how the content knowledge transforms into teaching in the classroom
achievement, science teachers, teaching (Gess-Newsome & Lederman, 1999). Research on the PCK of science teach-
experience ers has shown that teachers who possess similar content knowledge can
implement a specific topic for students in different ways. Researchers have
Muhammet Usak suggested that these differences stem mainly from teachers’ PCKs (Hashweh,
Kazan (Volga Region) Federal University, Russia
Harun Uygun 1987; Käpylave et al., 2009). Researchers have also indicated that the teachers’
The Hague, The Netherlands PCKs have a much more complex structure from this perspective. Because of
Muharrem Duran this reason, scholars (Gess-Newsome & Lederman, 1999; Kutluca & Mercan,
Independent Researcher, Turkey
2022; Loughran et al., 2001) have suggested that more than a single method
694
ISSN 1648–3898 /Print/ THE EFFECTS OF SCIENCE TEACHERS’ PEDAGOGICAL CONTENT KNOWLEDGE ON
STUDENTS’ ATTITUDES TOWARD SCIENCE AND THEIR ACHIEVEMENT
ISSN 2538–7138 /Online/ (pp. 694-705)
is required to evaluate teachers’ PCKs. The researchers also state that using the multi-method in PCK assessment
helps assess the complex structure of transforming a teacher’s knowledge into the teaching process. For example,
Louhgran et al. (2004) suggested that topic-specific studies are needed to understand how teachers transform their
knowledge to teach while teaching a specific science topic.
Given the complex nature of teachers’ PCK, researchers have suggested that there is always a need to conduct
new research studies to understand better the subject-specific nature of science teachers’ PCKs (Ekiz-Kiran et al.,
2021; Tufail, 2021). With this aspect, most studies have focused on the PCKs of preservice and in-service science
teachers. However, the number of studies conducted with in-service science teachers is very small compared to
those undertaken with n preservice science teachers (Frågåt et al., 2021). Although many studies have examined
science teachers’ PCKs (for example, Kutluca, 2021; Shin & Song, 2021), very few studies have studied the differ-
ences between science teachers’ PCKs and their students’ achievements and attitudes. Also, very little research has
examined how the teaching experience of science teachers influenced the PCKs of novice and experienced teachers.
For example, Lin (2017) focused on the difference between experienced and preservice elementary science
teachers’ content knowledge and PCKs. Results showed that the experienced had more content knowledge with
higher confidence than preservice teachers. However, there were no statistical differences between the experienced
and preservice teachers in predicting the students’ preconceptions about the topic. In another study, Goes, Fernan-
dez & Eilks (2020) examined the PCKs of prospective and in-service teachers. Their results revealed that teaching
experience caused differences in the PCKs of teachers. According to their findings, experienced teachers had a
more advanced repertoire of instructional strategies. In their study, preservice teachers focused on traditional and
content-focused approaches, while experienced teachers considered the application of the content. In a recent
study, Gao et al. (2021) examined the interactions among the PCK components of middle school science teachers.
Their research revealed that instructional strategies and science content knowledge were most frequently con-
nected with other PCK components. Abukari et al. (2022) assessed science educators’ PCKs and the impact of their
PCKs on their students’ PCK development. Results revealed that educators’ PCKs had a very strong positive influence
on their students’ PCK development. Fauth et al. (2019) explored the relationships between teacher competence,
instructional quality, and student outcomes in elementary science classrooms in a new study. They found that the
PCKs of science teachers were not related to the achievement of elementary students.
Hanuscin et al. (2018) studied elementary teachers’ PCKs in a specific science topic and aimed to explore the
differences in teachers’ PCKs related to teaching experience. Their findings revealed that teachers had difficulties
with regard to the standards necessary for teaching the topic and did not develop activities to engage their stu-
dents in developing models regarding the subject. Their findings also revealed that teachers lacked assessment
strategies specific to the topic. They also found that the PCKs of elementary science teachers are not directly related
to the teaching experience and concluded that teachers’ PCKs are associated with the teaching experience with
more grade-level experience. These findings undoubtedly provide valuable information on the differences in the
PCK of novice and experienced science teachers in teaching a specific science topic. However, much less research
focused on the effects of teaching experience on the PCKs of science teachers and students’ achievement and
attitudes in the literature.
Another line of research in science teaching has aimed to increase students’ achievement and develop a
positive attitude toward science. To this parallel, scholars have indicated that students’ positive attitudes toward
science positively affect their science-related achievements (Bennett et al., 2001; Freedman, 1997; Martinez, 2002;
Weinburgh, 1995). Researchers well accept that many abstract concepts in science make it difficult for students
to learn. Because the mixtures consist of matter’s microscopic structure, properties of matter, aqueous solutions,
and solubility, students perceive it as complex (Salame & Nikolic, 2021).
Research Problem
Although some research has been conducted to examine the PCKs of science teachers, only a few studies
sought to explore how the teaching experience influenced teachers’ PKCs and students’ achievement and attitudes
toward science. Hence, this study aimed to examine the PCKs of science teachers affect students’ achievements and
attitudes. Thus, this study seeks to fill the gap in the field by revealing the relationship between science teachers’
subject-specific PCK levels and their students’ achievements and attitudes. The results obtained from this study
will contribute to the literature by adding new knowledge about science teachers’ PCKs and the effects of their
PCKs on students’ attitudes and achievements.
695
THE EFFECTS OF SCIENCE TEACHERS’ PEDAGOGICAL CONTENT KNOWLEDGE ON
(pp. 694-705) ISSN 2538–7138 /Online/
The main research question guides this study:

•• What are the effects of science teachers’ PCKs on students’ science achievements about mixtures and
attitudes toward science?
General Background
A mixed-methods approach was adopted using closed and open-ended questionnaire items to collect quan-
titative and qualitative data. A quantitative method was chosen to assess students’ achievement on mixtures and
reveal the relationships between students’ attitudes towards science and their teachers’ PCKs. Qualitative research
methods were also used to collect demographic information and observe teachers’ pre-lesson preparations and
classroom implementations. The data for this research was collected in the spring semester of the 2019 teaching year.
Participants
The study participants were seventh-grade students and their science teachers from two different middle
schools in Ankara, Republic of Turkey. Three classes from two schools were chosen randomly. A total of 46 students
(22 girls and 24 boys) were involved in the study. Students in each classroom were 21, 15, and 10, respectively. The
teachers of these classrooms agreed to participate in this study voluntarily. Teachers also agreed with the research-
ers’ classroom observations, video recordings, and interviews. All three teachers were male. Teacher1 had eight
years of teaching experience and taught the mixtures in previous teaching years. Teacher2 had only one year of
teaching experience and was teaching the mixtures for the first time. Teacher3 had 13 years of teaching experience
and taught the mixtures many times before this study.
This study aimed to obtain a rich data source using different measurement tools. Details regarding data col-
lection tools are as follows.
Achievement Test: The researcher developed an achievement test to determine students’ knowledge about
mixtures. The difficulty level of the questions was determined according to the explanations in the curriculum
and Bloom’s revised taxonomy (Krathwohl, 2002). A question pool was created to select questions about mixtures.
Thirty-five questions were chosen from this pool, and three science teachers revised the questions. These teachers
had a lot of experience teaching the topic “mixtures” at the middle school level. Their feedback helped increase the
validity of the test for using it in this study. Later, an achievement test with 30 questions was created by research-
ers. To validate this test, researchers asked for feedback from three science educators who enrolled as university
professors in science education on the test. After receiving the feedback from educators, the final version of the
test was completed by researchers. Students approximately answered the test in thirty minutes. The developed
achievement test was administered to 77 eighth-grade students for the pilot study.
Student perceptions questionnaire about teacher knowledge: A questionnaire with 18 items, originally developed
by Tuan et al. (2000), was translated into Turkish by Afacan et al. (2013). This questionnaire was used to determine
students’ perceptions of teacher knowledge. The reliability coefficient of the questionnaire was found to be 0.87
by the researchers, while the Cronbach alpha reliability coefficient was calculated as 0.94.
Attitude and perception questionnaire: A Likert-type questionnaire developed by Kaya (2002) consisted of 19
items and revealed the relationships between students’ attitudes and perceptions of science. The alpha reliability
coefficient of the questionnaire was calculated as 0.90 based on the data in this study. The first 12 items of the
questionnaire were aimed at determining students’ attitudes toward the science, and the other seven items were
used to measure students’ perceptions of the science.
696
ISSN 2538–7138 /Online/ (pp. 694-705)
Interview form: The interview form, including 15 questions, was developed by using the studies of Avraamidou
(2003) and Brunsberg (2013). Teachers were asked to answer the questions in writing. The answers that teachers did
write were used to qualitatively and quantitatively analyze teachers’ PCKs. A part of written responses was evalu-
ated as quantitative data. ANOVA analyzed these data, and the differences among teachers’ PCKs were examined.
Video observation form: Observations are one of the best data collection methods for qualitative studies (Pat-
ton, 2002). In this study, teachers’ lessons were observed and videotaped by one of the researchers. During the
observations, notes were taken to obtain detailed information about the implementation of teachers’ teaching. To
assess the teachers’ classroom practices, three different scholars evaluated the video recordings using an observa-
tion form with 36-items developed by Wischow (2010).
Data Analysis
Analysis of the quantitative data was conducted using SPSS statistical software. Parametric and non-parametric
techniques were used depending on the characteristics of the data collected. For example, any changes in the at-
titudes toward science according to gender were analyzed using independent groups t-test. At the same time, the
differences in students’ perceptions of teachers’ knowledge were examined with ANOVA. The differences between
the pre-test and post-test mean scores were analyzed with the Wilcoxon test. Regarding students’ perceptions
about teachers’ knowledge, POST HOC analyses were run after the ANOVA. For the POST HOC analysis, Dunnett’s
C analysis was used.
Analysis of qualitative data: The PCKs of science teachers about mixtures were examined using a content
analysis approach to reveal dark themes and distinct themes, as suggested by Corbin and Strauss (2008). The data
was sought to be defined with the content analysis, and hidden situations were revealed. The obtained data were
checked with other sources such as video and audio recordings, and corrections were made for missing or incorrect
parts. After the data completion and correction processes were completed, the data were analyzed using descriptive
and content analyses. Content analysis and descriptive and inferential statistics were used during the data analyses.
Research Results
Achievement
The pre-test and post-test scores were analyzed using the t-test and Wilcoxon test. The average scores in
pre-post-test results are given in Table 1.
Table 1
Pre-Test and Post-Test Scores
Class Pre-test scores Post-test scores
7L (T1) 17.00 20.70

7E (T2) 13.29 18.52
7K (T3) 10.07 18.20
Table 1 shows that each class’s pre-test and post-test scores differed. According to the results, the mean scores
of each category in the post-tests were higher than pre-tests. To compare the differences between pre and post-
tests, a t-test was used. The results of this analysis are given in Table 2.
697
(pp. 694-705) ISSN 2538–7138 /Online/
Table 2
Comparison of Pre-Test and Post-Test Scores of Classes with Paired t-Test
Means Difference sd t df p
7E -5.24 4.55 -5.28 20 .0001

7K -8.13 4.93 -6.40 14 .0001
7L -3.70 2.87 -4.08 9 .003
Table 2 shows a significant difference in favor of the post-tests between all classes’ pre-test and post-test
mean scores.
Since the number of students in each class is less than 30, a normal distribution might not be expected for
use in a parametric test. For this detail, a Kolmogorov-Smirnov analysis was used to test the normal distribution
for pre-test and post-test. According to the results of the Kolmogorov-Smirnov test, a normal distribution was not
provided for pre-test and post-test. Hence, the differences between the two dependent groups were analyzed
using the Wilcoxon test, one of the non-parametric tests.
According to the results of both parametric and non-parametric tests, it was found that there were significant
differences in favor of the post-test in all three classes (p values for t-test: .000, .000, and .003; p values for Wilcoxon
test: .000, 001 and .008). When the results of the tests were combined, it was found that students’ achievement did
increase in all three classes.
Table 3
ANOVA Results of Post-Test Scores
Sum of Squares df Mean Square F p
Intergroups
42.718 2 21.359 1.823 .174
Within Groups 503.738 43 11.715
Total 546.457 45
From table 3, it appears that there was no statistically significant difference between the post-test scores of
the classes (p =.174). The significance value for the post-test scores according to the results of the F test was calcu-
lated as p =.174> .05. Thus, our results showed no significant difference between the post-test scores of each class.
Students’ Perceptions of Teachers’ Knowledge
The teacher perception questionnaire consisted of 28 statements in a 5-point Likert type. The highest score
that will be obtained from the questionnaire is 140. ANOVA analysis was used to analyze the questionnaire data
(see Table 4). Our results showed a significant difference in students’ views about their teachers’ knowledge.
Table 4
Descriptive Results of the Student Perception Questionnaire about Teachers’ Knowledge
N Interval Average SD Change Distortion Flatness
Class L 11 39.00 112.81 13.75 189.16 -.23 -1.34
Class K 20 42.00 101.00 11.58 134.00 .23 -.68
Class E 21 82.00 88.86 24.32 591.23 -.22 -.90
698
ISSN 2538–7138 /Online/ (pp. 694-705)
From Table 4, it is noted that the average score of the L class (112.81) is higher than the other two classes (K =
101.00, E = 88.86). Because the variances of the three classes are not equal (p = .001), different POST HOC analyzes
were run after the ANOVA. For this analysis, Dunnett’s C analysis was preferred in the study.
Table 5
Results of Student Perception Questionnaire
Sum of squares df Means square F p
Inter groups 4332.31 2 2166.16 6.53 .003
Within Groups 16262.21 49 331.88

Total 20594.52 51
In table 5, it appears that the significance value for students’ perceptions according to the F test was significant
(p = .003 <0.05). According to the results, students’ views of teachers’ knowledge differed significantly according
to the classes.
Attitudes toward Science
The attitude questionnaire consisted of 19 statements in a 5-point Likert type. The highest score that will
be obtained from the questionnaire is 95. According to the descriptive statistics regarding attitudes, the average
scores of L, K, and E classes are 44.82, 39.35, and 42.86, respectively. Similarly, the average scores of the classes
regarding the perception questionnaire are 26, 25.75, and 26.67, respectively. There was no significant relation-
ship between attitude and perception scores. One of the classes (L class) had the highest scores in attitudes.
Another class (K class) had a lower average score than other classes in attitude and perception.
In addition, the Skewness values of the scores for both attitude and perception vary between -2 and +2. In
terms of Skewness values, only the accepted limits of the K class (between -2 and +2) slightly exceed (2.18 and
2.86). Therefore, the distribution of attitude and perception scores can be considered normal.
According to the equality of variances test for science attitude and perception, p values are more signifi-
cant than .05 for both attitude (.284) and perception (.127). In this case, the variances of the groups can be
considered equal. As a result, it is possible to conduct an ANOVA analysis to test whether there is a difference
between students’ attitudes and perceptions of science. ANOVA results regarding attitude and perception are
given in Tables 6 and 7.
Table 6
Results of Attitude Questionnaire
Students’ Attitudes
Intergroup 242.94 2 121.47 1.49 .237

Within Groups 4008.76 49 81.81
Total 4251.69 51
699
(pp. 694-705) ISSN 2538–7138 /Online/
Table 7
Results of Perception Questionnaire
Students’ Perceptions
Intergroups 9.03 2 4.51 .204 .817

Within Groups 1086.42 49 22.17
Total 1095.44 51
As given in Table 6 and Table 7, there is no significant difference among all classes for both attitude and per-
ception towards the science (Attitude: .237> .05; Perception: .817>.05). According to a 0.05 significance level and
two between groups and 49 degrees of freedom within groups, the critical value of F = 3.19 in the F-distribution
chart is compared with the F statistics values 1.49 and .204 in the ANOVA table. Since 1.49 <3.19 and .204 <3.19,
it was found that there is no significant difference between the means of the three groups.
Interview Data
The interview form consisted of 15 statements in a 5-point Likert type. The highest score that will be ob-
tained from the questionnaire is 75. The teachers in this study were interviewed before teaching about mixtures.
During these interviews, teachers were asked questions and asked to fill out an interview form consisting of
15 questions. In this form, teachers provided written answers to the questions about PCK. Since the interview
form consisted of 15 items, the highest score obtained from the form was 75. Teachers graded the form as 68,
69, and 59, respectively. According to these scores, T1 and T3 received almost the same score (68 and 69), while
teacher T2 received a relatively low score (59).
Table 8
Analysis of Teachers’ Scores Based on Interview Results with ANOVA
Intergroup 4.93 2 2.47 6.07 .005

Within Groups 17.07 42 .41
Total 22.00 44
Some examples from the interviews with teachers are below.
For example, the fourth item in the interview form is: “What are the limitations you encounter when teach-
ing this concept?”. Teachers answered this question as follows:
Teacher-1: “For example, I will give the example of water and sugar on dissolution, but we cannot give
examples such as alcohol and sugar, oil and sugar. We go through certain measures. This situation limits
me. We have to explain the concepts superficially. This kind of explanation limits me, for example. I
have to explain the lesson in their words, which is a limitation.”
Teacher-2: “When describing the heterogeneous mixture, one should not go into details—for example,
emulsion and suspension. Also, dilute, and concentration-related formulas should not be used.”
Teacher-3: “It is possible to confuse the concepts homogeneous-heterogeneous and concentrated-

dilute with each other.”
700
ISSN 2538–7138 /Online/ (pp. 694-705)
The quotations above show that the first teacher emphasized the limitations he encountered while teaching
mixtures. In contrast, the second teacher emphasized the concepts that should not explain in the courses. The
third teacher pointed out the same question by giving examples of difficult concepts for the students.
For another question in the interview form (How will you evaluate individual differences? Can you give us
an example?), teachers gave the following responses. These are:
Teacher-1: “Of course, not every child is the same, but I will have attempts to make every student know
and learn the basic information about the subject of mixtures. Some will understand the subject once.
Some will realize it with videos. Others will recognize the homework they prepare at home. I will work
for each student. If needed, I will give additional lessons.”
Teacher-2: “Giving performance homework depending on the students’ abilities. To have additional
studies done.”
Teacher-3: “I intend to assign students with weak interests, especially while experimenting. In addition,
I am thinking of teaching the lesson to students who are below the general level of the class in a way
to support their answers with positive reinforcements by providing them with examples while work-
ing on the subject, keeping their interest alive, and taking into account different types of intelligence.”
These quotations indicate that teacher T1 pointed out the conditions under which students with different
success levels can understand the lesson and the sacrifice he will make for this. In contrast, teacher T2 plans to
evaluate individual differences with homework. The other teacher, T3, intended to overcome individual differ-
ences with the additional training practices he would make.
To understand the differences among the PCKs of three teachers for the interview form, we used the Levene
test for further analysis. The variances are equal in the homogeneity test of the variances according to the Levene
test results of the scores the teachers got from the interview (.781> .05). According to the ANOVA analysis, there
is a difference between the groups (.005 <.05). According to a 0.05 significance level and two between groups
and 42 degrees of freedom within groups, the F = 3.22 critical value in the F-distribution chart is compared with
the F statistic value of 6.07 in the ANOVA table 8. Since 6.07> 3.22, it was decided that there is a significant dif-
ference between the means of the three groups with 95% confidence levels.
According to the results of Bonferroni analysis after ANOVA, no difference was found between the PCK of
the first and third teachers. The PCK of both T1 and T3 was higher than T2. It is understood from here that the
PCK of T2, with only one year of teaching experience, is low.
Video Observations
If the teacher implemented a teacher’s behavior in the observation form, it was scored with 0, 1, 2, 3, and 4.
Since the form consisted of 36 items, the highest score was 144, and the total score obtained from three experts
was 432. Accordingly, the teachers received T1; 295, T2; 211, and T3; 259, respectively, from the experts. ‘When it
was conducted ANOVA and Bonferroni analyses, it was found that there is a difference between the PCKs of T1
and T2 in favor of T1 (mean difference = 2.33; p = .0001). These results indicate no significant difference between
T1 and T3 and the PCKs of T2 and T3.
Discussion
This study aimed to examine the PCKs of science teachers who have different teaching experiences and
investigate the effects of teachers’ PCKs on students’ achievement and attitudes toward science. This study
examined the PCKs of three science teachers with different teaching experiences, their students’ achievements
in mixtures, and their attitudes toward science.
The results showed that teachers with different teaching experiences on a particular science topic did not
significantly affect students’ achievement and attitudes. The result of this study is consistent with other stud-
ies showing that teachers’ PCKs do not have a positive influence on students’ achievement. (Fauth et al., 2019;
701
(pp. 694-705) ISSN 2538–7138 /Online/
Gess-Newsome, 2017). On the other hand, our results related to student achievement are not consistent with
some of the findings of Gess-Newsome (2017), who found that academic content knowledge had a statisti-
cally significant impact on student achievement as a predictor of this variable. However, the other findings
of Gess-Newsome (2017) found no statistical significance between teachers’ general pedagogical knowledge
and teacher practice. The lack of a relationship between teachers’ PCK and student achievement may depend
on many factors. For example, the achievement test we used in this study may not have been appropriate to
measure student achievement on this topic. It is possible that the researchers in this study did not control some
factors affecting student achievement.
Further studies should focus on the aspects that examine the relationship between science teachers’ PCKs
and student achievement. In addition, one of the teachers in this study stated that he mainly used smartboards
when teaching the mixtures. He also believed that the smartboards would strongly impact student achievement.
However, the use of the smartboard did not affect student achievement in this study.
We found no relationship between students’ attitudes toward science and teachers’ PCKs. The results show
no statistically significant difference between science teachers’ PCKs and students’ perceptions of science. This
finding can be attributed to many factors. The research findings suggest that science teachers’ PCKs are not the
main factor affecting students’ attitudes. There may be other factors that influence student attitudes and achieve-
ment. Some researchers (Nuangchalerm, 2017) pointed out that one of the most important factors that make
students learn science lessons could be teachers’ behavior. For example, one of the teachers who participated in
our study stated that although he did not spare any sacrifice for students’ learning during class, he tried to be a
good example. Despite this teacher, her efforts to teach science were not enough to improve student attitudes.
The results related to teaching experience showed that more teaching experience in science class did not
positively affect students’ achievement and attitude. Although researchers (Ekiz-Kiran et al., 2021; Kutluca, 2021;
Mikeska et al., 2021; Özel, 2012) have pointed out that experienced teachers may have a sophisticated level in
some components of PCK, such as students’ difficulties and teaching strategies, the results of this study showed
that teaching experience did not affect students’ achievement and attitude. On the other hand, the other data
sources in this study, including the interview forms and video observations, showed some differences among
the PCKs of teachers with teaching experience. For example, a teacher with only one year of teaching experience
received low scores in analyzing the interview forms and video observations. Another interesting finding is that
a teacher with eight years of teaching experience scored higher on the interview form and video observations
analysis than a teacher with thirteen years of experience. These results confirm that our findings are consistent
with Hanuscin, Cisterna, and Lipsitz (2018), who indicated that expertise in a particular science topic does not
transfer to teaching a different science topic at a different grade level. They also emphasized that the nature
of the relationship between teaching experience and PCK needs to be reconsidered by researchers. In general,
researchers assume that PCK develops through teaching experience; however, the results of this study do not
support this assumption.
Because there is little research examining the PCKs of science teachers with different teaching experiences,
the results of this study provide new insights into the literature by looking at the PCKs of science teachers with
varying teaching experiences and the attitudes and performance of their students. From this perspective, the
results of this study make several contributions to the current literature. The present study provides a compre-
hensive assessment of teachers’ PCKs based on their teaching experiences and an understanding of how sci-
ence teachers’ PCKs differ while teaching the same subject. From these perspectives, the results contribute to
our understanding of science teachers’ PCKs and provide a basis for understanding how teachers’ PCKs vary by
teaching experience and examining how student achievement and attitudes influence teachers’ PCKs.
Looking at the nature of PCKs, there is still a need to explore science teachers’ PCKs further. By their very
nature, teachers’ PCKs may depend on various factors. Like teachers’ PCKs, many factors can also influence stu-
dents’ performance and attitudes. We recommend further research on teachers’ PCKs and students’ achievements
and attitudes. Further studies still need to examine science teachers’ PCKs and their students’ achievement and
attitudes.
It should be noted that teachers’ PCKs are an ongoing process that will continue to be the focus of future
research in teacher education. Moreover, PCKs are an evolving knowledge structure, and teachers’ PCKs may
702
ISSN 2538–7138 /Online/ (pp. 694-705)
decrease or increase during the teaching process. Therefore, we suggest that further research should closely
examine the relationships between teachers’ PCKs and their students’ achievement and attitudes. Future re-
search should also be conducted to identify other relationships between teachers’ PCKs and other factors in
the instructional context.
References
Abukari, M. A., Bayuo, J., Alagbela, A. A., & Bornaa, C. S. (2022). Pedagogical content knowledge of science tutors and its influence
on their trainees. Contemporary Mathematics and Science Education, 3(1), Article ep22008. https://doi.org/10.30935/
conmaths/11830
Afacan, O., Karakus, M., & Usak, M. (2013). Turkish adaptation of the scale of “student perceptions of teachers’ knowledge (SPOTK)”
and examining the aspect of some variables. Eğitim Bilimleri Araştırmaları Dergisi, 3(1), 185-200.
Avraamidou, L. (2003). Perspectives on learning, learning to teach and teaching elementary science [Doctoral dissertation,
The Pennsylvania State University]. The Pennsylvania State University Electronic Theses and Dissertations for
Graduate School. https://etda.libraries.psu.edu/catalog?f%5Bdegree_name_ssi%5D%5B%5D=PHD&f%5Bkeyword_
ssim%5D%5B%5D=learning&f%5Byear_isi%5D%5B%5D=2003&sort=last_name_ssi+asc%2C+title_ssi+asc
Bennett. J., Rolhiick. M., Green. G., & White, M. (2001). The development and use of an instrument to assess students’ attitude to
the study of chemistry. International Journal of Science Education, 23(8), 833-845.
Brunsberg, S. L. (2013). A study about the level of a teacher’s content knowledge, pedagogical content knowledge, instructional
practices, and demographics and their effects on students’ literacy achievement, North Dakota State University, ProQuest,
UMI Dissertations Publishing.
Corbin, J., & Strauss, A. (2008). Basics of qualitative research: Techniques and procedures for developing grounded theory (3rd Edition).
Sage.
Duran, M., Usak, M., Hsieh, M. Y., & Uygun, H. (2021). A new perspective on pedagogical content knowledge: intellectual and emotional
characteristics of science teachers. Revista de Cercetare si Interventie Sociala, 72, 9-32, https://doi.org/10.33788/rcis.72.1
Ekiz-Kiran, B., Boz, Y., & Oztay, E. S. (2021). Development of preservice teachers’ pedagogical content knowledge through a PCK-
based school experience course. Chemistry Education Research and Practice, 22(2), 415-430.
Fauth, B., Decristan, J., Decker, A. T., Büttner, G., Hardy, I., Klieme, E., & Kunter, M. (2019). The effects of teacher competence
on student outcomes in elementary science education: The mediating role of teaching quality. Teaching and Teacher
Education, 86, Article 102882. https://doi.org/10.1016/j.tate.2019.102882
Frågåt, T., Henriksen, E. K., & Tellefsen, C. W. (2021). Preservice science teachers’ and in-service physics teachers’ views on the
knowledge and skills of a good teacher. Nordic Studies in Science Education, 17(3), 277-292.
Fre e d m a n . M . P. ( 1 9 9 7 ) . R e l a t i o n s h i p a m o n g l a b o r a t o r y i n s t r u c t i o n , a t t i t u d e t o w a rd s c i e n c e, a n d
a c h i e v e m e n t i n s c i e n c e k n o w l e d g e . J o u r n a l o f R e s e a rc h i n S c i e n ce Te a c h i n g , 3 4 ( 4 ) , 3 4 3 - 3 5 7 .
https://doi.org/10.1002/(SICI)1098-2736(199704)34:4%3C343::AID-TEA5%3E3.0.CO;2-R
Gao, S., Damico, N., & Gelfuso, A. (2021). Mapping and reflecting on integration of the components of pedagogical content
knowledge (PCK) for teaching natural selection: A case study of an experienced middle-school science teacher. Teaching
and Teacher Education, 107, Article 103473. https://doi.org/10.1016/j.tate.2021.103473
García-Pérez, D., Fraile, J., & Panadero, E. (2021). Learning strategies and self-regulation in context: How higher education students
approach different courses, assessments, and challenges. European Journal of Psychology of Education, 36(2), 533-550.
Geer, R., & Barnes, A. (2007). Cognitive concomitants of interactive board use and their relevance to developing effective research
methodologies. International Education Journal, 8(2), 92-102.
Gess-Newsome, J., & Lederman, N. G. (1999). Examining pedagogical content knowledge. Kluwer Academic Publishers.
Gess-Newsome, J., Taylor, J. A., Carlson, J., Gardner, A. L., Wilson, C. D., & Stuhlsatz, M. A. M. (2017). Teacher pedagogical
content knowledge, practice, and student achievement. International Journal of Science Education, 41(7), 944-963.
https://doi.org/10.1080/09500693.2016.1265158
Goes, L. F., Fernandez, C., & Eilks, I. (2020). The development of pedagogical content knowledge about teaching redox reactions
in German chemistry teacher education. Education Sciences, 10(7), 170.
Goldberg, M. (2001). An interview with Linda Darling-Hammond: Balanced optimism. Phi Delta Kappan, 89(9), 687-690.
Guskey, T. R. (2003). What makes professional development effective? Phi Delta Kappan, 84(10), 748-750.
Hanuscin, D. L., Cisterna, D., & Lipsitz, K. (2018). Elementary teachers’ pedagogical content knowledge for teaching structure and
properties of matter. Journal of Science Teacher Education, 29(8), 665-692.
Hashweh, M. Z. (2005). Teacher pedagogical constructions: A reconfiguration of pedagogical content knowledge. Teachers and
Teaching, 11(3), 273-292.
Hashweh, M. Z. (1987). Effects of subject matter knowledge in the teaching of biology and physics. Teaching and Teacher
Education, 3(2), 109-120.
703
(pp. 694-705) ISSN 2538–7138 /Online/
Kaya, O. N. (2002). İlköğretim 7. sınıf öğrencilerinin atom ve atomik yapı konusundaki başarılarına, öğrendikleri bilgilerin kalıcılığına,
tutum ve algılamalarına çoklu zekâ kuramının etkisi [The effect of the multiple intelligence theory on grade-7 students’
achievement, retention on their knowledge, attitude and perceptions in the topic of atom and atomic structure] [Master’s
thesis, Gazi University]. Gazi Universitesi. Council of Higher Education Thesis Center.
Käpyla, M., Heikkinen, J. P., & Asunta, T., (2009). Influence of content knowledge on pedagogical content knowledge: The case of
teaching photosynthesis and plant growth. International Journal of Science Education, 31(10), 1395–1415.
Kim, L. E., Jörg, V., & Klassen, R. M. (2019). A meta-analysis of the effects of teacher personality on teacher effectiveness and
burnout. Educational Psychology Review, 31(1), 163-195.
Kinskey, M., & Zeidler, D. (2021). Elementary preservice teachers’ challenges in designing and implementing socioscientific
issues-based lessons. Journal of Science Teacher Education, 32(3), 350-372.
Koballa, T. R. (1995). Children’s attitudes toward learning science. In S. Glynn & R. Duit (eds.), Learning science in the schools:
Research Reforming Practice, Mahwah, NJ:Earlbaum.
Krathwohl, D. R. (2002). A revision of Bloom’s taxonomy: An overview. Theory into Practice, 41(4), 212-218.
Kutluca, A. Y. (2021). An investigation of elementary teachers’ pedagogical content knowledge for socioscientific argumentation:
The effect of a learning and teaching experience. Science Education, 105(4), 743-775.
Kutluca, A. Y., & Mercan, N. (2022). Exploring the effects of preschool teachers’ epistemological beliefs on content-based
pedagogical conceptualizations and PCK integrations towards science teaching. European Journal of Science and Mathematics
Education, 10(2), 170-192. https://doi.org/10.30935/scimath/11661
Lin, J.-W. (2017). A comparison of experienced and preservice elementary school teachers’ content knowledge and pedagogical
content knowledge about electric circuits. Eurasia Journal of Mathematics, Science and Technology Education, 13(3), 835-
856. https://doi.org/10.12973/eurasia.2017.00646a
Loughran, J., Milroy, P., Berry, A., Gunstone, R., & Mulhall, P. (2001). Documenting science teachers’ pedagogical content knowledge
through PaP-eRs. Research in Science Education, 31(2), 289-307.
Martinez, A. (2002). Student achievement in science: A longitudinal look at individual and school differences. [Doctoral
dissertation]. Harvard University.
Mikeska, J. N., Brockway, D., Ciofalo, J., Jin, H., & Ritter, S. (2021). Examining variability in elementary science teachers’ pedagogical
content knowledge about phase change: Implications for teacher development and assessment. Journal of Science Teacher
Education, 32(4), 400-424.
Njiku, J., Mutarutinya, V., & Maniraho, J. F. (2021). Building mathematics teachers’ TPACK through collaborative lesson design
activities. Contemporary Educational Technology, 13(2).
Nuangchalerm, P. (2017). Relationship between preferred and actual opinions about inquiry-based instruction classroom. European
Journal of Science and Mathematics Education, 5(1), 67-73. https://doi.org/10.30935/scimath/9498
Özden, M., Usak M., Ulker, R., & Šorgo, A. (2013). Effects of lesson preparation methods on prospective primary teachers’ pedagogical
content knowledge. Journal of Environmental Protection and Ecology, 14(3A), 1432-1442.
Özel, M. (2012). Farkli öğretim deneyimine sahip fen ve teknoloji öğretmenlerinin kimyasal tepkimeler konusundaki pedagojik alan
bilgilerinin incelenmesi (How does the pedagogical content knowledge of elementary teachers with different years of experience
vary as they teach chemical reactions) [Doctoral dissertation, Gazi University]. Council of Higher Education Thesis Center.
Patton, M. Q. (2014). Qualitative research & evaluation methods: Integrating theory and practice. Sage Publication.
Salame, I. I., & Nikolic, D. (2021). Examining some of the challenges students face in learning about solubility and
the dissolution process. Interdisciplinary Journal of Environmental and Science Education, 17(3), Article e2237.
https://doi.org/10.21601/ijese/9333
Shin, C., & Song, J. (2021). A study on science teaching orientation and PCK components as they appeared in science lessons by
an experienced elementary teacher: Focusing on’Motion of objects’ and’Light and lens’. Journal of The Korean Association
for Science Education, 41(2), 155-169.
Shulman, L. S. (1986). Paradigms and research programs in the study of teaching: A contemporary perspective. In M. C. Wittrock
(Ed), Handbook of research on teaching. (3rd ed., pp. 3-36). Macmillan.
Shulman, L. S. (1987). Knowledge and teaching: Foundations of the new reform. Harvard Educational Review, 57(1), 1-22.
Simpson, R. D., Koballa, T. R. JR., Oliver, J. S., & Crawley, F. E. (1994). Research on the affective dimension of science learning. D.
White (Ed). Handbook of Research on Science Teaching and Learning. New York: Mac Millan Publishing Company; 211-235.
Tuan, H. L., Chang, H. P., Wang, K. H., & Treagust, D. F. (2000). The development of an instrument for assessing students’ perceptions
of teachers’ knowledge. International Journal of Science Education, 22(4), 385-398.
Tufail, I. (2021). Secondary school science teachers’ pedagogical content knowledge (PCK) in their classroom practice (Doctoral
dissertation, The University of Waikato).
Usak, M., Özden, M., & Eilks, I. (2011). A case study of beginning science teachers” subject matter (SMK) and pedagogical content
knowledge (PCK) of teaching chemical reaction in Turkey. European Journal of Teacher Education, 34(4), 407-429.
Veal, W. R., Tippins, D. J., & Bell, J. (1999). The evolution of pedagogical content knowledge in prospective secondary physics
teachers (No. ED443719). Indiana, University.
Vekli, G. S. (2021). What factors affect middle school students’ perceptions of inquiry learning towards science?. Pedagogical
Research, 6(4), Article em0108. https://doi.org/10.29333/pr/11301
Wan, Z. H. (2021). Exploring the effects of intrinsic motive, utilitarian motive, and self-efficacy on students’ science learning in
the classroom using the expectancy-value theory. Research in Science Education, (3), 647-659.
Weinburgh, M. (1995). Gender differences in student attitudes toward science: A meta-analysis of the literature from 1970 to
704
ISSN 2538–7138 /Online/ (pp. 694-705)
1991. Journal of Research in Science Teaching, 32(4), 387-398. https://doi.org/10.1002/tea.3660320407

Wischow, E. D. (2010). Interactions between teachers’ existing pedagogical content knowledge and novel subject matter
knowledge [Doctoral dissertation, Purdue University]. Purdue e-Pubs. https://docs.lib.purdue.edu/dissertations/AAI3444880/
Xin, J. F., & Sutman, X. F. (2011). Using the smart board in teaching social stories to students with autism. Teaching Exceptional
Children, 43(4), 18-24.
Received: February 10, 2022 Revised: June 02, 2022 Accepted: July 10, 2022
Cite as: Usak, M., Uygun, H., & Duran, M. (2022). The effects of science teachers’ pedagogical content knowledge
on students’ attitudes toward science and their achievement. Journal of Baltic Science Education, 21(4), 694-705.
Muhammet Usak PhD Science Education, Kazan (Volga Region) Federal University, Kazan,
(Corresponding author) Russia.
E-mail: musaktr@gmail.com
ORCID: http://orcid.org/0000-0002-6537-9993
Harun Uygun PhD Science Education, Cultura Building, Wassenaarseweg 20 2596CH The
Hague, The Netherlands.
E-mail: harunuygun@yahoo.com
Muharrem Duran PhD Science Education, Independent Researcher, Yenimahalle, Ankara,
Turkey.
E-mail: muharremduran@gmail.com
705
THE EFFECT OF STUDENTS’

ATTITUDE TOWARDS SCIENCE,
TECHNOLOGY, ENGINEERING,
AND MATHEMATICS ON 21ST
CENTURY LEARNING SKILLS:
Abstract. It is crucial to educate students A STRUCTURAL EQUATION
on the basis of science, technology, MODEL
engineering, and mathematics (STEM)
education to develop 21st century learning
skills. However, there is little research on the
contribution of the attitude towards STEM
specific discipline to 21st century learning Shao-Rui Xu,
skills. This study aimed to examine the
Shao-Na Zhou
extent to which students’ attitude towards
science, technology, engineering, and
mathematics (STEM) contributes to 21st
century learning skills. Data were collected
from 779 students from three primary
schools and two secondary schools with Introduction
the S-STEM questionnaire. Quantitative
data were analyzed with structural The sustainability of economic growth and the achievement of prosper-
equation modelling. The results revealed ity for countries are facing more and more challenges in the 21st century. One
that (a) students’ science attitude, and of the challenges is the shortage of the highly-competent human resource
engineering and technology attitude had with 21st century skills, such as creativity, critical thinking, collaboration and
positive effects on 21st century learning communication (the 4-Cs), to prepare individuals to survive and compete in
skills, and (b) students’ science attitude and the digital era (International Technology and Engineering Educators Associa-
mathematics attitude positively affected tion [ITEEA], 2020; Australian Education Council, 2015; Bybee, 2010). In the
21st century learning skills indirectly fields of science, engineering, mathematics, and technology (STEM), there is
through the mediating role of technology an increasing demand for creative individuals who can think, question, and
and engineering attitude, and (c) the school produce. In the workforce market, it was reported that the demand for STEM
year partially played a moderating role. occupations would increase by 8% between 2013 and 2025 (Caprile et al.,
The finding highlights the importance of 2015). However, the demand for the labor force in STEM-related fields is not
valuing STEM attitude across disciplines. being adequately met (Moakler & Kim, 2014). In contrast to the prospective
In particular, students’ engineering and demand in the labor market, there is a shortage of engineers in specific areas,
technology attitude is more conducive to such as software, electromagnetics, maintainability, structures, and manufac-
improving students’ 21st century learning turing engineering (Xue & Larson, 2015). Although there is a likelihood that
skills. It also suggests the need to pay the statistics could be altered over time, the current statistics clearly dem-
attention to the changing effect of STEM onstrate that a significant shortage of engineers would last for a long period
attitude on 21st century learning skills unless more students are encouraged to engage in the engineering field.
driven by the school year. It is argued that by integrating science, technology, engineering, and
Keywords: 21st century learning skills, mathematics (STEM) education through innovative projects, 21st century
mediating role, STEM attitude, structural learning skills will keep up with the development of society (Beswick & Fraser,
equation modelling 2019; Karahan, 2019). To equip students with 21st century learning skills,
STEM education, whose goal is to develop students’ skills in logical reasoning,
Shao-Rui Xu problem solving and collaboration, is a good medium to train students to
Guangdong Mechanical & Electrical meet the need of the 21st century workforce (Moore, 2009). STEM education
Polytechnic, China can synthesize the information that students have learned and strengthen
Shao-Na Zhou
South China Normal University, China their understanding. By integrating science, technology, engineering and
technology, STEM education embodies interdisciplinary learning, which is
the foundation of 21st century curricula. The demand for the development
706
ISSN 1648–3898 /Print/ THE EFFECT OF STUDENTS’ ATTITUDE TOWARDS SCIENCE, TECHNOLOGY, ENGINEERING,
ISSN 2538–7138 /Online/ MODEL
(pp. 706-719)
of 21st century learning skills requires the establishment of new pedagogical and content-based education objec-
tives and promotes students’ positive attitude towards STEM learning (Voogt et al., 2013). Based on these facts, it
is crucial to educate students on the basis of STEM education to develop 21st century learning skills.
Literature Review
21st century learning skills
The 21st century is an era that increasingly requires students to possess the skills such as critical thinking,
problem solving, collaboration and active learning, which are referred to as 21st century learning skills (Partner-
ship for 21st century skills, 2006; 2021). Students need to prepare to solve different kinds of problems arising from
unforeseen circumstances and handle issues that have not yet arisen. 21st century learning skills are considered
as lifelong learning skills which could enable students to get familiar with their changing living conditions and
become more responsive (OECD, 2005). The fundamental skills that individuals should possess in the 21st century
are life and career skills, knowledge, media, and technology skills, and learning and innovation skills (Partnership
for 21st century learning skills, 2019). Thus, 21st century learning skills are essential for personal and social success
(Rotherham & Willingham, 2010).
In the 21st century, students should be evaluated not only by testing the ability of answering questions,
but also by their level of knowledge and ability to apply their 21st century learning skills. According to the Metiri
Group and NCREL (2003), 21st century learning skills can be useful when the younger generation faces challenges
from the industrial society, the global economy, the influx of high technology, as well as the overflow of global
information. The purpose of education is to prepare students for a future that needs constant learning and active
use of 21st century learning skills (Krskova et al., 2020; Mutiani & Faisal, 2020; Park & Suh, 2020). Therefore, it is
important to augment students’ ability of 21st century learning skills in the current educational system, especially
in STEM education.
STEM Pipeline Leakage
In recent years, leaks from current STEM pipelines have been reported in many countries (Ball et al., 2017; Do-
erschuk et al., 2016). The STEM pipeline is a common metaphor used to express the flow of students and culminate
in STEM as a career (Allen-Ramdial & Campbell, 2014). As the school year grows, students tend to flee the STEM
pipeline. A number of studies have reported that older graders’ attitude towards STEM is less positive compared
to younger graders’ attitude (Potvin & Hasni, 2014a; Unfried et al., 2014). It is noted that in high education institu-
tions, the number of students choosing STEM-related majors is not up to expectations (Shapiro & Sax, 2011). It is
stated that more students abandon the STEM-related majors that they initially choose compared to other fields
(Reinhold et al., 2018).
The leakage of STEM pipelines could occur in different K-12 school years. It is stated that compared with upper
primary students, lower primary students have a more positive attitude towards STEM (Zhou et al., 2019). Unfried
et al. (2014) concluded that attitude towards science became decreasing after a longitudinal study from primary
school to middle school. Students’ attitude towards STEM is evident at thirteen years old and boosting students’
attitude towards STEM at a later age becomes difficult (Lindahl, 2007). Another longitudinal study has revealed
a steady decline in students’ attitude towards STEM as students transition into early high school (Speering et al.,
1996). The school year which has been considered as an important influencing factor on STEM attitude (Wiebe et
al., 2018), may be the source of leaks from the STEM learning pipelines.
STEM attitudes
Due to the continuous research on students’ attitude, the understanding of students’ attitude continues to
deepen (Luo et al., 2019). From the perspective of the STEM field, the literature of the research concentrates on
students’ attitude not only towards STEM as a whole, but also towards an independent STEM discipline (Unfried et
al., 2014). It is necessary to systematically collect data on the students’ attitude across different STEM fields (Min-
ner et al., 2012). An instrument was created by Erkut and Marx (2005) for assessing the attitude towards multiple
STEM fields. With this instrument, Johnpaul et al. (2018) further measured students’ attitude towards various STEM
707
MODEL ISSN 2538–7138 /Online/
(pp. 706-719)
disciplines to understand their differences and similarities. Another instrument was developed by Faber et al. (2013)
to investigate K-12 students’ attitude towards STEM disciplines, as well as 21st century learning skills and STEM
career pathways. Wiebe et al. (2018) addressed the gap by using the S-STEM questionnaire focusing on student
attitudes towards all STEM subjects.
There is also an impressive amount of research literature on students’ attitude towards independent STEM
disciplines (Gardner, 1975; Osborn et al., 2003; Potvin & Hasni, 2014). Typically, researchers are primarily concerned
with students’ interest in mathematics or science. Regarding mathematics, the Attitudes towards Mathematics Survey
(Miller et al, 1996) was developed to assess students’ attitude towards mathematics. The Test of Science-Related
Attitudes (TOSRA) was created to assess students’ attitude towards science during secondary education (Fraser,
1978). Affective Elements of Science Learning Questionnaire is another instrument to investigate students’ attitude
towards science (Williams et al., 2011). However, few studies have looked at students’ attitude towards technology
and engineering fields (Johnpaul et al., 2018). The technology aspect of STEM is inclined to be treated as a toolkit
that integrates technology into mathematics and science, rather than as an independent discipline of STEM (Ken-
nedy et al., 2018). Lederman and Lederman (2013) posed a question of whether only mathematics and science
were worthy of attention, instead of integrating technology and engineering as individual disciplines into STEM.
Factors influencing 21st century learning skills
The conceptual frameworks of 21st century learning skills are addressed by the organizations including the
Metiri Group and NCREL (2003), the Organization for Economic Cooperation and Development (2005), the Partner-
ship for 21st century skills (2006; 2021), and the American Association of Colleges and Universities (2007). All of the
conceptual frameworks concentrate on the demand to combine 21st century learning skills with the core content
of disciplines, including science, mathematics, technology and engineering. Increasing 21st century skills through
STEM education has been focused among educators (Bybee, 2010; Jang, 2016; Li et al., 2019). The acquisition of 21st
century learning skills calls for more innovative supports from STEM education to improve learning effectiveness. Li
et al. (2019) posited that students can develop thinking skills in a new way in STEM education and that these new
thinking skills are connected to 21st century skills. Accordingly, teachers are required to enhance learners’ positive
attitude through integrating science and engineering practices in their classrooms, which can explicitly improve
students’ 21st century learning skills (Kelley et al., 2020; NGSS Lead States, 2013; NRC, 2012).
According to a meta-analysis performed by Lent et al. (2018) using data from 143 studies, the social cognitive
career theory (SCCT) model has been applied in disciplinary STEM education, and the relationships between attitude
and career interest have received support overall. It has been reported that students’ positive attitude towards STEM
has a significant impact on the realization of STEM education objectives and 21st century learning skills acquisition
(Luo et al., 2019; Mahoney, 2010; Tseng et al., 2013). For STEM attitude, the existing research mainly focuses on its
effect on learning effectiveness (Han et al., 2021), or concentrates on the comparison of differences in STEM attitude
(Zhou et al., 2019). The other research also concentrates on the functional description of the effect of STEM attitude on
21st century learning skills learning (Akcanca, 2020). Contemporary educational standards indicate that students can
enhance 21st century skills and develop confidence through the integration of STEM subjects. As all of the conceptual
frameworks of 21st century learning skills address the critical underpinning of each core content discipline, maintain-
ing positive students’ attitude towards STEM specific discipline is one of the crucial supports. Therefore, it is necessary
to explore the effect of students’ attitude towards independent STEM disciplines on 21st century learning skills.
Research Aim
From the literature review, there is little previous research on the effect of the attitude towards STEM specific
discipline to 21st century learning skills. The extent to which students’ attitude towards STEM specific discipline
contributes to 21st century learning skills is a matter of concern for the present research. According to the cur-
rent situation of the primary and secondary school curriculum, science and mathematics are always independent
subjects, while technology and engineering are usually not. Few studies have looked at students’ attitude towards
technology and engineering (Johnpaul et al., 2018; Wiebe et al., 2018). However, in the teaching of science and
mathematics, technical skills and engineering thinking are often permeated. It is worth evaluating whether stu-
dents’ attitude towards STEM specific discipline directly affects 21st century learning skills, while students’ science
708
(pp. 706-719)
attitude and mathematics attitude also indirectly affect 21st century learning skills through the mediating role of
technology and engineering attitude. Considering that students’ technology attitude and engineering attitude were
rarely separated in the previous studies (Johnpaul et al., 2018; Wiebe et al., 2018), those were integrated together
as students’ attitude towards technology and engineering in the present research.
Given that students may be inclined to flee the STEM pipeline as the school year grows, the impact of the school
year is also an important factor. Whether the school year plays a moderating role in the effect of attitude towards
mathematics, science, engineering, and technology on 21st century learning skills will be investigated as well.
The framework of factors influencing 21st century learning skills is shown in Figure 1. The following hypotheses
were proposed in the current study:
H1: Students’ attitude towards mathematics, science, engineering, and technology has a positive direct
effect on the attitude towards 21st century learning skills, and students’ science attitude and math-
ematics attitude also affect 21st century learning skills through the mediating role of technology and
engineering attitude.
H2: School year plays a significant moderating role in the pathway of H1.
Figure1
The Framework of Factors Influencing 21st Century Learning Skills
Notes: The framework of factors influencing 21st century learning skills: Mat = mathematics attitude; E&T = engineering and technol-
ogy attitude; Sci = science attitude; 21CS = 21st century learning skills.
Assessment Tool
There have been several instruments designed to measure K-12 students’ attitude towards a single STEM sub-
ject, such as TOSRA (Fraser, 1978) and ATMS (Miller et al., 1996). Also, the assessment of students’ attitude towards
multiple subjects has been developed. Erkut and Marx (2005) created an instrument that measured 8th-grade
students’ attitude towards multiple STEM subjects including engineering, mathematics, and science. Unfried et
al. (2015) developed two questionnaires (S-STEM) to measure students’ attitude towards STEM including science,
engineering and technology, mathematics, and 21st century learning skills. One of the questionnaires Upper El-
ementary S-STEM is used to test 4th-grade through 5th-grade students and the other questionnaire Middle/High
S-STEM is used to test 6th-grade through 12th-grade students. Both S-STEM questionnaires consist of four subscales
with 37 items. Four subscales are listed as mathematics attitude (items from 1 to 8), science attitude (items from 9
to 17), engineering and technology attitude (items from 18 to 26), and 21st century learning skills (items from 27
to 37). Attitude towards 21st century learning includes items measuring students’ confidence in communication,
collaboration, and self-directed learning. Each S-STEM questionnaire uses a 5-point Likert-type response scale from
709
(pp. 706-719)
strongly disagree to strongly agree. The reliability and validity evidence on the S-STEM questionnaire have been
confirmed by Cronbach alpha value and confirmatory factor analysis (Luo et al., 2019; Unfried et al., 2015). Then
the Upper Elementary S-STEM questionnaire was extended by Zhou et al. (2019) to test the 1st-grade through
6th-grade students’ attitude towards STEM and proved to adequately cover the whole K-12 education system.
Therefore, S-STEM questionnaire was opted as an assessment tool for assessing the relation between students’ at-
titude towards STEM and 21st century learning skills in the present study. The Elementary S-STEM was for students
from grade 1 to 6, and the Middle/High S-STEM was for students from grade 7 to 12.
Participants
As the present study aimed to explore whether school year plays a significant moderating role in the relation
between STEM attitude and 21st century learning skills, students from primary schools and secondary schools were
recruited. The formal sample included first- to twelfth-grade students from three primary schools and two secondary
schools. One primary school and one secondary school locate in a province of southern China, and the other two
primary schools and one secondary school locate in a province of northern China. In total, 410 primary students in
12 classes from three primary schools and 435 secondary students in 12 classes from two secondary schools were
recruited for the research. The research program was introduced by one of our authors with the support of headteach-
ers of each school. Students were informed before doing the S-STEM questionnaire that their responses were only for
research purposes and their personal information would not be released. By eliminating blank and invalid responses,
a total of 779 students were included in the study, with 380 students in the primary school group and 399 students
in the secondary school group. Table 1 illustrates the statistics of the valid sample from the two groups. The sample
of participants in the primary schools included students from grade one to grade six (249 boys and 131 girls). Of the
participants in the primary group, 65.5% were male and 34.5% were female. In the secondary schools, there were 213
boys and 186 girls participating in the study. Of the secondary participants, 53.3% were male and 46.7% were female.
The participating primary and secondary schools are all national public schools. Students in these primary and sec-
ondary schools are taught in accordance with the curriculum established by the Ministry of Education (MOE) in China.
Table 1
Demographics of Students at Each Grade for the Research Sample
Group Grade No. of Sample No. of Male No. of Female
Primary 1st 24 21 3
2nd 60 44 16
3rd 61 41 20
4th 82 58 24
5 th
44 23 21
6th 109 62 47
Secondary 7th 39 18 21
8 th
75 33 42
9th 57 30 27
10 th
78 43 35
11th 69 44 25
12 th
81 45 36
Total 779 462 317
Data Analysis
Data analysis in this study was conducted using Mplus version 8.3. The Cronbach’s alpha coefficient was firstly
examined to confirm the reliability of the data. Then, confirmatory factor analysis (CFA) using the Structural Equa-
tion Modelling (SEM) technique was applied to confirm the reliability and validity of the measurement model for
710
(pp. 706-719)
the assessment. SEM is a statistical method to measure the direct and indirect effects between latent variables.
To test hypothesis 1, the structural model was applied to the data. In the structural model, the coefficient of the
path between constructs indicates the strength of the relationship and the R2 value explains the variation in the
endogenous variable caused by exogenous variables. The path coefficient and R2 value determine whether the
data represent the constructing model well (Wu & Chen 2017). In hypothesis 2, to evaluate the moderating role
of the school year in the moderated mediation model, the latent moderated structural equations (LMS) approach
was used to estimate the direct and indirect interaction effects, with respect to the estimates and 95% confidence
intervals (CI). The significance of the interaction effects was computed using bootstrapping procedures.
Research Results
Preliminary Analyses
Table 2 presents the descriptive statistics of measurement items of S-STEM questionnaire. In the primary
group, the skewness values of each item ranged between - .304 and - .083 and the kurtosis values of each item
ranged between - .859 and 1.18. In the secondary group, the skewness values of each item changed between -
.890 and - .269 and the kurtosis values of each item changed between - .556 and 2.25. According to the guidelines
provided by Kline (2005), the distribution of the data is considered as the univariate normality distribution if the
absolute skew is less than 3 and the absolute kurtosis is less than 10. On this basis, the data meet the assumption
of univariate normality.
The Cronbach’s alpha coefficient was calculated to measure the internal consistency reliability of the S-STEM
test. The Cronbach’s alpha coefficients for four scales of the whole test in Primary and Secondary are presented in
Table 2. Specifically, the Cronbach’s alpha coefficients of four scales for two groups were all greater than .807. Ac-
cording to the results, it can be argued that a sufficient level of internal consistency reliability for the instruments
from two groups was demonstrated (α < .5 unacceptable, .5 ≤ α < .6 poor, .6 ≤ α < .7 acceptable, .7 ≤ α < .9 good,
α > .9 excellent) (George & Mallery, 2003).
Table 2
Descriptive Statistics and Reliability
Cronbach’s
Groups Categories Min Max M SD Skewness Kurtosis
alpha
Mathematics 8 40 29.39 6.44 - .220 - .859 .807

Science 9 45 32.01 7.16 - .083 - .192 .885
Primary
Engineering 9 45 33.71 7.14 - .304 - .242 .853
21st century learning skills 11 55 42.69 8.57 - .301 1.180 .872
Mathematics 8 40 26.94 7.48 - .269 - .469 .926
Science 9 45 33.38 6.97 - .399 .431 .924
Secondary
Engineering 9 45 31.32 7.54 - .357 - .556 .930
21st century learning skills 11 55 42.39 8.21 - .890 2.25 .944
Measurement Model
The measurement model was evaluated by construct reliability and convergent validity using confirmatory
factor analysis (CFA). Construct reliability was assessed with composite reliability (CR) value and convergent validity
was assessed with average variance extracted (AVE) index. Given that the distribution of the data was considered
as the univariate normality distribution, the maximum likelihood (ML) estimation technique was applied for CFA.
For the one-dimensional measurement model, the CR values of the students’ attitude towards mathematics, sci-
ence, engineering and technology, and 21st century learning skills were .891, .904, .893 and .909 respectively. All
of the CR values were greater than the threshold value of .70 (Fornell & Larcker, 1981), indicating that the internal
711
(pp. 706-719)
structure of the latent factor with multiple indicators was reliable. The AVE index of students’ attitude towards
mathematics, science, engineering and technology, and 21st century learning skills were .516, .517, .482 and .479
respectively. Although the AVE index of students’ attitude towards engineering and technology and 21st century
learning skills were less than .5, both of the CR values were greater than .6, suggesting that the convergent validity
of the construct was still adequate (Lam, 2012). The reliability and convergent validity show that the measurement
model is acceptable.
The fit of the model was commonly assessed with c2 values, root mean square error of approximation (RMSEA),
comparative fit index (CFI) and standardized root mean square residual (SRMR). A good fit is indicated by 0 ≤ c 2/df ≤
3, 0 ≤ RMSEA ≤ .05, .95 ≤ CFI ≤ 1 and 0 ≤ SRMR ≤ .05 and an acceptable fit is indicated by 0 ≤ c2/df ≤ 5, .05 ≤ RMSEA
≤ .08, .90 ≤ CFI ≤ .95 and .05 ≤ SRMR ≤ .10. The fit indices of the present model (c2 = 1719.9, df = 623, c2/df = 2.76,
p < .001, RMSEA = .048, CFI = .926, and SRMR = .048) demonstrated that the measurement model was acceptable
(c2/df < 5, RMSEA < .08, CFI > .90 and SRMR < .10).
Structural Model
Testing for the Mediating Effect in Hypothesis 1
For Hypothesis 1, it was anticipated that students’ attitude towards mathematics, science, engineering, and
technology had positive effects on 21st century learning skills, and students’ mathematics attitude and science
attitude indirectly affect 21st century learning skills through the mediating role of engineering and technology
attitude. As expected, the structural model for hypothesis 1 applied an acceptable fit to the data (c2 = 1719.9, df
= 623, c2/df = 2.76, p < .001, RMSEA = .048, CFI = .926, and SRMR = .048). Figure 2 shows the structural model of
factors influencing 21st century learning skills.
Figure 2
The Effect of the STEM Attitude on 21st Century Learning Skills
Table 3 illustrates the direct and indirect effects of students’ attitude towards independent STEM disciplines
on 21st century learning skills. According to Table 3, students’ attitude towards engineering and technology
(β = .608, p < .0001), and science attitude (β = .154, p < .001) were positively associated with 21st century learning
skills. While students’ mathematics attitude had insignificant direct relationship with 21st century learning skills
(β = .033, p = .363). The Bootstrap method with 1000 resamples was applied to examine the mediating effect of
712
(pp. 706-719)
engineering and technology attitude in the relationship between students’ attitude towards mathematics and
science and their 21st century learning skills. The indirect effect of mathematics attitude on 21st century learning
skills with the mediating role of engineering and technology attitude was β = .186, p < .001, with 95% confidence
interval [.134 .238]. The indirect effect of science attitude on 21st century learning skills with the mediating role
of engineering and technology attitude was β = .356, p < .001, with 95% confidence interval [.278 .436]. The result
suggested that both mathematics attitude and science attitude had positive effects on 21st century learning skills
through attitude towards engineering and technology as a mediator.
Table 3
Results for the Direct and Indirect Effects in the Two Groups
Effects Relationship Point Estimate 95% bias-corrected CI
E&T 21st CS .608*** [.490, .717]

Direct Effect Mat 21st CS .033 [- .051, .121]
Sci 21st CS .154** [.043,.262]
Mat E&T 21st CS .186*** [ .134, .238]
Indirect Effect
Sci E&T 21st CS .356*** [.278, .436]
Notes: E&T = attitude towards engineering and technology; Mat = mathematics attitude; Sci = science attitude; 21st CS = 21st
century learning skills; CI = confidence interval.
*p < .05, **p < .01, ***p < .001
The R2 value for students’ attitude towards mathematics, science, engineering, and technology to 21st century
learning skills was .557; hence, 55.7% of the variance was explained. Based on the empirical analysis, it can be seen
that the model of hypothesis 1 was a good fit and acceptable (Hayes 2013; Preacher & Hayes 2004). The result con-
firmed that attitude towards engineering and technology was both the dominant factor and the mediating factor
for 21st century learning skills. And the evidence from the insignificant coefficient from mathematics attitude to
21st century learning skills attitude suggested that hypothesis 1 was partially supported.
Testing for the moderating effect of the school year in Hypothesis 2
Further analysis then shifted to estimating the moderating role of the school year, which constituted the
second hypothesis of the current study. Hypothesis 2 predicted that the school year would moderate pathways
in the mediation process of hypothesis 1. Considering that the direct effect of mathematics was not significant in
hypothesis 1, the moderating effect of the school year was examined in the pathways of the relationships between
science attitude and 21st century learning skills, and between engineering and technology attitude and 21st cen-
tury learning skills. According to Table 4, the interaction term (Sci × School Year) had a significant positive impact
on 21st century skills (β = .097, p < .05), indicating that the effect of science attitude on 21st century learning skills
was moderated by different school years. Meanwhile, another interaction term (E&T × School Year) had a significant
negative impact on 21st century learning skills (β = - .235, p < .001), indicating that the effect of students’ attitude
towards engineering and technology on 21st century learning skills was also moderated by different school years.
Figure 4(a) displays the interaction of science attitude and school year in predicting 21st century learning skills
and shows a greater predictive effect with the higher school year. For students at higher school year, a higher level
of science attitude was related to higher 21st century learning skills. While Figure 4(b) displays the interaction of
engineering and technology attitude and school year in predicting 21st century learning skills and the analysis of
the slope indicated the relation between engineering attitude and 21st century learning skills weakened as the
school year increased. For students at higher school year, a higher level of engineering and technology attitude
was related to lower 21st century learning skills.
713
(pp. 706-719)
Table 4
Conditional Process Analysis
Path Coefficient Boot SE p 95% CI
Outcome: 21st CS
Predictor:
School Year .961 .200 <.001 [ .787, 1.729]
E&T × School Year - .235 .038 <.001 [- .333, -. 156]
Sci × School Year .097 .042 .020 [ .014, .178]
Figure 4
The Moderating Role of the School Year
(a) the interaction of science attitude and school year in pre- (b) the interaction of engineering and technology attitude and
dicting 21st century learning skills school year in predicting 21st century learning skills
Further detailed data demonstrated different effects of students’ STEM attitude on 21st century learning skills
in the primary group and the secondary group. In general, the structural model fits were satisfactory with c2 = 1027,
df = 623, c2/df = 1.67, p < .001, RMSEA = .041, CFI = .924, and SRMR = .050 for the primary group, with c2 = 1458,
df = 623, c2/df = 2.34, p < .001, RMSEA = .058, CFI = .927, and SRMR = .058 for the secondary group (see Table 5).
Table 5
CFA Goodness-of-Fit Indices
Group c2 df c2/df p RMSEA CFI SRMR
Primary 1027 623 1.67 <.001 .041 .924 .050
Secondary 1458 623 2.34 <.001 .058 .927 .058
As seen in Table 6, in terms of direct effects, students’ attitude towards engineering and technology was posi-
tively directly associated with 21st century learning skills in both the primary group (β = .721, p < .0001) and the
secondary group (β = .549, p < .001). While Mathematics attitude significantly predicted 21st century learning skills
for the primary group (β = .160, p < .01), but not for the secondary group (β = - .049, p > .05). In contrast, science
attitude significantly predicted 21st century learning skills for the secondary group (β = .269, p < .001), but not for
the primary group (β = - .027, p > .05). The results indicated that direct effects of mathematics attitude and science
attitude on 21st century learning skills were not identical in different school years. In terms of indirect effects, the
effect of students’ mathematics attitude on 21st century skills through the mediating role of engineering and
technology attitude was significant both in the primary group (β = .127, p < .01) and in the secondary group (β =
.177, p < .001). As well, the indirect effect of students’ science attitude on 21st century skills through the mediating
role of engineering and technology attitude was significant both in the primary group (β = .475, p < .001) and in
the secondary group (β = .318, p < .001).
714
(pp. 706-719)
Table 6
Results for the Direct and Indirect Effects in Two Groups
Group Effects Relationship Point Estimate 95% bias-corrected CI
E&T 21st CS .721*** [.534, .855]

Direct Effect Mat 21st CS .160** [.046, .270]
Primary Sci 21st CS - .027 [- .160, .131]
Mat E&T 21st CS .127** [.045, .230]
Indirect Effect
Sci E&T 21st CS .475*** [.350, .610]
E&T 21st CS .549*** [.386, .708]
Direct Effect Mat 21st CS - .049 [- .178, .074]
Secondary Sci 21st CS .269*** [.118, .429]
Mat E&T 21st CS .177*** [.113, .252]
Indirect Effect
Sci E&T 21st CS .318*** [.216, .432]
Notes: E&T = attitude towards engineering and technology; Mat = mathematics attitude; Sci = science attitude; 21st CS = 21st
century learning skills; CI = confidence interval.
*p < .05, **p < .01, ***p < .001
Discussion
This present study aimed to identify the extent to which students’ attitude towards STEM specific disciplines
contributes to 21st century learning skills. The study hypothesized that students’ attitude towards mathematics,
science, engineering, and technology has a positive direct effect on the attitude towards 21st century learning
skills, and students’ science attitude and mathematics attitude also affect 21st century learning skills through the
mediating role of technology and engineering attitude, as well the school year plays a moderating role in the
above relationships.
The findings confirmed that STEM attitude positively affected 21st century learning skills, with a total direct
effect of .795, indicating that students’ positive attitude towards STEM had a significant impact on their development
of 21st century learning skills. This result supports the previous research (Akcanca, 2020), which emphasized the
importance of focusing on students’ STEM attitude in developing their 21st century learning skills. By comparing
students’ STEM discipline-specific attitude, both science attitude and attitude towards engineering and technol-
ogy had positive effects on 21st century learning skills, while mathematics attitude had no significant effect on
21st century learning skills. Previously, researchers have argued that attitude towards different STEM disciplines
may lead to varied effects, and it is necessary to systematically look at data on attitude of students in different
STEM fields (Minner et al., 2012). This viewpoint is clearly supported by the results of the hypothesis in the present
research. The current finding is consistent with previous research highlighting the importance of valuing STEM at-
titude across disciplines. As Johnpaul et al. (2018) emphasized, trends of students’ attitude towards different STEM
disciplines are not identical, so it is critical to attach importance to the STEM attitude of each discipline. From the
results of the research, students’ STEM attitude towards different disciplines contributes differently to 21st century
learning skills. In particular, students’ attitude towards engineering and technology dominates the impact on 21st
century learning skills with a coefficient of .608. It can be argued that students’ attitude towards engineering and
technology is more conducive to improving students’ 21st century learning skills.
According to the first hypothesis, another aim was to verify the mediation model with the engineering and tech-
nology attitude as a mediator. The outcomes demonstrated that both students’ mathematics attitude and science
attitude had positive effects on 21st century learning skills through attitude towards engineering and technology
as a mediator. The results of path analysis from students towards science attitude to engineering and technology
attitude (0.586) and from engineering and technology attitude to 21st century learning skills (0.608) were both
found to be significant. Therefore, the mediating effect of engineering and technology attitude between science
attitude and 21st century learning skills was significant. The outcome supports the fact that improving students’
science attitude could promote their 21st century learning skills by emphasizing their engineering and technology
715
(pp. 706-719)
attitude. Besides, the path coefficients from students’ mathematics attitude to engineering and technology attitude
(0.306) and from engineering and technology attitude to 21st century learning skills were both significant (0.608).
It suggests that although the direct effect of mathematics attitude on 21st century learning skills is not significant,
its indirect effect is significant through the mediating role of engineering and technology attitude. This result em-
phasizes the contribution of engineering and technology attitude to 21st century learning skills, not only in terms
of the direct effect but also in terms of the mediating effect. It could reduce the doubt of the previous study on
whether attitude towards engineering and technology should be treated as an independent disciplinary attitude
(Lederman & Lederman, 2013). Lederman and Lederman (2013) posed a question of whether only mathematics
and science were worthy of attention, instead of integrating technology and engineering as individual disciplines
into STEM. The concern is related to the lack of research on engineering and technology attitude as a STEM disci-
plinary attitude (Johnpaul et al., 2018). This research supports that attitude towards engineering and technology
is important for 21st century learning skills and should be studied as an important dimension of STEM disciplinary
attitude (Erkut & Marx, 2005; Johnpaul et al., 2018; Zhou et al., 2019; 2021). As well, it emphasizes the significance
of the relationship between disciplines, in particular, the important role of engineering and technology attitude
between science attitude and mathematics attitude and 21st century learning skills.
On the basis of previous outcomes, further research results confirmed the second hypothesis that school
year partially moderated pathways in the mediation process of the first hypothesis. It can be seen that the direct
effects of students’ science attitude and engineering and technology attitude on 21st century learning skills were
moderated by different school years. Given the dominant effect of engineering and technology attitude on 21st
century learning skills and its decreasing effect with school year, it may mostly contribute to leaks from current
STEM pipelines (Ball et al., 2017; Doerschuk et al., 2016). Since the effect of engineering and technology attitude on
21st century learning skills was negatively moderated by school year, it was not only directly weakened its direct
effect on 21st century learning skills, but also indirectly weakened the effect of science attitude and mathematics
on 21st century learning skills due to its mediating effect. The moderating role of the school year should not be
ignored in the indirect pathway with the mediating role of engineering and technology attitude. As many studies
have highlighted the fact that older students are less likely to have positive STEM attitudes than younger students,
it is critical to pay close attention to the changing effect of learning attitude on 21st century learning skills driven
by the school year (Potvin & Hasni, 2014a; Unfried et al., 2014).
This study aimed to examine the extent to which students’ attitude towards science, technology, engineering,
and mathematics (STEM) contributes to 21st century learning skills. The results of structural equation modelling
analysis revealed that (a) students’ science attitude, and engineering and technology attitude had positive effect
on 21st century learning skills, and (b) students’ science attitude and mathematics attitude positively affected
21st century learning skills indirectly through the mediating role of technology and engineering attitude, and (c)
the school year partially played a moderating role. The results highlight the importance of valuing STEM attitude
across disciplines. In particular, students’ engineering and technology attitude is more conducive to improving
students’ 21st century learning skills. The finding further emphasizes the important role of engineering and tech-
nology attitude between science attitude and 21st century learning skills and between mathematics attitude and
21st century learning skills. It also suggests the need to pay close attention to the change in the impact of STEM
learning attitude on 21st century learning skills driven by the school year.
Future work can be further expanded in the following three aspects: (1) The sample data covers a large span
through k - 12, with a limited number of participants in each school year group. Future studies could increase
the number of participants in each school year group to test for higher measurement reliability. (2) The impact
of attitude towards engineering and technology on 21st century learning skills without considering the effect
of gender as a moderating variable. Future research will continue to test whether gender plays a moderating
role in the moderated mediation model. (3) The data for evaluating 21st century learning skills were obtained by
students’ self-report from a subjective point of view. In the later research, it is worth retrieving multi-dimensional
data through the evaluation of students from their teachers and parents, or students’ mutual evaluation. Efforts
can also be made to design project-based programs for the evaluation of students’ 21st century learning skills
through practical problem-solving.
There were still some limitations in this study. Due to the lack of literacy and understanding ability of students
716
(pp. 706-719)
in grades one and two, it was difficult to complete every item of the questionnaire independently. Hence, providing
explanations for the questions to the first and the second graders may result in certain influences and limitations
on the consistency of the research data. In addition, the research data of each sample group were only collected
from two or three schools. The result only reflects the attitude and skills of students at a certain level, but not the
evaluation of a larger span of population, nor the attitude and skill differences of students at a finer level. Moreover,
the imbalance in the number of male and female students in the first grade will lead to the loss of reliable support
for the design of taking gender as an adjusting variable in the subsequent study.
Acknowledgement
The authors would like to acknowledge the help of the Editor and the anonymous reviewers. The research
is supported in part by the Education and Teaching Reform Research and Practice of Guangdong Mechanical &
Electrical Polytechnic under the Grant No. YJZD2021-06, and Electronics and Communication Teaching Steering
Committee of Guangdong Vocational Colleges on General Teachers.
References
Akcanca, N. (2020). 21st century learning skills: The predictive role of attitudes regarding STEM education and problem-based
learning. International Journal of Progressive Education, 16(5), 443-458. https://doi.org/10.29329/ijpe.2020.277.27
Allen-Ramdial, S., & Campbell, A.G. (2014). Reimagining the pipeline: Advancing STEM diversity, persistence, and success.
Bioscience, 64(7), 612. https://doi.org/10.1093/biosci/biu076
American Association of Colleges and Universities. (2007). College learning for the new global century. Washington.
Australian Education Council. (2015). National STEM School Education Strategy 2016-2026. http://www.educationcouncil.edu.au/
site/DefaultSite/filesystem/documents/National%20STEM%20School%20Education%20Strategy.pdf
Ball, C., Huang, K. T., Cotten, S. R., & Rikard, R. V. (2017). Pressurizing the STEM pipeline: An expectancy-value theory analysis of
youths’ STEM attitudes. Journal of Science Education and Technology, 26(4), 372-382. https://doi.org/10.1002/sce.21108
Beswick, K., & Fraser, S. (2019). Developing mathematics teachers’ 21st century competence for teaching in STEM
contexts. ZDM, 51(6), 955-965. https://doi.org/10.1007/s11858-019-01084-2
Bybee, R. W. (2010). Advancing STEM Education: A 2020 Vision. Technology and Engineering Teacher, 70(1), 30–35.
Caprile, M., Palmen, R., Sanz, P., & Dente, G. (2015). Encouraging STEM studies for the labour market. Brussels: European Parliament.
http://www.europarl.europa.eu/RegData/etudes/STUD/2015/542199/IPOL_STU%282015%29542199_EN.pdf
Doerschuk, P., Bahrim, C., Daniel, J., Kruger, J., Mann, J., & Martin, C. (2016). Closing the gaps and filling the STEM pipeline: A
multidisciplinary approach. Journal of Science Education and Technology, 25(4), 682-695. https://doi.org/10.1007/s10956-
016-9622-8
Erkut, S., & Marx, F. (2005). 4 schools for WIE (Evaluation report). Wellesley College, Center for Research on Women. http://www.
coe.neu.edu/Groups/stemteams/evaluation.pdf
Faber, M., Unfried, A., Wiebe, E. N., Corn, J., Townsend, L. W., & Collins, T. L. (2013, June). Student attitudes toward STEM: The
development of upper elementary school and middle/high school student surveys. In 2013 ASEE Annual Conference &
Exposition (pp. 23-1094). Atlanta, GA, United States. https://peer.asee.org/student-attitudes-toward-stem-the-development-
of-upper-elementary-school-and-middle-high-school-student-surveys
Fraser, B. (1978). Development of a test of science-related attitudes. Science Education, 62, 509-515. https://doi.org/10.1002/
sce.3730620411
Gardner, P. L. (1975). Attitudes to science: a review. Studies in Science Education,2,1-41. https://doi.org/10.1080/03057267508559818
George, D., & Mallery, P. (2003). SPSS for windows step by step: a simple guide and reference. Boston.
Hayes, A. (2013). Introduction to mediation, moderation, and conditional process analysis. New York.
Han, J., Kelley, T. & Knowles, J. (2021). Factors influencing student STEM learning: Self-efficacy and outcome expectancy, 21st
century skills, and career awareness. Journal for STEM Educational Research, 4, 117–137. https://doi.org/10.1007/s41979-
021-00053-3
International Technology and Engineering Educators Association (ITEEA). (2020). Standards for technological and engineering
literacy: Defining the role of technology and engineering in STEM education. Reston.
Jang, H. (2016). Identifying 21st century STEM competencies using workplace data. Journal of Science Education and Technology,
25(2), 284-301. https://doi.org/10.1007/s10956-015-9593-1
Johnpaul, K., Frances, Q., & Terry, L. (2018). The keys to STEM: Australian year 7 students’ attitudes and intentions towards science,
mathematics, and technology courses. Research in Science Education, 50(5), 1805-1832. https://doi.org/10.1007/S11165-
018-9754-3
717
(pp. 706-719)
Karahan, E. (2019). Stem education approach. In A. G. Balım (Ed.), New Approaches in Science Education (pp. 171-186).
Kelley, T. R., Knowles, J. G., Holland, J. D., & Han, J. (2020). Increasing high school teachers’ self-efficacy for integrated STEM
instruction through a collaborative community of practice. International Journal of STEM Education, 7, 1–13. https://doi.
org/10.1186/s40594-020-00211-w
Kennedy, J., Quinn, F., & Lyons, T. (2018). Australian enrolment trends in technology and engineering: putting the T and E back
into school STEM. International Journal of Technology and Design Education, 28(2), 553-571. https://doi.org/10.1007/s10798-
016-9394-8
Kline, R. B. (2005). Principles and practice of structural equation modeling. New York.
Krskova, H., Wood, L. N., Breyer, Y. A. & Baumann, C. (2020). F.I.R.S.T: Principles of discipline for 21st century learning skills. In L. Wood,
L. P. Tan, Y. A. Breyer, & S. Hawse (Eds.), Industry and Higher Education: Case Studies for Sustainable Futures (pp. 265-289).
Singapore. https://doi.org/10.1007/978-981-15-0874-5_12
Lam, L. W. (2012). Impact of competitiveness on salespeople’s commitment and performance. Journal of Business Research, 65(9),
1328-1334. https://doi.org/10.1016/j.jbusres.2011.10.026
Lederman, N. G., & Lederman, J. S. (2013). Is it STEM or “S & M” that we truly love? Journal of Science Teacher Education, 24(8),
1237-1240. https://doi.org/10.1007/s10972-013-9370-z
Lent, R. W., Sheu, H. B., Miller, M. J., Cusick, M. E., Penn, L. T., & Truong, N. N. (2018). Predictors of science, technology, engineering,
and mathematics choice options: A meta-analytic path analysis of the social–cognitive choice model by gender and race/
ethnicity. Journal of Counseling Psychology, 65(1), 17–35. https://doi.org/10.1037/cou0000243
Li, Y., Schoenfeld, A. H., diSessa, A. A., Graesser, A. C., Benson, L. C., English, L. D., & Duschl, L. D. (2019). On thinking and STEM
education. Journal for STEM Education Research, 2, 1–13 https://doi. org/10.1007/s41979-019-00014-x
Lindahl, B. (2007). A longitudinal Study of students’ attitudes towards science and choice of career. Paper presented at NARST Annual
Conference, April 15-18, 2007, New Orleans.
Luo, W., Wei, H. R., Ritzhaupt, A. D., Huggins-Manley, A. C., & Gardner-Mccune, C. (2019). Using the s-stem survey to evaluate a
middle school robotics learning environment: validity evidence in a different context. Journal of Science Education and
Technology, 28(4), 429-443. https://doi.org/10.1007/s10956-019-09773-z
Mahoney, M. P. (2010). Students’ attitudes toward STEM: development of an instrument for high school STEM-based programs.
Journal of Technology Studies, 36(1), 24–34. https://doi.org/10.21061/jots.v36i1.a.4
Metiri Group & NCREL. (2003). enGauge 21st century learning skills: Digital Literacies for a Digital Age. Chicago.
Miller, R. B., Greene, B. A., Montalvo, G. P., Ravindran, B., & Nichols, J. D. (1996). Engagement in academic work: The role of learning
goals, future consequences, pleasing others, and perceived ability. Contemporary Educational Psychology, 21(4), 388–422.
https://doi.org/10.1006/ceps.1996.0028
Minner, D., Ericson, E., Wu, S., & Martinez, A. (2012). Compendium of STEM Student Instruments PART II: Measuring Students’
Content Knowledge, Reasoning Skills, and Psychological Attributes. Abt Associates. https://cadrek12.org/sites/default/files/
Compendium%20of%20STEM%20instruments%20Part%202_11-20-12.pdf
Moakler, M. W., & Kim, M. M. (2014). College major choice in STEM: Revisiting confidence and demographic factors. Career
Development Quarterly, 62(2), 128–142. https://doi.org/10.1002/j.2161-0045.2014.00075.x
Moore, B. (2009). Emotional intelligence for school administrators: A priority for school reform? American Secondary Education,
37(3), 20-28. https://www.jstor.org/stable/41406313
Mutiani, M. & Faisal, M. (2020). Urgency of the 21st century learning skills and social capital in social studies. The Innovation of
Social Studies Journal, 1(1), 1-11. https://doi.org/10.20527/iis.v1i1.1256
National Institute of Literacy and the Small Business Administration. (1999). 21st century learning skills for 21st Century Jobs.
Washington.
National Research Council. (2012). A framework for K-12 science education: Practices, crosscutting concepts,
and core ideas. National Academies Press.
NGSS Lead States. (2013). Next generation science standards. National Academies Press.
OECD. (2005). The definition and selection of key competencies executive summary. Accessed 20 April 2019.
https://www.oecd.org/pisa/35070367.pdf
Organization for Economic Cooperation and Development. (2005). The definition and selection of key competencies: Executive
summary. Paris.
Park, E. K. & Suh, J. K. (2020). Active learning using google suites for education to promote 21st-century learning skills. Transforming
Teaching Through Active Learning A National Symposium, Miami, Florida.
Partnership for 21st century skills. (2006). A state leader’s action guide to 21 st century skills: A new vision for education. Tucson.
Partnership for 21st century Skills. (n.d.) http://www.p21.org/framework.
Potvin, P., & Hasni, A. (2014a). Analysis of the decline in interest towards school science and technology from grades 5 through
11. Journal of Science Education and Technology, 23(6), 784-802. https://doi.org/10.1007/s10956-014-9512-x
Potvin, P., & Hasni, A. (2014b). Interest, motivation and attitude towards science and technology at K-12 levels: A systematic
review of 12 years of educational research. Studies in Science Education, 50(1), 85-129. https://doi.org/10.1080/03057267
.2014.881626
Preacher, K. J., & Hayes, A. F. (2004). SPSS and SAS procedures for estimating indirect effects in simple mediation models. Behavior
Research Methods, Instruments, & Computers, 36(4), 717–731. https://doi.org/10.3758/BF03206553
Reinhold, S., Holzberger, D., & Seidel, T. (2018). Encouraging a career in science: A research review of secondary schools’ effects
on students’ STEM orientation. Studies in Science Education, 54(1), 1-35. https://doi.org/10.1080/03057267.2018.1442900
718
(pp. 706-719)
Rotherham, A. J., & Willingham, D. T. (2010). 21st-century skills. American Educator, 17(1), 17-20.
Shapiro, C., & Sax, L. (2011). Major selection and persistence for women in STEM. New Directions for Institutional Research, 2011
(152), 5-18. https://onlinelibrary.wiley.com/doi/10.1002/ir.404
Speering, W., & Rennie, L. (1996). Students’ perceptions about science: The impact of transition from primary to secondary school.
Research in Science Education, 26(3), 283-298. https://link.springer.com/article/10.1007/BF02356940
Tseng, K.-H., Chang, C.-C., Lou, S.-J., & Chen, W.-P. (2013). Attitudes towards science, technology, engineering and mathematics
(STEM) in a project-based learning (PjBL) environment. International Journal of Technology and Design Education, 23(1),
87-102. https://doi.org/10.1007/s10798-011-9160-x
Unfried, A., Faber, M., Stanhope, D. S., & Wiebe, E. (2015). The development and validation of a measure of student attitudes
toward science, technology, engineering, and math (s-stem). Journal of Psychoeducational Assessment, 33(7), 622-639.
https://journals.sagepub.com/doi/10.1177/0734282915571160
Unfried, A., Faber, M., & Wiebe, E. (2014). Gender and student attitudes toward STEM. Presented at the AERA Annual Meeting,
Philadelphia, PA. https://eval.fi.ncsu.edu/wp-content/uploads/2016/03/AERA-2014-Unfried-Faber-Wiebe.pdf
Wiebe, E., Unfried, A., & Faber, M. (2018). The relationship of STEM attitudes and career interest. Eurasia Journal of Mathematics,
Science and Technology Education, 14(10), em1580. https://doi:10.29333/ejmste/92286
Williams, K., Kurtek, K., & Sampson, V. (2011). The affective elements of science learning: A questionnaire to assess-and improve-
student attitudes toward science. Science Teacher, 78(1), 40-45. https://www.proquest.com/scholarly-journals/affective-
elements-science-learning/docview/860711736/se-2?accountid=13741
Xue, Y., & Larson, R. C. (2015). STEM crisis or STEM surplus? Yes and yes. Monthly labor review.
Voogt, J., Erstad, O., Dede, C. & Mishra, P. (2013). Challenges to learning and schooling in the digital networked world of the
twenty-first century. Journal of Computer Assisted Learning, 29, 403-413. https://doi.org/10.1111/jcal.12029
Zhou, S., Zeng, H., Xu, S., Chen, L., & Xiao, H. (2019). Exploring changes in primary students’ attitudes towards science, technology,
engineering and mathematics (stem) across genders and grade levels. Journal of Baltic Science Education, 18(3), 466-480.
https://dx.doi.org/10.33225/jbse/19.18.466
Zhou, S., Chen, L., Xu, S., Lu, C., Li, Q., & Li, D. (2021). Primary students’ performance of STEM domain-specific self-efficacy belief
and expectancy-value belief. Journal of Baltic Science Education, 20(4), 677-690. https://doi.org/10.33225/jbse/21.20.677
Received: May 14, 2022 Revised: June 20, 2022 Accepted: July 27, 2022
Cite as: Xu, S.-R., & Zhou, S.-N. (2022). The effect of students’ attitude towards science, technology, engineering, and
mathematics on 21st century learning skills: A structural equation model. Journal of Baltic Science Education, 21(4), 706-719.
Shao-Rui Xu MSc, Lecturer, School of Electronics and Communication, Guangdong

Mechanical & Electrical Polytechnic, Guangzhou 510550, China.
E-mail: jayxee@hotmail.co.uk
Shao-Na Zhou PhD, Associate Professor, School of Physics and Telecommunication
(Corresponding author) Engineering, South China Normal University, Guangzhou 510006, China.
E-mail: zhou.shaona@m.scnu.edu.cn
719
ASSESSMENT OF KNOWLEDGE
INTEGRATION IN STUDENT
LEARNING OF BUOYANT FORCE
Abstract. Students’ problem-solving Yi Zou,

ability depends on their understanding Lizhen Jin,
of related scientific concepts. Therefore,
Yanbing Li,
the modeling and assessment of students’
understanding of specific scientific
Tao Hu
concepts is important to promote students’
problem-solving ability, as it can find
students’ understanding difficulties
and explore breakthrough strategies
accordingly. Inspired by the theory of
knowledge integration and combined
with the situational characteristics of
science education in China, this study Introduction
established a conceptual framework
about buoyant force, which was applied to With the increasingly close relationship between science and social
model students’ different understandings living, to develop students’ ability to solve problems in real contexts
of it. And based on the established has become the fundamental goal of science education (Murcia, 2009;
framework, an assessment of buoyant National Research Council, 2008; Özdem, 2010). However, many students
force was designed and tested among 622 can successfully solve problems in typical and familiar contexts, but when
Chinese lower-secondary school students. they face problems in complex or unfamiliar contexts, they are often
Through the analysis of the test data and helpless (Alonso & Marcelo, 1992; Chiu et al., 2007; Kim & Pak, 2002). The
the interview outcomes, it was found reason for this phenomenon lies in the traditional instruction to a great
that students’ understanding of buoyant extent, which has the characteristics of memorizing rules and formulas
force could be divided into three levels of by rote (Bloom et al., 1956; Rivet & Krajcik, 2010). As a result, students
knowledge integration including novice, develop a tendency to rely on pattern matching rather than concept
intermediate, and expert. Furthermore, understanding when solving problems.
the results demonstrate that an emphasis Recognizing the weaknesses of students’ understanding of scien-
on the nature of buoyant force can be an tific concepts and the shortcomings of the traditional instruction, many
effective strategy to help students achieve modeling and assessment studies have been explored to characterize
a deeper conceptual understanding of and identify students’ conceptual understanding on a specific topic, so
buoyant force, leading to a more integrated that teachers can grasp students’ learning difficulties and then adopt
knowledge structure. new targeted teaching methods to improve students’ conceptual
understanding on the topic (Bao & Fritchman, 2021; Demkanin, 2020;
Keywords: assessment of knowledge
Demkanin et al., 2022; Gilbert & John, 2012; Linn, 2000; Scalise, 2012).
integration, buoyant force, central idea,
Inspired by the theory of knowledge integration, this study explored an
conceptual framework, scientific concept
alternative modeling and assessment approach to clearly probe features
understanding
of students’ conceptual understanding in learning of buoyant force by
establishing a conceptual framework and designing and implementing
Yi Zou, Lizhen Jin a test accordingly.
Zhejiang Normal University, China In perspective of knowledge integration, students’ knowledge struc-
Yanbing Li, Tao Hu
East China Normal University, China
ture is the key factor to determine their problem-solving performance
(Champagne et al., 1981; Chi et al., 1981; Gerace et al., 2001; Schoenfeld &
Herrmann, 1982; Snyder, 2000). The knowledge structure about a specific
720
ISSN 1648–3898 /Print/ ASSESSMENT OF KNOWLEDGE INTEGRATION IN STUDENT LEARNING OF BUOYANT FORCE
(pp. 720-738)
topic of experts is a logically integrated conceptual framework, which connected the related ideas, concep-
tions, and contextual variables through certain reasoning pathways (Bao & Fritchman, 2021; Champagne et
al., 1981; Chi et al., 1981; Eylon & Reif, 1984; Schoenfeld & Herrmann, 1982). Such a conceptual framework
can be activated by contextual features when experts encounter problems, and then some specific strategic
reasoning pathways emerge, which makes experts have a great probability to solve problems smoothly (Dai
et al., 2019; Duit & Treagust, 2003; Nie et al., 2019; Xie et al., 2021; Xu et al., 2020). On the other hand, the
knowledge structure about the same topic of novices is loose and lacking organization, which only con-
tains some knowledge fragments such as calculation equations and related variables. When novices solve
problems, they tend to focus on the variables in the context, and then directly match these variables with
the equations in memory (Champagne et al., 1981; Chi et al., 1981; Eylon & Reif, 1984; Gerace et al., 2001;
Schoenfeld & Herrmann, 1982). If the matching fails, they have no other strategies to solve the problems.
In summary, helping students construct more integrated conceptual frameworks for the particular concept
is the key to improve their problem-solving ability (Dai et al., 2019; Nie et al., 2019; Xie et al., 2021; Xu et al.,
2020), and clearly presenting an ideal conceptual framework to analyze and assess the characteristics of
students’ knowledge structures is the basic and prerequisite work. Accordingly, the two goals of this study
were: (1) Construct a conceptual framework on the topic of buoyant force from the visual angle of the knowl-
edge integration and apply it to analyze students’ understanding in learning buoyant force. (2) Employ an
assessment based on the conceptual framework to test students’ understanding statuses of buoyant force
and make some inferences on their knowledge structures.
The Conceptual Framework of Buoyant Force
Informed by many existing knowledge integration studies, identifying a central idea is the most impor-
tant step to build a conceptual framework for a specific concept (Dai et al., 2019; Kubsch et al., 2018; Nie et
al., 2019; Xie et al., 2021; Xu et al., 2020). The central idea is like an anchor point with strong adsorption force,
which can link other relevant ideas and conceptions according to certain logical relationships (Dai et al., 2019;
Nie et al., 2019; Xie et al., 2021; Xu et al., 2020). Once a conceptual framework has been established, it can
be used as a guidance to design and employ some assessment instruments to accurately map out students’
conceptual understanding on a specific topic, as it provides core scientific principles and explanatory mecha-
nisms on the topic (Bao & Fritchman, 2021; Dai et al., 2019; Nie et al., 2019; Xie et al., 2021; Xu et al., 2020).
It should be noted that the choice of the central idea is not fixed, the form and structure of the conceptual
framework can also be diversified, it mainly depends on the purpose and emphasis of the study and the views
of experts in the field (Dai et al., 2019; Duit & Treagust, 2003; Nie et al., 2019; Xie et al., 2021; Xu et al., 2020).
This study aimed to model and assess students’ understanding of the concept of buoyant force, found out
the difficulties, and provided empirical evidence for exploring the corresponding breakthrough strategies.
Therefore, the selection of the central idea was carried out on the basis of reviewing the relevant studies
on students’ learning difficulties of buoyant force and sufficient expert consultation and demonstration.
The floating and sinking of an object is a common phenomenon in daily life, but students’ under-
standing about buoyant force formed through daily experience often has many mistakes, such as big/heavy
objects sink and small/light things float, hollow objects float and objects with air in them float, objects with
holes sink (Besson, 2004; Kim & Kim, 2012; Libarkin et al., 2003; Loverude et al., 2003; Radovanović & Sliško,
2013; Smith et al., 1985; Suat, 2008; Tomo, 2021; Wagner et al., 2013; Yin et al., 2008). From these miscon-
ceptions, it can be seen that the concept of buoyant force seems to involve many variables, such as mass,
weight, volume, shape, and so on. Some studies have pointed out that buoyant force is a difficult concept
for students because it is closely related to too many other scientific concepts and principles, and students
usually can’t deal with too many knowledge elements at the same time unless they can integrate them in a
certain effective way (Gao et al., 2020; Halford et al., 1986; Minogue & Borland, 2016; Mullet, 1988; Paik et al.,
2017; Suat, 2008; Yin et al., 2014). In this regard, many studies have shown that understanding the nature of
buoyant force may be such an effective way (Loverude et al., 2003; Paik et al., 2017; Young & Meredith, 2017).
In other words, the most important thing in learning about the buoyant force is to understand its essential
meaning, that is, the buoyant force is an effect force, which is caused by the effect of liquid gravity on the
surfaces of an immersed object (Kim & Paik, 2021; Lima & Monteiro, 2013). With this understanding basis,
the interpretation of buoyant force related phenomena and the solution of buoyant force related problems
721
(pp. 720-738)
are correct (Paik et al., 2017; Radovanovic, 2012; Young & Meredith, 2017).
The above findings of literature review were consistent with the results of expert consultation and
demonstration. Most of the experts who participated in this study mentioned that the biggest difficulty for
students in learning buoyant force is that they cannot completely understand the definition of the buoyant
force, which is usually expressed as “the net force exerted by liquid pressures on the surface of an object”
or simply as “Fb = ΔF” (Kim & Paik, 2021; Lima & Monteiro, 2013). In fact, this definition not only reflects a
mathematical relationship, but also contains the nature of buoyant force, when problems related to buoy-
ant force need to be solved, the most important thing is to understand that the buoyant force is an effect
force, which is caused by the gravity acting on the liquid. Based on the above studies’ review and the expert
opinions, this study finally selected the nature of the buoyant force as the central idea and expressed it as
“NoB” for convenience of expression.
After identifying the central idea, this study continued to construct the conceptual framework of buoy-
ant force through expert consultation and demonstration. Since buoyant force is an effect force, it can be
compared with other forces on the object in the vertical direction (the Chinese students participating in
this study have only taken introductory level courses, so the net force only considers which in the vertical
direction here.). Such force analysis can be linked with the motion state of the object in the liquid, so as to
form a series of reasoning pathways to solve buoyant force related problems. When an object is partially or
completely immersed in a liquid, it will appear in five states in the vertical direction: emerged, suspended,
sunk, emerging, sinking. For the case of the object is emerged or suspended, which shows that the object
is in equilibrium, the equation Fb = mg can be obtained according to the principle of two force balance. In
terms of the object is sunk, which also shows that the object is in equilibrium, the equation Fb = mg-Fs can
be obtained according to the principle of two force balance and the principle of force synthesis (In science
education in China, unless otherwise specified, it is usually assumed that the space between the bottom of
the object and the bottom of the container is filled with liquid.). The Fs represents the support force of the
bottom of the container to the object. In the case where the object is emerging or sinking, it is no longer in
equilibrium, so the force analysis of the object from the perspective of balance force is no longer effective.
At the time, we need to turn our attention to the Archimedes principle, which asserts that the buoyant force
of an object immersed in a liquid is equal to the gravity of the displaced liquid (Kim & Paik, 2021; Lima &
Monteiro, 2013; Minogue & Borland, 2016; Young & Meredith, 2017). Imaging without the object, the same
immersed space will be occupied by the same volume of liquid, the weight of those liquid is supported
by other parts of the liquid. Therefore, when an object is emerging or sinking, the buoyant force can be
calculated by the formula Fb = ρgv. Of course, this formula is also applicable to the case that the object is
emerged, suspended, or sunk.
This study was conducted among Chinese students. Based on the above discussion and careful review
of the course content of buoyant force theme, a conceptual framework was established as shown in Figure
1. The top component is the central idea, which expresses the nature of buoyant force. When solving buoy-
ant force related problems, experts often have formed a basic understanding that buoyant force is an effect
force, which together with other forces in the vertical direction determines the state of an object in liquid.
Then, they begin to analyze the forces acting upon the object according to its state. If the object is emerged
or suspended, they will calculate the buoyant force by applying Fb = mg according to the principle of two
force balance. If the object is sunk, they will calculate the buoyant force by applying Fb = mg-Fs according to
the principle of two force balance and the principle of force synthesis. If the object is emerging or sinking,
they will calculate the buoyant force by applying Fb = ρgv according to the Archimedes principle. What’s
more, in the process of state judgment and forces analysis, experts will always call to mind that Fb = ρgv is
applicable to any of the above situations to deal with the contexts where the gravity and the support force
acting on the object cannot be obtained.
722
(pp. 720-738)
Figure 1
Conceptual Framework of Buoyant Force
The middle layer includes different buoyant force calculation equations obtained after analysis of the states
and forces of the object in liquid. The reasoning process also shows that the central idea is closely tied to some
other scientific concepts and principles, such as two force balance principle, force synthesis principle and Archi-
medes principle. These correlative components are listed in “related concepts & principles” in Figure 1. The bottom
layer involves the contextual variables directly corresponding to the calculation equations of buoyant force, such
as mass, gravitational acceleration, density of liquid, volume of the displaced liquid, and Fs. As to great majority
problems related to buoyant force, the task goal is usually to calculate the buoyant force or needs to be achieved
by calculating buoyant force, as shown in the box on the right in Figure 1.
Different components of the conceptual framework are linked by numerous arrows. The solid arrows represent
the reasoning pathways of experts on solving buoyant force related problems. These all pass through the central
idea and link to other components, forming a well-integrated knowledge structure. Meanwhile, the dashed arrows
tend to ignore the central idea, and make weak connections between the different components. These indicate
that the novices’ knowledge structure is fragmented, mechanical memory and matching contextual variables with
equations are their habitual behaviors when solving problems. The arrows in Figure 1 are two-way arrows. This
means that a link can be originated from either side. It is also possible that some students have only constructed
one direction link between certain components. These students can be considered at certain stages between
novice level and expert level.
Modeling Student Understanding Using the Conceptual Framework
According to the representative pathways of the conceptual framework shown in Figure 1, this study divided
students’ understanding of buoyant force and their ability to solve related problems into three levels:
1) Novice level: This is the lowest level of understanding of buoyant force. Students at this level either have
low cognitive functions, often miss information, and activate wrong knowledge in solving problems
related to buoyant force, or at best have established direct relations between the buoyant force calcu-
lation equations and the contextual variables (the components of bottom layer). For instance, when
given density of liquid, gravitational acceleration, and volume of the displaced liquid, these students
723
(pp. 720-738)
at best can only use Fb = ρgv to calculate the buoyant force mechanically. But they will be baffled when
any of these three context variables is not provided in the problem.
2) Intermediate level: Students at this level can connect the contextual variables with the middle layer
components. They have more strategies and better performance than novices in calculating buoyant
force, especially in applying the relevant concepts and principles to obtain the unknown variables,
but they still don’t understand the essential meaning of buoyant force. As an example, students at this
level can judge that the object sunk at the bottom of a container is in equilibrium, then use Fb = mg-
Fs accordingly. When the variables involved in these equations are not directly given in the problem,
they can also obtain these required variables by analyzing relevant information and applying correct
reasoning pathways. But when the object sticks to the container, they still tend to use Fb = mg-Fs to
calculate the buoyant force. For the sake of establishing some more effective problem-solving meth-
ods, these students need to have a deeper understanding of the central idea to make their knowledge
structures more integrated.
3) Expert level: This is the highest level of understanding of buoyant force. Students at this level have
formed a well-integrated knowledge structure, which drives them to connect the contextual variables
with the central idea, as well as many of the related concepts, principles, and the intermediate reason-
ing pathways. When solving problems, these students have a greater chance of success, because they
can follow through the related connections all the way to the nature of buoyant force with any given
contextual variables, then get the effective problem-solving strategies according to some specific
reasoning pathways.
Under the guidance of the constructed conceptual framework and the discussions above, a single-choice
assessment was developed to probe the characteristics of students’ knowledge structures of buoyant force. Fur-
thermore, this study explored students’ thinking processes and reasoning pathways through follow-up interviews.
Design of the Buoyant Force Test
Based on the constructed conceptual framework, a test consists of 12 single-choice questions (there are 4
options for each question) was designed to assess students’ understanding of buoyant force. All the 12 questions
were adapted from the ones commonly used in instruction. For each question, tried to simplify the calculation
involved and avoid information irrelevant to the conceptual framework. The questions were designed into 3 sets
with varying degrees of knowledge connectedness and contextual familiarity, including single-link familiar ques-
tion set, multilink familiar question set, and integrated-link unfamiliar question set.
The single-link familiar set contains 3 questions (Q3, Q5, Q8) that the students often encounter in their daily
learning, which can be solved by directly using one of the following three equations: Fb = mg, Fb = mg-Fs, Fb = ρgv.
In these questions, the contextual variables such as m, g, Fs, ρ, v are given directly, students can directly match
them with the equations to reach the conclusions. It was expected that expert and intermediate level students
can successfully answer most of these questions, and so can novice students, unless they forget the questions or
make calculation errors.
The multilink familiar set includes 4 questions (Q1, Q4, Q10, Q11) that the students often encounter too, but
these questions all require multiple steps to correctly obtain the contextual variables by connecting the related
concepts & principles and the intermediate reasoning & processes in order to get the answers. For example, Q1 asks
the students to first analyze the gravity of an object, and then use the equation Fb = mg-Fs to get the result. It was
expected that in responding to these questions the students at expert level should be capable of answering all of
them correctly, the students at intermediate level should often be able to answer a large fraction of them correctly.
Meanwhile, the students at novice level can’t succeed in obtaining proper answers under normal circumstances.
The integrated-link unfamiliar set contains 5 questions (Q2, Q6, Q7, Q9, Q12). These questions were designed
to evaluate whether students can successfully apply the central idea in relatively unfamiliar problems. In other
words, to answer these questions correctly, students need to understand the essential meaning of buoyant force
and be capable of applying it in unfamiliar contexts successfully, which include contexts such as the object in outer
space, the object sticks to the bottom of container, the object be sunk into the mud, and so on. It was expected
724
(pp. 720-738)
that when answering these questions, students at different levels may perform differently. The students at novice
level almost need to guess, so they may not answer any questions correctly. The students at intermediate level
should also have difficulty with these questions but may succeed in some of them. Meanwhile, the students at
expert level should be able to solve most of these questions by applying the central idea.
The test questions and the predictive performance of students with different levels of knowledge integration
on these questions can be summarized as shown in Table 1.
Table 1
Assessment Questions Used in this Study
Knowledge integration Single-link familiar Multilink familiar Integrated-link unfamiliar
Q3,Q5,Q8 Q1,Q4,Q10,Q11 Q2,Q6,Q7,Q9,Q12
Novice level Occasionally successful Rarely successful Guessing
Intermediate level Mostly successful Often successful Sometimes successful
Expert level Always successful Always successful Mostly successful
Research Procedure
The participants of this study were 622 8th grade students taking an introductory level science course, who
were from a lower-secondary school in Jinhua City, Zhejiang Province, China. Jinhua is a moderately developed
city in China, and this lower-secondary school is at the medium quality level in Jinhua. The average age of these
students was 13.6 years old, including 333 girls and 289 boys. And 402 of these students were from 10 ordinary
classes and the remaining 220 students were from 6 better classes. The academic performance of students from
ordinary classes was different from that of students from better classes, because they were divided in different types
of classes according to their entrance grades. The difference of students’ academic performance was acceptable,
and even the main basis for the selection of participants in this study, which was more conducive to acquire the
ideal study results, that was, students would show different levels of understanding on buoyant force. All the 622
students took the test in December 2021 within 40 minutes. From September 2021 to October 2021, these students
had learned about buoyant force in their science courses. The teaching content and strategies of buoyant force
adopted by the science teachers of these students were similar (see Figure 8 in Appendix).
After the test, a variety of quantitative and qualitative methods were applied to explore and verify the students’
conceptual understanding of buoyant force. In order to determine the assessment structures related to the differ-
ent question sets, exploratory factor analysis (EFA) was carried out on students’ test data. Statistical significances
of comparisons between class level (the better class vs the ordinary class) and question set were determined with
two-way ANOVA and further explored through student t tests. The size of differences among different question sets
were measured with Cohen’s d effect sizes. Additionally, 48 of the 622 students who participated in the test were
randomly selected to continue to attend think-out-loud interviews. They were required to explicate the reason-
ing process when dealing with questions in detail, so that their knowledge structures could be interpreted more
deeply. Each interview lasted for about 30 minutes and was audiotaped.
Research Results
Quantitative Study on Students’ Knowledge Structure Development
Firstly, an exploratory factor analysis (EFA) was conducted on all students’ test data. It is a scree plot of the
EFA eigenvalues as shown in Figure 2. The results show that there are three factors with eigenvalues greater than
1, which explain 49% of the cumulative variance (i.e., 18%, 16%, and 15% for factors 1, 2 and 3, respectively). And
Figure 3 provides the factor loading of all the questions on these three factors.
725
(pp. 720-738)
Figure 2
Scree Plot of EFA Eigenvalues of All Students’ Test Data
The assessment test contains three categories that properly match the design of the three question sets, which
was confirmed in the EFA result. As shown in Figure 3, factor 1 represents single-link familiar question set, factor 2
represents multilink familiar question set, and factor 3 represents the integrated-link unfamiliar question set. The
results also show that students’ responses on the single-link familiar question set are moderately correlated to their
responses on multilink familiar question set (.429). However, the students’ proficiency in solving the single-link
familiar question set (.244) and the multilink familiar question set (.221) is less connected to the integrated-link
unfamiliar question set. It may represent a less integrated (or more fragmented) knowledge structure, especially
the lack of a deep understanding of the central idea.
Figure 3
Factor Loadings for EFA of All Students’ Test with Three-Factor Solution
726
(pp. 720-738)
Figure 4 shows the performance of students in better classes and ordinary classes across the different ques-
tion sets. And the details are listed in Table 2. Performance differences between class levels were compared using
two-way ANOVA (class level and question set). The main effects of class level (F(1, 1860) = 157.163, p < .001, η2p =
.078) and question set (F(1, 1860) = 533.980, p < .001, η2p = .365) were measured. The influence of the question set
is shown in Table 2 below. We can see that whether in the better class or in the ordinary class, students perform
best in the single-link familiar question set and worst in the integrated-link unfamiliar question set, which are
consistent with the expectations shown in Table 1. From the above analysis results, it can be seen that the question
design shown in Table 1 can distinguish the scores of students with a high level of statistical significance. Directly
comparing performances for each class level reveals students in better class with higher scores than students in
the ordinary class, both within each question set (t(S) (620) = 6.719, p< .001, d = .563), (t(M) (620) = 6.640, p< .001,
d = .557), (t(I) (620) = 8.431, p < .001, d = .707) and overall (t(218) = 10.378, p < .001, d = .870). The differences can
be predicted because students in different classes have different levels of academic performance. However, the
relative performance of the two classes at different levels on the different question sets is quite similar (two almost
parallel trend lines as shown in Figure 4). The results of analysis of ANOVA also confirmed this, indicating that there
is no significant interaction between class level and question set (F(2, 1860) = 2.636, p < .072, η2p = .003).
Figure 4
Students’ Performance across Different Question Sets
Table 2
Summary of Students’ Performance in Different Question Sets at Different Levels of Classes, and Comparison of T-Test Results
of Two Groups of Students
Better class Ordinary class

Question set
M SE M SE t d
Single-link familiar .982 .005 .857 .014 6.719*** .563

Multilink familiar .860 .017 .692 .016 6.640*** .557
Integrated-link unfamiliar .511 .021 .313 .013 8.431*** .707
Total .745 .012 .575 .010 10.378* .870
Notes: *p < .05, ***p < .001; SE: the standard deviation of a sampling distribution.
In order to research the different responses of students with different overall performance levels in the three
question sets, the students were assigned to five performance groups, each containing 20% of the total sample. The
727
(pp. 720-738)
performance level was determined by the students’ scores in the test, which reflect their overall learning results of
buoyant force. As shown in Figure 5, the scores on integrated-link unfamiliar question set are consistently below
the scores on multilink familiar question set and single-link familiar question set for all groups.
The students with low total scores (<20%) can successfully answer most of these questions on single-link famil-
iar question set but fail to obtain correct answers on multilink familiar question set and integrated-link unfamiliar
question set. This is not different from the expected performances of novice students listed in Table 1. With the
increase of the total score, performance gaps between the integrated-link unfamiliar question set and the other
two question sets are more pronounced, but the performance gap between single-link familiar question set and
multilink familiar question set is closing, indicating that students in this range have begun to perform well on simple
and more complex typical questions using memorization but without establishing a deep understanding. With the
further improvement of the total score, performance gaps between the different question sets are less pronounced,
indicating that students have developed partially integrated knowledge structures that allow them to achieve
occasional success in some unfamiliar situations. At last, students with the highest scores (>80%) have formed a
well-integrated knowledge structure, which drives them perform well in three question sets. These results reveal
a general progression of student knowledge integration, which is very consistent with the discussion in Table 1.
Figure 5
Students’ Scores in Different Question Sets. The Error Bars Represent Standard Error
The knowledge structures of expert-level students were further explored using exploratory factor analysis
(EFA), constructed the correlation matrix of all project scores, and calculated the eigenvalues. As shown in Figure
6, it is a scree plot of the EFA eigenvalues. The results show that the first eigenvalue can explain nearly 64% the
variance. This indicates that students’ performance can be fully explained by a single salient factor, which means a
quite integrated knowledge structure (See Figure 7 for the loading of all the questions on the single factor). When
compared to the knowledge structure of the all students, which has three distinctive factors, this merging effect
further indicates that these expert-level students have formed a well-integrated knowledge structure. Considering
the meaning of analyzing these students’ data is controversial because they got most of the questions correct, the
characteristics of their knowledge structure need to be further interpreted in combination with the subsequent
qualitative analysis.
728
(pp. 720-738)
Figure 6
Scree Plot of EFA Eigenvalues of High-Level Students’ Test Data
Figure 7
Factor Loadings for EFA of High-Level Students’ Test with one Factor
Summarizing these results and combining with the discussion in Table 1, students at the three levels of
knowledge integration perform differently on questions with differing degrees of knowledge connectedness and
contextual familiarity. Since the average total score of the test can be used as a serviceable indicator of the different
knowledge integration levels, it is proposed to divide the scores of 0% - 20%, 20% - 80% and 80% - 100% as the
novice-intermediate-expert levels of knowledge integration, which is summarized in Table 3.
729
(pp. 720-738)
Table 3
Summary of Total Score and Question Set Score of Each Knowledge Integration Level Based on Test Data. Standard Errors are
Given in Parentheses.
Knowledge integration level N Single-link typical Multilink typical Integrated-link atypical
Novice 125 .616(.017) .286(.017) .181(.017)
Intermediate 372 .964(.010) .834(.010) .305(.010)
Expert 125 .997(.017) .970(.017) .819(.017)

Notes: standard error is the standard deviation of a sampling distribution.
The assessment results have reflected that the intermediate level of knowledge integration is the state that
most students were in, where they have more strategies and better performance than novices in calculating buoyant
force, but they still rarely succeed in the integrated-link unfamiliar questions. This clearly shows that most students
lack a well-integrated knowledge structure and understanding of the nature of buoyant force. In response to this
issue, it is critical to helping students construct conceptual frameworks for particular concepts to promote their
knowledge integration level and improve their problem-solving ability.
Qualitative Study on Students’ Reasoning Process
48 of the 622 students were randomly selected to participate in think-out-loud interviews to ascertain the
possible reasoning pathways and knowledge structures. According to students’ responses, three different levels
of students’ understanding were determined and described below.
Novice level: These students can only link the contextual variables directly to related equations in their memory
without meaningful reasoning processes, and often lose information or activate not proper knowledge. It can be
seen that the knowledge structure of these students is largely fragmented, which is not only not connected with
the essential meaning of buoyant force, but also with the relevant concepts and principles. Therefore, novices are
able to correctly solve a limited number of simple familiar questions using memorized equations. The following
are excerpts from interviews with students who exhibit this level of problem-solving behavior:
Student A: (Response to question 1) “In this question, the object is sunk at the bottom of the container, well, the equation
seems to be Fb = mg-Fs, right? I remember there should be this equation. But the gravity of the object is unknown
here, so I directly consider the buoyant force here is equal to Fs.” (Response to question 2) “I really didn’t know
how to solve the question. This is the case in outer space, the question gives both the mass of the object and the
volume of the displaced liquid, I didn’t know whether to use the equation Fb=mg or Fb =ρgv. Finally, I randomly
chose Fb =ρgv.”
Student B: (Response to question 12) “I’ve never seen such a question. The object is stuck to the bottom of the container, is
this the same as when it is sunk at the bottom of the container? Should I use the equation Fb=mg-Fs, right? But
the variables in the question do not correspond to this equation, So I just guess an answer.”
Both student A and student B answered most of the questions in single-link familiar set, but they did badly
in multilink familiar set and integrated-link unfamiliar set. As student A mentioned, the context that the object is
sunk at the bottom of the container prompted him to recall the equation Fb = mg-Fs. However, when he found that
the gravity of the object was not given in the question, he created a new equation Fb = Fs to calculate the buoy-
ant force incorrectly. This thinking process shows that the student only remembered some equations of buoyant
force. The same is true for student A in question 2. He only thought of the corresponding equations Fb = mg and
Fb = ρgv according to the mass and the volume given in the question, but the correct solution to this question
should consider that the liquid will not produce buoyant force on the object, because the liquid is not affected by
gravity in outer space.
Meanwhile, student B recalled the equation Fb = mg-Fs cued by the context that the object is stuck to the
bottom of container. Then, the student attempted to seek out all the required variables in the question to solve
for the answer. Unfortunately, the variables given in this question were not gravity and support force, then, the
730
(pp. 720-738)
student didn’t know how to solve it and can only guess one answer. In fact, students need to apply the central idea,
that is, the essential meaning of buoyant force, and then get answer by calculating the force difference between
the upper and lower surfaces of the object in this question.
The performances the two students demonstrated here support the viewpoint that novices’ knowledge
structure is decentralized with simple connections between contextual variables and related equations, leading
them to rely on pattern matching as the main strategy to solve problems.
Intermediate level: These students can make sufficient connections between contextual variables and related
concepts and principles but have great difficulties in understanding and applying the central idea. They seem to
predominantly pay attention to how to get the variables involved in the equations by connecting related concepts
and principles, and then they can use the equations to calculate the buoyant force. But when the solution of the
problem requires students to understand the nature of buoyant force, they often feel confused. For instance, student
C successfully solved all the questions in multilink familiar set but had trouble with the questions in integrated-link
unfamiliar set.
Student C: (Response to question 6) “I can’t make sure the answer of this question. The object is sunk into the mud at the bot-
tom of the container, I think I should use the equation Fb = mg-Fs, I remembered that Fs is always upward, but it
seems to be downward here. If I still use this equation to calculate the buoyant force, the direction of the buoyant
force seems to be downward, but we all know that the direction of buoyant force should be upward.” (Response
to question 2) “This question should be solved by equation Fb =mg or Fb =ρgv. But I was shocked to find that the
answers calculated by these two equations were different. Is the buoyant force of objects in outer space different
from that on earth?”
Generally speaking, the intermediate level students appear to have more developed understanding than the
novices, they are often to perform effective reasoning by applying the related concepts and principles to obtain
the unknown variables. However, their knowledge structures are still not integrated around the central idea and
their problem-solving strategies are still relied on applying equations. The following student’s responses further
confirmed this finding:
Student D: (Response to question 9) “When the object is stationary, Fb=mg-Fs, so options A and B are wrong. When the object
is emerging, Fb ≠mg, so option C is wrong. The answer is D.” (The correct answer is A.)
Students similar to student C and student D have a good understanding of the connection between buoyant
force and the related concepts and principles, so they can often analyze the states and forces of the object, and
obtain the variables not directly given in the question through certain transformation methods. As stated in the
conceptual framework of buoyant force, these students’ knowledge structures need to be more integrated with
the central idea to develop some more valid problem-solving strategies.
Expert level: These students are able to solve most of all 12 questions using multiple strategies. For the ques-
tions in integrated-link unfamiliar set in particular, they seem to be able to apply it correctly to get the keys to
answer the questions on the basis of a deep understanding of the central idea.
Student E: (Response to question 2) “The origin of buoyant force is that the liquid is affected by gravity, this is A. In outer space,
the liquid is not affected by gravity, so the buoyant force is zero.”
Student F: (Response to question 2) “In this question, the buoyant force is obviously zero. Because the g=0 holds forever in
outer space, it doesn’t matter whether the equation Fb=mg or Fb =ρgv is used, the final answer is zero. So, the
answer is A.” (Response to question 7) “The buoyant force is the net force on the upper and lower surfaces of an
object in a liquid, and the object is in equilibrium in this question, so I can get the answer as long as I analyze the
forces on the object in the vertical direction. After calculation, the answer should be D.”
As shown in these excerpts, the problem-solving performances of these students are like that of experts.
They often started by understanding the essential meaning of buoyant force, and then chose the appropriate ap-
proaches. They formed such a basic understanding that buoyant force is an effect force caused by the gravity of
the liquid. Through this basic understanding, they can integrate contextual variables, calculation equations, and
related concepts and principles to solve the problems successfully.
731
(pp. 720-738)
The number at each level of the 48 interviewed students included 19 novices, 20 intermediate, and 9 experts.
Their average performance on different question sets is shown in Table 4. Considering the small sample sizes, such
results are only used as a supplementary reference.
Table 4
The Average Performance of Interviewed Students at Different Levels
Total Single-link typical Multilink typical Integrated-link atypical

Knowledge integration level
(%) (%) (%) (%)
Novice 38.16 77.19 40.79 12.63
Intermediate 62.08 88.33 76.25 35.00
Expert 99.07 100.00 100.00 97.78
Putting these interview outcomes together, it is quite evident that the novice level students’ knowledge struc-
tures were fragmented. Therefore, they can only apply memorized equations to solve the simple questions in familiar
contexts. The students at intermediate level had deeper understanding about buoyant force and could solve most
multilink familiar questions. Nevertheless, the knowledge structures of these students were still lack of connection
with the central idea. As a result, it was difficult for them to solve problems in complex and unfamiliar contexts.
The expert level students had established well-integrated knowledge structures so that they could obtain multiple
problem-solving strategies by using the central idea. Therefore, they were able to solve most problems effectively.
Discussion
In this study, a conceptual framework of buoyant force was established according to the theory of knowledge
integration to guide the modeling and assessment of students’ knowledge structures in understanding buoyant
force. The results show that this approach can effectively identify and present students’ understanding of buoyant
force into three different levels of knowledge integration, including novice, intermediate, and expert.
For the novice level students, since they only rely on scattered mnemonic pattern matching to solve related
problems, establishing the awareness of concept understanding is the first thing they need to strive to do when
learning a related topic. Specifically, they should give up the learning tendency of directly memorizing calculation
equations and corresponding variables and try to construct the connection between relevant knowledge elements
by looking for certain logical relationships and reasoning pathways. Thus, they can solve problems analytically
through meaningful reasoning and explanation, rather than through inefficient and error prone mechanical memory.
For the intermediate level students, they seem to be locally connected among the calculation equations of
buoyant force and related concepts & principles, but they still have the tendency of pattern matching in the process
of solving problems, even they can successfully choose the proper calculation equations and get the required vari-
ables. In other words, they may just be more familiar with calculation equations and remember some corresponding
methods to promote the successful application of these equations than novices, but they do not understand the
root and essence of the connections between the relevant knowledge elements they have established. Therefore,
they need to further understand the central idea to find the nature of the connections between the knowledge
elements. Thus, they can effectively solve the problems that need to be solved by applying the central idea rather
than simply pattern matching.
For the expert level students, they seem to have developed a deep understanding of the central idea, es-
tablished effective connections among most key contextual variables and reasoning pathways, and formed a
comprehensive knowledge structure. This means that they have found the right way to learn scientific concepts.
What they need to do is to explore more logical relationships and reasoning pathways to link more knowledge
elements, so as to further expand the formed conceptual framework. As a result, they may solve more complex
and difficult related problems.
In this study, the different problem-solving behaviors of the three levels students reveal a process from surface
to deep understanding of buoyant force, from memorizing equations to building more networked relationships,
and finally to the fully integrated knowledge structure. Through this development process, students show that they
732
(pp. 720-738)
reduced context dependency, used the central idea more explicitly, and improved accuracy on problem-solving
tasks. As is shown by this study, understanding the nature of buoyant force seems to be the key to make this process
happen, as it exactly reflects the root causes of various buoyant force related phenomena. Unfortunately, a few
Chinese students have understood the essential meaning of buoyant force and formed an integrated knowledge
structure, so as to achieve better results in the assessment questions. This shows that the emphasis on understand-
ing the nature of buoyant force and establishing a conceptual framework accordingly may be an effective strategy
to help students form an integrated knowledge structure, leading to better problem-solving ability.
Conclusions
Overall, this study shows that the method of constructing the conceptual framework is effective in modeling
and analyzing students’ understanding in buoyant force learning. The results also show that it is necessary to em-
phasize the essential meaning of buoyant force to help students develop more integrated knowledge structure and
better problem-solving performance. In addition, this study also has some limitations that need to be duly noted.
On the one hand, buoyant force is a very complex concept, which is related to many other knowledge elements,
so more diversified conceptual frameworks need to be constructed and verified in order to investigate students’
understanding of buoyant force more comprehensively. On the other hand, this study was carried out in the Chi-
nese context, whether the constructed conceptual framework of buoyant force, the designed test questions, and
the study results are applicable to a wider range need to be further tested.
References
Alonso, & Marcelo. (1992). Problem solving vs. conceptual understanding. American Journal of Physics, 60(9), 777-778.
https://doi.org/10.1119/1.17056
Bao, L., & Fritchman, J. C. (2021). Knowledge integration in student learning of Newton’s third law: Addressing the action-
reaction language and the implied causality. Physical Review Physics Education Research, 17(2), Article 020116. https://doi.
org/10.1103/PhysRevPhysEducRes.17.020116
Besson, U. (2004). Students’ conceptions of fluids. International Journal of Science Education, 26(14), 1683-1714.
https://doi.org/10.1080/0950069042000243745
Bloom, B. S., Engelhart, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D. R. (1956). Taxonomy of educational objetives: The classification
of educational goals: handbook I: cognitive domain (No. 373.19 C734t). New York, US.
Champagne, A. B., Gunstone, R. F., & Klopfer, L. E. (1981). A perspective on the differences between expert and novice performance
in solving physics problems. Research in Science Education, 12(1), 71-77. https://doi.org/10.1007/BF02357016
Chi, M. T., Feltovich, P. J., & Glaser, R. (1981). Categorization and representation of physics problems by experts and novices.
Cognitive Science, 5(2), 121-152. https://doi.org/10.1207/s15516709cog0502_2
Chiu, M.-H., Guo, C.-J., & Treagust, D. F. (2007). Assessing students’conceptual understanding in science: An introduction about a national
project in Taiwan. International Journal of Science Education, 29(4), 379–390. https://doi.org/10.1080/09500690601072774
Dai, R., Fritchman, J. C., Liu, Q., Xiao, Y., & Bao, L. (2019). Assessment of student understanding on light interference. Physical Review
Physics Education Research, 15(2), Article 020134. https://doi.org/10.1103/PhysRevPhysEducRes.15.020134
Demkanin, P. (2020). The ways the theory of science education can evolve. Journal of Baltic Science Education, 19(6), 860-863.
http://oaji.net/articles/2020/987-1606572136.pdf
Demkanin, P., Novotná, S., & Sukeľová. T. (2022). Strategies and challenges of physics curriculum-refraction of light as an example
of brain-friendly curriculum design. A paper presented in the 16th International Technology, Education and Development
Conference (March7-8) (pp. 1282-1289). https://doi.org/10.21125/inted.2022.0386
Duit, R., & Treagust, D. F. (2003). Conceptual change: A powerful framework for improving science teaching and learning.
International Journal of Science Education, 25(6), 671-688. https://doi.org/10.1080/09500690305016
Eylon, B. S., & Reif, F. (1984). Effects of knowledge organization task performance. Cognition and Instruction, 1(1), 5-44.
https://doi.org/10.1207/s1532690xci0101_2
Gao, Y., Zhai, X., Andersson, B., Zeng, P., & Xin, T. (2020). Developing a learning progression of buoyancy to model
conceptual change: A latent class and rule space model analysis. Research in Science Education, 50(4), 1369–1388.
https://doi.org/10.1007/s11165-018-9736-5
Gerace, W. J., Dufresne, R. J., Leonard, W. J., & Mestre, J. P. (2001). Problem solving and conceptual understanding. A paper presented
in the 2001 Physics Education Research Conference (p. 33), New York.
Gilbert, & John, K. (2012). A coherent approach to the development of science education science learning and instruction:
733
(pp. 720-738)
Taking advantage of technology to promote knowledge integration. Studies in Science Education, 48(2), 217-221.
https://doi.org/10.1080/03057267.2012.720769
Halford, G. S., Brown, C. A., & Thompson, R. M. (1986). Children’s concepts of volume and flotation. Developmental Psychology,
22(2), 218-222. https://doi.org/10.1037/0012-1649.22.2.218
Kim, E., & Pak, S. J. (2002). Students do not overcome conceptual difficulties after solving 1000 traditional problems. American
Journal of Physics, 70(7), 759-765. https://doi.org/10.1119/1.1484151
Kim, S., & Paik, S. H. (2021). Archimedes’ balance approach applied to buoyant force. The Physics Teacher, 59(2), 125-127.
https://doi.org/10.1119/10.0003469
Kim, Y., & Kim, J. (2012). Analysis of the middle school students’ conceptions about buoyancy. Journal of Science Education, 36(2),
369-380. https://doi.org/10.21796/JSE.2012.36.2.369
Kubsch, M., Nordine, J., Neumann, K., Fortus, D., & Krajcik, J. (2018). Measuring integrated knowledge–a network analytical
approach. A paper presented in the 13th International Conference of the Learning Sciences (ICLS), London, UK.
https://repository.isls.org/handle/1/627
Libarkin, C. J., Crockett, C. D., & Sadler, P. M. (2003). Density on dry land: Demonstrations without buoyancy challenge student
misconceptions. The Science Teacher, 70(6), 46-50. https://doi.org/10.2307/24156089
Lima, F. M. S., & Monteiro, F. F. (2013). Buoyant force in a nonuniform gravitational field. Revista Brasileira de Ensino de Física, 35(3),
1-4. https://doi.org/10.1590/S1806-11172013000300030
Linn, M. C. (2000). Designing the knowledge integration environment. International Journal of Science Education, 22(8), 781-796.
https://doi.org/10.1080/095006900412275
Loverude, M. E., Kautz, C. H., & Paula, P. R. L. (2003). Helping students develop an understanding of Archimedes’ principle. I.
Research on student understanding. American Journal of Physics, 71(11), 1178-1187. https://doi.org/10.1119/1.1607335
Minogue, J., & Borland, D. (2016). Investigating students’ ideas about buoyancy and the influence of haptic feedback. Journal of
Science Education and Technology, 25(2), 187-202. https://doi.org/10.1007/s10956-015-9585-1
Mullet, E. (1988). Archimedes’ effect, information integration and individual differences. International Journal of Science Education,
10(3), 285-301. https://doi.org/10.1080/0950069880100305
Murcia, K. (2009). Re-thinking the development of scientific literacy through a rope metaphor. Research in Science Education,
39(2), 215-229. https://doi.org/10.1007/s11165-008-9081-1
National Research Council. (2008). Research on future skill demands: A workshop summary. Washington (DC): National Academies
Press. https://pubmed.ncbi.nlm.nih.gov/20669417/
Nie, Y., Xiao, Y., Fritchman, J. C., Liu, Q., Han, J., Xiong, J., & Bao, L. (2019). Teaching towards knowledge integration in learning force
and motion. International Journal of Science Education, 41(16), 2271-2295. https://doi.org/10.1080/09500693.2019.1672905
Özdem. (2010). An investigation of elementary students’ scientific literacy levels. Journal of Baltic Science Education, 9(1), 6-19.
http://www.scientiasocialis.lt/jbse/?q=node/183
Paik, S., Song, G., Kim, S., & Ha, M. (2017). Developing a four-level learning progression and assessment for the concept of buoyancy.
Eurasia Journal of Mathematics Science & Technology Education, 13(8), 4965-4986. https://doi.org/10.12973/eurasia.2017.00976a
Radovanovic, J. (2012). Approximate value of buoyant force: A water-filled balloon demonstration. The Physics Teacher, 50(7),
428-429. https://doi.org/10.1119/1.4752051
Radovanović, J., & Sliško, J. (2013). Applying a predict–observe–explain sequence in teaching of buoyant force. Physics Education,
48(1), 28-34. https://doi.org/10.1088/0031-9120/48/1/28
Rivet, A. E., & Krajcik, J. S. (2010). Contextualizing instruction: Leveraging students’ prior knowledge and experiences
to foster understanding of middle school science. Journal of Research in Science Teaching, 45(1), 79-100.
https://doi.org/10.1002/tea.20203
Scalise, K. (2012). Science learning and instruction: Taking advantage of technology to promote knowledge integration. Science
Education, 96(6), 1136-1138. https://doi.org/10.1002/sce.21025
Schoenfeld, A. H., & Herrmann, D. J. (1982). Problem perception and knowledge structure in expert and novice mathematical
problem solvers. Journal of Experimental Psychology Learning Memory & Cognition, 8(5), 484-494. https://doi.
org/10.1037/0278-7393.8.5.484
Smith, C., Carey, S., & Wiser, M. (1985). On differentiation: A case study of the development of the concepts of size, weight, and
density. Cognition, 21(3), 177-237. https://doi.org/10.1016/0010-0277(85)90025-3
Snyder, J. L. (2000). An investigation of the knowledge structures of experts, intermediates, and novices in physics. International
Journal of Science Education, 22(9), 979-992. https://doi.org/10.1080/095006900416866
Suat, U. (2008). Changing students’ misconceptions of floating and sinking using hands-on activities. Journal of Baltic Science
Education, 7(3), 134-146. http://www.scientiasocialis.lt/jbse/?q=node/156
Tomo. D. (2021). Promoting students’ conceptual change by integrating the 3-2-1 reading technique with refutation text in the
physics learning of buoyancy. Journal of Turkish Science Education, 18(2), 290-303. https://doi.org/10.36681/tused.2021.66
Wagner, D. J., Carbone, E., & Lindow, A. (2013). Exploring student difficulties with buoyancy. A paper presented in the Physics
Education Research Conference 2013 (July 17-18), Portland, OR.
https://www.compadre.org/Repository/document/ServeFile.cfm?ID=13152&DocID=3699
Xie, L., Liu, Q., Lu, H., Wang, Q., Han, J., Feng, X., & Bao, L. (2021). Student knowledge integration in learning mechanical
wave propagation. Physical Review Physics Education Research, 17(2), Article 020122. https://doi.org/10.1103/
PhysRevPhysEducRes.17.020122
734
(pp. 720-738)
Xu, W., Liu, Q., Koenig, K., Fritchman, J., Han, J., Pan, S., & Bao, L. (2020). Assessment of knowledge integration in student
learning of momentum. Physical Review Physics Education Research, 16(1), Article 010130. https://doi.org/10.1103/
PhysRevPhysEducRes.16.010130
Yin, Y., Tomita, M. K., & Shavelson, R. J. (2014). Using formal embedded formative assessments aligned with a short-term learning
progression to promote conceptual change and achievement in science. International Journal of Science Education, 36(4),
531-552. https://doi.org/10.1080/09500693.2013.787556
Young, D. E., & Meredith, D. C. (2017). Using the resources framework to design, assess, and refine interventions on pressure in fluids.
Physical Review Physics Education Research, 13(1), Article 010125. https://doi.org/10.1103/PhysRevPhysEducRes.13.010125
Appendix
Funding
This study was supported by the National Educational Science Planning Project of China (No. BHA210121),
the Zhejiang Educational Science Planning Project (No.2021SCG307), and the Open Research Fund of College of
Teacher Education, Zhejiang Normal University (No. jykf22013).
The Process of Instruction on Buoyant Force in Chinese lower-secondary schools
The instruction of buoyant force in Chinese lower-secondary schools is delivered in four lectures, each lasting
45 minutes (see Figure 8). In the first lesson, the teacher guides the students to understand the definition of buoy-
ant force and know that every object immersed in a liquid receives an upward buoyant force. It should be noted
that the teacher usually pays more attention to introducing the definition of buoyant force from the perspective
of mathematical relationship, but only very briefly mentions the nature of buoyant force without any explanation
and verification.
In the second lesson, the teacher introduces the Archimedes principle to the students, and guides the stu-
dents to verify through experiments that the buoyant force is equal to the gravity of the displaced liquid. Then,
the students will get the equation Fb = ρgv.
In the third lesson, under the guidance of the teacher, students analyze the forces of the object when it is
emerged, suspended, or sunk in the liquid. Then, the students will get the two other equations, Fb = mg, and Fb =
mg-Fs.
In the fourth lesson, the teacher guides students to solve various problems in different contexts to help them
get familiar with the relationship between the contextual variables and the equations.
Figure 8
The Process of Instruction on Buoyant Force in Chinese Lower-Secondary Schools
In general, there is no obvious logical clue in the instruction of buoyant force, most teachers usually spend
less time on explaining the essential meaning of buoyant force but focus on the training of applying equations to
solve problems in different contexts, so that students can be familiar with some fixed problem-solving patterns.
735
(pp. 720-738)
The Assessment Questions Used in This Study
(Tips: if gravity acceleration and water density are involved in the question, g is taken as 10N / kg, and water
density is taken as 1.0 × 103 kg/m3; and in all cases that involving the object is sunk, unless otherwise specified, it is
considered that the space between the bottom of the object and the bottom of the container is filled with liquid.)
1.When an object is placed on the desktop and remains stationary, the supporting force of the desktop to it is
40N. When the object is put into the water, it is sunk and receives 30N supporting force from the bottom of the
container. Please calculate the buoyant force of this object in the water. (B)
A.5 N B.10 N C.30 N D.40 N
2. As shown in the figure, in outer space, if we put a small ball with volume 3 ×10-6m3 and mass 2 g into a cup with
water, it would be suspended. What is the buoyant force of this small ball in the water at this time? (A)
A.0 N B.0.01 N C.0.02 N D.0.03 N
3. If we gently put an object into a cup full of water, 80 ml of water would overflow from the cup.
Please calculate the buoyant force of this object in the water. (A)
A.0.8 N B.1.8 N C.2.8 N D.3.8 N
4.As shown in the figure, there is an appropriate amount of water in the container. If we place the wooden block
A with a gravity of 5N in the container and press the wooden block B with a gravity of 3N on wooden block A,
wooden block A would be emerged on the water with wooden block B. What is the buoyant force of wooden block
A in the water at this time? (A)
A.3 N B.5 N C.8 N D.10 N
5. James puts a small ball with a mass of 0.04kg into a container with water. When the small ball is stationary, the
state of it in the water is emerged. Please calculate the buoyant force of this ball in the water. (D)
A.0.1 N B.0.2 N C.0.3 N D.0.4 N
736
(pp. 720-738)
6. As shown in the figure, the object with a volume of 600cm3 and a gravity of 3N is sunk into the mud at the bot-
tom of the container. It is known that the volume of the object trapped in the mud is 200cm3, the total embedding
force of the mud on it is 7 N, and the force exerted by the water pressures on the upper surface of the object is 5N.
If we want to salvage the object out of the mud, how much force is needed at least? (D)
A.9 N B.10 N C.11 N D.15 N
7. As shown in the figure, the hemispherical object with a volume of 700 cm3 and a gravity of 4N is stationary
at the bottom of a container with water. If we know that the force exerted by the water pressures on the upper
surface of the object is 5N, what is the force exerted by the water pressures on the lower surface (spherical part)
of the object? (D)
A.2 N B.4 N C.6 N D.8 N
8. The gravity of the object is 7N. When we put the object into the water, it would sunk and receive 6N supporting
force from the bottom of the container. Please calculate the buoyant force of this object in the water. (A)
A.1 N B.6 N C.7 N D.13 N
9. As shown in the figure, put the mouth of a lidless and bottomless beverage bottle facing downward, put the
table tennis ball (the diameter is slightly larger than the diameter of the bottle mouth) into the bottle and inject
water. You can see a small amount of water flowing out of the bottle mouth. At this time, the table tennis ball is
stationary still, and then block the bottle mouth with your hand. After a while, the table tennis ball will float up.
Which of the following analysis is correct? (A)
A. the direction of the net force of liquid on the table tennis ball is vertical
downward when it is stationary.
B. The supporting force of the table tennis ball in the figure
is balanced with the gravity when it is stationary.
C, In the process of table tennis emerging, the buoyant force
it receives is equal to the gravity it receives.
D. In the process of table tennis emerging, the buoyant force
it receives remains unchanged.
10. As shown in the figure, the object is composed of a balloon and a stone (the stone is hung with the string under
the balloon), and the object is suspended in the water. The mass of the stone is 0.4kg, the mass of the balloon is
0.1kg (including the air mass inside the balloon) and the mass of the string is 0.05kg. Please calculate the buoyant
force of this object in the water. (C)
A.5.5 N B.5 N C.4 N D.1 N
737
(pp. 720-738)
11. As shown in the figure, James puts the cuboid block with a total volume of 180 cm3 into a container with water.
When the block is stationary, the state of it in the water is emerged, and its volume above the water surface is
80cm3. What is the buoyant force of the cuboid block in the water? (B)
A.0.8 N B.1 N C.1.8 N D.2 N
12. As shown in the figure, the cuboid block with a volume of 600cm3 and a gravity of 3N is stuck to the bottom of
the container, and there is no gap between block bottom and the bottom of the pool. It is known that the viscous
force on the object is 5N and the downward pressure on the upper surface of the object is 4N. If we want to salvage
the object from the water, how much force is required at least? (B)
A.18 N B.12 N C.6 N D.3 N
Cite as: Zou, Y., Jin, L., Li, Y., & Hu, T. (2022). Assessment of knowledge integration in student learning of buoyant force. Journal
of Baltic Science Education, 21(4), 720-738. https://doi.org/10.33225/jbse/22.21.720
Yi Zou PhD, Lecturer, College of Teacher Education, Zhejiang Normal University,

(Corresponding author) Jinhua, China.
E-mail: zouyi88@zjnu.edu.cn
Lizhen Jin MS Student, College of Teacher Education, Zhejiang Normal University,
Jinhua, China.
E-mail: jinlizhen99@zjnu.edu.cn
Yanbing Li PhD, Professor, College of Teacher Education, East China Normal
University, Shanghai, China.
E-mail: ybli@kcx.ecnu.edu.cn
Tao Hu PhD Student, College of Teacher Education, East China Normal University,
Shanghai, China.
E-mail: hutao6277@163.com
738
INFORMATION
FOR CONTRIBUTORS
EDITORIAL AND REVIEW PROCEDURES
Journal of Baltic Science Education (JBSE) publishes original scientific research articles in the field of Natural Science Education and
related areas for all educational levels in the Baltic and other countries. It is possible to publish special (thematic) issues of JBSE. The
papers should be submitted and will be published in English. JBSE will promote to establish contacts between researchers and practical
educators both in the Baltic countries and countries around.
The authors of the manuscripts are responsible for the scientific content and novelty of the research materials. Articles, published
before in other international journals or papers’ collections will not be accepted for publication in JBSE.
As a publication that represents a variety of cross-disciplinary interests, both theoretical and practical, the JBSE invites manuscripts on
a wide range of topics, especially in the following areas:
• Didactics of natural sciences. • Philosophical, political, economical and social aspects
• Theory and practice in natural science teacher of natural science education.
education. • The supplementary natural science education.
• Integrated natural science education. • ICT in natural science education.
• Natural science and technological literacy. • The standardisation of natural science education etc.
• General and professional natural science education.
MANUSCRIPTS GUIDELINES
The structure of the research paper presented to the Journal of Baltic Science Education should be as follows: abstract - short report of the
investigation; introduction inc. aim and subject of the research; research methodologies and methods; results of the research incl. discussion;
conclusions; list of references in APA style (7th Ed.).
The papers should be submitted in English. If English is a second language for the author, please consider having the manuscript
proof read and edited before submitting.
The text must be elaborated in Word for Windows, using 12 point Times New Roman letters. An article should not exceed 7-10 A4 pages,
included figures, tables and bibliography. Publishing of longer articles should be negotiated separately. Texts margins: top and bottom 20mm,
left - 25mm, right - 20mm. The title: capital letters, 14pt, bold; space between the title and the author’s name is one line interval. Author’s name
and surname: small letters, 12pt, bold. Under the name, institution: 11 pt, italics; space between the title and the text: 1 line interval. Abstract
– about 100-150 words - precedes the text. The text: 12pt Single or Auto spacing, in one column. Key words: no more than five words. The
language must be clear and accurate. The authors have to present the results, propositions and conclusions in a form that can suit scientists
from different countries.
Titles of the tables and figures: 11 pt, small letters. Space between figures or tables and the text: 1 line interval. Introduction, titles of
chapters and subchapters: 12pt, bold, small letters. Numbers: Arabic, subchapters numbered by two figures (1.1, 1.2, etc.). Figures, tables and
captions should be inserted within the manuscript at their appropriate locations. Diagrams and graphs should be provided as finished black
and white line artwork or electronic images. When there are a number of illustrations, the author should endeavour to reduce the amount of
text to accommodate the illustrations in the limited space available for any article.
References in the text should be presented in brackets (Knox, 1988; Martin, 1995). If necessary, the page can be indicated: (Martin,
1995, p.48). The list of references should be presented after the text. The Words List of References: 11pt, bold, small letters. The references
should be listed in full at the end of the paper in the following standard form:
For books: Saxe, G.B. (1991). Cultural and Cognitive Development: Studies in Mathematical Understanding. Hillsdale, NJ: Erlbaum.
For articles: Bekerian, D.A. (1993). In Search of the Typical Eyewitness. American Psychologist, 48, 574-576.
For chapters within books: Bjork, R.A. (1989). Retrieval Inhibition as an Adaptive Mechanism in Human Memory. In: H.L. Roediger III & F.I.M.
Craik (Eds.), Varieties of Memory & Consciousness (pp. 309-330). Hillsdale, NJ: Erlbaum.
On a separate page, author - related data should be presented in English: name, surname, degree and academic title, institution, full
correspondence address in the clearest and most complete form /ordinary post and e-mail addresses /, position (to ensure anonymity in the
review process). The author (authors) should confirm in writing, that the manuscript has not been published in other journal or
handed over (transferred) to other journal for publication.
EDITORIAL AND REVIEW PROCEDURES

Manuscripts will be sent anonymously to reviewers with expertise in the appropriate area. All manuscripts will be rewieved by two
experts before JBSE’s accept them for publication. This process usually takes about two months. The journal co-editors will make minor
editorial changes; major changes will be made by the author(s) prior to publication if necessary. JBSE’s redaction will sent to author(s)
only one correcture which must be sent back within 2 weeks. JBSE will not review submissions previously published elsewhere through
print or electronic medium.
Manuscripts submitted to the JBSE cannot be returned to authors. Authors should be sure to keep a copy for themselves. Authors’
signatures should be at the end of the paper and its second checked proofs.
Manuscripts, editorial correspondence (and other correspondence for subscription and exchange), and any questions should be
sent to editor-in-chief or to regional redactors.
Journal`s requirements for the authors are available online: http://www.scientiasocialis.lt/jbse/files/JBSE_requirements_2019.pdf
739
ISSN 1648-3898 /Print/, ISSN 2538-7138 /Online/
Compiled by: Vincentas Lamanauskas

Linguistic Editor: Ilona Ratkevičienė
Cover design by: Jurgina Jankauskienė
Layout design by : Linas Janonis

August 30 2022. Publishing in Quires 10,5. Edition 90
Publisher Scientia Socialis Ltd.,

Donelaicio Street 29, LT-78115 Siauliai, Lithuania
E-mail: scientia@scientiasocialis.lt
Phone: +370 687 95668
http://www.scientiasocialis.lt

Printing Šiauliai printing house
9A P. Lukšio Street
LT-76207 Šiauliai, Lithuania
Phone: +370 41 500 333.
Fax: +370 41 500 336
E-mail: info@dailu.lt
https://siauliuspaustuve.lt/

Journal of Baltic Science Education, Vol. 21, No. 4, 2022

Uploaded by

Document Information

Copyright

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Journal of Baltic Science Education, Vol. 21, No. 4, 2022

Uploaded by

Copyright:

Vol.21, No.

Published since 2002 Address:

ISSN 1648–3898 (Print) © Scientia Socialis Ltd., a member of the Lithuanian

ISSN 2538–7138 /Online/

INFORMATION FOR CONTRIBUTORS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 739

PROBLEMS OF PSYCHOLOGY IN THE 21ST CENTURY . . . . . . . . . . . . . . . . . . . . . . . . . . . .

A PERSONAL REFLECTION ON COVID-19 ONLINE TEACHING, LEARNING, AND

Introduction and Background

Teaching •• Face-to-Face •• Screen-to-Screen

Assessment •• Uploading assignments to Moodle •• Online reflections

Teaching Style Compromised or Not?

Concerns Related to Online Learning from a Student Perspective

Concerns Related to Online Learning From a Personal Perspective

THE JIGSAW TECHNIQUE IN

ACHIEVEMENT, METACOGNITION ISSN 2538-7138 /Online/

Research Aim and Research Questions

Physics achievement .70 .65

Pre- Post- Pre- Post-

Conclusions and Implications

Task Number 1 – Motion of the freely falling object

Task Number 2 – Velocity of the freely falling object

Task Number 3 – Distance the object falls

Task Number 4 – Correlation between the velocity and the distance

Task Number 5 – Weightlessness

Appendix 2. Scheme of Jigsaw steps

Research with Pre-service Primary School Teachers

Education Context of the Study

Research Aim and Research Questions

Instruments and Procedures

Conceptual area of the PNM Questions in the test Reference on research

Griffiths & Preston, 1992

Conceptual area of the PNM Questions in the test Reference on research

Griffiths & Preston, 1992

Test characteristics TEA LEA Total

Number of respondents 197 170 367

The Occurrence of Correct Responses in Both Study Groups

TEA LEA group

Q1 77.66 74.71 .255

No. Misconception % of TEA % of LEA p

No. Misconception % of TEA % of LEA p

When a piece of coal is hammered to dust, small parts of some

Limitations of the study

Conclusions and Implications

Appendix - The Multiple-Choice test

1. Can we see individual atoms?

2. Are atoms alive?

4. What is the shape of the atoms?

5. Are all atoms of the same size?

6. Do all atoms have the same weight?

8. Do atoms in a solid move?

9. Do the atoms in an iron rod enlarge when the rod is heated?

13. Does an atom need to be broken to free an electron?

Eva Hejnová PhD, Assistant Professor, Department of Physics, Faculty of Science,

USING A MACHINE LEARNING

AFFECTING READING, ISSN 2538-7138 /Online/

MATHEMATICS, AND SCIENCE

Research Aim and Research Questions

Instrument and Procedures

Common Features of Three Subjects in China

School 2 SMINS Instructional time: Science courses 0~2210

4 ST059Q03TA Number of class periods in science 0~40

7 ST060Q01NA number of class periods per week. 10~80

Common Features only Two Subjects in China

Common Features of Three Subjects in the United States

School 3 ST060Q01NA number of class periods per week. 10~80

8 SMINS Instructional time: Science courses 0~2210

Common Features only Two Subjects in the United States