Journal of Baltic Science Education, Vol. 18, No. 5, 2019

Vol.18, No.
5, 2019
ISSN 1648-3898 /Print/
ISSN 2538–7138 /Online/
The International Journal of the Scientia Socialis Ltd., & SMC “Scientia Educologica”
Journal of Baltic Science Education, Vol. 18, No. 5, 2019
Editorial Board
ISSN 1648–3898 /Print/
Editor-in-Chief ISSN 2538–7138 /Online/
Prof., Dr. Vincentas Lamanauskas Siauliai University, Lithuania
Deputy Editor-in-Chief
Prof., Dr. Andris Broks University of Latvia, Latvia
Deputy Editor-in-Chief
Prof., Dr. Miia Rannikmäe University of Tartu, Estonia
Executive Secretary
Dr. Laima Railienė SMC “Scientia Educologica”, Lithuania
Editors:
Prof., Dr. Boris Aberšek University of Maribor, Slovenia
Prof., Dr. Agnaldo Arroio University of Sao Paulo, Brazil
Prof., Dr. Martin Bilek Charles University, Czech Republic
Dr. Paolo Bussotti University of Udine, Italy
Prof., Dr. Bulent Cavas Dokuz Eylul University, Turkey
Prof., Dr. Salih Cepni Uludag University, Turkey
Prof., Dr. Hana Čtrnáctová Charles University, Czech Republic
Dr. Peter Demkanin Comenius University, Slovakia
Dr. André du Plessis Nelson Mandela University, South Africa
Prof., Dr. Jack Holbrook University of Tartu, Estonia
Prof., Dr. Ryszard M. Janiuk Maria Curie Sklodowska University, Poland
Dr. Milan Kubiatko Jan Evangelista Purkyně University in Ústí nad Labem, Czech Republic
Dr. Todar Lakhvich Belarusian State Medical University, Republic of Belarus
Prof., Dr. Jari Lavonen University of Helsinki, Finland
Dr. Rita Makarskaitė-Petkevičienė Vilnius University, Lithuania
Prof., Dr. Aadu Ott Göteborg University, Sweden
Prof., Dr. Paul Pace Malta University, Malta
Prof., Dr. Valfrids Paškevičs Daugavpils University, Latvia
Prof., Dr. Jongwon Park Chonnam National University, Korea
Prof., Dr. Raffaele Pisano University of Lille, France
Prof., Dr. Pavol Prokop Comenius University in Bratislava, Slovakia
Prof., Dr. Konstantinos Ravanis University of Patras, Greece
Dr. Alona Rauckienė-Michaelsson Klaipėda University, Lithuania
Dr. Dušica Rodić University of Novi Sad, Serbia
Prof., Dr. Heimo Saarikko Helsinki University, Finland
Dr. Hae-Ae Seo Pusan National University, Korea
Dr. Uladzimir K. Slabin University of Oregon, USA
Prof., Dr. Borislav V. Toshev Sofia University, Bulgaria
Dr. Georgios Tsaparlis University of Ioannina, Greece
Dr. Muhammet Usak Science and Medical Education Research Center, Turkey
A scientific journal JBSE issued by the Scientia Socialis Ltd., in cooperation with SMC Scientia Educologica, Lithuania,
emphasizes theoretical, experimental and methodical studies in the field of science education. JBSE is an international
academic journal. In order to maintain the high standards appropriate to such a journal, all contributions received are
submitted for anonymous review by two experts, additionally to review by the Editor. The decision of the Editor on
the acceptance of articles is final and no correspondence can be entered into on reasons for rejection of a submitted
contribution.
Published since 2002 Address:

The journal is published bimonthly. Scientia Socialis, Ltd.
IF: 1.024 (2018) Donelaičio Street 29, LT-78115 Siauliai, Lithuania
H Index: 12 (2018) E-mail: mail.jbse@gmail.com
SJR: 0.387 (2018) Phone: +370 687 95668
ICDS: 10.7 (2017) Home page: http://www.scientiasocialis.lt/jbse/
ICV: 151.38 (2018)
© Scientia Socialis Ltd., Lithuania, 2019
ISSN 1648–3898 (Print)
ISSN 2538–7138 (Online)
The articles appearing in this journal are indexed/abstracted in Copernicus Index, EBSCO:
650 Academic Search Premier, Web of Science Core Collection (Social Sciences Citation Index ),
SCOPUS, ProQuest, Road, Crossref, and DOI.
ISSN 1648–3898 /Print/ CONTENTS
CONTENTS
Editorial
INDUSTRY 4.0 AND SCIENCE EDUCATION

Bulent Cavas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 652
Articles
INDONESIAN PRIMARY SCHOOL AND MIDDLE SCHOOL STUDENTS’ ATTITUDES TOWARD

SCIENCE: FOCUS ON GENDER AND ACADEMIC LEVEL
Rahmi Qurota Aini, Arif Rachmatullah, Minsu Ha . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 654
DICHOTOMOUS KEYS IN THE BOTANICAL LEARNING OF NON-VISUAL (BLIND) PEOPLE

Branko Andjić, Stanko Cvijetićanin, Simon Hayhoe, Rade Grujičić, Danijela Stešević . . . . . . . . . . . . 668
RECONCEPTUALIZING AND FIELD TESTING THE SCIENTIFIC LITERACY FRAMEWORK BY

EXPLORING THE ASPECT OF SCIENTIFIC LITERACY IN TURKISH SCIENCE CURRICULUM
Mustafa Cansiz, Nurcan Cansiz . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 681
EFFECTS OF MICROTEACHING MULTIPLE-REPRESENTATION PHYSICS LESSON STUDY ON PRE-

SERVICE TEACHERS’ CRITICAL THINKING
Billy A. Danday, Sheryl Lyn C. Monterola . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 692
THE INFLUENCE OF SELECTED VARIABLES ON LOWER SECONDARY SCHOOL STUDENTS’

CONCEPTS ABOUT DISEASES
Jana Fancovicova, Milan Kubiatko . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 708
THE RELATION BETWEEN TEACHERS’ ATTITUDES TOWARDS RENEWABLE ENERGY SOURCES

AND CRITICAL THINKING DISPOSITIONS
Gokhan Guven, Nevin Kozcu Cakir . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 717
BIBLIOMETRIC ANALYSIS OF THE WOS LITERATURE ON RESEARCH OF SCIENCE TEACHER

FROM 2000 TO 2017
Jianqiang Ye, Dimei Chen, Lingxin Kong . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 732
EVALUATION, VALIDATION AND MODIFICATION OF SCIENCE MOTIVATION QUESTIONNAIRE

FOR UPPER SECONDARY SCHOOL
Vanda Janštová, Andrej Šorgo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 748
TEACHER-DEMONSTRATION AND STUDENT HANDS-ON EXPERIMENTS IN TEACHING

INTEGRATED SCIENCES
Mirjana Maričić, Stanko Cvjetićanin, Branko Anđić . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 768
EXAMINATION OF PERCEPTIONS OF GIFTED STUDENTS ABOUT CLIMATE CHANGE AND

GLOBAL WARMING
Fatma Mutlu, Oğuzhan Nacaroğlu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 780
SCIENCE-EDU-COMMUNICATION: TRENDS REVEAL IN 20 YEARS OF SCIENCE

COMMUNICATION RESEARCH
Leon Yufeng Wu, Nathan M. Truong, Hsin-Yen Lu, Yuen-Hsien Tseng, Chun-Yen Chang . . . . . . . . . . . 793
Information
INFORMATION FOR CONTRIBUTORS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 806
PROBLEMS OF PSYCHOLOGY IN THE 21ST CENTURY. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 807
651
INDUSTRY 4.0 AND SCIENCE EDUCATION
Bulent Cavas
Dokuz Eylul University, Turkey
Industry 4.0 and its applications are one of the most important issues of countries that want to keep their
competitiveness in the field of scientific, technological and innovation in today’s world. The Countries that are not
able to adapt to Industry 4.0 processes will not be able to develop scientifically and technologically or keep up
with current innovation processes. One of the aims of this paper is to explain the concept of industry 4.0 to our
readers in a comprehensible manner and to show how much science education is actually related to industry 4.0.
In a general sense, Industry 4.0 or in other words, the 4th Industrial Revolution is an expression that includes
many modern automation processes, large databases that communicate with each other, robotic devices and dark
factories, in short, quality production technologies. In addition, Industry 4.0 includes modern technologies such as
additive manufacturing, autonomous robots, big data, artificial intelligence, augmented reality, system integration,
internet of things, cybersecurity, cloud computing, machine learning, deep learning etc. These technologies play
an important role especially in the formation of new smart factories.
Considering the factories that were known in the past, it was seen that there were workers working at the
different machines and these workers took part in the control and decision making processes. However, today,
there are factories in which even the lights do not light up, and instant decision-making processes are carried out
automatically by modern robotic devices. In such factories, the data obtained from all processes are continuously
subjected to high level analysis and used to create new business models to increase the efficiency of the factory.
After providing information about the concept and processes of Industry 4.0, it is useful to briefly discuss the
historical paradigm in the transition of industry 1.0 to 4.0. Industry 1.0, known as the first industrial revolution,
took place between the 18th and 20th centuries and mainly used mechanical production systems using water
and steam power. The main objective here is to increase production by increasing the working comfort of work-
ers. Some technologies, such as steam trains, industrial weaving devices, can be defined as industry 1.0 products.
Industry 2.0 had been effective since the early 20th century. The emergence of electricity as an important
power source in this period caused the end of the use of water and steam power in the factories. It is one of the
important initiatives of this period that the machines required for the factories are designed to use electrical power.
At the beginning of this period, the concepts of efficiency and productivity emerged. It has begun to focus on the
processes that affect production, such as which worker will work on which machine and what needs have to be
paid attention to when the product produced by this worker moves to the next stage. Extra processes to improve
product output and quality are considered important functions of the factories.
Industry 3.0 seems to have taken place towards the end of the 20th century. During this period, the discov-
ery of electronic circuits and the integration of these circuits to each other began to make workers’ work easier
in factories, and in some cases they started to take their place. Towards the end of this period, it is seen that fully
automate machines emerged. During the Industry 3.0 period, software requirements appeared to be largely evident.
In particular, the software that will work on the developed electronic hardware has gained importance. In addition,
more in-depth studies have been carried out on concepts such as product process management, follow-up and
operational resources within the factories, and it has been emphasized that digital technology and automation
software should be used in place of manual processes.
652
https://doi.org/10.33225/jbse/19.18.652
By the time of Industry 4.0 (in the past few decades), a new era of technology has emerged in which machines
in factories communicate and interact with each other via the Internet of Things (IoT). In this period, it is seen
that machines play an active role in decision-making processes by communicating with instant real data. In this
developing process, we see that there are systems in which the factories can plan their production in the most
efficient way and they can take control and decide with instant data. It is seen that the factories will increase their
production processes through Industry 4.0 to the highest level and in a controlled manner.
In the section above, the industry 1.0 - 4.0 paradigm is tried to be summarized. Maybe, it is better to continue
to ask questions to understand the situation between industrial developments and science education: To what
extent these developments in the industry have taken place in science education?
In response to this question, we can begin by explaining the change in schools. We can take an example
from science classrooms. When the classes in the Industry 1.0 period and today’s science classes are examined,
it is impossible to say that there have been huge changes. Today, it is seen that most students are still passively
trying to understand the information coming from their teachers as much as they can in their classrooms in most
countries. The situation was the same in the past. The science topics taught to students are more or less the same
when compared. What about the development of phones after invention by Graham Bell in 1876? While it was not
possible to call for a person without a cable connection in the past, we are now able to talk as much as we want at
any location. In short, we carry high-tech computers in our pockets. It is easy to say that this incredible change in
the industry has not been the same in science education.
The second important question can be connected with science education: Current science education in
countries in terms of content and capacity provides individuals to meet today’s digital expectations?
The importance of science and technology education comes to the forefront at this point, considering that the
workers who will work in factories in the future should be equipped with high-level technological knowledge and
experience. It is clear that this knowledge and experiences should be provided from the early stages of education.
It is very important that qualified manpower equipped with scientific and technological knowledge and experience
will increase the competitiveness of that country in science and technology fields. The ministries of education have
great responsibilities in training qualified manpower who can adapt to this incredible transformation in the industry.
Otherwise, we will have a great number of people who have problems with finding a job which requires any skills.
From this point of view, radical changes in science education are inevitable. Priority should be given to systems
where students can produce alternative solutions using the knowledge and skills they have gained, especially
when faced with complex scientific problem situations. When developing alternative solutions, students are re-
quired to communicate effectively with their friends, teachers and stakeholders in the outside world rather than
by individual studies. In addition, high-level thinking skills such as critical thinking and complex problem solving
need to be developed.
As a final remark for this paper, it is inevitable that some of the complex skill sets, where even one university
degree is not sufficient, will be questioned by the leaders of industry in science and technology. For that reason,
individuals who know how to be imaginative, creative, adaptable, flexible will win the race among the others. It is
also clear that these skills are easily developed using science education opportunities.
Received: September 20, 2019 Accepted: October 05, 2019
Bulent Cavas PhD, Professor, Dokuz Eylul University, Distance Education

Application and Research Center & Department of Science
Education, Turkey.
E-mail: bulentcavas@gmail.com
Website: http://people.deu.edu.tr/bulent.cavas
653
INDONESIAN PRIMARY
SCHOOL AND MIDDLE SCHOOL
STUDENTS’ ATTITUDES TOWARD
ISSN 2538-7138 /Online/
SCIENCE: FOCUS ON GENDER
AND ACADEMIC LEVEL
Abstract. Student attitudes toward science Rahmi Qurota Aini,

have been one of the longstanding topics
Arif Rachmatullah,
in science education research. Even though
Indonesia has a unique educational system
Minsu Ha
in which cultural, religious, and cognitive
aspects are incorporated in the science K-12
curriculum, a limited number of studies
have explored Indonesian students’ atti-
tudes towards science. This research aimed
to examine students’ attitudes toward Introduction
science and the interaction between aca-
demic grade level and gender in Indonesian Science is an essential part of human life, necessary for understanding
primary and middle school. Thirty items the nature of phenomena, inventing new technologies, and optimizing
from five components of the BRAINS instru- decision-making to improve our lives. Science, technology, engineering,
ment were administered to 1587 Indone- and math (STEM) capabilities and skills, which are fostered in science classes
sian students from fourth grade through are also in high demand for the future global workforce. According to the
ninth grade to measure their attitude Workforce in the Future report (PwC, 2017), resource scarcity and climate
toward science. Rasch analysis, two-way change will increase the demand for jobs in areas such as alternative energy,
ANOVA, and structural equation modeling new engineering processes, and waste management. Dramatic changes
path analysis were used to answer the toward machine-oriented techniques and artificial intelligence will also
proposed research questions. The current necessitate the creation of new professions. UNESCO (2017) reported
research found that Indonesian students’ that gender equality in science would promote new solutions, generate
attitudes toward science were signifi- diverse innovation, reduce bias, and broaden new research in science.
cantly affected by academic level, however, However, the gender gap ratio between science researchers from 2008 to
gender only affected three components of 2014 was reported to be 70% males and only 30% females (UNESCO, 2017).
attitude (control belief, attitude toward the Female interest in STEM is believed to originate in school and the report by
behavior, and intention). Female students UNESCO (2017) also indicated that the gender gap in STEM was apparent
showed a higher attitude toward science in secondary education. Therefore, studies of gender differences have been
than male students in general. The trends in undertaken for more than 20 years to address this issue.
every component of the students’ attitudes Given the emergence of the importance of gender equity in science,
decreased from primary school to mid- it is essential to consider educational factors, particularly in the psycho-
dle school. The present research provides logical influences on students’ attitudes toward science. Several studies
a deeper discussion by considering the have pointed out that one of the personal factors substantially influencing
socio-cultural and educational history of students’ choices regarding course and career aspirations was their attitude
Indonesia. toward science (Khishfe & Boujaoude, 2016; Masnick, Valenti, Cox, & Osman,
2010; Osborne, Simon, & Collins, 2003; Prieto & Dugar, 2016; Uitto, 2014).
Keywords: academic level, gender, middle
UNESCO (2015) also reported that one of the crucial factors predicting STEM
school, primary school, science attitude.
career interest itself was attitude toward science in school.
Psychological studies have found a relationship between attitudes,
Rahmi Qurota Aini
Kangwon National University, Republic of Korea
intention, and belief. Theory Reasoned Action and Behavior (TRAPB)
Arif Rachmatullah proposed by Ajzen and Fishbein (1991, 2005, and 2010) is one of the
North Carolina State University, USA well-known findings. TRAPB suggests a framework for understanding the
Minsu Ha
Kangwon National University, Republic of Korea
specific behavior of a human by considering the interest and perception
controlling the behavior. According to this theory, it is possible to access
654
ISSN 1648–3898 /Print/ INDONESIAN PRIMARY SCHOOL AND MIDDLE SCHOOL STUDENTS’ ATTITUDES TOWARD
ISSN 2538–7138 /Online/ (P. 654-667)
attitude-relevant information from different types of beliefs. Therefore, exploring students’ attitudes toward
science is also helpful in identifying prospective approaches to science teaching and learning (Elster, 2007).
Previous research indicated that the attitude of male students toward science was more positive than that of
female students (Dawson, 2000; Denessen, Hasselman, & Louws, 2015; DeWitt & Archer, 2015; Toma, Greca, &
Orozco Gómez, 2019).
Interestingly, 67.5% of the students enrolled in STEM disciplines at Indonesian universities are female and
32.5% are male (UNESCO, 2015). This gender percentage, which is opposite that found in previous research,
requires further investigation of the Indonesian students’ attitudes toward science and the interaction between
each attitude’s construct. It is important to investigate this issue to provide insight into gender equity in science
education.
Literature Review
Attitude towards Science
In a psychological context, attitude is defined as the latent variable in an individual’s response toward
some object, person, or other aspects (Ajzen, 1989). Major research from the 1960s in science education has
acknowledged the importance of attitude and its effect on learning science (Jones, Howe, & Rua, 2000; Koballa
& Glynn, 2013; Osborne, Simon, & Collins, 2003). Previous research on science attitudes has identified students’
interest in science, science-related activities, and enjoyment of science-learning experiences as important fac-
tors (Khishfe & Boujaoude, 2016).
A study by Anderhag, Hamza, and Wickman (2015) used the term “taste”, rather than attitude, to examine
how students come to enjoy science class, see it as a concern in their everyday lives, and choose careers in sci-
ence. Furthermore, Osborne, Simon, and Collins (2003) described that attitude was different from interest in
science, since attitude itself refers to a more general area of an individual relationship with the environment,
is more behavioral, and has a relationship between intention. Thus, attitudes toward science also have the at-
tributes of feeling, belief, value, and opinion about science, which distinguishes as scientific attitude (Koballa &
Glynn, 2013; Osborne, Simon, & Collins, 2003).
According to Summers and Abd-El-Khalick (2018), the concept of attitude toward science has shifted from
individual’s preference (like or dislike) and negative or positive feelings about science to a cognitive orientation.
TRAPB by Ajzen and Fishbein (1991, 2005, 2010) also put its role in this context. Summers and Abd-El-Khalick
(2018, p.12) simplified it by stating that “the more favorable the attitude and the subjective norm, and the greater
the perceived control, the stronger the person’s intention is to perform the behavior in question.” Furthermore, Sum-
mers and Abd-El-Khalick (2018) developed an instrument to measure students’ attitude toward science called
Behaviors, Related Attitudes, and Intentions toward Science (BRAINS), as well as developing the components
of attitude toward behavior. They used Ajzen’s TRAPB theoretical framework to understand science-related be-
havior and predict students’ attitudes and behavioral intentions. Components of BRAINS relate to five TRAPB’s
components, which are behavioral belief, normative belief, control belief, attitude toward behavior, and intention.
Gender Issue in Science and Attitude towards Science
Research in science education indicates that gender has an essential role in influencing attitudes toward
science. In the 1960s, gender inequality in science education was acknowledged and studies on gender and
science education began (Brootman & Moore, 2008). Baker (2002) pointed out that the gender issue in science
education was recognized as a crucial problem in the late 1980s, when the number of scholarly articles about
gender began to increase. This was also supported by the finding by Osborne, Simon, and Collins (2003) that
gender was one of the factors that influenced students’ science attitudes in general. According to Weinburgh
(1995), who conducted a meta-analysis study between 1970 and 1991 by examining gender differences in
attitudes toward science and their correlation with achievement in science, the attitude of male students in
science was more positive than female students. A similar result was found by a meta-analysis study conducted
by Becker (1989). Because of these issues, several authors have attempted to identify gender discrimination in
the teaching and practice of science (Brotman & Moore, 2008; Brickhouse & Potter, 2001; Howes, 2002).
Early studies indicated that female students had less science attitude than male students (Catsambis, 1995;
655
(P. 654-667) ISSN 2538–7138 /Online/
Simpson & Oliver, 1990). A more current report by several authors (Jones, Howe, & Rua, 2000; Miller, Blessing, &
Schwartz, 2006) also indicated that male students had a higher science attitude than females, even though there
were also big gaps of in certain subjects, such as physics and biology. Contrasting results have also been found
in several studies, indicating that female students had more positive attitudes or equal proportions of positive
attitudes toward science as male students (Boone, 1997; Harwell, 2000; Murphy & Beggs, 2003; Said, Summers,
Abd-El-Khalick, & Wang, 2016). According to Koballa and Glynn (2013), sociological reasons for gender inequality
in science students might be the result of different cultural expectations of the genders by people close to the
students (i.e., parents, peers, and teachers) and their experiences in science. According to Baker (2002 p. 662),
gender and equity issues in science education need to be addressed by examining the gender, class, cultural,
religious ethnicity to answer where the gender and equity in science education are.
Gender, Culture, and Educational Issue in Indonesia
Indonesia is the largest Muslim country in the world, about 87% of its population is Muslim (Statistics Indo-
nesia, 2018). In the colonial era (beginning in the 16th century), a woman’s role was as a mother and wife, and at
that time, only women from elite backgrounds attended formal school (Rinaldo, 2011). That was also impacted
by the role of gender in Indonesian culture. For example, Javanese culture, the most prominent ethnic group in
Indonesia, had three prominent Java beliefs about women. The first was in the saying ‘konco wingking’ (literally
meaning “a companion whose place is in the back part of the house”), the second was ‘suargo nunut neraka
katut’ (meaning “a wife will go wherever her husband goes, to hell or heaven, making her a mere follower of the
husband’s deed”), and the third said women’s roles are ‘manak, masak, macak’ (meaning “to breed, to cook, and
to adorn herself for her husband)” (Arimbi, 2010 p.61). Thus, women’s roles in Javanese culture were to manage
the household and be obedient to their husbands.
In the 1880s, support for Indonesian education for girls and women gained momentum from the actions
of the Indonesian national heroine for women’s emancipation, Raden Adjeng Kartini. She began to read many
books and wrote famous pieces about Indonesian women at that time, which commenced the Indonesian literacy
culture campaign for women. The reformation of women’s rights in Islam started in the mid-1990s when political
subjectivities were shaped (Rinaldo, 2011). The period in the 1990s after Indonesian independence was a vital
period for Muslims in Indonesia where the Indonesian constitution guaranteed equal rights for both genders
and a women’s independent organization was founded (Arimbi, 2010; Rinaldo, 2011). The words “gender qual-
ity” and “feminism” were famous among students because of the network of women’s NGO in Indonesia in the
mid-1990s, even though the stereotypical female activists were from elite backgrounds. Over the last decade,
the gender quality issue has been raised again by Muslim women.
Regarding the current situation for women in Indonesia, their participation in decision-making roles is
still low, with 18% of women’s participating in the Indonesian parliament in 2009-2014 (UNDP Indonesia, 2017;
JICA report, 2011). Women still accounted for a bigger proportion of the unemployment rate in 2014, when
6.2% of females and 5.7% of the males were unemployed (JICA report, 2011; UNDP Indonesia, 2017). In general,
women are also paid less than men, even though it depends on their academic level (UNDP Indonesia, 2017;
Purnastuti, Miller, & Salim, 2013). The stereotype of “women’s place is in the house” contributes to the female
unemployment rate in Indonesia.
As mentioned previously, in Indonesia, female students still accounted for a larger proportion of students
enrolled in STEM disciplines in 2014 in selected universities, where females and males comprised 68% and
32.5% of the enrollment, respectively. However, the data reported that the female students enrolled in STEM
fields in higher education were concentrated in biology or health fields and a bigger proportion of students
in the engineering field were comprised of males (UNESCO, 2015). Based on the UNESCO report, prospective
female students chose the health field and prospective male students chose the engineering field as their career
aspirations starting from 15 years old.
Indonesian K-4 to 9 Science Curriculums
Another factor that influences students’ attitudes toward science is the curriculum. Osborne, Simon, and
Collins (2003) described curriculum as variables that influenced students’ attitudes toward science, even though
the more significant effect on attitude was the teacher him/herself. In addition, it has been hard to generalize
656
ISSN 2538–7138 /Online/ (P. 654-667)
which attitude or curriculum impacted attitudes toward science. Evidence by Osborne, Simon, and Collins (2003,
p.1070) highlighted that effective science teaching could be the main reason for negative attitudes toward
school science.
Current Indonesian curriculum (K-2013) has four core competencies which are covered using a student-
centered scientific approach. Science subject begins in the fourth grade in primary school until ninth grade
of middle school level where it is taught as one integrated science subject. At the high school level, science
subjects are separated into biology, chemistry, and physics. The general principles of the Indonesian curriculum
are comprehensive and continuous (Dharma, 2008), where the content in all grades is the same but the depth
and scope are different. The subjects are presented continuously at all educational grade levels. This indicates
that Indonesia follows the spiral curriculum, where the difficulty construct level increases in every grade. The
students learn the same topics or sub-subjects at every grade level with different degrees of difficulty, starting
from a concrete concept to an abstract concept and from general to specific concepts. Science teaching and
learning also follows a continuum from surface knowledge to constructed knowledge (Biggs & Collin, 2014).
Research Focus
The current research examined Indonesian students’ attitudes toward science with the following research
questions:
1. What is the extent of the effect of gender, academic grade level, and the interaction between gender
and academic grade level on Indonesian students’ attitude toward science?
2. What are the aspects of attitude toward science that influence Indonesian students’ attitudes toward
science?
Research Methodology
General Background

Quantitative research was performed, and the data were collected in August 2017. The researcher asked the
school officer to inform the teacher and students about this research and the school administrators arranged
the classes and students who were willing to participate in the research. Before completing the questionnaire,
the students read the conditions of research involvement. An IRT-Rasch analysis was performed for data vali-
dation and the person measure of IRT-Rasch was used in further analyses. Subsequently, two-way ANOVA, the
correlation of gender and grade to students’ attitudes toward science, and structural equation modeling (SEM)
were examined.
Participants
The data of the current research were gathered from 1587 Indonesian students from fourth grade through
ninth grade in primary and middle schools located on the western Java Island. According to Decile (2010), many
researchers from educational studies consider that 30-500 subjects are appropriate enough for a parametric
test. Furthermore, sample size also considered as enough criterion for statistical significance and meaningful to
represent the population. The participants were gathered from four schools, with both public schools and private
schools at each grade level. From the 1587 total participants, 806 (51%) were female, 764 (49%) were male, and
17 did not indicate gender information (missing). Ninety-seven percent of the total participants were Muslim,
and some were of another religion (Christian, Catholic, and Hindu). Regarding the frequencies of participants at
the primary level, 148 students were in the fourth grade, 128 were in the fifth grade, and 103 were in the sixth
grade. In the middle school level, 361 students were in the seventh grade, 380 were in the eighth grade, and
450 were in the ninth grade.
Research Instrument and Instrument Validation
The BRAINS Instrument developed by Summers and Abd-El-Khalick (2018) and measured on a 5-point
Likert-scale (from strongly disagree to strongly agree) was used in this research. The main reason that the BRAINS
657
(P. 654-667) ISSN 2538–7138 /Online/
instrument was used is because it standardizes attitude measurements and removes gaps between grades and
ages from the internal consistency. The BRAINS instrument also has a strong theoretical framework built on
Ajzen’s TRAPB (1991, 2005, 2010).
The BRAINS Instrument has five dimensions (constructs) with a total of 30 items translated into the Indonesian
language. The Behavioral Beliefs dimension works to control positive or negative attitudes toward behavior (Ajzen
& Dasgupta, 2015). The behavioral belief dimension is a belief about the effects of learning science or becoming
a scientist as measured by nine items such as “We live in a better world because of science” or “Scientists usually
like to go to work, even when they have a day off.” The Normative Beliefs dimension refers to individual or group
approval for performing the behavior. This belief stems from social (for example, family or friends) approval or
disapproval of engagement in science (Ajzen & Fishberin, 2005; Summers & Abd-El-Khalick, 2018). It has three
items, including “My family encourages my interest in science.” The Control Beliefs dimension refers to the basic
perception of behavioral control. This belief is perceived to stem from self-efficacy or effort and that this belief
and actual control (i.e., skills or abilities) can facilitate performance (Ajzen & Fishberin, 2005). In this context, an
individual’s perceived ability to learn science or the effort put forth affects science learning (Summers & Abd-
El-Khalick, 2018). The Control Beliefs dimension has six items about perceived self-efficacy, such as “Science is
easy for me.” The Attitude toward Behavior dimension refers to the control of responding favorably or unfavorably
toward science (Ajzen & Fishbein, 2005). The attitude refers to different aspects of science and connects with
individuals’ lives. This attitude toward behavior serves as the connection between belief factors and an indi-
vidual’s intention. Examples of items in this dimension are “I really like science,” and “Science is one of the most
interesting school subjects.” The Intention dimension involves motivational factors of an individual’s choice or
conscious plan which are reflected in the performance of an individual’s behavior. It is also an indicator of how
much effort individuals are willing to put forth in order to perform their behavior (Ajzen, 1991). The intention
dimension includes students’ intention to pursue science, such as a career in science. One of the samples from
the six items of intention is “I would enjoy working in a science-related career.”
The responses to the item in the BRAIN Instrument were examined in terms of dimensionality, item fit,
and reliability. To determine which was the better way to interpret the instrument, dimensional analyses were
conducted using Rasch analysis. As a result, the five-dimension model was used, in which the Akaike Informa-
tion Criterion (AIC), final deviance, and chi-squared analyses met the benchmark after being compared with the
one-dimensional analyses. Criteria from Adams and Wu (2010) was used to select the dimensionality model by
choosing the lowest AIC and final deviance (Table 1).
Table 1. Dimensionality test.
Final Deviance AIC
One-Dimension 117287 117529

Five-Dimension 115783 116053

The IRT-based Rasch analysis and Classical Test Theory (CTT) were used to determine reliability. Cronbach’s
alpha coefficient was determined using the CTT methods. Cronbach’s alpha coefficient was above .663, which
suggests acceptable use of the instrument (Behavioral Beliefs, .769; Control Beliefs, .648; Normative Beliefs, .663;
Attitude towards Behavior, .790; and Intention, .735). Based on the IRT-based Rasch analysis, the coefficient obtained
from Separation Reliability to indicate how fit the parameter separated from the analysis was .993. Furthermore,
all coefficients of EAP (Expected A Posteriori) indicating item reliability were above .752 (Behavioral Belief, .798;
Control Beliefs, .787; Normative Beliefs, .752; Attitude towards Behavior, .878; and Intention .845).
The amount of unexpected variation in the response items was also analyzed by the item fit by the range
of MNSQ and the items were fit into the range of .50-1.50, as recommended by Wright and Linacre (1994). The
miss fitting is found in BRAIN30 item by outfit MNSQ 1.56, but it was still considered acceptable to use (Wright
and Linacre, 1994). Differential item functioning (DIF) was used to identify any items biased in terms of gender or
grade (Table 2). Based on criteria by Zwick, Thayer, and Lewis (1999), the range of DIF was below .5. However, the
DIF result for BRAIN item 8 was considerable, since the gap from benchmark is not a huge difference.
658
ISSN 2538–7138 /Online/ (P. 654-667)
Table 2. Psychometrical properties based on Rasch Modeling in the BRAINS instrument.
Outfit Cronbach’s α if
Dimension Item Estimate Infit MNSQ DIF Gender DIF Grade
MNSQ Item Deleted
BRAIN02 –.459 1.07 1.07 .754 .08 .03

BRAIN03 –.164 1.03 1.02 .751 .28 .06
BRAIN08 .182 1.25 1.16 .765 .00 .66
BRAIN19 –.032 .92 .92 .736 .00 .22
Behavioral Beliefs BRAIN21 –.138 .98 .97 .741 .00 .10
BRAIN25 .484 1.13 1.11 .759 .14 .18
BRAIN26 –.196 .90 .91 .734 .05 .14
BRAIN27 –.198 0.87 .87 .727 .02 .02
BRAIN29 .521 1.12 1.11 .756 .21 .16
BRAIN05 –.364 .97 .98 .597 .07 .36
BRAIN06 –.184 1.18 1.15 .620 .25 .18
BRAIN10 .643 .94 .94 .590 .00 .07
Control Beliefs
BRAIN12 –.131 1.24 1.19 .641 .17 .38
BRAIN14 .303 .94 .94 .599 .08 .13
BRAIN18 –.267 .89 .90 .586 .14 .04
BRAIN09 .147 1.20 1.17 .742 .03 .39
Normative Beliefs BRAIN17 –.055 .91 .91 .482 .11 .36
BRAIN22 –.092 .91 .91 .472 .06 .00
BRAIN01 –.271 .93 .94 .756 .13 .16
BRAIN07 –.048 .88 .89 .749 .00 .07
Attitude towards BRAIN15 –.149 .81 .82 .739 .00 .16

Behavior BRAIN23 .207 1.23 1.22 .787 .03 .13
BRAIN24 .149 .79 .80 .717 .05 .06
BRAIN30 .112 1.56 1.32 .797 .20 .43
BRAIN04 .019 .95 .94 .680 .08 .09
BRAIN11 .044 1.36 1.30 .741 .00 .32
BRAIN13 –.049 1.12 1.10 .707 .12 .16
Intention
BRAIN16 –.382 .88 .89 .688 .15 .05
BRAIN20 .055 .87 .87 .675 .00 .12
BRAIN28 .314 .98 .98 .695 .13 .12
Data Analysis
Dimensionality analysis was performed using ACER ConQuest version 4.5.0, whereas the item fit and IRT-based
Rasch analyses used WINSTEP version 4.0.1. Furthermore, to examine the interaction and correlation between
academic levels and gender on students’ attitudes toward science, two-way ANOVA and Pearson’s Correlation were
performed using SPSS version 22. Following the analysis of dimensionality, the five-dimensional Rasch model was
applied to every person. The five attitudes toward science constructs and path analyses were conducted using
SEM to evaluate the students’ attitude toward the science model fit construct. SEM analysis was performed with
AMOS version 2.4.
659
(P. 654-667) ISSN 2538–7138 /Online/
Research Results
Two-way ANOVA was used to examine the effect of academic level and gender for each attitude toward the
behavior subscale, as shown in Table 3. The mean difference between every academic level and gender are shown
in Table 3. The significant results in the Attitude toward Science variable on grade and gender are discussed.
Table 3. The results from two-way ANOVA test of gender and grade.
Grade Gender Grade x Gender

Variable
F p-value ηp2 F p-value ηp2 F p-value ηp2
Overall 11.09 <.001 .034 5.93 .015 .004 .20 .961 .001
Behavioral Beliefs 4.79 <.001 .015 2.39 .122 .002 .32 .904 .001
Normative Beliefs 5.95 <.001 .019 0.09 .768 .000 1.67 .138 .005
Control Beliefs 17.44 <.001 .053 4.13 .042 .003 1.16 .327 .004
Attitude towards Behavior 11.53 <.001 .036 6.13 .013 .004 1.67 .138 .005
Intention 8.16 <.001 .026 7.29 .007 .005 0.90 .480 .003
Relationship of Academic Level and Gender on Students’ Attitudes toward Science
Behavioral beliefs. A significant effect of academic level was found on the student’s behavioral beliefs (F [5, 1558]
= 4.79, p <.001, ηp2 = .015). In contrast, not significant on gender on behavioral beliefs (F [5, 1558] = 2.39, p <.122,
ηp2 = .002) was found. The combined analyses also found no significant interaction between academic level and
gender (F [5, 1558] = .32, p <.904, ηp2 = .001). The significant result on academic level indicated that the differences
between the students’ behavioral beliefs were affected on an academic level. The mean differences are shown in
Figure 1, where the means in primary school increased but fell sharply at the middle school level.
Normative beliefs. A significant effect of academic level was found on the student’s normative beliefs (F [5, 1558]
= 5.95, p <.001, ηp2 = .019) and gender was not significant (F [5, 1558] = .09, p <.768, ηp2 = .000). No interaction between
academic level and gender was found in normative beliefs (F [5, 1558] = 1.67, p <.138, ηp2 = .005). This result was
similar to the behavioral beliefs, where only an effect of the academic level was seen. The same trend of a dramatic
decrease from sixth grade to seventh grade seen in the behavioral beliefs was also seen in the normative beliefs.
Control Beliefs. Academic level significantly affected the student’s control beliefs (F [5, 1558] = 17.44, p <.001,
ηp2 = .053), as did gender (F [5, 1558] = 4.13, p <.042, ηp2 = .003). However, no significant interaction between aca-
demic level and gender was found (F [5, 1558] = 1.16, p <.327, ηp2 = .004). The results indicate that students’ control
beliefs were affected by academic level and gender. The mean differences are shown in Figure 1, where relatively
higher control beliefs in females than males were seen at every academic level and both the female and male
values decreased steadily from fourth to ninth grade.
Attitude towards Behavior. Academic level had a significant effect on students’ attitude toward behavior
(F [5, 1558] = 11.53, p <.001, ηp2 = .036), as did gender (F [5, 1558] = 6.13, p <.013, ηp2 = .004). In contrast, no signifi-
cant interaction between academic level and gender was found in attitude toward behavior (F [5, 1558] = 1.67,
p <.138, ηp2 = .005). Similar trends in the mean differences of attitude toward behavior and control beliefs were
found, where females had a higher attitude toward behavior, and it decreased over the grades (Figure 1).
Intention. Significant effects of academic level regarding the student’s intention in academic level (F [5, 1558] =
8.16, p <.001, ηp2 = 0.026) and gender (F [5, 1558] = 7.29, p <.007, ηp2 = .005) on the students’ intentions were found.
However, no significant interaction between academic level and gender was found (F [5, 1558] = .90, p <.480, ηp2
= .003). This result indicates that academic level and gender affected students’ intention. A significant two-way
interaction was seen between the academic level and gender. The similar trend is also found in the mean of differ-
ence in overall academic level. Interestingly, female students’ intention in 9th grade was higher than in 7th grade.
660
ISSN 2538–7138 /Online/ (P. 654-667)
The mean differences of person measures between students’ attitude toward science according to academic
level and gender are presented in Figure 1. Except for the control belief and attitude toward behavior components,
the results showed a trend of increasing attitude by primary grade. Interestingly all components decreased from
sixth grade in the primary school level to seventh grade in the middle school. Regarding gender, at the end of
primary and middle school (sixth grade and ninth grade, respectively), female students’ attitudes toward science
were relatively higher than male. Students’ attitudes toward science in each construct will be discussed further.
Figure 1. The interaction between academic level and gender on students’ attitude toward science.
Correlations between the Five Constructs of Attitude toward Science according to Academic Level and Gender
Correlation coefficients were examined to determine the association of each construct. All attitude constructs
toward science were significantly correlated (p < .05) with samples from every grade level in primary and middle
school and gender (Table 4).
661
(P. 654-667) ISSN 2538–7138 /Online/
Table 4. The correlation coefficients (r) for every variable.

(Notes: ** p-value < .01; * p-value < .05)
Academic x Gender
Component I Component II Primary Middle
Female Male Female Male
Normative Beliefs .528** .405** .458** .454**

Control Beliefs .409** .425** .391** .329**
Behavioral Beliefs
Attitude toward Behavior .524** .587** .525** .527**
Intention .560** .435** .471** .440**
Control Beliefs .254** .245** .317** .221**
Normative Beliefs
Attitude toward Behavior .649** .534** .640** .566**

Intention .608** .298** .576** .471**
Control Beliefs Attitude toward Behavior .318** .456** .397** .342**

Intention .570** .618** .561** .523**
Attitude toward Behavior Intention .648** .590** .639** .623**
A weaker association between each component was found in the correlation coefficients between the samples.
At the primary school level, the lowest correlation was found between normative beliefs and control beliefs (r = .245)
for both males and females, whereas the highest correlation was found between normative beliefs and attitude
toward behavior (r = .649) for females and between control beliefs and intention (r = .618) for males. Regarding
the middle school level, the lowest coefficient was found between normative beliefs and control beliefs for both
females (r = .317) and males (r = .221). For females in middle school, the highest coefficient was found between
normative beliefs and attitude toward behavior (r = .640). For males in middle school, the highest coefficient was
found between attitude toward behavior and intention (r = .623). Overall, the correlation coefficients of male stu-
dents were much lower than those in the female students in both primary and middle school.
SEM (Structural Equation Modeling)
Path analysis was conducted (Figure 2) based on SEM. The chi-squared value for males and females was ex-
amined (p < .05). RMSEA, CFI, TLI, NFI, AGFI, sRMR, and CMIN/DF were also examined as fitness indices. The RMSEA
value (.051) was considered to fit the data well because it was below .08. The CFI (.995), GFI (.996), and TLI (.960)
values were also more than .09 and considered acceptable. The value for sRMR (.0185) was considered a good fit
because it was below .05. The path values are shown in Figure 2.
662
ISSN 2538–7138 /Online/ (P. 654-667)
Figure 2. Structure Equation Modeling of attitude toward science by gender

CMIN/DF = 5.044, sRMR = .0185, GFI = .996, AGFI = .956, NFI = .993, TLI = .960, CFI = .995, RMSEA = .051 (CI90 = .03 - .074)
Following the benchmark by Keith (1933), a small influence is a value between .05–.10, a moderate influence is
a value between .11-.25, and a large influence is a value of more than .25. Following those benchmarks in females,
the largest influences were found in control beliefs on attitude toward behavior (.52) and normative beliefs on
intention (.34). Interestingly, the influence of behavior beliefs on intention was not significant (.05). However, the
largest influence was found in control beliefs on attitude toward behavior (.38), followed by normative beliefs on
intention (.37). Of note, the influence of control beliefs in females was higher than in males.
Table 5. Standardized β values for each path compared by gender.
Female Male
Path z-score
β p-value β p-value
Academic Level à Control Beliefs –.24 <.001 –.21 <.001 .625

Academic Level à Normative Beliefs .01 .782 –.09 .014 –1.957*
Academic Level à Behavioral Beliefs –.05 .164 –.06 .095 –.208
Behavioral Beliefs àAttitude towards Behavior .22 <.001 .32 <.001 2.231**
Control Beliefs à Attitude towards Behavior .52 <.001 .38 <.001 –3.173***
Normative Beliefs à Attitude towards Behavior .13 <.001 .18 <.001 1.228
Normative Beliefs à Intention .34 <.001 .37 <.001 .923
Attitude towards Behavior à Intention .31 <.001 .40 <.001 1.983**
Control Beliefs à Intention .25 <.001 .09 .004 –3.322***
Behavioral Beliefs à Intention .09 .001 .05 .130 –.962
(Notes: *** p-value < .01; ** p-value < .05; * p-value < .10)
663
(P. 654-667) ISSN 2538–7138 /Online/
Further analysis using z-scores to compare the standardized β values for both genders found that the five paths
were significantly different in both genders, as shown in Table 5. First, male students (β =.32) had significantly higher
β values than female students (β =.22) in the path from behavioral beliefs to attitude towards behavior (p-value
< .05). A similar significant difference was also found between male and female (β =.40 and β =.31, respectively)
paths from attitude toward behavior to intention. However, the female students (β =.01) had a significantly higher
β value than the male students (β= –.09) in the path from academic level to normative beliefs (p < .10). In addi-
tion, a significantly higher β value in females than males (β =.52 and β =.38, respectively) was found in the path
of control beliefs to attitude toward behavior and in the path from control beliefs to intention (β =.25 and β =.09,
respectively) with a p-value <.05. No significant effects were found in the rest of the full sample.
Discussion

The results of this research on Indonesian students’ attitudes toward science and its effect on educational
level and gender were consistent with previous results from Murphy and Beggs (2003) and Said, Summers, Abd-El-
Khalick, and Wang (2016), where attitude toward science declined along with academic grade level. Thus, from the
five BRAIN constructs, the overall trend of attitude toward science plunged from primary school to middle school.
Collectively, behavioral beliefs and normative beliefs had the same trend, where they first increased in primary
school, then dropped in middle school. Control beliefs and attitude toward behavior decreased in every grade, start-
ing from the fourth primary grade until the ninth grade in middle school. The intention scores fluctuated but also
decreased between primary and middle school. In the study by Said, Summers, Abd-El-Khalick, and Wang (2016),
the students’ attitudes toward science also decreased with age. Based on two-way ANOVA, the effect of academic
level was significant in every component of attitude toward science. Even if the school, teacher, and learning experi-
ence between each academic grade level is considered, the subject difficulty by grade is more obvious to discuss
because it is directly related to the Indonesian curriculum. From the aspect of subject difficulty, students might feel
burdened and perceive science to be difficult. Moreover, according to Murphy and Beggs (2003), interest in science
could become a consequence of repetition of topic and national test assessment. Regarding religion, a study by
Koul (2016) described students’ attitudes toward school science and religious outlook and reported that students’
personal beliefs could affect their attitude toward science, as well as their ability to learn science. This belief also
needs to be discussed, since personal beliefs toward science and religion could lead to conflict and contrast (Koul,
2016), where the two frameworks could become opposing and lead to unacceptance.
The second point of note from the academic grade level results is that the students’ control beliefs and at-
titudes toward behavior were the two lowest components among all of the attitudes toward science components.
This is necessary because the control belief perceived from themselves, as their ability about themselves toward
learning science. Examples of BRAINS items in this component are “I cannot understand science even if I try hard”
and “Science is easy for me.” Improving students’ self-efficacy would help the students to believe in themselves.
Several ways have been suggested to foster students’ self-efficacy. Artino (20102) recommended explicit feedback
from the teacher to shape students’ self-monitoring and activate metacognition. Teachers can also use Bandura’s
(1997) four principal strategies to foster students’ self-efficacy by an inactive approach to students’ mastery ex-
periences, vicarious (observational) experiences, social persuasions, and physiological and psychological states.
Furthermore, science learning in the previous class could impact this result and the academic results need to be
seriously considered. A previous study of Indonesian participants by Kristiani, Susilo, and Aloysius (2015) showed
that attitude toward science and cognitive learning were correlated. By considering students’ control beliefs and
attitude toward behavior, we might better understand their impact on students’ cognitive learning in the classroom.
Regarding gender influence, the attitude of female students toward science was relatively higher than males,
as shown in Figure 1. This result was consistent with previous research by Khishfe and BouJaoude (2016) that
examined Lebanese students’ attitudes toward behavior by considering gender and religion. That study reported
that female students took science-related jobs because the subject was seen as a prestigious one, it was a good
future career, and because of their interest in science. This result was also supported by a study by Najafi, Ebrahi-
mitabass, Dehghani, and Rezaei (2012) on attitudes related to science in Iran, where female students more highly
favored the study of environmental issues. The major two findings were that students were aware that science
could have beneficial, as well as harmful effects on the environment and their interest in a science and technology
career was aimed at overcoming those issues. A previous study by (Rohaeti, & Prodjosantoso, 2018) that examined
Indonesian science processing skills showed a significant gap, where female undergraduate students had higher
664
ISSN 2538–7138 /Online/ (P. 654-667)
scores than males. Furthermore, in this study, the positive attitude of female students toward science might
have come from the social perception of science-related careers and their salary in a particular industry. Based
on the two-way ANOVA results, gender only significantly affected control beliefs, attitude toward behavior, and
intention. Gender had no interaction with normative beliefs, suggesting that the individual belief that came from
social approval to engage in a behavior was not significant. Hence, behavioral beliefs, where the individual belief
regarding the consequences of science learning, becoming a scientist, and the relevance of science, was also not
significantly associated with gender. As mentioned earlier, the gender role in Indonesia has been moving forward
and education has become more accessible for women, indicated by the fact that female students accounted for
a greater proportion of STEM-enrolled students in 2014 in the selected university. Furthermore, an older study
by Steinkamp (1982) reported male students’ attitudes toward science in developed countries such as Japan, the
USA, Australia, and New Zealand. This finding emphasized that the role of women in a science-related career in
a developed country in Indonesia was higher and might also come from their attitude toward science in school.
This research found differences in behavioral beliefs on intention between females and males, where, for
females, the influence was significant. This evidence is important because female students believe in science,
as well as its benefits, and the consequences of becoming scientists more than males. For instance, they have a
substantial belief that science makes a better world to live in and that science is important for their future success.
Conclusions and Implications
This research aimed to explore Indonesian students’ attitude toward science based on empirical evidence. The
results showed two important elements that must be considered. First of all, regarding the academic grade level,
the students’ attitude toward science declined from the fourth grade to the ninth grade. Specifically, the decline
occurred between the sixth primary grade and the seventh middle school grade. Although the results showed
that academic grade level was significant in all components of attitude toward science, according to the structure
equation modeling, the component of control belief had large influence on attitude toward behavior and impacted
on their intention. Along with component attitude toward behavior, the trend of control belief component was
gradually decreasing from the 4th grade to 9th grade. Control belief component is defined about students’ belief
in their capabilities to do science-related tasks. This highlights the importance of facilitating classroom teaching
strategies to enhance students’ self-efficacy and help them believe in themselves.
Secondly, this research also found the interaction of gender and the Indonesian students’ attitudes toward
science. In this case, gender was the only significant influence on control beliefs, attitude toward behavior, and inten-
tion components. Moreover, Indonesian female students had a significantly higher overall attitude toward science
than males. The result might indicate that gender roles in Indonesia have moved forward to where education is
more accessible for women and provides a model of attitudes toward science in each gender. It also provides more
insight into why a greater proportion of female students are enrolled in STEM disciplines in Indonesia. Moreover,
this research was also helpful in identifying prospective approaches in science teaching and learning in the future.
Science education researchers have been aware of gender inequality issues since 1960. Given the fact that
future jobs in science demand gender equality to promote diverse innovation, science teachers also need to be
aware of the essential role science classrooms play in this issue. Interestingly, the results of this study showed that
Indonesia has the potential to reduce the gender gap in science. However, the research also revealed the next
challenge, which is the declining student attitudes toward science with academic grade level. Middle school is a
crucial period for students to shape and establish their attitudes. It also means the science teachers should try to
exert positive effects on students’ attitudes in science. Further research on the Indonesian learning environment
is also needed, since how science is taught can be one of the solutions to increasing students’ science attitude.
References
Ajzen, I. (1989). Attitude structure and behavior. In A. R. Pratkanis, S. J. Breckler, and A. G. Greenwald (Ed.), Attitude structure and
function. Hillsdale, NJ: Lawrence Elbaum Associates.
Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50(2), 179-211.
Ajzen, I., & Dasgupta, N. (2015). Explicit and implicit beliefs, attitudes, and intentions. In P. Haggard, and B. Eitam (Eds.). The sense
of agency (pp. 115-144). New York, NY: Oxford University Press.
Ajzen, I., & Fishbein, M. (2005). The influence of attitudes on behavior. In D. Albarracín, B. T. Johnson, & M. P. Zanna (Eds.). The
handbook of attitudes (pp. 173–221). Mahwah, NJ: Erlbaum.
665
(P. 654-667) ISSN 2538–7138 /Online/
Anderhag, P., Hamza, K. M., & Wickman, P. O. (2015). What can a teacher do to support students’ interest in science? A study of
the constitution of taste in a science classroom. Research in Science Education, 45(5), 749-784.
Arimbi, D. A. (2010). Reading contemporary Indonesian muslim women writers: Representation, identity, and religion of muslim
women in Indonesian fiction. Amsterdam, Netherlands: Amsterdam University Press.
Artino, A. R. (2012). Academic self-efficacy: From educational theory to instructional practice. Perspectives on Medical Educa-
tion, 1(2), 76-85.
Baker, D. (2002). Where is gender and equity in science education? Journal of Research in Science Teaching, 39(8), 659-663.
Bandura, A. (1997). Self-efficacy: The exercise of control. New York, NY: W H Freeman.
Becker, B. J. (1989). Gender and science achievement: A reanalysis of studies from two meta-analyses. Journal of Research in
Science Teaching, 26(2), 141-169.
Biggs, J. B., & Collis, K. F. (2014). Evaluating the quality of learning: The SOLO taxonomy (Structure of the observed learning outcome).
New York, NY: Academic Press.
Boone, W. J. (1997) Science attitudes of selected middle school students in China: A preliminary investigation of similarities and
differences as a function of gender. School Science and Mathematics, 97(2), 96–103.
Brickhouse, N. W., & Potter, J. T. (2001). Young women’s scientific identity formation in an urban context. Journal of Research in
Science Teaching, 38(8), 965-980.
Brotman, J. S., & Moore, F. M. (2008). Girls and science: A review of four themes in the science education literature. Journal of
Research in Science Teaching, 45(9), 971-1002.
Catsambis, S. (1995). Gender, race, ethnicity, and science education in the middle grades. Journal of Research in Science Teach-
ing, 32(3), 243-257.
Dawson, C. (2000). Upper primary boys’ and girls’ interests in science: Have they changed since 1980? International Journal of
Science Education, 22(6), 557-570.
Delice, A. (2010). The sampling issues in quantitative research. Educational Sciences: Theory and Practice, 10(4), 2001-2018.
Denessen, E., Vos, N., Hasselman, F., & Louws, M. (2015). The relationship between primary school teacher and student attitudes
towards science and technology. Education Research International, 2015, 1-7.
DeWitt, J., & Archer, L. (2015). Who aspires to a science career? A comparison of survey responses from primary and secondary
school students. International Journal of Science Education, 37(13), 2170-2192.
Dharma, A. (2008, August). Indonesian basic education curriculum: Current content and reform. Roundtable Discussion Retrac
Governing Board Meeting (Vol. 27). Symposium conducted at Retrac Governing Board Meeting in Institut Aminuddin Baki,
Malaysia.
Elster, D. (2007). Student interests—the German and Austrian ROSE survey. Journal of Biological Education, 42(1), 5-10.
Fishbein, M., & Ajzen, I. (2010). Predicting and changing behavior: The reasoned action approach. New York, NY: Taylor & Francis
Harwell, S. H. (2000). In their own voices: Middle level girls’ perceptions of teaching and learning science. Journal of Science
Teacher Education, 11(3), 221-242.
Howes, E. V. (2002). Connecting girls and science: Constructivism, feminism, and science education Reform. New York, NY: Teachers
College Press.
Japan International Cooperation Agency [JICA]. (2011). Country Gender Profile in Indonesia. Retrieved from https://www.jica.
go.jp/english/our_work/thematic_issues/gender/
Jones, M. G., Howe, A., & Rua, M. J. (2000). Gender differences in students’ experiences, interests, and attitudes toward science
and scientists. Science Education, 84(2), 180-192.
Keith, T. Z. (1993). Causal influences on school learning. In: H. J. Walberg (Ed.), Analytic Methods for Educational Productivity (pp.
21–47). Greenwich, CT: JAI Press.
Khishfe, R., & BouJaoude, S. (2016). Lebanese students’ conceptions of and attitudes towards science and related careers based
on their gender and religious affiliations. International Journal of Science and Mathematics Education, 14(1), 145-167.
Koballa Jr, T. R., & Glynn, S. M. (2013). Attitudinal and motivational constructs in science learning. In S. K. Abell, and N. Lederman
(Ed.), Handbook of research on science education (pp. 89-116). New York, NY: Routledge.
Koul, R. (2006). Religious outlook and students’ attitudes towards school science. Journal of Beliefs & Values, 27(3), 251-267.
Kristiani, N., Susilo, H., & Aloysius, D. C. (2015). The correlation between attitude toward science and cognitive learning result of
students in different biology learnings. Journal of Baltic Science Education, 14(6).
Masnick, A. M., Valenti, S. S., Cox, B. D., & Osman, C. J. (2010). A multidimensional scaling analysis of students’ attitudes about
science careers. International Journal of Science Education, 32(5), 653-667.
Miller, P. H., Blessing, S. J., & Schwartz, S. (2006). Gender differences in high-school students’ views about science. International
Journal of Science Education, 28(4), 363-381.
Murphy, C., & Beggs, J. (2003). Children’s perceptions of school science. School Science Review, 84, 109-116.
Najafi, M., Ebrahimitabass, E., Dehghani, A., & Rezaei, M. (2012). Students’ attitude towards science and technology. Interdisciplin-
ary Journal of Contemporary Research in Business, 3(10), 129-134.
Osborne, J., Simon, S., & Collins, S. (2003). Attitudes towards science: A review of the literature and its implications. International
Prieto, E., & Dugar, N. (2017). An enquiry into the influence of mathematics on students’ choice of STEM careers. International
Journal of Science and Mathematics Education, 15(8), 1501-1520
Purnastuti, L., Miller, P. W., & Salim, R. (2013). Declining rates of return to education: Evidence for Indonesia. Bulletin of Indonesian
Economic Studies, 49(2), 213-236.
666
ISSN 2538–7138 /Online/ (P. 654-667)
PricewaterhouseCoopers [PwC]. (2017). Workforce in the future: The competing forces shaping 2030. Retrieved from https://www.
pwc.com/gx/en/services/people-organisation/publications/workforce-of-the-future.html
Rinaldo, R. (2011). Muslim women, moral visions: Globalization and gender controversies in Indonesia. Qualitative Sociology, 34(4),
539-560.
Rohaeti, E., & Prodjosantoso, A. K. (2018). Undergraduate students’ science process skills in terms of some variables: a perspective
from Indonesia. Journal of Baltic Science Education, 17(5), 751-764.
Said, Z., Summers, R., Abd-El-Khalick, F., & Wang, S. (2016). Attitudes toward science among grades 3 through 12 Arab students
in Qatar: findings from a cross-sectional national study. International Journal of Science Education, 38(4), 621-643.
Simpson, R. D., & Oliver, J. S. (1990). A summary of major influences on attitude toward and achievement in science among
adolescent students. Science Education, 74(1), 1-18.
Statistics Indonesia. (2010). Population According to Region and Religion. Retrieved from https://sp2010.bps.go.id/index.php/
site/tabel?tid=321
Steinkamp, M. W. (1982, March). Sex-related differences in attitude toward science: A quantitative synthesis of research. Paper pre-
sented at the Annual Meeting of the American Educational Research Association, New York, NY.
Summers, R., & Abd-El-Khalick, F. (2018). Development and validation of an instrument to assess student attitudes toward science
across grades 5 through 10. Journal of Research in Science Teaching, 55(2), 172-205.
Toma, R. B., Greca, I. M., & Orozco Gómez, M. L. (2019). Attitudes towards science and views of nature of science among primary
school students in terms of gender, cultural background and grade level variables. Research in Science & Technological
Education. Advance online publication. doi:10.1080/02635143.2018.1561433.
Uitto, A. (2014). Interest, attitudes, and self-efficacy beliefs explaining upper-secondary school students’ orientation towards
biology-related careers. International Journal of Science and Mathematics Education, 12(6), 1425-1444.
UNDP Indonesia. (2017). UNDP Gender Equality Strategy 2017-2020. Report. Retrieved from http://www.id.undp.org/content/
indonesia/en/home/library/sustainable-development-goals/undp-gender-equality-strategy-2017-2020.html.
UNESCO. (2015). Complex formula: Girls and Women in Science, Technology, Engineering and Mathematics in Asia. UNESCO Bangkok
Office: The United Nations Educational, Scientific and Cultural Organization.
UNESCO. (2017). Cracking the code: Girls’ and women’s education in science, technology, engineering and mathematics (STEM).
France: The United Nations Educational, Scientific and Cultural Organization.
Weinburgh, M. (1995). Gender differences in student attitudes toward science: A meta-analysis of the literature from 1970 to
1991. Journal of Research in science Teaching, 32(4), 387-398.
Wright, B. D., & Linacre, J. M. (1994). Reasonable mean-square fit values. Rasch Measurement Transactions, 8(3), 370.
Zwick, R., Thayer, D.T., Lewis, C. (1999). An empirical bayes approach to Mantel-Haenszel DIF analysis. Journal of Educational
Measurement, 36, 1, 1-28.
Received: May, 2019 Accepted: September 10, 2019
Rahmi Qurota Aini B.Ed, Master Student, Division of Science Education, College of
Education, Kangwon National University, 1 Kangwondaehak-gil,
Chuncheon-si, Gangwon-do, 24341 Republic of Korea.
E-mail: qurota.rahmi@gmail.com
Arif Rachmatullah M.Ed, Doctoral Student, Department of STEM Education, North
Carolina State University, Raleigh, North Carolina 27695, USA.
E-mail: arachma@ncsu.edu
Minsu Ha PhD., Associate Professor, Division of Science Education, College
(Corresponding author) of Education, Kangwon National University, 1 Kangwondaehak-
gil, Chuncheon-si, Gangwon-do, 24341 Republic of Korea.
E-mail: msha@kangwon.ac.kr
667
DICHOTOMOUS KEYS IN THE
BOTANICAL LEARNING OF NON-
ISSN 1648-3898 /Print/ VISUAL (BLIND) PEOPLE
Abstract. Recently, the research about Branko Andjić,

innovative approaches in the education of Stanko Cvijetićanin,
blind (non-visual- NV) students has intensi- Simon Hayhoe,
fied; however the use of dichotomous keys Rade Grujičić, Danijela Stešević
(DK) in their botanical education has not
yet been examined. This research explored
the contribution of two self-generated
dichotomous keys (DK) for plant identifica-
tion (a digital version- DDK and a printed
version- DPK), to the botanical education
of NV students. The research included
100 students, with an average age of 24,
divided into 2 groups. Group E1 determined Introduction
the plants using the DDK while in Group
The report by the World Health Organization, “Universal Eye Health”,
E2 plants were identified using the DPK.
highlights the need for more intensive research in order to improve the
The DDK contributed more than the DPK in
quality of life and education of NV people (WHO, 2013). The fact is that only
helping the NV participants to acquire the
a small number of NV people (i.e. people born without sight and taught
quality and durability knowledge they need without visual references in school) are provided with a contemporary
to identify different plant groups (woody, education, as it requires significant material resources (Hashemi et al., 2017;
bushy and herbaceous plants). The NV has Livingston, McCarty, & Taylor 1997; Lamichhane, 2016, 2017). Education in
the greatest success in the identification the field of Natural Sciences is one of the main challenges in the education of
of woody plants, and the least success in NV people (Fraser & Maguvhe, 2008). Across a large number of studies, it has
the identification of herbaceous plants. been concluded that with the help of assistive educational technology, NV
The members of both groups (E1, E2) had students can achieve the same quality of knowledge in different natural sci-
possitive opinion about the contribution ences as students who have no visual impairment (Freire, Linhalis, Bianchini,
of the applicable dichotomous key to their
Fortes, & Pimental, 2010; Rice, Aburizaiza, Jacobson, Shore, & Paez, 2012).
In the biological education of NV students, it is especially demanding
knowledge, the activities in it. Due to this
to adapt the educational content in topics related to biodiversity, because
fact, both versions of DKs are recommend-
so much of it is perceived visually. By contrast, the biodiversity education
ed as new assistive tools in the botanical
of NV students should be based on touch, hearing and smell (Smith, 1998;
education of NV students. Smith, Polloway, Patton, & Dowdy, 1998). In the case of NV people, the ab-
Keywords: botanical education, plants sence of the visual sense is compensated for by the better development of
identification, dichotomous keys, non-visu- other senses, primarily touch, hearing and smell (Morin-Parent, Beaumon,
al people, quasi-experimental design. Théoret, & Lepag, 2017). Therefore, they acquaint themselves with their
environment only partially, including in their approach to biodiversity.
Research has shown that insufficient knowledge of biodiversity has the
Branko Andjić following consequences: the incomplete interaction of NV people with the
University of Montenegro, Montenegro environment; a negative impact on mental health and social skills, as well
Stanko Cvijetićanin
University of Novi Sad, Serbia as limiting self-confidence (Binns et al., 2012). In order to mitigate these
Simon Hayhoe consequences, it is necessary to allow NV students to perceive their environ-
Bath University, United Kingdom ment based on their available sensory experiences, insofar as it is possible.
Rade Grujičić, Danijela Stešević
University of Montenegro, Montenegro The most common ways to educate NV people about biodiversity
668
ISSN 1648–3898 /Print/ DICHOTOMOUS KEYS IN THE BOTANICAL LEARNING OF NON-VISUAL (BLIND) PEOPLE
(P. 668-680)
are the verbal-textual methods, the application of models and use of sensory gardens. Some researchers have
pointed out that the acquisition of knowledge by the verbal-textual method causes verbalism in NV students. They
interpret the content, but do not understand it because they have not adopted it through a sensory experience.
They partially understand the concepts which explain the biodiversity surrounding them (Andersen, Dunlea, &
Kekelis, 1993; Andersen, Dunlea, & Kekelis, 1984). By applying three-dimensional models in the teaching of NV
people, the effects of verbal-textual methods are partially but not entirely removed.
Sensory gardens for NV learners are places which are specially adapted to NV visitors, through providing
the maximum sensory stimulation, allowing them to encounter every part of garden by exploring them with
their senses. One of the main aims of sensory gardens is the education of NV people and their preparation for
task solving in everyday life. Through visits to sensory gardens, NV people are able to experience the richness of
various tactile, fragrant and listening experiences, enabling them to explore, identify and understand their sur-
roundings (Chawla & Heft, 2002; Mount & Cavet, 1995). The consequence is positive effects on the psychological
and social well-being of the NV individual (Hussein, 2017). Söderback et al. (2004) pointed out that horticultural
therapy and staying in nature increase the emotional, cognitive well-being, sensory functioning and the social
inclusion of NV people. Due to the fact that sensory gardens are mostly concentrated in large cities and near major
health rehabilitation centers, they are barely available to NV people who do not live in their immediate vicinity.
A review of the previous research indicated that the contribution of dichotomous keys (DK) for plant
identification to knowledge on biodiversity has been tested only on students without any visual impairment.
Recent studies (Anđić, Cvijetićanin, Maričić, & Stešević, 2018; Knight & Davies, 2014) have confirmed the positive
contribution of DK to knowledge on biodiversity. Some researchers have suggested that NV students can achieve
the same quality of knowledge in different natural sciences, as students who have no visual impairment (Freire
et al., 2010; Rice et al., 2012), which forms the basic idea of this research. Thus, one question arises: If DKs make
a positive contribution to the knowledge of people without visual impairment, will they also make a positive
contribution to the environmental education of NV people when using the example of plants?
The aim of this research was to determine the relation between the contribution of the deliberately gener-
ated DKs and the quality and durability of the botanical knowledge of the NV participants, being needed for
plant identification (recognizing and naming), as the basis for the sensory exploration of plants. In addition,
within this aim, this research hoped to further determine the opinion of the NV participants on the impact of
the applied DKs, thus it examined:
1. The similarities and differences in the quality and durability of NV knowledge in identifying plants from
different groups (herbaceous, bushy and woody).
2. The opinions of NV participants on the contributions of the specific DK used to:
•• the knowledge they need to identify plants;
•• their desire to learn about plants from their surroundings and the wider environment;
•• the application of the acquired knowledge in everyday life;
•• the application of the acquired knowledge in the biodiversity education of the NV.
The basic hypothesis of this research was that both the generated DKs could be used as new assistive tools
in the botanical education of NV students.
It was assumed that due to the use of educational software with speech technology in the DDK, the NV par-
ticipants would acquire the better quality and longer-lasting knowledge that they need for the identification of
plants, and that they would have a more positive opinion on its application, compared to those NV participants
who learned using the DPK (a DK printed in Braille).
General Background
The quasi-experimental design was used in the research. It was realized on the basis of an experiment with
parallel groups over a period of 18 months (January 2017 to June 2018), and had two main focuses: 1) the con-
tribution of the use of DKs to the quality and durability of the botanical knowledge of the NV participants; 2) the
opinion of the NV participants about the applied DKs.
669
(P. 668-680)
Participants
The research included 100 NV participants whose median average age was 24, from Montenegro and Austria.
The minimum sample size was determined by G* power program, following the inputs: one-tailed t test, the effect
size of d = 80, desired power of .80, the error rate of .05. Results indicated that the minimum number of participants
was 45 per group. The final size of the sample was determined on the basis of previous research that examined DK
efficiency but which examined students and enthusiasts without visual impairment, and research in the field of the
science education of NV students, as well as on the basis of the number of NV who wanted to voluntarily participate
in the research and general recommendations for sample size in educational researches (Cohen, Manion, & Mor-
rison, 2008). All the participants had a visual acuity of less than 3/60 and a narrowing of the field of view of 10° for
their better visual eye (Yang et al., 2016). The demographic characteristics of the sample is presented in Table 1.
Table 1. Demographic characteristics of the sample (Total N=100).
Variables Type N %
Male 57 57
Gender
Female 43 43
Less than 7 1 1
7-11 12 12
11-15 14 14
15-19 32 32
Age (years)
19-25 10 10
25-30 17 17
30-40 5 5
40-50 9 9
The NV were divided into two groups (E1 and E2), which were equal in the number of NV people (each group
had 50 NV people), and which were based on the number of plant species that they could identify on the PRT.
Approval of and agreement for this research was provided by each institution from which NV participants
were engaged, including schools, universities and societies. The institution managers and administrative staff,
the parents of minor participants, and all the participants themselves were made familiar with the research and
procedures within it. All participants were included on a voluntary basis. The anonymity of all participants and
confidentiality were guaranteed. All participants were reminided of the guarantees regarding confidentiality and
anonymity at every stage of the research process, and were sought permission to record questionnaires and use
the questionnaire data.
Research Design
The research was divided into the following phases:

1. Questionnaire 1- semi‑structured questionnaire in oral form (adapted to the NV) was used to
examine the opinions of the NV participants about plant species that could be identified on the
basis of the sensory perception of plants. Moreover, one aim of this questionnaire was to examine
the way in which the NV participants had learnt about plants prior to their involvement in this
research.
2. The level of prior botanical knowledge of each NV participant was assesed by using a non-stand-
ardised pre-test (PRT), which was based on the sensory perception and identification of fresh
material of plant species listed in Questionnaire 1.
3. Descriptions of the morphological plant characteristics by NV participants – these descriptions were
obtained by giving each NV student the fresh material of one plant species, to be used to examine
the plant based on the senses of touch, smell and hearing. The NV participant then described all
the reproductive and vegetative plant organs without being required to identify the plant species.
670
(P. 668-680)
The descriptions were recorded using an ICD-UX560 Dictaphone. On average, each NV participant
took around 15 minutes to study the morphological characteristics of the one plant species, but
the time for giving their description was not limited. Each NV participant gave a description of all
one hundred plant species. This phase of the research lasted 9 months, because the species did
not belong to the same aspect of flora. Some of them flower/produce reproductive structures in
spring, some in summer, and some in autumn.
4. Selecting the morphological characteristic of plant for DK – in the creation of the DK, was only
used the morphological characteristic of plants which were described by NV participants as the
basis for sensory perception and which had a scientific relevance. To begin with, all the descriptions
were transcribed and after that coded separately. The coding method used a grounded theory
approach (Glaser & Strauss, 1967; Strauss & Corbin, 1990) and was similar to the coding method
used in other studies in teaching biology to blind students (Fraser & Maguvhe, 2008). Codes with
a frequency of greater than 85 (out of 100) were used to create the content of both DKs.
5. The creation of particular DKs (the DDK and the DPK) by using specific taxonomical markers (the
morphological characteristics of plants) obtained in the previous phase of the research. The DDK
was created using education software and speech technology, while the DPK was printed in Braille.
6. Formation of groups – The NV participants were divided into two groups (E1 and E2). The groups
were equal according to NV knowledge on the PRT and in terms of the number of participants.
7. Implementation of DDK and DPK in plants determination - The determination lasted for three weeks,
consisting of 8 periods of 60 minutes of teaching. Each NV participant independently determined
the plants using the relevant DK and fresh plant species.
8. The opinion of the NV participant about the used DK – structured questionnaire (Questionnaire 2) in
oral form (adapted to the NV) was asked with the aim of examining the opinion of the NV students
on the DK which they used for plant determination. The intent was to examine the opinions of NV
participants about the contribution of the applied DK to the knowledge they needed to identify
plants; their motivation to learn about plants, and the opinions of the NV participants about the
implementation of the DDK and the DPK in the botanical education of NV students.
9. The examination of the new knowledge of the NV participants – the new knowledge about plant
identification was examined using a non-standardised post-test (POT). It was realized immediately
after finishing the phases of the implementation of the DDK and the DPK in plant determination.
10. Knowledge durability of NV participants – this was examined through a non-standardised re-test
(RET)- which was realized two months after finishing the phases of the implementation of the DDK
and the DPK in plant determination.
Approach
In Group E1, the NV identified plants using a DDK and in E2 they used a DPK. Both groups performed the deter-
mination in a natural environment (parks and walking grounds). In the first round of determination, the researcher
identified one species using the DK with each NV participant, in order to demonstrate to them the basic principle
of the function of the DK that was used. After that, the NV received the fresh plant material of the next plant from
the researcher and independently performed the determination using their specific assigned DK.
In determining, the plant species which have major morphological details and vegetative, reproductive organs
were determined first, gradually shifting to smaller plant species. The accuracy of the determination was checked
by the researcher. When a NV person accurately identified the plant, the researcher led them to a location where
the plant grew in its natural habitat, so that the NV person could complete a mental image of the environment in
which the plant species were growing.
Description of the DDK and the DPK
The DDK and the DPK were created by the researchers in three languages (English, German and Serbian) and
had the exact same content (100 plant species from the environment of the NV participants). The one-hundred
plant species were selected on the basis of the various plant species about which NV students learn at inclusive
pre-university level in Austria and Montenegro, as well as the fact that a similar number of species was used in
671
(P. 668-680)
previous similar research with students and enthusiasts without visual impairments. The selected plant species
grow in the environment of the NV participants in both Austria and Montenegro.
The keys hold the names of the plants which are used in everyday life, as well as their Latin names. In the DDK
(Figure 1), educational software with speech technology was applied, while the DPK contents from the DDK were
presented with the text in Braille. In both DKs, determination is performed on the same principle. The NV gradually
turn from one claim to another. The claims describe the morphological characteristics of the given plant species,
moving from the general to the specific characteristics of the plant. At the end of the determination, the NV are
focused on a claim which summarizes the properties of the plant from the previous claims and designates the plant.
Figure 1. Example of simple determination through the DDK.
Research Instruments
A testing technique was used to analyze knowledge. Testing was carried out in the environment in which
the NV were identifying the plants. On all tests, the NV were supposed to identify the plant species based on the
sensory perception of fresh plant material. On the PRT, participants were supposed to identify only those plant
species which most of the NV study group claimed that they were able to identify (four plants) in Questionnaire
1. Based on those responses, the PRT had four questions. The results of the PRT represented one of the criteria for
equalizing the groups. The POT and the RET consisted of a total of 24 questions. Eight questions related to the
identification of woody, eight dealt with shrubs and eight concerned herbaceous plants. On the POT and the RET,
the participants were supposed to identify the same plant species from different groups of plants. In the selection
of plants, we considered all the plants that the majority of NV participants had identified in Questionnaire 1 and
those plants which are most common in the natural environment of the NV participants. The examination of the
participants’ knowledge was the same for all the tests, and was carried out according to the following principles:
•• Every participant received one example of a fresh plant species, which was provided by the researchers.
The NV participant was supposed to identify the plant species based on sensory exploring.
•• On average, the NV participants took around 3 minutes to identify one plant species. The time allowed
for the POT and the RET was two school classes (90 minutes) each. In this process, it was considered
that all the NV participants had enough time for sensory plant identification. The PRT lasted only one
school class.
•• The NV participants answered in oral form. Their answers were checked by the researchers and they
were written on the record sheet.
•• Each answer provided by the NV participants was evaluated as either correct or incorrect because
theywere only supposed to identify (name) the plants.
All the tests used in this research were non-standardized because of the lack of standardized test for testing
672
(P. 668-680)
the knowledge of NV people in terms of plants identification. Both questionnaires were in oral form adapted to the
NV (Ratanasukon, Tongsomboon, Bhurayanontachai, & Jirarattanasopa, 2016). Questionnaire 1 had 10 items with 5
questions and it examined the way in which NV participants had learned about plants prior to the realization of this
research, as well as the plant species that they could identify. Questionnaire 2 had twenty items and four blocks of
questions (in total 16 question). In the first block of questions, opinions about the contribution of the applied DK to
the quality of the knowledge needed to identify the plants on the part of the NV participant that was examined. In
the second block of questions, their opinions about the activity in the applied DK were examined. In the third block
of questions, their opinions about the impact of the applied DK on their motivation to learn about plants from their
surroundings and beyond, as well as the impact of the knowledge acquired through the DK on their everyday life
were examined. In the fourth block of questions, the NV were supposed to give their opinion on the possibility of
using the DK in the further botanical education of other NV. The questions in Questionnaire 2 were open, ordinal
(using the Likert scale of assessment) and combined in type. The Likert scale featured five points: (1 = I don’t have
an opinion, 2 = Disagree, 3 = Partially Disagree, 4 = Agree, 5= Agree Strongly). The questions were read to the NV
participants as well as the available answers (to questions of an ordinal or combined type) and clarified if that was
necessary. The NV participants answered the questions orally and the researchers wrote these on the record sheet.
Data Analysis
The contribution of the DDK and the DPK to the quality and durability of the knowledge of the NV in Groups
E1 and E2 was measured based on the number of accurately identified plant species on the POT and the RET. The
difference and similarity in knowledge between the groups on the PRT, POT, or RET, were obtained using a non-
parametric Mann-Whitney U test and an independent t test. Also, the Mann-Whitney U test was used for analyzing
differences in opinions among NV participants between Groups E1 and E2 on the questionnaire, the Kolmogorov-
Smirnov and Shapiro-Wilk tests were used to test the normality of whether the obtained data on the three tests
corresponded to a normal distribution. To determine the difference in knowledge between the POT and the RET
within one group, the Wilcoxon test was used. The analysis of the questionnaire was performed by exploratory
factor analysis, principal component analysis (the Barlett sphericity test and the Kaiser-Meyer-Olkin test -KMO) and
Varimax rotation. The internal consistency of the factors in the questionnaire was computed using the Cronbach
Alpha test (α).
Research Results
The results of Mann–Whitney U test, indicate (U = 2469.000; Z = -2.856; p = .936) that there was no sta-
tistically significant difference between the NV participants in Groups E1 and E2 in the claims about the method
of learning about plants before the realization of this research. In Questionnaire 1, all the NV participants claimed
that they rarely learned about plants on the basis of their personal sensory experience. The NV participants learned
about plants from books (E1: 53%, E2 55%); from family / friends (E1: 21%; E2: 23%); from the media (E1: 18%, E2: 14%)
and from personal experience (E1: 8%; E2: 7%).
The Knowledge of NV participants in terms of Identifying Plants before using the DDK and the DPK
Most NV (E1: 90%; E2: 92%) claimed that they could identify 4 plant species: the common daisy (Bellis perennis
L.), the dandelion (Taraxacum officinale L.), the pine (Pinus sp.) and the platan (Platanus sp.). 17% of the NV in the
whole sample correctly identified only one plant (Pinus sp.), while the other plants listed in Questionnaire 1 were
not identified on the basis of the fresh plant material provided. The Mann-Whitney U nonparametric test found
that there was no difference in the knowledge of the NV in Groups E1 and E2 in relation to the indicated plants
(U = 3286.000; Z = -4.228; p = 1.717). This was confirmed by the independent t-test (t = 8.236; df =99; p = 1.336).
The Knowledge of NV participants in terms of Identifying Plants immediately after the application of the
DDK and the DPK
The values of the Kolmogorov-Smirnov normality test and the Shapiro-Wilk test showed that the obtained
data did not have a normal distribution (see Table 2).
673
(P. 668-680)
Table 2. Tests of normality of distribution in Groups E1 and E2 on the POT and the RET.
Kolmogorov-Smirnov Shapiro-Wilk
Test Group
D df p W df p
POT E1 .188 50 .002 .985 50 .002

E2 .158 50 .002 .9623 50 .005
RET E1 .189 50 .003 .915 50 .003
E2 .123 50 .002 .948 50 .002
The difference in the knowledge of the NV participants between Groups E1 and E2 was used to determine the
results as an independent t test. In Group E1, the NV participants identified 21 out of 24 plant species, while in Group
E2, they identified exactly half (12 out of 24). The difference in the number of identified plants was as follows: 2
woody pants (t = 3.971; df =99; p = .0001); 3 shrubs (t = 3.126; df =99; p = .0001) and 4 herbaceous plants (t = 4.023;
df =99; p = .0001). The NV in Group E1 were more successful in identifying the fir (Abies alba Mill.), the spruce (Picea
abies (L.) Karst), the hawthorn (Crataegus monogyna Hawthorn), the yew (Taxus baccata L.), the oleander (Nerium
oleander L.), the broadleaf plantain (Plantago major L.), the narrowleaf plantain (Plantago lanceolata L.), St John’s-
wort (Hypericum perforatum L.), and white clover (Trifolium repens L.).
The Knowledge of NV participants in terms of Identifying Plants two months after the application of the
DDK and the DPK
Both the Kolmogorov-Smirnov normality test and the Shapiro-Wilk test, shown above in Table 2, demonstrated
that the obtained data did not have a normal distribution. Group E1 showed more durable knowledge and man-
aged to identify 17 out of the 24 species. Group E2 identified 8 out of 24 species. The differences in the number
of identifiable plants between Groups E1 and E2 was as follows: 2 woody (t = 3.759; df = 99; p = .0001); 2 shrubs (t
= 4.823; df = 99; p = .0001) and 5 herbaceous plants (t = 4.129; df = 99; p = .0001). The NV in Group E1 were more
successful in identifying the fir (Abies alba Mill.), the hornbeam (Carpinus orientalis Mill), the hawthorn (Crataegus
monogyna Hawthorn), the yew (Taxus baccata L.), the broadleaf plantain (Plantago major L.), the narrowleaf plantain
(Plantago lanceolata L.), St John’s-wort (Hypericum perforatum L.), slag (Malva sylvestris L.), and white clover (Trifolium
repens L.). The results of the Wilcoxon test, Table 3, show the existence of differences in the number of accurately
identified plants from all groups of plants on the post-test and re-tests within Groups E1 and E2.
Table 3. The difference in the knowledge of the NV participants between the POT and the RET in each group,
the Wilcoxon test.
Group E1 Group E2
Willk λ F p Willk λ F p
Woody .745 .082 .017 .835 .071 .012

Shrubs .972 .335 .014 .865 .224 .009
Herbaceous .791 .213 .009 .887 .119 .018
Total .795 .238 .019 .823 .158 .023
The Opinions of the NV participants about the Applied DKs
Exploratory factor analysis, principal component analysis (KMO = 740; Barlett sphericity test = 437.205; df =
66; p = .000) and Varimax rotation, demonstrated that there were four latent factors that explain 72.61% of total
variance. For further analysis, four specific factors were taken: Factor 1: The opinion of the NV on the contribution
674
(P. 668-680)
of the applied DK to their knowledge, needed to identify the plants (that explain 19.62% of total variance); Factor
2: The opinion of the NV on the activity in the DK(that explain 18.15% of total variance); Factor 3: The opinion of
the NV on the impact of the applied DK on their motivation to learn about plants and the importance of applying
this knowledge in their daily lives (that explain 17.46% of total variance) and Factor 4: The opinion of the NV on
the opportunities for using the DKs in their botanical education, (that explain 17.38% of total variance). The range,
average values, dispersion of results and Cronbach Alpha coefficient by factors are presented in Table 4.
Table 4. Range, average values, dispersion of results and Cronbach Alpha coefficient by factors.
Factor N Mmin Mmax M SD α
The opinion of the NV participants on the contribution of the applied DK to their

100 2 5 4.255 .682 .84
knowledge, needed to identify the plants
The opinion of the NV participants on the activity in the DK 100 1 5 3,561 .713 .81
The opinion of the NV participants on the impact of the applied DK on their
motivation to learn about plants and the importance of applying this knowledge in 100 1 5 3.228 .801 .81
their daily lives
The opinion of the NV participants on the opportunities for using the DKs in the
100 1 5 3.11 .905 .86
botanical education of NV students
The difference in opinions among NV participants between Groups E1 and E2 was confirmed by the Mann-
Whitney U test: the contribution of the applied DK to their knowledge, needed to identify the plants. (U = 1726.000;
Z = -4.255; p = .001); activities in the DK (U = 2044.000; Z = -5.111; p = .001); the desire to learn about plants in the
future using the DK (U = 1768.000; Z = -4.052; p = .000) and the implementation of the relevant DK in the botanical
education of NV students (U = 1556.000; Z = -3.859; p = .001). The difference in the opinions of the NV participants
in Group E1 and Group E2 was also confirmed by the percentage of NV participants who selected the option I fully
agree in answering questions using the Likert scale, (Figure 2).
Figure 2. The difference between Groups E1 and E2 in the choice of the claim I fully agree on the impact of the
DK.
675
(P. 668-680)
Discussion
The tactile sense of the NV individual was the primary source for the collection of information on the basis
of which the plants were identified. In terms of the process among NV participants in the identification of plants,
it can be concluded that identification does not begin by defining the shape and size of the plant, but rather with
the sensations of touching. Using the example of the leaf, the first reported feature is the surface (bare, hairy,
flaky rough or fine, very rough, and so on). Then they determined the firmness (soft, medium, hard plastic), thick-
ness (thin, medium, thick), the nature of the edge of the leaf (unstressed, toothed, wavy) and finally its shape.
The senses of smell and hearing helped them collect information to identify a small number of plants. Thus, for
example, the sense of smell helps in the identification of aromatic plants rosemary and mother’s thymus. Further-
more, Poison ivy (Hedera helix L.) was identified by the characteristic “blunt” sound of the fruits when touching.
Most NV could not identify any of the plant species they stated on the Questionnaire 1, based on fresh
herbal material, before using the specifically generated DK. A small number of NV identified only one plant
species (pine) from the group of woody plants. These data indicate that prior to the determination by means
of the DDK and the DPK the NV (PRT) were primarily taught about the plants from the environment verbally,
without sufficient sensory experience, which led to the appearance of verbalism (Greenaway & Dale, 2017). This
assumption is in line with the responses received from the NV in the survey. Their main source of knowledge
about plants was books, while the sensual (personal) experience had little influence.
Immediately after determining with the DDK and the DPK, the NV on average acquired the good knowl-
edge they needed to identify the woody, or shrub plants, and significantly less knowledge for the identification
of herbaceous plants. This can be explained by the grating orientation thresholds and its value of 0.96mm for
NV (Alary et al., 2009). The morphological details of the woody and bushy plants are larger and beyond their
threshold range, so that they heard the touch of the NV participants. The herbaceous plants are generally smaller
in size, have finer morphological features comparing to woody and scaly plants. In both groups, most NV claim
that the easiest to identify were plants of larger dimensions, and the most difficult plants of smaller dimensions,
as well as plants that have similar structures.
A possible reason for the easier identification of the woody and shrubs plants is that the NV had heard
most about these plant groups in their everyday life, but had not had the opportunity to investigate them. This
probably caused internal motivation to experience them sensually and learn more about them. Most NV in both
groups claim that during the determination they placed more attention and focused on the plants they had
heard or read about because they wanted to find out more about them. The results obtained are in correlation
with the research by Stagg and Donkin (2016) in which enthusiasts without visual impairment participated. In
that research, for most enthusiasts, the woody plants were more engaging in terms of their determination by
the generated DK, and relatively easy to identify. As one of the reasons for these results, these plants are more
popular among enthusiasts than other plants.
Two months after the DDK and DPK determination, the NV participants, on average in both groups, achieved
poorer results when identifying plants compared to the immediate POT. When comparing the quality of the
knowledge of the NV related to the group of plants, it was noticed that the NV had forgotten some plant species
from all the groups of plants. It is assumed that one of the reasons for forgetting is the effect of the active and
passive forgetting process, as well as the non-repeating of plant content between the POT and the RET. The pos-
sible reason for the fact that some plant species were more quickly forgotten was that these plants did not leave
powerful imagery ratings for the NV participants; they were not interesting, attractive during the determination
process. This assumption is correlated with neuroscience studies that have examined long-term memory func-
tions in NV. In these studies, it was concluded that there is no difference in the durability in memory between
NV and those without visual impairment. NV memorize objects and environments that left them with a strong
sensory rating for a longer period of time than those which left a weaker imagery rating, unless these imagery
ratings are based on the visual sense (Beni & Cornoldi, 1988; Zimler & Keenan, 1983).
All the NV identified all the plant species listed in Questionnaire 1 on the POT and the RET, even though
they could not identify them on the basis of fresh plant material on the PRT. On average, the NV were not suc-
cessful on either the POT or the RET in the identification of similar plant species; for example, they confused
the identification of the white clover (Trifolium repens L.) and the red clover (Trifolium pratense L.) as well as the
676
(P. 668-680)
species from the group of grasses. The small dimensions of these plants and similar structures probably caused
the confusion of the NV during the identification. The sense of hearing and smell was not helpful in these cases,
because both plants have neither a distinctive odor nor create a distinctive sound when touched. Similar research
was undertaken by Stagg and Donkin (2016), which involved students with no visual impairments. The students
who used the DK could not recognize similar plant species due to their low visibility.
In the opinion of most NV in both groups, both generated DKs contributed (“fully agree”) to improving
their knowledge of plants. They consider that the generated DK which they applied was an innovative way to
learn about plants in the environment. The possible reason for unanimity in this answer is that the systemati-
cally explored the plants for the first time through the application of the relevant DK, which probably caused a
positive opinion about DKs in general. With the generated DK the NV were enabled to replace verbal learning
with research learning through the observation of plant species using their tactile, smell and hearing senses.
The NV in both groups fully agreed with the claim that the generated DK which they used would be suitable for
the botanical education of other NV. According to the opinion of the NV in Group E1, the DDK could be easily
be applied in the botanical education of NV students in elementary and secondary schools. The NV in Group E2
considered that the DPK would be more successful in secondary education than in primary education.
The reason for the greater contribution of the DDK in comparison to the DPK to the knowledge of the NV
needed to identify plants is probably the fact that DDK uses education software with speech technology. This
made it possible for the NV in Group E1 to actively communicate with the tablet through the right use of the
touch screens, while listening to what they touched, flipping through the contents of the screen while listening,
starting again from what they had heard last. This, among other things, made it possible for them to quickly
determine the plants, unlike the NV participants in Group E2, who had to read each claim in Braille. This is in cor-
relation with the research by Cassia et al. (2009), in which it was concluded that the use of assistive technology
and educational software contributed more to the quality of the knowledge of NV students and students in
comparison to the traditional way of learning, in which the verbal-textual method dominates. Moving from one
claim to another in the DDK was facilitated by a simple click that influenced concentration (Anđić et al., 2018)
and allowed the NV in Group E1 to focus on the tactile, auditory and smell senses while exploring the plant more
than the NV in Group E2 (Röder et al., 1999). These assumptions are in correlation with many studies in the field
of neuroscience which have shown that NV encode auditory verbal material better than those without visual
impairment, causing greater brain activity and better memory performance (Röder, Rösler, & Neville, 2001; Kujala,
Alho, Paavialinen, Summala, & Näätänen, 1992). Given that the NV using it determined plants more quickly, the
DDK indirectly causes the NV to receive faster feedback about the accuracy of their determination than the NV
in Group E2, which makes it easier to correct errors in their determination process. Continuous feedback has a
motivational impact, stimulating further activity in the determination process, which leads to the transformation
of external to internal motivation (Csikszentmihalyi, Abuhamdeh, & Nakamura, 2005).
A possible reason for the better contribution of the DDK than the DPK is in the means of presenting the
content and activities within the DK. The auditory presentation of the content made the DDK more attractive and
engaging for the NV than a written presentation in the DPK. This assumption correlates with the differences in
opinion of the NV in Groups E1 and E2. For most of the NV in Group E1, learning with the DDK was easy, interesting,
and it would be relatively easy to use the DDK in learning, which is why they wanted to continue to learn about
plants in the future. Most of the NV in Group E2 claimed that their activities in the DPK were tiring after a short
time and that their concentration deteriorated during the determination process. They would use the DPK to
determine the plants in the future, but the determination would take a shorter time and include a longer break.
Most NV in both groups expressed the need to get to know as many plant species as possible in their
immediate surroundings and further afield. Basically, for both groups, the reason for this is the desire for new
knowledge, but also the curiosity of getting to know their surroundings, and also distant areas they have read
or heard about. They consider (“I fully agree”) that the applied generated DK helped them complete their mental
picture of the landscape in their environment through the identification of plants, and that, based on listen-
ing to information on the plant species that grow in other environments; they could imagine other areas more
clearly than before.
When asked ‘’In what way does new knowledge about the plants contribute to your daily life?’’, most of
the NV thought that based on plant identification it would be easier for them to find their way in parks and
677
(P. 668-680)
green areas, which made them feel safer, more independent and encouraged them to stay longer outdoors.
This indicates that both generated DK can minimize the perception of environments as unfriendly from the side
of NV, which is mostly caused by the lack of adaptable teaching and learning materials and negative attitudes
towards them (Odame, Hwedie, Nketsia, Peprah, & Nanor, 2019). The obtained opinions of the NV participants
are in correlation with the results of the research by Söderback et al. (2004) and Johnson (2012), in which they
examined the influence of the sensory garden and gardening on the everyday life of their NV study group
and concluded that gardening and plant exploring encourages individualistic and idiosyncratic creativity and
thinking while offering diverse learning skills that enhance both land management at the domestic level and
environmental interaction.
This research represents a pioneer study exploring the implementation of the generated DKs in the botanical
education of NV students. Both the generated DKs (the DDK and the DPK) are potentially innovative assistive
tools in the botanical education of NV students because they contributed to their knowledge in identifying all
the plants species that they could not identify previously, before using the generated DK. The NV participants
considered that both the generated DKs had helped them to acquire new knowledge that would help them to
more easily orient themselves and feel safer when they are outdoors. The contribution of the DDK and the DPK
to the quality and durability of NV participant knowledge needed to identify different groups of plants (woody,
bushy and herbaceous) was unequal. The way the content was presented to the NV groups in the generated
DK, the DK design, the activities in it, the speed of determination by using the DK, and the speed of obtaining
feedback on the accuracy of the determination not only affect the quality and durability of NV knowledge, but
also the desire (motivation) of NV participants to investigate the plants in their surroundings and the wider en-
vironment by using a DK. Therefore, in the botanical education of NV students, priority should be given to the
DDK as compared to the DPK. The way of learning for NV participants, using both the generated DK alongside
the parallel sensory research of plants on fresh plant species might be used as a new form of explicit instruction,
but it could also be used in creating implicit instructions to help NV students determine and learn about plants.
It is necessary to significantly increase the extent of the research in this area in order to confirm these claims. In
this way, such research would contribute to the adaptation of botanical content to the needs of NV people and
to innovation in biology education for NV students.
Acknowledgements
This research was supported by Council of Europe, Programme Democratic and Inclusive School Culture
in Operation (619259 - GA.DGII, PMM No. 1852).
References
Alary, F., Duquette, M., Goldstein, R., Elaine Chapman, C., Voss, P., La Buissonnière-Ariza, V., & Lepore, F. (2009). Tactile acuity
in the blind: A closer look reveals superiority over the sighted in some but not all cutaneous tasks. Neuropsychologia,
47(10), 2037–2043.
Andersen, E., Dunlea, A., & Kekelis, L. (1993). The impact of input: Language acquisition in the visually impaired. First Lang,
13(1), 23–50.
Andersen, E.S., Dunlea, A., & Kekelis L. S. (1984). Blind children’s language: Resolving some differences. Journal of Child Lan-
guage, 11(1), 645–664.
Anđić, B., Cvijetićanin, S., Maričić, M., & Stešević, D. (2018). The contribution of dichotomous keys to the quality of biological-
botanical knowledge of eighth grade students. Journal of Biological Education, 53(3), 310-326.
Beni, De., R., & Cornoldi, C. (1988). Imagery limitations in totallycongenitally blind subjects. Journal of Experimental
Psychology:Learning, Memory, and Cognition, 14(1), 650–655.
Binns, A. M., Bunce, C., Dickinson, C., Harper, R., Tudor-Edwards, R., Woodhouse, M., … Margrain, T. H. (2012). How Effective is
Low Vision Service Provision? A Systematic Review. Survey of Ophthalmology, 57(1), 34–65.
Cássia, A. C, Monteiro, G. B., Rabello, S., Gasparetto, M. E., & Carvalho, K. M. (2009). Assistive technology applied to education
of students with visual impairment. Revista Panamericana de Salud Publica, 26(1), 148-152.
678
(P. 668-680)
Cohen, L., Manion, L., & Morrison, K. (2008). Research Methods in Education (7th ed.). London, United Kingdom: Routledge Falmer.
Chawla, L., & Heft, H. (2002.) Children’s competence and the ecology of communities: A functional approach to the evaluation
of participation. Journal of Environmental Psychology, 22(1-2), 201–216.
Csikszentmihalyi, M., Abuhamdeh, S., & Nakamura, J. (2005). Flow. In A. J. Elliot & Dweck, C. S. (Eds.), Handbook of competence
and motivation (pp. 598-608). New York, NY, US: Guilford Publications.
Fraser, W. J., & Maguvhe, M. O. (2008). Teaching life sciences to blind and visually impaired learners. Journal of Biological
Education, 42(2), 84–89.
Freire, A. P., Linhalis, F., Bianchini, S. L., Fortes, R. P. M., &. Pimentel, M. G. C. (2010). Revealing the whiteboard to blind students:
An inclusive approach to provide mediation in synchronous e-learning activities. Computers & Education, 54(4), 866–876.
Glaser, B. G., & Strauss, A. L. (1967). The discovery of grounded theory: Strategies for qualitative research. Chicago, IL: Aldine
Publishing Company.
Greenaway, R., & Dale, N.J. (2017). Congenital visual impairment’. In L. Cummings (Ed.), Research in Clinical Pragmatics, (p. 441-469).
Cham, Switzerland: Springer International Publishing..
Hashemi, H., Yekta, A., Jafarzadehpur, E., Doostdar, A., Ostadimoghaddam, H., & Khabazkhoob, M. (2017). The prevalence of
visual impairment and blindness in underserved rural areas: a crucial issue for future. Eye, 31(8), 1221–1228.
Hussein, H. (2011). The influence of sensory gardens on the behaviour of children with special educational needs. Asian Journal
of Environment – Behaviour Studies, 2(4), 77-93.
Johnson, S. (2012). Reconceptualising gardening to promote inclusive education for sustainable development. International
Journal of Inclusive Education, 16(5-6), 581–596.
Knight, K., & Davies, R. S. (2014). Using a Mobile Dichotomous Key iPad application as a scaffolding tool in a museum setting.
Interactive Learning Environments, 24(4), 814–828.
Kujala, T., Alho, K., Paavilainen, P., Summala, H., & Naèaètaènen, R. (1992). Neuronal plasticity in processing of sound loca-
tion by the early blind: an event-related potential study. Electroencephalography and Clinical Neurophysiology/Evoked
Potentials Section, 84(5), 469-472.
Lamichhane K. (2016). Individuals with visual impairments teaching in Nepal’s Mainstream Schools: A Model for Inclusion.
International Journal of Inclusive Education, 20(1), 16–31.
Lamichhane, K. (2017). Teaching students with visual impairments in an inclusive educational setting: A case from Nepal.
International Journal of Inclusive Education, 21(1), 1–13.
Livingston, P. M., McCarty, C. A., & Taylor, H. R. (1997). Visual impairment and socioeconomic factors. British Journal of Oph-
thalmology, 81(7), 574–7.
Morin-Parent, F., de Beaumont, L., Théoret, H., & Lepage, J.-F. (2017). Superior non-specific motor learning in the blind. Scientific
Reports, 7(1), doi: 10.1038/s41598-017-04831-1.
Mount H, & Cavet, J. (1995). Multi-sensory environments: An exploration of their potential for young people with profound
and multiple learning difficulties. British Journal of Special Education, 22(1), 52–55.
Odame, L., Osei-Hwedie, B., Nketsia, W., Opoku, M. P., & Nanor Arthur, B. (2019). University preparation and the work capabili-
ties of visually impaired graduates in Ghana: a tracer study. International Journal of Inclusive Education, 1–18. doi:10.10
80/13603116.2019.1609102.
Ratanasukon, M., Tongsomboon, J., Bhurayanontachai, P., & Jirarattanasopa, P. (2016). The Impact of Vision Impairment (IVI)
Questionnaire; Validation of the Thai-Version and the Implementation on Vision-Related Quality of Life in Thai Rural
Community. PloS one, 11(5), e0155509. doi:10.1371/journal.pone.0155509.
Rice, M. T., Aburizaiza, A. O., Jacobson, R. D., Shore, B. M., & Paez, F. I. (2012). Supporting accessibility for blind and vision-
impaired people with a localized gazetteer and open source geotechnology. Transactions in GIS, 16(2), 177–190.
Röder, B., Rösler, F., & Neville, H. J. (2001). Auditory memory in congenitally blind adults: A behavioral-electrophysiological
investigation. Cognitive Brain Research, 11(2), 289-303.
Röder, B., Teder-Sälejärvi, W., Sterr, A., Rösler, F., Hillyard, S. A., & Neville, H. J. (1999). Improved auditory spatial tuning in blind
humans. Nature, 400(6740), 162–166.
Smith, D. J. (1998). Inclusion: Schools for all students. Albany, NY: Wadsworth Publishing Company.
Smith, T. E. C.,. Polloway, E. D., Patton, J. R. & Dowdy, C. A. (1998). Teaching students with special needs in inclusive settings.
Boston: Allyn and Bacon.
Söderback, I., Söderström, M., & E. Schälander. (2004). Horticultural therapy: the “healing garden”and gardening in rehabilita-
tion measures at Danderyd hospital rehabilitation clinic, Sweden. Pediatric Rehabilitation, 7(4), 245–260.
Stagg, B. C., & Donkin, M. E. (2016). Apps for angiosperms: The usability of mobile computers and printed field guides for UK
wild flower and winter tree identification. Journal of Biological Education, 51(2), 123–135.
Strauss, A., & Corbin, J. (1990). Basics of qualitative research: Grounded theory procedures and techniques. Newbury Park, CA:
Sage Publications.
Weinläder, H. G. (1991). Psychological factors with an impact on orientation and mobility in blind children. European Journal
of Special Needs Education, 6(3), 201–208.
WHO (2013). Draft action plan for the prevention of avoidable blindness and visual impairment 2014–2019. Universal eye health:
a global action plan 2014–2019. Geneva: World Health Organization. Retrieved 11/03/2019 from: https://www.who.int/
blindness/AP2014_19_English.pdf
679
(P. 668-680)
Yang, W.Y., Li, J., Zhao, C. H., . Qian, D. J., Niu, Z., Shen, W.,… & Pan, C.-W. (2016). Population-based assessment of visual impair-
ment among ethnic Dai adults in a rural community in China. Scientific Reports, 6(1). doi: 10.1038/srep22590.
Zimler, J., & Keenan, J.M. (1983). Imagery in the congenitally blind: How visual are visual images? Journal of Experimental
Psychology: Learning, Memory, and Cognition, 9(1), 269–282.
Received: June 06, 2019 Accepted: September 18, 2019
Branko Anđić M.Sc., PhD Student, Faculty of Natural Sciences and Mathematics,
Džordža Vašngtona 88, 81 000 Podgorica, Montenegro.
E-mail: brankoan01@gmail.com
Website: https://me.linkedin.com/in/branko-andjic-409b0513a
Stanko Cvjetićanin PhD, Full-time Professor, Department of Natural Science and
Mathematics (Education), University of Novi Sad, Trg Dositeja
Obradovića 3, 21 000 Novi Sad, Serbia; tel/fax: +38125412030,
E-mail: tozchemy@eunet.rs
Website: http://www.pef.uns.ac.rs/index.php/2015-03-25-09-38-
47/2015-03-27-07-36-28/katedra-za-prirodno-matematicke-nauke-i-
menadzment-u-obrazovanju/24-prof-dr-stanko-cvjeticanin
Simon Hayhoe PhD, Full-time Professor, Department of Education, Bath University,
Claverton Down, Bath Somerset BA2 7AY, UK.
E-mail: s.j.hayhoe@bath.ac.uk
Website: https://researchportal.bath.ac.uk/en/persons/simon-hayhoe
Rade Grujičić M.Sc., PhD student, Faculty of Mechanical Engineering, University of
Montenegro, Džordža Vašingtona bb, 81000 Podgorica, Montenegro.
E-mail: rade.grujicic@yahoo.com
Website: https://www.ucg.ac.me/radnik/900644-rade-grujicic
Danijela Stešević PhD, Associate Professor, Faculty of Natural Sciences and Mathematics,
(Corresponding author) Džordža Vašngtona 88, 81 000 Podgorica, Montenegro.
E-mail: danijela.stesevic@ucg.ac.me
Website: https://www.ucg.ac.me/radnik/170653-danijela-stesevic
680
RECONCEPTUALIZING AND
FIELD TESTING THE SCIENTIFIC
LITERACY FRAMEWORK BY ISSN 1648-3898 /Print/
EXPLORING THE ASPECT ISSN 2538-7138 /Online/
OF SCIENTIFIC LITERACY
IN TURKISH SCIENCE
CURRICULUM
Abstract. The aim of this research was
twofold. The first aim was to revise the exist-
Mustafa Cansiz, ing scientific literacy framework developed
Nurcan Cansiz by Boujaoude in accordance with the
current literature about scientific literacy.
The second aim was to explore how the
aspects of scientific literacy were reflected
in the Turkish science curriculum using the
revised framework. The impact of science
Introduction and technology on environment resulted in
the need for both scientifically and environ-
The education systems all over the world are under the pressure of mentally literate individuals and therefore,
the industrial revolution. In considering what kind of education meets the any framework examining scientific literacy
needs of today, educators should focus on the curriculum to prepare young should include the interaction between sci-
individuals for this purpose. Bayliss (1999, p. 9) argued that “many people in ence, technology, society, and environment
education are realizing that the time for tinkering with the traditional cur- as an aspect. Moreover, a new aspect was
riculum is over”. There is a need for such a curriculum which fosters creativity suggested for the framework, the affective
and imagination as today’s industrial world requires. The skills and knowledge side of science, considering the affective
required for the needs of today should be integrated into the curriculum of outcomes of science education. Finally,
different disciplines. Science education and the skills required for today’s the science curriculum from grades 3 to 8
world have a lot in common (e.g. creativity, productivity, and critical thinking) was analyzed using the revised framework.
which makes science education more critical for educating students to handle Results showed that the revisions on the
the challenges of the day. It has been emphasized that scientific literacy is framework resulted in a more rigorous
one of the characteristics of the citizens of the societies to meet the needs of framework which evaluates scientific lit-
today (Cansiz & Turker, 2011; Choi, Lee, Shin, Kim, & Krajcik, 2011). Although eracy more precisely. Moreover, the results
scientific literacy has been emphasized since the 1950s (Choi et al., 2011), about the analysis of science curriculum
it has multiple meanings (Pearson, Moje, & Greenleaf, 2010). Two visions of indicated that the knowledge of science
scientific literacy have shaped these definitions (Roberts, 2007). One vision and the investigative nature of science were
has focused on understanding key scientific concepts, principles, and facts; reflected in the objectives of the curriculum
the other has emphasized the use of knowledge in situations students face more than other aspects. The results and
in daily life as citizens (Roberts, 2007). implications were discussed.
The developments in science and technology produce controversial is- Keywords: affective domain, document
sues in which people need to make decisions. Most of the controversial issues analysis, science curriculum, science educa-
include an environmental aspect because those developments influence the tion, scientific literacy framework.
environment and its components too. For example, as the nuclear chemistry
and physics develops and technology advances, nuclear power plants have
emerged. Then, the impact of nuclear power plants on environment and
Mustafa Cansiz, Nurcan Cansiz
organisms have begun to take place in the agenda of scholars all over the Artvin Çoruh University, Turkey
world. Such problems require thorough investigation and decision-making
by a variety of stakeholders. The public is also among those stakeholders. This
necessitates that public, as the stakeholders, should be equipped with the
681
(P. 681-691) ISSN 2538–7138 /Online/
knowledge and skills to make informed decisions about social and environmental problems. That is, there exists
a need for both scientifically and environmentally literate citizens in today’s world (Bybee, 2008). Therefore, there
is a need for such a science curriculum which targets to achieve this goal. In deciding whether the curriculum is
competent in preparing scientifically literate individuals, a framework covering all aspects of scientific literacy is
needed. Such a framework might have the capacity to assess the science curriculum in terms of scientific literacy.
Chiappetta, Fillman, and Sethna (1991) proposed four aspects for the construct scientific literacy. BouJaoude (2002)
used Chiappetta et al.’s (1991) study as the base for developing the scientific literacy framework to evaluate the
curriculum. This framework has been widely used in many studies in recent years too (e.g. Cansiz & Turker, 2011;
Erdoğan & Köseoğlu, 2012; Forjan & Sliško, 2017; Kılıç, Haymana, & Bozyılmaz, 2010; Wei & Chen, 2017). However,
it requires modification since the characteristics of scientifically literate individuals as well as the definition of sci-
entific literacy has been shaped by the fast-changing world. Therefore, as being one of the aims of this research,
the framework was reconceptualized and a revised framework was suggested for the use by the researchers to
analyze science curriculum and other science-related documents.
Realizing the fact that science education in primary and middle school has a key role in building a society
with scientifically literate citizens; the science curriculum at these levels of education should have the necessary
elements of scientific literacy (Cansiz & Cansiz, 2019). In a curriculum, there are three main components: objec-
tives, instruction, and assessment (Anderson, 2010). Objectives have been considered as the most specific aspect
of the curriculum as they follow the philosophy, aims, and goals (Behar-Horenstein, 2010). In other words, they act
as the starting point for teaching. Teachers look at the objectives and decide on methods for teaching, classroom
activities, teaching materials and the way to assess the students’ learning. Therefore, the knowledge, skills, and
values emphasized in the objectives should be determined carefully. Since the objectives describe what the learn-
ers are able to do after the instruction, their role in fostering scientific literacy becomes more critical. Being one of
the prime components of the curriculum, if the objectives include the knowledge and skills that are required for
educating scientifically literate individuals, then the curriculum might achieve its purpose. That is, if the objectives
are written in a way that emphasizes the aspects of scientific literacy, then the problems in educating scientifically
literate students might be diminished. Therefore, objectives should be analyzed if they target the intended learn-
ing outcomes in terms of scientific literacy which was the other aim of this research.
Existing Scientific Literacy Framework developed by Boujaoude
Boujaoude (2002) developed the framework to analyze science curricula based on Chiappetta et al. (1991)
and Chiappetta, Sethna, and Fillman’s (1993) work. Chiappetta et al.’s (1991) scientific literacy themes are 1) the
knowledge of science, 2) the investigative nature of science, 3) science as a way of thinking, and 4) interaction of
science, technology, and society.
The first one, the knowledge of science, refers to the content aspect. When the focus is to teach scientific knowl-
edge and make students remember and use that knowledge, it refers to the first theme (Chiappetta et al., 1993).
Chiappetta et al. underlined that this theme covers the transfer of scientific knowledge from teacher to students.
The second theme, namely the investigative nature of science refers to what is known as science process skills.
This theme aims to evaluate whether students are engaged with science process skills and use them actively
(Chiappetta et al., 1993). This is important in that it emphasizes the activities which foster students to reason, to
interpret, and to evaluate; and encourages them to involve activities by exploring scientific knowledge rather than
rote memorization of scientific knowledge.
The third theme in Chiappetta et al.’s framework is science as a way of thinking. The governing idea in this
theme is about the nature of science and scientific knowledge. This theme is particularly important to figure out
how scientists carry out scientific inquiries and use reasoning skills. (Chiappetta et al., 1993). The knowledge em-
phasizing the historical development of scientific knowledge, scientists’ approaches to their work, and the role of
evidence has been sorted under this category (Chiappetta et al., 1993).
The last theme in the framework is the interaction of science, technology, and society. If the scientific endeavor
intends to demonstrate the impacts of science on the public, it is related to this theme. The advantages and dis-
advantages of science and technology on society, cases illustrating the science or technology-related social issues
have been categorized under this category (Chiappetta et al., 1993).
Boujaoude (2002) used Chiappetta et al.’s work as the foundation of his study and adapted the four aspects
of scientific literacy to analyze science curricula. The author made three modifications to the framework. First, he
682
ISSN 1648–3898 /Print/ RECONCEPTUALIZING AND FIELD TESTING THE SCIENTIFIC LITERACY FRAMEWORK BY
ISSN 2538–7138 /Online/ (P. 681-691)
added the issues: the use of science in making every day personal decisions, resolve science-related problems in
daily life, and engage with moral and ethical topics in science to the aspect interaction of science, technology, and
society. Second, the author suggested analyzing the different domains (e.g. general science and physics) of scien-
tific literacy separately defending that one cannot be scientifically literate in all subject areas. Thirdly, he preferred
using science as a way of knowing instead of science as a way of thinking since the former covers the epistemology
of science (Boujaoude, 2002). The framework Boujaoude adapted is given in Table 1.
Table 1. Scientific Literacy Framework developed by Boujaoude (2002).

Aspects of Scientific Literacy
Aspect 1. The knowledge of science:
- Facts, concepts, principles, laws, hypotheses, theories, and models of science
Aspect 2. The investigative nature of science:

- Using methods and process of science such as observation, measuring, classifying, inferring, recording and analyzing data,
Communicating using a variety of means such as writing, speaking, using graphs, tables, and charts, making calculations, and experimenting
- Emphasis on hands-on minds-on science
Aspect 3. Science as a way of knowing:

- Emphasizes thinking, reasoning, and reflection in the construction of scientific knowledge and the work of scientists
- Empirical nature in science
- Ensuring objectivity of science
- Use of assumptions in science
- Inductive and deductive reasoning
- Cause and effect relationships
- The relationship between evidence and proof
- Role of self-examination in science
- Describes how scientists experiment
Aspect 4. Interaction of science, technology, and society:

- Impact of science on society
- Inter-relationships between science, society, and technology
- Careers
- Science-related social issues
- Personal use of science to make everyday decisions, solve everyday problems, and improve one’s life
- Science-related moral and ethical issues
In this research, the existing scientific literacy framework was revised, and the Turkish science curriculum was
analyzed with it. Overall, the following research questions guided this research:
1. Is the existing scientific literacy framework consistent with the current literature about scientific literacy?
2. How does the Turkish science curriculum reflect the aspects of scientific literacy?
General Background
This research utilized document analysis. It is a type of qualitative research method and includes a systemic
practice for assessing printed and electronic materials (Bowen, 2009). This research was conducted during the 2018-
2019 fall semester after the 2018 science curriculum had been released by the Turkish Ministry of National Education.
The Unit of Analysis: The Turkish Science Curriculum at a Glance
In this research, the recent science curriculum in Turkey was analyzed. It was released in 2018 for grades 3 to
8. Science is formally introduced to the students first in 3rd grade in primary school. There is a national curriculum
for all science teachers to follow. Topics in the curriculum have been deepened in each grade with increasing
complexity. At the end of 8th grade, students would take 792 hours of science with 326 objectives. Until the end
of middle school, science is taught including chemistry, physics, and biology by the science teachers. In secondary
school, these three disciplines are taught separately. The recent curriculum includes new trends in science educa-
683
(P. 681-691) ISSN 2538–7138 /Online/
tion as a result of reforms such as socio-scientific issues, science-technology-engineering-mathematics practices,

and values education.
Data Analysis
This research occurred in five steps. In the first step, the authors of this research had reviewed the literature
about scientific literacy and then they evaluated the degree to which the existing scientific literacy framework was
consistent with the current literature about scientific literacy. In the second step, the existing aspect of the framework
was revised, and a new aspect was suggested based on the literature. Third, the two researchers reached consensus
on each aspect of the framework and their descriptions. Moreover, they scrutinized each objective independently
and assigned it to one of the five aspects. In the next step, they compared their analyses and found that there were
28 inconsistencies out of 326 decisions. Therefore, the interrater reliability was calculated as 91.4%. Then, Cohen’s
Kappa (Cohen, 1960) was administered to test the significance of this interrater reliability. The interrater reliability
was statistically significant (Cohen’s Kappa = .78, p < .001). Based on Landis and Koch’s (1977) criteria, the level of
agreement was substantial. In the final step, the researchers focused on inconsistent decisions and negotiated
on them. The process of data analysis was iterated until a convincing agreement was reached on categorization.
And finally, all inconsistencies were resolved, and a full agreement was obtained. The summary of the research
procedure was given in Figure 1.
Figure 1. The summary of the research procedure.
Research Results
The first purpose of this research was to reconceptualize the scientific literacy framework. Therefore, the result
based on the first research question was presented initially. This is followed by presenting result based on the second
research question which was about to examine the objectives of science curricula using the revised framework.
The Revised Scientific Literacy Framework
Based on the recent literature on science education and scientific literacy, it was concluded that some adapta-
tions and additions were required to the original framework to better assess scientific literacy. First, environment
was added to the aspect of the interaction of science, technology, and society. The impact of science and technology
on the environment was emphasized by Science-Technology-Society-Environment approach (Aikenhead, 2003;
Fensham, 1988a, 1988b; Pedretti, 2003; Pedretti & Nazir, 2011). Science and technology do not only influence society
but also have an impact on the environment. One of the major goals of science education is to understand the
interaction between science, technology, society, and the environment. Especially as a result of the advancement
of new technologies, the concerns about environmental issues increased. This led educators to develop such a
science curriculum which fosters individuals to cultivate an understanding of these issues and skills to deal with
684
ISSN 2538–7138 /Online/ (P. 681-691)
environmental challenges. Environmental issues are complex in nature and this requires higher-order thinking
skills such as interpreting, reasoning, evaluating, analyzing, and problems solving. As this issue is getting more
important, the last aspect of the scientific literacy framework is changed as the interaction of science, technology,
society, and environment. Secondly, there existed a need for another aspect which we call affective side of science.
This aspect emphasizes empathy, commitment, interests, attitudes, appreciations, values, and feelings about
science. Education is not only for cognitive outcomes but also for affective outcomes (Stone & Glascott, 1997).
Social and emotional skills such as having empathy, being commitments, having interests and positive attitudes,
emphasizing values and feelings should be included in science curriculum. Science curriculum is key to cultivate
those skills. Therefore, the affective side of science was added to the framework. The final form of the framework is
provided in Table 2.
Table 2. Revised Scientific Literacy Framework.
Aspects of Scientific Literacy Descriptors
Aspect 1: The knowledge of science

- Facts, concepts, principles, laws, hypotheses, theories, and models of science
Aspect 2: The investigative nature of science

- Using methods and process of science such as observation, measuring, classifying, inferring,
recording and analyzing data,
- Communicating using a variety of means such as writing, speaking, using graphs, tables, and
charts, making calculations, and experimenting
- Emphasis on hands-on minds-on science
Aspect 3: Science as a way of knowing

- Emphasizes thinking, reasoning, and reflection in the construction of scientific knowledge and the
work of scientists
- Empirical nature in science
- Ensuring objectivity of science
- Use of assumptions in science
- I nductive and deductive reasoning
- Cause and effect relationships
- The relationship between evidence and proof
- Role of self-examination in science
- Describes how scientists experiment
Aspect 4: Interaction of science, technology,

- Impact of science and technology on society and environment
society, and environment
- Inter-relationships between science, society, technology, and environment
- Careers
- Science-related social issues and environmental issues
- Personal use of science to make everyday decisions, solve everyday problems, and improve one’s
life
- Science-related moral and ethical issues
- Use of science to discuss environmental issues and being involved in the decision-making process
on these issues
Aspect 5: Affective side of science

- Having environmental awareness
- Taking responsible action for the environment and its components
- Being responsible for personal, local, and global issues
- Proposing action plans for social improvement and following with commitment.
- Valuing ethical standards
- Cooperating in group activities (displays teamwork)
Results of the Analysis of Science Curricula with the Revised Scientific Literacy Framework
With the aim of exploring how sufficient the Turkish science curriculum to prepare scientifically literate indi-
viduals, the objectives were examined and categorized based on the revised framework (see Table 2). The results
were organized based on each aspect.
685
(P. 681-691) ISSN 2538–7138 /Online/
Aspect 1: The knowledge of science
This aspect of scientific literacy was matched with scientific facts, ideas, doctrines, regulations, propositions,
models, and prototypes. When the objectives included descriptions, explanations, or model preparations, they
were classified under this aspect. These objectives were observed in the middle school curriculum more. In grade
3, 33.3% (12 out of 36) of the objectives were categorized under this aspect. A typical example of an objective is
explains the basic functions of the five sense organs in human beings. With this objective, students are expected to
explain the facts related to the sense organs’ functions. That is, it emphasizes the knowledge underlying the sense
organs. In grade 4, 26.1% (12 out of 46) of all objectives were assigned to this aspect. This percentage increased
in grade 5 to 37.5% (15 out of 40 objectives), and in grade 6 to 42.6% (26 out of 61 objectives). In grade 7 and 8,
though, it decreased to 41.9% and 32.4%, respectively. The frequencies, percentages, and the total number of
objectives in each grade were presented in Table 3.
Table 3. The frequencies and percentages in each grade for the knowledge of science.
Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8
f % f % f % f % f % f %
The knowledge of science 12 33.3 12 26.1 15 37.5 26 42.6 31 41.9 22 32.4

Total number of objectives 36 46 40 61 74 68
Aspect 2: The investigative nature of science
Generally speaking, the use of science process skills comes to prominence in this aspect. Besides, ways of
communications such as writing, speaking, graphical representations, tables, and illustrations are emphasized.
The objectives in this category were found in all grades with an increasing number from grade 3 to 7, except for
grade 8. In the third grade, 38.9% of all objectives (14 objectives out of 36) emphasized the investigative nature of
science. In these objectives, students were expected to participate actively in the classroom by either doing experi-
ments or hands-on activities. A typical example objective is does experiments to find out how a force acting on an
object causes it to stop or start moving. It is evident that students were expected to carry out simple experiments in
the classroom using science process skills. This certainly goes beyond knowing the subject matter. Students were
expected to transfer their science content knowledge to a particular situation in a different context. The percent-
age, frequencies and the total number of objectives for all grades are given in Table 4.
Table 4. The frequencies and percentages in each grade for the investigative nature of science.
f % f % f % f % f % f %
The investigative nature of 14 38.9 15 32.6 17 42.5 21 34.4 26 35.1 24 35.3

science
Aspect 3: Science as a way of knowing
This aspect is mostly associated with the reasoning in producing scientific knowledge, the nature of science,
and the work of scientists. Moreover, the role of evidence in the development of knowledge is considered within
this aspect. It was not emphasized as much as the aspect 1 and 2 in the science curricula. With 23.9%, it had the
highest percentage in grade 4 and with only 6.8% it had the lowest percentage in grade 7 among others. A typical
objective regarding science as a way of knowing aspect is discusses the opinions about the structure of the cell from
686
ISSN 2538–7138 /Online/ (P. 681-691)
the past to the present in light of the technological developments. Here, students needed to know the history of the
structure of the cell. They also needed to comprehend how scientists’ work, experiments, and discussions have
shaped the cell theory. They also required to understand how technological developments led to the modifica-
tions of the cell theory by making inferences based on evidence. That is, they were expected to consider science
as a way of knowing rather than just a body of knowledge. The percentages, frequencies and the total number of
objectives can be found in Table 5 for each grade.
Table 5. The frequencies and percentages in each grade for science as a way of knowing.
f % f % f % f % f % f %
Science as a way of knowing 3 8.3 11 23.9 5 12.5 6 9.8 5 6.8 10 14.7

Aspect 4: Interaction of science, technology, society, and environment
The interaction between science, technology, society, and environment is also among the aspects which are
not emphasized in the objectives adequately. It includes the dynamic relationship between science, technology,
society, and environment. The impact of science and technology on society and the environment or how the
needs of society shape technology are included in this aspect. It is observed that the science curriculum of grade 8
included more objectives emphasizing this aspect (27.9%). On the other hand, in grade 7, this aspect was reflected
the least (14.9%) among other grades. Two examples of objective regarding this aspect are discusses the reasons
and results of global climate change and gives examples of the innovative applications of solar energy in daily life and
technology. The percentages, frequencies and the total number of objectives are presented in Table 6 for all grades.
Table 6. The frequencies and percentages in each grade for the interaction of science, technology, society,
and the environment.
f % f % f % f % f % f %
The interaction of science, technology,

9 25.0 10 21.7 7 17.5 12 19.7 11 14.9 19 27.9
society, and environment
Aspect 5: The affective side of science
In the final version, we have added a new aspect to the framework. This aspect is associated with having
environmental awareness, taking responsible action for the environment and its components, being responsible
for personal, local, and global issues, proposing action plans for social improvement and following with commit-
ment, valuing ethical standards, and cooperating in group activities. This aspect was seen in the objectives of
science curricula too. That is, 8.3% of objectives (3 out of 36) in grade three, 4.3% in grade four (2 out of 46), 4.1%
of objectives (3 out of 74) in grade seven and 2.9% of the objectives (2 out of 68) in grade eight were categorized
under this aspect. Typical objectives are pays attention to the waste control in a near environment and take care of a
plant or an animal and report its growth. The percentages, frequencies and the total number of objectives can be
found in Table 7.
687
(P. 681-691) ISSN 2538–7138 /Online/
Table 7. The frequencies and percentages in each grade for the affective side of science.
f % f % f % f % f % f %
The affective side of science 3 8.3 2 4.3 0 0.0 0 0.0 3 4.1 2 2.9
Table 8 provides the overall results. The emphasis on each aspect in each grade can be compared easily by
investigating frequencies, percentages, and the total number of objectives.
Table 8. Summary of the results for five aspects of scientific literacy.
f % f % f % f % f % f %
The knowledge of science 12 33.3 12 26.1 15 37.5 26 42.6 31 41.9 22 32.4

The investigative nature of science 14 38.9 15 32.6 17 42.5 21 34.4 26 35.1 24 35.3
Science as a way of knowing 3 8.3 11 23.9 5 12.5 6 9.8 5 6.8 10 14.7
Interaction of science, technology, society, 9 25.0 10 21.7 7 17.5 12 19.7 11 14.9 19 27.9
and environment
Affective side of science 3 8.3 2 4.3 0 0.0 0 0.0 3 4.1 2 2.9
Note: In the curriculum, some objectives focus on more than one aspect of scientific literacy. Therefore, the sum of the number of objec-
tives in each column should not necessarily be equal to the total number of objectives!
To sum up, it is obvious that Turkish science curriculum released in 2018 includes objectives emphasizing the
knowledge and the investigative nature of science aspects with a higher frequency than the other three aspects of
the scientific literacy.
Discussion
The purpose of this research was to revise the scientific literacy framework developed by Boujaoude (2002)
in terms of (1) understanding the interaction of science, technology, society, and environment (2) affective side of
science including empathy, commitment, interests, attitudes, appreciations, values, and feelings. A second purpose
was to analyze the science curricula in Turkey for primary and middle schools.
In this regard, we modified existing aspects of the framework and we also suggested an additional aspect in
order to enhance its capacity to evaluate scientific literacy. The revised aspect underlined the interaction between
science, technology, society, and environment. Khishfe (2014) emphasized raising students both scientifically and
environmentally literate to take an active role in problems that affect the society as a result of the developments
in science and technology. Especially, the rise in environmental problems we face in daily life leads to the need for
education to increase awareness and concern for the environment within society. Such education is required since
people are expected to participate in the decision-making process on issues including the environment. This is one
of the essentials of scientific literacy. However, without having an understanding of environmental problems, their
causes and effects, it is not possible to discuss and propose solutions. Therefore, environmental literacy has been
considered within the construct of scientific literacy (Heiskanen, 2006). Qualified and responsible citizens should
also be aware of the environment and its components to be scientifically literate. The additional aspect was the
affective side of science. This aspect is associated with having environmental awareness, taking responsible action
for the environment and its components, being responsible for personal, local, and global issues, proposing action
plans for social improvement and following with commitment, having values and empathy. The recent definition of
scientific literacy includes the affective dimension. For example, when OECD (2016) describes scientific literacy, the
attitudinal aspect is considered an aspect of scientific literacy. The attitudinal aspect includes using scientific way
688
ISSN 2538–7138 /Online/ (P. 681-691)
of thinking in daily life situations, attentiveness to environmental problems, and attitudes towards science (OECD,
2016). That is, scientific literacy does not only requires having scientific knowledge and skills but also depend on
affective domain outcomes such as attitude, values, awareness, empathy, responsibility, or commitment. Especially,
positive attitudes and behaviors toward the environment have become critical issues in educating students as
scientifically literate since people deal with environmental problems more than ever.
The analysis of the Turkish science curriculum with the revised scientific literacy framework yielded important
results in terms of the potential of the curriculum to educate students as scientifically literate. The aim of the science
curriculum is no longer associated with the achievement rather a broader term scientific literacy is incorporated
into the curriculum. Scientific literacy also covers students’ success in science, but it includes more than knowing
science content knowledge. In order to maximize the potential of the science curriculum in terms of scientific literacy,
it is vital to balance the emphasis on its aspects in the curriculum. Since scientific literacy is a multidimensional
construct, ignoring some aspects or overemphasizing one aspect would definitely be reflected in the educational
outcomes. It will also hinder the accomplishment of the goals of the curricula. This research found that the Turkish
science curriculum lacks the balance in terms of the aspects of scientific literacy. The emphasis is more on the aspects
the knowledge of science and the investigative nature of science. The aspect science as a way of knowing was also not
emphasized as it should be. The last aspect, the affective side of science, was almost neglected in the curriculum
from grades 3 to 8. This creates an important problem since scientific literacy is a whole with its all aspects. Leaving
one aspect behind results in the failure of scientific literacy. As the future citizens of the society, students should
be equipped with the five aspects at least basically because they will engage in science-related issues in their lives
although they will not have science-related careers (Krajcik & Sutherland, 2010). The curriculum that is dominated
by the knowledge of science and the investigative nature of science may not help students to grasp the dynamic
interaction between science, technology, society, and the environment. This is because science is presented as a
body of knowledge and rarely applied to the real world (Pedretti, 2003). The real-world context and issues help
students comprehend the interaction between science, technology, society, and environment (Ramsey, 1993). The
inclusion of issues such as global warming, genetically modified foods, nuclear power plants, and cloning in science
curricula might help students realize how science and technology impact their lives and environment. Instead of
isolating science from the real world, an authentic and contextualized science teaching and learning should be
embraced by the educators, teachers and curriculum developers. It is necessary if the aim is to prepare students
to take an active role in tomorrow’s decision-making process. Moreover, the affective side of science is not usually
given prominence in the curriculum. The outcomes of science education also include educating students as being
emphatic, committed, responsible for the environment, and having values, attitudes, and awareness. The societies
need citizens who have these social and emotional skills in addition to cognitive skills. Therefore, science curricula
should also give emphasis to the affective outcomes.
This research was designed to revise the existing scientific literacy framework based on current scientific literacy
literature. Within the scope of its aim, this research also analyzed the extent to which Turkish science curriculum
reflects the aspects of scientific literacy in a balanced way using the revised framework.
The comparison of the existing framework with current scientific literacy literature showed that the existing
framework was inadequate to reflect the “environment” aspect within its fourth aspect of “Interaction of science,
technology, and society”. Therefore, the name of this aspect was changed as “Interaction of science, technology,
society, and environment” in the revised framework and corresponding descriptors were added. This research also
suggested a completely new aspect to the existing framework as “the affective side of science” with correspond-
ing descriptors considering the affective outcomes of science education. Moreover, the results emerging from the
analyses of the Turkish science curriculum showed that the knowledge of science and the investigative nature of
science were reflected in the objectives of the curriculum more frequently compared to other aspects. That is, the
Turkish science curriculum did not reflect the aspects of scientific literacy in a balanced manner.
The findings from this research make several noteworthy contributions to the current science education lit-
erature. First, improving the original aspects and offering a new one to the existing framework may lead to have a
more rigorous framework which might help researchers to assess scientific literacy more precisely in future research.
This is essential to obtain a more comprehensive understanding of scientific literacy. Scientific literacy does not only
refer to knowledge and skills in science but also includes values and attitudes. Therefore, this research addressed
689
(P. 681-691) ISSN 2538–7138 /Online/
the need for this issue and revised the scientific literacy framework. Second, it provided empirical evidence for the
validity of the revised framework by field-testing it. Although the validity was established by analyzing the Turkish
science curriculum, the framework is not specific to the Turkish context. Therefore, international researchers can
make use of the revised framework to analyze the science curriculum in their countries and inform authorities
about the effectiveness of their curriculum. Third, the results of field-testing provided empirical evidence on the
inadequacy of Turkish science curriculum regarding preparing scientifically literate students. It is a well-known
fact that Turkey scores well below the OECD average in all PISA (The Programme for International Student Assess-
ment) applications which evaluate 15-year-old students’ scientific literacy together with mathematics and reading
literacy. Preparing students as scientifically literate is accepted as the ultimate goal of science education worldwide.
Citizens of any societies should be scientifically literate at a basic level in order to survive in a rapidly changing and
highly competitive world of today. Students should possess the necessary knowledge and skills associated with
scientific literacy. One way to achieve this is to improve science education at K-12 levels. In the process of teaching
and learning science, the aspects of scientific literacy need to be approached holistically. That is, both knowledge
and skill aspects should be included in the curriculum. There is certainly a need for further research on this issue
to explore what other factors, in addition to the curriculum, may influence scientific literacy. As a result, curriculum
developers, policymakers, and educators can make necessary revisions accordingly and might improve science
education which prepares the students for the future.
References
Aikenhead, G. S. (2003). STS education: A rose by any other name. In R. Cross (Ed.), A vision for science education: Responding to
the work of Peter J. Fensham (pp. 59 – 75). New York: Routledge Press.
Anderson, L. W. (2010). Taxonomies of objectives and learning. In C. Kridel (Ed.). Encyclopedia of curriculum studies (pp. 839-841).
London: Sage Publications.
Bayliss, V. (1999). Opening minds: Education for the 21st century. The final report of the RSA project redefining the curriculum. London:
Royal Society of Arts.
Behar-Horenstein, L. S. (2010). Objectives in curriculum planning. In C. Kridel (Ed.) Encyclopedia of curriculum studies (pp. 615-
616). London: Sage Publications.
BouJaoude, S. (2002). Balance of scientific literacy themes in science curricula: The case of
Lebanon. International Journal of Science Education, 24(2), 139–156.
Bowen, G. A. (2009). Document analysis as a qualitative research method. Qualitative Research Journal, 9(2), 27-40.
Bybee, R. W. (2008). Scientific literacy, environmental issues, and PISA 2006: The 2008 Paul F-Brandwein Lecture. Journal of Science
Education and Technology, 17(6), 566-585.
Cansiz, N. & Cansiz, M. (2019). Evaluating Turkish science curriculum with PISA scientific literacy framework. Turkish Journal of
Education, 8(3), 217-236.
Cansiz, M., Turker, N. (2011). Scientific literacy investigation in science curricula: The case of Turkey. Western Anatolia Journal of
Educational Sciences, Special Issue, 359-366.
Chiappetta, E. L., Fillman, D. A., & Sethna, G. H. (1991). Procedures for conducting content analysis of science textbook. Houston, TX:
University of Houston, Department of Curriculum and Instruction.
Chiappetta, E., Sethna, G., & Fillman, D. (1993). Do middle school life science textbooks provide a balance of scientific literacy
themes? Journal of Research in Science Teaching, 30, 787–797.
Choi, K., Lee, H., Shin, N., Kim, S., & Krajcik, J. (2011). Re-conceptualization of scientific literacy in South Korea for the 21st Century.
Journal of Research in Science Teaching, 48(6), 670 –697.
Erdoğan, M. N., & Köseoğlu, F. (2012). Analysis of high school physics, chemistry and biology curriculums in terms of scientific
literacy themes. Educational Sciences: Theory and Practice 12(4), 2899-2904.
Fensham, P. J. (1988a). Approaches to the teaching of STS in science education. International Journal of Science Education, 10(4),
346 – 356.
Fensham, P. J. (1988b). Familiar but different: Some dilemmas and new directions in science education. In P. J. Fensham (Ed.),
Developments and dilemmas in science education (pp. 1 – 26). New York: Falmer Press.
Forjan, M., & Sliško, J. (2017). Simplifications and idealizations in high school physics in mechanics: A study of Slovenian cur-
riculum and textbooks. European Journal of Physics Education, 5(3), 20-31.
Greenberg, M. T., Weissberg, R. P., O’Brien, M. U., Zins, J. E., Fredericks, L., Resnik, H., & Elias, M. J. (2003). Enhancing school-based
prevention and youth development through coordinated social, emotional, and academic learning. American Psycholo-
gist, 58(6/7), 466-474.
Heiskanen, E. (2006). Encounters between ordinary people and environmental science – A transdisciplinary perspective on
environmental literacy. The Journal of Transdisciplinary Environmental Studies, 5(1-2), 1–13.
Khishfe, R. (2014). A reconstructed vision of environmental science literacy: The case of Qatar. International Journal of Science
Education, 36(18), 3067-3100.
690
ISSN 2538–7138 /Online/ (P. 681-691)
Kılıç, G. B., Haymana, F., & Bozyılmaz, B. (2010). Analysis of the elementary science and technology curriculum of Turkey with
respect to different aspects of scientific literacy and scientific process. Education and Science, 33(150). 52-63.
Krajcik, J. S., & Sutherland, L. M. (2010). Supporting students in developing literacy in science. Science, 328(5977), 456-459.
Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159-174.
Lederman, N. & Niess, M. (1998). Survival of the fittest. School Science and Mathematics, 98(4), 169–172.
OECD (2016). PISA 2015 Assessment and analytical framework: Science, reading, mathematics and financial literacy. Paris: PISA
OECD Publishing.
Pearson, P. D., Moje, E., & Greenleaf, C. (2010). Literacy and science: Each in the service of the other. Science, 328(5977), 459-463.
Pedretti E. (2003). Teaching science, technology, society and environment (STSE) education. In D. L. Zeidler (Ed.), The role of moral
reasoning on socioscientific issues and discourse in science education (pp. 219-239). Dordrecht: Springer.
Pedretti, E. and Nazir, J. (2011). Currents in STSE education: Mapping a complex field, 40 years on. Science Education, 95(4), 601-626.
Ramsey, J. (1993). The science education reform movement: Implications for social responsibility. Science Education, 77(2), 235–258.
Roberts, D. A. (2007). Scientific literacy/ science literacy. In S. K. Abell & N. G. Lederman (Eds.), Handbook of Research on Science
Education (pp. 729-780). London: Lawrence Erlbaum Associates.
Stone, S. J., & Glascott, K. (1997) Teaching strategies: The Affective side of science instruction. Childhood Education, 74(2), 102-104.
Wei, B., & Chen, B. (2017). Examining the senior secondary school chemistry curriculum in China in view of scientific literacy.
In L. L. Liang, X. Liu & G. W. Fulmer (Eds.) Chinese Science Education in the 21st Century: Policy, Practice, and Research (pp.
133-148). Springer, Dordrecht.
Received: March 29, 2019 Accepted: September 20, 2019
Mustafa Cansiz PhD, Assistant Professor, Artvin Çoruh University, Faculty of

(Corresponding author) Education, Department of Mathematics and Science Education,
Artvin, Turkey.
E-mail: mustafacansiz@gmail.com
Nurcan Cansiz PhD, Assistant Professor, Artvin Çoruh University, Faculty of
Education, Department of Mathematics and Science Education,
Artvin, Turkey.
E-mail: nurcansiz7911@gmail.com
691
EFFECTS OF MICROTEACHING
MULTIPLE-REPRESENTATION
PHYSICS LESSON STUDY ON
PRE-SERVICE TEACHERS’
CRITICAL THINKING
Abstract. A research on the effects of the Billy A. Danday,

microteaching Multiple-Representation Sheryl Lyn C. Monterola
Lesson Study (MRLS) on pre-service Physics
teachers’ critical thinking was conducted
using a pre-test-post-test quasi-experimen-
tal design. Eighteen fourth year Bachelor
of Secondary Education-Physical Science
majors participated in the research. The
experimental group employed the micro-
teaching MRLS while the control group
implemented the Traditional Instructional Introduction
Planning Approach (TIPA). Data were
The prevalent educational milieu is characterized by an amalgamation
gathered from multiple sources such as
of various pedagogical principles and social dynamics involving the learners,
researcher-made written tests, interviews, teachers, and other stakeholders. Technological developments in the modern
diaries, and field notes. The Mann-Whitney time inevitably shape educational paradigms, hence, instructional practices
U test was utilized to ascertain statistical must adapt to these changes that essentially foster student discourse and
difference between the experimental and critical thinking. Emphasizing critical thinking within teacher education
anticipates a better response to societal issues by carrying out effective
the control group. Results revealed signifi-
measures (Williams, 2005). Scholars have established an explicit relation of
cant differences in the scores between the
critical thinking and problem-solving (Buckley, 2012; Maebius, 1990; Sensi-
two groups in the overall critical thinking baugh, 2015; Tseng, 2008; Williams, 2005). Buckley (2012) has even asserted
and on its sub-domains. Findings indicate that critical thinking is a prerequisite to problem-solving.
beneficial effects of the microteaching With the overwhelming challenges faced by pre-service teachers each
MRLS in developing pre-service teachers’ day, acquiring sound critical thinking is necessary for effective instructional
planning, implementation, and reflection. Research, however, has shown
critical thinking.
a dismal state of pre-service teachers’ critical thinking level (Akdere, 2012;
Keywords: critical thinking, lesson study,
Grosser & Nel, 2013; Qing, Jing, & Yan, 2010). Gashan (2015) has declared that
multiple representations, physics educa- pre-service teachers hold insufficient understanding of critical thinking skills,
tion, pre-service teachers, science educa- that they have uncertain knowledge about it despite their positive views
tion. about the place of critical thinking in the educative process. Furthermore,
the pre-service teachers have struggled in substantiating their knowledge
of their pupils’ capacity to probe, integrate, and assess information (Meister,
Billy A. Danday 2011). Initiatives to resolve these issues have been carried out but these re-
Leyte Normal University, Philippines vealed mixed impressions. Various approaches have not consistently ensued
Sheryl Lyn C. Monterola desirable results (Akyuz & Samsa, 2009; Behar-Horenstein & Niu, 2011; Goyak,
University of the Philippines, Philippines
2009; Qing, Jing, & Yan, 2010; Sulaiman, 2013).
One major challenge that confronts pre-service teachers is addressing
692
ISSN 1648–3898 /Print/ EFFECTS OF MICROTEACHING MULTIPLE-REPRESENTATION PHYSICS LESSON STUDY ON
PRE-SERVICE TEACHERS’ CRITICAL THINKING
ISSN 2538–7138 /Online/ (P. 692-707)
learning difficulties with Physics due to the numerous abstract concepts involved as well as the ubiquitous pres-
ence of mathematical symbols and processes (Alias & Ibrahim, 2013; De Cock, 2012; Erinosho, 2013; Kozhevnikov,
Hegarty, & Mayer, 1999). Learners often struggle to discern connections among mathematical, graphical, verbal,
and pictorial representations (Bal, 2015; Gulkilik & Arikan, 2012; Nguyen & Rebello, 2009; Ogunleye, 2009; Snetinova
& Koupilova, 2012; Soong, Mercer, & Er, 2009).
A deficiency in teaching competence may be manifested in learners’ misconceptions and difficulties associated
with multiple representations. Literature has shown a close association of the teachers’ knowledge and student
learning (Hightower, Delgado, Lloyd, Wittenstein, Sellers, & Swanson, 2011; Sadler, Sonnert, Coyle, Cook-Smith, &
Miller, 2013; Tchoshanov, Lesser, & Salazar, 2008). This connotes that a productive teaching entails a firm scaffolding
of teachers’ knowledge. Instructional practices maximizing the use of multiple representations can be a potential
driver in alleviating perennial conceptual, procedural, and mathematical difficulties (Akkus & Cakiroglu, 2009; Bal,
2015; Banerjee, 2010; Gulkilik & Arikan, 2012; Leigh, 2004; Ozmantar, Akkoc, Bingolbali, Demir, & Ergene, 2010).
A collaborative instructional planning approach such as the Lesson Study (LS) may be considered in con-
fronting these academic difficulties involving multiple representations. Lesson study is a cooperative educational
approach employed by in-service Japanese educators that can pose potential advances in attaining desirable
learning outcomes through careful collaborative instructional planning (Fernandez, 2002; Lewis, 2002; Lewis, Perry,
& Hurd, 2009). Incorporating the Lesson Study in the pre-service teacher curriculum offers explicit benefits such
as improving habits, competencies, and skills (Gurl, 2009; Elipane, 2012), transferring Nature of Science (NOS) into
classroom practice, and deepening the NOS pedagogical content knowledge of prospective teachers (McDowell,
2010). Moreover, engaging in a lesson study can promote student achievement and critical thinking (Barrett, Riggs,
& Ray, 2013; Lucenario, Yangco, Punzalan, & Espinosa, 2016; Quilario, 2014; and Teele, Maynard, & Marcoulides, 2015).
Reckoning the initiatives to cast light upon the constructs of lesson study, multiple representations, and criti-
cal thinking, a paucity of information sheds light on pre-service Physics teaching. A microteaching Physics Lesson
Study employing technology-driven representations is yet unknown. Further, little is known of the effects of a
multiple-representation-based collaborative instructional planning on the critical thinking of prospective Physics
teachers, either as an overall competence, or as a specific proficiency.
Multiple Representations and Learning
Representations are vital elements of a good Physics instruction. When at least two representations are
employed, either simultaneously or at some point during the entire duration of instruction, it is called a multi-
representational system (Ainsworth, 2006). Kohl and Finkelstein (2006) categorized basic representations into four:
1. Verbal – spoken words, written text, phrases or sentences
2. Graphical – graphs, diagrams that have labeled axes
3. Pictorial – pictures, images, sketches, diagrams
4. Mathematical - numerical calculations, equations, formulae, or any quantitative reasoning
The evolving technological education offers a more advanced combination of representations which Ainsworth
(2006) called a multi-media system. A regular multi-media system can present images, words, simulations, sound,
mathematical symbols, and diagrams. Notably, this multi-media system contains basic representations defined
by Kohl and Finkelstein (2006).
Learning with multiple representations can be a complicated task for learners due to the numerous information
they contain. Ainsworth (2006) cited Palmer’s (1977) five compositions of an external representation as shown in
Figure 1. The Cognitive Theory of Multimedia Learning describes five intellectual processes associated with multiple
representations and multimedia instruction. Mayer (n.d.) identified these processes which includes choosing words,
choosing images, organizing words, organizing images, and amalgamating. The theory maintains that learning is
established when learners systematically undergo these processes as they interact with representations.
693
EFFECTS OF MICROTEACHING MULTIPLE-REPRESENTATION PHYSICS LESSON STUDY ON
(P. 692-707) ISSN 2538–7138 /Online/
Figure 1. Compositions of external representation.
Kozhevnikov, Hegarty, and Mayer (1999) have reported that the use of images presents problem-solving im-
pediments. This may be due to the ineffective use of representations or failure of the students to visualize problems
in the form of illustrations or figures (Alias & Ibrahim, 2013; Kohl & Finkelstein, 2006).
Some scholars, however, have argued that instructional strategies utilizing representations pose beneficial
effects to students such as improving achievement, conceptual understanding, problem-solving strategy, use of
representations, and mathematics performance (Akkus & Cakiroglu, 2009; Banerjee, 2010; Kohl & Finkelstein, 2006;
Kurnaz & Saglam Arslan, 2014; Lucas, 2014). Larkin and McDermott (1978) classified the representations used by
the students into four stages as presented in the following figure.
Figure 2. Stages of utilization of representations.
Figure 2 connotes that problem-solving often allows students to undergo a transition from the lower stages
of employing representations (i.e. literal and naive representations) to higher stages such (i.e. scientific and alge-
braic representations). It can be noted that the last two stages indicate a more abstract depiction of the problem.
Understanding students’ representational skills and performance in Physics or in Mathematics in relation to
multiple representations is essential for a better grasp of Science instruction. Nevertheless, probing pre-service
teachers’ multi-representational competence and the methods of developing it are equally important. What lacks
in the literature is the initiative of scaffolding pre-service Physics teachers’ proficiency in the didactic use of multiple
representations, particularly the technology-driven ones.
Lesson Study
Lesson Study is a popular cooperative educational approach widely practiced in Japan. It comes from the
Japanese word jugyokenkyu. In English, the word jugyo is translated as “lesson” while kenkyu is translated as “study
or research”. The reverse word kenkyujugyo is literally translated as “study or research lessons” (Fernandez, 2002).
Various researchers and educators who have utilized the Lesson Study have exhibited the following features in
their process: (1) goal setting, (2) instructional planning, (3) designing the study, (4) implementing the instructional
plan, (5) discussing the implementation, (6) revising the instructional plan, (7) teaching the revised instructional
694
ISSN 2538–7138 /Online/ (P. 692-707)
plan, (8) documenting results, and (9) discussing the implementation of the new version of the instructional plan
(Fernandez, 2002; Cerbin & Kopp, 2006; Cavin, 2007; Chew & Lim, 2013).
Meanwhile, Cerbin, and Kopp (2006) have emphasized that lesson studies are primarily conducted not to
evaluate what the students have learned, rather, to find out how the students learn the lessons. Their interaction
with each other, with the teacher, and with the instructional materials are the chief considerations in the Lesson
Study framework.
As a professional development approach, Lesson Study poses potential advances in attaining desirable learn-
ing outcomes through careful collaborative instructional planning (Fernandez, 2002; Lewis, 2002; Lewis, Perry, &
Hurd, 2009). The integration of the Lesson Study in the teacher education curriculum has provided an array of
experiences that can improve skills, competencies, and habits (Elipane, 2012; Gurl, 2009), can successfully transfer
Nature of Science (NOS) into classroom practice, and can deepen the NOS pedagogical content knowledge of
pre-service teachers (McDowell, 2010). Moreover, the application of lesson study can have positive impact on
student achievement and critical thinking (Barrett, Riggs, & Ray, 2013; Lucenario, Yangco, Punzalan, & Espinosa,
2016; Quilario, 2014; Teele, Maynard, & Marcoulides, 2015).
The integration of technology into the Lesson Study framework has shown another positive perspective. Cavin
(2007) has claimed that the microteaching Lesson Study (MLS) can create a learning environment leading to the
appreciation of technology integration into a learner-centered teaching. Chew and Lim (2013) have found that
employing Lesson Study (LS) with the use of Geometer’s Sketchpad (GSP) in teaching mathematics can enhance
the technological pedagogical content knowledge of the pre-service teachers.
The lesson study may also be effectively utilized for curriculum design, implementation, and assessment. For
instance, Gutierez (2015) has applied the lesson study in identifying the issues associated with the implementation
of inquiry-based instruction in elementary science in the Philippines. Three major challenges have been reported
in engaging inquiry-based teaching among elementary Science teachers: (1) dearth in inquiry-based resources
and professional support; (2) giving more importance to content learning than inquiry-learning; and (3) the long
and laborious nature of inquiry-based didactic strategies. Meanwhile, some positive feedbacks have been derived
from the staff members who participated in the Lesson Study for the implementation of Understanding by Design
(UbD) curriculum in Neshaminy School District in Langhorne, Pennsylvania. These feedbacks include strengthening
teaching confidence, elimination of isolationism, re-evaluation of teaching strategies through self-reflection, and
empowerment of teachers (Kolenda, 2007).
Despite the growing number of studies involving lesson study, there still exist unexplained and less explored
fields of its application in the instructional milieu. For instance, the literature shows many advantages of lesson
study to Mathematics teachers and students. However, little is known about its application to other academic
domains, especially in the pre-service Physics teaching curricula. Moreover, a Lesson Study emphasizing the use
of technology-driven multiple representations in Physics instruction is still unexplored.
Critical Thinking
A major goal of science education around the globe is the attainment of critical thinking. The varying defini-
tions of this construct by various scholars converge to the contention provided by the American Philosophical As-
sociation (APA) Delphi Consensus as the process of persistent, self-directing judgment (Insight Assessment, 2013).
Therefore, critical thinking allows an individual to appraise a statement, problem, or issue by carefully examining
given facts and by looking “beyond” the data at hand through constant reflections while considering various pos-
sibilities to construct a belief system, make a definite decision, or do appropriate action.
As an essential domain of education and employment, measuring students’ critical thinking have propelled
academicians to craft valid and reliable instruments. Behar-Horenstein and Niu (2011) have revealed a predominant
use of single, quantitative measure of critical thinking, utilizing tests such as Watson-Glaser Critical Thinking Ap-
praisal (WGCTA), California Critical Thinking Skills Test (CCTST), and Cornell Critical Thinking Tests (CCTT). They have
asserted that qualitative measures are necessary in determining changes in students’ critical thinking. Therefore,
integrating open-ended questions to single-response items in a critical thinking test can enrich data that may
provide a deeper understanding of the changes that may take place.
In an attempt to measure critical thinking qualitatively, Facione and Facione (2013) have developed an all-
inclusive rubric internationally known for its applicability - The Holistic Critical Thinking Scoring Rubric (HCTSR).
It is primarily intended to assess the quality of thinking displayed in verbal presentations or written reports to
695
(P. 692-707) ISSN 2538–7138 /Online/
discriminate between strong and weak thinking which may be used in any training program or assessment pro-
cess. Facione and Facione (2013) contended that “its greatest value is obtained when used by trainees to assess the
quality of their own or another’s reasoning” (n.p.). The instrument scores an item ranging from 1 to 4 depending on
the quality of response.
Literature suggests the applicability of the WGCTA to measure pre-service teachers’ critical thinking. It can
be used to evaluate critical thinking ability of adult learners, trainees, and career-seekers (Akyuz & Samsa, 2009;
Behar-Horenstein & Niu, 2011; Gadzella, Stacks, Stephens, & Masten, 2005; Watson & Glaser, 2008). The WGCTA has
five components that test the examinees’ critical thinking ability - (1) Inference, (2) Recognition of Assumptions,
(3) Deduction, (4) Interpretation, and (5) Evaluation of Arguments.
The earlier versions of the WGCTA were composed of 100 items (Forms Ym and Zm) while the new versions
contained 80 items (i.e. Forms A and B) and 40 items (i.e. WGCTA – Form S), respectively, which are suitable for one
classroom period. These versions can be completed in a relatively shorter span of time (30 to 40 minutes) which
makes it practical enough to administer without altering the fundamental elements of constructs appraised in the
original forms (Watson & Glaser, 2008).
Efforts have been made to develop critical thinking among students by proposing and experimenting with varied
instructional interventions (Peirce, 2004). Mundilarto and Ismoyo (2017) utilized the problem-based learning (PBL)
while Barnett and Francis (2012) applied higher order thinking questions in classroom discourses. Desirable results
have been found considering that these pedagogical strategies require employment of various mental processes.
In the higher education, most of the researches on critical thinking development have been directed to the
investigation of instructional interventions employing quasi-experimental pretest-posttest design. However, these
studies were chiefly geared towards medical education fields (Behar-Horenstein & Niu, 2011). It has been reported
that certain instructional approaches can affect critical thinking skills of students such as inquiry-based learning,
concept mapping, scenario-based course exercises, active learning techniques, computer-assisted instruction,
structured web-based bulletin boards, guided instruction, and online instruction. Findings of the review revealed
mixed results of the studies; the effectiveness of the instructional strategies has differed from one study to another.
Central to this research was the development of pre-service teachers’ critical thinking. Studies in this area
have reported varied research techniques and claimed different results. Akdere (2012) has investigated the critical
thinking levels of Turkish pre-service teachers and other related constructs. It has been revealed that pre-service
teachers have a dismal level of critical thinking regardless of their encouraging outlook towards it. Correspond-
ingly, Qing, Jing, and Yan (2010) have disclosed that the effects of the inquiry-based and traditional approaches
are not significantly different on developing pre-service teachers’ critical thinking. They, however, concluded that
the inquiry-based chemical experiment in teaching is an effective approach in enhancing pre-service chemistry
teachers’ critical thinking. Other methodologies that have positive effects on pre-service teachers’ critical thinking
are the Cognitive-Infusion Intervention (Lang, 2006), problem-based learning model (Jatmiko, Prahani, Munasir,
Supardi, Wicaksono, Erlina, Pandiangan, Althaf, & Zainuddin, 2018), and reflective thinking-based teaching activi-
ties (Tican & Taspinar, 2015).
Literature has shown that despite the efforts to develop critical thinking among pre-service teachers, a solid
understanding of this area has not yet been fully achieved. Majority of the studies employed a single measure or a
purely quantitative approach in analyzing data which does not warrant a holistic assessment. There are aspects of
critical thinking that are not easily captured by quantitative instruments (Behar-Horenstein & Niu, 2011); qualitative
components of data need to be incorporated to meet such end. Likewise, there is a dearth of literature concerning
the development of valid and reliable domain-specific critical thinking instrument, especially in Physics. Scholars
often use existing general-content critical thinking tests, with the dominance of the WGCTA.
Problem of Research
Literature indicates a dismal level of critical thinking among pre-service teachers. Clearly, devising methods
to enhance critical thinking of pre-service teachers is imperative for a better system of instructional practice in
their future career. Hence, this research endeavored to determine if the microteaching Multiple-Representation
Lesson Study (MRLS) have significantly better effects on critical thinking of the pre-service Physics teachers than
the widely-used Traditional Instructional Planning Approach (TIPA). Further, this research was aimed to determine
if there is a significantly different effect of the two instructional approaches on the critical thinking components:
inference, recognition of assumptions, deduction, interpretation, and evaluation of arguments.
696
ISSN 2538–7138 /Online/ (P. 692-707)
One limitation of this research is the relatively small number of participants. This was mainly due to the lim-
ited number of the pre-service teachers taking Bachelor of Secondary Education Major in Physical Sciences in the
university where the research was conducted. However, it was ensured that data were appropriately analyzed and
findings were properly interpreted to answer the research problems.
General Background
This research utilized a mixed method approach which employed a pre-test-post-test quasi-experimental
design. The data were gathered last April-May, 2016 prior to the practice teaching of the pre-service Physics teach-
ers in a state university of the Eastern Visayas Region of the Philippines. The intervention lasted for five weeks.
Permission to conduct the research with the participation of the fourth year BSED-Physical Science majors
was sought from the university president and the university officials on March 17, 2016 through a letter signed by
the researcher and noted by the research adviser. During the preliminaries of the research on April 11, 2016, the
Informed Consent Form was administered to the participants to seek their permission on their participation in the
research. The pre-test for critical thinking were conducted shortly after the administration of the Informed Consent.
The 18 research participants were evenly divided into two by randomly assigning them to the experimental
or control group through a fishbowl technique. The nine participants in the control group were exposed to the
Traditional Instructional Planning Approach (TIPA) while the other nine were exposed to the experimental educa-
tional approach, the Multiple-Representation Lesson Study (MRLS). The MRLS group was further subdivided into
three sub-groups with three members each.
A total of nine topics representing the different areas of Physics were identified as shown in the following list.
Topic 1: Forces and Motion (Law of Acceleration)

Topic 2: Work and Energy (What is Work?: Calculating Work)
Topic 3: Work, and Energy (Work is a Method of Transferring Energy)
Topic 4: Heat and Temperature (Defining Heat and Temperature)
Topic 5: Heat and Temperature (Phase Change)
Topic 6: Electricity (Electric Current, Voltage and Resistance)
Topic 7: Electricity (Series and Parallel Connections)
Topic 8: Sounds (Propagation and Characteristics of Sound)
Topic 9: Colors of Light (Refraction of Light)
All the TIPA members wrote the instructional plans and prepared the instructional materials individually for
the nine topics. The three sub-groups in the MRLS group prepared the instructional plans and materials collab-
oratively for the three different topics. Microteachings were conducted for each topic and each sub-group shared
the instructional plans and materials that they have prepared to other sub-groups.
The research culminated on May 18, 2016. A post-test for critical thinking was administered to all the research
participants on the said date.
Participants
The participants of the research were 18 fourth year BSED-Physical Sciences students of a state university
in Region VIII, Philippines. The nine members of the TIPA group were from the ages 19 to 21 with an average age of
19.56 years. The MRLS group, on the other hand, had a mean age of 19.00 years, with age range of 18 to 20 years.
Both TIPA and MRLS had three male participants who comprised 33.3% of the group sample, and both groups
were composed of six female members who comprised 66.7% of the group sample. The MRLS and TIPA groups
had the same profile in terms of high school background; seven (77.8%) graduated from public high schools while
two (22.8%) came from private high schools. With respect to academic performance, the mean ratings in Science
were almost the same for the two groups, 1.89 for TIPA and 1.88 for MRLS. The mean ratings in Mathematics were
higher than in Science; the TIPA group had a slightly higher average rating than the MRLS group which was 1.72
against 1.77.
697
(P. 692-707) ISSN 2538–7138 /Online/
Initial comparability in Academic Performance

The ratings of the research participants in the 13 Science and two Mathematics courses that were taken prior
to the experiment were determined and analyzed to establish comparability in the academic performance. Table
1 presents the data.
Table 1. Mann-Whitney U test for the ratings in science and mathematics.
Subject/Group N Mean Rating Mean Rank Mann-Whitney U p
Science
MRLS 9 1.89 9.67
39.0 .894
TIPA 9 1.91 9.33
Math
MRLS 9 1.77 10.00
36.0 .690
TIPA 9 1.72 9.00
Results of the statistical analysis suggested that the median rating in Science of the MRLS (mean rank = 9.67)
group was not significantly different (p = .894) to that of the TIPA group (mean rank = 9.33). Similarly, the median
ratings in Mathematics between the two groups were not significantly different (mean rankMRLS = 10.00, mean rankTIPA
= 9.00, p = .690). These indicate that TIPA participants were comparable to MRLS participants in terms of academic
performance in Science and Mathematics before their participation in the research.
Instrument and Procedures
A. Pre-service Teacher Critical Thinking Inventory in Physics (PTCTIP)
The PPTCTI was a 60-item researcher-made test composed of both objective-type and open-ended items. It
aimed to assess the pre-service teachers’ level of thinking based on the six components of the WGCTA Form-S. Each
component comprised of twelve questions in the Physics context with the utilization of different representations.
One point was given for every correct response on the objective-type items while no point was given to a wrong
choice. Answers in the open-ended portion were given different credits ranging from one to four points based on
The Holistic Critical Thinking Scoring Rubric (Facione & Facione, 2013). The said rubric was modified and adapted
to rate participants’ explanations to the option they chose per item.
The PTCTI was validated by seasoned educators in Science and Mathematics Education coming from the UPD
College of Education and UP Open University. Their academic works involved researches on student Critical Think-
ing and production of instruments and materials that promote Critical Thinking Skills. The validated material was
pilot-tested to 56 BSED-Physical Sciences students of two state universities in Region VIII. The Cronbach’s alpha
of the 60-item instrument was calculated to be .923. The subtests also had acceptable reliability levels: Inference,
.701; Recognition of Assumptions, .771; Deduction, .784; Interpretation, .736; and Evaluation of Arguments, .807.
B. Interview Protocol
Researcher-made interview protocols were utilized to gather data about the experiences in instructional
planning approach and teaching demonstrations of the research participants from the two groups. These were
used at the end of the intervention in May 2016.
C. Diary
Each participant was asked to keep a diary to record their experiences during the two phases of the research.
Emphasis was placed on the challenges they have experienced with the instructional planning approach they
were exposed to and their coping mechanisms in overcoming the challenges they had mentioned in their record.
698
ISSN 2538–7138 /Online/ (P. 692-707)
The Microteaching Multiple-Representation Lesson Study (MRLS). The Microteaching MRLS framework was
modeled from the steps described by Cerbin and Kopp (2006). First, topics in Physics that were labeled “difficult”
and that required various representations were identified by the researcher and the subject teacher. The nine
identified topics in the different areas of Physics were then assigned to both the experimental and comparison
group. Second, members of each MRLS group designed the research lesson based on the goals that they have
formulated. They identified teaching strategies that were applied, the instructional materials that they used,
and evaluation techniques and procedures that measured student learning. Third, they decided how to inves-
tigate the learning process; that is, they devised a strategy to examine students’ manner of learning the lesson.
Fourth, the lesson plan was implemented. One of the group members executed the plan while the other mem-
bers observed how the students learned the topic. The subject teacher and the researcher also observed and
assessed the implementation of each plan. Attention was placed on the demonstrator’s delivery of the lesson,
student interactions, instructional materials, flow of the lesson, student activities, student discipline, and on
other aspects that affected attainment of the goals that were initially set. Fifth, Lesson Study members and the
facilitators (subject teacher and researcher) convened to discuss all observations and to assess the execution of
the lesson plan. The comments and observations were incorporated in the revised version of the plan. Sixth, the
revised lesson plan was implemented by the same pre-service teacher while the other members observed and
recorded their observations. Another post-lesson discussion followed and the instructional plan was modified
for the second time. Finally, the group documented the Lesson Study process to allow other MRLS sub-groups
to scrutinize their work and learn from it.
It must be noted that certain modifications of the Lesson Study framework were employed in this research. One
is that technology-driven multiple representations were integrated into the study lessons where a second version
of the lesson was required rather than optional. The second version of the lesson was implemented by the same
pre-service teacher who implemented the first version rather than another pre-service teacher. Other than that,
the non-participant “students”, that is, the pre-service teachers who played as “students” during the microteaching,
took part in the post-lesson assessment which followed immediately after the demonstration teaching of the first
version of the lesson. The traditional Lesson Study process lacked the student-perspective for it exclusively involved
the lesson study members with or without the facilitator or supervisor. The Multiple-Representation Lesson Study
afforded the MRLS participants to view the lesson in the eye of the “students” which was hardly observed in the
conventional Lesson Study process.
Figure 1 summarizes the Microteaching MRLS procedure.
Figure 1. Microteaching MRLS.
The Traditional Instructional Planning Approach (TIPA). The TIPA is the prevailing instructional approach applied
by the university in preparing the pre-service teachers for their future career. This has been practiced since the
university has offered undergraduate education programs. The TIPA reflects the predominant instructional practice
in the Department of Education of the Philippines.
In this research, the pre-service teachers in the TIPA group individually developed the nine lesson plans. The
subject teacher and the researcher collected and assessed each lesson plan. The corrected plans were returned
for modifications. The TIPA participants then proceeded to construct the instructional materials by themselves
after receiving the corrected plans. They were given the prerogative to choose the type of material used, either
technology-driven or conventional materials.
699
(P. 692-707) ISSN 2538–7138 /Online/
Both the TIPA and the Microteaching MRLS participants were allowed to use the resources of the university
such as the internet, books, and other printed and/or electronic resources. Likewise, both groups followed the same
schedule in the development of lesson plans per topic and the preparation of instructional materials, hence, an
approximately equal amount of time was devoted for the instructional intervention by the two groups. The entire
data collection procedure lasted for five weeks.
Table 2 provides the summary of the experiment. Explicit steps are shown for both groups.
Table 2. Summary of the experiment.
MRLS Group TIPA Group
•• Critical Thinking Pre-test •• Critical Thinking Pre-test

•• Orientation •• Orientation
•• Intervention (Multiple-representation Lesson Study) •• Intervention (Traditional Instructional Planning Approach)

•• Collaborative instructional planning
•• Collaborative material preparation •• Individual instructional planning
•• Microteaching (Tech-based 1st, and 2nd Versions) •• Individual material preparation
•• Class-based post-lesson discussions •• Feedback
•• Critical Thinking Post-test •• Critical Thinking Post-test
Data Analysis

This research employed the Mann-Whitney U Test to compare the pre-test and post-test scores of the MRLS
group and the TIPA group in the Critical Thinking Inventory. These were set at α=.05 level of significance.
Correspondingly, the qualitative data were analyzed using conceptual content analysis. The participants’
responses in the interview and the entries in their diary were coded based on the benefit or detriment, the chal-
lenges they encountered, and the coping mechanisms that they employed in the respective intervention they
were exposed to. Meanwhile, the responses in the open-ended portion which required the participants to explain
their choice in the objective part were evaluated using the Holistic Critical Thinking Scoring Rubric (HCTSR) by
Facione and Facione (2013).
Research Results
Initial comparability in critical thinking. Table 3 exhibits the pre-test results of the Critical Thinking Inventory.
Results revealed a slightly better performance of the MRLS group (Md = 118, Mn = 124, SD = 31.2) than the TIPA
group (Md = 115, Mn = 118, SD = 24.8). It must be noted that the highest possible score for the entire test was 300
points. Each of the five sub-tests or components was worth a maximum of 60 points.
Table 3. Pre-test results for the critical thinking inventory.
Mn
Test/Group N Md (Max.=300) SD (SD)2 η η2
(Max.=300)
Pre-test
MRLS 9 118 (39.3%) 124 (41.3%) 31.2 973
TIPA 9 115 (38.3%) 118 (39.3%) 24.8 615 .110 .012
Total 18 117 (39.0%) 121 (40.3%) 27.5 756
Note: Max. Score=300; Md=median; Mn=mean; SD=standard deviation; (SD)2=variance; and η and η2=measures of association
Though the pre-test median scores between the comparison (Md = 115) and experimental (Md = 118) groups
slightly differed, the Mann-Whitney U test revealed that the scores were not significantly different (U = 36.0, p = .691)
as exhibited in Table 4. This connotes that the two groups were comparable prior to the conduct of the research.
700
ISSN 2538–7138 /Online/ (P. 692-707)
Table 4. Mann-Whitney U test of the critical thinking inventory pre-test.
Test/Group N Mean Rank Mann-Whitney U Z p r (ES)
Pre-test
MRLS 9 10.0
36.0 -.398 .691 .094
TIPA 9 9.00
Table 5 presents the group mean scores of the participants and the Mann-Whitney U test analysis per criti-
cal thinking component. Pre-test results revealed no significant differences in the scores between the TIPA and
MRLS group in all the subtests or components, namely, Inference (U = 34.5, p = .954), Recognition of Assumption
(U = 36.0, p = .691), Deduction (U = 35.0, p = .626), Interpretation (U = 26.0, p = .199), and Evaluation of Arguments
(U = 38.0, p = .824). These indicate that the MRLS and the TIPA group were comparable in terms of making infer-
ences, recognizing assumptions, making deductions from given data, interpreting ideas, and evaluating arguments
prior to their participation in the study.
Table 5. Mann-Whitney U test per component of the critical thinking inventory pre-test.
MRLS TIPA
CT Component M-W U Z p r (ES)
Mn Mean Mn Mean
(Max.=60) Rank (Max.=60) Rank
Inference 31.3 (52.2%) 10.2 28.9 (48.2%) 8.83 34.5 -.533 .594 .126
Recognition of Assumptions 22.9 (38.2%) 9.00 24.7 (41.2%) 10.0 36.0 -.398 .691 .094
Deduction 24.3 (40.5%) 8.89 24.9 (41.5%) 10.1 35.0 -.487 .626 .115
Interpretation 27.7 (46.2%) 11.1 22.1 (36.8%) 7.89 26.0 -1.29 .199 .303
Evaluation of Arguments 17.8 (29.7%) 9.22 17.6 (29.3%) 9.78 38.0 -.222 .824 .052
Effects of the Microteaching MRLS on critical thinking. The post-test results, as shown in Table 6, were relatively
better than the Pre-test results for both groups. The MRLS group (Md = 246, Mn = 242, SD = 12.8) outperformed the
TIPA group (Md = 159, Mn = 152, SD = 15.4) in the overall critical thinking inventory. Scores at this evaluation period
were less dispersed than in the Pre-test for both groups as seen on the lower computed standard deviations. Data
revealed that variability in the group scores associated to instructional approach was 92%, a value indicating a
high association or effect.
Table 6. Post-test results for the critical thinking inventory.
Test/Group N Md (Max.=300) Mn (Max.=300) SD (SD)2 η η2
Post-test
MRLS 9 246 (82.0%) 242 (80.7%) 12.8 160
TIPA 9 159 (53.0%) 152 (50.7%) 15.4 236 .959 .920
Total 18 190 (63.3%) 197 (65.7%) 48.3 2337
Note: Max. Score=300; Md=median; Mn=mean; SD=standard deviation; (SD)2=variance; and η and η2=measures of association
Although both groups exhibited a considerable increase in group mean scores, the Mann-Whitney U test
shown in Table 7 indicated that the MRLS group scored significantly higher than the TIPA group (U=.0001, p<.001)
and recorded a large effect size (r = .844).
701
(P. 692-707) ISSN 2538–7138 /Online/
Table 7. Mann-Whitney U test of the critical thinking inventory post-test.
Test/Group N Mean Rank Mann-Whitney U Z p r (ES)
Post-test
MRLS 9 14.0
.000 -3.58 .0001** .844
TIPA 9 5.00
Note: **p<.01
The results of the post-test per critical thinking component are shown in Table 8. It can be gleaned from the
table that the scores of the MRLS group were significantly higher than the TIPA group (U ≤ 1.00, p < .001) across
all critical thinking components in the post-test. Moreover, large effect sizes ranging from r=.826 to r=.845 were
obtained. These results signified a positive effect of the microteaching MRLS in fostering critical thinking skills
among pre-service Physics teachers.
Table 8. Mann-Whitney U test per critical thinking component for the post-test.
Post-test
Critical Thinking Component Group
Mn (Max.=60) Mean Rank M-W U p
MRLS 52.1 (86.8%) 14.0

Inference .000 .0001**
TIPA 33.9 (56.5%) 5.00
MRLS 45.3 (75.5%) 14.0
Recognition of Assumptions .000 .0001**
TIPA 26.4 (44.0%) 5.00
MRLS 47.4 (79.0%) 14.0
Deduction .000 .0001**
TIPA 29.0 (48.3%) 5.00
MRLS 49.4 (82.3%) 13.9
Interpretation 1.000 .0001**
TIPA 32.8 (54.7%) 5.11
MRLS 47.4 (79.0%) 14.0
Evaluation of Arguments .000 .0001**
TIPA 29.6 (49.3%) 5.00
Note: *p<.05; **p<.01
Discussion
The Critical Thinking Inventory used in this research was in the Physics context - a domain-specific instrument.
The more favorable effects of the MRLS than the TIPA in the participants’ critical thinking can be ascribed to the
nature of the intervention. First, the collaborative lesson planning and instructional materials-preparation provided
the experimental group an opportunity to discuss different facets of specific Physics content. Knowles’ (1973, 1984)
theory of adult learning has stipulated that creating a cooperative learning climate and establishing collaborative
planning mechanisms can maximize learning. Conversely, Cerbin and Kopp (2006) have explained that lesson
study members can illuminate queries and uncertainties, can probe procedures, and can reflect about any stage
of the instructional process. This result is further justified by Laal and Laal’s (2012) contention that a fundamental
feature of a collaborative learning is an open discourse that allows learners to articulate and defend their ideas
leading them to create their own unique conceptual frameworks. Essentially, these endeavors are indications of
conceptual and procedural learning in a content-based critical thinking.
Participant E8 of the MRLS group expressed how collaborative work made an impact to her learning in the
following statement:
“…it’s very helpful that I have partners (that will) [who] help me, to give additional advices [pieces of advice], [to give] ad-
ditional ideas for [my] lesson and [on] how to deliver the lesson.” (E8; Interview, May 25, 2016) [giving pieces of advice or
recommendations]
702
ISSN 2538–7138 /Online/ (P. 692-707)
Although the TIPA group prepared nine sets of lesson plans and IMs while the MRLS group prepared only
three sets per sub-group, the individual preparation may have limited the TIPA participants to learn more on their
own or even when supported by the researcher’s and subject teacher’s feedback on their outputs. No learning,
or a very limited one, may have been derived from their peers due to the individualistic nature of the Traditional
Instructional Planning Approach (Mitcheltree, 2006). These results support the findings of Cajkler, Wood, Norton,
Pedder, and Xu, (2015), Hixon (2009), Kolenda (2007), and Quilario (2014). They all have expressed positive effects
of the Lesson Study like eradicating isolationism, ensuring a reflective learning, and providing an opportunity for
teamwork. Likewise, these results conform with the findings of Burroughs and Luebeck (2010) who have reported
that engaging in lesson study can provide the pre-service teachers with opportunities to be critical thinkers and
to think like real teachers.
Second, the microteaching MRLS have prompted the experimental group to utilize multiple representations
in their lesson plans which incited them to operate at a higher level thinking (Larkin & McDermott, 1978; Mayer,
n.d.). Moreover, the MRLS participants integrated multimedia systems in their lessons. Scholars have reported that
the employment of multiple representations through multimedia systems can function as rich learning resources
(Ozmantar, Akkoc, Bingolbali, Demir, & Ergene, 2010; Cairncross & Mannion, 2001). These findings support the
contentions of Laal & Laal (2012) and Klemm (1994) that a collaborative learning enterprise utilizing multiple
representations in the form of multimedia can promote the development of critical thinking skills. The Multiple-
representation Lesson Study has set a teaching-learning environment that adheres to this provision.
Third, the microteaching MRLS may have instilled a deeper sense of critical thinking in the experimental group.
By implementing the lesson plans to their peers, an “internal pressure” has pushed them to do their best through
careful planning and execution (Cavin, 2007; Cerbin & Kopp, 2006). They were challenged to provide activities that
require higher thinking. The class interaction compelled them to function at higher order thinking considering
that “students’” knowledge of content and pedagogy were comparable to them. More so that the researcher was
present during their microteaching. In an interview, Participant E5 disclosed his apprehension when he did his
demonstration teaching.
“E5: Yes, Sir. Based on the what we experienced as a teacher, we are [were] curious about [our] students
[felt conscious of the presence of] especially if they are just [our] classmates. They already know
the concepts, so [we] are [were] also . . . bagan nakukuan mo ba Sir in your demo. Nacu-curious [con-
scious] ka gihap imo ginyiyinakan. (…[we] tend to be affected in [our] demo, Sir. [We] also tend to
be conscious of what [we] say.)
I: Curious or conscious?
All: Conscious. [chuckled]
E5: Naco-conscious kami gihap hit amon classmate...tas presence gihap nimo Sir… (We tend to be con-
scious [intimidated]of our classmates… and your presence as well, Sir.)
I: Okay. So, you get intimidated…
E5: Yes, Sir.
I:…because you feel that they are more learned than you? So, that was a challenge for you?”…(everybody
nods) - (Interview, August 19, 2016) [feeling anxious about being observed]
Finally, during the conduct of microteaching MRLS, the participants had a full one hour of teaching. Moreover,
the researcher and/or their subject teacher consistently gave feedback on their outputs. Post-lesson discussions were,
likewise, ensured which ran from 30 minutes to one hour in the presence of the researcher and all group members.
Aside from pedagogical and technological aspects of microteaching, scrutiny in the content was emphasized. These
were so because microteaching MRLS possesses essential features of a collaborative learning engagement such
as sense of responsibility, social skills, direct interaction, and collective effort (Cajkler, Wood, Norton, Pedder, & Xu,
2015; Cerbin & Kopp, 2006; Hixon, 2009; Kolenda, 2007; Laal & Laal, 2012). These elements were lacking in the TIPA.
Results suggest that the microteaching MRLS can promote the development of the overall critical thinking
and all of its sub-skills. Additionally, results suggest that feedback from experts, interaction with fellow pre-service
teachers and students, length of preparation and of teaching, and availability of instructional technologies influ-
703
(P. 692-707) ISSN 2538–7138 /Online/
ence the enhancement of critical thinking of prospective teachers. Results further imply that persistent praxis is
necessary for a sustained development or acclimatization of a critical thinking culture. Likewise, results indicate
that asking for justifications on participants’ responses in the multiple-choice items of the Critical Thinking Inven-
tory can provide a holistic assessment of pre-service teachers’ critical thinking.
Certain implications to the teaching-learning process stem from these findings. The pre-service teachers can
be challenged to think critically in a host of learning endeavors by utilizing collaborative and multi-representation-
based activities. Science teaching can uphold the integration of multiple representations to explicate pre-service
teachers’ conceptions of scientific ideas and critical thinking skills. Verbal, pictorial, graphical, and mathematical
presentations may be utilized to instill inference-making, recognizing assumptions, deducting information or ideas,
interpreting data, and evaluating arguments among pre-service teachers.
Preparatory courses prior to practice teaching can prompt pre-service teachers to explore, develop, and uti-
lize an array of both technology-based and conventional instructional materials. They can be trained to evaluate
educational technologies based on certain parameters such as relevance to the topic and appropriateness to the
students or audience.
Pre-service teacher education can promote collaborative learning designs to support the collegial spirit
and suppress professional isolation among pre-service teachers. A learning environment emphasizing a healthy
exchange of ideas may be designed to exercise critical thinking skills.
To amplify the desirable effects of the MRLS, a provision of physical, professional, and technical support is
imperative. Supporting pre-service teachers with sufficient and functional teaching resources may establish an
atmosphere that spontaneously cultivates critical thinking; the primary concern of which must be the provision
of a teaching space conducive to learning. Likewise, giving adequate time for preparation, implementation of
lesson plans, and post-lesson discussions can possibly assist pre-service teachers towards a systematic teaching.
Opportunities like collaborative microteaching may scaffold their teaching competence and critical thinking that
will soon be in use during their practice teaching and beyond.

Acknowledgements
The authors would like to acknowledge the professional and financial assistance provided by the Leyte Normal
University, Commission on Higher Education, University of the Philippines Open University, DepEd Tacloban City
Division, Leyte National High School, Eastern Visayas State University, College of Education UP Diliman, and UP
NISMED in the completion of this research.
References
Ainsworth, S. (2006). Deft: A conceptual framework for considering learning with multiple representations. Elsevier: Learning
and Instruction 16, 183-198.
Akdere, N. İ. H. A. L. (2012). Turkish pre-service teachers’ critical thinking levels, attitudes and self-efficacy beliefs in teaching for
critical thinking. Yayımlanmamış Doktora Tezi, Ortadoğu Teknik Üniversitesi Sosyal Bilimler Enstitüsü, Ankara.
Akkus, O., & Cakiroglu, E. (2009). The effects of multiple representations-based instruction on seventh grade students’ algebra
performance. Lyon, France: INRP.
Akyuz, H., & Samsa, S. (2009). Critical thinking skills of preservice teachers in the blended learning environment. International
Journal of Human Sciences, 6(2), 538-550.
Alias, S., & Ibrahim, F. (2013). Difficulties in learning for solving problem of space-time special relativity. International Journal of
Business and Social Science, 4(5), 280-284.
Bal, A. (2015). Skills of using and transform multiple representations of the prospective teachers. Procedia - Social and Behavioral
Sciences 197, 582-588.
Banerjee, B. (2010). The effects of diagramming as a representational technique on high school students’ achievement in solving
math word problems. Published Dissertation. Ann Arbor, Michigan: UMI Dissertation Publishing.
Barnett, J. & Francis, A. (2012). Using higher order thinking questions to foster critical thinking: A classroom study. Educational
Psychology, 32(2), 201–211
Barrett, D., Riggs, L., & Ray, J. (2013). Teachers’ collaborative use of the lesson study approach to foster student achievement in
Geometry. International Journal of Social Sciences & Education, 3(4), 1188-1192.
Behar-Horenstein, L., & Niu, L. (2011). Teaching critical thinking skills in higher education: A review of the literature. Journal of
College Teaching & Learning, 8(2), 25-41.
Buckley, S. (2012). The role of computational thinking and critical thinking in problem solving in a learning environment. Euro-
pean Conference on e-Learning (pp. 63-70). Kidmore End, United Kingdom: Academic Conferences International Limited.
704
ISSN 2538–7138 /Online/ (P. 692-707)
Burroughs, E., & Luebeck, J. (2010). Pre-service teachers in mathematics lessonsStudy. The Mathematics Enthusiast, 7(2), 391-400.
Cairncross, S., & Mannion, M. (2001). Interactive multimedia and learning: Realizing the benefits. Innovations in Education and
Teaching International, 38(2), 156-164.
Cajkler, W., Wood, P., Norton, J., Pedder, D., & Xu, H. (2015). Teacher perspectives about lesson study in secondary school depart-
ments: A collaborative vehicle for professional learning and practice development. Research Papers in Education, 30(2),
192-213.
Cavin, R. (2007). Developing technological pedagogical content knowledge in preservice teachers through microteaching lesson
study. Published Dissertation. Ann Arbor, MI: ProQuest Information and Learning Center/UMI.
Cerbin, W., & Kopp, B. (2006). Lesson study as a model for building pedagogical knowledge and improving teaching. International
Journal of Teaching and Learning in Higher Education, 18(3), 250-257.
Chew, C., & Lim, C. (2013). Developing pre-service teachers’ technological pedagogical content knowledge for teaching math-
ematics with the geometer’s sketchpad through lesson study. Journal of Education and Learning, 2(1), 1-8.
De Cock, M. (2012). Representation use and strategy choice in physics problem solving. Physical Review Special Topics - Physics
Education Research, 8, 020117-1 - 020117-15.
Elipane, L. (2012). Integrating the essential elements of lesson study in pre-service mathematics teacher education. Published Dis-
sertation. Copenhagen: Department of Science Education, Copenhagen University.
Erinosho, S. (2013). How do students perceive the difficulty of physics in secondary school? An exploratory study in Nigeria.
International Journal for Cross-Disciplinary Subjects in Education, Special Issue 3(3), 1510-1515.
Facione, P., & Facione, N. (2013). The holistic critical thinking scoring rubric. Retrieved from Insight Assessment: http://www.
insightassessment.com/About-Us/Measured-Reasons/pdf-file/Holistic-Critical-Thinking-Scoring-Rubric-in-English-PDF/
(language)/eng-US.
Fernandez, C. (2002). Learning from Japanese approaches to professional development: The case of lesson study. Journal of
Teacher Education, 53(5), 393-405. doi:10.1177/002248702237394.
Gadzella, B., Stacks, J., Stephens, R., & Masten, W. (2005). Watson-Glaser critical thinking appraisal, form-S for education majors.
Journal of Instructional Psychology, 32(1), 9-12.
Gashan, A. (2015). Exploring saudi pre-service teachers’ knowledge of critical thinking skills and their teaching perceptions.
International Journal of Education & Literacy Studies, 3(1), 26-33.
Goyak, A. (2009). The effects of cooperative learning techniques on perceived classroom environment and critical thinking skills of
preservice teachers. Published Dissertation. Ann Arbor, MI: ProQuest LLC/UMI.
Grosser, M., & Nel, M. (2013). The relationship between the critical thinking skills and the academic language proficiency of
prospective teachers. South African Journal of Education, 33(2), 1-17.
Gulkilik, H., & Arikan, A. (2012). Preservice secondary mathematics teachers’ views about using multiple representations in
mathematics instruction. Procedia - Social and Behavioral Sciences 47, 1751-1756.
Gurl, T. (2009). An analysis of an adaptation of lesson study with preservice secondary mathematics teachers. Published Disserta-
tion. Ann Arbor, MI: ProQuest LLC/UMI.
Gutierez, S. (2015). Collaborative professional learning through lesson study: Identifying the challenges of inquiry-based teach-
ing. Issues in Educational Research, 25(2), 118-134.
Hightower, A., Delgado, R., Lloyd, S., Wittenstein, R., Sellers, K., & Swanson, C. (2011). Improving student learning by supporting
quality teaching: Key issues, effective strategies. Bethesda, MD: Editorial Projects in Education.
Hixon, M. (2009). Lesson study: A proposed intervention for professional development and student achievement. Published Dis-
sertation. Ann Arbor, MI: UMI Dissertation Publishing, ProQuest LLC.
Insight Assessment. (2013). California critical thinking skills test: CCTST test manual: “The gold standard” test of critical thinking.
San Jose, CA: California Academic Press.
Jatmiko,B., Prahani, B.K., Munasir, Supardi, Z.A.I., Wicaksono, I., Erlina, N., … Zainuddin. (2018). The comparison of oripa teach-
ing model and problem based learning effectiveness to improve critical thinking skills of pre-service physics teachers.
Journal of Baltic Science Education, 17(22), 300-319
Klemm, W. R. (1994). Using a formal collaborative learning paradigm for veterinary medical education. Journal of Veterinary
Medical Education, 21(1), 2-6. Retrieved January 15, 2018, from Digital Library and Archives: http://scholar.lib.vt.edu/
ejournals/JVME/V21-1/save.Klemm.html.
Knowles, M. (1973). The adult learner: A neglected species. Houston: Gulf Publishing.
Knowles, M. (1984). The adult learner: A neglected species (3rd ed.). Houston, TX: Gulf Publishing.
Kohl, P., & Finkelstein, N. (2005). Student representational competence and self-assessment when solving physics problems.
Physical Review Special Topics - Physics Education Research 1, 0101041-01010411.
Kohl, P., & Finkelstein, N. (2006). Effect of instructional environment on physics students’ representational skills. Physical Review
Special Topics - Physics Education Research 2, 010102-1 - 010102-8.
Kohl, P., & Finkelstein, N. (2006). Effects of representation on students solving physics problems: A fine-grained characterization.
Physical Review Special Topics - Physics Education Research 2, 0101061-01010612.
Kolenda, R. (2007). Japanese lesson study, staff development, and science education reform - The Neshaminy story. Science
Educator, 16(1), 29-33.
Kozhevnikov, M., Hegarty, M., & Mayer, R. (1999). Students’ use of imagery in solving qualitative problems in kinematics, 1-18.
Published Dissertation. Educational Resources Information Center (ERIC).
Kurnaz, M., & Saglam Arslan, A. (2014). Effectiveness of multiple representations for learning energy concepts: Case of Turkey.
705
(P. 692-707) ISSN 2538–7138 /Online/
Procedia - Social and Behavioral Sciences 116, 627 – 632.

Laal, M., & Laal, M. (2012). Collaborative learning: What is it? Procedia - Social and Behavioral Sciences 31, 491-495.
Lang, K. (2006). Effects of a cognitive-infusion intervention on critical thinking skills and dispositions of pre-service teachers.
Australian Aassociation for Research in Education Annual Conference (pp. 1-10). Adelaide: Australian Aassociation for Re-
search in Education .
Larkin, J., & McDermott, J. (1978). Re-representing textbook physics problems. Proceedings of the 2nd National Conference of the
Canadian Society for Computational Studies of Intelligence. Toronto: University of Toronto Press.
Leigh, G. (2004, March). Developing multi-representational problem solving skills in large, mixed-ability physics classes. Cape Town
Africa: Unpublished Dissertation, University of Cape Town. Retrieved November 5, 2015, from http://www.phy.uct.ac.za/
people/buffler/Leigh_MSc.pdf
Lewis, C. (2002). Does lesson study have a future in the united states? Nagoya Journal of Education and Human Development 1, 1-23.
Lewis, C., Perry, R., & Hurd, J. (2009). Improving mathematics instruction through lesson study: A theoretical model and North
American case. Journal of Mathematics Teacher Education, 12, 285-304. doi: 10.1007/s10857-009-9102-7.
Lucas, L. (2014, July 30). Supporting representation-rich problem-solving in high school physics. Retrieved November 17, 2015,
from DigitalCommons@University of Nebraska-Lincoln: Theses, Student Research, and Creative Activity: Department of
Teaching, Learning and Teacher Education: http://digitalcommons.unl.edu/teachlearnstudent/48
Lucenario, J., Yangco, R., Punzalan, A., & Espinosa, A. (2016). Pedagogical content knowledge-guided lesson study: Effects on
teacher competence and students’ achievement in Chemistry. Education Research International, 2016, 1-9.
Maebius, N. (1990). Relationships among critical thinking ability, locus of control, and clinical problem solving performance of
registered nurses. Published Dissertation. Ann Arbor, MI: University Microfilms International (UMI).
Mayer, R. (n.d.). Cognitive theory of multimedia learning. Retrieved from Cambridge University Press, Cambridge Core: https://
www.cambridge.org/core/books/cambridge-handbook-of-multimedia-learning/cognitive-theory-of-multimedia-learni
ng/24E5AEDEC8F4137E37E15BD2BCA91326.
McDowell, Amy, “Preservice Teachers’ Use of Lesson Study in Teaching Nature of Science.” Dissertation, Georgia State University,
2010. https://scholarworks.gsu.edu/msit_diss/65.
McDowell, A. (2010). Preservice teachers’ use of lesson study in teaching Nature of Science. Atlanta, GA: Dissertation, Georgia State
University. Retrieved January 26, 2016, from ScholarWorks@Georgia State University: http://scholarworks.gsu.edu/cgi/
viewcontent.cgi?article=1064&context=msit_diss.
Meister, D. (2011). Pre-service teachers’ impact on student learning: Three-year study of the impact on student learning project. Online
Submission, Paper presented at the Association of Teacher Educators Annual Conference, Orlando, Florida (February 15,
2011). Retrieved from http://files.eric.ed.gov/fulltext/ED524510.pdf.
Mitcheltree, M. (2006). Exploring lesson study as a form of professional development for enriching teacher knowledge and classroom
practices. Published Dissertation. Ann Arbor, MI: ProQuest Information and Learning Company/UMI.
Mundilarto, & Ismoyo, H. (2017). Effect of problem-based learning on improvement physics achievement and critical thinking
of senior high school student. Journal of Baltic Science Education, 16(5), 761-779.
Nguyen, D., & Rebello, N. (2009). Students’ difficulties in transfer of problem solving across representations. Physics Education
Research Conference (pp. 221-224). Ann Arbor, MI: American Institute of Physics.
Ogunleye, A. (2009). Teachers’ and students’ perceptions of students’ problem-solving difficulties in physics: Implications for
remediation. Journal of College Teaching & Learning, 6(7), 85-90.
Ozmantar, M., Akkoc, H., Bingolbali, E., Demir, S., & Ergene, B. (2010). Pre-service mathematics teachers’ use of multiple repre-
sentations in technology-rich environments. Eurasia Journal of Mathematics, Science & Technology Education, 6(1), 19-36.
Peirce, B. (2004). Ways to improve students’ thinking. In F. M. College, Handbook of Critical Thinking Resources for the Year of
Critical Thinking (p. 4). Maryland: Prince George Community College.
Qing, Z., Jing, G., & Yan, W. (2010). Promoting preservice teachers’ critical thinking skills by inquiry-based chemical experiment.
Procedia Social and Behavioral Sciences, 2, 4597–4603.
Quilario, K. (2014). Biology modelling-based lesson study: Effects on teacher self-efficacy and student critical thinking. Diliman,
Quezon City: Unpublished Master’s Thesis, UP College of Education.
Sadler, P., Sonnert, G., Coyle, H., Cook-Smith, N., & Miller, J. (2013). The influence of teachers’ knowledge on student learn-
ing in middle school Physical Science classrooms. American Education Research Journal, 50(5), 1020-1049. doi:
10.3102/0002831213477680.
Sensibaugh, C. (2015). Defining and assessing problem solving across a biochemistry curriculum. Published Dissertation. Ann
Arbor, MI: ProQuest LLC.
Snetinova, M., & Koupilova, Z. (2012). Students’ difficulties in solving physics problems. Proceedings of the 21st Annual Conference
of Doctoral Students Part III (pp. 93-97). Prague: Matfyzpress.
Soong, B., Mercer, N., & Er, S. (2009). Students’ difficulties when solving physics problems: Results from an ICT-infused revision
intervention. 17th International Conference on Computers in Education (pp. 361-365). Hongkong: Asia-Pacific Society for
Computers in Education.
Sulaiman, F. (2013). The effectiveness of PBL online on pre-service science. Wudpecker Journal of Educational Research, 2(6),
086 - 096.
Tchoshanov, M., Lesser, L., & Salazar, J. (2008). Teacher knowledge and student achievement: Revealing patterns. NCSM Journal
of Mathematics Education Leadership, Fall Issue, 39-49.
Teele, S., Maynard, D., & Marcoulides, G. (2015). The lesson study process - An effective intervention to reduce the achievement
706
ISSN 2538–7138 /Online/ (P. 692-707)
gap. US-China Education Review, 5(4), 229-243. doi: 10.17265/2161-623X/2015.04.001.

Tican, C. & Taspinar, M. (2015). The effects of reflective thinking-based teaching activities on pre-service teachers’ reflective
thinking skills, critical thinking skills, democratic attitudes, and academic achievement. The Anthropologist, 20 (1-2), 111-
120. doi: 10.1080/09720073.2015.11891730.
Tseng, M.-H. (2008). Data-driven problem solving and students’ critical thinking in a problem-based learninng environment . Ann
Arbor, MI: ProQuest LLC/UMI.
Watson, G., & Glaser, E. (2008). Watson-Glaser critical thinking appraisal: Short form manual. New York: Pearson Education.
Williams, R. (2005). Targeting critical thinking within teacher education: The potential impact on society. The Teacher Educator,
40(3), 163-187.
Received: March 30, 2019 Accepted: September 15, 2019
Billy A. Danday PhD, Associate Professor, College of Arts and Sciences, Leyte
(Corresponding author) Normal University, Tacloban City, Philippines.
E-mail: billydanday@gmail.com
PhD, Associate Professor, Division of Curriculum and Instruction,
Sheryl Lyn C. Monterola College of Education, University of the Philippines-Diliman,
Quezon City, Philippines.
E-mail: sheryllyn.monterola@gmail.com
707
THE INFLUENCE OF SELECTED
VARIABLES ON LOWER
SECONDARY SCHOOL
STUDENTS’ CONCEPTS ABOUT
DISEASES
Abstract. Children’s concept of biological

Jana Fancovicova,
phenomena differs from scientific thought
in many aspects. The revelation of wrong Milan Kubiatko
concepts contributes to better education,
especially when that information is de-
scribed in a specific field of science. Not only
children, but also adults often use anthro-
pomorphic and theological explanations to
express their ideas, which confirm their lack Introduction
of understanding of biological phenomena.
A great deal of research has proved retain- People have been aware of the value of health from ancient times.
ing of children’s misconceptions in the fields There are many factors, which can influence the health of people. The most
of zoology and botany. How accurate, frequent are bacteria and viruses. Bacteria are originators of many diseases
however, are their ideas about organisms of such as cholera, typhus, tuberculosis and others. Children have many wrong
a microscopic size? The aim of the research ideas about bacteria, which are perceived as a negative part of nature, but in
was to find out the influence of selected fac- many ways, bacteria also have a positive effect on human life. Many industrial
tors (perceived vulnerability, grade, gender activities are connected with the use of bacteria for human benefits (e.g.
and type of group) on concepts about preparation of food, preparation of medicaments) (Blaser, 2014). Viruses are
diseases among lower secondary school able to cause many diseases in humans, more than any parasites. This research
students. The research sample consisted of has worked with the concept of disease. The concept of “disease” is a synonym
400 students of lower secondary education for the viruses and bacteria and was used in this research.
from Slovakia. Data were collected through The current state of the research studies regarding concepts of microbes
Likert type questionnaires. The correlation and viruses. Boschhuizen, and Brinkman (1991) focused, for example, in their
was found between perceived vulnerability study on the issue of health concepts among lower secondary school stu-
to diseases, age and concepts about dis- dents. The researchers have quoted many common ideas about health, bac-
eases. The statistically significant difference teria and viruses. Simonneaux (2000) assumed that students’ conceptions are
in the results between boys and girls was themselves conditioned by the status given to diseases, by a hygiene-oriented
not confirmed. The higher impact should culture, by lay and school knowledge, personal experiences, socio-cultural
be aimed at their importance for humans mediation and linguistic confusions. Onyango-Ouma, Aagaard-Hansen, and
and nature for a better understanding of Jensen (2004) reported that students can modify and broaden their concepts
this concept. of health and illness through action-oriented health education. Myant and
Williams (2005) confirmed that children of all ages represent cold weather as
Keywords: concepts about diseases, lower
the main cause of colds. According to the researchers, the idea is explained
secondary school students, perceived vul-
by the children that there is a higher tendency to catch cold in the winter
nerability, quantitative research approach.
than in the summer. Another finding was that children often hear from their
parents that they will catch cold if they go out without having some protec-
Jana Fancovicova
tion for cold weather (sweater, warm clothes). The researchers also detected
University of Trnava, Slovakia the significant effects of age for an understanding of specific illnesses with
Milan Kubiatko explanations becoming more sophisticated and accurate with development.
J.E. Purkyne University, Czech Republic
Fernandes, Liamputtong, and Wollersheim (2015) examined how children
of refugees (1st - 4th year of primary school) perceive health and disease.
708
ISSN 1648–3898 /Print/ THE INFLUENCE OF SELECTED VARIABLES ON LOWER SECONDARY SCHOOL STUDENTS’
ISSN 2538–7138 /Online/ (P. 708-716)
Most children placed a major emphasis on the environment as a potential “guilty” disease. Children recognized
the need for warm clothing on cold days, as well as the importance of sun protection. They had an explanation
problem, however, as they did not know why the environment (especially the weather) was causing the disease.
In addition to the environment, children also mentioned the following causes of illness: lack of hygiene, sneezing
of strangers, eating at McDonald’s, overly sweet and savory meals. Mouratidi, Bonoti, and Leondari (2015) found
that older pupils have better concepts about illness and health than younger pupils, while the youngest pupils
had difficulties with representing the two concepts clearly. Piko and Bak (2006) demonstrated statistically insig-
nificant difference between boys and girls regarding concepts of illness and that children’s past experiences (e.g.
trivial infections) could play an important role in their concepts of disease causation. The gender differences were
examined in the study of Diaz, Soriano, and Belena (2016), but with the focus on germ aversion, which was higher
at girls. Flood et al. (2011) found a better understanding of influenza among older pupils. It is also possible to find
older research studies, for example Opie (1935), who realized that the people who suffered from tuberculosis have
correct concepts about this disease in comparison with people without previous experience. Pidgeon (1985) also
determined the amount of wrong ideas about diseases among pupils and their occurrence was influenced by
the age of respondents. Shiloh, Wade, Roberts, Alford, and Biesecker (2013) found out many findings, the most
interesting of which are: there are significant differences in risk perceptions and concern about diseases; there are
significant gender differences in risk perceptions and concern about common diseases. Similar results were pos-
sible to find in the studies of Duncan, Schaller, and Park (2009), Fukukawa, Oda, Usami, and Karahito (2014), Hill,
Boehm, and Prokosch (2016), Larki, Tahmasebi, and Reisi (2018) and Oda (2017). The similar research was done
by Welling, Conway, Bebruine, and Jones (2007) with the result, that men and women who perceived themselves
to be vulnerable to disease have stronger preferences for apparent health in dynamic faces than individuals who
perceive themselves to be relatively less vulnerable to disease.
Problem of the Research
The literature review revealed that the surveys which assessed participants’ ideas about diseases cover different
age groups, from primary school students (i.e. Fernandes, Liamputtong, & Wollersheim, 2015) till adult population
(i.e. Shiloh, Wade, Roberts, Alford, & Biesecker, 2013). The lower secondary school students as research sample are
rare in the published studies. At this age, students create their final concepts about different phenomena, so it is
important to find out, which topics are problematic for them and in the next step, try to eliminate the wrong ideas.
This is the reason why students from lower secondary schools were selected. The second reason is the lack of infor-
mation about bacteria and viruses in the biology curriculum at Slovakian lower secondary school. The biological
curriculum includes only fragments of information about diseases of organ systems. On the basis of this, students
have got wrong ideas about diseases and their originators. The bacteria are more mentioned than viruses, not
only in textbooks, but also in the media like TV and internet and it can influence better concepts about bacteria
among students. The third reason why this kind of research is important is the lack of research in the educational
environment about this topic and published studies did not adequately explain this phenomenon.
Research Focus
Research was focused on finding out the factors, which can influence concepts about diseases. The aim of
the research was to find out the influence of selected factors (perceived vulnerability, grade, gender and type of
group) on concepts about diseases among lower secondary school students.
General Background
The research was conducted in June 2018. The research design was quantitative, due to generalizing of the
findings and also to explain the examining phenomenon. The questionnaire was administered to all participants
in selected classes irrespective of the students’ concepts about diseases. Data analysis focused on interpreting
students’ responses considering their understanding of the diseases.
709
(P. 708-716) ISSN 2538–7138 /Online/
Research Sample
The research was conducted in June 2018 at four lower secondary schools in Slovakia. For the purpose of the
research, a non-random sample selection method was applied. Selection of participants was not intentional but
was based on teachers’ willingness to participate on the research in the accessible school in which researchers had
personal contacts with the director. Participants (n = 400), students of lower secondary education (202 girls and
198 boys), attended fifth (n = 92), sixth (n = 59), seventh (n = 75), eighth (86) and ninth (88) grades. The age of the
students ranged from 10 to 14 years old, with the mean age 12.91 (SD = 1.80).
Procedure
Anonymous questionnaires were administered in the last month of the school year 2017-18 by classroom
teachers during regular biology lessons. The management of the schools, the director and deputies, were informed
about the entire realization and agreed with the research. Students were randomly divided into two separate groups:
the first group with statements about bacterial disease, the second group with viral disease. They completed the
questionnaire in approximately 20 minutes.
Instruments
The questionnaire consisted of 45 5-point Likert type items (Cronbach’s alpha > .70) and were divided into
two parts:
1. Perceived vulnerability to disease scale (PVD) (Duncan, Schaller, & Park, 2009) was used to assess
the respondents’ self-perceived vulnerability to disease. This scale consisted of 15 items (Cronbach’s
alpha = .75); one subscale assesses beliefs about one’s own susceptibility to infectious diseases (Per-
ceived Infectability [PVD-PI] with 7 items); the second subscale assesses emotional discomfort in con-
texts that suggest an especially high potential for pathogen transmission (Germ Aversion [PVD-GA]
with 8 items). Items were rated on a five -point Likert scale from 1 (strongly disagree) to 5 (strongly
agree). Negatively worded items were scored in reverse order. The PVD-PI subscale was used to assess
inter-personal differences in perceived vulnerability to infectious diseases (item example: In general,
I am very susceptible to colds, flu and other infectious diseases). The PVD-GA subscale assesses some
behavior and emotional avoidance of some pathogen-relevant stimuli (item example: I prefer to wash
my hands quite soon after shaking someone’s hand).
2. The second part was focused on students’ concepts about viral or bacterial diseases (example item:
“Transmission of the disease occurs through physical contact with the infected person”). The items (30)
were in a positive and negative meaning. Negatively worded items were scored in reverse order. Students
had to decide and indicate to what extent they agreed or disagreed with a particular statement. The
questionnaire was created by the authors of research. All items were preparing in the concordance of
the curriculum. The content validity was determined by the teachers of biology (n = 4), who commented
also language level of the items. The reliability of the questionnaire was determined by the Cronbach’s
alpha coefficient, which value was sufficient (α = .65) for the first used questionnaires (Huizingh, 2007).
Demographic variables (gender, age of the respondents and also the incidence of illnesses) were obtained in
the introductory part of the questionnaire. Reported experiences with illnesses of respondents were asked: “How
many times have you been sick (not have a cold) from September up to the present?” (a) ever, b) 1-2 times, c) 3-4
times, d) 5 or more times). The students were also asked about specific diseases that they overcame between
September 2017 and June 2018.
Data Analysis
Age did not correlate with the incidence of illnesses (r = .04), which eliminated the prediction that younger
students were more likely to be ill and therefore the results would be affected by the frequency of the disease.
Since Perceived Infectability was positively correlated with the incidence of illnesses (r = .30; p < .05), the mean
disease susceptibility score was used in further analyses. The reliability of the measurements has been confirmed.
Through a correlation analysis, the positive correlation was confirmed between pathogenic resistance (Germ
710
ISSN 2538–7138 /Online/ (P. 708-716)
Aversion) and perceived infectability (r = .40; p < .05), thus disease susceptibility was excluded from further analysis
to prevent multicollinearity of data.
The analysis of covariance (ANCOVA) was used, in which the dependent variables were the concepts of dis-
eases, the categorical variables were gender and the type of group and the continuous variables were grade and
perceived vulnerability.
Research Results
The concepts of the disease score were significantly influenced by grade, perceived vulnerability to diseases,
type of group and gender (Table 1). The interaction term between the type of group and gender was not signifi-
cant (Table 1).
Table 1. Results of ANCOVA on concepts of diseases.
df SS MS F p
Intercept 1 38.85 38.85 543.60 < .001

Grade 1 1.54 1.54 21.54 < .001
Perceived vulnerability 1 .74 .74 10.41 < .001
Type of group 1 2.23 2.23 31.26 < .001
Gender 1 .40 .40 5.60 0.02
Type of group*Gender 1 .04 .04 .51 0.48
Error 394 28.16 .07
Total 399 33.79
Specifically, students more vulnerable to infectious diseases showed better concepts of diseases than their
less disease vulnerable counterparts (F = 10.41, p < .001) (Figure 1). Respondents who did not have frequent disease
experiences had imperfect concepts of disease.
Figure 1. Positive correlation between PVD and students’ concepts of diseases.
Bacterial diseases were better understood than viral diseases (F = 31.26, p < .001) (Figure 2). Bacterial diseases
received higher mean scores in concepts about diseases questionnaire.
711
(P. 708-716) ISSN 2538–7138 /Online/
Figure 2. Differences in mean scores of concepts of bacterial and viral diseases.
Girls manifested higher mean scores of concepts of diseases than boys (F = 5.60, p < .05) (Figure 3). Girls
scored significantly better in the concepts about diseases questionnaire.
Figure3. Differences in mean scores of concepts of diseases between males and females.
712
ISSN 2538–7138 /Online/ (P. 708-716)
Older students showed a better understanding of diseases than younger students (F = 21.54, p < .001) (Figure
4). The results look like consistent according to the grade of study, however, in the 7th grade was the decrease of
the score in comparison with the 6th grade.
Figure 4. Changes in concepts of diseases with respect to the students’ grade.
Discussion
This research examined relatively unstudied topic focused on the lower secondary school students’ con-
cepts about diseases and tried to find out the influence of selected variables on the students’ concepts about
diseases. The first variable was perceived vulnerability and there was the effort whether perceived vulnerability
to diseases should be associated with students’ concept of disease. This suggests that people with greater wor-
ries about pathogen transmission also have greater concerns about diseases and know more about bacteria
and viruses. The logic of this association lies in the sensitivity of the behavioral immune system that protects
humans against disease threat (Schaller, 2006). People with more perceived vulnerability to disease should
have fewer wrong ideas about viruses and bacteria. Findings supported this assumption. Similar results could
be found in the study of Duncan (2009). The indirect influence on perceived vulnerability to disease and the
concept level was confirmed in other studies (e.g. Gerrard, Gibbons, & Bushman, 1996; Prokop, & Kubiatko,
2014; Prokop, Usak, & Fancovicova, 2010; Timmermans et al., 2008). It can be assumed that people who have
a perceived vulnerability to disease are more interested in diseases, in the origin of every disease, have more
knowledge about bacteria and viruses and also about every possibility of being endangered by diseases in
outside as well as also inside environment. The perceived vulnerability is connected with the incidence of ill-
nesses. The respondents, who had more experiences with diseases, also had better ideas about diseases. The
amount of wrong ideas was less in the more experienced respondents with diseases. This can be explained
by the increased interest in their problems and on the basis of this fact, the wrong ideas were eliminated.
713
(P. 708-716) ISSN 2538–7138 /Online/
The other variable was gender and findings revealed no differences in concept about diseases with respect
to gender. They are supported by additional research (e.g. Piko & Bak, 2006). Differences were identified in
some cases, while girls had fewer incorrect ideas about diseases. Thandar et al. (2019) found out, for example,
that women had better knowledge and fewer wrong ideas about diseases. Similar results were shown in the
research of Suzuki et al. (2019), where female had a better knowledge of papillomavirus-related disease and
also about prevention of it. The inconsistency between research and other results could lie in the sample size,
more concretely in the age of the sample size. The age was relatively low in our study thus, the girls could
not realize the effect of diseases on their health and also on their potential children. Protectionistic behavior,
which is typical for woman, was not developed among younger girls who were attending secondary school.
This assumption is supported by the arguments of Draper, and Belsky (1990) concerning the development
of maternal care about children.
The assumption that students have better concepts about bacteria in comparison with viruses was
confirmed. The difference was statistically significant. This result was in all probability caused by the fact that
students positively evaluate different objects, which are pleasant, useful or which have some importance for
humans. Bacteria are mentioned, not only as harmful, but also as useful organisms for humans. These sug-
gestions are confirmed by other researchers (i.e. Hummel & Randler, 2012).
The grade of the respondents was another examined variable and our research confirmed that grade did
have a significant effect on the results. The youngest respondents had the highest amount of wrong ideas
about diseases and the oldest ones had fewer numbers of wrong ideas. Similar results could be found in the
study of Mouratidi, Bonoti, and Leondari (2015) and also Myant, and Williams (2005). Pidgeon (1985) also
stated that children’s knowledge of specific illnesses increases significantly during the school-age years. It
appears that children’s concepts of illnesses are a mixture of real knowledge and imaginative deformations.
The concepts of children, not only about diseases, are more complex with the higher age and include a low
amount of wrong ideas. This statement was also confirmed by Pidgeon (1985), who confirmed that there is,
in the school-age years, an increasing differentiation between external and internal explanations of illness.
Conclusions
The presented research found out the influence of selected factors (perceived vulnerability, grade, gender
and type of group) on concepts about diseases among lower secondary school students. Respondents vulner-
able to disease and also those students, who had more experience with diseases, had fewer wrong ideas about
diseases. This could have been caused by the low contact of students with the bacteria and viruses, which
caused the various diseases. The research fills the gap in this field of study, the number of research studies
is low, as it was mentioned in the theoretical background. It is possible to continue with the other variables,
which could influence the concepts about diseases.
The low understanding of diseases could be caused by the formal style of teaching, which is prevail-
ing in the Slovak educational system. Students do not have the possibility to understand the importance of
bacteria and viruses and it is difficult for students to understand how these microorganisms work. The higher
the amount of the mediated laboratory class, where students can see the life process of bacteria and viruses
and also their importance for humans and nature, the better the understanding of this concept. It is also
significant to emphasize the positive effect of bacteria on the human life and to advert on the influence of
the antibiotics, which are also eliminating useful bacteria.
Acknowledgments
This research was supported by the university grant no. 8/TU/2018 and by the Slovak Research and
Development Agency under contract No. APVV-14-0070 [grant number APVV-14-0070]
714
ISSN 2538–7138 /Online/ (P. 708-716)
References
Blaser, J. M. (2014). Missing microbes: How the overuse of antibiotics is fueling our modern plagues. New York: Henry Holt.
Boschhuizen, R., & Brinkman, F. G. (1991). A proposal for a teaching strategy based on preinstructional ideas of pupils. Envi-
ronmental education: The use of pupils’ ideas about cycles of nature and health. European Journal of Teacher Education,
14 (1), 45-56.
Diaz, A., Soriano, J. F., & Belena, A. (2016). Perceived vulnerability to disease questionnaire: Factor structure, psychometric proper-
ties and gender differences. Personality and Individual Differences, 101, 42-49. http://dx.doi.org/10.1016/j.paid.2016.05.036.
Draper, P., & Belsky, J. (1990). Personality development in evolutionary perspective. Journal of Personality, 58 (1), 141-161.
Duncan, L. A., Schaller, M., & Park, J. H. (2009). Perceived vulnerability to disease: Development and validation of a 15-item
self-report instrument. Personality and Individual Differences, 47, 541-546. http://dx.doi.org/10.1016/j.paid.2009.05.001.
Fernandes, S., Liamputtong, P., & Wollersheim, D. (2015). What makes people sick? Burmese refugee children’s perceptions of
health and illness. Health Promotion International, 30 (1), 151-161. https://doi.org/10.1093/heapro/dau082.
Flood, E. M. et al. (2011). Children’s perceptions of influenza illness and preferences for influenza vaccine. Journal of Pediatric
Health Care, 25(3), 171-179. https://doi.org/10.1016/j.pedhc.2010.04.007.
Fukukawa, Y., Oda, R., Usami, H., & Kawahito, J. (2014). Development of a Japanese version of the perceived vulnerability to
disease scale. The Japanese Journal of Psychology, 85 (2), 188-195. https://doi.org/10.4992/jjpsy.85.13206.
Gerrard, M., Gibbons, F. X., & Bushman, B. J. (1996). Relation between perceived vulnerability to HIV and precautionary sexual
behavior. Psychological Bulletin, 119 (3), 390-409.
Hill, S. E., Boehm, G. W., & Prokosch, M. L. (2016). Vulnerability to disease as a predictor of faster life history strategies. Adaptive
Human Behavior and Physiology, 2 (2), 116-133. https://doi.org/10.1007/s40750-015-0040-6.
Huizingh, E. (2007). Applied statistics with SPSS. London: Sage Publications.
Hummel, E., & Randler, C. (2012). Living animals in the classroom: A meta-analysis on learning outcome and a treatment–con-
trol study focusing on knowledge and motivation. Journal of Science Education and Technology, 21 (1), 95-105. https://
doi.org/10.1007/s10956-011-9285-4.
Larki, A., Tahmasebi, R., & Reisi, M. (2018). Factors predicting self-care behaviors among low health literacy hypertensive
patients based on health belief model in Bushehr District, South of Iran. International Journal of Hypertension, 2018, 7.
https://doi.org/10.1155/2018/9752736.
Mouratidi, P., Bonoti, F., & Leondari, A. (2015). Children’s perceptions of illness and health: An analysis of drawings. Health
Education Journal, 75 (4), 1-14. https://doi.org/10.1177/0017896915599416.
Myant, A. K., & Williams, M. J. (2005). Children´s concepts of health and illness: Understanding of contagious illness, non-
contagious illness and injuries. Journal of Health Psychology, 10 (6), 805-819.
Oda, R. (2017). Does perceived vulnerability to disease predict life-history strategy in Japanese adults? Letters on Evolutionary
Behavioral Science, 8 (2), 36-39. https://doi.org/10.5178/lebs.2017.62.
Onyango-Ouma, W., Aagaard-Hansen, J., & Jensen, B. B. (2004). Changing concepts of health and illness among children of
primary school age in Western Kenya. Health Education Research, 19 (3), 326-339. https://doi.org/10.1093/her/cyg034.
Opie, E. L. (1935). Present concepts of tuberculous infection and disease, their principles and application. American Review
of Tuberculosis, 35 (6), 617-630.
Pidgeon, V. (1985). Children’s concepts of illness: implications for health teaching. Maternal–Child Nursing Journal, 14 (1), 23-35.
Piko, B. F., & Bak, J. (2006). Children’s perceptions of health and illness: images and lay concepts in preadolescence. Health
Education Research, 21 (5), 643-653. https://doi.org/10.1093/her/cyl034.
Prokop, P., & Kubiatko, M. (2014). Perceived vulnerability to disease predicts environmental attitudes. Eurasia Journal of Math-
ematics, Science & Technology Education, 10 (3), 3-11. https://doi.org/10.12973/eurasia.2014.1017a.
Prokop, P., Usak, M., & Fančovičová, J. (2010). Risk of parasite transmission influences perceived vulnerability to disease
and perceived danger of disease-relevant animals. Behavioural Processes, 85 (1), 52-57. https://doi.org/10.1016/j.be-
proc.2010.06.006.
Schaller, M. (2006). Parasites, behavioral defenses, and the social psychological mechanisms through which cultures are
evoked. Psychological Inquiry, 17 (2), 96-101. http://dx.doi.org/10.1207/s15327965pli1702_2.
Shiloh, S., Wade, C. H., Roberts, J. S., Alford, S. H., & Biesecker, B. B. (2013). Risk perceptions and worry about common diseases:
A between- and within-subjects examination. Psychological Health, 28 (4), 434-449. https://doi.org/10.1080/08870446
.2012.737464.
Simonneaux, L. (2000). A study of pupils’ conceptions and reasoning in connection with ‘microbes’, as a contribution
to research in biotechnology education. International Journal of Science Education, 22 (6), 619-644. https://doi.
org/10.1080/095006900289705.
Suzuki, Y. et al. (2019). Current knowledge of and attitudes toward human papillomavirus-related disease prevention among
Japanese: A large-scale questionnaire study. The Journal of Obstetrics and Gynaecology Research, 45 (5), 994-1005. https://
doi.org/10.1111/jog.13929.
Thandar, M. M. et al. (2019). Women’s knowledge on common non-communicable diseases and nutritional need during
pregnancy in three townships of Ayeyarwaddy region, Myanmar: a cross-sectional study. Tropical Medicine and Health,
47, 12. https://doi.org/10.1186/s41182-019-0137-x.
715
(P. 708-716) ISSN 2538–7138 /Online/
Timmermans, D. R. M., Henneman, L., Hirasing, R. A., & van der Wal, G. (2008). Parents’ perceived vulnerability and perceived
control in preventing Meningococcal C infection: a large-scale interview study about vaccination. BMC Public Health, 8,
45. https://doi.org/10.1186/1471-2458-8-45.
Welling, L. L. M., Conway, C. A., Debruine, L. M., & Jones, B. C. (2007). Perceived vulnerability to disease is positively related to
the strength of preferences for apparent health in faces. Journal of Evolutionary Psychology, 5 (1), 131-139. https://doi.
org/10.1556/JEP.2007.1012 .
Received: July 08, 2019 Accepted: September 30, 2019
Jana Fancovicova PhD, Researcher, Department of Biology, Faculty of Education,

University of Trnava, Priemyselna 4, 918 43 Trnava, Slovakia.
E-mail: fankaj@gmail.com
Milan Kubiatko PhD, Researcher, Department of Biology, Faculty of Science, J.E.
(Corresponding author) Purkyne University, Ceske mladeze 8, 400 96 Usti nad Labem,
Czech Republic.
E-mail: mkubiatko@gmail.com
716
THE RELATION BETWEEN
TEACHERS’ ATTITUDES
TOWARDS RENEWABLE ENERGY ISSN 1648-3898 /Print/
SOURCES AND CRITICAL ISSN 2538-7138 /Online/
THINKING DISPOSITIONS
Abstract. In this research, it was aimed to

Gokhan Guven,
explore the relation between attitude to-
Nevin Kozcu Cakir wards renewable energy sources and criti-
cal thinking disposition. For this purpose,
relational survey model was used in the
research. The research group consisted of
468 prospective teachers studying in faculty
of education in a public university in the
Introduction fall semester of 2018-2019 academic year.
Renewable Energy Sources Attitude Scale in
One of the most important requirements for sustaining human life is order to evaluate prospective teachers’ at-
energy because life is an energy cycle, and energy is used in every activity titudes towards renewable energy sources,
and at every moment of daily life. However, the consumption of energy and Critical Thinking Disposition Scale for
sources, which has increased by the industrialization, the amount of con- evaluating critical thinking dispositions
sumed energy and the need for it are increasing rapidly due to the increase were used. In the research, the relation
in production technologies and population (Karakas, 2014). In order to meet between the data set for attitude towards
this need, countries are using different energy sources. For this reason, while renewable energy sources comprising of
determining energy policies, they aim to use energy sources that minimize variables which are application request,
environmental problems as well as meeting the energy needs in the best importance of education, country interest,
way. Among energy production sources, solar, wind, hydraulic, biomass, environmental awareness and invest-
geothermal, tidal and energy derived from waves are renewable, while the ments and the data set for critical thinking
energy obtained from nuclear, petroleum, coal and natural gas is classified disposition comprising of variables which
as non-renewable energy sources (Liarakou, Gavrilakis, & Flouri, 2009). are metacognition, flexibility, systematic-
In order to meet their energy needs, countries prefer fossil fuels such ity, tenacity-patience, open-mindedness
as coal and oil, due to the developments in the production technologies by the canonical analysis. As a result of the
and the cheapness of these sources. However, the combustion of these canonical correlation analysis, a significant
fuels causes the concentration of greenhouse gases such as CO2, CH4, CFCs, relation was found between the attitudes
halons, N2, ozone and peroxy acetylnitrate to increase in the atmosphere and towards renewable energy sources and
the surface temperature of the earth to rise. Many environmental problems the critical thinking dispositions and it was
such as global warming, acid rain, climate change, disasters and migrations found that the common variance shared
emerge as a result of increase and retention of heat spreading from the sur- between the data sets was 22.23%.
face of the Earth (Dincer, 1998; Koroneos, Spachos, & Moussiopoulos, 2003).
Keywords: renewable energy, critical
The emergence of such problems and the fact that fossil fuel reserves are
thinking disposition, prospective teachers,
limited and exhaustible raise the importance of renewable energy sources.
relational survey model. .
Renewable Energy Sources
Renewable energy is defined as the energy source that can exist in a Gokhan Guven, Nevin Kozcu Cakir
short time within the evolution of nature itself. These sources have an im- Mugla Sitki Kocman University, Turkey
portant place for the world and our country since they are renewable, have
minimum impact on environment, their operating and maintenance costs
are low, they are national and supply reliable energy (Haskok, 2005). The use
717
(P. 717-731) ISSN 2538–7138 /Online/
of clean and renewable energy sources becomes inevitable considering the extinction of natural resources, global
warming, climate change, the increase in greenhouse gases and energy demand, the possibility of exhaustion of
fossil fuels and uncertainty in oil prices (Apergis & Payne, 2010; Komor & Bazilian, 2005; Sarac & Bedir, 2014). The use
of renewable energy sources may reduce the carbon footprint resulting from the use of global energy. If the share
of renewable energy is doubled by 2030, the emission release can be reduced by approximately half. In addition,
when combined with energy efficiency, the average rise in global temperature can be kept below 2°C (1.5°C) and
climate change can be avoided before being a global disaster (Bayrac, 2010; Dogan, 2014). For this reason, European
countries aim to supply 20% of the energy from renewable energy sources by 2020 while USA aims 30% of the en-
ergy by 2030 (Balocco & Grazzini, 2007). However, when examining production of electric energy in Turkey in 2018,
it appears that approximately 71% of the electrical energy has been produced from fossil fuel sources, and 29% of it
from renewable energy sources. Also, Turkey is dependent on foreign countries because of the limited non-renewable
energy sources. Even the dependence on foreign source for oil (90%) and natural gas (95%) is quite high (Guner &
Turan, 2017). However, it can be said that it is a country that has rich regions in terms of renewable energy resources
and its geographical location makes the use of these resources possible (Baris & Kucukali, 2012; Benli, 2013; Yuksel &
Kaygusuz, 2011). In this context, among the most potential and the most used renewable energy sources, the solar,
hydraulic and wind are in the lead in Turkey (Cebesoy & Karisan, 2017). However, although 600 geothermal resources
have been identified in Turkey, whose temperatures reach until 1000C, the biomass obtained from potential urban
waste and the geothermal energy cannot be utilized sufficiently. In this respect, it is important to ensure that sustain-
able and renewable energy resources are used widely and effectively in order to reduce environmental pollution, to
eliminate environmental problems such as global warming and climate change and to leave a healthy world to future
generations (Karabulut & Alkan, 2010).
Education plays an important role in demonstrating positive attitudes and behaviour tendencies in individuals
who use renewable energy sources (Liarakou et.al., 2009). If individuals can have a positive attitude towards renewable
energy resources by education, it can be ensured that individuals refer alternative energy sources, use resources con-
sciously, get the idea of protecting the environment and develop awareness of responsibility towards the environment
(Koca & Bulut, 2015). It seems that the society has negative attitudes and behaviours towards the use of renewable
energy (Krohn & Damborg, 1999; Liarakou, et al., 2009; Wolsing, 2007). In addition, in the literature, in the researches
regarding the renewable energy sources, primary and secondary school students (Coker, Catlioglu, & Birgin, 2010;
Kilinc, Stanisstreet, & Boyes, 2009), high school students (Halder, Pietarinen, Nuutinen, & Pelkonen, 2010; Yuenyong,
Jones, & Yutakom, 2008; Zyadin, Puhakka, Ahponen, Cronberg & Pelkonen, 2012) and prospective teachers (Bilen,
Ozel & Surucu, 2013; Celikler, 2013; Firat, Sepetcioglu, & Kiraz, 2012; Guven & Sulun, 2017; Karatepe, Varbak, Kecebas,
& Yumurtaci, 2012; Zyadin, et.al., 2012) have been found to have negative attitudes.
In this context, to form environmental awareness, values, attitudes, skills and behaviours, which is necessary
for sustainable development, is possible only by training programs and equipped teachers (Guven & Sulun, 2017).
Celikler (2013) also emphasized that important teachers have important duties to create awareness in students who
have negative attitudes towards renewable energy. Teachers who have positive attitudes towards these resources
will help students learn and realize the benefits of renewable energy for society and the environment, and look out
for the interests of the country regarding energy policies (Halder, Nuutinen, Pietarinen, & Pelkonen, 2011; Liarakou,
et.al., 2008). Considering this situation, it is necessary to educate teachers who have positive attitudes towards re-
newable energy in order to raise individuals who question global environmental problems, search their effects and
causes and conduct ideas in this direction, do not harm the environment and are interested in and develop positive
attitudes towards the use of renewable energy sources. However, first of all, it is necessary to search for the reasons
why prospective teachers have negative attitudes towards renewable energy sources. When the researches in this
regard have been reviewed, the studies have reported that whether individuals have a critical thinking disposition can
affect their thoughts, evaluations, decisions and attitudes towards renewable energy sources (Lee, 2016; Opitz, 2016;
Sakschewski, Eggert, Schneider, & Bogeholz, 2014). In this respect, it has been thought that there may be a relationship
between prospective teachers’ attitudes towards renewable energy sources and their critical thinking dispositions.
Critical Thinking
The main objective of 21st century education is to raise individuals who are capable of meeting the demands
of the current age, adapt to their living standards, follow the technology, think critically and solve problems, are
creative, have communication power and who are open to cooperation. Among these competencies critical
718
ISSN 1648–3898 /Print/ THE RELATION BETWEEN TEACHERS’ ATTITUDES TOWARDS RENEWABLE ENERGY SOURCES
ISSN 2538–7138 /Online/ (P. 717-731)
thinking is in the lead (Colwill & Gallagher, 2007). Critical thinking is a process with universal values based on the
intuition, logic and experience used by the individuals in case of challenges during their lives (Tasci, 2005). This
process starts with analysis, continues with interpretation, self-regulation, inference and explanation, followed by
evaluation (Craft, 2003; Rudd, 2007).
Critical thinking is addressed in two basic dimensions. These are critical thinking skills and critical thinking
dispositions (Ozdemir, 2005; Walsh & Hardy, 1999). Critical thinking skill means that a person can think in a critical
way easily and ably with a mental effort, whereas tendency means a person’s desire for critical thinking (Zhang,
2003). Tendencies are related to attitudes in human nature and develop over time, influenced by factors such as
peer, environmental factors. Tendencies are powerful markers of critical thinking and, they are changeable, but
this change occurs slowly over time. Individuals with a low level of tendency for critical thinking do not address
complex problems, not seek different solutions to problems, not question judgments or not try to solve problems
(Irani, Rudd, Gallo, Ricketts, Friedel, & Rhoades, 2007). Moreover, individuals who do not have a critical thinking
disposition do not use these skills even if they have critical thinking skills. In contrast to this, individuals with these
tendencies are more enthusiastic for critical thinking. Therefore, the development of tendencies, which are paths for
critical thinking, is one of the main requirements for the individual to think critically (Tishman, Jay, & Perkins, 1993).
An individual with a critical thinking disposition has qualifications such as using reliable sources for a problem,
searching for reasons and options, being open-minded, taking or changing stance in cases where the evidence
and reasons are sufficient, searching for certainty, looking holistically at complex issues and handling the parts of
a whole in an organized way, being sensitive to others’ feelings, knowledge and culture levels (Ennis, 1985).
Instead of raising individuals who accept the given information without questioning, in the contemporary
education concept, it is aimed to educate individuals who know what, why and how to learn, who use the knowl-
edge they learn, develop and produce new knowledge (Senlik, Balkan, & Aycan, 2011) because critical thinkers
are open to new ideas, question the causes of a problem, reach reliable sources, try to determine the main point
without being obsessed with details, respect others’ opinions, pay regard to others, form their own views on a
scientific basis. In addition, critical thinking allows individuals to see incidents as multidimensional rather than one
dimensional (Girot, 2000). In this context, it can be said that critical thinking is an integral part of education (Norris,
1985). However, if the content and methods in the teaching process are formed in such a way as to provide skills
such as critical, creative and scientific thinking and reasoning, students will be able to think critically in a higher
level (Emir, 2012). Thus, when students’ critical thinking skills are included regularly in classes, students’ level of
critical thinking will increase (Besoluk & Onder, 2010). The teacher who adopts critical thinking will contribute to
the cognitive development of his/her students and will increase their positive attitudes towards critical thinking
(Seferoglu & Akbiyik, 2006). Given this situation the prospective teachers should have a tendency to think critically
(Erdem, Ilgan, & Celik, 2013; Tas, 2004; Yanpar-Yelken, 2009). However, when the literature is reviewed, it is seen
that the teachers’ critical thinking dispositions are low or moderate (Akar, 2007; Alper, 2010; Argon & Selvi, 2011;
Ay, Padem, & Eris, 2010; Besoluk & Onder, 2010; Can & Kaymakci, 2015; Cekic, 2007; Cetin, 2008; Dayioglu, 2003;
Gulveren, 2007; Guven & Kurum, 2008; Jatmiko, Prahani, Supardi, Wicaksono, Erlina, Pandiangan, & Althaf, 2018;
Kocak, Kurtlu, Ulas, & Epcacan, 2015; Korkmaz, 2009a; 2009b; Kokdemir, 2003; Kurum, 2002; Ozdemir, 2005; Sacli &
Demirhan, 2008; Sen, 2009; Turnuklu & Yesildere, 2005; Yaman & Yalcin, 2005; Zayif, 2008). In this context, consider-
ing the studies which have been reviewed, it can be said that the critical thinking dispositions of the prospective
teachers in our country are not sufficient.
Problem of Research
In summary, it is necessary to raise individuals and students who question global environmental problems,
search their effects and causes and conduct ideas in this direction, who are interested in and develop attitudes
towards the use of renewable energy sources that do not harm the environment. In addition, these individuals
and students should make inquiries about the problems, know what, why and how to learn, and they should tend
to think critically by generating new information. However, it is seen that both the attitudes of our prospective
teachers towards renewable energy sources and their critical thinking dispositions are not sufficient. Researches in
this regard also emphasize that thoughts, interests, evaluations, decisions and attitudes towards renewable energy
sources can be influenced by critical thinking dispositions of individuals (Lee, 2016; Opitz, 2016; Sakschewski et.al.,
2014). From this point of view, in the research, it is aimed to examine the relation between prospective teachers’
attitudes towards renewable energy sources and their critical thinking dispositions.
719
(P. 717-731) ISSN 2538–7138 /Online/
Research Model
In this research, relational survey model, which is a pattern of the survey model, was used. The survey model is
used to describe a past or present event or situation (Islamoglu, 2003). In the relational (correlational) pattern, which
is a type of survey model, it is tried to determine the presence and/or degree of the interchange between two or
more variables (Fraenkel, & Wallen, 2006; Karasar, 2016). The research group consists of the teachers who research
in the faculty of education in the Mugla Sitki Kocman University in the fall semester of 2018-2019 academic year.
Study Group
In this research, convenient case sampling method was used, which is one of the purposive samplings.
The research group included a total of 468 prospective teachers comprising of 155 individuals from the depart-
ment of early childhood education, 106 from the department of social sciences teaching, 94 from the department
of science teaching and 113 from the department of elementary teaching in the Mugla Sitki Kocman University. In
this case, according to the 95% confidence interval, it is seen that the sample size is sufficient to reduce sampling
error (Yazicioglu & Erdogan, 2014).
Of the research group, 324 (69.2%) were female and 144 (30.8%) were male prospective teachers. The dis-
tribution of the class level where the data was collected is; 1st grade n= 156 (33.3%), 2nd grade n= 120 (25.6%),
3rd grade n= 51 (10.9%), 4th grade n= 141 (30.1%). Stevens (2009) stated that the reliability of the findings in the
canonical correlation analysis can be achieved by 20 times more participants than the sum of the variables in each
set. When the scales used in this study are examined, there are 9 variables in total, 4 of which are application request,
importance of education, country interest, environmental awareness and investments in the data set of Renewable
Energy Sources Attitude Scale; and 5 of which are metacognition, flexibility, systematicity, tenacity-patience and
open-mindedness in the data set of Critical Thinking Disposition Scale. For the reliability of the findings, there must
be at least 180 participants (9x20 = 180) since there should be 20 times more participants than the total number of
variables. Considering the number of the sample group, it can be said that there are sufficient participants for the
reliability of the findings. Additionally, ethical issues were addressed. Thus, the required permission was obtained
from the university ethics commission, and the participants filled the consent forms.
Renewable Energy Sources Attitude Scale: Gunes, Alat and Gozum (2013) developed the Renewable Energy
Sources Attitude Scale, which has 4 sub-dimensions; application request (9 items), importance of education (6 items),
country interest (6 items), environmental awareness and investments (5 items). The scale consists of 26 items in
5-point Likert-type, 13 of which are positive and 13 of which are negative. The overall Cronbach’s Alpha reliability
coefficient value of the scale was .87; in this research, Cronbach’s Alpha reliability coefficient was calculated as .91.
Cronbach’s Alpha reliability coefficients calculated in line with the sample items belonging to sub-dimensions and
reliability study are shown in Table 1.
Table 1. Reliability coefficients of Renewable Energy Sources Attitude Scale sub-scales with sample items.
Cronbach’s Alpha Cronbach’s Alpha

Sub-scale Sub-Item
(Original Scale) (This Scale)
I would like to attend scientific meetings on

Application Request (AR) .97 .76
renewable energy resources.
I think education is an important tool to use
Importance of Education (IE) .80 .66
renewable energy correctly and effectively.
I do not think that converting renewable energy
Country Interest (CI) .78 .85
sources into energy will help development.
The use of renewable energy sources does not
Environmental Awareness and Investments (EAI) .72 .79
protect environment.
720
ISSN 2538–7138 /Online/ (P. 717-731)
Critical Thinking Disposition Scale [CTHD]: It used in the research was developed by Semerci (2016). The scale
consists of 49 items and 5-point Likert type. The scale has five sub-dimensions: metacognition (14 items), flexibility
(11 items), systematicity (13 items), tenacity-patience (8 items), open-mindedness (3 items). The overall Cronbach’s
Alpha reliability coefficient value of the scale was .96; in this study, Cronbach’s Alpha reliability coefficient was calcu-
lated as .95. Cronbach’s Alpha reliability coefficients calculated in line with the items belonging to sub-dimensions
and reliability study are shown in Table 2.
Table 2. Reliability coefficients of the Critical Thinking Disposition sub-scales with sample items.
Cronbach’s Alpha Cronbach’s Alpha

Sub-scales Sub-Item
(Original Scale) (This Scale)
Metacognition (MC) .90 .87 If there is something which I am bad at, I try to fix it.
Flexibility (FX) .89 .86 I can suggest more than one solution to solve a problem.
Systematicity (SYS) .90 .87 When I read any article, I can quickly find the main idea.
Tenacity-Patience (TP) .84 .83 I usually do what I do in a perfect and complete way.
Open-Mindedness (OM) .67 .63 I collect enough data before I decide.
Data Analysis

The relation between prospective teachers’ attitudes towards renewable energy sources and their critical
thinking disposition was searched by canonical correlation analysis. Canonical correlation examines the directional
relation between a X variable group (X1, X2, ... Xp) and a Y variable group (Y1, Y2, Y3, ... Yn) (Li, 2012). However,
the variable groups X and Y should be 2 or more than 2 in their own group. The reason why canonical correlation
analysis is preferred instead of multiple regression in the study is that in multiple regression analysis, the relation
between a single variable (Y) and two or more variables (X1, X2, ... Xp) (Cohen, 1968), while the canonical correla-
tion allows simultaneously examination of the connection of multiple numbers of Y variable with a large number
of X-variable and includes structural equation models (Bordens & Abbott, 2011; Henson, 2002; Knapp, 1978). The
most important feature of the canonical analysis is that the relation between two datasets can be demonstrated
by a single analysis and allows to keep the Type I error in minimum in the measurement process (Sherry & Henson,
2005; Thompson, 1991).
When the data set variables used in the research are examined; in the dimension of the attitude towards
renewable energy sources there are 4 variables; application request, importance of education, country interest,
environmental awareness and investments; and in the dimension of critical thinking disposition; metacognition,
flexibility, systematicity, tenacity-patience, open-mindedness; that is, 9 variables in total. The data sets formed here
were called set 1 and set 2 and the relation between two data sets was determined without taking as dependent
and independent variables (Stevens, 2012).
Before the canonical correlation analysis, it was searched whether the data showed normal distribution. Skew-
ness and Kurtosis values were examined for normality test. It was seen that the value of Kurtosis was between -.657
and +1.5, and the Skewness value changed between -.613 and +.337. According to Tabachnick and Fidell (2007), the
Skewness and Kurtosis values were between -1.5 and +1.5, indicating a normal distribution of data. The canonical
correlation analysis was performed with SPSS 20 program in the significance level of .05 and by syntax writing. A
general diagram of the canonical correlation analysis is shown in Figure 1.
721
(P. 717-731) ISSN 2538–7138 /Online/
Figure 1. Illustration of the first function in a canonical correlation analysis with four predictors and five crite-
rion variables. The canonical correlation is the simple Pearson r between the two synthetic variables,
which were linearly combined from the observed variables.
Research Results
Table 3 shows the correlation values of the sub-dimensions and descriptive values of the relation between
the attitudes of the prospective teachers towards renewable energy sources and their critical thinking dispositions.
Table 3. Relation between renewable energy sources attitude and critical thinking disposition.
Variables Mean SD 1 2 3 4 5 6 7 8 9
AR (1) 33.84 5.57 1

IE (2) 24.44 3.80 .72 1
CI (3) 21.95 6.11 .47 .57 1
EAI (4) 18.22 4.64 .48 .58 .79 1
MC (5) 56.12 6.89 .37 .33 .08 .08 1
FX (6) 43.91 5.88 .32 .31 .10 .08 .79 1
SYS (7) 50.92 6.99 .35 .33 .05 .06 .77 .77 1
TP (8) 30.86 5.05 .27 .23 -.01 -.02 .59 .58 .68 1
OM (9) 11.92 2.02 .26 .25 .04 .06 .48 .58 .58 .54 1
When the correlation values between the variables given in Table 3 were analysed, it was observed that the
relation among the application request, importance of education, country interest, environmental awareness and
investments in the first variable set changed between .47 and .79; the correlation coefficients of the relationships
between the variables metacognition, flexibility, systematicity, tenacity-patience and open-mindedness in the
second set of variables ranged between .79 and .48. It was determined that the correlation coefficients between
the first and the second variable sets changed between .37 and .02.
In the canonical correlation analysis, multivariate significance test, which indicates whether the canonical
model is statistically significant, was initially examined. These significance tests consist of four different tests: Pillais,
Hotellings, Wilks, and Roys. Each of these tests can be converted to the F test, so the significance of the canonical
model resulting from the analysis can be tested. The theoretical basis for each of these four tests is different, which
leads to a difference in the F value calculated for each test. However, since it is generally more applicable in research,
the comments were made based on the Wilks - test (Sherry & Henson, 2005; Stevens, 2012).
722
ISSN 2538–7138 /Online/ (P. 717-731)
Table 4. Multivariate test of significance.
Value Approx F Hypothesis df Eror df Significance of F
Pillais .22782 5.58058 20 1848 .000*

Hotellings .27878 6.37707 20 1830 .000*
Wilks .77770 5.99794 20 1523 .000*
Roys .20106 20
(S = 4, M = 0, N = 228 1/2; *p < .0001)
Table 4 shows that the canonical model obtained from the study was statistically significant [Wilks’s λ =.77770,
F (20, 1523) = 5.998, p<.001]. However, the significance of these tests does not give information about the strength
of the relation obtained. In the canonical correlation analysis, it is important to make an assessment of the effect
size as well as the significance of the model. In doing so, Wilks λ value, which is called as reverse effect size, is used.
Since Wilks λ value expresses the unexplained variance between the canonical variables in the model obtained
as a result of the analysis, “1-λ” value gives the common variance amount shared by the canonical variables and is
interpreted as r2 value in the regression analysis. In this case, “1-λ” value was calculated as .2223 for Wilks’ λ value
obtained. According to this situation, it can be said that the common variance shared between the data sets of
prospective teachers’ attitudes towards renewable energy sources and their critical thinking dispositions is 22.23%.
After determining whether the canonical model is statistically significant, the significance of each canonical function
in the model should be examined separately. Here, while the significance of the canonical model obtained from
canonical correlation is tested, the cumulative values of the canonical functions are processed. In a canonical model
where the cumulative values are statistically significant, while some of the canonical functions are significant, the
relation between the variables may be very low. Therefore, while interpreting the results of canonical correlation
analysis, the significance of each canonical function should be evaluated separately together with the canonical
model. In order to determine which canonical functions are significant, eigenvalues of canonical functions and
canonical correlation values are examined. In the research, as the result of the canonical correlation analysis used
to review the relationship between the data sets of the attitude towards the renewable energy sources and the
critical thinking disposition, four canonical functions were obtained. The eigenvalues and the canonical correlation
values of these functions are shown in Table 5.
Table 5. Canonical correlation analysis results between renewable energy sources attitude and critical think-
ing disposition.
Roods Eigenvalue % Cumulative % rc rc2
1 .252 90.269 90.269 .45 .201

2 .018 6.343 96.612 .13 .017
3 .007 2.397 99.009 .08 .007
4 .003 .991 100 .05 .003
According to the findings in Table 5, the canonical correlation value for the first-order canonical function was
found to be .45. According to this, the data sets of the attitude towards renewable energy sources and critical think-
ing disposition in the first canonical function share a variance of 20.1%. The canonical correlation value which was
disregarded in the first canonical function and revealed the maximum relation between two canonical variables
was calculated in the second order canonical correlation. It was determined that the canonical correlation value
calculated for the second order canonical function was .13 and that the attitude towards the renewable energy
sources and critical thinking disposition shared a variance of 1.7% in this function. After removing the common
variance shared by the data sets in the first two canonical functions, it was seen that the canonical correlation value
of the third canonical function was .08 and the common variance shared by the data sets for the third canonical
function is .7%. Finally, after removing the common variance shared by the data sets in the first three canonical
functions, it was determined that the canonical correlation value of the fourth canonical function was .05 and the
common variance shared by the attitude towards renewable energy sources and critical thinking disposition data
723
(P. 717-731) ISSN 2538–7138 /Online/
sets for the third canonical function was .3%. The analysis of the significance of each canonical function deter-
mined as the result of the analysis, also shed light on which functions that resulted from the canonical correlation
analysis should be interpreted. Here, regarding which canonical functions should be interpreted as the result of
the analysis, Tabachnick and Fidell (2007) stated it is appropriate to interpret only the statistically significant ones.
Regarding which canonical functions should be interpreted, Sherry and Henson (2005) stated that the canonical
value calculated for each function should be squared and the sum of the obtained values should be compared
with the value of “1-λ”. If the “1-λ” value is equal to or greater than the value obtained in the comparison made
here, that amount of function should be interpreted. The results of the dimension reduction analysis conducted for
the relation between prospective teachers’ attitude towards renewable energy sources and their critical thinking
disposition datasets are given in Table 6.
Table 6. Dimension reduction analysis.
Roods Wilks L. F Hypothesis df Eror df rc rc2 Significance of F
1 to 4 .777 5.997 20 1523.28 .448 .201 .000*

2 to 4 .973 1.038 12 1217.34 .132 .017 .410
3 to 4 .991 .7253 6 922.00 .081 .007 .629
4 to 4 .997 .6383 2 462.00 .052 .003 .529
(*p < .0001)
When Table 6 is examined, for the first canonical model (function 1 to 4) regarding Wilk’s λ values and Chi-square
values of four canonical functions obtained as a result of analysis, the canonical correlation coefficient calculated
between the attitude towards renewable energy sources and critical thinking disposition is statistically significant
[Wilks’s λ=.78, F (20, 1523.28) = 5.997, p<.001]. The correlation value of the first canonical function was found to
be .448. Accordingly, the shared variance between the data sets is 20.1%. After removing the first canonical func-
tion where the correlation between the canonical variables was highest, it was determined that the relationship
between the data sets for the remaining second canonical function (function 2 to 4) was not statistically significant
[Wilks’s λ =.973, F (12, 1217.34) = 1.038, p>.05]. According to the Wilks λ value of this relationship, which consists of
the cumulative values of the second, third and fourth canonical functions, the common variance shared between
the data sets is 3% [“1-λ” = .030]. The relation between the data sets for the third canonical function (function 3 to
4) after the removal of the second canonical function was not statistically significant [Wilks’s λ=.991, F (6, 922.00)
= .7253, p>.05]. According to the Wilks λ value of this relationship, which consists of the cumulative values of the
third and fourth canonical functions, the common variance shared between the data sets is 0.9% [“1-λ” = .009].
There was no statistically significant relationship between the data sets for the fourth canonical function (function
4 to 4) after the removal of the third canonical function [Wilks’s λ=.997, F (2, 462.00) = .6383, p>.05]. In this function,
where the relation between canonical variables is the weakest, the data sets of the attitude toward the renewable
energy sources and critical thinking disposition share only a .3% common variance [“1-λ” =.003].
To determine how application request (AR), importance of education (IE), country interest (CI) and environ-
mental awareness and investments (EAI) variables in the data set of the attitudes toward renewable energy sources
and metacognition (MC), flexibility (FX), systematicity (SYS), tenacity-patience (TP) and open-mindedness (OM)
variables in the data set of critical thinking disposition contribute to the relation between the canonical variables,
the standardized coefficients belonging to the first canonical function between the canonical variables and struc-
tural coefficients were researched. These obtained coefficients are given in Table 7.
In the presentation of the findings, the standardized coefficients of the canonical functions were shown as
“Coef” and the structural coefficients as “rs”. In Table 7, while variables of application request (AR), importance of
education (IE), country interest (CI) and environmental awareness and investments (EAI) are defined with the data
set of the attitudes towards renewable energy sources, the common variance shared by these variables with the
data set of critical thinking disposition is shown as “rc2” In addition, it is stated in the same table, by summing rc2
values in the first canonical function of AR, IE, CI and EAI variables included in the data set of the attitudes towards
renewable energy sources and MC, FX, SYS, TP and OM variables in the data set of critical thinking disposition, what
is the common variance shared by these variables with the data set in the canonical model. In order to determine
whether the variables they share with the data set are important or not, the value of .45 is taken as the criterion.
724
ISSN 2538–7138 /Online/ (P. 717-731)
Accordingly, it can be said that the contribution of the variables with the rs values .45 or above to their data sets are
important (Sherry, & Henson, 2005). In order for the variables in each data set to be a part of the canonical model,
their correlation should be greater than .30 (Tabachnick and Fidell, 2007).
Table 7. Canonical solution for renewable energy sources attitude and critical thinking disposition for func-
tions 1.
Function 1
Variables Coef. rs rs2(%)
AR -.706 -.86 .74

IE -.621 -.77 .60
CI .292 -.13 .02
EAI .329 -.14 .02
rc2 20.1
MC -.512 -.92 .85

FX .094 -.81 .65
SYS -.388 -.92 .85
TP -.156 -.77 .60
OM -.184 -.69 .47
Structure coefficients (rs ) greater than |.45| are underlined. Coef = standardized canonical function coefficient; rs = structure coeffi-
cient; = squared structure coefficient
When Table 7 is examined, it is determined that the contribution of AR and IE variables to the data set of the
attitudes towards renewable energy sources is over .45 in the first canonical function. The structural coefficient of
the CI and EAI variables was found to be below .45. Accordingly, it can be said that the contribution of AR and IE
variables for the first canonical function to the data set of the attitudes towards renewable energy sources is more
important than the CI and EAI variables. Similarly, the contribution of MC, FX, SYS, TP and OM variables in the first
canonical function to the critical thinking disposition data set is over .45. Thus, it can be said that the contribution
of MC, FX, SYS, TP and OM variables in the first canonical function to the critical thinking disposition data set is
important.
In canonical functions obtained from canonical correlation analysis, the direction of the relation among these
variables can be determined depending on the positivity or negativity of variables which have a significant contri-
bution to the data set (having a structural coefficient of .45 or more). In the first canonical function where the coef-
ficients of AR and IE variables are significant, when the signs of variables are examined it is seen that the variables are
negative. According to this, it can be said that there is a same-directional relationship between AR and IE variables.
When the variables of critical thinking disposition data set in first canonical function are examined; MC, FX, SYS, TP,
and OM all have negative signs and a same-directional relation. According to Table 7; there is a positive-directional
relation between AR and EI variables and MC, FX, SYS, TP and OM variables. According to this result; as the attitude
towards renewable energy sources in terms of the application request (AR) and the importance of education (IE)
increases, the metacognitive (MC), their flexibility (FX), their systematicity (SYS), their tenacity and patience (TP),
and their open mindedness (OM) among critical thinking dispositions increase. According to Table 7, rc2value for
the first canonical function was calculated as 20.01. This value reveals that the common variance shared between
the data sets of attitudes towards the renewable energy sources and the critical thinking disposition in the first
canonical function is 20.1%. In addition, the canonical correlation coefficients between the structural coefficients
of the first canonical function and the data sets for this function are given in Figure 2.
725
(P. 717-731) ISSN 2538–7138 /Online/
Figure 2. Canonical correlation results.
As a result, according to the data obtained from the participants, it was determined that the common vari-
ance shared by the data sets of the attitude towards renewable energy sources and the critical thinking disposition
was 22.23%. Based on the findings obtained from the canonical correlation analysis, the relation between attitude
towards renewable energy sources and critical thinking disposition is given in Figure 3.
Figure 3. The shared common variance by two data sets (22.23%).
Discussion
This research was carried out to reveal the relation between prospective teachers’ attitudes towards renew-
able energy sources and critical thinking dispositions.
Correlation analysis was performed to determine the relationship between the sub-dimensions of the scales.
As a result of the analysis, firstly, the application request, one of the sub-dimensions of the scale of the attitude
towards renewable energy sources, has found to have a low-level positive relation with metacognition, flexibility,
systematicity and tenacity-patience among the sub-dimensions of the scale for critical thinking dispositions. Ac-
cording to this result, the request of prospective teachers to have detailed knowledge about renewable energy
resources, their desire to attend scientific meetings and to prepare projects was observed to be related to their
awareness of their own cognitions, thinking flexibly, controlling their thoughts and acts in a planned manner,
and being confident against the barriers for the problems. When the literature is examined, it is seen that the pro-
spective teachers are willing for applications for renewable energy sources (Bilen, Ozel, & Surucu, 2013; Cebesoy
& Karisan, 2017; Karabulut & Alkan, 2010). In these studies, prospective teachers stated that they were willing to
use and apply these resources due to the reasons such as the fact that renewable energy sources do not harm the
environment, there is no danger of exhaustion like fossil fuels and the cost of plant installation is low. In addition,
726
ISSN 2538–7138 /Online/ (P. 717-731)
prospective teachers stated that renewable energy sources can be used for water heating and supplying energy
for electronic equipment in houses and they are willing to implement these systems (Cebesoy & Karisan, 2017).
In this direction, it can be said that metacognition, flexible thinking, systematicity and tenacity are important
factors in increasing the requests of prospective teachers. As a result of the analysis, secondly, the importance of
education, one of the sub-dimension of the scale of the attitude towards the renewable energy sources, was de-
termined to have a low-level positive relation with metacognition, flexibility, systematicity and tenacity-patience,
the sub-dimensions of the scale of the critical thinking dispositions. According to this result, it can be seen that
seeing the education as an important factor to use renewable energy sources properly and effectively, to provide
energy saving and get habits of saving has relationship with prospective teachers’ reflection of their thinking
process, their ability to think on this reflection, to act in a planned and organized manner by thinking distinctively
for the issues and questions and to be persistent in case of failures. Liarakou, Gavrilakis and Flouri (2009) reported
that education plays an important role in informing about renewable energy sources, raising educated individuals
and raising their awareness. In this context, it can be said that metacognition, flexible thinking, systematicity and
tenacity are important factors in providing an awareness that education is important for prospective teachers to
have sufficient knowledge about renewable energy sources. As a result of the analysis, lastly, it was determined
that the country interest, which is the sub-dimension of the scale of the attitude towards renewable energy re-
sources had a low-level positive relationship with flexibility, which is the sub-dimension of the scale of the critical
thinking dispositions. According to this result, it was seen that the thoughts of prospective teachers that the use
of renewable energy sources would increase the development level of the country, contribute to the economy of
the country and would be beneficial for development, had a relationship with the ability of prospective teachers
to develop different points of view for the different encounters and to think distinctively. In the literature on this,
it has been stated that countries have started to prefer renewable energy sources in their energy policies, and they
are in search of different energy sources (Karabulut & Alkan, 2010). Prospective teachers have thought that the use
of renewable energy sources will contribute to the country’s economy, create job opportunities, increase regional
development and reduce dependence on foreign countries (Cebesoy & Karisan, 2017). In this respect, it can be said
that flexible thinking and country interest are important factors in increasing the attitudes of prospective teachers
towards renewable energy resources.
As a result of the canonical correlation analysis conducted in the study, four canonical functions related to the
relationship between participants’ attitudes towards renewable energy sources and critical thinking dispositions
were obtained. Only one canonical function was found to be statistically significant among these functions. It has
been determined that the data sets of attitude towards renewable energy sources and critical thinking disposition
in the first canonical function, which is calculated in order to maximize the relation between the attitudes towards
renewable energy sources and the critical thinking disposition data sets, share a variance of 20.1%. In the second
canonical function, the canonical correlation value, which is not taken into account in the first canonical function
and reveals the maximum relation between the two canonical variables, is calculated. In the second canonical
function, the common variance shared by the attitudes towards the renewable energy sources and the critical
thinking disposition data sets was found to be 1.7%. After removing the common variance shared by the data sets
in the first two canonical functions, the common variance shared by the attitude and critical thinking disposition
data sets for the third canonical function was .7%. Similarly, after removing the common variance shared by the
first three canonical functions, it was determined that the common variance shared by the attitude and critical
thinking disposition data sets for the fourth canonical function was .3%.
In the canonical model consisting of the cumulative values of the canonical functions obtained from the
canonical correlation analysis, the common variance shared by the data sets of critical thinking disposition and
attitudes towards renewable energy sources was calculated as 22.23%. This result shows that there is a relation
between the prospective teachers’ attitudes towards renewable energy sources and their critical thinking. Studies
on this also indicate that the critical thinking of individuals can affect their thoughts, evaluations, decisions and
attitudes towards renewable energy sources (Lee, 2016; Opitz, 2016; Sakschewski, Eggert, Schneider, & Bogeholz,
2014). In this context, the fact that prospective teachers are aware of what he/she should do in order to solve many
environmental problems related to energy such as global warming, acid rain, climate change, disaster and migra-
tion, and what environmental impacts might be in case anything is not done, in a sense, their awareness of their
own metacognition may have positively affected their attitudes towards renewable energy sources. In addition, it
can be effective to reveal this kind of relationship that prospective teachers learn the scientific information about
727
(P. 717-731) ISSN 2538–7138 /Online/
the sources in the classes about the renewable energy sources in the teachers’ training programs, have estimations
for their environmental effects, associate the important connections and see the different thoughts by discussing.
In addition, prospective teachers’ ability to establish relation between different perspectives on environmental
problems in the courses related to renewable energy resources, and to seek answers to the problems in a multi-
faceted and conscious manner, and thus to carry out critical thinking in the classes, may also increase attitudes
towards these sources.
Conclusion and Implications
As a result of this research, it was seen that critical thinking disposition such as metacognition, flexibility, sys-
tematicity, tenacity-patience, and open-mindedness were effective in increasing the attitudes of the future teach-
ers towards renewable energy sources. Considering that it is important for the future students to bring sensitivity
and solution suggestions to environmental problems in line with the changes and developments in the world,
the importance of critical thinking skill which is one of the 21st century skills is emphasized with this research.
In line with the results of the study, critical thinking in teaching activities should be carried out in relation
to environmental factors in these courses related to renewable energy resources in teacher training programs. In
addition, these courses can be done about the renewable energy resources and their investments in the country’s
economy, regional development and interaction with the environment and it should be provided that prospec-
tive teachers think critically. In addition, it is recommended to search the factors that direct the attitudes towards
renewable energy sources and critical thinking dispositions of the prospective teachers. Qualitative studies can
also be conducted for in-depth study of the relationships obtained in this study. In this context, it can be suggested
to conduct new researches to examine the relationship between the attitudes towards renewable energy sources
and critical thinking dispositions by using different data collection techniques such as interview and observation.
Thus, more detailed information about the relation between attitudes towards renewable energy sources and
critical thinking dispositions can be obtained.
References
Akar, C. (2007). Critical thinking of elementary school students. Unpublished PhD Dissertation. Gazi University Institute of
Educational Sciences, Ankara.
Alper, A. (2010). Critical thinking disposition of pre-service teachers. Education and Science, 35(158), 14-27.
Apergis, N., & Payne, J.E. (2010). Renewable energy consumption and economic growth: Evidence from a panel of OECD
countries. Energy Policy, 38, 656-660.
Argon, T., & Selvi, C. (2011). Öğretmen adaylarının eleştirel düşünme eğilimleri ve çatışma yönetim stilleri (Prospective
teachers’ critical thinking dispositions and conflict management styles). Journal of Educational and Instructional
Studies in the World, 1(1), 93-100.
Balocco, C., & Grazzini G. (2007). Plant refurbishment in historical buildings turned into museum. Energy and Buildings,
39(6), 693-701.
Baris, K., & Kucukali, S. (2012). Availability of renewable energy sources in Turkey: Current situation, potential, government
policies and the EU perspective. Energy Policy, 42, 377-391.
Bayrac, H.N. (2010). Effects of energy use on global warming and prevention policies. Eskisehir Osmangazi University Journal
of Social Sciences, 11(2), 229-260.
Benli, H. (2013). Potential of renewable energy in electrical energy production and sustainable energy development of
Turkey: Performance and policies. Renewable Energy, 50, 33-46.
Besoluk, S., & Onder, I. (2010). Investigation of teacher candidates’ learning approaches, learning styles and critical thinking
dispositions. Elementary Education Online, 9(2), 679-693.
Bilen, K., Ozel, M., & Surucu, A. (2013). Pre-service science teachers’ awareness about renewable energy. Dumlupinar Uni-
versity Journal of Social Sciences, 36, 102-118.
Bordens, K. S., & Abbott, B. A. (2011). A process approach to research design and methods. New York: McGraw-Hill.
Can, S., & Kaymakci, G. (2015). Pre-service teachers’ critical thinking tendencies. Education Sciences, 10(2), 66-83.
Cebesoy, U. B., & Karisan, D. (2017). Investigation of preservice science teachers’ knowledge, teaching efficacy perceptions
and attitude towards renewable energy sources. Van Yuzuncu Yil University Journal of Education, 14(1), 1377-1415.
doi: 10.23891/efdyyu.2017.49.
Cekic, S. (2007). The analysis of the power of mathematics teaching degree students on the basis of certain variables. Unpub-
lished master thesis. Dokuz Eylul University Institute of Social Sciences, İzmir.
Celikler, D. (2013). Awareness about renewable energy of pre-service science teachers in Turkey. Renewable Energy, 60,
343-348. https://doi.org/10.1016/j.renene.2013.05.034.
728
ISSN 2538–7138 /Online/ (P. 717-731)
Cetin, A. (2008). The critical thinking ability of the pre-service teachers. Unpublished master thesis. Uludag University Institute
of Social Sciences, Bursa.
Cohen, J. (1968). Multiple regression as a general data-analytic system. Psychological Bulletin, 70, 426-433.
Coker, B., Catlioglu, H. & Birgin, O. (2010). Conceptions of students about renewable energy sources: A need to teach based
on contextual approaches. Procedia-Social and Behavioral Sciences, 2, 1488-1492.
Colwill, I., & Gallagher, C. (2007). Developing a curriculum for the twenty-first century: The experiences of England and
Northern Ireland. Prospects, 37(4), 411-425.
Craft, A. (2003). School and critical thinking. Teaching of Psychology, 30(3), 220-224.
Dayioglu, S. (2003). A descriptive study on the critical thinking levels of the students at the unit of English preparatory school at
Hacettepe University. Unpublished master thesis. Middle East Technical University Institute of Social Sciences, Ankara.
Dincer, I. (1998). Energy and environmental impacts: Present and future perspectives. Energy Sources, 20(4), 427–53.
Dogan, N. (2014). The role of renewable energy resources in fighting against global climate change: An assessment for
Turkey. IIB International Refereed Academic Social Sciences Journal, 5(15), 265-276.
Emir, S. (2012). Eğitim fakültesi öğrencilerinin eleştirel düşünme eğilimleri [Critical thinking dispositions of faculty students].
HAYEF Journal of Education, 9(1), 34-57.
Ennis, R. (1985). Goals for critical thinking curriculum. A. Costa (Ed.), Developing minds. (pp. 54-57). Alexandria, VA: Associa-
tion for Supervision and Curriculum Development.
Erdem, M., Ilgan, A., & Celik, F. (2013). Relationship between emotional intelligence and critical thinking skills of high school
teachers. Turkish Studies, 8(12), 509-532.
Firat, A., Sepetcioglu, H., & Kiraz, A. (2012). Analysis of the attitudes of teacher candidates about renewable energies.
Hacettepe University Journal of Education, Special 1, 216-224.
Fraenkel, J. R., & Wallen, N. E. (2006). How to design and evaluate research in education (6th ed.). New York: McGraw Hill.
Girot, E. A. (2000). Graduate nurses: Critical thinkers or better decision makers? Journal of Advanced Nursing, 31(2), 288-297.
Gulveren, H. (2007) Critical thinking skills of education faculty students and factors influencing critical thinking skills. Unpub-
lished PhD Dissertation. Dokuz Eylul University Institute of Educational Sciencies, İzmir.
Guner, E. D., & Turan, E.S. (2017). The impact of renewable energy sources on global climate change. Journal of Natural
Hazards and Environment, 3(1), 48-55.
Guven, G., & Sulun, Y. (2017). Pre-service teachers’ knowledge and awareness about renewable energy. Renewable and
Sustainable Energy Reviews, 80, 663-668.
Guven, M., & Kurum, D. (2008). The relationship between teacher candidates’ learning styles and critical thinking disposi-
tions. Elementary Education Online, 7(1), 53-70.
Gunes, T., Alat, K., & Gozum, A.I.C. (2013). Renewable for science teacher candidates enegy sources attitude scale: Validity
and reliability study. Journal of Educational Sciences Research, 3 (2), 269-289.
Halder, P., Nuutinen, S., Pietarinen, J., & Pelkonen, P. (2011). Bioenergy and the youth: Analyzing the role of school, home,
and media from future policy perspectives. Applied Energy, 88 (4), 1233-1240.
Halder, P., Pietarinen, J., Havu-Nuutinen, S., & Pelkonen, P. (2010). Young citizens’ knowledge and perceptions of bioenergy
and future energy implications. Energy Policy, 38, 3058-3066.
Haskok, A. S. (2005). The assessment of Turkey`s present energy resources. Unpublished master thesis. Eskisehir Osmangazi
University Graduate School of Natural and Applied Sciences, Eskişehir.
Henson, R. K. (2002, April). The logic and interpretation of structure coefficients in multivariate general linear model analyses.
Paper presented at the Annual Meeting of the American Educational Research Association, New Orleans.
Irani, T., Rudd, R., Gallo, M., Ricketts, J., Friedel, C., & Rhoades, E. (2007). Critical thinking instrumentation manual. Retrieved
from http://step.ufl.edu/resources/critical_thinking/ctmanual.pdf.
Islamoglu, H.A. (2003). Scientific research method. İstanbul: Beta Yayınları.
Jatmiko, B., Prahani, B.K., Supardi, Z.A., Wicaksono, I., Erlina, N., Pandiangan, P., & Althaf, R. (2018). The comparison of oripa
teaching model and problem-based learning model effectiveness to improve critical thinking skills of pre-service
physics teachers. Journal of Baltic Science Education, 17(2), 300-319.
Karabulut, A., & Alkan, M.A. (2010). An empirical study investigating the teaching of renewable energy sources which
are important in the global financial crisis environment at university level. What others manifest. World Econ Theory
Turbul Global Finance Crisis, 420–427.
Karakas, A. (2014). The relationship between electricity consumption, population and income in OECD and non-OECD
countries: A panel date analysis for 1990-2011 period. Turkish Studies, 9(2), 845-853.
Karasar, N. (2016). Scientific research method. Ankara: Nobel Yayın Dağıtım.
Karatepe, Y., Varbak, N., Kecebas, A., & Yumurtaci, M. (2012). The levels of awareness about the renewable energy sources
of university students in Turkey. Renewable Energy, 44, 174-179.
Kilinc, A., Stanisstreet, M., & Boyes, E. (2009). Incentives and disincentives for using renewable energy: Turkish students’
ideas. Renewable and Sustainable Energy Reviews, 13(5), 1089-1095.
Knapp, T.R. (1978). Canonical correlation analysis: A general parametric significance-testing system. Psychological Bulletin,
85, 410-416.
Koca, N., & Bulut, R. (2015). Opinions of prospective social studies teachers about energy resources of Turkey. Turkish Stud-
ies, 10(11), 1007-1022.
729
(P. 717-731) ISSN 2538–7138 /Online/
Kocak, B., Kurtlu, Y., Ulas, H., & Epcacan, C. (2015). Examining of the elementary class teachers’ critical thinking levels and
their attitudes towards reading. EKEV Academy Journal, 19(61), 211-228.
Kokdemir, D. (2003). Decision making and problem solving under uncertainty. Unpublished PhD Dissertation. Ankara Uni-
versity Institute of Social Sciencies, Ankara.
Komor, P., & Bazilian, M. (2005). Renewable energy policy goals, programs, and technologies. Energy Policy, 33, 1873-1881.
Korkmaz, O. (2009a). Teachers’ critical thinking level and dispositions. Journal of Kirsehir Education Faculty, 10(1), 1-13.
Korkmaz, O. (2009b). The influence of education faculties on students’ critical thinking level and disposition. The Journal
of Turkish Educational Sciences, 7(4), 879-902.
Koroneos, C., Spachos, T., & Moussiopoulos, N. (2003). Energy analysis of renewable energy sources. Renewable Energy, 28(2),
295-310.
Krohn, S., & Damborg, S. (1999). On public attitudes towards wind power. Renewable Energy, 16, 954-960.
Kurum, D. (2002). Critical thinking abilities of teacher trainees. Unpublished master thesis. Anadolu University Institute of
Educational Sciences, Eskişehir.
Lee, P.L. (2016). Misconceptions and biases in German students’ perception of multiple energy sources: Implications for
science education. International Journal of Science Education, 38(6), 1306-1056.
Liarakou, G., Gavrilakis, C., & Flouri, E. (2009). Secondary school teachers’ knowledge and attitudes towards renewable
energy sources. Journal of Science Education and Technology, 18(2), 120-129.
Norris, S.P. (1985). Synthesis of research on critical thinking. Educational Leadership, 8, 40- 45.
Opitz, S. (2016). Students’ progressing understanding of the energy concept: an analysis of learning in biological and cross-
disciplinary contexts. Unpublished PhD Dissertation Thesis. Christian-Albrechts Universität Kiel.
Ozdemir, S.M. (2005). Assessing university students’ critical thinking skills for some variables. The Journal of Turkish Edu-
cational Sciences, 3(3), 1-17.
Rudd, R.D. (2007). Defining critical thinking. Techniques: Connecting Education and Careers, 82(7), 46-49.
Sacli, F., & Demirhan, G. (2008). A determination and comparison of critical thinking levels of students in physical education
teacher training program. Hacettepe Journal of Sport Sciences, 19(2), 92-110.
Sakschewski, M., Eggert, S., Schneider, S., & Bogeholz, S. (2014). Students’ socioscientific reasoning and decision-making
on energy-related issues-development of a measurement instrument. International Journal of Science Education,
36(14), 2291-2313.
Sarac, E., & Bedir, H. (2014). Primary school teachers related to perceptions of renewable energy sources on the qualitative
research. Science Journal of Turkish Military Academy, 24(1), 19-45.
Seferoglu, S.S., & Akbiyik, C. (2006). Teaching critical thinking. Hacettepe University Journal of Education, 30, 193-200.
Semerci, N. (2016). The development of critical thinking disposition scale (CTHD): Study on the revision of validity and
reliability. Turkish Studies, 11(9), 725-2740.
Sen, U. (2009). An evaluation about Turkish teacher candidates’ critical thinking attitude’s in terms of difference variable.
Zeitschrift für die Welt der Türken Journal of World of Turks, 1(2), 69-89.
Senlik, N.Z., Balkan, O., & Aycan, S. (2011). Critical thinking dispositions of pre-service teacher: Mugla University example.
C.B.U. Journal of Science, 7(1), 67-76.
Sherry, A., & Henson, R. K. (2005). Conducting and interpreting canonical correlation analysis in personality research: A
user-friendly primer. Journal of Personality Assessment, 84, 37-48.
Stevens, J.P. (2012). Applied multivariate statistics for the social sciences. New York: Routledge.
Tabachnick, B.G., & Fidell, L.S. (2007). Using multivariate statistics (5th ed.). Boston: Allyn and Bacon.
Tas, A.M. (2004). Determination of curriculum standards of social studies teaching. Ankara University Journal of Faculty of
Educational Sciences, 37(1), 28-54.
Tasci, S. (2005). The problem-solving process in nursing. Journal of Health Sciences, 14, 73-78.
Thompson, B. (1991). A primer on the logic and use of canonical correlation analysis. Measurement and Evaluation in
Counseling and Development, 24, 80-95.
Tishman, S., Jay, E., & Perkins, D.N. (1993). Teaching thinking dispositions: From transmission to enculturation. Theory into
Practice, 32(3), 147-153.
Turnuklu, E.B., & Yesildere, S. (2005). A profile from Turkey: Critical thinking dispositions and abilities of pre-service math-
ematics teachers of 11-13 year. Ankara University Journal of Faculty of Educational Sciences, 38(2), 167-185.
Walsh, C.M., & Hardy, R.C. (1999). Dispositional differences in critical thinking related to gender and academic major. Journal
of Nursing Education, 38(4), 149-155.
Wolsing, M. (2007). Wind power implementation: the nature of public attitudes: Equity and fairness instead of ‘backyard
motives’. Renewable & Sustainable Energy Reviews, 11(6), 1188-1207.
Yaman, S., & Yalcin, N. (2005). Effectiveness on creative thinking skills of problem-based learning approach in science
teaching. Elementary Education Online, 4(1), 42-52.
Yanpar Yelken, T. (2009). The effects of materials development based on “creativity activities within a group” on teacher
candidates’ portfolios. Education and Science, 34(153), 83-98.
Yazicioglu, Y., & Erdogan, S. (2014). SPSS uygulamalı bilimsel araltırma yöntemleri [SPSS applied scientific research methods].
Ankara, Detay Yayıncılık.
730
ISSN 2538–7138 /Online/ (P. 717-731)
Yuenyong, C., Jones, A., & Yutakom, N. (2008). A comparison of Thailand and New Zealand students’ ideas about energy
related to technological and societal issues. International Journal of Science and Mathematics Education, 6, 293-311.
Yuksel, I., & Kaygusuz, K. (2011). Renewable energy sources for clean and sustainable energy policies in Turkey. Renewable
and Sustainable Energy Reviews, 15, 4132-4144.
Zayif, K. (2008). Critical thinking dispositions of teachers canditates. Unpublished master thesis. Abant Izzet Baysal University
Institute of Social Sciences, Bolu.
Zhang, L.F. (2003). Contributions of thinking styles to critical thinking dispositions. Journal of Psychology, 137(6), 517-543.
Zyadin, A., Puhakka, A., Ahponen, P., Cronberg, T, & Pelkonen, P. (2012). School students’ knowledge, perceptions, and at-
titudes toward renewable energy in Jordan. Renewable Energy, 45, 78-85.
Received: May 21, 2019 Accepted: September 28, 2019
Gokhan Guven Dr., Research Assistant in Department of Maths and Science

(Corresponding author) Education in Mugla Sitki Kocman University, Turkey.
E-mail: gokhanguven@mu.edu.tr
Nevin Kozcu Cakir Dr., Research Assistant in Department of Maths and Science
Education in Mugla Sitki Kocman University, Turkey.
E-mail: nkozcu@mu.edu.tr
731
BIBLIOMETRIC ANALYSIS
OF THE WOS LITERATURE
ON RESEARCH OF SCIENCE
TEACHER FROM 2000 TO 2017
Abstract. In order to explore the develop- Jianqiang Ye,

ment of research of science teacher(RST), Dimei Chen,
904 articles from the Web of Science (WoS)
Lingxin Kong
core set based on bibliometric methods
through R software were analyzed in this
research. Specifically, it examined the co-
occurrence relations of countries/regions,
major journals, most cited references,
and hot keywords from the macroscopic,
mesoscopic and microscopic level of RST.
The results showed that the core strength Introduction
of RST is mainly from traditional industrial-
ized countries such as the United States, The science teacher plays a significant role in improving students’
Australia, and Britain. And some top jour- scientific literacy, which is an important goal in science education in
nals in science education (such as Journal countries and international organizations. In the past century, science
of Research in Science Teaching, Science education has not only occupied an important position in the school
Education) has to pay more attention on curriculum, but also has become an important aspect that cannot be
RST, it may also appeal to lots of top jour- ignored in education policies of various countries (Abell, 2000). Stepping
nals in general teacher education (such as into the new century, with the promulgation of many “national scientific
Journal of Teacher Education, Teaching and standards” (e.g. the US NGSS), science education has once again triggered
Teaher Education). The research on science a wave of global reform (NRC, 2012). Among them, reformers begin to
teachers was guided by several educational realize that new science courses or innovative teaching techniques need
theories about teacher research, such as
to be implemented by science teachers, and they believe that the reform
the teacher epistemological belief, reflective
of science teachers is a crucial factor in promoting the reforms in science
practice, and PCK. Moreover, theories in
education (Abell, 2000). In science education, competent science teachers
science education such as scientific literacy,
are the decisive factors for students’ science learning (NRC, 1996; NRC,
scientific conceptual change also becomes
2012; Shaharabani & Tal, 2016). Therefore, science teachers are the key to
the theoretical basis for science teachers’
teaching practice and scientific inquiry in-
education reform of all kindergarten to Grade 12 science subjects (K-12).
structing. The knowledge, key competences, However, it is worth noting that until the advent of this century, the
dispositions, and professional development reform of international science education has always been carried out
of science teacher are the main keywords around science curricula and science learning, and there has been a lack
and hot topics in the field of RST. of corresponding attention to science teachers’ competences (Abell, 2000).
Keywords: science teacher research, bib- Rumberger (1985) pointed out that one of the important reasons why
liometric analyses, Web of Science. students’ scientific literacy has not been effectively improved is the lack
of competent science teachers. In its 19th issue in 2013, Science, a leading
international journal, published a special issue named Grand Challenges in
Jianqiang Ye, Dimei Chen
Science Education (Hines, Mervis, Mccartney, & Wible, 2013). The specific
Wenzhou University, China, content of the challenge is not only related to individual science learning,
Lingxin Kong the topic on science teachers has also received great attention. Among
East China Normal University, China
them, the topics related to science teachers’ professional development,
science teachers’ core skills and teaching strategies and science teachers’
732
ISSN 1648–3898 /Print/ BIBLIOMETRIC ANALYSIS OF THE WOS LITERATURE ON RESEARCH OF SCIENCE TEACHER
FROM 2000 TO 2017
ISSN 2538–7138 /Online/ (P. 732-747)
ICT skills, etc. This reveals that the RST is becoming a hot topic in the field of science education research at
present, and appeals to the attention of many academic institutions and scholars.
As a “special group” in the team of teachers, science teachers’ teaching idea, professional dispositions and
teaching behavior not only directly affect students’ learning outcome, but also have effect on their scientific
literacies such as scientific attitude, subject matter, scientific thinking skills and scientific methods (Barnhart
& Van, 2015; Maeng, 2016; Park, Chu, & Martin, 2016; Sakiz & Gonul, 2017; Sansone, 2017; Van Breukelen &
Van Meel, De Vries, 2017; Wallace & Brooks, 2013). As the organizer, participant and leader of science courses’
practice, the basic idea of science course can only be achieved by science teachers’ specific implementation
and organization, and finally transformed into the students’ realistic scientific literacy (Ingersoll, 2011). How-
ever, due to the backwardness of the economy in many countries, schools are not only lacking in hardware
equipment, but also a formal team of science teachers (Adams & Gupta, 2017). Many school-aged children
are rarely able to receive formal science education, and some children do not even receive informal science
education (Abell, 2000). It shows that science teachers play an irreplaceable part in the implementation of
science curriculum teaching, and competent science teachers are the vital factors in the reform of science
curriculum and promotion of students’ adaptation to the challenges caused by the development of science
and technology.
In addition to the lack of competent scientific faculty in teaching practice, the focus of RST in the sci-
ence education is not enough as well. And most of RST papers pay more attention on teachers’ attitudes,
beliefs and identities, and few of them systematically analyze the content of RST (Bryan, 2012; Cronin-Jones,
1991; Enochs & Riggs, 1990; Moore, 1973; Osborne, Simon, & Collins, 2003). For example, Cronin-Jones (1991)
explored the influence of teachers’ beliefs on science teaching practice in two different situations, and the
results turned out that although some elements of teachers’ belief structure do promote the implementa-
tion of the curriculum, their structure of belief is not consistent with the implementation philosophy of the
curriculum. Moore (1973) developed the science teaching attitude scale, which has been divided into three
dimensions: emotional attitude to science teaching, attitude to science content and process, and view about
science teachers’ role. Bryan (2012) pointed out the research on the science teacher’s belief has been formed
in a very complete and comprehensive research system in last two decades. However, Osborne, Simon and
Collins (2003) found that for some research on scientific attitude, primary and middle school students are the
main objects of research, and the research on the attitude of science teachers is much less. Therefore, analyz-
ing the research on science teachers has important reference value for further exploring science teachers’
status and understanding their functions in science teaching and learning practice.
Research Focus
Limited by the technique of statistical tools, the general reviews of RST are mainly based on the ways of
meta-analysis or meta-synthesis, which belong to qualitative analysis. Specifically, the contents of the review
were mainly about knowledge review, belief review and attitude review of science teachers, and there is little
review of RST from a macro perspective, letting alone quantifying the articles of RST from the perspective of
bibliometrics (Bryan, 2012; Osborne, Simon, & Collins, 2003; Schneider & Plasman, 2011). This research aimed
at exploring and analyzing the fundamental situation and attention of the RST in global science education
research since the beginning of the 21st century by using bibliometrics. In order to understand the above
content, the number of annual publications of the core collection of WoS literature on the science teacher
was counted firstly in this research. Based on collected data of literature, the major countries or regions that
occupy the leading and center in the field of science teacher research were further analyzed in the research
from the macroscopic perspective. Then, the core journals, key articles, highly cited references and highly
influential authors RST were analyzed from the mesoscopic level. Finally, the RST high frequency keywords
and main topic areas were analyzed from the micro perspective.
Therefore, the research questions were carried out as follows:
1. What are the main countries, main journals, and main articles of RST?
2. Do bibliometrics effectively explore the theme and keywords of RST?
733
FROM 2000 TO 2017
(P. 732-747) ISSN 2538–7138 /Online/
General Background
Several years ago, the collection and quantitative analysis of scientific literature were basically done
manually (Garfield, 1972). However, after the 21st century, due to the rapid development of information
technology, the processing of document data has accelerated (Chen, Ibekwe-Sanjuan, & Hou, 2010). R-studio
mapping software was used for bibliometric analysis and descriptive analysis in this research. The R environ-
ment provides many packages (e.g. the bibliometrix package) related to bibliometrics through its official
repository. The bibliometrix R package can be very useful for quantitative studies of bibliometrics by authors,
keywords, citation networks and even historiography (Aria & Cuccurullo, 2017). It can visualize the bibliography
of scientific literature by means of bibliometrics, which can provide researchers with intuitionistic data and
models, and facilitate them to discover some important information of research topics. This research used R
software to visualize and quantify the scientific literature in the RST field. The scientific literature from the WoS
core collection 2000-2017 was extracted. The purpose of this research was to explore the research actuality
and main topics in the RST field by performing quantitative analysis on these data.
Data collection
In this research, the data used for bibliometric analysis were collected from the core collection of WoS,
including the Social Sciences Citation Index (SSCI), Science Citation Index Expanded (SCI-E), Conference Pro-
ceedings Citation Index-Social Science & Humanities (CPCI-SSH) and on so on. For example, SSCI and SCIE are
authoritative citation index databases which are widely used in the humanities and social sciences and natural
sciences respectively, both of which are located on the WoS platform. The retrieval topic was set as “science
teacher” and the retrieval time of the data in this research was 2000-2017. Finally, a total of 904 publication
records were obtained using above parameters after eliminating non-English articles.
Data Analysis

The reference data in this research was dealt with matrix creation, data reduction and network matrix
creation, which were automatically processed by software R. Bibliometric analysis of above procedures mainly
includes co-citation, collaboration, co-occurrence, clustering, network mapping and so on (Aria & Cuccurullo,
2017). Finally, the descriptive result and the collaboration network of countries, authors, sources, key words
and hot topics of RST can be achieved via R. Additionally, the bibliometrix R package provides metrics such
as h-index, which quantifies the centrality and influence of countries/regions, authors and sources on RST
(Aria & Cuccurullo, 2017).
Research Results
The Annual Scientific Production of RST
The annual scientific production of RST articles can be divided into three stages. Firstly, from 2000 to
2004, the annual output of the research paper was kept at the lowest level with almost no significant increase.
However, things have changed dramatically since 2005 (the second stage). Specifically, the annual scientific
production of RST articles continued rapid growth until 2013. In the recent years (the third stage), although
the number of RST articles has a slight decline in 2014, it recovered rapidly in 2015 and reached an unprec-
edented peak in 2017. It can be found that the research about the science teacher has gained lots of attention
since 21century according to Figure 1.
734
FROM 2000 TO 2017
ISSN 2538–7138 /Online/ (P. 732-747)
Figure 1. The annual scientific production of RST from WOS.
Macroscopic Level
It can be seen from the Table 1 that the top 20 countries have been collected based on their publications
of RST. Only a simple statistics of the number of papers published by various countries on RST cannot represent
their influence and popularity in this field. Therefore, other key indicators such as SCP, MCP, TC and AAC have been
obtained by statistical calculation. Among them, SCP (Single Country Publications) represents the production of
each county; MCP (Multiple Country Publications) represents cooperation and production among multiple coun-
tries; MCP_Ratio represents MCP divided by the sum of articles; TC represents total citations and AAC represents
average article citations.
The number of published RST papers, SCP and TC values in the United States is far higher than those in other
countries or regions, indicating the centrality and leadership of the United States in the global RST. Specifically,
USA published 282 RST articles during 2000-2017, and all those RST papers have been cited more than 3000 times
(3319) since 2000, and the average article citations of RST articles reached to 12.293 times. Besides, some countries,
such as Turkey, Australia and United Kingdom have higher values on the above three indicators as well, demonstrat-
ing their significance in the field of ST. However, some countries have inconsistency among indicators of Article
publications, SCP and TC. For example, Spain has a fifth production of RST while the TC value ranks eighth (drop
by 60%) and AAC value eleventh (drop by 120%), and the value of TC and AAC even did not reach the average
level (138<280, 5.111 < 7.133). It can be found that Spanish scientific papers of RST have advantages in number,
but the overall quality of the paper in the field becomes less satisfactory. The same goes for Turkey which has the
second largest output of RST papers. However, Turkish total papers in AAC only reach to 4.982, which is far from the
average level of 7.133. In contrary to Spain, Singapore and the Netherlands have less production, but have a high
value of AAC. The value of AAC from Singapore and the Netherlands was cited firstly and secondly, respectively,
indicating their overall quality of the RST papers in the field are much higher.
735
FROM 2000 TO 2017
(P. 732-747) ISSN 2538–7138 /Online/
Table 1. Descriptive analysis of countries.
Country/Region NP NP% SCP MCP MCP_Ratio TC AAC
USA 282 0.40929 256 26 0.0922 3319 12.293
TURKEY 109 0.15820 94 15 0.1376 543 4.982
AUSTRALIA 37 0.05370 30 7 0.1892 287 7.757
UNITED KINGDOM 28 0.04064 25 3 0.1071 223 7.964
SPAIN 27 0.03919 25 2 0.0741 138 5.111
TAIWAN(CHINA) 17 0.02467 14 3 0.1765 186 10.941
CANADA 16 0.02322 13 3 0.1875 163 10.188
KOREA 14 0.02032 7 7 0.5000 74 5.286
SOUTH AFRICA 14 0.02032 12 2 0.1429 54 3.857
BRAZIL 12 0.01742 10 2 0.1667 17 1.417
ISRAEL 12 0.01742 11 1 0.0833 84 7.000
NETHERLANDS 12 0.01742 8 4 0.3333 156 13.000
CHINA(Mainland) 11 0.01597 8 3 0.2727 21 1.909
GERMANY 11 0.01597 7 4 0.3636 51 4.636
SWEDEN 8 0.01161 5 3 0.3750 98 12.250
THAILAND 7 0.01016 5 2 0.2857 31 4.429
SINGAPORE 6 0.00871 6 0 0 99 16.650
ARGENTINA 5 0.00726 2 3 0.6000 9 1.800
NEW ZEALAND 5 0.00726 4 1 0.2000 36 7.200
GREECE 4 0.00581 3 1 0.2500 26 4.000
Average Value 31 0.04623 27 5 0.2269 280 7.133

NOTE: SCP = Single Country Publications; MCP = Multiple Country Publications; MCP_atio = MCP divided by sum of articles; TC = Total
Citations; AAC = Average Article Citations
After clarifying the basic situation of each country in the RST paper, the research continued to use R to visual-
ize the cooperative relations between various countries/regions, seen as Figure 2. The nodes in the Figure 2 are
the main countries, the size of nodes and fonts is determined by their publications, and the lines between nodes
represent the co-occurrence relationship of countries/regions. The number and thickness of lines in the nodes
indicate the closeness or looseness of the links among different countries/regions. According to the correlation
analysis of main countries, the co-occurrence results show that the United States is absolutely central in terms of
node size, number of connections, and thickness, which are basically consistent with the parameters of SCP, and
MCP in Table 1.
736
FROM 2000 TO 2017
ISSN 2538–7138 /Online/ (P. 732-747)
Figure 2. The co-occurrence relations of countries/regions.
Mesoscopic Level
In the macroscopic level, the research focused on the countries and regions that published RST papers. In order
to further explore the sources and authors of the RST articles, the research continued to analyze the publications of
journals involved in RST, and the highly cited papers, the highly cited references of RST, and the top authors in RST.
Major journals
The research collected the top 20 journals which published the maximal RST articles (See Table 2). It can be
found that the publications of journals like the International Journal of Science Education (IJSE), Research in Science
Education (RSE), Journal of Research in Science Teaching (JRST), Science Education (SE) accounted for 70% of top 10
journals’ publications, indicating their leading effect and importance in the area of RST. It also can be found that
the top 20 journals were all most famous sources in the field of science education, indicating RST is one of the most
active areas. However, there are some journals such as Teaching and Teacher Education (TTE), Journal of Teacher
Education (JTE) and Teachers College Record (TCR), which are the leading journals in teacher and teaching research,
indicating RST is one of the most highly valued fields in teacher research as well. Moreover, Computers & Education,
and Innovation and Creativity in Education are two leading journals in the education field of computer-based learning.
In order to analyze the most influential journals in RST, five indexes were calculated by using R software, which
mainly include h index, g index, Total Citation (TC) and NP (Number of Publication) (see Table 2). All these indica-
tors, to a certain extent, reflect the influence and importance of journals in the field of RST. It can be seen from
the Table 2 that JRST and SE are far superior to other journals in all above indicators, indicating their considerable
influence and impact in RST. Besides, IJSE and RSE have optimistic values on all indicators as well. It is worth noting
that both TTE and JTE have entered the top 10 in the rank of all indicators. TTE and JTE are the two most influential
journals in the field of teacher research, indicating that the field of teacher education also pays great attention to
the research on science teachers. However, it can be also found that there have been only ten RST papers published
on JTE, which ranks the last of all journals. JTE’s IF ranks first in teacher education, just higher than SE and a little
below JRST, which are the two top journals in science education.
737
FROM 2000 TO 2017
(P. 732-747) ISSN 2538–7138 /Online/
Table 2. Top 20 active journals in the RST research.
IF
Source h g TC TC% NP NP%
(2017)
JOURNAL OF RESEARCH IN SCIENCE TEACHING 26 43 2032 20.68 68 6.78 3.210
SCIENCE EDUCATION 26 44 2050 20.86 65 6.48 3.035
INTERNATIONAL JOURNAL OF SCIENCE EDUCATION 21 34 1294 12.17 77 7.68 1.325
RESEARCH IN SCIENCE EDUCATION 16 23 777 7.91 74 7.38 1.568
JOURNAL OF SCIENCE EDUCATION AND TECHNOLOGY 9 17 335 3.41 29 2.89 1.375
TEACHING AND TEACHER EDUCATION 9 17 572 5.82 17 1.69 2.473
SCIENCE & EDUCATION 7 12 148 1.51 17 1.69 1.265
JOURNAL OF SCIENCE TEACHER EDUCATION 6 8 124 12.62 52 5.18 NULL
JOURNAL OF TEACHER EDUCATION 6 10 259 2.64 10 1.00 3.180

INTERNATIONAL JOURNAL OF SCIENCE AND MATHEMATICS
5 8 87 0.89 17 1.69 1.086
EDUCATION(IJSME)
COMPUTERS & EDUCATION 5 5 59 0.60 5 0.50 4.538
STUDIES IN SCIENCE EDUCATION 5 5 202 2.06 5 0.50 3.455

EURASIA JOURNAL OF MATHEMATICS SCIENCE AND TECH-
4 7 72 0.73 19 1.90 0.903
NOLOGY EDUCATION
TEACHERS COLLEGE RECORD 4 6 49 0.50 6 0.60 1.072
JOURNAL OF BALTIC SCIENCE EDUCATION 4 5 46 0.47 22 2.19 0.638
INNOVATION AND CREATIVITY IN EDUCATION 4 5 31 0.32 8 0.80 NULL
CHEMISTRY EDUCATION RESEARCH AND PRACTICE 3 5 26 0.27 8 0.80 1.621
RESEARCH IN SCIENCE & TECHNOLOGICAL EDUCATION 3 4 27 0.27 13 1.30 0.513
JOURNAL OF BIOLOGICAL EDUCATION 3 4 21 0.214 4 0.40 0.633
EURASIAN JOURNAL OF EDUCATIONAL RESEARCH 2 3 10 0.10 7 0.70 NULL

NOTE: h: h index; g: g index; TC: Total Citation; NP: Number of Publications; IF: 2017 ISI impact factor
Main references
The most cited research papers that above mentioned provide important achievements of scholars involv-
ing in the research of science teachers and their frontier researches of science teachers, and references below can
provide us with the important literature, basic theory and sources of standards for the above research (See Table 4).
Table 5 lists the 20 most cited references, and it can be concluded that all these references can be divided into four
categories: national standards of science education, science teachers’ knowledge (such as PCK, misconceptions,
conceptions of nature of science (NOS), self-efficacy belief, and research of science teaching practice.
The most cited reference is the National Science Education Standards (NSES) which was first published in 1996,
and its total citation has reached 176. Science teachers’ ability such as planning and designing the inquiry program,
guiding and facilitating science learning, engaging in ongoing assessment of teaching and learning, organizing and
managing classroom environments has been stressed in the chapter of the professional development of science
teachers in NSES. Additionally, many scholars paid attention to the knowledge and concept research of science
teachers since Shulman (1986, 1987) and Posner (1982) proposed the concept of PCK and theory of conceptual
change, respectively. For example, Magnusson, Krajcik and Borko (1999) proposed a framework on teachers’ PCK,
738
FROM 2000 TO 2017
ISSN 2538–7138 /Online/ (P. 732-747)
which contains knowledge of science curricula, assessment of scientific literacy, students’ understanding of core
ideas and instructional strategies. Van Driel, Verloop and Vos (1998) considered teaching experiences contribute to
the generation of PCK, and sufficient subject knowledge is an important basis for the formation of PCK. Van Driel,
Jong, Verloop (2002) investigated the development of PCK of 12 preservice chemistry teachers’ chemical thinking,
and the result showed that pre-service teachers were increasingly aware of the need for clear links between the
macro level and the micro level in the context of chemical teaching, and chemical teachers’ PCK mainly influenced
by their teaching experiences and their advisors.
Another long-term goal of science education is to develop students’ conception of NOS. Primary and second-
ary students’ conceptions of NOS have been emphasized in quite a few articles. Similarly, the research of science
teachers’ NOS also received much attention. Lederman (1992) conducted a review of researches of NOS owing
to lack of both empirical studies (both quantitative and qualitative), and it can be found that science teachers
have not developed a full understanding of NOS, and the disciplinary academic background of teachers does
not significantly affect their understanding of NOS. Abd-El-Khalick and Lederman (2000) summarized two ways
to improve conceptions of NOS: the first one is implicit attempts which utilized engagement in scientific inquiry
activities, and the second one is the explicit attempts which utilized elements from history of science, philosophy
of science and sociology of science.
Finally, research of science teachers’ teaching practice has been given aboard much attention since Schon
(1983) and Wenger (1998) respectively put forward the concepts of Reflective Practitioner and Community of Practice,
and their academic papers have high citations as well in RST (Table 4). For example, Del Carlo, Hinkhouse and Isbell
(2010) developed a framework for science teachers’ reflective practice based on Schon’ study, and elements such as
technical reflection and critical reflection become the core components of their reflective practitioner framework.
Moreover, Forbes and Skamp (2016) explored science teachers’ involvement in a professional development com-
munity of science practice, and the result showed that science teachers’ view and practice about how to address
the science teaching practice in secondary school changed significantly.
Table 3. Top 20 cited reference.
Cited References TC
NATIONAL RESEARCH COUNCIL, 1996, NAT SCI ED STAND 176
SHULMAN L. S., 1986, EDUC RES, V15, P4, DOI:10.3102/0013189X015002004 92
SHULMAN L. S., 1987, HARVARD EDUC REV, V57, P1 82

AMERICAN ASSOCIATION FOR THE ADVANCEMENT OF SCIENCE (AAAS), 1993, BENCHM
62
SCI LIT
PAJARES M. F., 1992, REV EDUC RES, V62, P307, DOI 10.3102/00346543062003307 56
MAGNUSSON S., 1999, EXAMINING PEDAGOGICA, P95 46
MILES M. B., 1994, QUALITATIVE DATA ANA 46
LEDERMAN N. G., 1992, J RES SCI TEACH, V29, P331, DOI 10.1002/TEA.3660290404 43
NATIONAL RESEARCH COUNCIL, 2000, INQ NAT SCI ED STAND 39
GROSSMAN P. L., 1990, MAKING TEACHER TEACH 38
ABD-EL-KHALICK F., 2000, INT J SCI EDUC, V22, P665, DOI 10.1080/09500690050044044 35
SCHON D. A., 1983, REFLECTIVE PRACTITIO 35

VAN DRIEL J. H., 1998, J RES SCI TEACH, V35, P673, DOI 10.1002/(SICI)1098-
35
2736(199808)35:6<673::AID-TEA5>3.0.CO
WENGER E., 1998, COMMUNITIES PRACTICE 35
LINCOLN Y., 1985, NATURALISTIC INQUIRY 33
NESPOR J., 1987, J CURRICULUM STUD, V19, P317, DOI 10.1080/0022027870190403 33
LAVE J., 1991, SITUATED LEARNING LE 32
739
FROM 2000 TO 2017
(P. 732-747) ISSN 2538–7138 /Online/
Cited References TC
GLASER B. G., 1967, DISCOVERY GROUNDED T 29
LEDERMAN N. G., 2002, J RES SCI TEACH, V39, P497, DOI 10.1002/TEA.10034 29
POSNER G. J., 1982, SCI EDUC, V66, P211, DOI 10.1002/SCE.3730660207 29
Leading scholars
The research further analyzed top 20 most productive and most cited authors in RST (see Table 4). It can be
found that Davis, Luft and Ritchie have higher values on publications. For example, their h index and g index are
higher than others. There are some exceptions like Berry and Kind. Their articles are highly cited, but they rank
very bottom other indicators. Because the output of their articles is not high, and especially in recent years there
has been no published article on RST. For example, Berry published three articles on RST in 2008, 2009, 2011 and
2013 respectively, and Bell published a total of four articles from 2011 to 2015. Both two of them scarcely have
academic production of RST in recent years.
The indicators of Davis and Luft are relatively high, their work in the field of RST will be further classified.
Firstly, Davis’s articles on RST dated back to 2004 and continued to 2017 based on the result of WoS. In the early
stage, Davis focused on science teachers’ knowledge development in their teaching practice, such as subject
matter knowledge (SMK) and PCK (Beyer & Davis, 2008; Davis, 2004). Subsequently, Davis’ research transferred to
the key teaching/pedagogical competences research of science teachers. For example, Davis (2010) developed a
project to cultivate science teachers’ ability of science teaching practice. Then, Beyer and Davis (2012) designed a
reform-based curriculum to develop preservice teachers’ pedagogical design capacity. Furthermore, Davis, Kloser
and Wells (2017) studied science teacher educators’ competence to cultivate novice science teachers’ ability of
participating in activities by using rehearsals. Secondly, according to the result of WoS, Luft’s articles on RST dated
back to 2004 and continued to 2017 like Davis. Luft pays more attention on the in-service science teachers’ profes-
sional development (e.g. the newly hired/beginning/novice secondary science teacher) (Luft, 2007, 2011, 2015). For
example, Luft (2011) explored beginning science teachers’ development through a two-year mixed methods study,
the result of science teachers’ knowledge, belief and practice were strengthened through an induction program.
Table 4. Top 20 most influential scholars publishing in RST.
Most productive authors Most cited authors
NAME TC NP TC /NP h g NAME TC NP TC /NP h g
DAVIS E. A. 201 9 22.33 6 9 NIESS M. L. 319 1 319.00 1 1
LUFT J. A. 302 8 37.75 6 8 LUFT J. A. 302 8 37.75 6 8
RITCHIE S. M. 89 8 11.13 5 8 BRYAN L. A. 265 3 88.33 3 3
DEMIRDOGEN B. 36 7 5.14 4 6 WINDSCHITL M. 232 4 58.00 2 4
MAENG J. L. 73 7 10.43 3 7 CRAWFORD B. A. 228 1 228.00 1 1
CLOUGH M. P. 70 6 11.67 4 6 DAVIS E. A. 201 9 22.33 6 9
FIRMAN H. 2 6 0.33 1 1 HANEY J. J. 191 2 95.50 2 2
KAYA O. N. 124 6 20.67 5 6 ABELL S. K. 185 2 92.50 2 2
MENSAH F. M. 25 6 4.17 2 5 LUEHMANN A. L. 168 2 84.00 2 2
EBENEZER J. 85 5 17.00 5 5 SETTLAGE J. 164 4 41.00 4 4
HERMAN B. C. 45 5 9.00 3 5 KIND V. 162 4 40.50 4 4
740
FROM 2000 TO 2017
ISSN 2538–7138 /Online/ (P. 732-747)
Most productive authors Most cited authors
NAME TC NP TC /NP h g NAME TC NP TC /NP h g
MARTIN S. N. 21 5 4.20 2 4 BERRY A. 156 4 39.00 4 4
OLSON J. K. 62 5 12.40 3 5 ATWATER M. M. 153 2 76.50 2 2
RAGONIS N. 18 5 3.60 3 4 VAN DRIEL J. 153 3 51.00 3 3
TOBIN K. 83 5 16.60 4 5 BELL R. 149 5 29.80 5 5
BELL R. L. 115 4 28.75 4 4 SOUTHERLAND S. 142 6 23.67 5 6
BERRY A. 156 4 39.00 4 4 VAN DRIEL J. H. 141 2 70.50 2 2
BIANCHINI J. A. 98 4 24.50 4 4 DE JONG O. 139 2 69.50 2 2
KIND V. 162 4 40.50 4 4 TSAI C. C. 135 4 33.75 4 4
ROTH W. M. 105 4 26.25 4 4 SOUTHERLAND S. A. 131 4 32.75 4 4

NOTE: h = h index; g = g index; TC = Total Citation; NP= Number of Publications; IF = 2017 ISI impact factor
Additionally, there are two authors (Niess and Crawford) published only one RST article, but the number of
citations of these two articles is as high as 319 and 228 respectively, ranking first and fifth. Firstly, Niess’s (2005)
article published in TTE, which has been cited most frequently (TC=319, TC per Year=24.86. Niess’s (2005) study
focused on TPCK of preservice science teacher, and Niess (2005) pointed out that preparing science teachers to
teach with technology is one of key competences of the science teacher in 21st century. Niess’s work in TTE, to
some extent, reveals his focus on science teachers in the research field of disciplinary teacher. Crawford (2007)
investigated five student teachers’ knowledge, beliefs, and efforts in enacting teaching science as inquiry over a
year interns in a high school, it turned out to be that the teacher’s complex belief has an important influence on
preservice teachers’ intentions and abilities.
There are also quite a few authors with high RST citations like Windschit, Abell, Van Driel and so on. Windschitl’s
(2003) outcome in SE paid an attention to the inquiry projects as well. Windschitl (2003) examined the influence of
pre-service teachers’ conception of scientific inquiry based on their experience in the course of science methods.
Windschitl (2003) advocated that the scientific inquiry experiences should be attached to the cultivation stage of
preservice education, and Windschitl believed that relevant experience in pre-service stage could be scaffolded
to promote individuals to deeply understand the nature of inquiry. Abell (2008) and Kind (2009) both focused on
PCK of science teacher since the notion of PCK has been developed in last two decades. Van Driel (2002) explored
the development of PCK of preservice chemistry teacher by focusing on chemical thinking, which involved the
transformation between the macro and micro levels. Additionally, Lumpe (2000) and Bryan (2003) examined and
assessed science teachers’ belief about science teaching context.
Microscopic Level
Most relevant keywords
After the macroscopic analysis of the country and the mesoscopic analysis of the journals and authors in the
field of RST, the study conducted a micro analysis of the RST based on researchers’ keywords and keyword-plus (see
Table 5). Considering that the abstract in many papers does not contain keywords, WoS can provide researchers with
keywords-plus. Therefore, the research collected all two parts of keywords for further research and analysis. It can
be founded that all those key words can be classified into four dimensions which are science teacher education,
science teachers’ knowledge, science teachers’ key competencies and dispositions. Firstly and most importantly,
science teacher education, and its frequency ranked first among all the keywords. Additionally, keywords like pro-
fessional development, teacher development, and teacher professional development all relate to science teacher
education, indicating that science teachers’ education and their professional development is a hotspot in the RST
field. Secondly, the research of science teachers’ knowledge such as PCK, SMK, TPACK, NOS and conceptual change
741
FROM 2000 TO 2017
(P. 732-747) ISSN 2538–7138 /Online/
has been paid much attention as well. Thirdly, keywords like inquiry, science teaching, argumentation, reflection,
discourse, teacher learning and action research aimed at science teachers’ key competencies, which also accounted
for a very large percentage of all keywords in RST articles. Finally, the other focus of RST is science teachers’ disposi-
tions which mainly include belief, self-efficacy, identity and equity.
Table 5. Most relevant keywords.
Author Keywords(DE) Articles Keywords-Plus (ID) Articles
Science teacher education 92 Education 160
Science education 63 Knowledge 100
Teacher education 38 Students 94
Nature of science 26 Beliefs 87
Pedagogical content knowledge 25 Conceptions 61
Professional development 23 Inquiry 60
Science teacher 19 Instruction 48
Science teaching 18 Classroom 46
Teacher beliefs 17 Reform 40
Science 16 Views 40
Inquiry 12 Science 36
Teacher development 12 Pedagogical content knowledge 35
Conceptual change 10 Professional-development 32
Pre-service science teachers 9 School 31
Self-efficacy 9 School science 30
Environmental education 8 Teachers 30
Pre-service teacher education 8 Framework 26
Teacher professional development 8 Attitudes 25
Chemistry education 7 Curriculum 25
Social relations of most relevant keywords
Although the most relevant keywords in Table 5 show their frequency in the RST field more intuitively, it does
not explain the relationship among these keywords. Therefore, the study continued to explore the relations among
the above keywords based on their co-occurence by social network analysis (see Figure 4). In the social network,
the relevance and closeness between the keywords can be judged based on the number and thickness of the links
between the keywords (Batagelj & Mrvar, 2004). It can be founded that keywords such as science education, teacher
education and science teacher education have lots of connections and these connections are very thick. For example,
keyword science teacher education is closely related to professional development, PCK, indicating RST focus on science
teachers’ PCK and professional development. It can be seen that keyword science education also becomes a very
hot keyword, which also has close relation with pre-service teachers, nature of science, teacher beliefs, identity, equity,
indicating that they are main topics in RST field (Luehmann, 2007; Nuangchalerm & Prachagool, 2010). Although
the links between some keywords (such as misconception and conceptual change) in the social network are not
as obvious as the keywords like teacher education, science education and science teacher education, the research
on the conceptual change of science teachers is still an important topic in the field of RST, and has received lots of
attention by many scholars (Kartal, Öztürk, & Yalvaç, 2011; Lawrenz, 1986; Stofflett, 1994 ).
742
FROM 2000 TO 2017
ISSN 2538–7138 /Online/ (P. 732-747)
Figure 3. Most relevant keywords.
Discussion
Today, with the deepening of globalization and the rapid development of information technology in the
world, how to innovate science teaching in school that can improve students’ science literacy, which puts forward
new challenges and requirements for the professional competence of science teachers in middle schools (Davis
& Petish, 2006). Davis and Petish stressed that the current content of teachers in science education needs to deal
with the challenges, which mainly include understanding of science subjects, subject matter, learner, teaching,
learning environment and professional development. Therefore, the research on science teachers has received the
attention and focus by many international scholars.
This research aimed to combine the international RST articles through the bibliometrics way, and then explore
the major countries, core journals, highly cited literature and hot topics and keywords in the RST field from the
macroscopic, mesoscopic and microscopic perspectives, respectively. It is helpful for researchers to clearly iden-
tify the status and characteristics of different countries or regions in the field of RST by analyzing the number of
publications and cooperation in the field of RST from a macro perspective. Additionally, the analysis of the most
influential RST papers and authors, as well as the core journals that publish RST papers, can help relevant readers
and scholars in the RST field grasp the leading articles, authors and major journals from the mesoscopic perspective.
Finally, analyzing the core topics and keywords in the RST field from a micro level can help researchers have access
to catch the current themes and trends in the RST field.
From the bibliometrics analyses’ results, it can be found that the publications on the research of science
teachers has increased significantly since the new century (Figure 1). It has become increasingly popular in both
developed and developing countries, where cooperation in the area of RST has become more frequent and closer
(Figure 2). And it can be also seen in Table 1 that more and more countries or regions focus on RST (e.g.,Thailand,
Indonesia, and China). The level of RST in a country relies on its research system and infrastructure of science edu-
cation, and the strength of RST also reflects the country’s scientific technology and industrial level. Additionally, the
number of publications in the RST field is rapidly increasing in developing countries such as China and Indonesia.
However, the influence and centrality of these countries in the field of RST still need a long time of continuous
efforts and developments. The leadership of traditional industrialized countries such as the United States, Canada
and Netherlands in the RST field is not only reflected in the high publications, but also in the high citation rate of
articles published (e.g. TC and AAC). And their cooperation with other countries is very close (see MCP). Develo-
743
FROM 2000 TO 2017
(P. 732-747) ISSN 2538–7138 /Online/
ping countries with high population densities such as China and Indonesia should have made a difference in RST
because these countries lack a large number of professional and competent science teachers. However, due to the
uneven development of the domestic region, the path of cooperation with developed countries seems feasible.
Quite a few journals have been able to provide access and channels for RST scholars (see Table 2). It can be
found that the attention to the science teachers is not only reflected in the field of science education, but also in
the journals of common teacher education. Additionally, the interdisciplinary and transdisciplinary education is also
given great attention to the science teacher. The research on science teachers has also been guided by educational
theories in the field of teacher research, such as Pajares’s teacher belief (1992), Schon’s (1983) reflective practice,
and Shulman’s PCK(1986, 1987) (see Table 3). Moreover, theories in science education such as Miles’(1994) scientific
literacy, Posner’s (1982) theory of scientific conceptual change provides the theoretical basis for science teachers’
teaching practice and scientific inquiry instructing. Nonetheless, it should be noted that RST is still not an entirely
mature and compelete area, and it is still developing by adoptting more conceptions and frameworks (e.g. science
teachers’ knowledge, key competences, dispositions, science teacher education and their professional development)
from other ﬁelds such as educational psychology, teacher education, and global cooperation (Table 5 and Figure
3). It can be found that the above contents show the following characterics in chronological order: the scientific
attitude of science teachers in the 1960s and 1970s, the pedagogical content knowledge of science teachers in
the 1980s, the science teachers’ view on nature of science in the 1990s to the 21st century. Recently, there has
been a gradual shift to the study of scientific thinking skills (such as scientific reasoning, scientific argumentation
and scientific explanation) and core competence (e.g. core scientific teaching practice, and instructional design
capacity) of science teachers. Generally speaking, the research of science teachers is complex and systematic, so it
is difficult to make a systematic review of all RST articles. And What needs to be clear is that the research of science
teachers must reflect nature of science, and the research should fully reflect the commonality, universality and
generality of science.
Conclusions
The aim of this research was to provide a systematic overview of WoS literature on science teacher published in
international journals by using bibliometric analysis with R software. Therefore, several conclusions were obtained
based on the research questions.
Firstly, at present, the core strength of RST is mainly from western developed countries such as the United
States, Turkey, Australia, Britain and Spain, which are the backbone of RST and have a high influence in this field.
Although developing countries such as China, Brazil, South Africa and Thailand ranked fairly well in the number of
publications, their TC and AAC values of RST papers were relatively low, and there was less cooperation and contact
between countries. Therefore, developing countries need to continuously invest human and material resources to
support domestic scholars to carry out in-depth learning and cooperation with other international scholars if they
want to achieve the same influence in this field as the developed countries.
Moreover, the international RST field has attracted some top journals of science education (such as JRST, SE
and IJSE), and the RST field also has been valued by top journals of teacher education (such as JTE and TTE). The
attention on RST in international science education journal reveals that science teachers have become the research
hotspots and trends in science education. However, it is worth noting that, unlike JRST and IJSE, which publish a
large number of RST articles, some teacher research journals such as JTE and TTE are still insufficiently concerned
about science teachers, indicating that the status and importance of science teachers in general teacher education
research field still need to be improved. Additionally, this section also analyzes the main articles of RST and the
main references for RST. The former research prefers the knowledge of science teachers (such as PCK, SMK, NOS)
and key competences like scientific practice, while the latter one mainly contains the national science education
standards (such as NGSS) and general teacher research results (such as Shulman’s view about teacher knowledge
and Schoen’s research of reflective practice), etc. These articles in science education and teacher education research
field provide important theoretical basis and reference value for international RST.
Secondly, the theme and keywords of RST was effectively explored by bibliometrics analyzing. Specifically, the
main keywords and their relations in the field of RST were summarized and analyzed. The main keywords mainly
involve several dimensions, such as knowledge, ability and professional development. Since the new century, the
international science teacher education has experienced three themes: firstly, theme of RST mainly refers to and
revolves around these documents and policies due to the science standards and documents issued by countries
744
FROM 2000 TO 2017
ISSN 2538–7138 /Online/ (P. 732-747)
represented by the United States. The second topic mainly involves the research of science teachers’ knowledge,
skills, abilities and professional dispositions. The third one focuses on science teachers’ education and professional
development by constructing different professional learning communities.
It can be concluded that the main content of these documents (NGSS) mainly includes the following key ele-
ments: big ideas (core conceptions), learning progression, science and engineering practice. And these standards
mainly focused on students’ science learning, which involved three- dimensional performance expectations: core
ideas, crosscutting concepts and practices. However, it is worth noting that the evaluation of these core concepts
has gradually concerned the scholars and the government. They pay much attention to develop the measurement
of evaluating students’ science learning and conducted the quasi-experiment research in the designed science
classroom.
Implications
Science education shoulders the important task of cultivating scientific and technological innovation talents
and improving citizens’ scientific literacy in the future, and science teachers with good competence become the
most important factors in the development of science education and the cultivation of students’ scientific literacy
(NRC, 1996, 2012; Shaharabani & Tal, 2016). Many studies have shown that the lack of competent science teachers
is one of the important reasons for the ineffective improvement of teenagers’ scientific literacy (Rumberger, 1985).
This phenomenon reveals that the research on science teachers is becoming a hot topic in the field of science edu-
cation research, and has appealed to many academic institutions and scholars (Hines et al., 2013). As the organizer,
participant and executor of science curriculum teaching and learning, the basic idea of science curriculum can only
be implemented through the specific implementation of science teachers, and finally transformed into the realis-
tic scientific literacy of students. However, at present, in both developed and developing countries, their science
education and science teacher preparation are insufficient to meet the demands and expectations of the govern-
ment, citizens and educators for the cultivation of scientific literacy (Ingersoll, 2011). It can be seen that science
teachers play an irreplaceable role in the implementation of science curriculum teaching, and competent science
teachers are a vital factor in the reform of science education and the development of students’ scientific literacy.
References
Alvermann, D. E., & Wilson, A. A. (2011). Comprehension strategy instruction for multimodal texts in science. Theory into Prac-
tice, 50(2), 116-124.
Aria, M., & Cuccurullo, C. (2017). bibliometrix: An R-tool for comprehensive science mapping analysis. Journal of Informetrics, 11(4),
959-975.
Barnhart, T., & van Es, E. (2015). Studying teacher noticing: Examining the relationship among pre-service science teachers’ ability
to attend, analyze and respond to student thinking. Teaching and Teacher Education, 45, 83-93.
Batagelj, V., & Mrvar, A. (2004). Pajek—analysis and visualization of large networks. Berlin, Heidelberg: Springer.
Beyer, C. J., & Davis, E. A. (2008). Fostering second graders’ scientific explanations: A beginning elementary teacher’s knowledge,
beliefs, and practice. The Journal of the Learning Sciences, 17(3), 381-414.
Beyer, C. J., & Davis, E. A. (2012). Developing preservice elementary teachers’ pedagogical design capacity for reform-based cur-
riculum design. Curriculum Inquiry, 42(3), 386-413.
Bryan, L. A. (2012). Research on science teacher beliefs. In B. Fraser, K. Tobin, & C. J. McRobbie (Eds.), Second international handbook
of science education (pp. 477-495). Dordrecht: Springer.
Chen, C., Ibekwe-SanJuan, F., & Hou, J. (2010). The structure and dynamics of co-citation clusters: A multiple-perspective co-
citation analysis. Journal of the American Society for information Science and Technology, 61(7), 1386-1409.
Crawford, B. A. (2007). Learning to teach science as inquiry in the rough and tumble of practice. Journal of Research in Science
Teaching, 44(4), 613-642.
Cronin-Jones, L. L. (1991). Science teacher beliefs and their influence on curriculum implementation: Two case studies. Journal
of Research in Science Teaching, 28(3), 235-250.
Davis, E. A. (2004). Knowledge integration in science teaching: Analyzing teachers’ knowledge development. Research in Science
Education, 34(1), 21-53.
Davis, E. A., & Smithey, J. (2009). Beginning teachers moving toward effective elementary science teaching. Science Educa-
tion, 93(4), 745-770.
Davis, E. A., Kloser, M., Wells, A., Windschitl, M., Carlson, J., & Marino, J. C. (2017). Teaching the practice of leading sense-making
discussions in science: Science teacher educators using rehearsals. Journal of Science Teacher Education, 28(3), 275-293.
Dawson, V. M., & Venville, G. (2010). Teaching strategies for developing students’ argumentation skills about socio-scientific issues
in high school genetics. Research in Science Education, 40(2), 133-148.
745
FROM 2000 TO 2017
(P. 732-747) ISSN 2538–7138 /Online/
Del Carlo, D., Hinkhouse, H., & Isbell, L. (2010). Developing a reflective practitioner through the connection between educational
research and reflective practices. Journal of Science Education and Technology, 19(1), 58-68.
Eick C, Dias M. (2005). Building the authority of experience in communities of practice: The development of preservice teachers’
practical knowledge through coteaching in inquiry classrooms. Science Education, 89(3), 470-491.
Enochs, L. G., & Riggs, I. M. (1990). Further development of an elementary science teaching efficacy belief instrument: A preservice
elementary scale. School Science and Mathematics, 90(8), 694-706.
Forbes, A., & Skamp, K. (2016). Secondary science teachers’ and students’ involvement in a primary school community of science
practice: How it changed their practices and interest in science. Research in Science Education, 46(1), 91-112.
Garfield, E. (1972). Citation analysis as a tool in journal evaluation. Science, 178(4060), 471-479.
Glenn, J. (2000). Before it’s too late: A report to the nation from the national commission on mathematics and science teaching for
the 21st century. Washington, DC: NCMST.
Hines, P. J., Mervis, J., Mccartney, M., & Wible, B. (2013). Grand challenges in science education. Plenty of challenges for all. Sci-
ence, 340(6130), 290-291.
Ingersoll, R. M. (2011). Do we produce enough mathematics and science teachers? Phi Delta Kappan, 92(6), 37-41.
Kartal, T., Öztürk, N., & Yalvaç, H. G. (2011). Misconceptions of science teacher candidates about heat and temperature. Proce-
dia-Social and Behavioral Sciences, 15, 2758-2763.
Lakshmanan, A., Heath, B. P., Perlmutter, A., & Elder, M. (2011). The impact of science content and professional learning communi-
ties on science teaching efficacy and standards-based instruction. Journal of Research in Science Teaching, 48(5), 534-551.
Lawrenz, F. (1986). Misconceptions of physical science concepts among elementary school teachers. School Science and Math-
ematics, 86(8), 654-660.
Lederman, N. G., Abd-El-Khalick, F., Bell, R. L., & Schwartz, R. S. (2002). Views of nature of science questionnaire: Toward valid and
meaningful assessment of learners’ conceptions of nature of science. Journal of research in science teaching, 39(6), 497-521.
Luehmann, A. L. (2007). Identity development as a lens to science teacher preparation. Science Education, 91(5), 822-839.
Luft, J. (2007). Minding the gap: Needed research on beginning/newly qualified science teachers. Journal of Research in Science
Teaching, 44(4), 532-537.
Luft, J. A., Firestone, J. B., Wong, S. S., Ortega, I., Adams, K., & Bang, E. (2011). Beginning secondary science teacher induction: A
two-year mixed methods study. Journal of Research in Science Teaching, 48(10), 1199-1224.
Luft, J. A., Dubois, S. L., Nixon, R. S., & Campbell, B. K. (2015). Supporting newly hired teachers of science: Attaining teacher pro-
fessional standards. Studies in Science Education, 51(1), 1-48.
Maeng, J. L. (2017). Using technology to facilitate differentiated high school science instruction. Research in Science Educa-
tion, 47(5), 1075-1099.
Magnusson, S., Krajcik, J., & Borko, H. (1999). Nature, sources, and development of pedagogical content knowledge for science
teaching. Examining pedagogical content knowledge (pp. 95-132). Dordrecht: Springer.
National Research Council. (1996). National science education standards. Washington, DC: National Academies Press.
National Research Council. (2000). Inquiry and the national science education standards: A guide for teaching and learning. Wash-
ington, DC: National Academies Press.
National Science Teachers Association. (1998). Standards for science teacher preparation. Arlington, VA: National Science Teachers
Association.
National Research Council. (2012). A framework for K-12 science education: Practices, crosscutting concepts, and core ideas. Wash-
ington, DC: National Academies Press.
Nuangchalerm, P., & Prachagool, V. (2010). Influences of teacher preparation program on preservice science teachers’ beliefs. In-
ternational Education Studies, 3(1), 87-91.
Osborne, J., Simon, S., & Collins, S. (2003). Attitudes towards science: A review of the literature and its implications. International
Pellegrino, J. W., Wilson, M. R., Koenig, J. A., & Beatty, A. S. (2014). Developing Assessments for the Next Generation Science Standards.
Washington, DC: National Academies Press.
Park, J., Chu, H. E., & Martin, S. N. (2016). Exploring how Korean teacher’s attitudes and self-efficacy for using inquiry and language
based teaching practices impacts learning for culturally and linguistically diverse students: Implications for science teacher
education. Eurasia Journal of Mathematics, Science & Technology Education, 12(7), 1799-1841.
Rumberger, R. (1985). The shortage of mathematics and science teachers: A review of the evidence. Educational Evaluation and
Policy Analysis, 7(4), 355-369.
Sadler, P. M., Sonnert, G., Coyle, H. P., Cook-Smith, N., & Miller, J. L. (2013). The influence of teachers’ knowledge on student learn-
ing in middle school physical science classrooms. American Educational Research Journal, 50(5), 1020-1049.
Sakiz, G. (2017). Perceived teacher affective support in relation to emotional and motivational variables in elementary school
science classrooms in Turkey. Research in Science & Technological Education, 35(1), 108-129.
Sansone, D. (2017). Why does teacher gender matter?. Economics of Education Review, 61, 9-18.
Schneider, R. M., & Plasman, K. (2011). Science teacher learning progressions: A review of science teachers’ pedagogical content
knowledge development. Review of Educational Research, 81(4), 530-565.
Shaharabani, Y. F., & Tal, T. (2017). Teachers’ practice a decade after an extensive professional development program in science
education. Research in Science Education, 47(5), 1031-1053.
Stofflett, R. T. (1994). The accommodation of science pedagogical knowledge: The application of conceptual change constructs
to teacher education. Journal of Research in Science Teaching, 31(8), 787-810.
746
FROM 2000 TO 2017
ISSN 2538–7138 /Online/ (P. 732-747)
Taylor, J. A., & Dana, T. M. (2003). Secondary school physics teachers’ conceptions of scientific evidence: An exploratory case
study. Journal of Research in Science Teaching: The Official Journal of the National Association for Research in Science Teach-
ing, 40(8), 721-736.
Windschitl, M. (2003). Inquiry projects in science teacher education: What can investigative experiences reveal about teacher
thinking and eventual classroom practice? Science Education, 87(1), 112-143.
Wallace, C. S., & Brooks, L. (2015). Learning to teach elementary science in an experiential, informal context: Culture, learning,
and identity. Science Education, 99(1), 174-198.
Van Breukelen, D., Van Meel, A., & De Vries, M. (2017). Teaching strategies to promote concept learning by design challenges. Re-
search in Science & Technological Education, 35(3), 368-390.
Van Driel, J. H., Verloop, N., & De Vos, W. (1998). Developing science teachers’ pedagogical content knowledge. Journal of Research
in Science Teaching: The Official Journal of the National Association for Research in Science Teaching, 35(6), 673-695.
Van Driel, J. H., Jong, O. D., & Verloop, N. (2002). The development of preservice chemistry teachers’ pedagogical content knowl-
edge. Science Education, 86(4), 572-590.
Jianqiang Ye PhD, Associate Professor, College of Chemistry and Materials

Engineering, Wenzhou University, Zhejiang 325035, China.
E-mail: yjqyszj@163.com
ORCID: https://orcid.org/0000-0002-0672-6385
Dimei Chen M.Ed., Professor, College of Chemistry and Materials Engineering,
(Corresponding author) Wenzhou University, Zhejiang 325035, China.
E-mail: cdm8203@163.com
Lingxin Kong PhD Candidate, College of Teacher Education, East China Normal
University, Shanghai, 200062, China.
E-mail: 13210920726@163.com
747
EVALUATION, VALIDATION AND
MODIFICATION OF SCIENCE
MOTIVATION QUESTIONNAIRE
FOR UPPER SECONDARY
SCHOOL
Abstract. Motivation to learn a school Vanda Janštová,

subject is crucial for achieving knowledge
Andrej Šorgo
and skills and, consequently, for good aca-
demic performance in the subject. There are
several approved questionnaires dedicated
to assessing motivation for science and bi-
ology but none for upper secondary school
Czech students. Therefore, existing Science
Introduction
Motivation Questionnaire was evaluated,
validated and modified in order to create
Biology teachers all around the globe are trying to find the best way
a Czech version of a Biology motivation
how to get their students involved in science, motivate them and ensure
questionnaire (BMQ). The original version
they gain knowledge, skills, and positive attitudes toward biology at the
of the questionnaire had 30 questions, ar-
same time. From the historical perspective someone can recognize continous
ranged in the following subscales: Internal
experimenting with many different methods, forms, strategies, technologies,
Motivation, External Motivation, Personal
etc. wishfully leading toward better Biology education (e.g. Michael, 2006).
Relevance, Responsibility, Self-efficacy and
However, it seems that one of the keys to better Biology education is motiva-
Anxiety. Validation was based on a survey
tion towards both the content and the process of aquiring knowledge, skills
of 517 upper secondary school Czech
and positive attitude. In order to gain knowledge about different types of
students. Analysis of the questionnaires
motivation (see bellow), with the aim of making biology education better,
revealed that, although the questionnaire
tested instruments are needed to evaluate motivation before and after the
could be used as a whole with satisfactory
intervention. Only then, and following the evidence provided by the test
Cronbach alpha values, the underlying
results, teachers and others involved in curriculum design, can introduce
constructs in the Czech version were differ-
changes leading toward better knowledge and appreciation of Biology as
ent from the six ones originally suggested.
an interesting discipline (Dohn, Fago, Overgaard, Madsen, & Malte, 2016),
Based on Exploratory Factor Analysis and
worth to be learned or chosen, when election is an option (Bathgate & Sc-
Structural Equation Modelling, 14 ques-
hunn, 2016). Hopefully, students motivated for Biology and related topics
tions retained in three constructs: a) Self–
were more likely to show career aspirations in STEM (Science, Technology,
efficacy, b) Responsibility and c) Motivation
Engineering and, Mathematics) as a plausible study choice and follow up
as a combination of Internal Motivation,
career (Šorgo et al., 2018).
External Motivation and Personal Rel-
Motivation is perceived as an internal state and can be recognized as a
evance. This shorter version of the BMQ
psychological process determining the intensity, direction and persistence
showed satisfactory Goodness of Fit Indices
of learning-related behaviour. If someone is motivated, he/she is moved
in contrast to the original 30-item version.
to do something (Beihler & Snowman, 1997; Ryan & Deci, 2000); therefore,
Keywords: biology motivation question-
motivation to learn is crucial for successful learning. Motivation to learn is
naire, questionnaire modification, EFA,
linked to achievement in the subject, with self-efficacy (Bandura, 1977) be-
secondary school students.
ing the motivational factor most related to achievement because it directly
influences the academic goals that students set for themselves (Zimmerman,
Bandura, & Martinez-Pons, 1992). There exists a number of studies connecting
Vanda Janštová self-efficacy with different aspects of learning and teching biology. Lawson,
Charles University, Czech Republic
Andrej Šorgo Banks, and Logvin (2007) connected SE with reasoning ability and find positive
University of Maribor, Slovenia correlatation; moderate correlations were established with academic perfor-
mance (Honicke & Broadbent, 2016).
748
ISSN 1648–3898 /Print/ EVALUATION, VALIDATION AND MODIFICATION OF SCIENCE MOTIVATION QUESTIONNAIRE
ISSN 2538–7138 /Online/ (P. 748-767)
To paraphrase Glynn, Brickman, Armstrong, and Taasoobshirazi (2011), motivation to learn biology can be
defined as an internal state that arouses, directs and sustains biology-learning behavior. The connection between
motivation, and different aspects of teaching and learning Biology was approved in numerous studies (e.g. Dyrberg,
Treusch, & Wiegand, 2017; Jeno, Grytnes, & Vandvik, 2017; Mahler, Großschedl, & Harms, 2017).
However, learning is not influenced only by the higher or lower levels of motivation but also by the type of
motivation. Deci and Ryan (1985) formulated a self-determination theory distinguishing between different types
of motivation: intrinsic and extrinsic motivation and amotivation. Additionally, extrinsic motivation consists of
several subtypes as reviewed by Ryan and Deci (2000) who further discriminate external control and true self-
regulation. According to theory, intrinsic motivation leads to performance of an activity for satisfaction, because
of interest, curiosity and will to explore, and is recognized as the preferred incentive for success in schoolwork.
However, many (if not most) school activities are extrinsically motivated, or more reliably, the motivational drive
is a combination of internal and external motives. Gagné and Deci (2005) recognized that activities that are not
intrinsically motivating require extrinsic motivation. Extrinsic motives can be internalized, meaning that they
can be passively and even actively accepted and integrated. In the educational context ‘it appears that intrinsic
motivation (for active exploring and learning) becomes weaker with each advancing grade’ (Ryan & Deci, 2000, p.
60), possibly because the extrinsic tangible rewards widely used in schools undermine internal motivation (Deci,
Koestner, & Ryan, 2001). Ryan and Deci (2000) described the further structuring of external motivation, including
external regulation, introjection, identification and integration. In education, motivation most of the time cannot
be separated from the educational context and activities leading to a concept of situational motivation as moti-
vation towards the specific activity in which students are engaged ‘here and now’ (Guay, Vallerand, & Blanchard,
2000; Vallerand, 1997). A Situation Motivational Scale consisting of four subscales - intrinsic motivation, identified
regulation, external regulation and amotivation - was developed by Guay et al. (2000). Guay et al. (2000) also con-
cluded that situational motivation connected to a specific academic activity was related to general motivational
orientation within the school context.
Various survey instruments - questionnaires - have been constructed to measure motivation, sometimes
including self-efficacy as well. Table 1 shows an overview of these questionnaires.
Table 1. A list of questionnaires used for assessment of motivation towards Biology and Science.
No. of
Authors Constructs included Subject Target group
Questions
Tuan, Chin, & Shieh, 2005 Self-efficacy, active learning strategies, Science Lower secondary school 35
performance goal, achievement goal, learning students
environment stimulation, science learning
value
Glynn, Taasoobshirazi, & Brickman, Internal motivation, external motivation, per- Science University students 30
2007, 2009 sonal relevance, responsibility, self – efficacy,
anxiety
Britner, 2008 Self-efficacy, self-concept, achievement goal Science Upper secondary school 59
orientation, science anxiety students
Shihusa & Keraro, 2009 Motivation to learn biology using the advanced Biology Secondary school students 20
organizer teaching strategy and the traditional
methods
Ekici, 2009, 2010 Like Glynn, Taasoobshirazi, & Brickman, Biology Upper secondary school 30
2007, 2009 students
Zeyer, 2010 Like Glynn, Taasoobshirazi, & Brickman, Science Upper secondary school 30
2007, 2009 students
Velayutham, Aldridge, & Fraser, 2011 Learning goal orientation, taks value, self- Science Lower secondary school 32
efficacy, self-regulation students
Glynn et al., 2011 Intrinsic motivation, self-determination, self- Science University students 25
efficacy, career motivation, grade motivation
Ates & Saylan, 2015 Internal motivation, external motivation, grade Biology University students 30
motivation, responsibility, self-confidence, test
anxiety
749
(P. 748-767) ISSN 2538–7138 /Online/
No. of
Authors Constructs included Subject Target group
Questions
Mavrikaki, Andressa, & Dermitzaki, self-efficacy, biology learning value, active Biology Upper secondary school 34
2015 learning strategies, performance goal, students
achievement goal, learning environment
stimulation
The focus of this research was the motivation among Czech upper secondary school students for learning
biology. The aim was to translate and adapt Science Motivation Questionnaire (SMQ) (Glynn, Taasoobshirazi, &
Brickman, 2007, 2009) to Czech environment. In this research, the questionnaire (although originally applied to
university students) was applied to upper secondary school students and was adapted for biology by simply chang-
ing the word Science to Biology (Ekici, 2009; Glynn et al., 2011; Zeyer, 2010). The SMQ, where the word Science was
changed and the words Pedagogy/Didactics were used to measure the difference between motivation toward
the science courses and the pedagogically/didactically oriented courses of prospective teachers, was used in an
international research by Šorgo et al. (2017). Despite the existence of revised version of SMQ (Science motivation
questionnaire II; Glynn et al., 2011) the original SMQ was chosen in this research. In the revised version of SMQ, the
major changes of the innitial version were exclusion of the Anxiety dimension from the set of items and formation of
three new constructs: Career motivation, Self-determination, and Grade motivation, replacing External Motivation,
Personal Relevance, and Responsibility. In this way the questionnaire was shortened to 25 items. However, due to
the orientation toward the students’ college science we made informed decision to retain the original version of
the questionnaire which can be seen as more related to the original theories of Deci and Ryan (1985).
From the practical point of view, any research instrument has to balance the accuracy and length. The length
of a questionnaire has been shown to negatively influence the number of respondents who complete it. Also the
time spent by answering the questions positioned later in the questionnaire is shorted and the answers are more
uniform compared to the answers positioned closer to the beginning of the questionnaire (Galesic & Bosnjak, 2009).
Although Rolstad, Adler, and Rydén (2011) also concluded the longer the questionnaire the lower the respondent
rates, the content of a questionnaire can have an impact as well. The trend of shortening questionnaires can be
seen in different areas, e.g. psychology (Rammstedt & John, 2007).
Aim and Purposes of the Research
The aim of the research was to provide a validated (Czech) tool suitable for exploring the motivation of upper
secondary school students to learn biology (adapted from Glynn et al., 2007, 2009) not only by translating it but
with a test of validity of underlying theoretical construct.
Differences between students, based on varied characteristics and traits such as gender and residence, were
explored but were not considered as factors in the following models. The reason does not lie in the unimportance
of recognizing such differences; however, the aim of this research was to find a universal instrument valid for ex-
ploring motivation at the secondary school level. As is the case in majority of such studies the aim was to find a
robust instrument for assessment of general trends and not individual differences between different strata of the
sample (e.g. gender, enrolment in a classroom, etc.).
In conducting the research, answers to the following research questions were sought:
a) In what ways are the constructs transferred from SMQ (Internal Motivation, External Motivation, Personal
Relevance, Responsibility, Self – efficacy and Anxiety) to BMQ valid in the Czech version?
b) Is it possible to shorten the original questionnaire without losing too much information (if yes, how)?
General Background
For this research, the quantitative approach was chosen to find the answers to the research questions. The
constructs of the Czech version of Biology motivation questionnaire were explored using questionnaires completed
750
ISSN 2538–7138 /Online/ (P. 748-767)
by Czech upper secondary school students. Exploratory factor analysis and confirmatory factor analysis were used
to explore the construct of motivation. An alternative model was developed and described.
Sample and Sampling
The sampling was performed during the practical courses that were offered at the Department of Teaching
and Didactics of Biology, Faculty of Science of Charles University, Prague (Czech Republic) to schools from all
parts of the Czech Republic as a voluntary activity from 2012 to 2014. During the course, the students (groups
of 10 – 25) were asked to take part in the research by completing the BMQ in paper and pencil format. Students
were given basic information about the purpose of the questionnaire, as well as the information that responding
to the questionnaire was voluntary and anonymous, and no benefits or disadvantages were foreseen for those
who participated or those who did not. Completion of the questionnaire was supervised by the University staff.
Typically, the task took 20 minutes. Most students (about 90%) agreed and provided a completed questionnaire.
The sample represents students accompanied with their teachers who chose practical courses offered by
the Faculty of Science, Charles University, Prague. The sample consisted of 517 upper secondary school students
from 20 different schools in the Czech Republic (seven from Prague, 13 from other towns from all Czech districts;
30 % of the students were from middle size towns, 29 % from Prague and big towns and 41 % from little towns
and villages). Only questionnaires with no missing data were used in further analysis. This sample consisted of 332
(64%) girls and 185 (36%) boys, from year level ten (N = 53; 10%), 11 (N = 82; 16%), 12 (N = 211; 41%) and 13 (N =
171; 33%); the students were 15 – 19 years old, and in the last four years of upper secondary school (years 10 – 13
in the Czech Republic, after having finished nine years of compulsory education). A minority of the students were
from Prague (N = 133; 26%) and most were from other towns (N = 384; 74%). Approximately one in five students
attended an elective biology course (N = 106; 21%); the rest of the students (N = 411; 79%) had only compulsory
biology lessons.
The construction of the new version of Biology Motivation Questionnaire (BMQ) started from 30 questions
from the Science Motivation Questionnaire (SMQ) developed by Glynn et al. (2007, 2009). The questionnaire was
based on a 5-point Likert scale, as follows: never (1), rarely (2), sometimes (3), usually (4), always (5). Five items
(Table 2: 4, 6, 11, 12, and 16) are negatively coded, and were recoded to be included in statistical tests; however,
for clarity, these are presented in Table 2 in the format as answered by respondents. In the tables, these items are
marked with an asterisk. The initial difference between the original questionnaire and the new version was in the
leading statement, where the word Science was replaced by the word Biology. The leading sentence is as follows:
‘In order to better understand what you think and feel about your biology courses, please respond to each of the
following statements from the perspective of: When I am in a (high) school biology course…’. Later in the question-
naire items, the word Science was replaced by the word Biology.
The questionnaire was translated to the Czech language by two translators proficient in both languages,
and two university teachers of different Biological subjects in context of education. When in doubt conciliar ap-
proach was applied to find the best match between Czech translation and the original. Content validation was
not questioned, because the intention was not to develop a questionnaire from the scratch, however sometimes
small adjustments were necessary to accommodate differences between college and secondary school levels, and
differences in school systems and content of the syllabi (e.g. Biology versus Science).
The initial questionnaire BMQ had six subscales following constructs as suggested by Glynn et al. (2009). The
constructs were as follows: Internal Motivation (IM); External Motivation (EM); Personal Relevance (PR); Responsibil-
ity (RE); Self-efficacy (SE); and Anxiety (AN).
The constructs applied in the theoretical models of motivation and the factors involved in it and their theoreti-
cal base were described elsewhere (e.g. Glynn et al., 2007, 2009; Ryan & Deci, 2000; Vallerand, 1997; Vallerand et al.,
1992), they are presented only briefly here, as follows. Internal Motivation (IM): (intrinsic motivation in Introduction
section) refers to the circumstances when an activity is done for its own sake, for the pleasure and satisfaction of
the participant (Vallerand et al., 1992, p. 1004). As mentioned by Vallerand et al. (1992), internal motivation can be
further divided as IM to know, IM toward accomplishments and IM to experience stimulation. External Motivation
(EM): (extrinsic motivation in Introduction section) refers to behavior as a mean to an end, an activity ‘done in order
751
(P. 748-767) ISSN 2538–7138 /Online/
to attain some separable outcome’ (Deci & Ryan, 1985; Ryan & Deci, 2000, p. 60). Further structuring is mentioned
in Introduction. Personal Relevance (PR): was understood as described by Glynn et al. (2007, p. 1095) as ‘relevance
of learning science to personal goals’ e.g. future life and career. Responsibility (RE): refers to self-determination
for learning science, in a sense of students’ belief that they have control over their learning (Glynn et al., 2007, p.
1090). Self-efficacy (SE): was understood as defined by Bandura (1977) as an expectation of personal efficacy to
be able to finish an activity. Self-efficacy determines the amount of effort expended, the duration of the behavior
and therefore the achievement in the subject. Anxiety (AN): refers to assessment anxiety (Glynn et al., 2007, p.
1090), as a debilitating tension associated with grading in science. In case the level of assessment anxiety is high,
it hinders students’ motivation and therefore achievement (Cassady & Johnson, 2002), what can be understood as
amotivation in a sense of Ryan and Deci (2000).
Reliability of the scale and subscales was calculated as Cronbach’s alpha. Reliability of the entire scale was
.91, showing appropriate internal consistency. Alphas of the subscales and items are presented in Table 3 in the
Results section.
Data Analysis
Evaluation of the BMQ and its Constructs
For evaluation of the BMQ and its underlying constructs, three plausible options were available (Figure 1).
Models were adapted from Pekrun, Goetz, Frenzel, Barchfeld and Perry (2011). The first model is based on the
prediction that motivation is univariate. The second model is based on the prediction that motivation follows
discrete constructs predicted by a theory (Ryan & Deci, 2000) and applied by Glynn et al., (2007, 2009). The third
model is based on the prediction that theoretical constructs (Glynn et al., 2007, 2009) are predictors of a single
second order factor.
Figure 1. Theoretical models for evaluation of BMQ (adapted from Pekrun et al., 2011).
752
ISSN 2538–7138 /Online/ (P. 748-767)
Statistical Procedures
To explore construct validity of the instrument two methods were chosen: Exploratory factor analysis (EFA) and
confirmatory factor analysis (CFA) (see Schmitt, 2011) for overview of factor analysis usage and recommendation on
using EFA). CFA was used to determine whether the responses of the participants followed the pre-existing factor
models for the BMQ. Principal Component Analysis (PCA) with Direct Oblimin rotation calculated with SPSS ver. 24
was chosen, because correlation between constructs was expected. SEM analyses by use of AMOS ver. 24 package
were performed to explore model fits. Upon discovering poor-fit for initial models, exploratory factor analysis (EFA)
was used to develop alternative models. SEM and CFA were interchangeably used to establish the fit of the newly
trimmed questionnaire. Only some of the key steps (initial and final models) are presented in the article, and not
all steps in every statistical method applied (e.g. Cronbach’s alpha if item deleted, step by step improvement of the
models by connecting error terms, and similar). The reason is to shorten the article to a reasonable length, however,
if someone is interested in post-hoc analysis or in comparative study anonymized data are available on request.
Reliability and Validity of a Measurement Model
Models were built by use of raw data collected from 517 respondents to the BMQ. No missing data was in
the data set. Reliability of each construct was assessed by the inspection of factor loadings on each construct and
Cronbach’s alpha. Factor loadings with values greater than .4, and alphas above the .7 level were considered ap-
propriate for the models describing motivation and its’factors.
The Construct Validity was checked by analysing Fitness Indices as follows: RMSEA (Root Mean Square of
Error Approximation), GFI (Goodness of Fit Index), AGFI (Adjusted Goodness of Fit), CFI (Comparative Fit Index),
NFI (Normed Fit Index) and Chisq/df (Chi Square/Degrees of Freedom). There is no agreement among researchers
on which fitness indices to use. Threshold values for each index were compiled from several sources (Kline, 2015;
Šumak et al., 2017; Šumak & Šorgo, 2016). A modification procedure with correlation of error terms was used to
improve model fit (Hooper, Coughlan, & Mullen, 2008).
SPSS 24 and AMOS 24 software were applied for calculation. Figures were created using CorelDRAW X6.
Weakness of the Research
The major weakness of the research is one shared with a majority of similar studies: every Czech student
does not have an equal chance to be included in a sample, which is the assumption behind random sampling.
The sample consisted of students who might not themselves have been motivated for biology; however, they had
motivated teachers with the resources to travel to the workshops at Charles University. However, the sample used
was sufficiently large to allow robust conclusions based on applied statistical methods. Retesting of the shortened
version of the questionnaire on a different sample and, hopefully, random sampling, should be the next steps.
Research Results
Descriptors from the Biology Motivation Questionnaire
Students were asked to provide answers to the initial statement, as follows.

In order to better understand what you think and feel about your biology courses, please respond to each of
the following statements from the perspective of: ‘When I am in a (high) school biology course…’
Frequencies of their answers and values of mean (M), standard deviation (SD), median (Mdn) and Mode
(Mod) are provided in Table 2. Items J4, J6, J11, J12, and J16 (denoted by an asterisk) were reverse coded for follow
up analyses; however, in Tables 2, 4 and 7, these are presented in the format as received from the respondents.
Information about subscales as defined by Glynn et al. (2007, 2009), and Cronbach’s alphas for each construct are
given in Table 3.
753
(P. 748-767) ISSN 2538–7138 /Online/
Table 2. Frequencies of answers and descriptive statistics on the Biology Motivation Questionnaire (N = 517).
No. of
Statement F1% F2% F3% F4% F5% M SD
Statement
J1 I enjoy learning biology. 13.9 44.7 24.4 10.6 6.4 2.51 1.06
J2 Biology I learn relates to my personal goals. 21.7 25.0 23.0 20.3 10.1 2.72 1.28
I like to do better than the other students in biol-
J3 58.8 19.5 12.4 5.4 3.9 1.76 1.11
ogy assessments.
I am nervous about how I will do in biology
J4 14.5 18.0 28.6 21.7 17.2 3.9 1.29
assessments.*
If I am having trouble learning biology, I try to
J5 26.1 29.8 28.0 10.8 5.2 2.39 1.14
figure out why.
I become anxious when it is time to sit a biology
J6 17.8 33.5 25.9 12.0 10.8 2.65 1.22
assessment.*
Achieving a good biology grade (Achieved /
J7 39.1 34.0 18.0 6.6 2.3 1.99 1.02
Merit / Excelence) is important to me.
J8 I put enough effort into learning biology. 29.2 40.8 17.0 9.1 3.9 2.18 1.07
J9 I use strategies that ensure I learn biology well. 8.7 20.3 31.5 24.2 15.3 3.17 1.17
I expect to do as well as or better than other
J10 30.6 40.6 19.9 7.2 9 1.7 2.09 .97
students in biology course.
J11 I worry about failing biology assessments.* 11.4 27.3 31.3 15.1 14.9 2.95 1.22
I am concerned that the other students are better
J12 20.9 27.5 22.6 15.9 13.2 2.73 1.31
in biology. *
I think about how biology will affect my overall
J13 22.1 24.0 25.5 16.8 11.6 2.72 1.30
subject or certificate endorsement.
Biology I learn is more important to me than the
J14 30.6 29.0 25.3 8.9 6.2 2.31 1.17
grades I receive.
I think about how learning biology can help my
J15 32.7 21.3 17.4 17.8 10.8 2.53 1.38
career.
J16 I hate taking biology assessments.* 13.2 26.1 28.2 18.4 14.1 2.94 1.24
It is my fault if I do not understand the biology
J17 37.9 37.3 16.2 4.6 3.9 1.99 1.04
ideas.
I am confident I will do well on the written and
J18 15.7 44.5 30.6 7.0 2.3 2.36 .91
practical biology assessments.
J19 I find learning biology interesting. 34.0 31.9 21.9 6.8 5.4 2.18 1.14
J20 Biology I learn is relevant to my life. 20.9 33.5 26.3 14.1 5.2 2.49 1.13
I believe I can master the knowledge and skills in
J21 22.1 45.6 25.5 5.8 5 1.0 2.18 .87
the biology course.
J22 Biology I learn has practical value for me. 20.7 29.4 30.6 14.1 5.2 2.54 1.12
I prepare well for the biology assessments (both
J23 12.8 38.3 27.1 15.7 6.2 2.64 1.08
internal and external)
J24 I like biology that challenges me. 10.8 21.5 28.8 21.9 17.0 3.13 1.24
I am confident I will do well in the biology as-
J25 13.9 38.7 35.6 9.1 2.7 2.48 .94
sessments.
I believe I can earn ‘excellence’ grades in the
J26 10.4 26.5 30.4 16.8 15.9 3.01 1.22
biology course.
Understanding biology gives me a sense of
J27 27.9 33.3 21.1 12.8 5.0 2.34 1.16
accomplishment.
I think about how learning biology can help me
J28 21.9 24.0 23.4 19.5 11.2 2.74 1.30
get a good job.
754
ISSN 2538–7138 /Online/ (P. 748-767)
I think about how biology I learn will be helpful

J29 24.6 33.3 27.5 10.6 4.1 2.36 1.09
to me.
J30 I think about how I will use biology I learn. 21.3 33.1 29.4 10.4 5.8 2.46 1.11
Note: M mean, SD standard deviation, Mdn median
Table 3. Subscales of the BMQ and their Cronbach’s alphas.
Subscales of BMQ Items Alphas
Internal motivation (IM): 1, 14, 19, 24, 27 .84
External motivation (EM) 3, 7, 13, 15, 28 .73 (.75 if 13 deleted)
Personal relevance (PR) 2, 20, 22, 29, 30 .89
Responsibility (RE) 5, 8, 9, 17, 23 .75 (.82 if 17 deleted)
Self-efficacy (SE) 10, 18, 21, 25, 26 .83
Anxiety (AN) 4, 6, 11, 12, 16 .66 (.72 if 16 deleted)
Whole questionnaire 1 to 30 .91
All subscales except the anxiety subscale (AN) passed the .7 threshold; however, it was possible to raise the
alpha level of this subscale to acceptable levels by deleting one item.
From the initial calculations of Cronbach’s alphas, it can be concluded that the questionnaire and its subscales
(constructs) can be used for studies about high school students’ motivation towards biology.
Principal Component Analysis of the BMQ
Initial checking for suitability of the matrix for factorial analyses reveals the values for the Kaiser-Meyer-Olkin
(KMO) measure of sampling adequacy as .947 and Bartlett’s Test of Sphericity as Chi-square = 9323.24; df = 435; p
< .0001. Principal component analysis with Oblimin rotation based on Eigenvalue > 1 criteria revealed five compo-
nents. Based on the results of the parallel analysis (Hayton, Allen, & Scarpello, 2004), only the first three components
should be retained. However, when considering alphas, only the first, the third and the fourth component passed
the .7 threshold. Even more, the second component had negative value of alpha. Three items (11, 18, and 6) load
on two components, and the loading of item 9, did not exceed the .4 threshold (Table 4). When considering cor-
relations between components, a mixture of negative and positive correlations between principal components
can be discerned (Table 5).
Table 4. Principal component structure and item loadings of the BMQ.
No. of
Motivation PC1 PC2 PC3 PC4 PC5
Statement
J29 PR .94
J30 PR .93
J28 EM .86
J15 EM .75
J2 PR .68
J20 PR .65
J22 PR .60
J27 IM .54
J19 IM .46
J24 IM .45
755
(P. 748-767) ISSN 2538–7138 /Online/
No. of
Motivation PC1 PC2 PC3 PC4 PC5
Statement
J14 IM .43
J5 RE .43
J4 AN .72
J13 EM -.66
J11 AN .66 -.42
J12 AN .64
J3 EM -.60
J7 EM -.48
J25 SE -.83
J26 SE -.77
J10 SE -.69
J21 SE -.67
J18 SE -.62 .48
J6 AN .49 -.51
J23 RE -.57
J8 RE -.51
J16 AN -.47
J1 IM .43 -.44
J9 RE
J17 RE .92
Variance 37.64 11.56 6.15 3.87 3.46
Eigenvalue 11.29 3.47 1.85 1.16 1.04
Alpha .94 -.53 .80 .79 NA
Note: Cumulative variance 62.68 %.
Table 5. Component correlation matrix.
Component 1 2 3 4 5
1
2 -.31
3 -.33 .01
4 -.37 .11 .26
5 .26 .04 -.21 -.17
Based on the findings from the initial analyses of principal components, we concluded on the basis of the suf-
ficiently high value of Cronbach’s alpha (.91), that the initial SMQ applied as BMQ could be suitable for assessment
of motivation towards Biology if applied as an entire instrument or a particular extracted construct. However, the
resulting constructs from the described analysis do not follow the theoretical reasoning of the authors of the SMQ
(Glynn et al., 2007, 2009), showing a mixture of motivational types, so we continued the SEM analysis, as follows.
CFA of initial BMQ using SEM
The procedure was repeated using all thirty items in six proposed subscales (Glynn et al., 2009), building three
models based on SEM statistics (Figures 2, 3, 4) following Pekrum et al. (2011).
756
ISSN 2538–7138 /Online/ (P. 748-767)
Figure 2. CFA diagram of Model 1.
757
(P. 748-767) ISSN 2538–7138 /Online/
Figure 3. Initial model (M2) based on a modified SMQ (Glynn et al., 2009).
Note: (IM Internal Motivation, EM External Motivation, PR Personal Relevance, RE Responsibility, SE Self-efficacy and AN Anxiety)
758
ISSN 2538–7138 /Online/ (P. 748-767)
Figure 4. Model 3 assuming that theoretical constructs are predictors of a single second-order variable.
Note: (IM Internal Motivation, EM External Motivation, PR Personal Relevance, RE Responsibility, SE Self-efficacy and AN Anxiety)
The resulting models (M1-M3) do not fit well (Table 6) according to recommendations compiled from several
sources (see Šumak & Šorgo, 2016).
759
(P. 748-767) ISSN 2538–7138 /Online/
Table 6. Model fit summary of the models (M1-M3) based on a modified SMQ (Glynn et al., 2009).
Fit index Recommendation M1 M2 M3
χ2 Non-significant 3260.8 1995.7 2563.1
Degrees of freedom (df) n/a 405 390 399
χ2/df < 3.00 8.05 5.02 6.4
Goodness-of-fit index (GFI) > .90 .59 .78 .69

Adjusted Goodness-of-fit index
> .80 .53 .74 .65
(AGFI)
Comparative fit index (CFI) > .90 .69 .83 .76
Root mean square error of approxi-
< .08 .12 .09 .10
mation (RMSEA)
Normed fit index (NFI) > .80 .66 .80 .73
Parsimony normed fit index (PNFI) > .60 .61 .71 .67
Note: M1: One construct model; M2 = Initial model based on SMQ with six constructs (30 items); M3 = second order model
From inspection of the models (Figures 2 – 4), it was concluded that Model 2 (M2) showed the best Fit, so it
was considered for the improvement. It can be recognized that correlations between the construct Anxiety (AN)
and all other constructs, except Self-efficacy (SE) are negative or none in the case of internal motivation (IM). This
finding can be attributed with considering Anxiety as Amotivation in a sense of Self Determination Theory. All
other correlations in the model fall in the positive range. The highest value in the model (.98) is between External
Motivation (EM) and Personal Relevance (PR), allowing us to conclude that both constructs form or are a part of
a single factor. The same is probably true for other motivational constructs (IM, EM, PR, RE), other than SE, where
all correlations are above the .8 level. Additionally, standardized factor loadings of some items on the proposed
constructs are below suggested levels (Kline, 2015).
Building a Model
With the procedure ‘Alpha if item deleted’, eight variables were excluded from the initial pool, leaving us with
22 variables considered for further analysis (Table 7). The Cronbach’s alpha for the reduced questionnaire was excel-
lent at .95. KMO at .96 and Bartlett’s Chi-square test = 7791.83; 231 df; p < .001 allowed further principal component
analysis. Based on Eigenvalue >1, three components, explaining 63.7% of variance were extracted; however, by
parallel analysis, only the first two components should be retained, explaining 58.5% of variance.
Table 7. Three-component Model 1.
Motivation PC1 PC2 PC3
PR .83
PR .82
EM .81
PR .76
IM .75
PR .70
EM .69
760
ISSN 2538–7138 /Online/ (P. 748-767)
Motivation PC1 PC2 PC3
IM .65
PR .64
IM .62
IM .57
RE .45
SE .80
SE .75
SE .67
SE .63
AN .50
RE .72
EM .71
IM .65
RE .64
RE .63
Variance 48.80 9.66 5.26
Eigenvalue 10.74 2.11 1.16
Alpha .94 .82 .83
From the item loadings to the components (Table 7), we can conclude that only Self-efficacy (SE) forms a
stable construct (PC2), and even the single remaining AN (J16) item ‘I hate taking biology assessments’ can be,
by the opinions of the authors, considered as Self-efficacy, not as Anxiety, because it is more probable that it in-
dicates anger rather than fear. Personal Relevance (PR) is the main constituent of the first component (five items)
and is combined with four items of Internal Motivation (IM), two items of External Motivation (EM), and one item
of Responsibility (RE). Responsibility (three items) is the leading idea of the third component, accompanied with
one item each from the IM and EM pools.
After analysis of the factor loadings on the components, we made the decision to exclude from the question-
naire the last remaining Anxiety item, therefore theoretical construct Amotivation was excluded from the model.
With this intervention, the Cronbach’s alpha of the scale stayed the same at the .95 value. However, the explained
variance of the matrix rose to a value of 65.04% when three components were considered and to 59.64% when
two components were considered. All items included in the second analysis, with the excluded AN item, stayed
in the same components; however, small differences in loadings do appear. These differences are too minor to be
considered as important.
From the component correlation matrix (Table 8), it can be seen that the components are correlated, with
the highest correlation between the first and the third component. These correlations allow us to speculate that
motivation is in reality a complex mixture of theoretically predicted constructs, thus allowing further shortening
of the instrument.
761
(P. 748-767) ISSN 2538–7138 /Online/
Table 8. Component correlation matrix.
Component 1 2
2 .36
3 .58 .22
A Three-construct model
A three-construct model, based on three components identified by EFA (PCA) analysis of the initial BMQ,
excluding Anxiety (AN), was used as a basis for further shortening of the BMQ. By use of modification indices
and deletion of some items in the AMOS program, we retained 14 items to be included in a new model (Table
9, Figure 5). PCA analysis revealed a matrix structure (KMO = .940; Chi Square = 4836; df = 91; p < .001) suitable
for proceeding with calculations. Cronbach’s alpha of the shortened instrument was .93. Based on eigenvalue >
1, three components were extracted (for the correlations, see Table 10); however, only the first two components
passed the stricter thresholds calculated by parallel analysis (Hayton, Allen, & Scarpello, 2004). Cronbach’s alphas
for all three components passed the .7 threshold value, and it was not possible to raise this by deletion of any item.
Table 9. Principal components structure and item loading of the shortened 14-item BMQ.
NNo. of
Motivation Statement PC 1 PC 2 PC 3
Statement
JJ15 EM I think about how learning biology can help my career. .91
JJ20 PR Biology I learn is relevant to my life. .87
JJ2 PR Biology I learn relates to my personal goals. .87
JJ22 PR Biology I learn has practical value for me. .84
JJ28 EM I think about how learning biology can help me get a good job. .78
JJ19 IM I find learning biology interesting. .59
JJ1 IM I enjoy learning biology. .55
JJ26 SE I believe I can earn ‘excellence’ grades in the biology course. .84
JJ10 SE I expect to do as well as or better than other students in biology course. .81
JJ21 SE I believe I can master the knowledge and skills in the biology course. .76
JJ23 RE I prepare well for the biology assessments .89
JJ9 RE I use strategies that ensure I learn biology well. .73
JJ8 RE I put enough effort into learning biology. .72
JJ7 EM Achieving a good biology grade is important to me. .62
Variance 54.13 9.74 7.21
Eigenvalue 7.58 1.36 1.01
Alpha .93 .78 .82
The model was confirmed by PCA analysis (Table 9) with apropriate reliability coeficients, explaining 71.08% of
variance. However, correlations between components are still high (Table 10), revealing the possibility of reducing
the model even further; however, we did not take this step, owing to achievement of the stricter threshold level of
762
ISSN 2538–7138 /Online/ (P. 748-767)
the set of fit indices. Three constructs can be recognized from this model. The first is a mixture of Personal Relevance,
External and Internal Motivation. The construct does not follow distinctions between Internal Motivation and the
various kinds of External Motivation. The second construct is Self-efficacy, and the third one Responsibility, both
with low factor loadings (Figure 5).
Figure 5. Three-component SEM model.
Table 10. Component Correlation Matrix.
Component 1 2
2 .45
3 .61 .40
Discussion
The SMQ (Glynn et al., 2007, 2009) has been used for upper secondary school students (Zeyer, 2010) and adapted
for measuring motivation to learn biology (Ekici, 2009, 2010). It was used to measure Czech upper secondary school
students’ motivation for learning biology and also explored the construct itself, since Glynn et al. (2009) suggested
that some items needed revision to improve the construct validity. The original SMQ (Glynn et al., 2007, 2009) was
used, despite the existence of the revised SMQII (Glynn et al., 2011), which was used for university students.
763
(P. 748-767) ISSN 2538–7138 /Online/
Originally, the SMQ and the derived BMQ consisted of the following constructs: Internal Motivation, External
Motivation, Personal Relevance, Responsibility, Self-efficacy and Anxiety (Glynn et al., 2007, 2009). The whole
SMQ (BMQ) construct or individual constructs can be used as described (Ekici, 2009; Glynn et al., 2007, 2009;
Zeyer, 2010); however, the connections between variables are far from the initial constructs. Our analysis showed
that there should be fewer constructs included in the Czech version of the BMQ. Very high covariance between
External Motivation and Personal Relevance, as well as among Internal Motivation, External Motivation, Personal
Relevance and Responsibility revealed that these factors are in fact most probably parts of a single, underlying
motivational construct. Therefore, it was concluded that only Self-efficacy could be considered a firmly based
factor, as proposed by Bandura (1977), whose model combined personal accomplishment, vicarious experience,
verbal persuasion and psychological states. Academic self-efficacy, student beliefs about their academic capabil-
ity, have been shown to be an effective predictor of learning and also of motivation to learn, as it determines
students’ aspirations and therefore academic accomplishment. It was shown to be a predictor of science achie-
vement (Britner & Pajares, 2001, 2006). Students with low self-efficacy approach difficulties almost as threats,
whereas students with high self-efficacy approach difficulties as ‘challenges to be mastered’ (Bandura, 1993).
Self-efficacious students also persist longer and work harder when they have to overcome difficulties, compared
to students with lower self-efficacy (Bandura, 1997). Based on the results, Personal Relevance, Internal Motiva-
tion and External Motivation were combined into the second factor. Responsibility remained as the leading
construct for the third and last proposed factor. Internal motivation emerged as only marginally important in
our construct. Based on this research, it is suggested that anxiety should be excluded from the BMQ. Although
it is an aspect of negative motivation (Bandura, 1986), which can be stronger for females than males in the case
of science (Mallow, 1994), it correlated negatively with the remaining BMQ constructs. This is understandable
when one considers that anxiety can be paralyzing. The negative correlation is predictable, since as one of the
remaining constructs was Self-efficacy, which has been shown to correlate negatively with anxiety (Pajares &
Kranzler, 1995a, 1995b). This research shows that anxiety should not be included, or not as it was in the original
SMQ. It might be more appropriate to measure anxiety by itself, as suggested by (Cassady & Johnson, 2002) and
not as a part of motivation.
A shortened instrument suitable for exploring upper secondary school students’ motivation for studying
biology based on the SMQ (Glynn et al., 2007, 2009) was introduced. Mavrikaki et al. (2015) also provided a valid
questionnaire for measuring motivation to learn biology, although this is more than twice as long as the one
described in this research. Since time can be one limitation on research conducted in schools, an adequate, but
shorter version can be useful for both researchers and teachers.
Conclusions / and Implications
Based on analysis including the BMQ completed by 517 Czech upper secondary school students, a shorter
version of the Czech BMQ was suggested: 14 questions instead of the original 30 in the SMQ. Such a question-
naire is suitable for measuring the motivation of upper secondary school students to learn biology. This reflects
the constructs behind the BMQ, which showed that three factors should be retained: Self-efficacy itself, a factor
combining Personal Relevance, Internal Motivation and External Motivation, and a third factor consisting mostly
of Responsibility and containing one question included in the original External Motivation construct. Anxiety
questions were excluded, based on the analysis. Hopefully, the construct described above will be useful when
investigating motivation to learn biology.
Disclosure statement
No potential conflict of interest was reported by the authors. Neither the Agency nor its employees had any
influence on the work in progress.
Primary anonymised data for secondary analyses is available on request from the authors in electronic form
as an Excel file. In the case of data usage, it is expected that the publication source will be properly cited.
764
ISSN 2538–7138 /Online/ (P. 748-767)
Acknowledgements
The work was supported by Charles University Research Centre program UNCE/HUM/024 Centrum didaktického
výzkumu v přírodních vědách, matematice a jejich mezioborových souvislostech (to VJ), and the Slovenian Research
Agency, research core funding No. P2-0057 (to AŠ). The authors are also grateful to Petr Janšta for his help with
the figures.
References
Ates, H., & Saylan, A. (2015). Investigation of pre-service science teachers’ academic self-efficacy and academic motivation
toward biology. International Journal of Higher Education, 4(3), 90. https://doi.org/10.5430/ijhe.v4n3p90.
Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84(2), 191–215. https://
doi.org/10.1037/0033-295X.84.2.191.
Bandura, A. (1986). Social foundations of thought and action: A social-cognitive view. Englewood Cliffs, NJ: Prentice Hall.
Bandura, A. (1993). Perceived self-efficacy in cognitive development and functioning. Educational Psychologist, 28(2), 117–148.
https://doi.org/10.1207/s15326985ep2802_3.
Bandura, A. (1997). Self-efficacy: The exercise of control. New York, NY, US, Macmillan.
Bathgate, M., & Schunn, C. (2016). Disentangling intensity from breadth of science interest: What predicts learning behaviors?
Instructional Science, 44(5), 423–440. https://doi.org/10.1007/s11251-016-9382-0.
Beihler, R. F., & Snowman, J. (1997). Psychology applied to education. Boston: Houghton Mifflin Company.
Britner, S. L. (2008). Motivation in high school science students: A comparison of gender differences in life, physical, and earth
science classes. Journal of Research in Science Teaching, 45(8), 955–970. https://doi.org/10.1002/tea.20249.
Britner, S. L., & Pajares, F. (2001). Self-efficacy beliefs, motivation, race, and gender in middle school science. Journal of Women
and Minorities in Science and Engineering, 7(4), 271–285. https://doi.org/10.1615/JWomenMinorScienEng.v7.i4.10.
Britner, S. L., & Pajares, F. (2006). Sources of science self-efficacy beliefs of middle school students. Journal of Research in Sci-
ence Teaching, 43(5), 485–499. https://doi.org/10.1002/tea.20131.
Cassady, J. C., & Johnson, R. E. (2002). Cognitive test anxiety and academic performance. Contemporary Educational Psychol-
ogy, 27(2), 270–295. https://doi.org/10.1006/ceps.2001.1094.
Deci, E. L., Koestner, R., & Ryan, R. M. (2001). Extrinsic rewards and intrinsic motivation in education: Reconsidered once again.
Review of Educational Research, 71(1), 1–27. https://doi.org/10.3102/00346543071001001.
Deci, E. L., & Ryan, R. M. (1985). The general causality orientations scale: Self-determination in personality. Journal of Research
in Personality, 19(2), 109–134. https://doi.org/10.1016/0092-6566(85)90023-6.
Dohn, N. B., Fago, A., Overgaard, J., Madsen, P. T., & Malte, H. (2016). Students’ motivation toward laboratory work in physiology
teaching. Advances in Physiology Education, 40(3), 313–318. https://doi.org/10.1152/advan.00029.2016.
Dyrberg, N. R., Treusch, A. H., & Wiegand, C. (2017). Virtual laboratories in science education: students’ motivation and experi-
ences in two tertiary biology courses. Journal of Biological Education, 51(4), 358–374. https://doi.org/10.1080/002192
66.2016.1257498.
Ekici, G. (2009). Adaptation of the biology motivation questionnaire to Turkish. Çağdaş Eğitim Dergisi, 34(365), 6–15.
Ekici, G. (2010). Factors affecting biology lesson motivation of high school students. Procedia - Social and Behavioral Sciences,
2(2), 2137–2142. https://doi.org/10.1016/j.sbspro.2010.03.295.
Gagné, M., & Deci, E. L. (2005). Self-determination theory and work motivation. Journal of Organizational Behavior, 26(4),
331–362. https://doi.org/10.1002/job.322.
Galesic, M., & Bosnjak, M. (2009). Effects of questionnaire length on participation and indicators of response quality in a web
survey. Public Opinion Quarterly, 73(2), 349–360. https://doi.org/10.1093/poq/nfp031.
Glynn, S. M., Brickman, P., Armstrong, N., & Taasoobshirazi, G. (2011). Science motivation questionnaire II: Validation with sci-
ence majors and nonscience majors. Journal of Research in Science Teaching, 48(10), 1159–1176. https://doi.org/10.1002/
tea.20442.
Glynn, S. M., Taasoobshirazi, G., & Brickman, P. (2007). Nonscience majors learning science: A theoretical model of motivation.
Journal of Research in Science Teaching, 44(8), 1088–1107.
Glynn, S. M., Taasoobshirazi, G., & Brickman, P. (2009). Science motivation questionnaire: Construct validation with nonscience
majors. Journal of Research in Science Teaching, 46(2), 127–146.
Guay, F., Vallerand, R. J., & Blanchard, C. (2000). On the assessment of situational intrinsic and extrinsic motivation: The situ-
ational motivation scale (SIMS). Motivation and Emotion, 24(3), 175–213. https://doi.org/10.1023/A:1005614228250.
Hayton, J. C., Allen, D. G., & Scarpello, V. (2004). Factor retention decisions in exploratory factor analysis: A tutorial on parallel
analysis. Organizational Research Methods, 7(2), 191–205. https://doi.org/10.1177/1094428104263675.
Honicke, T., & Broadbent, J. (2016). The influence of academic self-efficacy on academic performance: A systematic review.
Educational Research Review, 17, 63–84. https://doi.org/10.1016/j.edurev.2015.11.002.
765
(P. 748-767) ISSN 2538–7138 /Online/
Hooper, D., Coughlan, J., & Mullen, M. (2008). Structural equation modelling: Guidelines for determining model fit. The Elec-
tronic Journal of Business Research Methods, 6(1), 53-60.
Jeno, L. M., Grytnes, J.-A., & Vandvik, V. (2017). The effect of a mobile-application tool on biology students’ motivation and
achievement in species identification: A self-determination theory perspective. Computers & Education, 107, 1–12. https://
doi.org/10.1016/j.compedu.2016.12.011.
Kline, P. (2015). A handbook of test construction (Psychology Revivals): Introduction to psychometric design. London, Great Britain,
Routledge.
Lawson, A. E., Banks, D. L., & Logvin, M. (2007). Self-efficacy, reasoning ability, and achievement in college biology. Journal of
Research in Science Teaching, 44(5), 706–724. https://doi.org/10.1002/tea.20172.
Mahler, D., Großschedl, J., & Harms, U. (2017). Opportunities to learn for teachers’ self-efficacy and enthusiasm. Education
Research International, Article ID 4698371, 1-17. https://doi.org/https://doi.org/10.1155/2017/4698371.
Mallow, J. V. (1994). Gender-related science anxiety: a first binational study. Journal of Science Education and Technology, 3(4),
227–238. https://doi.org/10.1007/BF01575898.
Mavrikaki, E., Andressa, H., & Dermitzaki, I. (2015). Adaptation of the students’ motivation towards science learning (SMTSL)
questionnaire to measure Greek students’ motivation towards biology learning. International Journal of Biology Educa-
tion, 4(2), 78-93. https://doi.org/10.20876/ijobed.16761.
Michael, J. (2006). Where’s the evidence that active learning works? Advances in Physiology Education, 30(4), 159–167. https://
doi.org/10.1152/advan.00053.2006.
Pajares, F., & Kranzler, J. (1995a). Role of self-efficacy and general mental ability in mathematical problem-solving: A path
analysis. Contemporary Educational Psychology, 20, 426-443.
Pajares, F., & Kranzler, J. (1995b). Self-efficacy beliefs and general mental ability in mathematical problem-solving. Contem-
porary Educational Psychology, 20(4), 426–443. https://doi.org/10.1006/ceps.1995.1029.
Pekrun, R., Goetz, T., Frenzel, A. C., Barchfeld, P., & Perry, R. P. (2011). Measuring emotions in students’ learning and perfor-
mance: The Achievement Emotions Questionnaire (AEQ). Contemporary Educational Psychology, 36(1), 36–48. https://
doi.org/10.1016/j.cedpsych.2010.10.002.
Rammstedt, B., & John, O. P. (2007). Measuring personality in one minute or less: A 10-item short version of the Big Five Inven-
tory in English and German. Journal of Research in Personality, 41(1), 203–212. https://doi.org/10.1016/j.jrp.2006.02.001.
Rolstad, S., Adler, J., & Rydén, A. (2011). Response burden and questionnaire length: Is shorter better? A Review and Meta-
analysis. Value in Health, 14(8), 1101–1108. https://doi.org/10.1016/j.jval.2011.06.003
Ryan, R. M., & Deci, E. L. (2000). Intrinsic and extrinsic motivations: Classic definitions and new directions. Contemporary
Educational Psychology, 25(1), 54–67. https://doi.org/10.1006/ceps.1999.1020.
Shihusa, H., & Keraro, F. N. (2009). Using advance organizers to enhance students’ motivation in learning biology. Eurasia
Journal of Mathematics, Science & Technology Education, 5(4), 413–420.
Schmitt, T. A. (2011). Current methodological considerations in wxploratory and confirmatory factor analysis. Journal of Psy-
choeducational Assessment, 29(4), 304–321. https://doi.org/10.1177/0734282911406653.
Šorgo, A., Dojer, B., Golob, N., Repnik, R., Repolusk, S., Pesek, I., … Špur, N. (2018). Opinions about STEM content and classroom
experiences as predictors of upper secondary school students’ career aspirations to become researchers or teachers.
Journal of Research in Science Teaching, 55(10), 1448–1468. https://doi.org/10.1002/tea.21462
Šorgo, A., Lamanauskas, V., Šašić, S. Š., Ersozlu, Z. N., Tomažič, I., Kubiatko, M., … Usak, M. (2017). Cross-national study on rela-
tions between motivation for science courses, pedagogy courses and general self-efficacy. Eurasia Journal of Mathematics,
Science and Technology Education, 13(10), 6497–6508. https://doi.org/10.12973/ejmste/76970.
Šumak, B., Pušnik, M., Heričko, M., & Šorgo, A. (2017). Differences between prospective, existing, and former users of interac-
tive whiteboards on external factors affecting their adoption, usage and abandonment. Computers in Human Behavior,
72(Supplement C), 733–756. https://doi.org/10.1016/j.chb.2016.09.006.
Šumak, B., & Šorgo, A. (2016). The acceptance and use of interactive whiteboards among teachers: Differences in UTAUT
determinants between pre- and post-adopters. Computers in Human Behavior, 64, 602–620. https://doi.org/10.1016/j.
chb.2016.07.037
Tuan, H.-L., Chin, C.-C., & Shieh, S.-H. (2005). The development of a questionnaire to measure students’ motivation towards sci-
ence learning. International Journal of Science Education, 27(6), 639–654. https://doi.org/10.1080/0950069042000323737.
Vallerand, R. J. (1997). Toward a hierarchical model of intrinsic and extrinsic motivation. Advances in Experimental Social Psy-
chology, 29, 271–360. https://doi.org/10.1016/S0065-2601(08)60019-2.
Vallerand, R. J., Pelletier, L. G., Blais, M. R., Briere, N. M., Senecal, C., & Vallieres, E. F. (1992). The academic motivation scale:
A measure of intrinsic, extrinsic, and amotivation in education. Educational and Psychological Measurement, 52(4),
1003–1017. https://doi.org/10.1177/0013164492052004025.
Velayutham, S., Aldridge, J., & Fraser, B. (2011). Development and validation of an instrument to measure students’ motivation
and self-regulation in science learning. International Journal of Science Education, 33(15), 2159–2179. https://doi.org/1
0.1080/09500693.2010.541529.
Zeyer, A. (2010). Motivation to learn science and cognitive style. Eurasia Journal of Mathematics, Science & Technology Educa-
tion, 6(2), 121–128.
766
ISSN 2538–7138 /Online/ (P. 748-767)
Zimmerman, B. J., Bandura, A., & Martinez-Pons, M. (1992). Self-motivation for academic attainment: The role of self-
efficacy beliefs and personal goal setting. American Educational Research Journal, 29(3), 663–676. https://doi.
org/10.3102/00028312029003663.
Vanda Janštová PhD, Assistant Professor, Department of Teaching and Didactics of

(Corresponding author) Biology, Charles University Faculty of Science, Prague, Czech Republic,
Department of Teaching and Didactics of Biology, Charles University
Faculty of Science, Viničná 7, Prague 2, 12844, Czech Republic.
E-mail: vanda.janstova@natur.cuni.cz
Website: https://www.natur.cuni.cz/biology/teaching/people/vanda-
janstova?set_language=en
Andrej Šorgo PhD, Professor, Faculty of Natural Sciences and Mathematics,
University of Maribor, Koroška cesta 160, 2000 Maribor, Slovenia.
E-mail: andrej.sorgo@um.si
Website: http://biologija.fnm.uni-mb.si/index.php?option=com_conte
nt&view=article&id=55&Itemid=7&lang=en
767
TEACHER-DEMONSTRATION
AND STUDENT HANDS-ON
EXPERIMENTS IN TEACHING
INTEGRATED SCIENCES
Abstract. The widely explored but still Mirjana Maričić,

unresolved question about the contribution Stanko Cvjetićanin,
of the application of different ways of per-
forming experiments in the integrated nat-
Branko Anđić
ural sciences is addressed in this research.
The aim was to determine the contribution
of demonstration hands-on experiments
(DHE) and student hands-on experiments
(SHE) in relation to conventional teaching Introduction
method (CTM) on the quality and durabil-
ity of 3rd grade students (between 9 and The beginning of the application of laboratory work in scientific educa-
10 years of age) from primary school. The tion is related to the introduction of demonstration experiments (experiment
research involved 180 students, further performed by a teacher or student with advanced experimental skills in front
divided into three groups: E1 (experimen- of the whole class), while the beginning of the application of laboratory work
tal group 1, where content was learned within school practice (curricula) is related to the introduction of student
through DHE), E2 (experimental group 2, experiments (experiment performed by students independently) (Hodson,
where content was learned through SHE) 1993; Hofstein & Lunetta, 2004). Demonstration and student experiments can
and C (control group, where content was be laboratory, or simple hands-on experiments (Demircioğlu & Yadigaroğlu,
learned through CTM). The results of the re- 2011). A laboratory experiment involves a set of procedures that teacher or
search point to the fact that priority should students perform with the application of laboratory equipment and materi-
be given to the DHE and SHE over the CTM als under the supervision of teachers in order to obtain certain information
in the realization of air-related content in about studied phenomenon (Salameh El Rabadi, 2013). Simple hands-on
the 3rd grade. SHE should be used more experiments also refer to a set of procedures that teacher or students per-
than DHE when it comes to teaching this form (with the guidance of teacher), but their application does not require
specific content. expensive equipment and laboratory equipment, and they can be realized
with the help of simple, inexpensive, easily accessible materials and resources
Keywords: demonstration hands-on ex-
(Hırça, 2013, Sadi & Cakiroglu, 2011). However, they differ from the science
periments, student hands-on experiments,
laboratory in the fact that they do not represent experiments through which
integrated sciences, primary school, quality
students discover new scientific knowledge, but acquire knowledge, which is
of knowledge
already known to science, but is new and unknown to students themselves
(Ruby, 2001). They also differ from the CTM (the usual teacher’s transfer of
content), because students have the opportunity to: communicate with ma-
terials, make their own observations, expand their knowledge, understand
causal and consequential relations and understand the basis of the scientific
research and develop experimental skills (Akani, 2015; Kibirige, Maake, &
Francis-Mavhunga, 2014; Sadi & Cakiroglu, 2011).
Mirjana Maričić, Stanko Cvjetićanin Relevant Research

University of Novi Sad, Serbia
Branko Anđić
University of Montenegro, Montenegro Research which have included the contribution of the application of
laboratory and hands-on experiments (demonstration and student experi-
ments) are particularly numerous in science education i.e. science subject
768
ISSN 1648–3898 /Print/ TEACHER-DEMONSTRATION AND STUDENT HANDS-ON EXPERIMENTS IN TEACHING
INTEGRATED SCIENCES
ISSN 2538–7138 /Online/ (P. 768-779)
teaching. It has been done with students from the fifth grade of primary school and onwards, because in this
case laboratory work mostly refers to student activities within the school laboratory, or with the help of labora-
tory equipment in the classroom, when interacting with various materials in small groups, or individually for the
purpose of examining a scientific phenomenon. In this field, such research has been done within the framework
of cognitive and affective domains, and everything which could have been claimed as a part of the field of science
subject teaching. The results of these recent research have shown that there are positive effects:
•• SHE in relation to the CTM to: achievement, learning outcomes of low performing students, mastery of
chemistry concepts, science process skills students’ attitudes (Ateş & Eryilmaz, 2011; Cardak, Onder, &
Dikmenli, 2007; Ojediran, Oludipe, & Ehindero, 2014; Okam & Idris-Zakari, 2017; Sadi & Cakiroglu, 2011),
student’s understanding of subjects (Demircioğlu & Yadigaroğlu, 2011), students’ performance (Kibirige
et al., 2014), scientific thinking (Hugerat, Najami, Abbasi, & Dkeidek, 2014). For example, Cardak et al.
(2007) have examined the impact of student experiments within the experimental group in comparison
to the traditional approach within the control group. The results of the research have shown better
achievements of the students from the experimental group who studied the same contents with the
application of experiments.
•• DHE in relation to the CTM to: elimination of misconception (Sert-Çibik, Diken, & Darçin, 2008), students’
performance (Udo, 2010), on student’s achievement (Sola & Ojo, 2007). For example, the research by
Sert-Çibik et al. (2008), has examined the influence of group work and demonstration experiments
(E - experimental group) in comparison to traditional teaching (C - control group). Group work with a
demonstration approach to performing experiments has positively influenced the removal of student’s
misconceptions, i.e. E group students have shown better results.
•• SHE in comparison to the DHE to: the achievement of students in learning (Musasia, Ocholla, & Sakwa,
2016; Wachanga & Gowland-Mwaangi, 2004), student performance, quality and durability of knowledge
(Irinoye, Bamidele, Adetunji, & Awodele, 2015; Logar & Savec-Ferk, 2011), development of scientific
skills and students attitudes towards subject and science (Bilgin, 2006; Obadović, Rančić, Cvjetićanin,
& Segedinac, 2013). In addition, the student experiments has proven to be as effective as the teacher’s
demonstration experiments on the conceptual understanding of the content being studied (McKee-
Vickie, Williamson, & Ruebush, 2007). For example, the research conducted by McKee-Vickie et al. (2007),
has examined the impact of inquiry hands-on experiments within the experimental group in comparison
to inquiry demonstration experiments within the control group, on the conceptual understanding the
content of Reactions of Calcium. The results of the study have indicated that the students of both groups
showed almost equal conceptual understanding of the content they were studying.
•• SHE and DHE in relation to the CTM to: students’ achievement (Sola & Ojo, 2007), student’s performance
(Udo, 2010). For example, a research conducted by Udo (2010) has studied the impact of applying
guided discovery in the experimental group 1 (E1), the student-centered demonstration within the
experimental group 2 (E2) and the application of the method of exposure within the control condition
(C). The students from the Group E1 have shown the best achievements, followed by the students from
the Group E2 and the students from the Group C.
Students who are a part of the integrated science education system (from the 1st to the 4th grade) do not
go into school laboratories but learn the content of biology-physics-chemistry-ecology in an integrated form,
mainly in their classrooms. In the teaching process, when the content permits, they can use certain safe laboratory
equipment, or easily accessible, simple materials (with the supervision of teachers), which enable them to acquire
scientific knowledge in practice. In this field, research involving the application and contribution of simple hands-
on experiments (demonstration and students) using the above-mentioned equipment, has been very rare and
represents an unexplored field in world scientific education. The research on the application and contribution of
the application of simple hands-on experiments in integrated science education within the cognitive and affective
domains are significantly less represented. The results of these research have shown positive effects:
•• SHE in relation to the CTM to: quantum and quality of knowledge (Cvjetićanin, Segedinac, & Halaši, 2010),
understanding of the subject (Cakici & Yavuz, 2010), student interest and motivation (Golubović-Ilić,
2011). For example, Cakici and Yavuz (2010) have explored the impact of a constructivist approach (with
experiments) within the experimental group in relation to the traditional approach within the control
group. The results of the research have indicated a much better success (understanding the content
of Matter) of students from the experimental group.
769
INTEGRATED SCIENCES
(P. 768-779) ISSN 2538–7138 /Online/
•• SHE in comparison to DHE to: quality and retention of student knowledge (Cvjetićanin & Maričić, 2017;
Cvjetićanin, Obadović, & Rančić, 2015). For example, a research by Cvjetićanin et al. (2010) has studied
the influence of student hands-on experiments which has been realized within the experimental group
in comparison to traditional teaching which has been carried out within the control group. The results
have shown a better achievement of the students of the experimental group in terms of quantum and
quality of knowledge.
Identifying the Research Problem – Focus and Aim
Most previous studies related to the examination of the contribution of DHE in comparison to SHE to the quality
and durability of student knowledge, i.e. DHE and SHE in contrast to CTM (learning through instruction, without
hands-on experiments) have been carried out within individual subject teaching (mostly Physics and Chemistry).
With classroom students, such research has been significantly less widely conducted in the processing of various
elements of the integrated natural sciences. Based on a detailed analysis of the available literature, no studies have
been found on the comparative contribution of DHE and SHE in relation to CTM in the integrated sciences. In all
the analyzed studies, the application of DHE and SHE has been examined; SHE versus DHE, SHE versus CTM or
SHE versus CTM. No studies have been found which have examined the difference in the contribution of DHE and
SHE in relation to CTM to the quality and durability of students’ knowledge at different cognitive levels (under the
Anderson-Krathwohl-Bloom taxonomy).
The content of the integrated natural sciences is significantly more complex in the 3rd grade of primary school.
Content related to the topic of the air has a special place. Students need to understand various concepts and
properties of air (air movement, mass, volume, pressure, and so on), the influence of temperature on air, and air as
an insulator. For this reason, the question arises: When will students achieve better and more lasting knowledge,
if they study this content using DHE, SHE or CTM (verbal-textual method)?
The following aim thus comes into focus: Determining the contribution of DHE and SHE in relation to the CTM
to the quality and durability of 3rd grade student knowledge about air-related content in the integrated sciences
at all levels of the Anderson-Krathwohl-Bloom taxonomy. From the set goal, the following tasks can be identified:
1. Examine whether DHE contribute more in relation to the CTM to gaining better quality and more last-
ing knowledge of 3rd grade students on air at all cognitive levels?
2. Examine whether SHE contribute more in relation to the CTM to gaining better quality and more lasting
knowledge of 3rd grade students on air at all cognitive levels?
3. Examine the quality and durability of 3rd grade students’ knowledge of air on all cognitive levels by
the way in which experiments are carried out (when the students themselves perform or when these
experiments are demonstrated to them)?
Research Design
The research was quasi-experimental in character and was carried out according to the experimental design
with parallel groups, in the school year 2017/2018 in a time period of three months, through 8 phases (Table 1).
Table 1. Phases of research.
1 2 3 4 5 6 7 8
One school One school One school One school One school
3 weeks 1 week 2 weeks
hour hour hour hour hour
Analysis of pedagogical Revision Revision of

Creating Re-test (follow
documentation of prior Pre-test Training newly acquired Post-test
groups up test)
(AG) knowledge knowledge
770
INTEGRATED SCIENCES
ISSN 2538–7138 /Online/ (P. 768-779)
1 2 3 4 5 6 7 8
One school One school One school One school One school
3 weeks 1 week 2 weeks
hour hour hour hour hour
Second grade: 4.13

E1 group DHE
World around us: 4.06
previous the same newly acquired the same the same
Second grade: 4.21
knowledge pre-test E2 group SHE knowledge of post-test re-test
of the air the air
Second grade: 4,24 C group CTM
Sample
The research included students from the 3rd grade (between 9 and 10 years of age) from nine classes in four
primary schools (N = 180 students) in the municipality of Novi Sad - area of the Autonomous Province of Vojvodina
(Republic of Serbia). The sample of the research was of a convenience character, i.e. the students who were available
to the researcher at that particular moment, were selected. The research has been approved by the management
of selected schools with the consent of: pedagogue, psychologist and teachers of selected classes. The research
involved exactly 180 pupils, since these schools have nine 3rd grade classes. Three numerically uniform groups of
students were formed out of these nine classes. As there were between 20 and 25 students in each class, each of
the three groups consisted of 60 students. In addition to the criteria of numerical uniformity, the researchers have
paid attention for the selected classes to have: approximate average grades at the end of the 2nd grade; approximate
average grades from the subject The World around Us; and approximate previous knowledge of air-related content
from the previous grades (at pre-test). Data were obtained by one-way ANOVA analysis and it showed that there is
no statistically significant difference between these criteria (F = .084, p > .05).
Research Instruments
All groups of students were given the same tests: pre-test, post-test and re-test. Since there are no standardized
tests for checking students’ knowledge of the contents of integrated natural sciences in the system of education of the
Republic of Serbia, the researchers had themselves created tests with the help of teachers who taught in classes. The
validity of the tests was provided by examination and confirmation of two teachers who have a working experience of
over 10 years, and by examination and confirmation of two methodologists of integrated natural sciences - experts in
this field. The values of the Cronbach Alpha coefficient for each test were: pre-test α = .81, post-test α = .83, re-test α = .86.
Anderson, Krathwohl and Bloom taxonomy (Anderson, Krathwohl, & Bloom, 2001) was used to create test item, while
the manual Smart tests: Teacher-made tests that help students learn (Walker & Schmidt, 2004) was used for their design
at cognitive levels. The pre-test was different from the post-test and re-test by the contents that were examined. Items
based on air-related content from the previous classes were in the pre-test, while items based on air-related content from
the 3rd grade were on the post-test and re-test. Post-test and re-test differed only in the formulation of items, while in
structure and content they were very similar. In all tests, the quality of students’ knowledge was tested with two items of
different structures, at each cognitive level. Each item carried a higher number of points at the next - higher cognitive level.
The pre-test was designed and applied before the application of new content in order to provide the last criteria
for balancing groups and determine the quality of the existing student knowledge at all cognitive levels on air-related
content, and provide an updated basis for upgrading, expanding and adopting new knowledge of air-related content.
Pre-test results showed researchers whether the students had adopted the basic concepts of air in the previous classes,
since they are necessary for the acquisition of new knowledge about these contents. The pre-test consisted of 12 items,
created according to the model of the authors (Blagdanić, Kovačević, & Jović, 2016; Kukić & Aćimović, 2016). The post-
test was designed and applied after the pre-test in order to determine the quality of the newly acquired knowledge
of students at all cognitive levels on air-related content. The re-test was designed and applied two months after the
application of the post-test in order to determine the durability of the students’ newly acquired air-related content at all
cognitive levels. Post-test and re-test consisted of 12 items, created according to the model of the authors (Blagdanić,
Jović, Kovačević, & Petrović, 2016; Ralić-Žeželj, 2016).
771
INTEGRATED SCIENCES
(P. 768-779) ISSN 2538–7138 /Online/
Procedure and Treatment
For the purpose of this research, the following themes were selected: Non-living nature and Materials and their use.
The air-related content that was realized in the groups C, E1, E2, and processed within the aforementioned teaching
topics consisting of the following teaching units: The shape and pressure of the air, Changes arising from heating and
cooling of air (change temperature, volume, air flow...), Air heat insulator. By character, the selected content relates to
basic knowledge in physics and chemistry, but in an integrated form. The same teacher worked in all groups (C, E1
and E2). In the groups E1 and E2, the same experiments were performed (10 experiments in total per group), and they
were selected and adapted to the cognitive, mental and physical characteristics of students of the 3rd grade. The level
of complexity of the experiment increased (Figure 1) with each lesson. The basic methodic rule was employed. The
simplest experiments were performed first (Example 1), followed by the experiments of average complexity (Example
2) and ending with more complex experiments (Example 3). Before each class the teacher complied with the basic
methodological rule and performed experiments in his cabinet in order to see if they could be applied.
Example 1.
Name of the experiment: Warm and cool air!
Necessary equipment: balloon, and empty glass bottle, deeper bowl with warm water.
Description of the experiment: Put a balloon on the bottle, merge it in warm water and keep it like that for a
minute! What goes on? After that put the bottle in cold water and see what happens.
Pupil’s tasks: Does air have a shape? How does the temperature affect expanding and contracting of air?
Example 2.
Name of the experiment: Air mass
Necessary equipment: two identic balloons and a weighing scale.
Description of the experiment: put the balloons on a weighing scale. After that, remove one balloon and blow
it, tie it and put on a weighing scale.
Pupil’s tasks: What happened to the balloon when it was blown and why? What happened to the weighing scale
during the experiment and why? Based on this, which conclusion can you make?
Example 3.
Name of the experiment: Air insulator
Necessary equipment: three plastic bottles, jug with warm water; thermometer.
Description of the experiment: Put one glass inside another, and then pour the warm water in to reach the top
of the glass. Take a third glass and pour the warm water in, to reach the top of the glass. After that, use the
thermometer to measure the temperature of the water in the glass (both in regular and double glass). Measure
the temperature in three-minute intervals. Enter data into the table.
Time (minutes) Water temperature in double cup (° C) Water temperature in a plain cup (° C)
In the beginning (0 minutes)

3
6
9
12
Student’s tasks: How did the temperature act in the regular and how in the double glass? What is between
two glasses? What can you conclude based on it, what attribute does the air have? Give some examples from
everyday life, where can this characteristic of air be applied.
Figure 1. Examples of experiments with student’s tasks.
772
INTEGRATED SCIENCES
ISSN 2538–7138 /Online/ (P. 768-779)
Treatment in the group C
In the group C, the air-related contents were realized in a conventional way (CTM) using the verbal-textual
method. The teacher taught new air-related content to pupils using heuristic approach, following the material from
the textbook. Group C students did not perform experiments. After each explained air-related concept, students
noted the most important information into their notebooks. After finishing teaching, the teacher would revise
with students all the taught concepts.
Treatment in the group E1
Before the demonstration of each experiment in the group E1, the teacher explained the following to students:
which equipment and materials are necessary for the experiment, how to demonstrate the experiment, and how
to observe it. The frontal approach was used. After each experiment carried out by the teacher, a brief discussion
of the results was initiated, and the students correctly recorded the views in their special notebooks (experiment
notebooks). After all experiments were demonstrated, the common important conclusions about the air were
made, which the teacher wrote on the board, and the students in their regular notebooks at the end of the class.
Treatment in the group E2
Group E2 members were split into heterogeneous groups, composed of three students of different levels
of knowledge (good, average and low) and that did not change during classes in which air-related content was
taught. Before the beginning of each lesson, during which the pupils acquired new air-related content through
conducting an experiment, the teacher placed on the table of each group the necessary material (equipment) for
experiments and an instructive sheet for each group member in particular. Prior to performing the experiment, the
students were supposed to carefully read the instructions from the instruction sheet, and then talk in the group
about what is expected of them. The instructions contained the following information: the equipment and material,
the description of how to conduct the experiment and the tasks which the students were supposed to complete
(based on their results and observations after the experiment was performed). When the teacher estimated that the
students were familiar with the text and were ready to work, they approached the experimental phase. All groups
of students performed experiments of the same level of complexity at all classes. Upon completion of the work of
the group, a representative of each group presented the results that his/her group obtained from the experiment.
After results of all groups were presented, a discussion was initiated in which the students corrected the wrong
conclusions and made the correct ones. The students then checked their notes to see whether they wrote down
the course of the experiment correctly, whether their answers and conclusions were correct and if they were not,
they were corrected. After that students used their regular notebooks to write down the common conclusions
about air, which came as a result of all group’s work.
Data Analysis
Cronbach alpha coefficient was used in order to determine the reliability of the tests. One-way ANOVA analysis
was applied to determine if there is a statistically significant difference between the average grades of students
at the end of the 2nd grade, the average grades of students from the subject The World around Us and the grades
they achieved on the pre-test. One-way ANOVA analysis was also applied to compare differences between mean
values of all groups (E1, E2 and C) at pre-test, post-test and re-test. To determine the statistically significant differ-
ence between the quality and the durability of the knowledge of each of the two groups separately (E1 and E2, E1
and C, E2 and C), the Scheffe post-hoc test was used. Wilcoxon test was used in order to determine the difference
between pre-test and post-test and the difference between post-test and re-test in all three groups. SPSS program,
version 22, was used for statistical analysis of data.
Research Results
The results of Spearman’s correlation coefficient showed a moderate correlation between the average grades
of students at the end of the 2nd grade and the grades they achieved at the pre-test (ρ = .463 with p < .001), and
773
INTEGRATED SCIENCES
(P. 768-779) ISSN 2538–7138 /Online/
between the average grades of students from the subject The World around Us and their grades at pre-test (ρ =
.481, p < .001). One-way ANOVA analysis showed that there are no statistically significant differences between the
students’ knowledge of C, E1 and E2 group at all cognitive levels at the pre-test (p > .05). Statistically significant
difference is present at post-test and re-test at the level of: analysis, evaluation and synthesis (Table 2).
Table 2. Statistically significant differences between students of C, E1 and E2 group at all cognitive levels at
post-test and re-test - one-way ANOVA.
Sum of
Cognitive level Group SD SE F p
Squares AS
Post-test
90.989 2 46.214
Between groups
Analysis Within groups 1119.622 175 6.162 8.097 .0001
Total
1221.503 178
397.230 2 231.122
Between groups
Within groups 1331.334 168 8.112 26.623 .00001
Evaluation
Total
1882.656 171
675.655 2 354.298
Between groups
Synthesis Within groups 2098.297 178 10.925 29.103 .00001
Total
2773.131 181
Re-test
110.993 2 54959
Between groups
Analysis Within groups 752.795 175 4.189 12.996 .0001
Total
859.876 180
492.878 2 249.022
Between groups
Within groups 1352.524 176 7.703 32.003 .00001
Evaluation
Total
1871.124 179
747.205 2 373.031
Between groups
Synthesis Within groups 1698.997 177 9.71 39.025 .00001
Total
2447.122 179
The Scheffe post-hock test showed that there is a statistically significant difference between the quality and
durability of E1 and E2 group knowledge at the level of evaluation and synthesis at post-test and at the level of
analysis, evaluation and synthesis at re-test (Table 3).
Table 3. Statistically significant differences in the quality and retention of knowledge between students of
C, E1 and E2 group - Scheffe post-hock test.
95% Confidence Interval

Dependent Mean Difference
(I) Group (J) Group SE p
Variable (I-J) Lower Upper
Bound Bound
Post-test
E1 -1.54868 .45838 .011 -2.3912 -.1092
C
E2 -1.68687 .45938 .0001 -2.7979 -.5397
C -1.54868 .45838 .011 .1092 2.3818
E1
Analysis E2 -.42098 .48087 .630 -1.5665 .7025
C -1.68687 .45838 .0001 ,5396 2,8166
E2
E1 -.42098 .45838 .630 -,7103 1,5785
774
INTEGRATED SCIENCES
ISSN 2538–7138 /Online/ (P. 768-779)
95% Confidence Interval

Dependent Mean Difference
(I) Group (J) Group SE p
Variable (I-J) Lower Upper
Bound Bound
E1 -2.04890* .521874 .0001 -3.3385 -.7588

C
E2 -3.86990* .52196 .00001 -5.1565 -2.5812
C -2.04890* .521874 .0001 .7593 3.3407
E1
Evaluation E2 -1.84030* .54292 .001 -3.1212 -.5398
C -3.86990 .521874 .00001 2,5899 5,1722
E2
E1 -1.84030 .521874 .001 ,5403 3,1235
E1 -1.91997* .62887 .002 -3.3984 -.3680
C
E2 -4.73953* .62887 .0001 -6.3054 -3.1881
C -1.91997* .62887 .002 .3678 3.4685
E1
Synthesis E2 -2.81423* .61885 .001 -4.3695 -1.2659
C -4.73953 .62887 .0001 3,1896 6,2955
E2
E1 -2.81423 .62887 .001 1,2698 4,392
Re-test
E1 -1.49957* .37523 .001 -2.4265 -.5738
C
E2 -1.80110* .37531 .003 -2.7253 -.8718
C -1.49957* .37523 .001 .5738 2.4269
E1
Analysis E2 -2.80188* .61798 .013 -4.3097 -1.2598
C -1.80110 .37523 .003 ,8788 2,7551
E2
E1 -2.80188 .37523 .013 ,5456 2,8155
E1 -2.16921* .50844 .000 -3.4276 -.9157
C
E2 -4.05849* .50849 .002 -5.3146 -2.8027
C -2.16921* .50844 .0001 .9151 3.4269
E1
Evaluation E2 -1.88697* .50865 .0001 -3.1431 -.6309
C -4.05849 .50844 .002 2,8033 5,3157
E2
E1 -1.88697 .50844 .0001 ,6323 3,1442
E1 -2.11493* .56579 .001 -3.5109 -.7177
C
E2 -4.96821* .56581 .0001 -6.3647 -3.5712
C -2.11493* .56579 .0001 .7178 3.5109
E1
Synthesis E2 -2.85296* .56581 .001 -4.2496 -1.4549
C -4.96821 .56579 .0001 3,5735 6,3702
E2
E1 -2.85296 .56579 .001 1,4578 4,2492
The Coefficient of variation (CV) showed significantly lower values at the level of analysis, evaluation and syn-
thesis in the E1 and E2 groups compared to the C group at the post-test, as well as the lower values at the level of
analysis, evaluation and synthesis in the E2 group compared to E1 and C group at the re-test (Table 4).
Table 4. CV in all groups (C, E1 and E2) at individual cognitive levels at post-test and re-test.
CV (%)
Group Analysis Evaluation Synthesis
Post-test
C 15.73 17.22 18.75
E1 4.98 5.88 6.68
E2 3.73 4.34 5.12
Re-test
C 19.01 19.63 20.01
E1 11.95 12.91 14.08
E2 4.12 4.97 5.03
775
INTEGRATED SCIENCES
(P. 768-779) ISSN 2538–7138 /Online/
The Wilcoxon test showed that there is no statistically significant difference between the pre-test and the
post-test results of the students C group at all cognitive levels. Statistically significant difference can be found in
the E1 group at the analysis level (Z = -1.782, p = .026, r = .187), i.e. in the group E2 at the analysis (Z = -1.824, p
= .018, r = .192), evaluation (Z = -1.763, p = .012, r = .173) and synthesis level (Z = -1.886, p = .026, r = .190). When
analyzing the difference between post-test and re-test results, it can be noted that there is a statistically significant
difference only at the analysis level in the E1 group (Z = -1.801, p = .039, r = .188).
When comparing students’ success in each group (E1, E2, and K) at the same cognitive level at the pre-test,
post-test and re-test, it can be noticed that there is a statistically significant difference in group E1 and group E2 in
the number of students who have correctly solved the items at the cognitive level of: application (E1: F = 10.212,
p = .012, E2: F = 101.031, p = .001), analysis (E1: F = 1.92, p = .0001, E2: F = 6.989, p = .001) evaluation (E1: F = 8.981,
p = .001, E2: F = 8.122, p = .001) and synthesis (E1: F = 11.292, p = .011, E2: F = 12.783, p = .001) on these tests. This
difference is not present in the C group (p > .05).
Discussion
The results of the research showed that the high average grades of students at the end of the 2nd grade, the high
average grades in the subject of The World around Us, as well as revising the previous knowledge of the air-related
content is not a reliable criterion for the students to achieve good pre-test results. The poor knowledge of students
in groups C, E1 and E2 at pre-test is most likely the result of the previous learning method through which students
acquired these contents, as well as the process of forgetting and interfering with other content (Rather, 2010). In
conversation with teachers of students who participated in the research, as well as with the students themselves, it
was concluded that the air-related content, as well as other contents of natural sciences in the 1st and 2nd grade, were
realized through the application of the CTM, without use of experiments. In addition, the results showed that there was
no difference in the quality of knowledge between students of all three groups at all cognitive levels at the pre-test.
At the level of analysis, evaluation and synthesis, students C, E1 and E2 group showed lower knowledge. The results
obtained on the pre-test are similar to the results of numerous research in which the efficiency of the application of
CTM has been examined and compared with the laboratory method on the quality of students’ knowledge in science
subject teaching and integrated science education (Cvjetićanin & Maričić, 2017; Golubović-Ilić, 2011).
Students of all three groups achieved better post-test results, which can be noticed on the basis of a compara-
tive analysis of the total number of points achieved on the pre-test and post-test. Statistically significant difference
between pre-test and post-test results of students is noticed within the E1 and E2 groups. However, it is not statistically
significant in the group C, which leads to the conclusion that CTM cannot enable students to acquire knowledge at
higher cognitive levels (Ivić, Pešikan, & Antić, 2001). The obtained results are consistent with the results of most of
the previous research on the contribution of experiments (DHE and SHE) to the quality of students’ knowledge of the
contents of natural sciences in relation to CTM, or some teaching method in science subject teaching and integrated
science education. Thus, for example, in a research by Musasia et al. (2016) students of control and experimental group
who studied physics contents using traditional approach and practical (experimental) activities have achieved bet-
ter results at post-test than at pre-test. The same results were obtained in the research conducted by Tauel and Erol
(2008), where a difference in the quality of student knowledge of magnetism has been studied, when students were
taught in a conventional, lecturing way and with cooperative learning. In contrast to the aforementioned, research
conducted by Uside, Barchok, and Abura (2013) found that between the results of the pre-test and the post-test, no
statistically significant difference was noticed in the students’ knowledge of physical content, given they studied
using experiments and in a traditional way. Students of all three groups achieved similar knowledge at the level
of knowledge, understanding and application. Statistically significant difference between the quality of students’
knowledge was noticed at the level of analysis, evaluation and synthesis in groups C and E1 and groups C and E2. The
difference in success between students in group E1 and E2 was noticed only at the level of evaluation and synthesis,
i.e. students in the E2 group were more successful than E1 students in the correct solution of two or one item at these
levels. In addition, it can be noted that DHE significantly influenced the quality of E1 students’ knowledge of the level
of analysis and slightly at the level of evaluation and synthesis, when comparing their results from the pre-test and
post-test. Similar results have also been found in the research Sert-Çibik et al. (2008) in which demonstration experi-
ments significantly influenced students’ success in relation to the conventional approach. However, the results of some
research disagree with the results of this and other research. Thus, for example, research by Cvjetićanin et al. (2015)
has shown that demonstration experiments contributed to the quality of student knowledge about the materials
776
INTEGRATED SCIENCES
ISSN 2538–7138 /Online/ (P. 768-779)
at the level of analysis, as there was difference in knowledge at this level between students to whom experiments
were demonstrated and students who conducted same experiments independently. The results of the research
have shown that students of E1 and E2 groups have higher uniformity of knowledge at the level of: analysis, evalua-
tion and synthesis, than C group students, whereas this uniformity is higher in the E2 group than in the E1 group at
the level of evaluation and synthesis. The obtained results point to the fact that SHE contribute more to the better
quality of student knowledge of 3rd grade on air-related content when compared to DHE. The results of this research
are similar to the results of the research (Wachanga & Gowland-Mwaangi, 2004) in which both experimental groups
of students (1EG and 3EG), who studied chemical contents with the application of experiments with a cooperative
approach, have achieved considerably better results than the students of both control groups (2CG and 4CG), who
have been taught the same contents with the application of regular learning methods. In a research by Rendler and
Hulde (2007), students who performed hands-on experiments have gained more durable knowledge from students to
whom these experiments have been demonstrated. Contrary to what was stated, in the Logar and Savec-Ferk (2011)
research, results have shown that demonstration experiments had a greater effect on students’ knowledge quality
than students’ hands-on experiments.
The results of the research showed that there was no difference in the C and E2 groups between the post-test
and the re-test results in terms of the quality of students’ knowledge at every cognitive level. Statistically significant
difference is noticed in the group E1 but only at the level of the analysis. At the lower cognitive levels (knowledge,
understanding and application), students of all three groups achieved similar knowledge. Students of the E2 group
were more successful at the level of analysis, evaluation and synthesis, i.e. they showed more durable knowledge than
students from the C and E1 groups. None of the students in the group E1 at the re-test did both items at the analysis
level, while only 11.66% answered correctly to one item. When comparing students’ knowledge of air on all cognitive
levels at post-test and re-test, it is noted that students in C, E1 and E2 groups achieved lower success at re-test from
success they achieved at the post-test. These results are in correlation with research results (Cvjetićanin & Maričić,
2017) in which the students of the control and experimental group displayed lower knowledge at the re-test in rela-
tion to the post-test. Bearing in mind the effect of spontaneous and active forgetting, this was to be expected (Glynn,
Britton, & Yeany, 2012; Robbins, Schwartz, & Wasserman, 2001). The students did not revise the air-related content
between the post-test and the re-test i.e. they studied other contents during that time period, which affected the
air-related content (Sternberg & Zhang, 2001). The obtained results are similar to the results of numerous research.
For example, in the research Cvjetićanin et al. (2015), E group students (student experiments) achieved more lasting
knowledge in relation to the students from the K group (demonstration experiments) at higher cognitive levels. The
results of the Badeleh (2011) research are similar, where the students of the experimental group achieved more lasting
knowledge of chemistry contents from the students of the control group at all cognitive levels. In addition, the results
of the research have shown that there is a statistically significant difference (as opposed to post-test) between the
knowledge of students E1 and E2 groups at the level of analysis, evaluation and synthesis. The students in group E2
showed greater uniformity in their knowledge at these cognitive levels, as compared to the students in groups C and E1.
When comparing students’ success in each group, especially, on pre-test, post-test, and re-tests at the same
level of knowledge, it can be noted that there is no statistically significant difference in the number of students who
correctly solved the items at the first two cognitive levels. Statistically significant difference is noticed within the E1
and E2 groups at the cognitive level of application, analysis, evaluation, and synthesis, while it does not exist within
C group. These data confirm the fact that simple hands-on experiments (SHE and DHE) contribute more to a higher
quality and durability of students’ knowledge than CTM.
Conclusions
This research confirms the fact that students achieve better knowledge using hands-on experiments than when
they study the same content using the CTM. In this study, unlike in previous (available) research, the difference in the
contribution of DHE and SHE comparing to the CTM in terms of both the quality and durability of student knowl-
edge in the 3rd grade of primary school education on air at different cognitive levels has been examined for the first
time. The majority of students who studied using the CTM achieved the highest quality and durability of knowledge
at the cognitive level of application. The means of conducting the experiment affects the quality and durability of
students’ knowledge of the topic of the air. DHE significantly contributes to the quality of students’ knowledge at the
level of analysis and partly at the level of evaluation, while their impact on the durability of knowledge at these levels
decreases over time. Unlike DHE, SHE affects the quality of knowledge at higher cognitive levels, including those of
777
INTEGRATED SCIENCES
(P. 768-779) ISSN 2538–7138 /Online/
analysis, evaluation and synthesis. Their contribution does not decrease over time; that is, these experiments equally
influence the durability of students’ knowledge. Therefore, it is suggested that when processing this content, prefer-
ence should be given to hands-on experiments in relation to CTM, i.e. SHE as opposed to DHE.
Acknowledgements
Funding: This research was done within the following projects:

•• Ministry of Education, Science and Technological Development of the Republic of Serbia: Quality of the
education system of Serbia in the European perspective [Grant number: 179010].
•• Ministry of Science Montenegro – HERIC project [Grant number: 01-2864].
References
Akani, O. (2015). Laboratory teaching: Implication on students’ achievement in chemistry in secondary schools in Ebonyi State of
Nigeria. Journal of Education and Practice, 6(30), 206-213.
Anderson, L. W., Krathwohl, D. R., & Bloom, B. S. (2001). A taxonomy for learning teaching and assessing: A revision of Bloom’s taxonomy
of educational objectives. New York: Longman Publishing.
Ateş, Ö., & Eryilmaz, A. (2011). Effectiveness of hands-on and minds-on activities on students’ achievement and attitudes towards
physics. Asia-Pacific Forum on Science Learning and Teaching, 12(1), 1-22.
Badeleh, A. (2011). The effect of laboratory training model of teaching and traditional method on knowledge, comprehension,
application, skills-components of achievement, total achievement and retention level in chemistry. Journal on School Educa-
tional Technology, 7(1), 19-29.
Bilgin, I. (2006). The effects of hands-on activities incorporating a cooperative learning approach on eight grade students’ science
process skills and attitudes toward science. Journal of Baltic Science Education, 9(1), 27-37.
Blagdanić, S., Kovačević, Z., & Jović, S. (2016). Istražujemo svet oko nas 1 – Radna sveska za prvi razred osnovne škole [We are exploring
the world around us 1 - A workbook for the first grade of primary school]. Beograd: BIGZ školstvo.
Blagdanić, S., Jović, S., Kovačević, Z., & Petrović, A. (2016). Priroda i društvo 3 – Radna sveska za treći razred osnovne škole [Nature and
society 3 - The workbook for the third grade of primary school]. Beograd: BIGZ školstvo.
Cakici, Y., & Yavuz, G. (2010). The effect of constructivist science teaching on 4th grade students’ understanding of matter. Asia-Pacific
Forum on Science Learning and Teaching, 11(2), 1-19.
Cardak, O., Onder, K., & Dikmenli, M. (2007). Effect of the usage of laboratory method in primary school education for the achieve-
ment of the students’ learning. Asia-Pacific Forum on Science Learning and Teaching, 8(2), 1-11.
Cvjetićanin, S., Segedinac, M., & Halaši, T. (2010). Značaj primene metode eksperimenta u razrednoj nastavi [The importance of the
application of experimental method in classroom teaching]. Nastava i vaspitanje, 2, 173–189.
Cvjetićanin, S., Obadović, D., & Rančić, I. (2015). The efficiency of student-led and demonstration experiments in initial physicschemi-
cal education in primary school. Croatian Journal of Education, 17(3), 11-39.
Cvjetićanin, S., & Maričić, M. (2017). The contribution of demonstration and student-led experiments on the students’ knowledge
quality in the third grade of primary school. Journal of Baltic Science Education, 16(5), 634-650.
Demircioğlu, G., & Yadigaroğlu, M. (2011). The effect of laboratory method on high school students’ understanding of the reaction
rate. In A. G. Balim, H. Aydin, E. Akpinar & G. Ünal Çoban (Eds.), World Conference on New Trends in Science Education (pp. 509-
516). Izmir, Turkey: Dokuz Ejlül Üniversity Institute.
Glynn, S., Britton, B., & Yeany, R. (2012). Psychology of learning science. Oxon: Routledge.
Golubović-Ilić, I. (2011). Stavovi i mišljenja učenika o primeni laboratorijsko-eksperimentalne metode u nastavi prirode i društva
[The attitudes and opinions of students on the application of laboratory-experimental method in teaching of science and
society]. Pedagogija, 4, 674-685.
Hırça, N. (2013). The influence of hands on physics experiments on scientific process skills according to prospective teachers’ experi-
ences. European Journal of Physics Education, 4(1), 1-9.
Hodson, D. (1993). Re-thinking old ways: Towards a more critical approach to practical work in school science. Studies in Science
Education, 22(1), 85–142.
Hofstein, A., & Lunetta, V. (2004). The laboratory in science education: Foundations for the twenty first century. Science Education,
88, 28–54. doi: 10.1002/sce.10106.
Hugerat, M., Najami, N., Abbasi, M., & Dkeidek, I. (2014). The cognitive acceleration curriculum as a tool for overcoming difficulties
in the implementation on inquiry skills in science education among primary school students. Journal of Baltic Science Educa-
tion, 13(4), 523-532.
Irinoye, J., Bamidele, E. F., Adetunji, A. A., & Awodele, B. A. (2015). Relative effectiveness of guided inquiry and demonstration meth-
ods on students’ performance in practical chemistry in secondary schools in Osun State, Nigeria. Advances in Social Sciences
Research Journal, 2(2), 21-30.
Ivić, I., Pešikan, A., & Antić, S. (2001). Aktivno učenje 2 [Active Learning 2]. Beograd: Institut za psihologiju.
Kibirige, I., Maake, R., & Francis-Mavhunga, F. (2014). Effect of Practical Work on Grade 10 Learners’ Performance in Science in Mank-
weng Circuit, South Africa. Mediterranean Journal of Social Sciences, 5(23), 1568-1577. doi: 10.5901/mjss.2014.v5n23p1568.
778
INTEGRATED SCIENCES
ISSN 2538–7138 /Online/ (P. 768-779)
Kukić, M., & Aćimović, M. (2016). Svet oko nas 2 - Radna sveska za drugi razred osnovne škole [The world around us 2 - The workbook
for the second grade of primary school]. Čačak: Pčelica.
Logar, A., & Savec-Ferk, V. (2011). Students’ hands-on experimental work vs lecture demonstration in teaching elementary school
chemistry. Acta Chimica Slovenica, 58(4), 866–875.
McKee-Vickie, K., Williamson, M., & Ruebush, E. (2007). Effects of demonstration laboratory on student learning. Journal of Science
Education and Technology, 16(5), 395-400. doi: 10.1007/s10956-007-9064-4.
Musasia, M, A., Ocholla, A, A., & Sakwa, T. W. (2016). Physics practical work and its influence on students’ academic achievement.
Journal of Education and Practice, 7(28), 129-134.
Obadović, D., Rančić, I., Cvjetićanin, S., & Segedinac, M. (2013). The impact of implementation of simple experiments on the pupils’
positive attitude in learning science contents in primary school. The New Educational Review, 34(4), 138-150.
Ojediran, I. A., Oludipe, D. I., & Ehindero, O. J. (2014). Impact of laboratory-based instructional intervention on the learning outcomes
of low performing senior secondary students in physics. Creative Education, 5(4), 197-206. doi: 10.4236/ce.2014.54029.
Okam, C. C., & Idris-Zakari, I. (2017). Impact of Laboratory-based Teaching Strategy on Students’ Attitudes and Mastery of Chemistry.
International journal of innovative research & development, 6(1), 112-121.
Ralić-Žeželj, R. (2016). Maša i Raša - Priroda i društvo – Radna sveska za treći razred osnovne škole [Masha and Rasa - Nature and society
- The workbook for the third grade of primary school]. Beograd: Klett.
Rather, A. R. (2010). Psychology of learning and development. Devon: Discovery Publishing House.
Randler, C., & Hulde, M. (2007). Hands-on versus teacher-centered experiments in soil ecology. Research in Science &Technological
Education, 25(3), 329-338.
Robbins, S., Schwartz, B., & Wasserman, E. (2001). Psychology of learning and behavior. New York: W.W. Norton & Company.
Ruby, A. M. (2001). Hands-on Science and Student Achievement. Santa Monica, CA: RAND Corporation.
Sadi, Ö., & Cakiroglu, J. (2011). The effects of hands-on activity enriched instruction on students’ achievement and attitudes towards
science. Journal of Baltic Science Education, 10(2), 87-97.
Salameh El Rabadi, E. G. (2013). The effect of laboratory experiments on the upper basic stage students’ achievement in physics.
Journal of Education and Practice, 4(8), 62-71.
Sert-Çibik, A., Diken, H. E., & Darçin, E. S. (2008). The effect of group work and demonstrative experiments based on conceptual
change approach: Photosynthesis and respiration. Asia-Pacific Forum on Science Learning and Teaching, 9(2), 1-22.
Sola, A. O., & Ojo, O. E. (2007). Effects of project, inquiry and lecture-demonstration teaching methods on senior secondary students’
achievement in separation of mixtures practical test. Educational Research and Review, 2(6), 124-132.
Sternberg, R., & Zhang, L. (2001). Perspectives on thinking, learning, and cognitive styles. New Jersey: Lawrence Erlbaum Associates.
Tauel, Z., & Erol, M. (2008). Effects of cooperative learning on instructing magnetism: analysis of an experimental teaching sequence.
Latin-American Journal of Physics Education, 2(2), 124-136.
Udo, M. E. (2010). Effect of guided-discovery, student-centred demonstration and the expository instructional strategies on students’
performance in chemistry. African Research Review, 4(4), 389-398.
Uside, O. N., Barchok, K. H., & Abura, O. G. (2013). Effect of discovery method on secondary school student’s achievement in physics
in Kenya. Asian Journal of Social Sciences & Humanities, 2(3), 351-358.
Wachanga, S., & Gowland-Mwaangi, J. (2004). Effects of the cooperative class experiment teaching method on secondary school
students’ chemistry achievement in Kenya’s Nakuru district. International Education Journal, 5(1), 26-36.
Walker, C., & Schmidt, E. (2004). Smart tests: Teacher-made tests that help students learn. Ontario: Pembroke Publishers.
Mirjana Maričić MSc, PhD Student, Faculty of Education, University of Novi Sad,
(Corresponding author) Podgorička 4, 25 000 Sombor, Serbia.
E-mail: mimiunt@hotmail.com
Website: http://www.pef.uns.ac.rs/
Stanko Cvjetićanin PhD, Professor, Faculty of Education, University of Novi Sad,
Podgorička 4, 25 000 Sombor, Serbia.
E-mail: tozchemy@eunet.rs
Website: http://www.pef.uns.ac.rs/
Branko Anđić MSc, PhD Student, Faculty of Natural Sciences and Mathematics,
Džordža Vašingtona bb, 81 000 Podgorica, Montenegro.
E-mail: brankoan@yahoo.com
Website: https://www.ucg.ac.me/pmf
779
EXAMINATION OF PERCEPTIONS
OF GIFTED STUDENTS ABOUT
CLIMATE CHANGE AND GLOBAL
WARMING
Abstract. The aim of this research was to Fatma Mutlu,

examine the perceptions of gifted students Oğuzhan Nacaroğlu
about climate change and global warm-
ing from socio-scientific topics. Phenom-
enology design, which is a qualitative
research design, was used in the research.
The sample group consisted of 53 gifted
students studying in a Science and Art Introduction
Center located in Anatolia in the 2018-2019
academic year. As data collection tool, the It is important for societies to adapt to the rapid changes in science and
draw-write-tell technique was used. The technology and to have new scientific knowledge related to this because
drawings of the students were grouped into developments in science and technology cause scientific knowledge to
four categories as causes of climate change change and develop. As scientific knowledge changes and evolves today, the
and global warming, their results, recom- needs and expectations of the societies change in this direction (Türkoğlu
mendations and unacceptable drawings; & Öztürk, 2019). Besides, some scientific developments have led to discus-
whereas, the compositions of the students sions and dilemmas with no simple consequences (Eş, Mercan, & Ayas, 2016;
and the focus group interviews were col- Sadler, 2004) in the societies (Kılınç, Demiral, & Kartal, 2017). Therefore, these
lected under three categories including scientific developments that cannot be considered separately from the society
causes, results and recommendations as should be considered and evaluated with the participation of scientists and
a result of content analysis. The research other individuals in the society (Sürmeli, Duru, & Duru, 2017). In this case,
results suggested that while the perception socio-scientific issues based on the interaction of science and society emerge
levels of gifted students about the causes (Kolstø, 2001; Topaloğlu & Kıyıcı, 2017).
and possible results of climate change and Socio-scientific issues are the subjects which have scientific basis, are
global warming were high, their perception valuable for society (Ratcliffe & Grace, 2003), have multiple solutions (Zeidler
level about the solution of these problems & Nichols, 2009), are open-ended and mostly controversial (M. Genç & T. Genç,
was quite low. In addition, it was deter- 2017) and have effective moral values in decision making process (Nuhoğlu,
mined that students had misconceptions 2014). Bakırcı, Artun, Şahin, and Sağdıç (2018) defined the socio-scientific
about climate change and global warming. issues as subjects that are taken into consideration by the society and sci-
Besides, the research results showed that ence, have no definitive solution, and include moral, political and ethical
the draw-write-tell technique was a useful concerns. Climate change and global warming, organic farming, the use of
tool in determining the students’ percep- genetically modified organisms, studies in genetic engineering, and cloning,
tion about socio-scientific subjects. gene therapy, and stem cell researches can be given as examples to socio-
Key words: climate change, draw-write- scientific subjects (Atasoy, 2018; Klop & Severiens, 2007). Climate change
tell technique, gifted students, global and global warming from socio-scientific subjects are primary ones among
warming. the most important problems encountered by the world in recent years and
it has reached to a dimension that threatens not only human beings but all
living creatures (Aydın, 2017).
Fatma Mutlu Global warming is defined as the increase of the average earth tem-
Inonu University, Turkey perature at the parts of atmosphere close to the earth surface naturally or by
Oğuzhan Nacaroğlu
Erciyes University, Turkey the effect of human (Aksay, Ketenoğlu, & Kurt, 2005). This increase is caused
by the greenhouse effect developing due to the increase of ratios of some
gases to values higher than normal values (Çepel, 2008). In fact, 85% of the
780
ISSN 1648–3898 /Print/ EXAMINATION OF PERCEPTIONS OF GIFTED STUDENTS ABOUT CLIMATE CHANGE AND
GLOBAL WARMING
ISSN 2538–7138 /Online/ (P. 780-792)
greenhouse effect is composed of water vapor, 12% is composed of small water molecules in the atmosphere,
and about 3% is composed of chlorofluorocarbons, methane, nitrogen oxides, ozone and CO2, formed mostly by
human activities. The most important reason in the increase of these gases is the rapid economic growth starting
with the industrial revolution and in parallel, the use of more fossil fuels (Kadıoğlu, 2001). Climate change, on the
other hand, refers to the changes in climate systems such as humidity, precipitation and temperature along with
the increase in greenhouse gases (Doğan & Tüzer, 2011). According to the report announced by the United Nations
Intergovernmental Panel on Climate Change, it is stated that the climate crisis will reach irreversible dimensions in
the next 20 years unless necessary measures are taken for climate change and global warming (Yıldızoğlu, 2018).
Similarly, in the Global Risks report of the World Economic Forum (2018), climate change and global warming con-
stitute one of the important topics. When the report is examined, it is seen that more than half of the top 10 risks
are caused by climate change and global warming. Therefore, it is necessary to raise awareness in all individuals
about climate change and global warming and to conduct works in order for the individuals in all segments as a
requirement of socio-scientific subjects not only for the scientist to take necessary steps (Aksan & Çelikler, 2013).
Besides, it is also important to determine the individuals’ knowledge levels about climate change and global warm-
ing and their perception style on this concept in order to take the necessary steps.
In recent years, emphasizing the human-environment relationship within the educational activities given
to individuals in the school is highlighted. The special purposes of the updated 2018 Science curriculum include
bringing basic information about environmental sciences, adopting scientific process skills and scientific research
approach and finding solutions to the problems encountered in these fields in the process of discovering the na-
ture and understanding of the human-environment relationship, recognizing the interaction between individual,
environment and society, and developing sustainable development awareness on society, economy and natural
resources (MEB, 2018). Besides, when the literature is reviewed, it is seen that individuals in different teaching lev-
els have significant lack of knowledge and different perceptions about the concept of climate change and global
warming (Aydın, 2017; Bahar, 2000; Eroğlu, 2009; Yalçın, 2010). For example, in the study conducted by Aydın (2017)
to determine the knowledge level of the university students about global warming it was determined that the
students had serious lack of knowledge. Therefore, it is necessary to find out the thoughts of individuals in different
teaching levels about climate change and global warming constituting one of the most significant problems of
today’s world and to take the necessary steps to determine and eliminate their misconceptions, if they have any.
One of these institutions is the Science and Art centers (BILSEM) where gifted children are educated by flexible
programs (Öztürk, Eş, & Turgut, 2017).
Research Focus
BILSEMs are independent private education institutions that are opened to enable gifted students to develop
and use their talents at the highest level (MEB, 2017). Gifted students who succeed in science and art exams study
at BILSEMs, are subject to some programs such as integration, support education, individual talent recognition
(BYF), special talent development (ÖYG) and project. The main purpose of these programs is to give gifted students
the opportunity to become aware of their talents and to use them. For this purpose, activities are carried out in
each program to provide students with different skills. In these institutions, it is also aimed to multi-dimensionally
develop individuals who can bring important ideas in solving the world problems (Clark, 2015). In line with this goal,
it is necessary to examine the thoughts of BILSEM students about climate change and global warming constitut-
ing one of the today’s most important problems and to determine their misconceptions if there is any and thus to
take necessary steps. However, it is necessary to use data variation technique to determine the perceptions of the
students about concepts rather than a single measurement tool (Dinç & Üztemur, 2017). One of these techniques
is the draw-write-tell technique that includes drawing, writing and explaining that concept by students while
explaining the concepts (Üztemur & Dinç, 2018).
The draw-write-tell technique is based on the students’ drawing on the axis of certain concepts, then writing
down their thoughts on this concept and finally expressing their thoughts about the concepts. With the draw-
ings, perceptions of the students about concepts can be revealed (Pınar & Yakışan, 2016) and the students can
also express their opinions and thoughts more willingly and clearly (Özsoy & Ahi, 2014). In addition, conducting
necessary interviews with the students along with the drawings (Ersoy & Türkkan, 2010) and asking them to write
a composition explaining their drawings will allow to examine their perceptions about the concepts in more detail.
In the literature, there are many studies exploring the knowledge level of individuals about socio-scientific issues
781
GLOBAL WARMING
(P. 780-792) ISSN 2538–7138 /Online/
(Türkoğlu & Öztürk, 2019); whereas, no research was found investigating the knowledge level of gifted children
studying in BILSEM about climate change and global warming concepts from socio-scientific subjects. Besides,
while data collection tools such as questionnaire and attitude scale have been generally used in the studies inves-
tigating the socio-scientific subjects (M. Genç & T. Genç, 2017), no study has been found based on draw-write-tell
technique. In this respect, this research was considered as important in terms of contributing to the literature.
This research was conducted to determine the perceptions of gifted children about climate change and global
warming from socio-scientific subjects and to examine the effectiveness levels of the draw-write-tell technique
in the examination of socio-scientific subjects. Within the frame of this general purpose, answers were sought to
the following questions:
1. What are the perceptions of gifted students studying in BILSEM about climate change and global
warming?
2. Is the draw-write-tell technique effective in determining the perception levels of the students about
climate change and global warming?
Research Design
In this research, phenomenology (phenomenon-science) design among qualitative research method designs
was used. Phenomenology provides an opportunity to examine in depth the facts and events which are aware of
but not fully understood according to the experiences of individuals (Christensen, Johnson, & Turner, 2015). In addi-
tion, phenomenological studies reveal the perspective of individuals by focusing entirely on individual perceptions
(Tanyaş, 2014). In this research, the phenomenology design was also preferred to determine how the perceptions
of gifted students studying in BILSEM are about climate change and global warming.
Research Group
The sample group consisted of 53 gifted children studying in BILSEM in 2018-2019 academic year. Necessary
permissions were obtained from students and their families within the scope of the research. BILSEM was also con-
tacted to conduct the research. The maximum variation sampling method was preferred to give the opportunity
to access rich data from purposeful sampling methods in order to determine the sample group (Yıldırım & Şimşek,
2013). In this context, the research was conducted with BILSEM students studying in different programs. Table 1
shows the demographic data of the students included in the sample group:
Table 1. Demographic information of the participants.
Personal characteristics f %
Female 25 47.16
Gender
Male 28 52.84
Support 20 37.73
BILSEM group BYF 21 39.62
ÖYG 12 22.65
6-10 age 24 45.28
Age
11-17 age 29 54.72
When Table 1 was examined, 47.16% of the participants were female and 52.84% were male. Likewise, the
students studying in the programs of support (37.73%), recognition of individual skills (BYF) (39.62%), and im-
provement of special talents (OYG) (22.65%) were included in the sample group in order to ensure the variation.
In addition, the ages of the participants ranged from 6 to 17 years. 24 of the participants were between 6-10 years
old and 29 of them were between 11-17 years old.
782
GLOBAL WARMING
ISSN 2538–7138 /Online/ (P. 780-792)
Data Collection Tool
In this research, multi-method technique was used in data collection process. The multi-method technique
is the use of more than one data collection methods while collecting information about an event, phenomenon
or problem (Clark & Statham, 2005). In addition, this technique includes drawing, writing and explaining steps for
the individuals to express themselves more easily (Darbyshire, MacDougall, & Schiller, 2005; Freeman & Mathison,
2009). In this research, the draw-write-tell technique was used to collect the data and three stages were followed.
In the first stage, the students were asked to make four drawings about the things the “climate change and
global warming” concept evoked in them. Figure 1 shows sample student drawings. In the second stage, they were
asked to write down a composition by considering the drawing which was the closest to their thoughts among those
made for “climate change and global warming” concept. They were ensured to state their reasons of choosing this
drawing, what “climate change and global warming” concept mean in themselves and their solution suggestions for
these global problems in the composition. In the third stage, focus group interviews were conducted with all students
participating in the research. The focus group interviews took a week and the participants were asked to read what
they wrote in the compositions and to explain what they emphasized about the climate change and global warming
and their reasons. While conducting the interviews which lasted for approximately 20 minutes, a quiet environment
was preferred and the interviews were then converted into a text and codes, categories and themes were determined.
Figure 1. Example student drawings.
Data Analysis

The summative content analysis was performed to analyze the data. Summative content analysis usually
aims to compare the counting of keywords or content and to interpret the content in this context. The summative
content analysis is based on expressing data that are similar to each other in the form of code, category and theme
that the reader can understand (Yıldırım & Şimşek, 2011; Gökçe, 2006). While analyzing data, the answers of each
participant were numbered first as P1, P2, P3… P53. Then, each drawing was examined in detail and transferred
into computer environment. 197 drawings drawn by the students for climate change and global warming concept
were determined and the drawings were grouped under certain categories (Table 2). The categories are presented
in detail in the results section. In the second and third steps of the research, codes, categories and themes for the
students’ responses were determined.
Studies have been conducted to control factors threatening the validity and reliability of the study (Yıldırım &
Şimşek, 2013). In this context, gifted children studying in different programs (Support, BYF, OYG) were determined
(variation). In preparing the interview form, expert opinion was consulted (internal validity). The interviews were
conducted in a quiet environment (internal reliability). Expert opinion was consulted when determining code,
category and theme in data analysis. While forming the results, direct citations were included. By ensuring the
consistency of the results and findings, properly discussion of the data was ensured (External reliability). In addi-
tion, the analyses were presented to the opinion of more than one researcher to provide internal reliability of the
research and the cases with consensus and disagreement were revealed (Miles & Huberman, 1994).
783
GLOBAL WARMING
(P. 780-792) ISSN 2538–7138 /Online/
Research Results
Results for the Students’ Drawings
In the first stage of the research, the drawings were used in order to determine the perceptions of the stu-
dents about climate change and global warming concept. Table 2 shows frequency and percentage values of the
students’ drawings.
Table 2. Frequency values of the students’ drawings.
Codes f f%
Changing of seasons 35 17.76

Melting of glaciers 24 12.18
Depletion of ozone layer 22 11.16
Greenhouse effect 12 6.09
Increase of air temperature 11 5.58
Greenhouse gases 10 5.07
Increase of factories 10 5.07
The use of perfume, deodorant 8 4.06
Increase of the number of vehicles 6 3.04
Air pollution 6 3.04
Cutting of trees 5 2.53
Decrease of water resources 5 2.53
Narrowing of habitats 5 2.53
Floods 5 2.53
Destruction of living creatures 4 2.03
Reduction of biodiversity 3 1.52
Destruction of lands 2 1.01
Disruption of food chain 2 1.01
Drought 2 1.01
Desertification 2 1.01
Snowfall 2 1.01
Disruption of the economy 2 1.01
Garbage 2 1.01
Exhaust gases 2 1.01
Destruction of vegetation 1 0.50
Storms 1 0.50
Decrease of snowfall 1 0.50
Population growth 1 0.50
Public transport 1 0.50
Destruction of trees 1 0.50
Carbon-dioxide gas 1 0.50
The use of electric vehicles 1 0.50
Sea pollution 1 0.50
Urbanization 1 0.50
784
GLOBAL WARMING
ISSN 2538–7138 /Online/ (P. 780-792)
When Table 2 was examined, the drawings of the students were mostly found in the “changing of seasons”
code (f=35). In the same way, it was also seen that the number of drawings under the codes “melting of glaciers”
(f=24) and “depletion of ozone layer” (f=22) was high. The drawings were categorized as causes, results, and solu-
tion suggestions of climate change and global warming (Table 34) and the drawings that did not have direct con-
nection with climate change and global warming were gathered under the category of insufficient associations.
Table 3. Theme, categories and codes for the drawings.
Theme Category Code f
Greenhouse effect 12
Greenhouse gases 10
Increase of factories 10
Increase of the number of vehicles 6
Air pollution 6
Cutting of trees 5
Causes
Garbage 2
Exhaust gases 2
Destruction of vegetation 1
Population growth 1
Destruction of trees 1
Carbon-dioxide gas 1
Changing of seasons 35
Melting of glaciers 24
Increase of air temperature 11
Decrease of water resources 5
Climate change Narrowing of habitats 5

Global warming Floods 5
Destruction of living creatures 4
Reduction of biodiversity 3
Results
Destruction of lands 2
Disruption of food chain 2
Drought 2
Desertification 2
Snowfall 2
Disruption of the economy 2
Storms 1
Decrease of snowfall 1
Public transport 1
Solution Proposals
The use of electric vehicles 1
Depletion of ozone layer 22
Use of perfume, deodorant 8
Insufficient drawings
Sea pollution 1
Urbanization 1
When Table 3 was examined, it was seen that the drawings of the students were mostly directed to greenhouse
effect (f=12) in the category of the causes of global warming. This was followed by the drawings about greenhouse
785
GLOBAL WARMING
(P. 780-792) ISSN 2538–7138 /Online/
gases (f=10) and increase of factories (f=10). Besides, while six drawings were determined for the increase of the
number of vehicles and air pollution, five drawings were determined stating that cutting of trees caused climate
change and global warming.
Figure 2. The drawing of P42, P33 and P39
Similarly, it was seen that the students’ drawings focused on the changing of seasons (f=35) concept in the
category of the results of climate change and global warming. This was followed by melting of glaciers (f=24) and
increase of air temperature (f=11). It was also seen that the number of drawings in the solution suggestions about
climate change and global warming was quite low. The students presented the suggestions with only their draw-
ings of public transport (f=1) and the use of electric vehicles (f=1).
Figure 3. The drawing of P38, P41 and P10.
Besides, it was seen that the students’ perception on the presence of a connection of the depletion of the ozone
layer with climate control and global warming was quite high (11.6% and they made drawings about the depletion
of ozone layer. There were drawings about the use of perfume and deodorant causing global warming (4.06%).
Figure 4. The drawing of P37, P30 and P11.
786
GLOBAL WARMING
ISSN 2538–7138 /Online/ (P. 780-792)
The Results Obtained for Focus Group Opinions and Written Statements
Written statements and the results from the focus group opinions were given together and the participants
were asked to explain what they emphasized in the texts by reading compositions they wrote about climate change
and global warming and to explain their reasons. Table 4 shows the obtained results.
Table 4. Written statements and focus group opinions.
Theme Category Code f
Depletion of ozone layer (20),

Factory wastes (4),
Causes
Greenhouse effect (3), Environmental pollution (3), 33
Air pollution (2),
Approach of sun to the world
Changing of seasons (6),

Climate change and Global Extinction of species (4),
warming Melting of glaciers (3),
Results 16
End of humanity,
Disruption of natural balance,
Increase of air temperature
Factories should be equipped with filters
Vehicles should be used less
Suggestions 4
Chemical substances should not be used
Environment should not be harmed
When written statements and the students’ opinions were examined, most of the students (37.23%) considered
the cause of climate change and global warming as depletion in the ozone layer. For example, P2 expressed his/her
opinion on this issue as “Ozone layer is thinning. Global warming and climate change are connected with each other.”;
P10 expressed his/her opinion as “The picture I draw is related to the depletion of the ozone layer. It causes to global
warming.”. Similarly, P16 expressed his/her opinions as “Dirty gas exhaled by factory chimneys causes the depletion
of ozone layer to grow by polluting the air. Therefore, global warming occurs…”; on the other hand, P30 express his/
her opinion as “The deodorants people use and car exhausts reduce the ozone layer, cause ozone layer depletion, and
start the global warming. Therefore, harmful rays of the sun pass through the ozone layer.” and shows the depletion of
the ozone layer as the cause of climate change and global warming. The example student opinions showing the
depletion of the ozone layer as the cause of climate change and global warming are as follows:
The hole in the ozone layer is expanding. Thus, global warming is occurring (P35),
Global warming, exhaust gases from the cars, the depletion of the ozone layer, and gases from the factories causes the
depletion of the ozone layer and therefore the harmful sun rays come onto the world and global warming occurs (P42),
The exhaust gases emitted from cars raise into the space, cause depletion of the ozone layer, and lead to global warming.
In order to stop global warming, cars should be used less and we should prefer to walk (P29),
Global warming refers to the depletion of the ozone layer for me (P53).
When the participants’ opinions were examined, it was seen that they expressed that factory wastes also
caused climate change and global warming. P26 expressed his/her opinion about this situation as “not attaching
filter on the factory chimneys is one of the most effective factors triggering global warming. The smoke emitted from
factory chimneys causes the global warming and climate change. Disposing the wastes of materials used in the factories
is the cause of global warming.”; whereas, P39 stated that “To me, disposing the factory wastes to the environment
causes global warming.”. Similarly, P41 stated that “We pollute the air without realizing. This reduces our life process.
Another reason for air pollution is the increase of factories. This also affects the climate change.” and P52 expressed
his/her thoughts as “Since the number of factories is high, sun rays are trapped between the atmosphere and the world.
Solution is to attach filters on factory chimneys.” and showed the factory wastes as the cause of climate change and
global warming.
787
GLOBAL WARMING
(P. 780-792) ISSN 2538–7138 /Online/
When the students’ opinions were examined, it was seen that only three students emphasized the greenhouse
effect which is the most important reason for climate change and global warming. About this subject, P8 expressed
that “… Because greenhouse effect causes global warming the most.” and P32 expressed his/her thoughts as “the first
thing seen about global warming is greenhouse. Here, the sun rays penetrate into the greenhouse. But a bit of radiation
comes out. The remaining sun rays come down again. They raise to get out again from the glass. During this process, the
greenhouse starts to heat up. This can be given as an example for global warming.”. Likewise, P33 who highlighted the
greenhouse effect stated his/her opinion as “The one which is closest to my thought is the greenhouse effect. Because
our atmosphere is getting dirty. It cannot get rid of carbon-dioxide. Therefore, the earth is getting polluted and climate
change occurs with global warming.”.
While P6 who stated that environmental pollution causes climate change and global warming expressed
his/her thoughts as “Factories and cars pollute the environment. The number of cars increases. Number of factories
increases. The environment becomes more polluted. Global warming increases.”, P15 stated that “I think people should
not harm the environment and should not use chemicals. Otherwise, our world will be in an irreversible form due to
global warming and climate change arising from environmental pollution.”. Similarly, P50 who emphasized the envi-
ronmental pollution as the cause of climate change and global warming stated his/her opinion as “Glaciers melt in
global warming. A major reason for this is the environmental pollution and wasteful behavior.”.
P22 who stated that air pollution causes climate change and global warming expressed his/her opinion as
“Dirty smoke causes climate change and global warming.” While P48 stated as “In my opinion, it is the air pollution.
For this, we should use renewable energy sources instead of coal”.
P13 who stated that the approach of Sun to the Earth causes climate change and global warming expressed
his/her opinion as “Global warming occurs as the sun approaches to earth.”
It was seen that the participants focused on changing of seasons, extinction of species and melting of glaciers
events for the results of climate change and global warming (Table 3). While P4 who emphasized that the seasons
changed stated his/her opinion as “I think about hot winter and cold summer in climate change; melting of glaciers
in global warming.”, P14 stated his/her opinion as “In the picture I drew, a child is playing with snow in summer while
the other child is sunbathing. I mean the seasons changed their places. If this continues, the climate will change and
living creatures may die.”. The example student opinions expressing the results of change of season are as follows:
It is not snowing as it used to be. This is because of the climate change. But, the rainfall has increased thanks to the measures
taken in recent years. But the glaciers are still in danger and necessary precautions for this are not taken (P36),
What I understand from the picture I drew is the snowfall in the spring and extreme climate changes (P40),
In my opinion, climate change is the changing of seasons. I mean, it is living winter in summer and summer in winter. The
average temperature of earth increases as a result of global warming (P46),
While P9 expressing that the species extinct as a result of climate change and global warming stated his/her
opinions as “According to a video that I have watched, we may lose some species.” P5 stated as “Plants have begun to
disappear due to climate change.”. Similarly, P23 expressing that the glaciers are melting expressed his/her thoughts
as “ In the picture I drew, I drew melting of glaciers and thus the destruction of the home of polar bears. Poles are the
places mostly affected by global warming.”; whereas, P49 stated that “Glaciers melt if global warming occurs. The world
gets quite hot. Storms break out frequently. People get hurt a lot.”
While P1 who expressed that climate change and global warming can cause the end of the humanity stated
his/her opinions as “Global warming can bring an end to mankind. Climate change may have effects that affect our way
of life. Therefore, I believe that precautions should be taken.”, P19 who stated that the natural balance is disrupted as a
result of these events expressed his/her opinions as “The natural balance is disrupted with global warming. This leads
to climate change. While there used to be cleaner air and many more living creatures, now their number is very small.”.
Besides, P47 who stated that the air temperature has increased expressed his/her opinions as “the air temperature
is increasing in the picture I drew. This is the result of global warming. In order to prevent air temperature change as a
result of global warming, necessary precautions should be taken”.
788
GLOBAL WARMING
ISSN 2538–7138 /Online/ (P. 780-792)
Discussion
In this research, it was aimed to examine the perceptions of gifted children studying in BILSEM about climate
change and global warming by using the draw-write-tell technique. In this context, it was determined that 53 BIL-
SEM students drew 197 drawings about climate change and global warming (Table 2). This indicated that the gifted
children had different and rich perception levels (Ataman, 2004). When the drawings about climate change and
global warming were examined, it was observed that the number of drawings concerning the changing of seasons
and melting of glaciers were the highest and the number of drawings about the depletion of ozone layer causing
climate change and global warming was also high (Table 2). The drawings emphasizing the relationship between
the depletion of ozone layer and global warming pointed out that more sunlight will reach to the surface of the
earth as a result of depletion of the ozone layer and thus cause global warming. Likewise, the students stated in
their written statements and in the focus group interviews that depletion of the ozone layer causes climate change
and global warming (Table 3). The emergence of such a result indicated that the students had misconceptions.
This result is supported by numerous studies (Aksan & Çelikler, 2013; Arsal, 2010; Aydın, 2014; Boyes & Stanisstreet,
1997; Demirbaş & Pektaş, 2009; Eroğlu & Aydoğdu, 2016; Erdoğan & Özsevgeç, 2012; Oluk & Oluk, 2007; Pekel, Kaya,
& Demir, 2007; Selvi & Yıldız, 2009; Summers, Kruger, Childs, & Mant, 2001). The common proposal emphasized in
these studies is to determine and remove lack of knowledge and misconceptions of students regarding climate
change and global warming. Thus, it is important to determine and remove misconceptions of gifted students
who constitute the most important labor force of communities and have a significant potential for generating
solutions to current world problems regarding climate change and global warming, which comprise one of to-
day’s most important problems. Because it is stated that gifted students who like solving current world problems
by questioning them would take on significant tasks to generate solutions to global problems in the near future
(Schreglmann, 2016). In this context, it is primarily required to determine the underlying reasons of lack of knowl-
edge and misconceptions in students. This is thought to be caused especially by the effect of misconceptions of
science teachers regarding climate change and global warming. For example, Eroğlu and Aydoğdu (2016) stated
that science teacher candidates had incomplete and incorrect information on some subjects although they had
knowledge level higher than the average about global warming. Therefore, it is important first to determine and
eliminate the misconceptions of the teachers and teacher candidates about climate change and global warming
which are the most important problems of our era (Aksan & Çelikler, 2013; Aydın, 2017). In addition, it is stated that
news in the media that may lead to misconceptions in students regarding climate change and global warming
pose a problem (Biçer & Vaizoğlu, 2015) and it is important especially for advisors and families to help students
access true and reliable sources on this matter. Additionally, it has been stated that climate change, global warm-
ing and ozone layer are evaluated together by students due to teaching the aforementioned subjects together
(Pekel & Kırık, 2016). In this context, when examining the Ministry of National Education (MNE) 8th grade science
schoolbook; it is seen that the subjects of depletion of ozone layer and global warming are taught together un-
der the chapter of “Matter Cycles”. In addition, when examining the content of the chapter, it is determined that
there are statements such as, “Technology makes life easier. However, some products that are brought in life with
technology may cause depletion of ozone layer and deterioration of the greenhouse effect”. It is thought that this
would cause the development of misconceptions in students by evaluating depletion of ozone layer, greenhouse
effect and consequently climate changes together. In this respect, it is considered important to review and update
schoolbooks that are prepared according to curricula.
When the drawings of the students were categorized, it was observed that the solution suggestions about
climate change and global warming were very low (Table 3). In this context, it was determined that the drawings
were intended to give importance to electric vehicles and public transport. When the written statements of the
students and the data of the focus group interviews were examined, it was found that they could offer very few
suggestions such as factories should be equipped with filters, vehicles should be used less, chemical substances
should not be used, and environment should not be damaged. This is seen as a major deficiency. Taking necessary
precautions for climate change and global warming and raising awareness of everyone about precautions are
important (Yıldızoğlu, 2018). Besides, it was determined that the students made more drawings in terms of the
causes and results of climate change and global warming. It was an important result that students emphasized
mostly greenhouse effect and greenhouse gases in the formation of these problems in their drawings. This result
is supported by some studies (Aydın, 2014; Eroğlu & Aydoğdu, 2016; Kılınç, Stanisstreet, & Boyes, 2008; Koulaidis
& Christidou, 1999). Besides, not mentioning water vapor which has more greenhouse effect approximately three
789
GLOBAL WARMING
(P. 780-792) ISSN 2538–7138 /Online/
thousand times than carbon-dioxide gas (CO2) causing global warming and mentioning a little about CO2 are strik-
ing as a shortcoming. This confirmed that the students had incomplete knowledge about the causes of climate
change and global warming. Likewise, changing of seasons, melting of glaciers, increase of air temperature, decrease
of water resources, narrowing of habitats, and floods were involved in the drawings of students as the results of
climate change and global warming. This showed that students generally had knowledge about the results of
climate change and global warming. This result is similar to the study by Aydın (2017).
Conclusions
In the light of all these evaluations, it was found that the gifted children had misconceptions about climate
change and global warming constituting one of the most important environmental problems of our era and they
had lack of knowledge especially about solution suggestions for these problems. The research results suggested
that while the perception levels of gifted students about the causes and possible results of climate change and
global warming were high, their perception level about the solution of these problems was quite low. Therefore,
it is important to examinate the underlying causes of information deficiencies and misconceptions about global
warming and climate change, which is one of the biggest problems of today’s gifted students who have the po-
tential to produce solutions to world problems. In this context, it is considered important to update the curricula
for gifted students and especially the gifted students in the project program should be supported to produce
projects on socio-scientific issues. Likewise, the results included that the draw-write-tell technique was effective
in determining the perception, misconceptions and knowledge levels of the students about climate change and
global warming. Asking students to draw, write down their opinions about the subject and then express this matter
verbally in order to determine their perception about climate change and global warming provides an opportunity
to examine profoundly the students’ perspectives about the subject.
Recommendations
Necessary measures should be taken to raise awareness of students and to eliminate their misconceptions
about climate change and global warming among the most important problems of today. In this context, explaining
climate change and global warming to students through methods such as project-based learning, argumentation,
cooperative learning and so on is important.
In order for students to reach scientific and accurate information about global warming and greenhouse effect,
these subjects should be included more in the curriculum. In order to eliminate the misconceptions of students
about the reasons, results and prevention of environmental problems such as global warming and climate change,
practices about real life-based teaching of course contents can be developed.
It is recommended to carry out project studies with students in order to increase awareness about climate
change and global warming and to reduce our ecological footprint.
Qualitative research method was used in research but quantitative research method designs can also be used
for larger samples.
References
Aksan, Z., & Çelikler, D. (2013). İlköğretim öğretmen adaylarının küresel ısınma konusundaki görüşleri [Pre-service elementary teach-
ers’ opinions about global warming]. Eskişehir Osmangazi University, Journal of Social Sciences, 14(1), 49-67.
Aksay, S.C., Ketenoğlu, O., & Kurt, L. (2005). Küresel ısınma ve iklim değişikliği [Global warming and climatic change]. Selçuk University,
Journal of Science Faculty, 25, 29-41.
Arsal, Z. (2010). İlköğretim öğretmen adaylarının sera etkisi ile ilgili kavram yanılgılarının belirlenmesi [The greenhouse effect mis-
conceptions of the elementary school teacher candidates]. Elementary Education Online, 9(1), 229-240.
Ataman, A. (2004). Üstün zekâlı ve üstün özel yetenekli çocuklar [Gifted and talented children]. Şirin M. R., Kulaksızoğlu A., Bilgili
A.E. (Ed.). Gifted children selected articles book (pp.155-168). İstanbul: Çocuk Vakfı.
Atasoy, S. (2018). Öğretmen adaylarının yaşam alanlarına göre yerel sosyobilimsel konularla ilgili informal muhakemeleri [Student
teachers’ ınformal reasoning of local socioscientific ıssues according to the living places]. FEAD, 6(1), 60-72.
Aydın, F. (2014). Ortaöğretim öğrencilerinin küresel ısınma konusundaki bilgi düzeylerinin belirlenmesi [Determination of knowledge
level of high school students on global warming subject]. Turkish Journal of Education, 3(4), 1527.
Aydın, F. (2017). Üniversite öğrencilerinin küresel ısınma hakkındaki bilgi düzeylerinin belirlenmesi [Determınıng the level of knowl-
edge of unıversıty students on global warmıng]. Journal of Social Sciences and Humanities, 1(1), 118-132.
790
GLOBAL WARMING
ISSN 2538–7138 /Online/ (P. 780-792)
Bahar, M. (2000). Üniversite öğrencilerinin çevre eğitimi konularındaki ön bilgi düzeyi, kavram yanılgıları [University students’ pre-
conceptions about topics of environmental education]. V. International Symposium on Ecology and Environmental Problems,
Poster Work, TUBİTAK, Ankara.
Bakırcı, H., Artun, H., Şahin, S., & Sağdıç, M. (2018). Ortak bilgi yapılandırma modeline dayalı fen öğretimi aracılığıyla yedinci sınıf
öğrencilerinin sosyobilimsel konular hakkındaki görüşlerinin incelenmesi [Investigation of opinions of seventh grade students
about socio-scientific ıssues by means of science teaching based on common knowledge construction model]. Journal of
Qualitative Research in Education, 6(2), 207-237.
Biçer, B. K., & Vaizoğlu S.A. (2015). Determination of nursing students’ knowledge and awareness about global warming / climate
change. Hacettepe University Journal of Nursing, 2(2), 30-43.
Boyes, E., & Stanisstreet, M. (1997). Children’s models of two major global environmental ıssues (ozone layer and greenhouse effect).
Research in Science and Technological Education, 15(1), 19-28.
Çepel, N. (2008). Ekolojik sorunlar ve çözümleri [Ecological problems and solutions]. Ankara: TUBİTAK Populer Science Books.
Christensen, L. B., Johnson, R. B., & Turner, L. A. (2015). Research methods, design, and analysis. Ankara: Anı yayıncılık.
Clark, A., & Statham, J. (2005). Listening to young children: Experts in their own lives. Adoption and Fostering, 29(1), 45-56.
Clark, B. (2015). Growing as gifted, developing children’s potential at home and school. Ankara: Nobel Yayıncılık.
Darbyshire, P., MacDougall, C., & Schiller, W. (2005). Multiple methods in qualitative research with children: More insight or just
more?, Qualitative Research, 5(4), 417-436.
Demirbaş, M., & Pektaş, H.M. (2009). İlköğretim öğrencilerinin çevre sorunu ile ilişkili temel kavramları gerçekleştirme düzeyleri [El-
ementary students’ levels of realization of basic concepts related with environment problem]. Necatibey Faculty of Education
Electronic Journal of Science and Mathematics Education, 3(2), 195-211.
Dinç, E., & Üztemur, S. (2017). Investigating student teachers’conceptions of social studies through the multi-dimensional structure
of the epistemological beliefs. Educational Sciences: Theory & Practice, 17(6), 2093-2142.
Doğan S., & Tüzer M. (2011). Küresel iklim değişikliği ve potansiyel etkileri [Global climate change and ıts potential effects]. Cum-
huriyet University Journal of Economics and Administrative Sciences, 12(1), 21-34.
Erdoğan, A., & Özsevgeç, L.C. (2012). Kavram karikatürlerinin öğrencilerin kavram yanılgılarının giderilmesi üzerindeki etkisi: sera
etkisi ve küresel ısınma örneği [The effects of concept cartoons on eliminating students’ misconceptions: greenhouse effect
and global warming]. Turkish Journal of Education, 1(2), 1-13.
Eroğlu, B. (2009). Fen bilgisi öğretmen adaylarının küresel ısınma hakkındaki bilgi düzeylerinin belirlenmesi [Determination of pre-service
science teachers’ knowledge level about global warming]. MS Thesis, Gazi University, Turkey.
Eroğlu, B., & Aydoğdu, M. (2016). Fen bilgisi öğretmen adaylarının küresel ısınma hakkındaki bilgi düzeylerinin belirlenmesi [De-
termination of pre-service science teachers’ knowledge level about global warming]. Uludağ University Journal of Faculty of
Education, 29 (2), 345-374.
Ersoy, A.F., & Türkkan, B. (2010). İlköğretim öğrencilerinin çizdikleri karikatürlere yansıttıkları sosyal ve çevresel sorunların incelen-
mesi [Analyzing social and environmental ıssues elementary school students reflect in their cartoons]. Education and Science,
35(156), 96-109.
Eş, H., Mercan, S., & Ayas, C. (2016). Türkiye için yeni bir sosyobilimsel tartışma: Nükleer ile yaşam [A new socio-scientific issue for
Turkey: Life with nuclear]. Turkish Journal of Education, 5(2), 47-59.
Freeman, M., & Mathison, S. (2009). Researching children’s experience. New York: Guilford Publications.
Genç, M., & Genç, T. (2017). Türkiye’de sosyo-bilimsel konular üzerine yapılmış araştırmaların içerik analizi [The content analysis of
the researches about socio-scientific ıssues in Turkey]. Kafkas Journal of Educational Research, 4(2), 19-26.
Gökçe, O. (2006). İçerik analizi kuramsal ve pratik bilgiler [Content analysis theoretical and practical information]. Ankara: Siyasal
Yayınları.
Kadıoğlu, M. (2001). Bildiğimiz havaların sonu küresel iklim değişimi ve Türkiye [The last we know the weather, global climate change
and Turkey]. Güncel Yayıncılık. İstanbul.
Kılınç, A., Stanisstreet, M., & Boyes, E. (2008). Turkish students’ ıdeas about global warming. International Journal of Environmental
& Science Education, 3(2), 89-98.
Kılınç, A., Demiral, U., & Kartal, T. (2017). Resistance to dialogic discourse in SSI teaching: The effects of an argumentation‐based
workshop, teaching practicum, and induction on a preservice science teacher. Journal of Research in Science Teaching, 54(6),
764-789.
Klop, T., & Severiens, S. (2007). An exploration of attitudes towards modern biotechnology: A study among dutch secondary school
students. International Journal of Science Education, 29(5), 663-679.
Kolstø, S.D. (2001). Scientific literacy for citizenship: Tools for dealing with the science dimension of controversial socioscientific
issues. Science Education, 85(3), 291-310.
Koulaidis, V., & Christidou, V. (1999). Models of students thinking concerning the greenhouse effect and teaching implications.
Science Education, 83, 559-576.
MEB. (2018). Fen bilimleri dersi öğretim programı (İlkokul ve Ortaokul 3, 4, 5, 6, 7 ve 8. Sınıflar) [Science course curriculum (Primary and
Middle School 3, 4, 5, 6, 7 and 8 Grades)]. Ankara, Turkey.
MEB. (2017). 2017-2018 bilim ve sanat merkezleri öğrenci tanılama kılavuzu [2017-2018 science and art centers student diagnostic
guide]. Retrieved from https://orgm.meb.gov.tr.
Miles, M. B., & Huberman, M. A. (1994). An expanded sourcebook qualitative data analysis. London: Sage Publication.
Nuhoğlu, H. (2014). Güncel sosyobilimsel konulara yönelik sistem dinamiği temelli kurulan öğrenci modellerinin değerlendirilmesi
[Evaluation of student models on current socioscientific topics based on system dynamics]. Educational Sciences: Theory &
Practice, 14(5), 1957-1975.
791
GLOBAL WARMING
(P. 780-792) ISSN 2538–7138 /Online/
Oluk, E. A., & Oluk, S. (2007). Yüksek öğretim öğrencilerinin sera etkisi, küresel ısınma ve iklim değişikliği algılarının analizi [Analysis
of under graduated students’ perceptions concerning green house effect global warming and climate change]. Dokuz Eylül
University Buca Journal of the Faculty of Education, 22, 45-53.
Özsoy, S., & Ahi, B. (2014). İlkokul öğrencilerinin geleceğe yönelik çevre algılarının çizdikleri resimler aracılığı ile belirlenmesi [El-
ementary school students’ perceptions of the future environment through artwork]. Educational Sciences: Theory & Practice,
14(4), 1557-1582.
Öztürk, N., Eş, H., & Turgut, H. (2017). How gifted students reach decisions in socio-scientific issues? warrants, information sources
and role of media. International Online Journal of Educational Sciences, 9(4),1111-1124.
Pekel, F., O., Kaya, E., & Demir, Y. (2007). Farklı lise öğrencilerinin ozon tabakasına ilişkin düşüncelerinin karşılaştırılması [A comparatıve
study of dıfferent hıgh school students’ perceptıons about ozone layer depletıon]. Kastamonu Education Journal, 15(1), 169-174.
Pekel, F., & Kırık, Ö.T. (2016). Middle school students’ cognitive structures about global warming and ozone layer depletion. Journal
of Theory and Practice in Education 12(1), 308-357.
Pınar, E., & Yakışan, M. (2016). İlkokul öğrencilerinin çevre kavramları ile ilgili çizimlerinin analizi [Analyze of the drawings on envi-
ronmental concepts of the primary school students]. Trakya University Journal of Education, 8(1), 97-113.
Ratcliffe, M., & Grace, M. (2003). Science education for citizenship: Teaching socio-scientific ıssues. Maidenhead: Open University Press.
Sadler, T. (2004). Informal reasoning regarding SSI: A critical review of research. Journal of Research in Science Teaching, 41(5), 513-536.
Schreglmann, S. (2016). Content analysis of higher education thesis made about gifted students in Turkey (2010-2015). Journal of
Gifted Education Research, 4(1), 14-26.
Selvi, M., & Yıldız, K. (2009). Biyoloji öğretmeni adaylarının sera etkisi ile ilgili algılamaları [Pre-servıce bıology teachers’ perceptıons
of the greenhouse effect]. The Journal of Turkish Educational Sciences, 7(4), 813-852.
Summers, M., Kruger, C., Childs, A., & Mant, J. (2001). Understanding the science of environmental ıssues: development of a subject
knowledge guide for primary teacher education. International Journal of Science Education, 23(1), 33-53.
Sürmeli, H., Duru, N. & Duru, R. (2017). Nükleer enerji ve nükleer santraller konusuna yönelik öğretmen tutumlarının farklı değişkenler
açısından incelenmesi [Investigating teachers’ attitudes towards nuclear energy and nuclear power plants ın terms of different
variables]. Necatibey Faculty of Education Electronic Journal of Science and Mathematics Education 11(1), 293-319.
Tanyaş, B. (2014). Nitel araştırma yöntemlerine giriş: Genel ilkeler ve psikolojideki uygulamaları [An ıntroduction to qualitative re-
search methods: General principles and applications in psychology]. Research and Method Discussions in Psychology, Special
Issue, 5, 25-38.
Topaloğlu, M.Y., & Kıyıcı, F.B. (2017). Ortaokul öğrencilerin hidroelektrik santrali hakkındaki görüşleri [Middle school students’ opinions
about hydroelectric power plants]. Ahi Evran University Kırşehir Journal of Education, (KEFAD),18, 159-179.
Türkoğlu, A., & Öztürk, N. (2019). Sosyo-bilimsel konulara ilişkin fen bilgisi öğretmen adaylarının zihinsel modelleri [Pre-service sci-
ence teachers’ mental models of socio-scientific ıssues]. Başkent Unıversıty Journal of Educatıon, 6(1),127-137.
Üztemur, S., & Dinç, E. (2018). Ortaokul öğrencilerinin epistemolojik inançlarının keşfedilmesinde öğrenci merkezli bir yaklaşım: Çiz-
Yaz-Anlat Tekniği [A student-centered approach to explore middle school students’ epistemological beliefs: Draw-Write-Tell
Technique]. Journal of History Culture and Art Research, 7(3), 566-592. doi: 10.7596/taksad.v7i3.1579.
World Economic Forum (2018). The Global Risks Report 2018. Retrieved from https:// www.weforum.org/reports/the-global-risks-
report-2018.
Yalçın, F. (2010). A study on determining the misconceptions and level of knowledge in global warming and greenhouse effect of the
primary school students. MS Thesis, Gazi University Institute of Educational Sciences Ankara, Turkey.
Yıldırım, A., & Şimşek, H. (2011). Sosyal bilimlerde nitel araştırma yöntemleri [Qualitative research methods in the social sciences].
Ankara: Seçkin Yayınları.
Yıldırım, A., & Şimşek, H. (2013). Sosyal bilimlerde nitel araştırma yöntemleri [Qualitative research methods in the social sciences].
Ankara: Seçkin Yayıncılık.
Yıldızoğlu, E. (2018). İklim değişikliği: En kritik 20 yıl [Climate change: The 20 most critical years]. Retrieved from https://www.bbc.
com/turkce/haberler-dunya-45938984.
Zeidler, D. L., & Nichols, B. H. (2009). Socioscientific issues: Theory and practice. Journal of Elementary Science Education, 21(2), 49-58.
Fatma Mutlu PhD, Associate Professor, Department of Science Education,

(Corresponding author) Faculty of Education, İnonu University, Malatya, Turkey.
E-mail: fatma.mutlu@inonu.edu.tr
Oğuzhan Nacaroğlu PhD Student, Department of Science Education, Erciyes University,
Kayseri, Turkey.
E-mail: onacaroglu44@gmail.com
792
SCIENCE-EDU-COMMUNICATION:
TRENDS REVEAL IN 20 YEARS
OF SCIENCE COMMUNICATION ISSN 1648-3898 /Print/
RESEARCH ISSN 2538-7138 /Online/
Leon Yufeng Wu, Abstract. By investigating scholarly output

Nathan M. Truong, in science communication from 1997 to
Hsin-Yen Lu, 2018, this research sought evidence that
Yuen-Hsien Tseng, science education has been increasingly
Chun-Yen Chang focusing on communication methods to
reach the public. Through an automatic
scientometric method, this study analyzed
1300 articles published in two leading
journals in the field of science communica-
Introduction
tion. As a result, seven trends were revealed
“Science is an integrative force and Science Education can help to under- and categorized into three themes: Public
stand the changing world” (Lakhvich, 2010, p. 164). However, understanding engagement with science (PES); Media
the world has become ever more challenging for citizens than it has been and science (MS); and Issues in science (IS).
in the past, because people have been inundated with incomplete and er-
Furthermore, PES and MS scholarly output
roneous information from the media (L. Y. Wu et al., 2015). If the media at a
certain level represents the “skin of culture” (Kerckhove, 1995), and science were found increased significantly. The
content in the media has not been communicated well to the public, then the findings confirmed the goal of this research.
goal of science education to help promote understanding of the changing However, it then suggested a research area
world is hindered. A gap was found between the role for science education
of bridging science education and science
in communicating science to the public (Lewenstein, 2015), and the goal of
public understanding of science (Bauer, Allum, & Miller, 2007; Martin, 2017). communication that is currently less ex-
Science education as a research field, is a field in which research trends plored. Given increased focus towards PES
have been both qualitatively and quantitatively analyzed, thus providing the and MS, these fields are primed for further
research community with nuanced data of the field’s goals and directions
collaboration to more engage the public in
(Cavas, Cavas, Ozdem, Rannikmae, & Ertepinar, 2012; T.-C. Lin, Lin, & Tsai,
2014; Tsai & Wen, 2005). Teaching and learning in science, along with top- science learning.
ics of interest and trends in science education have been documented and Keywords: CATAR, scientometric analysis,
conveyed trends over the past decades towards topics of context of student science communication, science education,
learning and conceptual change (Chang, Chang, & Tseng, 2010; Lee, Wu, &
science-edu-communication.
Tsai, 2009; Tsai & Wen, 2005). One such trend was the increasing international
collaborative science education research, as indicated by the increased output
from non-English-speaking countries over the last decade (Tsai & Wen, 2005).
Science communication, on the other hand, has engaged with the con-
cepts of scientific literacy or understanding, public understanding of science,
and science and its relationship with society, shifting from a deficit oriented Leon Yufeng Wu
Chung Yuan Christian University, Taiwan
model to a more interactive process of engagement, since the 1960’s (Bauer Nathan M. Truong, Hsin-Yen Lu,
et al., 2007; Bucchi & Trench, 2008). Diverse in its research fields, science com- Yuen-Hsien Tseng, Chun-Yen Chang
munication encompasses media, communication techniques, and the process National Taiwan Normal University, Taiwan
of relaying science to groups that interpret new information and science in
793
(P. 793-805) ISSN 2538–7138 /Online/
a variety of ways. It has grown to encompass research that overlaps with fields such as nutrition and environment,
but continues to lack robust communication between researchers within the various fields concerning science
communication (Logan, 2001).
While collaboration between the fields of science education and science communication is largely absent and
their research mainly distinct in development, the two fields are largely aligned in their objectives and comple-
mentary in their methods (Baram-Tsabari & Osborne, 2015). Both fields wish to develop increased broad scientific
literacy and competency in critical thinking and analysis so that individuals can publicly engage in the discussion
of science and its effects on society (Feinstein, 2015). Science education and communication are alike in substance
but differ in style and setting. Science communication focuses on broad public education while science educa-
tion communicates science on a more intimate level in classrooms and through teachers; both fields are distinct
in their research practices and collaborations, even though formalized disciplinary boundaries do not prevent the
fields from crossing (Davis & Russ, 2015; Lewenstein, 2015). Despite sharing so much of the same-spirited ideas,
both fields actually do not contribute to each other’s research in a continuously collaborative manner as their
overlapping goals would suggest. Thus, there is a risk in conducting research that has already been addressed in
the respective fields of science communication or education (Baram-Tsabari & Osborne, 2015; Lewenstein, 2015).
In seeking to educate the public through science communication, there has been a growing shift from a
deficit-oriented model towards one that stresses scientific discussion and conversation with the public (Buc-
chi, 2008; McNeil, 2013). The deficit-oriented model broadly views the scientific community as an expert entity
that educates the public through gradual dispersion of information through media and communication outlets.
Though experts are the main communicators, the act of communication with non-experts will result in changes to
the original science content and message (Claessens, 2008; Hilgartner, 1990; McNeil, 2013; Sturgis & Allum, 2004;
Suldovsky, 2016). Science communication, in tension with the deficit-oriented model, has in recent years shifted
towards a model that regards the public as an interactive partner in discussion in science-related issues (Baram-
Tsabari & Osborne, 2015; Bucchi, 2013). This shift, in many ways, is congruent with science education’s emphasis
on activity-based learning and active discussion in science classrooms, a major theme that will guide research in
communication and education for years to come (Baram-Tsabari & Osborne, 2015). The emphasis on a more col-
laborative science communication model emphasizing conversation between scientists and the public promises
to expand the diversity of stakeholders involved in developing science communication. While this shift in science
communication model focus is present, the deficit model will still play a significant role because the public will still
need to rely on information derived from scientific experts and interactive modes of communication also serve to
further enrich the information science communicators convey (Hilgartner, 1990).
Since science communication not only involves accurately communicating science concepts with precision,
but also appealing to peoples’ values and the “entertainment” component of peoples’ reception, the media is a
distinctive component of science communication and often times the bridge through which the scientific commu-
nity communicates (Friedman, Dunwoody, & Rogers, 1999). Media can have a negative role in terms of educating
the public as studies have shown that the manner in which media communicates science affects the cognitive
acquisition of scientific concepts (Berinstein, 2006). The ongoing conversation about the purpose of science com-
munication might tap into the goals of science in the public sphere and beg the question as to how effectively
and in what modality science literacy may be achieved (Bauer et al., 2007).
While the direction of science communication research is unclear in literature and form, the trend studies
carried out in science education give form to how this closely related field has developed over time (Chang et al.,
2010; T.-C. Lin et al., 2014; Tsai & Wen, 2005). These studies have been carried out in a systematic sense categorizing
research under predetermined categories, as well as through automated computer algorithms that cluster research
through citation-linked coupling. The findings of these studies have informed researchers of the trends in science
education, revealing patterns that otherwise would not be available unless through large manual analysis. These
broad analyses also allow scholars new to a field to access the large trends of a given field and target research in
conjunction with a significant topic (Cavas et al., 2012).
Purpose and Research Questions
By investigating scholarly output in science communication from 1997 to 2018, this research sought evidence
that science education has been increasingly focusing on communication methods to reach the public. Instead of
looking at research trends in science education, this research proposed a trend analysis in science communication
794
ISSN 1648–3898 /Print/ SCIENCE-EDU-COMMUNICATION: TRENDS REVEAL IN 20 YEARS OF SCIENCE
ISSN 2538–7138 /Online/ (P. 793-805)
over the previous 21 years, with the goal of revealing prominent trends in science communication to allow coupling
of these findings with other research fields, particularly science education. This analysis sought to elucidate the
emerging trend in which the goals of science education and communication research were shared, offering fresh
insights into the shifts of science communication as well as future areas of collaboration between two fields that
historically have not interacted but could stand to benefit in doing so (Baram-Tsabari & Osborne, 2015; Lewenstein,
2015; Ogawa, 2011; L. Y. Wu et al., 2015).
The research’s aim was to provide and clarify areas of potential further research in conjunction with fields
related to science education and science communication and provide a foundation for future research areas and
trends. Research questions to be answered were as follows:
1. What are the major research trends in the field of science communication?
2. How does science education play a role in the research trends of science communication?
Analysis Tool Selection
Traditionally, trends analysis studies have sought to take large amounts of research papers and sort them
into pre-determined categories based on arbitrary or previously utilized standards (Lee et al., 2009; Tsai & Wen,
2005). Since this research analyzed papers from a 21-year period, it was determined an automatic content analysis
method would serve as the most efficient and detailed method to evaluate the large quantity of research articles
while minimizing human error. Science education and tourism studies have made use of Content Analysis Toolkit
for Academic Research (CATAR) developed by Tseng (2018) to conduct bibliographic analysis.
CATAR’s use of multi-stage clustering and generic title labeling proves most useful to grouping papers into
broad categories and facilitating labeling (Tseng, 2010; Tseng & Tsay, 2013). This method also serves to minimize
influence of researchers’ personal interests and professional knowledge in reviewing literature. It has been previ-
ously used to obtain an overview of reoccurring research topics and track their trends over time and their relation-
ships to countries and institutions (Chang et al., 2010; Yuan, Gretzel, & Tseng, 2015). This information can then be
used to suggest further areas of study as well as give quantitative evidence in describing a field and its directions,
information, especially relevant to those unfamiliar with a field such as new students and researchers.
CATAR makes use of bibliographic coupling as a fundamental tool in grouping papers, and thus using Social
Sciences Citation Index journals was fitting for CATAR; papers with shared citations are clustered together and can
be assumed to have similar focuses. CATAR makes use of this idea through grouping papers with similar citations
in stages to provide specific categories for research analysis as well as more broad categories. CATAR is currently
limited to using World of Science (WoS) article data and data converted into the WoS formats. CATAR, while use-
ful to compile large amounts of data, also requires the user to select the quantity of coupling iterations used on
selected data, which is four in this research. More detailed explanations of CATAR can be found in the research of
Tseng and Tsay (2013).
The following steps provide a summary of the six main steps of CATAR to assist in understanding the results
that follow.
Step 1: Text Segmentation
Each record from the Thomson’s Web of Science contains about 40 fields of information ranging from paper
title, authors, publication years and journal title, etc. This step chooses the relevant pieces of data for analysis. We
identify at least seven of them for use in analysis. Below are their abbreviations, definitions, and examples:
AU: Authors, e.g., Harambam, J Aupers, S.
TI: Publication title, e.g. Contesting epistemic authority: Conspiracy theories on the boundaries of science.
SO: Journal title, e.g., Public Understanding of Science.
AB: Publication’s abstract.
C1: First author’s country, extracted from first author’s research institution.
795
(P. 793-805) ISSN 2538–7138 /Online/
CR: Normalized citations, e.g. Campbell Colin, 1972; Sociological YB Reli, V5, P119.
PY: Year of publication, e.g. 2015.
Step 2: Similarity Computation
After CATAR captures each article’s relevant information, the program used the common citations normalized
by each paper’s individual citations to calculate the similarities between other articles. The more of the same articles
each paper cites, the more likely the articles are discussing the same topic. This is the foundation of bibliographic
coupling that creates clusters of research topics.
Step 3: Multi-Stage Clustering (MSC)
A clustering algorithm called complete linkage clustering is used to discern the underlying themes of the col-
lection of papers and assign the journals to individual clusters (Salton, 1989). CATAR first recognizes each article
as a single cluster and then groups the most similar pairs of clusters into a larger cluster and repeats this rule until
a reasonable and manageable number of clusters are formed or can no longer be merged. For example, three
rounds could yield 35 clusters, whereas four rounds of clustering can yield nine clusters. There is a determined
similarity threshold for clusters to be grouped together, where pairs of clusters with a similarity larger than the
determined threshold are grouped and those not fulfilling the threshold are excluded, resulting in a smaller set of
resulting papers. This research utilized a similarity threshold of .05 and four rounds of clustering were carried out
to provide the final research topics.
Step 4: Cluster Labeling
While CATAR is an automated process, largely taking care of the intensive labor of analyzing large amounts of
research articles with human interpretation, analysts still need to read through titles and abstracts to determine
the broad topics covered by each cluster. CATAR uses a text mining approach to generate automatic descriptions
for each cluster. The algorithm filters through words in the titles and abstracts of each paper and extracts the most
repeated words and phrases and uses the five most correlated terms to serve as the descriptors for each cluster.
These words provide a basis for researchers to characterize clusters with relevant descriptors to label a research topic.
Step 5: Facet Analysis
Once the topics have been detected in cluster labelling, it becomes more possible to analyze multiple facets of
information from each article over a larger set of data. For example, one can know the topic distributions of all the
authors, elucidating their areas of expertise in addition to their individual productivity. One can also analyze yearly
distributions of each topic allowing for topic trends to emerge more clearly. The topic trends can also be quantified,
such as when determining the slope of a linear regression line that best fits the time series. These analyses are the
major focus of this research and allow for broad level systematic analysis of large sets of data.
Step 6: Visualization
The last step was to visually represent the clusters using Multi-Stage Clustering (MSC) and Multi-Dimensional
Scaling (MDS). Using the pre-calculated similarities between topics, the MSC organizes the topics to display a
structure that readily shows information about each cluster (like descriptors). The MDS technique also computes a
location for each cluster in Euclidean space to aid in visual interpretation. A topic map is then created where factors
such as cluster size, closeness, and spatial topic distribution are readily interpreted. The representations provide
different ways to explore and interpret the data. The two-dimensional visualization helps understand inter-cluster
relationships by distance and orientation.
796
ISSN 2538–7138 /Online/ (P. 793-805)
Journal Selection
To understand how science communication has developed since 1997, researchers must explore the large
trends of the field and the development of academic discourse, especially in conjunction with other relevant
major themes. In carrying out a trends study, factors such as related fields and topics also can provide useful
information in evaluating science communication’s influence or relationship to other research areas. Prior to this
research, there has been no significant quantitative trends analysis of science communication research journals.
Researchers had carried analysis studies in the related field of science education and others such as tourism and
general education (Chang et al., 2010; Lee et al., 2009; Tsai & Wen, 2005; Tseng & Tsay, 2013; Yuan et al., 2015) but
to date, science communication has been largely isolated in this course of research. To present a representative
and longitudinal overview of science communication over the years, this research conducted trends analysis of
science communication research pulling from 1300 articles dating from January 1, 1997 to June 23, 2018 in the
journals, Science Communication and Public Understanding of Science. These years were chosen to encompass the
earliest period of journal indexing (1997 for Science Communication and 1994 for Public Understanding of Science)
to the most-recent complete year that research was available, maximizing the broadest quantity of data. These
journals also have cross-disciplinary relevance, especially in the related topics of science education, thus providing
a foundation to find linkages with the two fields. The study chose these two journals due to their high citation rates
within the field of science communication. Additionally, their inclusion in the Social Sciences Citation Index of the
Web of Science (WoS) ensures the documents can be analyzed using CATAR. Currently, CATAR can only analyze WoS
database. Thus, data collection process excluded some other potential science communication related journals
such as International Journal of Science Education, Part B, and Journal of Science Communication.
The data for the publications analysis were downloaded from Thomson Reuter’s ISI Web of Science (WoS) dur-
ing mid-June 2018 to ensure that completed research articles up until 2018 would be included in the database of
articles. The type of the documents only included research articles, excluding commentaries, editorials, and short
research reports or book reviews to provide the most direct source of research.
Justification for Validity
The data set consisted of 1300 papers (i.e., PUS: 819; SC: 481) and was narrowed down to 393 articles after the
fourth clustering analysis (see Step 4 Cluster labelling) removed extraneous papers that were not categorized under
a broader topic. It is common during outlier removal in the MSC process for a manageable number of research
themes to analyze. A “long tail effect” (Figure.1) could explain why a large portion of the papers were removed in
clustering analysis (Tseng, 2018).
Figure 1. A demonstration of the Long Tail effect.
It is normal in clustering analysis because more than average of the papers usually deal with independent is-
sues, which is similar to a phenomenon in business such as online book sales. More than 40% of sales are produced
797
(P. 793-805) ISSN 2538–7138 /Online/
by a variety of customers’ personal interests (Brynjolfsson, Hu, & Smith, 2003). Thus, 393 articles were analyzed in
this broader trends study, excluding 907 of originally chosen papers, a reduction rate congruous with previous
studies and additionally explained below (Chang et al., 2010). Moreover, for assuring the least potential bias while
finalizing the research topics (see Step 3~5), two analysts with science communication domain knowledge read
together through the titles and abstracts to generate the topics.
Research Results
Seven Topics Emerge in Science Communication Research
Seven categories of science communication research trends emerged from Science Communication (SC) and
Public Understanding of Science (PUS) during the past two decades (i.e. 1997-2018). Figure 2 recorded the article
counts and percentages in each of the seven categories.
Figure 2. Seven topic categories in science communication research and their composition from journals of
PUS and SC from 1997 to mid-2018 (n = 393).
The spatial relatedness among the seven topics are mapped in Figure 3, where a circle represents a topic
and its size is proportional to the number of articles contained within the topic. The topic map was illustrated by
using Perl programming language (Tseng & Tsay, 2013). Relatedness between circles were presented in terms of
orientation and distance. The circles with overlap represent topics that have stronger relationships in terms of topic
similarity and degree of citation linking. Circles that are located at farther distances from each other indicate topics
that are mainly independent or largely not similar.
798
ISSN 2538–7138 /Online/ (P. 793-805)
Figure 3. Topic map rendered by multi-dimensional scaling (MDS) for multi-stage clustering of Science Com-
munication and Public Understanding of Science journal articles.
Table 1 shows detailed subtopics within the seven trends under the three themes.
Table 1. Topic themes encompassing the seven topics of science communication research and counts of
articles in subtopics.
Themes Abbreviations/Topics (Trends) Docs Subtopics (n)
Science in the News (21)

SRMJ/Science and it’s Representation in News Frames and Science (28)
78
Media and Journalism Media Representations (16)
Media and Science (MS) Media Coverage (13)
Effects of Mass Media (49)

MMPAS/Mass Media and Public Attitudes
96 Issue-related Coverage in Media (34)
towards Science
Risk and Uncertainty Perceptions (13)
SRMP/ Scientists Relationship with Media Scientists’ Participation (17)

29
and Public Scientists for Public Engagement (12)
Public Engagement with Science (24)

Public Engagement and Science SESC/ Science Engagement and Science Constructing Communication (32)
85
(PES) Citizenship Engagement Methodology (17)
Trust, Identities and Risk (12)
PUSSL/ Public Understanding of Science and Public Participation (15)

51
Science Literacy Attitudes toward Science (36)
799
(P. 793-805) ISSN 2538–7138 /Online/
Themes Abbreviations/Topics (Trends) Docs Subtopics (n)
BNM/ Biotechnology and Nanotechnology Genetically Modified Food and Biotechnology (13)
26
in Media Public and Nanotechnology (13)
Issues in Science (IS)
CCDP/ Climate Change and Disease Percep- Climate Change and Global Warming (14)
28
tions Climate Technology and Actions (14)
Figure 4 demonstrates the publication counts for development trends of the seven topics of science com-
munication research from 1997-2018.
Figure 4. The publication counts for development trends of the seven topics of science communication research
from 1997-2018.
Finally, Figure 5 merged the seven science communication research topics into three themes.
Figure 5. The publication counts for the development of overall trends for the seven science communication
topics from 1997-2018.
Trends Concerning Theme PES
There was an increase in Theme PES research, encompassing topics: “Public Perception and Understanding
of Science and Science Literacy (PUSSL),” “Science Engagement and Science Citizenship (SESC),” and “Scientists’
Relationship with Media and Public (SRMP)” overall since 1997. SRMP has maintained steady output during the 21
800
ISSN 2538–7138 /Online/ (P. 793-805)
year period (Figure 4). SESC has experienced general increases in research output since 1997 with a decline since
2014 and PUSSL has lower overall output, but has increased in publications for the duration from 1997-2018 (Figure
4). SESC, of the three topics in this theme, has the highest overall output during this period, and is a focal point of
the results given its high share of research output over the last decade.
Trends Concerning Theme MS
Overall, Media and Science as a theme has had overall increases in research output during the entire period of
1997-2018. Researches regarding “Science and its Representation in Media and Journalism (SRMJ)” remain constant
in output except an uptick in output during the 2009-2011 period. The topic of “Mass Media and Public Attitudes
Towards Science (MMPAS)” overall have increased, with sharp increases in output starting in 2006. The possible
reasons for this are discussed in the next section. The topic of media and science is a commonly present topic in
science communication and thus is reliably researched and consistently a topic of discussion. The trends of these
topics may also coincide with other factors in research and science communication.
Trends Concerning Theme IS
Biotechnology and Nanatechnology in Media (BNM) and Climate Change and Disease Perception (CCDP) all
covered material regarding prominent topics in science that are of particular interest to the public and remain
present in contemporary media. Climate change, health, disease, and biotechnology among other topics have
remained publicly prominent for some time. These topics are either seen as controversial, referencing aspects of
science that many in the public may fear, may not adequately understand, or target topics that are of immediate
relevance to individuals. BNM and CCDP have received increased publishing output most notably from 2006-2018
with a sharp uptick in publishing, though at a lower rate than those of Theme PES.
General Trends Across Three Themes
The trends across all topics were documented in Figure 4 and explained in following sections. CATAR automatic
content analysis generated seven main topics and three overarching themes based on the similarities between
topics as indicated by their proximity in the topic map or their relatedness as prominent topics in science. To en-
compass topics that largely engage in the interface between the public, science and scientists, Theme PES: Public
Engagement and Science was formed. Theme MS: Media and Science, focuses on the portrayal and coverage of
science in the media. Theme IS: Issues in Science covers topics that may not be directly related but nonetheless,
share the common trait of being prominently featured in media and discussions of science.
Discussion
RQ1: What are the major research trends in the field of science communication?
While this research organizes overall trends of science communication, it also serves the purpose of providing a
basis for linkages with other fields of research related to the analyzed research papers. This connection is especially
relevant in fields that previously have collaborated the fields of interest discussed in this research as well as other
fields where relationships may remain unexplored. This research discusses the linkages science communication
has with previous trends studies in science education.
Theme MS: Increased Media Attention to Science
The increase in Media themed research is a primary finding of this research. Research relating to Media and
Science showed a general upward trend of output over the last two decades, with the largest increase taking place
after 2006. Media is a fundamental component of science communication in today’s landscape given its role of
interfacing with the public as well as the duty scientists feel to interact with the media (Peters, 2013). Research
related to science’s representation and framing in the media (MMPAS) have shown steady presence with an in-
creased output in 2009-2011 (Figure 3), while research relating to mass media and public attitudes towards science
801
(P. 793-805) ISSN 2538–7138 /Online/
have increased in output throughout 1997-2018. These topics of research are primarily concerned with how media
frames or represents science and how that affects the ways science is perceived in the public. Given that media is
a primary vehicle for communication, the upward trend in research indicates that during the past two decades,
media has only grown in importance in science communication, especially given the role it plays in communicat-
ing and representing science (Nelkin, 1995). How science is presented and how the public feels about science are
increasingly topics of interest and consequently form a large theme in science communication research. In addi-
tion, journals such as Journal of Science & Popular Culture, Journal of Science Communication, International Journal
of Science Education, Part B, or other international journals provide further context to science in media
RQ2: How does science education play a role in the research trends of science communication?
Theme PES: Engaging in a Conversation with Science
One main trend exists in Theme PES: Public Engagement with Science. Over the course of the last two decades
that SC and PUS have been publishing research articles, there has been a consistent and steady increase in the
output of articles related to the science engagement and science citizenship (SESC). Topics relating to the scientists’
with the media and public (SRMP) also showed noted increases in output and subsequently steadying during the
past ten years. Research related to public understanding of science and science literacy have remained relatively
constant within the range of five to ten articles produced in each 3-year period. The topic of public knowledge and
understanding of science is an important and arguably necessary prerequisite to engaging the public in becoming
more active and invested citizens in the scientific process (Schafer, 2012). During the past two decades, this topic
has remained relatively constant in its research output, indicating that it remains an important topic, but lower in
output compared to research related to science engagement. In conjunction, research related to scientists’ rela-
tionship with the public and media has also, though at a lesser rate and output, increased in number. The trend of
increased science engagement with the public aligns with moving from deficit models of science communication
to more interactive discussion, reflected in the increased output in SESC and SRMP related to science engagement
and scientists relationship with the public, respectively.
As science communication research continues to delve more into how the field can engage the public in dis-
course recognizing that both the public and science communicators possess deficits in understanding the groups
they interact with, there promises to be more research to address public engagement (Bauer et al., 2007). The
increase in research regarding science engagement most likely indicates that science communication continues
to grapple with how best to engage the public to increase science understanding as well as how the public can
actively participate in science-related activities rather than continue with deficit-oriented avenues of consuming
science (Seakins & Dillon, 2013). The roles of scientists in communication also will continue to require clarification
as they shift from the role of experts in the deficit-oriented model of communication to participants in the public
engagement process (Lewenstein, 2011; Stocklmayer & Bryant, 2012). Reflecting increased focus on public engage-
ment, other outlets beyond PUS and SC have also recently promoted articles such as a 2014 issue of “Understanding
the Public Understanding of Science: Psychological Approaches” in Educational Psychologist or the emergence of Public
Communication of Science and Technology (PCST) Network to promote public understanding of and engagement
with science. As these trends indicate, increased interactive and engaged models of science communication will
continue to be an area of interest for future studies that can build upon understanding the complex interactions
of science with society (Baram-Tsabari & Lewenstein, 2017).
Theme IS: Issues in Science
In regards to the themes of Issues in Science, research trends analysis is less pronounced, but there is still an
increase in research in this theme. In Issues in Science, there are two main topics. Topics in BNM (Biotechnology and
Nanotechnology in Media) and CCDP (Climate Change and Disease Perceptions) have seen upticks in their research
output, indicating that research related to climate change as well as biotechnology remain and continue to be more
frequently discussed and relevant topics. The relevance of these topics is indicated by constant exposure to news and
discussions related to climate change as the yearly temperatures around the world continue to break records and
debates regarding climate change coverage continues (Busch, 2016). Biotechnology and nanotechnology continue
802
ISSN 2538–7138 /Online/ (P. 793-805)
to garner more interest in popular news outlets (Acarli, 2016), given technological breakthroughs and the potential
impact that these technologies can have on peoples’ lives and bodies (S.-F. Lin, Lin, Lee, & Yore, 2015). The general
upward trend for the topics in this theme indicate that issues generated in science remain intensely researched
or discussed. BNM and CCDP topics, as seen in Figure 3, are mostly unrelated in content indicated by their distinct
spacing on the topic map. Political, socio-cultural, or international events such as the release of science-related
documentary “An Inconvenient Truth” could trigger interests in topics among constant media coverage and could
have played a role in the discussion of climate change (Busch, 2016; Zimmerman & Bell, 2014). Advancements in
body-modifying treatments or other controversial ramifications of cutting-edge technology can likewise trigger
changes, positive and negative, in research focus (Kempner, 2008). These two of research, while present and con-
sistently relevant to science communication, could also be present in other enclaves of discussion such as widely
accessed news outlets or in broader media such that academic research does not target (Friedman et al., 1999).
Overall, the increased research output in science education related to conceptual change as well as socio-
scientific issues is complementary to research output in science communication relating to public engagement
in science as well as public attitudes towards science. These trends indicate an available space for cooperation
and increased dialogue between the fields’ researchers given that understanding conceptual change and how
individuals learn best is affected by the way scientific experts or communicators convey or frame science content
(Theme PES and MS). Also, increased research in socio-scientific issues is concerned with how science is represented
in media and how science educators may utilize to engage the public. The fields of science education and com-
munication have developed independently and have largely kept to their own respective communities (Baram-
Tsabari & Lewenstein, 2017; Feinstein, 2015), but in the most recent decade, as we have witnessed and analyzed
drastic upticks in research, science communication and education continue to share broad and specific goals of
engaging the learners in classroom and with the public to promote engagement in a more collaborative manner
(Mercer-Mapstone & Kuchel, 2017). This convergence points to the idea of a field of “Science-Edu-Communication”
as an area of great potential (P.-H. Wu, Wu, & Chang, 2016). There exists a substantial space for educators, media,
and those that specialize in mediating between the scientific community and the public to collaborate and further
broaden scientific literacy.
Conclusions
It was confirmed in this research that scholarly output of science communication has trended toward a
sharing goal of science education: promoting public understanding of science via ways of public engagement of
science and media representation of science. This research also found that there was much complementary space
for further collaboration with science communicators as the field increasingly focuses on research aligned with
Public Engagement (Theme PES) and Media (Theme MS). Science education’s research on conceptual change,
socio-scientific issues and learning contexts, were also complementary with science communications’ focus on
science engagement and public attitudes towards science as expertise in both fields only serves to enrich increas-
ing scientific understanding and literacy.
Therein exists a crucial need for science education to play a role in the future science communication de-
velopment. Changing the perception of science in the public and in educational settings remains fundamental
in engaging the public and thus collaborations with science education researchers is an area of promise. In the
future, with increased research collaboration between science education and communication trends studies to
quantify linkages in the two fields will be possible. This applies especially to the fields that would benefit from
understanding of how media portrays the science and how to best educate unfamiliar audiences with science
content. Also, as more avenues such as journals, conferences, and increased catalogues of research related to “sci-
ence edu-communication” are made available, researchers can further depict the direction and overall findings
of the interdisciplinary research field.
Acknowledgements
This research is supported by the Ministry of Science and Technology, Taiwan, R.O.C., under Grant No. MOST-
106-2511-S-033-010-MY2.
803
(P. 793-805) ISSN 2538–7138 /Online/
References
Acarli, D. S. (2016). Determining prospective biology teachers’ cognitive structure in terms of “Biotechnology”. Journal of Baltic
Science Education, 15(4), 494-505.
Baram-Tsabari, A., & Lewenstein, B. V. (2017). Science communication training: What are we trying to teach? International Journal
of Science Education, Part B, 7(3), 285-300. doi:10.1080/21548455.2017.1303756.
Baram-Tsabari, A., & Osborne, J. (2015). Bridging science education and science communication research. Journal of Research in
Science Teaching, 52(2), 135-144. doi:10.1002/tea.21202.
Bauer, M. W., Allum, N., & Miller, S. (2007). What can we learn from 25 years of PUS survey research? Liberating and expanding
the agenda. Public Understanding of Science, 16(1), 79-95. doi:10.1177/0963662506071287.
Berinstein, P. (2006). The truth, the whole truth, and nothing but the truth: Error and misinformation in media. Searcher, 14(8), 38-52.
Brynjolfsson, E., Hu, Y., & Smith, M. D. (2003). Consumer surplus in the digital economy: Estimating the value of increased product
variety at online booksellers. 49(11), 1580-1596. doi:10.1287/mnsc.49.11.1580.20580.
Bucchi, M. (2008). Of deficits, deviations and dialogues: Theories of public communication of science. In M. Bucchi & B. Trench
(Eds.), Handbook of Public Communication of Science and Technology (1 ed., pp. 57-76). New York: Routledge.
Bucchi, M. (2013). Style in science communication. Public Understanding of Science, 22(8), 904-915. doi:10.1177/0963662513498202.
Bucchi, M., & Trench, B. (2008). Handbook of Public Communication of Science and Technology. New York: Routledge.
Busch, K. C. (2016). Polar bears or people? Exploring ways in which teachers frame climate change in the classroom. International
Journal of Science Education, Part B, 6(2), 137-165. doi:10.1080/21548455.2015.1027320.
Cavas, B., Cavas, P., Ozdem, Y., Rannikmae, M., & Ertepinar, H. (2012). Research trends in science education from the perspective of
Journal of Baltic Science Education: A content analysis from 2002 to 2011. Journal of Baltic Science Education, 11(1), 94-102.
Chang, Y.-H., Chang, C.-Y., & Tseng, Y.-H. (2010). Trends of Science Education Research: An Automatic Content Analysis. Journal
of Science Education & Technology, 19(4), 315-331. doi:10.1007/s10956-009-9202-2.
Claessens, M. (2008). European Trends in Science Communication. In D. Cheng, M. Claessens, T. Gascoigne, J. Metcalfe, B. Schiele,
& S. Shi (Eds.), Communicating Science in Social Contexts: New Models, New Practices (pp. 27-38): New York: Springer.
Davis, P. R., & Russ, R. S. (2015). Dynamic framing in the communication of scientific research: Texts and interactions. Journal of
Research in Science Teaching, 52(2), 221-252. doi:10.1002/tea.21189.
Feinstein, N. (2015). Education, communication, and science in the public sphere. Journal of Research in Science Teaching, 52(2),
145-163. doi:10.1002/tea.21192.
Friedman, S. M., Dunwoody, S., & Rogers, C. L. (1999). Communicating Uncertainty: Media Coverage of New and Controversial Sci-
ence: Erlbaum Associates.
Hilgartner, S. (1990). The dominant view of popularization: Conceptual problems, political uses. 20(3), 519-539.
doi:10.1177/030631290020003006.
Kempner, J. (2008). The Chilling Effect: How Do Researchers React to Controversy? PLOS Medicine, 5(11), e222. doi:10.1371/
journal.pmed.0050222.
Kerckhove, D. D. (1995). The Skin of Culture: Investigating the New Electronic Reality. Toronto: Somerville House Pub.
Lakhvich, T. (2010). Science education: Regarding trends and main principles. Journal of Baltic Science Education, 9(3), 164-166.
Lee, M. H., Wu, Y. T., & Tsai, C. C. (2009). Research Trends in Science Education from 2003 to 2007: A content analysis of publica-
tions in selected journals. International Journal of Science Education, 31(15), 1999-2020. doi:10.1080/09500690802314876.
Lewenstein, B. V. (2011). Changing our ideas. International Journal of Science Education, Part B, 1(1), 17-21. doi:10.1080/215484
55.2011.554013.
Lewenstein, B. V. (2015). Identifying what matters: Science education, science communication, and democracy. Journal of Research
in Science Teaching, 52(2), 253-262. doi:10.1002/tea.21201.
Lin, S.-F., Lin, H.-s., Lee, L., & Yore, L. D. (2015). Are science comics a good medium for science communication? The case for
public learning of nanotechnology. International Journal of Science Education, Part B, 5(3), 276-294. doi:10.1080/2154845
5.2014.941040
Lin, T.-C., Lin, T.-J., & Tsai, C.-C. (2014). Research trends in science education from 2008 to 2012: A systematic content analysis of
publications in selected journals. International Journal of Science Education, 36(8), 1346-1372. doi:10.1080/09500693.201
3.864428.
Logan, R. A. (2001). Science mass communication: Its conceptual history. Science Communication, 23(2), 135-163. doi:10.1177/
1075547001023002004.
Martin, V. Y. (2017). Citizen science as a means for increasing public engagement in science. Science Communication, 39(2), 142-
168. doi:10.1177/1075547017696165.
McNeil, M. (2013). Between a Rock and a Hard Place: The Deficit Model, the Diffusion Model and Publics in STS. Science as Culture,
22(4), 589-608. doi:10.1080/14636778.2013.764068.
Mercer-Mapstone, L., & Kuchel, L. (2017). Core skills for effective science communication: A teaching resource for undergraduate
science education. International Journal of Science Education, Part B, 7(2), 181-201. doi:10.1080/21548455.2015.1113573.
Nelkin, D. (1995). Selling Science: How the Press Covers Science and Technology. New York: W.H. Freeman.
Ogawa, M. (2011). A new age of cooperation and collaboration between school science education research and science com-
munication research. International Journal of Science Education, Part B, 1(1), 9-11. doi:10.1080/09500693.2010.543708.
Peters, H. P. (2013). Gap between science and media revisited: Scientists as public communicators. Proceedings of the National
Academy of Sciences, 110(Supplement 3), 14102-14109. doi:10.1073/pnas.1212745110.
804
ISSN 2538–7138 /Online/ (P. 793-805)
Salton, G. (1989). Automatic text processing: the transformation, analysis, and retrieval of information by computer: Addison-Wesley
Longman Publishing Co., Inc.
Schafer, M. (2012). Taking stock: A meta-analysis of studies on the media’s coverage of science. Public Understanding of Science,
21(6), 650-663. doi:10.1177/0963662510387559.
Seakins, A., & Dillon, J. (2013). Exploring research themes in public engagement within a natural history museum: A modified
Delphi approach. International Journal of Science Education, Part B, 3(1), 52-76. doi:10.1080/21548455.2012.753168.
Stocklmayer, S. M., & Bryant, C. (2012). Science and the public—What should people know? International Journal of Science
Education, Part B, 2(1), 81-101. doi:10.1080/09500693.2010.543186.
Sturgis, P., & Allum, N. (2004). Science in society: re-evaluating the deficit model of public attitudes. Public Understanding of Sci-
ence, 13(1), 55-74. doi:10.1177/0963662504042690.
Suldovsky, B. (2016). In science communication, why does the idea of the public deficit always return? Exploring key influences.
Public Understanding of Science, 25(4), 415-426. doi:10.1177/0963662516629750.
Tsai, C. C., & Wen, M. L. (2005). Research and trends in science education from 1998 to 2002: a content analysis of publication in
selected journals. International Journal of Science Education, 27(1), 3-14. doi:10.1080/0950069042000243727.
Tseng, Y.-H. (2010). Generic title labeling for clustered documents. Expert Systems with Applications, 37(3), 2247-2254. doi:10.1016/j.
eswa.2009.07.048.
Tseng, Y.-H. (2018). Content Analysis Toolkit for Academic Research (CATAR). Retrieved from http://web.ntnu.edu.tw/~samtseng/
CATAR/Readme.html.
Tseng, Y.-H., & Tsay, M.-Y. (2013). Journal clustering of library and information science for subfield delineation using the biblio-
metric analysis toolkit: CATAR. Scientometrics, 95(2), 503-528. doi:10.1007/s11192-013-0964-1.
Wu, L. Y., Chang, C. Y., Liu, H. H., Wu, P. H., Lei, Y. C., & Lu, H. Y. (2015). Piloting a collaboration between education and broadcast
journalism in Taiwan. Science Communication, 37(4), 542-548. doi:10.1177/1075547015592068.
Wu, P.-H., Wu, L. Y., & Chang, C.-Y. (2016). Science Edu-communication: An exploration of the impact of science education initiated
television news on learners’ perceived Awareness, Enjoyment, Interest, Opinion Formation, and Understanding (AEIOU). Paper
presented at the International Conference of East-Asian Association for Science Education (EASE 2016), Tokyo.
Yuan, Y., Gretzel, U., & Tseng, Y.-H. (2015). Revealing the Nature of Contemporary Tourism Research: Extracting Common Subject
Areas through Bibliographic Coupling. International Journal of Tourism Research, 17(5), 417-431. doi:10.1002/jtr.2004.
Zimmerman, H. T., & Bell, P. (2014). Where young people see science: Everyday activities connected to science. International
Journal of Science Education, Part B, 4(1), 25-53. doi:10.1080/21548455.2012.741271.
Received: August 18, 2018 Accepted: October 06, 2019
Leon Yufeng Wu PhD, Assistant Professor, Chung Yuan Christian University,

200 Chung Pei Road, Chung Li District, Taoyuan City, Taiwan
32023, R.O.C.
E-mail: leonyifengwu@gmail.com
Nathan M. Truong MA, Research Assistant, National Taiwan Normal University
88 Sec. 4, Ting-Chou Road, Taipei, Taiwan 11677.
E-mail: truong.nathan@gmail.com
Hsin-Yen Lu MA, Research Assistant, National Taiwan Normal University
88 Sec. 4, Ting-Chou Road, Taipei, Taiwan 11677.
E-mail: happysky0323@gmail.com
Yuen-Hsien Tseng PhD, Professor, National Taiwan Normal University,
No.162, Sec. 1, Hoping East Rd., Da-an District, Taipei 10610,
Taiwan.
E-mail: samtseng@ntnu.edu.tw
Chun-Yen Chang PhD, Professor, National Taiwan Normal University,
(Corresponding author) 88 Sec. 4, Ting-Chou Road, Taipei, Taiwan 11677.
E-mail: changcy@ntnu.edu.tw
Website: http://changcy.com/
805
INFORMATION
FOR CONTRIBUTORS
EDITORIAL AND REVIEW PROCEDURES
Journal of Baltic Science Education (JBSE) publishes original scientific research articles in the field of Natural Science Education and
related areas for all educational levels in the Baltic countries. It is possible to publish special (thematic) issues of JBSE. The papers should
be submitted and will be published in English. JBSE will promote to establish contacts between researchers and practical educators
both in the Baltic countries and countries around.
The authors of the manuscripts are responsible for the scientific content and novelty of the research materials. Articles, published
before in other international journals or papers’ collections will not be accepted for publication in JBSE.
As a publication that represents a variety of cross-disciplinary interests, both theoretical and practical, the JBSE invites manuscripts on
a wide range of topics, especially in the following areas:
• Didactics of natural sciences. • Philosophical, political, economical and social aspects
• Theory and practice in natural science teacher of natural science education.
education. • The supplementary natural science education.
• Integrated natural science education. • ICT in natural science education.
• Natural science and technological literacy. • The standardisation of natural science education etc.
• General and professional natural science education.
MANUSCRIPTS GUIDELINES
The structure of the research paper presented to the Journal of Baltic Science Education should be as follows: abstract - short report of the
investigation; introduction inc. aim and subject of the research; research methodologies and methods; results of the research incl. discussion;
conclusions; list of references in APA style.
The papers should be submitted in English. If English is a second language for the author, please consider having the manuscript
proof read and edited before submitting. The preliminary text of the article can be sent as a.doc file in the attachment by e-mail: mail.
jbse@gmail.com
The text must be elaborated in Word for Windows, using 12 point Times New Roman letters. An article should not exceed 7-10 A4 pages,
included figures, tables and bibliography. Publishing of longer articles should be negotiated separately. Texts margins: top and bottom 20mm,
left - 25mm, right - 20mm. The title: capital letters, 14pt, bold; space between the title and the author’s name is one line interval. Author’s name
and surname: small letters, 12pt, bold. Under the name, institution: 11 pt, italics; space between the title and the text: 1 line interval. Abstract
– about 100-150 words - precedes the text. The text: 12pt Single or Auto spacing, in one column. Key words: no more than five words. The
language must be clear and accurate. The authors have to present the results, propositions and conclusions in a form that can suit scientists
from different countries.
Titles of the tables and figures: 11 pt, small letters. Space between figures or tables and the text: 1 line interval. Introduction, titles of
chapters and subchapters: 12pt, bold, small letters. Numbers: Arabic, subchapters numbered by two figures (1.1, 1.2, etc.). Figures, tables and
captions should be inserted within the manuscript at their appropriate locations. Diagrams and graphs should be provided as finished black
and white line artwork or electronic images. When there are a number of illustrations, the author should endeavour to reduce the amount of
text to accommodate the illustrations in the limited space available for any article.
References in the text should be presented in brackets (Knox, 1988; Martin, 1995). If necessary, the page can be indicated: (Martin,
1995, p.48). The list of references should be presented after the text. The Words List of References: 11pt, bold, small letters. The references
should be listed in full at the end of the paper in the following standard form:
For books: Saxe, G.B. (1991). Cultural and Cognitive Development: Studies in Mathematical Understanding. Hillsdale, NJ: Erlbaum.
For articles: Bekerian, D.A. (1993). In Search of the Typical Eyewitness. American Psychologist, 48, 574-576.
For chapters within books: Bjork, R.A. (1989). Retrieval Inhibition as an Adaptive Mechanism in Human Memory. In: H.L. Roediger III & F.I.M.
Craik (Eds.), Varieties of Memory & Consciousness (pp. 309-330). Hillsdale, NJ: Erlbaum.
On a separate page, author - related data should be presented in English: name, surname, degree and academic title, institution, full
correspondence address in the clearest and most complete form /ordinary post and e-mail addresses /, position (to ensure anonymity in the
review process). The author (authors) should confirm in writing, that the manuscript has not been published in other journal or
handed over (transferred) to other journal for publication.
EDITORIAL AND REVIEW PROCEDURES

Manuscripts will be sent anonymously to reviewers with expertise in the appropriate area. All manuscripts will be rewieved by two
experts before JBSE’s accept them for publication. This process usually takes about two months. The journal co-editors will make minor
editorial changes; major changes will be made by the author(s) prior to publication if necessary. JBSE’s redaction will sent to author(s)
only one correcture which must be sent back within 2 weeks. JBSE will not review submissions previously published elsewhere through
print or electronic medium.
Manuscripts submitted to the JBSE cannot be returned to authors. Authors should be sure to keep a copy for themselves. Authors’
signatures should be at the end of the paper and its second checked proofs.
Manuscripts, editorial correspondence (and other correspondence for subscription and exchange), and any questions should be
sent to editor-in-chief or to regional redactors.
Journal`s requirements for the authors are available online: http://www.scientiasocialis.lt/jbse/files/JBSE_requirements_2019.pdf
806
JBSE
Problems of Psychology in the 21st
Century is an international, periodical,
peer reviewed scientific journal, issued by
the Scientia Socialis, UAB in cooperation
with SMC “Scientia Educologica”.
Editor-in-Chief
Prof. dr. Vincentas Lamanauskas, Scientific

Methodical Centre „Scientia Educologica“,
Republic of Lithuania
Problems of Psychology in the 21st Century

(ISSN 2029-8587) is abstracted and/or indexed
in:
Index Copernicus -
http://journals.indexcopernicus.com
EBSCO - http://search.ebscohost.com
Editorial Board
Dr., prof. Ferda Aysan, Dokuz Eylul University, Turkey

Dr., prof. Serhiy Boltivets, Grigory Kostyuk Psychological Institute of the Ukrainian National
Academy of Pedagogical Sciences, Ukraine
Dr., assoc. prof. Irena Gailiene, SMC "Scientia Educologica", Republic of Lithuania
Dr., prof. Irakli Imedadze, Dimitri Uznadze Georgian Psychological National Society, Georgia
Dr. Julia Lakhvich, Belarusian State University, Republic of Belarus
Dr., prof. Vladimir S. Karapetyan, Armenian State Pedagogical University named after Kh.
Abovyan, Armenia
Dr., prof. Vincentas Lamanauskas, Scientific Methodical Centre „Scientia Educologica“, Republic of
Lithuania (Editor-in-Chief)
Dr., prof. Maria Ledzińska, University of Warsaw, Poland
Dr., prof. Aleksandr Lobanov, Belarusian State Pedagogical University, Republic of Belarus
Dr., prof. Guna Svence, Riga Teacher Training and Educational Management Academy, Latvia
Website: http://www.jbse.webinfo.lt/PPC/Problems_of_Psychology.htm
Website: http://www.scientiasocialis.lt/ppc/
807
ISSN 1648-3898 /Print/, ISSN 2538-7138 /Online/
Compiled by: Vincentas Lamanauskas

Linguistic Editor: Ilona Ratkevičienė
Cover design by: Jurgina Jankauskienė
Layout design by : Linas Janonis

15 October 2019. Publishing in Quires 10. Edition 200
Publisher Scientia Socialis Ltd.,

Donelaicio Street 29, LT-78115 Siauliai, Lithuania
E-mail: scientia@scientiasocialis.lt
Phone: +370 687 95668
http://www.jbse.webinfo.lt/centras.htm
http://www.scientiasocialis.lt

Printing Šiauliai printing house
9A P. Lukšio Street
LT-76207 Šiauliai, Lithuania
Phone: +370 41 500 333.
Fax: +370 41 500 336
E-mail: info@dailu.lt

Journal of Baltic Science Education, Vol. 18, No. 5, 2019

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Journal of Baltic Science Education, Vol. 18, No. 5, 2019

Uploaded by

Copyright:

Available Formats

Vol.18, No.

Published since 2002 Address:

ISSN 2538–7138 /Online/

INDUSTRY 4.0 AND SCIENCE EDUCATION

INDONESIAN PRIMARY SCHOOL AND MIDDLE SCHOOL STUDENTS’ ATTITUDES TOWARD

DICHOTOMOUS KEYS IN THE BOTANICAL LEARNING OF NON-VISUAL (BLIND) PEOPLE

RECONCEPTUALIZING AND FIELD TESTING THE SCIENTIFIC LITERACY FRAMEWORK BY

EFFECTS OF MICROTEACHING MULTIPLE-REPRESENTATION PHYSICS LESSON STUDY ON PRE-

THE INFLUENCE OF SELECTED VARIABLES ON LOWER SECONDARY SCHOOL STUDENTS’

THE RELATION BETWEEN TEACHERS’ ATTITUDES TOWARDS RENEWABLE ENERGY SOURCES

BIBLIOMETRIC ANALYSIS OF THE WOS LITERATURE ON RESEARCH OF SCIENCE TEACHER

EVALUATION, VALIDATION AND MODIFICATION OF SCIENCE MOTIVATION QUESTIONNAIRE

TEACHER-DEMONSTRATION AND STUDENT HANDS-ON EXPERIMENTS IN TEACHING

EXAMINATION OF PERCEPTIONS OF GIFTED STUDENTS ABOUT CLIMATE CHANGE AND

SCIENCE-EDU-COMMUNICATION: TRENDS REVEAL IN 20 YEARS OF SCIENCE

INFORMATION FOR CONTRIBUTORS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 806

PROBLEMS OF PSYCHOLOGY IN THE 21ST CENTURY. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 807

INDUSTRY 4.0 AND SCIENCE EDUCATION

Received: September 20, 2019 Accepted: October 05, 2019

Bulent Cavas PhD, Professor, Dokuz Eylul University, Distance Education

Abstract. Student attitudes toward science Rahmi Qurota Aini,

Attitude towards Science

Gender Issue in Science and Attitude towards Science

Gender, Culture, and Educational Issue in Indonesia

Indonesian K-4 to 9 Science Curriculums

Research Instrument and Instrument Validation

Table 1. Dimensionality test.

Final Deviance AIC

One-Dimension 117287 117529

Table 2. Psychometrical properties based on Rasch Modeling in the BRAINS instrument.

BRAIN02 –.459 1.07 1.07 .754 .08 .03

Attitude towards BRAIN15 –.149 .81 .82 .739 .00 .16

Grade Gender Grade x Gender

Relationship of Academic Level and Gender on Students’ Attitudes toward Science

Table 4. The correlation coefficients (r) for every variable.

Component I Component II Primary Middle

Female Male Female Male

Normative Beliefs .528** .405** .458** .454**

Control Beliefs Attitude toward Behavior .318** .456** .397** .342**

SEM (Structural Equation Modeling)

Figure 2. Structure Equation Modeling of attitude toward science by gender

Table 5. Standardized β values for each path compared by gender.

Academic Level à Control Beliefs –.24 <.001 –.21 <.001 .625

Conclusions and Implications

Received: May, 2019 Accepted: September 10, 2019

Abstract. Recently, the research about Branko Andjić,

Table 1. Demographic characteristics of the sample (Total N=100).

The research was divided into the following phases:

Description of the DDK and the DPK

Figure 1. Example of simple determination through the DDK.

POT E1 .188 50 .002 .985 50 .002

Woody .745 .082 .017 .835 .071 .012

The Opinions of the NV participants about the Applied DKs

Factor N Mmin Mmax M SD α

The opinion of the NV participants on the contribution of the applied DK to their

Conclusions and Implications

Received: June 06, 2019 Accepted: September 18, 2019

EXPLORING THE ASPECT ISSN 2538-7138 /Online/

Existing Scientific Literacy Framework developed by Boujaoude

Table 1. Scientific Literacy Framework developed by Boujaoude (2002).

Aspect 2. The investigative nature of science:

Aspect 3. Science as a way of knowing:

Aspect 4. Interaction of science, technology, and society:

The Unit of Analysis: The Turkish Science Curriculum at a Glance

Normative Beliefs .528 .405 .458 .454

Control Beliefs Attitude toward Behavior .318 .456 .397 .342