Journal of Baltic Science Education, Vol. 18, No. 1, 2019

Vol.18, No.
1, 2019
ISSN 1648-3898 /Print/
ISSN 2538–7138 /Online/
The International Journal of the Scientia Socialis Ltd., & SMC “Scientia Educologica”
Journal of Baltic Science Education, Vol. 18, No. 1, 2019
Editorial Board
ISSN 1648–3898 /Print/
Editor-in-Chief ISSN 2538–7138 /Online/
Prof., Dr. Vincentas Lamanauskas Siauliai University, Lithuania
Deputy Editor-in-Chief
Prof., Dr. Andris Broks University of Latvia, Latvia
Deputy Editor-in-Chief
Prof., Dr. Miia Rannikmäe University of Tartu, Estonia
Executive Secretary
Dr. Laima Railienė SMC “Scientia Educologica”, Lithuania
Editors:
Prof., Dr. Boris Aberšek University of Maribor, Slovenia
Prof., Dr. Agnaldo Arroio University of Sao Paulo, Brazil
Prof., Dr. Martin Bilek Charles University, Czech Republic
Dr. Paolo Bussotti University of Udine, Italy
Prof., Dr. Hana Čtrnáctová Charles University, Czech Republic
Dr. Peter Demkanin Comenius University, Slovakia
Dr. André du Plessis Nelson Mandela University, South Africa
Prof., Dr. Jack Holbrook University of Tartu, Estonia
Prof., Dr. Ryszard M. Janiuk Maria Curie Sklodowska University, Poland
Dr. Milan Kubiatko University of Zilina, Slovakia
Dr. Todar Lakhvich Belarusian State Medical University, Republic of Belarus
Prof., Dr. Jari Lavonen University of Helsinki, Finland
Dr. Rita Makarskaitė-Petkevičienė Vilnius University, Lithuania
Prof., Dr. Aadu Ott Göteborg University, Sweden
Prof., Dr. Paul Pace Malta University, Malta
Prof., Dr. Valfrids Paškevičs Daugavpils University, Latvia
Prof., Dr. Jongwon Park Chonnam National University, Korea
Prof., Dr. Raffaele Pisano Lille 3 University, France
Prof., Dr. Pavol Prokop Trnava University, Slovakia
Prof., Dr. Konstantinos Ravanis University of Patras, Greece
Dr. Alona Rauckienė-Michaelsson Klaipėda University, Lithuania
Dr. Dušica Rodić University of Novi Sad, Serbia
Prof., Dr. Heimo Saarikko Helsinki University, Finland
Dr. Hae-Ae Seo Pusan National University, Korea
Dr. Uladzimir K. Slabin University of Oregon, USA
Prof., Dr. Borislav V. Toshev Sofia University, Bulgaria
Dr. Georgios Tsaparlis University of Ioannina, Greece
Dr. Muhammet Usak Science and Medical Education Research Center, Turkey
A scientific journal JBSE issued by the Scientia Socialis Ltd., in cooperation with SMC Scientia Educologica, Lithuania,
emphasizes theoretical, experimental and methodical studies in the field of science education. JBSE is an international
academic journal. In order to maintain the high standards appropriate to such a journal, all contributions received are
submitted for anonymous review by two experts, additionally to review by the Editor. The decision of the Editor on
the acceptance of articles is final and no correspondence can be entered into on reasons for rejection of a submitted
contribution.
Published since 2002 Address:

The journal is published bimonthly. Scientia Socialis, Ltd.
IF: 0.638 (2017) Donelaičio Street 29, LT-78115 Siauliai, Lithuania
H Index: 10 (2016) E-mail: mail.jbse@gmail.com
SJR: 0.326 (2017) Phone: +370 687 95668
ICDS: 10.7 (2017) Home page: http://www.scientiasocialis.lt/jbse/
ICV: 143.26 (2015)
ISSN 1648–3898 (Print) © Scientia Socialis Ltd., Lithuania, 2018
ISSN 2538–7138 (Online)

The articles appearing in this journal are indexed/abstracted in British Education Index
(http://www.leeds.ac.uk/bei/bei.htm), Copernicus Index (http://journals.indexcopernicus.com), The Asian
Education Index (http://www.asian-education-index.com/sciences_index.php), EBSCO: Academic Search Premier
2 (http://search.ebscohost.com), Social Scisearch (Thomson Reuters) (http://science.thomsonreuters.com/index.html),
Journal Citation Reports / Social Sciences Edition (Thomson Reuters)
(http://thomsonreuters.com/products_services/scientific/Journal_Citation_Reports),
and SCOPUS (http://www.scopus.com)
ISSN 1648–3898 /Print/ CONTENTS
CONTENTS
Editorial
PROPOSALS FOR SMALL STEPS TOWARD REPRODUCIBILITY OF SCIENCE EDUCATIONAL

STUDIES
Andrej Šorgo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Articles
AGRICULTURAL STUDENTS’ KNOWLEDGE OF PHOTOSYNTHESIS AND THE CONTEXTUAL

FACTORS THAT INFLUENCE IT
Katja Gobec, Jelka Strgar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
THE IMPACT OF A SCIENCE METHODS COURSE ON FEMALE PRE-SERVICE TEACHERS’

BELIEFS WITHIN A SAUDI-ARABIAN CONTEXT
Amani K. Hamdan Alghamdi, Sun Young Kim . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
A STUDY OF FACTORS AFFECTING SCIENCE ACHIEVEMENTS OF JUNIOR HIGH SCHOOL

FEMALE STUDENTS
Chun-Chu Liu, Tse-Yu Wang . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
THE EFFECTS OF INQUIRY-BASED LEARNING AND LEARNING STYLES ON PRIMARY

SCHOOL STUDENTS’ CONCEPTUAL UNDERSTANDING IN MULTIMEDIA LEARNING
ENVIRONMENT
Dek Ngurah Laba Laksana, I Wayan Dasna, I Nyoman Sudana Degeng . . . . . . . . . . . . . . . . . . . . . 51
DEVELOPMENT AND VALIDATION OF A SURVEY INSTRUMENT (AKA) TOWARDS ATTITUDE,

KNOWLEDGE AND APPLICATION OF STEM
Bevo Wahono, Chun-Yen Chang . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
A FEASIBLE GUIDANCE FOR ORDERED MULTIPLE-CHOICE ITEMS IN STUDENTS’

HIERARCHICAL UNDERSTANDING LEVELS
King-Dow Su . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
IDENTIFICATION OF THE INTEREST OF TURKISH MIDDLE-SCHOOL STUDENTS IN STEM

CAREERS: GENDER AND GRADE LEVEL DIFFERENCES
Ayşegül Ergün . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
TEACHERS’ SELF-CONCEPT AND ITS BENEFITS FOR SCIENCE EDUCATION

Jelena Ž. Maksimović, Jelena S. Osmanović . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
THE IMPLEMENTATION OF MIND MAPS IN TEACHING PHYSICS: EDUCATIONAL

EFFICIENCY AND STUDENTS’ INVOLVEMENT
Zvezdan Z.Gagić, Sonja J. Skuban, Branka N. Radulović, Maja M. Stojanović, Olivera Gajić . . . . . . . . 117
ROLE AND MEANING OF FUNCTIONAL SCIENCE, TECHNOLOGICAL AND ENGINEERING

LITERACY IN PROBLEM-BASED LEARNING
Zvonka Cencelj, Metka Kordigel Aberšek, Boris Aberšek, Andrej Flogie . . . . . . . . . . . . . . . . . . . . 132
Information
INFORMATION FOR CONTRIBUTORS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
3
PROPOSALS FOR SMALL STEPS TOWARD REPRODUCIBILITY OF SCIENCE

EDUCATIONAL STUDIES
Andrej Šorgo
University of Maribor, Slovenia
It is evident that almost nobody is satisfied with contemporary (science) education, however, critique and
proposed solutions to the perceived educational problems are sometimes established from opposite reasoning.
Nowadays we can witness, for example, debates about position, role and effect of digital mobile technologies, so-
cial networks, and many other issues on behaviour and performance of students. Suggestions about their place in
science education are on a scale between transforming education toward digital technologies and calls to prohibit,
at least some of them, on the other end. Unfortunately enough argumentation too many times follows patterns
recognized as ‘The demon-haunted world’ (Sagan, 1995) and a vocabulary and argumentation in line with Frankfurt’s
(2005) famous essay ‘On bullshit’.
Optimally, argumentation for or against changes and follow up interventions should be empirically validated
in a way, that facts are clearly separated from opinions. Common sense is that the crucial part in the process toward
evidence-based interventions should be delegated to the researchers. However, it seems that they too many times
forget to provide enough clues for their primary research to be understood and their findings to be verified, mak-
ing their findings irreproducible. This does not mean that researchers are intentionally publishing falsified findings,
however “Humans are remarkably good at self-deception.” (Nuzzo, 2015, p. 182). Therefore, reproducibility should
be the key quality of empirical research.
There are many good reasons, why published studies should not be regarded as a one-time event but as a
brick in a cathedral of knowledge. The sentence, “No research paper can ever be considered to be the final word,
but there are too many that do not stand up to further study” cited from the collection of articles published under
an umbrella title “Challenges in irreproducible research” (Nature, n. d.) points to the problem. As an example, we can
once again consider educational technologies, which had substantially changed science education and our private
lives in last decades, as well. Therefore, findings and conclusions presented in pioneering papers are not necessarily
valid anymore, however, methodology, if properly presented, can allow repetition of the study. The same is true for
comparative studies, because something valid for one culture or population is not necessary valid under different
conditions. Additionally, statistics is a very vivid research field and with greater computer power allows procedures
which were not available at the time of the study or are out of the statistical horizons of a researcher. Repetition is,
however, not possible without access to the primary data.
I would like to present a short list of simple procedures that can transfer published papers from gravel in a
pathway toward to a keystone in an arch of knowledge. As a reader and potential user of your valuable research, I
would greatly appreciate that your papers follow at least majority of the listed:
•• Clearly concept your research questions in a way that they can be reused by yourself and the others in
follow up studies.
•• Hypotheses should not be presented as educated guesses about expected outcomes, but should be
regarded as research questions translated in equations presented by words and/or symbols, to be tested
by one or the other test.
•• It should be known who or what constitute your population (e.g. number of all secondary students of
particular age in a country) by providing exact population numbers and qualitative and quantitative
description of traits and population strata considered in a study.
4
https://doi.org/10.33225/jbse/19.18.04
•• Describe in enough details how well your sample represents your population, and what are sources of
possible bias.
•• Provide enough details of sampling procedure and data collection. Provide assumptions what can be
potential and actual sources of bias.
•• In some or the other way provide in appendix, supporting materials and/or link to the permanent page
access to the research instruments.
•• Provide permanent access to your primary data even after you have achieved your degree, have moved
to another institution or went retired.
•• Report statistical parameters in enough details to allow inclusion of your findings in secondary studies.
At least what you should provide in empirical educational studies is listed below:
•• For each described and tested group provide the exact sample size (n) as a discrete number.
•• Provide units for assessment for each condition and description of all applied covariates.
•• Provide indication of any assumption connected to your variables (normality, skewness, and
similar) and corrections (exclusion of outliers, handling with missing data, transformations, and
similar).
•• Ensure that the error bars are defined and state explicitly what they represent throughout the
figures (e.g. confidence intervals, standard deviation, standard error, range, etc.).
•• Provide a full descriptive statistics of each variable by inclusion of frequencies, measures of
central tendency (means, mode), variation (e.g. standard deviation) or associated estimates of
uncertainty (e.g. confidence intervals).
•• Provide names of all applied statistical test(s) and state whether they are one- or two-sided.
•• Provide outcomes of the test statistic by reporting exact numbers (e.g. F, U, Z, t, r) with confidence
intervals, effect sizes, degrees of freedom and p values.
•• Report exact p-values even if they are statistically non-significant whenever relevant.
•• When applicable present multilevel and complex models as graphics.
•• Provide estimates of effect sizes (e.g. Cohen’s d, Pearson’s r) with confidence intervals and refer-
ence to the calculations.
•• When reporting exploratory factorial analyses provide information about the initial set of items,
excluded items, and reasoning for retaining factors (parallel analysis), together with estimates
of reliability.
•• Order items in tables in some logical order.
•• Conclusions should be answers to the research questions.
•• Provide assumptions of limitations of the study.
Following this list of suggestions cannot guarantee that your paper is published, because nothing can replace
evidence of comprehensive background research, creativity in producing attractive ideas, and formal reasoning in
defending them. However, you should not be surprised if a number of reviewer’s recommendations will drop, and
the number of downloads and papers citing your published work is going to climb.
References
Frankfurt, H. G. (2005). On bullshit. Princeton, New Jersey. Princeton University Press.

Nature (n. d). Challenges in irreproducible research. Retrieved from https://www.nature.com/collections/prbfkwmwvz.
Nuzzo, R. (2015). How scientists fool themselves–and how they can stop. Nature News, 526 (7572), 182.
Sagan, C. (1995). The demon-haunted world: Science as a candle in the dark. New York. Random House.
Received: January 08, 2019 Accepted: January 31, 2019
Andrej Šorgo PhD, Associate Professor of Biology Didactics, University of Maribor,

Koroška cesta 160, 2000 Maribor, Slovenia.
E-mail: andrej.sorgo@um.si
5
AGRICULTURAL STUDENTS’
KNOWLEDGE OF
PHOTOSYNTHESIS AND THE
ISSN 2538-7138 /Online/
CONTEXTUAL FACTORS THAT
INFLUENCE IT
Abstract. This research aimed to assess Katja Gobec,

the knowledge of photosynthesis among
Jelka Strgar
students in agricultural education and
understand the influence of the contextual
factors on it. In photosynthesis, biomass is
produced, so understanding these process-
es is essential for successful agriculture. The
research sample consisted of 330 students
Introduction
in agricultural education in Slovenia. Data
were collected with a knowledge test and a
The process of photosynthesis is a topic about which students struggle
contextual questionnaire. Better knowledge
to comprehend adequately. According to previous research, students have
of photosynthesis was positively related
various misconceptions, misunderstandings, and naive and alternative
with higher awareness of the importance
conceptions about it (Waheed & Lucas, 1992). The same or similar problems
of biology as a science and of photosyn-
in knowledge of photosynthesis were reported at all educational levels:
thesis. There was no significant relation
elementary school (Skribe-Dimec & Strgar, 2017; Tekkaya & Balci, 2003),
between the knowledge of photosynthesis
secondary school (Barak, Sheva, & Gorodetsky, 1999; Jin & Anderson, 2012;
and positive attitudes toward biology
Lin & Hu, 2003; Svandova, 2013), pre-service teachers (Brown & Schwartz,
lessons. The average achievement on the
2009; Harman, 2012; Tekkaya, Capa, & Yilmaz, 2000), and practising teachers
knowledge test of photosynthesis was 56%:
(Domingos-Grillo, Reis-Grilo, Ruiz, & Mellado, 2012; Lenton & McNeil, 1993).
the highest on the item about energy and
Biology topics concerning cells, organelles, physiological and biochemical
the lowest on the item about the primary
processes, and transformation of energy present difficulties to many students
purpose of photosynthesis. Between these
(Lazarowietz & Penso, 1992; Marmaroti & Galanopoulou, 2006). The chemistry
two achievements was the knowledge of
of photosynthesis was perceived as one of the most difficult topics in school
reactants and products in photosynthesis,
biology (Johnstone & Mahmoud, 1980).
and the knowledge concerning chlorophyll.
Marmaroti and Galanopoulou (2006) have stated that photosynthesis is
Considering these results, improvements
one of the most essential of the processes that are indicators of life in plants,
should be made in teaching. For this
and that it has numerous different features in comparison to other biochemi-
population, more experimental and less
cal processes. This makes photosynthesis a central subject of biology courses
theory-based lessons were suggested. The
at all levels, especially at the secondary level of education.
findings could benefit biology and science
On completion of compulsory primary education, typically at the age of
teachers in agricultural education and help
15, Slovenian students continue education in secondary school and choose
improve national curriculum for biology in
from among programmes of either general, vocational, or technical educa-
agricultural programmes.
tion. Over 60% of students enrol in vocational or technical programmes.
Keywords: agricultural education, biology
In Slovenia, students in secondary agricultural education are enrolled
education, contextual factors in education,
either in a three-year vocational programmes of Country Farmer and Gardener
knowledge of photosynthesis
or four-year technical programmes of Agricultural-Entrepreneurial Technician
and Horticultural Technician. In vocational programmes, all biology content
Katja Gobec is integrated into the subject of Science. Most of the learning objectives in
Vocational College, School Centre Šentjur, the Science syllabus include biology content, and students have to acquire
Slovenia basic knowledge of ecosystems, physiology, genetics, and plant nutrition
Jelka Strgar
University of Ljubljana, Slovenia and be aware of the significance of plants (Poberžnik, Skvarč, Verovnik, &
Vičar, 2007). In these two educational programmes, many biology topics are
6
ISSN 1648–3898 /Print/ AGRICULTURAL STUDENTS’ KNOWLEDGE OF PHOTOSYNTHESIS AND THE CONTEXTUAL
ISSN 2538–7138 /Online/ (P. 6-18)
also integrated into various agricultural subjects. In technical programmes, there is a separate subject of Biology;
according to its syllabus, students should acquire general and fundamental knowledge to be able to understand
the laws of nature and perform their work in plant, animal, horticultural, and floricultural production and supple-
mentary activities. Photosynthesis is a compulsory part of their syllabus in topics about understanding the basic
concepts of life and ecological processes (Vičar, 2007).
Different teaching methods, models, and approaches impact students’ attitudes toward biology; and attitudes
influence students’ knowledge (George, 2000). Ajzen (2001) has explained how the conceptualisation of attitude
and its formation and activation are correlated with peoples’ accessible beliefs, and what the influence of affective
versus cognitive processes in the formation of attitudes is. The development of attitudes starts in primary school.
If students do not have positive experiences there, they tend to avoid science in their later education, resulting in
a lack of knowledge and commitment to science (Simpson & Oliver, 1990).
Attitudes toward science, and biology specifically, are well studied from different perspectives. A positive
correlation between students’ attitudes toward biology and their achievements and process skills performance
has been reported (Kubiatko, 2012; Nasr & Soltani, 2011; Owino, Yougungu, Ahmed, & Ogolla, 2015; Shaheen &
Kayani, 2017). In Slovenia, however, 15-year-olds have shown good knowledge of biology but a poor attitude to-
ward it (Štraus, Šterman Ivančič, & Štirn, 2016). Hansen and Birol (2014) have found that high-performing biology
programme students have more expert-like attitudes than low-performing students do.
A large amount of attitudinal research has focused on gender, age or year of study, either deliberately or as
a side finding. Some research has shown that girls have a significantly greater interest in biology than boys do as
well as more positive attitudes toward biology (Prokop, Prokop, & Tunnicliffe, 2007). The majority of studies found
no difference between boys and girls in attitudes toward biology (Nasr & Soltani, 2011; Šorgo & Špernjak, 2009;
Zeidan & Jayosi, 2015) or their attitudes toward plants (Fančovičová & Prokop, 2010). According to Šorgo and
Špernjak (2009), different age classes do not differ in the general directions of their expressed attitudes; they like
or dislike the same things, but they differ in the strength of their attitudes (p.131).
Students’ attitudes toward biology can be affected by different variables; in the case of plants, some students
lack positive attitudes toward them, but if the family has a garden, more positive attitudes toward plants emerge
(Fančovičová & Prokop, 2010). In Slovenia, it has been discovered that students in vocational and technical sec-
ondary education value biology more than students in general secondary education do (Šorgo & Špernjak, 2009).
Agriculture is a science that includes biology along with other sciences (Baird, Lazarowitz, & Allman, 1984;
Terry & Torres, 2006).) Agricultural and natural resources should be used to teach biology and incorporate agricul-
tural concepts into plant science (Connors & Elliot, 1995). Teachers have agreed that integrating more biology in
agricultural education would assist in students’ better understanding of science concepts and their application
(Mayer, 1987).
Problem of Research
National aims are to raise the level of food self-sufficiency and the number of farms whose agricultural pro-
duction is sustainable (Hrustel Majcen & Paulin, 2001; Strategija razvoja Slovenije, 2005). This requires good basic
knowledge of biology in farmers, especially of the process of photosynthesis which is at the basis of agricultural
production because it connects the living and non-living worlds (Waheed & Lucas, 1992). It also requires the
awareness of farmers about the importance of photosynthesis. According to Mayer (1987), a significant amount
of agriculture is conducted without any knowledge of the biological principles involved. In Slovenia, most future
farmers finish some level of agricultural education, which includes biology either as a separate subject or as biology
topics that are part of other agronomical subjects. Previous research indicated that students of different ages find
photosynthesis difficult to understand and have inadequate knowledge of it (Domingos-Grillo, Reis-Grilo, Ruiz, &
Mellado, 2012, Marmaroti & Galanopoulou, 2006; Özay & Öztaş, 2003; Svandova, 2014).
Research Focus
The main aim of this research was to explore the level of knowledge of photosynthesis among Slovenian
students in agricultural education. Research also focused on contextual factors affecting the knowledge of stu-
dents’: their attitudes toward photosynthesis, the importance of biology, and biology lessons and learning habits
of students. The following research questions were formulated:
7
(P. 6-18) ISSN 2538–7138 /Online/
•• How well do students in agronomical education understand the process of photosynthesis?

•• Is there any relation between the students’ knowledge of photosynthesis and the selected contextual
factors?
Methodology of Research
General Background
The research design was quantitative. Descriptive and correlational methods were used. Knowledge of
photosynthesis of Slovenian students in secondary agricultural education and contextual factors affecting their
knowledge were explored by administering a knowledge test and a contextual questionnaire. The research was
conducted in spring 2017 in three secondary agricultural schools in Slovenia.
Sample
The total number of participants is 330, which is 35% of all students enrolled in four secondary agricultural
programmes in Slovenia (Table 1). Included are all students who attended classes when this research was conducted.
They are students of all three grades in the vocational programmes of Country Farmer (10.6% of total participants)
and Gardener (19.8%) and students in all four grades in the technical programmes of Agricultural-Entrepreneurial
Technician (49.9%) and Horticultural Technician (19.7%). The Country Farmer and Agricultural-Entrepreneurial
Technician programmes are focused on farming while the Gardener and Horticultural Technician programmes are
focused on horticultural production. The proportion of female adolescents among participants is 51.5%; 66.1% of
participants come from a home with a farm.
The research was anonymous; all the parents of the students signed an agreement to allow their children to
participate in it.
Table 1. Distribution of participants according to the educational programme and year of study.
Participants
Educational programme
Year Age N f (%)
1 15 10 3.1
Country Farmer 2 16 13 3.9

3 17 12 3.6
1 15 25 7.6
Gardener 2 16 22 6.8
3 17 18 5.4
1 15 41 12.4
Agricultural-Entrepreneurial 2 16 51 15.4
Technician 3 17 39 11.8
4 18 34 10.3
1 15 18 5.4
2 16 17 5.2
Horticultural Technician
3 17 16 4.9
4 18 14 4.2
Total 330 100.0
8
ISSN 2538–7138 /Online/ (P. 6-18)
Instrument and Procedures
Data were collected using a set of demographic questions (educational programme, year of study, gender,
and farm at home), knowledge test about photosynthesis, and a contextual questionnaire about students’ at-
titudes toward photosynthesis, the importance of biology, and biology lessons and their learning habits.
The knowledge test contained nine multiple choice items. Participants had to choose one answer out of
three to five. The same items were used in research by Domingos-Grilo, Reis-Grilo, Ruiz, and Mellado (2012; ‘What
is the main purpose of photosynthesis?’), Marmaroti and Galanopoulou (2006; ‘Which pair of substances are the
reactants in photosynthesis?’, ‘Which substances are products of photosynthesis?’, ‘Which parts of the plant con-
tain chlorophyll?’, and ‘In which parts of the plant does photosynthesis take place?’), Çepni, Taș, and Köse (2006;
‘Into what type of energy do plants transform sunlight energy?’). An item used in a survey by Skribe-Dimec and
Strgar (2017; ‘What type of energy do plants receive from the sun?’) was added. Two pairs of items were compiled
into two new items, which acted as control items: ‘In which parts of the plant does photosynthesis take place,
and which parts of the plant contain chlorophyll?’, and ‘Which pair of substances is needed in photosynthesis,
and which pair of substances are the products of photosynthesis?’).
With multiple choice items, the following four conceptions in connection with photosynthesis were evalu-
ated:
1) The function of photosynthesis (item F - ‘What is the main purpose of photosynthesis?’).
2) Photosynthesis and energy (item E1 - ‘What type of energy do plants receive from the sun?’, and item
E2 - ‘Into what type of energy do plants transform sunlight energy?’).
3) The process of photosynthesis, (item P1 - ‘Which pair of substances are the reactants in photo-
synthesis?’, item P2 - ‘Which substances are the products of photosynthesis?’, and item P3 - ‘Which
pair of substances is needed in photosynthesis, and which pair of substances are the products of
photosynthesis?’).
4) Chlorophyll (item C1 - ‘In which parts of the plant does photosynthesis take place?’, item C2 - ‘Which
parts of the plant contain chlorophyll?’, and item C3 - ‘In which parts of the plant does photosynthesis
take place, and which parts of the plant contain chlorophyll?’).
The choice of items was based on the fact that biomass is produced in photosynthesis, so understanding
these processes is essential for successful agriculture.
The contextual questionnaire contained 22 statements assessing students’ attitudes toward photosynthesis,
the importance of biology, and biology lessons and learning habits of the students. Most of these statements were
adapted from the questionnaire used in The Relevance of Science Education study (The ROSE questionnaire, n.d.).
The contextual questionnaire used a 5-point Likert scale, ranging from strongly disagree (1) to strongly agree (5).
After each of the three agricultural schools confirmed their participation in the research, copies of tests and
questionnaires were delivered to them. Instructions for biology teachers were given in person and in writing.
Students completed the set of demographic questions, the knowledge test, and the contextual questionnaire
during their regular lessons of biology or science in approximately 20 to 25 minutes.
Data Analysis
For the knowledge test of photosynthesis, frequencies were calculated. Data were tested for normal distribu-
tion with the Kolmogorov-Smirnov test; considering that the distribution of data was not normal, non-parametric
testing was used. A chi-square test was used to identify statistically significant differences in the knowledge of
photosynthesis between students of different genders, years of study, educational programmes, and whether
they come from a farm or not. Post hoc pairwise comparisons were made using z-test and Bonferroni correction.
Data from the contextual questionnaire were subjected to Principal Component Analysis (PCA). The oblique
rotation (direct oblimin) method with Kaiser normalisation was conducted on 22 items. The value of the Kaiser-
Meyer-Olkin measure of sampling adequacy was .920, which means that the sample size was adequate for PCA.
Bartlett’s test of sphericity was highly significant (χ2 = 3021.279, df = 231, p < .001), indicating that correlations
between items were sufficiently large for PCA.
According to PCA, four components can be extracted. Seven items were loaded onto component 1 (positive
attitudes toward biology lessons); seven items were loaded onto component 2 (importance of biology); three
items loaded onto component 3 (positive attitudes toward photosynthesis), and five items loaded onto compo-
9
(P. 6-18) ISSN 2538–7138 /Online/
nent 4 (good learning habits). Cronbach’s alphas for three components were between .77 and .87 while for one
component it was .65 (Table 2). These four components explained 55.93% of the variance. Cronbach’s alpha for all
22 items was .92. Correlations between components were calculated using Spearman’s correlation coefficient (rs).
Table 2. Summary of principal component analysis results for the attitude toward biology questionnaire.
Rotated Factor Loading
Statement Component
1 2 3 4
I like the content taught in biology lessons. .857
I am looking forward to biology lessons. .800
I am very interested in the content of biology. .784
I understand everything in biology lessons. .668
I like the content I learn in biology. .550
I think it is important to learn the contents discussed in biology. .549
I finish my homework on time. .445
Progress in biology improves the quality of our lives. .731
Biology is useful in everyday life. .722
What I learn in biology lessons will benefit my everyday life. .718
I find it important to understand the contents of biology lessons. .655
Biology is important for society. .586
Everyone needs the basic biological knowledge. .564
Due to my biology lessons, I understand the world around me better. .458
Understanding photosynthesis will benefit my life. .778
Photosynthesis is interesting. .756
Understanding the essence of photosynthesis is important for general education. .666
I usually make a work plan for each day for what I have to do for school. .818
I make myself learn even when I could do some more interesting things. .751
I make myself work for school. .546
In the lessons, I always concentrate on school subjects. .528
I arrange my space, so I can learn without disturbances. .513
Eigenvalue 8.09 1.89 1.25 1.08
% of variance 36.77 8.59 5.68 4.89
α .87 .84 .65 .77
Number of items 7 7 3 5
Note. 1=Positive attitudes toward biology lessons, 2=Importance of biology, 3=Positive attitudes toward photosynthesis, 4=Good
learning habits.
10
ISSN 2538–7138 /Online/ (P. 6-18)
Results of Research
Students’ general Achievements on the Knowledge Test about Photosynthesis
According to the results of the knowledge test about photosynthesis (Table 3), which contained nine multiple
choice items, all participating students achieved an average of 55.9% of correct answers. In the part about energy,
which was checked by two items (E1, E2), students reached an average of 67.6% of the correct answers for those
two items. In the part about photosynthesis as a process, which was checked by three items (P1, P2, P3), students
achieved an average of 63.9% of correct answers for those three items. With the part about chlorophyll, which
was checked by three items (C1, C2, C3), students achieved an average of 51.0% of correct answers for those three
items. The content part of the function of photosynthesis was checked by one item (F), and the achievement of
the students here was 23.3%.
The students had the lowest achievements on item F (‘What is the main purpose of photosynthesis?’; 23.3%)
and item C3 (‘In which parts of the plant does photosynthesis take place, and which parts of the plant contain
chlorophyll?’; 33.6%). On two items (E2 – ‘Into what type of energy do plants transform sunlight energy?’ and C2
– ‘Which parts of the plant contain chlorophyll?’), the achievements were 56.4% and 57.3%, respectively, which is
slightly below 60.0%, which was a limit for adequate knowledge. For five items, the achievements ranged between
61.2% and 78.8% (P1 – ‘Which pair of substances are the reactants in photosynthesis?’; C1 – ‘In which parts of the
plant does photosynthesis take place?’; P2 – ‘Which substances are the products of photosynthesis?’; P3 – ‘Which
pair of substances are needed in photosynthesis, and which pair of substances are the products of photosynthesis?’,
and E1 – ‘‘What type of energy do plants receive from the sun?) (Table 3).
Table 3. Percentage of students’correct answers to the items on the knowledge test, and statistical significance
of differences between their answers according to gender, year of education, educational programme.
Total Gender Year of education Educational programme Farm at home
Female Male χ2 1. 2. 3. 4. χ2 CF G AET HT χ2 No Yes χ2
Statements f(%) f(%) f(%) p f(%) f(%) f(%) f(%) p f(%) f(%) f(%) f(%) p f(%) f(%) p
F - The main purpose

of photosynthesis is the 23.3 23.4 23.3 .972 25.5 17.5 25.9 27.1 .405 17.1 18.5 23.0 32.3 .208 26.8 21.6 .288
production of glucose.
E1 - Plants receive sunlight

78.8 78.5 79.1 .896 77.7 80.6 78.8 77.1 .950 51.4 70.8 85.5 84.6 < .001 78.6 78.9 .945
energy from the sun.
E2 - Plants transform
sunlight energy into chemi- 56.4 50.6 61.6 .044 50.0 65.0 45.9 68.8 .009 48.6 38.5 60.6 67.7 .003 50.0 59.6 .095
cal energy.
P1 – Carbon dioxide and

water are the reactants in 61.2 66.5 56.4 .061 58.5 74.8 54.1 50.0 .006 22.9 60.0 63.6 76.9 < .001 67.0 58.3 .124
photosynthesis.
P2 – Glucose and oxygen

are the products of photo- 64.5 64.6 64.5 .997 73.4 67.0 50.6 66.7 .013 42.9 55.4 64.2 86.2 < .001 69.6 61.9 .165
synthesis.
P3 - Carbon dioxide and

water are needed in photo-
synthesis and glucose, and 66.1 67.7 64.5 .541 68.1 74.8 57.6 58.3 .055 45.7 49.2 70.3 83.1 < .001 68.8 64.7 .460
oxygen is the products of
photosynthesis.
11
(P. 6-18) ISSN 2538–7138 /Online/
Total Gender Year of education Educational programme Farm at home
Female Male χ2 1. 2. 3. 4. χ2 CF G AET HT χ2 No Yes χ2
Statements f(%) f(%) f(%) p f(%) f(%) f(%) f(%) p f(%) f(%) f(%) f(%) p f(%) f(%) p
C1 - Photosynthesis takes
place in the green parts of 62.1 69.6 55.2 .007 74.5 60.2 63.5 39.6 .001 54.3 67.7 52.7 84.6 < .001 72.3 56.9 .006
the plant.
C2 - Chlorophyll is in the
57.3 58.2 56.4 .737 57.4 62.1 55.3 50.0 .538 51.4 56.9 55.2 66.2 .409 50.0 61.0 .056
green parts of the plant.
C3 - Photosynthesis takes
place in the green parts of
the plant and chlorophyll 33.6 34.2 33.1 .842 34.0 36.9 37.6 18.8 .118 28.6 35.4 23.0 61.5 < .001 34.8 33.0 .744
is in the green parts of the
plant.
Average performance 55.9 57.0 54.9 57.7 59.9 52.1 50.7 40.3 50.3 55.3 71.5 57.5 55.1
Note. CF=Country Farmer, G=Gardner, AET=Agricultural Entrepreneurial Technician, HT=Horticultural Technician; Statistically signifi-
cant values are shown in bold type.
Differences among Students’ Achievements by Gender, Year of Study, and Programme
The average achievement of female adolescents was slightly higher (57.0%) than that of male adolescents
(54.9%) (Table 3). We found two statistically significant differences in answers between genders. Item E2 on energy
was answered more correctly by male adolescents, while item C1 on chlorophyll was more correctly answered by
female adolescents.
The lowest average achievement of students on the knowledge test about photosynthesis was in the fourth
year (50.7%), followed by students in the third year (52.1%), then the first year (57.7%), and the highest was the
achievement of the second-year students (59.9%) (Table 3). Four statistically significant differences among the
answers of students of these four years were found. According to the post hoc test, in most of these four items, the
first-year and/or the second-year students significantly differed from the third-year and/or the fourth-year students.
Comparing students’ knowledge in four different educational programmes the highest average achievements
on the knowledge test of photosynthesis are held by the students in the educational programme of Horticultural
Technician (71.5%), followed by Agricultural-Entrepreneurial Technician students (55.3%), then in Gardener stu-
dents (50.3%); the lowest average exam performance was seen in the Country Farmer programme (40.3%) (Table
3). Students of these four educational programmes answered statistically significantly differently in seven out of
nine items. According to the post hoc test, in almost all of these seven items, the students in the Horticultural
Technician programme differed significantly from the students in the Country Farmer programme. Students in the
Gardener programme and those in the Agricultural-Entrepreneurial Technician programme differed significantly
from those in the Horticultural Technician programme in some items and from those in the Country Farmer pro-
gramme in other items.
The average achievements of the students who have no farm at home were higher (57.5%) than that of the
students coming from a farm (55.1%). There was only one statistically significant difference (Table 3). Students from
a farm had lower achievement on item C1, about chlorophyll.
Correlations between the Students’ Achievements on the Knowledge Test about Photosynthesis and
Contextual Factors that Influence Knowledge
The correlations between the students’ achievements on the knowledge test about photosynthesis and con-
textual factors that influence the knowledge were assessed (Table 4). The contextual factors were the components
12
ISSN 2538–7138 /Online/ (P. 6-18)
we obtained as a result of the PCA (attitudes toward photosynthesis, the importance of biology, and biology lessons
and good learning habits) on a contextual questionnaire that students completed. Students who scored higher
on the positive attitude toward photosynthesis (component 3 of PCA) also scored significantly higher on three
items on the knowledge test: two out of three items concerning chlorophyll (C2, C3), and one out of three items
concerning the process of photosynthesis (P1). All three significant correlations were positive and small (.12 < rs
< .23). Students who scored higher on the importance of biology (component 2 of PCA) also scored significantly
higher on five items on the knowledge test: all three items concerning chlorophyll (C1, C2, C3), and two out of three
items concerning the process of photosynthesis (P1, P3). All five significant correlations were positive and small
(.13 < rs < .20). There was no significant correlation between students’ positive attitudes toward biology lessons
(component 1 of PCA) and their achievement on the knowledge test about photosynthesis. Good learning habits
(component 4 of PCA) correlated significantly to one item concerning energy (E1), and two items concerning the
process of photosynthesis (P2, P3). These correlations were negative and small (-.15 < rs < -.23).
Table 4. Correlations between the students’ achievements on the knowledge test about photosynthesis and
the contextual factors that influence the knowledge.
F1-Positive F3-Positive
F2-Importance F4-Good
Statements attitudes toward attitudes toward
of biology learning habits
biology lessons photosynthesis
F - The main purpose of photosynthesis is the produc-

-.03 -.01 -.01 -.06
tion of glucose.
E1 - Plants receive sunlight energy from the sun. -.08 -.03 -.02 -.15**
E2 - Plants transform sunlight energy into chemical
.02 .08 .09 -.08
energy.
P1 – Carbon dioxide and water are the reactants in
.02 .18** .12* -.51
photosynthesis.
P2 – Glucose and oxygen are the products of photo-
-.07 .07 .04 -,21**
synthesis.
P3 - Carbon dioxide and water are needed in photo-
synthesis and glucose, and oxygen is the products of -.08 .13* .10 -,23**
photosynthesis.
C1 - Photosynthesis takes place in the green parts of
.08 .16** .07 -.03
the plant.
C2 - Chlorophyll is in the green parts of the plant. .07 .15** .23** -.03
C3 - Photosynthesis takes place in the green parts of
the plant and chlorophyll is in the green parts of the .03 .20** .18** -.04
plant.
* p < .05, ** p < .01
Correlations between the Students’ Achievements on the Knowledge Test about Photosynthesis
There were 22 significant correlations between the students’ knowledge of photosynthesis, which was tested
with nine items (Table 5). All of them were positive: fifteen were small (.11 < rs < .29), and seven were medium (.32
< rs < .55).
Items F (‘What is the main purpose of photosynthesis?’) and E1 (‘What type of energy do plants receive from
the sun?’) each correlated to just one item. Six items concerning energy, the process of photosynthesis, and chlo-
rophyll (E2, P1, P3, C1, C2, C3) each correlated to five or six other items. Item P2 (‘Which substances are products
of photosynthesis?’) correlated to all other items. All three items concerning the process of photosynthesis (P1,
P2, P3) inter-correlated. Among items concerning chlorophyll (C1, C2, C3), only item C3 correlated to both of the
other items (C1 and C2) while there was no correlation between C1 and C2. The two items concerning energy (E1
and E2) did not correlate with each other.
13
(P. 6-18) ISSN 2538–7138 /Online/
Table 5. Correlations between the students’ achievements on the knowledge test about photosynthesis.
Statements F E1 E2 P1 P2 P3 C1 C2
F - The main purpose of photosynthesis is the production of glucose. 1.00
E1 - Plants receive sunlight energy from the sun. .04 1.00
E2 - Plants transform sunlight energy into chemical energy. -.04 .04 1.00
P1 – Carbon dioxide and water are the reactants in photosynthesis. .03 .09 .14* 1.00
P2 – Glucose and oxygen are the products of photosynthesis. .11* .16** .18** .41** 1.00
P3 - Carbon dioxide and water are needed in photosynthesis and

.03 .10 .22** .55** .49** 1.00
glucose and oxygen are the products of photosynthesis.
C1 - Photosynthesis takes place in the green parts of the plant. .00 .02 .16** .25** .28** .35** 1.00
C2 - Chlorophyll is in the green parts of the plant. .07 -.01 .17** .29** .24** .26** .08 1.00
C3 - Photosynthesis takes place in the green parts of the plant and

.08 .09 .23** .32** .27** .28** .45** .46**
chlorophyll is in the green parts of the plant.
* p < .05, ** p < .01
Discussion
The first research question addressed in this paper has been how well the students in agricultural education
understand the process of photosynthesis. The average achievement of all participants on the knowledge test
about photosynthesis has been slightly under 60%, which was set as the limit of adequate knowledge. This is
in line with the previous research that showed many difficulties in learning this process at all educational levels.
The first item in the knowledge test was focused on the primary purpose of photosynthesis (item F). Less than
one quarter (23.3%) of students in agricultural education answered that the primary purpose of photosynthesis
is the formation of glucose. The majority of students held the idea that the main function of photosynthesis is the
formation of oxygen. As Skribe-Dimec and Strgar (2017) have pointed out, defining photosynthesis’ main function
is difficult. In evolution, the primary function of photosynthesis was the formation of glucose, and oxygen was
a by-product. However, for aerobic organisms, both products are equally important. In their research conducted
on the sample of elementary school students in Slovenia, more than one third (37.7%) answered that the main
purpose of photosynthesis is the production of glucose. This suggests that elementary school students (aged 11-
14) have a better understanding of the production of glucose in photosynthesis than agricultural students (aged
15-18) do. Others have also reported that students more often relate photosynthesis with oxygen and much more
rarely with glucose (Anderson, Sheldon, & Dubay, 1990; Svandova, 2014). This could be due to the predominant
presentation of photosynthesis, which stresses only both gasses (oxygen and carbon dioxide) but not also glucose
(Cañal, 1999). Carlsson (2002) has stated that between the ideas of energy flow and transformation of the matter
in the ecosystems, the latter is crucial for the more complex understanding of photosynthesis, and understanding
photosynthesis is necessary for understanding ecosystems. For sustainable agriculture, farmers should be well
aware of the circulation of matter in ecosystems.
The results related to the energy in photosynthesis (items E1 and E2) have shown that students in agricultural
education know very well that plants receive light energy from the sun (78.8%). The second most frequent answer
was heat. Anderson, Sheldon, and Dubay (1990) have come to similar conclusions. In other research (Marmaroti
& Galanopoulou, 2006; Waheed & Lucas, 1992), the frequency of students who answered light energy has been
similar to those who answered heat.
Over half of the participants in this research (56.4%) have answered correctly that in photosynthesis light
energy is transformed into chemical energy. Less frequent answers have been that light energy is transformed
into light or heat. According to these results and other reports (Carlsson, 2002; Eisen & Stavy, 1988; Marmaroti &
Galanopoulou, 2006; Waheed & Lucas, 1992), energy transformation in photosynthesis presents a greater problem.
The results related to chlorophyll (items C1, C2, and C3) have shown that 62.1% of students in agricultural
14
ISSN 2538–7138 /Online/ (P. 6-18)
education know that photosynthesis takes place in the green parts of the plant and 57.3% know that the green
parts of the plants contain chlorophyll. However, only a third (33.6%) has answered correctly on the combined
item, i.e. that photosynthesis takes place in the green parts of the plant where chlorophyll is. This leads to the
conclusion that participants poorly comprehend the role of chlorophyll in photosynthesis even though they
know that chlorophyll is a necessary constituent of photosynthesis. In the present and other research (Marmaroti
& Galanopoulou, 2006; Özay & Öztas; 2003), students frequently answered that photosynthesis takes place in
the leaves and that chlorophyll is in the leaves. According to Skribe-Dimec & Strgar (2017), the reason for these
answers might be textbooks that typically present photosynthesis with a picture of a plant leaf. The knowledge of
students in this research (aged 15-18) has been considerably higher in comparison to that reported by Marmaroti
and Galanopoulou (2006). At the same time, it was comparable with the knowledge of Slovenian students (aged
11-14) in research by Skribe-Dimec & Strgar (2017).
As a result of items about reactants and products of photosynthesis (items P1, P2, and P3), very similar pro-
portions of students in agricultural education have correctly recognised reactants and products (61.2%-66.1%).
According to Marmaroti and Galanopoulou (2006), students find it easier to understand what is produced in a
chemical reaction than what enters it, but the results of the present research are not in line with that conclusion.
Many authors have reported that students have difficulties in understanding the inputs and outputs of photosyn-
thesis as well as the process itself (Anderson, Sheldon, & Dubay, 1990; Eisen & Stavy, 1988; Griffard & Wandersee
2001; Harman, 2012; Haslam & Treagust, 1987).
In this research, only two statistically significant differences in the knowledge test about photosynthesis
considering gender have been observed. The first has been on the item ‘In which parts of the plant does photo-
synthesis take place?’, for which female adolescents’ achievement was higher, and on the item ‘Into what type of
energy do plants transform sunlight energy?’, for which male adolescents’ achievement was higher. Our results
have been in accordance with the findings of Baram-Tsabari and Yarden (2011) and Jones, Howe, and Rua (2000),
which have shown that male students have greater affinity for physics while female students have greater affinity
for biology. Svandova (2014) has also discovered better knowledge in male students in her research of photosyn-
thesis. In contrast, in research by Haslam and Treagust (1987), the knowledge of photosynthesis of female and
male students has been comparable.
Four statistically significant differences in the knowledge test about photosynthesis among answers of students
of four different years of study have been found. Knowledge of the first- and the second-year students has been
better than that of the third- and the fourth-year students. Significant differences in knowledge of photosynthesis
regarding the year of study have also been reported by Haslam and Treagust (1987) while Svandova (2014) has
not found any.
Students in the four educational programmes participating in this research have answered significantly dif-
ferently to seven out of nine items on the knowledge test about photosynthesis. Students in four-year agricultural
programmes have performed better in comparison to students in three-year programmes. This may be due to the
fact that students who enrol in three-year education have lower academic scores in elementary level of education.
There has been only one significant difference in knowledge of photosynthesis between agricultural students
coming from a farm and those who have no farm at home, which has been in favour of the latter. The expectation
had been that students coming from a farm would possess better knowledge. To better understand the reasons
that this was not the case, other contextual factors should be taken into consideration, such as socio-economic
status and the region in which they live.
The second research question was to determine whether there has been any relationship between the stu-
dents’ knowledge of photosynthesis and the selected contextual factors (recognising the importance of biology,
attitudes toward photosynthesis, attitudes toward biology lessons, good learning habits). Recognising the impor-
tance of biology and positive attitude toward photosynthesis have significantly positively correlated with higher
performance on the knowledge test about photosynthesis, especially on the items concerning chlorophyll and the
process of photosynthesis. In contrast, students’ positive attitudes toward biology lessons have not correlated with
knowledge. The same is true for general Slovenian 15-year olds, in PISA 2015, where Slovenian participants had the
second-to-last average motivation for and interest in learning (Štraus, Šterman Ivančič, & Štirn, 2016). In research
by Svandova (2014) no significant influences between attitudes toward biology and knowledge of photosynthesis
have been found. Usak et al. (2009) and Atik and Erkoç (2015) have reported that positive attitude toward biology
lessons and biology as a subject has been positively related with better knowledge of this subject. The present
research, however, has shown that good learning habits are even negatively correlated with agricultural students’
15
(P. 6-18) ISSN 2538–7138 /Online/
performance on the knowledge test. This result is difficult to explain. The plan is to explore the mentioned negative
correlation further on an additional sample of agricultural students. The finding has contradicted the research of
Duckworth and Seligman (2005) conducted at the middle school level, which had found self-discipline, including
some good learning habits, to be positively correlated with the year of study and standardised test scores. Adodo
and Oyeniyi (2013) and Udeani (2012) have also found significant positive correlations between learning habits
and knowledge among secondary school students.
Conclusions
This research has shown that students in secondary agricultural education in Slovenia do not have adequate
knowledge of photosynthesis. The knowledge has been related with the year of study and very strongly related to
the educational programme. One important finding is that students in the last two years of education possessed
less knowledge than students in the first two years do. This suggests that when they leave school and start their
professional career, their knowledge is not sufficient. Knowledge of photosynthesis has been only weakly related
to gender and the fact that a student comes from a home with a farm. Students with better knowledge of some
aspects of photosynthesis have also had a higher awareness of the importance of biology and more positive at-
titudes toward photosynthesis. In this research, the knowledge of photosynthesis has not been related to positive
attitudes toward biology lessons. There had been a negative relationship between knowledge and good learning
habits; this is difficult to explain and should be investigated further.
In order to improve the agricultural students’ knowledge of photosynthesis, more practical work in biology
classes should be considered. Teaching activities should relate this process to daily life, so students will compre-
hend the relationship between what they learn in class and their experience in practical fieldwork. This could be
also beneficial in strengthening students’ positive attitudes toward biology lessons. Aspects of photosynthesis that
students understand worse could help teachers in an orientation where to start making improvements.
The sample in this research was sufficiently large but limited to two regions in Slovenia where the three schools
that agreed to participate are located, so the results should be treated with some caution.
Because in Slovenia there is not much interest in attending agricultural programmes, knowledge of photo-
synthesis and the attitude of every student in agricultural education matters. These students will professionally
work in plant production and their attitudes toward photosynthesis, and their perception of the importance of
biology could affect their decisions in fieldwork.
Acknowledgements
Sincere gratefulness to Biotechnical Centre Naklo, School for Horticulture and Visual Art Celje and the School
Centre Šentjur for the logistic and supports. Special thanks to the teachers for their help and to all students who
participated in the research.
References
Adodo, S. O., & Oyeniyi, J. D. (2013). Student variables as correlates of secondary school students’ academic performance in
biology. International Journal of Science and Research, 2(7), 386–390.
Ajzen, I. (2001). Nature and operation of attitudes. Annual Review of Psychology, 52, 27-58.
Anderson, C. W., Sheldon, T. H., & Dubay, J. (1990). The effects of instruction on college nonmajors’ conceptions of respiration
and photosynthesis. Journal of Research in Science Teaching, 27(8), 761–776.
Atik, A. D., & Erkoç, F. (2015). The investigation of 9th grade students’ attitudes towards the science and the course of biology in
terms of various variables. Adiyaman University Journal of Educational Sciences, 5(2), 211–244.
Baird, J. H., Lazarowitz, R., & Allman, V. (1984). Science choices and preferences of middle and secondary school students in Utah.
Journal of Research in Science Teaching, 21(1), 47–54.
Barak, J., Sheva, B., & Gorodetsky, M. (1999). As ‘process’ as it can get: Students’ understanding of biological processes. International
Journal of Science Education, 21(12), 1281–1292.
Baram-Tsabari, A., & Yarden, A. (2011). Quantifying the gender gap in science interest. International Journal of Science and Math-
ematics Education, 9(3), 523–550.
Brown, M. H., & Schwartz, R. S. (2009). Connecting photosynthesis and cellular respiration: Preservice teachers’ conceptions.
Journal of Research in Science Teaching, 46(7), 791–812.
Cañal, P. (1999). Photosynthesis and ‘inverse respiration’ in plants: An inevitable misconception? International Journal of Science
Education, 21(4), 363–372.
16
ISSN 2538–7138 /Online/ (P. 6-18)
Carlsson, B. (2002). Ecological understanding 1: Ways of experiencing photosynthesis. International Journal of Science Education,
24(7), 681–699.
Çepni, S., Taș, E., & Köse, S. (2006). The effects of computer-assisted material on students’ cognitive levels, misconceptions and
attitudes towards science. Computers & Education, 46 (2), 192–205.
Connors, J. J., & Elliot, J. F. (1995). The influence of agriscience and natural resources curriculum on students’ science achievement
scores. Journal of Agricultural Education, 36 (3), 57–63.
Domingos-Grilo, P., Reis-Grilo, C., Ruiz, C., & Mellado, V. (2012). An action-research programme with secondary education teachers
on teaching and learning photosynthesis. Journal of Biological Education, 46(2), 72–80.
Duckworth, A. L., & Seligman, M. E. P. (2005). Self-discipline outdoes IQ in predicting academic performance of adolescents.
Psychological Science, 16 (12), 939–944.
Eisen, Y., & Stavy, R. (1988). Students’ understanding of photosynthesis. The American Biology Teacher, 50 (4), 208–212.
Fančovičová, J., & Prokop, P. (2010). Development and initial psychometric assessment of the plant attitude questionnaire. Journal
of Science Education and Technology, 19 (5), 415–421.
George, R. (2000). Measuring change in students’ attitudes towards science over time: An application of latent variable growth
modelling. Journal of Science Education and Technology, 9 (3), 213–225.
Griffard, P. B., & Wandersee, J. H. (2001). The two-tier instrument on photosynthesis: What does it diagnose? International Journal
of Science Education, 23 (10), 1039–1052.
Hansen, M. J., & Birol, G. (2014). Longitudinal study of student attitudes in a biology program. CBE Life Sciences Education, 13 (2),
331–337.
Harman, G. (2012). Knowledge and misconceptions of science student teachers about physical and chemical change. Journal of
Education and Instruction Research, 1(3), 130–146.
Haslam, F., & Treagust, D. F. (1987). Diagnosing secondary students’ misconceptions of photosynthesis and respiration in plants
using a two-tier multiple-choice instrument. Journal of Biological Education, 21 (3), 203–211.
Hrustel Majcen, M., & Paulin, J. (Ed.). (2001). Slovenski kmetijsko okoljski program [Slovenian agricultural environmental programme].
Ljubljana: Ministrstvo za kmetijstvo, gozdarstvo in prehrano.
Jin, H., & Anderson, C. W. (2012). A learning progression for energy in socio-ecological systems. Journal of Research in Science
Teaching, 49 (9), 1149–1180.
Johnstone, A. H., & Mahmoud N. A. (1980). Isolating topics of high perceived difficulty in school biology. Journal of Biological
Education, 14 (2), 163–166.
Jones, M. G., Howe, A., & Rua, M. (2000). Gender differences and students’ experiences, interests, and attitudes toward science
and scientists. Science Education, 84 (2), 180–192.
Kubiatko, M. (2012). The investigation of Czech lower secondary school pupils toward science subjects. Journal of Educational
and Social Research, 2 (8), 11–17.
Lazarowitz, R., & Penso, S. (1992). High school students’ difficulties in learning biology concepts. Journal of Biological Education,
26 (3), 215–224.
Lenton, G., & McNeil, J. (1993). Primary school teachers’ understanding of biological concepts: selected research findings. Journal
of In-Service Education, 19 (2), 27–34.
Lin, C., & Hu, R. (2003). Students’ understanding of energy flow and matter cycling in the context of the food chain, photosyn-
thesis, and respiration. International Journal of Science Education, 25(12), 1529–1544.
Marmaroti, P., & Galanopoulou, D. (2006). Pupils’ understanding of photosynthesis: A questionnaire for the simultaneous assess-
ment of all aspects. International Journal of Science Education, 28 (4), 383–403.
Mayer, V. V. (1987). Agriculture and biology education. In A. N. Rao (Ed.), Food, Agriculture and Education (pp. 31–39). Oxford:
Pergamon Press.
Nasr, A. R., & Soltani, A. K. (2011). Attitude towards biology and its effects on student’s achievement. International Journal of
Biology, 3 (4), 100–104.
Owino, O. A., Yungungu, A. M., Ahmed, O., & Ogolla, B. O. (2015). The relationship between students’ attitude towards biology
and performance in Kenya certificate of secondary education biology in selected secondary schools in Nyakach, Kenya.
Research Journal of Educational Studies and Review, 1(5), 111–117.
Özay, E., & Öztaş, H. (2003). Secondary students’ interpretations of photosynthesis and plant nutrition. Journal of Biological
Education, 37(2), 68–70.
Poberžnik, A., Skvarč, M., Verovnik, I., & Vičar, M. (2007). Katalog znanja: naravoslovje: za nove programe srednjega poklicnega
izobraževanja (SPI). [Catalogue of knowledge: Science for vocational secondary education]. Ljubljana: MŠZŠ, CPI, SSPSI,
2001. Ljubljana: ZRSS: Evropski socialni sklad: Evropska unija.
Prokop, P., Prokop, M., & Tunnicliffe, S. D. (2007). Is biology boring? Student attitudes toward biology. Journal of Biological Educa-
tion, 42(1), 36–39.
ROSE questionnaire. (n.d.) The relevance of science education. Retrieved from http://roseproject.no/key-documents/question-
naire.html.
Shaheen, N. K., & Kayani, M. M. (2017). Improving students’ attitude towards biology as a school subject: Do the instructional
models really work? Journal of Applied Environmental and Biological Sciences, 7 (1), 170–179.
Simpson, R. D., & Oliver, J.S. (1990). A summary of major influences on attitude toward and achievement in science among
adolescent students. Science Education, 74 (1), 1–18.
Skribe-Dimec, D., & Strgar, J. (2017). Scientific conceptions of photosynthesis among primary school pupils and student teachers
of biology. CEPS Journal, 7 (1), 49–68.
17
(P. 6-18) ISSN 2538–7138 /Online/
Svandova, K. (2014). Secondary school students’ misconceptions about photosynthesis and plant respiration: Preliminary results.
Eurasia Journal of Mathematics, Science and Technology Education, 10 (1), 59–67.
Šorgo, A., & Špernjak, A. (2009). Secondary school students’ perspectives on and attitudes towards laboratory work in biology.
Problems of Education in the 21st Century, 14, 123–134.
Strategija razvoja Slovenije. [Strategy for Development of Slovenia]. (2005). Ljubljana, Republika Slovenija, Urad Republike Slo-
venije za makroekonomske analize in razvoj, Retrieved from http://www.gov.si/umar/
Štraus, M., Šterman Ivančič, K., & Štirn, S. (Ed.). (2016). Nacionalno poročilo: PISA 2015. [National report. PISA 2015]. Ljubljana:
Pedagoški inštitut. Retrieved from http://novice.pei.si/wp-content/uploads/sites/2/2017/10/PISA2015NacionalnoPorocilo.
pdf.
Tekkaya, C., & Balci, S. (2003). Determined students’ misconceptions of photosynthesis and respiration. Journal of Hacettepe
University Education Faculty, 24, 101-107.
Tekkaya, C., Çapa, Y., & Yılmaz, Ö. (2000). Pre-service biology teachers’ misconceptions about biology. Journal of Hacettepe Uni-
versity Education Faculty, 18, 140-147.
Terry, R., & Torres, R. M. (2006). Are teachers ready to integrate science concepts into secondary agriculture programs? Journal
of Agricultural Education, 47(4), 102–113.
Udeani, U. (2012). The relationship between study habits, test anxiety and science achievement. Journal of Education and Practice,
3(8), 151–158.
Usak, M., Prokop, P., Ozden, M., Ozel, M., Bilen, K., & Erdogan, M. (2009). Turkish university students’ attitudes toward biology: The
effects of gender and enrolment in biology classes. Journal of Baltic Science Education, 8(2), 88–96.
Vičar, M. (2007). Katalog znanja: biologija: za nove programe srednjega strokovnega izobraževanja (SSI) in srednjega poklicno-
tehniškega izobraževanja (PTI). [Catalogue of knowledge: Biology for technical and vocational-technical secondary educa-
tion]. Ljubljana: MŠZŠ, CPI, SSPSI, 2001. Ljubljana, ZRSS: Evropski socialni sklad: Evropska unija.
Waheed, T., & Lucas, A. M. (1992). Understanding interrelated topics: Photosynthesis at age 14. Journal of Biological Education,
26(3), 193–200.
Zeidan, A. H., & Jayosi, M. R. (2015). Science process skills and attitudes toward science among Palestinian secondary school
students. World Journal of Education, 5 (1), 13–24.
Received: September 20, 2018 Accepted: December 21, 2018
Katja Gobec MSc, Lecturer, Vocational College, School Centre Šentjur, Cesta na
kmetijsko šolo 9, 3230 Šentjur, Slovenia.
E-mail: katja.gobec@guest.arnes.si
Website: http://www.sc-s.si
Jelka Strgar PhD, Associate Professor, Biotechnical Faculty, Department of
Biology, University of Ljubljana, Večna pot 111, 1000 Ljubljana,
Slovenia.
E-mail: jelka.strgar@bf.uni-lj.si
Website: http://www.bf.uni-lj.si
18
THE IMPACT OF A SCIENCE
METHODS COURSE ON FEMALE
PRE-SERVICE TEACHERS’ ISSN 1648-3898 /Print/
BELIEFS WITHIN A SAUDI- ISSN 2538-7138 /Online/
ARABIAN CONTEXT https://doi.org/10.33225/jbse/19.18.19
Abstract. This research examines the

impact of a science methods course on
Amani K. Hamdan Alghamdi, the beliefs of female pre-service teachers
Sun Young Kim (PSTs) in Saudi Arabia. Forty-seven female
PSTs enrolled in a diploma of education
programme at Imam Abdulrahman Bin
Faisal University (IAU) took a 16-week sci-
ence methods course aimed at promoting
their beliefs about their own self-efficacy,
Introduction science teaching strategies, and science
content knowledge (SCK). The PSTs com-
For many years, science educators have focused on science educa- pleted a 30-item questionnaire on science
tion reforms in order to engage students in the discourses and practices of teaching beliefs (five- point Likert Scale)
argumentation in science subjects (Erduran & Jimenez-Aleixandre, 2008; both before and after taking the course.
Sampson & Blanchard, 2012; Zembal-Saul, 2009). It has been assumed that Data analysis revealed that the PSTs’ beliefs
when students engage in well-structured discourse, they are also involved regarding their own self-efficacy changed
in metacognition. As a result of free-association drawn from their construc- after the course (statistically significant
tive experiences, their sense of intuitiveness is fostered, and they develop a t=2.792, p 0.01) with scores indicating
better understanding of science concepts (Berland & Reiser, 2009; Erduran & increased beliefs. Although increases
Jimenez-Aleixandre, 2008). Numerous case studies have provided empirical were also observed for beliefs regarding
evidence of the positive impact of argumentation on students’ learning in science teaching methods and strategies
science education (Lemke, 1990; McNeill & Krajcik, 2011; Venville & Dawson, and science content knowledge, they were
2010). However, the successful execution of a pedagogical revolution that very slight and not statistically significant.
promotes the concepts of argumentation in science education among Overall, mean scores fell within the ‘neither
students does not simply involve the development of knowledge; it is also agree nor disagree’ category for all three
about empowering in-service and pre-service science teachers by giving themes, ranging from 2.98 to 3.24. As one
them the necessary knowledge, skills and competencies to teach science of the first studies in Saudi Arabia on PSTs’
through argumentation (Simon, Johnson, & Johnson, 2008). science teaching beliefs, this research filled
Effective pedagogies and their successful execution by teachers are a a gap in the existing literature. Grounded
core component in strengthening the implementation process of argumen- in the moderate scores for all three themes,
tation in science education. Fostering teachers’ belief in their own abilities recommendations for future science educa-
to “organize and execute the courses of action” that are required to produce tion course design are tendered as are
argumentation in science education is one way to promote this concept suggestions for future research.
(Bandura, 1997, p. 3). Dewey (1933) referred to thought as being synonymous Keywords: pre-service teachers, science
with belief, defining the latter as “something beyond itself by which its value teachers’ beliefs, self-efficacy, teaching
is tested; it makes as assertion about some matter of factors or some principle strategy, teachers in Saudi Arabia.
or law” (p. 6). Dewey (1933) simplifed the necessity of belief because
Amani K. Hamdan Alghamdi
…it covers all the matters of which we have no sure knowledge and yet Imam Abdulrahman Bin Faisal University,
Saudi Arabia
which we are sufficiently confident of to act upon and also the matters we
Sun Young Kim
now accept as certainly true, as knowledge, but which nevertheless may Chosun University, South Korea
be questioned in the future. (p. 6)
19
(P. 19-38) ISSN 2538–7138 /Online/
Richardson (1994) described beliefs as one’s understandings of the world and the way it functions or should
function. These beliefs may be held consciously or unconsciously, and they guide a person’s practice. Science
teachers possess beliefs regarding their professional practice, with Lumpe, Czerniak, Haney, and Beltyukova (2012)
commenting on the critical nature of these motivational beliefs. But research indicates that acquiring new beliefs is
difficult. For example, Pajares (1992) distinguished between early-held beliefs and newly acquired ones, explaining
that the former ones are combined with one’s belief structure, and become difficult to change. Therefore, these
early-held beliefs greatly influence one’s perceptions and how one addresses new information. As a result, newly-
obtained beliefs are more “vulnerable,” although they become “robust” over time (Pajares, 1992, p. 317). Pajares
has also argued that beliefs are difficult to replace and noted that an individual can retain a belief that is derived
from incomplete or incorrect knowledge, despite sufficient evidence to the contrary.
Studies have suggested that engaging teachers in active learning activities has a significant role in changing
their beliefs and practices. In mathematics education, for example, Nelson and Hammerman (1996) found that
activities can challenge teachers “to confront their old ideas about the nature of mathematics, learning, and teach-
ing with new data, ideas, and experiences” (p. 5). Mansour (2009) introduced different sources of teachers’ beliefs,
explaining that some beliefs stem directly from the culture, while others come from experiences. Mansour further
noted that teachers’ experiences have a significant impact on their beliefs regarding the process of teaching and
learning. According to Mansour, other sources of teachers’ beliefs include “personality factors, prior learning or
teaching experiences, teacher education, teaching contexts, an apprenticeship through observation, and related
reading of either research findings or other materials” (2009, pp. 36, 37).
Bransford, Brown, and Cocking (2000) identified a barrier that restricts teachers from rethinking their subject
matter and thus changing their beliefs. They explained that:
…learning involves making oneself vulnerable and taking risks, and this is not how teachers often see their role ...
teachers generally are accustomed to feeling efficacious – to knowing that they can affect students’ learning – and
they are accustomed to being in control. When they encourage students to actively explore issues and generate
questions, it is almost inevitable that they will encounter questions that they cannot answer – and this can be
threatening. (p. 195)
Bransford et al. (2000) also suggested that it is very important to assist teachers to feel comfortable in the
role of learner by providing them with expertise in the subject matter and with developed technology that grants
them broader access to the expertise of other professionals.
The Impact of Science Methods Courses for Pre-service Science Teachers
In teacher preparation programmes, science methods courses play the most influential role in helping pre-
service teachers (PSTs) to establish their own teaching strategies and build a positive perception of science and
science teaching (Bursal, 2008). Through such courses, PSTs have opportunities to connect theory and practice
and integrate their teaching and learning perspectives (Anderson, 1997). Other studies have argued that science
methods courses are a good means of increasing efficacy in science teaching (Cantrell, Young, & Moore, 2003; Kim,
2012; Palmer, 2006). PSTs who possess a good sense of their own efficacy in science teaching will progress well as
teachers (Appleton & Kindt, 2002). In addition, when teaching science, such teachers make more use of inquiry
methods (Anderson et al., 2004). Cross (2009) argued that the beliefs held by math teachers regarding the practise
of teaching are more than one-dimensional. He maintained that research has redefined and categorized science
teachers’ beliefs into three dimensions: science teaching efficacy, science teaching strategies, and science content,
elaborated in the following sections.
Self-efficacy in Science Teaching
Self-efficacy is a powerful predictor of performance that is used interchangeably with self-confidence (Watters
& Ginns, 2000). Bandura (1997) defined self-efficacy as “the beliefs in one’s capabilities to organize and execute
the courses of action required to produce given attainments” (p. 3). Bandura then described four main sources of
self-efficacy: mastery experiences, in which a particular situation is successfully dealt with; vicarious experiences,
which refers to situations in which people compare their own abilities with those of others who have modelled
20
ISSN 1648–3898 /Print/ THE IMPACT OF A SCIENCE METHODS COURSE ON FEMALE PRE-SERVICE TEACHERS’
ISSN 2538–7138 /Online/ (P. 19-38)
the desired behaviour; verbal persuasion, which occurs when individuals receive positive feedback from their peers;
and physiological and affective states, which refers to individuals’ personal levels of fear, anxiety and stress.
In a similar sense, teachers’ self-efficacy refers to their belief that they have the ability as an individual to create
positive change in students’ learning (Gibson & Dembo, 1984). A lack of confidence impacts a teacher’s classroom
behaviour and teaching. Appleton and Kindt (2002) have found that new teachers with little belief in their own
abilities seldom use hands-on activities in teaching science, using instead reading and writing as teaching strategies.
Bursal (2008) has also argued that teachers with low self-efficacy rely on the use of teacher-directed instruction
and avoid science experiments and other inquiry activities.
In order to increase science teaching efficacy, pedagogical knowledge is a prerequisite, including under-
standing both the use of appropriate science teaching strategies, and content knowledge. According to Appleton
(1995), a constructivist science methods course increased teachers’ confidence. Furthermore, Settlage (2000) has
found that pre-service elementary teachers’ self-efficacy increased when they were given instructions about the
learning cycle. Furthermore, science content knowledge (SCK) has been shown to be one of factors that affects
PSTs’ confidence and self-efficacy (Schoon & Boone, 1998).
Science Teaching Strategies
The current focus in science teaching involves promoting inquiry-based instruction and engaging students
in reasoning (National Science Teachers Association [NSTA], 1998). Inquiry process skills, which include observ-
ing, classifying, measuring, communicating, predicting, inferring and experimenting, help students to engage in
meaningful learning. Science teaching reforms position science teachers as facilitators who enable students to
be actively involved in examining and solving real-world problems (Plevyak, 2007). The NSTA (2003) has stated
that inquiry-focused teachers prepare to teach through inquiry, understand the processes of multiple methods
of inquiry, and engage students in appropriate inquiries based in a scientific manner on their observations, data,
and inferences. In this sense, one of the roles of a pre-service teacher preparation programme is to provide ideal
contexts that allow PSTs to develop insights into science teaching strategies.
Science Content in Science Teaching
It is critical that science teachers understand both science content and the nature of science as a way of know-
ing. The NSTA has stated that “teachers of science understand and can articulate the knowledge and practices of
contemporary science. They can interrelate and interpret important concepts, ideas, and applications in their fields
of licensure; and can conduct scientific investigations” (2003, p. 4). SCK is considered to be a requirement in order
for teachers to develop inquiry-based science pedagogy or pedagogical content knowledge in science (Santau,
Maerten-Rivera, Bovis, & Orend, 2014). Davis (2004) has argued that teachers with adequate SCK will use authentic
inquiry in their teaching more often.
However, many studies have reported that science teachers lack adequate content knowledge, with this
deficiency resulting in lower-quality teaching (Kind, 2009). In addition, the literature indicates that elementary
science teachers possess the same science misconceptions as their students (Bursal, 2012). In order for teachers to
implement reform-based science teaching, the development of SCK, or, what science to teach, is essential, along
with pedagogical content knowledge, or, how to teach (Santau et al., 2014). When teachers possess sufficient SCK,
they will be better prepared to plan and implement inquiry lessons and better able to answer students’ questions
in a way that connects with students’ prior experience (Luera & Otto, 2005).
Pre-service Science Teacher Education in the Saudi Arabian Context
Saudi Arabia was founded in 1932. As the birthplace of Islam, this country holds considerable religious sig-
nificance; it also possesses a great deal of petroleum-based wealth. Reforming higher education in Saudi Arabia
(Hamdan Alghamdi & Alsalouli, 2013) has been at the centre of attention in recent years. In 1989, the Ministry of
Education (MOE) established 18 teacher colleges, with goals that included the preparation of new student teach-
ers and the provision of professional development for in-service teachers. PSTs in these teacher colleges must
complete general academic courses, courses in professional education, and additional credit hours in courses
on their major areas of study (Ministry Agency of Teachers’ Colleges as cited in Hamdan Alghamdi, 2015). Almost
21
(P. 19-38) ISSN 2538–7138 /Online/
all teacher education programmes in Saudi Arabia are structured in a similar way and involve two semesters of
theoretical courses and eight to 10 weeks of practicum in the second semester. Depending on their field of study
(whether BA or BSc), the PSTs attend micro-teaching seasons, take methods courses (e.g., science, math, religion,
literature, social studies) and teach their various subjects in schools for eight to 10 weeks.
Research Problem and Questions
Based on a review of the literature associated with science teachers’ education in general and science teach-
ers’ education in Saudi Arabia in particular, it appeared that little is known about the beliefs these teachers hold
about science. Few studies have examined the belief system of PSTs from the perspective of a science methods
course, and there seems to be a lack of focus on how teachers’ beliefs about science are formed. Research in this
area is necessary because there is an increasing demand for preparing science teachers, improving their teaching
skills, and increasing their understanding of the nature of science.
The beliefs and perspectives of pre-service and in-service science teachers play a critical role in shaping their
learning. This research attempted to better understand teachers’ perspectives and the influence of a science methods
course on such perspectives. In particular, this research examined the impact of teachers’ beliefs regarding their
own self-efficacy, methods and strategies for teaching science, and the content and nature of science. The study
was guided by the following research questions:
Research Question 1: What science teaching beliefs are currently held by female Saudi-Arabian PSTs studying for a
post-graduate diploma in education at Imam Abdulrahman Bin Faisal University (IAU)?
Research Question 2: What impact does a science methods course have on the beliefs of these female PSTs?
Research Methodology
Science Methods Course Context
This research involved female PSTs (i.e., teacher candidates) who were enrolled in a science methods course
titled “Science Teaching in the 21st Century” within their diploma of education programme. This course covered half
a year (i.e., one 16-week semester) and provided two credits (total 32 contact hours per semester). It required the
PSTs to spend two hours in their practicum and three hours in the classroom per week. According to the schedule,
this course addressed science teaching (Table 1) before the practicum. The course introduced the PSTs to science
concepts that might arise during their practicum, provided them with an understanding of how to develop SCK
using creative and critical thinking, and taught them how to promote an understanding of SCK through problem
solving. Furthermore, the PSTs were given opportunities to apply scientific and critical thinking in the context of
teaching science. Within a teaching context, the PSTs were encouraged to use constructivist ways of teaching,
such as the “Five E’s” model (Engage, Explore, Explain, Elaborate, Evaluate) and to reflect both on how learners
use scientific concepts and how teachers plan and evaluate concepts. Overall, the course focused on promoting
PSTs’ self-efficacy, their science teaching methods and strategies, and their SCK in order to allow them to develop
adequate pedagogical knowledge of and positive beliefs about science teaching.
Table 1. Content of the 16-week science methods course.
Week Content
1 Requirements for preparing a science teacher
2 Planning for science education
3 Evolution of concepts in science: how to acquire it, how to develop a concept, and how to promote it through problem solving
4 Scientific thinking: the concept of thinking, creative thinking and critical thinking
22
ISSN 2538–7138 /Online/ (P. 19-38)
Week Content
5 Applications of scientific and critical thinking in teaching science (part 1)
6 Applications of scientific and critical thinking in teaching science (part 2)
7 Applications of teaching and learning theories in science teaching
8 Constructivism in science education: the “Five E’s” model
9 Constructivism in science education: deduction methods
10 Constructivism methods in teaching science: the induction method
Basic rules of science and the scientific way of thinking (how learners use scientific concepts and how the teacher plans and evaluates
11
concepts): basic concepts and skills in science, discovery and exploration.
Basic science skills: observation, sensory use, graphics, comparison, classification, measurement, expression of results, reasoning,
12 prediction, asking default questions, understanding the relationship between objects, identifying and controlling variables, and using
scientific tools to obtain information
13 Assessment of learning outcomes in the science curriculum
14 Models of activities: methods and methods of science education (part 1)
Models of activities: methods and methods of science education (part 2); reading and libraries, scientific exhibitions, trips and scientific
15
visits, associations and scientific clubs
16 Models of activities: methods and methods of science education; radio and press school, lectures and scientific seminars
Sample
Potential respondents were informed of the research study through an informative letter and were asked
to participate. They were given the right to refuse participation or withdraw from the research if desired, with no
consequences. They were ensured of confidentiality and informed that ethical measures would be taken. For ex-
ample, their responses were not recorded under their names, so the results had no impact on their course marks or
final grades in the diploma of education programme. Ultimately, data were collected from 47 female participants
enrolled in the diploma of education programme at IAU, ranging in age from 26–44 (reflecting the lack of access
to male counterparts). For clarification, education in Saudi Arabia is gender segregated so female professors (i.e.,
authors of this study) have access to female students and are not allowed to teach male students.
The researchers developed a data-collection instrument – a questionnaire in the form of a science beliefs
scale – and used it to collect data about PSTs’ beliefs regarding their science teaching efficacy, science teaching
strategies and science content. The questionnaire, titled “Science Teaching Beliefs,” was constructed according to
the following three steps.
1) Identifying the themes of the questionnaire. A review of the existing literature on this topic revealed
a wide range of different measures that have been used to examine science teaching beliefs. For ex-
ample, Ambusaidi and Al-Baloushi (2012) and Bakir (2016) both examined teaching beliefs using three
approaches: a direct (explicit) approach, a discovery approach and a conceptual approach. On the other
hand, Sahin, Deniz, and Topçu (2016) adopted three rationales for beliefs; namely, knowledge beliefs,
beliefs regarding learning methods and techniques, and teachers’ beliefs regarding learning concepts.
Saylan, Armagan, and Bektas (2016) have identified more detailed rationales for science teachers’ beliefs,
which included simple knowledge and real knowledge beliefs, beliefs about how the quick assimila-
tion of science occurs, and beliefs about the innate abilities of a learner that cause learning to occur.
Within the same detailed orientation of beliefs, Alabdulkareem (2016) has adopted a typology of beliefs
23
(P. 19-38) ISSN 2538–7138 /Online/
divided into five dimensions: instruction, teaching, teaching objectives, key teaching practices, and
learners’ error-correction strategies. In contrast, other studies, such as that by Ibrahim (2012), focused
on the qualitative aspect of teachers’ beliefs, including science teachers’ beliefs regarding observation,
inference, the laws of science, scientific theories, and the nature of scientific knowledge. This qualitative
perspective is consistent with Al-Jazaeri’s (2009) research, which addresses teachers’ beliefs by focusing
on the nature of science, scientific knowledge, instruction, and learning.
After reviewing the orientation of these previous studies, the researchers chose to examine science
teachers’ beliefs within the following three themes:
•• Theme 1: Beliefs regarding their own self-efficacy
•• Theme 2: Beliefs regarding science teaching methods and strategies
•• Theme 3: Beliefs regarding science content knowledge
This structure covers the components that were investigated by the above-mentioned existing studies
but re-classifies them into three themes. For example, the components of instruction, teaching and
teaching objectives (Alabdulkareem, 2016) was merged into Theme 2: Beliefs regarding science teaching
methods and strategies. Similarly, the components of simple and real knowledge of science (Saylan et
al., 2016) and the nature of science (Ibrahim, 2012; Al-Jazaeri, 2009) were merged into Theme 3: Beliefs
regarding science content knowledge. In addition, the three themes within the chosen typology are
compatible with the simple educational experience of the study population – namely, the experience
of female PSTs studying for a post-graduate diploma in education in the Eastern province in Saudi
Arabia.
2) Building the questionnaire items. In the questionnaire, each theme comprised 30 items that were
graded by the respondents using a 5-point Likert scale as follows:
•• 5: Strongly agree
•• 4: Agree
•• 3: Neither agree nor disagree
•• 2: Disagree
•• 1: Strongly disagree
The 30 items (which were not posed as questions) were appropriately phrased to ensure that they were
understandable to the respondents. The higher the rating to each questionnaire item, the more closely
the respondents’ beliefs aligned with the desired pedagogical perspective, that being “constructivist.”
Furthermore, the 30 items focused on real and relevant issues involving beliefs that must be a concern
for science teachers.
3) Verifying the validity and reliability of the questionnaire. The content of the questionnaire was
verified by having four professors in the field of curricula, teaching methods and science assess the
instrument in terms of the following criteria: the accuracy of the wording, clarity of drafting, and linking
of each item to a single theme among the three main themes. The instrument was then modified based
on the experts’ observations. It should be mentioned here that the questionnaire was constructed and
validated after reading and analysing the international literature on PSTs’ beliefs.
Also, Cronbach’s alpha was calculated, and the parameter value was found to be .81 based on the 30
statements (items) within the questionnaire. This value indicates that the questionnaire is stable and
applicable to the community targeted in the research. Table 2 clearly shows that the correlation coef-
ficient values was positive and statistically significant at the .05 level, which is an indication of instru-
ment internal consistency.
24
ISSN 2538–7138 /Online/ (P. 19-38)
Table 2. Correlation coefficient between the three themes in the questionnaire and the overall degree of
respondents’ beliefs.
Beliefs regarding
Beliefs regarding Beliefs regarding
science teaching
Item Theme their own self- science Total
methods and
efficacy knowledge
strategies
1 Beliefs regarding their own self-efficacy

2 Beliefs regarding science teaching meth- .550*
ods and strategies
3 Beliefs regarding science knowledge Decline from .381* .397*
4 Total .767* .891* .699*
* Mean significance at the level of (.05)
** Mean significance at the level of (.01)
Data were collected before the science methods course, and then collected again after the course for com-
parison.
Data Analysis
To collect data for the first research question, the respondents were asked to complete the questionnaire
prior to taking the abovementioned course titled “Science Teaching in the 21st Century” (see Table 1). Using the
questionnaire, data were gathered on the PSTs’ beliefs regarding their own self-efficacy, science teaching methods
and strategies, and SCK. The researchers calculated both the respondents’ average response per item and their
responses for all items in the questionnaire using a 5-point Likert scale, which ranged from 1 (strongly agree) to 5
(strongly disagree). The average response was calculated by first calculating the range according to the following
relationship:
Range = Number of categories – 1
The value of the range is 4. The length of the category was specified using the following equation:
Length of category = Range/Number of categories
The value of the category length is 0.80. Thus, the categories used to interpret the values of the average
response for each belief item on the questionnaire were specified as follows:
•• Average ranging from 4.21–5.00: Beliefs that the research group strongly agrees with, on average
•• Average ranging from 3.41–4.20: Beliefs that the research group agrees with, on average
•• Average ranging from 2.61–3.40: Beliefs that the research group neither agrees nor disagrees with, on
average
•• Average ranging from 1.81–2.60: Beliefs that the research group disagrees with, on average
•• Average ranging from 1–1.80: Beliefs that the research group strongly disagrees with, on average
A factor analysis was not carried out because the three themes were not directly interrelated, per insights
from the literature review. However, a t test was applied due to the interrelation of the independent variables. The
internal consistency of the questionnaire was verified by calculating the correlation coefficient between the three
themes and the “overall agree” using an exploratory sample of 25 PSTs within the diploma of education programme
at IAU. The results of the correlation coefficient calculation are provided in Table 2. Data were analyzed using SPSS
Statistics software.
25
(P. 19-38) ISSN 2538–7138 /Online/
Research Results
Table 3 summarizes the overall results obtained when 47 Saudi female PSTs enrolled in the diploma of educa-
tion at IAU completed the questionnaire prior to taking the course titled “Science Teaching in the 21st Century” (see
Table 1). With an average score of 3.07 (neither agree nor disagree), the respondents’ ratings of the three themes
in the questionnaire were ordered as follows. Theme 3 (science knowledge) had the highest rating, with an aver-
age of 3.18. Theme 1 (self-efficacy) had an average rating of 3.08 and Theme 2 (teaching methods and strategies)
had the lowest rating, with an average score of 2.95. All ratings fell within the neither agree nor disagree category,
with scores ranging from 2.95–3.18. Also, the PSTs in this research held similar beliefs regarding each of the three
themes, with no considerable or clear difference relative to self-efficacy, teaching methods and strategies, or sci-
ence content knowledge.
Table 3. General average ratings for the three themes associated with PSTs beliefs about teaching science.
Category of
Item Theme Mean Mean (%)
mean
Neither agree nor

1 Beliefs regarding their own self-efficacy 3.08 62
disagree
2 Beliefs regarding science teaching methods and strategies 2.95 59 Disagree
Neither agree nor
3 Beliefs regarding science knowledge. 3.18 64
disagree
Neither agree nor
General mean ratings for Themes 1, 2 and 3 by female pre-service science teachers 3.07 61
disagree
The following section provides a detailed account of the results for each of the three themes per the first re-
search question, which was concerned with their beliefs before completing the science methods course (see Table 1).
Theme 1: Beliefs Regarding Their Own Self-efficacy (Pre-enrolment)

Table 4 profiles the results for Theme 1, the PSTs’ beliefs regarding their own self-efficacy as science educators
assessed before taking the course. First, the general average score for theme 1 was 3.08 out of a total value of 5,
reflecting the neither agree nor disagree category. Second, none of the items within the first theme fell within the
category of strongly agree. Respondents rated four items (40%) as agreed (items 1 through 4), three items (30%)
as neither agree nor disagree (items 5 through 7), and three items (30 %) as disagree (items 8 through 10).
Table 4. Ordered average ratings for 10 items in Theme 1 (pre- and post enrolment).
Before the course After the course

Theme 1 - Beliefs regarding their own self-efficacy
Category of Category of
Mean Mean
mean mean
Achieving apparent improvement in the level of learners is attrib-

1 4.17 Agree 4.34 Strongly agree
uted to the effort exerted by the teacher in teaching sciences.
Accurately implementing scientific experiments is a skill to be
2 3.89 Agree 4.15 Agree
mastered by a science teacher.
Implementing the images related to science inside the classroom
affects the learners’ understanding and comprehension.
Linking science concepts to real-life applications is a simple task
for a science teacher.
Difficulties related to working out the applications and problems of Neither agree nor
5 2.98 3.55 Agree
teaching science are attributed to a science teacher’s lack of skills. disagree
26
ISSN 2538–7138 /Online/ (P. 19-38)

Theme 1 - Beliefs regarding their own self-efficacy
Mean Mean
mean mean
Developing the high-order thinking skills related to teaching sci- Neither agree nor Neither agree nor
6 2.79 2.79
ence is difficult. disagree disagree
Motivating learners to study science content is difficult for a sci- Neither agree nor
7 2.64 2.34 Disagree
ence teacher. disagree
It is difficult to manage the classroom while explaining science
8 2.51 Disagree 2.53 Disagree
because of discussions and carrying out activities.
Overcoming the learners’ learning difficulties related to the study of Neither agree nor
9 2.45 Disagree 2.61
science is not an exhausting activity for the science teacher. disagree
Developing the learners’ attitudes towards studying sciences is
achieved at a weak rate.
Neither agree nor Neither agree nor
General mean 3.08 3.24
disagree disagree
Theme 2: Beliefs Regarding Science Teaching Methods and Strategies (Pre-enrolment)
Table 5 profiles the results for Theme 2 – beliefs regarding science teaching methods and strategies before
completing the science methods course. The overall average rating for this theme was 2.95 out of 5 (neither agree
nor disagree). Three items (30%) were rated as strongly agree or agree (items 1, 2 and 3). Three were rated as neither
agree nor disagree (items 4, 5 and 6). Two items were rated as disagree (items 7 and 8) or strongly disagree (items
9 and 10), four in total.
Table 5. Ordered average ratings for the 10 items in Theme 2.

Theme 2 - Beliefs regarding science teaching methods
and strategies Category of
Mean Mean Category of mean
mean
The impact of life applications on the learners’ understand- Strongly agree

1 4.30 4.15 Agree
ing of science is clear.
Giving the learner the opportunity to build his/her own
2 knowledge through studying science is important to achieve 4.13 Agree 4.17 Agree
meaningful learning.
Assimilating and understanding the laws of the science
3 subject is achieved by inferring the specific example of the 3.72 Agree 3.57 Agree
laws.
Providing the learners with knowledge and skills prior to
4 introducing scientific concepts clearly affects their assimila- 3.23 3.38
disagree disagree
tion.
Individualizing learning is an important requirement in
5 teaching science because of the clear variation in the levels 3.17 3.34
disagree disagree
of learners.
Depending on group work s will clearly affect the learners’ Neither agree nor Neither agree nor
6 3.13 3.32
assimilation and understanding. disagree disagree
Teaching science through indirect experiences is most
useful for saving time.
27
(P. 19-38) ISSN 2538–7138 /Online/

Theme 2 - Beliefs regarding science teaching methods
and strategies Category of
Mean Mean Category of mean
mean
Real learning of science occurs through the teacher assum-

8 ing the major role of explaining the lesson because of the 2.19 Disagree 2.06 Disagree
poor and low level of the learners.
A teacher implementing the scientific experience by himself
9 or herself wastes much time without achieving a tangible 1.79 Strongly disagree 1.77 Strongly disagree
result.
Giving the learners the opportunity to try self-learning in sci-
10 1.57 Strongly disagree 1.91 Disagree
ence is not clearly useful.
disagree disagree
Theme 3: Beliefs Regarding Science Content Knowledge (Pre-enrolment)
Still focused on the first research question, Table 6 profiles the results for Theme 3 – beliefs regarding science
content knowledge, before taking the course. The overall average rating for Theme 3 was 3.18 out of 5 reflecting
the neither agree, nor disagree category. No item was rated as strongly agree, but two items (20%) were rated as
agree (items 1 and 2). The majority (60%, n=6) of items were rated as neither agree nor disagree (items 3, 4, 5, 6, 7
and 8). Two items were rated as disagree (items 9 and 10).
Table 6. Ordered average ratings for the 10 items in Theme 3.

Theme 3 - Beliefs regarding science content knowledge
Mean Mean
mean mean
The focus of science textbooks on the application of life skills and

1 classroom activities is more important than theoretically elaborating 4.17 Agree 4.32 Strongly agree
on the topics.
2 Parents believe in the difficulty of the science subject content. 3.91 Agree 3.83 Agree
Precisely identifying learners’ learning difficulties of science is a Neither agree Neither agree nor
3 3.34 3.06
complex task because of the nature of science content. nor disagree disagree
Neither agree Neither agree nor
4 Teaching practical skills is easier than teaching scientific concepts. 3.34 3.34
nor disagree disagree
Implementing classroom activities related to science is easier than Neither agree Neither agree nor
5 3.32 3.13
implementing them in other courses. nor disagree disagree
More students tend to study science subjects rather than theoretical Neither agree Neither agree nor
6 3.23 3.36
courses. nor disagree disagree
7 Teaching science is more difficult than teaching other subjects. 2.79 2.91
Building scientific tests to measure learners’ achievement in science Neither agree Neither agree nor
8 2.72 2.94
is a simple task due to the diversity of science content. nor disagree disagree
Reading the science content textbooks is a simple task for students
and occurs without suffering.
Preparing the teaching aids of science-related subjects is more dif-
ficult compared with other subjects.
28
ISSN 2538–7138 /Online/ (P. 19-38)
Results for Research Question 2 (Comparing pre- and post-enrolment)
To verify the effectiveness of the science methods course (see Table 1) in modifying the science teaching
beliefs held by female pre-service science teachers at IAU in Saudi Arabia, the questionnaire administered before
the course was re-administered after they had completed the course.
Theme 1: Beliefs Regarding Their Own Self-efficacy after Completing the Course
Table 7 shows the results of a t test comparing the average ratings for the 10 items in Theme 1 before and
after the course. With t = 2.792, it is evident that there is a significant difference between the pre- and post-course
averages regarding beliefs associated with self-efficacy (with 46 degrees of freedom and a significance level of .01).
A detailed comparison was then performed in order to infer the impacts of the course on the participants’ beliefs in
relation to their own self-efficacy when teaching science. These results, outlined in Table 4, clearly demonstrate a
change in the responses for virtually all the items in Theme 1, the exception being item 6, “Developing high-order
thinking skills associated with teaching science is difficult.” Of the nine item scores that did change, four changed
categories (items 1, 5, 7, and 9). PSTs either gained more certainty (moving from neither agree or disagree to
agree or disagree, items 7 and 5 respectively and moving from agree to strongly agree for item 1) or lost certainty
(changing from disagree to not sure (neither agree nor disagree) (item 9).
Table 7. t-test results comparing Theme 1 item ratings before and after completing the course.
Number of
Application Mean SD df t value p
participants
Before course 30.82 3.749

47 46 2.792 .008
After course 32.40 3.411
Theme 2: Beliefs Regarding Science Teaching Methods and Strategies After Completing the Course
Table 8 shows the results of a t test comparing the average ratings for the 10 items in Theme 2 before and after
the course. The t value (t = .347) is not statistically significant at 46 degrees of freedom and a significance level of
.05. Thus, there are no statistically significant differences between the pre- and post-course averages regarding
beliefs related to science teaching methods and strategies. However, a minuscule change (increase) occurred from
an average rating of 29.53 in the first application to 29.82 in the second application. A detailed comparison was
then performed in order to infer the impacts of the course on the respondents’ beliefs in relation to science teach-
ing methods and strategies (see Table 5). There was an increase in average numerical ratings for five (50%) items
(items 2, 4, 5, 6 10) but only one item changed categories, with item 10 moving from strongly disagree to disagree.
Table 8. t-test results for comparing Theme 2 item ratings before and after completing the course.
Average of
Number of
Application the sum of SD df t value p
participants
the ratings

47 46 .347 .730
29
(P. 19-38) ISSN 2538–7138 /Online/
Theme 3: Beliefs Regarding Science Content Knowledge after Completing the Course
Table 9 shows the results of a t test comparing the average ratings for the 10 items in Theme 3 before and
after the course. A t value of t = 0.357 is not statistically significant at 46 degrees of freedom and a significance
level of .05. Thus, there were no statistically significant differences between the pre- and post-course averages
regarding beliefs related to science content knowledge. However, a change is noticeable from an average rating
of 31.80 in the first application to 32.04 in the second application. A detailed comparison was then performed in
order to infer the impacts of the course on the participants’ beliefs in relation to their beliefs regarding science
content knowledge (see Table 6). There was an increase in numerical scores for five (50%) of the SCK belief items
(items 1, 6, 7, 8 and 10) but only one item changed categories, with item 1 moving from agree to strongly agree.
Table 9. t-test results for comparing Theme 3 item ratings before and after completing the course.
Average of
Number of
participants
the ratings

47 46 .375 .709
Results: Overall Impact of the Science Methods Course on PSTs’ Science Teaching Beliefs
Table 10 provides the results of a t test comparing the average ratings the PSTs provided for the question-
naire’s 30 items as a whole, before and after the course. A t value of t = 1.336 indicates no statistically significant
differences between the pre- and post-course averages regarding beliefs related to science teaching in general
(at 46 degrees of freedom and a significance level of .05). Nonetheless, an overall increase in the ratings on the
questionnaire items prior to the course may be seen relative to the ratings given afterwards, increasing about two
points from 92.17 to 94.27.
Table 10. t-test for comparing overall results before and after PSTs took the course.
Average of
Number of
participants
the ratings

47 92 1.336 .188
Discussion

The discussion section begins with the import of the results for question one (organized by the three themes
of self-efficacy, teaching methods and strategies, and science content knowledge). This question focused on the
PSTs’ beliefs before completing the science methods course (see Table 1). Then, the discussion turns to the results
for question two, which was focused on the effectiveness of the science methods course in modifying the beliefs
held by female PSTs at IAU in Saudi Arabia. In short, before taking the course, the PSTs presented beliefs character-
ized as neither agree nor disagree (mean= 3.08), indicating a mental stance of uncertainty about their self-efficacy,
their beliefs about how to teach science and science content. After taking the course, the overall score was still
within the neither agree nor disagree category. The PSTs’ scores for self-efficacy improved statistically but not so
for science teaching methods or science content. The course was somewhat effective with room for improvement.
30
ISSN 2538–7138 /Online/ (P. 19-38)
Theme 1: Beliefs Regarding Self-efficacy (Pre-enrolment)
Table 4 profiles the results for Theme 1, the PSTs’ beliefs regarding their own self-efficacy as science educators
assessed before taking the course. On average, they indicated ambivalence around this issue, with a general aver-
age of 3.08 (neither agree nor disagree). The PSTs in this study had spent one year on post-graduate studies (only
one semester for this 16-week course), meaning their teaching experiences were limited. A general self-efficacy
score of 3.08 reflects this inexperience. These results are supported by earlier studies that reported first-level PSTs
having lower self-efficacy than the more experienced fourth-level PSTs (Aslan, Tas, & Ogul, 2016; Uyanik, 2016).
Overall results could suggest that these Saudi female PSTs lacked confidence regarding their efficacy as science
teachers. It could be disempowering to feel incapable of producing a desired result when teaching science (i.e.,
low self-efficacy). Teachers with higher levels of efficacy believe they can control, or at least strongly influence,
student achievement and motivation (Tschannen-Moran, Hoy, & Hoy, 1998).
In particular, three specific items scored as neither agree or disagree (items 5, 6 and 7). These pertained to
difficulties around (a) working out the applications and problems (activities) of how to teach science, (b) developing
students’ higher order thinking skills, and (c) motivating learners to study science. Not being sure about whether
these tasks will be difficult speaks to their immaturity as educators and their perceived skill sets. The more experi-
enced the science teacher, the higher their self-efficacy (Aslan et al., 2014; Uyanik, 2016). Feeling inadequate in
improving student motivation and performance (item 7) would lower self-efficacy because these two factors are
powerful motivators for teacher behaviour and efficacy (Tschannen-Moran et al., 1998).
The significance of this result can be further interpreted using a recent Spanish study. Arce, Bodner, and
Hutchinson (2014) compared the viewpoints of a “conventional” group of teachers with those orientated towards
developing and upgrading their science teaching methods (“correctional” teachers). Their research showed that
ideal science teaching practices (item 5) are those that are centred on the involvement of learners in both scien-
tific and discovery activities, which inherently develop learners’ high-level thinking skills (item 6), as they involve
an examination of the real-life applications of science (item 4); hence, such activities motivate learners to study
science (item 7).
None of the ten items for Theme 1 scored strongly agree but four scored agree (items 1, 2, 3 and 4). Together,
these items reflect a distinct sense of competency on the part of the teachers. They agreed that their teaching efforts
influence student academic improvement as does implementing images related to science when teaching. They
agreed that it is easy to link science concepts to real life and that teachers must know how to accurately imple-
ment science experiments. One interpretation of these results is that most of the items scored as agree pertain to
a focus on the learner rather than the teacher. This interpretation is compatible with existing studies that show a
positive correlation between a science teacher’s ability to make the learner the focal point and their belief in their
own self-efficacy in teaching. Bakir (2016) pointed out that discovery teaching in science (i.e., experiments, real-life
applications, and imagery, items 2, 3 and 4) is more acceptable for students than direct teaching or conceptual
teaching. And, Aydogdu and Peker (2016, p. 20) asserted that “there is a relationship between the science teachers’
beliefs … with regard to their teaching efficiency and the teaching activities which are learner-centred, and which
are practiced inside the classroom.”
The respondents disagreed with three items related to self-efficacy (items 8, 9 and 10). The PSTs disagreed
that they were ineffective in developing learners’ attitudes towards studying science; in other words, they felt they
were effective. They did not think it was difficult to carry out classroom management activities while facilitating
discussions and other activities. But they did feel that overcoming learners’ learning difficulties is exhausting. It
might be naïve for PSTs to perceive classroom management and developing students’ attitudes towards studying
science as easy tasks. This might be an instance of misplaced self-efficacy given that first-level PSTS have been
found to have lower self-efficacy than more experienced fourth-level PSTs (Aslan, Tas, & Ogul, 2016; Uyanik, 2016).
Theme 2: Beliefs Regarding Science Teaching Methods and Strategies (Pre-Enrolment)
Table 5 profiles the results for Theme 2, the PSTs’ beliefs regarding science teaching methods and strategies
assessed before taking the course. First, the respondents gave an overall average rating of 2.95 out of 5 for this
theme. At first glance, the numerical value 2.95 indicates that they generally rated items under this theme in the
disagreement category. But this apparent negative finding is contradicted when the items are reframed from their
original wording. The four items rated disagree or strongly agree (items 7, 8, 9 and 10), in concert with the three
31
(P. 19-38) ISSN 2538–7138 /Online/
rated agree and strongly agree (items 1, 2 and 3), actually represent the PSTs’ inclination to value student-centred,
constructivist learning.
Students explicitly agreed that (a) life applications have a visible impact on learners (item 1), (b) it is important
to allow students the chance to build their own knowledge (item 2) and (c) students can assimilate and learn sci-
ence concepts if they experience the chance to make inferences (item 3). Reframing disagree and strongly disagree
items reveals, indirectly, that PSTs felt (d) teaching science thorough indirect experiences does not save time, (e)
teacher-centered education does not ensure real learning (item 8), (f ) students do learn when teachers implement
science experiences (item 9), and (g) facilitating self-learning is useful (item 10).
Items scored as strongly agree or agree (items 1, 2 and 3) may be examples of teaching strategies that reflect
constructivist thinking, which is beyond conventional teaching techniques. Sahin et al. (2016) revealed a correla-
tion between the constructivist conception of science teaching and PSTs’ epistemological beliefs regarding sci-
ence content, which are reflected in their teaching methods and strategies. They concluded that epistemological
beliefs and conceptions of learning are important factors in PSTs’ conceptualizations of teaching science (see also
Saylan et al., 2016). Bursal (2012) reported that positively developing teachers’ beliefs towards science teaching
can occur through qualitative training in the main skills of science, including the key skills of inquiry and simple
practical experiments.
The researchers consider the items scored as neither agree nor disagree (items 4, 5, and 6) to be closely related
to a PST’s skill in diversifying the techniques of teaching science for individualized or group activities. These PSTs
may be unable to agree or disagree on these particular items because they lack teaching experience and need
more training. Alabdulkareem (2016) has reported that science teachers’ beliefs about key concepts were closely
related to science teaching strategies such as instruction, teaching, teaching objectives, key teaching practices,
and learners’ error-correction strategies. All of these strategies are affected by the science teachers’ beliefs about
science teaching methods.
As a general observation, Wong (2016) showed that two-year training provided to science teachers through
an online post-graduate programme caused positive and tangible modifications in teachers’ beliefs regarding
the effective teaching of science and how teaching is centred on the learner. The science teachers in his study
underwent positive changes in their beliefs, changes that were unaffected by the number of years of experience.
This finding implies that effective and planned training is a very important factor in convincing Saudi pre-service
science teachers to adjust incorrect or affirm vacillating beliefs regarding effective teaching strategies.
Theme 3: Beliefs Regarding Science Content Knowledge (Pre-enrolment)

Table 6 profiles the results for Theme 3, the PSTs’ beliefs regarding science content knowledge, assessed
before taking the course. The general overall rating for this theme was 3.18 out of 5, meaning neither agree
nor disagree. Sixty percent (n=6) of the items scored in this category. Being in a state of ambivalence about
their beliefs around science content knowledge is not surprising given that the PSTs had yet to take the science
methods course titled “Science Teaching in the 21st Century” (see Table 1). Ambivalent means uncertain due
to mental conflict. At this stage, they appeared to not have adequately examined their beliefs around science
content knowledge.
In particular, the scores for items 3 through 8 ranged from 2.72-3.34, meaning the PSTs were not sure if
they agreed or disagreed with these statements. Issues captured within these items pertained to the ease of (a)
identifying students’ learning difficulties vis-à-vis science content (item 3), (b) teaching science concepts relative
to practical skills (item 4), (c) implementing science classroom activities (item 5), (d) teaching science compared to
other subjects (item 7), and (e) creating assessment tools and tests (item 8). PSTs were also not sure about whether
more students study science than theoretical courses (item 6). Not knowing their stance on such fundamental
aspects of science content knowledge has the potential to handicap future science teaching.
In more detail, science content knowledge (SCK) has been shown to affect PSTs’ confidence and self-efficacy
(Schoon & Boone, 1998). Furthermore, we consider that rating these six items as neither agree nor disagree indi-
cates that the PSTs are vacillating around their beliefs of whether science content knowledge is difficult to teach,
learn and assess. Discussing the notion of accuracy of beliefs about science content knowledge, Tarmo and Bevins
(2016) showed that science teachers hold common inaccurate beliefs about the nature and content of science. They
concluded that these ‘inaccurate’ beliefs can negatively affect teaching science practices. We hold that ‘ambiguous’
beliefs can also have negative consequences.
32
ISSN 2538–7138 /Online/ (P. 19-38)
Some of our results can be explained somewhat using Ambusaidi and Al-Baloushi’s (2012) work. They per-
formed research within an Arabian environment in the Sultanate of Oman involving candidate teachers who were
to be employed as science teachers. They reported that candidate teachers’ belief in conceptual and direct teaching
ranked higher than their belief in discovery teaching, because conceptual teaching is theoretical in nature and
thus exempts the teacher from exerting much teaching effort (see also Bakir, 2016). The PSTs in our study were not
sure about the ease of teaching science concept knowledge (items 4, 5, and 7), perhaps because of its complex
nature (item 3). Yet, puzzlingly, they agreed that science textbooks should eschew theory and focus on life skills
applications (item 1, Table 6) (i.e., eschew conceptual teaching).
Two items were scored agree (items 1 and 2). Respectively, the PSTs agreed that it is more important for sci-
ence textbooks to cover topics by focusing on the application of life skills instead of theoretically elaborating on
topics. They also believed that parents appreciated the difficultly of science content and knowledge. Along similar
lines of thought, the PSTs disagreed that it was easy for students to read the science content in textbooks (item 9)
and they did not believe it was easy to prepare science-related teaching aids (item 10).
By way of explanation, science teachers form their beliefs as a result of societal factors, and such factors make
each society unique. Regarding science-related beliefs, Ozfidan, Cavlazoglu, Burlbaw, and Aydin (2017) reported
differences between science teachers’ beliefs in the United States and Turkey. Accordingly, the beliefs of the PSTs in
this research may, well be attributed to societal viewpoints on the difficulty of science content. Also, the belief that
the knowledge taught in a science class is difficult, and that parents firmly hold this belief, may reflect negatively
on students’ futures because it may hinder their learning and influence any progress in choosing science streams
and professions.
The ratings for items 9 and 10 (disagree) imply that respondents considered it difficult to communicate using
the language of SCK, whether in students’ reading or in the use of teaching aids that convey and translate scien-
tific concepts. This result could be attributed to the PSTs’ lack of experience with academic science content from a
pedagogical perspective. They had not yet taken the science methods course (see Table 1). This result aligns with
Sahin et al.’s (2016) work. They performed a forecasting study with primary education science teachers in Turkey
and reported that epistemological beliefs were key predictors of science teachers’ beliefs regarding SCK, along
with their beliefs about the methodology and concepts of science learning.
General Comments on Research Question One (Pre-enrolment)

Per Table 3, the fact that there was no considerable or clear difference between beliefs held for self-efficacy,
teaching methods and strategies, or science content knowledge can be attributed to the respondents’ lack of
teaching experience, such that their perspective on science-related beliefs is formed more through their learning
experiences as science students in their first degree than through their experiences as pre-service science teach-
ers (Diploma in Education). Since the respondents had only spent one year on post-graduate study (i.e., in teacher
education program), their self-efficacy was still not high enough. The low score for beliefs regarding science teaching
methods and strategies is acceptable from a logical standpoint, given that these beliefs are technical and require
specialized study. The PSTs in this research are in need of a science methods course.
These observations align with findings from existing studies indicating that the initial beliefs of PSTs – before
they complete academic or qualitative programmes focused on the professional preparation of a pre-service science
teacher – are inaccurate or unexamined and likely based on personal conceptions. In fact, this generalization also
applies to many in-service science teachers. Saylan et al. (2016) reported a correlation between the epistemologi-
cal beliefs of PSTs and their conceptions of the learning environment. Epistemological beliefs form from several
sources including: simple knowledge and real knowledge of science, how quickly science assimilation occurs, and
the innate capabilities and faculties of the learner and their impact on science learning. This viewpoint is consistent
with Yenice’s (2015) findings. He showed, through a forecasting study, that the key rationales held by PSTs include
a belief that science learning depends on trial and error, the abilities of learners, and the nature of the science
content. It is the researchers’ opinion that all of these beliefs position success in science teaching as relying on
variables for which the PSTs do not bear clear responsibility.
The discussion now shifts to address the import of any changes in Saudi female PSTs’ beliefs about science
teaching self-efficacy, methods, and content as a result of completing a science methods course (see Table 1), part
of the Diploma of Education at the Imam Abdulrahman Bin Faisal University (IAU).
33
(P. 19-38) ISSN 2538–7138 /Online/
Theme 1: Changes in Beliefs about Their Own Self-Efficacy After Completing the Course

There was a statistically significant difference between the pre- and post-course averages regarding beliefs
associated with their own self-efficacy. This result is supported by previous research that confirmed science
methods courses as a viable means for increasing efficacy in science teaching (Cantrell et al., 2003; Kim, 2012;
Palmer, 2006). The only self-esteem belief item that did not change dealt with developing students’ high-order
thinking skills (item 6, Table 4). The PSTs still felt uncertain about whether it would be difficult for them to de-
velop this skill set in students. Santos (2017) maintained that PST science education has largely disregarded
critical thinking, opting instead for an emphasis on a traditional pedagogy. In contrast, the science methods
course used in our research study intentionally incorporated critical thinking, with three weeks out of 16 set
aside for this topic (see Table 1). It could be argued that a change in this belief would require science teachers
to have more teaching experience and confidence in their teaching performance. It could also mean they need
more or different instruction on how to instil critical thinking skills or more convincing of its merit. Demir (2015)
affirmed that training teacher candidates in critical thinking is necessary if they are to appreciate its merits and
apply it when teaching.
Of the nine Theme 1 item scores that did change, four changed categories. PSTs gained more certainty about
the role their skill level plays in teaching science and they indicated that it is not as difficult to motivate students
to study science as they formally believed. They also increased their belief that improvement in learning can be
attributed to their teaching efforts. But they lost certainty around beliefs about how exhausting it is to overcome
learners’ learning difficulties. These four items tended to focus on the concept of difficulty, which means something
requires a lot of effort to achieve, deal with or understand. Assessing the level of difficulty of a task before actually
experiencing it is a challenge.
From another perspective, although results revealed tangible development between the pre- and post-course
assessment of PSTs’ beliefs about their own self-efficacy, this development could be attributed to the respondents
considering that they had attained a proper level of teaching experience after the course, perhaps with a degree
of exaggeration. This interpretation aligns with Saka, Bayram, and Kabapinar’s (2016) work with final-year Marmara
University teacher candidates, wherein they proposed that beliefs regarding high self-efficacy could be misplaced.
This possibility should be considered in future research and course design.
Theme 2: Changes in Beliefs about Science Teaching Methods and Strategies After Completing the Course

Regarding beliefs about science teaching methods and strategies, there was an increase in average score
ratings for half of the items (n=5) (items 2, 4, 5, 6 and 10 in Table 5) (not statistically significant). This moderate
score increase may reflect the short duration of the course (16 weeks), as the development of beliefs associated
with teaching methods and strategies requires deep academic learning and considerable classroom experience.
The amount of time teachers spend in professional development courses and programs can impact their learning
and teaching (Lumpe et al. 2012). But acquiring new beliefs is difficult (Pajares (1992), perhaps explaining the only
moderate increase in beliefs, with most currently still held in disagreement.
The five items in question (with most scored as disagree) reflected at best nominal increases and were
focused on both (a) the effectiveness of providing learners with knowledge and skills before the lesson, using
group work, and enabling self learning; and (b) the importance of individualized learning, and providing learners
with opportunities to build their own knowledge. Failing to appreciate the importance of these teaching strate-
gies implies that these Saudi PSTs had not yet fully embraced the constructivist pedagogy, as is recommended
by Simon et al. (2008). The constructivist science pedagogy can increase teachers’ confidence in their teaching
ability (Appelton, 1995).
Only one of these five items changed categories. The PSTs articulated lessening disagreement on the usefulness
of giving learners self-learning opportunities, moving from strongly disagree to disagree. In effect, they still did not
believe that it was useful to engage in this teaching strategy. However, movement away from strong disagreement
on this belief is encouraging because research shows that self-directed learning is a key component of effective
science learning. The NSTA (2003) believes science educators should use inquiry-based teaching strategies, which
includes students engaging in self-learning.
34
ISSN 2538–7138 /Online/ (P. 19-38)
Theme 3: Changes in Beliefs about Science Content Knowledge After Completing the Course
Regarding beliefs about science content knowledge, nominal score increases were observed for half (n=5)
of the items (items 1, 6, 7, 8 and 10 in Table 6) with most of these registering the sentiment of neither agree nor
disagree (not statistically significant). The PSTs became slightly more uncertain about the difficulty of teaching
science, preparing teaching aids, and creating assessment tools. As noted, assessing the level of difficulty of a task
before actually experiencing it is a challenge. This may explain why they are still uncertain about their beliefs about
these aspects of science content knowledge.
Only one belief item about science content knowledge changed categories, with item 1 moving from agree
to strongly agree. Students became even more convinced that it was important for science textbooks to focus on
the application of life skills and classroom activities rather than theoretically elaborate on science topics. This re-
sult implies that these Saudi PSTs favoured applied approaches to teaching instead of theoretical and conceptual.
Eschewing the latter short changes students’ understanding of science concepts since learning best happens with
a combination of conceptual and practical (experiential) learning. To quote Duit and Tesch (2010), when teaching
science, “hands-on needs to include minds-on” (p. 26). This includes textbooks.
From another perspective, their strengthened belief around textbooks may reflect an insufficiently developed
scientific perspective required to accurately assess science textbooks. In particular, there is a strong epistemological
structure among the concepts, facts, relationships and practical applications of SCK, which has been shown to affect
PSTs’ confidence (Schoon & Boone, 1998), in this case perhaps creating a false sense of confidence when it comes
to assessing textbooks. Direct classroom experience is required in order to understand this structural relationship,
and the PSTs in this research sample may not yet have sufficient experience to critically judge textbook resources.
Conclusions and Recommendations
Results for Theme 1 (beliefs regarding their own self-efficacy) were statistically significant. Respondents’
scores indicated an increase in beliefs about their own self-efficacy after taking the course, although PSTs’ scores
still fell within the neither agree nor disagree category. Although increases were also observed for Theme 2 (beliefs
regarding science teaching methods and strategies) and Theme 3 (beliefs regarding science content knowledge),
they were slight and not statistically significant. General scores before and after the course fell within the neither
agree nor disagree category for these themes as well. Two themes were found to be not statistically significant –
science teaching methods and science content knowledge. Respectively, future science education course planners
can be encouraged that in-service training can modify PSTs’ beliefs about effective teaching methods. Teacher
candidates’ beliefs regarding the epistemological (knowledge) structure of science can also be modified through
short-term micro-teaching.
Because teaching requires pedagogical content knowledge and knowledge of teaching strategies, it is impera-
tive that science methods courses provide PSTs with learning opportunities to develop this aspect of practice. While
respecting this imperative, the overall results of this study imply that it was difficult to establish a considerable and
significant modification in Saudi female PSTs’ beliefs over a short period of time. It may be that Saudi female PSTs
require more or different instruction before they begin teaching science. Curriculum planners and PST instructors
should take this into account when redesigning the course for future cohorts. Especially, they need to engage in
evidence-based course development and heed best practices. And, longitudinal studies are encouraged so as to
track science teachers’ beliefs and actual practice in full-time employment. This recommendation applies to both
male and female PSTs, with the latter the focus of our research.
As one of the first studies in Saudi Arabia on female PSTs’ beliefs regarding science education, this research
addresses a gap in the existing literature. The results and aforementioned recommendations are relevant to science
education curriculum theorists, researchers, specialists and practitioners.
References
Alabdulkareem, S. (2016). The impact of science teachers’ beliefs on teaching science: The case of Saudi science teachers.
Journal of Education and Learning, 5(2), 233-249.
Alghamdi Hamdan, A., & Alsalouli, M. (2012). Saudi elementary school science teachers’ beliefs about teaching science in the
new millennium. International Journal of Science and Mathematics Education, 11(2), 501-525.
35
(P. 19-38) ISSN 2538–7138 /Online/
Al-Jazaeri, K. (2009). Science teachers’ beliefs at the second school stage in Damascus about the nature of science and sci-
entific knowledge. Proceedings of the Second Scientific Conference at the Educational Sciences College in Jerash University
(Role of Arab Teacher in the Knowledge Flow Age) [pp. 454-493]. Jordan, Jerash: College of Educational Sciences, Jerash
National University.
Ambusaidi, K., & Al-Balushi, M. (2012). A longitudinal study to identify prospective science teachers’ beliefs about science
teaching using the draw-a-science-teacher-test checklist. International Journal of Environmental and Science Education,
7(2), 291-311.
Anderson, R. D. (1997). The science methods course in the context of the total teacher education experience. Journal of Sci-
ence Teacher Education, 8(4), 269-282.
Andersen, A. M., Dragsted, S., Evans, R. H., & Sørensen, H. (2004). The relationship between changes in teachers’ self-efficacy
beliefs and the science teaching environment of Danish first-year elementary teachers. Journal of Science Teacher Edu-
cation, 15(1), 25-38.
Appleton, K. (1995) Student teachers’ confidence to teach science: Is more science knowledge necessary to improve self-
confidence? International Journal of Science Education, 17, 3, 357-369.
Appleton, K., & Kindt, I. (2002). Beginning elementary teachers’ development as teachers of science. Journal of Science Teacher
Education, 13(1), 43-61.
Arce, J., Bodner, M., & Hutchinson, K. (2014). A study of the impact of inquiry-based professional development experiences
on the beliefs of intermediate science teachers about “best practices” for classroom teaching. International Journal of
Education in Mathematics, Science and Technology, 2(2), 85-95.
Aslan, D., Tas, I., & Ogul, G. (2016). Pre- and in-service preschool teachers’ science teaching efficacy beliefs. Educational Research
and Reviews, 11(14), 1344-1350.
Aydogdu, B., & Peker, M. (2016). Science and mathematics teaching efficacy beliefs of pre-school teachers. Universal Journal
of Educational Research, 4(11), 2541-2550.
Bakir, S. (2016). What are their beliefs? What is their approach in practice? What is the profile of their science teachers and
professors? International Journal of Environmental and Science Education, 11(5), 587-602.
Bandura, A. (1997a). Self-Efficacy: The Exercise of Control. New York: Freeman.
Bandura, A. (1997b). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84, 191-215.
Berland, L. K., & Reiser, B. J. (2009). Making sense of argumentation and explanation. Science Education, 93(1), 26-55.
Bransford, J. D., Brown, A.L., & Cocking, R. R. (Eds.). (2000). How People Learn: Brain, Mind, Experience, and School. Washington,
DC: National Academy Press.
Bursal, M. (2008). Changes in Turkish pre-service elementary teachers’ personal science teaching efficacy, beliefs and science
anxieties during a science method course. Journal of Turkish Science Education, 5(1), 99-112.
Bursal, M. (2012). Changes in American pre-service elementary teachers’ efficacy beliefs and anxieties during a science methods
course. Science Education International, 23(1), 40-55.
Cantrell, P., Young, S., & Moore, A. (2003). Factors affecting science teaching efficacy of pre-service elementary teachers. Journal
of Science Teacher Education, 14(3), 177-192.
Cross, D. I. (2009). Alignment, cohesion, and change: Examining mathematics teachers’ belief structures and their influence
on instructional practices. Journal of Mathematics Teacher Education, 12(5), 325-346.
Davis, E. A. (2004). Knowledge integration in science teaching: Analysing teachers’ knowledge development. Research in
Science Education, 34(1), 21-53.
Dede, H., Yilmaz, A., & Ilhan, N. (2017). Investigation of the self-efficacy beliefs of pre-service science teachers in terms of
following and using the innovations in the field of education. Journal of Education and Training Studies, 5(2), 21-30.
Demir, S. (2015). Evaluation of critical thinking and reflective thinking skills among science teacher candidates. Journal of
Education and Practice, 6(18), 17-21.
Dewey, J. (1933). How We Think: A Restatement of the Relation of Reflective Thinking to the Educational Process. Lexington, MA:
Heath.
Duit, R., & Tesch, M. (2010, July 25-31). On the role of the experiment in science teaching and learning: Visions and the reality
of instructional practice. Proceedings of the 7th International Conference on Hands-on Science (pp. 17-30). Rethymno, Crete.
Erduran, S., & Jimenez-Aleixandre, M. P. (Eds.). (2008). Argumentation in Science Education. New York, NY: Springer.
Gibson, S., & Dembo, M. H. (1984). Teacher efficacy: A construct validation. Journal of Educational Psychology, 76(4), 569.
Alghamdi, A. H. K., & Azam, S. (2018). Differentiation in a Saudi pre-service science teacher program. Journal of Baltic Science
Education, 18(3), 1-35.
Hamdan Alghamdi, A. K. (2013). Pre-service teachers’ preferred methods of assessment: A perspective from Saudi Arabia.
Australian Journal of Teacher Education, 38(4), 1-25.
Ibrahim, B. (2012). Science teachers’ beliefs in the UNRWA schools in Jordan about the nature of science and their relationship
to some variables. Arab Educational Magazine for Education, Tunisia, 32(1), 71-97.
Kazempour, M. (2014). I can’t teach science! A case study of an elementary pre-service teacher’s intersection of science ex-
periences, beliefs, attitude, and self-efficacy. International Journal of Environmental and Science Education, 9(1), 77-96.
Kim, S. Y. (2012). Pre-service biology teachers’ learning to teach science through science methods courses. Journal of the
Korean Association for Science Education, 32(9), 1427-1442.
Kind, V. (2009). Pedagogical content knowledge in science education: Perspectives and potential for progress. Studies in Sci-
ence Education, 45(2), 169-204.
36
ISSN 2538–7138 /Online/ (P. 19-38)
Lemke, J. L. (1990). Talking Science: Language, Learning and Values. Westport, CT: Ablex.
Lieberman, A., & Grolnick, M. (2005). Educational reform networks: Changes in the forms of reform. In M. Fullan (Ed.), Funda-
mental Change. New York: Springer.
Luera, G. R., & Otto, C. A. (2005). Development and evaluation of an inquiry-based elementary science teacher education
programme reflecting current reform movements. Journal of Science Teacher Education, 16(3), 241-258.
Lumpe, A., Czerniak, C., Haney, J., & Seltyukova, S. (2012). Beliefs about teaching science: The relationship between elementary
teachers’ participation in professional development and student achievement. International Journal of Science Educa-
tion, 34(2), 153-166.
Mansour, N. (2009). Science teachers - beliefs and practices: Issues, implications and research agenda. International Journal
of Environmental & Science Education, 4(1), 25-48.
McNeill, K. L., & Krajcik, J. (2011). Claim, Evidence and Reasoning: Supporting Grade 5-8 Students in Constructing Scientific Expla-
nations. Boston, MA: Pearson Education.
National Science Teachers Association (1998). NSTA position statement: The national science education standards: A vision
for the improvement of science and learning. Science Scope, 65(5), 32-34.
National Science Teachers Association (2003). Standards for science teacher preparation. Faculty Publications: Department
of Teaching, Learning and Teacher Education, 86.
Nelson, B. S., & Hammerman, J. K. (1996). Reconceptualizing teaching: Moving toward the creation of intellectual communities
of students, teachers, and teacher educators. In M. W. McLaughlin & I. Oberman (Eds.), Teacher learning: New policies,
new practices (pp. 3-21). New York: Teachers College Press.
Nilsson, P., & Loughran, J. (2012). Exploring the development of pre-service science elementary teachers’ pedagogical content
knowledge. Journal of Science Teacher Education, 23(7), 699-721.
Ozfidan, B., Cavlazoglu, B., Burlbaw, L., & Aydin, H. (2017). Reformed teaching and learning in science education: A compara-
tive study of Turkish and US teachers. Journal of Education and Learning, 6(3), 23-30.
Pajares, M. F. (1992). Teachers’ beliefs and educational research: Cleaning up a messy construct. Review of Educational Research,
62(3), 307-332.
Palmer, D. H. (2006). Sources of self-efficacy in a science methods course for primary teacher education students. Research
in Science Education, 36(4), 337-353.
Plevyak, L. H. (2007). What do pre-service teachers learn in an inquiry-based science methods course? Journal of Elementary
Richardson, V. (1994). The consideration of beliefs in staff development. In V. Richardson (Ed.), Teacher Change and the Staff
Development Process: A Case of Reading Instruction (pp. 90-108). New York: Teachers College Press.
Sahin, E., Deniz, H., & Topçu, M. (2016). Predicting Turkish pre-service elementary teachers’ orientations to teaching science
with epistemological beliefs, learning conceptions, and learning approaches in science. International Journal of Envi-
ronmental and Science Education, 11(5), 515-534.
Saka, M., Bayram, H., & Kabapinar, F. (2016). The teaching processes of prospective science teachers with different levels of
science-teaching self-efficacy belief. Educational Sciences: Theory and Practice, 16(3), 915-941.
Sampson, V., & Blanchard, M. R. (2012). Science teachers and scientific argumentation: Trends in views and practice. Journal
of Research in Science Teaching, 49, 1122-1148. doi:10.1002/tea.21037.
Santau, A. O., Maerten-Rivera, J. L., Bovis, S., & Orend, J. (2014). A mile wide or an inch deep? Improving elementary pre-service
teachers’ science content knowledge within the context of a science methods course. Journal of Science Teacher Educa-
tion, 25(8), 953-976.
Santos, L. F. (2017). The role of critical thinking in science education. Journal of Education and Practice, 8(20), 159-173.
Saylan, A., Armagan, F., & Bektas, O. (2016). The relationship between pre-service science teachers’ epistemological beliefs and
preferences for creating a constructivist learning environment. European Journal of Science and Mathematics Education,
4(2), 251-267.
Schoon, K. J., & Boone, W. J. (1998). Self-efficacy and alternative conceptions of science of pre-service elementary teachers.
Settlage, J. (2000). Understanding the learning cycle: Influences on abilities to embrace the approach by pre-service elemen-
tary school teachers. Science Education, 84(1), 43-50.
Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15, 4-14.
Simon, S., Johnson, S., & Johnson, S. (2008). Professional learning portfolios for argumentation in school science. International
Journal of Science Education, 30(5), 669-688.
Tarmo, A., & Bevins, S. (2016). Pre-service science teachers’ epistemological beliefs and teaching reforms in Tanzania. Cogent
Education, 3, 1.
Tschannen-Moran, M., Hoy, A. W., & Hoy, W. K. (1998). Teacher efficacy: Its meaning and measure. Review of Educational
Research, 68(2), 202-248.
Uyanik, G. (2016). Investigation of the self-efficacy beliefs in teaching science and attitudes towards teaching profession of
the candidate teachers. Universal Journal of Educational Research, 4(9), 2119-2125.
Venville, G. J., & Dawson, V. M. (2010). The impact of a classroom intervention on grade 10 students’ argumentation skills,
informal reasoning, and conceptual understanding of science. Journal of Research in Science Teaching, 47, 952-977.
Watters, J. J., & Ginns, I. S. (2000). Developing motivation to teach elementary science: Effect of collaborative and authentic
learning practices in pre-service education. Journal of Science Teacher Education, 11(4), 301-321.
37
(P. 19-38) ISSN 2538–7138 /Online/
Wong, S. (2016). Development of teacher beliefs through online instruction: A one-year study of middle school science and
mathematics teachers’ beliefs about teaching and learning. Journal of Education in Science, Environment and Health,
2(1), 21-32.
Yakar, Z., & Turgut, D. (2017). Effectiveness of lesson study approach on pre-service science teachers’ beliefs. International
Education Studies, 10(6), 36-43.
Yenice, N. (2015). An analysis of science student teachers’ epistemological beliefs and metacognitive perceptions about the
nature of science. Educational Sciences: Theory and Practice, 15(6), 1623-1636.
Zembal-Saul, C. (2009). Learning to teach elementary school science as argument. Science Education, 93, 687-719.
Received: August 02, 2018 Accepted: December 23, 2018
Amani K. Hamdan Alghamdi BSc–BEd, MA Ed, PhD, Associate Professor, College of Education,
Curriculum and Pedagogy, Imam Abdulrahman Bin Faisal
University, Saudi Arabia.
E-mail: akhalghamdi@iau.edu.sa
Sun Young Kim Ph.D, Associate Professor, College of Education, Department of
Biology Education, Chosun University, Gwangju, South Korea.
E-mail: sykim519@chosun.ac.kr; mhsysj@gmail.com
38
A STUDY OF FACTORS
AFFECTING SCIENCE
ACHIEVEMENTS OF JUNIOR ISSN 1648-3898 /Print/
HIGH SCHOOL FEMALE ISSN 2538-7138 /Online/
STUDENTS
Chun-Chu Liu, Abstract. This research was the first

research integrating decision making trial
Tse-Yu Wang
and evaluation laboratory (DEMATEL)
and structural equation modeling (SEM)
to analysis factors, including interest in
science, value in science and confidence in
science, affecting female students’ science
Introduction achievements from the Trends in Inter-
national Mathematics and Science Study
Regarding topics related to science and mathematics achievements of (TIMSS) 2011. The research constructed
students, the debate over whether there is a significant difference between two causal models base on mutual effect of
male and female students in science and mathematics achievements has value in science and confidence in science
been ongoing. Thus far, some scholars still believe that there is a significant by DEMATEL analysis. The causality of two
difference between male and female students in learning achievement, while
factors in model 1 was value in science
other scholars hold an opposite view that there is no significant difference
affected confidence in science and model 2
in learning achievement between male and female students (Preckel, Goetz,
vice versa. According to the results of SEM
Pekrun, & Kleine, 2008; White, 2007). With the continuous advancement in
research tools and techniques, many scholars have applied different research analysis, fit indices of CFI, GFI and AGFI in
methods and techniques to study and verify the differences in learning model 1 were 0.851, 0.796 and 0.745; in
achievement between male and female students (Anglin, Pirson, & Langer, model 2 were 0.818, 0.772, and 0.873. And
2008; Ary, Jacobs, Irvine, & Walker, 2018; Ertl, Luttenberger, & Paechter, 2017; results showed that female students had
Kerkhoven, Russo, Land-Zandstra, Saxena, & Rodenburg, 2016; Quinn & better fitness in model 1; in other words, the
Cooc, 2015; Yarnell et al., 2015). Meanwhile, these tools and techniques have causality of value in science to confidence
also made it possible for researchers to make progress in the exploration of in science might have an important effect
physiological and psychological factors that impact learning achievement on the science achievement of female stu-
as well as to obtain a greater understanding of the difference in learning dents. Lastly, these findings could provide
achievement between the two genders. In particular, following large-scale
not only a different perspective for future
assessments of students’ learning achievement, such as the Trends in Inter-
researches but also a useful insight as a
national Mathematics and Science Study (TIMSS) held by the International
reference for policy making and improve-
Association for the Evaluation of Educational Achievement (IEA) and the
Programme for International Student Assessment (PISA) held by the Organ- ment by relevant decision makers.
isation for Economic Co-operation and Development (OECD), many scholars Key words: decision making trial and
began to use results of the assessments to study the differences in learning evaluation laboratory, science achieve-
achievement between the two genders and attempted to explore the actual ment, structural equation modeling, TIMSS.
situation and influencing factors that impact the differences in performance
between male and female students, to solve the black box of performance
differences in learning achievement between the two genders (Mills & Simon, Chun-Chu Liu
Chang Jung Christian University, Taiwan
2015; Nosek et al., 2009; Patrick, 2012; Preckel, Goetz, Pekrun, & Kleine, 2008; Tse-Yu Wang
White, 2007). With more discovering of factors influenced the difference in National Sun Yat-sen University, Taiwan
learning achievement of male and female students, many research results
39
FEMALE STUDENTS
(P. 39-50) ISSN 2538–7138 /Online/
of learning achievement showed that there was no significant difference between male and female students.
Additionally, under the influence of various factors on learning achievement, there is no significant performance
difference in learning between the two genders. In other words, in terms of learning ability, there is no significant
innate advantage for either gender.
In terms of the outstanding academic performance by Taiwanese students in various international assessment
tests, it is easy to find that the performance of female students in science and mathematics is not inferior to that of
male students. For instance, in the TIMSS 2011, the assessment results of 8th-grade students indicated that female
students scored higher on mathematics and chemistry than did male students. However, according to the statis-
tics of the Ministry of Education of Taiwan, the number of female students in natural sciences, computer science,
industrial engineering, architecture, and urban planning accounts for less than 30% of all students in colleges and
universities in Taiwan (Ministry of Education Republic of China (Taiwan), 2017). In terms of career development,
very few female students continue to choose careers related to the majors or other scientific research that they
originally studied. This condition shows that Taiwan’s current social stereotype still exists and that there is still room
for improving gender equality in education in Taiwan. However, review of past relevant research revealed that the
discussions on science achievement have mostly focused on gender differences; few researchers have explored
and analysed female students. There is a significant difference between East Asian students and European and
American students in the psychological factor of self-enhancement, which has a more significant impact on learn-
ing achievement (Falk, Heine, Yuki, & Takemura, 2009; Green, Nelson, Martin, & Marsh, 2006; Heine, 2003; Heine &
Hamamura, 2007; Trope, 1986). Therefore, it is necessary to further clarify and discuss the causality between the
factors that impact the learning achievement of students in Taiwan and other East Asian countries.
The TIMSS assessment was held every four years, since Taiwan jointed TIMSS assessment in 1999, there were
five assessments in 1999, 2003, 2007, 2011 and the latest assessment was in 2015, respectively. However, in present
researches on gender difference in science and mathematics achievements included for Taiwan, most researches
still used TIMSS 2007 as the data to analyse the gender difference in science and mathematics achievements
without progressing (Tsai, Yang, & Chang, 2015). For this reason, this research first reviewed the relevant literature
and integrated the Decision Making Trial and Evaluation Laboratory (DEMATEL) method and Structural Equation
Modeling (SEM) method (Baye & Monseur, 2016; Mills & Simon, 2015; Neuschmidt, Barth, & Hastedt, 2008; Hoyle,
1995; Nosek et al., 2009; Pavešić, 2008; Reilly, Neumann, & Andrews, 2015; Tomson, 2008; Tsai, Yang, & Chang, 2015).
Then, a causal model for impact on science achievement was constructed through the DEMATEL method, and the
TIMSS 2011 assessment results on science achievement of Taiwanese 8th grade female students as well as the SEM
were used to test the model. Finally, based on the analysis results, the research proposed practical suggestions and
future research directions for decision makers and researchers of current educational institutions.
Literature Review
There had been many related researches and discussions before the TIMSS regarding the issue of gender dif-
ferences in mathematics and science achievements. Weinburgh (1995) used meta-analysis to analyse the literature
from 1970 to 1991 on gender differences in attitudes towards science. The results showed that male and female
students have significant differences in their attitudes towards science. For average students, male students have
a higher positive attitude towards science, while for high-achieving students, female students have a higher posi-
tive attitude towards science. Anglin, Pirson, and Langer (2008) used mindful learning as a moderating variable to
explore the differences in mathematics achievement between the two genders. In the absence of mindful learning,
male students outperform female students in terms of mathematics achievement.
However, when both male and female students are engaged through mindful learning, there is no difference
in their mathematics achievement. Therefore, this research believes that mindful learning can effectively enhance
female students’ mathematics learning performance and narrow the gap between the two genders with regard to
mathematics learning. Costes, Rowley, Britt, and Woods (2008) explored the difference between male and female
students in terms of self-perception and self-awareness in mathematics and science capabilities based on 4th, 6th
and 8th-grade students in schools in the southeast region of the United States (US). The results showed no signifi-
cant difference between male and female students in their mathematics and science capabilities. However, male
students tend to be affected by the stereotype that male students perform better in mathematics and science
than female students, which in turn affects their self-awareness. Under the influence of stereotype, the impact of
self-awareness of female students is not significant. Tomson (2008) used the TIMSS 2003 to analyse the differences
40
ISSN 1648–3898 /Print/ A STUDY OF FACTORS AFFECTING SCIENCE ACHIEVEMENTS OF JUNIOR HIGH SCHOOL
FEMALE STUDENTS
ISSN 2538–7138 /Online/ (P. 39-50)
in science achievement between the two genders of Australian eighth-grade students. The results showed that
there is a significant difference in science achievement between the two genders, and the educational level of the
parents as well as books in the family have a significant impact on the differences in science achievement between
the two genders. Pavešić (2008) used the TIMSS 2003 to analyse the difference between the two genders in the sci-
ence achievement of Slovenian eighth-grade students after the implementation of the new compulsory education
system. It was found that the new nine-year compulsory education system significantly narrowed the differences in
science achievement between the two genders. Lindberg, Hyde, Petersen, and Linn (2010) used the meta-analysis to
analyse 242 researches on the mathematics achievement of the two genders between 1990 and 2007. No significant
difference in mathematics learning achievement was found between the two genders; this research also analysed
the educational database of long-term surveys of adolescents in the US. The results also showed that there was no
significant difference between the two genders in mathematics performance. Wai, Cacchio, Putallaz, and Makel (2010)
collected 30 years of Scholastic Aptitude Test (SAT) scores from Duke University to analyse the low participation rate
of women in STEM careers. They found that women have better performance in language and writing ability, while
there are still more male students than female students with high scores on mathematics and science tests. This
finding showed that the effects of physiological factors continue to decrease for the low participation of women
in careers related to science, engineering, and mathematics. The low participation of women should be attributed
to social factors. Abu-Hilal, Abdelfattah, Shumrani, Dodeen, and Abdulajabber (2014) used the TIMSS 2007 assess-
ment results to explore the relationship between variables such as the self-concept, subject value, and mathematics
and science achievement of Saudi Arabia’s eighth graders. At the same time, they explored the difference in the
relationship models between male and female students. Their analysis showed no significant difference between
male and female students in the overall model, but the self-concept of male students in mathematics and science is
higher than that of female students. However, self-improvement in mathematics and science of female students is
better than that of male students. Reuben, Sapienza, and Zingales (2014) first conducted a test in mathematics and
science to examine the differences in abilities between male and female students. Their results showed that even
though there is no significant difference between male and female students in mathematics and science abilities, the
number of male students who are employed is twice as that of female students in job-seeking situations, indicating
that the negative impact of stereotype on female education and career choice may not decrease with the progress
of society but may actually increase. This result echoes the research result and conclusion of Nosek et al. (2009).
Review of relevant research showed that scholars believe that in recent years, there has been a significant
decrease in differences between the two genders in their learning performance. Some scholars have even further
discovered that in terms of performance in mathematics and science, women are superior to men in certain abili-
ties, such as verbal reasoning, in the content of textbooks (Ding, Song, & Richardson, 2007; Lohman & Lakin, 2009;
Preckel, Goetz, Pekrun, & Kleine, 2008; Spelke, 2005; Strand, Deary, & Smith, 2006). Therefore, even though many
researches on mathematics and science abilities and the learning achievement of male and female students have
confirmed that there are no innate differences in learning ability between the two genders, the current state of female
students in science learning and career development in science related fields still shows women at a disadvantage,
which indicates that there is still room for improvement for the development of science careers for female students.
In the case of Taiwan, gender-related issues in science learning have been taken more attention after imple-
menting the Gender Equality Education Act and renaming of the Gender Equality Education Committee in 2004.
However, compared with the exploration of the factors impacting mathematics and science learning achievement
of students, there has been a lack of research on issues related to gender differences or learning achievement of
women, and there is still a significant difference in social stereotypes and the confidence and value of learning
between male and female students. In other words, the notion that men are more suitable for mathematics and
science-related learning activities, mathematics and science-related research, and mathematics and science-related
careers also exists in Taiwan. In view of this, the research questions arose as follows:
•• What was the causality of factors that impact science achievements of female student based on TIMSS?
Therefore, considering the differences in self-enhancement between East Asian and European and American
students as well as the lack of research on issues such as the learning achievement of women, this research first
constructed a causal model for the impact of the relevant factors on science achievement of female students
through the DEMATEL method using the assessment results of the TIMSS 2011 as analysis data. It then verified the
model through SEM, to understand the correlations between the learning achievement and its influencing fac-
tors of Taiwanese students as well as the current inadequacy of female students and the factors and aspects that
required special attention in enhancing science learning achievement.
41
FEMALE STUDENTS
(P. 39-50) ISSN 2538–7138 /Online/
Decision Making Trial and Evaluation Laboratory (DEMATEL)
DEMATEL is a multi-criteria decision-making (MCDM) method developed and formally proposed by Gabus
and Fontela (1972). DEMATEL has been used to study and understand complex and difficult problems such as
race, famine, environmental protection and energy issues (Falatoonitoosi, Leman, Sorooshian, & Salimi, 2013).
The DEMATEL method first analyses problems through hierarchical structure and causality and then establishes a
network relationship map (NRM) like SEM to find the corresponding solutions (Fontela & Gabus, 1976; Kuo & Cheng,
2013; Lin, Chen, & Tzeng, 2009). Because the DEMATEL analysis method is mainly based on experts’ experience
and judgement to evaluate the correlations of the factors and therefore the correlations of the factors constructed
through this method can help researchers to clarify the correlations between unknown and complex factors, and
these correlations are more in line with reality (Fontela & Gabus, 1976; Gandhi, Mangla, Kumar, & Kumar, 2015;
Ilieva, 2017; Si, You, Liu, & Zhang, 2018).
According to the summary of Fontela and Gabus (1976) and Kumar and Dixit (2018), the analytical calculation
process of the DEMATEL method was mainly divided into the following five steps: defining the research questions
and factors and identifying relationships, establishing the initial relation matrix, establishing the normalized initial
relation matrix, establishing the total initial relation matrix, and calculating the degrees of prominence and relation
and plotting the causal diagram. The steps were described as follows:
1. Defining the research questions and factors as well as identifying causality:
The factors in the system as well as their definitions were determined through discussion, literature
review, and brainstorming, and the factors were used for paired comparison to evaluate the cognition
of impact on the factors of each respondent; the integer numbers 0, 1, 2 3, and 4 were used for evalu-
ation scale. Among them, 0 means no impact at all; 1 means low impact; 2 means medium impact; 3
means high impact; and 4 means extremely high impact.
2. Establishing the initial relation matrix:

After the identification of the factors in pairs with the help of H number of experts in the research field
and confirming the degrees of prominence and influence of each factor, the n.n non-negative matrix
was obtained as . Then, the scoring matrix of H number of experts was synthesized and
averaged, which is called initial relation matrix A = aij, as shown in formula (1):
(1)
where K=the number of respondents with 1 ≤ ik ≤ H; and N=the number of barriers criteria.
3. Establishing the normalized initial relation matrix:

When the initial relation matrix is normalized, a normalized relation matrix D can be obtained, as shown in
formula (2):
D=A.S (2)
where

4. Establishing the total initial relation matrix:
After the normalized initial relation matrix is established, the formula T=D/(I-D) can be used to calculate
the total initial relation matrix; I is the identity matrix.
5. Calculating the degrees of prominence and relation and plotting the causal diagram:
Calculating the sums of rows and columns according to the total initial relation matrix;
(ri + cj) is the prominence, denoting the degree of factor being affected; and (ri - cj) is the relation. If
positive, it means that this factor can impact other factors and is a “cause”. If negative, it means that this
42
FEMALE STUDENTS
ISSN 2538–7138 /Online/ (P. 39-50)
factor is an impacted factor and is a “result”. Finally, a causal diagram is plotted based on the degrees
of prominence and relation.
Currently, the DEMATEL method has been widely used in various fields. In addition to being used to explore
the correlations between multiple criterions, it has also been used for supplier selection, business strategy deriva-
tion, waste management and organizational improvement. Wang, Lin, Chung, and Lee (2012) applied the DEMATEL
method to analyse and improve the design of high-tech factory buildings. Li, Hu, Deng, and Mahadevan (2014) used
DEMATEL to identify key success factors for disaster management. Gandhi et al. (2015) and Li and Mathiyazhagan
(2018) applied the DEMATEL method to explore the key elements of supply chain management for heavy equip-
ment and automotive parts manufacturers. Uygun, Kaçamak, and Kahraman (2015) applied DEMATEL to analyse
and select the outsourcing suppliers of telecommunications companies. Kumar and Dixit (2018) evaluated the key
barriers to e-waste management with the DEMATEL method. Applying this method to manufacturing companies,
Quezada, López-Ospina, Palominos, and Oddershede (2018) constructed strategic maps and identified the relation-
ships between factors in the strategy map.
Data Collection
As to DEMATEL analysis, this research invited a total of 12 experts, including scholars with backgrounds re-
lated to this research, teachers from the national high school who teach science-related courses and women with
scientific backgrounds and working in science-related fields. The distribution of 12 experts was shown as Table 1.
Table 1. Aggregation of experts.
Experts’ background Number of experts
Scholars with backgrounds related to this research 4
Teachers from the national high school who teach science-related courses 4
Women with scientific backgrounds and working in science-related fields 4
Total 12
As to SEM analysis, from the TIMSS 2011 Science Achievement Assessment Report, there were 304,037 8th-
grade students participated in TIMSS 2011 of Taiwan, and 5,042 students were valid sample. In these 5,042 students,
the numbers of male students were 2,594, and female students were 2,448. Therefore, this research used 2,448
female students as the research sample to analyse the fitness of causal model.
In DEMATEL analysis, the research designed expert questionnaires based on the TIMSS 2011 Science Achieve-
ment Assessment Report. The four factors which were also the variables of the research including the interest in
science, value in science, confidence in science, and the science achievement of eighth-grade students. The fac-
tors were used for paired comparison to evaluate the cognition of impact on the factors of each respondent; the
integer numbers 0, 1, 2 3, and 4 were used for evaluation scale. Among them, 0 means no impact at all; 1 means
low impact; 2 means medium impact; 3 means high impact; and 4 means extremely high impact.
As to SEM analysis, the analysis variables and items in this research also were gathered from the TIMSS 2011
Science Achievement Assessment Report same as to the factors of DEMATEL analysis. Therefore, the four variables
to be analysed in this research also were the interest in science, value in science, confidence in science, and the
science achievement of eighth-grade students. According to the TIMSS 2011 Science Achievement Assessment
Report, there were five questions for the variable interest in science, six questions for the variable value in science,
and six questions for the variable confidence in science. The scores of science achievement were summed and
averaged. The research items of four variables were shown as Table 2.
43
FEMALE STUDENTS
(P. 39-50) ISSN 2538–7138 /Online/
Table 2. Research factors/variables and items.
Factor/Variable Items
I enjoy learning science.

I wish I did not have to study science.
Interest in science Science is boring.
I learn many interesting things in science class.
I like science.
I think learning science will help me in my daily life.

I need science to learn other school subjects.
I need to do well in science courses to get into the University of my Choice.
Value in science
I need to do well in science courses to get the job I want.
I would like a job that involves using science.
It is important to do well in science courses.
I usually do well in science courses.

Science is harder for me than for many of my classmates.
I am just not good at science.
Confidence in science
I learn things quickly in science courses.
My teacher tells me I am good at science.
Science is harder for me than any other subject.
Science achievement Average of all science test subjects

Source: TIMSS 2011 Science Achievement Assessment Report.
Therefore, the analysis procedure of the research was to construct the casual model by DEMATEL analysis in
the first place. Secondly, SEM analysis was conducted to analyse the fitness of the casual model.
Data Analysis
The research used Microsoft Excel 2010 to run all analytical processes of DEMATEL analysis included matrix
calculating and plotting the causal diagram. The IBM SPSS Amos 20.0 was used to run the fitness indices of SEM
analysis.
Results of Research
DEMATEL Analysis
As part of the DEMATEL analysis, after collecting the questionnaires from the 12 experts, the average initial
relation matrix (A) was calculated through the initial relation matrix of each expert, as shown in Table 3.
Table 3. The average initial relation matrix (A).
Interest Value Confidence

Factors Science achievement
in science in science in science
Interest in science 0 2.2 1.4 2.2
Value in science 1.8 0 2.2 2.6
Confidence in science 1.8 1.8 0 2.8
Science achievement 1 1.8 2 0
By normalization the average initial relation matrix (A), the normalized initial relation matrix (D) was calculated,
as shown in Table 4.
44
FEMALE STUDENTS
ISSN 2538–7138 /Online/ (P. 39-50)
Table 4. Normalized initial relation matrix.

Interest in science 0 0.333 0.212 0.333
Value in science 0.273 0 0.333 0.394
Confidence in science 0.273 0.273 0 0.424
Science achievement 0.152 0.273 0.303 0
After the normalized initial relation matrix (D) was calculated, the total initial relation matrix (T) was further
calculated using the identity matrix (I), as shown in Table 5.
Table 5. Total initial relation matrix.

Interest in science 0 1.971 0 2.383
Value in science 0 0 2.142 2.64
Confidence in science 0 2.053 0 2.585
Science achievement 0 0 0 0
Finally, the sums of rows and columns in the total initial relationship matrix (T) were calculated separately to
obtain the prominence and relation, as shown in Table 6.
Table 6. Degree of the total initial relation of factors.
Factors ri cj ri + cj ri - cj Group
Interest in science 4.354 0 4.354 4.354 cause
Value in science 4.782 4.024 8.806 0.758 cause
Confidence in science 4.638 2.142 6.78 2.496 cause
Science achievement 0 7.608 7.608 -7.608 effect
Based on the prominence and relation, the causal diagram between the four factors was plotted, as shown
in Figure 1.
45
FEMALE STUDENTS
(P. 39-50) ISSN 2538–7138 /Online/
Figure 1. Causal diagram of factors.
Analysis of SEM
The results of the DEMATEL analysis indicated that the three factors such as interest in science, value in sci-
ence and confidence in science could impact science achievement, and interest in science could impact value
in science. However, further analysis showed that value in science and confidence in science could impact each
other. Therefore, in the analysis of SEM, this research needed to put the factors in two models for further analysis,
as shown in Figure 2. In Model 1, the causality between value in science and confidence in science was that value
in science could impact confidence in science. In Model 2, the causality between value in science and confidence
in science was that confidence in science could impact value in science.
Figure 2. Analysis model of SEM.
46
FEMALE STUDENTS
ISSN 2538–7138 /Online/ (P. 39-50)
The analysis results of the two models were summarized in Table 7.
Table 7. Analysis results of two models.
Model 1 Model 2
Fitness indices
Female All students Female All students
χ2/df 28.309 84.536 24.298 78.193
RMSEA 0.107 0.129 0.099 0.124
CFI 0.851 0.779 0.818 0.719
GFI 0.796 0.709 0.772 0.647
AGFI 0.745 0.636 0.873 0.796
value-> confidence 29.341*** 44.884*** - -

CR
confidence -> value - - 4.832*** 12.013***
In Model 1, the research first used the data of all Taiwanese students in the TIMSS 2011 for analysis. The results
of the analysis showed that the degree of freedom ratio of chi-square test is 84.536, Root Mean Square Error of
Approximation (RMSEA) is 0.129, comparative fit index (CFI) is 0.779, goodness-of-fit (GFI) is 0.709 and adjusted-GFI
(AGFI) is 0.636. The composite reliability (CR) value for the path of value in science to confidence in science is 44.884,
and the p-value is significant. However, in analysing data of female students, the overall fitness index improved
significantly. The degree of freedom ratio of chi-square test was 28.309, RMSEA was 0.107, CFI was 0.851, GFI was
0.799, and AGFI was 0.745. The CR value for the path of value in science to confidence in science was 29.341; the
ρ value was significant.
In Model 2, in analysing the data of all students, the results showed that the degree of freedom ratio of the
chi-square test is 78.513, RMSEA was 0.124, CFI was 0.718, GFI was 0.646, and AGFI was 0.796, while the CR value for
the path of confidence in science to value in science was 12.013; the p value was significant. However, in analysing
data of female students, the overall fitness index was also significantly improved. The degree of freedom ratio of
chi-square test was 24.298, RMSEA was 0.099, CFI was 0.818, GFI was 0.772, and AGFI was 0.873. The CR value for
the path of confidence in science to value in science was 4.832; the p value was significant.
Discussion
Nowadays, the proportion of women in science and careers has continually increased in many countries. And
the issue of gender differences in science achievement was noticed and researched with it by governments and
scholars too. Moreover, since 1995, IEA held the first TIMSS assessment, it helped not only the research scope of
gender differences extend from single country to international comparison, but also explored more factors which
might affect gender differences of science achievement (Baye & Monseur, 2016; Miller, Eagly, & Linn, 2015; Mills &
Simon, 2015; Neuschmidt, Barth, & Hastedt, 2008; Nosek et al., 2009; Pavešić, 2008; Penner, 2003; Tomson, 2008).
However, results of these researches were all focus on differences of male and female students on respective
factors, and these results have merely explained the cause of gender differences in science achievement. That’s
Scantlebury (2012) considered that gender issue in science education still was part of the conversation. Although
some researches employed SEM to link science achievement with different factors and examined the invariant
across gender to explain gender differences in science achievement, there were measurement errors which existed
under the premise of examining male and female students with the same model (Abu-Hilal, Abdelfattah, Shumrani,
Dodeen, & Abdulajabber, 2014; Tsai, Yang, & Chang, 2015).
Therefore, this research used the DEMATEL method to construct a causal model that could impact science
achievements of female students through experts’ judgments at first and then applied the SEM to verify the model.
According to the analysis results of DEMATEL, the factors of cause were interest in science, value in science, confi-
47
FEMALE STUDENTS
(P. 39-50) ISSN 2538–7138 /Online/
dence in science, and science achievement was the factor of effect. Among these four factors, interest in science,
value in science and confidence in science all could impact science achievement and interest in science also could
impact value in science too. However, the two factors of value in science and confidence in science could impact
each other. In consideration of this result, it implied the causality of value in science and confidence in science
might have key impact to the science achievements of female students. This is one of the main research findings
in this research.
Furthermore, based on the result of DEMATEL analysis, two causal models occurred and compared the
analysis results of fitness and significance, and that female students have better results under the path of model
1. In other words, the science achievement of female students was impacted by the causality of value in science
to confidence in science. This is the second major research finding, progress and difference compared to previous
researches (Abu-Hilal, Abdelfattah, Shumrani, Dodeen, & Abdulajabber, 2014; Baye & Monseur, 2016; Miller, Eagly,
& Linn, 2015; Mills & Simon, 2015; Neuschmidt, Barth, & Hastedt, 2008; Nosek et al., 2009; Pavešić, 2008; Penner,
2003; Tomson, 2008; Tsai, Yang, & Chang, 2015).
Conclusions
In real life, the factors that the social sciences seek to explore often impact each other. Exploring how to make
the correlations between multiple factors better in line with actual situations has always been a very important
research topic. Therefore, this research was the first research for integrating MCDM method and statistical method to
propose a more practical approach to explore the relation between science achievement and factors influencing it
on students. This approach could provide different and realistic way to construct research models for future research.
As to research issue, present researches still focus on the difference between male and female students
in science and mathematics achievements and proposed some way to enhance the science and mathematics
achievements of female students. Few researches used female students as the main body of analysis in research.
The findings of this research included casual model and the causality of value in science and confidence in science
could provide not only more integral analysis model but also a different perspective and insight to explore the
research issues, like gender difference and science and mathematics achievements of female students for future
researches and also could be used as a reference for policy making and improvement by relevant decision makers.
These statements were also the importance and contribution of this research.
Acknowledgements
The authors would like to thank the support from Ministry of Science and Technology (MOST), Taiwan. (MOST
104-2511-S-309-001-).
References
Abu-Hilal, M. M., Abdelfattah, F. A., Shumrani, S. A., Dodeen, H., & Abdulajabber, A. S. (2014). Mathematics and science achieve-
ments predicted by self-concept and subject value among 8th grade Saudi students: Invariance across gender. International
Perspectives in Psychology: Research, Practice, Consultation, 3(4), 268-283.
Anglin, L. P., Pirson, M., & Langer, E. (2008). Mindful learning: A moderator of gender differences mathematics performance.
Journal of Adult Development, 15(3/4), 132-139.
Ary, D., Jacobs, L. C., Irvine, C. K. S., & Walker, D. (2018). Introduction to Research in Education. Boston, MA, US: Wadsworth, Cen-
gage Learning.
Baye, A & Monseur, C. (2016). Gender differences in variability and extreme scores in an international context. Large-scale As-
sessments in Education, 4(1), 1-16.
Costes, B. K., Rowley, S. J., Britt, A. H., & Woods, T. A. (2008). Gender stereotypes about mathematics and science and self-perceptions
of ability in late childhood and early adolescence. Merrill-Palmer Quarterly, 54(3), 386-409.
Ding, C. S., Song, K., & Richardson, L. I. (2007). Do mathematical gender differences continue? A longitudinal study of gender
difference and excellence in mathematics performance in the U.S. Educational Studies, 40(3), 279-295
Ertl, B., Luttenberger, S., & Paechter, M. (2017). The impact of gender stereotypes on the self-concept of female students in STEM
subjects with an under-representation of females. Frontiers in Psychology, 8, 1-11.
Falatoonitoosi, E., Leman, Z., Sorooshian, S., & Salimi, M. (2013). Decision-making trial and evaluation laboratory. Research Journal
of Applied Sciences, Engineering and Technology, 5 (13), 3476-3480.
Falk, C. F., Heine, S. J., Yuki, M., & Takemura, K. (2009). Why do Westerners self-enhance more than East Asians? European Journal
of Personality, 23(3), 183-203.
48
FEMALE STUDENTS
ISSN 2538–7138 /Online/ (P. 39-50)
Fontela, E. & Gabus, A. (1976). The DEMATEL observer, DEMATEL 1976 Report. Switzerland Geneva: Battelle Geneva Research Centre.
Gabus, A. & Fontela, E. (1972). World Problems an Invitation to Further Thought within the Framework of DEMATEL. Switzerland
Geneva: Battelle Geneva Research Centre.
Gandhi, S., Mangla, S. K., Kumar, P., & Kumar, D. (2015). Evaluating factors in implementation of successful green supply chain
management using DEMATEL: A case study. International Strategic Management Review, 3(1/2), 96-109.
Green, J., Nelson, G., Martin, A. J., & Marsh, H. (2006). The causal ordering of self-concept and academic motivation and its effect
on academic achievement. International Education Journal, 7(4), 534-546.
Heine, S. J. & Hamamura, T. (2007). In search of East Asian self-enhancement. Personality and Social Psychology Review, 11(1), 1-24.
Heine, S. T. (2003). Making sense of East Asian self-enhancement. Journal of cross-cultural psychology, 34(5), 596-602.
Hoyle, R. H. (1995). Structural equation modeling: Concepts, issues, and applications. Thousand Oaks, CA, US: Sage Publications.
Ilieva, G. (2017). Group Decision Analysis with Interval Type-2 Fuzzy Numbers. Cybernetics and Information Technologies, 17(1), 31-44.
Kerkhoven, A. H., Russo, P., Land-Zandstra, A. M., Saxena, A., & Rodenburg, F. J. (2016). Gender stereotypes in science education
resources: A visual content analysis. PLoS ONE, 11(11), 1-13.
Kumar, A. & Dixit, G. (2018). Evaluating critical barriers to implementation of WEEE management using DEMATEL approach.
Resources, Conservation & Recycling, 131, 101-121.
Kuo, S. Y., & Chen, S. C. (2013). Causal analysis of air passenger decision-making. Journal of Aeronautics, Astronautics and Aviation,
Series A, 45(2), 85-92.
Li, Y. & Mathiyazhagan, K. (2018). Application of DEMATEL approach to identify the influential indicators towards sustainable
supply chain adoption in the auto components manufacturing sector. Journal of Cleaner Production, 172, 2931-2941.
Li, Y., Hu, Y., Zhang, X., Deng, Y., & Mahadevan, S. (2014). An evidential DEMATEL method to identify critical success factors in
emergency management. Applied Soft Computing, 22, 504-510.
Lin, C. W., Chen, S. H., & Tzeng, G. H. (2009). Constructing a cognition map of alternative fuel vehicles using the DEMATEL method.
Journal of Multi-Criteria Decision Analysis, 16(1/2), 5-19.
Lindberg, S. M., Hyde, J. S., Petersen, J. L., & Linn, M. C. (2010). New trends in gender and mathematics performance: A meta-
analysis. Psychological Bulletin, 136(6), 1123-35.
Lohman, D. F. & Lakin, J. M. (2009). Consistencies in sex differences on the Cognitive Abilities Test across countries, grades, test
forms, and cohorts. The British Journal of Educational Psychology, 79(2), 389-407.
Miller, D. I., Eagly, A. H., & Linn, M. C. (2015). Women’s Representation in Science Predicts National Gender-Science Stereotypes:
Evidence from 66 nations. Journal of Educational Psychology, 107(3), 631-644.
Mills, J. D. & Simon, M. E. (2015). Exploring eighth-grade minority and female students’ statistical literacy skills using TIMSS. Middle
Grades Research Journal, 10(3), 1-22.
Ministry of Education Republic of China (Taiwan). (2017). e100-1 Enrollment. All Levels-by year & gender. Retrieved from https://
english.moe.gov.tw/cp-28-14508-95005-1.html
Neuschmidt, O, Barth, J., & Hastedt, D. (2008). Trends in gender differences in mathematics and science (TIMSS 1995-2003).
Studies in Educational Evaluation, 34 (2), 56-72.
Nosek, B. A., Symth, F. L., Sriram, N., Linder, N. M., Devos, T., Ayala, A., Bar-Anan, Y., Bergh, R., Cai, H., Gonsalkorale, K., Kesebir, S.,
Maliszewski, N., Neto, F., Olli, E., Park, J., Shiomura, K., Tulbure, B. T., Wiers, R. W., Somogyi, M., Akrami, N., Ekehammar, B.,
Vianello, M., Banaji, M. R., & Greenwald, A. G. (2009). National differences in gender-science stereotypes predict national
sex differences in science and math achievement. Proceedings of the National Academy of Science of the United States of
America, 106(26), 10593-10597.
Patrick, A. O. (2012). Unequal achievement of science undergraduates: Does sex influence the differences? US-China Education
Review B, 6, 578-594.
Pavešić, B. J. (2008). Science achievement, gender differences, and experimental work in classes in Slovenia as evident in TIMSS
studies. Studies in Educational Evaluation, 34(2), 94-104.
Penner, A. M. (2003). International gender×item difficulty interactions in mathematics and science achievement tests. Journal
of Educational Psychology, 95 (3), 650-655.
Preckel, F., Goetz, T., Pekrun, R., & Kleine, M. (2008). Gender differences in gifted and average-ability students comparing girls’ and
boys’ achievement, self-concept, interest, and motivation in mathematics. Gifted child quarterly, 52(2), 146-459.
Quezada, L. E., López-Ospina, H. A., Palominos, P., & Oddershede, A. M. (2018). Identifying causal relationships in strategy maps
using ANP and DEMATEL. Computers & Industrial Engineering, 118, 170-179.
Quinn, D. M. & Cooc, N. (2015). Science achievement gaps by gender and race/ethnicity in elementary and middle school. Trends
and Predictors, 44(6), 336-346.
Reilly, D., Neumann, D. L., & Andrews, G. (2015). Sex differences in mathematics and science achievement: A meta-analysis of
national assessment of educational progress assessments. Journal of Educational Psychology, 107(3), 645-662.
Reuben, E., Sapienza, P., & Zingales, L. (2014). How stereotypes impair women’s careers in science. Proceedings of the National
Academy of Science of the United States of America, 111(12), 4403-4408.
Scantlebury, K. (2012). Still part of the conversation: Gender issues in science education. In B. Fraser, K. Tobin, & C. McRobbie (Eds.),
Second international handbook of science education (pp. 499-512). New York, NY, US: Springer.
Si, S. L., You, X. Y., Liu, H. C., & Zhang, P. (2018). DEMATEL technique - A systematic review of the state-of-the-art literature on
methodologies and applications. Mathematical Problems in Engineering, 2018, article ID: 3696457, 1-33.
Spelke, E. S. (2005). Sex differences in intrinsic aptitude for mathematics and science? A critical review. American Psychologist,
60(9), 950-958.
49
FEMALE STUDENTS
(P. 39-50) ISSN 2538–7138 /Online/
Strand, S., Deary, I. J., & Smith, P. (2006). Sex differences in cognitive abilities test scores: A UK national picture. British Journal of
Educational Psychology, 76(3), 463-480.
Tomson, S. (2008). Examining the evidence from TIMSS: Gender differences in year 8 science achievement in Australia. Studies
in Educational Evaluation, 34(2), 73-81.
Trope, Y. (1986). Self-enhancement and self-assessment in achievement behaviour. In R. M. Sorrentino & E. T. Higgins (Eds.), Handbook
of motivation and cognition: Foundations of social behaviour (pp. 350-378). New York, NY, US: Guilford Press.
Tsai, L. T., Yang, C. C., & Chang, Y. J. (2015). Gender differences in factors affecting science performance of eighth grade Taiwan
students. The Asia-Pacific Education Researcher, 24(2), 445-456.
Uygun, Ö., Kaçamak, H., & Kahraman, Ü. A. (2015). An integrated DEMATEL and Fuzzy ANP techniques for evaluation and selection
of outsourcing provider for a telecommunication company. Computers & Industrial Engineering, 86, 137-146.
Wai, J., Cacchio, M., Putallaz, M., & Makel, M. C. (2010). Sex differences in the right tail of cognitive abilities: A 30 year examina-
tion. Intelligence, 38(4), 412-423.
Wang, W. C., Lin, Y. H., Chung, C. H., & Lee, M. T. (2012). DEMATEL-based model to improve the performance in a matrix organiza-
tion. Expert Systems with Applications, 39(5), 4978-4986.
Weinburgh, M. (1995). Gender differences in student attitudes toward science: A meta-analysis of the literature from 1970 to
1991. Journal of Research in Science Teaching, 32(4), 387-398.
White, B. (2007). Are Girls Better Readers than Boys? Which Boys? Which Girls? Canadian Journal of Education, 30(2), 554-581.
Yarnell, L. M., Stafford, R. E., Neff, K. D., Reilly, E. D., Knox, M. C., & Mullarkey, M. (2015). Meta-analysis of gender differences in
self-compassion. Self and Identity, 14(5), 499-520.
Received: September 21, 2018 Accepted: December 26, 2018
Chun-Chu Liu Ph. D., Professor, College of Continuing Education, Chang Jung
Christian University, No.1,Changda Rd., Gueiren District, Tainan City
71101, Taiwan.
E-mail: lcc@mail.cjcu.edu.tw
Tse-Yu Wang Ph. D. Candidates, Department of Business Management, National
Sun Yat-sen University, 70 Lienhai Rd., Kaohsiung 80424, Taiwan.
E-mail: d994010003@student.nsysu.edu.tw
50
THE EFFECTS OF INQUIRY-
BASED LEARNING AND
LEARNING STYLES ON ISSN 1648-3898 /Print/
PRIMARY SCHOOL STUDENTS’ ISSN 2538-7138 /Online/
CONCEPTUAL UNDERSTANDING https://doi.org/10.33225/jbse/19.18.51
IN MULTIMEDIA LEARNING
ENVIRONMENT
Abstract. This research aimed to identify
the effects of inquiry-based learning on the
conceptual understanding of students with
various learning styles in the multimedia
Dek Ngurah Laba Laksana, learning environment. This research was
I Wayan Dasna, factorial experiment with the quasi-ex-
I Nyoman Sudana Degeng perimental design. There were total of 157
primary school students in Bali, Indonesia
being involved as the subjects of the study.
There were two instruments used in this re-
search. The first instrument was a question-
naire adapted from Index of Learning Style
Introduction (ILS) developed by Felder and Soloman for
measuring the verbal-visual learning styles.
Learning is a complex process which needs to be proven empirically. It The second instrument was the conceptual
is not only seen from the teachers and students’ perspective, but also how understanding test for measuring students’
technology works in instructional process. Educational technology and conceptual understanding in the form of
instructional strategy are correlated with the philosophy of science in the multiple-choice test. The research discov-
effort of providing solutions for learning issues to improve scientific inquiry ered three findings as follows: (1) there is a
(Cilesiz & Spector, 2014). One of the concepts of philosophical-based learn- significant difference in terms of conceptual
ing is meaningful learning (Arends, 2013). According to Jansen and Merwe understanding between students who are
(2015), a meaningful learning is a model in a 21st century learning context. taught by inquiry-based learning and direct
A meaningful learning requires the learning outcome that also produces instruction strategy, (2) there is a significant
graduates who have the critical thinking ability, problem-solving skill, and difference in terms of students conceptual
who will be of service in the future. Further, in the context of meaningful understanding between visual and verbal
learning, a learning process that encourages conceptual understanding is students, (3) there is a significant interac-
a strong foundation that can facilitate students to have a higher level of tion between different learning strategies
thinking (Degeng, 1988; Llewellyn, 2013; Reigeluth, 1999; Reigeluth, Beatty, (inquiry based learning and direct instruc-
& Myers, 2017). tion strategy) and students’ learning styles
Another problem that commonly arises in meaningful learning is the (visual verbal) toward students conceptual
students’ low achievement (Anderson & Krathwohl, 2001). Natural science understanding.
as one of the subjects taught in elementary schools is considered to be Keywords: conceptual understanding,
difficult for students (Jonstone, 1991; Llewellyn, 2013). This perception was inquiry-based learning, learning style,
supported by research findings which stated that a number of misconcep- multimedia learning.
tion occurred in natural science subject (Akbas & Gencturk, 2011; Thomp-
son & Logue, 2006). One of the misconceptions took place in the topics of
photosynthetic and organ systems of the human body (Laksana, Degeng, Dek Ngurah Laba Laksana
& Dasna, 2017; Svandova, 2014). Citra Bakti College of Education, Indonesia
The importance of the conceptual understanding in learning has I Wayan Dasna,
I Nyoman Sudana Degeng
reinforced the use of inquiry-based learning (Choi, Lee, & Jung, 2008; NRC, State University of Malang, Indonesia
2000). Inquiry-based learning has regarded as the foundation of an inductive
51
THE EFFECTS OF INQUIRY-BASED LEARNING AND LEARNING STYLES ON PRIMARY SCHOOL
STUDENTS’ CONCEPTUAL UNDERSTANDING IN MULTIMEDIA LEARNING ENVIRONMENT
(P. 51-62) ISSN 2538–7138 /Online/
approach that is highly suitable to be applied for developing students’ conceptual understanding (Prince & Felder,
2007). Therefore, as cited in NRC (2000), inquiry-based learning is superior in changing one’s conception according
to the constructivist view. The inquiry-based learning has commonly been used during learning since it has many
advantages to the development of students’ conceptual understanding and reasoning skill in its problem-solving
function (Kischner, Sweller, & Clark, 2006).
There are various kinds of inquiry-based learning strategies. One of them that is applicable for beginners is
5E inquiry model (Acish, Yalcon, & Turgut, 2011; Laksana, 2017). Such model includes the engage, explore, explain,
elaborate, and evaluate (Bybee, 2006). A concept attainment learning model is appropriate for a learning concept
with a clear set of attributes. This strategy has been proven to allow students to develop their definitions and
understandings based on the approach used (Magee & Ryan, 2012).
There are three types of inquiry learning strategy, namely guided inquiry, free inquiry, and modified inquiry
(Spencer & Tracy, 2012). Among those three, guided inquiry has been highly recommended since it is more superior
to the other types of inquiry strategies (Bunterma et al., 2014). Kirschner, Sweller, and Clark (2006) have pointed out
another reason for the importance of guidance stating that students learn a little amount through a constructiv-
ist approach. Most teachers try to apply the constructivist approach by providing appropriate guidance. Another
study has also suggested that guided inquiry is highly recommended to be implemented in learning, especially
for young learners in primary school level (Bunterma, 2014; Koksal & Berberoglu, 2014; Laksana, 2017).
Ideally, the application of inquiry strategy requires hands-on activities, in which the students actively investigate
real phenomena (Levitt, 2002). Hands-on activities in scientific learning cannot be separated from the availabilities
of learning instruments and materials, as well as the consideration of time that includes the preparation stage.
The hands-on activities for beginners could also cause harm such as accident so that there is a limited option for
hands-on activities. Hence, a deeper investigation in learning management alternative, namely an integration of
media technology or multimedia in the implementation of learning is necessary to be done (Spencer & Tracy, 2012).
The advancement of computer-based multimedia technology has a potential to create learning materials
that could support the learning process (Mayer, 2007). Such advancement allows the learning media in an inquiry
class to be designed based on real phenomena either in the form of laboratory fact video or daily life video (Bass,
Contant, & Carin 2009). Some inquiry processes, such as identification of the problem, hypothesis, experiment,
observation and evaluation, classification, explanation, and drawing conclusion has been proven that it could be
well facilitated by using the aid of a computer (Bransford, Brown, & Cocking, 2000; Chinn & Silver, 2002).
The advantages of using computer-based media in learning via inquiry strategy are a shorter period of the
experiment, the more complex design of the experiment, and more focus on theoretical aspect (Chinn & Silver,
2002). Further, Spencer and Tracy (2012) have stated that, by using a computer, the accuracy of the experimental
activities and results (either a real or simulation phenomenon) could be controlled by the learning designer so
that it could be adjusted with the theory.
Besides, the efficiency in designing the conceptual visualization, either static or dynamic, is one of the supe-
riorities of computer-based media in supporting the theoretical aspect. It is supported by Bass et al. (2009) who
has stated that the best way for elementary or secondary school students to be able to learn science is through
experiences. However, it is not practical, economical, or safe. Simulation of experiences using a computer could
be an effective alternative. Other findings also has pointed out the importance of multimedia learning integra-
tion into inquiry strategies (Bruckermann, Aschermann, Bresges, & Schlüter, 2017). The integration of multimedia
technology with inquiry-based learning has a potential to improve students’ conceptual understanding (So &
Kong, 2008; Turkmen, 2006). Hence, it is essential to perform a study about the development of a learning design
that integrates multimedia into inquiry-based learning to enhance students’ conceptual understanding (Hong,
Hwang, Tai, & Tsai, 2017).
Learning and teaching process does not only relate to the strategy of information delivery that is limited to
an integration of multimedia, more importantly, students’ characteristics also need to be a concern (Reigelut et
al., 2017). The characteristics of young learners’ learning style are different from adults’ (Bransford et al., 2000). In
accordance with Piaget’s theory of cognitive development (Piaget, 2000) stating that young learners (aged 7-11
years) have had the ability of logical thinking, but only about concrete objects. They still have difficulties to think
abstractly (Piaget, 2000). In line with such problem, a certain strategy is necessary for a certain learning condi-
tion, which leads to effective, efficient, and interesting results of learning (Degeng, 2013; Reigeluth et al., 2017).
A learning condition includes the characteristics of the field of study and the students’ characters. One important
character to be studied is students’ learning style. In relation to multimedia learning, verbal-visual learning style
52
ISSN 1648–3898 /Print/ THE EFFECTS OF INQUIRY-BASED LEARNING AND LEARNING STYLES ON PRIMARY SCHOOL
ISSN 2538–7138 /Online/ (P. 51-62)
has a strong disposition to be explored. Such kind of learning style relates to the process of receiving information
into students’ cognitive structures (Mayer, 2007; Mariano, 2014).
Based on the aforementioned explanation, inquiry learning strategy is potential to be a significant strategy
for improving elementary school students’ conceptual understanding. Students with different learning styles
would obtain different learning benefits so that it is necessary to perform a research about inquiry strategy. The
influence of verbal-visual learning style is highly relevant to be explored since it has a strong correlation with the
designing of teaching materials in the multimedia learning environment. Thus, it is important to investigate the
effects of inquiry based learning and learning styles in multimedia learning environment on the improvement of
students’ conceptual understanding.
Problem of Research
This present research aimed at examining learning styles as one of the factors that affects students’ conceptual
understanding in multimedia learning environment. Verbal and visual styles play a key role to the inquiry based
learning and direct instruction. Specifically, the research problems were formulated as follows: (1) is there any
significant difference on the students’ conceptual understanding between those who are treated by using inquiry
based strategy and direct instruction in multimedia assisted instruction? (2) is there any significant difference
on students’ conceptual understanding between those who have verbal and visual dimension, (3) is there any
significant effect of the interaction between the learning strategy (inquiry-based learning and direct instruction)
and learning styles (verbal and visual) toward students’ conceptual understanding?
Research Focus
This research focuses on (1) finding out the effects of the learning strategy (inquiry based learning and direct
instruction) toward students’ conceptual understanding, (2) finding the effect of the learning styles upon the
students’ conceptual understanding.
General Background of Research
This present research utilized quasi-experimental design. It is based on the consideration that the inability of
the researcher to conduct individual randomization. This design was also chosen because the participants were
taken from four intact classes. In this respect, the randomization was done through cluster sampling
Therefore, this research divided students into two groups namely experimental and control group. This
research was a quasi experimental design. In addition, this research utilised the two-factor analysis experiment.
With that rationale, intact groups were used. Both the experimental and control group consisted of two classes.
This research was conducted from January to July 2017.
Table 1. Scheme of factorial experiment 2x2.
Learning Strategy
Inquiry based learning Direct instruction
Verbal Group 1 Group 2

Learning Style
Visual Group 3 Group 4
Table 1 provides a clearer picture of the effect of the variables used in this research. By utilising this design,
the main and interactional effect from each variable could be shown clearly in accordance to what is stated in the
research hypothesis. The main effect of the variables were divided into two types, namely (1) the main effect of
53
(P. 51-62) ISSN 2538–7138 /Online/
the learning strategy without considering the learning styles, and (2) the main effect of the learning styles without
considering the learning strategy. The interactional effect provides information in regards to the interaction between
the learning strategy and the learning styles to the dependent variable that becomes the focus of the research.
Sample of Research
The participants were selected by considering the population of the research, they are fifth graders from
all elementary schools in South Kuta, Bali, Indonesia. The schools involved covered those that were categorized
as favourite and non-favourite schools. From all participants, the researcher decided the sample through cluster
sampling. There were total 157 primary school students involved in this research coming from 4 classes in two
different schools. The distribution of the subjects can be seen in Table 2.
Table 2. Distribution of the participants.
Number of Students
School Name Class Treatment Involved Following full
M F n M F n
VA IBL 18 24 42 17 23 40
SDN 4 Jimbaran
VB DI 27 15 42 26 13 39
VA IBL 25 19 44 25 15 40
SDN 6 Jimbaran
VB DI 19 23 42 17 21 38
N 89 81 170 85 72 157
Note: M (Male), F (Female); IBL (Inquiry Based Learning); DI (Direct Instruction)
This research was conducted through several stages, they cover : (1) developing and trying out the instru-
ments, (2) administering the pre-test of conceptual understanding, (3) administering the questionnaire about
verbal-visual learning style, (4) conducting two different instructional strategy, (5) conducting the posttest, and
(6) analysing the data.
There were two instruments used to measure the research variable that became the primary unit analysis,
they are: (1) questionnaire to receive students’ learning style, (2) achievement test to measure students’ conceptual
understanding.
The questionnaire used in this research was a result of adaptation from Index of Learning Styles (ILS) developed
by Felder and Solomon (Felder & Silverman, 1988; Felder & Spurlin, 2005). The questionnaire consisted of 44 ques-
tions that measure 4 learning style types: sensing-intuitive, verbal visual, active-reflective, and sequential-global.
Each learning style is measured from 11 items in the ILS questionnaire. Specifically, the researcher adapted the 11
items that measure the students verbal-visual dimension. An adaptation was conducted due to this dimension
was originally developed and tried out to secondary schools and universities (Felder & Silverman, 1988; Felder &
Spurlin, 2005) with different context from Indonesia. The reliability test showed the score was 0.832. This reflected
that the instrument was reliable to measure the learning style. In addition, contructed validity was tested by using
factor analysis. This showed that the factors were valid. This test was conducted by involving 346 primary school
students as participants.
On the other hand, an achievement test was conducted to measure students’ conceptual understanding. The
form of the test was a multiple choice test. This test was developed by the researcher, thus C2 operational words
from Bloom’s taxonomy were used (Anderson dan Krathwolh, 2001). In addition, there were 20 item in total for
the test. For this test, there were 199 participants involved. The reliability from alpha Cronbach test showed that
the conceptual understanding was 0.977 with 20 questions were valid where the range was from 0.216 -0.425.
This research examined the effect of inquiry based learning by using 5E model and direct instruction in
54
ISSN 2538–7138 /Online/ (P. 51-62)
multimedia learning environment. It focused on natural science subject in primary school with specific course on
“human organ system, photosynthesis, and objects characteristics”. Moreover, this was conducted in 4 weeks that
cover 8 meetings in total.
Table 3-a. Activities based on 5E model in multimedia integrated inquiry strategy.
Phase Teacher Activity in 5E Model Inquiry with Multimedia
Engage 1. Raising students curiosity and attract their interests by displaying an infographic model of a topic on multimedia
2. Determining students’ prior knowledge of a certain concept
3. Inviting learners to express their thought
4. Inviting learners to question themselves
Explore 1. Encouraging interaction among learners by grouping them and ask them to explore a topic in multimedia displays (using
tablet, laptop, smartphone)
2. Questioning learners to lead them to make arguments
3. Giving students time to have a cognitive conflict
Explain 1. Encouraging learners to use their daily experience and those that they got during the engage and explore phases to
construct an explanation.
2. Delivering questions that help learners to give explanation
3. Asking for supporting evidence regarding their explanation
4. Giving learners time to compare their ideas with other students and revising their work
5. Introducing some terminologies and giving alternative explanation after the learners express their ideas through multi-
media displays
Elaborate 1. Focusing learners attention to the relation between the new concept and their experience
2. Encouraging learners to use what they have learned during the activity
3. Giving reinforcement to the learners with the use of scientific terms and description which have been introduced previ-
ously (displayed in infographic mode)
4. Asking questions to help learners conclude their ideas based on evidences and data gathered
Evaluate 1. Observing and assessing learners understanding, performance, and skills

2. Interviewing learners regarding the assessment and improvement
3. Encourage learners to assess themselves
Inquiry strategy in multimedia learning environment had a different phase or steps from direct instruction.
These two strategies are different in terms of its syntax. Table 3-a and 3-b present the syntax of the two strategies
in multimedia learning environment.
Table 3-b. Activities based on direct instruction in multimedia integrated inquiry strategy.
Phase Teacher Activity in direct Instruction with Multimedia
Presentation Teacher delivers the objective of the study and motivates students to learn
Teacher confirms students initial knowledge and concept needed for the learning
Teacher presents the core information, theme, and demonstrate how to use multimedia
Teacher confirms students’ knowledge and clarifies the wrong concept
Practice Teacher delivers assisted exercise to students with the utilization of multimedia
Teacher assigns an independent exercise
Evaluation Teacher checks students’ independent exercise
Monitoring and feedback Teacher gives feedback based on the evaluation and repetition as necessary
Teacher assigns a homework for students as an apperception of the following meeting
55
(P. 51-62) ISSN 2538–7138 /Online/
Data Analysis

Two analyses were done in this research, i.e. 2x2 factorial analysis of covariance (ANCOVA) and descriptive
analysis. The factorial ANCOVA analysis aimed to test the research hypothesis where, (1) there are significant effects
of the educational strategy (inquiry based learning and direct instruction) toward students’ conceptual understand-
ing, (2) there are significant effects of the learning styles upon the students’ conceptual understanding, and (3)
there is an interactional effect between the educational strategy and learning styles toward students’ conceptual
understanding. Descriptive analysis was performed to describe the students’ conceptual understanding and the
standard deviation.
Results of Research
A hypothesis test was done via a 2x2 factorial Analysis of Covariance (ANCOVA), in which the scores of con-
ceptual understanding pre-test were utilized as a covariate. Three hypotheses were tested in this research. The
summary of descriptive statistics and ANCOVA test results can be seen in Table 4 and Table 5.
Table 4 shows that the average score of conceptual understanding of the student group that has visual learn-
ing style and learned by using inquiry learning strategy after being controlled by the covariate variable was 78.77.
The average score of conceptual understanding of students who have visual learning style and learning via direct
instruction after being controlled by the covariate variable was 50.77. Meanwhile, the average score of conceptual
understanding of the student group that has verbal learning style and learned by using inquiry learning strategy
after being controlled by the covariate variable was 57.22. The average score of conceptual understanding of
students who have verbal learning style and learning via direct instruction after being controlled by the covariate
variable was 70.60.
Table 4. Descriptive statistics of the research findings.
Learning Strategy M SD n
Verbal Learning Style 70.60 10.14 25

Direct instruction group Visual Learning Style 50.77 9.57 52
Total 57.21 13.46 77
Inquiry group Visual Learning Style 78.77 10.51 53
Total 71.50 14.20 80
Total Visual Learning Style 64.90 17.26 105
N 64.49 15.55 157
The first hypothesis testing showed the results of 2x2 factorial ANCOVA test as shown in Table 5 reveals that
there was a significant different between students who learned by using inquiry strategy and those who learned
using direct instruction strategy (F value = 24.976, p = .0001).
Table 5. Calculation on 2x2 factorial ANCOVA test results.
Source Type III SS df MS F p
Corrected Model 28579.594a 4 7144.899 118.956 .0001
Intercept 49373.474 1 49373.474 822.022 .0001
Conceptual understanding
5617.539 1 5617.539 93.527 .0001
Pre-test
56
ISSN 2538–7138 /Online/ (P. 51-62)
Source Type III SS df MS F p
Learning strategy 1499.593 1 1499.593 24.967 .0001
Learning style 763.013 1 763.013 12.703 .0001
Learning strategy *
16290.644 1 16290.644 271.224 .0001
Learning Style
Error 9129.642 152 60.063
Total 690675.000 157
Corrected Total 37709.236 156
a. R Squared = .758 (Adjusted R Squared = .752)

The results of a further study of LSD are presented in Table 6. The results of the LSD further study show a sig-
nificance value less than 0.05 and the posttest average value of inquiry group was better than the direct instruction
group (71.50 > 57.22; as shown in Table 4).
Table 6. LSD test on learning strategy.
Learning strategy Mean Difference (I-J) Std. Error p
Direct instruction Inquiry -6.581* 1.317 .0001
Inquiry Direct instruction 6.581* 1.317 .0001
*. The mean difference is significant at the .05 level.

b. Adjustment for multiple comparisons: Least Significant Difference (equivalent to no adjustments)
The results of analysis on the second hypothesis can be identified from the results of 2x2 factorial ANCOVA
test as presented in Table 5. The analysis results pointed out that a significant difference was found between the
conceptual understanding of students with visual learning style and the students with verbal learning style (F value
= 12.073). The results of the LSD advanced study are presented in Table 7. Such results showed a significance value
less than 0.05. The students who have visual learning style have better average posttest value than the students
who have verbal learning style (64.90 > 63.65; as shown in Table 4).
Table 7. LSD test on learning style.
Learning Style Mean Difference (I-J) Std. Error p
Verbal Learning Style Visual Learning Style 5.190* 1.456 .0001
Visual Learning Style Verbal Learning Style -5.190* 1.456 .0001
*. The mean difference is significant at the .05 level.

b. Adjustment for multiple comparisons: Least Significant Difference (equivalent to no adjustments)
The third hypothetical test results revealed that the interaction between inquiry strategy and learning style
has an influence on students’ conceptual understanding. The results of ANCOVA test presented in Table 4 showed
such influence of interaction (F = 271.224).
Discussion
The discussion covered three main things. The first discussion was about the influence of the learning strategies
(inquiry strategy and direct instruction strategy) on students’ conceptual understanding. Second, the discussion is
57
(P. 51-62) ISSN 2538–7138 /Online/
about the effect of learning styles (verbal and visual dimensions) on students’ conceptual understanding. Lastly, the
discussion is about the effect of the interaction between learning strategy and learning style on students’ conceptual
understanding.
The first research finding showed that students with inquiry strategy and students with direct instruction strategy
have a significantly different conceptual understanding. The results revealed that inquiry strategy is better than direct
instruction strategy. Inquiry strategy is effective in strengthening students’ conceptual understanding since, in applying
the strategy, the teacher gave questions that range from the easiest to the most complex. Therefore, students could
build the construction of stronger understanding. Besides, the inquiry strategy allows students to find answers and
build a deeper understanding independently and to share conceptions in groups of scientific discussion. Also, this
strategy requires a teacher to explain the concept based on the materials sourced from textbooks, scientific journals,
or the internet in order to construct more in-detail students’ conceptual understanding. Additionally, the elaboration
stage of this strategy allows a teacher to make a new conceptual connection from experience, find similarities and
differences, and create a simple flowchart showing the process of a topic in order to make it easier to be understood
and memorized by the students. Furthermore, the multimedia exploration allows students to access their understand-
ing in anytime so that they could present the concept in their words.
The research findings were in line with the results of the research about the superiorities of inquiry strategy in
improving students’ conceptual understanding, which was conducted previously (Bruckermann et al., 2017; Kirna,
2010; Pandey et al., 2011; Sever & Guven, 2015). Inquiry strategy encourages students to have more group activities.
Inquiry strategy includes several learning activities, namely elicitation of students’ initial ideas (pre-inquiry), review
of students’ initial ideas (while inquiry), negotiation of definition (post-enquiry), a conceptual application in a new
situation, and drawing conclusion and reflection. The component of inquiry is group activities, such as exploring,
hypothesis testing, data collection, and providing an alternative explanation based on empirical evidence found by
the students. The activities performed by the students assisted them to give more meaning to the learning process
in their cognitive structures (Bass et al., 2009; Bransford et al., 2000).
Additionally, the teacher’s role in inquiry strategy is as a facilitator, and the teacher develops a structure on cog-
nitive knowledge based on the ideas of the students (Pandey et al., 2011). Hence, inquiry strategy provides a larger
portion of students’ independent learning activities. It is supported by Bransford et al. (2000) who has emphasized that
students’ initial experience and hypothesis hold an important role in learning, while the cognitive research showed
that students’ previous knowledge affects all aspects of their information processing.
In this research, the inquiry learning was accompanied by teacher guidance. The teacher guided the students
by directing the discussion by providing investigation questions. Additionally, the teacher also assists the students
by giving real cases in order to help students in exploring their fundamental knowledge. The results of this study
are in agreement with the inquiry learning concept, i.e. learning with an inductive approach (Felder & Prince, 2007).
Inquiry learning as an inductive approach is applied to develop students’ conceptual understanding. Further, the
inquiry strategy cannot be given independently as it challenges the students to connect their world with the topic
in the curriculum. During its implementation, the inquiry does not only aim for personal competency, but also for
improving students’ involvement in their communities and creating social interactions (Bruckermann et al., 2017).
Thus, the learning strategy was implemented with supervision since the students were still in elementary school
level (Magee & Ryan, 2012). Another finding also revealed that guided inquiry provides more advantages than other
inquiries or strategies that do not provide guidance and scaffolding (Arnold et al., 2014; Bunterma et al., 2014; Kawalkar
& Vijapurkar, 2013; Laksana, 2017; Lazonder & Harmsen, 2016).
The research findings are also in line with the results found by NRC (2000) stating that inquiry strategy could
develop the necessary conceptual understanding and thinking ability in solving problems. Besides, Joyce et al. (2009)
showed the superiorities of inquiry strategy, namely enhancing scientific skill, creative thinking ability, independent
learning, appreciating multi-definition, and developing awareness of tentative knowledge. Inquiry learning strategy
is highly suitable for changing students’ conceptions in accordance with the constructivist view.
This research also revealed that an integration of multimedia with learning encourages the effectiveness of learn-
ing. The multimedia is designed to be used by the students in the inquiry group. Each stage of inquiry is integrated
with the multimedia, which includes a video about a concept such as the human blood circulation process. Such result
is in agreement with the previous finding stating that multimedia technology makes learning more efficient (Heinich,
2002; Spector, 2009; and Wang, 2008). Also, Spector (2012) has performed a study presenting that an integration of
multimedia technology in learning is potential to improve learning efficiency and outcome. Such potential could be
seen on augmented reality-based learning and game-based learning, as well as multimedia-based learning.
58
ISSN 2538–7138 /Online/ (P. 51-62)
Learning multimedia are components that can be used to support the learning process. It is based on a percep-
tion that learning can be better, effective, and pleasant if it is backed up by learning media that could attract students’
interest and attention (Clark & Mayer, 2003; Degeng, 2013). Multimedia are highly prospective and very suitable to
be used during learning (Passerini, 2007, Reigeluth, 1999; and Reigeluth & Carr-Cheliman, 2009). The utilization of
multimedia cannot be separated with various studies that integrated technology with multimedia in a particular
learning strategy. Many researchers have highlighted the importance of an integration of technology with multime-
dia and recommended to use multimedia which is supported by an inquiry strategy that is based on the learning
environment (Kim et al., 2007).
The second finding of this research was in relation to the learning style variable and its connection with students’
conceptual understanding. Such finding presented a significant difference between the conceptual understandings
of students with visual learning style and those with verbal learning style. It was found from the inquiry group that
the students with visual learning style have better conceptual understanding than the students with verbal learning
style. However, such fact was not found in the direct instruction group. On the contrary, in direct instruction group, the
students with verbal learning style have better conceptual understanding than the students with visual learning style.
Such finding is in good agreement with Felder & Solomon (2007; Mayer, 2007) who explained that visual learners
have the ability to strongly remember by looking at pictures, diagram, a flow diagram, timeline, film, and demonstra-
tion. Students with visual learning style who learned using either inquiry strategy or direct instruction strategy could
make a conceptual map, a flowchart of the process, find keywords, and visualize their understanding so that they
have more holistic understanding compared with those with verbal learning style.
Nevertheless, when the teacher role is dominant, especially in direct instruction group, the students with ver-
bal learning style could understand the words, write, and understand the teacher explanation more easily than the
students with visual learning style (Huit, Monneti, & Hummel, 2009; Mayer, 2007). It is shown by the research finding
that even though the multimedia given to inquiry and direct instruction groups is similar, the teacher has a more
dominant role in presenting information when it comes to direct instruction strategy.
Learning by using inquiry strategy and multimedia is beneficial for students with visual learning style since they
could connect one concept with another and try to find their similarities and differences during the exploration and
elaboration stages. Multimedia, especially in the form of pictures, provides an opportunity for students with visual
learning style to easily understand the topic they learn (Clark & Mayer, 2003).
The research findings are in line with other results which state that the academic achievement level of students
with visual learning style is higher than that of students with verbal learning style (Choi et al., 2008; Felder & Brent,
2005; Litzinger, 2007). Conforming to such matter, Clark & Mayer (2003) reported that the provision of multimedia
technology containing visual stimulus results in better understanding of students with visual learning style than
those with verbal learning style. Hence, students with visual learning style have a better conceptual understanding
when studying in a multimedia learning environment, yet the students with verbal learning style also could follow
the lesson without difficulties (Pallapu, 2007).
Another finding in this research also revealed that a significant effect of the interaction between learning strategy
and learning style was found on students’ conceptual understanding level. Such interaction concluded that students
with verbal learning style had a better conceptual understanding when learned by using a direct instruction than
the inquiry strategy. Meanwhile, students with visual learning style had a better understanding when they learned
by using inquiry strategy than direct instruction.
The direct instruction is beneficial for students with verbal learning style since they have ta better ability to un-
derstand sentences and words, either spoken or written, rather than students with visual learning style (Becker, 1998;
Huit, 2009). In conformity to such findings, Parkinson & Redmond (2002) found that students with verbal learning
style could acoustically understand a complex text, show a higher reading ability, and better language ability than the
students with visual learning style. On the other hand, a direct instruction method is not beneficial for students with
visual learning style as their needs to understand a topic through imagination, picture, graphic, and the conceptual
map is not fulfilled since most of the materials are delivered in speaking although multimedia use accompanies it.
Students with visual learning style benefit greatly from inquiry learning strategy because they could connect
one concept with another through a diagram, a flow of process, and a conceptual map during the exploration and
elaboration processes. Hence, they could obtain a holistic understanding, even more, when accompanied by multi-
media (Felder & Solomon, 2007). This is also supported by Hong et al. (2017) who found that there was a significant
improvement of students who learn through inquiry based learning and technology assistance like iPad. Riding &
Douglas (1993; Wang, 2008) found that students with visual learning style are more superior compared with the stu-
59
(P. 51-62) ISSN 2538–7138 /Online/
dents with verbal learning style in learning that involves diagram, and motion and pictures, while the students with
verbal learning style are better than those with visual learning style in learning with text. Visual learners use more
diagrams to illustrate their understanding than the verbal learners. Meanwhile, inquiry learning strategy is not benefi-
cial for students with verbal learning style. There was a low amount of spoken explanation given in this research. The
students learned independently and in small groups most of the time. Mayer (2007; Pallapu, 2007) pointed out that
the use of multimedia in learning is advantageous for students who have visual learning style since it could support
the construction of their knowledge in understanding the concept.
Conclusions
The research revealed different conceptual understandings were found between students who learned using
inquiry strategy and those who learned using direct strategy in multimedia learning. The conceptual understanding
of students that learned using inquiry strategy is higher than those who obtained direct instruction strategy. This
is because : (1) students started to construct a firmer concept when teacher delivered series of questions ; (2) the
elaboration process comprised the relation between the new concept and students’ experience ; (3) students’ expe-
rienced was re-accessed through multimedia exploration so that they expressed the concept on their own language
Second, there are different conceptual understandings between verbal learners and visual learners. The concep-
tual understanding of visual learners is better than that of verbal learners. It was because the visual learners have
longer retention from looking at pictures, diagram, flow, movies, or demo.
Third, different learning strategies (inquiry and direct strategies) and learning styles (verbal and visual) show
an influence of interaction on students’ conceptual understanding. The interaction pattern indicates that learning
strategy and style strengthen the conceptual understanding of visual learners, yet they slow down the conceptual
understanding of verbal learners. The information on the developed multimedia should be used for students who
have verbal learning style. Based on the findings on this research, multimedia should be designed not inhibit the
verbal style. It could be adapted since the visual learners were not inhibited from the multimedia.
As for future researchers, it is important to (1) conduct deeper investigation on other aspects of learning achieve-
ment such as the retention to conceptualize learning, application of the concept, analysis, synthesis, and creation, (2)
the developed multimedia in this research functions very well for fifth grader students. However, the effectiveness of
its usage should be seen more deeply specifically for students in grade I,II, III. Besides, an empirical study on different
course subject like social science is necessarily to be done
Acknowledgements
The highest gratitude I address to the chairman of Citra Masyarakat Mandiri Foundation as the source of funds for
this study. My highest appreciation also goes to the head of SDN 4 Jimbaran and SDN 6 Jimbaran who have warmly
permitted the researcher to conduct this study in these schools
References
Abdullah, S., & Abbas, M. (2006). The effect of inquiry-based computer simulation with cooperative learning on scientific thinking
and conceptual understanding. Malaysia Online Journal of Instructional Technology, 3(2), 1-16.
Acish, S., Yalcon, S. E., & Turgut, U. (2011). Effects of the 5E learning model on students’ academic achievements in movement and
force issues. Procedia Social and Behavioral Sciences, 15, 2459–2462.
Akbas, Y., & Gencturk, E. (2011). The effect of conceptual change approach to eliminate 9th grade high school students’ misconcep-
tions about air pressure. Educationals Sciences: Theory & Practice, 11(4), 2217-2222.
Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching, and assessing. New York: Addison Wesley Longman.
Arends, R. I. (2013). Learning how to teach (9th Ed). Boston: McGraw Hill.
Arnold, J. C., Kremer, K., & Mayer, J. (2014). Understanding students’ experiments - What kind of support do they need in inquiry
tasks? International Journal of Science Education, 36(16), 2719-2749
Bass, J. E., Contant, T. L., & Carin, A. A. (2009). Teaching science as inquiry 7th. Boston: Pearson Education.
Becker, D. A. (1998). The impact of student verbal/visual learning style preference on implementing groupware in the classroom.
JALN, 2(2), 61-69.
Bransford, J. D., Brown, A. L., & Cocking, R. R. (2000). How people learn: Brain, mind, experience, and school. Washington: National
Academy Press.
Bruckermann, T., Aschermann, E., Bresges, A., & Schlüter, K. (2017). Metacognitive and multimedia support of experiments in inquiry
learning for science teacher preparation. International Journal of Science Education, 39(6), 701-722.
60
ISSN 2538–7138 /Online/ (P. 51-62)
Bunterma, T., Leeb, K., Ng Lan Kong, J., Srikoon, S., Vangpoomyaic, P., Rattanavongsa, J., & Rachahoon, G. (2014). Do different levels
of inquiry lead to different learning outcomes? a comparison between guided and structured inquiry. International Journal
of Science Education, 36(12), 1937-1959.
Bybee, R. W. (2006). Scientific inquiry and science teaching. In L. B. Flick & N. G. Lederman. (Eds.), Scientific inquiry and nature of sci-
ence: implications for teaching, learning, and teacher education (pp. 1-14). Dordrecht, Netherlands: Springer.
Chang, H., Quintana, C., & Krajcik, J. S. (2010). The impact of designing and evaluating molecular animations on how well middle
school students understand the particulate nature of matter. Science Education, 94(1), 73–94.
Chapman, N., & Chapman, J. (2004). Digital multimedia (2nd Ed). London: John Wiley & Sons.
Chinn, C. A., & Silver, C. E. (2002). Authentic inquiry: Introduction to the special section. Science Education, 86(2), 175-218.
Choi, I., Lee, S. J., & Jung, J. W. (2008) Designing multimedia case-based instruction accomodating student’s diverse learning style.
Journal of Educational Multimedia and Hypermedia, 17(1), 5-25.
Cilesiz, S., & Spector, J.M. (2014). The philosophy of science and educational technology research. In J.M. Spector, M.D. Merrill, J.
Elen, & M.J. Bishop (Eds). Handbook of research on educational communications and technology 4th edition (pp. 875-884). New
York: Springer Science.
Clark, R. C., & Mayer, R. E. (2003). E-learning and the science of instruction. San Francisco: Jossey-Bass.
Degeng, I N. S. (2013). Ilmu pembelajaran: Klasifikasi variabel untuk pengembangan teori dan penelitian [Instructional Science: The
variable classification for research and theory development]. Bandung, Indonesia: Aras Media.
Felder, R. M., & Brent, R. (2005). Understanding student different. Journal of Engineering Education, 94(1), 57-72.
Felder, R. M., & Silverman. (1988). Learning and teaching styles in engineering education. Engineering Education, 78(7), 674-681.
Felder, R. M., & Solomon, B. A. (2007). Learning styles and strategies. Retrieved from http://www.ncsu.edu/felder-public/ILSdir/
styles.htm.
Felder, R. M., & Spurlin, J. (2005). Applications, reliability and validity of the index of learning styles. International Journal Engineering
Education, 1, 103-112.
Fraser-Abder, P. (2011). Teaching budding scientists-fostering scientific inquiry with diverse learner in grades 3-5. United State of America:
Pearson Education.
Gillies, R. M., Nichols, K., Burgh, G., & Haynes, M. (2014). Primary students’ scientific reasoning and discourse during cooperative
inquiry-based science activities. International Journal of Educational Research, 63, 127–140.
Heinich, R., Molenda, M., Russel, J. D., & Smaldino, S. E. (2002). Instructional media and technology for learning (7th ed.). New Jersey.
Prentice Hall.
Hong, J. C., Hwang, M. Y.,Tai, K. H., & Tsai, C. R. (2017). An exploration of students’ science learning interest related to their cognitive
anxiety, cognitive load, self confidence and learning progress using inquiry based learning with an iPad. Science Education,
4(6), 1193-1212.
Huit, W. G., Monneti, D. M., & Hummel, J. H. (2009). Direct approach to instruction. In C. M. Reigeluth, & A. A. Carr-Cheliman (Eds.),
Instructional-design theories and models: Building a common knowledge base, (pp. 73-98). New York: Routledge.
Jansen, C., & Merwe, P. (2015). Teaching practice in the 21st century: Emerging trends, challenges and opportunities. Universal Journal
of Educational Research, 3(3), 190-199.
Jonstone, A. H. (1991). Why is science difficult to learn? Thinks are seldom what they seem. Journal of Computer Assisted Learning,
7, 75-83
Joyce, B., Weil, M., & Calhoun, E. (2009). Models of teaching (8th ed.). New Jersey: Pearson Education.
Kawalkar, A., & Vijapurkar, J. (2013). Scaffolding science talk: The role of teachers’ questions in the inquiry classroom. International
Journal of Science Education, 35(12), 2004-2027
Kim, M. C., Hannafin, M. J., & Bryan, L. A. (2007). Technology-enhanced inquiry tools in science education: An emerging pedagogical
framework for classroom practice. Science Education, 91(6), 1010–1030.
Kirna, I M. (2010). The effect of the use of hypermedia in instruction using learning cycle strategy on the understanding and application
of chemistry concepts of junior high school student with two different learning styles (Unpublished doctoral dissertation). State
University of Malang, Indonesia.
Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why minimal guidance during instruction does not work: an analysis of the failure
of constructivist, discovery, problem-based, experiential, and inquiry-based teaching. Educational Psychologist, 41(2), 75-86.
Koksal, E. A., & Berberoglu, G. (2014). The Effect of guided-inquiry instruction on 6th grade Turkish students’ achievement, science
process skills, and attitudes toward science. International Journal of Science Education, 36(1), 66-78.
Kuhltau, C. C., Maniotes, L. K., & Caspari, A. K. (2007). Guided inquiry: Learning in 21st century. London: Libraries Unlimited Publishing.
Laksana, D. N. L. (2017). The effectiveness of inquiry based learning for natural science learning in elementary school. Journal of
Education Technology, 1(1), 1-5.
Laksana, D. N. L., Degeng, I. N. S., & Dasna, I. W. (2017). Why teachers faces misconception: A study toward natural science teachers
in primary schools. European Journal of Education Studies, 3(7), 667-679.
Lazonder, A. W., & Harmsen, R. (2016). Meta-analysis of inquiry-based learning: Effects of guidance. Review of Educational Research,
86(3), 681-718.
Lee, W. W., & Owens, D. L. (2004). Multimedia-based instructional design: Computer-based training, web-based training, distance
broadcast training, performance based solution (2nd ed.). San Francisco: Pfeiffer A Wiley Inprint.
Levitt, L. E. (2002). An analysis of elementary teacher’s belief regarding the teaching and learning of science. Science Education,
86(1), 1-22.
Litzinger, T. A., Lee, S. A., Wise, J. C., & Felder, R. M. (2007). A psychometric study of the index of learning style. Journal of Engineering
Education, 94(4), 309-319.
61
(P. 51-62) ISSN 2538–7138 /Online/
Llewellyn, D. (2013). Teaching hingh school science through inquiry and argumentation (2nd ed.). California: Corwin a SAGE Company.
Magee, P. A., & Ryan, F. (2012). Collaborating to improve inquiry-based teaching in elementary science and mathematics method
courses. Science Educational International, 23(4), 353-365.
Mariano, G. (2014). Breaking it down: knowledge transfer in a multimedia learning environment. International Journal of Teaching
and Learning in Higher Education, 26(1), 1-11.
Mayer, R. E. (2007). Multimedia Learning (9th ed.). New York: Cambridge University Press.
Muller, D. A., Lee, K. J., & Sharma, S. D. (2008). Coherence or interest: Which is most important in online multimedia learning? Austral-
ian Journal of Educational Technology, 24(2), 211-221.
NRC - National Research Council. (2000). Inquiry and the national science education standards: A guide for teaching and learning. DC:
National Academies Press.
Pallapu, P. (2007). Effects of visual and verbal learning styles on learning. Institute for Learning Styles Journal, 1, 34-39.
Pandey, A., Nanda, G. K., & Ranjan, V. (2011). Effectiveness of inquiry training model over conventional teaching method on academic
achievement of science students in india. Journal of Innovative Research in Education, 1(1), 7-20.
Park, S., & Lim, J. (2007) Promoting positive emotion in multimedia and hypermedia learning using visual illustrations. Journal of
Educational and Hypermedia, 16(2), 141-162.
Parkinson, A., & Redmond, J.A. (2015). The impact of cognitive style and educational computer environment of learning performance.
Retrieved August 15, 2016, from https://www.researchgate.net/ publication/253079863.
Passerini, K. (2007). Performance and behavioral outcome in technology-supported learning: The role of interactive mutimedia.
Journal of Education Multimedia and Hypermedia, 16(2), 183-210.
Piaget, J. (2000). Commentary on Vygotsky. New Ideas in Psychology, 18, 241–259.
Prince, M., & Felder, R. (2007). The many faces of inductive teaching and learning. Journal of College Science Teaching, 36(5), 14-20.
Reigeluth, C. M. (1999). What is instructional design theory and how is it changing? In C.M. Reigeluth (Ed.). Instructional design theory
and model (pp. 5-29). London: Lawrence Erlbaum Associates.
Reigeluth, C. M., & Carr-Cheliman, A. A. (2009). Theories for different outcomes of instruction. In C. M. Reigeluth, & A. A. Carr-Cheliman
(Eds.), Instructional-design theories and models: Building a common knowledge base (pp. 195-197), New York: Routledge.
Reigeluth, C. M., Beatty, B. J., & Myers, R. D. (2017). Instructional-design theories and models: The learner-centered paradigm of educa-
tion. New York: Routledge.
Riding, R. J., & Douglas, G. (1993). The effect of cognitive style and mode of presentation on learning performance. The British Journal
of Educational Psychology, 63(2) 297-307.
Sever, D., & Guven, M. (2015). Effect of inquiry-based learning approach on student resistance in a science and technology course.
Educational Sciences: Theory & Practice, 14(4), 1601-1605.
So, W. M., & Kong, S. C. (2008). Approaches of inquiry learning with multimedia resources in primary classroom. Journal of Computer
in Mathematics and Science Teaching, 26(4), 329-354.
Spector, J. M. (2009). Adventures and advances in instructional design theory and practice. In L. Moller, J.B. Huett, D.M. Harvey (Eds).
Learning and instructional technologies for the 21st century (pp. 1-14). New York: Springer.
Spector, J. M. (2012). Foundations of educational technology: Integrative approaches and interdisciplinary perspectives. New York:
Routledge.
Spencer, T. L., & Tracy M. W. (2012). Creating a love for science for elementary students through inquiry-based learning. Journal of
Virginia Science Education, 4(2), 18-25.
Svandova, K. (2014). Secondary school students’ misconceptions about photosynthesis and plant respiration: Preliminary results.
Eurasia Journal of Mathematics, Science and Technology Education, 10(1), 59-67
Thompson, F. & Logue, S. (2006). An exploration of common student misconceptions in science. International Education Journal,
7(4), 553-559.
Turkmen, H. (2006). What technology plays supporting role in learning cycle approach for science education. The Turkish Online
Journal of Educational Technology, 5(2), 1303-6521.
Wang, L. (2008). Developing and evaluating an interactive multimedia instructional tool: Learning outcome and user experiences
of optometry students. Journal of Educational Multimedia and Hypermedia, 17(1), 43-57.
Received: October 06, 2018 Accepted: January 05, 2019
Dek Ngurah Laba Laksana PhD, Lecturer of Primary Teacher Education, Citra Bakti
College of Education, Indonesia.
E-mail: laba.laksana@gmail.com
I Wayan Dasna PhD, Lecturer of Instructional Technology, State University of
Malang, Indonesia.
I Nyoman Sudana Degeng PhD, Professor, Lecturer of Instructional Technology, State
University of Malang, Indonesia.
62
DEVELOPMENT AND
VALIDATION OF A SURVEY
INSTRUMENT (AKA) TOWARDS ISSN 1648-3898 /Print/
ATTITUDE, KNOWLEDGE AND ISSN 2538-7138 /Online/
APPLICATION OF STEM https://doi.org/10.33225/jbse/19.18.63
Bevo Wahono
Chun-Yen Chang
Abstract. The development and deploy-

Introduction
ment of STEM education in every continent
and country are different. An instrument
Innovation and development of education are needed to answer the
demands of rapid changes in the progress of era and technology. STEM to access the current situation of the ap-
education was present to respond to these needs. STEM is a unique con- plication, knowledge, as well as attitude
tent area that acknowledges the interdependence of science, technology, towards STEM education, is necessary. This
engineering and mathematics (English, 2016). Recently, in the field of edu- research aimed to design and develop a
cation research, STEM education has grown to be a trend. The first term of valid instrument that can be used to assure
STEM that emerged in America more than a decade ago (Honey, Pearson, the quantitative degree of attitude, knowl-
& Schweingruber, 2014), has now expanded throughout the Western as edge, and application of STEM (AKA) by sci-
well as into Asia. During this time, efforts have been made by researchers ence teachers. Research data were collected
and educational policymakers in each country to develop a reform and from 137 Indonesian secondary school
initiative to shape teaching and learning and try applying STEM on many
science teachers. In order to determine the
educational units in their respective regions (Siew, Amir, & Chong, 2015;
validity of the scale, reliability test, explora-
Wahono, Rosalina, Utomo, & Narulita, 2018). Moore et al. (2014) argued
tory factor analysis (EFA), as well as the
that the connections between the subjects and real-world problems are
a connecting way to incorporate some or all of the four disciplines of sci- content and face validity from experts were
ence, technology, engineering, and mathematics (STEM) into unit or les- used. Results showed that the designed and
son or even into one class to promote school curriculum integrated STEM developed AKA instrument was adequate
education. Honey et al. (2014) asserted that far from being a single, STEM to reliabilities and validities as well as can
education approach composes some level of connection that involves a be used to collect data. The development of
space of many experiences. The experiences could take place in a broad AKA instrument enables users worldwide to
range such as throughout a curriculum, entire school or a single course, in obtain information about the development
one or several class periods, or even be covered in any out of school teach- of STEM as well as the problems and chal-
ing and learning activities. However, every single type of integrated STEM lenges faced by science teachers in the field.
education encourages different things such as outcomes, resource needs,
Further work is also suggested.
design approaches, and application challenges.
Keywords: science teachers, STEM educa-
Therefore, teachers are a vital key as at the main guard of STEM imple-
tion, survey instrument.
mentation in most countries around the world (Aslam, Adefila, & Bagiya,
2018; Watermayer & Montgomery, 2018), especially teachers who teach
science subjects. Researches of Ruiz et al. (2014) and Kola (2013) showed Bevo Wahono
the importance of science teaching in education levels. They argued that National Taiwan Normal University, Taiwan
University of Jember, Indonesia
the purpose is to prepare technologists and scientists needed for the de- Chun-Yen Chang
velopment of innovation and research as a foundation for the economic National Taiwan Normal University, Taiwa
welfare of an emerging economy as well as the development of many
63
(P. 63-76) ISSN 2538–7138 /Online/
nations. However, the development and deployment of STEM Education in every continent and country are
different. Some countries are already implementing it at an advanced level, while some countries are still in the
introduction and trial stages (National Audit Office, 2010). Therefore, an instrument to evaluate the attitude,
knowledge and even application (AKA) of STEM Education is necessary.
The current developed AKA instrument encompasses three domains at once that include STEM attitude,
knowledge and application by the teacher. Based on review literature, some previous studies have also devel-
oped related instruments on STEM. Lam, Doverspike, Zhao, Zhe, and Menzemer (2008) developed an instrument
to know the STEM career interest in middle school students. Meng, Idris, Eu, and Daud (2013) also developed a
survey instrument to elicit perceptions of secondary school students on the STEM-related subjects in the school
assessment practices. Ibrahim, Aulls, and Shore (2017) developed an instrument to survey the STEM for bachelor
students’ achievement values of inquiry engagement. Nevertheless, those kinds of STEM survey instrument have
a function for eliciting information from students.
Other studies developed instruments that focused on the teachers. El-Deghaidy and Mansour (2015) have
developed an instrument focused on science teachers’ perceptions of STEM Education. Another research, Ven-
nix, Brok, and Takonis (2016) have developed a survey questionnaire to assess the perceptions of STEM-based
outreach learning activities both for the students and teachers in secondary education. However, this instrument
has not been comprehensive enough for general use. The instrument has a specialty such as only to elicit data
regarding teacher perception of STEM-based outreach learning activities. That instrument would be a problem
to obtain the teachers’ perception of STEM in general, without preceded by any STEM activities. For instance,
an instrument used to get teachers’ attitude, knowledge, and application simultaneously regarding STEM
education as well as the problems and challenges faced by teachers on the particular area, country, or global.
Moreover, there is limited research that develops survey instruments focusing specifically on science teachers.
One solution is to re-evaluate, modify, combine, and reconstruct the instruments (Summers & Abd-El-Khalick,
2018) into a single instrument by convergence with an existing STEM education framework: a framework for
K12 science education (NRC, 2012). However, these approaches cannot overcome the ingrained problems of
existing instruments. The AKA instrument simply answers the needs of science teachers, as we have known the
vital role of a science teacher (Kola, 2013; Ruiz et al., 2014) in quantitatively measuring the levels of attitude,
knowledge, and application of STEM.
Teachers’ Attitude, Knowledge and Application of STEM
Attitude, knowledge, and application (AKA) are the three main words that often appear in the assessment,
especially in education. The definition also has a broad meaning. The first aspect introduced here is the attitude.
Maio and Haddock (2014) argued that the term attitude can be defined in ample ways. Pryor, Pryor, and Kang
(2016) defined attitude as negative, positive, or neutral feeling regarding some behavior or object. Then, in term
of direction and strength, the attitude can vary such as a spectrum from extremely unfavorable to extremely
favorable. Ajzen (2005) and Maio and Haddock (2014) explained that an overall assessment of an object in several
conditions such as unpleasant or pleasant, and bad or good is a definition of the term attitude. Therefore, in this
research “attitude” indicates whether the science teacher agrees or disagrees with the application of STEM, at
the time of the students’ classroom learning, and the students’ sense of curiosity towards STEM, as well as what
is the teacher thinking and feeling about STEM. Al-Salami, Makela, and de-Miranda (2017) proved that primary
teachers need to develop both skills and attitudes toward interdisciplinary teaching. To determine secondary
teachers’ attitudes regarding teaching of integrated STEM-related subject, Thibaout, Knipprath, Dehaene, and
Depaepe (2018) have developed a valid questionnaire in alignment with a theoretical framework encouraged
by Van-Aalderen, Walma and Asma (2012). The finding of the validation research provided any evidence for a
framework composing of three conditions, namely perceived control, cognition, and emotion. In the field of
STEM education, the study on science teachers related to the attitudes is relatively limited.
The next aspect introduced is knowledge. The definition of knowledge also has a broad meaning. Not
Surprising if many discussions of teacher knowledge, just how difficult it is to probe this aspect of teaching
practice by a quantitative survey. Thomson (1998) defined the term of knowledge as many specific meanings.
Firstly, as familiarity or awareness obtained by an experience of a thing, person, or a fact, or as a person’s range
of any information. Secondly, as a practical or theoretical understanding of a language, subject, etc. or as an
64
ISSN 1648–3898 /Print/ DEVELOPMENT AND VALIDATION OF A SURVEY INSTRUMENT (AKA) TOWARDS ATTITUDE,
ISSN 2538–7138 /Online/ (P. 63-76)
amount of what was known. Thirdly, a right justified belief; a specific understanding as opposed to a perception
or opinion. Furthermore, Shulman (1986) divided teacher knowledge into three forms: propositional knowl-
edge, case knowledge, and strategic knowledge. In the process of acquiring such knowledge, Biggam (2001)
argued that there are many ways to gain knowledge. It could be obtained through some experiences. Moreover,
knowledge could also be obtained from rational thought. In addition, knowledge can also be more specialized
or even expandable. Nowadays, information and communication technology greatly influences the way to com-
municate and work of a knowledge (Binckly et al., 2012). Thus, in this research, the researchers restricted the
term teachers’ STEM knowledge to all information held by a science teacher about STEM education regarding
the extent of the term STEM. The knowledge weather STEM education is one kind of teaching method or not,
as well as their knowledge of the way to apply STEM in the classroom, including the interconnectedness of one
discipline with another. Koehler and Mishra (2009) said that the teacher needs to be knowledgeable in each
discipline and understand how they interconnect with each other.
STEM teachers have content knowledge that includes the scientific method, evidence-based reasoning,
principles of engineering design and constraints, and mathematical theories and constructs, and technology
applications that support their content knowledge. However, pedagogy is also one important thing. Pedagogy is
the knowledge of how students learn classroom management skills, lesson planning and assessment (Koehler &
Mishra, 2009). The challenge is to have sufficient STEM content knowledge, and effective pedagogical knowledge
to make the learning effective, challenging and engaging. Several previous studies attempted to measure the
knowledge, such as Lam, Doverspike, Zhao, Zhe, and Menzemer (2008), have tried to extract insight information
about STEM, especially among high- school students. They elicit information by surveys through workshops on
the knowledge and beliefs of students and parents on STEM Education. Gosselin and Macklem-Hurst (2002) also
surveyed to access the level of knowledge of students in high school. Nevertheless, information about STEM
teacher knowledge is still very limited.
Furthermore, the term of application, practice, and implementation are words that have the same relative
meaning. Those show the meaning of realization or performance of some activities. Meanwhile, the term of prac-
tice is a real implementation or the usedness of a belief, method, or idea, as opposed to related-theories toward
the term. Then, the implementation is the process of putting a decision or plan into effect or execution. In this
current research, the researchers would like to use the term application rather than two other words. The term
application is more appropriately applied to describe the teacher’s STEM performance in the classroom. Many
researchers have addressed the application of STEM in the classroom, but few still discuss it from the teacher’s
point of view, especially on a quantitative application. Han, Yalvac, Capraro, and Capraro (2015), have studied
using a qualitative case study approach regarding teachers’ implementation and understanding of STEM-related
activities. They stated that in applying STEM in the classroom the teacher should pay attention to the academic
level of the students, and the teacher should prepare as best as possible and try hard. English (2016), Herschbach
(2011), as well as Kelley and Knowles (2016) asserted that whilst the quantity of STEM education over countries
is significantly increasing, however, still not much was known about methods or approaches for the application
of STEM education instruction. Overall means that, to understand the current situation in the field, especially in
STEM education, this instrument is strongly necessary.
Research Focus
Any limitation of the existing STEM survey instruments on some previous researches (El-Deghaidy & Mansour,
2015; Meng, Idris, Eu, & Daud, 2013; Vennix, Brok, & Takonis, 2016), regarding scope and purpose as well as the need
of an appropriate tool to access the latest progress on STEM education were the concern to be addressed in this
current research. Then, this research focused on the design and development of a valid instrument that is used to
determine the general quantitative degree of attitude, knowledge, and application of STEM by science teachers.
This instrument serves as an essential tool and reference for evaluating and understanding the general descrip-
tion of STEM progress. The AKA instrument enables users to obtain data and information on the development of
STEM, and the problems and challenges faced by science teachers worldwide. The worthiness of the instrument
is determined by: (1) how does the development process of AKA instrument, and (2) how does the validity of the
AKA instrument.
65
(P. 63-76) ISSN 2538–7138 /Online/
General Background
This research attempted to develop an instrument used to scrutinize the quantitative degree of STEM attitude,
knowledge, and application among science teachers. This research is categorized as Research and Development
(R & D). There were four phases to get the final instrument namely planning, construction, quantitative evalua-
tion, and validation. The ultimate instrument was qualified as a survey instrument used to determine the general
quantitative degree of attitude, knowledge, and application on STEM simultaneously by science teachers on the
secondary school. The quality of the instrument was determined based on reliability and validity. The quantita-
tive way by Statistical Package for the Social Sciences (SPSS) was done in term accessing the reliability, while the
validity was determined by both quantitative (SPSS) and qualitative ways (analysis of experts’ opinions). Data were
collected from Indonesian science teachers of secondary schools in different provinces from February to April 2018.
Sample
Samples taken for this test pilot were 137 participants who are secondary school science teachers in Indonesia.
Table 1 here, points out a demographic data of respondents. This demographic data comprised gender, education
background, teaching experience, and specialization of teaching. The respondents were secondary school science
teachers from eight provinces in Indonesia for validity and reliability testing. In terms of ethical concerns, the re-
searcher ensured the confidentiality of the participated teachers’ identities in the current research. The purpose
of the research was explained as well in detail, and all the respondents were assured of the confidentiality of their
responses. Furthermore, there is no coercion in the retrieval of data and could be interpreted as all participants
sincerely filling out and involved in this research. The subjects of the teachers were Chemistry, Physics, Biology,
and Integrated Science subject both on junior and senior high school.
Table 1. Statistics data of the respondents.
Variable Category Quantity Percentage (%)
Gender Male 51 37.22

Female 86 62.78
Education Bachelor 93 67.78

Master 44 32.22
Teaching Experience <10 Years 97 70.80

>10 Years 40 29.20
Area of Specialization Integrated Science 51 37.22

Biology 52 38.00
Physics 18 13.13
Chemistry 16 11.65
Procedure
There were four phases used in the design and development of this instrument. These phases were bypassed
in order to obtain accurate instruments in measuring target variables (Creswell, 2005). Those four phases are plan-
ning, construction, quantitative evaluation, and validation. The four stages of the research are shown in Figure 1.
66
Physics 18 13.13
Chemistry 16 11.65
Procedure
There were four phasesDEVELOPMENT
used in theAND
design and development
VALIDATION of this instrument.
OF A SURVEY INSTRUMENT TheseATTITUDE,
(AKA) TOWARDS phases
were bypassed in order
ISSN 2538–7138 /Online/ to obtain accurate instruments in measuring target
KNOWLEDGE ANDvariables (Creswell,
APPLICATION OF STEM
(P. 63-76)
2005). Those four phases are planning, construction, quantitative evaluation, and validation. The
four stages of the research are shown in Figure 1.
Planning Quantitative Evaluation
Purpose Pilot Study
Target Group Reliability Test

Final Instrument
Shape Face & content

main way
Domain (item pool) Factor Analysis
hidden way
Construction Validation
Figure 1. Research Phases of developing AKA instrument (adapted from Creswell, 2005).
Figure 1. Research Phases of developing AKA instrument (adapted from Creswell, 2005).
The first phase was planning. At this stage, determining the purpose of making the instrument as well as the
The first phase was planning. At this stage, determining the purpose of making the instrument
target group to be tested, were the main focus. As the researchers discussed before, the aim was to provide a basic
as well as the target group to be tested, were the main focus. As the researchers discussed before, the
tool and reference for evaluating and understanding the general description of STEM progress. The second stage
aimwaswas to providestage.
the construction a basic
On thistool
step,and reference determined
the researchers for evaluating
the shapeand understanding
of the the identified
survey instrument, general
description
the use of aof5-point
STEM progress.
Likert-type The
scale, second
divided stagedomain
the STEM was theintoconstruction
four sections and stage. Onitems
created this about
step, each
the
researchers determined
domain (item pool) and the
othershape of theissues
additional survey instrument,
to strengthen theidentified theThe
survey results. usethird
of astage
5-point
was Likert-type
to determine
scale,
anddivided
conduct athe STEM
pilot study todomain into four sections
obtain information about the and created of
deficiencies items about each
the instrument domain
made based(item pool)
on feedback
andfrom
other
the additional
respondent. Theissues
last to strengthen
stage the The
was validation. survey results.
validation wasThe third using
conducted stageface
wasandtocontent
determine and
validation
by experts as well as by exploratory factor analysis using SPSS. Actually, the quantitative phase and validation phase
were performed simultaneously, and the result of this step would go back to the construction phase. The cycles
did not finish until the researchers got the final instrument (hidden way). In the result of the research below, the
researchers in detail provided what already had done in each phase, especially for the second to the fourth phase.
Data Analysis
In terms of to know the internal consistency in each domain created in the pilot study, the researchers used the
reliability of the instrument. The internal consistency of the instrument was known from the value of the Cronbach’s
Alphas. Three main domains and five subdomains were accessed using this test. The domains were STEM attitude,
STEM knowledge, and STEM application. The subdomains were STEM Science-Technology, Science-Engineering,
Science-Mathematics, Science-Technology-Engineering, Science-Technology-Math, Science-Engineering-Math, and
Science-Technology-Engineering-Math. Then, the next step was validation. In the validation phase, there were two
types of validation done. The first validation of the instrument was assessed by three experts who are as content
and face instrument validators. Those experts have experience in the development of an instrument and also know
about STEM Education. Two of them are professors, and another one is a doctor in the science education area.
They were using a 5-point Likert-type scale with a range of one to five to get their level of agreement. From the
validation the results obtained the degree of approval from the validators as well as some suggestions and input.
Feedback from validation was used to improve the research instrument for the better. The second validation was
to perform exploratory factor analysis using the SPSS for windows version 22 that was by analyzing the loading
factor of each domain item instrument.
67
(P. 63-76) ISSN 2538–7138 /Online/
Results of Research
Development of Domain and Item Pool
The results of this development, the researchers titled “Survey of Science Teachers’ Attitude, Knowledge, and
Application (AKA) of STEM.” The development of this instrument began with STEM domain grouping into three
main groups, namely STEM knowledge (SK), STEM Application (SAp), and STEM Attitude (SAt). STEM knowledge
was a domain that reveals the element of knowledge of respondents to STEM education. The second domain was
the STEM Application. In this domain, these questions lead to information about the extent to which STEM ap-
plications were in the classroom by teachers of science teachers. STEM application was derived into several other
small parts of STEM Science-Technology (SAp-ST), Science-Engineering (SAp-SE), Science-Mathematics (SAp-SM),
Science-Technology-Engineering (SAp-STE), Science-Technology-Math (SAp-STM), Science-Engineering-Math (SAp-
SEM), and Science-Technology-Engineering-Math (SAp-STEM). The last domain was a STEM Attitude (SAt). The SAt
explores the information about attitude or respondents’ view towards STEM education.
The STEM knowledge domain consisted of four question items that were representative of a single construct.
In the beginning, we developed six questions, however, finally after reliability and validity test, they became four.
The STEM application domain consisted of 23 question items consisting of six constructs. The 23 items were also
the result after the reliability and validity test. While the last domain, STEM attitude consisted of three question
items that were part of a construct. The final question items and each construct are shown in an appendix.
Reliability of Instrument
The analysis of instrument reliability level was done after obtaining data of the respondents’ test results us-
ing Cronbach’s Alpha method. Reliability test is an index showing the extent to which measurement tools can be
trusted or relied upon. Below (Table 2), shows a resume of the reliability test results of the instrument.
Table 2. Reliability of the instrument.
STEM Domain Internal Consistency (alpha)
STEM Knowledge (SK) .908

STEM Application (SAp)
Science-Technology (SAp-ST) .819
Science-Engineering (SAp-SE) .792
Science-Math (SAp-SM) .811
Science-Technology-Engineering (SAp-STE) .793
Science-Technology-Math (SAp-STM) .724
Science-Engineering-Math (SAp-SEM) .684
Science-Technology-Engineering-Math (SAp-STEM) .865
STEM Attitude (SAt) .866
Table 2 above shows that the value of internal consistency alpha of all STEM domains was greater than .6. The
highest internal value of the highest three domains was in the domain SK (.908) and the lowest value is in the SAp
domain (.865). As for fractions of STEM application, the highest value was at SAp-STEM (.865) and the lowest in
SAp-SEM domain (.684). These values mean that the scale reliability of Cronbach’s alpha was similar to with values
greater than .6 considered acceptable. Therefore, the AKA instrument was to be acceptable of internal consistency
among domains.
68
ISSN 2538–7138 /Online/ (P. 63-76)
Validity of Instrument
Development of the survey instrument was then validated to provide confidence, whether the instrument
was feasible to use to obtain the right data or not. This research used two types of validation. The first, content
and face validity was done using a 5-point Likert-type scale by three experts. This validity was conducted to see
how the instrument looks like or appearance. Table 3 shows the results of the content and face validity, which
concludes how much expert-approved or agreed on the presence and content of the instrument. Moreover, this
validity was done to determine the readability, accuracy, and suitability of the instrument content. As a part of
this validity, the experts gave some suggestions and comments as well. Second, the validation was verified by
doing an exploratory factor analysis (EFA). There were some reasons why the researchers were doing EFA. Firstly,
there was a set of underlying variables called factors that for aggregation of observed variables could indicate
the interrelationships among those variables. Secondly, the authors want to know whether these items hang or
swing together to create a construct. Finally, the authors want to know whether the questions in the survey had
similar patterns of responses.
Table 3. Content and face validity.
Validation Score
No Criteria
Expert 1 Expert 2 Expert 3 Mean±SD
1 Systematic preparation of instruments 5 5 4 5±0

2 Clarity and legibility of every item in the sentence 4 4 4 3.5±.7
3 Coherence between items within each domain 4 4 3 4±0
Completeness of items based on the purpose of developing the
4 4 4 4 4±0
instrument
The accuracy of all items in measuring knowledge, attitude and ap-
5 4 3 4 3.5±.7
plication of STEM
6 Ease of use 5 5 4 5±0
7 Scores obtained from the survey will illustrate the real conditions 4 3 4 3.5±.7
Does not contain SARA (racial, ethnic, and religious issues), violate
8 5 5 5 5±0
intellectual property, pornography and bias (gender, territory, etc.)
Total 35 33 32 35±1.4
Percentage of validity 87.5% 82.5% 80% 83.33%
Note: Scale is 1 to 5; The framework validity is adopted and modified from Burton & Mazerolle (2011) and BSNP (2016).
The result was that the average expert agreement rate is 83.33%. Based on BSNP (2016), the agreement scores
from the experts is 83.33% in the good level. No criteria items scaled below three, which means that all experts
agree on the form and content of the instrument. However, experts still provided some notes and comments to
refine the instrument. The experts’ notes and comments are summarized in table 4 below.
Table 4. Comments, expert advices and improvements.
Domain Comments Improvements
Identities/ Demog- Expert 1 Please insert the aim of research in the questionnaire Write down the aim of the research on the introduc-
raphy tion of this questionnaire
Expert 2 It is better not use “name”, but initials only Change the item name to an initial name
STEM Knowledge Expert 1 I often see my friends doing STEM class, this item should Removed
be removed because it will make the respondent confused
Expert 3 I know everything about STEM I know the term of STEM
69
(P. 63-76) ISSN 2538–7138 /Online/
Domain Comments Improvements
STEM Application Expert 1 Please give some examples of term the design something I often ask students to design something related to
on the SAp-SE item the topic of science (ex: design replica of DNA, atom,
etc.)
Expert 2 The meaning of technology not only as a tool, but also as a My students are actively involved using simple
system of thinking in producing something technology or a particular procedure to produce
something in learning
The number of items for SAp-STEM is too much unbal- Reduce the number of items from five to four items
anced with other domains
Give an example for this item, I often invite students to use I often invite students to use all possible technologies
all possible technologies to collect data on learning in the to collect data on learning in the science classroom
science classroom (ex: using a thermometer and use mathematical com-
putation to make a decision)
STEM Attitude Expert 3 The two items are too few, add at least one more Added one item “I am very interested to know more
how to properly integrate the mathematical, tech-
nological and engineering approaches in teaching
science in the classroom”
Complementary Expert 2 The sentence structure of the item “Based on your knowl- Fixed the sentence structure and added word current
edge, provide some of the possible difficulties... is not ap- and ability. Based on your current knowledge and
propriate for the respondent, please add the word “current abilities, provide some of the possible difficulties
and ability”
Sentences or words in italic words are the result of a change
Those notes from the experts became valuable advice for researchers. Well known that what some people
thought is good, may be different from what others think. Therefore, here, the role of experts was needed. However,
some notes are not included in the table because researchers had a difference in understanding, but the numbers
are very limited. For example, one of the comments is not to use the word “always” in the survey sentences. After
carefully considering and relying on the original goal, the researchers ignored the note. Thus, after revising and
accommodating the comments and suggestions from these experts, the progress of the instruments improved.
In addition to the face and content validation by experts, the researchers also performed a construct valida-
tion using exploratory factor analysis method. In terms of the construct validity, explanatory factor analysis was
performed by using the IBM SPSS 22.0 for windows program. Based on Hair, Black, Babin, Anderson, and Tatham
(2006), only any factors with an eigenvalue higher than one were included as representative. Then, Keiser-Meyer-
Olkin (KMO) of sampling adequacy test .822 indicating that the variables are highly factorable (Table 4). The result
of Bartlett’s Test of Sphericity is significant (p< .05). The finding indicates that variables were correlated. The Bartlett
test was statistically significant, then based on Pallant (2005), the value of KMO found is higher than the recom-
mended value of .60, and that means the researchers continued the factor analysis.
Table 4. Sampling adequacy test.
KMO Measure of sampling adequacy .822
Approx. Chi-Square 996.724

Bartlett’s Test of Sphericity df 55
Sig. (p) .0001
Analysis result of a construct validation using exploratory factor analysis method shows three factors as shown
in table 5 below. From the analysis, results obtained information that all items have a factor loading score higher
than .5. However, one of the items (SAP_STEM4) gets a not-so-good value of .65, but the value is still eligible to
include the item into the analysis.
70
ISSN 2538–7138 /Online/ (P. 63-76)
Table 5. Cronbach’s α and factor loading for the main domain of STEM.
Items Factor I Factor II Factor III
SK, α value =0.90

SK1 .90
SK2 .91
SK3 .89
SK4 .75
SAP, α value =0.86
SAP_STEM1 .83
SAP_STEM2 .88
SAP_STEM3 .84
SAP_STEM4 .65
SAt, α value =0.86
SAt1 .88
SAt2 .86
SAt3 .83
Percentage of Variance 45.54% 18.10% 14.09%
Total variance explained = 77.74%
The largest percentage of variance is in factor one, which is 45.54% of the total variance explained about
77.74%. Finally, using principal component analysis with Varimax Rotation Method items SK1, SK2, SK3, and SK4
were shown to belong to Factor 1 as the values are larger than .3. Item SAP_STEM1, SAP_STEM 2, SAP_STEM3
and SAP_STEM4 belong to Factor 2. Then, items SAt1, SAt2 and SAt3 belong to Factor 3. Factor 1 refers to STEM
knowledge, factor 2 refers to STEM application and factor 3 refers to STEM Attitude.
Final Instrument
The final instrument was the result after several revisions based on expert input and statistical analysis from
the pilot study. The instrument consisted of three parts: the introduction, the core questionnaire items, and ad-
ditional or complementary questionnaire items. The introductory section contained any matters relating to the
purpose of making the instrument, how to use the instrument, how to provide and calculate survey scores and
instructions for how to obtain data easily. For instance, how do you score the survey? Each item response is scored with
a value of one assigned to strongly disagree, all the way to five for strongly agree. For each construct, the participant’s
responses are averaged. For example, the four questions under SK (STEM Knowledge) are averaged to produce one SK
(STEM Knowledge) Score.
The core questionnaire item consisted of items to access respondents’ demographic data totaling eight ques-
tions and the items to elicit teachers’ AKA regarding STEM education. The example item to access respondents’
demographic data was about “the range of teachers’ teaching experience (< 10 Years or > 10 Years)”. Then, the items
to explore STEM knowledge, attitude, and applications totaling 30 questions. The example item to get information
about teachers’ STEM attitude was “I strongly agree to implement the mathematical, technological and engineering
approaches in teaching science in the classroom.” Next, an example item to elicit information about teachers’ STEM
knowledge was “I know the term of STEM.” Finally, the sample item to get information regarding STEM application was
“I usually teach science content using any kinds of technologies, engineering and mathematical context simultaneously.”
While the last part consisted of questions to access the application model of STEM and two open questions
about opinions and obstacles or challenges of the implementation of STEM. For instance, “based on your current
knowledge and abilities, please provide some of the possible difficulties that will be faced when applying the integration
of mathematical, engineering, and technology approaches in science classroom learning!” The full form of core items
regarding STEM knowledge, attitude and application by science teachers of the final instrument is shown in the
appendix of this paper.
71
(P. 63-76) ISSN 2538–7138 /Online/
Discussion
Exploratory factor analysis indicated that a three-factor was formed namely STEM knowledge, STEM application,
and STEM attitude, which was calculated from 77.74% of the total variance. The analyses of the result showed that
a sum of items loaded into factors that corresponded to the theoretical structure. Cronbach’s alpha value of each
domain or construct was all over .60. Important to note that Cronbach’s alpha was evaluated similarly to scale reli-
ability with values between .7 and .9 considered good. Then, the Cronbach’s alpha values higher than .6, however,
considered acceptable. Dillon and Goldstein (1984) and Joreskog (1971) asserted that the scale reliability also referred
to as construct of reliability, was measured based on results of the exploratory factor analysis.
In addition, the distinguishment between this development process and previous similar process of developing
an instrument is the content and face validation of the experts to elicit their approval or agreement. Dorrusen, Lenz,
and Blavoukos (2005) said that researchers are necessary to assess the reliability and validity of the information pro-
vided by some interviewees in surveying research and expert interviews. Furthermore, they claim that inter-expert
agreement is fundamental in the validation process of a new instrument. The result of the validity of the face and
content performed by the expert showed the average approval value of 85%. Based on the analysis results of the
factors and the expert agreement indicated that the developed instrument was reliable and valid, it is feasible to be
used to collect data. However, any researches or replicated researches are still necessary to make this instrument more
meaningful. Lin and Tsai (2017) suggested that replicated studies with national random samples may be meaningful
in consolidating the findings of a study.
In terms of the item’s context in this AKA instrument, the authors referred to several definitions that have been
raised by previous research studies. An item related to STEM knowledge, for example, was “I know the term of STEM.”
Based on Thomson (1998), this item belonged to the category of knowledge that originates from a person’s awareness
or familiarity with something or a condition. Surely, in this case, was the awareness of the STEM education knowledge.
Next one, the example item to get information about teachers’ STEM attitude was “I strongly agree to implement the
mathematical, technological and engineering approaches in teaching science in the classroom.”The item could be classified
into a statement that was predicted to explore information regarding person’s attitude due to it was related to feelings
of like or dislike, agree or disagree about a condition (Pryor, Pryor, & Kang, 2016), in this case, of course, again is related
to the STEM education. Finally, an example item on the STEM application domain, e.g., “I usually teach science content
using any kinds of technologies, engineering and mathematical context simultaneously.” Authors argue that the sentence
could extract information about teacher’s STEM performance or activity in the classroom. This argument is under the
definition from Thomson (1998) which stated that the application is an action of implementation of something on a
surface. Thus, all items contained in AKA’s instrument were relevant, valid, and feasible concerning its construct and
definition to be used in assessing some important aspect on science teacher regarding STEM education in the field.
Teachers are a vital component in an educational process. Ibrahim and Aulls (2017) found that teachers’ roles
in class included the teachers’ roles as motivators and facilitators as well as teachers’ roles as mentors and models.
Furthermore, the part of the teacher in STEM teaching and learning approach is to aid the students to develop a
conception or abstractions and to decontextualize concepts for implementation in a variety of authentic contexts
on different real-world problems (Moore et al. (2014). Thus, knowledge of STEM is absolutely required by science
teachers. Quantitative knowledge can aid teachers towards better understanding and attitude. A good attitude led
an impact on STEM application in everyday teaching and learning process. Various studies have proven that STEM
applications by teachers are affected by several attributes such as the level of knowledge and attitudes of teachers
themselves (Han, Yalvac, Capraro, & Capraro, 2015; Thibaut, Knipprath, Dehaene, & Depaepe, 2018). On the other hand,
STEM is becoming a trend lately because of its reliability and potential in improving the quality of learning. However,
not all the teachers understand and accept STEM. Many teachers were indicated to have misunderstandings and
misconceptions toward STEM education. For instance, teachers show weak control of students and sometimes just sit
down and see what students are doing without any intervening (Han, Yalvac, Capraro, & Capraro, 2015). Inevitably, an
investigation or survey of the STEM attitude, knowledge, and application of the teacher becomes very important in
providing information on current conditions and for better planning in the future. WHO (2008) states that the survey
would be a tool to gather information about specific information or a specific small topic. Thus, they also suggested
that the results of investigation through a survey could be a baseline to represent data collected, at a point in time
before any intervention is carried out.
Then, some contributions of this current research to the literature. Firstly, this instrument is specially used to
assess a general quantitative degree regarding attitude, knowledge, and application of STEM education simultane-
72
ISSN 2538–7138 /Online/ (P. 63-76)
ously by science teachers. Secondly, this research used factor analysis and content and face validity from experts to
determine AKA’s instrument validity which is the step rarely funded in some similar instrument developments. Then,
the final results of the AKA instrument gave us a quantitative way to measure the progress of STEM Education.
Conclusions
The results showed that the designed and developed AKA instrument was valid and could be used to collect
data. The AKA instrument allows for collecting quantitative data on a large-scale; and by using an amalgamated instru-
ment, the field now has, a baseline to start from, or refer to, for any interventions of STEM education. The development
of AKA instrument enables users worldwide to obtain information about the progress of STEM education and the
problems as well as any challenges which would face the science teachers in the field. Furthermore, this instrument
was being a prototype for the emergence of other similar instruments that may differ in some respects depending
on the purpose of the development and where the instrument is used. Some of the things mentioned above reflect
the limitations of this instrument. For instance, the purpose of this instrument was to know the general quantitative
level of STEM, so it is unsuitable for data collecting from a STEM workshop’ pre-test and post-test or other similar
activities. Another limitation is that this instrument has only been tested in one country namely Indonesia. Then,
in the next future research, larger samples from different area and cultures should be explored. It will take several
repetitions, especially in different target countries to obtain more accurate results. However, a recommendation for
further research is to test the psychometric properties of the instrument on a variety of cultural contexts and demo-
graphic data of the targeted respondents. Nonetheless, researchers believed that the AKA instrument could be used
for researchers that are interested in, for instance, to explore some factors which affect science teachers to integrate
STEM in their classroom on any areas or countries.
Acknowledgements
The authors would like to express the gratefulness to Terrence from the Science Education Center, NTNU, who
have helped in the English editing process. This study was supported in part by the Ministry of Science and Technology
(MOST), Taiwan, Republic of China., by the number of grant 106-2511-S-003-050-MY3, A Joint Adventure in Science
Education Research & Practice (STEM 2TV) and the National Taiwan Normal University Subsidy for Talent Promotion
Program. We also would like to say thank you, for having received funding from the Ph.D. Degree Training of the 4 in
1 project of University of Jember, the Ministry of Research Technology and Higher Education Indonesia, and Islamic
Development Bank (IsDB).
References
Ajzen, I. (2005). Attitudes, personality, and behavior. London: McGraw-Hill Education.

Allum, N., Sturgis, P., Tabounarazi, D., & Brunton-Smith, I. (2008). Science knowledge and attitudes across cultures: A meta-analysis.
Public Understanding of Science Journal, 17 (1), 35-54.
Al Salami, M.K., Makela, C.J., de Miranda, M.A. (2017). Assessing changes in teachers’ attitudes toward interdisciplinary STEM teach-
ing. International Journal of Technology and Design Education, 27 (1), 63-88.
Aslam, F., Adefila, A., & Bagiya, Y. (2018). STEM outreach activities: An approach to teachers’ professional development. Journal of
Education for Teaching, 44 (1), 58-70.
Bano, R., Al Shammari, E., Fatima, S.B., & Al-Sahmmari, N. (2013). A comparative study of knowledge, attitude, practice of nutrition
and non-nutrition student towards a balanced diet in Hail University. Journal of Nursing and Health Science, 2 (3), 29-36.
Biggam, J. (2001). Defining knowledge: An epistemological foundation for knowledge management. Proceedings of the 34th Annual
Hawaii International Conference on System Sciences. (pp. 1-7). Maui, United State of America: IEEE Explore.
Binckly, M., Erstad, O., Herman, Miller-Ricci, M., J., Raizen, M., & Rumble, M. (2012). Defining twenty first century skills. In P. Griffin, B.
McGaw, & E. Care (Eds.), Assessment and Teaching of 21st Century Skills (pp. 17-66). Dordrecht: Springer.
BSNP. (2016). Instrumen penilaian buku teks pelajaran tahun 2016 [The instrument of a textbook assessment on 2016 teaching
period]. Jakarta: Kemdikbud RI.
Burton, L. J., & Mazerolle, S.M. (2011). Survey instrument validity part I: Principles of survey instrument development and validation
in athletic training education research. Athletic Training Education Journal, 6 (1), 27-35.
Creswell, J. W. (2005). Educational research: Planning, conducting, and evaluating quantitative and qualitative research (2nd ed.). Upper
Saddle River, NJ: Pearson Education.
Dillon, W., & Goldstein, M. (1984). Multivariate analysis: Method and applications. New York: John Wiley & Sons.
Dorrusen, H., Lenz, H., & Blavoukos, S. (2005). Assessing the reliability and validity of expert interviews. European Union Politics, 6
(3), 315–337.
73
(P. 63-76) ISSN 2538–7138 /Online/
El-Deghaidy, H., & Mansour, N. (2015). Science teachers’ perceptions of STEM education: Possibilities and challenges. International
Journal of Learning and Teaching, 1 (1), 51-54.
English, L. D. (2016). STEM education K-12: Perspectives on integration. International Journal of STEM Education, 3 (3), 2-8.
Gosselin, D. C., & Macklem-Hurst, J. L. (2002). Pre-/post knowledge assessment of an earth science course for elementary/middle
school education majors. Journal of Geoscience Education, 50 (3), 169-175.
Guskey, T. R. (2003). Analyzing lists of the characteristics of effective professional development to promote visionary leadership.
NASSP Bulletin, 87 (637), 4-20.
Hair, J. F., Black, W. C., Babin, B. J., Anderson, R. E., & Tatham, R. L. (2006). Multivariate data analysis (6th Ed.). Upper Saddle River, NJ:
Pearson Prentice Hall.
Han, S., Yalvac, B., Capraro, M.M., & Capraro, R.M. (2015). In-service teachers’ implementation and understanding of STEM project-
based learning. Eurasia Journal of Mathematics, Science & Technology Education, 11 (1), 63-76.
Herschbach, D.R. (2011). The STEM initiative: Constraints and challenges. Journal of STEM. Teacher Education, 48 (1), 96–112.
Honey, M., Pearson, G., & Schweingruber, H. (2014). STEM integration in K-12 education: Status, prospects, and an agenda for research.
Washington, DC: The National Academies Press.
Ibrahim, A., Aulls, M.W., & Shore, B.M. (2017). Teachers’ roles, students’ personalities, inquiry learning outcomes, and practices of
science and engineering: The development and validation of the McGill attainment value for inquiry engagement survey in
STEM disciplines. International Journal of Science and Mathematics Education, 15 (3), 1195–1215.
Joreskog, K.G. (1971). Statistical analysis of sets of congeneric tests. Psychometrika, 36, 109-133.
Kelley, TR., & Knowles, JG. (2016). A conceptual framework for integrated STEM education. International Journal of STEM Education,
3 (11), 2–11.
Kola, A. J. (2013). Importance of science education to national development and problems militating against its development.
American Journal of Educational Research, 1 (7), 225-229.
Lam, P., Doverspike, D., Zhao, J., Zhe, J., & Menzemer, C. (2008). An evaluation of STEM program for middle school students on learn-
ing disability related IEPs. Journal of STEM Education, 9 (1&2), 21-29.
Lin, T. J., & Tsai, C. C. (2017). Developing instruments concerning scientific epistemic beliefs and goal orientations in learning science:
A validation study. International Journal of Science Education, 39 (17), 2382–2401.
Loehlin, J. (1998). Latent variable models: An introduction to factor, path, and structural analysis. Lawrence Erlbaum: Associates
Publishers.
Maio, G., & Haddock, G. (2014). The psychology of attitudes and attitude change. London, England: Sage Publications.
Meng, C. C., Idris, N., Eu, L. K., & Daud, M. F. (2013). Secondary school assessment practices in science, technology, engineering and
mathematics (STEM) related subjects. Journal of Mathematics Education, 6 (2), 58-69.
Moore, T. J., Stohlmann, M. S., Wang., H. H., Tank, K. M., Glancy, A., & Roehrig, G. H. (2014). Implementation and integration of en-
gineering in K-12 STEM education. In J. Strobel, S. Purzer, M. Cardella (Eds.), Engineering in precollege settings: Research into
practice, (pp. 35–59). West Lafayette: Purdue University Press.
National Audit Office. 2010. Educating the next generation of scientists. London: The Stationery Office.
National Research Council (NRC). 2012. A Framework for K12 science education: Practices, cross, cutting concept, and core ideas. Wash-
ington: National Academies Press.
Pallant, J. (2005). SPSS survival manual. (2nd Ed.). New York: Open University Press.
Pryor, B. W., Pryor, C. R., & Kang, R. (2016). Teachers’ thoughts on integrating STEM into social studies instruction: Beliefs, attitudes,
and behavioral decisions. The Journal of Social Studies Research, 40, 123–136.
Roelens, K., Verstraelen, H., Egmond, K.V., & Temmerman, M. (2006). A knowledge, attitudes, and practice survey among obstetrician-
gynaecologists on intimate partner violence in Flanders, Belgium. BMC Public Health, 6 (238), 1-10.
Ruiz, M. A. O., Osuna, L. V., Salas, B. V. A., Wienner, M. S., Garcia, J. S., Cordova, E. C., Nedev, R., & Ibarra, R. (2014). The importance of
teaching science and technology in early education levels in an emerging economy. Bulletin of Science, Technology & Society,
34 (3), 87-93.
Shulman, L.S. (1986). Those who understand: Knowledge growth in teaching. Educational Research, 15 (2), 4-14.
Siew, N. M., Amir, N., & Chong, C. L. (2015). The perceptions of pre-service and in-service teachers regarding a project-based STEM
approach to teaching science. SpringerPlus, 4 (8), 1-20.
Summers, R., & Abd-El-Khalick, F. (2018). Development and validation of an instrument to assess student attitudes toward science
across grades 5 through 10. Journal of Research in Science Teaching, 55 (2), 172–205.
Thibaut, L., Knipprath, H., Dehaene, W., & Depaepe, F. (2018). How school context and personal factors relate to teachers’ attitudes
toward teaching integrated STEM. International Journal of Technology and Design Education, 28 (3), 631-651.
Thomson, D. (1998). The concise Oxford dictionary: 9th edn. England: Oxford University Press.
Van Aalderen-Smeets, S. I., Walma van der Molen, J. H., & Asma, L. J. (2012). Primary teachers’ attitudes toward science: A new theo-
retical framework. Science Education, 96 (1), 158-182.
Vennix, J., Brok, P. den., & Taconis, R. (2017). Perceptions of STEM-based outreach learning activities in secondary education. Learn-
ing Environment Research, 20, 21–46.
Wahono, B., Rosalina, A. M., Utomo, A. P., Narulita, E. (2018). Developing STEM based student’s book for grade XII Biotechnology
topics. Journal of Education and Learning, 12 (3), 450-456.
Watermayer, R., & Montgomery, C. (2018). Public dialogue with science and development for teachers of STEM: Linking public
dialogue with pedagogic praxis. Journal of Education for Teaching, 44 (1), 90-106.
WHO. (2008). A guide to developing knowledge, attitude and practice surveys. Switzerland: World Health Organization.
74
ISSN 2538–7138 /Online/ (P. 63-76)
Appendix
Table 1. Items in the final instrument
Scales
Items Neither
Strongly Strongly
Disagree Agree or Agree
disagree Agree
Disagree
Attitude Toward STEM Education (SAt)
•• I strongly agree to implement the mathematical, technological and engineering ap-

proaches in teaching science in the classroom
•• I am sure, students will gain more values if I integrate mathematical, technological and
engineering approaches in teaching science in the classroom
•• I am very interested to know more how to apply the mathematical, technological and en-
gineering approaches simultaneously and properly in teaching science in the classroom
STEM Knowledge (SK)
•• I know the term of STEM
•• I aware that STEM in education is no kind of teaching method
•• I have enough knowledge about weaknesses and advantages of integrated STEM
•• I am convinced that STEM compile of integrated science, technology, engineering and
math or compile of at least two of those disciplines.
Application (SAp)
Application of Science-Technology (SAp-ST)
•• I often teach a science subject with a variety of electronic tools
•• My students are actively involved using simple technology or a particular procedure to
produce something in learning
•• Sometimes, I teach by asking students to use the internet in the classroom
•• I use a ready-made technology tool (not made by myself)
Application of Science-Engineering (SAp-SE)
•• As a science teacher, in one term, I make learning media by myself
•• I often ask students to design something related to the topic of science (ex: design
replica of DNA, atom, etc.)
•• I always create questions or assignments related to the students’ ability to design and
create a concept or model in the form of project tasks
Application of Science-Math (SAp-SM)
•• Sometimes, I ask students to collect observational data on a science class in the form
of numbers (quantitative data)
•• In science learning that produces data, I always lead the students to be analyzed using
simple statistics
•• I often invite students to think carefully with mathematical thinking to make a decision
Application of Science-Technology-Engineering (SAp-STE)
•• Sometimes, I design and make my own a simple technology or using a procedure to
produce something that can be used in learning (ex: design and make a lever tool,
measuring instruments, etc.)
•• In science learning in the classroom, I often ask the students to design and create any
simple tools or models
•• Using engineering and technological context to explain more about the particular
science topic
75
(P. 63-76) ISSN 2538–7138 /Online/
Application of Science-Technology-Math (SAp-STM)

•• In my class, I usually use a technology tool to mathematically analyzing of data from
observation (ex: use calculator, computer, mobile phone, ect)
•• I often invite students to use all possible technologies to collect data on learning in
the science classroom (ex: using a thermometer and use mathematical computation
to make a decision)
•• I often explain the lessons about complex calculations in the science classroom and
show it by Power Point or other learning technologies
Application of Science-Engineering-Math (SAp-SEM)
•• I often ask students to use or recycle the goods around to be more useful and economical
value, especially for the learning process of science
•• In the learning or practicum, students often use the analysis of profitability, graphics,
and spatial ability of space, especially in engineering or creating a particular model or
academic product.
•• Although I am a science teacher, I always teach to make a work which is made by the
students themselves and there is an element of mathematical calculation in the process
Application of Science-Technology-Engineering-Math (SAp-STEM)
•• I usually teach science content using technology, engineering and mathematical context
simultaneous
•• I train students using mathematical thinking, design of planning and also technological
assistance to solve various problems in decision making on science learning
•• In one term, I prepare or ask students to bring simple materials to design a particular
model, together to search for information through the website or following a particular
procedure to produce something and calculate the appropriate form (maximum length,
breadth ideal, etc. for the design)
•• I often combine by many ways the technology, design-engineering, and mathematics
approaches into a single learning topic of science
Model of the Application < 25% 26-50% 51-75% 76-100%
•• In general, what percentage of you teach using technology tools, designing technology
or designing certain concepts with students and analyzing mathematically on science
learning in the classroom?
•• In general, what percentage of you integrate the mathematical, technological and engi-
neering approaches in teaching science in the classroom in each semester?
Open-Question the Possibility of Difficulty Open-ended questions
•• Please give your opinion on the integration of mathematical, engineering, and techno-
logical approach in a science learning process in the classroom!
•• Based on your current knowledge and abilities, please provide some of the possible
difficulties that will be faced when applying the integration of mathematical, engineering,
and technology approaches in science classroom learning!
Noted: In the appendix cited above, these categories (SAt, SK, and etc.) were removed, so that participants were not oriented to the
constructs when answering the survey questions.
Received: November 27, 2018 Accepted: January 10, 2019
Bevo Wahono MA, PhD Student, Graduate Institute of Science Education, National
Taiwan Normal University, Taiwan & Lecturer at Faculty of Teacher
Training and Education, University of Jember, Indonesia.
E-mail: bevo.fkip@unej.ac.id
Chun-Yen Chang PhD, Chair Professor, Director of Science Education Center, National
Taiwan Normal University, Taiwan.
Email: changcy@ntnu.edu.tw
Website: http://changcy.com
76
A FEASIBLE GUIDANCE
FOR ORDERED MULTIPLE-
CHOICE ITEMS IN ISSN 1648-3898 /Print/
STUDENTS’ HIERARCHICAL ISSN 2538-7138 /Online/
UNDERSTANDING LEVELS https://doi.org/10.33225/jbse/19.18.77
Abstract. This research focuses on stu-

King-Dow Su dents’ 5 hierarchical levels of Ordered Mul-
tiple-Choice (OMC) items for their extensive
conceptualized understanding in the
particulate nature of matter (PNM) chem-
istry. The basic framework for OMC items is
Introduction to link students’ conceptual understanding
levels with possible cognitive responses.
Students’ core conceptual understanding has become an important Developed as the substantial learning
learning target for constructing scientific basic knowledge in the STEM perspective, OMC item approaches benefit
program. Science educators have worried about how to evaluate students’ students with dominant formulations in
understanding levels effectively with different alternative choices, by which directing a new impetus on individual men-
they could facilitate cognitive performances and improve their learning ef- tal constructions. This research attempts
fectiveness (Bell, 2007). The best encounter of conceptual understanding is to establish the validity and reliability
in creating an environment of possible choice items via the major pathway through assessing students’ understanding
to conduct students’ different thinking levels (Neumann, Viering, & Boone, levels. The main methodology of innovative
2013). For example, there are some low performing students who rely on OMC design manifests students’ verified
rote memorization without target conceptual understanding to comprehend responsive accumulations; with 5 hierarchi-
abstract and complicated chemical phenomena (Grove, Hershberger, & Bretz, cal mental conceptualizations from naive
2008). They may not develop mental cognition to understand chemical understanding up to complete systemic
knowledge sufficiently and fail to pass university-level general chemistry PNM understanding. OMC findings offer
exams (Bhattacharyya & Bodner, 2005; Ferguson & Bodner, 2008). students more choices to determine the
The use of ordered multiple-choice (OMC) was not a readily defined most appropriate corresponding answer
category of conceptualization, nor was its discussion and understanding towards different conceptualization levels
limited to STEM program aspects. Serving initially as an assessment tool for and to set their individual responses at the
models of students’ cognitive development, the unique feature of OMC items medium understanding levels. All con-
could be linked to discover students’ responses of core understanding levels tribution of this research gives students’
in choice items. Researchers have observed students’ understanding levels future perspectives in more collaborative
to build-up individual cognitive skills and construct multiple educational engagements with further administering
perspectives for effective test items in chemistry learning. Hadenfeldt et al. OMC items.
(2013) developed students’ conceptual understanding levels as an approach
Keywords: Ordered Multiple-Choice
for logical reasoning instrument in many university chemistry courses. Briggs
(OMC), particulate nature of matter, hierar-
and Alonzo (2009) integrated OMC items to elicit individual responses to
chical levels.
appropriate understanding levels, and to analyze students’ developmental
insights for more elaborate perspective. From OMC item choices to individual
cognitive performance, students’ responses of concept understanding levels
King-Dow Su
dominate the most crucial and substantial factors in evaluating students’ Hungkuo Delin University of Technology,
concept responses, also provide an overview of corresponding particular Taiwan & Chung Yuan Christian University,
models to investigate the validity of the OMC results (Briggs et al., 2006; Taiwan
Hadenfeldt et al., 2013; Özmen, 2013).
77
(P. 77-89) ISSN 2538–7138 /Online/
Judged from OMC item choices to students’ responses of concept understanding levels, instructors could
follow a consistent design of high learning assessment to promote students’ individual performance cognition.
Selected as a starting model design for students’ core conceptual understanding, OMC item choices reflect particular
multiple-choice items which are different from the traditional multiple-choice items (TMC). The five hierarchical
levels based on OMC instrument will require effective choices from which students make alternative information of
a scientific concept they hold and retain more advantages on their way toward developing a deeper understanding
of the respective target concept (Hadenfeldt et al., 2013). Students quickly and reliably facilitate their responsive
options related to one level across a set of OMC items compared to their developments of typical TMC choice items.
Thus, OMC item choices with hierarchical levels are featured by a more detailed assessment for students’ cognitive
coalition for reliable scientific concepts.
The present research of OMC item choices puts emphasis on students’ concept understanding levels with re-
gards to the potential basis in PNM chemistry learning design. It has been a new tactic study for university students
to follow a complete OMC command in developing their responsive levels with cognitive learning performance
(Yakmaci-Guzel, 2013). Since few researches offered satisfactory OMC items in formulating students’ different
understanding levels, the learning dilemma in need have existed to reassess students’ enacting and reconstruct-
ing students’ mental potentiality in chemistry problem-solving skills (Yakmaci-Guzel, 2013). The coalition of OMC
items with PNM chemistry knowledge would inspire students’ reconstruction of conceptual understanding levels
for more particular scientific engagements (Bunce & Gabel, 2002; Sanger, Phelps, & Fienhold, 2000). In particular,
this research sets up a functional assessment instrument for OMC items designed on students’ responsive under-
standing levels of PNM chemistry.
Research Purposes
To fit the above learning requirement, this research developed newly designed OMC items to examine students’
conceptual understanding levels in PNM chemistry learning. In response to students’ scientific understanding levels
of core concepts, this research proposed three fundamental research purposes, as follows:
1) To set up OMC items for students’ assessment of conceptual understanding levels
2) To construct validity and reliability of OMC items as an effective diagnostic instrument
3) To evaluate students’ PNM domains of understanding levels with OMC diagnostic instrument.
Research Questions
According to the above purposes, three research questions were designed in assessing students’ conceptual
performance as below:
1) What effective OMC items can be best designed for students’ conceptual understanding?
2) To what extent of diagnostic function do OMC items fulfill the most practicable performance of validity
and reliability?
3) What substantial PNM domains of students’ understanding levels can be accessed with OMC diagnostic
instrument?
Research Focus
There were three research focuses for students to go hand in hand with five hierarchical conceptual levels
rendered as their target learning. After they have finished extensive understanding levels of PNM conceptual map,
all students have a better participation in both OMC items and individual cognitive learning performance in class
which could not be otherwise acquired. Since instructors’ designation of OMC items is crucial to students’ response
and participation of conceptual understanding, the present research modifies Wilson’s four building blocks of the
construct map, items’ design, outcome space, and measurement model (2005) to develop an instrument assessing
students’ chemistry equilibrium based on designs of OMC items. To consider the importance of diagnostic assess-
ment for flexible OMC items could be explicated by an illustrated participation of students’ individual conceptual
understanding levels.
78
ISSN 1648–3898 /Print/ A FEASIBLE GUIDANCE FOR ORDERED MULTIPLE-CHOICE ITEMS IN STUDENTS’
ISSN 2538–7138 /Online/ (P. 77-89)
General Background
The primary research background enacted in this research specifically stressed students’ conceptual under-
standing levels obtained through diagnostic assessment of OMC items. It suggested not only students’ readiness
and response of PNM pre-knowledge (Shepard et al., 2005) but also more possible diagnostic assessments for
instructors to make further research requirements. To be different from both formative and summative assessment,
diagnostic assessment provides the instructors with learning information about what students should learn and
be able to do (Green & Johnson, 2010). For example, it often causes many students to get confused whenever
they are confronted with complicated problems of PNM chemistry equilibrium in class. With the OMC diagnostic
instrument, students could make correct choice items directly or indirectly for instructor’s scaffolding learning
framework. It would be hard for students to mediate the abstract behavior of the particulate of atoms or molecules
in matters because of their poor visibility at macroscopic conditions. Before the class instructions, students are
required to conduct a test in surf the net of OMC instrument during 2017/3~2018/5 to formulate their feedback
and conceptual understanding levels represented by SPSS analysis.
Participants
All 548 participants were selected through two stages of qualification tests as research samples (Su, 2016).
There were 173 university students who attended the author’ course, Nature Science, at the first stage of qualifica-
tion tests in October 2016. With the design of the full-scale experiment, the pilot test guided these 173 students
(aged from 19 to 22 years) to gather validity and reliability data of the OMC items. The remaining 375 students
were chosen for the normal participation of OMC diagnostic instrument at the second stage of qualification tests
in the 2017 academic year. Table 1 shows students’ background and characteristic analysis listed by characteristic,
variance, number and percentage, such as enrollment (sport excellence, multi-star recommendation, learning
achievement test, assigned test, uniform test and other tests). Since the participants of this research were limited
to undergraduates in Taiwan, it should be careful that the approach might not go beyond the academic context
within law inferences. All qualification tests at two stages would be carried out by participants for special precau-
tions without violating local laws and legal agreement. It confirmed with the ethical procedures that all participants
performed their publication in accordance with the full result understanding not to be anonymous and illegal as
the ethical approval did (Taber, 2014) in 2017.
Instruments and Procedures
Each design of five hierarchical understanding levels was aligned with a corresponding level for students to
assess their mental conception. The design framework of each corresponding understanding level can be a modified
pattern of OMC instrument. The ordered multiple-choice (OMC) items of this research included four parts: major
construct map, test items’ design, outcome space, and measurement models. It is a constructed exercise of mental
performances based on Wilson (2005). An exploration of the development of OMC diagnostic instrument with
validity and reliability provides a substantial basis for designing students’ corresponding level of mental framework.
Table 1. Students’ background and characteristic analysis (N=375).
Characteristic Variance Number Percentage (%)
School 1. CYCU 359 95.73

2.TCU 16 4.27
Science 1. Major 24 6.40
2. Non-major 351 93.60
79
(P. 77-89) ISSN 2538–7138 /Online/
Characteristic Variance Number Percentage (%)
Enrolment 1. Sport excellence 7 1.87

2. Multi-star recommendation 39 10.40
3. Learning achievement test 117 31.20
4. Assigned test 146 38.93
5. Uniform test 17 4.53
6.Other tests 49 13.07
Gender 1. Male 136 49.45
2. Female 239 50.55
Age 1. 18 81 21.60
2. 19 225 60.00
3. 20 43 11.47
4. 21 14 3.73
To construct effective OMC diagnostic instrument, the lecturer first needs to define what proper instructional
component should be accepted by students. Next, determine how to help students in getting more PNM progress
to promote their understanding levels. In response to the initial requirements, Briggs et al. (2006) proposed the
special test items as ordered multiple-choice (OMC), in discovering students’ diagnostic assessment of five different
understanding levels. One can approach five hierarchical OMC understanding levels, step by step from an ordered
research finding (Hadenfeldt et al., 2013). There were five hierarchical conceptual levels for PNM understanding,
indicated with scores and statements from level 1 to level 5 in Table 2. Starting from the naive concept of level 1
with basic PNM knowledge, students constructed their conceptual understanding in level 2 with the hybrid con-
cept to explain PNM scientific phenomena. Equipped with both simple and differentiated particulate conception,
students proceeded to the intermediate developments of level 3 and level 4. Finally, students can use the advanced
particulate conception in level 5 to explain macroscopic PNM properties.
Table 2. Summary of five hierarchical OMC conceptual levels indicated with scores and statements.
Corresponding Level Score Items Participant Statement
The lowest level that students do not have much experience of

1. Naive concept 1
how complicated PNM is.
Students have learned about PNM and explained scientific
2. Hybrid concept 2
phenomena using a hybrid model.
3. Simple particulate concept 3 Using a simple particulate model to explain PNM phenomena.
Using a differentiated particulate model to understand particulate
4. Differentiated concept 4
concept fine structure of sub-atoms.
Using an interaction of differentiated particulate to explain macro-
5. Systemic particulate concept 5
scopic properties of matter.
As for the validity and reliability of OMC test items, the difficulty index of test items p value was determined
by students’ fractional answering rate. The descriptive statistics of p values were indicated in four difficulty indices:
(1) p < 30% to be regarded as the difficulty test item, (2) 30% ≤ p<50% as the difficulty towards easy test item, (3)
50% ≤ p < 70% as the easy towards difficult test item, and (4) 70% ≤ p as the easy test item. Each separate p value
was measured as an average percentage of cumulative scores for test items in students’ conceptual understanding
level. The function of p value was aimed at understanding students’ distribution of difficulty index in OMC test items.
All 21 OMC draft test items were adapted from research resources of Su (2015), Nyachwaya et al. (2011), Margel,
Eylon and Scherz (2008). It was demonstrated that these draft test items had been already scrutinized, deleted
and revised by four renowned chemistry professors and two science professors to provide the content validity of
students’ achievement test items. Furthermore, the corresponding OMC test items were answered by three senior
80
ISSN 2538–7138 /Online/ (P. 77-89)
chemistry professors with more exact analyses of inter-rater reliability as Kendall’s coefficient of concordance. The
analyses opened up four possibilities for the following statistic coefficient: (1) When the coefficient was below
0.4, the analyses were developed from slight to fair agreement, (2) the coefficient between 0.4-0.6, developed to
be substantial agreement, (3) the coefficient between 0.6-0.8, developed to be consistent agreement, and (4) the
coefficient above 0.8, developed to be almost perfect agreement (Marozzi, 2014).
Data Analysis
From the analyses of Table 2, 19 OMC test items could be scored in 5 assessed statements according to stu-
dents’ conceptual understanding levels. The total score items covered the statements from level 1 to level 5 with
different points from corresponding participants. All university participants were scored for assigning 19 OMC test
items with 95 points for the total score. They could answer 19 test items within the time span of 45 minutes in class.
It was available for participants to administer their OMC instrument via Wi-Fi from their cell phones from Wi-Fi or
with a computer provided in the classroom. The aggregate of participants’ responses to set up statistical informa-
tion would be stored in an excel file, which eventually had been transmuted into the file of SPSS 22.0 Windows
software to analyze each open-ended choice item.
Results of Research
Hierarchical Correspondence of OMC Instrument
In response to students’ OMC options, three renowned chemistry professors were independently charged
to examine the fulfillment of assignments and the differentiated validity of PNM understanding levels. Kendall’s
coefficient of concordance (W) was used to determine
statistic agreement by the three professors. The research finding W value 0.613 (c2=33.107, p=.016) indicated
that the substantial responses of OMC options in understanding levels were agreed upon by three professors. The
OMC instrument included a set of 19 test items which were assigned to assess university students’ PNM under-
standing level. Figure 1 was a detailed demonstration for No. 17 test item as one of the correct choices in OMC
options. As seen in
Figure 1 from the summary of No. 17 test item, choice item A was a simple description of particulate model for
PNM phenomena in conjunction with Level 3; choice item B was a hybrid model for PNM scientific phenomena with
Level 2; choice item C was a naive model for not complicated PNM phenomena with Level 1; choice item D was an
interaction of differentiated particulate model for macroscopic properties of matter with Level 5. Students could
make out their own subsequent choice items developed from their understanding levels of hierarchical correspon-
dence. These 19 test items were all terminological usages for OMC instrument which were published at Chinese
Internet station (web address: https://goo.gl/forms/LUPm5mAgcn4xfzMv2). The aforementioned 19 test items
provided a well-designed and profound instruction for students to participate actively OMC options for guidance.
81
(P. 77-89) ISSN 2538–7138 /Online/
17. Consider chemistry equilibrium reaction equation between dinitrogen tetroxide and nitrogen dioxide,
N ₂O₄(g)⇌ 2NO₂(g), in Figure (a), at 100°C the flask is definitely reddish-brown owing to a large
amount of NO₂(g) present; and Figure (b), at 0°C the equilibrium is shifted toward colorless N ₂O₄(g). Which of
the following chemistry equilibrium reaction between 100°C and 0°C changes will be the correct choice item?
○A. At 100°C Figure (a) products a large amount of NO₂ (g) present. (Level 3)
○B. At 0°C equilibrium shifting to left due to colorless in Figure (b). (Level 2)
○C. At both Figure (a) and Figure (b) finds it is a endothermic reaction. (Level 1)
○D. The chemistry equilibrium reaction relates to chemical bond formation and breaking, so higher temperature
will favor chemical bond breaking. (Level 5)
Figure 1. Summary of OMC instrument in No. 17 test item.
Conceptual Response and Analyses
In order to give more assessments for inspiring students’ conceptual response, this research carried out the
pilot test for 173 university students in the first stage. The existing OMC qualities and functions justified students’
major conceptual response within different understanding levels. All 19 test items were designed for constructing
the OMC structure and composition in PNM. Both the function and choices of 19 test items described in Table 3
characterized students’ conceptual response of understanding levels within two major corresponding distribution
areas. The corresponding distribution of p values to difficulty index has been analyzed in the following four statistic
items: (1) p < 30% in reference to 1 difficulty test item; (2) 30% ≤ p <50 reference to
Table 3. 173 students’ responses of choice item mean score and fraction percentage for OMC test items.
Test Item
Score 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
Mean 2.78 2.01 2.80 2.72 2.42 3.29 1.68 2.19 1.40 4.28 2.77 4.17 3.60 2.79 3.55 2.39 2.42 3.40 3.42
P(%) 55.6 41.5 56.0 54.3 48.4 65.9 33.6 43.8 28.0 85.7 55.4 83.4 72.0 55.7 71.0 47.7 48.3 68.1 68.3
6 difficulties towards easy test items; (3) 50% ≤ p < 70% to 8 easy towards difficult test items; (4) 70% ≤ p to 4
easy test items. In particular, all p values highlighted two statistic distribution areas, which ranged from difficulty
towards easy test items to easy towards difficult test items. Table 3 illustrates what responsive difficulty index for 173
university students’ engagement would set up the relevant distribution of p values with their separate test items.
The second stage gave an impetus for 375 undergraduate students to participate their authentic OMC
instrument diagnoses of conceptual understanding level during 2017 academic year. While students were not
expected to give a response of the same conceptual understanding level, each OMC instrument diagnosis would
be considered to mark students’ major response numbers of each test item in Table 4. As the second stage was
administered, each choice item
82
ISSN 2538–7138 /Online/ (P. 77-89)
Table 4. Choice item analysis for 375 students’ response numbers of each test item.
Choice Test Item

Item 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
1 121 52 101 46 62 19 78 69 105 19 47 112 139 22 85 28 102 33 57

2 38 146 108 128 225 115 139 47 139 145 131 72 104 87 88 184 84 72 84
3 52 126 88 118 42 129 72 209 49 165 116 77 110 147 121 73 112 96 79
4 164 51 78 83 25 20 62 23 82 46 81 114 22 88 81 91 77 91 155
5 21 92 24 27 31 83
required students to mark their separate responses of conceptual understanding level. For example, most
students inclined toward No. 5 test item 2 in their choice items. Meanwhile students chose relatedly few choice
items with the disliked No. 6 test item 1 and No. 10 test item 1. Because differentiated phenomena illustrated that
students could make their refinement of OMC instrument for conceptual response, their different understanding
levels in assigned OMC test items would detect the core cognition of scientific matter within the whole progresses
of Table 4.
With regard to 375 students’ understanding levels, their distribution in Table 5 illustrates that the hierarchical
levels of the L4 and L5 were sparser than those of the L1-L2 and L3 for PNMCE OMC. Accordingly, students’ identifica-
tion toward higher hierarchical levels remained in a more difficult position to attain than toward lower hierarchical
levels in the OMC options. Students were required to make their responses of OMC 19 test items related to one
level across a set of the scattering plot in Figure 2. Thus, each response option gave students to contemplate their
thinking for a particular choice of hierarchical understanding levels. Students’ speculation on respective OMC items
manifested that L3 hierarchical understanding levels accumulated 47.4% from the total OMC options in conjunc-
tion with students’ medium choice of understanding levels.
Table 5. Distribution of understanding levels for 375 students’ responses of PNMCE OMC choice items.
Level Test Item

Item 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
L1 103 78 46 129 154 158 92 188 65 47 22 101 112

L2 121 146 128 139 187 147 85 84
L3 254 126 196 118 246 129 78 283 328 184 236 87 88 274 102 292 220
L4 101 83 92 31 121
L5 310 191 139 88 81 77 83 155
To obtain additional OMC options, L1 and L2 hierarchical understanding levels received 19.0% and 13.6%
from the total OMC items, in conjunction with low-understanding level students. For students with L1 and L2 hier-
archical understanding levels, we need to strengthen their PNM construction to avoid their particulate confusion
with hybrid conceptualization. It was calculated that L4 and L5 hierarchical understanding levels were considered
to have 6.5% and 13.5% from the total OMC items, in conjunction with high-understanding level. The immediate
improved remedy for L4 and L5 hierarchical understanding levels was to assign students with more instructions for
particulate concept structure of sub-atoms and macroscopic properties of matter. Of all 5 respective understanding
levels, students with L4 and L5 hierarchical understanding were ranked as the advanced group who administered
OMC item as specified and prominent participation in the scattering plot of Figure 2.
83
(P. 77-89) ISSN 2538–7138 /Online/
Figure 2. The scattering plot of understanding levels for 375 students’ responses of 19 test items.
By conducting the previous 375 students’ choices across 19 test items, 5 major percentages of hierarchical
understanding levels were carried out to identify their responsive item numbers, with the differentiated order mark
L3(3241, 45.5%)> L1(1295, 18.2%)> L5(1041, 14.6%) >L2(1037, 14.5%)>L4(511, 7.2%) in Figure 3. The authentic
distribution of 375 students’ responsive choices goes deep into 5 order marks of hierarchical understanding levels.
It deserved careful attention that the order mark L3 occupied the most dominant percentage (45.5%) of total 375
students’ responsive choices. The distributive test items of L3 understanding levels demonstrated students’ familiar
knowledge for the simple particulate model of PNM phenomena. Figure 3 also indicated other responsive item
numbers of the lower students’ 32.7% understanding levels for L1 and L2might be compared with those of the
higher students’ 21.8% understanding levels for L4 and L5.
Figure 3. Distribution of understanding levels for 375 students’ responses across 19 test items.
In order to find out students’ divided distribution of item aspects, this study conducted the descriptive sta-
tistics of mean scores and standard deviation to measure four subscales QD, QMD, QME and QE in Table 6. It was
available to demonstrate four subscales QD (difficulty test item), QMD (difficulty towards easy), QME (easy towards
difficult) and QE (easy) with the total mean score of 2.94 and the standard deviation of 0.69 measured students’
four different item aspects from difficult to easy. Students’ participation of OMC test items validated a successful
design to assess their authentic understanding levels and learning achievements from basic broad degrees to
84
ISSN 2538–7138 /Online/ (P. 77-89)
the final depth of item aspects. The employment of mean scores (M) and standard deviations (SD) offered an im-
portant instrument to inspect students’ learning effectiveness and understanding levels for their item aspects in
class. Instructors would likewise apply all results of OMC item aspects to improve students’ learning of the textual
structure and meaningful contents.
Students’ Divided Distribution of Item Aspects
Table 6. Mean scores (M), standard deviations (SD), and aspects for students’ item tests.
Item Aspect M SD Number Test Item
QD 1.64 .47 2 7, 9
QMD 2.52 .30 9 1, 2, 3, 5, 6, 8, 11, 16, 17
QME 3.42 .40 6 4, 13, 14, 15, 18, 19
QE 4.16 .92 2 10, 12
Students’ ANOVA and Relative Analysis
Table 7 would convey more accounts of OMC blocking variable with the guidance of a series of ANOVAs for
the multi-variants of the Wilks’ Lambda parameter in four subscales of item aspects. Accordingly, Table 7 suggested
a brief summary of individual OMC item aspects to go with the discussion of the F-ratios, p-values and relative
analysis (r). To regard the school as an example of the independent variable in Table 7, there appeared a significant
difference (F=6.742, p=.010) of dependent variable QD (difficulty test item) which illustrated that QD of CYCU school
students was superior to that of TCU school students in Taiwan, and the grades of CYCU (M=1.66, SD=.48) got higher
scores than those of TCU school students (M=1.34, SD=.30). In contrast to QD of CYCU’ school students’ grades, the
data shows that all dependent variables QMD, QME and QE were estimated without significant differences in Table
7. In developing more accurate evaluations of OMC item aspects, this research conducted Pearson’s differentiate
relative analysis (r) with students’ four item aspects of dependent variable, represented the relative analysis range
between .044 and .133 in Table 7. Judged from four dependent variables, only the QD dependent variable received
significant negative relation (r=-.133, p<.01) as QD towards difficulty item aspect administered TCU school students
to have an inclination for reducing consciousness learning achievement. Other three dependent variables, QMD,
QME and QE did not provide significant relative analysis.
Table 7. 375 students’ understanding level of item aspects for one-way ANOVAs in PNMCE.
Item Aspect
Experimental Blocking Analysis of
Course Variable Variance
QD QMD QME QE
PNMCE School F-ratio 6.742 .710 3.614 2.426

(1.CYCU p-value .010* .400 .058 .120
2. TCU) r -.133** .044 -.098 .080
Science F-ratio .169 3.231 .600 4.198
(1. Major p-value .682 .073 .439 .041*
2.Nonmajor) r .021 -.093 -.040 .106
Enrollment F-ratio 2.006 .990 .730 2.226
(1.Sport test p-value .007 .424 .599 .051
2. Star test r .044 -.075 .070 .021
3.School test
4.Indicate test
5. Present test
85
(P. 77-89) ISSN 2538–7138 /Online/
Item Aspect
Experimental Blocking Analysis of
Course Variable Variance
QD QMD QME QE
6. Other)
Gender F-ratio .182 .000 1.541 8.068
(1. Male p-value .667 .990. .215 .005**
2. Female) r .022 .001 .064 .146**
Age F-ratio .151 1.520 1.925 .546
(1.18 2.19 p-value .963 .196 .106 .702
3. 20 4.21 r -.008 .011 .039 -.034
5. 22)
Note: *p< .05; **p< .01.
In the area of science, science major students (M=4.19, SD=.91) received higher scores than those of non-
science major students (M=3.79, SD=.98). Taking science as an independent variable, there appeared a significant
difference (F=4.198, p=.041) of dependent variable QE (easy test items) in Table 7. Four dependent variables marked
students’ science range of relative analysis between .021 and .106 in Table 7. Only the QE dependent variable re-
ceived significant positive relation (r=.106, p< .05) which meant major science students to have more inclinations
for good consciousness learning achievement than those of non-major science students. There were no more
significant relative analyses for other three dependent variables, QMD, QME and QD.
In the area of gender, male students (M=4.26, SD=.86) scored higher than female students (M=3.99, SD=1.00).
Taking gender as an independent variable, a significant difference (F=8.068, p=.005) of dependent variable QE (easy
test items) is shown in Table 7. There were four dependent variables for students’ gender range of relative analysis
between .001 and .146 in Table 7. The only manifestation might be attributed to QE dependent variable of signifi-
cant positive relation (r= .146, p< .01) for requiring more male students to perform inclined perceptions of learning
achievement than female students did. No more significant relative analysis existed for other three dependent
variables, QMD, QME and QD. In both the enrollment and age areas, there did not appear any significant differences
(p>.05) for ANOVAs and relative analysis accorded with four dependent variables QMD, QME, QE and QD in Table 7.
Accordingly, in the case of five blocking variables with four dependent variables, the final statistical analysis
appeared to indicate that students’ good command of performed perceptions would affect their learning achieve-
ment. It is yet to be determined whether there were not any significant differences within the relative analyses of
four dependent variables. More science major and male gender students would only take the easy test items in the
performed assumption of dependent variable for both significant differences and significant relation. CYCU school
students had a good command of the difficulty of test items which illustrated more performances of beneficial
scores than those of TCU school students. A continuous research confirmed that TCU school students made up a
significant negative relation in QD dependent variable towards difficulty item aspect.
Discussion
A critical problem for this research is to adapt and validate students’ consistent agreement by three senior
chemistry professors. Through the designed content of inter-rater reliability for Kendall’s coefficient of concordance,
it has been analyzed that there are four agreements to be accorded with the statistic coefficients: (1) from slight to
fair agreement, (2) substantial agreement, (3) consistent agreement, and (4) almost perfect agreement (Marozzi,
2014). A comparison has been made for this study to focus upon the similarity of the consistent agreement under
the four agreements of corresponding OMC instrument with the functional validity and reliability in this study.
Students’ response of the consistent agreement requires a subsequent self-development of choice items from their
understanding levels of hierarchical correspondence within 19 test items. Further researches on the hierarchical
correspondence of OMC instrument would clarify students’ five understanding levels starting from (1) naive concept
(2) hybrid concept (3) simple particulate concept (4) differentiated concept and (5) systemic particulate concept
up to the most perfect level (Hadenfeldt et al., 2013).
86
ISSN 2538–7138 /Online/ (P. 77-89)
Most college students, imbued with the cognitive performances of conceptual response, have better diagnostic
responses of ordered multiple-choice (OMC) instruments than those of traditional multiple-choice (TMC) items in
their substantial PNM domains of understanding levels (Hadenfeldt et al., 2013; Othman, Treagust, & Chandrasegaran,
2008; Su, 2015). It has been accepted that students could make assigned choice items in different OMC understand-
ing levels as their conceptual response of non-linear scientific learning (Neumann, Viering, Boone, & Fischer, 2013).
In fact, students’ responsive distribution of choice items varies with their different conceptual performances of
assigned OMC test items in core scientific structure and composition of matter. The responsive focus of students’
conceptual analyses makes a foremost concentration on L3 understanding levels, to be the highest percentage
for their entire engagement in the simple particulate model of PNM phenomena. For other analyses of students’
conceptual response, only 21.8% students give their responsive distribution on both higher understanding levels
of L4 and L5, and 32.7% students occupy their responsive distribution on both lower understanding levels of L1
and L2. More endeavor for students of lower understanding levels would be needed in their refinement of PNM
conceptual response to accumulate their potential impact of higher understanding levels by flexible test item
designs (Wilson, 2005; Yakmaci-Guzel, 2013).
Students’ divided distribution of item aspects provides an intrinsic factor to assess their authentic under-
standing levels in four subscales starting from QD, QMD, QME and mediating up to QE. This research makes more
applications in detail for students to participate their improved learning of the textual structure and meaningful
contents. Several scholars have explored how the most complicated and abstract nature of atoms and molecule
gives different relevant bearing for students PNM misconceptions in their understanding levels (Othman et al., 2008;
Özmen, 2013). Additionally, students’ responsive distribution of OMC test items often misplaced their understanding
performances either stuck in fragmented mental models or in confused obstacles without cognitive development
of chemistry structures and behavior of matter (Margel, Eylon, & Scherz, 2008; Nakhleh, Samarapungavan, & Sa-
glam, 2005; Yezierski & Birk, 2006). They also advocated many disciplinary methods for students to take favorable
advantages of OMC items in their developed response of diagnostic interpretations (Briggs, Alonzo, Schwab, &
Wilson, 2006; Alonzo & Steedle, 2009).
In the corresponding case of the above four dependent variables with five blocking variables, students’ com-
mand of difficulty test items reflects a dominant dependent variable in both significant differences and signifi-
cant relation. For example, science major and male students are good at the easy test items, and the students of
CYCU school make higher scores than those of TCU school towards difficulty test items. As Yakmaci-Guzel (2013)
emphasized this assertion, more flexible discourses of consistent dependent variable should be discovered in
developing students’ problem-solving performances. A close coalition of dependent variables with students’ per-
formances is related to the correspondent fulfillment of responsive analysis. There would be more engagements
in students’ manipulating the functional skills of ordered multiple-choice (OMC) items, which include the cases of
semi-structure interviews (Alonzo & Steedle, 2009; Coll & Treagust, 2003), diagnostic assessment (Treagust & Chiu,
2011) and traditional multiple-choice (TMC) items (Briggs et al., 2006; Othman et al., 2008).
A favorable goal for five hierarchical correspondence of OMC instrument allows students to accumulate
their PNM understanding level, set up by instructors’ flexible OMC textual designs. Students are encouraged to
have direct control of increasing learning response and assess their potential impact of complicated and abstract
concepts in different understanding levels. For further framework of the functional skills, the research has to me-
diate the performance of lower understanding level students in differentiating the correct answer for particulate
concept structure of sub-atoms and macroscopic properties of matter as higher understanding level students do
in OMC instrument.
Conclusions
The key function of OMC instrument serves to facilitate students’ correct manipulation in PNM conceptual
understanding levels. At first, this research set up the disciplinary design of students’ mental development based
on OMC choice items with respect to PNM conceptual understanding. The full-fledged design of five hierarchical
conceptual levels for OMC items led students to follow a noticeable mental conceptualization on a convenient
scale. A close look at students’ OMC items would elucidate to a large degree in this research that their cognitive
response of conceptual understanding was different from one matrix level of TMC items. The whole procedure
also provided an effective assessment, in which senior chemistry professors examined the corresponding validity
and inter-rater reliability to reflect students’ understanding levels in detail.
87
(P. 77-89) ISSN 2538–7138 /Online/
The three findings of OMC instrument research offer several benefits. First, the implementation of OMC
functioned successfully in class to assess and detect different levels of students’ PNM understanding and learning
progress. Second, as students fulfilled individual performances of conceptual understanding, they could accu-
mulate more effective core chemical concept. Third, for further impact of advanced OMC items, this study offered
instructors an easy- to-check tool in scrutinizing students’ previous selections of OMC items which mostly had
been hidden within their perceptive organization for interactive potentials. More flexibilities of OMC instrument
would be taken into alternative measurements for both students of lower and higher understanding levels. The
remaining finding of OMC instrument would not be restricted to the single assumption in adjusting to only one
approach of students’ understanding levels.
The survey of students’ PNM understanding levels made up more extended approaches for further OMC in-
struments. Scholars have agreed that many disciplinary designs would be explored for students to take favorable
advantages of OMC items in their utmost development of individual responsive understanding levels. Although
students’ conceptual responses were apt to act out for non-linear scientific learning, more adjustments of OMC
items would be in great need to revise and adapt students’ misfitting breakdown in overlapping concepts in the
future. An important indicator of OMC items would be needed in the extended study to link more students’ ad-
vanced performances with different understanding levels to carry out the feasible guidance as the fundamental
framework of the functional skills.
Acknowledgements
A short but sincere thank must also be given to the patronage of the Ministry of Science and Technology,
MOST in Taiwan (under Grant No. MOST 105-2511-S-237-002-MY2). Without their help and financial support, this
research could not have been completed in the present form.
References
Alonzo, A. C., & Steedle, J. T. (2009). Developing and accessing a force and motion learning progression. Science Education, 93
(3), 389−421.
Bhattacharyya, G. & Bodner, G. M. (2005). It gets me to the product: how students propose organic mechanisms. Journal of
Chemical Education, 82, 1402-1407.
Bell, B. (2007). Classroom assessment of science learning. In S. K. Abell and N. G. Lederman (Eds.), Handbook of research on science
education, Mahwah, NJ: Lawrence Erlbaum Associates, pp. 965-1006.
Briggs, D. C., & Alonzo, A. C. (2009). The psychometric modeling of ordered multiple-choice item responses for diagnostic assess-
ment with a learning progression. Learning Progressions in Science Conference: Current Challenges and Future Directions,
Iowa City, IA, June, pp.293-316.
Briggs, D. C., Alonzo, A. C., Schwab, C., & Wilson, M. (2006). Diagnostic assessment with ordered multiple-choice items. Educational
Assessment, 11(1), 33-63.
Bunce, D. M., & Gabel, D. (2002). Differential effects of males and females of teaching the particulate nature of chemistry. Journal
of Research in Science Teaching, 39, 911–927.
Coll, R. K., & Treagust, D. E. (2003). Learners’ mental models of metallic bonding: A cross-age study. Science Education, 87, 685–707.
Ferguson, R. L., & Bodner, G. M. (2008). Making sense of arrow pushing formalism by chemistry majors enrolled in organic chem-
istry. Chemistry Education Research and Practice, 8, 99-110.
Green, S. K., & Johnson, R. L. (2010). Assessment is essential, New York: McGraw-Hill.
Grove, N. P., Hershberger, J. W., & Bretz, S. L. (2008). Impact of a spiral organic curriculum on student attrition and learning. Chemi-
cal Education Research and Practice, 9, 157–162.
Hadenfeldt, J. C., Bernholt, S., Liu, X., Neumann, K., & Parchmann, I. (2013). Using ordered multiple-choice items to assess students’
understanding of the structure and composition of matter. Journal of Chemical Education, 90, 1602-1608.
Margel, H., Eylon, B. S., & Scherz, Z. (2008). A longitudinal study of junior high school students’ conceptions of the structure of
materials. Journal of Research in Science Teaching, 45(1), 132–152.
Marozzi, M. (2014). Testing for concordance between several criteria. Journal of Statistical Computation and Simulation, 84 (9),
1843-1850.
Nakhleh, M. B., Samarapungavan, A., & Saglam, Y. (2005). Middle school students’ beliefs about matter. Journal of Research in
Science Teaching, 42, 581−612.
Neumann, K., Viering, T., Boone, W. J., & Fischer, H. E. (2013). Towards a learning progression of energy. Journal of Research in
Science Teaching, 50, 162−188.
Othman J., Treagust, D. F., & Chandrasegaran, A. L. (2008). An investigation into the relationship between students’ conceptions
of the particulate nature of matter and their understanding of chemical bonding. International Journal of Science Educa-
tion, 30(11), 1531–1550.
88
ISSN 2538–7138 /Online/ (P. 77-89)
Özmen, H. (2013). A cross-national review of the studies on the particulate nature of matter and related concepts. Eurasian
Journal of Physic and Chemistry Education, 5(2), 81-110.
Potgieter, M., & Davidowitz, B. (2011). Preparedness for tertiary chemistry: Multiple applications of the
Chemistry Competence Test for diagnostic and prediction purposes. Chemistry Education Research
and Practice, 12, 193–204.
Sanger, M., Phelps, A., & Fienhold, J. (2000). Using a computer animation to improve students’ conceptual understanding of a
can-crushing demonstration. Journal of Chemical Education, 77(11),
1517–1520.
Shepard, L., Hammerness, K., Darling-Hammond, L., & Rust, F. (2005). Assessment. In L. Darling-Hammond and J. Bransford (Eds.),
Preparing teachers for a changing world: what teachers
should learn and be able to do, San Francisco, CA: Jossey-Bass, pp. 275-326.
Su, K. D. (2015). The development and application for a three-tier diagnostic instrument to assess college students’ misconcep-
tions in chemistry equilibrium. Chinese Journal of Science Education, 23(4), 321-352.
Su, K. D. (2016). Strengthening strategic applications of problem-solving skills for Taiwan students’ chemistry understanding.
Journal of Baltic Science Education, 15 (6), 662-679.
Taber, K. S. (2014). Ethical considerations of chemistry education research involving ‘human subjects.’
Chemistry Education Research and Practice, 15, 109-113.
Treagust, D. F., & Chiu, M. H. (2011). Diagnostic assessment in chemistry. Chemistry Education Research and Practice, 12(2), 119-120.
Wilson, M. Constructing Measures: An Item Response Modeling Approach; Lawrence Erlbaum: Mahwah, NJ, 2005.
Yakmaci-Guzel, B. (2013). Preservice chemistry teachers in action: An evaluation of attempts for changing high school students’
chemistry misconceptions into more scientific conceptions. Chemistry Education Research and Practice, 14, 95-104.
Yezierski, E. J., & Birk, J. P. (2006). Misconceptions about the particulate nature of matter. Using animations to close the gender
gap. Journal of Chemical Education, 83, 954-960.
King-Dow Su Chemistry PhD, Professor, Department of Hospitality Management

and Center for General Education, Hungkuo Delin University of
Technology, NO.1, Lane 380, Ching-Yun Road, Tu-Cheng District.,
New Taipei City, Taiwan 23646, R.O.C. & Center for General Education,
Chung Yuan Christian University, 200 Chung Pei Road, Chung Li
District, Taoyuan City, Taiwan 32023, R.O.C.
E-mail: su-87168@mail. hdut.edu.tw
89
IDENTIFICATION OF THE
INTEREST OF TURKISH MIDDLE-
SCHOOL STUDENTS IN STEM
CAREERS: GENDER AND GRADE
LEVEL DIFFERENCES
Abstract. In Turkey, it is predicted that the

need for STEM employment in the 2016-
Ayşegül Ergün
2023 period will be close to one million and
that about 31% of this need will not be met.
Therefore, the identification of students’
interest in STEM careers in middle-school
is regarded as important. The aim of this Introduction
research was to identify the interest of
middle-school students in STEM careers. Today, the developments in information and communication technolo-
The sample of the research was made up gies which took place with globalization have influenced the economies of
of 892 students who received education in countries just like all areas of our lives. The need for countries which wish
four middle-schools in the Aegean region to have a say in global economy for generations which can develop new
of Turkey. In this research which was of the technologies and have the competence to produce goods of higher value
descriptive survey model, the STEM Career which are also innovative has also increased. In this respect, the primary goal
of countries is to educate qualified individuals who have careers in the areas
Interest Questionnaire was used to collect
of science, technology, engineering and mathematics (STEM) (Ministry of
data. As a result of research, it was deter-
National Education [MoNe], 2016).
mined that the interest of male students in
In the USA, STEM has become a government policy (National Research
STEM careers is more positive compared to Council [NRC], 2010) and the interest in STEM disciplines and STEM education
the female students. It was found that the in many European countries has increased (Corlu, Capraro, & Capraro, 2014).
interest in careers in terms of grade level did However, a decrease has taken place in the number of students who choose
not display any differences in the area of the areas of physical sciences, engineering and mathematics in university
technology, whereas it displayed differenc- education in numerous European countries (European Commission, 2004).
es in the areas of science, engineering and Similarly, it is stated that the students who were placed in STEM areas in
mathematics. In addition, it was identified universities in Turkey from 2000 until 2014 has decreased as well (Akgun-
that the interest in STEM careers displayed duz, 2016). When the rate of those students who have chosen STEM fields
differences in terms of grade level and that in higher education in 2015 in OECD countries is taken a look at, it can be
seen that Turkey with 18% has remained below the OECD average of 27%. It
as the grade level increases, the interest in
has been stated that only 2% of those who have just started higher educa-
STEM careers decreases.
tion have chosen science, mathematics, statistics, while the other 2% have
Keywords: gender differences, grade level
chosen information and communication technologies and that this is the
differences, STEM career interest, STEM second lowest rate among the OECD countries (OECD, 2017). With respect to
education. the paper of the Turkish Industry and Business Association (TUSIAD) (2017),
it is predicted that the need for STEM employment in the 2016-2023 period
will be close to one million and that about 31% of this need will not be met.
Ayşegül Ergün In Turkey, the STEM areas determined by the Assessment, Selection and
Manisa Celal Bayar University, Turkey Placement Center in line with the International Standard Classification of
Education, are positive and natural sciences and engineering are dealt with
as production and construction science educational areas (OSYM, 2017). As
90
ISSN 1648–3898 /Print/ IDENTIFICATION OF THE INTEREST OF TURKISH MIDDLE-SCHOOL STUDENTS IN STEM
ISSN 2538–7138 /Online/ (P. 90-104)
examples to departments within these educational areas, physics, chemistry, biology, mathematics, computer,
architecture and engineering can be given. In the conducted researches, it has been concluded that the number
of students who prefer programs related to STEM fields in higher education institutions and graduate from these
programs are quite lower than expected (Morganson, Jones, & Major, 2010; Shapiro & Sax, 2011). Similarly, it has
been determined that the rate of students who had just enrolled in higher education and placed in STEM fields in
2016-2017 in Turkey has been quite low. This rate is lower than 1% in the area of information and communication
technologies, 3.06% in the fields of natural sciences, mathematics and statistics and 8.66% in the fields of engi-
neering and engineering works. As a result, it has been determined that career rates in STEM fields are in general
quite low (Kizilay, 2018; OSYM, 2017). In the USA, the demands about labor force in STEM fields cannot be met in a
sufficient manner (Knezek, Christensen, & Tyler-Wood, 2011; Moakler & Kim, 2014). In Turkey, it has been observed
that the rate of students who graduated from the STEM areas of universities in 2013-2016 was around 17% and it
is estimated that about 31% of STEM labor force demand in 2016-2023 will not be supplied (TUSIAD, 2017).
The interest in STEM fields brings academic success and career preference in STEM fields as well. Therefore,
the interest of students in STEM fields being higher is extremely efficient in their making career preferences in
these fields. Childhood and juvenescence periods are quite important in the formation and development of career
interest. During primary school, children improve their interest in school and extracurricular activities (Tracey &
Ward, 1998). In juvenescence, typical interest areas for adults start taking shape in parallel with the development
of higher cognitive functions (Tracey & Ward, 1998; Tracey, 2001). It is stated that in particular 13-14 years of age in
juvenescence is a crucial period in the development of profession interest (Gottfredson, 1996).
It is stated that decisions about STEM career goals and wishes are taken during the middle-school years. There-
fore, informing middle-school students about STEM fields and careers will be effective in developing their STEM
career interests and choosing the accurate career (Knight & Cunningham, 2004; Wyss, Heulskamp, & Siebert, 2012).
The research results showed that students need to be guided towards their STEM careers starting with middle-
school years (Christensen & Knezek, 2017; Wyss et al., 2012). In this respect, identifying STEM career interests of
middle-school students was regarded as important.
Literature Review
In recent decades, the number of researches conducted on the relation between students’ STEM career interest
and STEM career pursuits has increased. One of the reasons for not preferring STEM careers is that students do not
get familiar with the career opportunities in STEM fields in early ages and thus, they do not acquire the required
knowledge to be able to have a career in a STEM field (Christensen & Knezek, 2017).
Since the researchers think that high-school years are critical in terms of shaping STEM career interest, more
researches carried out in the high-school level have been seen in the literature (Bahar & Adiguzel, 2016; Korkut-Owen
& Eraslan Capan, 2017; Lichtenberger & George-Jackson, 2013; Owen & Capan, 2018; Sadler, Sonnert, Hazari, & Tai,
2012; Wang, 2013). However, there were also researches in which STEM career interest of middle-school students
has been determined (Karakaya, Avgin, & Yilmaz, 2018; Knezek, Christensen, Tyler-Wood, & Gibson, 2015; Unlu &
Dokme, 2018; Yerdelen, Kahraman, & Tas, 2016).
Relation between Gender and STEM Career Interest
Numerous researches in the literature have dealt with the effect of gender on success and career interest
in STEM areas. In these researches, it was concluded in general that males’ STEM career interest is more positive
compared to females and that they choose STEM careers more. The result of these researches in which the relation
between gender and choosing STEM careers have been determined show that males in general have more of a
tendency to choose these areas compared to females (Catsambis, 1994; Christensen & Knezek, 2017; Knezek et al.,
2011; Sadler et al., 2012; Su, Rounds, & Armstrong, 2009; Unfried, Faber, & Wiebe, 2014). In recent years, different
research results have been seen. For instance, it has been determined in a research that STEM career interest does
not display differences in terms of gender (Yerdelen et al., 2016) or that female students’ STEM career interest is
higher than the male students (Karakaya et al., 2018). In these researches, while the role of gender gap in students’
STEM academic success can be questioned by comparing various data and test scores, it is accepted without doubt
that women are underrepresented in STEM careers (Blickenstaff, 2005). In Turkey, it has been determined that there
were more males in STEM areas in higher education in 2002-2012 and that in particular engineering is seen as a
91
(P. 90-104) ISSN 2538–7138 /Online/
male dominant career. However, it has been stated that during these years, the gap between genders in natural
sciences within STEM areas has been closed in general (Korkut-Owen, Kelecioglu, & Owen, 2014).
It is stated that the attitudes of female students toward science lessons and careers in the field of science is
more negative compared to male students (Cannon & Simpson, 1985; Weinburgh, 1995). In addition, it has been
observed that female students’ STEM areas interest has decreased in a speedier manner compared to male students
during high-school years (Brotman & Moore, 2008; Wells, Sanchez, & Attridge, 2007). Similarly, it has been stated
that while the rate of male students who are interested in a STEM career during high-school years remains stable,
this rate decreases for female students and that the inequality in STEM careers takes place more in engineering than
science in favor of male students (Sadler et al., 2012). In addition, it has been determined that the attitudes of female
students toward engineering and technology is less positive in all educational levels compared to male students
(Unfried et al., 2014). According to the outcomes of another research, it has been identified that the acquisitions of
female students in science, mathematics, technology and engineering fields are higher compared to male students
and that the STEM career interest of male students is higher compared to female students (Knezek et al., 2011).
In the outcomes of the research, it has been stated that STEM career areas which female and male students
are interested in also display differences. For instance, there are researches which show that females are choos-
ing natural sciences more and that male students choose engineering, production and construction areas more
(Korkut-Owen et al., 2014; Sadler et al., 2012). However, it has been determined that female students in general
are interested in biological sciences and male students are more interested in physical sciences (Osborne, Simon,
& Collins, 2003; Sadler et al., 2012). In another research, it has been defined that the interest of secondary school
female students who participate in their universities’ summer camp activities organized under the guidance of
female role models in science and mathematics developed more and that they showed more interest in possible
STEM careers in higher education (Dubetz & Wilson, 2013).
In a research carried out with middle-school students, the impact of female and male students’ STEM career
interest on their tendencies toward mathematics, science and engineering and creative dispositions has been ana-
lyzed. It has been reported that creative tendencies and disposition towards science explains 33% of the variance
in STEM career interest for male students and that disposition toward science and mathematics explains 36% of the
variance in STEM career interest for female students. It has been determined that tendencies towards engineering
have a closer relation with female students’ STEM career interest (Knezek et al., 2015). As an outcome of another
research, it has been stated that the difference between skills concerned to the perception of career in the field of
science is more effective compared to the difference in genders (Knezek et al., 2011).
Relation between Grade Level and STEM Career Interest
In the literature, there are research results which aimed at how STEM career interest differs in different edu-
cational levels and different grade levels (Karakaya et al., 2018; Unfried et al., 2014; Unlu & Dokme, 2018; Wiebe,
Unfried, & Faber, 2018; Yerdelen et al., 2016). In these researches, it has been determined in general that the at-
titudes of students towards STEM careers are not constant during elementary and middle-school and that they
display differences.
The results of a research in which the attitudes of elementary, secondary and high school students towards
STEM fields have been analyzed have displayed that older students have a less positive attitude concerning all
STEM fields compared to younger students (Unfried et al., 2014). With respect to the outcomes of another research
conducted with a similar sample, it has been determined that the students’ STEM career interest decreases with
age and tends to get balanced during their high-school years. The researchers stated that these results might have
emerged due to the fact that students being more mature during their high-school years and understanding their
skills, interest fields and STEM careers better may result in their focusing on certain careers within or outside STEM
fields (Wiebe et al., 2018).
In a research carried out in Turkey with 6th, 7th and 8th grade students, a significant difference was deter-
mined in grade levels in terms of STEM career interests of students (Karakaya et al., 2018). In a research in which the
correlation between STEM profession interest and attitudes of middle-school students from a low socioeconomic
level with grade level was analyzed, it was determined that the students had a positive attitude towards STEM
and that their attitudes did not display differences in terms of grade level. In addition, it has been determined that
STEM profession interest with the exception of physical sciences did not display a significant difference between
grade levels. A positive correlation was found between the students’ STEM profession interest and attitudes. The
92
ISSN 2538–7138 /Online/ (P. 90-104)
researchers stated that middle-school students at a low socioeconomic level in Turkey have limited knowledge
on STEM career choices and that sufficient guidance should be given to them during their middle-school years on
STEM careers (Yerdelen et al., 2016).
It was seen that, the number of researches identifying STEM career interest in middle-school level in the litera-
ture is less compared to high-school level. In addition, in researches in which the effect of gender and grade level on
STEM career interest are analyzed, different results have been obtained. In this respect, it was aimed at contributing
to the related literature with the findings to be obtained in this research which aims at identifying the STEM career
interest of Turkish middle-school students. It is expected that the results to be obtained will guide researchers and
educational policy makers in terms of what needs to be done about developing STEM career interest in early ages.
Problem Statement
Countries take steps towards the integration of STEM in their educational policies with the purpose of meet-
ing the labor force. Educational policy makers give importance to the acquisition of critical thinking, cooperation,
creativeness and communication skills which are required by the labor force of 21st century, besides their attempts
in developing the interest in STEM careers. It is being aimed at making the acquisition of 21st century skills by giv-
ing place to STEM education in primary, middle and high school curricula as well (NRC, 2010; Rynearson, Douglas,
& Diefes-Dux, 2014). In this respect, national policy makers underline developing the interest of students in STEM
context, learning and developing their careers more.
In Turkey, the first step in terms of STEM integration has been taken with the ‘Science and Technology’ lesson
syllabus in 2005. In 2017, ‘Science and Engineering Applications’ unit and ‘Engineering and Design Skills’ have been
included in the engineering component of STEM education of the Physical Sciences Lesson Syllabus (MoNe, 2017).
In the program published in 2018, ‘Science, Engineering and Entrepreneurship Applications’ has been included in all
units (MoNe, 2018). One of the general purposes of physical sciences curricula is to develop career awareness and
entrepreneurship skills related to physical sciences (MoNe, 2017; MoNe, 2018). The interest of students in STEM careers
is an important factor which determines their career preference in the future. The researches have shown that, students’
choosing a STEM career and continuing their university education in the same area depends on their interest in STEM
careers (Astin & Astin, 1992; Maltese & Tai, 2011). It is stated that students’ interest in and objectives about careers in
their middle-school years affects both their academic performance and the career they choose in the future (Tai, Liu,
Maltese, & Fan, 2006). According to the researchers, the interest and expectations of eighth grade students in career
plays a significant role in choosing careers in particular related to science and engineering (Tai et al., 2006). For this
reason, the identification of students’ interest in STEM careers in middle-school was regarded as important in terms
of receiving education in the STEM fields they are interested in during high-school and university and participating
in STEM labor force. In this respect, it was aimed at identifying the interest of middle-school students in STEM careers
in this research. For this aim, answers to the following research questions were sought:
1. At which level is the STEM career interest of middle-school students?
2. Does the STEM career interest of middle-school students display a significant difference in terms of
STEM fields?
gender?
grade level?
Design
In the research, the descriptive survey model was used with the purpose of identifying the STEM career inter-
est of middle-school students. Researches conducted in the descriptive survey model which are widely used in
social sciences aim at displaying the attitudes, views or behaviors of individuals about the studied subject (Creswell,
2008). The research was carried out in the 2016-2017 academic year. In order to speed up the research in terms of
determining the sample group, the convenience sampling method was used. The data of the research was collected
through the STEM Career Interest Questionnaire (Kier, Blanchard, Osborne, & Albert, 2014).
93
(P. 90-104) ISSN 2538–7138 /Online/
Sample
The sample of the research was made up of 892 students who received education in the 2016-2017 academic
year in four middle-schools in a rural district of a city in the Aegean region of Turkey, which has a medium level of
socioeconomic structure. In the determination of the sample, the case sampling method among the purposeful
sampling types which can be easily accessed was used. In this sampling method, since the researcher chooses a
close and easily accessible case, this speeds up the research and makes it practical (Yildirim & Simsek, 2016). The
scale prepared by Yazicioglu and Erdogan (2004, p.50) was used to determine the size of the sample. Since there
are about 5000 students in the population, the sample size required for a 97% confidence level on the chart of 5000
people is 880 people. Therefore, it was aimed at reaching at least 880 students and 892 volunteering students were
reached. The distribution of demographic data related to the research sample is presented in Table 1.
Table 1. Demographic characteristics of the sample.
Grade Level Female Male Total Percentages
5 100 110 210 23.50

6 118 94 212 23.80
7 102 86 188 21.10
8 166 116 282 31.60
Total 486 406 892 100.0
As it can be seen in Table 1, 54.48% of the sample consisted of female students and 45.52% of the sample
consists of male students. 23.50% of the sample consisted of 5th graders, 23.80% consisted of 6th graders, 21.10%
consisted of 7th graders and 31.60% consisted of 8th graders.
Data Collection Tool: STEM Career Interest Questionnaire (STEM-CIQ)
In the research, the questionnaire developed by Kier et al. (2014) and adapted in Turkish by Unlu et al. (2016)
was used to identify STEM career interest of middle-school students. The questionnaire adapted to Turkish was
of a “5-point Likert scale” (strongly disagree to strongly agree) and consisted of four sub-dimensions as science,
technology, engineering and mathematics and a total of 40 questions with 10 questions in each sub-dimension.
The questionnaire’s total Cronbach’s alpha value was stated as .93 and the value of the sub-dimensions was stated
successively as .86, .88, .94 and .90 (Unlu et al., 2016). In this research, questionnaire was given to 368 8th grade
students and its validity and reliability researches were completed. According to the fit indexes obtained as a result
of the confirmatory factor analysis done for construct validity (χ2/df = 3.02, RMSEA= .06, SRMR= .07, CFI= .89, NNFI=
.96), it was determined that the conformity of the structure of the model with data was good. As a result of the
reliability analysis, the questionnaire’s general Cronbach’s alpha value was determined as .88 and the values of the
sub-dimensions were determined successively as .82, .80, .83 and .86. According to Buyukozturk (2017), since the
Cronbach’s alpha value being .70 and over means that it is acceptable for the reliability of the measurement tool,
it can be stated that the questionnaire is reliable.
Data Analysis
In order to determine whether the data obtained in the research displayed normal distribution or not, the
skewness and kurtosis values of data obtained from the questionnaire and the sub-dimensions and the results of
the analysis are presented in Table 2.
94
ISSN 2538–7138 /Online/ (P. 90-104)
Table 2. Skewness and Kurtosis values related to the questionnaire and the sub-dimensions.
Sub-dimensions and the

Skewness Kurtosis
STEM-CIQ
Science -0.597 0.068

Technology -0.790 1.054
Engineering -0.404 0.042
Mathematics -1.097 1.040
STEM-CIQ -0.360 0.082
The skewness and kurtosis values being between +1.5 and -1.5 expressed that the data displays normal dis-
tribution (Tabachnick & Fidell, 2007). When the values in Table 2 were analyzed, it was determined that the data
obtained from the questionnaire and the sub-dimensions displayed normal distribution. Therefore, in the analysis
of data, the independent sample t test and One Way ANOVA among the parametric tests were used. In order to
determine the effect of gender on STEM career interest, Cohen’s d value which shows the extent of the effect was
calculated. The size of the effect was evaluated as independent from the d sign and can have any value; the 0.2, 0.5
and 0.8 values which d may have are evaluated successively as small, medium and great effects (Green & Salkind,
2005; p.169). In order to determine the effect of grade level on STEM career interest, the eta square (η2) value which
shows the effect dimensions was calculated. The 0.01 value eta square may have is interpreted as small, 0.06 value
as medium and 0.14 value as great effect size (Green & Salkind, 2005, p.157).
Since the score value which belongs to each of the questions in the questionnaire is between 1.00 and 5.00,
as the obtained scores get closer to 5.00, level of STEM career interest was accepted as high and as it gets closer
to 1.00, it was accepted as low. The average score obtained by dividing the number of marked questions with the
sum of the question scores of the marked questions can give an idea to the researchers about the STEM career
interest of students (Turgut & Baykul, 1992). Therefore, scores below 2,60 received from the STEM career interest
questionnaire point out to negative interest and scores which are 2.60 and over point out to positive interest. The
score intervals about the questionnaire questions are presented in Table 3.
Table 3. The score intervals related to the questionnaire questions.
Likert -Type Scale Response Point Point Range
Strongly Agree 5.00 4.20-5.00

Agree 4.00 3.40-4.19
Neither Agree nor Disagree 3.00 2.60-3.39
Disagree 2.00 1.80-2.59
Strongly Disagree 1.00 1.00-1.79
Results of Research
Results Related to the First Research Question
In order to determine STEM career interest levels of middle-school students, descriptive analysis was done.
The lowest and highest score values the students received from the questionnaire and the sub-dimensions and
the mean (M) and standard deviation (SD) of the scores are presented in Table 4.
95
(P. 90-104) ISSN 2538–7138 /Online/
Table 4. Results of descriptive analysis of the middle-school students’ STEM career interest.
Sub-dimensions and the

N Min Max M SD
STEM-CIQ
Science 892 2.20 5.00 4.06 .564

Technology 892 1.00 5.00 3.97 .619
Engineering 892 1.00 5.00 3.54 .763
Mathematics 892 1.00 5.00 4.11 .695
STEM-CIQ 892 2.33 5.00 3.93 .502
According to the findings in Table 4, it can be stated that average scores of the students’ related to the ques-
tionnaire and sub-dimensions are over (M=2.60) value and thus, their STEM career interest was positive. It was seen
that their career interest in terms of the areas were mathematics, science, technology and engineering from the
highest to the lowest. The students’ career interest levels in terms of STEM fields are shown in Figure 1 in graphics.
Figure 1. STEM fields career interest levels.
Results Related to the Second Research Question
With the purpose of determining whether STEM career interests of middle-school students display significant
differences or not in terms of fields, the One Way ANOVA test was done. The results of the analysis are presented
in Table 5.
Table 5. Results of the One way ANOVA test related to STEM fields career interests.
Eta square
Sources of Variation SS df M F p
(η2)
Between Groups 179.734 3 59.911 135.378 .0001** .10

.443
Within Groups 1577.246 3564
Total 1756.980 3567
Note. **Level p-values of < .01.
According to the analysis results in Table 5, the middle-school students’ STEM career interests displayed statisti-
cally significant differences in terms of STEM fields [F(3-3564)=135.378; p<.01]. The test result showed that the effect of
the calculated effect size (η2 = .10) of the fields on this difference is of medium scale. In order to determine which
STEM fields this difference in question is related to, firstly the equality of the variances was checked. The Levene
test result was determined that the variances were equal (p>.05). Therefore, the Post Hoc Tukey test was done and
the results of the analysis are presented in Table 6.
96
ISSN 2538–7138 /Online/ (P. 90-104)
Table 6. Results of the Post Hoc Tukey test related to the difference between the STEM fields career interest
scores.
Field Field p
Technology .008**
Science Engineering .0001**
Mathematics .396
Science .008**
Technology Engineering .0001**
Mathematics .0001**
Science .0001**
Engineering Technology .0001**
Mathematics .0001**
According to Table 6, there was no significant difference between the science career interest of middle-school
students and their mathematics career interest. On the other hand, a significant difference was found between
the interest levels of science and technology and engineering fields in favor of the field of science. A significant
difference was found between mathematics field interest and technology and engineering fields interest levels
in favor of mathematics field. In addition, a statistically significant difference was found between technology and
engineering fields interest levels in favor of the technology field.
Results Related to the Third Research Question
The results of the independent sample t test analysis done with the purpose of determining whether the
middle-school students’ STEM career interests displayed a significant difference in terms of gender or not are
presented in Table 7.
Table 7. Results of the independent sample t test analysis of STEM career interest in terms of gender.
Sub-dimensions
and the Gender N M SD df t p Cohen’s d
STEM-CIQ
Female 486 4.12 0.55

Science 890 3.03 .002** 0.20
Male 406 4.00 0.56
Female 486 3.89 0.65
Technology 890 -3.96 .0001** 0.27
Male 406 4.06 0.57
Female 486 3.39 0.74
Engineering 890 -6.86 .0001** 0.46
Male 406 3.73 0.75
Female 486 4.16 0.62
Mathematics 890 2.29 .022* 0.15
Male 406 4.05 0.77
Female 486 3.89 0.49
STEM-CIQ 890 -2.12 .034* 0.14
Male 406 3.96 0.52
Note. *Level p-values of < .05. **Level p-values of < .01.
According to Table 7, the students’ career interest displays a statistically significant difference in the fields
of science [t(890)=3.03; p<.01], technology [t(890)=-3.96; p<.01], engineering [t(890)=-6.86; p<.01] and mathematics
[t(890)=2.29; p<.05]. In addition, it was seen that STEM career interest displayed a statistically significant difference
in terms of gender as well [t(890)=-2.12; p<.05]. When the Cohen’s d effect size (d) values were analyzed, it was found
97
(P. 90-104) ISSN 2538–7138 /Online/
that the gender effect on engineering career interest is of medium scale and its effect on career interest in other
sub-dimensions and the questionnaire in general is of small scale. The average career interest scores of the students’
related to the questionnaire and the sub-dimensions are shown in Figure 2 in graphics.
Figure 2. Average career interest scores of female and male students related to the questionnaire and the
sub-dimensions.
When the average interest scores in Figure 2 are analyzed, it can be seen that the difference is in favor of female
students in science and mathematics fields and in favor of male students in technology and engineering fields.
According to the career interest, average scores obtained from the whole questionnaire, STEM career interest of
the male students was found to be higher compared to the female students.
Results Related to the Fourth Research Question
Firstly, a descriptive analysis was done to determine whether the middle-school students’ STEM career interests
displayed differences in terms of grade levels. According to the grade levels, the frequency, mean (M) and standard
deviation (SD) values related to STEM CIQ and sub-dimensions are presented in Table 8.
Table 8. Results of the descriptive analysis results related to STEM CIQ and sub-dimensions in terms of grade
level.
Sub-dimensions
and the Grade Level N M SD
STEM-CIQ
Science 5th grade 210 4.13 .47

6th grade 212 4.22 .51
7th grade 188 4.07 .52
8th grade 282 3.89 .63
Technology 5th grade 210 4.02 .58
6th grade 212 4.02 .69
7th grade 188 3.91 .56
8th grade 282 3.94 .61
98
ISSN 2538–7138 /Online/ (P. 90-104)
Sub-dimensions
and the Grade Level N M SD
STEM-CIQ
Engineering 5th grade 210 3.74 .61

6th grade 212 3.55 .72
7th grade 188 3.45 .88
8th grade 282 3.45 .77
Mathematics 5th grade 210 4.41 .57
6th grade 212 4.29 .56
7th grade 188 4.03 .61
8th grade 282 3.82 .78
STEM-CIQ 5th grade 210 4.07 .45
6th grade 212 4.02 .48
7th grade 188 3.86 .47
8th grade 282 3.78 .52
The results of the One Way ANOVA test analysis done with the purpose of determining whether there is a
statistically significant difference between the average values obtained in Table 8 is presented in Table 9.
Table 9. Results of the One Way ANOVA test related to STEM CIQ and sub-dimensions in terms of grade level.
Sub-dimensions Eta
Sources of p
and the SS df M F square Tukey
Variation
STEM-CIQ (η2)
Science Between Groups 14.576 3 4.859 16.025 .0001** .05 5>8

Within Groups 269.242 888 .303 6>7 and 8
7>8
Total 283.818 891
Technology Between Groups 2.010 3 .670 1.750 .155 .01 No significant
Within Groups 340.017 888 .383 difference
Total 342.027 891

Engineering Between Groups 12.212 3 4.071 7.119 .0001** .02 5>6-7 and 8
Within Groups 507.808 888 .572
Total 520.020 891

Mathematics Between Groups 51.309 3 17.103 39.960 .0001** .12 5>7 and 8
7>8
Total 431.381 891
STEM-CIQ Between Groups 13.627 3 4.542 19.067 .0001** .06 5>7 and 8
Total 225.174 891

According to Table 9, a significant difference was not found in technology career interest in terms of grade
level [F(3-888)=1.750; p>.05]. Fields in which career interests displayed differences in terms of grade level were de-
termined as science [F(3-888)=16.025; p<.01], engineering [F(3-888)=7.119; p<.01], mathematics [F(3-888)=39.960; p<.01]
and STEM [F(3-888)=19.067; p<.01]. The difference in the average interest scores obtained from STEM CIQ and the
sub-dimensions in terms of grade levels are shown in Figure 3 in graphics.
99
(P. 90-104) ISSN 2538–7138 /Online/
Figure 3. Difference in the average interest scores obtained from STEM CIQ and the sub-dimensions in terms
of grade levels.
According to the graphics in Figure 3 and the results of the Tukey test in Table 9, the science career interest
of fifth and seventh grades was higher compared to eighth grades, whereas science career interest of sixth grades
was higher compared to seventh and eighth grades. A significant difference was not found between science career
interests of fifth and sixth grades. According to the results of the analysis related to the career of engineering, it was
seen that the fifth grades have the highest interest level and that there is a significant difference between the interest
levels of fifth grades and sixth, seventh and eighth grades. While engineering interest levels of seventh and eighth
grades is the same, a significant difference was not found between the interest levels of sixth, seventh and eighth
grades. According to the results of mathematics career interest, while a significant difference was not found between
fifth and sixth grades in terms of interest level, the interest level of fifth grades was found to be higher compared
to seventh and eighth grades. Similarly, it was seen that the mathematics interest level of sixth grades is higher
compared to seventh and eighth grades. It was determined that the lowest mathematics career interest belonged
to eighth grades. When the average scores in Figure 3 are analyzed, it can be seen that as the grade level increases,
mathematics career interest decreases.
According to Figure 3, STEM career interest level in general decreased as grade level increased. The interest levels
of fifth and sixth grades were found to be higher compared to seventh and eighth grades. According to the eta square
(η2) values in Table 9, while the effect of grade level on mathematics and STEM career interest is of medium scale, its
effect on science, technology and engineering career interest is of small scale. Therefore, it can be stated that grade
level affects mathematics and STEM career interest more.
Discussion
In this research which aimed at identifying the STEM career interest of middle-school students, the answers to
four questions were sought. The first of these was at which level the students’ interest in STEM careers was. As a result
of the research, it can be stated that the students’ STEM career interest is positive according to the career interest
average scores related to each of the STEM fields and all of the fields. The interest levels of the fields were identified
as mathematics, science, technology and engineering from the highest to the lowest. When it is considered that
the students do not receive any education related to engineering in their middle-school years and that they are not
familiar to careers in this field, the finding that the lowest interest being in the field of engineering is expected. In
100
ISSN 2538–7138 /Online/ (P. 90-104)
the results of other researches, it was determined that the knowledge and career wishes of middle-school students
about engineering were lower compared to mathematics and science and that a majority of the students did not
wish to be engineers similar to the finding of this research (Gulhan & Sahin, 2018; Karakaya et al., 2018; Mooney &
Laubach, 2002; Spencer, 2011). In the other researches, a few possible reasons were stated as to why the students
did not choose engineering as a career (Besterfield-Sacre, Moreno, Shuman, & Atman, 2001). These are that, a major-
ity of the students had negative stereotypes about engineers (such as nerds become engineers) or that they overly
exaggerated the positive stereotypes about engineers (such as, engineers must be geniuses). It is considered that
the students not knowing what engineers do is another possible reason why they do not wish to have a profession
in the area of engineering.
The second question to which answers have been sought in the research was whether the STEM career inter-
ests of middle-school students displayed a significant difference in terms of STEM fields. As a result of the analyses,
it was found that the difference between the fields is statistically significant and that the effect of these fields on
this difference is of medium scale. As a result of a research, it was identified that the knowledge and career wish of
middle-school students about engineering is lower compared to mathematics and science (Mooney & Laubach, 2002).
Similarly, it was found in this research that there is no significant difference between the science and mathematics
career interests of the students and that their science and mathematics career interests are more positive compared
to technology and engineering career interests as well. This result can be clarified with the fact that although sci-
ence and mathematics lessons are given weight to in the curriculum in Turkey starting with kindergarten, there are
not sufficient lessons on technology and engineering fields. It can be stated that the Increasing Opportunities and
Improving Technology Movement (FATIH in Turkish) Project initiated in 2010 in Turkey plays a role in the findings
obtained in the research about technology career interest being more positive compared to engineering. The aim of
this project is to create individuals who have 21st century skills and a society centered on production (MoNe, 2010).
Within the scope of the FATIH Project, interactive boards, wide band Internet connections have been provided to
schools and students and teachers have been given tablet computers. It is stated that, teachers and students obtained
information technology tools which contributed greatly to the environment which can be used for STEM education
with this project (MoNe, 2016).
The third question to which answers were sought was whether STEM career interests of middle-school students
changed in terms of gender or not. As a result of the analyses, it was determined that the students’ career interests
about STEM as a whole and about its all fields displayed a difference in terms of gender. It was seen that the effect of
gender on the field of engineering is of medium scale and small scale on other STEM fields and STEM careers. It was
seen that this difference in career interest is in favor of female students in the fields of science and mathematics and
in favor of male students in the fields of technology and engineering. It was concluded that STEM career interest of
male students is more positive compared to female students. This finding is in line with the results of many researches
in the literature (Catsambis, 1994; Christensen & Knezek, 2017; Knezek et al., 2011; Sadler et al., 2012; Su et al., 2009;
Unfried et al., 2014). It was stated that the attitude of female students towards engineering and technology is less
positive compared to male students in all of the educational levels (Knezek et al., 2011; Unfried et al., 2014). In this
research, the engineering and technology career interests of female students were found to be lower compared to
the male students as well. This finding also explained why STEM career interest of female students is lower compared
to male students. Similarly, the researchers stated that the tendencies in terms of engineering are more closely related
to STEM career interests of female students (Knezek et al., 2015). It was stated that female students prefer traditional
careers such as nursing and psychology more and fields such as mathematics, computer sciences and engineering
less (NSF, 2013). In other researches, it was stated that male students are more inclined to the field of engineering
as well (Korkut-Owen et al., 2014; Korkut-Owen & Mutlu, 2016). One of the reasons why female students’ interest in
engineering is lower compared to male students might be not having sufficient role models who have a career in
STEM fields (McCrea, 2010). In the results of the research, it was stated that individuals who have family members
with STEM careers have more of a chance of choosing STEM branches (Moakler & Kim, 2014; Oware, Capobianco, &
Diefes-Dux, 2007). The other reasons as to why female students’ engineering career interest is lower compared to
male students might be that they regard engineering as ‘difficult,’‘boring,’‘one-dimensional’ and ‘male work’ and think
that engineers only deal with construction work (Gulhan & Sahin, 2018).
The fourth question of the research to which answers were sought was whether STEM career interest of middle-
school students displayed differences in terms of grade level. According to the analyses, a significant difference was
not found in technology career interest in terms of grade level. It is considered that, students being able to access
information technology tools in all classes with the FATIH Project and much electronic content being presented to
101
(P. 90-104) ISSN 2538–7138 /Online/
students during lessons do not change their interest in technology. Fields which displayed differences in career inter-
est in terms of grade level were determined as science, engineering, mathematics and STEM. In the research, it was
seen that while grade level affected mathematics and STEM career interest in a medium scale, it affected science,
technology and engineering career interest in a small scale. It was seen that STEM career interest in general decreased
as grade level increased. It was stated in the results of other researches that older students had a less positive attitude
towards STEM careers compared to younger students (Unfried et al., 2014), the interest of students towards STEM
careers decreased with age and that this interest started to get balanced in their high-school years as well (Wiebe et
al., 2018). In a research in the literature, it was stated that STEM career interest did not change in terms of grade level
as different from the finding of this research (Yerdelen et al., 2016). The reason why STEM career interest decreases as
grade level increases might be the insufficient guidance provided to students about STEM occupations.
Conclusions
Within the scope of this research in which the STEM career interest of middle-school students was identified, it
was found that the STEM career interest of the male students is more positive compared to the female students. In
addition, it was determined that career interest in the areas of engineering and technology displayed differences in the
favor of the male students and career interest in the areas of physical sciences and mathematics displayed differences
in the favor of the female students. It is considered that STEM education can be effective in the development of the
career interest of the female students in the areas of engineering and technology in a positive manner. Through the
STEM education applications to be carried out at schools and environments other than schools, the students need to
be taught the connection of engineering with physical sciences, mathematics and technology. It is considered that
introducing students with STEM careers at early ages and giving them STEM education to develop their STEM career
interest can be an effective approach. Within the scope of the research, it was concluded that the STEM career interest
of middle-school students displayed differences in terms of grade level and that STEM career interest decreased as
grade level increased. In all grade levels, the STEM career interest of students can be developed in a positive manner
by giving them sufficient occupational guidance about their STEM careers. In line with this purpose, students can
be introduced with role models who have careers in STEM areas by organizing seminars or video conferences with
STEM experts.
Limitations of the Research and Recommendations
This research is limited with 892 middle-school students living in a district of medium level socioeconomic
structure in Turkey. Researchers can conduct researches in different countries, different regions and different grade
levels to identify or compare STEM career interest. The effect of variables other than gender and grade level on STEM
career interest can be identified. In this research, suggestions have been made on the reasons for differences in STEM
career interest based on the results of previous studies. Qualitative researches which can support quantitative results
related to the subject can be conducted and detailed results might be obtained.
References
Akgunduz, D. (2016). A research about the placement of the top thousand students in STEM fields in Turkey between 2000 and
2014. Eurasia Journal of Mathematics, Science & Technology Education, 12 (5), 1365-1377.
Astin, A. W., & Astin, H. S. (1992). Undergraduate science education: The impact of different college environments on the educational
pipeline in the sciences. Final Report. Los Angeles: Higher Education Research Institute.
Ayar, M. C. (2015). First-hand experience with engineering design and career interest in engineering: An informal STEM educa-
tion case study. Educational Sciences: Theory & Practice, 15 (6), 1655-1675.
Bahar, A., & Adiguzel, T. (2016). Analysis of factors influencing interest in STEM career: Comparison between high ability and
motivated American and Turkish high school students. Journal of STEM Education, 17 (3), 64-69.
Besterfield-Sacre, M., Moreno, M., Shuman, L. J., & Atman, C. J. (2001). Gender and ethnicity differences in freshmen engineering
student attitudes: A cross-institutional study. Journal of Engineering Education, 90 (4), 477-489.
Bishop, A. E. (2015). Career aspirations of high school males and females in a science, technology, engineering, and mathematics
program. Doctoral dissertation. University of Maryland.
Blickenstaff, J. C. (2005). Women and science careers: Leaky pipeline or gender filter? Gender and Education, 17 (4), 369-386.
Brotman, J. S., & Moore, F. M. (2008). Girls and science: A review of four themes in the science education literature. Journal of
Research in Science Teaching, 45 (9), 971-1002. doi:10.1002/tea.20241.
Buyukozturk, S. (2017). Data analysis handbook for social studies: Statistics, research pattern SPSS applications and interpretation
(23th Ed.). Ankara, Turkey: Pegem Academy.
102
ISSN 2538–7138 /Online/ (P. 90-104)
Cannon, R. K., & Simpson, R. D. (1985). Relationships among attitude, motivation, and achievement of ability grouped, seventh-
grade, life science students. Science Education, 69 (2), 121-138.
Catsambis, S. (1994). The path to math: Gender and racial-ethnic differences in mathematics participation from middle school
to high school. Sociology of Education, 67, 199-215.
Christensen, R. & Knezek, G. (2017). Relationship of middle school student STEM interest to career intent. Journal of Education in
Science, Environment and Health (JESEH), 3 (1), 1-13.
Corlu, M. S., Capraro, R. M., & Capraro, M. M. (2014). Introducing STEM education: Implications for educating our teachers in the
age of innovation. Education and Science, 39 (171), 74–85.
Creswell, J. W. (2008). Educational research planning, conducting and evaluating quantitative and qualitative research. Upper Sad-
dle River, NJ: Pearson/Merrill Prentice Hall.
Dubetz, T., & Wilson, J. A. (2013). Girls in engineering, mathematics and science, GEMS: A science outreach program for middle-
school female students. Journal of STEM Education, 14 (3), 41-47.
European Commission. (2004). Europe needs more scientists: Report by the high level group on increasing human resources for sci-
ence and technology. Brussels. European Commission.
Gottfredson, L. S. (1996). Gottfredson‘s theory of circumscription and compromise. In D. Brown & L. Brooks (Eds.), Career choice
and development (3rd ed., pp. 179-232), San Francisco: Jossey-Bass.
Green, S. B., & Salkind, N. J. (2005). Using SPSS for Windows and Macintosh: Analyzing and understanding data (4th Edition), New
Jersey: Pearson.
Gulhan, F. & Sahin, F. (2018). Why STEM education? Investigation of middle school 5th grade students’ career choices in STEM
fields. Journal of STEAM Education, 1 (1), 1-23.
Karakaya, F., Avgin, S. S., & Yilmaz, M. (2018). Middle school students’ interest in science technology engineering and mathematics
(STEM) professions. Ihlara Journal of Educational Research, 3 (1), 36-53.
Kier, M. W., Blanchard, M. R., Osborne, J. W., & Albert, J. L. (2014). The development of the STEM career interest survey (STEM-CIS).
Research in Science Education, 44 (3), 461-481.
Kizilay, E. (2018). Career and employment in STEM fields in Turkey. Journal of International Social Research, 11 (56), 570-574.
Knezek, G., Christensen, R., & Tyler-Wood, T. (2011). Contrasting perceptions of STEM content and careers. Contemporary Issues
in Technology and Teacher Education, 11 (1), 92-117.
Knezek, G., Christensen, R., Tyler-Wood, T., & Gibson, D. (2015). Gender differences in conceptualizations of STEM career interest:
Complementary perspectives from data mining, multivariate data analysis and multidimensional scaling. Journal of STEM
Education: Innovations & Research, 16 (4), 13-19.
Knight, M. & Cunningham, C. (2004). Draw an Engineer Test (DAET): Development of a tool to investigate students’ ideas about
engineers and engineering. Paper presented at the annual meeting of the American Society for Engineering Education, Salt
Lake City, Utah.
Korkut-Owen, F., Kelecioglu, H., & Owen, D. W. (2014). A decade of change gender trends in university enrollment: Implications
for career counseling. International Journal of Human Sciences, 11 (1), 794-813.
Korkut-Owen, F., & Mutlu, T. (2016). Gender differences on selecting STEM areas in Turkey. Journal of Education for Life, 30 (2), 53-72.
Korkut-Owen, F., & Eraslan Capan, B. (2017). Reasons for science, technology, engineering and mathematics selection among
high school students. Journal of Education for Life, 31 (2), 23-40.
Lichtenberger, E., & George-Jackson, C. (2013). Predicting high school students’ interest in majoring in a STEM field: Insight into
high school students’ postsecondary plans. Journal of Career and Technical Education, 28 (1), 19-38.
Maltese, A. V., & Tai, R. H. (2011). Pipeline persistence: Examining the association of educational experiences with earned degrees
in STEM among US students. Science Education, 95 (5), 877-907.
McCrea, B. (2010). Engaging girls in STEM. THE Journal. Retrieved from https://thejournal.com/articles/2010/09/08/engaging-
girls-in-stem.aspx.
Ministry of National Education (MoNe) (2010). Retrieved from http://fatihprojesi.meb.gov.tr/.
Ministry of National Education (MoNe) (2016). STEM education report, Ministry of National Education General Directorate of
Innovation and Education Technologies. Ankara. Retrieved from http://yegitek.meb.gov.tr/STEM_Egitimi_Raporu.pdf.
Ministry of National Education (MoNe) (2017). Science curriculum (primary and middle schools 3, 4, 5, 6, 7 and 8). Ankara. Retrieved
from http://mufredat.meb.gov.tr.
Ministry of National Education (MoNe) (2018). Science curriculum (primary and middle schools 3, 4, 5, 6, 7 and 8). Ankara. Retrieved
from http://mufredat.meb.gov.tr/ProgramDetay.aspx?PID=325.
Moakler, M. & Kim, M. M. (2014). College major choice in STEM: Revisiting confidence and demographic factors. Career Develop-
ment Quarterly, 62, 128-143.
Mooney, M. A. & Laubach, T. A. (2002). Adventure engineering: A design centered, inquiry based approach to middle grade sci-
ence and mathematics education. Journal of Engineering Education, 91 (3), 309-318.
Morganson, V., Jones, M., & Major, D. (2010). Understanding women’s underrepresentation in science, technology, engineering,
and mathematics: The role of social coping. The Career Development Quarterly, 59, 169-179.
National Research Council. (2010). Exploring the intersection of science education and 21st century skills: A workshop summary.
Washington, DC: National Academies Press.
National Science Foundation, National Center for Science and Engineering Statistics (2013). Women, minorities, and persons
with disabilities in science and engineering: 2013. Special Report NSF 13-304. Arlington, VA. Retrieved from http://www.
nsf.gov/statistics/wmpd/.
OECD, Organization for Economic Co-operation and Development. (2017). Turkey - country notes - training in 2017 at a glance:
OECD indicators. Retrieved from http://www.oecd.org/education/skills-beyond-school/EAG2017CN-Turkey-Turkish.pdf.
103
(P. 90-104) ISSN 2538–7138 /Online/
Osborne, J., Simon, S., & Collins, S. (2003). Attitudes towards science: A review of the literature and its implications. International
Journal of Science Education, 25 (9), 1049-1079.
Evaluation, Selection and Placement Center (OSYM in Turkish) (2017). Number of students at undergraduate level according to
the classification of areas of education and training. Retrieved from https://istatistik.yok.gov.tr/.
Oware, E., Capobianco, B., & Diefes-Dux, H. (2007, June) Gifted students perceptions of engineers? A study of students in a summer
outreach program. Paper presented at the 2007 Annual American Society for Engineering Education Conference & Exposition,
Honolulu, Hawaii. Retrieved from https://peer.asee.org/2656.
Owen, F. K., & Capan, B. E. (2018). Planning of the selection of science, technology, engineering and mathematics fields: beliefs
about career choice. Hacettepe University Journal of Education, 33 (4), 915-933. doi: 10.16986/HUJE.2017032884.
Rynearson, A. M., Douglas, K. A., & Diefes-Dux, H. A. (2014, June). Engineering teaches problem solving: Teachers’ perceptions
of student learning through engineering lessons. Paper Presented at the American Society for Engineering Education Annual
Conference and Exposition, Indianapolis, Indiana. Retrieved from https://peer.asee.org/19898.
Sadler, P. M., Sonnert, G., Hazari, Z., & Tai, R. (2012). Stability and volatility of STEM career interest in high school: A gender study.
Science Education, 96 (3), 411-427. doi: 10.1002/sce.21007.
Shapiro, C. A., & Sax, L. J. (2011). Major selection and persistence for women in STEM. New Directions for Institutional Research,
(152), 5-18. doi:10.1002/ir.v2011.152.
Spencer, M. E. (2011). Engineering perspectives of grade 7 students in Canada. Master thesis. Queen’s University Kingston, Ontario,
Canada.
Su, R., Rounds, J., & Armstrong, P. I. (2009). Men and things, women and people: A meta-analysis of sex differences in interests.
Psychological Bulletin, 135 (6), 859-884.
Tabachnick, B. G. & Fidell, L. S. (2007). Using multivariate statistics, 5th ed. Needham Height, MA: Allyn & Bacon.
Tai, R. H., Liu, C. Q., Maltese, A. V., & Fan, X. (2006). Planning early for careers in science. Science, 312, 1143-1144.
Tracey, T. J. G., & Ward, C. C. (1998). The structure of children’s interests and competence perceptions. Journal of Counseling Psy-
chology, 45 (3), 290. doi: 10.1037/00220167.45.3.290.
Tracey, T. J. G. (2001). The development of structure of interests in children: Setting the stage. Journal of Vocational Behavior, 59,
1-16. doi: 10.1006/jvbe.2000.1787.
Turgut, F., & Baykul, Y. (1992). Scaling techniques. Ankara, Turkey: OSYM Publications.
Turkish Industry and Business Association’s (TUSIAD). (2017). STEM requirement in Turkey towards 2023. İstanbul: TUSIAD pub-
lication. Retrieved from https://www.tusiadstem.org/images/raporlar/2017/STEM-Raporu-V7.pdf.
Unfried, A., Faber, M., & Wiebe, E. (2014). Gender and student attitudes toward STEM. Presented at the AERA Annual Meeting, Phila-
delphia, PA. Retrieved from https://eval.fi.ncsu.edu/wp-content/uploads/2016/03/AERA-2014-Unfried-Faber-Wiebe.pdf.
Unlu, Z. K., Dokme, I., & Unlu, V. (2016). Adaptation of the science, technology, engineering, and mathematics career interest
survey (STEM-CIS) into Turkish. Eurasian Journal of Educational Research, 63, 21-36. doi: 10.14689/ejer.2016.63.2.
Unlu, Z. K., & Dokme, I. (2018). Multivariate assessment of middle school students’ interest in STEM career: A profile from Tur-
key. Research in Science Education, Online first, 1-15. doi: 10.1007/s11165-018-9729-4.
Wang, X. (2013). Why students choose STEM majors: Motivation, high school learning, and postsecondary context of sup-
port. American Educational Research Journal, 50 (5), 1081-1121.
Weinburgh, M. (1995). Gender differences in student attitudes toward science: A meta-analysis of the literature from 1970 to
1991. Journal of Research in Science Teaching, 32 (4), 387-398.
Wells, B., Sanchez, H. A., & Attridge, J. M. (2007). Modeling student interest in science, technology, engineering and mathematics.
2007 IEEE Meeting the Growing Demand for Engineers and Their Educators 2010-2020 International Summit, 50 Papers, 1-17.
doi: 10.1109/MGDETE.2007.4760362.
Wiebe, E., Unfried, A., & Faber, M. (2018). The relationship of STEM attitudes and career interest. EURASIA Journal of Mathematics,
Science and Technology Education, 14 (10), 2-17. doi: 10.29333/ejmste/92286.
Wyss, V. L., Heulskamp, D., & Siebert, C. J. (2012). Increasing middle school student interest in STEM careers with videos of scien-
tists. International Journal of Environmental Science Education, 7 (4), 501-522.
Yazicioglu, Y., & Erdogan, S. (2004). SPSS applied scientific research methods. Ankara, Turkey: Detay Publishing.
Yerdelen, S., Kahraman, N., & Tas, Y. (2016). Low socioeconomic status students’ STEM career interest in relation to gender, grade
level, and STEM attitude. Journal of Turkish Science Education. 13 (Special Issue), 59-74.
Yildirim, A., & Simsek, H. (2016). Qualitative research methods in social studies. Ankara, Turkey: Seckin Publishing.
Received: December 11, 2018 Accepted: January 18, 2019
Ayşegül Ergün PhD, Assistant Professor, Faculty of Education, Department of

Mathematics and Science Education, Manisa Celal Bayar University, 45900,
Demirci, Manisa, Turkey.
E- mail: ergunaysegul@gmail.com
104
TEACHERS’ SELF-CONCEPT AND
ITS BENEFITS FOR SCIENCE
EDUCATION ISSN 1648-3898 /Print/
Abstract. Self-concept, understood as an

Jelena Ž. Maksimović,
impression or idea one has about oneself,
Jelena S. Osmanović
represents a significant precondition for
the professional improvement of teachers,
as well as for the advancement of teaching
itself and students’ development of their
own positive self-concept. This research
Introduction
aimed to examine the idea of self-concept
Highly qualified teachers are central to quality education. A teacher’s with the purpose of determining the man-
quality is estimated on the basis of their teaching and conduct, both of which ner in which primary and secondary school
largely depend on the teacher’s ability to adapt to current circumstances. teachers perceive themselves. The factors
Self-awareness and understanding of oneself are crucial to the success of the to be analyzed in relation to the variables,
process of teachers’ quality improvement, which means that they represent work experience and pay satisfaction,
significant factors that constitute the profession of teaching itself. In other
were defined by means of the scale of
words, teachers who understand and know their own character are better
at adapting themselves to various situations and are more self-confident estimates and factor analysis. The obtained
in teaching than those who lack these abilities. This further contributes to results showed that accepting oneself,
teachers’ job satisfaction (Shastri, 2015). Self-awareness and understanding self-improvement, self-satisfaction and job
of oneself are the basis of the notion of self-concept (Durodolu, 2018), which satisfaction, as well as the initiative and
may be explained as a system of attitudes one has towards oneself accord- openness to change varied depending on
ing to Prasad (2014). Certain authors have offered more ample explanations,
work experience and pay satisfaction. Work
which include additional characteristics of self-concept, so that it is defined
as: a system of attitudes and knowledge about oneself, as well as perceptions, experience, not pay satisfaction, proved to
characteristics and attributes used when describing oneself (Peralta Sánchez have an impact on the teachers’ perception
& Sánchez Roda, 2003); a complex, organized and dynamic system of learned of the relationship with their colleagues,
opinions, attitudes and emotions that individuals ascribe to themselves con- whereas the perception of personal and
sidering them to be true (Mehrad, 2016); an organized cognitive structure teaching qualities did not depend on the
comprised of attitudes, beliefs and values, independent of experience and
aforementioned variables.
activities, which are combined to help in the realization of particular and
specific habits, abilities, perspectives, ideas and emotions (Mishra, 2013). Keywords: empirical research, practice,
The term self refers to “psychological mechanisms or processes that teachers, self-concept, science education.
enable individuals to think about themselves consciously” (Leary & Tangney,
2012, p. 6). Self includes both the one who thinks (“I think”) and the object
of their thoughts (“about myself”). Mental concepts or ideas regarding the
queries of who one is, who one was and who one will be occupy an impor-
tant part of these thoughts. These mental concepts are actually the gist of Jelena Ž. Maksimović,
Jelena S. Osmanović
self-concept (Oyserman et al., 2012). Self-concept is present in the character University of Niš, Serbia
of each individual and that individual’s experience. It may be defined as a
general and conscious understanding of oneself. In fact, it is the ability to
105
(P. 105-116)
conceptualize and represent oneself, which is independent of quotidian experiences (Shastri, 2015).
Teachers’ self-concept is the factor that determines their conduct and behaviour (Shastri, 2015). In the light
of the profession, teachers’ self-concept includes a series of convictions, attitudes and suppositions they have
regarding their teaching achievements, relationship with students and colleagues and values of their profession
(Villa & Calvete, 2001). Teachers’ self-concept is based on several dimensions. They are: (1) social self-concept, re-
lated to an individual’s ideas of other people’s perception of that individual; (2) teachers’ efficiency, reflected in the
acquisition of professional skills, such as self-reliance, adaptability, personal efficiency, etc.; (3) resolving academic
issues through the use of knowledge, facts and data with the purpose of solving professional and institutional
problems; and (4) self-confidence, which is based on teachers’ estimate of their own values and which includes
their convictions (“I am competent”, “I am useful”) and emotions (success, despair, pride, shame). These dimensions
are interconnected and complementary. Those teachers who perceive themselves in a positive way have a positive
self-concept. Actually, the premise “I can do this” triggers the individual’s ability to choose and try novel things, as
well as their desire to succeed (Shastri, 2015). Moreover, the significance of teachers’ having a positive self-concept
is evident in their endeavour to help their students create a positive image of themselves, but also accept their
own flaws and constraints. The teachers who do not have a positive self-concept lack the self-confidence neces-
sary for them to encourage their students to create their own positive self-concept (Villa & Calvete, 2001). This is
very important, especially as regards the fact that self-concept is crucial and central to the process of personality
development. A positive self-concept is the basis of an overtly good personal, social and professional engagement,
personal satisfaction and the ability of each individual to be comfortable “in their own skin” (Esnaola et al., 2008).
Self-concept is frequently viewed through the prysm of self-confidence (Mishra, 2013), which is actually self-
conviction of one’s own abilities that enables one to deal with unpredictable situations and challanges (Asmaa,
2016). Considering the fact that self-confidence means the expression of one’s personal value (Gabryś-Barker, 2010),
it follows that a highly confident person has a positive self-concept. Since teachers form and shape future genera-
tions, they are responsible for the development of the positive self-concept and self-confidence (Mishra, 2013). In
other words, the development of teachers’ self-concept is also the strategy for the improvement of the quality of
education in general (Villa & Calvete, 2001). Self-confidence can be developed by means of creating positive attitudes
to oneself, as well as by reconsidering the system of values on which self-concept is based (Gabryś-Barker, 2010).
Self-confidence is improved by teachers’ acceptance of their own mistakes and their determination to overcome
and correct them (Mbuva, 2016). Self-concept may be improved by the development of reflective abilities regarding
one’s own qualities and their base. This may either influence their further development or prevent it completely
(Gabryś-Barker, 2010). Another problem is the fact that self-concept is resistant to change (Živković, 2013), which
means that a person struggles to maintain the already existing self-concept even when it is changed, i.e. an indi-
vidual still holds tight onto obsolete and negative perceptions, which leads to self-deception and impairs further
advancement (Sampthirao, 2016). A young teacher with a negative self-concept is unlikely to accept a different
view in the future. This is also caused by the fact that self-concept is a relational construct – it develops under the
influence of the perception that individuals have regarding their relationship with other people that are important
to them (Živković, 2013). Relationships with other people mostly influence two aspects of self-concept: the image
of oneself, related to the characteristics that one believes one has, and self-confidence, or one’s estimate of what
is valuable and significant in one’s own character (Sampthirao, 2016).
Self-concept also depends on age, gender, education, experience, academic advancement, social and eco-
nomic status, intelligence, as well as on certain institutional variables, such as school environment, organizational
ambience, interpersonal relationships, etc., and on the factors such as success and failure, social feedback and
self-respect. Moreover, self-concept is conditioned by teachers’ competence. Teachers who feel competent enough
and prepared to improve their communicative and teaching skills communicate better with their students and
have an overall better understanding of their own selves, i.e. they create a positive self-concept (Mishra, 2013).
Self-concept is one of the most significant factors that determines one’s success. The person who knows
their own abilities and talents, develops a positive attitude towards them and believes in their own success, is
the person who can improve and succeed. The same premise is valid for teachers and their self-concept. Teachers’
attitude towards their own selves is crucial to the process of teaching and learning since it directs this process to
teachers’ identification of their own abilities, talents, interests and motivation, which in turn have an impact on
the process of teaching (Sadeghi et al., 2015). Self-concept is also a teacher’s awareness of their own personality.
However, a distorted self-image may have a negative effect on teachers’ teaching competences (Mbuva, 2016).
106
ISSN 1648–3898 /Print/ TEACHERS’ SELF-CONCEPT AND ITS BENEFITS FOR SCIENCE EDUCATION
(P. 105-116)
The creation of a positive image of their own personality, i.e. a positive self-image, is an imperative and obliga-
tion for any contemporary teacher. The teachers with a positive image of themselves, with a strong belief in their
own abilities and engaged in a continuous self-improvement are likely to be not only more successful teachers,
but also more successful in their relationship with colleagues, students and other persons of importance. Besides,
teachers being role-models for their students, their positive self-concept is a precondition for the development
of students’ positive self-concept.
Knowing one own self, one’s abilities, characteristics, skills and knowledge, as well as constrains and drawbacks
is undoubtedly central to successful teaching; therefore, this research was focused upon the study of self-concept
of teachers, which was examined through the perceptions that teachers have regarding their own competences,
interpersonal relationships, hazards, initiatives, satisfaction and acceptance of their own personalities. The aim of
this research was to motivate teachers, who educate the future scholars, to reconsider their own personalities and
expectations and to reevaluate their own work.
General Background
Self-concept is a complex and multidimensional notion. It is primarily the image one has of oneself. People
frequently wonder: Who are we? What are we like? What is our value and self-worth? One often relies on pieces
of information contained in the statements made by other people about them, i.e. on their behaviour towards
others. However, it is of utter importance that the information about one`s own personality be provided by that
person with self-criticism. Owing to the fact that teaching represents one demanding, complex and challenging
profession, self-concept is crucial in the context of the contemporary education system.
The following hypotheses were postulated:
1. Teachers’ views on their relationships with colleagues are expected to vary depending on work experi-
ence and pay satisfaction.
2. Teachers’ views on the acceptance of their own selves and self-improvement are expected to vary
depending on work experience and pay satisfaction.
3. Teachers’ views on the satisfaction with their own selves, their achievements and their own profession
are expected to vary depending on work experience and pay satisfaction.
4. Teachers’ views on their teaching and personal qualities are expected to vary depending on work
experience and pay satisfaction.
5. Teachers’ views on the initiative and openness to change are expected to vary depending on work
experience and pay satisfaction.
The research was conducted on the territory of the Republic of Serbia in the course of the academic year of
2017/2018. The sample was constructed on the basis of the voluntary participation of teachers in the empirical
research. Therefore, it consisted of 442 respondents from the territory of the Republic of Serbia. The sample was
defined concerning the following factors: work experience (up to 5 years; 6 – 15 years; 16 – 25 years; over 25 years)
and pay satisfaction (examined in two categories: yes and no). The sample was the convenience sample.
Instruments and Procedures
The method used was descriptive with the scaling technique and Likert Scale (5- strongly agree, 4- agree,
3- neither agree nor disagree, 2- disagree, 1- strongly disagree). The teachers’ reflections on their self-concept in-
cluded the following segments: estimate of teachers’ competence, perception of interpersonal relationships with
colleagues, acceptance of risks and taking initiative, assessment of relationships with students, estimate of job
satisfaction and self-acceptance. The scale, consisting of 33 items, constructed for the purposes of this research in
particular and used to test the teachers’ self-concept, was examined by Factor Analysis. However, the adequacy
of the data to be analyzed using Factor Analysis had to be first determined. Thus, Kaiser-Meyer-Oklin (KMO) and
Bartlett’s test were applied. The data are adequate for Factor Analysis if the value of KMO is higher than .600, and
the value of Bartlett’s test lower than .050 (the value is supposed to be statistically significant) (Pallant, 2011).
107
(P. 105-116)
Data Analysis
The nature of the research and the choice of the research instruments determined the statistical tools to be
used. Therefore, the descriptive, inferential statistics and multivariate statistical method were used. Five main fac-
tors were distinguished by Factor Analysis. The parametric statistics (t test and ANOVA test) was used in the further
data processing in accordance with the determined independent research variables.
Results of Research
The scale consisting of 33 items, used to test the teachers’ self-concept, was examined by Factor Analysis.
However, the adequacy of the data to be analyzed using Factor Analysis had to be first determined. Thus, Kaiser-
Meyer-Olkin (KMO) and Bartlett’s test were applied. The data are adequate for Factor Analysis if the value of KMO
is higher than .600, and the value of Bartlett’s test lower than .050 (the value is supposed to be statistically signifi-
cant) (Pallant, 2011).
Table 1. KMO and Bartlett’s test.
Kaiser-Meyer-Olkin Measure of Sampling Adequacy .745
Approx. Chi-Square 8846.797

Bartlett’s Test of Sphericity df 528
Sig. .001
Table 1 shows that the KMO value was 0.745, while the value of Bartlett’s test was statistically significant,
p=.001, which meant that the data were adequate for Factor Analysis.
Table 2. Structure matrix of the extracted factors.
Rotation Sums
Component Initial Eigenvalues Extraction Sums of Squared Loadings of Squared
Loadings(a)
% of % of
Total Cumultive % Total Cumultive % Total
Variance Variance
1 7.465 22.620 22.620 7.465 22.620 22.620 4.815

2 3.718 11.268 33.888 3.718 11.268 33.888 4.482
3 3.251 9.852 43.740 3.251 9.852 43.740 3.191
4 2.479 7.511 51.251 2.479 7.511 51.251 4.805
5 2.052 6.219 57.470 2.052 6.219 57.470 3.011
6 1.599 4.845 62.315 1.599 4.845 62.315 3.696
7 1.307 3.960 66.275 1.307 3.960 66.275 1.790
Table 2 shows the extracted factors. The number of the factors to be extracted had to be based upon Kaiser
test, according to which only those factors whose eigenvalues were 1 or more had to be taken into consideration.
Those values were determined in 7 factors (7.465, 3.718, 3.251, 2.479, 2.052, 1.599, 1.307).
108
(P. 105-116)
Figure1. Scree Plot.
The scree plot presented in Figure 1 also proves that 7 factors were extracted by Factor Analysis with the vari-
max rotation. The point where the slope of the curve was clearly leveling off (the “elbow”) indicated the number of
factors that should be generated by the analysis (the break clearly visible at factor 8). However, since 7 appeared
to be rather a lot of factors, it was decided to reduce them to 5, which described over 50% of the total cumulative
variance.
The structure matrix of the rotated factors shown below was obtained by the oblimin factor rotation.
Table 3. Structure matrix of rotated factors.
Component

1 2 3 4 5
p10 .860
p11 .854
p12 .802
p14 .694
p13 .627
p31 .869
p30 .849
p32 .820
p29 .803
p33 .753
p28 .857
p24 .789
p26 .764
p25 .688
p27 .622
109
(P. 105-116)
Component

1 2 3 4 5
p2 .742
p6 .727
p3 .698
p4 .697
p1 .640
p7 .639
p9 .634
p5 .626
p15 .792
p18 .738
p16 .706
p19 .603
Extraction Method: Principal Component Analysis.
Rotation Method: Oblimin with Kaiser Normalization.a
a. Rotation converged in 9 iterations.
Since it was decided to retain 5 factors, the following items were excluded: p21 (I completely trust my students),
p22 (I usually enjoy teaching), p23 (Generally speaking, my students respect me), p20 (I am self-confident in the relation-
ship with my students), p17 (I find it difficult to take the initiative) and p8 (I am prone to questioning my ideas and skills).
The following factors were included in the research:
The first factor comprised these items: I feel that my colleagues and I are part of the same team; I am pleased
with the interpersonal relationships with my colleagues at work; I feel that I am respected; I am accepted by others;
I enjoy sharing experience and cooperating with my colleagues. This factor was named Relationship with colleagues.
The second factor comprised the following items: I know what I want and what I am able to do; I accept myself
although I am not perfect; I learn from my own mistakes; I feel free and I am not afraid to be what I am and to accept
the consequences; I learn continually. This factor was named Accepting and improving oneself.
The third factor comprised these items: I cannot stand my own self; I consider myself a failure; I find my teach-
ing unsatisfactory; It is very hard for me to be successful in the present circumstances; I wish I could change my
job. This factor was named Self-satisfaction and job satisfaction.
The fourth factor comprised the following items: Generally speaking, I consider myself a good teacher; I am
known to be an efficient teacher; I think that I have good teaching skills/abilities; I am confident of my abilities; I
feel quite competent; I feel that I am a worthy person; My colleagues consider me a competent teacher; I feel self-
confident. This factor was named Teaching and personal qualities.
The fifth factor comprised the following items: I enjoy thinking about new plans; I love to take risks; I am not
concerned about changes; I readily accept any criticism coming from either my colleagues or my students. This
factor was named Initiative and openness to change.
The success in teaching depends largely on the ability of every individual teacher to become aware of their
own characteristics, abilities and qualities, so as to attain a true image of themselves, including all possible flaws
and drawbacks of their character, which is the first step towards them being overcome. This further proves the
significance of the study of self-concept. Presented through separate factors (relationship with colleagues, ac-
cepting and improving oneself, self-satisfaction and job satisfaction, teaching and personal qualities, initiative
and openness to change), teachers’ self-concept was studied in relation to the stated variables: work experience
and pay satisfaction.
110
(P. 105-116)
Table 4. Differences in teachers’ views on their relationship with colleagues as regards work experience.
Work experience M SD F df p
0-5 yrs 21.949 2.621

6-15 yrs 21.312 2.644
Relationship with colleagues 3.445 3 .017
16-25 yrs 22.139 2.346
over 25 yrs 22.374 3.126
Table 4 presents the differences in teachers’ views on their relationship with colleagues as regards work
experience. The results were estimated by F test. The value p=.017 indicated that this difference was statistically
significant. It meant that teachers having different views on interpersonal relationships with colleagues, coopera-
tion with colleagues, team work, mutual approval and respect depended on work experience. The results also
proved that the most experienced teachers (over 25 years of teaching) valued most the aforementioned elements
(cooperation, approval and respect, team work), i.e. relationship with colleagues (M=22.374).
Table 5. Differences in teachers’ views on their relationship with colleagues as regards pay satisfaction.
Pay satisfaction primanjima M SD t df p
Yes 22.083 1.556

Relationship with colleagues .629 56.803 .532
No 21.899 2.756
Table 5 presents the differences in teachers’ views on their relationship with colleagues as regards pay satisfac-
tion. The results were estimated by t-test. The obtained value p=.532 proved that pay satisfaction did not determine
any statistically significant difference in teachers’ views on their relationship with colleagues, which was further
proved by the similar arithmetic mean for the categories yes (M=22.083) and no (M=21.899).
Table 6. Differences in teachers’ views on self-acceptance and self-improvement as regards work experience.
0-5 yrs 23.063 1.917

6-15 yrs 21.023 4.195
Self-acceptance and self-improvement 5.483 3 .001
16-25 yrs 22.062 3.134
over 25 yrs 21.099 5.679
Table 6 presents the differences in teachers’ views on accepting and improving oneself as regards work experi-
ence. The results were estimated by ANOVA F test for the independent variable work experience. The difference in
teachers’ views was statistically significant (p=.001). The concept of accepting and improving oneself was related
to the acceptance of one’s own imperfection, flaws and mistakes and continuous efforts at overcoming these
drawbacks and improving one’s personal and professional qualities. The obtained results proved that the least
experienced teachers (up to 5 years of teaching) valued most this concept (M=23.063), which means that they
were the most willing to accept and improve themselves.
Table 7. Differences in teachers’ views on self-acceptance and self-improvement as regards pay satisfaction.
Pay satisfaction primanjima M SD t df p
Yes 20.278 4.694

Self-acceptance and self-improvement -2.298 440 .022
No 21.872 3.922
Table 7 presents the differences in teachers’ views on self-acceptance and self-improvement as regards pay
satisfaction, i.e. whether they were satisfied with their salaries or not. The obtained results proved that pay satisfac-
111
(P. 105-116)
tion had an impact on whether teachers accepted themselves as they really were, whwther they became aware
of their own qualities and drawbacks and whether they undertook certain steps with the purpose of continuous
learning about their own personalities and improving themselves. The value p=.022 meant that the difference was
statistically significant. However, the obtained data showed that the teachers who were not satisfied with their
salaries valued the concept of self-acceptance and self-improvement (M=21.872) more than the teachers who were
satisfied with their salaries (M=20.278).
Table 8. Differences in teachers’ views on self-satisfaction and job satisfaction as regards work experience.
0-5 yrs 7.633 1.956

6-15 yrs 8.227 2.517
Self-satisfaction and job satisfaction 7.373 3 .001
16-25 yrs 8.653 3.646
over 25 yrs 10.208 6.187
Table 8 presents the differences in teachers’ views on self-satisfaction and job satisfaction as regards work
experience. The results were estimated using F test. The value p=.001 proved that the difference in teachers’ views
on self-satisfaction and job satisfaction was statistically significant. This concept included the statements used
to examine whether teachers regarded themselves as successful in teaching, whether the present circumstances
impaired their success, whether they assessed their teaching as satisfactory or whether they were so unsatisfied
by themselves that they wanted to change their job. The estimated arithmetic means proved that teachers who
had been teaching for more than 25 years valued most the aforementioned statements (M=10.208), which means
that they were less satisfied with their own personalities and their job more frequently.
Table 9. Differences in teachers’ views on self-satisfaction and job satisfaction as regards pay satisfaction.
Pay satisfaction M SD t df p
Yes 6.861 1.885

Self-satisfaction and job satisfaction -5.287 67.853 .001
No 8.827 4.013
Table 9 shows the differences in teachers’ views on self-satisfaction and job satisfaction as regards pay satis-
faction. The results were estimated by means of t-test. The obtained value p=.001 proved that this difference was
statistically significant since teachers’ different views regarding satisfaction with their own personalities, success in
teaching and quality of their teaching depended on the fact whether they were satisfied with their salaries or not.
The teachers who were not satisfied with their salaries valued most the statements related to self-satisfaction and
job satisfaction (M=8.827) in comparison to those teachers who were satisfied with their job (M=6.861).
Table 10. Differences in teachers’ views on teaching and personal qualities as regards work experience.
0-5 yrs 36.013 2.289

6-15 yrs 35.539 3.505
Teaching and personal qualities .710 3 .546
16-25 yrs 35.819 3.389
over 25 yrs 36.132 3.092
Table 10 shows the differences in teachers’ views on teaching and personal qualities as regards work experi-
ence. The results were estimated by means of F test. The obtained value p=.0546 (p>.050) indicated that there
was no statistically significant difference in teachers’ views on their personal and teaching qualities in relation to
work experience. It meant that teachers similarly valued their competence, efficiency, teaching skills and abilities,
self-confidence and self-worth regardless of the years of teaching (5, 6-15, 16-25 or over 25 years).
112
(P. 105-116)
Table 11. Differences in teachers’ views on teaching and personal qualities as regards pay satisfaction.
Yes 35.722 2.337

Teaching and personal qualities -.296 47.986 .768
No 35.847 3.259
Table 11 presents the differences in teachers’ views on their teaching and personal qualities as regards pay
satisfaction. The results were estimated by means of t-test. The obtained value p=.768 proved that the difference
in teachers’ views was not statistically significant, and that pay satisfaction did not have a significant influence on
teachers’ assessment of their teaching and personal qualities, which was proved by the similar arithmetic means
for both categories, i.e. M=35.722 for the teachers who were satisfied with their salaries, and M=35.847 for the
teachers who were not satisfied with their salaries.
Table 12. Differences in teachers’ views on initiative and openness to change as regards work experience.
0-5 yrs 16.241 2.243

6-15 yrs 15.883 2.702
Initiative and openness to change 4.478 3 .004
16-25 yrs 15.097 2.890
over 25 yrs 16.187 2.878
Table 12 presents the differences in teachers’ views on initiative and openness to change as regards work
experience. The results were estimated by means of F test. Readiness to take initiative and openness to change
included the elements such as considering new plans, not showing concern about changes, readiness to take risks
and accept criticism from both colleagues and students. The obtained value p=.004 showed that the difference in
teachers’ views on the aforementioned elements was statistically significant with regard to work experience. The
obtained results also showed that the teachers with 5 years of teaching experience valued most these elements
(M=26.241).
Table 13. Differences in teachers’ views on initiative and openness to change as regards pay satisfaction.
Yes 16.639 1.313

Initiative and openness to change 3.705 69.087 .001
No 15.675 2.841
Table 13 presents the differences in teachers’ views on initiative and openness to change as regards pay sat-
isfaction. The results were estimated by means of t-test. The obtained value p=.001 indicated that the difference
was statistically significant, which meant that teachers’ views on risks, initiative, changes, etc., were significantly
determined by pay satisfaction. The teachers who were satisfied with their salaries valued the concept of initiative
and openness to change (M=16.639) more than the teachers who were not satisfied with their salaries (M=15.675).
Discussion
Self-concept depends on age, gender, education, experience, academic advancement, social and economic
status, intelligence, as well as on certain institutional variables, such as school environment, organizational ambi-
ence, interpersonal relationships, etc., and the factors such as success and failure, social feedback and self-respect.
Moreover, self-concept is conditioned by teachers’ competence. Teachers who feel competent enough and prepared
to improve their communicative and teaching skills communicate better with their students and have an overall
better understanding of their own selves, i.e. they create a positive self-concept (Mishra, 2013).
This empirical research tested the teachers’ self-concept on the basis of two variables, work experience, already
emphasized by the aforementioned author, and another one, pay satisfaction, which has rarely been explored.
113
(P. 105-116)
The obtained results proved that teachers’ perception on their relationship with colleagues was determined
by their work experience, but not by pay satisfaction, which is understandable considering the fact that relation-
ships with colleagues represent interpersonal relationships that may develop into friendship. The results stated
that the teachers with the teaching experience of over 25 years valued most their relationships with colleagues,
reflected in various forms of team work, sharing and cooperation, approval and respect, as well as mere delight in
these relationships with colleagues. This result is partially explained by the fact that these teachers, owing to their
long teaching experience, managed to create firm, valuable and recognized relationships with their colleagues,
realizing thus the importance of cooperation and team work for any success and achievement. The obtained re-
stults therefore partly proved the hypothesis: Teachers’ views on their relationships with colleagues are expected to
vary depending on work experience and pay satisfaction.
Accepting and improving oneself includes the elements such as knowing one’s own wishes and abilities, ac-
cepting oneself despite one’s imperfections, learning from one’s own mistakes and continuous learning, so that
teachers present their true self in every situation. The results of the research showed that teachers’ views varied
depending on work experience and pay satisfaction. The teachers with 5 years of teaching experience valued most
this concept, which is partly explained by the fact that they were generally young teachers, willing to dedicate
themselves to self-improvement in order to be respected and competent teachers and ready to prove their own
worth, aware of the fact that professional success was based on the acceptance of the paradigm of life-long learning
and education. The results also showed that the teachers with lower pay satisfaction valued this concept more than
the teachers who were satisfied with their salaries. This result does not correspond to the expected result, i.e. that
the teachers who are not satisfied with their salaries will not be highly motivated to improve themselves, or that
the appropriate financial support will contribute to a higher level of motivation for teachers’ self-improvement. It
is clear that, regardless of the fact that a number of teachers were dissatisfied with their salaries, they still recog-
nized and acknowledged the importance of permanent education, self-improvement, learning from one’s own
mistakes, overcoming one’s own weaknesses and flaws, etc. precisely because they knew that their continuous
improvement and education might not be feasible without the adequate financial support. The obtained results
confirmed the hypothesis: Teachers’ views on the acceptance of their own selves and self-improvement are expected
to vary depending on work experience and pay satisfaction.
The results of the research also proved that self-satisfaction and job satisfaction depended on work experience
and pay satisfaction. The teachers with the longest teaching experience (over 25 years) were the least satisfied
with themselves and their job, which may be explained by the fact that they had difficulty adapting to innovations
and changes required from them, and these were mostly older teachers who could hardly “keep pace” with the
novel ideas. Moreover, these teachers were more likely to suffer from the burnout syndrome, which further bred
dissatisfaction. Self-satisfaction and job satisfaction were lower in those teachers who were not satisfied with their
salaries. Pay is the mark of a job’s worth, social acknowledgment and approval, so that pay dissatisfaction inevitably
breeds job dissatisfaction and dissatisfaction with one’s own personality. The obtained results thus confirmed the
hypothesis: Teachers’ views on the satisfaction with their own selves, their achievements and their own profession are
expected to vary depending on work experience and pay satisfaction.
Considering teaching and personal qualities, the obtained results proved that work experience and pay satis-
faction did not significantly determine teachers’ perceptions of efficiency, teaching skills and abilities, competence
and personal worth, self-confidence, etc. These results are partly justified by the fact that teachers, regardless
of work experience and pay, trust their own ideas, develop the sense of their own worth and feel competent in
teaching since teaching is the profession that requires certain skills and talents. Therefore, the obtained results
disproved the hypothesis: Teachers’ views on their teaching and personal qualities are expected to vary depending on
work experience and pay satisfaction.
Initiative and openness to change depend on work experience and pay satisfaction. The teachers with the
least work experience (5 years) most considered new plans and were least concerned about changes and criticism.
Thus, these teachers were most willing to change, take risks and initiative, which is partly explained by the fact that
they were mainly young teachers still endeavouring to find their own manner of teaching, try out various teach-
ing methods and not yet suffering from burnout syndrome. The results also proved that these ideas were more
valued by the teachers who were satisfied with their salaries than by the teachers with low pay satisfaction, which
further proves that the teachers dissatisfied with their pay are not willing to take initiative or be open to change.
The obtained results confirmed the hypothesis: Teachers’ views on the initiative and openness to change are expected
to vary depending on work experience and pay satisfaction.
114
(P. 105-116)
Numerous scholars have studied the same issues (Asmaa, 2016; Mbuva, 2016; Mishra, 2013; Sadeghi et al.,
2015; Sampthirao, 2016), which has provided the grounds for this empirical research. However, two independent
variables, work experience and pay satisfaction, were used to perceive the Serbian teachers’ self-concept, which
further contributes to the future similar studies.
Conclusions and Implications
Improvement and quality achievement are based on the observation of the current circumstances, charac-
teristics, advantages, success, but also of certain weaknesses, drawbacks and flaws to be corrected. This is valid
for people, as well, since it is only by knowing themselves that individuals can change, improve and develop.
Self-concept, understood as the “image of oneself”, is based on knowing one’s own personality. This research was
conducted with the purpose of realizing what kind of “self-image” teachers have, i.e. their self-concept. Considering
the fact that it is a multidimensional concept, teachers’ views on relationships with colleagues, self-satisfaction and
job satisfaction, self-acceptance and self-improvement, initiative and openness to change, personal and teach-
ing qualities with regard to work experience and pay satisfaction were studied. The obtained results showed that
self-acceptance and self-improvement, self-satisfaction and job satisfaction, as well as initiative and openness to
change varied depending on work experience and pay satisfaction. Relationships with colleagues varied depending
on work experience, but not on pay satisfaction, whereas the views on personal and teaching qualities depended
on the aforementioned variable.
This research may motivate teachers to think about themselves, to consider their own qualities, values, strengths
and their drawbacks, but also to consider their own personalities in the context of other people, both their students
and colleagues. Self-reflection is very significant from the teachers’ viewpoint, since they have to be reflective prac-
titioners and reflect on their own selves and their job and life, in general, in order to be able to fulfil contemporary
requirements. Only the teachers who know their own selves will be able to transform their “imagined self” into
their “true self”, i.e. to create the best version of their own personalities. This is in turn reflected in their success in
teaching and educating their students since teachers are the persons that their students appreciate and respect.
References
Asmaa, B. (2016). On studying teachers’ self-esteem based on revised Janis scale application. Linguistic and literary broad
research and innovation, 5 (1), 34-47.
Durodolu, O. O. (2018). Information literacy, self-concept and metacognitive ability of teacher-librarians at the University of
Zululand. Library philosophy and practice. Retrieved 10/06/2018, online source: https://digitalcommons.unl.edu/cgi/
viewcontent.cgi?article=4759&context=libphilprac
Esnaola, I., Goñi, A., & Madariaga, J. M. (2008). El autoconcepto: Perspectivas de investigación [Self-concept: perspectives of
investigation]. Revista de Psicodidáctica, 13 (1), 69-96.
Gabryś-Barker, D. (2010). On teacher beliefs, self-Identity and the stages of professional development. Lingvarm Arena, 1 (1),
25-42.
Leary, M. R., & Tangney, J. P. (2012). The self as an organizing construct in the behavioural and social sciences. In M. R. Leary,
& J. P. Tangney (Eds.), Handbook of self and identity (pp. 1-18). New York: The Guilford Press.
Mbuva, J. (2016). Exploring teachers’ self-esteem and its effects on teaching, students’ learning and self-esteem. Journal of
Higher Education Theory and Practice, 16 (5), 59-68.
Mehrad, A. (2016). Mini literature review of self-concept. Journal of Educational, Health and Community Psychology, 5 (2), 62-66.
Mishra, S. (2013). Self-Concept and scholastic competence of secondary school teachers in relation to some personal variables.
IOSR Journal of Computer Engineering, 12 (1), 23-28.
Oyserman, D., Elmore, K., & Smith, G. (2012). Self, self-concept and identity. In M. R. Leary, & J. P. Tangney (Eds.), Handbook of
Self and Identity (pp. 69-104). New York: The Guilford Press.
Pallant, J. (2011). SPSS survival manual: A step by step guide to data analysis using the SPSS program. 4th Edition, Allen & Unwin,
Berkshire.
Peralta Sánchez, F. J., & Sánchez Roda, M. D. (2003). Relationships between self-concept and academic achievement in primary
students. Electronic Journal of Research in Educational Psychology and Psychopedagogy, 1 (1), 95-120.
Prasad, R. (2014). Effect of teacher training programme on self-concept, self-confidence, teaching competency, and role com-
mitment of special teacher trainees. American Journal of Educational Research, 2 (12), 22-30.
Sampthirao, P. (2016). Self-concept and interpersonal communication. The International Journal of Indian Psychology, 3 (6),
178-189.
Sadeghi, A., Azizi, S., & Poor, S. M. (2015). Investigating the relationship between positive self-concept and success of academic
staffs at the University of Guilan, Iran. Psychology, 6, 2155-2160.
115
(P. 105-116)
Shastri, F. C. (2015). A study on the self-concept of teachers working in government, aided and unaided colleges in Rajkot.
International Journal of Information Research and Review, 2 (3), 571-575.
Villa, A., & Calvete, E. (2001). Development of the teacher self-concept evaluation scale and its relation to burnout. Studies in
Educational Evaluation, 27, 239-255.
Živković, P. (2010). Professional development and teachers professional identity: Self-Assessment in Republic of Serbia. Journal
of Education and Instructional Studies in the World, 3 (1), 150-158.
Villa, A., & Calvete, E. (2001). Development of the teacher self-concept evaluation scale and its relation to burnout. Studies in
Educational Evaluation, 27, 239-255.
Jelena Ž. Maksimović PhD, Associate Professor, University of Niš, Department of Pedagogy,

Ćirila i Metodija 2, 18 000 Niš, Serbia.
E-mail: jelena.maksimovic.filfak.ni.ac.rs
Website: https://www.filfak.ni.ac.rs/nastavno-osoblje/pedagogija/
item/137-jelena-maksimovic
Jelena S. Osmanović Teaching Assistant, University of Niš, Department of Pedagogy, Ćirila I
Metodija 2, 18 000 Niš, Serbia.
E-mail: jelena.osmanovic.filfak.ni.ac.rs
Website: https://www.filfak.ni.ac.rs/nastavno-osoblje/pedagogija/
item/143-jelena-osmanovic
116
THE IMPLEMENTATION OF MIND
MAPS IN TEACHING PHYSICS:
EDUCATIONAL EFFICIENCY AND ISSN 1648-3898 /Print/
STUDENTS’ INVOLVEMENT ISSN 2538-7138 /Online/
Abstract. Primary school students gener-

Zvezdan Z.Gagić,
ally lack motivation for learning physics,
Sonja J. Skuban, which they perceive as a difficult subject.
Branka N. Radulović, In order to overcome this problem, it is
Maja M. Stojanović, necessary to apply appropriate teaching
Olivera Gajić approach. The aim of this research was to
assess the educational efficiency of mind
maps in physics and students’ involvement
when this approach is used. A pre-test –
post-test control group design was used.
The sample of this research consisted of
Introduction 113 seventh-grade students divided into
an experimental and a control group. The
Students usually lack motivation for learning subjects which they students in the experimental group were
perceive as difficult to learn. That is, most students want to get good marks taught physics using mind maps and the
with the least amount of effort. Although some subjects are more complex students in the control group were taught
and abstract than the others, using an appropriate teaching approach can using conventional teaching approach. The
change students’ perception on how difficult learning these subjects is. data were collected by using two physics
Physics is usually regarded as difficult to learn, so students are often unmo- knowledge tests and perceived mental
tivated to learn this subject. Besides, students have great difficulty in gaining effort scale. The data were analyzed using
meaningful learning on this subject because lecturing-based instruction is ANOVA, t-test and chi-square test. The ob-
the most common teaching approach (Stanivuk, Bogdanović, & Skuban, tained results showed that the educational
2015; Stanivuk, Skuban, & Bogdanović, 2015; Tanel & Erol, 2008, according efficiency of teaching with the use of mind
to Márquez, Manuel, Llinás, & Macías, 2017). The use of appropriate teaching maps was greater than the efficiency of
approach can enhance students’ acquiring of physics contents at all levels conventional teaching approach. Besides
of education (Cvjetićanin, Obadović, & Rančić, 2015; Zouhor, Bogdanović, that, the students’ involvement in the
Skuban, & Pavkov-Hrvojević, 2017). For this reason, in the last decades, teach- experimental group was higher than the
ing approaches have been developed based on a constructivist approach one in the control group. The implementa-
to teaching and learning (Dhindsa & Anderson, 2011). The aim of these ap- tion of mind maps in teaching physics in
proaches is to increase the performance of students taking into account the primary schools can increase students’
limit of working memory and encouraging active participation of students in motivation for learning physics and lower
creating their own knowledge. If knowledge is presented as a link between their mental effort.
concepts and propositions (Douma, Ligierko, & Romano, 2009), then it is Keywords: educational efficiency, mind
clear that there is a need to introduce techniques of visual representation maps, students’ involvement, teaching
of knowledge (Meyer, 2010). There are numerous techniques for visualizing physics.
information in teaching process. Some of them are: conceptual maps, mind
maps, conceptual diagram, visual metaphor, semantic networks, etc. (Ep-
Zvezdan Z.Gagić, Sonja J. Skuban,
pler, 2006; Parikh, 2015). A concept map is a top-down diagram showing Branka N. Radulović,
the relationships between concepts, including cross connections and their Maja M. Stojanović, Olivera Gajić
manifestations (Eppler, 2006). Since concepts are very clearly connected University of Novi Sad, Serbia
to each other, concept maps represent knowledge structures as a whole
117
THE IMPLEMENTATION OF MIND MAPS IN TEACHING PHYSICS: EDUCATIONAL EFFICIENCY
AND STUDENTS’ INVOLVEMENT
(P. 117-131) ISSN 2538–7138 /Online/
(Nousiainen, 2012). According to Usta and Ültay (2016), McClure, Sonak and Suen have emphasized that concept
maps can be used as a learning strategy, as a teaching strategy, as a strategy for planning curriculum, and as a
means of assessing students’ understanding of science concepts (Usta & Ültay, 2016). Soika and Reiska (2014) have
emphasized in their research that concept mapping can be used for assessment in science education (Soika &
Reiska, 2016). Beside the conceptual maps developed by J. D. Novak (1998), there are also mind maps. Mind maps
were first constructed by T. Buzan (Buzan & Buzan, 1996). Buzan used Habert’s ideas to develop mind mapping as
a method of note-taking based on the idea of making notes as brief as possible and as “interesting to the eye” as
possible by using visual effects (Abi-El-Mona & Adb-El-Khalick, 2008). Mind mapping is used in order to represent
knowledge by organizing it in a form of network or other non-linear diagram (Dhindsa & Anderson, 2011). Mind
maps are composed of a central idea, keywords (edges) and nodes (Kedaj, Pavlíček, & Hanzlík, 2014). The central
idea can be a physical phenomenon or a concept that is treated during a particular class. The keywords are branch-
ing from the central idea to specific details that may be presented in the form of images, formulas or experiment
sketches. Images or sketches are most often represented in color. In this way, both brain hemisphere activation is
achieved (Buzan & Buzan, 1996; Seyihoglu & Kartal, 2010). Mind maps can be used in all situations involving the
need for learning and any form of thinking (Kovačević & Segedinac, 2007). According to them, this can be: planning,
organizing, analyzing and solving problems, designing projects, preparing speeches and presentations, writing,
making notes, lecturing, and similar. From the above, special benefits of using mind maps in teaching are apparent.
One advantage of mind maps is emphasizing student-created representation of knowledge as compared to one
being imposed by the visual tool itself (Abi-El-Mona & Adb-El-Khalick, 2008). This approach involves brainstorming
ideas (Adodo, 2013) which is of particular importance for the teaching of natural sciences, especially physics. In
consistency with modern constructivist learning approach, mind mapping can be used to emphasize the active
involvement of the student who constructs new knowledge by making connections between new content and prior
knowledge (Dhindsa & Anderson, 2011). It has been shown that students perceive mind maps as “fun, interesting,
and a motivating approach to learning” (Goodnough & Woods, 2002; according to Keleş, 2012). Besides, teachers
also enjoy using mind maps and they believe that this approach fosters students’ motivation in learning science
(Keleş, 2012). Since there was no similar research carried out in sixth-grade physics teaching, the aim of this research
was to examine educational efficiency and students’ involvement when mind maps are used.
Cognitive Load and Mental Effort
Cognitive load is a term that is directly related to the topology of human cognitive architecture, which in its
composition distinguishes multiple types of memory. These memory types are: sensory, working, and long-term
memory (Tulving, 1985). The load that the individual (student), who through instruction seeks to adopt the mate-
rial, is exposed to, directly depends on the capacity of the working memory that can save seven plus/minus two
meaningful units (information) (Miller, 1955). Cognitive load is described as a multidimensional construct and
represents a load imposed on a cognitive system of an individual during the learning of a new teaching content
or in solving a particular problem or task (Plass, Moreno, & Brünken, 2010; Sweller, Ayres, & Kalyuga, 2011). Re-
searchers also define multiple types of cognitive load: intrinsic, extraneous and germane cognitive load (Kalyuga,
2009; Kirschner, 2002; Sweller, 1994). The cognitive load that arises from the very nature of the material and the
structure of the information that must be adopted through learning is an intrinsic cognitive load. Therefore, the
level of this load is determined by the very interactivity of the elements that are exposed to students (Sweller,
Ayres, & Kalyuga, 2011). Thus, a higher interactivity of the elements leads to a greater occupancy of the capacity
of the working memory, and therefore a higher intrinsic load (Chandler & Sweller, 1991). Extraneous cognitive load
represents the cognitive load that the individual (student) is exposed to as the result of inadequate external teach-
ing approach or procedure during the presentation of different contents (Paas, Tuovinen, Tabbers, & Van Gerven,
2003; Plass, Moreno, & Brünken, 2010; Sweller, Ayres, & Kalyuga, 2011). Great attention was focused on examining
this part of cognitive load in order to find a way to reduce it to the appropriate level. Germane cognitive load is
the type of cognitive load that is placed before an individual when the information that an individual needs to
adopt is presented in such a way that learning is improved, that is, the construction and automation of cognitive
schemes in the long-term memory is facilitated (van Merriënboer & Sweller, 2005). Germane load depends on how
information is passed on to an individual, as well as on the activities that he/she conducts in the learning process,
118
ISSN 1648–3898 /Print/ THE IMPLEMENTATION OF MIND MAPS IN TEACHING PHYSICS: EDUCATIONAL EFFICIENCY
ISSN 2538–7138 /Online/ (P. 117-131)
so that the load itself improves the learning process. In addition to the defined types of cognitive load, it, as a
multidimensional construct, includes causal and evaluating factors.
According to Paas and Van Merriënboer (1994) and Choi, Van Merriënboer and Paas (2014), causal factors can
be defined as task (environment) characteristics, learner characteristics and their interactions while the assessment
factors are mental load, mental effort and performance. So, mental effort as an aspect of cognitive load, which
we used in our research, refers to the cognitive capacity that is actually allocated to accommodate the demands
imposed by the task; thus, it can be considered to reflect the actual cognitive load (Paas, Tuovinen, Tabbers & Van
Gerven, 2003).
In their paper, Paas and Van Merrienboer gave a formula according to which it is possible to calculate the ef-
ficiency of the applied teaching approach using the standardized values of achievement and mental effort (Paas
& Van Merriënboer, 1993). Moreover, Paas and his associates determined the formula which makes it possible to
calculate the students-involvement (Paas, Tuovinen, Van Merrienboer, & Aubteen Darabi, 2005). Calculating the
involvement it can determine motivational effects of teaching approach and predict which instructional configura-
tions will maximize learning and transfer (Paas, Tuovinen, Van Merrienboer & Aubteen Darabi, 2005). Involvement
indicates on the learners’ involvement or on the retention of students’ attention on the learning materials. Taking
into account the students-involvement and educational efficiency in determining the influence of the applied
teaching approach on the students’ performance, positive values will indicate a positive effect while negative
values will indicate an adverse effect of the applied teaching approaches: conventional teaching approach and
approach based on the use of mind maps.
Problem of Research
The task of education is to enable students to aquire long-lasting and applicable knowledge. A number of
primary school students in the Republic of Serbia have low achievement in physics which is reflected in bad marks
in this subject (Zouhor, Bogdanović, Skuban, & Pavkov-Hrvojević, 2017). Besides, the results of PISA and TIMSS tests
showed that students from the Republic of Serbia achieve lower results than students from other countries (OECD,
2012; Provasnik, Malley, Stephens, Landeros, Perkins, & Tang, 2016). In order to increase students’ achievement
on these tests, it is necessary, among other things, to change the teaching approach. The introduction of new
teaching approaches can significantly contribute to students’ achievement. Therefore, the aim of this research was
oriented towards examining the new approach in physics teaching in primary school. The researchers considered
it particularly important to examine the efficiency of teaching approaches in primary school because students in
the Republic of Serbia then learn basics about physical concepts within regular classes for the first time. Incorrectly
learned (or not learned) concepts can cause major problems in the correct understanding of the world around us,
as well as in solving certain problem situations. In the literature (Abi-El-Mona & Adb-El-Khalick, 2008; Akinoglu &
Yasar, 2007) it is indicated that there is a positive influence of the application of mind maps as a teaching approach
on students’ achievement. It is considered important and interesting to examine the influence of teaching approach
based on mind maps on students’ achievement and perceived mental effort. Knowing the standardized values of
performance and mental effort, the efficiency and involvement can be determined.
Research Focus
The stated aim of the research was operationalized in the following research questions:
1. Is there a difference between students in the experimental (E) and control (C) groups in terms of their
achievement on the post-test, depending on the applied teaching approach?
2. Is there a difference between students in the E group and C group in the perception of mental effort,
depending on the applied teaching approach?
3. Are the efficiency and involvement for the approach based on the use of mind maps greater than
conventional teaching approach?
119
(P. 117-131) ISSN 2538–7138 /Online/
General Background
The starting point of this research lied in the constructivist approach to the application of mind maps in the
teaching physics. The researchers applied constructivist learning environment, PowerPoint was used for preparing
presentations, while mind maps were used for teaching the content. According to Dhindsa and Anderson (2011),
using mind maps should help students “comprehend and better integrate the scientific information into stable
knowledge structures in memory”. Similar studies carried out within the teaching of different school subjects gave
positive results. The research was carried out in two primary schools in the Republic of Serbia. According to the
curriculum determined by the Ministry of Education, Science and Technological Development of the Republic of
Serbia, Physics, as a teaching subject, is first introduced in the sixth grade (students are aged 12). It is therefore
important that students adopt the first concepts of Physics correctly with, if possible, a lower mental effort. The
research was carried out for 8 school weeks during February and March 2016. The use of mind maps in teaching
Statics (equilibrium) was explored.
Sample
The research included 113 sixth-grade primary school students, out of which 66 (58.4%) boys and 47 (41.6%)
girls. The research was conducted in two primary schools, “Đorđe Natošević” in Novi Sad and “Miroslav Antić” in
Futog in Republic of Serbia, during the period of February-March 2016. All students were volunteers in the research
and their privacy was respected during the research. Students filled out tests anonymously, using codes instead
of names. Informed consent was obtained from all individual participants included in the study. The size of the
sample was limited by the number of sixth-grade students that could be taught by the researcher and another
teacher involved in this research. The teacher was specially trained to work with students in the experimental
group. The teacher took an active part in preparing the material and, with the help of researcher, the teacher
did all the necessary preparation in order to use this teaching approach. This preparation was carried out within
two weeks. Researchers decided to include only one teacher in order to avoid the influence of different teachers’
competences. Used sample was valid for all tests performed in this research. All the requirements for carrying out
tests performed in this research were checked (with additional tests where needed) and the check showed that
all the requirements were met.
In this research the techniques of testing and scaling were applied. Testing was performed in order to evaluate
students’ achievement, while scaling was performed in order to assess perceived mental effort.
Instruments that were designed and applied in this research were:
- the pre-test - a test for determining prior knowledge of students in groups C and E (before the introduc-
tion of the experimental factors).
- The post-test - a test for determining knowledge of students in groups C and E after the introduction
of the experimental factors.
- Perceived mental effort scale.
The items for knowledge tests were adapted from literature related to physics for primary schools, used in
the Republic of Serbia (Čaluković, 2014/a, 2014/b; Kapor & Šetrajčić, 2009; Mitorvić, 2013/a, 2013/b). These items
were positively reviewed by three university professors who are specialized for such areas of physics. The Cronbach
alpha coefficient was .76 for the pre-test, .79 for the post-test, and .86 for the perceived mental effort scale. The
obtained values of this coefficient indicated that knowledge tests and perceived mental effort scale which have
been applied have acceptable reliability.
The following are the examples of the test items in the level of knowledge, comprehension and application,
respectively:
•• Lever is:
(a) A rigid body capable of rotating around a fulcrum
120
ISSN 2538–7138 /Online/ (P. 117-131)
(b) Any rigid body

(c) Fixed rigid body
(d) None of the above
•• The buoyant force acting at the body immersed in liquid is greater than the buoyant force acting on
the same body immersed in gas because:
(a) Density of gas is greater than density of liquid
(b) Density of liquid is greater than density of gas
(c) Density of gas and density of liquid are equal
(d) None of the above
•• An apple is hung on a lever with fulcrum in O (Figure 1). Weight of the apple is acting on the lever at point
A. Calculate the value of the force acting at point B, if it makes the lever in equilibrium (when horizontal).
(a) 0.8N
(b) 1N
(c) 1.2N
(d) 0.6N
Figure 1. Illustration of stated problem.
Before the beginning of the research, the students were given a pre-test with the aim to determine their initial
knowledge about the themes Pressure and Motion of the body under the influence of gravity. Friction force. The theme
Pressure was chosen with the aim to determine the understanding of the definition of the concept of pressure and
the force acting on a specific surface. Teaching theme Motion of the body under the influence of gravity. Friction force
was a theme that preceded the theme Equilibrium, and for this reason special attention was paid to the students’
understanding of the concept of frictional force. Based on students’ achievements in the pre- test, students were
divided into two groups (E group and C group). To the students in the C group, the teaching units were presented
with conventional teaching approach, in terms of explicit teaching through lectures and teacher-led demonstrations
planned according to the sixth-grade curriculum. The students in the E group were taught the same teaching units
with the use of mind maps. The researcher, who carried out a pedagogical experiment, created mind maps for the
planned teaching units (an example is given in Appendix). Four mind maps were created using PowerPoint. After
the formation of the groups and preparation of the teaching material for the students in the E group, a pedagogical
experiment with parallel groups was performed. The students in the E group created their own mind maps based
on given examples and the teacher helped them to master mind maping. After working on the planned topic, final
testing was conducted in order to observe the results of applying the particular instruction.
Data Analysis

The mean score and standard deviation on the pre-test and the post-test were determined for both groups.
The statistical significance of the obtained difference in students achievement, as well as in perceived mental effort
on the post-test between groups was tested by ANOVA, t test, and chi-square test. To determine the efficiency of
the applied teaching approach based on the use of mind maps, Cramer’s V indicator for the chi-square test and
121
(P. 117-131) ISSN 2538–7138 /Online/
the eta-square for ANOVA were used. The influence of applied teaching approach on students’ performance and
perceived mental effort was considered. Standardized values of students- achievement and perceived mental effort
were calculated based on the educational efficiency and students’ involvement caused by the applied teaching
approach. Data analysis was carried out using the SPSS 20.0 software.
Besides analysing students achievement and mental effort, educational efficiency and students involvement
R−P
E=
2
were calculated. According to Paas and Van Merriënboer (1993) efficiency can be calculated by using the formula ,
where R is the standardized value of perceived mental effort, and P is the standardized value of students’ achieve-
ment. Since this is an absolute value, the sign (positive or negative) for efficiency is determined based on the
position of the spot on the graph as explained in Cerniglia (2012) and Županec, Radulović, Pribićević, Miljanović,
and Zdravković (2018). Positive values of efficiency indicate that the applied teaching approach resulted in higher
students’ achievement with low values of perceived mental effort. Therefore, this approach is considered suitable
for students. Involvement is a quantity that can also be calculated by knowing the standardized values of students’
P+R
achievement and perceived mental effort, as . I=
2
Results of Research
Students’ Achievement
Students’ mean achievement and standard deviation on pre-test for both groups are shown in Table 1.
Table 1. Students’ achievement on the pre-test.
Group M SD
C 9.86 3.49
E 10.37 3.03
The use of a one-way ANOVA showed that there was no statistically significant difference in students’ achieve-
ment in the pre-test F(df = 1, N = 112) = 0.693, p = .407 so the groups were considered as equal. The obtained result
showed that the students of both groups had similar previous knowledge about themes Pressure and Motion of the
body under the influence of gravity. Friction force. Knowledge in this area is important for a proper understanding
of the balance of the body. After equalization of groups, teaching with the use of different methods began. Table
2 shows students’ mean achievement and standard deviation on post-test for both groups.
Table 2. Students’ achievement on the post-test.
Group M SD
C 9.46 3.00
E 12.63 2.74
ANOVA showed a statistically significant difference in students’ achievement, depending on the applied
teaching instruction F(1,112) = 34.445, p < .0001, eta squared 0.237. The value of eta squared indicated the great
influence of the applied teaching instruction on students’ achievement. The obtained results showed that the
students in the E group accomplished higher achievement in the post-test than the students in the C group. This
difference was confirmed by the chi-square test, χ2 (df = 2, N = 113) = 28.40, p < .0001, V = 0.50. The distribution of
the achieved number of points in the post-test for both groups is shown in the form of Figure 2.
122
ISSN 2538–7138 /Online/ (P. 117-131)
67.9%
70%
57.9%
60%
50% 42.1%
Frequency
40%
30%
17.9%
20% 14.3%
10%
0.0%
0%
Up to 6 points From 7 to 12 points From 13 to 18 points
C E
Figure 2. The distribution of the achieved number of points in the post-test.
As can be seen from Figure 2, more students from E group than from C group achieved between 13 and 18
points. Also, more students from C group than from E group achieved less than 13 points. According to obtained
results students from E group achieved better results than students from C group. The value of Cramer-s V coef-
ficient indicated great efficiency of applied teaching approach.
In order to complete the data about the efficiency of applied teaching approach on students- achievement,
the paired-samples t-test was used. The achievement of the students in the E group from the initial (M = 10.51, SD
= 3.26) increased to (M = 12.63, SD = 2.74), t (df = 56) = 5.23, p < .0001. In the case of the students in the C group
there was no statistically significant difference between their achievement on the pre- and post-test, t (df = 55)
= 1.62, p = .111. The research also showed that there was no statistically significant difference in the students’
achievement in relation to students’ gender, F(1,112) = 0.675, p = .413. Girls achieved M = 11.36, SD = 3.21 while
boys achieved M = 10.85, SD = 0.41. Descriptive statistics showed that about 51% of girls and about 55% of boys
achieved from 7 to 12 points, and about 30% of boys and 40% of girls achieved 13 and more points. Although
there was a difference of about 10% in favor of girls, this difference is not statistically significant. With regard
to cognitive levels, a statistically significant difference was found in students’ achievement between groups for
all three levels, depending on the applied teaching approach. Figure 3 shows the comparative achievement of
the students in both groups according to the observed cognitive levels. One-way ANOVA showed a statistically
significant difference in students’ achievements, depending on the applied teaching instruction for the cognitive
level of remembering F(1,112) = 17.105, p < .0001, eta squared 0,134. The average achievement of the students
in the E group on this group of questions was M = 4.82, SD = 1.21, while for the students in the C group it was M
= 3.82, SD = 1.36. This difference was confirmed by the chi-square test, χ2 (df = 5, N = 113) = 17.24, p = .004, phi =
0.391. Within the group of questions related to the first cognitive level (knowledge of facts), it was expected that
the students provide answers to questions related to the definition of the resultant force, lever, buoyant force, and
units for torque. This group of questions tested their knowledge of basic definitions from a given field of physics.
The obtained data indicated the existence of oversight in adopting basic concepts of the student in the C group.
Without properly learned basic concepts, the higher cognitive levels cannot be successfully achieved, and the
formation of misconceptions is also possible.
There was a significant difference in the students’ achievements depending on the applied teaching approach
for the cognitive level of understanding F(1,112) = 14.654, p < .0001, eta squared 0,117. The students in the E group
achieved higher scores (M = 4.30, SD = 1.30) than the students in the C group (M = 3.29, SD = 1.51). This difference
123
(P. 117-131) ISSN 2538–7138 /Online/
was confirmed by the chi-square test, χ2 (df = 6, N = 113) = 16.43, p = .012, phi = 0.381. This group of questions
related to the second cognitive level (understanding) examined the comprehension of the basic connections be-
tween the studied basic concepts. For example, the students were required to notice the connection between the
buoyant force and the density of the fluid, then, the very effect of the force and the state of equilibrium and similar.
As it was the case in the previous two levels of knowledge, there was a significant difference in students’
achievements, depending on the applied teaching approach, for the cognitive level of applying F(1,112) = 25.812,
p < .0001, eta squared 0,189. The eta coefficient indicated the great influence of the applied teaching approach on
students’ achievement. The average achievement of the students in the E group on this group of questions was M
= 3.51, SD = 1.23, while for the students in the C group it was M = 2.36, SD = 1.18. This difference was confirmed by
the chi-square test, χ2 (df = 6, N = 113) = 22.876, p < .001, V = 0.450. Within this group of questions related to third
cognitive level (applying), the students were required to apply acquired knowledge on concrete examples. For
example, the traction engine power of a car that is moving uniformly in a straight line was given and the students
were supposed to determine the friction force that acts on the car; or a body was hung at some distance from the
axis of rotation of the lever and the task was to find the force needed in order to bring the lever back in the state
of equilibrium. Within the questions of this cognitive level students needed to apply the analysis, synthesis and
evaluation in order to successfully solve tasks.
80%
68.4%
70%
59.6%
53.6%
60%
49.1%
46.4%
45.6%
42.9%
50%
41.1%
Frequency
35.7%
40%
30.4%
26.3%
23.2%
23.2%
30%
21.1%
19.3%
20%
5.3%
5.3%
10%
3.6%
0%
C E C E C E
I cognitive level II cognitive level III cognitive level
Up to 2 points From 2 to 4 points From 4 to 6 points
Figure 3. Students’ achievement for different cognitive levels.
Mental Effort
One-way ANOVA was used to examine the statistical significance in the perceived mental effort of the students,
depending on the applied teaching instruction F(1,112) = 4.062, p = .046. The average perceived mental effort of
the students in the E group was M = 2.56, SD = 0.99; for the students in the C group it was M = 2.92, SD = 0.88. This
difference was confirmed by the chi-square test, χ2 (df = 4, N = 113) = 17.51, p < .001, V = 0.394. Figure 4 shows the
distribution of perceived mental effort for both groups.
124
ISSN 2538–7138 /Online/ (P. 117-131)
70%
60.7%
60%
50%
40%
Frequency
36.8%
29.8%
30%
19.6%
20% 15.8%
14.0%
10.7%
10%
5.4%
3.6% 3.5%
0%
very easy easy neither easy nor difficult very difficult
difficult
C E
Figure 4. Mental effort.
As seen in Figure 4, more students in the E group than in the C group perceived lower levels of mental effort.
This confirmed the initial hypothesis that there is a difference in the perceived mental effort among the groups
and that the students in the E group perceived lower levels of mental effort than the students in the C group.
The research did not show statistically significant gender difference in the perception of mental effort, F(1,112)
= 1.953, p = .165. It was found that the girls perceived the mental effort of M = 2.59, SD = 1.05, and the boys M =
2.84, SD = 0.86.
Observed at cognitive levels (Figure 5), the chi-square test showed that there is a statistical significance for all
three levels. For the cognitive level of remembering (χ2 (df = 4, N = 113) = 10.70, p < .05), it was shown that more
students in the E group than in the C group rated questions as very easy, while more students in the C group than
in the E group rated the same questions as neither easy nor difficult and very difficult. For the cognitive level of
understanding (χ2 (df = 4, N = 113) = 6.88, p = .142) it was shown that about half of the students in the E group
rated questions as (very) easy, while half of the students in the C group rated the same questions as neither easy
nor difficult. For the cognitive level of applying (χ2 (df = 4, N = 113) = 2.48, p = .648) it was shown that a few more
students in the E group than in the C group rated the questions as easy or neither easy nor difficult, while more
students in the C group than in the E group rated the same questions as difficult.
125
(P. 117-131) ISSN 2538–7138 /Online/
60%
51.8%
50%
42.9%
42.1%
37.5%
35.1%
40%
33.3%
32.1%
31.6%
30.4%
Frequency
26.3%
26.3%
30%
23.2%
19.3%
17.5%
15.8%
20%
12.5%
12.5%
12.3%
10.7%
10.7%
10.7%
10.5%
10.5%
10.5%
7.1%
7.1%
7.1%
10%
5.3%
3.6%
3.5%
0%
C E C E C E
I cognitive level II cognitive level III cognitive level
very easy easy neither easy nor difficult difficult very difficult
Figure 5. Mental effort for different cognitive levels.
Educational Efficiency and Students‘ Involvement
Efficiency and involvement can be calculated by knowing standardized value of students- achievement and
perceived mental effort. Efficiency is the quantity that indicates the value of students’ achievement and the mental
effort required to solve the knowledge test. The positive values of calculated students’ involvement indicate the
motivational effects of the applied teaching approach. Therefore, if greater students’ motivation is achieved, while
not specifying the type of motivation, when applying a teaching approach, then students will make more effort to
master the given material and thus achieve better results in the knowledge test. Figure 6 shows a diagram contain-
ing the standardized values of students’ achievement and perceived mental effort.
Figure 6. Educational efficiency and students’ involvement.
126
ISSN 2538–7138 /Online/ (P. 117-131)
Based on the obtained values of achievement and perceived mental effort, the values for educational efficiency
and students’ involvement were calculated. For the E group the gained values were EE = 0.468 and IE = 0.213, while
for the C group they were EC = -0.528 and IC = -0.164. With conventional approach, a negative involvement value
was achieved, which indicated a lower interest of students in resolving post-test tasks. A teaching approach involv-
ing the use of mind maps and constructivist learning environment caused a positive value of involvement which
indicated that students had expressed higher interest in the content itself, as well as in resolving the post-test tasks.
The acquired values of educational efficiency and students’ involvement indicated that the teaching approach
based on the application of mind maps is more suitable for students than the conventional teaching approach.
Discussion
The obtained results showed that the students taught the content of Statics by the use of mind maps achieved
better results, while perceiving lower levels of mental effort, than the students taught by conventional teaching
approach. That is, the educational efficiency of teaching with the use of mind maps was greater than the efficiency
of conventional teaching approach, as well as students’ involvement.
Numerous studies were carried out to examine the use of mind maps in different teaching subjects and they
gave similar results. Besides, various researchers indicated that teachers perceived benefits of using the conceptual
maps and mind maps. Nousiainen and Koponen (2011), Seyihoglu and Kartal (2010) and Ünal Çoban (2013) pre-
sented the teachers’ opinion on the application of mind maps. The results of those studies showed that teachers
had a positive opinion on the application of mind maps. Keleş in his research (2012) compared conceptual maps
and mind maps. More than half of the respondents said that mind maps were more useful. They allow students to
express themselves, enhance creativity and increase the interest for learning.
The results of this research are divided into three parts. The first part consists of the results related to knowledge
tests. Within this part statistically significant influence of the applied teaching approach on the students’ achieve-
ment was obtained. Namely, the students of the E group showed significantly higher achievements in the post-test
(Figure 2). Significant differences in students- achievements were achieved both on the overall knowledge test
and on individual cognitive levels. The obtained result is in accordance with the results presented by Akinoglu and
Yasar (2007). This research shows the positive influence of using mind maps on students’ achievement and their
attitudes. Similar results were obtained in the research carried out by Abi-El-Mona and Adb-El-Khalick (2008). In this
research, eighth-grade students who applied mind maps showed a statistically significantly higher achievement
on the knowledge test than the students from the control group. This significant difference was reflected on all of
the target categories (conceptual understanding and practical reasoning) and levels of achievement. Importantly,
it was shown that mind maps are useful on higher cognitive levels as well. They help students to memorize facts,
but also to understand content and to be prepared to apply knowledge. It was shown that sixth grade students in
a school in Turkey improved their academic achievement in science class with the use of mind maping (Çömek,
Akinoğlu, Elmaci, & Gündoğdu, 2016). Besides, researchers in Korea showed that the use of mind maps in science
teaching improved junior high school students’ creative thinking skills (Yoon & Kang, 2015). Although physics
contents are taught within Science, the findings about the efficiency of the use of mind maps for teaching different
natural sciences (for example, biology, physics, chemistry) can be useful. Based on that insight, science teachers
can improve teaching practice by selecting and planning appropriate teaching approach for particular content.
The second part within the research results was related to examining mental effort of students. The results
showed that the students in the E group perceived a lower mental effort compared to the students in the C group.
This applies both to the general and to the individual cognitive levels. Therefore, teaching approach which includes
mind maps is more convenient for students. In the Republic of Serbia, only three studies examined the mental effort
of students in the physics teaching: Radulović (2015), Radulović, Stojanović and Županec (2016) and Radulović and
Stojanović (2015) and none of them was carried out in a primary school. These studies showed that the multimedia
environment has a positive influence on decreasing perceived mental effort. Therefore, mind maps were applied
in the PowerPoint program. The PowerPoint program was used for an easier visualization of mind maps, while the
maps themselves were used as a teaching approach. Kwon, Shin and Park (2018) found that students’ higher levels
of cognitive engagement are facilitated with both instructor-provided and student-generated graphic organizers.
Despite all the benefits of using mind maps, including students perceiving lower levels of mental effort, some findings
showed that this approach did not have any effect on students’ attitude in science education (Çömek et al., 2016).
The third part within the research results was related to the calculation of the educational efficiency and
127
(P. 117-131) ISSN 2538–7138 /Online/
students’ involvement in the context of applied teaching approaches. The values of educational efficiency and
students’ involvement undoubtedly indicated the positive influence of the applied teaching approach on students’
performance. Higher students’ motivation for learning physics contents resulted in better results in post-test. Parikh
(2016) showed that the use of mind maps in social sciences is an efficient teaching approach. Similarly, Kontrova
(2014) showed that mind mapping is an efficient tool in mathematics education. Moreover, it was shown that the
use of mind maps can effect students’ motivation (Jones, Ruff, Snyder, Petrich, & Koonce, 2012).
In order to gain insight into the use of teaching approach based on the use of mind maps in physics class,
it is necessary to consider all of the aforementioned results. Since all three parts described within the research
results show that this approach is useful, it can be suggested that this approach is appropriate for teaching phys-
ics contents. Moreover, since it was shown that there was no statistically significant difference in the students’
achievement nor mental effort in relation to students’ gender, the researchers propose this teaching approach as
equally suitable for girls as for boys.
Conclusions
The advantages of the use of mind maps in different teaching subjects are well known, but this research fo-
cuses on teaching physics to primary school students, making it specific. Primary school students are commonly
taught physics contents, together with other natural sciences within one teaching subject – Science. In case when
physics is being taught as a specific teaching subject it is not popular. In order to overcome primary school students’
lack of motivation for learning physics, which they perceive as a difficult subject, and to achieve better results, an
appropriate teaching approach should be used. The implementation of mind maps as a teaching approach leads
to an increase in students’ achievement while at the same time mental effort is decreased. The values of mental
effort indicate that the applied instruction is suitable for students.
The limitations of this research are as follows: the sample comprised only sixth-grade students, the research
was conducted in already-formed classes rather than in randomly chosen groups, and the research covered only
one topic.
It is important to examine the efficiency of new teaching approaches and present it clearly to teachers so that
they have a better insight into the possibilities of the approach itself. If teachers are informed about the results of
contemporary pedagogical research, they can apply suggested approaches, which can result in the increase of
students’ motivation and achievement. The results of the research have shown that the application of mind maps
can serve as a useful tool in physics education by helping students understand concepts more easily. It can be
suggested that teachers should advise students to use mind maps in order to enhance students’ performance and
motivation for learning, while decreasing mental effort. The teachers can be successful in using the approach based
on the use of mind maps only if they have knowledge about it. Since many teachers are not well prepared to use
mind maps, it can be suggested that additional training for teachers can be useful. Besides, teachers do not often
have the opportunity to learn about mental effort. Since that knowledge can improve teaching practice, it could
be useful to present the teachers with the information about how students perceive mental effort. Although the
results of this and other studies on using mind maps are promising, they are not conclusive. Consequently, more
research should be conducted in order to test the effect of mind mapping on the larger number of students, in
different types of schools, and for different age groups.
Acknowledgements
This research was funded by Grant No. 179010 financially supported by the Ministry of Education, Science
and Technological Development of the Republic of Serbia.
References
Abi El Mona, I., & Adb El Khalick, F. (2008). The influence of mind mapping on eighth graders’ science achievement. School Science
and mathematics, 108 (7), 298-312.
Adodo, S. O. (2013). Effect of mind-mapping as a self-regulated learning strategy on students’ achievement in basic science and
technology. Mediterranean Journal of Social Sciences, 4(6), 163-172. doi:10.5901/mjss.2013.v4n6p163.
Akinoglu, O., & Yasar, Z. (2007). The effects of note taking in science education through the mind mapping technique on students’
attitudes, academic achievement and concept learning. Journal of Baltic Science Education, 6 (3), 34-42.
128
ISSN 2538–7138 /Online/ (P. 117-131)
Brinkmann, A. (2003). Graphical knowledge display–mind mapping and concept mapping as efficient tools in mathematics
education. Mathematics Education Review, 16, 35-48.
Buzan, T., & Buzan, B. (1996). The mind map book: How to use radiant thinking to maximize your brain’s untapped potential. New
York: Dutton Adult.
Cerniglia, A. J. (2012). Instructional Efficiency and Learner Involvement. Doctoral work. Retrieved, from http://andrewcerniglia.
com/?p=411#comments
Chandler, P., & Sweller, J. (1991). Cognitive load theory and the format of instruction. Cognition and Instruction, 8, 293–332.
Choi, H. H., Van Merriënboer, J. J., & Paas, F. (2014). Effects of the physical environment on cognitive load and learning: towards
a new model of cognitive load. Educational Psychology Review, 26(2), 225-244.
Cvjetićanin, S., Obadović, D., & Rančić, I. (2015). The efficiency of student-led and demonstration experiments in initial physics-
chemistry education in primary school. Croatian Journal of Education, 17 (3), 11-39.
Çömek, A., Akinoğlu, O, Elmaci, E., & Gündoğdu, T. (2016). The effect of concept mapping on students’ academic achievement
and attitude in science education. Journal of Human Sciences, 13(1), 348-363.
Čaluković, N. (2014/a). Fizika 7, Zadaci, testovi i laboratorijske vežbe za sedmi razred osnovne škole [Physics 7, Tasks, tests and
laboratory exercises for the 7th grade of elementary school] Beograd: Krug.
Čaluković, N. (2014/b). Fizika 7, udžbenik za 7. razred osnovne škole [Physics 7, a textbook for the 7th grade elementary school].
Beograd: Krug.
Davies, M. (2011). Concept mapping, mind mapping and argument mapping: what are the differences and do they matter? Higher
education, 62(3), 279-301. doi: 10.1007/s10734-010-9387-6.
Dhindsa, H. S., & Anderson, O. R. (2011). Constructivist-visual mind map teaching approach and the quality of students’ cognitive
structures. Journal of Science Education and Technology, 20(2), 186-200. doi 10.1007/s10956-010-9245-4
Douma, M., Ligierko, G., & Romano, J. (2009, August). Creating online mind maps and concept maps. Paper presented at the 25th
Annual Conference on Distance Teaching & Learning. Madison, WI.
Eppler, M. J. (2006). A comparison between concept maps, mind maps, conceptual diagrams, and visual metaphors as complemen-
tary tools for knowledge construction and sharing. Information visualization, 5(3), 202-210. doi:10.1057/palgrave.ivs.9500131.
Goodnough, K., & Woods, R. (2002, April). Student and teacher perceptions of mind mapping: A middle school case study. Paper
presented at the American Educational Research Association Annual Meeting, New Orleans, LA.
Jones, B. D., Ruff, C., Snyder, J. D., Petrich, B., & Koonce, C. (2012). The effects of mind mapping activities on students’ motivation.
International Journal for the Scholarship of Teaching and Learning, 6(1), 1-21.
Kalyuga S. (2009). Cognitive load factors in instructional design for advanced learners. New York: Nova Science.
Kapor, D., & Šetrajčić, J. (2009). Fizika [Physics]. Beograd: Zavod za udžbenike.
Kedaj, P., Pavlíček, J., & Hanzlík, P. (2014). Effective mind maps in e-learning. Acta Informatica Pragensia, 3(3), 239-250. doi:
10.18267/j.aip.51.
Keleş, Ö. (2012). Elementary teachers’ views on mind mapping. International Journal of Education, 4(1), 93-100. doi:10.5296/ije.
v4i1.1327
Kirschner, P. A. (2002) Cognitive load theory: Implications of cognitive load theory on the design of learning. Learning and Instruc-
tion, 12, 1–10. http://dx.doi.org/10.1016/S0959-4752(01)00014-7.
Kontrova, L. (2014). Mind mapping as efficient tools in mathematics education. Komunikacie, 16(3), 74-78.
Kovačević, J., & Segedinac, M. (2007). A contribution to the educational reform: Mind maps. Zbornik Matice srpske za drustvene
nauke, (122), 191-201.
Kwon, K., Shin, S., & Park, S. J. (2018). Effects of graphic organizers in online discussions: Comparison between instructor-provided
and student-generated. Educational Technology Research and Development, 66(6), 1479-1503.
Márquez, T., Manuel, L., Llinás, J. G., & Macías, F. S. (2017). Collaborative learning: Use of the Jigsaw technique in mapping concepts
of physics. Problems of Education in the 21st Century, 75 (1), 92-101.
Meyer, R. (2010). Knowledge visualization. In D. Baur, M. Sdlmair, R. Wimmer, Y-X. Chen, S. Streng, S. Boring, A. De Luca, & A. Butz
(Eds.), Trends in Information Visualization (pp. 23-30). Munich: University of Munich, Department of Computer Science,
Media Informatics Group.
Miller, G. A. (1955). The magical number seven, plus or minus two some limits on our capacity for processing information. Psy-
chological Review, 101(2), 343-352.
Mitrović, M. (2013/a). Fizika 7, udžbenik za 7. razred osnovne škole [Physics 7, a textbook for the 7th grade elementary school].
Beograd: Saznanje.
Mitrović, M. (2013/b). Praktikum Fizika 7, zbirka zadataka i eksperimentalnih vežbi za 7. razred osnovne škole [Practicum of Physics
7, a collection of tasks and experimental exercises for the 7th grade elementary school]. Beograd: Saznanje.
Novak, J. (1998). Learning, creating and using knowledge. Concept maps as facilitative tools in schools and corporations. Mahwah:
Lawrence Erlbaum.
Nousiainen, M. (2012). Making concept maps useful for physics teacher education: Analysis of epistemic content of links. Journal
of Baltic Science Education, 11 (1), 29-42.
Nousiainen, M., & Koponen, I. T. (2011). Pre-service physics teachers’concept maps representing relational structure of physics
concepts. Journal of Baltic Science Education, 10 (3) 183-194.
OECD (2012), PISA 2009 Technical Report, PISA, OECD Publishing. http://dx.doi.org/10.1787/9789264167872-en.
Paas, F. G., & Van Merriënboer, J. J. (1993). The efficiency of instructional conditions: An approach to combine mental effort and
performance measures. Human Factors: The Journal of the Human Factors and Ergonomics Society, 35 (4), 737-743.
129
(P. 117-131) ISSN 2538–7138 /Online/
Paas, F., Tuovinen, J. E., Tabbers, H., & Van Gerven, P. W. (2003). Cognitive load measurement as a means to advance cognitive
load theory. Educational Psychologist, 38(1), 63-71.
Paas, F. G., & Van Merriënboer, J. J. (1994). Instructional control of cognitive load in the training of complex cognitive tasks. Edu-
cational Psychology Review, 6(4), 351-371.
Paas, F., Tuovinen, J. E., Van Merrienboer, J. J., & Aubteen Darabi, A. (2005). A motivational perspective on the relation between
mental effort and performance: Optimizing learner involvement in instruction. Educational Technology Research and
Development, 53(3), 25-34.
Parikh, N. D. (2015). Mind map and concept map as complementary tools for teaching. The International Journal of Indian Psy-
chology, 2(4), 147-158.
Parikh, N. D. (2016). Effectiveness of teaching through mind mapping technique. The International Journal of Indian Psychology,
3(3), 148-156.
Plass, J.L., Moreno, R. & Brünken, R. (2010). Cognitive load theory. Cambridge: University Press.
Provasnik, S., Malley, L., Stephens, M., Landeros, K., Perkins, R., & Tang, J.H. (2016). Highlights From TIMSS and TIMSS Advanced
2015: Mathematics and Science Achievement of U.S. Students in Grades 4 and 8 and in Advanced Courses at the End of High
School in an International Context (NCES 2017-002). U.S. Department of Education, National Center for Education Statistics.
Washington, DC. Retrieved [date] from http://nces.ed.gov/pubsearch.
Radulović, B. (2015). Kognitivno opterećenje učenika drugog razreda gimnazija u nastavi fizike [Cognitive load of the second grade
physics students at high school]. Doctoral dissertation, Novi Sad: University of Novi Sad, Faculty of Sciences, Department
of Physics.
Radulovic, B., & Stojanovic, M. (2015). Determination instructions efficiency of teaching methods in teaching physics in the case
of teaching unit “Viscosity. Newtonian and Stokes Law”. Acta Didactica Napocensia, 8(2), 61.
Radulović, B., Stojanović, M., & Županec, V. (2016). The effects of laboratory inquiry-based experiments and computer simulations
on high school students ‘performance and cognitive load in physics teaching. Zbornik Instituta za pedagoska istrazivanja,
48(2), 264-283. doi: 10.2298/ZIPI1602264R.
Seyihoglu, A., & Kartal, A. (2010). The views of the teachers about the mind mapping technique in the elementary life science
and social studies lessons based on the constructivist method. Educational Sciences: Theory and Practice, 10(3), 1637-1656.
Siew, N. M., & Mapeala, R. (2016). The effects of problem-based learning with thinking maps on fifth graders’ science critical
thinking. Journal of Baltic Science Education, 15(5), 602-616.
Soika, K., & Reiska, P. (2014). Using concept mapping for assessment in science education. Journal of Baltic Science Education,
13(5), 662-673.
Stanivuk, Lj., Bogdanović, I., & Skuban, S. (2015). Stavovi učenika tehničke škole o učenju u nastavi fizike [The attitudes of
technical-school students about learning physics]. Pedagogija, 70(3), 327-337.
Stanivuk, Lj., Skuban, S., & Bogdanović, I. (2015). Stavovi učenika tehničke škole o oceni iz fizike I motivacija za učenje sadržaja
fizike [The attitudes of technical-school students towards assessments and motivation for studying physics]. Nastava i
vaspitanje, 64(4), 735-751.
Sweller, J., Ayres P. & Kalyuga S. (2011). Cognitive Load Theory. New York: Springer.
Sweller, J. (1994). Cognitive load theory, learning difﬁculty, and instructional design. Learning and Instruction, 4, 295–312 http://
dx.doi.org/10.1016/0959-4752(94)90003-5
Tulving, E. (1985). How many memory systems are there? American Psychologist, 40, 385-398.
Ünal Çoban, G. (2013). The effects of inquiry supported by argument maps on science process skills and epistemological views
of prospective science teachers. Journal of Baltic Science Education, 12(3), 271-288.
Usta, N. D., & Ültay, N. (2016). Prospective chemistry teachers’ abilities of creating concept maps: Hydrocarbons example. Journal
of Baltic Science Education, 15(1), 58-67.
Van Merriënboer, J. J. G. & Sweller, J. (2005). Cognitive load theory and complex learning: Recent developments and future direc-
tions. Educational Psychology Review, 17(2), 147–177. doi: 10.1007/s10648-005-3951-0.
Yoon, H., & Kang, S. (2015). The effect of integrated mind map activities on the creative thinking skills of 2nd year students in
junior high school. Journal of the Korean Chemical Society, 59(2), 164-178.
Zouhor, A. M. Z., Bogdanović, I., Skuban, S., & Pavkov-Hrvojević, M. (2017). The effect of the modified Know-Want-Learn strategy
on sixth-grade students’ achievement in physics. Journal of Baltic Science Education, 16(6), 946-957.
Županec, V., Radulović, B., Pribićević, T., Miljanović, T., & Zdravković, V. (2018). Determination of educational efficiency and stu-
dents’ involvement in the flipped biology classroom in primary school. Journal of Baltic Science Education, 17(1), 162-176.
130
ISSN 2538–7138 /Online/ (P. 117-131)
Appendix
A mind map: the meaning of the term equilibrium of the body.
Received: January 17, 2018 Accepted: January 30, 2019
Zvezdan Z. Gagić MSc, PhD Student, Faculty of Sciences, University of Novi Sad, Trg
Dositeja Obradovića 4, Novi Sad, Serbia.
E-mail: zvezdangagic@gmail.com
Sonja J. Skuban PhD, Associate Professor, Faculty of Sciences, University of Novi Sad, Trg
(Corresponding author) Dositeja Obradovića 4, Novi Sad, Serbia.
E-mail: sonja.skuban@df.uns.ac.rs
Branka N. Radulović PhD, Research Associate, Faculty of Sciences, University of Novi Sad, Trg
E-mail: branka.radulovic@df.uns.ac.rs
Maja M. Stojanović PhD, Full Professor, Faculty of Sciences, University of Novi Sad, Trg
E-mail: maja.stojanovic@df.uns.ac.rs
Olivera Gajić PhD, Full Professor, Faculty of Philosophy, University of Novi Sad, Dr
Zorana Đinđića 2, Novi Sad, Serbia.
E-mail: gajico@ff.uns.ac.rs
131
ROLE AND MEANING OF
FUNCTIONAL SCIENCE,
TECHNOLOGICAL AND
ENGINEERING LITERACY IN
PROBLEM-BASED LEARNING
Abstract. Literacy, in particular functional Zvonka Cencelj,

literacy in various fields and especially in Metka Kordigel Aberšek,
the field of STEM, is becoming an increasing Boris Aberšek,
problem in modern-day society. The ques- Andrej Flogie
tion arises, can the school system develop
the kind of functional literacy in students,
which allows them to fulfil, in real life, their
personal and professional needs related to
the competence of reading and creating
written texts from the field of science, tech- Introduction
nology and engineering?
The present research aimed to explore how In today’s society literacy, especially functional literacy, is becoming a
students in today’s schools are trained in serious issue (OECD, 2000, PISA, 2015). Today’s society requires the educa-
tional system to train as many people as possible to achieve higher cognitive
functional literacy, especially in the fields
ways of thinking (OECD, 2009, 2011, 2014), therefore it is necessary to take
of technology and engineering, and what
into consideration the distinctive features of the educational field, where
kind of literacies in the field of STEM they the student is not only the object of teaching but also the subject of its own
are able to (or should be able to) master control and change, and structuralism as the fundamental premise must be
competently. The present research showed replaced with cognitive science (Bermudez, 2010, Aberšek, Borstner & Bregant,
that students achieve relatively poor 2014). It should be emphasized that acquiring knowledge is a complex and
deliberate process, in which conclusions, summaries and predictions are
results in the area of functional literacy,
formed, and the knowledge acquired in this way enables the anticipation
both regarding their science literacy, and
of situations and the choice of appropriate reactions in these situations. The
especially their technology and engineer- starting point for cognitive psychologists in trying to provide answers to the
ing literacy, which is a result of a lack of questions of how the human brain memorizes and learns, therefore, is to look
competence on behalf of mother-tongue for an internal mechanism that controls human thinking and the acquisition
teachers to develop this kind of functional of knowledge, whereby functional literacy is of key importance (OECD, 2000,
PISA, 2015, PIRLS, 2016).
literacy. Functional literacy should be
In today’s science, reading is understood as an active process in which
developed by teachers of individual areas
the reader constructs meaning from a text by assimilating information from
of STEM subjects. the text with their existing background knowledge and previous knowledge
Keywords: engineering functional literacy, about the subject of the text. In doing so, sophisticated readers apply a num-
functional literacy, key competence, prob- ber of cognitive strategies (Fisher, Frey & Lapp, 2009) in order to decide what
lem solving, science functional literacy. is important for them in the text, organize what they have read into networks
of hierarchical concepts, relate new knowledge to what they already know,
and evaluate what they have read. These strategies include: asking ques-
Zvonka Cencelj tions, developing connections, and, which is especially important, creating
Primary School Vransko-Tabor, Slovenia
Metka Kordigel Aberšek, inferential bridges, which refer to the forming of logical connections between
Boris Aberšek, Andrej Flogie information in the text (Aberšek et al, 2017).
University of Maribor, Slovenia There is no doubt about the above mentioned in the science of read-
ing, and so the question arises, to what extent school curricula are able to
132
ISSN 1648–3898 /Print/ ROLE AND MEANING OF FUNCTIONAL SCIENCE, TECHNOLOGICAL AND ENGINEERING
ISSN 2538–7138 /Online/ (P. 132-146)
develop such reading competence in students. It seems, namely, that school systems are still not done away with
the idea of functional literacy as a relatively monolithic competence. In today’s world, every expert/scientific field
has developed its own specific text types. Consequently, for studying chemistry, biology and physics textbooks
(e-textbooks), specific functional literacies are needed. The same could be established for studying engineering
and technical texts. For these, too, the reader/student needs specific functional literacies. All these literacies are
covered when speaking about functional literacy in the field of STEM.
STEM Literacy/STEM Literacies
STEM literacy is a term which became topical in the last few decades, as it becomes more and more apparent
that general functional literacy at higher levels overlaps with competences of the so-called subject-specific (con-
tent area) literacies. This doctrine focuses on science, engineering and technological literacy (STE literacy), which
encompasses ethical use of science, knowledge about the rules of science, technology and engineering, and the
ability to understand and use engineering and technology as applied sciences. Being a relatively new term, STEM
literacy is still to be defined precisely. One of the most frequently used definitions is the following: “STEM literacy
is the ability to identify, apply, and integrate concepts from science, technology, engineering, and mathematics to
understand complex problems and to innovate to solve them” (Balka, 2011: 7).
Such an understanding of STEM literacy, however, appears too narrow, as it focuses primarily on the scientific
and developmental engineering aspect of STEM literacy. From the perspective of the general population and the
idea of lifelong learning (which is necessary in the fields of engineering and technology, as these two disciplines
are evolving at an exponential rate and have a crucial impact on an individual’s everyday life), and, of course, for
the purpose of education and acquiring knowledge and competence in schools, the above definition is incomplete
for at least two reasons:
•• because it is based on the assumption about a single type of functional literacy, which should therefore
apply to all natural science disciplines (from biology to chemistry, physics, applied science, etc.), and
•• because it fails to include the notion of competences, related to literacy in the strict sense of the word,
i.e., the ability to communicate about scientific phenomena in general, and especially the ability to
communicate through the written channel.
The problem arising from the assumption about STEM literacy as a monolithic competence was quickly noted,
and soon after the turn of the millennium definitions of literacy began to emerge that relate to the individual
scientific disciplines/areas of the acronym STEM. Zollman (2012) defined four types of literacies: scientific literacy,
mathematical literacy, engineering literacy and technological literacy. In this context he described engineering
literacy as the ability to systematically and creatively apply scientific and mathematical principles to practical ends
such as the design, manufacture, and operation of efficient and economical structures, machines, processes, and
systems (OECD, 2011a). And he described technological literacy as the ability to demonstrate creativity and innova-
tion, communicate and collaborate, conduct research and use information, think critically, solve problems, make
decisions, and use technology effectively and productively (ISTE, 2000).
At the end of the second decade of the new millennium, therefore, it seems that science can no longer refer
to one single STEM literacy, but rather to several STEM literacies. While science, mathematical, engineering and
technological literacy may well refer to competences sharing common roots and a set of common attributes, they
are ultimately different kinds of literacy competences, which serve different goals, lead to different results and
must therefore be developed systematically, each of them separately (Zollman, 2012).
The above-mentioned definition of technological literacy, as developed by the International Society for Tech-
nology and Education, is a step closer to the solution of the second problem, which was mentioned in relation to
the original definition of STEM literacy. It focuses attention on the process of knowledge acquisition, as the term
“ability to communicate” the knowledge and problem solutions in the field of technology suggests that today stu-
dents acquire information and comprehensive knowledge (at school) mainly by looking for it in relevant sources.
This kind of teaching is the only way to achieve the goals mentioned in the definition of engineering and
technological literacy, namely, to develop creativity and innovation, to collaborate and to conduct research in the
framework of solving didactical problems by using technology effectively and productively. Because of this, in
problem-based learning situations students can develop the ability to independently deal with science, engineering
133
(P. 132-146) ISSN 2538–7138 /Online/
and technology-related texts, the ability to understand such texts, and also the ability to use and critically evaluate
them. In other words, in a problem-solving oriented didactic approach to technical education, students need to
develop their technological and engineering literacy in a stricter communication-related sense of the word – what
they need is functional literacy in the field of engineering and technology.
At first glance, it seems that everything is now in place, the school curriculum will enable students to become
literate in all areas covered by the curricula of their elementary and further education. But a more detailed reflec-
tion quickly points to some issues. The initiator of developing disciplinary literacies (and in this context, also STEM
literacies) is the International Literacy Association, ILA (formerly the International Reading Association). The ILA
primarily addresses L1 teachers (teachers of mother tongue) by providing them with “a set of guidelines for the
implementation of the English Language Arts Common Core State Standards. Among these guidelines are recom-
mendations for the use of challenging texts, teaching basic skills (such as phonological awareness and fluency),
focusing on comprehension, and developing writing skills and acquiring disciplinary literacy” (IRA, 2012). The
problem in this context is not the purpose of developing disciplinary literacies, but in defining what a challenging
text is in the field of the individual discipline. Or, more specifically, to what extent can mother-tongue teachers
understand and critically evaluate demanding texts from the fields of physics, chemistry, engineering, technology?
Believing that this could be done by mother-tongue teachers who definitely do not possess enough knowledge
in the areas of physics, chemistry, engineering, or technology, is simply naive.
The present research focused on the problem of STE(M) literacy in the Slovene compulsory school curriculum.
The central research question was the question of whether a L1 teacher (mother-tongue teacher), who is respon-
sible for developing general functional literacy, can also develop particular functional literacies. The question was
analysed on the case of science, technology and engineering literacy (STE). In this research functional literacy in
the field of engineering and technology (and, by way of analogy, also in other fields) is understood as:
1. Information literacy in the field of STE, which is defined as the ability to find and manage information,
and the ability to critically evaluate and ethically apply that information to solve a problem;
2. Functional reading literacy in the field of STE, which refers to
•• reading an expository and explanatory text from the field of (S)TE, understanding it (in the verbal
and visual code), and using the knowledge acquired in this way to solve problems,
•• reading/writing text types such as description of a procedure, instructions for use, building or
manufacturing instructions, understanding such texts and use what was read to successfully
and safely use a new product, make a product, assemble furniture, etc. (Aberšek, et al., 2014,
Dolenc, Aberšek, Kordigel Aberšek, 2015).
“In today’s world, to be literate requires students to be able to use and understand vocabulary specific to each
domain, to critically interpret and analyse multiple types of texts, and to express those understandings in creative
ways” (Lefever-Davis, Pearman, 2015). This kind of literacy, as defined by Lefever-Davis and Pearman, cannot be
taught by mother-tongue teachers, despite the fact that teaching reading and writing (of all kinds of texts) is their
responsibility in the majority of existing school systems. As results of this research point out, the mother tongue-
teacher does not have the competence to develop all STEM literacies.
General Background
Of particular interest to this research was the question of how worldwide education systems, in which only
mother-tongue teachers are responsible (as part of implementing the mother tongue and literature curriculum)
for the development of reading/learning strategies and functional reading, meet the requirements for functional
literacy in the field of STE. As part of the research, the reading speed and reading comprehension of technical
expository texts and procedural texts (instructions for use and manufacturing instructions), was measured. The
target groups in this research were primary school students of the sixth and seventh grades (age 13-14 years). At
this age, students should already be able to use reading to search for information, read critically and learn inde-
pendently with the help of reading.
134
ISSN 2538–7138 /Online/ (P. 132-146)
The main objectives of this research were:

•• To analyze the state of reading comprehension and the speed of reading of expository texts from the
field of engineering and technology;
•• To analyze the state of reading comprehension and the speed of reading of manufacturing instructions
for a product;
•• To analyze the comparison of the state of reading comprehension and reading speed between the
two types of texts;
•• To analyze the state of reading comprehension and reading speed and use it as a basis for defining the
level of (S)TE functional literacy.
The basic research method applied was mainly a quantitative method of pedagogical research, while a survey
and action research were used to collect data. Basic information about the students was obtained through the use
of a questionnaires and a non-experimental research.
Sample Selection
The pilot research was carried out in classes of ‘Science and Technology’ (ST classes), as part of a thematic area
called ‘Making products from paper materials’. Science and Technology is a compulsory subject in Slovene com-
pulsory education and is taught in the sixth, seventh and eighth grades. The pilot research included 108 students
from a smaller town, settled in a peripheral region in Slovenia, who attended the sixth or seventh grade at the same
primary school in the school year 2017/2018. Out of these, 68 were students of three divisions of the sixth grade
(63%) and 40 were students of two divisions of the seventh grade (37%). There were 18 more sixth-grade students
in comparison to the number of seventh-grade students (16.7%). According to gender, 55 boys (51%) and 53 girls
(49%) were included in the research.
Procedures and Instruments
The research measured the reading speed, reading comprehension and effective reading speed of reading
technical expository texts and manufacturing instructions. Students’ knowledge and the efficiency of their work were
tested by quantitatively measuring and calculating (according to eq. 1 and 2) reading time and reading understand-
ing for expository texts and for manufacturing instructions in the field of engineering and technology education.
The research task was the analysis of overall results in reading speed and understanding of technical expository
texts and manufacturing instructions, which provided an insight into the level of functional technological and
engineering literacy on the sample included in this survey.
Regarding the analysis of reading speed, the research presented:
•• types of readers from the perspective of the speed of reading technical expository texts, manufacturing
instructions, and reading both text types,
•• differences in reading speed from the perspective of reading technical expository texts, manufacturing
instructions, and both text types.
The students were also tested for their effective reading speed, which depends on the number of read words
per minute and the degree of understanding the read text. The research presented:
•• types of readers from the perspective of effective reading speed in reading a technical expository text,
manufacturing instructions, and both text types,
•• differences in effective reading speed when reading a technical expository text, manufacturing instruc-
tions, and both text types.
After discussing and coordinating with teachers of the school subject ‘Slovene language and literature’, stu-
dents learned about different reading strategies on a variety of examples in their mother-tongue classes. After
learning different reading strategies, a questionnaire was carried out in order to identify the existing situation – the
ability of transferring the general functional literacy acquired in mother-tongue classes to the level of functional
literacy in the field of engineering and technology. Using scientific measuring instruments, the reading time and
understanding of texts was measured, and the effective reading speed was calculated.
In order to test the speed of reading and reading comprehension, a questionnaire for different difficulty levels
was prepared. The questions were related to the technical expository text and the manufacturing of the product. To
135
(P. 132-146) ISSN 2538–7138 /Online/
evaluate the students’ responses and measure the reading time, an expert evaluation group was created, consisting
of two engineering and technology teachers and two mother-tongue teachers.
The expert evaluation group reviewed the student’s answers to the posed questions, which were related to
reading technical expository texts and manufacturing instructions. The degree of correctness of the answers was
coordinated by the evaluation group following a mutual consultation. In order to ensure objectivity, the question-
naire contained only basic information about the student (student identification number, gender and grade). In this
way, members of the evaluation group were unable to include any personal views in the evaluation of the students’
responses. This expert evaluation group also participated in the preparation of the questionnaire for measuring
reading speed, reading comprehension and effectiveness of reading, which were adopted from international stud-
ies such as OECD, PIRLS, PISA (OECD, 2000, 2011, 2014, PIRLS, 2016, PISA 2006, 2015), thus ensuring the validity of
the scientific instrumentation used. The measuring instrument contained three main sets.
1. Student information (gender, grade)

2. Reading speed test and text comprehension test:
•• speed of reading (HBS) and reading comprehension (VBS) test for a technical expository text,
•• speed of reading (HBN) and reading comprehension (VBN) test for instructions for manufactur-
ing a product,
3. Speed and effectiveness of reading technical expository texts (EBS) and manufacturing instructions
for a product (EBN).
Data Analysis
The reading speed was tested by measuring the time needed to read a technical expository text and the time
needed to read manufacturing instructions for a product. All the students began to read at the same time. As the
students finished reading, the precise time was checked, and the measurements (individual reading times) were
recorded. The number of read words per minute was calculated by calculating the quotient between the number
of words in the text and reading time in minutes:
On the basis of studying relevant literature (Buzan, 2009; Schmitz, 2012), groups (types) of readers were
formed according to the number of words read per minute. When forming the groups, the students’ age, their
level of development and the difficulty level of the text were also taken into account. Depending on the degree of
difficulty of the text and the age of readers, they were categorized into four groups (Table 2).
Table 2. Types of readers according to the number of words read per minute (adapted from Buzan, 2009;
Schmitz, 2012).
Reader Reading speed (wpm)
1. Very slow up to 49
2. Average from 50 to 79
3. Fast from 80 to 110
4. Very fast above 111
Since in addition to being fast, a good reader also has to be good at understanding and remembering what
they read, the number of read words per minute (Table 2), and the percentage of correct answers (Table 3), were
used to calculate the effective reading speed (the product of the number of read words per minute and the per-
centage of correct answers), on the basis of which four types of readers were discerned (Table 4).
136
ISSN 2538–7138 /Online/ (P. 132-146)
Table 3. Types of readers according to the level of text understanding.
Number Reader Understanding (%)
1. Poor up to 44%
2. Average from 45% to 60%
3. Functionally literate from 61% to 75%
4. Very good above 76%
Table 4. Types of readers according to effective reading speed.
Number Reader Effective reading speed
1. Poor up to 22.5
2. Average from 22.6 to 48
3. Functionally literate from 48.1 to 82.5
4. Very good above 82.6
The level of understanding in reading technical expository texts and manufacturing instructions was tested
using questions related to the text that was read.
1. Reading a technical expository text
The text for testing the speed of reading and reading comprehension of technical expository texts contained
682 words and 10 photographs. The text included a description of the procedure for making a wheat starch paste
adhesive, information about paper materials, and 9 words, which were new to six-grade students.
2. Reading manufacturing instructions
The text contained 568 words with added 32 photographs showing the tools and materials needed for mak-
ing (manufacturing) the product and the individual phases of the procedure.
According to the total number of points achieved and the difficulty level, the expert evaluation group was
designed, based on a comparison with rating scales of other expert teams, a four-level scale to measure the practi-
cal understanding of manufacturing instructions was formed (Table 5).
Table 5. Types of readers according to connection between theory and practice.
Reader Percentage of acquired points (v %)
1. Poor up to 50%
2. Satisfactory from 51% to 70%
3. Good from 71% to 90%
4. Very good above 91%
Microsoft Excel 2016 was used to organize the obtained data. For the statistical processing of the organized
data, the SPSS 22.00 software package was used.
137
(P. 132-146) ISSN 2538–7138 /Online/
Research Results
The research results, i.e., reading speed, reading comprehension and effective reading speed of reading
technical expository texts and manufacturing instructions, are presented in this chapter.
Analysis of Reading Speed
The comparison between the speed of reading a technical expository text and the speed of reading manu-
facturing instructions is shown in Table 6.
Table 6. Wilcoxon signed-rank test for reading a technical expository text and manufacturing instructions
Instructions HBN – technical expository text

Reading speed (HB_) HBS Mean rank
HBN < HBS 78.7 55.33

Arithmetic mean difference
HBN > HBS 21.3 51.43
(N in %)
HBN = HBS 0
Z - 5.40
Asymp. Sig (p)
.0001
(2-tailed)
Instructions VBN– technical expository text

Reading comprehension (VB_) Mean rank
VBS
VBN < VBS 45.37 52.39

VBN > VBS 54.62 56.25
(N in %)
VBN = VBS 0
Z -1.152
Asymp. Sig (p)
.249
(2-tailed)
Instructions EBN– technical expository text

Effective reading speed (EB_) Mean rank
EBS
EBN < EBS 49.07 58.73

EBN > EBS 50.93 50.43
(N in %)
EBN = EBS 0
Z -0.520
Asymp. Sig (p)
.603
(2-tailed)
Legend:
HBS - Reading speed of technical expository text; HBN - Reading speed of instructions for manufacturing a product.
VBS - Reading comprehension for a technical expository text; VBN - Reading comprehension for instructions for manufacturing a
product.
EBS - Effectiveness of reading technical expository texts; EBN - Effectiveness of reading manufacturing instructions for a product.
Figure 1 shows types of readers according the speed of reading two types of texts, namely, technical exposi-
tory texts and manufacturing instructions, and also the average speed of reading both text types.
138
ISSN 2538–7138 /Online/ (P. 132-146)
Figure 1. Types of readers according the speed of reading technical expository texts and manufacturing
instructions and average speed of reading both text types.
Table 7 shows statistical values of reading speed.
Table 7. Results of reading speed.
Reading speed Technical expository text Manufacturing instructions Total
Arithmetic mean 88.54 82.42 85.48

Standard deviation 17.08 14.88 14.92
Minimum 39.92 48.48 47.01
Maximum 121.07 120.85 120.55
I. quartile (Q1) 71.71 71.08 73.83
Median 84.81 83.23 86.61
III. quartile (Q3) 99.08 91.43 94.31
Table 8 shows the correlation between the speed of reading technical expository texts, manufacturing in-
structions, and both text types.
Table 8. Spearman’s correlation coefficient for the speed of reading technical expository texts, manufacturing
instructions, and both text types.
Spearman’s correla-
1 .730 .930
tion coefficient
Technical expository text
Asymp. Sig. (p)
/ .0001 .0001
(2-tailed)
.730 1 .920
tion coefficient
Manufacturing instructions
Asymp. Sig. (p)
.0001 / .0001
(2-tailed)
139
(P. 132-146) ISSN 2538–7138 /Online/
Analysis of Reading Comprehension
Based on the Wilcoxon test for comparing reading comprehension in students (Table 6), there was no sta-
tistically significant difference regarding the percentage of achieved points and in turn, regarding the reading
comprehension, between technical expository texts and manufacturing instructions (p = .249). This is true in spite
of the fact that 45.37% of students achieved a higher number of points for their understanding of a technical ex-
pository text, and 54.62% of students achieved better results for understanding manufacturing instructions. The
mean rank for reading a technical expository text was 52.39% of achieved points, and 56.25% of achieved points
for reading manufacturing instructions.
Types of readers according to their understanding of a technical expository text, manufacturing instructions,
and both text types, are shown in Figure 2.
Figure 2. Types of readers according to their understanding of a technical expository text, manufacturing instructions,
and both text types
Regarding the level of understanding what was read in a technical expository text, 25% of the students were
poor readers, and 25.90% of students were poor readers with regard to their reading comprehension of manufactur-
ing instructions. From the point of view of reading comprehension of both text types, 27.80% of the students were
labelled as poor readers. These were students who achieved up to 45% of possible points in answering questions
about the read text (Figure 2). The percentage of average readers was also smaller, by 16.70%, with regard to read-
ing comprehension of manufacturing instructions in comparison to understanding what was read in a technical
expository text (32.40% average readers for manufacturing instructions; 49.10% average readers for technical
expository texts). The percentage of good readers was larger in the case of reading manufacturing instructions
(36.10%) than in the case of reading a technical expository text (14.80%). With regard to reading comprehension
of a technical expository text, 11.10% of the students were very good readers, and only 5.60% of the students were
very good readers in the case of understanding what was read in manufacturing instructions. These were students
who achieved between 60% and 75% of possible points (good reader), or more than 75% of points (very good
reader) when answering questions about the read text.
From the perspective of reading comprehension of both text types, 27.80% of the students were poor readers,
49.10% were average readers, 19.40% were good readers, and only 3.70% were very good readers.
In comparison to the level of understanding a technical expository text, the students achieved better results
(2.06% higher on average) in the case of reading and understanding manufacturing instructions (Table 9). The aver-
age percentage of achieved points after reading a technical expository text was 50.53% of the maximum number of
140
ISSN 2538–7138 /Online/ (P. 132-146)
points, and 52.59% in the case of reading and understanding manufacturing instructions. After reading a technical
expository text, the top 25% students (Q4) achieved, on average, more than 60,32% of possible points, and more
than 64.32% of points in the case of manufacturing instructions. For technical expository texts, the lowest-scoring
25% of students (Q1) achieved up to 39.88% of the maximum number of points, and up to 39.58% of points (poor
readers) in the case of manufacturing instructions.
From the perspective of reading comprehension of both text types, the students achieved an average of
51.42% of possible points. The bottom 25% (Q1) achieved between 16.22% and 43.69% of the maximum number
of points (poor readers) with respect to reading and understanding both text types. The top 25% of students (Q4)
achieved between 59.46% and 83.33% of points after reading both text types (average to very good readers).
Table 9. Results of the analysis of reading comprehension for technical expository texts, manufacturing
instructions and both text types.

Minimum 9.22 9.38 16.22
Maximum 89.68 91.67 83.33
I. quartile (Q1) 39.88 39.58 43.69
Median 48.02 57.29 52.48
III. quartile (Q3) 60.32 64.32 59,46
Spearman’s rank correlation coefficient (Table 10) was used to determine the relation between reading com-
prehension and reading speed. The research established that there was a slight statistically significant positive
correlation between the achieved percentage of points when reading manufacturing instructions and the speed
of reading a technical expository text (r = .190, p = .049); between the speed of reading both text types and the
percentage of achieved points when reading manufacturing instructions (r = .192, p = .047); and a borderline sta-
tistically significant correlation between the speed of reading manufacturing instructions and the percentage of
achieved points in the reading comprehension of manufacturing instructions (r = .163, p = .092). The test further
confirmed that there was a strong statistically significant correlation between the percentage of achieved points
after reading both text types and the percentage of achieved points after reading manufacturing instructions (r =
.652, p = .0001), and a very strong statistically significant correlation between the percentage of achieved points
after reading a technical expository text and the percentage of achieved points after reading both text types (r =
.805, p = .0001).
Table 10. Results of reading comprehension.
Reading comprehension Technical expository text Manufacturing instructions Total
1 .132 .805
tion coefficient
Technical expository text
Asymp. Sig. (p)
/ .173 .0001
(2-tailed)
.132 1 .652
tion coefficient
Asymp. Sig. (p)
.173 / .0001
(2-tailed)
.018 .190 .085
Reading speed for t. exposition coefficient
tory text Asymp. Sig. (p)
.856 .049 .384
(2-tailed)
141
(P. 132-146) ISSN 2538–7138 /Online/
Reading comprehension Technical expository text Manufacturing instructions Total
- .059 .163 .034
Reading speed for manu- tion coefficient
facturing instructions Asymp. Sig. (p)
.546 .092 .728
(2-tailed)
-.009 .163 .034
Reading speed for both tion coefficient
text types Asymp. Sig. (p)
.924 .092 .728
(2-tailed)
Analysis of Effective Reading Speed
Based on Wilcoxon’s criterion for comparing the effective reading speed when reading a technical expository
text and when reading manufacturing instructions (Table 6), it was found that 49.07% of the students achieved a
higher effective reading speed when reading manufacturing instructions, and 50.93% when reading a technical
expository text. The negative z-score (Z = - .520) implied that there were more negative differences than positive
ones, i.e., that more often the students achieved a higher effective reading speed when reading manufacturing
instructions, however, this difference was not a statistically significant one (p = .603).
Types of readers according to the effective reading speed when reading a technical expository text, manu-
facturing instructions, or reading both text types, are shown in Figure 3. Measuring the effective reading speed
in reading technical expository texts showed that 7.40% of the students were poor readers. These were students
whose answers to the questions about the read text were insufficient, and their reading speed was low (small
number of read words per minute). In the case of reading manufacturing instructions, there were 14.80% of poor
readers. In comparison, there were also more (by 14.90%) average readers in the case of technical expository
texts (55.60%) than in the case of manufacturing instructions (40.70%). There were 9.20% less of good readers of
technical expository texts (34.30%) in comparison to the percentage of good readers of manufacturing instruc-
tions (43.50%). In addition, it was measured that 2.80% of the students achieved very good results in reading a
technical expository text (effective reading speed above 82.5), and 0.90% were very good readers in the case of
reading manufacturing instructions.
Figure 3. Types of readers according to effective reading speed.
142
ISSN 2538–7138 /Online/ (P. 132-146)
When reading a technical expository text, the students reached an average effective reading speed of 44.84
(Table 11), and an average effective reading speed of 43.69 (average reader) when reading manufacturing instruc-
tions. The individual student’s effective reading speed for technical expository texts deviated from the average
effective reading speed by a standard deviation of 17.80, and by a 17.00 standard deviation in the case of manu-
facturing instructions. 25% of the students with the lowest score (Q1) reached effective reading speeds between
7.6 and 33.34 when reading a technical expository text; and between 7.61 and 30.38 when reading manufacturing
instructions (poor reader – average reader). 25% of the highest-scoring students (Q4) reached an effective reading
speed for technical expository texts between 55.09 and 99.85, and an effective reading speed for manufacturing
instructions between 56.88 and 85.00 (good – very good reader).
From the point of view of reading both text types, the average effective reading speed was 44.11. The individual
student’s effective reading speed deviated from the average effective reading speed by a standard deviation of
13.98. 50% of the students reached effective reading speeds lower than 43.23 (poor – average readers). The bot-
tom quarter (Q1) reached effective reading speeds between 12.92 and 35.23, while the top 25% of the students
(Q4) reached an effective reading speed between 52.51 and 85.91 of read words per minute.
Table 11. Results of effective reading speed.
Reading speed T. expository text Manufacturing instructions Total

Minimum 7.60 7.61 12.92
Maximum 99.85 85.00 85.91
I. quartile (Q1) 33.34 30.38 35.23
Median 42.60 45.11 43.23
III. quartile (Q3) 55.09 56.88 52.51
Below, the results of the Spearman correlation test (Table 12) are presented, which was used to measure
the correlation between the effective reading speed of a technical expository text, the effective reading speed of
manufacturing instructions, and the effective reading speed when reading both text types.
Table 12. Spearman’s correlation coefficient for effective reading speed.
Effective reading speed Expository text Manufacturing instructions Total
1 .258 .795
tion coefficient
Expository text
Asymp. Sig. (p)
/ .007 .0001
(2-tailed)
.258 1 .767
tion coefficient
Asymp. Sig. (p)
.007 / .0001
(2-tailed)
It was established that a weak positive statistically significant correlation existed between the effective reading
speed of manufacturing instructions, and the effective reading speed of a technical expository text (r = .258, p =
.0001); a strong (very strong) correlation existed between the effective reading speed of a technical expository text
(r = .795, p = .0001) and the effective reading speed of reading both text types; and a high statistically significant
correlation existed between the effective reading speed of manufacturing instructions and the effective reading
speed of both text types (r = 0767, p = .0001) – those students, who had higher effective reading speeds when
reading manufacturing instructions, also had a higher reading speed in the case of reading a technical expository
text, as well as a higher effective reading speed considered from the perspective of reading both text types.
143
(P. 132-146) ISSN 2538–7138 /Online/
Discussion
In this research, the notion of functional literacy in the field of engineering and technology was defined, and the
specifics of reading competence and the concept of understanding technical texts from the field of engineering
and technology were described. A relevant circumstance in this research was the fact that in Slovenia, developing
functional literacy is the responsibility of mother-tongue teachers as part of the curriculum in the framework of
the school subject called ‘Slovene language and literature’. The research focused on the following question: is it
realistic to expect of mother-tongue teachers to develop in the mother-tongue class functional literacies for read-
ing so-called “demanding texts” from the field of engineering and technology? The results of the present research
provide a rather straightforward answer. A comparison between the types of readers according to the type of text
they were asked to read, shows that both in the case of reading a technical expository text, as well as in the case
of reading manufacturing instructions (Figure 1), there were 0.90% of slow readers (reading up to 50 words per
minute – Table 3). Table 3 also shows that, regarding the reading of manufacturing instructions, there were 32.40%
of average readers (reading between 50 and 80 words per minute), 54.60% of fast readers (reading between 80 and
110 words per minute), and only 12% of very fast readers, who read more than 110 words per minute. In the case of
reading a technical expository text, however, there were more average readers (40.70%) and fast readers (55.60%),
and less very fast readers (2.80%). From the point of view of the speed of reading both text types, there were 1.90%
of slow readers, 37% of average readers, 53.70% of fast readers, and 7.40% of very fast readers. A comparison of
reading speed between technical expository texts and manufacturing instructions (Table 6), shows that 78.70%
of the students reached a higher reading speed when reading a technical expository text, while 21.30% of the
students were faster when reading manufacturing instructions. The mean rank for reading a technical expository
text was 55.33 read words per minute, and 51.43 read words per minute in the case of manufacturing instructions
(on average, students read 3.9 words per minute more when reading a technical expository text than they did
when reading manufacturing instructions). On the basis of the calculated Z-score (Z = – 5.40) and the asymptote
significance value (p = .0001 (p < .05)), it was concluded that there were statistically significant differences between
the reading speed when reading a technical expository text and the reading speed when reading manufacturing
instructions. The standard deviation of the number of read words per minute from the average (Table 7) was 14.88
read words per minute in the case of reading a technical expository text. The average reading speed when reading
a technical expository text was 88.44 words per minute and 82.42 words per minute when reading manufacturing
instructions. Students who scored in the bottom 25% read between 39.92 and 71.71 words per minute on average
when reading a technical expository text (between 48.48 and 71.08 words per minute on average when reading
manufacturing instructions). Students who scored in the top 25% read between 99.08 and 121.07 words per minute
on average when reading a technical expository text (between 91.43 and 120.85 words per minute on average
when reading manufacturing instructions).
The reading speed results for both text types tell the same tale (Table 7): the average reading speed calcu-
lated on the sample was 85.48 of read words per minute for technical expository texts. The individual student’s
reading speed deviated from the average by 14.92 of read words per minute. Students who scored in the bottom
25% read between 47.01 and 73.83 words per minute. Students who scored in the top 25% read between 94.31
and 120.55 words per minute.
Concerning the reading speed for technical expository texts, manufacturing instructions, and both text
types, the Spearman correlation coefficient (Table 8) implies a very high statistically significant positive correlation
between reading a technical expository text, reading manufacturing instructions, and reading both text types
(r = .930; r = .920), and a high correlation between the reading speed when reading a technical expository text and
the reading speed when reading manufacturing instructions (r = .730).
These results serve as persuasive arguments in answering the question of whether a mother-tongue teacher
can successfully teach science, technology and engineering literacy. The answer, of course, is no. Students can learn
the majority of elements of the engineering and technical language only from their subject teacher of technology.
Generally, teachers should be able to teach them to read two types of texts: expository and explicatory texts from
the field of engineering and technology (definition, description, comparison – contrast, cause – effect, problem –
problem solution, and description of the process), and procedural texts describing processes (instructions for use,
manufacturing instructions). These are two key types of texts, which basically differ in the significance of the concept
of time for understanding the text and for successfully using the information from the text (i.e., the chronological
sequence of information). It depends on the latter whether the student/reader will be able to achieve the goal of
144
ISSN 2538–7138 /Online/ (P. 132-146)
reading, more specifically, whether they will be able to solve the problem posed (by themselves or by the teacher)
before the reading. According to the definition of engineering literacy, students should be able to demonstrate
creativity and innovation, communicate and collaborate, conduct research and use information, think critically,
solve problems, make decisions, and use technology effectively and productively (ISTE, 2000).
The research clearly shows that problems exist regarding literacy, especially functional literacy, which has been
confirmed also by the findings of most other (international) studies in this area. It is a task for education to foster
awareness of the fact that teaching literacy is not only the duty of mother-tongue teachers, but also, especially
with regard to functional literacy, a responsibility of teachers of individual expert areas – in the case of the present
research, science and technology teachers. In order to achieve this, the following is recommended:
1. Collaboration between the mother-tongue teacher and the subject teacher – the objective is developing
functional literacy, which is carried out at the school level, however, the entire education chain should
also be included in this process, including the competent ministry, research institutions, schools that
train educators, as well as educational institutions themselves (primary schools, secondary schools,
universities).
2. Training teachers for collaborative forms of work. At the same time, this principle (cross-curricular
cooperation and communication) should also be introduced at a global, paradigmatic level.
3. Development of strategies for collaborative and research learning with an emphasis on methods for
independent reading to support innovative approaches to research and collaboration, which, with the
emergence of technology in schools, are no longer an option, but a necessity.
Conclusions
It is evident from the present research that in schools, a shift from understanding functional literacy as a
monolithic competence to so-called functional literacies in subject-specific fields, is required. Functional literacies
in individual subject areas can only be the result of cross-curricular co-operation between a teacher of the Slovene
language (i.e., mother-tongue teacher) and a subject teacher, or teachers, with the share of responsibility leaning
towards the latter as the level of education progresses. It should be remembered, of course, that only properly
qualified teachers are able to competently perform their role in this process. Therefore, teachers need to be trained
in at least two areas, namely, in the field of functional literacy, in this case STE functional literacy, and in collaborative
work. In order to achieve this, teacher training methods have to be adapted, and in turn, STE teachers have to learn
how to teach their students reading strategies, so they can successfully learn from STE texts. In order to achieve
this goal, teacher training programs, first of all, have to equip future teachers with the awareness that every teacher
is also a teacher of reading, and that a STE teacher is responsible for developing STE reading competence in his
students. Secondly, research programs for STE teachers and lifelong learning courses have to equip their students/
teachers with metacognitive STE reading skills, as well as with knowledge about the didactics of STE scaffolding
strategies for reading both types of STE texts: for reading informative, explicatory, expository texts and for reading
procedural texts, which describe engineering and technical processes. This is a huge shift in STE teacher training
education, which is probably impossible to occur in a short span of time, because it requires, above all, changes
in the prejudiced attitudes of teacher training staff, which often include a firmly rooted conviction that for a STE
teacher the only important knowledge is scientific knowledge of the selected scientific field, while everything else
that will eventually be needed in the process of students’ knowledge acquisition, is someone else’s responsibility.
References
Aberšek, B., Barle Lakota, A., Borstner, B., Bregant, J., Dolenc, K., Flogie, A., Gartner, S., Kordigel Aberšek, M., Rutar Leban, T., Šverc,
A., Štraus, M., & Ploj Virtič, M. (2017). Cognitive science in education and alternative teaching strategies. Newcastle upon Tyne:
Cambridge Scholars.
Aberšek, B., Borstner, B., & Bregant, J. (2014). The virtual science teacher as a hybrid system: cognitive science hand in hand with
cybernetic pedagogy. Journal of Baltic Science Education, 13(1), 75–90.
Balka, D. (2011). Standards of mathematical practice and STEM. Stillwater: OK: School Science and Mathematics Association.
Bermudez, J. L. (2010). Cognitive Science. Cambridge: Cambridge University Press.
Buzan, T. (2009). Hitro branje [Fast reading]. Ljubljana: Mladinska knjiga.
Dolenc, K., Aberšek, B., & Kordigel Aberšek, M. (2015). Online functional literacy, intelligent tutoring systems and science educa-
tion. Journal of Baltic Science Education, 14(2), 162-17.
145
(P. 132-146) ISSN 2538–7138 /Online/
Dreher, M. J. (2002). Children searching and using information text: A critical part of comprehension. In Block, C. C., & Pressley, M.
(Eds.), Comprehension instruction: Research-based best practices (pp. 89-304). New York: Guilford.
Fisher, D., Frey, N., & Lapp, D. (2009). Learning cycles that deepen students’ interaction with text. Voices from the Middle, 22 (4), 15-19.
Goldman, S. R., & Rakestraw, J. S. (2000). Structural aspects of constructing meaning from text. In M. L. Kamil, P. B. Mosenthal, P. D.
Pearson, & R. Barr (Eds.), Handbook of reading research (Vol. 3, pp. 311-335). Mahwah, NJ: Lawrence Erlbaum.
IRA (2012). Adolescent literacy. Newark, DE: International reading association.
ISTE (2000). Standards for technology literacy: Contents for the study of technology. Reston: International Technology Education
Association.
Kordigel Aberšek, M., Aberšek, B., & Flogie, A. (2017). Writing versus typing during science teaching: Case study in Slovenia.
Journal of Baltic Science Education, 17(1), 84-96.
Lefever-Davis, S., & Pearman, C.J. (2015). Reading, writing and relevancy: Integrating 3R’s into STEM. The Open Communication
Journal, 9(1), 61-64.
OECD (2000). Literacy in the Information Age: Final Report of the International Adult Literacy Survey. Paris: OECD Publications Service.
OECD (2009). Creating effective teaching and learning environments: First results from TALIS – Executive Summary. Paris: OECD
Publishing.
OECD (2011a). Education Policy Analysis. Paris: OECD Publishing.
OECD (2011). OECD Guide to Measuring the Information Society 2011. Paris: OECD Publishing.
OECD (2014). TALIS 2013 Results: An international perspective on teaching and learning. Paris: TALIS, OECD Publishing.
PIRLS (2016). Assessment framework. Boston: TIMSS & PIRLS International Study Center.
PISA (2006). Science competencies for tomorrow’s world. Paris: OECD Publications Service.
PISA (2015). Pisa 2015: Draft reading literacy framework. Paris: OECD Publications Service.
Schmitz, W. (2012). Hitrejše branje – boljše razumevanje [Faster reading - better understanding]. Tržič: Avrora.
Zollman, A. (2012). Learning for STEM Literacy: STEM Literacy for Learning First. School Science and Mathematics, 112(1), 12-19.
Received: September 29, 2018 Accepted: February 02, 2019
Zvonka Cencelj Teacher, Primary School Vransko-Tabor, Vransko 23, 3305 Vransko,
Slovenia.
E-mail: zvonka.cencelj@gmail.com
Metka Kordigel Aberšek PhD, Professor, University of Maribor, Faculty of Education, Koroška 160,
2000 Maribor, Slovenia.
E-mail: metka.kordigel@um.si
Boris Aberšek PhD, Professor, University of Maribor, Faculty of Natural Science and
Mathematics, Koroška 160, 2000 Maribor, Slovenia.
E-mail: boris.abersek@um.si
Andrej Flogie PhD, Assistant Professor, University of Maribor, Faculty of Natural
Science and Mathematics, Koroška 160, and Institute Anton Martin
Slomšek, Vrbanska 30, 2000 Maribor, Slovenia
E-mail: andrej.flogie@z-ams.si
146
INFORMATION
FOR CONTRIBUTORS
EDITORIAL AND REVIEW PROCEDURES
Journal of Baltic Science Education (JBSE) publishes original scientific research articles in the field of Natural Science Education and
related areas for all educational levels in the Baltic countries. It is possible to publish special (thematic) issues of JBSE. The papers should
be submitted and will be published in English. JBSE will promote to establish contacts between researchers and practical educators
both in the Baltic countries and countries around.
The authors of the manuscripts are responsible for the scientific content and novelty of the research materials. Articles, published
before in other international journals or papers’ collections will not be accepted for publication in JBSE.
As a publication that represents a variety of cross-disciplinary interests, both theoretical and practical, the JBSE invites manuscripts on
a wide range of topics, especially in the following areas:
• Didactics of natural sciences. • Philosophical, political, economical and social aspects
• Theory and practice in natural science teacher of natural science education.
education. • The supplementary natural science education.
• Integrated natural science education. • ICT in natural science education.
• Natural science and technological literacy. • The standardisation of natural science education etc.
• General and professional natural science education.
MANUSCRIPTS GUIDELINES
The structure of the research paper presented to the Journal of Baltic Science Education should be as follows: abstract - short report of the
investigation; introduction inc. aim and subject of the research; research methodologies and methods; results of the research incl. discussion;
conclusions; list of references in APA style.
The papers should be submitted in English. If English is a second language for the author, please consider having the manuscript
proof read and edited before submitting. The preliminary text of the article can be sent as a.doc file in the attachment by e-mail: mail.
jbse@gmail.com
The text must be elaborated in Word for Windows, using 12 point Times New Roman letters. An article should not exceed 7-10 A4 pages,
included figures, tables and bibliography. Publishing of longer articles should be negotiated separately. Texts margins: top and bottom 20mm,
left - 25mm, right - 20mm. The title: capital letters, 14pt, bold; space between the title and the author’s name is one line interval. Author’s name
and surname: small letters, 12pt, bold. Under the name, institution: 11 pt, italics; space between the title and the text: 1 line interval. Abstract
– about 100-150 words - precedes the text. The text: 12pt Single or Auto spacing, in one column. Key words: no more than five words. The
language must be clear and accurate. The authors have to present the results, propositions and conclusions in a form that can suit scientists
from different countries.
Titles of the tables and figures: 11 pt, small letters. Space between figures or tables and the text: 1 line interval. Introduction, titles of
chapters and subchapters: 12pt, bold, small letters. Numbers: Arabic, subchapters numbered by two figures (1.1, 1.2, etc.). Figures, tables and
captions should be inserted within the manuscript at their appropriate locations. Diagrams and graphs should be provided as finished black
and white line artwork or electronic images. When there are a number of illustrations, the author should endeavour to reduce the amount of
text to accommodate the illustrations in the limited space available for any article.
References in the text should be presented in brackets (Knox, 1988; Martin, 1995). If necessary, the page can be indicated: (Martin,
1995, p.48). The list of references should be presented after the text. The Words List of References: 11pt, bold, small letters. The references
should be listed in full at the end of the paper in the following standard form:
For books: Saxe, G.B. (1991). Cultural and Cognitive Development: Studies in Mathematical Understanding. Hillsdale, NJ: Erlbaum.
For articles: Bekerian, D.A. (1993). In Search of the Typical Eyewitness. American Psychologist, 48, 574-576.
For chapters within books: Bjork, R.A. (1989). Retrieval Inhibition as an Adaptive Mechanism in Human Memory. In: H.L. Roediger III & F.I.M.
Craik (Eds.), Varieties of Memory & Consciousness (pp. 309-330). Hillsdale, NJ: Erlbaum.
On a separate page, author - related data should be presented in English: name, surname, degree and academic title, institution, full
correspondence address in the clearest and most complete form /ordinary post and e-mail addresses /, position (to ensure anonymity in the
review process). The author (authors) should confirm in writing, that the manuscript has not been published in other journal or
handed over (transferred) to other journal for publication.
EDITORIAL AND REVIEW PROCEDURES

Manuscripts will be sent anonymously to reviewers with expertise in the appropriate area. All manuscripts will be rewieved by two
experts before JBSE’s accept them for publication. This process usually takes about two months. The journal co-editors will make minor
editorial changes; major changes will be made by the author(s) prior to publication if necessary. JBSE’s redaction will sent to author(s)
only one correcture which must be sent back within 2 weeks. JBSE will not review submissions previously published elsewhere through
print or electronic medium.
Manuscripts submitted to the JBSE cannot be returned to authors. Authors should be sure to keep a copy for themselves. Authors’
signatures should be at the end of the paper and its second checked proofs.
Manuscripts, editorial correspondence (and other correspondence for subscription and exchange), and any questions should be
sent to editor-in-chief or to regional redactors.
Journal`s requirements for the authors are available online: http://www.scientiasocialis.lt/jbse/files/JBSE_requirements_2017.pdf
147
ISSN 1648-3898 /Print/, ISSN 2538-7138 /Online/
Compiled by: Vincentas Lamanauskas

Linguistic Editor: Ilona Ratkevičienė
Cover design by: Jurgina Jankauskienė
Layout design by : Linas Janonis

15 March 2019. Publishing in Quires 9,5. Edition 200
Publisher Scientia Socialis Ltd.,

Donelaicio Street 29, LT-78115 Siauliai, Lithuania
E-mail: scientia@scientiasocialis.lt
Phone: +370 687 95668
http://www.jbse.webinfo.lt/centras.htm
http://www.scientiasocialis.lt

Printing Šiauliai printing house
9A P. Lukšio Street
LT-76207 Šiauliai, Lithuania
Phone: +370 41 500 333.
Fax: +370 41 500 336
E-mail: info@dailu.lt

Journal of Baltic Science Education, Vol. 18, No. 1, 2019

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Journal of Baltic Science Education, Vol. 18, No. 1, 2019

Uploaded by

Copyright:

Available Formats

Vol.18, No.

Published since 2002 Address:

ISSN 1648–3898 (Print) © Scientia Socialis Ltd., Lithuania, 2018

ISSN 2538–7138 (Online)

ISSN 2538–7138 /Online/

PROPOSALS FOR SMALL STEPS TOWARD REPRODUCIBILITY OF SCIENCE EDUCATIONAL

AGRICULTURAL STUDENTS’ KNOWLEDGE OF PHOTOSYNTHESIS AND THE CONTEXTUAL

THE IMPACT OF A SCIENCE METHODS COURSE ON FEMALE PRE-SERVICE TEACHERS’

A STUDY OF FACTORS AFFECTING SCIENCE ACHIEVEMENTS OF JUNIOR HIGH SCHOOL

THE EFFECTS OF INQUIRY-BASED LEARNING AND LEARNING STYLES ON PRIMARY

DEVELOPMENT AND VALIDATION OF A SURVEY INSTRUMENT (AKA) TOWARDS ATTITUDE,

A FEASIBLE GUIDANCE FOR ORDERED MULTIPLE-CHOICE ITEMS IN STUDENTS’

IDENTIFICATION OF THE INTEREST OF TURKISH MIDDLE-SCHOOL STUDENTS IN STEM

TEACHERS’ SELF-CONCEPT AND ITS BENEFITS FOR SCIENCE EDUCATION

THE IMPLEMENTATION OF MIND MAPS IN TEACHING PHYSICS: EDUCATIONAL

ROLE AND MEANING OF FUNCTIONAL SCIENCE, TECHNOLOGICAL AND ENGINEERING

INFORMATION FOR CONTRIBUTORS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147

PROPOSALS FOR SMALL STEPS TOWARD REPRODUCIBILITY OF SCIENCE

Frankfurt, H. G. (2005). On bullshit. Princeton, New Jersey. Princeton University Press.

Received: January 08, 2019 Accepted: January 31, 2019

Andrej Šorgo PhD, Associate Professor of Biology Didactics, University of Maribor,

Abstract. This research aimed to assess Katja Gobec,

•• How well do students in agronomical education understand the process of photosynthesis?

Country Farmer 2 16 13 3.9

Total 330 100.0

Instrument and Procedures

Rotated Factor Loading

I like the content taught in biology lessons. .857

I am looking forward to biology lessons. .800

I am very interested in the content of biology. .784

I understand everything in biology lessons. .668

I like the content I learn in biology. .550

I think it is important to learn the contents discussed in biology. .549

I finish my homework on time. .445

Progress in biology improves the quality of our lives. .731

Biology is useful in everyday life. .722

What I learn in biology lessons will benefit my everyday life. .718

I find it important to understand the contents of biology lessons. .655

Biology is important for society. .586

Everyone needs the basic biological knowledge. .564

Due to my biology lessons, I understand the world around me better. .458

Understanding photosynthesis will benefit my life. .778

Photosynthesis is interesting. .756

Understanding the essence of photosynthesis is important for general education. .666

I make myself work for school. .546

In the lessons, I always concentrate on school subjects. .528

I arrange my space, so I can learn without disturbances. .513

Eigenvalue 8.09 1.89 1.25 1.08

% of variance 36.77 8.59 5.68 4.89

α .87 .84 .65 .77

Students’ general Achievements on the Knowledge Test about Photosynthesis

Total Gender Year of education Educational programme Farm at home

Female Male χ2 1. 2. 3. 4. χ2 CF G AET HT χ2 No Yes χ2

F - The main purpose

E1 - Plants receive sunlight

P1 – Carbon dioxide and

P2 – Glucose and oxygen

P3 - Carbon dioxide and

Total Gender Year of education Educational programme Farm at home

Female Male χ2 1. 2. 3. 4. χ2 CF G AET HT χ2 No Yes χ2

Differences among Students’ Achievements by Gender, Year of Study, and Programme

F - The main purpose of photosynthesis is the produc-

F - The main purpose of photosynthesis is the production of glucose. 1.00

E1 - Plants receive sunlight energy from the sun. .04 1.00