Journal of Baltic Science Education, Vol. 16, No. 5, 2017

Vol.16, No.
5, 2017
ISSN 1648-3898 /Print/
ISSN 2538–7138 /Online/
The International Journal of the Scientific Methodical Center “Scientia Educologica”

Journal of Baltic Science Education, Vol. 16, No. 5, 2017
Editorial Board
ISSN 1648–3898 /Print/
Editor-in-Chief ISSN 2538–7138 /Online/
Prof., Dr. Vincentas Lamanauskas Siauliai University, Lithuania
Deputy Editor-in-Chief
Prof., Dr. Andris Broks University of Latvia, Latvia
Deputy Editor-in-Chief
Prof., Dr. Miia Rannikmäe University of Tartu, Estonia
Executive Secretary
Dr. Laima Railienė SMC “Scientia Educologica”, Lithuania
Editors:
Prof., Dr. Boris Aberšek University of Maribor, Slovenia
Prof., Dr. Agnaldo Arroio University of Sao Paulo, Brazil
Prof., Dr. Martin Bilek Charles University, Czech Republic
Dr. Paolo Bussotti University of Udine, Italy
Prof., Dr. Bulent Cavas Dokuz Eylul University, Turkey
Prof., Dr. Hana Čtrnáctová Charles University, Czech Republic
Dr. Peter Demkanin Comenius University, Slovakia
Dr. André du Plessis Nelson Mandela Metropolitan University, South Africa
Prof., Dr. Peter Heering University of Flensburg, Germany
Prof., Dr. Jack Holbrook University of Tartu, Estonia
Prof., Dr. Ryszard M. Janiuk Maria Curie Sklodowska University, Poland
Dr. Milan Kubiatko University of Zilina, Slovakia
Dr. Todar Lakhvich Belarusian State Medical University, Republic of Belarus
Prof., Dr. Jari Lavonen University of Helsinki, Finland
Prof., Dr. Rita Makarskaitė-Petkevičienė Lithuanian University of Educational Sciences, Lithuania
Prof., Dr. Aadu Ott Göteborg University, Sweden
Prof., Dr. Paul Pace Malta University, Malta
Prof., Dr. Valfrids Paškevičs Daugavpils University, Latvia
Prof., Dr. Jongwon Park Chonnam National University, Korea
Prof., Dr. Raffaele Pisano Lille 3 University, France
Dr. Pavol Prokop Trnava University, Slovakia
Prof., Dr. Konstantinos Ravanis University of Patras, Greece
Dr. Alona Rauckienė-Michaelsson Klaipėda University, Lithuania
Prof., Dr. Heimo Saarikko Helsinki University, Finland
Dr. Hae-Ae Seo Pusan National University, Korea
Dr. Uladzimir K. Slabin University of Oregon, USA
Prof., Dr. Borislav V. Toshev Sofia University, Bulgaria
Dr. Georgios Tsaparlis University of Ioannina, Greece
Dr. Muhammet Usak Science and Medical Education Research Center, Turkey
A scientific journal JBSE issued by the SMC Scientia Educologica in cooperation with Scientia Socialis, Lithuania, empha-
sizes theoretical, experimental and methodical studies in the field of science education. JBSE is an international academ-
ic journal. In order to maintain the high standards appropriate to such a journal, all contributions received are submitted
for anonymous review by two experts, additionally to review by the Editor. The decision of the Editor on the acceptance
of articles is final and no correspondence can be entered into on reasons for rejection of a submitted contribution.
Published since 2002 Address:

The journal is published bimonthly. SMC “Scientia Educologica”
IF: 0.479 (2016) Donelaičio Street 29, LT-78115 Siauliai, Lithuania
H Index: 10 (2016) E-mail: mail.jbse@gmail.com
SJR: 0.302 (2016) Phone: +370 687 95668
ICDS: 10.6 (2016) Home page: http://www.scientiasocialis.lt/jbse/
ICV: 143.26 (2015)
ISSN 1648–3898 (Print) © SMC „Scientia Educologica“ & Scientia Socialis, Lithuania,
The Associated Member of Lithuanian Scientific Society,
ISSN 2538–7138 (Online) European Society for the History of Science (ESHS) and ICASE
The articles appearing in this journal are indexed/abstracted in British Education Index
(http://www.leeds.ac.uk/bei/bei.htm), Copernicus Index (http://journals.indexcopernicus.com), The Asian
Education Index (http://www.asian-education-index.com/sciences_index.php), EBSCO: Academic Search Premier
630 (http://search.ebscohost.com), Social Scisearch (Thomson Reuters) (http://science.thomsonreuters.com/index.html),
Journal Citation Reports / Social Sciences Edition (Thomson Reuters)
(http://thomsonreuters.com/products_services/scientific/Journal_Citation_Reports),
and SCOPUS (http://www.scopus.com)
ISSN 1648–3898 /Print/ CONTENTS
CONTENTS
Editorial
USER-FRIENDLY REPORTING OF RESEARCH FINDINGS FROM A QUESTIONNAIRE

Andrej Šorgo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 632
Articles
THE CONTRIBUTION OF DEMONSTRATION AND STUDENT-LED EXPERIMENTS

ON THE STUDENTS’ KNOWLEDGE QUALITY IN THE THIRD GRADE OF PRIMARY SCHOOL
Stanko Cvjetićanin, Mirjana Maričić . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 634
THE VALIDITY AND EFFECTIVENESS OF PHYSICS INDEPENDENT LEARNING MODEL
TO IMPROVE PHYSICS PROBLEM SOLVING AND SELF-DIRECTED LEARNING SKILLS OF STUDENTS
IN OPEN AND DISTANCE EDUCATION SYSTEMS
Paken Pandiangan, I Gusti Made Sanjaya, Budi Jatmiko . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 651
THE EFFECTS OF MULTIPLE INTELLIGENCES BASED INSTRUCTION ON STUDENTS’ PHYSICS
ACHIEVEMENT AND ATTITUDES
Deniz Gurcay, Hatice Ozturk Ferah . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 666
PHYSICS ANXIETY OF PRE-SERVICE TEACHERS AND THEIR SELF-EFFICACY BELIEFS:
DIFFERENCES ACCORDING TO GENDER AND PHYSICS ACHIEVEMENT
Serap Çalışkan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 678
INFORMATIONAL VALUE OF SUBMICROSCOPIC REPRESENTATIONS IN SLOVENIAN
CHEMISTRY TEXTBOOK SETS
Špela Hrast, Vesna Ferk Savec . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 694
THE ECOLOGICAL WORLDVIEWS AND LOCAL ENVIRONMENTAL CONCERNS AMONG
SECONDARY SCHOOL TEACHERS
Lwun-Syin Lwo, Jim- Hua Fu, Cheng-Chieh Chang . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 706
EDUCATIONAL EFFECTIVENESS OF THE 5E MODEL FOR SCIENTIFIC ACHIEVEMENT OF
STUDENTS WITH HEARING IMPAIRMENT
Zahida Parveen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 723
TACTIC FULFILLMENTS OF THREE CORRELATIONS FOR PROBLEM-SOLVING MAPS AND
ANIMATED PRESENTATIONS TO ASSESS STUDENTS’ STOICHIOMETRY PERFORMANCES
King-Dow Su . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 733
THE APPLICATION OF CONCEPT MAPS IN THE TEACHING OF POLLINATION AND
POLLINATORS IN ELEMENTARY SCHOOL
Jelena D. Stanisavljević, Mirka М. Bunijevac, Ljubiša Ž. Stanisavljević . . . . . . . . . . . . . . . . . . . . . . . 746
EFFECT OF PROBLEM-BASED LEARNING ON IMPROVEMENT PHYSICS ACHIEVEMENT AND
CRITICAL THINKING OF SENIOR HIGH SCHOOL STUDENT
Mundilarto, Helmiyanto Ismoyo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 761
HISTORY OF CHEMISTRY AS A PART OF ASSESSMENT OF STUDENTS’ UNDERSTANDING
OF THE LAW OF CONSERVATION OF MASS
Vesna D. Milanovic, Dragica D. Trivic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 780
THE EFFECTS OF CURRICULUM, GENDER AND STUDENTS’ FAVORITE SCIENCE SUBJECT ON
INDONESIAN HIGH-SCHOOL STUDENTS’ CONCEPTIONS OF LEARNING SCIENCE
Arif Rachmatullah, Sariwulan Diana, Minsu Ha . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 797
EXPLORING STUDENTS’ UNDERSTANDING IN MAKING A DECISION ON A SOCIOSCIENTIFIC
ISSUE USING A PERSUASIVE GRAPHIC ORGANISER
Mastura Halim, Rohaida Mohd Saat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 813
Information
INFORMATION FOR CONTRIBUTORS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 825

PROBLEMS OF PSYCHOLOGY IN THE 21ST CENTURY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 826
QUALITY ISSUES AND INSIGHTS IN THE 21ST CENTURY . . . . . . . . . . . . . . . . . . . . . . . . . . . . 827

631
USER-FRIENDLY REPORTING OF RESEARCH FINDINGS FROM A QUESTIONNAIRE
Andrej Šorgo
University of Maribor, Slovenia
Hooray! You have succeeded in publishing an article based on your own, yet unpublished questionnaire. This
can certainly be regarded as a personal success. However, many articles based on questionnaires are not particu-
larly reader-friendly. I am sure that these authors are not being deliberately evil; however, in their path to fame,
they forget that their writing is not a secret diary but something that should communicate readily with the reader.
An enormous number of research articles in the field of education are published on a daily basis, many of
them based on the application of questionnaires as research instruments. In early October 2017, a quick search of
Thomson Reuters Web of Science database using the search term “education*” yielded 849,962 hits, mainly, but not
exclusively listed in the following databases and categories: Science Citation Index Expanded - Education, Scientific
Disciplines (40 hits) and Social Sciences Citation Index - Education & Educational Research (235 hits), Education,
Special (40 hits) Psychology, Educational (53 hits). A similar search in Elsevier’s Scopus database yielded 1,900,897
hits, from 1017 journals listed in the SNIP database. Google Scholar, a search engine not protected by subscription,
as is the case with academic databases, produced 5,370,000 hits for education and 4,130,000 for questionnaire. These
numbers (leaving aside sources not listed in established databases, books, conference proceedings, grey-papers,
duplicate titles listed in different databases and so on--just to illustrate that the number of papers is indeed huge) far
exceed the range that could possibly be read by any single living person. In addition, if someone is inspired to read
your particular paper, then you should think of this person as a friend, and it is not polite to be rough with friends.
Assume that you have implemented advice on how to make your research findings visible by choosing a
catchy title and composing a striking initial sentence for your abstract, and have succeeded in attracting someone
to open a web-page offering your abstract, along with the information that the price for your article is let’s say,
$26.95. If the abstract concludes with a statement like this: “To improve X, we make recommendations for the
future use (application, etc) of X in the conclusion,” you can be sure that you will make no friends this way. There
are many guidelines telling us how to write a structured abstract. Briefly, the logical order of an abstract should be
as follows: Background, Purpose, Sample, Methodology, Results and Conclusions.
Assume that your article using a questionnaire as a data-acquisition method has passed the abstract threshold
and someone with the same problem wants to use a questionnaire to answer a similar research question. It isn’t
that this person doesn’t trust you; however, the cornerstones of research methodology are reproducibility and
replicability (Pashler & Wagenmakers, 2012). In addition, it is impossible to verify your findings unless you include
the questionnaire in the text (perhaps as an appendix or supplement) or refer directly to the full text of your in-
strument, if published elsewhere. At this point, I suggest using a pre-tested and applied questionnaire whenever
possible. This makes it much easier to compare your findings with those achieved by the rest of the world. An extra
bonus is that when you get around to writing up the discussion, this becomes a much easier task.
Most of the time, results from a questionnaire are reported in tables, which are sometimes incomprehensible.
One doesn’t have to seek up-to-date references to find recommendations for making tables more user-friendly.
For example, Weiner’s (1992) recommendations are as follows:
•• Arrange the rows and columns in a table in a way that makes sense (this apply for graphs too).
Example: It is fact that statements in a questionnaire follows an order from the beginning to the end. If
the authors of the questionnaire succeed in finding enough victims, they will assemble sufficient data.
Nowadays, this data is usually transferred to one of a range of statistical programmes, any of which will
632
follow commands to produce tables from the numbers. Thanks to the admirable cut and paste function,
these are easily transferred to one of the word processors and modified according rules chosen by the
journal editors (e. g. APA 6th edition). If such tables survive scrutiny by two anonymous reviewers, they
will be published as part of your paper, which can be a good excuse to open a bottle of something
sparkling.
However, using too many tables ends in treating the reader as an enemy. You can rest assured that
almost nobody is interested in the presentation of descriptors or calculations in the order in which they
were collected. Therefore, to make your tables and derived graphs user-friendly, employ sort command
to arrange the calculations into an order. Size, the largest first, can be a good choice. If size is not ap-
propriate, use some other natural order, such as time, and tell a story not a list.
•• Round up your figures.
Weiner (1992) give us three reasons for this recommendation: a) Human beings cannot readily under-
stand more than two digits; b) We can almost never justify more than two digits of accuracy statistically;
c) We seldom care about accuracy to more than two decimal places.
To raise statistical literacy, I will simply quote Weiner (1992, p. 20) as a revelation:
“The standard error of any statistic is proportional to one over the square root of the sample size. God did this, and
there is nothing we can do to change it. Thus, suppose we would like to report a correlation as .25. If we don’t want
to report something that is inaccurate, we must be sure that the second digit is reasonably likely to be 5 and not 6 or
4. To accomplish this, we need the standard error to be less than .005. But since the standard error is proportional to
1/√n, the obvious algebra (1/√n ~ .005 à 1/.005 = 200) yields the inexorable conclusion that a sample size of the order
of 2002, or 40,000, is required to justify the presentation of more than a two-digit correlation. A similar argument can
be made for all other statistics.”
As a conclusion, if readers dislike your work, it may not be because your findings are underwhelming, but
simply that they have not been presented in a readily understandable manner.
References
Pashler, H., & Wagenmakers, E. J. (2012). Editors’ introduction to the special section on replicability in psycho-
logical science: A crisis of confidence? Perspectives on Psychological Science, 7 (6), 528-530.
Wainer, H. (1992). Understanding graphs and tables. Educational Researcher, 21 (1), 14-23.
Received: October 04, 2017 Accepted: October 19, 2017
Andrej Šorgo PhD, Associate Professor of Biology Didactics, University of Maribor,

Koroška cesta 160, 2000 Maribor, Slovenia.
E-mail: andrej.sorgo@um.si
633
THE CONTRIBUTION OF
DEMONSTRATION AND
STUDENT-LED EXPERIMENTS
ISSN 2538-7138 /Online/
ON THE STUDENTS’
KNOWLEDGE QUALITY IN THE
THIRD GRADE OF PRIMARY
SCHOOL
Abstract. The aim of this research is to

establish the contribution of demonstration
and student-led experiments on the quality
Stanko Cvjetićanin,
of students’ knowledge about the air-re- Mirjana Maričić
lated content in the third grade of primary
school. This research included 120 students
from Serbia, divided into two groups:
control (C) and experimental (E). In group Introduction
C the experiments were demonstrated by
the teacher, whereas those same experi- Children start exploring from the earliest age, but first organized knowl-
ments were independently conducted by edge about nature, they learn in an integrated form during their preschool
the students’ inside smaller groups in group education, and then in primary school. In accordance with the educational
E. The quality of the students’ knowledge system of the Republic of Serbia, students in the first four grades of primary
was examined after the application of the school study natural sciences in an integrated form through mandatory
experiments with the post-test, while the subjects: The World around Uѕ (first and second grade) and Nature and So-
quality of the duration of knowledge was ciety (third and fourth grade). They can also choose from optional subjects
examined with the retest. The students (intended to expand the knowledge) such as: Guardians of the Nature and
in the group E acquired the knowledge Hands in the dough (taught in all four grades).
of the higher quality and more durable Students in the first four grades of primary school become familiar
knowledge about the air-related content with studying nature through different methods and processes, but acquire
then the students’ from the group C on the scientific approach much better through direct practical experience (Eshach,
cognitive levels: analysis, evaluation and 2006) when they: measure, touch and manipulate different materials, draw,
synthesis. Therefore, the priority should make charts and write data in (Ateş & Eryılmaz, 2011, Karamustafaoğlu,
be given to the student-led experiments 2011). Research activities have a central and distinct role in natural sciences
over the demonstration, when processed teaching which can be neither efficient nor relevant without their use in the
air-related content in the third grade of process (Katcha & Wushishi, 2015, Hart, Mulhall, Berry, Loughran & Gunstone,
primary school, because their application 2000). They usually imply the use of the problem approach and the usage
contributes to the increase of the students’ of certain scientific skills, but are mostly based on conducting experiments
knowledge quality. (Koray & Serdar Köksal, 2009). Laboratory work (research activities) is care-
Keywords: demonstration experiments, fully organized in teaching practice and presented to students through
primary school, student knowledge quality, the laboratory-experimental method, combined with different teaching
student-led experiments, third grade. methods in the classroom and outside of it. Being subjected to it students
learn independently (more than through other methods), look, conduct ex-
periments and perform tasks which additionally activate them to participate
in the teaching process (Cvjetićanin, 2009, Özmen, Demircioğlu, Burhan,
Naseriazar & Demircioğlu, 2012). This kind of activity, primarily based on
Stanko Cvjetićanin, noticing, gathering and analyzing data through experiments can provide
Mirjana Maričić students with a deeper understanding of nature and enable developing their
University of Novi Sad, Serbia
problem-solving skills and research abilities. Furthermore it can teach them
how to make generalizations about the most important scientific elements,
634
ISSN 1648–3898 /Print/ THE CONTRIBUTION OF DEMONSTRATION AND STUDENT-LED EXPERIMENTS ON THE
STUDENTS’ KNOWLEDGE QUALITY IN THE THIRD GRADE OF PRIMARY SCHOOL
ISSN 2538–7138 /Online/ (P. 634-650)
to acquire it and thus form corresponding scientific knowledge (Tamir, 1977). Over the last couple of years there
have been a large number of experimental and observational research on the scientific laboratory work in schools
as well as at universities (Hofstein & Lunetta, 1982, Hofstein & Lunetta, 2003). Research on their performance in
subject teaching at primary, high schools and universities are especially notable. These research have mostly
dealt with the contribution of laboratory work on students (students at universities and in schools) knowledge
(McKee-Vickie, Williamson & Ruebush, 2007, Myers & Dyer, 2006); achievement; attitude towards nature (Hugerat,
Najami, Abbasi & Dkeidek, 2014, Ogundiwin, Asaaju, Adegoke & Ojo, 2015) interest to study certain subjects and
achieving better performance (Gendjova, 2007, Salameh El-Rabadi, 2013), i.e. (that is, in other words) having
greater success in mastering specific content: ‘’Reproduction, growth and development in living things“ (Cardak,
Onder & Dikmenli, 2007); effect on developing scientific and cognitive skills (Odubunmi & Balogun, 1991, Khan &
Iqbal, 2011); eliminating misconceptions (Sert-Çibik & Diken, 2008); understanding certain content and forming
alternative concepts (Özmen, Demircioğlu, Burhan, Naseriazar & Demircioğlu, 2012); increasing motivation; active
thinking; finding a suitable way of work (Stavreva-Veselinovska, Koleva-Gudeva & Djokic, 2011); developing crea-
tive and logic thinking (Koray & Serdar Köksal, 2009). Results of listed works have indicated a positive contribution
and application of laboratory work and conducting experiments, not only in courses, but also in classes in primary
and high school. Authors Ogundiwin, Asaaju, Adegoke & Ojo, (2015) researched the effect of application of group
research laboratory work on students’ performance in biology (high school) and the results showed that there is a
significant statistical difference in students of the experimental group who learned through the aforementioned
strategy. Salameh El-Rabadi (2013) conducted a research on the effect of laboratory experiments on students’
achievement in the 10th grade physics classes, and the results indicated that students of the experimental group,
who have studied through experiments, had a greater success when compared to the control group. Apart from
the mentioned research, it is also important to mention the contribution of those who dealt with the efficiency of
hands-on activities and research based learning as key aspects of laboratory-experimental work. These research
studied their influence on students’ achievement (Sadi & Cakiroglu, 2011); quality and duration of the acquired
knowledge (Logar & Savec-Ferk, 2011); acquiring and developing scientific skills and attitudes towards science
and specific subject content: biology, chemistry and physics (simple electric circuit) (Ateş & Eryilmaz, 2011, Ergül,
Şimşekli, Çaliş, Özdølek, Göçmençelebi & Şanli, 2011); change of misconceptions (Unal, 2008); students’ interest
and activity (Holstermann, Grube & Bögeholz, 2010, Maxwell, Lambeth & Cox, 2015); conceptual understanding of
the content and skill improvement (Şimşek & Kabapinar, 2010, Turpin & Cage, 2004). Results have shown a positive
contribution of the application of hands-on activities and IBL on students’ achievement (Bilgin, 2006), but have
also indicated that teachers/instructors can organize experiments, which include cheap and available materials in
order to improve students’ skills (Hırça, 2013). Authors Sadi & Cakiroglu, (2011) researched the influence of hands-
on activities on students’ achievement and their attitudes towards science, as well as their results in post-SAT test
which have shown higher students’ achievement in members of the experimental group who have studied through
hands-on activities, whereas there was no difference in students’ attitudes in both groups.
Research based on the application and contribution of laboratory-experimental work in the first four grades
of primary school are considerably rare. Few research have studied the influence of conducting experiments on
students’ understanding of the presented content (Cakici & Yavuz, 2010), efficiency and attitudes of students towards
its application, as well as the interest and motivation of students to learn the subject in question (Golubović-Ilić,
2011), the influence on both quantity and quality of students’ knowledge (Cvjetićanin, Obadović & Rančić, 2015,
Cvjetićanin, Segedinac & Halaši, 2010). The research in question proved positive contribution of application of the
laboratory method, but have also laid the ground for further research related to this phenomenon (Cvjetićanin,
Obadović & Rančić, 2015). If the focus is set to a single segment i.e. a single category in the experiment division, it
can be noticed the lack of research on their separate contribution such as: the contribution of application of basic,
comparative, model, long-term experiments etc. One of the most important experiment divisions for this research
is the one based on who conducts the experiment. These experiments can be either demonstration (performed by
a teacher, professor or a better trained student in front of the entire class) or student-led (performed by students:
individually, in pairs or in smaller groups) (Cvjetićanin & Segedinac, 2007). The efficiency of student-led experiments
compared to traditional (lecture) teaching methods was researched by Golubović-Ilić and Cvjetićanin, Segedinac &
Halaši, while a comparative analysis of application of demonstration and student-led experiments was presented
by Cvjetićanin, Obadović & Rančić. The research of Golubović-Ilić (2011) noted a positive contribution of student-
led experiments on achievement, attitudes, interest and motivation of third grade students when compared to
traditional teaching approach applied in the control group. The research of Cvjetićanin, Segedinac & Halaši, (2010)
635
THE CONTRIBUTION OF DEMONSTRATION AND STUDENT-LED EXPERIMENTS ON THE
(P. 634-650) ISSN 2538–7138 /Online/
studied the influence of student-led experiments on the quantity and quality of students’ knowledge in fourth
grade, and the results have shown better achievement for students of the experimental group when compared to
the control group in which students learned through traditional methods. The research of Cvjetićanin, Obadović
& Rančić, (2015) (which is of the highest relevance for this research) dealt with the efficiency of student-led and
demonstration experiments on the quality of students’ knowledge in the fourth grade and the results proved that
the students’ from the experimental group displayed better knowledge than students from the control group who
only saw the experiments demonstrated.
Focus and Aim of Research
Based on the analysis of the available research conducted on the contribution of the laboratory-experimental
method in teaching natural sciences in the first four grades of primary school, it can be seen that there is a small
number of comparative analysis done on the contribution of demonstration and student-led experiments on the
quality of students’ knowledge in realization of specific natural sciences content in different grades. All content is
vertically connected through the first four grades of the primary school and becomes gradually more complex,
which changes the aims and tasks of teaching. Students are taught air-related content in all four grades, but the
matter does not become complex before third grade, so it is very suitable for this type of research. Reasons to do
so are the following: students already have some knowledge of the air from previous grades and the complexity of
the content is suitable for application, making it suitable for determining the contribution of the methods which
are being assessed (laboratory-experimental method in this work).
The question is: Does the way of conducting experiment (by demonstration, or when students’ perform them)
contribute to the quality of students’ knowledge about air-related content in the third grade of primary school?
This question follows the basic aim of the research which is: Establishing the contribution of demonstration and
student-led experiments on the quality of students’ knowledge about the air-related content, in third grade of
primary school. The aim is followed by the next goals:
1. Determining if there are statistically significant differences in the quality of knowledge in students
who acquired air-related content through demonstration experiments when compared to those who
acquired it through independently conducted experiments;
2. Analyzing and establishing the contribution of demonstration i.e. student-led experiments on the
quality of acquired knowledge about the air-related content in different cognitive levels (knowledge,
understanding, application, analysis, evaluation and synthesis);
3. Determining if there are statistically significant differences in quality of duration of students’ knowledge
for the students who acquired air-related content through demonstration experiments when compared
to those who acquired it through independently conducted experiments.
Methodology of Research
General Background and Design
Demonstration and student-led experiments are fully developed experimentation models in all methodology
of teaching sciences (Cvjetićanin, 2009, De Zan, 2005) in the first four grades of primary school. Although this is a
well-known fact they are rarely applied in practice, and even less scientifically assessed for their contribution and
significance. For the needs of this research 16 experiments about air-related content were chosen (for four teach-
ing units). The same 16 experiments were prepared for the both groups of students (C and E), but so that each
group conducts it in a different way in order to determine their contribution. Their suitability was checked and
confirmed by the teachers who were a part of the research, as well as by the experienced methodologists in the
area of teaching sciences. The experiments’ model in this research was tested through the experimental design as
the best way for establishing their contribution.
An experimental research has been conducted. The main aim of the experimental research (research with an
experimental design) is proving the cause-effect relation i.e. causal reasoning between the variables (Fraenkel &
Wallen, 2003). This kind of research has an experimental factor (cause, independent variable) which is applied in
the educational process in order to cause a change as a consequence of the experimental factor (dependent vari-
able) (Knežević-Florić, Ninković, 2012). The research used the draft of the parallel-group experiment which implies
636
ISSN 2538–7138 /Online/ (P. 634-650)
two ways (which are compared) realized at the same time with at least one experimental and one control group in
order to establish their contribution. The research was conducted in a three and a half months’ time span (fourteen
weeks) of 2015/2016 (from the beginning of March to mid-June), and was realized through the following phases:
1. Preparatory phase: includes the analysis of pedagogical documentation in order to establish the GPA
(grade point average) in the end of the previous grade, as well as their GPA in the subject The World
around Us in the end of the second grade. Experiments which had been applied in realization of new
air-related knowledge were identified in this phase. Items for the pre-test were chosen in the end of this
phase, in order to establish the quality of the existing air-related knowledge. This phase was introduced
three weeks before the realization of phase two and lasted for the entire period;
2. Phase of revising the previously acquired air-related knowledge (acquired by the students in the pre-
vious two grades). This phase lasted for a single school lesson and was realized after the pedagogical
documentation had been reviewed;
3. Phase of measuring the initial state (quality of the existing air-related knowledge): includes measuring
the dependent variable through a pre-test. This phase lasted for a single school lesson and was realized
in the first lesson which followed revising previously acquired knowledge;
4. Group-creation phase (C and E): in this phase students were split into two groups which were made
equal in the following criteria: GPA in the end of the second grade; GPA in the subject The World around
Us in the end of the second grade; results (knowledge quality) acquired in pre-test and the number of
students (C group n=60, E group n=60). This phase was realized after the pre-test had been finished,
and it lasted for a week;
5. Phase of the experimental factor realization: introduction of the chosen treatments–experiments into
the experimental (16 student-led experiments) and the control group (16 demonstration experiments)
which are then compared. This phase lasted for two weeks (four school lessons). At each school lesson
is treated one unit, and within each teaching unit were performed four experiments;
6. The newly-acquired knowledge revision phase (acquired by students through experiments). This phase
lasted for a single school lesson and was realized in the very next class after the experimental factor
ended. The teacher went through all new air-related content with students. Doing that, special atten-
tion was put to gradually going from one level to another (a higher one) in asking questions. After the
revision was realized, items for the post-test and retest were chosen;
7. Phase of the measuring the final state (quality of the newly acquired air-related knowledge): which
includes measuring the dependent variable through a post-test. This phase was realized in the class
(lasting for a single school lesson) which immediately followed the lesson in which the newly acquired
air-related knowledge was revised;
8. Phase of the measuring the quality of knowledge duration: which included measuring the dependent
variable through a retention test – retest. This phase was realized a month after the post-test and lasted
for a single school lesson;
9. Final phase: comparing the state of the dependent variable (quality of students’ knowledge) based on
the results of the pre-test, post-test and the retest. This phase was realized after the retest was realized,
and it lasted for a week.
The basic aim of the experimental design is to establish the difference between the state before introducing
the experimental factor (initial knowledge – in this case results of the pre-test) and the state following its application
(final state – in this case results of the post-test and retest). The difference (between the knowledge quality before
and after introducing the treatment) is seen as the effect of the experimental factor (in this case the contribution
of the treatment i.e. demonstration and student-led experiments) (Knežević-Florić, Ninković, 2003).
The independent variable was: application of experiments (demonstration and student-led) in processing
air-related content in the third grade of primary school. The dependent variable was: the third grade students’
quality knowledge about air-related content on the six cognitive levels: knowledge, understanding, application,
analysis, evaluation and synthesis.
Sample
Participants of the research were 120 third grade students from six classes of two primary schools in the area
of the Autonomous Province of Vojvodina (Republic of Serbia). The students were split into two groups: C (group in
637
(P. 634-650) ISSN 2538–7138 /Online/
which experiments were demonstrated by the teacher) and E (group in which students independently conducted
experiments based on written instructions given by the teacher).
Instrument and Procedures
After the data of the students’ GPA in the end of the second grade was gathered through the analysis of the
pedagogical documentation (in group C it is 4.03 and in group E it is 4.08), and the students’ GPA in the subject The
World around Us in the end of the second grade (in group C it is 4.27 and in group E it is 4.15), it was started with
choosing items for further measuring. This research used pre-test, post-test and retest.
Pre-test
The pre-test is a combination of items from four sources, and was comprised of 12 items in total. Items were
taken from the authors: Kukić & Aćimović (2016), Stokanović & Lukić (2016), Tadijin (2006) and Životić (2016). Items
were designed to measure the quality of air-related knowledge acquired in previous grades at all cognitive levels.
An example of items at the levels of understanding and synthesis in the pre-test is given below (Figure 1).
LEVEL OF UNDERSTANDING
1: Wind is the movement of warm and cold air.
With the blue colour paint the arrow which shows the movement of the cold air,
and with the red colour paint the arrow which shows the movement of the warm air.
LEVEL OF SYNTHESIS
2: Look closely at the picture and answer the question.
What should you do so the bee could survive in the jar?
________________________________________________________________________
Explain your answer:
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
Figure 1: An example of items at the levels of understanding and synthesis in the pre-test (Stokanović & Lukić,
2016, Tadijin, 2006).
Post-test
The post-test is a combination of items from six sources and was comprised of 12 items in total. Items were
taken from the authors Andjelić, Erić & Vićentijević (2010), Blagdanić, Jović, Kovačević & Petrović (2016), Marinković
& Marković (2011), Matanović, Vlahović, Joksimović & Djurdjević (2015), Munitlak, Šikl-Erski & Holond (2016) and
Ralić-Žeželj (2016). Items were designed to measure the quality of the newly acquired air-related knowledge at all
638
ISSN 2538–7138 /Online/ (P. 634-650)
cognitive levels. An example of items at the levels of understanding and synthesis in the post-test is given below
(Figure 2).
1. Look at the picture on which are shown the experiments. What do we prove with these experiments. Circle the letter in front of the correct answers.
a) The air is all around us

b) The water is good solvent
c) The air has the shape of the space in which it is located
a) The air expands on heating

b) The air is moving in nature
c) The air is all around us
LEVEL OF SYNTHESIS
2: Why bird and fox in the winter is not cold?
______________________________________ __________________________________________
______________________________________ __________________________________________
Why bird feathers in the spring change, a fox sheds?
______________________________________ __________________________________________
______________________________________ __________________________________________
Figure 2: An example of items at the levels of understanding and synthesis in the post-test (Andjelić, Erić &
Vićentijević, 2010, Marinković & Marković, 2011).
Re-test

The retest is a combination of items from six sources and was comprised of 12 items in total. Items were taken
from the authors: Andjelić, Erić & Vićentijević (2010), Blagdanić, Jović, Kovačević & Petrović (2016), Marinković &
Marković (2011), Matanović, Vlahović, Joksimović & Djurdjević (2015), Munitlak, Šikl-Erski & Holond (2016) and Ralić-
Žeželj (2016). Items were designed to measure the quality of duration of the newly acquired air-related knowledge at
all cognitive levels. Based on the structure and content these items were similar to those given in the post-test, they
measured the same newly acquired air-related knowledge through items which were only formulated in a different
way. An example of items at the levels of understanding and synthesis in the post-test is given below (Figure 3).
639
(P. 634-650) ISSN 2538–7138 /Online/
1: Look at the picture on which are shown the experiments. What do we prove with this experiments. Circle the letter in front of the correct answers.
a) The air expands on heating

b) The air takes up space
c) The air is dissolved in the water
a) The wind moves the balloon

b) The air shrinks in cooling
c) When heated the air becomes lighter
LEVEL OF SYNTHESIS
2: Is it cold for eskimos in the igloo?
_____________________________________________________________________________________________
_____________________________________________________________________________________________
Why?
_____________________________________________________
_____________________________________________________
_____________________________________________________
_____________________________________________________
_____________________________________________________
_____________________________________________________
_____________________________________________________
_____________________________________________________
_____________________________________________________
_____________________________________________________
_____________________________________________________
_____________________________________________________
______________
Figure 3: An example of items at the levels of understanding and synthesis in the retest (Andjelić, Erić &
Vićentijević, 2010, Marinković & Marković, 2011).
All Tests
The items in tests were created based on the Anderson, Krathwohl & Bloom taxonomy which is a revised Bloom
taxonomy (Anderson, Krathwohl & Bloom, 2001). In choosing items (from the previously mentioned sources) which
would follow all cognitive levels, it was used the collection Smart tests: Teacher-made tests that help students learn
(Walker, Schmidt, 2004), which gives the basics of each cognitive level with item examples to suit them. Validity of
the tests was ensured by assessment, discussion and approval of several teachers with considerable experience
(over ten years), as well as assessment, discussion and approval of several experts on teaching natural sciences, as
experts in this area. Reliability of the test was calculated using the Cronbach’s alpha coefficient (alpha coefficient)
which was .86 for all tests. According to this coefficient, all values which are above .70 are acceptable and consid-
ered reliable. In all tests (pre-test, post-test and retest) the quality of students’ knowledge was measured through
two items for each cognitive level, with the items different in structure. In valuation of the items the applied rule
was for items of higher cognitive level to bear more points. At the cognitive level of knowledge the students were
supposed to recall new air-related knowledge, to detect and reproduce the information, idea and the principle
similarly to what they learned. In items at the level of understanding the students were asked to notice and connect
640
ISSN 2538–7138 /Online/ (P. 634-650)
main properties of air, understand the most important ideas, explain and interpret the learnt content and different
air-related processes (such as movement and pressure), as well as to make logical conclusions based on available
information on air i.e. to spot causal relations in air behavior (such as the influence of warmth on air weight). The
items at the level of application were constructed to make students solve problems by using the newly acquired
air-related content in learning context or in a new situation, followed by application of knowledge about air as an
isolator, the importance of clear air for living beings, protection of air from pollution etc. At the level of analysis
students were demanded to differentiate between important and insignificant information on the impact external
factors have on air (on its movement, pressure, weight…), parse the given information to establish the parts on
all acquired air-related knowledge, causal relations, causes and effects of different factors on air and its influence
on living beings. At the level of evaluation students were supposed to compare and discover similarities and dif-
ferences between air, oxygen and CO2, as well as the differences between gas and liquid matter of state i.e. gas
and solid matter of state. They were supposed to predict air behavior based on images showing different seasons,
to discover and explain how living beings would act in given conditions. In items on the synthesis level students
were supposed to creatively, or divergently use the acquired air-related knowledge to create a new concept on
mutual relationship of air, water and soil. They were also asked to make generalizations based on information of
the influence of different substances (materials) on air quality and how it would affect beings, environment etc.
Treatment
The chosen content for establishing the contribution of the way of conducting the experiments in the third
grade of primary school on the quality of students’ knowledge was Gasses as a matter of state-air. Content realized
in both groups C and E is comprised of the following teaching units:
1. Air pressure and movement (changes in state and weight);
2. Changes which occur due to heating and cooling of the air (changes in temperature, weight, movement…);
3. Air as an isolator;
4. The properties of air that are important for the living world and human activity.
The Treatment in the Control Group
The teacher in group C gave explanations to students before each experiment: students were informed which
equipment and materials were necessary for the experiment, how the experiment was going to be conducted as
well as how they should observe. This involved frontal instructions. After the experiment was demonstrated and
conclusions reached, students wrote down the process of the experiment in a separate notebook (experiments
notebook). The teacher wrote important conclusions on the board during the discussion so students can write
them down in their notebooks in the end.
The Treatment in the Experimental Group
In group E the teacher made heterogeneous groups of three students which differed in the initial knowledge:
good, average and bad. Members of the group were the same in all experiments. Before each lesson in which the
experiments were to be conducted the teacher would put the required equipment and materials on every desk. Each
member of the group would get a paper with experiment instructions. Students in group E were asked to carefully
read the instructions before the experiment, and repeat aloud to each other which tasks have to be conducted and
how to conduct the experiment. The paper had the information about the needed equipment and material, how
the experiment has to be conducted and the questions which have to be answered based on the results. After the
teacher makes sure that all students know what has to be done, the students proceed to conducting experiments.
All groups conducted experiments of same complexity. The level of complexity increased with each lesson. Inside
their groups, students tried to answer the questions based on the results they reached. After groups finish, a student
from each group presents the results in front of the class. When the presentation is done, the teacher encourages
students to discuss, correct the wrong conclusions and make the right ones. Students check if they wrote down
the correct procedure into their notebooks and if their conclusions were correct. After that, they rewrite into their
notebooks conclusions written on the board, reached as the joint effort of all groups.
An example of experiments (for both groups) for some of teaching units is given below (Figure 4).
641
(P. 634-650) ISSN 2538–7138 /Online/
Experiment for the unit: Changes which occur due to heating and cooling of the air
Necessary materials and supplies:

•• bowl with water
•• slot
•• glass bottle
•• balloon
Description of the experiment:

Take the bowl with water and put it on the heated slot. When the water is heated, put in the glass bottle, on which
outfall is pulled deflated balloon. Observe what happens with the balloon.
Think and answer:
•• Why the balloon inflated?
•• Based on the knowledge about the behavior of solids to heat, try to explain the behavior of air.
•• Provide examples of everyday life, where you can see the phenomenon described.
•• Note down the answers in your experiment notebook.
Experiment for the unit: The properties of air that are important for the living world and human activity.
Necessary materials and supplies

•• two candles
•• glass cup
•• matches
Description of the experiment:

Put on the table two candles and lit them up, using the matches. After that, one of the candles cover with the glass
cup. Now inhale and hold your breath for a while. What do you feel? Can you stay long in this condition?
Think and answer:
•• When the candle has been lit, which process has happened?
•• When the candle has been covered, what has happened and why?
•• What is needed from the air so the candle could burn?
•• Why is air important for all living beings?
•• Note down the answers in your experiment notebook.
Figure 4: An example of experiments for some of the teaching units.
Data Analysis
The acquired data were used for:

1. Balancing the groups: analyzing the GPA in the end of the second grade, students’ GPA in the subject
The World around Us in the end of the second grade, and results (knowledge quality) of the pre-test in
order to establish:
•• If there is a statistically significant difference between the students of groups C and E in the GPA in the
end of the second grade and their GPA in the subject The World around Us in the end of the second
grade, by establishing the following values: arithmetic mean (M ( X )), standard deviation (SD) and the
independent t-test;
•• Spearman’s correlation coefficient was applied to establish the level of correlation between the students’
GPA in the end of the second grade and their GPA in the subject The World around Us in the end of the
second grade and the results of the pre-test. The quality of students’ knowledge about air was analyzed
from two aspects – marks and the scored number of points in pre-test;
2. Analysis of the quality of newly acquired air-related content displayed by the students in the post-test
in order to establish:
•• The contribution of demonstration and student-led experiments on the quality of students’ knowledge
i.e. the difference in students’ knowledge between groups C and E, after the experimental part of the
642
ISSN 2538–7138 /Online/ (P. 634-650)
research was analyzed by calculating and comparing the GPA (of marks) and number of scored points
in post-test (for all cognitive levels). If there is a statistically significant difference in results i.e. quality
of students’ knowledge in groups C and E in the post-test, was acquired through the non-parametric
Mann-Whitney test and the independent t-test, whereas the Kolmogorov–Smirnov normality test was
used to establish if the acquired data correspond to the normal distribution;
3. Analysis of the quality of duration of the newly acquired air-related content displayed by the students
in the retest in order to establish:
•• The contribution of demonstration and student-led experiments on the quality of duration of students’
knowledge i.e. the difference in students’ knowledge between groups C and E after the experimental
part of the research was analyzed by calculating and comparing the GPA (of marks) and the number of
scored points in the retest (for all cognitive levels). If there is a statistically significant difference in results
i.e. quality of duration of students’ knowledge in groups C and E in the retest, was acquired through
the non-parametric Mann-Whitney test and the independent t-test, whereas the Kolmogorov–Smirnov
normality test was used to establish if the acquired data correspond to the normal distribution;
4. Comparative analysis of the quality of students’ knowledge about air-related content (all tests):
•• One-way ANOVA was used to establish if there is statistically significant difference in the quality of stu-
dents’ knowledge in C and E group at cognitive levels in pre-test and post-test as well as on the retest.
Results of Research
Balancing the Groups
Based on the values of the t-test it was established that there was no statistical significance in GPA difference
between the groups C and E in the end of the second grade (t=.598, p=.613). Furthermore, there was no statistically
significant difference in students’ GPA in the subject The World around Us in the end of the second grade (t=.603,
p=.592). The third parameter for balancing groups was the quality of students’ knowledge in pre-test of both groups
(marks and the number of scored points).
The quality of students’ knowledge in C and E group on the pre-test at different cognitive levels is shown in
the table (Table 1).
Table 1. Differences in the quality of students’ knowledge in C and E group on the pre-test at the same cogni-
tive levels.
Cognitive level Group M(X ) SD t p
E 3.4532 1.787
Knowledge .6621 .4130
C 3.633 1.705
E 4.496 .537
Understanding .7050 .4770
C 4.414 .496
E 5.338 1.151
Application .9230 .5930
C 5.903 1.097
E 8.322 2.216
Analysis .5850 .5650
C 8.407 2.086
E 0.947 1.421
Evaluation .9110 .9030
C 0.901 1.381
E 0.383 2.783
Synthesis .4130 .6110
C 0.414 1.951
643
(P. 634-650) ISSN 2538–7138 /Online/
The results acquired in the pre-test show that the students in groups C and E share similar knowledge about
air-related content in previous grades in all cognitive levels (t-test significance is above .05 for every cognitive
level) (Table 1). Neither C nor E group students solved the syntheses item correctly. Bad results were also present
at both analysis and evaluation level.
There should be correlation between the GPA of students in the end of second grade, GPA in the subject
The World around Us in the end of second grade and the quality of their knowledge on the pre-test. Correlation
level was established based on value of the Spearman’s correlation coefficient. Results have shown a moderate
connection between the students’ GPA in the end of the second grade and marks they got in the pre-test (ρ=.497
with p <.001). The same situation emerges in correlation with the total number of scored points in testing (ρ=.528
with p<0.01). Similar results appeared in establishing correlation between the students’ GPA in the subject The
World around Us in the end of the second grade and marks they got in the pre-test (ρ=.512 with p<.001), as well
as between their GPA in the subject The World around Us in the end of the second grade and the total number of
points they scored in the pre-test (ρ=.518 with p<.001). Based on the intensity of the correlation coefficient of re-
searched phenomena (variables), it can be spot moderate correlation which was unexpected (correlation intensity
significantly lower than expected).
Analysis of the Quality of the Students’ New Knowledge of the Air-Related Content
The contribution of the way of conducting an experiment to the quality of students’ knowledge about air in
C and E group on the post-test at different cognitive levels is shown in the table (Table 2).
Table 2. Differences in the quality of students’ knowledge in C and E group on the post-test at the same
cognitive levels.
Cognitive level Group M(X ) SD CV (%) t p

E 2.889 .86 20.218
C 2.668 .94 21.13
E 3.792 0.98 21.22
C 3.688 1.31 23.62
E 5.087 6.102 18.87
C 4.581 5.755 19.56
E 9.225 1.88 5.23
Analysis .9060 .0630
C 5.141 2.09 6.38
E 5.344 1.64 4.98
Evaluation 4.976 .0001
C 2.298 7.23 19.03
E 3.771 2.14 4.62
Synthesis 8.865 .0001
C 0.987 9.12 15.97
The results obtained from post-test show that the students from the E group performed better, meaning they
acquired the knowledge of higher quality when compared to the students in the C group.
After analyzing the quality of students’ knowledge of both groups at some cognitive levels it is possible to
say that C and E group students shared similar results in: knowledge (t=.7970, p=.0730), understanding (t=.8550,
p=.4030), application (t=.6810, p=.3870) and analysis (t=.9060, p=.0630). The E group students (Table 2) performed
better than C group students in evaluation (t=4.976, p=.0001) and synthesis (t=8.865, p=.0001). If this is compared
to the number of students’ in C group who successfully completed pre-test and post-test items at the level of analy-
sis, it can be seen that this number is higher at the post-test. On the pre-test, none of C group students managed
644
ISSN 2538–7138 /Online/ (P. 634-650)
to successfully complete both items at the analysis level and only 8.33% have completed only one, while 34.77%
completed both items on the post-test and 43.42% completed only one. The progress of C group at the evaluation
level is higher than on the pre-test. Although none of C group students completed both items at the evaluation
level, 26.64% completed only one. At the synthesis level 6.66% did only one and the rest did not do anything at
this level. The E group outperformed C group in solving both items with 53.42% and only one with 41.76% at the
evaluation level. Their success was by far the best at the level of synthesis. Although only 13.33% managed to
complete both items, 39.84% did only one.
In order to determine if the existing differences (in favor of students from E group) are statistically significant,
it’s tested the hypothesis of the normal data distribution.
The obtained values of the Kolmogorov-Smirnov test of normality of the students in the C and E group at the
post-test and the retest are shown in the table (Table 3).
Table 3. Results of students C and E group on the Kolmogorov–Smirnov normality test.
One-Sample
Kolmogorov- Group N Kolmogorov-Smirnov Z Asymp. Sig. (2-tailed)
Smirnov test
Post-test
E 60 .9020 .0001
The number of C 60
scored points Retest
E 60 .8740 .0001
C 60
Based on the obtained value of the Kolmogorov–Smirnov normality test (Z=.9020, p=.0001) it was noted that
data does not have normal distribution and that statistical significance of the difference (Table 3) between students
in groups C and E in post-test should be confirmed through the Mann-Whitney non-parametric test.
Statistical significance of difference in the quality of students’ knowledge between groups C and E in post-test
is acquired through the Mann-Whitney non-parametric test, and it is shown in the table (Table 4).
Table 4. Difference in the quality of students’ knowledge in C and E group (post-test) on the Mann-Whitney
non-parametric test.
The total number of points
Mann-Whitney U 4385.000
Wilcoxon W 10888.000
Z -3.908
Asymp. Sig. (2-tailed) .0001
The significance obtained through the Mann-Whitney test (p=.0001) shows that there is a statistically signifi-
cant difference in results of the post-test in groups C and E (Table 4).
Analysis of the Quality of Duration of the Students’ New Knowledge of the Air-Related Content
Quality of duration of the students’ knowledge in both groups on the retest at different cognitive levels is
shown in the table (Table 5).
645
(P. 634-650) ISSN 2538–7138 /Online/
Table 5. Differences in the quality of duration of students’ knowledge in C and E group on the retest at the
same cognitive levels.
Cognitive level Group M(X ) SD CV (%) t p
E 2.921 .94 22.72

D 2.813 81 25.58
E 3.926 1.42 18.87

D 3.711 1.35 20.56
E 4.988 .910 16.79

D 3.962 .7.44 18.04
E 9.022 1.43 4.79

Analysis 1.957 .0020
D 5.121 7.11 18.36
E 5.022 1.29 5.12

Evaluation 5.226 .0001
D 2.003 6.98 19.27
E 3.887 2.18 4.95

Synthesis 5.885 .0001
D 0.793 8.81 15.89
Results showed that E group acquired a more durable knowledge about air-related content than group C (Table
5) i.e. demonstration experiments did not contribute to durability (quality of duration) of the students’ knowledge
comparing to student-led experiments.
Unlike on the post-test students of both groups displayed similar amount of knowledge at the first three
cognitive levels. They were equal in knowledge (t=.7570, p=.7240), understanding (t=.8860, p=.3460) and applica-
tion (t=.9720, p=.1140). Significant difference was at the levels of analysis (t=1.957, p=.0020), evaluation (t=5.226,
p=.0001) and synthesis (t=5.885, p=.0001). Although the post-test showed no statistically significant difference in
results at the cognitive level of analysis, on the retest it appeared as significant. Two items at the level of analysis
(on the retest) were correctly solved by 13.22% students in group C, and by 44.94% in group E. One correctly solved
item in the group C was acquired by 6.66% of the students, while in group E it was 39.96%. None of the students
in group C managed to solve both items at the level of evaluation, while 11.66% did only one. In group E 23.33%
of the students answered both items at the level of evaluation, and only 15% answered only one. At the level of
synthesis none of the students in group C managed to solve any of the items. In group E 18,33% of the students
solved one of the items at the level of synthesis but no one did both on the retest.
Based on the obtained value of the Kolmogorov–Smirnov normality test (Z=.8740, p=.0001) it was noted that
data does not have normal distribution and that statistical significance of the difference (Table 3) between students
in groups C and E in retest should be confirmed through the Mann-Whitney non-parametric test.
Statistical significance of difference in the quality of duration of students’ knowledge between groups C and
E in the retest is acquired through the Mann-Whitney non-parametric test, and it is shown in the table (Table 6).
Table 6. Difference in the quality of duration of students’ knowledge in C and E group (retest) on the Mann-
Whitney non-parametric test.
The total number of points
Mann-Whitney U 3983.000
Wilcoxon W 10106.000
Z -3.116
Asymp. Sig. (2-tailed) .0001
646
ISSN 2538–7138 /Online/ (P. 634-650)
Statistical significance obtained through the Mann-Whitney test (p=.0001) shows that there is a significant
statistical difference between the results of the retest in groups C and E (Table 6).
Comparative Analysis of the Quality of Students’ Knowledge (ANOVA Analysis)
Based on the comparison of the quality of students’ knowledge in each group (C and E) at the same cognitive
levels in the pre-test, post-test and the retest it was concluded that there is no significant difference in number of
students who solved items at the level of knowledge and understanding (value of F-test at both levels is higher than
.005). Statistically significant difference between the groups in the post-test and the retest was noted at levels of
application (E: F=98.883, p=.0001; C: F=9.145, p=.0001), analysis (E: F=7.221, p=.001; C: F=1.87 p=.0001), evaluation
(E: F=7.425, p=.001; C: F=9.112, p=.0001) and synthesis (E: F=10.986, p=.001; C: F=10.561, p=.0001).
Discussion
Application of the laboratory-experimental method in teaching sciences in the first four grades of pri-
mary school contributes to the quality of students’ knowledge. This is confirmed through research conducted by
Cvjetićanin, Obadović & Rančić, (2015) and Cvjetićanin, Segedinac & Halaši, (2010). Based on the available research
done on the contribution of the laboratory-experimental method in teaching sciences in the first four grades of
primary school, it can be concluded that these were mostly done as comparative analyses of student-led experi-
ments and the traditional (lecture) method (Cvjetićanin, Segedinac & Halaši, 2010, Golubović-Ilić, 2011). There is a
small number of comparative research done on the contribution of demonstration and student-led experiments
that they have on the quality of students’ knowledge in the first four grades of primary school. This is agreed upon
by authors Cvjetičanin, Obadović & Rančić (2015). Since the research done in this area is rare and there is no research
done about the contribution of demonstration and student-led experiments in air-related content for the third
grade students, this research was conducted as a scientific contribution to the mentioned thematics.
Previous to the application of the experimental factor, both groups had been balanced in the aforementioned
criteria. Students from both groups (C and E) had shown similar knowledge in the pre-test, with worse results in
higher cognitive levels (analysis, evaluation and synthesis). The reason for decreased results at higher cognitive
levels is probably due to the way in which the air-related content was acquired in previous levels, as well as the
process of forgetting the previously learnt. In discussion with teachers, it was concluded that they have realized
air-related content in classroom (previous classes) through verbal and written (textual) methods (the traditional,
lecture approach), without using the laboratory-experimental method. This starting point has its justification in
results of numerous research (in the comparison of the efficiency of application of the traditional, lecture approach
with laboratory-experimental method), which have shown higher achievement and improvement in students who
gained knowledge through the experimental-laboratory method, in subject (Cardak, Onder & Dikmenli, 2007,
Odubunmi & Balogun, 1991, Ogundiwin, Asaaju, Adegoke & Ojo, 2015), as well as in the first four grades of primary
school (Cvjetićanin, Segedinac & Halaši, 2010, Golubović-Ilić, 2011).
The students from both groups shared the similar knowledge on the levels: knowledge, understanding, ap-
plication and analysis, while on the higher cognitive levels (evaluation and synthesis) students from the group
E were more successful, they adopted the knowledge of the higher quality then the students from the group C.
When the results of the C group students from the pre-test and the post-test are compared, it can be noted that
the demonstration of experiments had a significant contribution on the increase of the quality of the students’
knowledge at the level of analysis and partly at the level of evaluation, while only slightly affected the quality of
the students’ knowledge at the level of synthesis. Research conducted by Cvjetićanin, Obadović & Rančić, (2015)
acquired results which (unlike the listed data) shows that demonstration experiments had less contribution on the
quality of knowledge in group C, because there was a statistically significant difference even at the level of analysis
in favor of students who conducted experiments independently, which was not the case in this research. Research
of McKee-Vickie, Williamson & Ruebush (2007), that was aimed to prove the influence of demonstration experiments
on students’ knowledge, has confirmed the positive contribution of demonstration and hands on experiments, i.e.
a higher level of knowledge (conceptual understanding of the content) after the intervention in both groups. They
have also shown that, when compared to hands-on experiments, demonstration experiments did not decrease
students’ (biology students’) conceptual understanding of the phenomena in question. The significance obtained
through the Mann-Whitney test shows that there is a statistically significant difference in results of the post-test
647
(P. 634-650) ISSN 2538–7138 /Online/
in groups C and E, in favor of the students from the group E, who have acquired the knowledge of higher quality.
The obtained results demonstrate the fact that the student-led experiments contribute more to the quality of the
knowledge than the demonstration experiments. Contrary to this conclusion, the research of Logar & Savec-Ferk
(2011) showed that students who learned through demonstration experiments acquired higher knowledge quality
when compared to those who conducted the experiments independently.
On the retest, the students from the C and E group shared the similar knowledge on the levels: knowledge,
understanding and application, while the students from the group E were more successful on the higher cogni-
tive levels (analysis, evaluation and synthesis) i.e. they adopted the more durable knowledge than the students’
from the group C. The obtained results were confirmed by the research of the following authors: Cvjetićanin,
Obadović & Rančić, (2015), in which both groups displayed similar knowledge in lower cognitive levels, with
the students who conducted student-led experiments being notably better in levels of analysis, evaluation and
synthesis when compared to the students who only saw those experiments demonstrated. When the results of
the students’ knowledge about the air-related content are compared at all cognitive levels based on the post-test
and retest, it can be seen that the students from both groups have acquired the knowledge of the lower quality
on the retest, then the knowledge acquired on the post-test. This was expected due to spontaneous forgetting
(Robbins, Schwartz & Wasserman, 2001). Students did not revise previously learnt between the post-test and the
retest, but did study different subject and thus, their air-related knowledge was disrupted (Sternberg & Zhang,
2001). Statistical significance obtained through the Mann-Whitney test shows that there is a significant statistical
difference between the results of the retest in groups C and E in favor of the students from the group E, who have
acquired the knowledge of higher durability.
Based on the comparison of the quality of students’ knowledge in each group (C and E) at the same cognitive
levels in the pre-test, post-test and the retest, it was concluded that there is no significant difference in number
of students who solved items at the levels of knowledge and understanding. Statistically significant difference
between the groups in the post-test and the retest was noted at levels of: application, analysis, evaluation and
synthesis. These results point to the fact that the application of experiments in both groups led to acquiring new
and perfecting existing knowledge (McKee-Vickie, Williamson & Ruebush, 2007) about air-related content. Similar
results were obtained by authors Cvjetićanin, Obadović & Rančić (2015).
Conclusions
The way of conducting experiments (demonstration or student-led) contributes to the quality of the third
grade students’ knowledge about the air-related content. Demonstration experiments have a significant contribu-
tion to the increase of the quality of students’ knowledge about air at the level of analysis, and partially at the level
of evaluation. However, their contribution to the increase in knowledge duration decreases by time at all levels
i.e. the students were not as successful at these levels while solving items during the retest. Unlike demonstration
experiments, student-led experiments contribute to the increase and duration of students’ knowledge quality at
the levels of analysis, evaluation and synthesis. More than demonstration experiments, student-led experiments
contribute to the students’ capability to: analyze, estimate, compare, rearrange, formulate, organize, etc. i.e. they
more contribute to the acquisition of the quality of students’ knowledge. The reason for higher contribution of
student-led experiments to the quality of the students’ knowledge about the air can be found in the facts: that
this type of experiment gives the students a chance to independently research, they follow what happens more
actively, in smaller groups they discuss mutually and obtain the conclusions of the results together, unlike those
students who merely watch the experiments during their demonstration by the teacher. These reasons clearly
point to the fact that realization of air-related content in the third grade of primary school (as well as in realization
of other natural sciences content) should be primarily done through student-led experiments. In this manner, in
addition to acquiring more quality knowledge, students develop higher experimental skills, get acquainted with
stages of scientific research and learn how to observe natural processes and phenomena through experiments.
Bearing in mind a relatively small amount of research done in this area, there is a need for more research
conducted with larger samples and related to the different content of the integrated science education in order to
establish the contribution of demonstration and student-led experiments on the quality of students’ knowledge
in realization of different content in teaching integrated natural sciences as a part of primary school.
648
ISSN 2538–7138 /Online/ (P. 634-650)
References
Anderson, L. W., Krathwohl, D. R., & Bloom, B. S. (2001). A taxonomy for learning teaching and assessing: a revision of Bloom’s tax-
onomy of educational objectives. New York: Longman Publishing.
Andjelić, V., Erić, V., & Vićentijević, M. (2010). Priroda i društvo – Radna sveska za treći razred osnovne škole [Nature and society - The
workbook for the third grade of primary school]. Kragujevac: Izdavačka kuća “ATOS”.
Ateş, Ö., & Eryilmaz, A. (2011). Effectiveness of hands-on and minds-on activities on students’ achievement and attitudes towards
physics. Asia-Pacific Forum on Science Learning and Teaching, 12 (1), 1-22.
Bilgin, I. (2006). The effects of hands-on activities incorporating a cooperative learning approach on eight grade students’ science
process skills and attitudes toward science. Journal of Baltic Science Education, 9 (1), 27-37.
Blagdanić, S., Jović, S., Kovačević, Z., & Petrović, A. (2016). Priroda i društvo 3 – Radna sveska za treći razred osnovne škole [Nature
and society 3 - The workbook for the third grade of primary school]. Beograd: BIGZ školstvo.
Cakici, Y., & Yavuz, G. (2010). The effect of constructivist science teaching on 4th grade students’ understanding of matter. Asia-
Pacific Forum on Science Learning and Teaching, 11 (2), 1-19.
Cardak, O., Onder, K., & Dikmenli, M. (2007). Effect of the usage of laboratory method in primary school education for the achieve-
ment of the students’ learning. Asia-Pacific Forum on Science Learning and Teaching, 8 (2), 1-11.
Cvjetićanin, S. (2009). Metodika nastave poznavanja prirode I [Methodics of teaching natural sciences I]. Sombor: Pedagoški fakultet.
Cvjetićanin, S., Obadović, D., & Rančić, I. (2015). The efficiency of student-led and demonstration experiments in initial physics-
chemical education in primary school. Croatian Journal of Education, 17 (3), 11-39.
Cvjetićanin, S., & Segedinac, M. (2007). Primena eksperimenta u saznajnom procesu nastave poznavanja prirode [Application of
experiment in the cognitive process in teaching of nature]. Inovacije u nastavi, 20 (3), 132-141.
Cvjetićanin, S., Segedinac, M., & Halaši, T. (2010). Značaj primene metode eksperimenta u razrednoj nastavi [The importance of
the application of experimental method in classroom teaching]. Nastava i vaspitanje, 2, 173–189.
De Zan, I. (2005). Metodika nastave prirode i društva [Methodics of teaching nature and society]. Zagreb: Školska knjiga.
Fraenkel, J. R., & Wallen, N. E. (2003). How to design and evaluate research in education. New York: McGraw-Hill.
Ergül, R., Şimşekli, Y., Çaliş, S., Özdølek, Z., Göçmençelebi, Ş., & Şanli, M. (2011). The effects of inquiry-based science teaching on
elementary school students’ science process skills and science attitudes. Bulgarian Journal of Science and Education Policy
(BJSEP), 5 (1), 48-68.
Eshach, H. (2006). Science literacy in primary schools and pre-schools. New York: Springer Netherlands. doi: 10.1007/1-4020-4674-X.
Gendjova, A. (2007). Enhancing students’ interest in chemistry by home experiments. Journal of Baltic Science Education, 6 (3), 5-15.
Golubović-Ilić, I. (2011). Stavovi i mišljenja učenika o primeni laboratorijsko-eksperimentalne metode u nastavi prirode i društva
[The attitudes and opinions of students on the application of laboratory-experimental method in teaching of science and
society]. Pedagogija, 4, 674-685.
Hart, C., Mulhall, P., Berry, A., Loughran, J., & Gunstone, R. (2000). What is the purpose of this experiment? Or can students learn
something from doing experiments? Journal of Research in Science Teaching, 37 (7), 655-675.
Hırça, N. (2013). The influence of hands on physics experiments on scientific process skills according to prospective teachers’
experiences. European Journal of Physics Education, 4 (1), 1-9.
Hofstein, A., & Lunetta, V. N. (1982). The role of the laboratory in science teaching: Neglected aspects of research. Review of
Educational Research, 52 (2), 201–217.
Hofstein, A., & Lunetta, V. N. (2003). The laboratory in science education: Foundations for the twenty-first century. Science Educa-
tion, 88, 28–54. doi: 10.1002/sce.10106.
Holstermann, N., Grube, D,, & Bögeholz, S. (2010). Hands-on activities and their influence on students’ interest. Research in Sci-
ence Education, 40 (5), 743-757.
Hugerat, M., Najami, N., Abbasi, M., & Dkeidek, I. (2014). The cognitive acceleration curriculum as a tool for overcoming difficul-
ties in the implementation on inquiry skills in science education among primary school students. Journal of Baltic Science
Education, 13 (4), 523-532.
Karamustafaoğlu, S. (2011). Improving the science process skills ability of science teachers using I diagrams. Eurasian Journal of
Physics and Chemistry Education, 3 (1), 26-38.
Katcha, M. A., & Wushishi, D. I. (2015). Effects of laboratory equipment on secondary school students’ performance and attitude
change to biology learning in federal capital territory, Abuja, Nigeria. Journal of Education Research and Behavioral Sciences,
4 (9), 250-256.
Khan, M., & Iqbal, M. Z. (2011). Effect of inquiry lab teaching method on the development of scientific skills through the teaching
of biology in Pakistan. Language in India, 11 (1), 169-178.
Knežević-Florić, O., Ninković, S. (2012). Horizonti istraživanja u obrazovanju [The horizons of the research in education]. Novi Sad:
Filozofski fakultet, Odsek za pedagogiju.
Koray, O., & Serdar Köksal, M. (2009). The effect of creative and critical thinking based laboratory applications on creative and
logical thinking abilities of prospective teachers. Asia-Pacific Forum on Science Learning and Teaching, 10 (1), 1-13.
Kukić, M., & Aćimović, M. (2016). Svet oko nas 2 - Radna sveska za drugi razred osnovne škole [The world around us 2 - The workbook
for the second grade of primary school]. Čačak: Pčelica.
Logar, A., & Savec Ferk, V. (2011). Students’ hands-on experimental work vs lecture demonstration in teaching elementary school
chemistry. Acta Chimica Slovenica, 58 (4), 866–875.
649
(P. 634-650) ISSN 2538–7138 /Online/
Marinković, S., & Marković, S. (2011). Priroda i društvo – Radna sveska za treći razred osnovne škole [Nature and society - The work-
book for the third grade of primary school]. Beograd: Kreativni centar.
Matanović, V., Vlahović, B., Joksimović, S., & Djurdjević, M. (2015). Priroda i društvo 3b – Radna sveska za 3. razred osnovne škole
[Nature and society 3b - The workbook for the 3. grade of primary school]. Beograd: Eduka d.o.o.
Maxwell, O. D., Lambeth, T. D., & Cox, T. J. (2015). Effects of using inquiry-based learning on science achievement for fifth-grade
students. Asia-Pacific Forum on Science Learning and Teaching, 16 (1), 1-31.
McKee-Vickie, K., Williamson, M., & Ruebush, E. (2007). Effects of demonstration laboratory on student learning. Journal of Science
Education and Technology, 16 (5), 395-400. doi: 10.1007/s10956-007-9064-4.
Munitlak, M., Šikl-Erski, A., & Holond, A. (2016). Priroda i društvo 3 – Radna sveska za treći razred osnovne škole [Nature and society
- The workbook for the third grade of primary school]. Beograd: Novi Logos.
Myers, E. B., & Dyer, E. D. (2006). Effects of investigative laboratory instruction on content knowledge and science process skill
achievement across learning styles. Journal of Agricultular Education, 47 (4), 52-63. doi: 10.5032/jae.2006.04052.
Odubunmi, O., & Balogun, T. A. (1991). The effect of laboratory and lecture teaching methods on cognitive achivement in inte-
grated science. Journal of Research in Science Teaching, 28 (3), 213-224. doi: 10.1002/tea.3660280303.
Ogundiwin, O. A., Asaaju, O. A., Adegoke, A. I., & Ojo, A. T. (2015). Effect of group investigative laboratory strategies on students’
achievement in biology. Pyrex Journal of Research in Environmental Studies, 2 (4), 35-41.
Özmen, H., Demircioğlu, G., Burhan, Y., Naseriazar, A., & Demircioğlu, H. (2012). Using laboratory activities enhanced with concept
cartoons to support progression in students’ understanding of acid-base concepts. Asia-Pacific Forum on Science Learning
and Teaching, 13 (1), 1-29.
Ralić-Žeželj, R. (2016). Maša i Raša - Priroda i društvo – Radna sveska za treći razred osnovne škole [Masha and Rasa - Nature and
society - The workbook for the third grade of primary school]. Beograd: Klett.
Robbins, S., Schwartz, B., & Wasserman, E. (2001). Psychology of learning and behavior. New York: W.W. Norton & Company.
Sadi, Ö., & Cakiroglu, J. (2011). The effects of hands-on activity enriched instruction on students’ achievement and attitudes
towards science. Journal of Baltic Science Education, 10 (2), 87-97.
Salameh El Rabadi, E. G. (2013). The effect of laboratory experiments on the upper basic stage students achievement in physics.
Journal of Education and Practice, 4 (8), 62-71.
Sert-Çibik, A., & Diken, H. E. (2008). The effect of group works and demonstrative experiments based on conceptual change ap-
proach: Photosynthesis and respiration. Asia-Pacific Forum on Science Learning and Teaching, 9 (2), 1-22.
Şimşek, P., & Kabapinar, F. (2010). The effects of inquiry-based learning on elementary students’ conceptual understanding of
matter, scientific process skills and science attitudes. Procedia Social and Behavioral Sciences, 2 (2), 1190-1194.
Stavreva-Veselinovska, S., Koleva-Gudeva, L., & Djokic, M. (2011). The effect of teaching methods on cognitive achievement in
biology studying. Procedia Social and Behavioral Sciences, 15, 2521–2527.
Sternberg, R., & Zhang, L. (2001). Perspectives on thinking, learning, and cognitive styles. New Jersey: Lawrence Erlbaum Associates.
Stokanović, Lj., & Lukić, G. (2016). Svet oko nas 2 – Radna sveska za drugi razred osnovne škole [The world around us 2 - The work-
book for the second grade of primary school]. Beograd: Novi Logos.
Tadijin, A. (2006). Svet oko nas - Radna sveska iz Sveta oko nas za drugi razred [The world around us - The workbook form the World
around us for the second grade]. Novi Sad: DOO “Školska knjiga”.
Tamir, P. (1977). How are laboratories used? Journal of Research in Science Teaching, 14 (4), 311-316. doi: 10.1002/tea.3660140408.
Turpin, T., & Cage, B. N. (2004). The effects of an integrated, activity-based science curriculum on student achievement, science
process skills, and science attitudes. Electronic Journal of Literacy through Science, 3, 1-17.
Unal, S. (2008). Changing students’ misconceptions of floating and sinking using hands-on activities. Journal of Baltic Science
Education, 7 (3), 134-146.
Walker, C., & Schmidt, E. (2004). Smart tests: Teacher-made tests that help students learn. Ontario: Pembroke Publishers.
Životić, B. (2016). Maša i Raša - Svet oko nas 2 – Radna sveska za drugi razred osnovne škole [Masha and Rasa - The world around
us 2 - The workbook for the second grade of primary school]. Beograd: Klett.
Received: May 17, 2017 Accepted: August 05, 2017
Stanko Cvjetićanin PhD, Professor, Faculty of Education, University of Novi Sad, Podgorička
(Corresponding author) 4, 25 000 Sombor, Serbia.
E-mail: tozchemy@eunet.rs
Website: http://www.pef.uns.ac.rs/
Mirjana Maričić MSc, PhD Student, Faculty of Education, University of Novi Sad,
Podgorička 4, 25 000 Sombor, Serbia.
E-mail: mimiunt@hotmail.com
Website: http://www.pef.uns.ac.rs/
650
THE VALIDITY AND
EFFECTIVENESS OF PHYSICS
INDEPENDENT LEARNING ISSN 1648-3898 /Print/
MODEL TO IMPROVE PHYSICS ISSN 2538-7138 /Online/
PROBLEM SOLVING AND SELF-

DIRECTED LEARNING SKILLS
OF STUDENTS IN OPEN
AND DISTANCE EDUCATION
Abstract. The Physics Independent
SYSTEMS Learning (PIL) model is an authentic
problem-based model designed teaching
guide for improving self-directed learning
and problem-solving skills in open and
distance education. This research is aimed
Paken Pandiangan, to analyze the validity and effectiveness
I Gusti Made Sanjaya, of PIL model. This research was conducted
Budi Jatmiko using focus group discussions of experts
that consisted of three science education
experts and applies quasi-experiment of
one group pre-test and post-test design to
144 students in East Java, Indonesia. Before
applying the lesson using the PIL model,
the students are given pre-test and after
Introduction accomplishing the learning, the students
are given post-test. The data collected from
Open and Distance Education Systems (ODES) in various countries today pre-test and post-test then is further ana-
apply dual pattern learning that is face-to-face tutorial and online tutorial lyzed by means of validity coefficient (rα),
teaching. During online pattern learning, the students can use internet-based Cronbach’s alpha (α), pair t-test, n-gain and
information and communication technology, either virtually or via video ANOVA. The result of research shows that rα
conference. Meanwhile, during face-to-face tutorial pattern, the students and = .75 and α = .92 is for content validity; rα
tutors conduct a meeting in such a limited period. The application of face-to- = .79 and α = .99 is for construct validity,
face tutorial on ODES in Indonesia today generally uses problem-based learn- so that PIL model is validity and reliability
ing and cooperative learning models (Tutorial Guidelines for Student, 2016). qualified. In addition, there are increasing
The problem-solving skills and self-directed learning skills using PBL scores of physics problem solving skills and
model and CL model of ODES achieved from physics learning are still low and self-directed learning skills of α = 5% with
under average level. Such a low level of students’ achievement theoretically moderate category of n-gain consistent in
is due to several issues, those are: lack of involvement, lack of convincing a limited trial test and of high category in a
and autonomy support (Brophy, 2013); low Achievement motivation, lack broader trial test for all groups.
of interest and absence of academic motivation (Legault, Green-Demers, & Keywords: PIL model, model validity,
Pelletier, 2006), lack of outcome and ability (Zimmerman & Schunk, 2011), model effectiveness, physics problem-
low intentionality and students’ poor self-regulatory (Schunk, 2005), lose solving skills, self-directed learning skills,
initiative and lacking the intention (Yew & Schmidt, 2009), learning model electricity, magnetism.
affects students’ conceptual development positively (Adeyemi & Adeyemi,
2014); the students are less capable of being more participated, lacking self-
confidence prior to the course and have such a poor responsibility as well as Paken Pandiangan
less-supportive learning environment (Pless, Maak, & Stahl, 2011). Indonesia Open University, Indonesia
The defects of PBL model and CL model can be fixed by developing I Gusti Made Sanjaya,
Budi Jatmiko
an alternative model that may improve physics problem solving skills and State University of Surabaya, Indonesia
self-directed learning skills of students in ODES. The corresponding teach-
651
THE VALIDITY AND EFFECTIVENESS OF PHYSICS INDEPENDENT LEARNING MODEL TO
IMPROVE PHYSICS PROBLEM SOLVING AND SELF-DIRECTED LEARNING SKILLS OF
STUDENTS IN OPEN AND DISTANCE EDUCATION SYSTEMS ISSN 2538–7138 /Online/
(P. 651-665)
ing model to be developed is Physics Independent Learning (PIL) that conforms to the learning characteristics of
adults and it is the innovation of PBL model and CL model with some modifications. Therefore, it is applicable to
all students at various academic levels, education and culture levels and ages as assigned to ODES implementation
(“Regulation of the Minister of Education and Culture of the Republic of Indonesia,” 2012).
PIL model development corresponds to the demand of curriculum and the development of globalization
era in 21st century. PIL model is an innovative physics-learning model that may encourage every individual person
to solve problem and teach the stages of problem solving. This is to be able to think critically and independently
(Silva, 2009) in which during the physics learning, individual. PIL model development applies scientific approach
by means of investigation, assignment, collaboration, discussion, and presentation method, so that it can act as
bridge for the gap between the required competence of graduates and the real condition as demanded by the
21st century and Indonesian National Qualifications Framework based curriculum (Adams, Vista, Scoular, Awwal, &
Griffin, 2015; “Regulation of the Minister of Education and Culture of the Republic of Indonesia,” 2013).
The importance of physics problem solving skills and self-directed learning skills of physics learning at uni-
versity level is based on the characteristics of physics subject. This is because physics is assumed relatively difficult
and complex. This indicates that problem solving is a critical element of learning physics although the students
have good ability to solve problems there is evidence that quite basic conceptual understanding is still very weak
(Prahani, Pandiangan, & Nasir, 2015). Curriculum demand and globalization era development require education
institutions to do a useful innovation for 21st century skills based education (Adams, Vista, Scoular, Awwal, & Griffin,
2015). Education and Culture Minister’s Regulation No.73 regarding Indonesian National Qualifications Framework
(INQF) of higher education requires universities to compose a curriculum that the students have such a superior
competence accompanied by various skills those correspond to the demand of 21st century, such as problem solv-
ing skills and self-directed learning skills (Griffin & Care, 2015; “Regulation of the Minister of Education and Culture
of the Republic of Indonesia,” 2013). Learning in the 21st century requires competent human resources, and the
students are directed to achieve learning skills and innovation, such as problem solving skills, self-directed learning
skills, critical thinking, creative thinking, responsible, and able to learn independently (Tucker, 2014)
During physics teaching, the problem-solving skills of students are generally weak. There are factors those af-
fect such a weak skill of students in physics problem solving, namely: the research of Brad (2011) shows that while
students apply basic strategies well, they use a trial approach, they give up when facing difficulties and have a lack
of metacognitive ability, which is a signal to be considered, greater attention should be given to the needs of the
students, giving more emphasis on reasoning and understanding, so that students can improve their self-directed
learning; the result of research done by Jatmiko, Widodo, Budiyanto, Wicaksono, & Pandiangan (2016) indicates
that General Physics learning integrates learning and laboratory and emphasizes on reasoning that can improve
procedural and non-procedural problems solving skills.
In addition to problem solving, one of very important features of 21st learning is life skills and career. Life skills
implementation in 21st century can be summarized in several matters those are simpler, such as: motivation, self-
directed learning skills (Ellinger, 2004), critical thinking and problem solving skills (Cash, 2017), communication and
collaboration (Zimmerman-Oster & Burkhardt, 2000), leadership and responsibility (Reimers, 2009). The research of
Pandiangan, Jatmiko, and Sanjaya (2016) toward 40 students (two classes) at Indonesia Open University’s ODE in
Bangkalan-Indonesia indicates that PBL and CL models application to improve physics problem solving skills is still
low and self-directed learning skills is under average. The impact of PBL and CL models influence to improve the
skills to solve physics problems looks significant at moderate level only on the following indicators: self-confident
and love to learn, while initiative and persistence, responsibility, discipline and curiosity, time organization and
learning pace management indicators all are low.
Based on the aforementioned descriptions, the question then is how are the validity, reliability, and effective-
ness of PIL against the achievement of physics problem solving skills and self-directed learning skills research?
To answer such question, PIL model has been developed and a research about the effectiveness of PIL model
application on electricity and magnetism subjects against the achievement of physics problem solving skills and
self-directed learning skills research of Bachelor Program of class 2016 at Indonesia Open University has been done.
PIL model learning refers to problem solving plot of John Dewey and self-directed learning process plot of
Knowles those supported by the newest learning theories. Pandiangan, Jatmiko, & Sanjaya (2016) have formulated
the syntax of PIL model learning, of which includes: (1) initiation and persistence, (2) responsibility, (3) self and group
investigation, (4) analysis, (5) presenting and discussion, and (6) strengthening and evaluation. The main purpose
of this developed PIL model is to create a learning model that can serve as the manual for tutors and students to
652
ISSN 1648–3898 /Print/ THE VALIDITY AND EFFECTIVENESS OF PHYSICS INDEPENDENT LEARNING MODEL TO
ISSN 2538–7138 /Online/ STUDENTS IN OPEN AND DISTANCE EDUCATION SYSTEMS
(P. 651-665)
plan and perform learning that they will be able to improve the achievement of physics problem solving skills and
self-directed learning skills of students in ODES.
The indicators of physics problem solving in this research are adapted from Bradford (2015), those are: (1)
problem formulation, (2) variable identification, (3) form of hypothesis, (4) data analysis, and (5) conclusion draw-
ing. Meanwhile, the indicators of self-directed learning readiness are adapted from Guglielmino and Guglielmino
(1991), those are: (1) initiation and persistence, (2) responsibility, (3) discipline and great curiosity, (4) confidence
and strong desire to learn, (5) able to organize time and set the pace of learning, and (6) love to learn and meet
the planned target and revision. The indicators of physics problem solving skills and self-directed learning skills
respectively are being trained in each stage of PIL model syntax phase presumed to be able to improve physics
problem solving skills and self-directed learning skills of students in open and distance education.
Problem of Research
The most primary issue of this research is how to analyze the validity and effectiveness of PIL model against
electricity and magnetism subjects can improve the achievement of physics problem solving skills and self-directed
learning skills of students in ODES. PIL model is effective when its content validity and construct validity are valid
and reliable. PIL model is effective when the learning process is able to reach such a significant improvement of
physics problem solving skills and self-directed learning skills (statistically). PIL model validity is formulated ac-
cording to the following validity formula, rα = [(Average Square people - Average Square residual)/ (Average Square
people + (k-1) Average Square residual)] and Cronbach’s alpha α = k rα/[1+(k-1) rα] (Malhotra, 2011). The criteria of
PIL model validity and reliability are shown in Table 1.
Table 1. Validity and reliability of PIL model criteria
Check Scale Statistics Category
ra ≤ r table Invalid
Validity Single measures interrater correlation coefficient-ICC (ra)
ra > r table Valid
a < .6 Unreliable
Reliability Cronbach’s alpha/average measures interrater correlation coefficient-ICC (a)
.6 ≤ a ≤ 1.0 Reliable
PIL model effectiveness to improve physics problem solving skills and self-directed learning skills of students
in ODES is decided by the normalized gain score, namely: n-gain = (post-test score – pre-test score)/ (maximum
score – pre-test score) (Hake, 1999). According to the following criteria: (1) when n-gain > .70 (high); (2) when
.30 < n-gain < .70 (moderate); and (3) when n-gain < .30 (high). This research is aimed to analyze PIL model validity
and effectiveness to improve physics problem solving skills and self-directed learning skills of students in ODES.
Research Focus
The focus of this research is to analyze the validity and effectiveness of PIL model against the improvement
of physics problem solving skills and self-directed learning skills. The issues are: 1) is PIL model that has been for-
merly developed validity and reliability qualified? 2) is there any significant improvement (statistically) of physics
problem solving skills and self-directed learning skills before and after learning using PIL model? 3) how much is
the improvement of physics problem solving skills and self-directed learning skills of students in open and distance
education? 4) is there any different improvement of physics problem solving skills and self-directed learning skills
after learning using PIL model among all groups?
General Background
This study was conducted in the Odd Semester of the Academic Year 2016/2017 within 15 weeks using elec-
tricity and magnetism topics. This research is emphasized on the analysis of PIL model validity and effectiveness
653
(P. 651-665)
fulfillment by analyzing the teaching impact using PIL model to improve physics problem solving skills and self-
directed learning skills of students in ODES before and after learning using PIL model. In this research, PIL model
validity is calculated according to the difference between the score of average square people and average square
residual, and the reliability of PIL model is calculated according to Cronbach’s alpha. Meanwhile, the effectiveness
of physics problem solving skills and self-directed learning skills is calculated according to: (a) the significant score
difference between pre-test and post-test, (b) the number of n-gain score can be categorized into low, moderate,
and high categories for both physics problem solving skills and self-directed learning skills.
Sample
This research is held toward 144 Open University students of Surabaya regional office from a population of
225 students in three different regions in which there are two groups of students respectively. Determination of
number of samples is based on Slovin’s formula, i.e. Sample = [population / (1 + e2 x population)] with error toler-
ance e = 5 % (Sevilla, Ochave, Punsalan, Regala, & Uriarte, 1984). During a limited trial test, it uses one region with
two groups, namely group-1 and group-2 of 22 students respectively. They have the same comprehensive level of
physics problem solving skills and self-directed learning skills with respect to electricity and magnetism. Whereas,
during a broad trial test, it uses two regions with two groups, namely group-3, group-4, group-5, and group-6 of
which consist of 25 students respectively with the same comprehensive level of physics problem solving skills and
self-directed learning skills with respect to electricity and magnetism.
This research was conducted using focus group discussions of experts that consisted of three science educa-
tion experts and applies quasi-experiment of one group pre-test and post-test design. This research is classified as
quasi experiment research using one group pre-test – post-test design, namely: O1 X O2 (Fraenkel, Wallen, & Hyun,
2012). Learning that applies PIL model (X) is firstly validated by three experts of physics education (two experts
have doctoral and professorial education background of physics education, 1 expert has pure physics doctoral
education, and the three of them have the skills of ODES and ICT) before it is certified to be valid and reliable. Fur-
thermore, PIL model that has been valid and reliable according to the expert shall be used in physics learning of
electricity and magnetism subjects. The research is held by giving pre-test (O1) before the group of students learns
about electricity and magnetism subject and the test is based on physics problem solving skills and self-directed
learning skills indicators. The group of students then shall learn using PIL (X) model.
The process of learning is performed by means of PIL model and the PIL model learning wares that shall
include syllabus, learning plan, textbooks for students, and worksheets of students. According to the evaluation
done by the validating officers, PIL model and PIL model learning wares have been certified to have been validity
qualified both of content and construct, and reliable. The process of learning applied in this research, during both
limited and broad tests uses PIL model. Finally, after the learning process is accomplished, all groups of students
are provided with post-test (O2), which subjects and problems are similar to those of pre-test.
Data Analysis
The experts according to the validity of both content and construct validate PIL model and PIL model wares.
The validity of content is the illustration of need and newness, while the validity of construct is the illustration of
consistence between PIL model with the theory/empiric and the consistence among model components (Plomp
& Nieveen, 2007). In order to analyze the validity and reliability of PIL model, single measures inter-rater coefficient
correlation and Cronbach’s alpha are used, accordingly. But, to analyze the impact of PIL model teaching toward
physics problem solving skills and self-directed learning skills, the collected scores of pre-test and post-test are
tested using pair t-test or non-parametric analysis of Wilcoxon’s test (Gibbons & Chakraborti, 2011).
The selection of testing method shall depend on the meeting of normality assumption for pre-test and post-
test scores. When the normality assumption for the achieved score is met, the pair t-test then will be applied. When
it is not met, the non-parametric analysis shall be used. In addition, the calculation of n-gain adapted from Hake
654
(P. 651-665)
(1998) is made to decide the level of improvement. IBM SPSS Statistics 22 software is used to test the impact of PIL
model teaching toward the improvement of problem solving skills and self-directed learning skills. Furthermore,
in order to analyze the consistence of PIL model teaching impact toward physics problem solving skills and self-
directed learning skills of the six groups, it uses variant analysis (ANOVA). The testing method shall depend on the
compatible result of normality assumption and variant homogeneity tests of n-gain.
Result of Research
PIL model and its wares validation is done through a series of focus group discussion (FGD) activities in which
three experts of physics education are invited as the validating officers. Details of validity and reliability scores of
each PIL model component item and its ware are shown in Table 2.
Table 2. Validity and Reliability of PIL Model.
Content validity Construct validity

Item
Validity (ra) Cronbach’s Alpha (a) Validity (ra) Cronbach’s Alpha (a)
1. PIL Model .75 Valid .92 Reliable .79 Valid .99 Reliable
2. Syllabus .85 Valid .99 Reliable .99 Valid .99 Reliable
3. Teaching plan .74 Valid .96 Reliable .99 Valid .99 Reliable
4. Teaching Materials .79 Valid .98 Reliable .73 Valid .96 Reliable
5. Students worksheet .70 Valid .92 Reliable .98 Valid .99 Reliable
The scores of pre-test and post-test during the limited trial and broad trial of all groups are shown in Figure
1 and Figure 2. The grey bar indicates the pre-test while the shaded bar indicates the post-test.
Figure 1: The average scores of the student’s pre-test and post-test in terms of physics problem solving skills
of all groups.
Figure 1 shows that the average score between pre-test and post-test associated with the physics problem
solving skills of electricity and magnetism subjects of all groups is improved. The average score of pre-test, post-test
and n-gain with respect to the indicator of physics problem solving skills of all groups is shown in details in Table 3.
655
(P. 651-665)
Table 3. The average scores of the pre-test, post-test, and the n-gain of physics problem solving skills of all
groups.
Problem solving skills indicator

Number of the
Group Scores
Students Formulating of Variable Form a Analyze Draw
the Problem Identification Hypothesis the Data Conclusions
Pre-test 28.71 28.18 29.92 42.10 36.93

Group-1 22 Post-test 84.55 78.18 84.62 69.72 76.93
n-gain .78 .70 .78 .44 .62
Pre-test 27.80 26.59 26.97 28.41 26.25
Group-2 22 Post-test 80.23 73.18 82.12 65.51 73.07
n-gain .73 .64 .75 .51 .61
Pre-test 45.23 41.82 46.21 29.83 30.68
Group-3 25 Post-test 88.64 94.32 87.95 68.58 87.73
n-gain .79 .86 .78 .56 .81
Pre-test 28.56 41.82 31.36 24.83 26.48
Group-4 25 Post-test 87.27 94.32 86.97 68.24 82.39
n-gain .85 .88 .82 .57 .78
Pre-test 38.03 39.09 37.95 27.95 32.05
Group-5 25 Post-test 87.35 90.68 87.20 66.76 85.34
n-gain .78 .83 .78 .53 .75
Pre-test 38.71 40.68 41.59 28.52 33.52
Group-6 25 Post-test 87.20 90.45 87.95 68.30 84.20
n-gain .80 .84 .79 .55 .76
The teaching outcomes achieved by all groups with respect to self-directed learning skills are shown in Fig-
ures 2 and Table 4.
Figure 2: The average scores of the student’s pre-test and post-test in terms of the self-directed learning skills
of all groups.
Figure 2 shows that the average score between the pre-test and post-test of self-directed learning skills of
electricity and magnetism subjects for all groups are also improved. The average score of pre-test, post-test and
n-gain with respect to the indicator of self-directed learning skills of all groups is shown in details in Table 4.
656
(P. 651-665)
Table 4. The average scores of the pre-test, post-test, and the n-gain of self-directed learning skills of all
groups.
Self-directed learning skills indicator

Group N Scores
A B C D E F
Pre-test 48.94 39.22 42.56 45.23 42.21 44.09

Group-1 22 Post-test 79.39 83.38 81.14 74.43 77.92 80.71
n-gain .58 .72 .66 .52 .61 .64
Pre-test 44.85 37.53 42.05 46.25 43.38 39.74
Group-2 22 Post-test 76.82 83.90 79.15 75.11 74.81 79.48
n-gain .54 .74 .63 .52 .55 .65
Pre-test 33.33 34.03 34.43 35.68 32.99 34.55
Group-3 25 Post-test 83.33 86.75 88.64 78.86 79.48 80.84
n-gain .73 .77 .80 .65 .67 .68
Pre-test 40.00 35.19 37.05 41.93 43.25 42.66
Group-4 25 Post-test 85.30 87.01 88.41 79.09 81.30 81.95
n-gain .76 .80 .82 .66 .68 .69
Pre-test 37.58 35.97 36.82 39.55 37.40 34.74
Group-5 25 Post-test 84.55 85.84 88.18 78.52 77.53 78.44
n-gain .75 .78 .81 .66 .63 .65
Pre-test 37.88 35.45 37.27 39.09 38.31 38.44
Group-6 25 Post-test 85.00 87.40 88.52 78.64 80.26 79.09
n-gain .75 .80 .82 .65 .67 .64
N= Number of students; A= Initiation and persistence; B= Responsibility; C= Discipline and great curiosity; D= Confidence and strong
desire to learn; E= Able to organize time and set the pace of learning; F= Love to learn and meet the planned target and revision
The average n-gain scores of physics problem solving skills and self-directed learning skills of all groups are
shown in Figure 3.
Figure 3: The average scores of the n-gain for both the problem solving skills and self-directed learning skills
of all groups.
657
(P. 651-665)
Figure 3 shows that the average scores of the n-gain for physics problem solving skills of all groups, from 1
to 6, respectively are: .68; .65; .76; .78; .73; and .75. The average n-gain scores of both groups in the limited test
respectively are categorized as moderate, while the average n-gain scores of the next four groups in the wider test
respectively are categorized as high. On the other side, the average n-gain scores for self-directed learning skills
of all groups respectively are .64; .62; .72; .74; .71; and .72. The average n-gain scores of both groups in the limited
test respectively are categorized as moderate, while the average n-gain scores of the next four groups in the wider
test respectively are categorized as high.
Pre-test and post-test normality test for all groups are held by means of one-sample Kolmogorov-Smirnov Z
test of IBM SPSS Statistics 22 software as shown in Table 5.
Table 5. The normalized post-test and pre-test of physics problem solving skills and self-directed learning
skills for all groups.
Physics problem solving skills Self-directed learning skills
Group Test N Asymp. Asymp.

Std. Normal Std. Normal
Mean Sig. Mean Sig.
Deviation Distribution Deviation Distribution
(2-tailed) (2-tailed)
posttest 22 78.34 4.88 .01 no 79.81 7.84 .92 yes

1
pretest 22 32.26 6.44 .58 yes 43.51 5.13 .96 yes
posttest 22 74.49 6.63 .28 yes 78.48 7.40 .82 yes
2
pretest 22 26.10 5.21 .90 yes 41.98 4.80 .83 yes
posttest 25 82.25 8.75 .77 yes 82.50 8.57 .33 yes
3
pretest 25 36.26 7.83 .94 yes 34.80 5.54 .25 yes
posttest 25 92.02 9.56 .33 yes 84.66 5.10 .99 yes
4
pretest 25 38.82 6.18 .25 yes 39.23 4.90 .89 yes
posttest 25 82.50 8.57 .33 yes 82.55 6.06 .71 yes
5
pretest 25 80.38 4.02 .90 yes 36.87 4.39 .95 yes
posttest 25 33.54 6.61 .67 yes 83.09 6.17 .91 yes
6
pretest 25 81.35 4.87 .92 yes 37.68 3.61 .65 yes
N= Number of students
Table 5 shows that post-test and pre-test of physics problem skills and self-directed learning skills are normally
distributed to all groups, unless for group-1 that is not normally distributed in the post-test. Therefore, in order
to analyze the impact of PIL model teaching further, Wilcoxon test is applied to group-1 that is not normally dis-
tributed, while pair t-test is applied for other groups those are normally distributed. Table 6 and Table 7 shows the
results obtained in Wilcoxon test and pair t-test after meeting the normality assumption of pre-test and post-test.
Table 6. The Wilcoxon test result of the physics problem solving skills of group-1.
N Z p
Post-test and Pre-test 22 - 4.108 < .0001

*p < .05 (2-tailed)
658
(P. 651-665)
Table 7. The pair t-test result of the physics problem solving skills and self-directed learning skills of group-1
up to group-6.
Physics problem solving skills Self-directed learning skills
Pair N Std. Std.

Mean Error t df p Mean Error t df p
Mean Mean
Pair 1 22 Wilcoxon test 36.30 2.22 16.32 21 < .0001

Pair 2 22 48.39 1.87 25.95 21 < .0001 36.50 2.21 16.52 21 < .0001
Pair 3 25 45.99 1.77 25.92 24 < .0001 47.70 2.41 19.80 24 < .0001
Pair 4 25 53.20 2.69 19.79 24 < .0001 45.43 1.34 33.91 24 < .0001
Pair 5 25 46.85 1.52 30.86 24 < .0001 45.68 1.57 29.15 24 < .0001
Pair 6 25 46.15 1.39 33.18 24 < .0001 45.41 1.49 30.40 24 < .0001
Table 6 shows that the score provided by Z is -4.108 with significance level of p < .05. This clearly indicates
that PIL model does have an impact on improving the physics problem solving skills of group-1. Table 7 also shows
that t scores of physics problem solving is t = 25.95 for degrees of freedom, df = 22, t = 25.92, t = 19.79, t = 30.86,
t = 33.18 for degrees of freedom, df = 25. t score of self-directed learning skills is 16.32 and t = 16.52 for degrees
of freedom, df = 22, t = 19.80, t = 33.91, t = 29.15, and t = 30.40 for degrees of freedom, df = 25. Such scores are
considered significant since p < .05.
The consistence of PIL model impact on the skills of physics problem solving and self-directed learning then
is further analyzed by means of ANOVA after meeting the normality assumption and variant homogeneity as
shown in Table 8.
Table 8. The results of ANOVA of physics problem solving skills and self-directed learning skills of all groups.
n-gain all groups Sum of Squares df Mean Square F p
Physics problem solving skills Between Groups .208 5 .042 1.979 .302
Within Groups 1.446 138 .010
Total 1.654 143
Self-directed learning skills Between Groups .274 5 .055 1.608 .704
Within Groups 2.096 138 .015
Total 2.370 143
Table 8 shows that F count gives F = 1.98 < Ftable (5; 138) = 2.28 with significance level p = .302 > .05 for physics prob-
lem solving skills. Therefore, there are strong indications that the impact of PIL model on the physics problem solving
skills for the group is not different at the 5% level of significance. Table 8 also shows that F = 1.61 < Ftable (5;138) = 2.28
with significance level P = .704 > .05 for self-directed learning skills. Consequently, there are strong indications that
the impact of PIL model on the self-directed learning for the group is not different at the 5% level of significance.
Discussion
The Validity and Reliability of PIL Model
PIL model validation is held through FGD activities by experts those are science education expert, physics
education expert, and open and distance education practitioner, so that valid PIL model that meets the aspect of
necessity and newness based on strong empirical theory in which there is inter-consistence among its structural
components is achieved. This PIL model validation is in line with the result of research done by Murgado-Armenteros,
Torres-Ruiz, and Vega-Zamora (2012) in which it states that one product validation can be performed through FGD
659
(P. 651-665)
activities by the experts, either face to face or online. By taking the availability of facilities and infrastructures into
account, this PIL model validation is taken face to face, in which three ODES experts are involved.
The result of FGD activities as shown in Table 2 indicates that PIL model and its components those model
rational, theoretical and empirical support, model syntax, social systems, reaction principle, supporting systems,
instructional & accompanying impacts are categorized as valid and reliable. Malhotra (2011) states that a product is
considered as valid when it has ICC single measure of rα = .754 > rtable. In this research, the result of rα = .75 > rtable for
content validation and of rα = .79 > rtable for construct validation are obtained, which means that the developed PIL
model has such a high validation of both content and construct. It is so with the Syllabus, teaching plan, teaching
materials, and university student’s worksheet those are valid and reliable. This also corresponds to the research of
Plomp and Nieveen (2007) which state that a product does have a good quality when referring to content validity
and construct validity those are valid and able to illustrate necessity, newness, inter-component consistence of
the model and both theoretical and empirical support.
PIL model that is valid already of both content and construct must be tested for its consistence in order that
the model is stable and can be routinely used. According to Sarstedt and Mooi (2014), reliability of certain product
can be stable when it meets the following qualifications: stability of the measurement, internal consistency reli-
ability, and inter-rater reliability. PIL model is considered as reliable when the coefficient of Cronbach’s alpha (α)
and Cronbach’s alpha if item deleted is > .60 (Malhotra, 2011). According to the result of FGD, internal consistence
reliability and inter-rater reliability of PIL model are found to be all reliable as shown by Table 2 with Cronbach’s
alpha is: .996 for content validity; .993 for construct validity. The internal consistence reliability is indicated with
the value of Cronbach’s alpha if items deleted of component are .972 – .976 respectively for content validity and
.925 – 1,000 for construct validity. It means that the developed PIL model has such a high reliability of both content
validity and construct validity.
The wares of PIL model that is developed those syllabus, teaching plan, teaching materials, and university
student’s worksheet are the wares of PIL model that is compatible to be used in complementing PIL model. It is to
make it corresponds to the need, has the newness feature and is supported by strong theoretical and empirical
ground, to have inter-component consistence (Plomp & Nieveen, 2007), good literacy and appropriate to be a
teaching plan of PIL model to improve the physics problem solving skills and self-directed learning skills (Dunlap,
2005) of the students in open and distance education (Blaschke, 2012).
PIL model is categorized as valid of both content and construct validity, so that it can be used as the refer-
ence to make a plan of physics problem solving skills and self-directed learning skills training. This is in line with
the research held by Seechaliao, Natakuatoong, and Wannasuphoprasit (2012) in which it is stated that such valid
teaching model can assist the researcher and practitioner to design a teaching according to the teaching principles
those have been understood. The valid teaching model can be used as the reference for the academics and practi-
tioner to plan a teaching program (Kimbell & Stables, 2007). The valid PIL model can provide the practitioners with
a chance to apply it on physics teaching by involving scientific process and product, so that it can be used to train
physics problem solving skills and self-directed learning skills (Pandiangan, Jatmiko, & Sanjaya, 2016).
The Effectiveness of PIL Model and Its Application
A good teaching model must have specific characteristics and purpose and meet validity, practicability, and
effectiveness aspects. Honebein and Honebein (2015) state that effective teaching is possible when the teaching
process is designed according to the core principles of teaching plan theory. Such an effective teaching can be
achieved when a lecturer has an appropriate strategy to deliver his/her knowledge to the students structurally
and be able to integrate theory and practice into the process of learning (Hughes, 2005). A teaching is categorized
as effective when the tutor has a good level of knowledge and comprehension of teaching (Roscoe & Chi, 2007),
the students actively participate in the learning (Eom, Wen, & Ashill, 2006), available infrastructures of laboratory
equipment’s/ computer simulation (Beatty, 2013), and the learning achievement of students is improved as they
have a good respond to the learning (Zimmerman & Schunk, 2012). Based on activity theory, increased student
engagement can improve learning outcome, with respect to effectiveness, such learning can be measured accord-
ing to the improved achievement of the students and their respond to the learning (Jatmiko, Widodo, Budiyanto,
Wicaksono, & Pandiangan, 2016; Liaw, 2008).
The improvement of student’s problem-solving skills can be seen from the n-gain of physics problem solv-
ing skills with respect to electricity and magnetism obtained from the calculation of scores achieved in pre-test
660
(P. 651-665)
and pro-test given to the students before and after teaching in which PIL model is applied. According to Figure 1,
before learning about electricity and magnetism using PIL model, the students have not had any competence of
problem-solving skills yet. Their average scores are still under the standard score that is of 32.26 within the score
range from 0 to 100 for group-1; of 26.10 for group-2; of 36.80 for group-3; of 27.53 for group-4; of 32.60 for group-5;
and of 35.59 for group-6. This is because the students have not been familiar yet to do the thinking activities as
demanded by PIL model in physics problem solving skills, as follows, problem formulation, variable identification,
hypothesis forming, data analysis, and conclusion drawing.
This research result is supported by the research done by Mestre (2001) in which it is stated that problem
solving skills in physics learning at higher education level is based on the characteristics of physics subject that is
assumed to be relatively difficult and complex. Problem solving skills in physics that is abstract is categorized as
low and difficult to understand (Snyder, 2000). The preliminary research result to 83 students of Open University
Stratum-1 program with respect to physics problem solving skills is still low (Pandiangan, Jatmiko, & Sanjaya, 2015).
This is due to the lack of understanding of students about the model and theory of physics (Clement, 1993) and
poor competence of understanding science literacy (Yore & Treagust, 2006).
After learning about electricity and magnetism using PIL model, the students achieve good physics problem
solving skills competence, which average scores are 78.34 for group-1; 74.49 for group-2; 82.60 for group-3; 81.14
for group-4; 80.99 for group-5; and 81.35 for group-6. It means that the average scores have been improved before
and after the application of PIL model as much as 142.20%, as follows: 184.93% for group-1, 72.63% for group-2,
185.71% for group-3, 105.26% for group-4, 152.22% for group-5, and 108.91% for group-6. These improved scores
of physics problem solving skills of all groups are significant and consistent at the real level of 5% with respective
n-gain of groups are .68 for group-1, .65 for group-2, .76 for group-3, .78 for group-4, .73 for group-5 and .75 for
group-6. Such results indicate that teaching using PIL model does have an impact on the skills of problem solv-
ing that is really improved. The level of such PIL model teaching impact on the physics problem solving skills of
all groups is consistent significantly at the real level of 5%. Group-1 and Group-2 are in the moderate category of
limited test, while group-3, group-4, group-5, and group-6 are in the high category of wider test.
Such improvement of competence according to the indicator of physics problem solving skills using PIL model
is due to several causes. Those causes among others are: the students have been trained and directed to reach
the indicator of physics problem solving skills (Bradford, 2015); course syllabus (Jenkins, Bugeja, & Barber, 2014),
teaching materials (Crouch & Mazur, 2001), good worksheet (Bakırcı, Bilgin, & Simsek, 2011), good teaching environ-
ment and tool/simulation technique (Sedrakyan & Snoeck, 2012) those provide the students with positive effects
while making hypothesis and creating good analyzing skills. Further, the learning initiated with having a problem
(Engeström, 2001), trying a simulation before predicting what may happen (Tao & Gunstone, 1999), deciding the
purpose of experiment, formulating the problem, identifying variables, and making hypothesis (Bradford, 2015)
can grow motivation, initiative and persistence of the students to solve the problem. An explanation about natural
symptoms according to the data obtained during the process of investigation is a highly decisive factor to ensure
a successful teaching of physics practicum (Champagne, Klopfer, & Anderson, 1980).
Referring to the achieved results as aforesaid, teaching syntax that has been formulated according to physics
problem solving skills is supported by the newest empirical data and teaching theory. Such an improved phys-
ics problem solving skills using PIL model is also supported by several results of research, as follows: (1) learning
effectiveness may be due to the quality of teaching (Palardy & Rumberger, 2008), facilities and infrastructures
availability (Piccoli, Ahmad, & Ives, 2001), active participation of students and students’ response (Liaw, 2008;
Tschannen-Moran & Hoy, 2001); (2) an effective lecturer knows how to assist his/her students in doing their in-
vestigation using knowledge, curriculum, and staged teaching to deal with complexity that they may encounter
during the class (Rubin, Chamot, Harris, & Anderson, 2007); (3) while solving a problem, it is important to build
such knowledge in social aspects according to what is needed by the students and how they should learn (Asheim,
Coenen, & Vang, 2007; Hmelo-Silver, 2004), social interaction with others in which they may generate new ideas
to improve their intellectual development based on former experience (Csikszentmihalyi, 2014; Ostrom, 2014),
the students then become more active in class discussions and are able to maintain a good learning habit (Baxter
Magolda, 2003; Kong, 2014).
This result of research is also supported by several theories of teaching with respect to physics problem solving
skills of PIL model. Those theories are as follows: motivation theory, which states that a person will be motivated
when what he or she does can attract the attention of students (Arends, 2012); top-down process, which states that
we’d better be off choosing complex problems to solve and then finding basic skills as required (Moreno, 2010);
661
(P. 651-665)
zone of proximal development, which states that students will learn the best concept when such concept is in the
closest development zone (Moreno, 2010; Slavin, 2006); and scaffolding, which states that the students should
be given complex, difficult, and realistic assignments and provided with gradual assistance to solve the problems
(Slavin, 2006). Theoretical analysis states that one of creative problem-solving methods those are often suggested
is to analyze and register the main characteristics of problem elements (Moreno, 2010).
As the case with physics problem solving skills, the improvement of university student’s self-directed learning
skills can be indicated from the n-gain. The n-gain is calculated from those scores achieved in pre-test and post-test
given to the student. According to Figure 2, before learning about electricity and magnetism by means of PIL model,
the students have such a low skill of self-directed learning, namely as much as 43.51 in group-1, 41.97 in group-2,
34.29 in group-3, 39.91 in group-4, 36.74 in group-5, and 37.77 in group-6 within the score range from 0 to 100. This
is since the students have not yet been used to do active thinking as thinking activities of self-directed learning
required by PIL model, namely initiation and persistence, responsibility, discipline and great curiosity, confidence
and strong desire to learn. They should also be able to organize time and set the pace of learning, and love to learn
and meet the planned target and revision. The result of this research is supported by Benegas and Flores (2014),
who states that undergraduate introductory physics course, especially electric circuit, has a complex concept to be
verbally explained only. Therefore, it will be less effective and practical to improve self-directed learning. The result
of preliminary study to 83 students of Open University Stratum-1 bachelor program with respect to self-directed
learning skills is still underrated (Pandiangan, Jatmiko, & Sanjaya, 2005).
After learning about electricity and magnetism using PIL model, the students have had a good competence
of self-directed learning skills with average scores of 79.81 for group-1; 78.48 for group-2; 83.53 for group-3; 84.22
for group-4; 82.55 for group-5; and 83.39 for group-6 respectively. It means that the average scores are improved
before and after PIL model application by 96.73%, which is 104.35% for group-1; 82.26% for group-2; 105.26% for
group-3; 103.3% for group-4; 83.74% for group-5; and 93.18% for group-6. These improved scores of self-directed
learning skills of all groups are significant and consistent at real level of 5% with respective n-gain of groups are
.64 for group-1; .62 for group-2; .72 for group-3; .74 for grou-4; .71 for group-5; and .72 for group-6. Such results
indicate that teaching using PIL model does have a real impact on the improved skills of self-directed learning. The
level of such PIL model teaching impact on the self-directed learning skills of all groups is consistent significantly
at real level of 5%. Group-1 and group-2 are in moderate category of limited test, while group-3, group-4, group-5,
and group-6 are in high category of wider test.
Such improvement of competence according to the indicator of self-directed learning skills using PIL model is
due to several causes. Those causes among others are: (1) the students have been trained and directed to reach the
indicator of self-directed learning skills (Stewart, 2007); (2) both formal and informal trainings may change someone’s
habit to leave his/her old habit and adapt with new technology to create better products and these technology,
together with the quality of the training process, should be a major challenge for the coming time (Daniel, Cano, &
Cervera, 2015; Walkington & Sherman, 2013); (3) problem presentation that will effectively encourage the students
to be more independent, initiated and persisted to learn (Biggs, 2011); (4) the students have the competence of
understanding formulations, graphs, illustrations, review and extend existing frameworks on modeling to develop
a new framework that describes model-based reasoning in introductory and upper-division physics laboratories,
table reading and relating variables that may help the students to simplify problems, so that the students can learn
more independently (Zwickl & Hu, 2015); (5) the students can individually build their own knowledge and develop
meanings according to former experiences, either personally or in social context (Greenspan, 2015).
The result of this research is also supported by several teaching theories with respect to self-directed learning
skills of PIL. Those theories among others are: advanced organizer theory, which states that the preliminary state-
ment about a subject to be learnt will provide new information structures and relate them to the information that
the students have had previously (Moreno, 2010; Slavin, 2006); modeling theory, which states that the students can
learn through explanation and observation of others (Arends, 2012; Moreno, 2010); cognitive apprenticeship, which
states that a student’s process of learning can be taken gradually until he/she reaches the level of expertise while
interacting with an expert whose knowledge is higher, either older or at the colleagues (Arends, 2012; Slavin, 2006).
Conclusions
By virtue of the aforementioned research and discussion, learning that applies PIL model and its wares on
electricity and magnetism subjects is validly, reliably, and effectively qualified to improve the physics problem solv-
662
(P. 651-665)
ing skills and self-directed learning skills of students at open and distance education systems. The validity, reliability
and effectiveness of teaching that applies PIL model and its wares to improve physics problem solving skills and
self-directed skills are based on: (1) the validity of the developed PIL model is valid based on single measure inter-
rater correlation coefficient rα > rtable and Cronbach’s alpha .6 < α < 1.0; (2) there is such a significant improvement
(statistically) of physics problem solving skills and self-directed learning skills before and after applying PIL model;
(3) n-gain scores of physics problem solving skills and self-directed learning skills are categorized as moderate in
the limited test and high in the wider test; (4) there is not any difference level of improvement between physics
problem solving skills and self-directed learning skills (there is no difference of n-gain) for both tests in all groups.
It means that PIL model and its wares are consistent to improve physics problem solving skills and self-directed
learning skills of the students at open and distance education systems.
Acknowledgements
The authors would like to express their gratitude to the Republic of Indonesia Government via Research,
Technology, and Higher Education Ministry for the support, facility and infrastructures to this research, especially
to the Indonesia Open University. The authors would like to send their gratitude also to Surabaya State University
for the opportunity to hold this research. This work was supported in part by a grant from the Research Institute
and Community Service of the Indonesia Open University (No.: 5344 /UN31.2/DN/2017).
References
Adams, R., Vista, A., Scoular, C., Awwal, N., & Griffin, P. (2015). Automatic coding procedures for collaborative problem solving
Assessment and Teaching of 21st Century skills (pp. 115-132): Springer.
Adeyemi, A. M., & Adeyemi, S. B. (2014). Personal factors as predictors of students’ academic achievement in colleges of educa-
tion in South Western Nigeria. Educational Research and Reviews, 9 (4), 97.
Arends, R. I. (2012). Learning to teach. New York: McGraw-Hill Companies, Inc.
Asheim, B., Coenen, L., & Vang, J. (2007). Face-to-face, buzz, and knowledge bases: Sociospatial implications for learning, innova-
tion, and innovation policy. Environment and Planning C: Government and Policy, 25 (5), 655-670.
Bakırcı, H., Bilgin, A. K., & Simsek, A. (2011). The effects of simulation technique and worksheets on formal operational stage in
science and technology lessons. Procedia-Social and Behavioral Sciences, 15, 1462-1469.
Baxter Magolda, M. B. (2003). Identity and learning: Student affairs a role in transforming higher education. Journal of College
Student Development, 44 (2), 231-247.
Beatty, K. (2013). Teaching & researching: Computer-assisted language learning: Routledge.
Benegas, J., & Flores, J. S. (2014). Effectiveness of tutorials for introductory physics in argentinean high schools. Physical Review
Special Topics - Physics Education Research, 10 (1), 1-10.
Biggs, J. B. (2011). Teaching for quality learning at university: What the Student Does: McGraw-Hill Education (UK).
Blaschke, L. M. (2012). Heutagogy and lifelong learning: A review of heutagogical practice and self-determined learning. The
International Review of Research in Open and Distributed Learning, 13 (1), 56-71.
Brad, A. (2011). A Study of the problem solving activity in high school students: Strategies and self-regulated learning. Acta
Didactica Napocensia, 4 (1), 21-30.
Bradford, A. (2015). Science & the scientific method: A ddefinition. Live Science. Retrieved from http://www.livescience.com/21456-
empirical-evidence-a-definition.html.
Brophy, J. E. (2013). Motivating students to learn: Routledge.
Cash, R. M. (2017). Advancing differentiation: Thinking and learning for the 21st century: Free Spirit Publishing.
Champagne, A. B., Klopfer, L. E., & Anderson, J. H. (1980). Factors influencing the learning of classical mechanics. American Journal
of Physics, 48 (12), 1074-1079.
Clement, J. (1993). Using bridging analogies and anchoring intuitions to deal with students’ preconceptions in physics. Journal
of research in science teaching, 30 (10), 1241-1257.
Crouch, C. H., & Mazur, E. (2001). Peer instruction: Ten years of experience and results. American Journal of Physics, 69 (9), 970-977.
Csikszentmihalyi, M. (2014). Toward a Psychology of optimal experience. Springer.
Daniel, S. J., Cano, E. V., & Cervera, M. G. (2015). The future of MOOCs: Adaptive learning or business model? Universities and
Knowledge Society Journal, 12 (1), 64-73.
Dunlap, J. C. (2005). Changes in students’ use of lifelong learning skills during a problem-based learning project. Performance
Improvement Quarterly, 18 (1), 5-33.
Ellinger, A. D. (2004). The concept of self-directed learning and its implications for human resource development. Advances in
Developing Human Resources, 6 (2), 158-177.
Engeström, Y. (2001). Expansive learning at work: Toward an activity theoretical reconceptualization. Journal of Education and
Work, 14 (1), 133-156.
Eom, S. B., Wen, H. J., & Ashill, N. (2006). The determinants of students’ perceived learning outcomes and satisfaction in university
663
(P. 651-665)
online education: An empirical investigation. Decision Sciences Journal of Innovative Education, 4 (2), 215-235.
Fraenkel, J., Wallen, N., & Hyun, H. (2012). How to design and evaluate research in education (8th edt.). New York: McGram-Hill
Companies: Inc.
Gibbons, J. D., & Chakraborti, S. (2011). Nonparametric statistical inference (5 ed.). Tuscaloosa: CRC Press.
Greenspan, Y. F. (2015). A guide to teaching elementary science: Ten easy steps. Springer.
Griffin, P., & Care, E. (2015). Assesment and teaching of 21st century skills: Methods and approach. New York: Springer.
Guglielmino, L. M., & Guglielmino, P. J. (1991). Expanding your readiness for self-directed learning: A workbook for the learning
preference assessment. Organization design and development.
Hake, R. (1998). Interactive-engagement versus traditional methods: A six-thousand-student survey of mechanics test data for
introductory physics courses. American Journal of Physics, 66 (1), 64-74.
Hake, R. (1999). Analyzing change/gain score. American educational research association’s division measurement and research
methodology. CA USA: Indiana University Press.
Hmelo-Silver, C. E. (2004). Problem-based learning: What and how do students learn? Educational Psychology Review, 16 (3), 235-266.
Honebein, P. C., & Honebein, C. H. (2015). Effectiveness, efficiency, and appeal: Pick any two? The influence of learning domains
and learning outcomes on designer judgments of useful instructional methods. Educational Technology Research and
Development, 63 (6), 937-955.
Hughes, J. (2005). The role of teacher knowledge and learning experiences in forming technology-integrated pedagogy. Journal
of Technology and Teacher Education, 13 (2), 277.
Jatmiko, B., Widodo, W., Budiyanto, M., Wicaksono, I., & Pandiangan, P. (2016). Effectiveness of the INQF-based learning on a
general physics for improving student’s learning outcomes. Journal of Baltic Science Education, 15 (4), 441-451.
Jenkins, J. S., Bugeja, A. D., & Barber, L. K. (2014). More content or more policy? A closer look at syllabus detail, instructor gender,
and perceptions of instructor effectiveness. College Teaching, 62 (4), 129-135. doi:10.1080/87567555.2014.935700.
Kimbell, R., & Stables, K. (2007). Researching design learning: Issues and findings from two decades of research and development
(Vol. 34): Springer Science & Business Media.
Kong, S. C. (2014). Developing information literacy and critical thinking skills through domain knowledge learning in digital
classrooms: An experience of practicing flipped classroom strategy. Computers & Education, 78, 160-173.
Legault, L., Green-Demers, I., & Pelletier, L. (2006). Why do high school students lack motivation in the classroom? Toward an
understanding of academic amotivation and the role of social support. Journal of Educational Psychology, 98 (3), 567.
Liaw, S.-S. (2008). Investigating students perceived satisfaction, behavioral intention, and effectiveness of e-learning: A case
study of the blackboard system. Computers & Education, 51 (2), 864-873.
Malhotra, N. K. (2011). Review of marketing research: Special issue–marketing legends: Emerald Group Publishing Limited.
Mestre, J. P. (2001). Implications of research on learning for the education of prospective science and physics teachers. Physics
Education, 36 (1), 44.
Moreno, R. (2010). Educational psychology. USA: John Wiley & Sons, Inc.
Murgado-Armenteros, E. M., Torres-Ruiz, F. J., & Vega-Zamora, M. (2012). Differences between online and face to face focus groups,
viewed through two approaches. Journal of Theoretical and Applied Electronic Commerce Research, 7 (2), 73-86.
Ostrom, E. (2014). Collective action and the evolution of social norms. Journal of Natural Resources Policy Research, 6 (4), 235-252.
Palardy, G. J., & Rumberger, R. W. (2008). Teacher effectiveness in first grade: The importance of background qualifications, at-
titudes, and instructional practices for student learning. Educational evaluation and policy analysis, 30 (2), 111-140.
Pandiangan, P., Jatmiko, B., & Sanjaya, I. G. M. (2015). Investigasi besiapan belajar mandiri mahasiswa program inservice training
pada pendidikan terbuka dan jarak jauh [Investigation readiness independent student learning inservice training program
in open and distance education]. Paper presented at the National Seminar on Physics, Jember.
Pandiangan, P., Jatmiko, B., & Sanjaya, I. G. M. (2016). Pentingnya keterampilan pemecahan masalah dan kesiapan belajar mandiri
mahasiswa pada pendidikan terbuka dan jarak Jauh untuk menyongsong abad 21 [The importance of physics problem solv-
ing skill and self-directed learning skill in open and distance education to welcome to the 21st century]. Paper presented
at the National Seminar on Science Education 2016, Surabaya.
Piccoli, G., Ahmad, R., & Ives, B. (2001). Web-based virtual learning environments: A research framework and a preliminary as-
sessment of effectiveness in basic IT skills training. MIS Quarterly, 401-426.
Pless, N. M., Maak, T., & Stahl, G. K. (2011). Developing responsible global leaders through international service-learning programs:
The ulysses experience. Academy of Management Learning & Education, 10 (2), 237-260.
Plomp, T., & Nieveen, N. (2007). An introduction to educational design research. Paper presented at the Proceedings of the Seminar
Conducted at the East China Normal University [Z]. Shanghai: SLO-Netherlands Institute for Curriculum Development.
Prahani, B. K., Pandiangan, P., & Nasir, M. (2015). Penilaian: Penghargaan prestasi kelompok penyelesaian masalah kolaborasi
[Assessment: Collaboration problem solving group achievements]. Paper presented at the National Seminar on Science
Education 2015, Surabaya.
Regulation of the Minister of Education and Culture of the Republic of Indonesia, 24 C.F.R. (2012).
Regulation of the Minister of Education and Culture of the Republic of Indonesia, 73 C.F.R. (2013).
Reimers, F. (2009). 14 Educating for global competency. International Perspectives on the Goals of Universal Basic and Seconda-
ryEeducation, 183.
Roscoe, R. D., & Chi, M. T. (2007). Understanding tutor learning: Knowledge-building and knowledge-telling in peer tutors’ ex-
planations and questions. Review of Educational Research, 77 (4), 534-574.
Rubin, J., Chamot, A. U., Harris, V., & Anderson, N. J. (2007). Intervening in the use of strategies. Language Learner Strategies, 30, 29-45.
664
(P. 651-665)
Sarstedt, M., & Mooi, E. (2014). A Concise guide to market research: The process, data, and methods using IBM SPSS statistics (2 ed.).
New York: Springer.
Schunk, D. H. (2005). Self-regulated learning: The educational legacy of Paul R. Pintrich. Educational Psychologist, 40 (2), 85-94.
Sedrakyan, G., & Snoeck, M. (2012). Technology-enhanced support for learning conceptual modeling. Enterprise, Business-Process
and Information Systems Modeling (pp. 435-449): Springer.
Seechaliao, h., Natakuatoong, O., & Wannasuphoprasit, W. (2012). The validation of an instructional design and development model
based on engineering creative problem solving principles to develop reative thinking skills of undergraduate engineering
students. IPEDR, 30. Retrieved from http://www.ipedr.com/vol30/19-ICEMI%202012-M00032.pdf.
Sevilla, C. G., Ochave, J. A., Punsalan, T. G., Regala, B. P., & Uriarte, G. G. (1984). An introduction to research methods. Quezon City:
Rex Printing Company.
Silva, E. (2009). Measuring skills for 21st-century learning. Phi Delta Kappan, 90 (9), 630-634.
Slavin, R. E. (2006). Educational psychology: Theory and practice (8 ed.). Boston: Allyn and Bacon.
Snyder, J. L. (2000). An investigation of the knowledge structures of experts, intermediates and novices in physics. International
Journal of Science Education, 22 (9), 979-992.
Stewart, R. A. (2007). Investigating the link between self-directed learning readiness and project-based learning outcomes: The
case of international masters students in an engineering management course. European Journal of Engineering Education,
32 (4), 453-465.
Tao, P.-K., & Gunstone, R. F. (1999). The process of conceptual change in force and motion during computer-supported physics
instruction. Journal of Research in Science Teaching, 36 (7), 859.
Tschannen-Moran, M., & Hoy, A. W. (2001). Teacher efficacy: Capturing an elusive construct. Teaching and Teacher Education, 17
(7), 783-805.
Tucker, S. Y. (2014). Transforming Pedagogies: Integrating 21st century skills and web 2.0 technology. Turkish Online Journal of
Distance Education, 15 (1), 166-173.
Tutorial Guidelines for Student. (2016). Jakarta: Karunika.
Walkington, C., & Sherman, M. (2013). Using adaptive learning technologies to personalize instruction: The impact of relevant
contexts on performance and learning outcomes. Journal of Educational Psychology, 105 (4), 932-945.
Yew, E. H., & Schmidt, H. G. (2009). Evidence for constructive, self-regulatory, and collaborative processes in problem-based
learning. Advances in Health Sciences Education, 14 (2), 251-273.
Yore, L. D., & Treagust, D. F. (2006). Current realities and future possibilities: Language and science literacy - empowering research
and informing instruction. International Journal of Science Education, 28 (2-3), 291-314.
Zimmerman-Oster, K., & Burkhardt, J. C. (2000). Leadership in the making: Impact and insights from leadership development programs
in US Colleges and Universities. Executive Summary.
Zimmerman, B. J., & Schunk, D. (2011). Motivational sources and outcomes of self-regulated learning and performance. Handbook
of self-regulation of learning and performance, 49-64.
Zimmerman, B. J., & Schunk, D. H. (2012). Self-regulated learning and academic achievement: Theory, research, and practice: Springer
Science & Business Media.
Zwickl, B. M., & Hu, D. (2015). Model-based reasoning in the physics laboratory: Framework and initial results. Physical Review
Special Topics - Physics Education Research, 11 (2), 1-12.
Paken Pandiangan Associate Professor, Researcher, Faculty of Training and Teacher

Education, Indonesia Open University, Jalan Cabe Raya, Pondok Cabe,
Jakarta, 15418 Indonesia.
E-mail: pakenp@ecampus.ut.ac.id
Website: http://www.ut.ac.id/
I Gusti Made Sanjaya Associate Professor, Researcher, Mathematic and Natural Science
Faculty, State University of Surabaya, Jalan Ketintang, Surabaya, 60231,
Indonesia.
E-mail: igustimadesanjaya@gmail.com
Website: http://www.unesa.ac.id/
Budi Jatmiko Professor, Researcher, Postgraduate School, State University of
(Corresponding author) Surabaya, Jalan Ketintang, Surabaya, 60231 Indonesia.
E-mail: budijatmiko@unesa.ac.id
Website: http://pasca.unesa.ac.id/
665
THE EFFECTS OF MULTIPLE
INTELLIGENCES BASED
INSTRUCTION ON STUDENTS’
PHYSICS ACHIEVEMENT AND
ATTITUDES
Abstract. The aim of this research is to ex- Deniz Gurcay,

amine the effects of multiple intelligences Hatice Ozturk Ferah
based instruction on the 9th grade stu-
dents’ achievements and attitude towards
the Force and Motion topics in physics. The
participants of this research are composed
of randomly selected 95 ninth graders,
Introduction
who were divided into two experimental
and two control classes. The experimen-
Recent studies conducted in science education have focused on
tal groups were exposed to lesson plans
identification and elimination of difficulties encountered by students in
based on the multiple intelligences for six
learning science in order to raise the quality of learning. Some studies have
weeks. In the control groups, traditional
emphasized that students regard physics as a difficult subject to learn and
instruction was conducted. Force and Mo-
an abstract one (Fonseca & Conboy, 1999; Ornek, Robinson & Haugan, 2008;
tion Achievement Test, Force and Motion
Whitelegg & Parry, 1999). As reasons for their views, they cite the fact that
Attitude Scale, Revised Student Multiple
they have performed many different things such as experiments, formulas,
Intelligences Profile Questionnaire and
concepts, graphs and calculations at the same time while they are learning
Students’ Views about MI-based instruc-
physics (Angell, Guttersrud, Henriksen & Isnes, 2004). However, one of the
tion Questionnaire were used as research
reasons why students have such negative opinions about physics is con-
instruments in this research. As a result of
cerned with the methods, which teachers employ while teaching physics
this research, it was found that multiple
(Zacharia, 2003). Contemporary understanding of education requires that
intelligences based instruction had a sig-
different characteristics of individuals be taken into consideration and that
nificant effect on students’ achievements
educational activities be planned in accordance with this (Rubin & Herbert,
in the Force and Motion topics. However,
1998). In this sense, teachers should use learning and teaching methods that
multiple intelligences based instruction
take into account the individual differences of students instead of traditional
had any significant effect on students’ at-
teaching in order to reduce students’ negative attitudes towards physics
titudes towards force and motion topics. In
learning and increase their success. In recent years, efforts have been made
addition, a large majority of the students in
to create more efficient learning environments by taking into account the
the experimental group found the courses
effects of individual differences in learning and various teaching methods
based on multiple intelligences more
have emerged in this regard.
enjoyable and found more instructive.
Gardner (1983) proposed the multiple intelligences (MI) theory in
Keywords: force and motion, multiple his book “Frames of Mind: The Theory of Multiple Intelligences”. MI theory
intelligences, physics education. goes beyond the classical point of view in the theory and is defined as an
individual’s ability to introduce a product, which is considered worthwhile
in one or more cultures, and solve a problem they encounter in their daily
or professional lives effectively and efficiently (Gardner, 1983). In this sense,
Deniz Gurcay, Gardner (1983) does not restrict individuals’ capacities to their IQs only and
Hatice Ozturk Ferah
Hacettepe University, Turkey defines seven intelligences, which he argues people possess. Moreover,
Gardner also maintains that this number may be inadequate in expressing
the multiplicity of human abilities (Chekley, 1997). In support of this, in an
666
ISSN 1648–3898 /Print/ THE EFFECTS OF MULTIPLE INTELLIGENCES BASED INSTRUCTION ON STUDENTS’ PHYSICS
ISSN 2538–7138 /Online/ (P. 666-677)
interview held by Chekley (1997) with Gardner, Gardner mentioned the existence of an eighth intelligence in his
work titled “Intelligences Reframed”, which was published in 1999, and he reframed the MI theory in a way that
would cover this field of intelligence (Gardner, 1999). The MI defined by Gardner are as follows: verbal-linguistic
intelligence, logical-mathematical intelligence, musical-rhythmic intelligence, visual-spatial intelligence, bodily-
kinesthetic intelligence, interpersonal intelligence, intrapersonal intelligence, and naturalistic intelligence. Chan
(2006, p.326) indicated that “Regardless of the classification, this pluralistic view of human cognitive abilities sug-
gests that the theory of MI provides useful framework for understanding individuals’ basic competencies, as well
as their unique strengths”.
The MI theory has become important in educational settings as it broadens the traditional view of intelligence
as mainly composed of verbal-linguistic and logical-mathematical abilities (Christison, 1999). The MI, except verbal-
linguistic intelligence and logical-mathematical intelligence, on the other hand, are not given proper attention
or taken seriously within the educational system. Thus, many students who fail to demonstrate the traditional
academic intelligences have been remaining unrealized (Campbell et al., 1996). MI theory emphasizes the impor-
tance of creating student-centered learning environments and individualizing teaching (Kagan & Kagan, 1998).
Moreover, MI-based learning environments take into consideration students’ different learning capacities, show
them that there are various ways of being smart (Christodoulou & Kunkel, 2009) and offer to them diverse ways
of learning (Haggarty, 1995). In learning environments, where students’ MI are taken into consideration, students
will discover their own competences and personalize information meaningfully. It is evident that the quality of
learning will increase in learning environments where students’ individual characteristics are taken into account
(Isik & Tarim, 2009).
Chapman (1993) and Armstrong (1994) indicated that the MI theory makes great contributions to education.
Moreover, they suggested that teachers need to expand their repertoire of techniques, tools and strategies beyond
the typical linguistic and mathematical ones predominantly used in classrooms. MI theory may also serve as a
framework for teachers to explore their teaching styles and to help them in making decisions about teaching and
learning experiences for students (Goodnough, 2001). Moreover, Kagan and Kagan, (as cited in Ozdemir, Guneysu,
& Tekkaya, 2006, p.74) indicate “The MI theory can be used to match teaching to how students learn, to encourage
students to stretch their abilities to develop all their intelligences as fully as possible, and honor diversity”. Science
teachers could be able to help their students in improving their MI and enable them, by offering different ways of
learning, to learn science by doing and experiencing through MI-based instruction.
To be able to enrich learning environments, it will be useful for teachers to have information about the students’
MI profiles before they come to the learning environment. Based on the view that standardized measurements
will measure only some of the competences, Gardner (1993) recommends that alternative testing techniques be
used in the assessment of MI. However, student self-reports conducted through paper-pencil checklists are con-
sidered important sources of data in determining students’ MI profiles and it is emphasized that such self-report
measures can be used to identify students’ MI profiles (Armstrong, 1994; Chan, 2001). Moreover, Gardner states that
to measure intelligence, a separate scale needs to be developed for each intelligence and respondents need to be
untroubled while they are dealing with materials and methods that measure these MI dimensions (Gardner, 1993).
When the relevant literature was reviewed, it was observed that generally validity and reliability analyses of
the self-report MI measurement instruments had not been conducted. There are limited numbers of MI instruments
whose validity and reliability have been reported (Shearer, 1994; Chan, 2001, 2006; Tirri & Nokelainen, 2008; Mc-
Clellan & Conti, 2008; Teele, 1997). Student Multiple Intelligences Profile (SMIP-24), which was developed by Chan,
(2001, 2006) was administered to 1560 Chinese primary and middle school students. According to the results of the
confirmatory factor analysis, the model demonstrates good fit when the fit indices are taken into consideration.
Multiple Intelligences Developmental Assessment Scale (MIDAS), which was developed by Shearer (2005), has
so far been administered to 8497 students aged 9-12, 1347 college and university students and 1071 adults. As a
result of the confirmatory factor analysis, model exhibited good fit. Teele Inventory of Multiple Intelligences (TIMI)
(Teele, 1997) was administered from primary school level to the university level and it was stated to be reliable.
When studies concerning MI are examined, various studies that investigate students’ MI profiles are encoun-
tered. In a study conducted by Uysal and Eryilmaz (2006), seventh and tenth grade students’ dominant intelligences
were investigated and interpersonal intelligence was found to be the dominant intelligence for both class levels.
In another study conducted by Demir and Aybek (2012), on the other hand, it was found that the most dominant
intelligences of ninth graders were intrapersonal, interpersonal and visual-spatial intelligences. Razak and Zaini
(2014) concluded in a study they conducted that students’ dominant intelligences were bodily-kinesthetic, inter-
667
(P. 666-677) ISSN 2538–7138 /Online/
personal, visual-spatial and musical-rhythmic intelligences. Moreover, Ozdemir et al., (2006) stated that fourth
grade students’ dominant intelligences were logical-mathematical, visual-spatial and interpersonal intelligences.
Some researches indicated the positive evidences of MI-based instruction for the academic achievement
and attitudinal behaviors of the students at high school. The MI theory used in the educational settings increases
students’ attitudinal behaviors (Campbell, 1992). Moreover, The MI theory improves students’ learning quality
(Campbell, 1992). The Project Zero research group founded by Howard Gardner conducted a study entitled Schools
Using Multiple Intelligence Theory (SUMIT) to determine the benefits of MI for students at schools. According to
the results of this longitudinal study, which was implemented in 41 schools that had been carrying out MI-based
instruction for at least three or four years, standard test achievements of the students of 80% of the schools im-
proved and they developed positive attitudes (Kornhaber, Fierros & Veenema, 2004). Today, some of the schools
(e.g. The Key School in Indianapolis, The Matter School in Boston) have based their curricula entirely on MI theory
owing to these positive aspects of the MI theory (Blythe & Gardner, 1990; Hatch, 1993; Hoerr, 1994). In addition,
as far as the Turkish Education System is concerned, both the science curriculum and the physics curriculum 2007
emphasize the importance of the use of various active learning methods, notably the MI theory, which considers
individual differences and enables construction of knowledge (MNE, 2007, 2013).
When studies investigating the effects of MI-based instruction on students’ achievements and attitudes
were examined, it was found that these studies involved science such as biology and chemistry and mathematics.
However, there is limited number of experimental studies investigating whether or not MI-based instruction had a
significant effect on students’ achievements and attitudes towards chemistry, biology and mathematics (Dillihunt
& Tyler, 2006; Ozdemir et al., 2006; Ucak, Bag & Usak, 2006; Isik & Tarim, 2009). In addition, studies that investigated
the effects of MI-based instruction on students’ achievements and attitudes towards physics have been also found
to be limited (Gurcay & Eryilmaz, 2005).
The aim of this research is to examine whether or not the MI-based instruction has a significant effect on 9th
grade students’ achievements in and attitudes towards the force and motion concepts in the physics course in
comparison with the traditional instruction. For this purpose, answers were sought to the following questions:
1. Is there a significant effect of MI-based instruction on 9th grade students’ achievements in the force
and motion?
2. Is there a significant effect of MI-based instruction on 9th grade students’ attitudes towards the force
and motion?
3. Is there a significant difference in the experimental and control groups’ MI scores before and after the
intervention?
4. What are the students’ views concerning MI-based instruction?
General Background
Quasi-experimental pre-test, post-test control group design was used in this research. Since it was impossible
to assign the students in the experimental and control groups in the two schools randomly to classes, this research
is quasi-experimental (Shadish, Cook & Campbell, 2002). Therefore, this research was conducted in 2013-2014
academic year, with four classes from two schools, 2 were experimental classes with MI based instruction and two
were control classes with traditional instruction, to elicit the effects of MI based instruction on students’ achieve-
ment and attitudes. Moreover, it lasted 8 weeks, as a week pre-test, six-weeks treatment and a week post-test.
Sample
The participants of this research are comprised of 9th grade students attending two primary schools in
Ankara-Turkey. Four classes, one experimental group and one control group from each school, were assigned ran-
domly. Ninety-five students, 50 in the control group and 45 in the experimental group, participated in this research.
The proportion of male (53.7) and female students (46.3) was at about the same level. Most of the students (68.4
%) were aged 15, 16.8 % were aged 16 and 14.7 % were aged 14. All of the subjects voluntarily participated in this
research. Before the research, they are informed about the research methodology. They fulfilled the consent form
and their names are kept anonymous.
668
ISSN 2538–7138 /Online/ (P. 666-677)
Instruments and Procedures
In order to collect the data in this research, Force and Motion Achievement Test (FMAT), Force and Motion
Attitude Scale (FMAS), Revised Student Multiple Intelligences Profile Questionnaire (SMIP-24) and Students’ Views
about MI-based instruction Questionnaire (SVQ) were used.
Force and Motion Achievement Test (FMAT)
This test was developed by the researchers to measure the ninth grade students’ achievement in the force
and motion concepts. The topics covered in this test involved motion in one dimension, classification of motion,
position, distance, displacement, velocity, speed, linear motion, acceleration, force, friction and Newton’s laws of
motion, which had to be taught according to the ninth grade curriculum. In order to develop FMAT, first objectives
of the ninth grade were taken into consideration and 47 multiple choice questions with five choices were developed.
The questions in the test were taken from the physics books and university entrance examinations administered in
Turkey. For the content validity of the test, an expert in physics education, a physics professor and a physics teacher
checked whether the test items matched students’ levels, the physics curriculum and then necessary changes were
made. Furthermore, at the end of this process, test items were administered to five ninth grade students; feedback
about to what extent the items were understood was obtained and necessary changes were performed.
The pilot study of the test was conducted on 124 students at 9th grade. As a result of the item validity analysis
and reliability analysis, 22 items were removed from the test, taking into consideration the item difficulty, reliability
and discrimination index and thus the final test have 25 items. Items whose item discrimination index is below
.30 should be removed from the test (Ding, Chabay, Sherwood & Beichner, 2006). Therefore, the items in this test
whose item discrimination indices were below .30 were removed. As a result of this pilot study, the average item
difficulty of the 25-item FMAT was calculated to be .50, whereas its item discrimination index was .40 and Cronbach
Alpha reliability coefficient was .73. The minimum score that could be taken from the test was 0 while the maximum
score was 25. The students were given a class hour (40 minutes) to answer the test. FMAT was administered to the
experimental and control groups as pre-test and post-test. The Cronbach Alpha reliability coefficient calculated
as .78 for the pre-test scores and .81 for the post-test scores of FMAT.
Force and Motion Attitude Scale (FMAS)
In order to measure the ninth grade students’ attitudes towards the force and motion FMAS was used. This
scale was developed by Taslidere (2012) in order to measure students’ attitudes towards simple electric circuits.
However, he stated that this scale could be used to measure students’ attitudes toward other physics topics since it
is a content based scale. Therefore, we made wording on the original scale. FMAS is a 5-point Likert scale consisting
of 24 items that involves the following statements: “I totally agree, I agree, I am undecided, I disagree and I totally
disagree”. The scores that could be received from FMAS vary between 24 and 120. Taslidere (2012) determined the
Cronbach Alpha reliability coefficient of the scale as .93. On the other hand, Cronbach Alpha reliability coefficients
of FMAS were calculated for its pilot-, pre-test-, and post-test administrations as .92.
Revised Student Multiple Intelligences Profile Questionnaire (SMIP-24)
In this research, SMIP-24, developed and then revised by Chan (2001; 2006), was used to determine students’
MI dimensions. First, MI scales developed by various researchers were examined, taking into account criteria such
as validity and reliability analyses, item numbers in the scale and their conformity with students’ readiness (Shearer,
1994; Teele, 1997; Chan, 2006; Tirri & Nokelainen, 2008; McClellan & Conti 2008). Then it was decided that SMIP-24
(Chan, 2006) be used. SMIP-24 is a 24-item self-report questionnaire consisting of three items for each of the eight
MI dimensions. Participants responded themselves using a five-point scale ranging from zero (least descriptive)
to five (most descriptive). The lowest score that could be taken for each MI dimension was zero while the highest
score was 15.
Within the scope of this research, first SMIP-24 was adapted to Turkish. For this purpose, two experts in English
translated English version of the questionnaire into Turkish. During the process of translation, special attention
was paid to conceptual, idiomatic and field correspondence. Following the completion of the translation process,
669
(P. 666-677) ISSN 2538–7138 /Online/
a language expert was consulted and then another English expert was asked to analyze the questionnaire in terms
of grammar and translate the questionnaire items back into English, the source language. It was determined sub-
sequent to the translation practices that there were no meaning changes or meaning losses. After the completion
of the translation process, assistance was received from two experts in Turkish language and literature department
to test the conformity of the questionnaire with Turkish. Vocabulary changes suggested by the experts were made.
A group of five students read the questionnaire items and they were asked if the items were comprehensible.
Then, the pilot study of SMIP-24 was carried out with 432 students attending 9th grade. The students were given
25 minutes to answer the test. Confirmatory factor analysis was performed in the study conducted by Chan (2006)
to test the construct validity of SMIP-24. The fit indices belonging to the model are given in Table 1.
Table 1. Fit indices of SMIP-24 (Chan, 2006).
RMSEA CFI S-RMR GFI ECVI df
3.25 .056 .909 .044 .935 1.204 448
In this research, Confirmatory Factor Analysis (CFA) was performed to determine the construct validity of
SMIP-24 through the pilot study. The fit indices calculated are given in Table 2. When Table 2 was investigated,
fit statistics in general were similar with the original SMIP-24 questionnaire and acceptable for the model fit.
Normed chi-square value less than 5 indicates an acceptable fit between a hypothesized model and sample data
(Schumacker & Lomax, 2004). The Standardized Root Mean Square Residuals (SRMR) value less than .1 and the Root
Mean Squared Error of Approximation (RMSEA) value less than .08 regarded as acceptable fit (Browne & Cudeck,
1993). Comparative Fit Index (CFI) and Goodness of Fit Index (GFI) values are regarded acceptable if they are greater
than 0.9 (Bentler 1995). Generally, fit indices of this research revealed that there is a good model fit. In addition, it
has been stated that when the CFI value is .85 and above, it is acceptable (Bollen, 1989). Therefore, in this research
CFI value shows a reasonable fit.
Table 2. Fit indices of the pilot study.
RMSEA CFI S-RMR GFI ECVI df
2.19 .056 .880 .063 .900 1.710 222
Cronbach Alpha was calculated for each MI dimension to determine the internal consistency coefficient of
SMIP-24. Cronbach Alpha coefficients calculated by Chan (2006) were as follows: for verbal-linguistic intelligence
.61, for logical-mathematical intelligence .60, for visual-spatial intelligence .61, for bodily-kinesthetic intelligence
.67, for intrapersonal intelligence .73, for interpersonal intelligence .72, for musical-rhythmic intelligence .76, and
for naturalistic intelligence .77. In the pilot study of the questionnaire, the Cronbach Alpha coefficients of the MI
dimensions were .65 for the verbal-linguistic intelligence, .66 for the logical-mathematical intelligence, .70 for the
visual-spatial intelligence, .74 for the bodily-kinesthetic intelligence, .75 for the intrapersonal intelligence, .78 for
the interpersonal intelligence,.73 for the musical-rhythmic intelligence, and .80 for the naturalistic intelligence. In
this research, on the other hand, Cronbach Alpha coefficients for each MI dimensions were as follows: .55 for the
verbal-linguistic intelligence, .61 for the logical-mathematical intelligence, .63 for the visual-spatial intelligence, .74
for the bodily-kinesthetic intelligence, .75 for the intrapersonal intelligence, .75 for the interpersonal intelligence,
.73 for the musical-rhythmic intelligence, and .80 for the naturalistic intelligence.
Students’ Views about MI-Based Instruction Questionnaire (SVQ)
At the end of the treatment, five open-ended questions were prepared by the researchers to determine the
students’ views concerning MI-based instruction. At the end of the treatment, only the students in the experimental
670
ISSN 2538–7138 /Online/ (P. 666-677)
group were asked to respond in writing to the questions on the SVQ. Response format was open-ended. The ques-
tions on the questionnaire were as follows: “Your physics classes have been conducted according to the MI-based
instruction. What would you say about these physics classes compared with your previous physics classes?”, “Are
there any aspects which you find inadequate concerning the way your physics classes are being taught?”, “Would
you like your physics classes to continue to be conducted according to MI-based instruction?”, “What could be the
advantages and disadvantages of your physics classes being taught according to the MI-based instruction?”, “What
aspects of you has the implementation of your physics classes according to the MI-based instruction improved?”
Treatment
At first, in this quasi-experimental pre-test-post-test control group research, the experimental and control
groups were given the pre-tests. After the pre-tests, the experimental groups were exposed to the treatment, and
then all the groups were given the post-tests. Before the treatment, all of the classes were given the FMAS, FMAT
and SMIP-24 as pre-tests. After the pre-tests, the students in the experimental group were exposed to the MI-based
instruction for six weeks. Meanwhile, the control groups were exposed to the traditional teaching method. The
students in the control group were taught the force and motion in the physics course through techniques such
as direct teaching, discussion and question-answer for six weeks. No materials other than course books and note-
books were used in control group classes. These classes were taught by the teacher. Moreover, students, listened
to the teacher and asked questions when they did not understand a subject. In addition, homework was assigned
after the courses. Then the same tests were administered as post-tests to all of the classes. Furthermore, SVQ was
administered to the experimental group after the treatment. Thus, this research lasted 8 weeks, as a week pre-test,
six-week treatment and a week post-test.
The students in the experimental group were exposed to MI-based lesson plans developed by the re-
searcher. The MI-based lesson plans, handouts, materials and teachers’ guidebook were prepared from the books
of different researchers (Bellanca, 1997; Campbell & Campbell, 1999; Dillihunt & Tyler 2006). Objectives for the force
and motion unit included in the ninth grade curriculum in Turkey were taken into consideration in the lesson plans.
In every course, the activities intended to reach diverse MI dimensions. Table 3 shows MI dimensions intended
to be covered every week through activities. The activities in MI lesson plans were selected and prepared among
the activities given by Dillihunt and Tyler (2006). Before the implementation of the MI-based lesson plans, views
of two field experts and two physics teachers were taken concerning the compatibility of the activities with the
curriculum, their compatibility with the relevant intelligence dimension and with the students’ level of readiness.
Necessary changes were made based on experts’ views.
Table 3. Targeted MI dimensions in the activities.
Weeks MI Dimensions Used in Lesson Plans
1st week Verbal-Linguistic, Logical-Mathematical, Visual-Spatial, Intrapersonal, Interpersonal, Musical-Rhythmic

2nd week Verbal-Linguistic, Logical-Mathematical, Visual-Spatial, Intrapersonal, Interpersonal
3rd week Verbal-Linguistic, Visual-Spatial, Bodily-Kinesthetic, Intrapersonal, Interpersonal, Naturalistic
4th week Verbal-Linguistic, Logical-Mathematical, Visual-Spatial, Bodily-Kinesthetic, Intrapersonal
5th week Verbal-Linguistic, Logical-Mathematical, Visual-Spatial, Bodily-Kinesthetic Intrapersonal, Interpersonal, Naturalistic,
6th week Verbal-Linguistic, Logical-Mathematical, Visual-Spatial, Intrapersonal, Interpersonal Musical-Rhythmic,

During implementation of the MI-based lesson plans, the students in the experimental group were given
activities of MI dimensions. All the students performed the activities at the same time and at the end of the activi-
ties shared their performance with their friends. The activities were prepared in such a way as to allow students to
participate actively and engage in group work. Moreover, students were assigned homework covering various MI
dimensions at the end of the classes. Table 4 shows examples of activities given in the activities concerning each
MI dimension:
671
(P. 666-677) ISSN 2538–7138 /Online/
Table 4. MI related activities in the lesson plans.

Verbal-Linguistic Intelligence Activities: Discussions, dialogues, word games, poetry, storytelling, reading, oral presentations
Logical-Mathematical Intelligence Activities: Socrating questioning, scientific investigations, problem solving, logical-sequential
assignments
Visual-Spatial Intelligence Activities: Graphic rich environments (visual and graphic organizers, pictures, posters, charts,
graphs, diagrams)
Bodily-Kinesthetic Intelligence Activities: Role plays, creative movement (mime, dance, drama, table techniques –body used
to communicate)
Intrapersonal Intelligence Activities: Reflection time/opportunity (journals), independent instructional assignments
Interpersonal Intelligence Activities: Social cooperative learning, opportunity to help others
Musical-Rhythmic Intelligence Activities: Curriculum songs (creating melodies, songs, rap, cheers, jingles etc.), poetry,
poems
Naturalistic Intelligence Activities: Environmental issues, outdoor activities
Data Analysis
Multivariate statistical analysis (MANCOVA) and descriptive analyses were conducted to examine whether or
not MI-based instruction had a significant effect on students’ achievements in and attitudes towards the force and
motion unit, when the FMAT- and FMAS- pre-test scores were taken under control. Before the MANCOVA analysis,
assumptions of MANCOVA were checked.
Results of Research
Table 5 shows results of the FMAT pre-test, post-test descriptive statistics of experimental and control groups.
An increase of 3.42 points is observed between the pre-test and post-test scores of the experimental group (Table
5). On the other hand, there is an increase of 1.7 points between the pre-test and post-test scores of the control
group. This increase is lower than the increase achieved in the experimental group.
Table 5. Results of FMAT descriptive statistics.
Experimental Group Control Group
FMAT Pre-test Post-test Pre-test Post-test

Mean 14.91 18.33 14.76 16.46
Standard Deviation 4.41 3.46 4.27 4.75
N 45 45 50 50
Table 6 shows results of the FMAS pre-test and post-test descriptive statistics of the experimental and control
groups. An increase of 4.17 points is observed between the pre-test and post-test scores of the experimental group
(Table 6). However, an increase of 3.34 points is seen between the pre-test and post-test scores of the control group.
This increase is lower than the increase achieved in the experimental group.
Table 6. Results of FMAS descriptive statistics.
FMAS Pre-test Post-test Pre-test Post-test
Mean 80.47 84.64 77.98 81,32

N 45 45 50 50
672
ISSN 2538–7138 /Online/ (P. 666-677)
The students’ MI pre-test and post-test scores in the experimental group are given in Figure 1. As a result of
the t-test, significant increases were observed in the students’ musical-rhythmic intelligence (p= .000), bodily-
kinesthetic intelligence (p= .014) and naturalistic intelligence (p= .013) pre-test-post-test scores (p< .05). There
was not any significant difference between pre-test and post-test scores for the other MI dimensions. Moreover, in
Table 1, it is seen that MI-based instruction improved students’ all MI scores. The highest increase was observed in
the students’ musical-rhythmic intelligence scores, whereas the lowest increase was observed in the interpersonal
intelligence scores.
Figure 1: Experimental group students’ pre-test-post-test MI scores.
The students’ MI dimensions pre-test and post-test scores in the control group are given in Figure 2. As a result
of the t-test, a significant increase was not observed between the pre-test and post-test MI scores of the students
in the control group concerning each intelligence area. Moreover, according to Figure 2, an increase is observed in
the verbal-linguistic, logical-mathematical, visual-spatial, bodily-kinesthetic, musical-rhythmic and intrapersonal
intelligence scores of the students in the control group, where classes were taught using traditional teaching
techniques. A decrease is observed in the average interpersonal intelligence pre-test scores, but no changes are
observed in the average naturalistic intelligence scores.
Figure 2: Control group students’ pre-test-post-test MI scores
673
(P. 666-677) ISSN 2538–7138 /Online/
MANCOVA Results
Since there was not any missing data, missing data analysis was not performed. Assumptions of independence
of observations, normality, multicollinearity, equality of variances, and homogeneity of regressions were tested
and verified for MANCOVA analysis.
In this research, the instruction method FMAT pre-test scores and FMAS pre-test scores were independent
variables, whereas FMAT post-test and FMAS post-test scores were dependent variables. FMAT pre-test scores and
FMAS post-test scores were used as covariates to statistically equalize the differences between the experimental
and control groups. According to the multivariate test results, null hypothesis was rejected ( l= 0.912; df= 90; F=
4.35; p= .016 ). This means, there is a significant mean difference between MI-based instruction and traditional
instruction on the collective dependent variables of FMAT post-test and FMAS post-test scores. Follow-up analyses
are necessary to determine which dependent variable is responsible for the variance. To test the effect of “method”
on each dependent variable, a univariate analysis of covariance (ANCOVA) was conducted as follow-up test. The
results showed a statistically significant effect of the “method” on the FMAT post-test (F= 8.29; df= 1; p= .005) and
statistically no significant effect of the “method” on the FMAS post-test (F= .409; df= 1; p= .52).
Students’ Views about MI Based Instruction Questionnaire (SVQ)
When students were asked to compare the classes conducted the MI-based instruction with their previous
classes, 57.7 % of the students in the experimental group stated that the classes conducted the MI-based instruc-
tion were both enjoyable and instructive. 20 % of the students responded to the same question saying that they
understood the subjects better in this way. When the students were asked if there were any shortcomings with
regard to the implementation of the classes, 84.4 % of the students stated that there were no shortcomings. More-
over, when they were asked whether they wanted their classes to continue through practices based on the MI
dimensions, 97.7 % of the students stated that they wanted to continue their classes in this way. When they were
asked if experimental classes had any disadvantages, 84.4 % of the students responded that they did not notice any
disadvantages. 60 % of the students stated that they found the classes both comprehensible and enjoyable while
31.1 % of them emphasized, that handouts were useful because they increased their participation in the classes
and added that these were advantages of the implementation. Finally, when the students were asked what aspects
of them improved in the experimental classes, 24.4 % of the students responded that it helped them understand
the subjects better. 15.5 % responded that it improved their problem solving skills and 13.3 % stated that it raised
their interest in the classes.
Discussion
According to the MANCOVA results of this research, MI-based instruction has a significant effect on the students’
achievements in force and motion topics. This result supports the results of other studies. The effect of MI-based
instruction on high school ninth grade students’ achievements and attitudes towards physics was also investigated
by Gurcay and Eryilmaz (2005) and according to the findings of the research, MI-based instruction had a significant
effect on high school first year students’ achievements in physics but that it did not have a significant effect on
their attitudes. Moreover, Ozdemir et al., (2006) also investigated the effect of MI-based instruction, compared
with the traditional instruction, on fourth grade students’ achievements in Diversity of Living Things Concepts
and their retention of knowledge and found that it had a significant effect on both their achievements and their
retention of knowledge. Dillihunt and Tyler (2006) stated that MI-based instruction increased fifth grade students’
mathematics achievements significantly. Likewise, Douglas, Burton and Reese-Durham (2008) found that MI-based
instruction had a significant effect on eighth grade students’ mathematics achievements compared with the direct
instruction. In addition, Campbell and Campbell (1999) emphasized based on the studies they conducted, that
the achievement scores in reading, language, mathematics, science and social studies had improved significantly.
Although the results of the studies mentioned above belong to different disciplines and different age groups,
they support the results of this research related to the student achievements. In another study, Ucak et al. (2006)
investigated if there was a significant difference between MI-based instruction and traditional instruction on the
seventh graders’ understanding and attitudes towards “The Structure of Material and Its Transformation”. The results
of Ucak et al. (2006)’s study, in which a 4-week treatment was conducted, indicated that MI-based instruction had
674
ISSN 2538–7138 /Online/ (P. 666-677)
a significant effect both on students’ achievements and on the improvement of their attitudes. The results of our
research do not support Ucak et al. (2006)’s results concerning attitude. This is thought to have stemmed from the
sample selected. The other reason of this might be due to inadequacy of the duration of intervention. Although,
it was determined that most of the students in the experimental group had positive thoughts about MI based
instruction, in this study no significant difference in students’ attitudes towards physics was determined compared
to traditional learning. Therefore, it could also be thought that 6-week research period might not be enough to
observe significant change in order that the positive opinions of the students to change their attitudes towards
physics learning in a significant way. Research results show that, more time is needed for students’ thoughts to
lead a change in their behaviors since there are many variables that are in relation with the development of posi-
tive attitudes (Wilkins & Ma, 2010).
Goodnough (2001) reported, as a result of her qualitative research, that most students (85%) enjoyed using
MI-based instruction and they liked working collaboratively with others on projects. Moreover, most of the students
implied that MI-based instruction helped them to enjoy science more. These results also support the results of this
research. The students in the experimental group (60%) reported that they learned physics and enjoyed physics.
Moreover, majority of the students were in favor of the MI-based instruction. In MI based learning environments,
the students had the chance to learn physics by using their strong intelligence. Moreover, they also had the chance
to learn physics in a variety of ways. They also actively participated in the learning process. Because of that reason
students might have positive views about MI based learning environments.
In this research, students’ MI dimensions with the lowest mean scores were logical-mathematical intelligence,
verbal-linguistic intelligence and musical-rhythmic intelligence in both the experimental and the control groups
before and after the treatment. On the other hand, intrapersonal intelligence, interpersonal intelligence and
naturalistic intelligence were found as the dominant intelligences. The findings of this research bear similarity to
the results of various studies in the literature. In Chan’s (2001) study, which was conducted on 192 students rang-
ing from the seventh grade to the 12th grade, MI dimensions with the highest mean scores were interpersonal
intelligence and intrapersonal intelligence, whereas the intelligence dimension with the lowest mean scores were
verbal-linguistic intelligence, bodily-kinesthetic and visual-spatial intelligence. When studies concerning MI are
examined, research results show that students’ dominant intelligence is interpersonal intelligence (Demir & Aybek,
2012; Razak & Zaini, 2014; Ozdemir et al.; 2006, Uysal & Eryilmaz, 2006). However, students with strong logical-
mathematical intelligence and verbal-linguistic intelligence at schools are regarded as having an advantage over
others, whereas those who are weak in these MI dimensions are regarded as disadvantaged. The results of this
research show that most of the students were disadvantaged in schools.
Moreover, according to the results of this research, a significant difference was observed between the musical-
rhythmical intelligence, bodily-kinesthetic intelligence and naturalistic intelligence pre-test and post-test scores
of the students in the experimental group. However, it was seen that there was no significant increase in the MI
dimensions pre-test and post-test scores of the students in the control group. A study conducted by Ozdemir et
al., (2006) also demonstrated that MI-based instruction strengthened students’ multiple intelligences. It was stated
that the intelligence where the most variation was observed was the musical-rhythmic intelligence area and that
according to the t-test results, a significant increase occurred between the pre-test and post-test results in the
students’ musical-rhythmic intelligence scores (Ozdemir et al., 2006). Physics classes, in particular, are conducted in
traditional classes through activities of logical-mathematical intelligence and verbal-linguistic intelligence (Chris-
tison, 1999). This means that the students are exposed to the activities of their weak intelligence dimensions. Yet,
through MI-based instruction, improvements can be made in MI dimensions where students are weak and at the
same time, the quality of their learning can be raised through activities in intelligence where they are strong so
that more enjoyable classes can be achieved (Gardner, 1989; Goodnough, 2001; Gurcay & Eryilmaz, 2005).
Conclusion and Implications
The results of this research indicate that MI-based learning environment has a significant effect, compared
with the traditional instruction, on the improvement of students’ academic achievements but that it does not
have a significant effect on their attitudes. A majority of the students in the experimental group stated that their
interest and achievement in the physics classes increased and added that they found MI-based physics instruction
more enjoyable and instructive.
Teachers need to know their students’ individual characteristics and create learning environments suiting
675
(P. 666-677) ISSN 2538–7138 /Online/
them. Moreover, teachers should be aware that all children have strengths. By the help of MI theory, teachers can
incorporate all types of the intelligences in teaching and learning. If teachers use the MI-based instruction, students
could reveal their strengths to show what they have learned. Moreover, valid and reliable self-report checklists,
whose validity and profiles can help teachers to see their students’ strong and weak intelligences. Due to the fact,
that the number of studies conducted in physics and other branches of science concerning the effects of MI-based
instruction is limited, there is a need for qualitative and quantitative studies that will be carried out on larger samples.

References
Angell, C., Guttersrud, Ø., Henriksen, E. K., & Isnes, A. (2004). Physics: Frightful, but fun, pupils’ and teachers’ views of physics and
physics teaching. Science Education, 88, 683-706.
Armstrong, T. (1994). Multiple intelligences in the classroom. Virginia, USA: Association for Supervision and Curriculum Development.
Baumgartner, H., & Homburg, C. (1996). Applications of structural equation modeling in marketing and consumer research: A
review. International Journal of Research in Marketing, 13 (2), 139-161.
Bellanca, J. (1997). Active learning handbook for the multiple intelligences classroom. USA: SkyLight Training and Publishing.
Bentler, P. M. (1995). EQS structural equations program manual. Encino, CA: Multivariate Software
Blythe, T., & Gardner, H. (1990). A school for all intelligences. Educational Leadership, 47 (7), 33-37
Bollen, K. A. (1989). Structural equations with latent variables. New York: John Wiley & Sons, Inc.
Browne, M.W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In: Bollen, K.A., & Long, J.S. (Eds.), Testing structural
equation models (pp. 136-162). Beverly Hills, CA: Sage
Campbell, B. (1992). Multiple intelligences in action. Childhood Education, summer, 197-202.
Campbell, L., Campbell, B., & Dickinson, D. (1996). Teaching and learning through multiple intelligences. Massachusetts. USA. A
Simon and Schuster Company.
Campbell, L., & Campbell, B. (1999). Multiple intelligences and student achievement: Success stories from six schools. Virginia, USA:
Association for Supervision and Curriculum Development.
Checkley, K. (1997). The first seven ….and the eight: A conversation with Howard Gardner. Educational Leadership, 55 (1), 8-13.
Chan, D. W. (2001). Assessing giftedness of Chinese secondary students in Hong Kong: A multiple intelligences perspective.
High Ability Studies, 12, (2), 215-234.
Chan, D. W. (2006). Perceived multiple intelligences among male and female Chinese gifted students in Hong Kong: The structure
of the student multiple intelligences profile. National Association for Gifted Children, 50, 325-338.
Chapman, C. (1993). How to develop multiple intelligences in the classroom. Illinois, USA. IRI / Skylight Publishing Inc.
Christison, M. A. (1999). Teaching the whole student. ESL Magazine, 2 (5), 10-13.
Christodoulou, J. A., & Kunkel, C. D. (2009). Applying multiple intelligences. School Administrator, 66 (2), 22-27.
Demir, R., Aybek, B. (2012). Investigation of the ninth grade students’ learning styles and multiple intelligences. International
Journal of Curriculum and Instruction Studies, 2, 27-40.
Dillihunt, M. L., & Tyler, K.M. (2006). Examining the effects of multiple intelligences instruction on math performance. Journal of
Urban Learning, Teaching and Research, 2, 131-150.
Ding, L., Chabay, R., Sherwood B., & Beichner, R. (2006). Evaluating electricity and magnetism assessment tool: Brief electricity
and magnetism assessment. Physical Review Special Topics - Physics Education Research, 2(1), 1-7.
Douglas, O., Burton, K. S., & Reese-Durham, N., (2008). The effects of the multiple intelligence teaching strategy on the academic
achievement of eighth grade math students. Journal of Instructional Psychology, 35, 182-187
Fonseca, J. M. B., & Conboy, J. E. (1999). Introductory physics for nonphysics majors-A case study. Journal of College Science
Teaching, February, 272–277.
Gardner, H. (1983). Frames of mind the theory of multiple intelligences. New York, USA: Basic Books.,
Gardner, H. (1989). To open minds: Chinese clues to the dilemma of contemporary education. New York: Basic Books.
Gardner, H. (1993). Multiple intelligences theory in practice. New York: Basic Books.
Gardner, H. (1999). Intelligence reframed. New York: Basic Books.
Goodnough, K. C. (2001). Multiple intelligences theory: A framework for personalizing science curricula. School Science and
Mathematics, 101 (4), 180-193.
Gurcay, D., & Eryilmaz, A. (2005). Effect of physics achievement of students teaching based on multiple intelligences. Hacettepe
University Journal of Education, 29, 103-109.
Haggarty, B. A. (1995). Nurturing multiple intelligences: A guide to multiple intelligences theory and teaching. New York. Addison
Wesley.
Hatch, M. J. (1993). The dynamics of organizational culture. Academy of Management Review, 18, 657–93
Hoerr, T. R. (1994). How the new city applies the multiple intelligences. Educational Leadership, 52 (3), 29-33.
Hoerr, T. R. (2000). Becoming a multiple intelligences school. Retrieved April 19, 2001 from http://www.ascd.org./readingroom/
books/hoerr00.html.
Isik, D., & Tarim, K. (2009). The effects of the cooperative learning method supported by multiple intelligence theory on Turkish
elementary students’ mathematics achievement. Asia Pacific Education Review, 10, 464-474.
Jesse L. M. W. & Xin M. (2003). Modeling change in student attitude toward and beliefs about mathematics. The Journal of Edu-
cational Research, 97 (1), 52-63.
676
ISSN 2538–7138 /Online/ (P. 666-677)
Kagan, S., & Kagan, M. (1998). Multiple intelligences: The complete MI book. San Clemente: Kagan Publishing.
Kornhaber, M. L., Fierros, E., & Veenema, S. (2004). Multiple intelligences: Best ideas from theory and practice. Needham Heights,
MA: Allyn & Bacon
McClellan, J. A., Conti, G. J. (2008). Identifying the multiple intelligences of your students. Journal of Adult Education, 37 (1), 13-32.
Ministry of National Education (MNE), (2007). Secondary Physics Curriculum. Retrieved from: http://ogm.meb.gov.tr/belgeler/
fizik9.pdf.
Ministry of National Education (MNE), (2013). Secondary Physics Curriculum. Retrieved from: http://mebk12.meb.gov.tr/
meb_iys_dosyalar/31/01/972850/dosyalar/2013_07/05032334_fizik_912.pdf.
Ornek, F., Robinson W. R., & Haugan, M. P. (2008). What makes physics difficult? International Journal of Environmental & Science
Education, 3 (1), 30 – 34.
Ozdemir, P., Guneysu, S., & Tekkaya C. (2006). Enhancing learning through multiple intelligences. Journal of Biological Education,
40 (3), 74-78.
Razak, N. A., & Zaini, N. (2014). Multiple intelligences scores of science stream students and their relation with reading compe-
tency in Malaysian University English Test. English Language Teaching, 7 (2), 63-72.
Rubin, J., & Herbert, M. (1998). Peer teaching-model for active learning. College Teaching, 48, 26–30.
Schumacker, R. E., & Lomax, R. G. (2004). A beginner’s guide to structural equation modeling. Mahwah, NJ: Lawrence Erlbaum
Associates.
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference.
Houghton Mıfflın Company, New York.
Shearer, C. B. (1994). The MIDAS: Professional manual. Ohio: MI Research and Consulting, Inc.
Shearer, C. B. (2005). Large scale factor analysis of the multiple intelligences developmental assessment scales. Paper presented
at the annual meeting of the American Educational Research Association, Montreal, Canada.
Ucak, E., Bag, H., & Usak, M. (2006). Enhancing learning through multiple intelligences in elementary science education. Journal
of Baltic Science Education, 10 (2), 61-69.
Uysal, E., & Eryilmaz, A. (2006). A study on self-evaluation of multiple intelligences dimensions in the seventh and tenth grade
students. Hacettepe University Journal of Education, 30, 230-239.
Taslidere, E., & Eryilmaz, A. (2012). Development of attitude scale towards simple electric circuits and assessment of students’
attitudes. Journal of Turkish Science Education, 9 (1), 31-46.
Teele, S. (1997). The multiple intelligences school. Redlands CA: Citrograph Printing.
Tirri, K., & Nokelainen, P. (2008). Identification of multiple intelligences with the multiple intelligences questioannaire 3. Psychol-
ogy Science Quarterly, 50 (2), 206-221.
Whitelegg, E., & Parry, M. (1999). Real-life contexts for learning physics: meanings, issues, and practice, Physics Education, 34, 68–72.
Zacharia Z. (2003). Beliefs, attitudes, and intentions of science teachers regarding the educational use of computer simulations
and inquiry-based experiments in physics. Journal of Research in Science Teaching, 40, 792–823.
Deniz Gurcay PhD., Associate Professor, Hacettepe University, Faculty of Education,

(Corresponding author) Department of SSME, Physics Education, Beytepe, Ankara, Turkey.
E-mail: denizg@hacettepe.edu.tr
Hatice Ozturk Ferah MSc., Doctoral Student, Hacettepe University, Faculty of Education,
Department of SSME, Physics Education, Beytepe, Ankara, Turkey.
E-mail: haticehtc@gmail.com
677
PHYSICS ANXIETY OF PRE-
SERVICE TEACHERS AND
THEIR SELF-EFFICACY BELIEFS:
DIFFERENCES ACCORDING
TO GENDER AND PHYSICS
ACHIEVEMENT
Abstract. The aim of this research is to de- Serap Çalışkan

termine the predictive level of physics self-
efficacy with regard to physics anxiety and
the correlations between the physics anxi-
ety and self-efficacy of pre-service teachers,
also to examine whether physics anxiety
and self-efficacy beliefs differ depending
on pre-service teachers’ achievement in
physics and their gender. The research was
conducted with 344 pre-service teachers
enrolled at a state university in Turkey. The Introduction
data obtained were analyzed using means
(M), standard deviation (SD), simple regres- Importance of affective dimension in teaching and learning process is
sion analysis, the one-way multivariate obvious. In this case, while learning can be affected by emotions and feelings,
analysis of variance (one-way MANOVA) emotions and feelings are also impacted by learning (Alsop &Watts, 2000).
and as a follow-up test, the univariate It is often said that learning and performance are associated with positive
analysis of variance (univariate ANOVA). feelings (Jerusalem & Pekrun, 1999). In this context, emotional factors and
The outcome of the research indicating the success are inseparably dependent upon each other and ultimately, those
physics anxiety of the pre-service teach- who are interested in the cognitive achievement of their students must also
ers was a significant predictor of their take emotional factors into consideration (Güngör, Eryılmaz & Fakıoğlu, 2007).
physics self-efficacy, significant differences Although cognitive skills are important determinants of student achievement,
between physics anxiety and self-efficacy in recent years affective variables have earned recognition as being striking
depending upon the level of achievement factors that have an impact on perseverance and achievement in the areas of
in the physics course, also showing that mathematics and science (Singh, Granville & Dika, 2002). Among the promi-
in general, female pre-service teachers nent affective variables defined in the literature can be cited students’ aca-
had significantly higher levels of anxiety demic self-concepts, attitudes, confidence, anxiety, motivation, self-esteem
compared to males and that with respect and locus of control (Higbee & Thomas, 1999). Similarly, in a study conducted
to self-efficacy, differences were in favor by Güngör, Eryılmaz and Fakıoğlu (2007), the researchers relate that the af-
of male pre-service teachers only in the fective variables that are suggested in studies on science and mathematics
dimension of problem-solving in physics. education as being the factors that are commonly employed and may have
Keywords: physics anxiety, physics a correlation with physics learning and physics achievement are attitudes
self-efficacy, pre-service teachers’ gender, toward physics (the interest in and importance placed on physics), student
physics achievement. motivation in the physics class (student motivation), achievement motivation
in physics class (achievement motivation), physics anxiety (anxiety about the
physics course and physics exams), physics self-efficacy, physics self-concept
(self-concept) and the locus of control in physics (locus of control). At the
Serap Çalışkan same time, it is possible to find literature on affective factors in the areas of
Dokuz Eylül University, Turkey science and mathematics but this type of study relating to physics education
is very few (Güngör et al., 2007). This is despite the fact that there are many
powerful stories of how many students find physics to be tasteless, boring
678
ISSN 1648–3898 /Print/ PHYSICS ANXIETY OF PRE-SERVICE TEACHERS AND THEIR SELF-EFFICACY BELIEFS:
ISSN 2538–7138 /Online/ (P. 678-693)
and stuffy (Alsop & Watts, 2000). In this sense, the importance of conducting research on examining the relationship
between learning physics and principal affective variables such as interest, love, placing importance, self-efficacy,
motivation and anxiety is very clear. Also, such research conducted with pre-service teachers may have a separate
significance for the teachers of tomorrow. This is because emotional growth is a developmental requirement for
teachers and is one of the key characteristics of effective teaching (Senler, 2016). In the light of this, physics anxiety
and physics self-efficacy affective variables were chosen in this research.
Anxiety and Physics Education
The term “anxiety” has existed for more than a century. Furthermore, the concept of “science anxiety”
was invented by Mallow in 1977 at Loyola University Chicago. The first Science Anxiety Clinic was founded at
Loyola University Chicago. Science anxiety has been defined as a widespread or indeterminate phobia in the sci-
ence learning setting (Mallow, 1978). The choices students make about the courses they take at the university or
in their future careers is generally related to how comfortable they feel in dealing with certain topics. Because of
the phenomenon that is well-known today as science anxiety, it is not rare for students to shy away from science
courses or perform poorly in their science studies (Udo, Ramsey & Mallow, 2004). Chiarelott and Czerniak (1987)
conducted a study with schoolchildren of the ages 9-14 in which they reported that science anxiety is associated
with science achievement, a high level of anxiety leads to low achievement in science, and girls have a higher
level of science anxiety than boys. It was also set forth that chi-square analysis showed that the higher level of
science anxiety found in girls in previous studies by Mallow (1994) and Udo, Ramsey, Reynolds-Alper & Mallow
(2001) displayed even greater differences. Like other negative emotions, anxiety is not only related to the concept
of learning physics but is the product of messages such as, “If I don’t pass this physics course, I will never be able
to graduate” that generate anxiety and lower performance in the physics class (Mallow & Greenburg, 1983). Com-
plaints of students about physics anxiety and low academic performance in physics class are frequently heard and
it is known that situations that suggest failure stimulate reactions of anxiety. In this context, the student who is
confronted with a threatening situation while studying for physics class may rightly feel anxious or uncomfortable.
Also, the lack of interest students with physics anxiety show towards the class may play an obstructive role and
lead to low achievement and to barriers such as thoughts of abandoning related career plans (Şahin, 2014). It is
thus obvious how important students’ anxiety levels are in terms of their physics courses. At the same time, while
there are numerous studies in the literature on anxiety concerning science and mathematics (Akman, İzgi, Bağçe,
& Akıllı, 2007; Akgün, Gönen, & Aydın, 2007; Beilock, Gunderson, Ramirez, & Levine, 2010; Brownlow, Jacobi, &
Rogers, 2000; Chiarelott & Czerniak, 1987; Czerniak & Chiarelott, 1985; Ho, Şentürk, Lam, Zimmer, Hong, Okamoto,
Chiu, Nakazawa, & Wang, 2000; Kuan & Tek, 2007; Mallow, 1994; Sharon & Corenna, 1990; Udo, Ramsey, & Mallow,
2004; Yüksel & Geban, 2016; Uluçınar Sağır, 2012; Uysal & Dede, 2016), there is also a study of note that explores
whether or not physics teaching causes science anxiety depending upon gender (Udo, Ramsey, Reynolds-Alpert, &
Mallow, 2001) and a study that seeks to determine the most appropriate structural equation model of the relation-
ship between affective variables and the physics achievement of freshmen high school physics students (Güngör
et al., 2007), where the 12 affective variables selected in the subscales of the affective characteristics scale include
physics exam anxiety and physics course anxiety. Furthermore, there is only a negligible number of studies on the
specific concept of physics anxiety in the literature (Cari, Suparmi, & Handhika, 2016; Dilek, Şahin, Güler, & Eslek,
2013; Şahin, 2014). Treating physics anxiety as a separate concept might contribute to gaining knowledge about
the complex interaction between the anxiety felt towards physics and achievement (Şahin, 2014).
Self-Efficacy and Physics Education
Bandura (1977) defines the belief individuals have in their ability to take the necessary action to cope
with a situation as self-efficacy belief. An individual’s expectation about his/her own success or failure affects that
individual’s perception of self-efficacy. An individual with a high perception of self-efficacy makes more of an effort
to succeed at something compared to an individual with a low perception of self-efficacy, acting with determina-
tion and perseverance. At the same time, an individual with a high perception of self-efficacy is less afraid of trying
something out, of bringing something to fruition (Senemoğlu, 2005). While students with high levels of self-efficacy
tend to have higher levels of achievement in a subject, those with low self-efficacy beliefs are likely to be less suc-
cessful (Stipek, 1998). Israel (2007) asserts that a person with a high sense of self-efficacy feels more self-confident
679
(P. 678-693) ISSN 2538–7138 /Online/
and that this self-confidence is reflected in his/her behavior. A student, for example, with a high perception of
self-efficacy will be an active participant in class, will take the time to study topics in physics and develop various
learning strategies. In this context, the most important factor that impacts academic achievement is the level of
self-efficacy (Tezer & Aşıksoy, 2015). Self-efficacy is a powerful predictor variable of achievement (Lane, Lane, &
Kyprianou, 2004; Schunk, 1991). According to Shaw, students in a class as well as the teacher coming into that class
bring along with them not only their prior thoughts about physics but also their prior thoughts on their ability to
be successful in the physics class (Shaw, 2004). In the light of this, the thoughts of students and teachers about
their own abilities point to their self-efficacy beliefs and in this sense, self-efficacy becomes an important variable
in physics teaching, and a topic that has produced an increasing number of studies in the literature on physics
and self-efficacy in recent years (Borrachero, Brigido, Costillo, Bermejo, & Mellado, 2013; Çalışkan, Selçuk, & Özcan,
2010; Fencl & Scheel, 2004; Maskan, 2010; Nissen & Shemwell, 2016; Selçuk, Çalışkan, & Erol, 2008; Tanel, 2013; Tezer
& Aşıksoy, 2015; Yener, Aydın, & Köklü, 2012; Yerdelen & Peşman, 2013). For example, in a study that examines the
variables of the present study in terms of the relationship between the physics self-efficacy of pre-service teachers
and gender and achievement, many significant differences were found in favor of male pre-service teachers and
pre-service teachers with high perceptions of achievement (Selçuk, Çalışkan, & Erol, 2008).
In the light of this information, the limitation in the literature is that these variables (anxiety, self-efficacy,
course achievement and gender) have not previously been examined together with respect to the physics course.

Aim of Research
The research was carried out to examine the predictive level of physics self-efficacy with regard to physics
anxiety and the correlations between these variables among pre-service teachers, and whether or not the physics
anxiety and self-efficacy beliefs of pre-service teachers displayed any differences in terms of achievement in the
physics course or depending upon gender. The research questions examined in the research were:
1) What is the predictive level of physics self-efficacy with regard to physics anxiety of pre-service teachers?
2) What is the correlations between physics anxiety and self-efficacy? 3) Does the physics anxiety of pre-service
teachers show significant differences in terms of the level of their achievement in the physics course? 4) Does the
physics self-efficacy of pre-service teachers show significant differences in terms of the level of their achievement
in the physics course? 5) Does the physics anxiety of pre-service teachers show significant differences in terms
of their gender? 6) Does the physics self-efficacy of pre-service teachers show significant differences in terms of
their gender?
General Background
The relational survey model was used in the research. This model determines the degree of covariance be-
tween two or more variables, expressing variance through symbolization using measurements or assigned values
in order to analyze the relationship between the variables (Karasar, 2000, p.81). In the present research, the vari-
ables of self-efficacy and anxiety were symbolized with the help of scales in an effort to provide a representation
of the interaction between these variables and academic achievement in physics and gender. The research was
conducted in the Spring term of the academic year 2015-2016. The researcher collected the data over a period of
approximately 3 weeks.
Research Sample
The purposeful sampling approach of criterion sampling was used in the research. In the criterion sam-
pling approach, units to be observed may comprise persons or objects having certain determined qualities. As
such, units that meet the determined sampling criteria are taken into the sample (Büyüköztürk, Çakmak, Akgün,
Karadeniz, & Demirel, 2012). In this research, the sampling criterion was the enrollment of the pre-service teachers
in the physics course that was a part of their department’s curriculum. Data was collected from 344 of the total
of 351 pre-service teachers who were taking General Physics or the course Physics II given in the first- or second-
year programs during the spring term during which the research took place at the faculty of education of a state
680
ISSN 2538–7138 /Online/ (P. 678-693)
university in Turkey, where the researcher worked as an instructor. Third- and fourth-year pre-service teachers who
had taken the physics course more than 1 or 2 years ago were not included in the research.
Instruments and Procedure
The Physics Anxiety Rating Scale (PARS) and the Physics Self-Efficacy Scale (PSES) were used as data col-
lection tools in the research. The implementation of the data collection tools took approximately 10 minutes. Both
of the data collection tools contained questions at the beginning about the student’s gender, the department the
student was studying in, and the grade received on the last or nearest physics test. Participation was voluntary in
the implementation of the scales and the pre-service teachers were asked not to write their names on the ques-
tionnaires. It was explained to the participants before they started to fill out either one of the questionnaires that
the data collected would be used entirely for scientific research and they were urged to answer each question
candidly.
Physics Anxiety Rating Scale (PARS)
Developed by Şahin, Çalışkan and Dilek (2015), PARS initially comprised 60 items, but in line with the
recommendations of an expert (a physics educator), 10 items were removed and revisions were made in two of the
items, resulting in a trial questionnaire of 50 items in the form of a 5-item Likert-type scale (from 1: I definitely do
not agree to 5: I definitely agree). The scale was implemented with 495 pre-service teachers and the data obtained
were analyzed for validity and reliability. To examine the factor structure of PARS, an exploratory factor analysis with
Varimax rotation was carried out. Eliminating factor cross loadings and factors with loadings of less than 0.40 led
to the exclusion of 18 items from the analysis. Using the factors with eigen values greater than 1 and the scree plot
test, four factors were formed that included 32 items. A 4-factor model was decided on as the best factor structure
for PARS. The final form of the scale has a maximum point of 160, and a minimum of 32. The higher the score, the
higher the level of anxiety. Descriptions and sample items concerning PARS subscales are given in Table 1.
Table 1. Descriptions and sample items concerning PARS subscales.
Subscales Descriptions Sample Items
Physics course/test anxiety (C/TA) Being anxious in physics class or when studying for *Among all the other courses, the course which
a physics exam makes me most anxious is physics.
*I am usually stressed out before a physics exam.
Anxiety about lack of physics knowl- Anxiety about not being able to explain physics con- *Helping a primary school student with her/his phys-
edge (LPKA) cepts to others or about displaying physics content ics project tenses me up.
knowledge to others. *If my instructor asked me to explain a physical
event from daily life, I would be worried.
Mathematics anxiety (MA) Anxiety about having a shortage of the mathemati- *I worry about not being able to remember the
cal knowledge that is needed to be successful in mathematical formulas of physics laws.
the physics course, not being able to remember the *When I open a physics book, seeing a page full of
mathematical relationships between physical vari- formulas without any explanation scares me.
ables, and having to study from a physics textbook
that is full of mathematical formulas.
Physics laboratory anxiety (PLA) Anxiety about performing a physics experiment in *Explaining the findings of an experiment that I con-
the laboratory ducted in the physics lab to the instructor stresses
me out.
*Physics experiments make me very tense.
In Table 2, the number of items for each subscale, calculated using Cronbach’s Alpha reliability coefficients,
are presented.
681
(P. 678-693) ISSN 2538–7138 /Online/
Table 2. Results of the reliability calculations concerning PARS.
Subscales Number of items Cronbach’s Alpha Coefficient
C/TA 9 .92
LPKA 8 .85
MA 8 .86
PLA 7 .83
PARS 32 .95
Physics Self-Efficacy Scale (PSES)
The PSES was developed by the researcher (Çalışkan, 2007) in order to determine the physics self-efficacy
beliefs of pre-service teachers. This scale, containing 5-choice Likert type items with choices of “Strongly Agree”,
“Agree”, “Undecided”, “Disagree”, and “Strongly Disagree,” consists of a total of 24 items. To investigate the factor
structure of PSES, an exploratory factor analysis with Varimax rotation was carried out. Items in the scale are grouped
in 4 dimensions and explain 56.68% of total variability. The highest score that can be obtained from this scale is
120, and the lowest score is 24. Descriptions and sample items concerning PSES subscales are given in Table 3.
Table 3. Descriptions and sample items concerning PSES subscales.
Subscales Descriptions Sample Items
Problem solving self-efficacy belief Self-judgments about how successful the individual “I have every confidence that I can ultimately solve a
(PSSEB) can be in solving physics problems. physics problem, no matter how hard it may be.”
“I am certain that I can come up with the needed
formulas to solve a physics problem.”
Self-efficacy belief towards achieve- Self-judgments about how successful the individual “I believe that I can get a 70 or better in the physics
ment (SEBTA) can be in the physics class. tests.”
“I don’t believe that I can be successful in physics
class.”
Self-efficacy belief in applying phys- Self-judgments about how successful the individual “I am sure I can write up a simple problem about a
ics knowledge (SEBAPK) can be in applying physics knowledge to different subject I’ve learned in physics.”
situations. “I believe I can effectively explain something I’ve
learned in physics class to my friend.”
Self-efficacy belief in recalling phys- Self-judgments about how successful the individual “I believe I can remember the important formulas I
ics knowledge (SEBRPK) can be in remembering physics knowledge when it learned in physics when I need to.”
is needed. “I believe I can remember the basic knowledge I
gained in physics class when I need to.”
In Table 4, the number of items for each subscale, calculated using Cronbach’s Alpha reliability coefficients,
are presented.
Table 4. Results of the reliability calculations concerning PSES.
Subscales Number of items Cronbach’s Alpha Coefficient
PSSEB 10 .91
SEBTA 5 .84
SEBAPK 6 .76
SEBRPK 3 .70
PSES 24 .94
682
ISSN 2538–7138 /Online/ (P. 678-693)
Data Analysis
In the event that the ANOVA results were significant, the post-hoc multiple comparison Bonferonni test was
used to find out which group the difference stemmed from. The independent variables in the study were the level
of achievement in physics and gender; the dependent variables were physics anxiety (C/TA, LPKA, MA, PLA) and
physics self-efficacy (PSSEB, SEBTA, SEBAPK, SEBRPK), and the control variable was the department the pre-service
teachers were studying in. Before examining each independent variable’s effect on the dependent variables, the
MANOVA hypotheses were tested. It was found that the scores related to the dependent variables showed univari-
ate and multivariate normal distribution. For each dependent variable, the assumption of homogeneity of variance
was tested with Levene’s test and Box’s M test was employed to determine homogeneity in variance-covariance
matrices. The effect size of the independent variables on the dependent variable scores was measured with the
partial eta-squared statistic (η2). The partial eta-squared values, according to Stevens’ (1992) recommendation, were
taken as small for η2 ≤ 0.01, medium for η2= 0.06 and large for η2= 0.14. The analysis of the data was performed at
a significance level of .05.
Results of the Research
Regression Analysis for Physics Anxiety and Self-Efficacy
To ascertain the relationship between the physics anxiety and physics self-efficacy of the pre-service teach-
ers, one of the two variables was set up as dependent, the other as an independent variable and the relationship
between them was examined using simple regression analysis and expressed as a mathematical equation. To
examine the predictive level of physics self-efficacy with regard to physics anxiety, the total physics anxiety and
self-efficacy scores were considered and, defining physics anxiety as the independent (X-variable) and physics
self-efficacy as the dependent variable (Y-variable), whether or not the pre-service teachers’ physics anxiety was
a significant predictor of their physics self-efficacy was explored, and in this context, simple regression analysis
was used to identify the relationship between the two variables. Before the analysis, a scatter diagram was drawn
up to understand whether there was a linear relationship between physics anxiety and physics self-efficacy and,
as can be seen from the figure below, it was seen that there was a linear relationship between the two variables.
125,00
100,00
SE_total
75,00
50,00
R Sq Linear = 0,234
25,00
50,00 75,00 100,00 125,00 150,00
ANX_total
Figure 1: Scatter diagram and regression line for physics anxiety and physics self-efficacy.
After this step, linear regression analysis was performed on the physics anxiety and physics self-efficacy
variables.
683
(P. 678-693) ISSN 2538–7138 /Online/
Table 5. of simple regression analysis on predictability of physics self-efficacy according to physics anxiety.
Standard
Variable B β t p
DeviationB
Fixed 121.722 4.416 - 27.565 .0001*

Physics Anxiety -.480 .047 -.484 -10.227 .0001*
R=.484, R2=.234
F(1,342)=104.602 p<.001
*p<.001

A review of the analysis results in the table show that the physics anxiety of the pre-service teachers was a
significant predictor of their physics self-efficacy (R= .484, R2= .234, F (1,342)=104.602, p=.0001). It can be said that
physics anxiety accounted for 23% of the total variance with respect to physics self-efficacy. It can also be seen
from the results of the t test related to the significance of the regression coefficients that physics anxiety was a
significant predictor of physics self-efficacy. The results of the regression analysis showing the regression equation
or the mathematical model of predicted physics self-efficacy is given below: “Physics Self-efficacy=121.722-480
Physics Anxiety.”
Correlation between Physics Anxiety and Self-Efficacy
In order to examine the sub-dimensions of the correlation between the physics anxiety and self-efficacy of
the pre-service teachers, the Pearson product-moment correlation coefficients displaying the relationship between
the physics anxiety and self-efficacy subscale scores were calculated. The correlation coefficients were evaluated
as recommended by Büyüköztürk (2012), with absolute values of .70 - 1.00 signifying a high correlation, .70 - .30
signifying a moderate correlation and .30 - .00 signifying a low correlation.
Table 6. Level of correlation between the pre-service teachers’ physics anxiety and physics self-efficacy.
PSSEB SEBTA SEBAPK SEBRPK C/TA LPKA MA PLA
PSSEB 1
SEBTA .812** 1
SEBAPK .747** .696** 1
SEBRPK .767** .683** .672** 1
C/TA -.364** -.439** -.284** -.264** 1
LPKA -.378**
-.387 **
-.283 **
-.316 **
.522** 1
MA -.497** -.514** -.434** -.394** .618** .652** 1
PLA -.262** -.238** -.191** -.177** .439** .619** .529** 1
**p<.01

A review of the correlations of the subscales of physics anxiety and physics self-efficacy in Table 6 reveals a
negative and low level of correlation in all the subscales of physics laboratory anxiety and physics self-efficacy (r<
.30). However, it was found that outside of the laboratory anxiety dimension, all of the anxiety and self-efficacy
scales displayed a negative, moderate and significant correlation. The highest negative and significant correlations
in the table were between physics course achievement and self-efficacy with regard to solving physics problems
and mathematical anxiety (r= -.514, r= -.497, p<.01, respectively).
Physics Course Achievement and Physics Anxiety
To determine the level of the pre-service teachers’ physics course achievement, at the time they were filling out
the questionnaire, they were asked to write down the last grade they received on the mid-term. The physics course
achievement grades (PCAG) written down were classified as follows: PCAG<50: “unsuccessful” (US), 50<PCAG<70:
684
ISSN 2538–7138 /Online/ (P. 678-693)
“mediocre” (MS) and PCAG>70: “successful” (S). An answer was sought to the question, “Does the physics anxiety of
pre-service teachers show significant differences in accordance with the level of their achievement in the physics
course?” Table 7 shows the descriptive statistics for PSES subscales according to the pre-service teachers’ PCAG’s.
The mean values for the dependent variables measured in the study were compared with the one-way MANOVA,
with the physics course achievement level taken as the fixed variable. It was seen that the multivariate effect was
significant [Wilks’ Lambda (Λ)=.813, F(4,338)=9.238, p<.0001, η2=.10]. The differences between the variables in terms
of achievement level were determined with the one-way ANOVA. The results revealed significant differences in the
following dimensions: C/TA [F(2, 341)=21.985, p<.0001, η2=.11], LPKA [F(2, 341)=16.68, p<.0001, η2=.09], MA [F(2,
341)=31.05, p<.0001, η2=.15] and PLA [F(2, 341)=9.79, p<.0001, η2=.05]. While significant differences were found in
all the physics anxiety scales according to physics achievement levels, when the eta squared values related to the
effect of physics achievement were examined, it was seen that the effect was moderate in the course and knowl-
edge deficiency anxiety dimension and small in the laboratory anxiety dimension, but large in the mathematics
anxiety dimension.
Table 7. Descriptive statistics according to the pre-service teachers’ physics anxiety PCAG.
Scale PCAG N M SD
US 114 29.42 4.41

C/TA MS 204 27.38 4.44
S 26 23.46 2.93
US 114 21.51 4.92
LPKA MS 204 19,71 5.32
S 26 15.15 4.85
US 114 27.29 5.39
MA MS 204 23.61 6.27
S 26 17.65 6.73
US 114 20.89 4.29
PLA MS 204 20.47 4.42
S 26 16.81 3.32

Differences according to physics achievement levels were examined with the Bonferroni test. The results
of the Bonferroni test are presented in Table 8. According to the results of the analysis, the physics self-efficacy
of the pre-service teachers who were successful in the physics course in the PSSEB, SEBTA, SEBAPK and SEBRPK
dimensions was significantly higher than that of the pre-service teachers who were mediocre or unsuccessful.
Similarly, it was determined that there were significant differences between the physics anxiety of the mediocre
and unsuccessful pre-service teachers in the first three dimensions, that the anxiety of the mediocre pre-service
teachers was significantly lower, but that there were no significant differences between the physics anxiety of the
mediocre and unsuccessful pre-service teachers in the PLA dimension.
Table 8. Results of the Bonferroni test according to the physics achievement levels of pre-service teachers.
Physics Anxiety Level of

US MS S
Subscales Achievement
US .000* .000*
C/TA MS .000* .000*
S .000* .000*
US .009* .000*
LPKA MS .009* .000*
S .000* .000*
685
(P. 678-693) ISSN 2538–7138 /Online/
Physics Anxiety Level of

US MS S
US .000* .000*
MA MS .000* .000*
S .000* .000*
US 1.000 .000*
PLA MS 1.000 .000*
S .000* .000*
*p<.001
Physics Course Achievement and Physics Self-Efficacy
An answer was sought to the question, “Does the physics self-efficacy of pre-service teachers show significant
differences according to their gender?” Table 9 shows the descriptive statistics for PSES subscales according to the
pre-service teachers’ PCAG’s. The mean values for the dependent variables measured in the study were compared
with the one-way MANOVA, with the physics course achievement level taken as the fixed variable. It was seen that
the multivariate effect was highly significant [Wilks’ Lambda (Λ)=.548, F(4,338)=29.678, p<.0001, η2=.26]. The differ-
ences between the variables in terms of achievement level were determined with the one-way ANOVA. A review
of the results revealed that there were significant differences between the dimensions PSSEB [F(2, 341)=94.857,
p<.0001, η2=.36], SEBTA [F(2, 341)=131.503, p<.0001, η2=.43], SEBAPK [F(2, 341)=62.527, p<.0001, η2=.27] and SEBRPK
[F(2, 341)=54.852, p<.0001, η2=.24]. While significant differences were found in all the physics anxiety dimensions
according to physics achievement levels, when the eta squared values related to the effect of physics achievement
were examined, it was seen that the effect was significantly large in all the subscales.
Table 9. Descriptive Statistics according to the pre-service teachers’ physics self-efficacy PCAG’s.
Scale PCAG N M SD
PSSEB US 114 24.32 6.40

MS 204 32.38 6.05
S 26 39.81 6.54
SEBTA US 114 11.89 3.65
MS 204 17.45 3.40
S 26 21.38 2.56
SEBAPK US 114 8.46 2.39
MS 204 10.70 1.83
S 26 12.42 1.92
SEBRPK US 114 18.29 4.61
MS 204 22.09 3.15
S 26 25.15 3.54

Differences according to physics achievement levels were examined with the Bonferroni test. The results of
the Bonferroni test are presented in Table 10. According to the results of the analysis, the physics self-efficacy of the
pre-service teachers who were successful in the physics course in the PSSEB, SEBTA, SEBAPK and SEBRPK dimensions
were significantly higher than the pre-service teachers who were mediocre or unsuccessful. Similarly, there were
significant differences between the physics self-efficacy of the mediocre and unsuccessful pre-service teachers in
all the dimensions, and it was found that the pre-service teachers with mediocre grades were significantly higher
in self-efficacy than the unsuccessful pre-service teachers.
686
ISSN 2538–7138 /Online/ (P. 678-693)
Table 10. Results of the Bonferroni test according to the physics achievement levels of pre-service teachers.
Physics Self-Efficacy Level of

US MS S
US .000* .000*
PSSEB MS .000* .000*
S .000* .000*
US .009* .000*
SEBTA MS .009* .000*
S .000* .000*
US .000* .000*
SEBAPK MS .000* .000*
S .000* .000*
US .000* .000*
SEBPRK MS .000* .000*
S .000* .000*
*p<.001
Gender and Physics Anxiety
An answer was sought to the question, “Does the physics anxiety of pre-service teachers show significant
differences according to their gender?” Table 11 shows the descriptive statistics for PARS subscales according to
the pre-service teachers’ gender.
Table 11. Descriptive statistics related to the pre-service teachers’ physics anxiety according to their gender.
Female (n=250) Male (n=94)

Scales Subscales
M SD M SD
C/TA 28.04 4.50 27.00 4.77

LPKA 20.37 5.15 18.87 5.85
PARS
MA 24.88 6.46 23.03 6.58
PLA 20.87 4.19 18.92 4.70
The mean values related to physics anxiety were compared with the one-way MANOVA, with gender taken as
the fixed variable. It was seen that the multivariate effect (physics anxiety) was significantly low [Wilks’ Lambda (Λ)=
.961, F(4,339)=3.481, p= .008, η2= .04]. The differences between the variables according to gender were determined
with the one-way ANOVA. The results showed small-sized significant effects in the following dimensions: LPKA
[F(1, 342)=5.334, p= .022, η2= .02], MA [F(1, 342)=5.573, p= .019, η2= .02] and PLA [F(1, 342)=13.716, p= .000, η2=
.04]. Also, the anxiety scores of the female pre-service teachers were significantly higher than those of the male
pre-service teachers. At the same time, in the C/TA [F(1, 342)=3.576, p= .059, η2= .01] dimension, it was seen that
there were no significant differences between the pre-service teachers in terms of gender.
Gender and Physics Self-Efficacy
An answer was sought to the question, “Does the physics self-efficacy of pre-service teachers show significant
differences according to their gender?” Table 12 shows the descriptive statistics for PSES subscales according to
the pre-service teachers’ gender.
687
(P. 678-693) ISSN 2538–7138 /Online/
Table 12. Descriptive statistics related to the pre-service teachers’ physics self-efficacy according to their
gender.
Female (n=250) Male (n=94)

Scales Subscales
M SD M SD
PSSEB 29.73 6.99 31.71 9.25

SEBTA 15.61 4.43 16.69 4.82
PSES
SEBAPK 10.06 2.20 10.15 2.82
SEBPRK 21.10 3.97 20.95 4.99
The mean values related to physics anxiety and physics self-efficacy were compared with the one-way MANOVA,
with gender taken as the fixed variable. It was seen that the multivariate effect (physics self-efficacy) was signifi-
cantly moderate [Wilks’ Lambda (Λ)= .952, F(4,339)=4.272, p= .002, η2= .05]. The differences between the variables
according to gender were determined with the one-way ANOVA. An examination of the one-way ANOVA results
for physics self-efficacy showed that there were significant differences in the PSSEB dimension [F(1, 342)=4.566,
p= .033, η2= .01] and that the male pre-service teachers had a higher level of self-efficacy than their female coun-
terparts. An examination of the other dimensions showed that in all of the dimensions SEBTA [F(1, 342)=3.865, p=
.050, η2= .01]; SEBAPK [F(1, 342)= .087, p= .769, η2= .00] and SEBPRK [F(1, 342)= .080, p= .777, η2= .00], the physics
self-efficacy of the female pre-service teachers did not display significant differences according to gender.
Discussion

The research was carried out to examine the predictive level of physics self-efficacy with regard to physics
anxiety and the correlations between these variables among pre-service teachers, and whether or not the physics
anxiety and self-efficacy beliefs of pre-service teachers displayed any differences in terms of achievement in the
physics course or depending upon gender. An examination of the correlation between the pre-service teachers’
physics anxiety and their self-efficacy showed that physics anxiety was a significant predictor of physics self-efficacy.
Although a negative and low correlation was seen in all of the subscales of physics laboratory anxiety and physics
self-efficacy, in general, the level of correlation between the dimensions of anxiety and self-efficacy was moderate,
negative and significant in all dimensions except laboratory anxiety. The highest negative and significant relation-
ship in the table was between mathematics anxiety and achievement in the physics course and self-efficacy with
regard to solving physics problems. This outcome is an expected one and is supported in the literature by numer-
ous studies that report, similar to the present research, a negative correlation between self-efficacy and anxiety
(Cocoradã & Pãlãşan, 2014; Durndell & Haag, 2002; Efe, Efe & Yücel, 2016; Kurbanoğlu & Akın, 2012; Senler, 2016). On
the other hand, it may be emphasized that the results of the present research are at first in the literature on physics
education. The outcome revealed in the present research that indicated a high correlation between self-efficacy in
problem-solving and physics course achievement and mathematics anxiety may stem from the fact that students
generally regard mathematics as a course that is full of mathematical formulas and because mathematics is an
important element in problem-solving in physics. This perception is supported by the way that problem-solving
has been a fundamental part of teaching physics over the years, with the ensuing result that students believe that
they have only learned physics when they can solve problems (McDermott, 1993). In a research by Şahin (2014),
the researcher reported that the literature did not provide a definitive clue in this context but that a deficiency in
students’ perception of their mathematical knowledge and talent could lead to a fear of physics itself and to poor
achievement in the course. In this context, mathematics may play a role in students’ anxiety about the physics
course (Şahin, 2014). At the same time, the outcome that there is a negative and low correlation between physics
laboratory anxiety and physics self-efficacy may be related to the fact that the pre-service teachers are limited to
only 2 hours of laboratory time in their weekly schedule or may not have taken it at all.
It was seen that in all of the dimensions, the physics anxiety of the pre-service teachers who were successful
in the course was significantly lower than that of the pre-service teachers who were unsuccessful or mediocre. In
the C/TA, LPKA, MA dimensions, there were significant differences between the physics anxiety of the mediocre
and poor performers among the pre-service teachers and it was seen that the pre-service teachers who performed
688
ISSN 2538–7138 /Online/ (P. 678-693)
at a mediocre level had significantly lower levels of anxiety. No research was discovered in the literature in which
the relationship between physics anxiety and physic course achievement had been studied. In the sole research
by Şahin (2014), in which the relationship between students’ academic average in all courses and physics anxiety
was explored, it was reported that students with high or low academic achievement display significantly greater
anxiety than students who performed at a mediocre level. This indicates that a low level of anxiety may provide
motivation and increase achievement, but that a high level of anxiety may be a barrier to achievement. At the same
time, parallel to the results of the present research, there are studies in the literature that point to a general negative
relationship between anxiety and achievement (Awan, Azher, Anwar, & Naz, 2010; Griggs, Rimm-Kaufman, Merritt,
& Patton, 2013; El-Anzi & Owayed, 2005; Karimi & Venkatesan, 2009; Ma, 1999; Sherman & Wither, 2003; Stankov,
2010; Woodard, 2004; Zakaria & Nordin, 2008; Zakaria, Zain, Ahmad, & Erlina, 2012). As Şahin (2014) has expressed,
hearing complaints from students about their physics anxiety and their low level of achievement in physics class
is a common event and situations that endanger success stimulate a reaction of anxiety. In this case, there are two
things that can be done. In other words, either the pre-service teachers should try to increase their perception of
physics self-efficacy or find ways of reducing their physics anxiety. Firstly, it is up to the teacher/instructor to refrain
from creating the perception that the course is difficult, too complex or hard to pass. In this context, Woodard’s
(2004) suggestions about how to reduce anxiety in mathematics class may be adapted to the physics class since
mathematics may play a role in university students’ physics anxiety in the physics courses they take (Şahin, 2014).
When physics students do not feel threatened and are presented with a comfortable class setting, cooperative
group work can be undertaken. In this type of group work, students see that others are experiencing the same
problems and begin to believe that with a bit of determination and help, they will indeed be able to succeed in
physics. At the same time, if students are given a second chance at a test or exam, they will feel that they have not
altogether lost and will continue to keep trying. This way, the students will feel that the instructor is paying atten-
tion to them and is working to have them succeed. According to Woodard (2004), a decelerated teaching plan is
useful in helping students to better understand the material.
It was found in this research that there were significant differences in all of the dimensions of physics self-
efficacy according to the level of achievement in physics and that the physics self-efficacy of pre-service teachers
who performed well in the physics course was higher than that of their counterparts who were unsuccessful or
performed at a mediocre level. It was also seen that pre-service teachers who performed at a mediocre level had
higher levels of self-efficacy compared to pre-service teachers who performed poorly. This is an expected result and
in this context, it is consistent with the results of studies that demonstrated that self-efficacy is a strong predictor
of academic achievement (Lane, Lane, & Kyprianou 2004; Schunk 1991) and that it is in strong correlation with
achievement (Metallidou, & Vlachou, 2007; Witt-Rose, 2003). There are also a few studies in the literature on physics
education that show that as the academic achievement of physics pre-service teachers increase, their self-efficacy
will also be high (Çalışkan, Selçuk, & Özcan, 2010) as well as those that reveal that there is a correlation between
the physics self-efficacy of pre-service teachers and their perception of physics achievement (Selçuk, Çalışkan, &
Erol, 2008). On the other hand, contrary to the results of this research, Shaw (2004) reports that there is no signifi-
cant relationship between the physics course grades of university physics students and their physics self-efficacy.
Additionally, while Abak, Eryılmaz and Fakıoğlu (2002) assert that there is no significant relationship between the
physics achievement of university first-year students and their perception of physics self-efficacy, in general, the
literature points to a positive relationship between self-efficacy and achievement, emphasizing that the percep-
tion of self-efficacy affects academic performance through its impact on countless behavioral and psychological
processes (Bandura, 1986). It is also reported that the mastery experiences of students whose self-efficacy is a basic
source of their achievement in science are particularly impacted (Britner & Pajares, 2006).
A review of the findings on the effects of gender on the anxiety and self-efficacy of pre-service teachers
reveals that in all the dimensions outside of C/TA, where there is no significant difference between the genders in
terms of physics anxiety, female pre-service teachers display a higher level of physics anxiety compared to their
male counterparts. Similarly, Kuan and Tek (2007) have examined the relationship between science anxiety and
gender, discovering that outside of physics class anxiety, the Science Anxiety Scale showed that in all of the other
dimensions, defined as danger anxiety, science test anxiety, mathematics or science problem-solving anxiety,
squeamish anxiety and performance anxiety, females were significantly more anxious than males; Hensley (1996),
too, reached the conclusion that the level of science anxiety was higher among females than males. Chiarelott and
Czerniak (1987) also asserted in a research they conducted with 9-14 year-old schoolchildren that girls experienced
a higher level of science anxiety than boys. Similarly, Mallow (1994) and Udo, Ramsey, Reynolds-Alper (2001) report
689
(P. 678-693) ISSN 2538–7138 /Online/
that the conclusion they had drawn in their previous studies that science anxiety was at a higher level in girls com-
pared to boys was even more pronounced in the results of the chi-square analysis. Zoller and Ben-Chaim (1990)’s
research showed, similar to the results revealed in the C/TA dimension in the present research, that test anxiety
did not show a significant difference between the genders. It was observed that as far as physics self-efficacy was
concerned, only in the physics problem-solving dimension were male pre-service teachers on a higher level; in all
the other dimensions, there was no significant difference between the female and male pre-service teachers in
terms of their self-efficacy. In the literature, the suggestion of Junge and Dretzke (1995) that females’ self-efficacy
in work requiring quantitative skills is lower than males is supported by the present research, which reveals that in
physics, which requires quantitative work such as problem-solving, males had a higher level of self-efficacy than
the females in the research. Britner and Pajares (2006) however found in their research conducted with middle
school pupils that the self-efficacy of girls in science were stronger in the emotional arousal domain but that boys
were stronger in mastery experiences, a result set forth that is consistent with the present research’s conclusion
that science self-efficacy does not exhibit a difference according to gender. Similar to the results of the present
research, Griggs, Rimm-Kaufman, Merritt and Paton (2013) showed in their research that mathematics and science
self-efficacy levels did not display significant differences according to gender.
Conclusions and Implications

An examination of the results of the research and the discussion may lead to the conclusion that, based on the
higher negative correlation between physics self-efficacy in the dimensions of course achievement and problem-
solving and between physics anxiety in the dimension of mathematics anxiety compared to the other dimensions.
At the same time, the findings suggest that pre-service teachers with higher levels of self-efficacy and lower levels
of anxiety are more successful in the physics course. In this context then, it can be said that levels of anxiety and
self-efficacy impact achievement. For this reason, physics teachers or faculty instructors should recognize the im-
portance of reducing mathematics anxiety, especially in their physics classes. Moreover, methods and techniques
of relieving physics anxiety should be examined through experimental studies with special attention being paid
to this so far scantly explored aspect of physics education through more frequent and varied research.
The present research is the finding that, as is expressed frequently in the literature, female students have a
higher level of science anxiety than their male counterparts. The higher level of anxiety that females carry in the
science or physics class may be associated with whether they had a male or female science or physics teacher in
their school years, the methods used in the classes in teaching physics, the nature of physics or science, and the
fact that scientists dealing with physics throughout history have predominantly been males. Another conclusion of
the research is that when it is considered that males display a higher level of self-efficacy in quantitative work such
as problem-solving, it may be necessary, in order to increase the self-efficacy of female students and reduce their
anxiety, for teachers/instructors to come to class with different teaching methods and go beyond concentrating
on mathematical processes in physics, giving the students alternative measuring/evaluating methods (e.g., con-
cept maps, conceptual change texts, structured grids) so that students are more widely introduced to conceptual
physics and everyday life experiences about physics concepts.
Limitations
The author thinks that certain limitations to the present research must be expressed. Since the data for the
research were collected using scales developed to identify a situation at any given point in time, the common
changes in self-efficacy and anxiety, it may be useful to conduct further experimental research that looks into how
changes may be achieved with different teaching interventions. Furthermore, qualitative studies conducted at dif-
ferent educational levels may be conducted in order to make an in-depth research of the sources of self-efficacy
beliefs and anxieties and their causal relationships in the context of physics class.
References
Abak, A., Eryılmaz, A., & Fakıoğlu, T. (2002). Determining university students’ selected affective characteristics. V. National Science
and Mathematics Education Congress, Ankara, Turkey.
Akgün, A., Gönen, S., & Aydın, M. (2007). The investigation of anxiety levels of primary school science and mathematics teacher
students’ according to some variables. Electronic Journal of Social Sciences, 6 (20), 283-299.
690
ISSN 2538–7138 /Online/ (P. 678-693)
Akman, B., İzgi, Ü., Bağçe, H., & Akıllı, H. İ. (2007). The effect of elementary students’ attitude towards science on their levels of
test anxiety. Education and Science, 32 (146), 3-11.
Alsop, S., &Watts, M. (2000). Facts and feelings: Exploring the affective domain in the learning of physics. Physics Education,
35,132-138.
Awan, R. N., Azher, M., Anwar, N. M., & Naz, A. (2010). An investigation of foreign language classroom anxiety and its relationship
with students’ achievement. Journal of College Teaching & Learning, 7 (11), 33-40.
Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84, 191-215.
Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice Hall.
Beilock, S. L., Gunderson, E. A., Ramirez, G., & Levine, S. C. (2010). Female teachers’ math anxiety affects girls’ math achievement.
PNAS, 107 (5), 1860-1863.
Britner, S. L., & Pajares, F. (2006). Sources of science self-efficacy beliefs of middle school students. Journal of Research in Science
Teaching, 43 (5), 485-499.
Brownlow, S., Jacobi, T., & Rogers, M. (2000). Science anxiety as a function of gender and experience. Sex Roles, 42 (1/2),119-131.
Borrachero, A. B., Brígido, M., Costillo, E., Bermejo, M. L., & Mellado, V. (2013). Relationship between self-efficacy beliefs and emotions
of future teachers of Physics in secondary education. Asia-Pacific Forum on Science Learning and Teaching, 14 (2), Article 5, 1.
Büyüköztürk, Ş. (2012). Data analysis manual for social sciences (16th ed.). Ankara: Pegem Academy Publishing Company.
Büyüköztürk, Ş., Çakmak, E. K., Akgün, Ö. E., Karadeniz, Ş., & Demirel, F. (2012). Scientific Research Methods (13th ed.). Ankara:
Pegem Academy Publishing Company.
Cari, C., Suparmi, A., & Handhika, J. (2016). Student’s preconception and anxiety when they solve multi representation concepts
in Newton laws and it’s application. 8th International Conference on Physics and its Applications (ICOPIA) (pp.1-5), Indonesia.
Chiarelott, L., & Czerniak, C. (1987). Science anxiety: Implications for science curriculum and teaching. The Clearing House, 60
(5), 202-205.
Cocoradã, E., & Pãlãşan, T. (2014). Computer anxiety and computer self-efficacy for the high school students. The 10 th International
Scientific Conference e-Learning and Software for Education (pp. 102-109), Bucharest.
Czerniak, C., & Chiarelott, L. (1985). Science anxiety among elementary school students: Equity issues. Journal of Educational
Equity and Leadership, 5 (4), 291-308.
Çalışkan, S. (2007). The effects of problem solving strategies teaching on the physics achievement, attitude, self-efficacy, and the
strategy usage. (Unpublished doctoral thesis), Dokuz Eylül University, Turkey.
Çalışkan, S., Selçuk, G. S., & Özcan, Ö. (2010). Self-efficacy beliefs of physics student teachers’: effects of gender, class level and
academic achievement. Kastamonu Education Journal, 18 (2), 449-466.
Dilek, U., Şahin, M., Guler, B., & Eslek, S. (2013). An investigation of university students’ anxiety in physics courses. 2nd World
Conference on New Trends in Science Education. Cracow, Poland.
Durndell, A., & Haag, Z. (2002). Computer self efficacy, computer anxiety, attitudes towards the Internet and reported experience
with the Internet, by gender, in an East European sample. Computers in Human Behavior, 18, 521-535.
Efe, H. A., Efe, R., & Yücel, S. (2016). A comparison of Swiss and Turkish pre-service science teachers’ attitudes, anxiety and
self-efficacy regarding educational technology. Universal Journal of Educational Research, 4, 1583 - 1594. doi: 10.13189/
ujer.2016.040711.
El-Anzi, & Owayed, F. (2005). Academic achievement and its relationship with anxiety, self-esteem, optimism, and pessimism in
Kuwaiti students. Social Behavior and Personality: An International Journal, 33 (1), 95-104.
Fencl, H. S., & Scheel, K. R. (2004). Pedagogical approaches, contextual variables, and the development of student self-efficacy
in undergraduate physics courses. AIP Conference Proceedings, 720 (1), 173-176.
Griggs, M. S., Rimm-Kaufman, S. E., Merritt, E. G., & Paton, C.L. (2013). The Responsive Classroom approach and fifth grade stu-
dents’ math and science anxiety and self-efficacy. School Psychology Quarterly: The Official Journal Of The Division Of School
Psychology, American Psychological Association, 28 (4), 360-373.
Güngör, A., Eryılmaz, A., & Fakıoğlu, T. (2007). The relationship of freshmen’s physics achievement and their related affective
characteristics. Journal of Research in Science Teaching, 44 (8), 1036-1056.
Hensley, C. A. (1996). Analysis of the interrelationships between science anxiety and the variables of gender, course of study, parent back-
ground, achievement, test anxiety, and attitude toward science (Unpublished doctoral thesis), The Ohio State University, USA.
Higbee, J. L., & Thomas, P. V. (1999). Affective and cognitive factors related to mathematics achievement. Journal of Developmental
Education, 23 (1), 8–10.
Ho, H-Z., Senturk, D., Lam, A. G., Zimmer, J. M., Hong, S., Okamoto, Y., Chiu, S-Y., Nakazawa, Y., & Wang, C-P. (2000). The affective and
cognitive dimensions of math anxiety: A cross-national study. Journal for Research in Mathematics Education, 31 (3), 362-379.
Israel, E. (2007). Self-regulated education, science achievement and self-efficacy (Unpublished doctoral thesis), Dokuz Eylül Uni-
versity, Turkey.
Jerusalem, M., & Pekrun, R. (Eds.). (1999). Emotion, motivation und leistung. Gottingen: Hogrefe.
Junge, M. E., & Dretzke, B. J. (1995). Mathematical self-efficacy gender differences in gifted/talented adolescents. Gifted Child
Quarterly, 39, 22–28.
Karasar, N. (2000). Scientific research methods (10th Ed.). Ankara: Nobel Publication.
Karimi, A., & Venkatesen, S. (2009). Mathematics anxiety, mathematics performance and academic hardiness in high school
students. International Journal of Educational Sciences, 1 (1), 33-37.
Kuan, F. L., & Tek, O. E. (2007). Science anxiety among form four students in Penang: A gender comparison. Journal of Science and
Mathematics Education in Southeast Asia, 30 (1), 84-109.
691
(P. 678-693) ISSN 2538–7138 /Online/
Kurbanoğlu, N. İ., & Akın, A. (2012). The relationships between university students’ organic chemistry anxiety, chemistry attitudes,
and self-efficacy: A structural equation model. Journal of Baltic Science Education, 11 (4), 347-356.
Lane, J., Lane, A. M., & Kyprianou, A. (2004). Self-efficacy, self-esteem and their impact on academic performance. Social Behavior
and Personality: An International Journal, 32 (3), 247-256.
Ma, X. (1999). A meta-analysis of the relationship between anxiety toward mathematics and achievement in mathematics. Journal
for Research in Mathematics Education, 30 (5), 520-540.
Mallow, J. V. (1978). A science anxiety program. American Journal of Physics, 46 (8)..
Mallow, J. V. (1994). Gender-related science anxiety: A first binational study. Journal of Science Education and Technology, 3, 227–238.
Mallow, J. V., & Greenburg, S. L. (1983). Science anxiety and science learning. The Physics Teacher, 21 (2), 95-99.doi:10.1119/1.2341214.
Maskan, A. (2010). An evaluation of prospective physics and maths teachers’ self-efficacy beliefs toward physics. Journal of Uludag
University Faculty of Education, XXIII (1), 31-42.
Metallidou, P., & Vlachou, A. (2007). Motivational beliefs, cognitive engagement, and achievement in language and mathematics
in elementary school children. International Journal of Psychology, 42 (1), 2-15.
Nissen, J. M., & Shemwell, J. T. (2016). Gender, experience, and self-efficacy in introductory physics. Physical Review Physics Educa-
tion Research, 12 (2), 020105-1-020105-16.
Schunk, D. (1991). Self-efficacy and academic motivation. Educational Psychologist, 26 (3 & 4), 207-231.
Selçuk, G. S., Çalışkan, S., & Erol, M. (2008). Physics self-efficacy beliefs of student teachers’: The relationships with gender and
achievement perception, Balkan Physics Letters (Special Issue: Turkish Physical Society 24th International Physics Congress),
648-651.
Senemoğlu, N. (2005). Development learning and from teaching-theory to application. Ankara: Gazi Kitabevi.
Senler, B. (2016). Pre-service science teachers’ self-efficacy: The role of attitude, anxiety and locus of control. Australian Journal
of Education, 60 (1) 26–41.
Sharon, W., & Corenna, C. (1990). Science anxiety: Relation with gender, year in chemistry class, achievement, and test anxiety.
Paper presented at the Annual Meeting of the Midwestern Educational Research Association, Chicago.
Shaw, K. A. (2004). The development of a physics self-efficacy instrument for use in the introductory classroom. AIP Conference
Proceedings, 720 (1), 137-140.
Sherman, B. F., & Wither, D. P. (2003). Mathematics anxiety and mathematics achievement. Mathematics Education Research
Journal, 15 (2), 138-150.
Singh, K., Granville, M., & Dika, S. (2002). Mathematics and science achievement: Effects of motivation, interest, and academic
engagement. Journal of Educational Research, 95, 323-332. http://dx.doi.org/10.1080/00220670209596607.
Stankov, L. (2010). Unforgiving Confucian culture: A breeding ground for high academic achievement, test anxiety and self-doubt.
Learning and Individual Differences, 20, 555-563.
Stevens, J. C. (1992). Applied multivariate statistics for the social sciences. Hillsdale, NJ: Lawrence Erlbaum.
Stipek, D. (1998). Motivation to learn from theory to practice (3rd ed.). USA: Allyn and Bacon.
Şahin, M. (2014). The relationship between pre-service teachers’ physics anxiety and demographic variables. Journal of Baltic
Science Education,13 (2), 201-215.
Şahin, M., Çalışkan, S., & Dilek, U. (2015). Development and validation of the physics anxiety rating scale. International Journal of
Environmental and Science Education, 10 (2), 183-200.
Tanel, R. (2013). Prospective physics teachers’ self-efficacy beliefs about teaching and conceptual understandings for the subjects
of force and motion. Journal of Baltic Science Education,12 (1), 6-20.
Tezer, M., & Aşıksoy, G. Y. (2015). Engineering students’ self-efficacy related to physics learning. Journal of Baltic Science Education,
14 (3), 311-326.
Udo, M. K., Ramsey, G. P., Reynolds-Alpert, S., & Mallow, J. V. (2001). Does physics teaching affect gender-based science anxiety?
Journal of Science Education and Technology, 10, 237–247.
Udo, M. K., Ramsey, G. P., & Mallow, J. V. (2004). Science anxiety and gender in students taking
general education science courses. Journal of Science Education and Technology,13 (4), 435-446.
Uluçınar, S. Ş. (2012). The primary school students’ attitude and anxiety towards science. Journal of Baltic Science Education, 11
(2), 127-140.
Uysal, F., & Dede, Y. (2016). Mathematics anxiety and beliefs of Turkish pre-service elementary teachers. Eurasia Journal of Mat-
hematics, Science & Technology Education, 12 (8), 2171–2186.
Uzuntiryaki, E. (2008). Exploring the sources of Turkish pre-service chemistry teachers’chemistry self-efficacy beliefs. Australian
Journal of Teacher Education, 33 (6), 12-28.
Witt-Rose, D. L. (2003). Student self-efficacy in college science: An investigation of gender, age, and academic achievement. Publica-
tion Manual of the American Psychological Association, 5th edition.
Woodard, T. (2004). The effects of math anxiety on post-secondary developmental students as related to achievement, gender,
and age. Inquiry, 9 (1), 1-5.
Yener, D., Aydın, F., & Köklü, N. (2012). Examination of students’ self-efficacy who were applied animation-simulation in physics
laboratory. 10th International Congress on Science and Mathematics Education, Nigde University, Nigde.
Yerdelen, S. D., & Peşman, H. (2013). Relations of gender and socioeconomic status to physics through metacognition and self-
efficacy. The Journal of Educational Research, 106 (4), 280-289. doi: 10.1080/00220671.2012.692729.
Yüksel, M. & Geban, Ö. (2016). Examination of science and math course achievements of vocational high school students in the
scope of self-efficacy and anxiety. Journal of Education and Training Studies, 4 (1), 88-100.
692
ISSN 2538–7138 /Online/ (P. 678-693)
Zakaria, E., & Nordin, N.M. (2008). The effects of mathematics anxiety on matriculation students as related to motivation and
achievement. Eurasia Journal of Mathematics, Science and Technology Education, 4 (1), 27-30.
Zakaria, E., Zain, N. M., Ahmad, N. A., & Erlina, E. (2012). Mathematics anxiety and achievement among secondary school students.
American Journal of Applied Sciences, 9 (11), 1828-1832.
Zoller, U., & Ben-Chaim, D. (1990). Gender differences in examination-type preferences, test anxiety, and academic achievements
in college science education a case study. Science Education, 74 (6), 597-608.
Received: July 02, 2017 Accepted: August 25, 2017
Serap Çalışkan Assoc. Prof. Dr., Dokuz Eylül University, Faculty of Education,
Department of Mathematics and Science Education, Buca, 35150 Izmir,
Turkey.
E-mail: serapcaliskan77@gmail.com
693
INFORMATIONAL VALUE
OF SUBMICROSCOPIC
REPRESENTATIONS IN
SLOVENIAN CHEMISTRY
TEXTBOOK SETS
Abstract. This research focuses on the Špela Hrast,

informational value of submicroscopic rep- Vesna Ferk Savec
resentations (SMRs) in chemistry textbook
sets for Slovenian primary schools (Grade
8 and Grade 9, students’ age 13-15 years).
For the purpose of analysis a holistic criteria
related to the accompanying descriptors
of SMRs in textbook sets was developed.
The research revealed four main holistic Introduction
descriptors accompanying SMRs, i.e.
direct descriptor (D), indirect descriptor (I), Learning in the information age is characterised by abundant prolif-
combined descriptor (C), and SMRs without eration of information from various sources that are communicated to the
learners by a variety of means, which are usually supported by information-
descriptors (W), which support learners’
communication technology (ICT), such as the internet, social media, virtual
recognition of SMRs’ informational value
classrooms, etc. However, the textbook sets continue to have the central role
on different levels by providing different in supporting effective teaching and learning of science as they should be,
accompanying add-ons of SMRs. In-depth by their intent, synchronised with national curriculum for specific subject at
analysis identified 14 main categories of certain educational levels. They are used for studying at school as well as at
the underlying representational types of home (Gkitzia, Salta & Tzougraki, 2011). Therefore, significant attention has
been paid to textbook analysis in science education. For example, Devetak,
SMR add-ons. The significance of the re-
Vogrinc, and Glažar (2010) studied explanations of states of matter in Slove-
search is in initiating a discussion about the
nian science textbooks; Majidi and Mäntylä (2011) studied the knowledge
holistic criteria for the description of SMRs organisation in magnetostatics in Finnish textbooks; Laçin-Şimşek (2011)
integrated in the textbook sets, because in studied female scientists in Turkish science and technology textbooks;
practice the learner perceives each SMR as Mumba, Chabalengula, and Hunter (2007) studied inquiry levels and skills
a whole. in Zambian high school chemistry textbooks.
The understanding of chemistry is based on creating mental images
Keywords: chemistry learning, chemis-
for corresponding molecular phenomena. The mental images are internal
try textbook sets, informational value,
representations that can be in chemistry visualised through special sym-
submicroscopic representations, textbook bolic systems, so-called external representations of the particulate nature
analysis. of matter, which are referred to as submicroscopic representations (SMRs)
in this research (Gilbert, 2005; Gilbert, Reiner & Nakhleh, 2008). Researchers,
starting with Johnstone (1991), pointed out that the representing of science
concepts and processes is based on representations of three levels: macro-
scopic (observable phenomena), submicroscopic or particulate (different
Špela Hrast, representations of atomic, molecular, and particle structure), and symbolic
Vesna Ferk Savec (mathematical and chemical symbols). Thereby, the interpretation of the
University of Ljubljana, Slovenia
phenomenon, perceived on a macroscopic level by the use submicroscopic
representations, is considered to be one of the fundamental ideas of con-
694
ISSN 1648–3898 /Print/ INFORMATIONAL VALUE OF SUBMICROSCOPIC REPRESENTATIONS IN SLOVENIAN
ISSN 2538–7138 /Online/ (P. 694-705)
temporary chemistry teaching (Eilks, 2013), because visualisations support students when connecting the three
levels of concept representations (Al-Balushi & Al-Hajri, 2014; Barke & Wirbs, 2002; Ferk Savec, Sajovic & Wissiak
Grm, 2009; Kelly & Jones, 2008).
It is often assumed that students comprehend and efficiently learn with the use of SMRs, because experienced
chemists (e.g. authors of textbook sets) can simultaneously use them as a part of a triple representation of chem-
istry concepts (submicroscopic, macroscopic and symbolic level), as defined by Johnstone (1991). However, the
abundant presence of SMRs in a textbook does not ensure efficient learning (Furió‐Más, Calatayud, Guisasola et al.,
2005; Gkitzia et al., 2011; Harrison, 2001). Researchers (Hinze, Willamson, Shultz et al., 2013; Stull, Gainer, Padalkar
et al., 2016; Stieff, Scopelitis, Lira et al., 2016) have found that in order to efficiently use the visualisations it is crucial
to develop students’ representational competence. According to Stieff et al. (2016), representational competence
comprises a distinct set of skills for constructing, selecting, interpreting, and using disciplinary representations
for communicating, learning, or problem solving. Research evidence (Ferk, Vrtacnik, Blejec et al., 2003; Kozma,
Chin, Russell et al., 2000; Stull, Hegarty, Dixon et al., 2012), indicates that students’ successful learning with SMRs
is significantly impacted by representational competence in chemistry. Kozma and Russell (2005) elaborated its
role in learning chemistry as in order to achieve expertise in the domain, students have to master a specific set of
skills, e.g. the ability to analyse features of a representation, transform one representation into another, generate
different representations, clarify the usefulness of a given representation, and explain the distinctive affordances
of different representations.
This research focuses on the descriptors of SMRs in textbook sets from the perspective of supporting learners
in the effective learning of chemistry. Thus far, several authors developed the criteria for evaluating of chemical rep-
resentations in school textbooks from various aspects (Hinze, Williamson, Deslongchamps et al., 2013; Nyachwaya &
Gillaspie, 2016). For example, Gkitzia et al. (2011) proposed five criteria (C1-C5): (C1) the type of the representation;
(C2) the interpretation of the surface features; (C3) the representation’s relationship to the text; (C4) the existence
and the properties of a caption; (C5) the degree of correlation between the components comprising a multiple
representation. Kapıcı and Savaşcı-Açıkalın (2015) used the rubric developed by Gkitzia et al. (2011) and focused
only on the particulate nature of matter by using three criteria (C1–C3): (C1) the type of the representation; (C2)
relatedness to text; (C3) properties of captions. All of the above criteria examined particular characteristics one
by one. Because in practice the learner perceives each SMR as one whole, there is a need to develop more holistic
criteria for the description of SMRs integrated into textbook sets, including various possible add-ons. The following
research questions (RQ) were stated:
1st RQ: What are the holistic descriptors of SMR add-ons in Slovenian chemistry textbook sets?
2nd RQ: What are the underlying representational types of holistic descriptors of SMR add-ons in Slovenian
chemistry textbook sets?
3rd RQ: What proportion of representations belong to a particular category of holistic descriptors of SMR
add-ons in Slovenian chemistry textbook sets?
4th RQ: What proportion of representations belong to a particular category of the underlying represen-
tational types of holistic descriptors of SMR add-ons in Slovenian chemistry textbook sets?
General Background
In Slovenia, students enter primary education at age 5-6 (Grade 1) and finish it by the age of 14-15 (Grade 9).
After primary school, they enter different secondary schools that last from two to four years. Primary education is
compulsory for all students in Slovenia.
There is a national curriculum that all teachers should implement in their classrooms; furthermore, textbooks’
approval by the National Commission for Textbook Approval at the Ministry of Education, Science and Sport de-
pends on following of the curriculum objectives. In other words this means, that various authors write textbooks in
accordance with national curriculum and these are submitted to the National Commission for Textbook Approval
for approval. In this research, we focused only on the chemistry textbooks in primary school (Grade 8 and Grade 9,
students’ age 13-15 years) that were approved by the National Commission for Textbook Approval at the Ministry
of Education.
695
(P. 694-705) ISSN 2538–7138 /Online/
Sample
The approved textbook sets for chemistry in Grades 8 and 9 of Slovenian primary schools in the 2016/17
school year were analysed. A list of analysed textbook sets is shown in Table 1.
Table 1. The list of the analysed textbook sets.
Year of
Number of
publication Grade/
Textbook set* Pages Special
Author(s) Publisher (Edition) Learner’s
title Textbook/ Section**
Textbook/ age
workbook
workbook
Gabrič, A., Glažar, S. A., Graunar, 2014 (1st Ed.)/

Kemija danes 1 DZS 125/106 No 8/13
M., Slatinek-Žigon, M. 2013 (1st Ed.)
Sajovic, I., Wissiak Grm, K., Godec,
Zavod RS za
Kemija 8, i-učbenik A.,Kralj, B., Smrdu, A., Vrtačnik, M., 2014 264 No 8/13
šolstvo
Glažar, S.
Vrtačnik, M., Wissiak Grm, K. S., 2015 (1st Ed.)/ 240/92,
Moja prva kemija Modrijan Yes 8, 9/13, 14
Glažar, S. A., Godec, A. 2014 (1st Ed.) 61
Devetak, I.,Cvirn Pavlin, T., Jamšek, 2010 (1st Ed.)/
Peti element 8 ROKUS KLETT 103/71 No 8/13
S. 2010 (1st Ed.)
2003 (1st Ed.)/
Pogled v kemijo 8 Kornhauser, A., Frazer, M. MK 140/126 No 8/13
2004 (1st Ed.)
Od atoma do 2012 (2nd Ed.)/
Smrdu, A. JUTRO 128/160 No 8/13
molekule 2012 (2nd Ed.)
Graunar, M., Podlipnik, M., Mirnik,
J. (textbook) 2016(1st Ed.)/
Kemija danes 2 DZS 152/96 No 9/14
Dolenc, D., Graunar, M., Modec, B. 2016 (1st Ed.)
(notebook)
Jamšek, S., Sajovic, I., Wissiak
Zavod RS za
Kemija 9, i-učbenik Grm, K., Godec, A., Boh, B., 2014 271 No 9/14
šolstvo
Vrtačnik, M., Glažar, S.
Devetak, I.,Cvirn Pavlin, T., 2011 (1st Ed.)/
Peti element 9 ROKUS Klett 77/ 79 No 9/14
Jamšek, S. 2011 (1st Ed.)
2005(1st Ed.)/
Pogled v kemijo 9 A. Kornhauser, M. Frazer MK 140/115 No 9/14
2006 (1st Ed.)
Od molekule do 2013 (2nd Ed.)/
Smrdu, A. Jutro 128/152 No 9/14
makromolekule 2013 (2nd Ed.)
The term “textbook set*” refers to all materials for students in the written or electronic form. The term “special section**” denotes a sec-
tion that intends to clarify notations and characteristics used in SMRs.
Development of the Criteria for the Analysis of SMRs in Textbook Sets
In order to derive holistic descriptors and the underlying representational types, two researchers separately
analysed the SMRs in the textbook sets of Table 1. In this manner, 283 pages (10% of all analysed textbook set
pages) were randomly selected and analysed to develop a rubric for the evaluation of SMRs in the textbook sets.
The development of the rubric was based on the recognition of different possible holistic descriptors regarding the
SMR add-ons (such as without, direct, indirect, combined) and, consequently, the categorisation of the underlying
representational types that are characterised by the occurrence of a compounds’ name and the triple-nature repre-
sentational level (submicroscopic, macroscopic, symbolic) of the SMR add-ons. The descriptions of the triple-nature
representational levels of the SMR add-ons were found to be presented in various modes, such as: (1) for the sub-
microscopic level of the SMR add-ons - pictorial or textual, (2) for the macroscopic level of the SMR add-ons - pictorial,
textual, and their integrated notation, and (3) for the symbolic level of the SMR add-ons - structural formula or other
symbolic notations and their integrated notation. The criteria and their description were then constructed based
696
ISSN 2538–7138 /Online/ (P. 694-705)
on these core characteristics. Each researcher independently grouped the characteristics into individual criteria.
Finally, to reduce bias issues, through discussion, reconstruction and agreement, both researchers came to the final
version of the rubric, which enabled a 95% inter-rater reliability about the categorisation of the analysed items.
The outcome of the whole analysis is the development of a fully-fledged typology specifically designed for SMRs
in textbook sets. The developed criteria were used in the analysis of the chemical representations of the entire
sample of chemistry textbook sets (Table 1).

Results of Research
The results of the analysis are presented with regard to the research questions stated, the subchapters are
named accordingly.
The Holistic Descriptors of SMR Add-ons in Slovenian Chemistry Textbook Sets
In the analysis, it was found that the informational value of SMRs for the learners differs with regard to various
SMR add-ons. To assign the occurrence of add-ons accompanying SMRs, four descriptors were proposed.
1st descriptor: direct SMR add-ons (D)
The direct SMR add-ons enable the learner a direct and unambiguous recognition of particles. Thereby, vari-
ous types of explanatory keys can be used. For example, pictorial, textual, integrated structural, or other symbolic
notations used in the explanatory key.
Table 2. Examples of direct SMR add-ons.
Pictorial SMR add-on Textual SMR add-on Integrated symbolic notation
2nd descriptor: indirect SMR add-ons (I)
Indirect SMR add-ons do not enable the learner a direct recognition of particles. The nature of the particles
can be derived e.g. based on the compound’s name, structure-properties relation or symbolic SMR add-ons, etc.
Table 3. Examples of indirect SMR add-ons.
Name and pictorial macroscopic SMR

Name Symbolic SMR add-on
add-on
697
(P. 694-705) ISSN 2538–7138 /Online/
3rd descriptor: combined SMR add-ons (C)
Combined SMR add-ons enable the learner a direct recognition and provide other information. It is a combi-
nation of the 1st and the 2nd descriptor.
Table 4. Examples of combined SMR add-ons.
Name, pictorial macroscopic and textual Symbolic SMR add-on and integrated
Name and pictorial SMR add-on
SMR add-on symbolic notation
4th descriptor: without SMR add-on (W)
The SMRs without add-ons.
Table 5. Example of SMR without add-on/s.
Without add-ons
The Underlying Representational Types of Holistic Descriptors of SMR Add-ons in Slovenian Chemistry Text-
book Sets
Analysis of textbooks revealed 1419 cases of SMRs, which could be categorised into 47 representational
types of SMR add-ons. However, further analysis revealed that 90% of cases (N=1272) belong to only 14 catego-
ries. Therefore, the remaining 33 representational types were ascribed to a group of combined representational
type representing 10% of all cases (N=147); consequently, there are 15 or fewer cases in each of the recognised
categories. A detailed description of representational types of SMR add-ons in analysed textbook sets is presented
in Table 6; additionally, the SMRs with add-on examples of each representational type are presented in Table 7.
698
Table 6. Description of SMR add-ons in analysed textbook sets.
Description
Submicroscopic level Macroscopic level Symbolic level

Representational
Descriptor
type
Name
Other
Textual
Textual
formula
Pictorial
Pictorial
symbolic
notations
Structural
Integrated
Integrated
I
Direct (D)
II

III
IV

V
Indirect (I) VI
VII
VIII
IX

X
XI
Combined (C)
XII
XIII
Direct or Indirect or Combined
XIV Group of combined representational type including 10% of all cases
(D,I,C)
Without (W) XV
(P. 694-705)
699
700
Table 7. Examples of cases of representational types of SMR add-ons.
Descriptor Examples of representational types

(P. 694-705)
Direct (D)
Representational type I
Representational type II
Indirect (I)
Representational type III Representational type IV Representational type V Representational type VI
Representational type VII Representational type VIII Representational type IX

Combined (C)
Representational type X Representational type XI Representational type XII Representational type XIII
Direct or Indirect
or Combined
(D,I,C)
Representational type XIV Representational type XIV Representational type XIV

Without (W)
Representational type XV
ISSN 2538–7138 /Online/ (P. 694-705)
Proportion of Representations in a Particular Category of Holistic Descriptors of SMR Add-ons

in Slovenian Chemistry Textbook Sets
As can be seen from Table 8, the textbook set analysis revealed that students’ learning of chemistry with SMRs
is rather rarely (14.80% of cases) facilitated by implementation of SMRs accompanied with a direct add-on descriptor
- D (4.44%) or its combination with indirect add-on descriptors - C (10.36%); the details about the categorisation are
elaborated in the 1st RQ. With the use of such descriptors of SMR add-ons, a direct and unambiguous recognition
of particles is enabled for learners.
It was found that the majority (55.67%) of SMRs can be categorised as SMRs with an indirect add-on descriptor
- I. In these cases, the recognition of the informational value of a particular SMR is supported by SMR add-ons and
the learners’ previous experience, knowledge, and their representational competence.
Surprisingly, it was found that as many as one third of SMRs (29.53%) do not include any information that
would support learners’ recognition of SMRs - no descriptors attached to the SMR - W. In these cases, the recognition
of the informational value of particular SMRs depends entirely on learners’ previous experience, knowledge, and
their representational competence.
Table 8. Proportion of SMRs with add-ons descriptors.
General
Descriptor
N f (%)
Direct (D) 63 4.44

Indirect (I) 790 55.67
Combined (C) 147 10.36
Without (W) 419 29.53
SUM 1419 100.00
Proportion of Representations in a Particular Category of the Underlying Representational Types

of Holistic Descriptors of SMR Add-ons in Slovenian Chemistry Textbook Sets
As described earlier, the textbook set analysis revealed 1419 cases of SMRs that were categorised into a total
of 47 representational types of SMR add-on/s; 90% of cases (N=1272) belong to only 14 categories. The details
about the categorisation are elaborated in the 2nd RQ.
From Table 9, it can be derived that the most commonly occurring representational type of holistic descriptors
of SMR add-ons in Slovenian chemistry textbook sets is the representational type XV (29.53%). SMR cases categorised
in the representational type XV do not have any descriptors attached to the SMR; therefore, that representational
is categorised as the descriptor without SMR add-ons (W).
The second most frequently used SMR representational type in the analysed textbook sets is representational
type III (16.35%), followed by representational type IV (16.77%). Both these representational types are categorised
as the descriptors of indirect SMR add-ons (I) with the name of the compound attached to the SMR, whereby rep-
resentational type IV also includes a further indirect SMR add-on, i.e., the rational formula of a compound.
The textbook set analysis revealed that approximately two thirds (62.65%) of the SMRs belong to only three
different representational types; however, from the SMRs categorised in the remaining third (37.35%), a variety
of SMR add-on combinations could be observed, including 33 different combinations in the combined represen-
tational type XIV.
701
(P. 694-705) ISSN 2538–7138 /Online/
Table 9. Proportion of SMRs with add-ons representational types.
General
Descriptor Representational type
N f (%)
Direct (D) I 24 1.69
II 24 1.69
Indirect (I) III 232 16.35
IV 238 16.77
V 83 5.85
VI 72 5.07
VII 27 1.90
VIII 25 1.76
IX 21 1.48
Combined (C) X 51 3.59
XI 23 1.62
XII 17 1.20
XIII 17 1.20
Direct or Indirect or Combined (D,I,C) XIV 146 10.29
Without (W) XV 419 29.53
SUM 1419 100.00
Discussion
Textbooks have the central role in the learning of science and can be used for studying at school as well as at
home (Gkitzia et al., 2011). Hinze, Rapp, Williamson et al. (2013) suggests that understanding the kinds of informa-
tion and inferences that the visualisations in various learning materials provide requires explicit instruction and
practice. Consequently, researchers have recently paid more attention to examining the importance of the possible
features and notations of visualisations for the learners with eye-trackers, e.g. Williamson, Hegarty, Deslongchamps
et al. (2013) studied students’ use of ball-and-stick images versus electrostatic potential maps when considering
electron density, positive charge, proton attack, and hydroxide attack; O’Keefe, Letourneau, Homer et al. (2014)
examined how the integration of multiple representations was associated with learning in a multimedia simulation;
Ferk Savec, Hrast, Devetak et al. (2016) examined some of the features of the explanatory key, such as coloured
versus black-and-white explanatory key, and pictorial versus textual explanatory key, etc.
Previous research proposed various criteria for evaluating of representations in school textbooks, e.g. the
five criteria (C1-the type of the representation; C2-the interpretation of the surface features; C3-their relationship
to the text; C4-the existence and the properties of a caption; C5-the degree of correlation between the compo-
nents comprising a multiple representation proposed) of Gkitzia et al. (2011); or three criteria (C1-the type of the
representation; C2-relatedness to text; C3-properties of captions) of Kapıcı and Savaşcı-Açıkalın (2015), thereby
examining only the particulate nature of matter.
The presented research focuses on studying the presence of descriptors of SMRs in textbook sets to support
efficient learning and, through that, to be acquainted with learners’ opportunities regarding textbook sets. As
previous research considered particular characteristics of different features of representations one by one, the
most valuable added value of this research is in recognising the need to develop more holistic criteria for the
description of SMRs integrated into textbook sets, as in practice the learner perceives each SMR as one whole.
702
ISSN 2538–7138 /Online/ (P. 694-705)
The novelty of the presented research is based on that, the analysis of the textbook sets that revealed four main
holistic descriptors accompanying SMRs, i.e. direct descriptor (D), indirect descriptor (I), combined descriptor (C)
and SMRs without descriptors (W). These descriptors support learners’ recognition of SMRs’ informational value to
a different level; namely, they provide different accompanying add-ons in relation to SMRs. It was found that more
than half of SMRs can be categorised as SMRs with an indirect add-on descriptor (I), with which the learner needs
to rely on his/her own previous experience, knowledge, and representational competence when recognising the
informational value of the particular SMR. In contrast, it was found that as much as one third of SMRs does not
include any information that would support learners’ recognition of SMRs, as no descriptors are attached to the SMR
(W). Further analysis of textbooks revealed 1419 cases of SMRs that could be categorised into 47 representational
types of SMR add-ons. It was found that 90% of cases belong to only 14 categories of underlying representational
types of holistic descriptors of SMR add-ons; each of categories includes more than 15 cases of SMRs (more than
1% of SMRs in each of the categories).
It would be valuable if the future research could examine the usefulness of the SMRs in the particular 14 cat-
egories postulated in the presented research from the learner’s perspective, e.g. with the use of an eye tracker. The
results of such research could then serve as a foundation for future development of the textbook sets and other
learning materials in order to support students’ learning. It would also be valuable to further examine the types
of supports and practice that are most helpful for students at different levels when learning with SMRs. It might
be interesting to focus especially on fostering representational competence, e.g. by asking students to choose or
construct representations that will be most useful for reasoning about particular chemical problems or relation-
ships, as suggested by Kozma and Russell (1997). It would also be interesting to study the order in which certain
representational types of holistic descriptors of SMR add-ons first appear in textbook sets, and their placement
in the learning process, e.g. whether SMRs appear in the textbooks before being integrated into homework and
exams (Sanger, 2000).
Conclusions
In this research, chemistry textbook sets for Slovenian primary schools were examined from the perspec-
tive of the information value of SMRs. The added value of this research from the international perspective, is in
recognising the need to develop more holistic criteria for the description of SMRs integrated into textbook sets,
as in practice the learner perceives each SMR as one whole. From this point of view, the novelty of the presented
research is also the analysis of the textbook sets, that revealed four main holistic descriptors accompanying SMRs,
i.e. direct descriptor (D), indirect descriptor (I), combined descriptor (C) and SMRs without descriptors (W). These
descriptors support learners’ recognition of SMRs’ informational value on a different level through providing dif-
ferent accompanying add-ons of SMRs. It was found that as much as one third of SMRs do not include descriptors
attached to the SMR. In these cases, the recognition of the informational value of a specific SMR depends entirely
on learners’ previous experience, knowledge, and their representational competence. The majority of SMRs can
be categorised as SMRs with an indirect add-on descriptor, such as name, pictorial macroscopic SMR add-on, sym-
bolic SMR add-on, etc. The recognition of the informational value of a particular SMR in such cases is supported
by SMR add-ons as well as with learners’ previous experience, knowledge, and their representational competence.
The analysis also revealed that approximately two thirds of the SMRs belong to only three different underlying
representational types of SMR add-ons.
It would be valuable to examine the usefulness of the most commonly used categories of SMRs from the
learner’s perspective, by using an eye tracker. The results of such research could then serve as a foundation for
future development of the textbook sets and other learning materials in order to support students’ learning.
Acknowledgements
This work was supported by the Faculty of Education University of Ljubljana, project – framework “Interni
razpis za financiranje raziskovalnih in umetniških projektov 2015/16 / An internal call for funding for research
and art projects 2015/16/”, project title “Pojasnjevanje uspešnosti reševanja kemijskih nalog na submikro ravni ter
preučevanje kompetentnosti bodočih učiteljev kemije za njihovo poučevanje / Clarifying the success of solving
chemical tasks at the submicro level and studying the competence of future chemistry teachers for their teaching/”.
703
(P. 694-705) ISSN 2538–7138 /Online/
References
Al-Balushi, S. M., & Al-Hajri, S. H. (2014). Associating animations with concrete models to enhance students’ comprehension
of different visual representations in organic chemistry. Chemistry Education Research and Practice, 15 (1), 47-58.
Barke, H. D., & Wirbs, H. (2002). Structural units and chemical formulae. Chemistry Education Research and Practice, 3 (2), 185-200.
Devetak, I., Vogrinc, J., & Glažar, S. A. (2010). States of matter explanations in Slovenian textbooks for students aged 6 to
14. International Journal of Environmental and Science Education, 5 (2), 217-235.
Eilks, I. (2013). Teacher pathways through the particulate nature of matter in lower secondary school chemistry: Continuous
switching between different models or a coherent conceptual structure? In G. Tsaparlis & H. Sevian (Eds.), Concepts of
matter in science education, (pp. 213-230). New York, NY: Springer.
Ferk Savec, V., Hrast, Š., Devetak, I., & Torkar, G. (2016). Beyond the use of an explanatory key accompanying submicroscopic
representations. Acta Chimica Slovenica, 63 (4), 864-873.
Ferk Savec, V., Sajovic, I. & Wissiak Grm, K. S. (2009). Action research to promote the formation of linkages by chemistry stu-
dents between the macro, submicro, and symbolic representational levels. In J. K. Gilbert (Ed.), Multiple representations
in chemical education, (Models and modeling in science education, vol. 4) (pp. 309-331). Berlin: Springer.
Ferk, V., Vrtačnik, M., Blejec, A., & Gril, A. (2003). Students’ understanding of molecular structure representations. International
Furió‐Más, C., Luisa Calatayud, M., Guisasola, J., & Furió‐Gómez, C. (2005). How are the concepts and theories of acid–base
reactions presented? Chemistry in textbooks and as presented by teachers. International Journal of Science Education, 27
(11), 1337-1358.
Gilbert, J. K. (2005). Visualization: A metacognitive skill in science and science education. In Visualization in science educa-
tion (pp. 9-27). Springer Netherlands.
Gilbert, J. K., Reiner, M., & Nakhleh, M. (2008). Visualization: Theory and practice in science education. United Kingdom: Springer.
Gkitzia V., Salta K., & Tzougraki C. (2011). Development and application of suitable criteria for the evaluation of chemical
representations in school textbooks. Chemistry Education Research and Practice, 12 (1), 5–14.
Harrison, A. G. (2001). How do teachers and textbook writers model scientific ideas for students? Research in Science Educa-
tion, 31 (3), 401-435.
Hinze, S. R., Rapp, D. N., Williamson, V. M., Shultz, M. J., Deslongchamps, G., & Williamson, K. C. (2013). Beyond ball-and-stick:
Students’ processing of novel STEM visualizations. Learning and Instruction, 26, 12-21.
Hinze, S. R., Williamson, V. M., Deslongchamps, G., Shultz, M. J., Williamson, K. C., & Rapp, D. N. (2013). Textbook treatments
of electrostatic potential maps in general and organic chemistry. Journal of Chemical Education, 90 (10), 1275-1281.
Hinze, S. R., Williamson, V. M., Shultz, M. J., Williamson, K. C., Deslongchamps, G., & Rapp, D. N. (2013). When do spatial abilities
support student comprehension of STEM visualizations? Cognitive Processing, 14 (2), 129-142.
Johnstone, A. H. (1991). Why is science difficult to learn? Things are seldom what they seem. Journal of Computer Assisted
Learning, 7 (2), 75-83.
Kapıcı, H. Ö., & Savaşcı-Açıkalın, F. (2015). Examination of visuals about the particulate nature of matter in Turkish middle
school science textbooks. Chemistry Education Research and Practice, 16 (3), 518-536.
Kelly, R. M., & Jones, L. L. (2008). Investigating students’ ability to transfer ideas learned from molecular animations of the
dissolution process. Journal of Chemical Education, 85 (2), 303.
Kozma, R. B., & Russell, J. (1997). Multimedia and understanding: Expert and novice responses to different representations of
chemical phenomena. Journal of Research in Science Teaching, 34 (9), 949-968.
Kozma, R., & Russell, J. (2005). Students becoming chemists: Developing representational competence. In J. Gilbert (Ed.),
Visualization in science education (pp. 121 – 146). London, UK: Kluwer.
Laçin-Şimşek, C. (2011). Women scientist in science and technology textbooks in Turkey. Journal of Baltic Science Education, 10
(4), 277-284.
Majidi, S., & Mäntylä, T. (2011). Knowledge organization in physics text books: A case study of magnetostatics. Journal of Baltic
Science Education, 10 (4), 285-299.
Mumba, F., Chabalengula, V. M., Wise, K., & Hunter, W. J. (2007). Analysis of New Zambian high school physics syllabus and
practical examinations for levels of inquiry and inquiry skills. Eurasia Journal of Mathematics, Science & Technology
Education, 3 (3), 213-220.
Nyachwaya, J. M., & Gillaspie, M. (2016). Features of representations in general chemistry textbooks: a peek through the lens
of the cognitive load theory. Chemistry Education Research and Practice, 17 (1), 58-71.
O’Keefe, P. A., Letourneau, S. M., Homer, B. D., Schwartz, R. N., & Plass, J. L. (2014). Learning from multiple representations: An
examination of fixation patterns in a science simulation. Computers in Human Behavior, 35, 234-242.
Sanger, M. J. (2000). Using particulate drawings to determine and improve students’ conceptions of pure substances and
mixtures. Journal of Chemical Educatiom, 77 (6), 762-766.
Stieff, M., Scopelitis, S., Lira, M. E., & Desutter, D. (2016). Improving representational competence with concrete models. Sci-
ence Education, 100 (2), 344-363.
704
ISSN 2538–7138 /Online/ (P. 694-705)
Stull, A. T., Gainer, M., Padalkar, S., & Hegarty, M. (2016). Promoting representational competence with molecular models in
organic chemistry. Journal of Chemical Education, 93 (6), 994-1001.
Stull, A. T., Hegarty, M., Dixon, B. L., & Stieff, M. (2012). Use it or lose it: Representational translation with concrete models.
Cognition & Instruction, 30 (4), 404-434.
Williamson, V. M., Hegarty, M., Deslongchamps, G., Williamson III, K. C., & Shultz, M. J. (2013). Identifying student use of ball-and-
stick images versus electrostatic potential map images via eye tracking. Journal of Chemical Education, 90 (2), 159-164.
Špela Hrast M.Sc. Doctoral Student, Teaching Assistant for Chemical Education,
University of Ljubljana, Faculty of Education, Kardeljeva ploščad 16,
1000 Ljubljana, Slovenia.
E-mail: spela.hrast@pef.uni-lj.si
Vesna Ferk Savec Ph.D. Associate Professor for Chemical Education, University
of Ljubljana, Faculty of Education, Kardeljeva ploščad 16, 1000
Ljubljana, Slovenia.
E-mail: vesna.ferk@pef.uni-lj.si
705
THE ECOLOGICAL
WORLDVIEWS AND LOCAL
ENVIRONMENTAL CONCERNS
AMONG SECONDARY SCHOOL
TEACHERS
Abstract. This research aims to explore the Lwun-Syin Lwo,

tendency of the New Ecological Paradigm Jim- Hua Fu,
and environmental concerns on local Cheng-Chieh Chang
environmental issues of secondary school
teachers. Two hundred seventy teachers
were selected by stratified random and
cluster sampling from 13 schools to par-
ticipate in the survey. The seven teachers
were interviewed on their concepts and
beliefs on the ecological worldviews and Introduction
local environmental concerns. The results
A person’s ecological views are generally subjective beliefs about hu-
of the research include: With high pro-
man–nature relationship that forms a conceptual basis for more specific
environmental beliefs, most of the teachers
attitudes, beliefs, and behaviors toward the environment. Ecological views
expressed their concerns about the environ-
are embedded in various cultural messages, and these beliefs are a compo-
ment. However, teachers believe that “Hu- nent of individual or group-level environmental mentality (Raudsepp, 2001).
man will eventually learn enough about Throughout history, the way in which people conceive nature–human relation
how nature works to be able to control it,” has changed dramatically, and different cultures also have divergent notions
which is labeled as anti-NEP item. It seems regarding the role that man plays in nature (Bechtel, Asai, Corral-Verdugo,
that teachers mostly notice to “human & Riesle, 2006). The human exception paradigm (HEP) or dominant social
learn” not to “human control”. It also seems paradigm (DSP), and the new environmental paradigm (NEP) were introduced
that teacher’ environmental worldviews as a 12-item measure of basic environmental belief system by Dunlap and
may be affected by their career training. Van Liere (1978). The HEP is a worldview that conceives human beings as
The results of the research provide some entities superior to all other organisms. Such vision establishes that humans
are independent from nature and have a strong confidence in science as a
implication for strengthening teachers’
means to control nature or solve problems regarding man’s adaptation to
environmental endorsement and therefore
the environment (Dunlap, Van Liere, Merting, & Jones, 2000). On the other
to affect people’s environmental ethics and
hand, the NEP considers human as just one component of the ecological
friendly behaviors. network, subject to the rules of the interdependence and diversity, which
Keywords: environmental education, local oblige this species to maintain a balance with the rest of the components of
environmental concerns, new ecological nature and limit the impact of its activity on earth. The HEP is conceived as
paradigm survey. an anthropocentric system of beliefs, since it places the human beings in the
center of everything, while the NEP is conceived as an eco-centric system of
beliefs (Bechtel, Asai, Corral-Verdugo, & Riesle, 2006).
Rapid, extensive, and ongoing environmental change increasingly
Lwun-Syin Lwo, demands that humans intervene in ecosystems to maintain or restore eco-
Jim- Hua Fu,
Cheng-Chieh Chang system services and biodiversity (Hobbs, Hallett, Ehrlich & Mooney, 2011).
National Taiwan Ocean University, Taiwan One of the main challenges to society is to find a balanced relation between
human and natural environments. Sustainable development is an accepted
706
ISSN 1648–3898 /Print/ THE ECOLOGICAL WORLDVIEWS AND LOCAL ENVIRONMENTAL CONCERNS AMONG
ISSN 2538–7138 /Online/ (P. 706-722)
concept for development aimed at finding a balance between providing the needs for the present and future hu-
man society and protecting the environment (World Commission on the Environment and Development, 1987). In
general, individuals’ awareness, views, and behaviors toward the environment are influenced by political ideology,
economical settings, and social structures within certain cultures (Huang & Yore, 2003). However, these factors are
also grouped in school settings. Schools have always been viewed as an institution for the transfer of knowledge
and culture to the future generation. Children obtain knowledge and develop attitudes toward the environment
mostly from general school curricula (Michail, Stamou, & Stamou, 2006). Schools can also become a place that
generates or produces knowledge on the solution and management of local environmental problems (Meinardi
& Revel-Chion, 2005). Hence, education is a crucial agent for achieving sustainable development and creating an
environmentally literate society that is motivated and equipped to influence decision-making (Orr, 1994; UNESCO-
UNEP, 1990) and ‘the ultimate aim of environmental education (EE) is to enable people to understand the complexities
of the environment and the need for nationals to adapt their activities and pursue their development in ways which
are harmonious with the environment’ (p.12) (UNESCO, 1978). The study of ecological worldviews and concerns is
fundamental to protect and improve the natural environment’s condition (Cordano, Welcomer & Scherer, 2003). In
recent decades, educators have become more aware of the role they have to play in conceptualizing environmental
issues and developing cognition concerning the environment in the future generations (Petegem, Blieck, & Pauw,
2007), since they are relatively accessible, active, and influential (UNESCO, 1988). Teachers’ beliefs and conceptions
of environment influence children’s environmental conception, especially in the elementary and secondary schools,
where teachers directly teach the domain knowledge of science and nature. Therefore, knowing secondary school
teachers’ ecological worldview and their local environmental concerns can shed some light as to how to make
the environmental education easier and capture the needs to protect the environment. Opinions from teachers
can provide worthy reference for education policymakers to empower teachers’ environmental endorsement and
foster public environmental concerns and behaviors.
Taiwan underwent industrialization and urbanization when it experienced rapid economic growth. However,
economic development without much environmental consideration has dramatically changed its natural environ-
ment (Hsu, & Roth, 1996; López, 2017). Economic progress in Taiwan has also led to greater public awareness of
environmental protection affairs. More and more people are becoming aware of the severity of the environmental
problem in Taiwan. Several studies have also pointed to the same conclusions (Hsu, & Roth, 1996; Huang & Yore,
2003). Taiwan’s population density is ranked as the second highest in the world and has caused a shortage of living
space and low quality of living standards. Most people in Taiwan feel that the environment quality in the area they
live is unsatisfactory. Environmental education has been an urgent concern since the 1980s. Both the government
and all levels of schools have initiated many efforts to promote EE in the past decade. Curriculum reform that took
place in the 1990s has defined infusing EE into the curriculum of every level of school. Moreover, the government
issued the Environmental Education Act in 2010 to advance citizens’ understanding of the interdependent relation-
ship between individuals, society and the environment, to raise the nation’s environmental ethics and responsibility,
so as to maintain the ecological balance of the environment, respect life, and foster social justice, and cultivate
environmentally aware citizens and environmental studies communities to achieve sustainable development
(Executive Yuan, 2010). On the whole, almost all the teachers have begun to take a note and engage themselves
in the EE. However, only a handful of teachers had received formal EE or training. For the most part, many teachers’
environmental perspectives and beliefs are still unknown.
Therefore, the problem area of the research concerns the lack of current empirical evidence about second-
ary teachers’ ecological worldview and their local environmental concerns. It is important to survey ecological
worldview and their local environmental concerns of secondary school teachers in Taiwan. The significance of the
research has two folds. First, the research reveals current teachers’ ecological worldviews that are recognized as
the foundation of teachers’ cognition, attitudes, and behavior toward EE. Second, the research is to determine the
local environment concerns and opinions, which secondary teachers have.
New Ecological Paradigm Scale Development and Trend
During the spring and summer of 1976, Dunlap and Van Liere (1978) surveyed two samples (general public
sample, GPS, and environmental organization sample, EOS) of Washington State residents, and found that a set
of 12 items could measure three facets of the new social paradigm or worldview (limits to growth, balance of
nature, anti-anthropocentrism). The NEP scale exhibited a good deal of internal consistency (coefficient α of 0.813
707
(P. 706-722) ISSN 2538–7138 /Online/
for GPS and 0.758 for EOS), and strongly discriminated between environmentalists and the general public. The
initial work of Dunlap and Van Liere (1978) was widely accepted by many researchers, and numerous studies used
the NEP scale as a measure of general ecological worldview in various populations and contexts. It has become a
valuable research instrument employed in hundreds of studies in dozens of nations (Dunlap, 2008; Dunlap, Van
Liere, Merting, and Jones, 2000; Harrway, Broughton-Ansin, Deaker, Jowett and Shephard, 2012; Bozzolasco, 2017).
The 12-item scale has been used mostly often with samples of the general public, but it has also been used
with samples of specific sectors such as farmers (Albrecht, Bultena, Hoiberg, & Nowak, 1982) and members of
interest groups. Arcury and Christianson (1990) compared the 1984 and 1988 Kentucky statewide environmental
worldviews in response to environmental problem and water shortage and found it was obvious that the local
environmental events could change people’s worldview. Noe & Snow (1990) evaluated the degree to park visitors
who support an ecological view of man and nature, as opposed to those who favor anthropocentric view of man
controlling nature more. Schultz, Unipan, and Gamba (2000) examined the environmental attitudes of foreign-born
Latino American high school students and their findings suggested that culture is an important determinant of
environmental attitudes.
The scale has also been used as the basic instrument in the international and cross-cultural researches. Corral-
Verdugo and Amendariz (2000) measured Mexicans on their beliefs about human–environment relations by the
NEP–HEP scale. The results pointed to a more holistic view of human relations with the environment, in contrast to
the dualistic vision of some western countries. Gooch (1995) compared the environmental beliefs and attitudes in
Estonia, Latvia, and Sweden. The results showed that the expected correlations between support for NEP, distrust
of science and technology, post-material values, and concern for environmental conditions were only partially
supported by the results of the Swedish study, and were not supported at all in case of the Baltic samples. LaLonde
and Jackson (2002) conducted an international survey via the Internet in which respondents not only completed
the NEP scale, but also commented critically and in-depth on the scale items. They distributed questionnaires using
e-mail. The survey was completed by 222 respondents from people living in 23 countries. The authors included
respondents’ characteristics, such as occupation and income to demonstrate the respondents’ diversity. However,
religion was considered as a significant characteristic in their study. Bostrom et al. (2006) conducted three surveys
in Bulgaria in 1998, 1999, and 2000 to provide evidence of high environmental concern, and that proximity to a
major petrochemical plant is associated with greater concerns. Their findings showed that the risk perceptions of
specific environmental problems are high. Bechtel, Asai, Corral-Verdugo, and Riesle (2006) compared the worldviews
in four different countries. Responses of 1358 undergraduates from USA, Japan, Mexico, and Peru to the HEP and
the NEP scale were analyzed. Results of this study produced a finding regarding the diversity of the environmental
belief structures among different national groups.
Dunlap, Van Liere, Merting, and Jones (2000) revised their NEP scale design to improve upon the original one
in several respects: (1) to tap a wider range of facets of ecological worldview, (2) offer a balanced set of pro- and
anti-NEP items, and (3) avoid outmoded terminology. The new scale was renamed as New Ecological Paradigm and
consisted of 15 items, since they tried to update and broaden the scale’s content. Since the production of Dunlap
and Van Liere’s (1978) original NEP scale, new issues have become increasing important within the environmen-
tal debate, in particular, the biotech rights, intrinsic value of nature, and moral obligations to future organisms
(Trobe & Acott, 2000). Dunlap argued that his NEP scale would capture the change in public opinion towards an
orientation that people should live in harmony with nature rather than use nature as a resource solely for human
purpose (Bostrom, Barke, Turaga, & O’Connor, 2006). Survey participants rated each of the NEP items on a Likert
scale ranging from 1 (strongly disagree) to 5 (strongly agree). The eight odd-numbered items were worded so that
agreement indicated a pro-ecological view and the seven even-numbered ones were worded so that disagreement
indicated a pro-ecological worldview.
After a pre-test with college students, the new set of NEP items was used in 1990 mail survey of a represen-
tative sample of Washington State residents and 676 completed questionnaires. Dunlap, Van Liere, Merting, and
Jones (2000) examined the degree of internal consistency of the 15-item scale. The coefficient α was very good,
with a value of 0.83. They suggested that future research would be to compare the degree to which the NEP beliefs
are organized coherently across different populations, including comparison of patterns of multidimensionality
when distinct dimensions emerge, as well as the degree to which the resulting worldviews influence a range of
environmental attitudes, beliefs, and behaviors. After factor analysis, they indicated that it would “be more appropri-
ate to treat the NEP as multidimensional”(p. 436) .The questions relate to five hypothesis facets: Balance of Nature
(items 3,8,13), Eco-crisis(items 5,10,15), Anti-Exemptionalism (items 4, 9,14), Limit to Growth (items 1,6,11) and
708
ISSN 2538–7138 /Online/ (P. 706-722)
Anti-anthropocentrism (items 2,7,12). They also claimed that the NEP possesses predictive, content, and construct
validities in some evidence (Cordano, Welcomer, and Scherer, 2003; Shephard, Mann, Smith, and Deaker, 2009).
The revised scale is still widely used by many researchers on environmental or ecological worldviews around
the world. Sherburn and Devlin (2004) investigated the relationships between academic major, environmental con-
cern, and the presence of a campus arboretum. Seventy college students completed a series of surveys including
the Environmental Preference Questionnaire, the Environmental Concern Scale, and the New Ecological Paradigm
Scale. They found that environmental studies majors were significantly more likely to value and use the arboretum
than were the other groups. Rideout, Hushen, McGinty, Perkins, and Tate (2005) conducted a longitudinal study
of environmental perspective among college students, resident-student opinion using New Ecological Paradigm
scale. Their sample subjects were taken through systematic alphabetic sampling (N=305) and a blanket e-mail
distribution of survey for voluntary response (N=659). The results were found to be positive with 54.4% and 61.1%
agreement with each NEP viewpoint, respectively. A two-way ANOVA indicated that this effect of sampling method
was significantly a gender effect (F(1, 960)=14.955, p<0.001). Mean values for women, 52.9 (systematic) and 55.3
(e-mail), were higher than those for men, 49.5 (systematic) and 51.6 (e-mail), indicating greater support for NEP
beliefs among the female respondents.
Manoli, Johnson, and Dunlap (2007) revised and validated the New Ecological Paradigm Scale for use with
upper elementary students. They interviewed 5-grade students to revise the NEP Scale for use with children. After
two-year validation and modification with large numbers of students, they suggested that a 3-dimensional modi-
fied NEP Scale for children with 10 instead of 15 items and revised wording was appropriate for use with children
aged 10-12years.
Lee (2008) examined the environmental attitudes of African American college students by using the 15-item
NEP Scale. The author also attempted to determine students’ everyday environmental behaviors such as recycling
and conservation and investigated major information sources for local, national, and international environmental
issues. In general, African American students were modestly pro-environmental, as determined by the NEP Scale.
Television and the Internet were the students’ major sources of information on environmental news.
Shephard, Mann, Smith, and Deaker (2009) surveyed 539 students by NEP Scale and found that age (p<.05),
gender (p<.001) and ethnicity (p<.001) demonstrated significant differences and identified four factors and used
their contributory NEP items to describe four consequential implied “tendencies” (to conserve, to recycle, to be
cautious about the future and to support animal and plant rights) by factor analysis. Harrway, Broughton-Ansin,
Deaker, Jowett and Shephard (2012) used the NEP to monitor changes in the ecological worldviews and contributory
sustainability tendencies of first-year undergraduate students in New Zealand. Totally, 360 participants completed
the surveys at three time points. They found women holding a stronger pro-ecological worldview than men and
students undertaking a zoology program having stronger pro-ecological attitudes than those in the other four
groups. Finally, they concluded that the NEP is a valuable research instrument for their study. However, their four-
factor model identified from the NEP is quite a new interpretation of the ecological worldviews.
Teachers’ Environmental Knowledge and Beliefs
The goal of EE and sustainability is’ to educate people to be environmentally responsible citizenship and to foster
environmental literacy for all’ (p. 571) (Shephard, Mann, Smith, and Deaker, 2009). Meinardi and Revel-Chion (2005)
claimed that EE should aim at three main objectives: (1) increase the knowledge about the different environmental
problems that affect different regions of the country, (2) promote changes as to how to teach science, by means
of transformation of the scientific conceptions of teacher, in addition to their ideas and teaching practices, and (3)
contribute to the raising teacher’s awareness in the performance of his/her role as orienting agent or facilitator of
the learning process, as well as in the strategies used, and the capability in handling students’ difficulties.
Michail, Stamou, and Stamou (2006) examined 155 Greek primary school teachers’ understanding of three
current environmental issues (acid rain, ozone-layer depletion, and greenhouse effect) and the emerging images
of nature. The study revealed that the teachers held several environmental knowledge gaps and misconceptions
about the three phenomena. Using the media as the major environmental information source in which environ-
mental issues were constructed as environmental risks, teachers became environmentally educated with respect
to common lay terms instead of scientific terms. Moreover, the image of nature emerging from their ideas about
the three environmental issues was that of the romantic archetype, which prevails in postindustrial societies.
Such a view gives a conceptualization of nature as balance, under which the greenhouse effect and acid rain are
709
(P. 706-722) ISSN 2538–7138 /Online/
seen as exclusively human-induced “disturbances.” Hsu and Roth (1996) conducted a study to assess the teachers’
environmental literacy and analyzed the predictors of teachers’ responsible environmental behavior. About 300
secondary school teachers, in Hualien County of eastern Taiwan, were randomly selected, and 157 effective response
data were analyzed. The results showed that secondary teachers in Hualien exhibited a relatively higher level of
knowledge of environmental action strategies, skills in using environmental action strategies, and intention to act.
They also found that a great majority of the teachers lived and taught in areas where they grew up, and teachers
living in urban areas took more environmental action than those living in rural areas. Their study also found that
the three most popular sources of teachers’ environmental information were newspapers (36.6%), TV (25.6%), and
books and magazines (24.4%).
Fransson and Garling (1999) tried to determine: (1) the correlations between determinants, such as sociode-
mographic or psychological factors, and environmental concern, and (2) an impact of environmental concern on
environmentally responsible behavior. After reviewing and analyzing previous research to determine whether
environmental concern plays an important role in behavior changes, they reviewed five background factors of
environmental concern, including age, social class, residence, political ideology, and gender. Their final summary
was concluded as “it has been suggested that younger, more educated individuals with liberal political ideologies
living in urban areas are most environmentally concerned.” However, such a statement must be made with great
caution. The relationships between social-demographic factors and environmental concern are generally weak.
The teachers play increasingly important roles in EE and pro-environmentalism. However, most teachers may
still have environmental knowledge gaps and misconceptions. The environmental concepts or ecological worldviews
are influenced by a person’s background and training experience. Thus, knowing the teachers’ worldviews may
provide good information for the policymakers or administrators to work on EE projects to increase endorsement
of the NEP attitudes, beliefs, and even their behaviors.
A survey was carried out or the collected data and interviews were conducted to comprehend their in-
depth understanding in 2015.
General Background
To measure whether people’s attitudes toward nature are actually changing, it is necessary to be able to val-
idly and reliably measure people’s beliefs and value systems (Trobe & Acott, 2000). Although factual knowledge of
environmental issues is not the sole determinant of worldview or behavior, it can have some influence over such
broad perspectives in the individual. A large base of factual ecological data is, of course, essential in the initial
scientific understanding leading to greater public awareness of ecological problems (Rideout, Hushen, McGinty,
Perkins & Tate, 2005).
The six items of personal information on EE are gender, age, religion, pre-service major, subject taught, and
training experience. Most of these items were cited from literature and the rest were considered to be the teachers’
special background, which may correlate to their worldview. All the items were checked by five experts, including
three professors, who had taught environmental courses and two science teachers in middle schools. Dunlap, Van
Liere, Merting, and Jones (2000) claimed that the NEP items primarily tap the “primitive beliefs” about the nature of
earth and humanity’s relationship with it. All the 15 items were translated into Chinese by the authors. They were
also checked by two English professors and three professors who had taught environmental courses in college level.
Sample Selection
A stratified random sample was taken to draw the participants with respect to the area and size of the schools.
After selecting the schools, 270 teachers were randomly selected to answer the questionnaire. There are 817 sec-
ondary school teachers in Keelung and the rate of selection was over 33%. Of the 245 questionnaires returned, 240
were valid. The return rate was 90.7%, making up 29% of the population of secondary school teachers in Keelung.
The distribution of teachers drawn by stratified sampling revealed more females than males, with the age range
mostly between 30 and 49 years. Most of them had no religious belief or were Buddhists, and a majority of them
had college degree, majored in social science, languages, or science. Most of the teachers had never taken any
710
ISSN 2538–7138 /Online/ (P. 706-722)
environmental courses, and over half of them had participated in environmental conferences, and few had joined
environmental groups or activities.
Development of New Ecological Paradigm Scale
With respect to the consistency and dimension of the scale, the internal consistency (Cronbach’s α) of the
scale in this study was 0.714 and was lower than that of Rieout et al. (2005) sampling of Ursinus college students
by systematic survey (0.828) and e-mail (0.816); however, it is still acceptable. The Kaiser–Meyer–Olkin value of the
present scale was 0.770 and was sufficient for analyzing the factors of the inventory. From Table 1, similar to the
findings of other studies (Shephard, Mann, Smith, and Deaker, 2009; Harrway, Broughton-Ansin, Deaker, Jowett and
Shephard, 2012) the analysis resulted in the 15 items loading into four factors where the eigenvalues were greater
than 1.0. The four factors cumulatively explain 51.598% of the total variance for the present study.
Table 1. Rotated component matrix.
Component
1 2 3 4
4 .707 -.159 .154 -.088

8 .637 .266 .033 .201
2 .630 .000 -.017 .084
12 .613 .230 .145 -.038
10 .529 .470 .026 .226
3 .005 .741 -.082 .157
7 .110 .714 .172 -.007
14 .082 -.031 -.703 .032
1 .080 -.103 .616 .170
9 .152 .145 .570 .074
5 .117 .423 .479 .222
11 .163 .411 .420 .315
13 -.008 .151 .196 .783
15 .034 .278 .194 .661
6 .450 -.312 -.129 .580
Extraction method: Principal Component Analysis. Rotation method: Varimax with Kaiser Normalization.
A rotation converged in five iterations.

The four factors showed in Table 2 were similar to Shephard et al. (2009), but still having a little difference in
facets and items in each facet.
Table 2. Comparisons of the distributions of items to factors with Dunlap et al. & Shepherd et al.
Shepherd et al.
Facets Present study Dunlap et al. 2000 study
2009 study
1. The reality of limits to growth 1,5,9,11,14 1,6,11 1,6,11
2.Anti-anthropocentrism 2,4,8,10,12 2,7,12 4,8,14
3. The fragility of nature’s balance 3,7 3,8,13 3,5,9,10,13,15
4.Rejection of exemptionalism 4,9,14 2,7,12
5.The possibility of an eco-crisis 6,13,15 5,10,15
711
(P. 706-722) ISSN 2538–7138 /Online/
The third part of the inventory contained nine items about local environmental issues. The questions were
collected according to the newspapers and other media, and were reviewed by five professors and 14 experts in
the focus panel. After arriving at a consensus on every item, nine items were selected as important local environ-
ment issues.
Open-Ended Question and Interview Questions
There was an open-ended question at the end of the questionnaire for extra comments on the worldview
or environmental concerns. Semi-structured questions were used to learn about the teachers’ deep concerns and
perceptions on the worldview. The questions were also reviewed by seven experts, including three professors in
education and environment, two elementary school principals and two mentor teachers, to construct its expert
validity. The interviews took about one hour and were done at each teacher’s respective schools. The whole process
was recorded by digital recorder and transcribed into text.
Results of Research
The main part of the analysis was twofold: first, to check the whole frequency distributions for the New Eco-
logical Paradigm scale of teachers and the comparative analysis by the teachers’ different variables in NEP; Second,
to analyze the local environmental concerns.
Data Analysis on Pro-Environment
The first analysis was aimed at determining the agreement or disagreement on each item, and the means,
deviations, and the percentage of pro-NEP. The total percentage of pro-NEP was calculated as the sum of “strongly
agree” and “agree” in odd items and “strongly disagree” and “disagree” in even items. The results are shown in Table 3.
Table 3. Frequency distributions for the New Ecological Paradigm Scale.
Agreement or disagreement SAa MA U MD SD M % of Pro-

level and the Percentage N (%) N (%) N (%) N (%) N (%) (SD) NEP
1. We are reaching the threshold 68 123 34 15 0 4.02

of the number of people that Earth 79.6
(28.3) (51.3) (14.2) (6.3) (0) (.823)
can support
2. Humans have the right to modify 10 31 39 117 43 3.63

the natural environment to suit 66.7
(4.2) (12.9) (16.3) (48.8) (17.9) (1.050)
their needs
3. When humans interfere with na- 134 97 2 2 5 4.47

ture, they often produce disastrous 96.2
(55.8) (40.4) (0.8) (0.8) (2.1) (.754)
consequences
4. Human ingenuity will ensure 11 45 70 79 35 3.34

that we do not make Earth as an 47.5
(4.6) (18.8) (29.2) (32.9) (14.6) (1.082)
uninhabitable place
5. Humans are severely abusing 136 97 5 1 1 4.53

97.1
the environment (56.7) (40.4) (2.1) (.4) (.4) (.606)
6. Earth has plenty of natural 18 94 41 69 18 2.90

resources if we just learn how to 36.3
(7.5) (39.2) (17.1) (28.8) (7.5) (1.129)
develop them
7. Plants and animals have as 139 98 2 0 1 4.56

98.7
much rights as human to exist (57.9) (40.8) (.8) (0) (.4) (.561)
8. The balance of nature is strong 5 12 29 121 73 4.02

enough to cope with the impacts of 80.8
(2.1) (5.0) (12.1) (50.4) (30.4) (.903)
modern industrial nations
712
ISSN 2538–7138 /Online/ (P. 706-722)
9. Despite our special abilities, 123 106 6 2 3 4.43

humans are still subject to the laws 95.5
(51.3) (44.2) (2.5) (0.8) (1.3) (.705)
of nature
10. The so-called “ecological 6 17 19 124 74 4.01

crisis” faced by mankind has been 82.5
(2.5) (7.1) (7.9) (51.7) (30.8) (.948)
greatly exaggerated
11. Earth is like a spaceship with 134 102 1 3 0 4.53

98.6
very limited room and resources (55.8) (42.5) (.4) (1.3) (0) (.578)
12. Humans are meant to rule over 3 8 9 105 115 4.34

91.7
the rest of nature (1.3) (3.3) (3.8) (43.8) (47.9) (.807)
13. The balance of nature is very 94 119 16 8 3 4.22

88.8
delicate and can be easily upset (39.2) (49.6) (6.7) (3.3) (1.3) (.811)
14. Humans will eventually learn 96 123 15 5 1 1.72

enough about how nature works to 2.5
(40.0) (51.3) (6.3) (2.1) (.4) (.711)
be able to control it
15. If things continue on their

present course, we will soon 129 95 12 2 2 4.45
93.4
experience a major ecological (53.8) (39.6) (5.0) (.8) (.8) (.707)
catastrophe
Total 3.94 77.06

SA=Strongly Agree; MA=Mildly Agree; U=Unsure; MD=Mildly Disagree; and SD=Strongly Disagree
The total percentage of pro-NEP was 77.06, showing that most of the teachers have NEP worldview. They re-
vealed high means and percentage in NEP items, especially in items 7, 11, and 5. Furthermore, most of the teachers’
perceptions had low means in HEP, especially in items 14, 6, and 4. It is worth noting the lowest mean of item 14.
Among the five hypothesis facets which Dunlap, Van Liere, Merting, and Jones (2000) proposed, the highest
facet was “The fragility of nature’s balance” with a mean of 4.24 and the lowest one was the “Rejection of exemp-
tionalism” with a mean of 3.16. The distribution on each facet is shown in Table 4.
Table 4. Distribution on the five facets.
Mean of
Facets Minimum Maximum Mean of facet SD
item
1. The reality of limits to growth 6 15 11.44 1.67 3.82
2. Anti-anthropocentrism 7 15 12.53 1.69 4.18
3. The fragility of nature’s balance 6 15 12.71 1.69 4.24
4. Rejection of exemptionalism 5 14 9.49 1.44 3.16
5. The possibility of an eco-crisis 8 15 12.98 1.64 4.33
Total mean 44 73 59.16 5.57 3.94
Comparison of Variables
While testing the different demographic variables on teachers’ worldview, we found only one variable show-
ing a significant difference, which is the teachers’ subject taught. After Scheffé comparison, teachers who taught
physical education revealed the lowest mean (Table 8). There were no differences among gender, age, educational
background, religious belief, course taken, participation in environmental conference or activities, or joining various
environmental groups. The statistical analysis by t-test and ANOVA is shown in Tables 5 and 6.
713
(P. 706-722) ISSN 2538–7138 /Online/
Table 5. The t-test for gender, religious, course, conference, group, and activity.
Variable N M SD F t-value p
Male 63 58.33 6.754 5.349 −1.200 .233

Female 177 59.45 5.068
Without religious belief 104 59.17 5.336 1.146 .036 .971

With religious belief 136 59.15 5.756
Haven’t taken any course on EE 153 58.83 5.387 .401 −1.213 .226
Have taken course(s) on EE 87 59.74 5.856
Haven’t participated in any conference on EE 84 58.69 6.615 13.400 −.876 .383

Have participated in conference(s) on EE 156 59.41 4.916
Haven’t joined any environmental group 219 59.07 5.561 .061 −.766 .445
Have joined environment-related activity 21 60.05 5.687
Haven’t participated in any activity for the environment 173 59.01 5.655 .329 −.656 .513
Have participated in activity for the environment 67 59.54 5.355
Table 6. ANOVA for age, religious, degree, major, teaching area.
Variable SS df MS F p
Age Between groups Within 109.621 4 27.405 .883 .475

groups 7296.363 235 31.048
Total 7405.983 239
Religion Between groups Within 265.005 5 53.001 1.737 .127
groups 7140.979 234 30.517
Total 7405.983 239
Degree Between groups Within 65.108 2 32.554 1.051 .351
groups 7340.875 237 30.974
Total 7405.983 239
Major Between groups Within 5.747 4 1.437 .046 .996
groups 7400.236 235 31.490
Total 7405.983 239
Teaching area Between groups Within 524.067 6 87.344 2.957 .008**
groups 6881.917 233 29.536
Total 7405.983 239
**p <0.01
Table 7. Scheffé comparison on teaching area.
LSD
Teaching area N M SD
comparison
1 Language 93 59.98 4.943 1>6

2 Math 47 58.26 5.899 2>6
3 Social science 28 59.07 4.729 3>6
4 Nature science 36 60.28 5.740 4>6
5 Art and literacy 11 59.18 5.618 5>6
6 Physical Education 13 53.77 6.313 6>6
7 Comprehensive activities 12 59.00 6.620 7>6
Total 240 59.16 5.567
714
ISSN 2538–7138 /Online/ (P. 706-722)
Local Environmental Concerns
(1) When asked, “Do you care about our environmental issues and events?” the percentage of teachers who
answered that they “Care” and “very much” was 89.6%. Only 10.4% of the teachers stated that they cared little
about the environmental issues, and no one expressed that they never cared. The actual numbers and percentage
distribution are shown in Table 8.
Table 8. Number and percentage of care about environmental issues.
Frequency N %
Very much 21 8.8

Care 194 80.8
Care little 25 10.4
Never care 0 0
Total 240 100.0
(2) When asked, “Where did you get your environmental information?” and instructed to rank the highest 3 by
order, the top three sources were found to be newspaper (30.9%), TV (29.7%), and network (20.6%). The weighed
values, percentage distribution, and the final rank are shown in Table 9.
Table 9. Sources of environmental information.
Ordered by first three ranks

% of after
Sources Final rank
weighedb
Rank 1 Rank 2 Rank 3 Sum (weighed)a
Newspaper 85 74 42 445 30.90 1

TV 84 64 48 428 29.72 2
Network 49 50 50 297 20.63 3
Books and Magazines 6 21 36 96 6.67 4
School 8 9 12 54 3.75 5
Tour or visiting 3 5 23 42 2.92 6
Government documents 3 7 11 34 2.36 7
Groups or Society 0 7 6 20 1.39 8
Broadcast 1 1 10 15 1.04 9
Friends 1 1 3 8 .56 10
Others 0 0 0 0 0 11
Total 100
a = (R1*3+R2*2+R1), b =a/ (n*6)*100
(3) When asked, “What are the most serious problems in Taiwan that need our efforts to solve?” and instructed
to rank the highest 3 by order, the five most serious environmental problems were found to be air pollution (20%),
major river pollution (16.5%), toxic waste (14.9%), trash pollution (13.68%), and shortage of resources (11.38%).
The weighed values, percentage distribution, and final rank are shown in Table 10.
715
(P. 706-722) ISSN 2538–7138 /Online/
Table 10. Most serious environmental problems in Taiwan.
Ordered by first three ranks

% of after
Problems Rank
weighedb
Rank 1 Rank 2 Rank 3 Sum (weighed) a
Air pollution 57 39 40 289 20.0694 1

Major river pollution 20 69 40 238 16.5278 2
Toxic waste 40 29 37 215 14.9306 3
Trash pollution 38 23 37 197 13.6806 4
Shortage of resources 35 22 15 164 11.3889 5
Maintain drain 30 24 23 161 11.1806 6
Wild-animal extinction 8 8 15 55 3.8194 7
Soil-layer sink 2 14 19 53 3.6806 8
Over population 9 6 7 46 3.1944 9
Noise pollution 1 6 7 22 1.5278 10
Total 100
a = (R1*3+R2*2+R1), b =a/ (n*6)*100
(4) When asked, “Whether the highway between Ilan and Hualien need to be constructed?”, 61.3% teachers
disagreed or strongly disagreed. There was still a high percentage (25.8%, over a quarter) of teachers who were
neutral when there was conflict between economic development and environment. The actual numbers and
percentage distribution are shown in Table 11.
Table 11. Agreement to construction of highway.
Agreement N %
Strongly Agree 5 2.1

Mildly Agree 26 10.8
Neutral 62 25.8
Mildly Disagree 93 38.8
Strongly Disagree 54 22.5
Total 240 100.0
(5) When asked, “What do you think about the quality of everyday lives in Taiwan?”, only 22.5% of the teachers
were satisfied or strongly satisfied with the living quality. On the other hand, 63.4% of the teachers were unsatisfied
or strongly unsatisfied. The actual numbers and percentage distribution are shown in Table 12.
Table 12. Satisfaction on living quality.
Satisfied N %
Very satisfied 1 0.4

Satisfied 53 22.1
Neutral 34 14.2
Unsatisfied 143 59.6
Strongly unsatisfied 9 3.8
Total 240 100.0
(6) When asked, “Did you ever talk about our environmental issues in your classroom?”, 85% of the teachers
716
ISSN 2538–7138 /Online/ (P. 706-722)
stated that they often or sometimes discussed environmental issues in their class. The actual numbers and percent-
age distribution are shown in Table 13.
Table 13. Discussion of environmental issues in the classroom.
Frequency N %
Often 46 19.2
Sometimes 158 65.8
Seldom 33 13.8
Never 3 1.3
Total 240 100.0
(7) When asked, “Do you think most teachers care about the protection of endangered animals?” 95% of the
teachers agreed or strongly agreed that most teachers care about the protection of endangered animals. The actual
numbers and percentage distribution are shown in Table 14.
Table 14. Attitude toward protection of endangered animals.
Agreement N %
Strongly Agree 66 27.5

Mildly Agree 162 67.5
Mildly Disagree 12 5.0
Strongly Disagree 0 0
Total 240 100.0
(8) When asked, “Do you think most teachers know that whale sharks are protected animals?”, only 6.7% of the
teachers thought that most teachers know that whale sharks are protected animals and 52.5% teachers answered
“a little.” Over 40% of the teachers thought that most teachers “mostly do not know” or “have no idea.” The actual
numbers and percentage distribution are shown in Table 15.
Table 15. Knowledge on the fact that whale sharks are protected animals.
Known N %
Absolutely 16 6.7
A little 126 52.5
Mostly do not know 88 36.7
Have no idea 10 4.2
Total 240 100.0
(9) When asked, “Do you think if people know that whale sharks are protected animals, it may affect the be-
havior of eating whale sharks?”, only 14.2% of the teachers thought that people would stop eating whale sharks if
they know whale sharks are protected animals. Most teachers thought that knowing whale sharks are protected
animals will slightly change or stop people from eating them. The actual numbers and percentage distribution
are shown in Table 16.
717
(P. 706-722) ISSN 2538–7138 /Online/
Table 16. Effect on the eating of whale sharks.
Influence N %
Yes 34 14.2
A little 192 80.0
Absolutely not 14 5.8
Total 240 100.0
The Open-Ended Question and Interviews
The last question was open to any answers or comments associated with the topic of worldview and en-
vironmental issues. Comments were coded as Q (questionnaire) and a number. The seven interviewees expressed
their opinions and viewpoints on worldview and environmental issues. Their opinions were retyped and coded as
W (interview) and a number. After that, three linked sub-processes (Huberman & Miles, 1994) for qualitative find-
ings, data reduction, and data display, followed by conclusion drawing and verification, were employed. The most
important points were drawn as follows:
Recognition of Humans in Harmony with Nature
Most of the teachers did not believe that humans should control everything just because they have the ability
to process thoughts. Other species need to be respected as they have the same rights to live on Earth.
“All animals have the same rights to live on Earth...” (Q160)
“Human being is only one of the many living organisms; if we think human is an exception from other species, we will easily
come to neglect the rights of other species and the meaning of nature.”(W01)
“Human is the only living thing with thought, but we still need to respect the existence of other species.”(W04)
“I agree that only human can think; however, nature does not exist for human beings only.”(W05)
“It is common sense that we have to protect and respect all life.”(Q05)
Concerns for Shortage of Natural Resources and Support for Setting Limitations on Economic Development
Most of the teachers understood the shortage of natural resources and agreed to set limitations on the
economic development. While faced with the dilemma of economic development and protecting the environment,
they agreed that the government should show its power and resolution to balance the needs of the people and
protect the environment. Some of their statements include:
“Natural resources are not unlimited; the needs for modernization have resulted in over exploitation.”(W02)
“I personally hope to set limitations on economic development, though it is not easy.” (W03)
“I agree that we should have laws to set limitations. In Taiwan, many people would look for loopholes to exploit resources,
because they think nature can always recover from losses. But we have to enforce the laws, otherwise it is meaningless to
have them.”(W05)
“Our policies are still too conservatory, the environment assessment for construction is too lax, and the damage is getting
worse.”(Q74)
“Natural resources are not infinite, even if they were, we still can’t waste them.”(W06).
Living Environment in Taiwan is not satisfactory
Teachers believed that low living quality was not only the public’s problem but the government should
bear the main responsibility. They considered that policies with long-term outlook that foster sustainable devel-
opment need to be set up, and people need to know where the government stands on environment policy. The
government also has to get people to back its policies on environment. Some of the teachers’ comments are:
“Damage to the environment is everyone’s problem, people, or government. The government needs to have proper
policies and the resolution to enforce those policies.”(Q02)
718
ISSN 2538–7138 /Online/ (P. 706-722)
“The population is so large and the land so over developed, and the government doesn’t manage the resources
effectively.”(Q48)
“The living environment is not satisfactory due to air and water pollution, all main rivers are dirty, noise is loud in many
places, mountains are exploited. If the government is determined to make a balance between economic development and
protection of the environment, I think we will be fine.”(W03)
“Taiwan is only a small island, but there are so many highways and roads.”(Q62)
“Our government can set up more sites and utilities for environmental education, such as museums, reservation areas,
and riverside parks.”(T03)
“The government should improve the quality of drinking water and air, and garbage problem to make our living environ-
ment better.”(Q44)
“More awareness events can be sponsored by government and non-government groups.”(Q131).
Changes in Behaviors toward the Environment
Most of the teachers did not think that environmental knowledge can directly change people’s behavior,
as selfishness and habits are not easily removed or changed. However, many teachers believed that the knowledge
will influence a person’s behavior.
“People care too much about their own benefits. Most are not concerned about public affair, therefore, policy on protection
of environment is hard to execute.”(Q45)
“Earth strikes back. If people do not pay more attention to the environment, someday we will destroy ourselves. The world
is like a village, everyone is responsible for doing their part about global warming.”(W01)
“Though many people are aware of the importance of environmental protection, but following policy is another issue. We
need more environmental education to help make the policy works.”(Q73)
“If people have awareness of the environment, they will accept the concepts of protecting the environment. I believe that
more than half the people in Taiwan can change their dietary habits and not eat endangered animals.”(Q121).
The Need for EE
Most teachers did not think EE in Taiwan is good enough. They expressed their concerns about the environ-
ment; however, they admitted that their knowledge on the environment and ecological systems are insufficient.
“Most teachers are concerned about environmental issues. I, myself, know only a little about the policy on environmental
protection. I hope our government can let us know more about the policy and hold more environmental conferences or
activities.”(W01)
“Teachers are intellectuals and they are always concerned about this issue.”(W03)
“As an educator, I support the policy.”(Q158)
“I don’t think we have enough knowledge about the environment. The government can provide more information, courses,
and pedagogical materials on environmental education for us.”(W02)
These comments reflect the teachers’ worldviews and deep concerns about the environment. They hope to
see the government manage the economic affair, but at the same time, give more considerations on the environ-
ment. They also hope to use their knowledge about the environment and ecological system, so as put their pro-
environment attitudes into actions.
Discussions
The percentage of agreement with NEP in this study is 77.06, which is higher than the percentage obtained
by Dunlap et al. (2000) from Washington state residents in 1990 (66.5%) and that by Rieout et al. (2005), collected
from the students of Ursinus College by systematic survey (54.4%) and by e-mail (61.1%). The result obtained
from our study demonstrated the teachers’ widespread concern for environmental issues in Taiwan.
The teachers’ awareness of the possibility of an eco-crisis is the highest in the five facets, and the result re-
vealed that there is a considerable anxiety for environmental problems in Taiwan. However, in the present study,
after factor analysis, four facets within 15-item scale similar to Shephard et al. (2009); and in item 14, “Humans
will eventually learn enough about how nature works to be able to control it”, 91.3% of the teachers expressed
“agree” or “strongly agree.” Only 2.5% of the teachers agreed with the NEP side. This phenomenon is quite dif-
719
(P. 706-722) ISSN 2538–7138 /Online/
ferent from the earlier studies, and can be explained in two respects: 1) item 14 is a compound sentence, after
translation into Chinese language, the weigh in the first part becomes the main focus, meaning more emphasis
in “Humans will eventually learn enough about how nature works…” than “to be able to control it.” Hence, it is
quite reasonable that most teachers believe in human’s knowledge and the potential of learning, and 2) cultur-
ally speaking, nearly all teachers in Taiwan were taught to strive for the best and that was what they have been
doing since childhood. Teachers work hard every day and they encourage their students to do the same. It is
acceptable to explain the teachers’ beliefs in “work hard and learn” better than proposal by Dunlap, Van Liere,
Merting, and Jones (2000) in HEP.
The demographic variables have almost no substantial explanatory value in explaining the worldview or
environmental concerns. Our finding of no gender difference is consistent with that of the results of Steger and
Witt (1989) and Gooch (1995); however, it contradicts those research findings that support the statement that
women are more environmentally conscious than men are (Milbrath, 1984; Rieout et al, 2005; Scott & Willits, 1994).
Theoretical explanations for gender differences involve the increased knowledge and tolerance for technologi-
cal advances, support for economic growth, and lower perception of environmental risks among men (Blocker
& Eckberg, 1997). In this study, the teachers’ social status and the economic situation in Taiwan are only slightly
influenced by the economic growth. Another phenomenon worth emphasizing is that most of the teachers in
Taiwan keep their neutral positions in religious beliefs when they become teachers. Hence, the religious factor
has little effect in their worldview and environmental concerns.
Teachers care about environmental issues and discuss environmental topics in their classroom, showing a
high percentage (89.6% and 85.0%, respectively). Over half of the teachers (61.3%) disagreed with the construc-
tion of the highway between Ilan and Hualien, because there is a mountain range in the area and the construc-
tion may destroy the entire ecological system. Furthermore, the teachers are not satisfied with the quality of
everyday lives in Taiwan. While they are aware that whale sharks are protected animals, they believe that most
people would not stop eating whale sharks just by the fact that whale sharks are protected animals. Overall,
teachers were found to be more concerned about the environmental ethics, and believe that educating people
about the environmental ethics is still a long process.

Conclusions and Recommendations
According to the finding for this research, that NEP which includes attitudes, beliefs, and behaviors got the
percentage of agreement 77.06, which is a high percentage. It describes the teachers’ widespread concern for
environmental issues in Taiwan. The teachers’ awareness of the possibility of an eco-crisis is the highest in the
five facets. Factor analysis found that 91.3% of the teachers expressed “agree” or “strongly agree.” with the NEP
side. This phenomenon can be explained that most teachers believe in human’s knowledge and the potential
of learning, and culturally speaking are taught to strive for the best and that was what they have been doing
since childhood. Teachers care about environmental issues and discuss environmental topics in their classroom,
showing a high percentage (89.6% and 85.0%, respectively). Overall, teachers are found to be more concerned
about the environmental ethics, and believe that educating people about the environmental ethics is still a long
process. Teachers understand the shortage of natural resources and agree to set limitations on the economic
development. They can suggest that the government should show its power and resolution to balance the needs
of the people and protect the environment.
The research provides some implications for scholars who are concerned about the environment and EE. We
believe that a long-term observation of teachers’ or public’s ecological beliefs and attitudes is worth discovering.
Authors can also glean insights that can help us understand about environment and ecological systems become
the most important part for teachers teaching students. So teachers can identify and care about environmental
issues and discuss in their classroom. These insights can be used by teachers and researchers in future to support
students’ learning about envirotmental topics.
In the future, it is expected that further research can be conducted in different fields, and also that deeper
analyses and interviews can continue to explore the environmental and ecological systems that influence educa-
tional system. We expect that in the following years there will be more research regarding ecological worldviews
and local environmental concerns.
720
ISSN 2538–7138 /Online/ (P. 706-722)
References
Arcury, T.A., & Christianson, E. H. (1990). Environmental worldview in response to environmental problems Kentucky 1984 and
1988 compared. Environment and Behavior, 22 (3), 387-407.
Albrecht, D., Bultena, G., Hoiberg, E., & Nowak, P. (1982). The new environmental paradigm scale. The Journal of Environmental
Education, 13 (3), 39-43.
Bechtel, R. B., Asai, M., Corral-Verdugo, V., & Riesle, A.G. (2006). Across-cultural study of environmental belief structures in USA,
Japan, Mexico, and Peru. International Journal of Psychology, 41 (2), 145-151.
Blocker, T., & Eckberg, D. (1997). Gender and environmentalism: Results from the 1993 general social survey. Social Science
Quarterly, 78, 841-858.
Bostrom, A., Barke, Turaga, R. M., & O’Connor, R.E. (2006). Environmental concerns and the new environmental paradigm in
Bulgaria. The Journal of Environmental Education, 37 (3), 25-40.
Bozzolasco, A. M. (2017). Using a mixed-methods approach to understand urban children’s nature conceptions, ecological worldviews,
and environmental perceptions and preferences before and after attending an environmental education program (Doctoral
dissertation, Montclair State University).
Cordano, M., Welcomer, S. A., & Scherer, R.F. (2003). An analysis of the predictive of the new ecological paradigm scale. The Journal
of Environmental Education, 34 (3), 22-28.
Corral-Verdugo, V., & Amendariz, L.I. (2000). The “new environmental paradigm” in a Mexican community. The Journal of Envi-
ronmental Education, 31 (3), 25-32.
Dunlap, R. (2008). The new environmental paradigm scale: From marginality to worldwide use. The Journal of Environmental
Education, 40 (1), 3-18.
Dunlap, R., & Van Liere, K. (1978). The new environmental paradigm. The Journal of Environmental Education, 9 (4), 11-19.
Dunlap, R. E., Van Liere, K. D., Merting, A. G., & Jones, R. E. (2000). Measuring endorsement of the new ecological paradigm: A
revised NEP scale. Journal of Social Issues, 56 (3), 425-442.
Executive Yuan (2010). The Environmental Education Act. 2012/08/29 retrieved from http://law.epa.gov.tw/en/laws/674865564.
html.
Frasson, N., & Garling, T. (1999). Environmental concern: Conceptual definitions, measurement methods, and research findings.
Journal of Environmental Psychology, 19, 369-382.
Gooch, G. D. (1995). Environmental beliefs and attitudes in Sweden and the Baltic States. Environment and Behavior, 27 (4), 513-539.
Harrway, J., Broughton-Ansin, F., Deaker, L., Jowett, T., & Shephard, K. (2012). Exploring the use of the revised new ecological
paradigm scale (NEP) to monitor the development of students’ ecological worldviews. The Journal of Environmental Educa-
tion, 43 (3), 177-191.
Hobbs, R. J., Hallett, L. M., Ehrlich, P. R., & Mooney, H. A. (2011). Intervention ecology: Applying ecological science in the twenty-
first century. Bioscience, 61 (6), 442-450.
Hsu, S., & Roth, R. E. (1996). An assessment of environmental knowledge and attitudes held by community leaders in Hualien
area of Taiwan. The Journal of Environmental Education, 28 (1), 25-32.
Huberman, A. M., & Miles, M. B. (1994). Data management and analysis methods. In N. K. Denzin, & Y. S. Lincoln (Eds.), Handbook
of qualitative research (pp. 428-444). Thousand Oaks, CA: Sage Publications.
Huang, H. P., & Yore, L. D. (2003). A comparative study of Canadian and Taiwanese grade 5 children’s environmental behaviors,
attitudes, concerns, emotional dispositions, and knowledge. International Journal of Science and Mathematics Education,
1, 419-448.
LaLonde, R., & Jackson, E. L. (2002). The new environmental paradigm scale: Has it outlived its usefulness? The Journal of Envi-
ronmental Education, 33 (4), 28-36.
López, A. (2017). Deconstructing Chipotle: Media as environmental education. Teaching Media Quarterly, 4 (3), 6.
Lee, E. B. (2008). Environmental attitudes and information sources among African American college students. The Journal of
Environmental Education, 40 (1), 29-42.
Manoli, C. C., Johnson, B., & Dunlap, R. E. (2007). Assessing children’s environmental worldviews: Modifying and validating the
new ecological paradigm scale for use with children. The Journal of Environmental Education, 38 (4), 3-13.
Meinardi, E., & Revel-Chion, A. (2005). The critical condition of environmental education in Argentina: The need for conceptual
and methodological innovations and new educational policies. Journal of Science Education, 6 (2), 103-105.
Meichtry, Y., & Harrell, L. (2002). An environmental education needs assessment of K-12 teachers in Kentucky. The Journal of
Environmental Education, 33 (3), 21-27.
Michail, S., Stamou, A. G., & Stamou, G. (2007). Greek primary school teachers’ understanding of current environmental issues:
An exploration of their environmental knowledge and images of nature. Science Education, 91, 244-259.
Milbrath, L. (1984). Environmental Vanguard for a new society. Albany, NY: State University of New York Press.
Noe, F. P., & Snow, R. (1990). The new environmental paradigm and further scale analysis. The Journal of Environmental Education,
21 (4), 20-26.
Orr, D. (1994). Earth in mind: On education, environment and the human prospect. Washington, DC: Island.
Petegem, P. V., Blieck, A., & Pauw, J. B. (2007). Evaluating the implementation process of environmental education in pre-service
teacher education: Two case study. The Journal of Environmental Education, 38 (2), 48-55.
Raudsepp, M. (2001). Environmental belief systems: Empirical structure and a typology. TRAMES: A Journal of the Humanities &
Social Science, 5 (3), 234-254.
721
(P. 706-722) ISSN 2538–7138 /Online/
Rideout, B. R., Hushen, K., McGinty, D., Perkins, S., & Tate, J. (2005). Endorsement of the new ecological paradigm in systematic
and e-mail samples of college students. The Journal of Environmental Education, 6 (2), 15-22.
Schultz, P. W., Unipan, J. B., & Gamba, R. J. (2000). Acculturation and ecological worldview among Latino Americans. The Journal
of Environmental Education, 31 (2), 22-27.
Scott, D., & Willit, F. K. (1994). Environmental attitudes and behavior: A Pennsylvania survey. Environment and Behavior, 26 (2),
239-260.
Shephard, K., Mann, S., Smith, N., & Deaker, L. (2009). Benchmarking the environmental values and attitudes of students in New
Zealand’s post-compulsory education. Environmental Education Research, 15 (5), 571-587.
Sherburn, M., & Devlin, A.S. (2004). Academic major, environmental concern, and arboretum use. The Journal of Environmental
Education, 35 (2), 23-36.
Steger, M. A. E., & Witt, S. L. (1989). Gender differences in environmental orientations: A comparison of publics and activities in
Canada and the U.S. The Western Political Quarterly, 42, (4), 627-649.
Trobe, H. L., & Acott, T. G. (2000). A modified NEP/DSP environmental attitudes scale. The Journal of Environmental Education, 32
(1), 12-21.
UNESCO-UNEP (1990). Environmentally educated teachers: The priority of priorities? Connect, 15 (1) 1-3.
UNESCO (1978). Intergovernmental conference on environmental education. Final report. 14–26 October, UNESCO and UNEP:
Tbilisi (USSR).
Van Liere, K., & Dunlap, R. (1981). Environmental concern: Does it make a difference how it is measured? Environment & Behavior,
13 (6), 651-676.
World Commission on the Environment and Development (1987). Our common future. Oxford, England: Oxford University Press.
Received: May 18, 2017 Accepted: September 12, 2017
Lwun-Syin Lwo PhD., Professor, Institute of Education & Center of Teacher Education,
National Taiwan Ocean University, No.2, Beining Rd., Jhongjheng
District, Keelung City 202, Taiwan (R.O.C).
E-mail: lol@mail.ntou.edu.tw
Jim- Hua Fu Graduate Student, Institute of Education, National Taiwan Ocean
University.
E-mail: fjh115_2@hotmail.com
Cheng-Chieh Chang PhD., Associate Professor, Institute of Education & Center of Teacher
(Corresponding author) Education, National Taiwan Ocean University, No.2, Beining Rd.,
Jhongjheng District, Keelung City 202, Taiwan (R.O.C).
E-mail: changjac@email.ntou.edu.tw
722
EDUCATIONAL EFFECTIVENESS
OF THE 5E MODEL FOR
SCIENTIFIC ACHIEVEMENT OF ISSN 1648-3898 /Print/
STUDENTS WITH HEARING ISSN 2538-7138 /Online/
IMPAIRMENT
Abstract. This experimental research

Zahida Parveen
aimed to find the effectiveness of the 5E
model of education for achievement in
science among students with hearing
impairment at the elementary level. An
instructional package was developed
following the 5E model for one unit from
a science subject in grade VIII. A science
test was developed by the researcher to
Introduction
test the achievement of students of grade
VIII with hearing impairment. This test was
Cognitive abilities are pre-requisites for learning science rather than
validated by experts, pilot tested, sub-
hearing ability. The importance is to what extent impairment in hearing ability
jected to item analysis and the reliability
affects cognitive development, or intellectual abilities. Leading researchers
of the test was calculated. The test with 38
have concluded that deaf people are not intellectually deficient (Moores,
items was used as a pre-test and post-test
1979); they have the same range of cognitive ability as other students (Bursuck
during the study. Thirty-four students of
& Friend, 1996; Paul & Jackson, 1993). According to Rittehouse and Spiro (as
grade VIII were selected from a school and
cited by Moores, 1979), deaf people exhibit the same level of intelligence as
were random assigned to two groups.
normal hearing people. Thus, there are no reasons why science cannot be
One group was randomly assigned as the
taught to these students. Hundreds of examples are found in the literature
experimental group and the other served as
about deaf people who have made their careers in science education. Science
the control group. The research followed a
education expands the experiential background for students with disabilities
pre-test-post-test control group design. The
who have limited experiences (Patton & Andre, 1989) and is recommended as
performance of both groups was compared
the most valuable subject for students with sensory and physical disabilities
to evaluate the effectiveness of instructions
(Mastropieri, Scruggs, Boon, & Carter, 2001; Atwood & Oldham, 1985; Hadary,
based on the 5E model. A significant im-
Hadary & Cohen, 1978; Linn & Thier, 1975).
provement was found in the performance
Difficulties exist due to hearing loss in communication and learning, but
of the experimental group as compared to
this does not mean that teachers should not make efforts to teach science to
the control group. It was concluded that
these students. The teaching methodology can encourage and motivate these
hearing impaired children can perform
students towards choosing a career. If emphasis is placed on multisensory
better in science if they are taught through
active learning and the use of different strategies based on teaching problem
approaches promoting construction of
solving, a significant rise in the achievement of science among these children
knowledge, such as in the 5E model.
can be observed (Luckner & Carter, 2001). However, the lack of teacher train-
ing of special educators in science teaching is a contributing factor for low Keywords: hearing impairment, scientific
scientific achievement of students with hearing impairment (Malik, 2014). achievement, 5E model.
Elementary teachers do lack practical exposure to science, which may lead
to their limited knowledge about science (Atwater, Gardner & Knight, 1991).
Special education teachers report that most of the time, science is not taught
Zahida Parveen
and if taught, less time is allocated to students with disabilities as compared University of Education, Pakistan
to general education science students (Patton, Polloway, & Cronin, 1990).
Further, 42% of special education teachers are not trained for teaching science
723
(P. 723-732) ISSN 2538–7138 /Online/
and 38% of special children do not receive any science instruction. Cawley (1994) highlights the issues in science
education for students with disabilities. According to him, special educators have little experience of science, while
science teachers have little knowledge of adaptations for disabilities. This suggests a need to train science teachers
with sufficient knowledge of teaching deaf students.
Constructivist and activity-based teaching approaches are found to better enable the acquiring of science
concepts among students with hearing impairment (Barman, (1991). Hands on experiences are beneficial for
these students, as is the case for students without disabilities, to develop understanding of complex processes of
science. According to Chira (1990), deaf students can be taught science through active explorations. Lang (1995)
also emphasized the use of such instructional strategies to cognitively engage deaf students. Research evidence
supports the use of a three-phase learning cycle, based on active learning, in science instruction to students with
hearing impairment (Charles & Jill, 1991).
The 5E model, based on a learning cycle of five phases, is an innovation resulted from studies at the Centre of
Biological Sciences Curriculum Studies (BSCS, 1997). The 5E model is viewed from a conceptual change (Volkmann,
Brown, West, Lankford, & Abell, 2009) and inquiry learning perspectives (Wilson, Taylor, Kowalski, & Carlson, 2010).
The sequence within the model for the teaching process is applicable in the development of programs, units and
lessons. The 5E instructional model consists of five phases:
Engage. The teacher or a curriculum task accesses the learners’ prior knowledge and helps learners become
engaged in a new concept through the use of short activities that promote curiosity and elicit prior knowledge.
Explore. Exploration experiences provide students with a common base of activities within which current
concepts (i.e., misconceptions), processes, and skills are identified and conceptual change is facilitated.
Explain. The explanation phase focuses students’ attention on a particular aspect of their engagement and
exploration experiences and provides opportunities to demonstrate their conceptual understanding, process
skills, or behaviors.
Elaborate. Teachers challenge and extend students’ conceptual understanding and skills.
Evaluate. The evaluation phase encourages students to assess their understanding and abilities and provides
opportunities for teachers to evaluate student progress toward achieving the educational objectives” (p. 1)
This model is found to improve achievement in science among students in general. Ozsevgec (2006) inves-
tigated the effectiveness of guided materials about “Force and Motion” for elementary students, based on the 5E
model. He concludes that guided material increases the level of academic success of elementary students and their
attitudes become more positive and retainable. Hirca, Çalik, and Seven (2011) investigate the effect of instruction,
based on the 5E model, on conceptual change. They conclude that instruction, based on the 5E model, is more
effective for students to comprehend related concepts and in overcoming their own alternative conception, as
compared to a traditional one. Caprico (1994) reports student gains when taught through a constructivist meth-
odology. Brad (2000) compares instruction based on a constructivist approach with a teacher centered method
and found a higher degree of academic achievement in favor of the student-centered approach. Positive outcomes
using 5E activities are also reported by Moseley, Reinke, and Bookout (2002), Boddy, Watson, and Aubsson (2003),
Einsenkraft (2003), Clark (2003), Evans (2004), Newby (2004), Akar (2005), Cardak, Dikmenli & Saritas (2008) as well
as Wilson, Taylor, Kowalski & Carlson (2010).
Deaf students can perform equally as well as their counterparts with the required adaptation and appropriate
strategies used by teachers. Moores (1987), quoting an example of a study by Karchmer and Belmont on short term
memory, indicates that deaf students performed at a lower level than hearing students. However, after teaching
through appropriate strategies to deaf subjects, they then find that deaf children can function at the same level as
hearing children. They conclude that the lower initial performance of deaf students is not because of a cognitive
deficit, but through not knowing which strategy to apply. Research related to learning styles and cognition of deaf
students show that no differences are found among students with and without hearing. Lang (1995) concludes
from the results of a study, based on Grasha-Reichmann Student Learning Styles Scales, those adolescents with
and without deafness have similar learning styles. This study also indicates that deaf adolescents value clarity,
organization, and structure.
There are many research studies supporting the use of constructivist approaches with hearing impaired
students. Elefant (1980), in conducting an experiment on deaf students implementing an Inquiry Development
Program, concludes that deaf students are successful in acquiring inquiry skills. Boyd and George (1973) also
observe higher scores for deaf group using hands-on activities. Lang, Stinson, Basile, Kavanagh, and Liu (1999)
conclude that higher participation of deaf students in activities leads to higher grades. In a study by Quinsland
724
ISSN 1648–3898 /Print/ EDUCATIONAL EFFECTIVENESS OF THE 5E MODEL FOR SCIENTIFIC ACHIEVEMENT OF
ISSN 2538–7138 /Online/ (P. 723-732)
(1986), deaf students’ learning is enhanced by an experiential activity, where they act out the different parts of
the human heart. The factual learning by the students is significantly better than that of a control group which are
guided to learn the same material through a traditional lecture. Akynoglu and Tandogan (2007) determine that
the use of active learning models enhances the academic achievement and develops positive attitudes towards
the learning of science.
Brown et al., (2002) designed a project ‘Classroom of the Sea’ (COS). From their study, they concluded that a
constructivist environment was helpful in increasing deaf students’ knowledge, attitude, behavior (KAB) and self-
efficacy related to science. Bybee and Hendricks (1972) worked on developing science concepts among preschool
deaf students using materials from the Science Curriculum Improvement Study (SCIS). The basic teaching sequence
of SCIS (exploration, invention, and concept application) was used with adaptations made for deaf in the teaching
method. The study reported a positive gain in achievement. Sunal and Sunal (1982) reported positive gains in the
achievement of science among students with hearing impairment using adapted information from the Houghton
Mifflin Science series to incorporate a multi-sensory approach to teaching science.
Achievement by Pakistani students with hearing impairment is underestimated in terms of their learning.
Classroom learning of students with hearing impairment is inadequate (Noreen & Aziz, 2000). Science education
encourages children to think imaginatively and logically, but only if the concepts are acquired accurately. It is
found in a study that students with hearing impairment acquire weak concepts of science as compared to their
counterparts (Nazir & Iram, 2001). Similarly, Bashir & Akram (1998) find that achievement of students with hear-
ing impairment in Science, Social Studies and Mathematics is lower than their counterparts. In another study of
general science concepts, Shah & Anjum (2009) find misconceptions among hearing impaired children regarding
basic scientific concepts.
Afzal & Naz (1998) found the possible reasons for the low achievement of students with hearing impairment
in science including the inability of the teachers to communicate effectively with students and a lack of application
of various teaching methods in the classrooms. Iqbal and Sharif (2006) conducted a study of problems faced by
children with hearing impairment in learning of science at VIII grade. They concluded that the syllabus of science
at VIII grade was not fulfilling the modern demands and students were not satisfied with the teaching methods of
science teachers. It was found that the students’ view was that their science curriculum was lacking in the develop-
ment of scientific thinking, problem solving and observation skills. Moreover, this research recommended the use
of laboratory and hands on activities to enhance scientific achievement among students with hearing impairment.
In Pakistan, science teachers of students with hearing impairment at elementary level have set low profiles for
these students about science learning (Malik, 2014). A major problem is observed in schools with the appointing
of special education teachers as science subject teachers of HIC. These teachers lack subject knowledge as well as
being unaware of specific teaching methods for science. This is a result of the thinking that, in Pakistan, students
with hearing impairment are not to be provided with the option to opt for science beyond elementary level. The
Board of Intermediate and Secondary Education (BISE) offers tailoring, embroidery, type writing, carpet weaving,
rug weaving, drawing, bakery, hosiery, computer science and physical education. BISE specifies that achieve-
ments of students with hearing impairment are to be assessed for knowledge, comprehension and application
levels (Board of Intermediate and Secondary Education Lahore, 2010). It is a fact that their inability to listen has a
significant impact on their learning, but research shows that the cognition of children with hearing impairment
is comparable with their peers without hearing loss. They may face difficulties in learning science due to poor
hearing, inappropriate instruction, teachers’ unfriendly attitude and parents holding a low-profile set. Schools are
not providing necessary support to science teachers to teach science to children with hearing impairment in an
effective way. It is also a possible reason why teachers are not ready to accept the challenge of teaching science
to children with hearing impairment. According to Fontana & Lari (2001), a large majority of head teachers (74%)
in Pakistan give lowest priority to science studies in the curriculum for students with special needs. They further
highlight the need for research in experimenting with teaching methods and learning approaches.
As the 5E model is found to increase the scientific achievement among general students, it is predicted to be
appropriate for disadvantaged students. Hence, in this research, an intervention, based on 5E model, is developed
to evaluate its effect on science achievement among students with hearing impairment. Further, this research is
delimited to only the first three levels (Knowledge, Comprehension and Application) of cognitive domain, as in
Pakistan, BISE (2010) limits assessment to these levels for students with hearing impairment at elementary level.
Additionally, it’s effectiveness for both genders are explored.
725
(P. 723-732) ISSN 2538–7138 /Online/
Null Hypotheses
The research was conducted to test the following hypothesis:

1. There was no significant difference in achievement in science between students with hearing impair-
ment receiving instructions based on the 5E model and those receiving regular classroom instructions.
2. There was no significant difference in achievement in science between male and female students with
hearing impairment receiving instructions based on 5E model of learning and those taught through
regular classroom instructions
3. There is no significant difference in achievement scores of the control and experimental group on the
basis of components of the cognitive domain.
General Background
The target population of this research was students of grade VIII with hearing impairment, studying in Lahore
schools. Children with hearing impairment were taught in a separate school system in Pakistan. They had low
scientific literacy skills as compared to their counterparts (Shaukat & Rubab, 2007). Their potential for learning
was underestimated by the school administrations and science teachers (Shehbaz & Batool, 2008). Therefore,
science was not taught to children with hearing impairment beyond elementary level. The scope of this research
was to evaluate the learning of science among deaf students when a 5E instructional model was used instead of
the traditional lecture method. This research covered a period of ten months in total, including development of
achievement test and intervention, and conduction of experiment. However, the experiment based on 5E model
covered twenty-four class sessions of forty minutes each across six weeks. The study followed pre-test post-test
control group design.
Sample
Thirty-four students with hearing impairment of grade VIII from Hamza Foundation Academy for the Deaf were
selected for the purpose of the research. The age of these students ranged between middle and late adolescence.
The probable reason for this variation in age within the grade was late admission of some students in schools.
Criteria for the Selection of Academy
The Hamza Foundation Academy for the Deaf was selected for this study for the following reasons.
1. An appropriate number of students studying in Class VIII.
2. An appropriate number of female students in class VIII.
3. The school administration was eager to cooperate in the research.
4. All relevant data of students were attainable.
Thirty-four students of 8th class were selected for the research. These students were divided into three sections
within the school, but the researcher divided them into two groups for science instructions. A list of students from
the class teachers was obtained and an ID code was assigned to each student. These students were divided into
two groups of seventeen students through random assignment. One group was randomly selected as the control
group. This group received regular classroom instructions. The other group was selected as the experimental group.
This group received specially designed instructions based on the 5E model. For this purpose, a science teacher
from the same academy was trained through few sessions and provided with a set of developed intervention
materials by the researcher.
1. Achievement Test
One chapter “classification of organisms” was selected from the syllabus of class VIII, and, for this, a test was
developed to measure the achievement of students with hearing impairment. In addition, the researcher personally
726
ISSN 2538–7138 /Online/ (P. 723-732)
visited the schools to discuss the content for the study with the science teachers. This was based on the textbook
from the Punjab text-book board for teaching science to grade VIII as all schools followed the same textbook.
The major topic of “Classification of organisms” was selected on the following basis:
1. Most of the teachers agreed to use this topic.
2. The topic of classification is seen as a base for building higher concepts indicated in the syllabus.
A table of specifications was developed consisting on six sub-topics, using Bloom’s taxonomy for the cogni-
tive domain, and forty-three test items were constructed. This test was presented to science education experts in
constructivist approaches and education for the deaf. Their suggestions were incorporated in modifications to the
test. Further, this test was pilot tested with 49 students with hearing impairment from three schools of Lahore and
then subjected to item analysis. After expert validation and item analysis, the instrument was reduced to 38 items
and was used as a pre-test and post-test during the study. The Cronbach alpha of the test was 0.83.
Development of Intervention
A set of instructions were developed by the researcher, based on the 5E model and validated by an expert of
constructivist approaches. One lesson was pilot tested with a group of fourteen students with hearing impairment
in a Goverment Secondary School of Special Education for the Hearing Impaired Children (Boys). These students
were not included in the final study. One lesson for each sub-topic (six) from the unit “Classification of organisms”
was developed according to the phases of the 5E instructional model. Each lesson consisted of Instructional Ob-
jectives, Content Focus, Overview, Key words, Required Materials, Classroom Layout, Instructional Procedure (5Es)
and Closure.
Conduction of Experimentation
The control group was taught by its science teacher through a traditional lecture method, involving direct
teaching, question and answer technique, drill and practice technique, and using total communication with the
hearing impaired. The experimental group was taught using the 5E model of instruction. A 40 minutes class period
was specified for each day for both groups. The treatment was given to the experimental group for six weeks. The
school schedule was followed in the research for time and day allocation. There were four class periods per week
for science subject. One lesson for each sub-topic was prepared. There was no time boundary for the completion of
a lesson. Consumption of class periods for each lesson was kept flexible according to hearing disability of students
because once the lessons were developed according to 5E model; it was easy to adapt already planned lessons
for time and other changes. In this way, six sub-topics were covered in twenty-four class periods. The science test,
with thirty-eight items developed by the researcher, was conducted with both the control group and experimental
group prior to the treatment. After the treatment, the same test was administered as a post-test to both groups.
Results of Research
The scores of students on the achievement test were analyzed to find differences among the control group
and experimental group on the pre-test, the post-test and gained scores. An independent sample t- test was car-
ried out to find the significance of these differences.
Table 1. Comparison of control group and experimental group on pre-test, post-test and gain scores.
Group Pre-test Post-test Gain scores
Control N=17
M 36.88 55.12 18.24
SD 15.93 19.36 6.58
N=17
Experimental M 35.18 71.00 35.82
SD 13.18 13.38 9.52
727
(P. 723-732) ISSN 2538–7138 /Online/
Group Pre-test Post-test Gain scores
MD 1.70 15.88 17.58
t-value 0.34 82.78 6.27

p value 0.73 .009 .0001
Table 1indicates the results of an independent sample t-test on pre-test, post-test and gain scores for the
control group and the experimental group. Results show that there were no significant differences among the
control group and the experimental group on pre-test scores. However, significant difference is observed between
post-test scores and gain scores of the control groups and the experimental group.
Comparison Based on Gender
Table 2. Gender wise comparison of control group and experimental group on pre-test, post-test and gain
score.
Group Pre-test Post-test Gain score
Girls control N=5

M 41.20 57.40 16.20
SD 22.81 24.55 4.19
Girls Exp N=6
M 40.50 74.00 33.50
SD 18.16 14.64 11.34
MD 0.70 16.60 17.30
T 0.05 1.39 3.15
p .95 .19 .01
Boys Control N=12
M 35.08 54.17 19.08
SD 12.96 17.96 7.11
Boys Exp N=11
M 32.27 69.36 37.09
SD 9.31 13.07 8.71
MD 2.81 15.19 18.01
T 0.59 2.30 5.45
p .56 .03 .0001
Results showed that there was a significant difference in gain scores of girls of both groups and boys of both,
control and experimental groups (Table 2). Girls and boys in the control group and experimental group performed
equally on pre-test. Boys in experimental group improved significantly on the post-test as compared to boys in the
control group, while this difference was not significant for girls of both groups on the post-test.
Comparison of Achievement Scores by Levels of Cognitive Domain
Students’ performance for each Bloom’s level of cognitive domain on the achievement test was analyzed to
find difference in performance by the control group and the experimental group over different component of the
test. An independent sample t-test was used to find the differences.
728
ISSN 2538–7138 /Online/ (P. 723-732)
Table 3. Comparison of control group and experimental group performance on levels of cognitive domain
on the achievement test.
Group Pre-test Post-test
Know Comp App Know Comp App
Control N=17
M 15.24 12.59 9.24 22.41 19.24 13.71
SD 7.00 5.07 6.67 8.04 6.08 6.86
Experimental N
M 14.71 12.29 8.53 22.65 24.00 24.35
SD 6.88 5.37 3.31 5.76 4.07 6.35
MD 0.53 0.30 0.71 0.24 3.76 10.64
T 0.22 0.16 0.39 0.09 2.68 4.69

p .82 .87 .70 .92 .01 .0001
Before the intervention, the control group and experimental group students performed equally on the three
cognitive levels (knowledge, comprehension, and application), while after intervention the experimental group
showed good performance on comprehension and application component of cognitive domain as compared to
performance of the control group (Table 3). It was concluded that 5E instruction improved comprehension and
application abilities. Achievement in the knowledge component was the same in the control group and experi-
mental group after the different treatments.
Discussion
The research was designed to assess the effectiveness of an inquiry oriented, 5E model on achievement in
science among students with hearing impairment. Findings indicated that students who received instructions
based on the 5E model gained a significant higher achievement in science as compared to their counterparts who
were taught through a lecture method. The results of this study were consistent with earlier studies conducted on
students with hearing impairment with reference to inquiry based instructions (Grant, Rosenstein & Knight, 1975,
Hadary, Haushatter, & Rosenberg, 1976, Boyed & George, 1973). The effectiveness of instruction, based on the 5E
model, was evident from the literature that supported the findings of the present study (Ozsevgec, 2006; Ozdilex
& Ozkan, 2009; Ergin, Kanli & Unsal, 2008; Wilson et al. 2010).
Instruction, based on the 5E model, was equally applicable for girls and boys in the experimental group. No
differences in achievement were found among girls and boys receiving the intervention. Boys in the experimental
group performed better on the post-test as compared with the control group boys while the same result was given
by the girls. These findings contrasted with the study by Zaleta (2014), who studied the impact of gender on sci-
ence process skills of general students. He found that girls gained higher scores than boys when taught through
an inquiry curriculum.
The intervention proved to be more effective for application and comprehension components of the cogni-
tive domain, although no difference was found between students of the control group and experimental group on
the knowledge component of the cognitive domain, where both groups gained almost equally. This finding was
supported by research by Mao, Chang, and Barufaldi (1998). They found in their experimental study that inquiry
group students performed better on higher order skills (comprehension and application) than knowledge reten-
tion, when compared with their counterparts in a control group.
The pilot instructional testing session, prior to the research, was very exciting for the researcher, because it
provided the courage to complete the intervention. The students with hearing impairment were found to be very
enthusiastic throughout the study as they actively participated in activities. On the verbal request of the head of
the school wing, a brief session, after the study, was conducted by the researcher for the science teachers of the
academy to give them some understanding of this constructivist approach. This phase was really an enriching
729
(P. 723-732) ISSN 2538–7138 /Online/
and exciting experience. It was felt during the study that teachers were willing to teach using methods other than
traditional one if the administration asked them to do so. Although they were still reluctant to use inquiry in their
daily lessons, they liked the approach.
The willingness of teachers to use teaching methods, based on inquiry (like 5E), indicates a change in their
perception and attitude towards science learning among students with hearing impairment. Studies focusing
on the effects of inquiry oriented teaching methods on attitudinal changes among teachers of hearing impaired
students are recommended for future research.
Conclusions

The present research concludes that the students receiving inquiry-based instructions, based on the approach
associated with the 5E model, performed better based on the post-test and their gain scores are significantly
higher when compared with scores by the control group. The null hypotheses are therefore rejected. This research
strengthens the idea that where inquiry instruction can be successfully implemented in science classrooms of
students with hearing impairment, they can achieve at a higher level. These findings may also raise teacher’s im-
pression about the capabilities of hearing impaired students when the focus is on teaching methodologies which
can enhance scientific achievement. Using a teaching approach based on the 5E model is shown to be effective
for hearing impaired students.
This research has implications for curriculum developers in Pakistan to reset the subject options available for
the students with hearing impairment, after the elementary level, according to their potential for learning. Noting
the student gains using a more inquiry-based teaching approach suggest that offering science beyond the elemen-
tary level can be strongly recommended. To further educational developments of hearing impaired students this
study recommends promoting teacher training programs for science teachers of the deaf, with emphasis on the
use of inquiry method, highly equipped science labs for hearing impaired institutes and appointment of teach-
ers with a master’s degree in science as science teachers in schools for the hearing impaired. Further, researchers
in science education need to explore other more effective strategies to enhance scientific achievement among
hearing impaired students.
References
Afzal, A., & Naz, M. (1998). A study of the problems faced by the teachers of hearing impaired children. (Unpublished master’s thesis).
University of the Punjab, Pakistan.
Akar, E. (2005). Effectiveness of 5e learning cycle model on students’ understanding of acid-base concepts. (Unpublished master’s
thesis). Retrieved from: http://etd.lib.metu.edu.tr/upload/12605747/index.pdf.
Akynoglu O., & Tandogan, R.O. (2007). The effects of problem-based active learning in science education on students’ academic
achievement, attitude and concept learning. Eurasia Journal of Mathematics, Science & Technology Education, 3 (1), 71-81
Retrieved from: http://www.ejmste.com/v3n1/EJMSTEv3n1_Akinoglu.pdf.
Atwater, M. M., Gardner, C., & Knight, C. R. (1991).Beliefs and attitudes of urban primary teachers toward physical science and
teaching physical science. Journal of Elementary Science Education, 3 (1), 3-12.
Atwood, R. K., & Oldham, B. R. (1985). Teachers’ perceptions of mainstreaming in an inquiry oriented elementary science program.
Science Education, 69 (5), 619-624.doi: 10.1002/sce.3730690504.
Barman, C. R. (1991). Integrating science into the K-8 curriculum of deaf children. Final Report to the Indiana Commission for Higher
Education. (67). Retrieved from: http://files.eric.ed.gov/fulltext/ED343779.pdf.
Bashir, R., & Akram, F. (1998). The study of differential achievements of hearing impaired children and normal children of grade 5.
(Unpublished mater’s thesis). University of the Punjab, Pakistan.
Board of Intermediate and Secondary Education. (2010). Secondary school certificate: Important instructions. BISE Lahore.
BSCS, (1997). BSCS biology: A human approach (first edition). Dubuque, IA: Kendall/Hunt Publishing Company.
Boddy, N., Waston, K., & Aubsson, P. (2003). A Trial of the Five Es: A referent model for constructivist teaching and learning. Research
in Science Education, 33 (1), 27-42. Retrieved from: http://www.springerlink.com/content/r03926m2812212j7/fulltext.pdf.
Boyd, E., & George, K. D. (1973). The effect of science inquiry on the abstract categorization behavior of deaf children. Journal of
Research in Science Teaching, 10 (1), 91-99.
Brad, H. (2000). Teacher-centered Instruction versus Students-centered instruction. American School Board Journal, 1-5.
Brown, S. W., Babb, I., Johnson, P. R., Scheifele, P. M., Lang, H., Zheng, D., et al. (2002). Classroom of the Sea: Problem-based learn-
ing for the deaf. Proceedings of the International Conference on Computers in Education. Auckland, New Zealand: IEEE Press.
Bursuck, W. D., & Friend, M. (1996). Including students with special needs: A practical guide for classroom teachers. London: Allyn
& Bacon.
730
ISSN 2538–7138 /Online/ (P. 723-732)
Bybee, R. W., & Hendricks, P. A. (1972). Teaching science concepts to preschool deaf children to aid language development. Sci-
ence Education, 56 (3), 303-310.doi: 10.1002/sce.3730560304.
Bybee, R. W., Taylor, J. A., Gardner, A., Scotter, P., Carlson Powell, J., Westbrook, A., et al. (2006). The BSCS 5E instructional model:
origins and effectiveness. Retrieved from: http://science.education.nih.gov/houseofreps.nsf/b82d55fa138783c2852572c
9004f5566/$FILE/Appendix%20D.pdf.
Caprico, M. W. (1994). Easing into constructivism, connecting meaningful learning with students` experience. Journal of College
Science Teaching, 23 (4), 210-212.
Cardak, O., Dikmenli, M., & Saritas, O. (2008). Effect of 5E instructional model in student success in primary school 6th year circula-
tory system topic. Asia-Pacific Forum on Science Learning and Teaching, (9) 2. Retrieved from: https://www.eduhk.hk/apfslt/
download/v9_issue2_files/cardak.pdf.
Cawley, J. F. (1994). Science for students with disabilities. Remedial and Special Education, 15 (2), 67.
Chira, S. (1990). Wherein balloons teach the learning process. Perspectives in Education and Deafness, 8 (4), 5-7.
Clark, 1. (2003). Soils ain’t soils. Investigating: Australian Primary & Junior Science Journal, 19 (4), 13-16.
Einsenkraft, A. (2003). Expanding the 5E Model. The Science Teacher, 70 (6), 56-59. Retrieved from http://www.its-about-time.
com/iat/5e.pdf.
Elefant, E. F. (1980). Deaf children in an inquiry training program. The Volta Review, 82 (5), 271-279.
Ergin, I., Kanli, U., & Unsal, Y. (2008). An example for the effect of 5E model on the academic success and attitude levels of
students’:“Inclined projectile motion”. Journal of Turkish Science Education, 5 (3), 47-59. Retrieved from http://www.tused.
org/internet/tused/archive/v5/i3/text/tusedv5i3s3.pdf.
Evans, C. (2004). Learning with inquiry minds: students are introduced to the unit on gas laws and properties of gases using the
5E model. The Science Teacher, 71 (1), 4.
Fontana, D., & Lari, Z. (2001). The curriculum in special needs education in Pakistani schools. International Journal of Special
Education, 16 (1). Retrieved from http://www.internationalsped.com/documents/fullissue161.doc.
Grant, W. D., Rosenstein, J., & Knight, D. L. (1975). A project to determine the feasibility of BSCS’s Me Now for hearing impaired
students. American Annals of the Deaf, 120, 63-69.
Hadary, D. E., Hadary, S. H., & Cohen, S. H. (1978). Laboratory science and art for blind, deaf, and emotionally disturbed children: A
mainstreaming approach. Baltimore: University Park Press.
Hadary, D. E., Haushalter, R., & Rosenberg, R. (1976). Breaking sound barriers for the deaf child. Science and Children, 14, 33.
Hirca, N., Çalik, M., & Seven, S. (2011). Effects of guide materials based on 5E model on students’ conceptual change and their
attitudes towards physics: A case for ‘work, power and energy unit. Journal of Turkish Science Education, 8 (1), 153-158.
Retrieved from: http://www.tused.org/internet/tused/archive/v8/i1/text/tusedv8i1a8.pdf.
Iqbal, S., & Sharif, A. (2006). A Study of problems faced by children with hearing impairment in learning science at VIII grade. (Unpub-
lished master’s thesis). University of the Punjab, Pakistan.
Lang, H. G. (1995). Best practices: Science education for deaf. Retrieved from http//deafed.net/PublishedDocs/LANGSciencelit-
review5.doc.
Lang, H. G., Stinson, M. S., Basile, M., Kavanagh, F., & Liu, Y. (1999). Learning styles of deaf college students and instructors’ teach-
ing emphases. Journal of Deaf Studies and Deaf Education, 4 (1), 16-27. Retrieved from: http://paperity.org/p/39750204/
learning-styles-of-deaf-college-students-and-instructors-teaching-emphases.
Linn, M. C., & Thier, H. D. (1975). The effect of experiential science on development of logical thinking in children. Journal of
Research in Science Teaching, 12 (1), 49-62.doi: 10.1002/tea.3660120108.
Luckner, J. L., & Carter, K. (2001). Essential competencies for teaching students with hearing loss and additional disabilities.
American Annals of the Deaf, 146 (1), 7-15.
Malik, A. (2014). A study of problems faced by students with hearing impairment in learning science. (Unpublished master’s thesis).
University of Management and Technology, Pakistan.
Mastropieri, M. A., Scruggs, T. E., Boon, R., & Carter, K. B. (2001). Correlates of inquiry learning in science. Remedial and Special
Education, 22 (3), 130–137.
Mao, S. L., Chang, C. Y., & Barufaldi, J. P. (1998). Inquiry teaching and its effects on secondary-school students’ learning of earth
science concepts. Journal of Geoscience Education, 46, 363-367. Retrieved from: http://www.nagt.org/files/nagt/jge/ab-
stracts/mao-chang-baruf.pdf.
Moores, D. F. (1979). Educating the deaf: Psychology, principles, and practices. Boston: Houghton Mifflin company.
Moores, D. F. (1987). Educating the deaf: Psychology, principles, and practices (3rded.). Boston: Houghton Mifflin company.
Moseley, C., Reinke, K., & Bookout, V. (2002). The effect of teaching outdoor environmental education on preservice teachers’
attitudes toward self-efficacy and outcome expectancy. The Journal of Environmental Education, 34 (1), 9-15.
Nazir, M., & Iram, S. (2001). A study of the scientific concepts of hearing impaired children of class 7th. (Unpublished master’s thesis).
University of the Punjab, Pakistan.
Noureen, A., & Aziz, R. (2000). A study of interpersonal relationships between teachers and students in the institutions of hearing
impaired children. (Unpublished master’s thesis). University of the Punjab, Pakistan.
Ozdilek, Z., & Ozkan, M. (2009). The effect of applying elements of instructional design on teaching material for the subject of
classification of matter. The Turkish Online Journal of Educational Technology-TOJET, 8 (1). 84-96. Retrieved from http://eric.
ed.gov/PDFS/ED503906.pdf.
731
(P. 723-732) ISSN 2538–7138 /Online/
Ozsevgeç, T. (2006). Determining effectiveness of student guiding material based on the 5E model in ‘‘force and motion’’ unit.
Journal of Turkish Science Education, 3 (2), 24–27. Retrieved from http://www.tused.org/internet/tused/archive/V3/i2/text/
tusedv3i2s3.pdf.
Patton, J. R., & Andre, K. E. (1989). Individualizing for science and social studies. In J. Wood (Ed), Mainstreaming: A practical ap-
proach for teachers (pp. 301-351). Columbus, OH: Merrill.
Patton, J., Polloway, E., & Cronin, M. (1990). A survey of special education teachers relative to science for the handicapped. (Unpub-
lished manuscript). University of Hawaii, Honolulu.
Paul, P. V., & Jackson, D. W. (1993). Towards a psychology of deafness. Boston: Allyn & Bacon.
Quinsland, L. K. (1986). Experiential learning vs. lecture learning with postsecondary hearing-impaired learners: A study of the
potential need for change to occur in instructional methodology (Doctoral Dissertation), Walden University.
Shah, A. M., & Anjum, N. A. (2009). A study of general science concepts of students with hearing impairment at secondary level.
(Unpublished master’s thesis). University of the Punjab, Pakistan.
Shehbaz, F. S., Batool, B. S. (2008). A study of the perceptions of the teachers of hearing impaired children about the capabilities of
hearing impaired children (Unpublished master’s thesis). University of the Punjab.
Shoukat, N., & Rubab, T. (2007). Comparative of study of scientific literacy of student with and without hearing (Unpublished master’s
thesis). University of the Punjab, Pakistan.
Sunal, C. S., & Sunal, D. W. (1982). Adapting science for hearing impaired early adolescents (Final Report). Morgantown: West Virginia
University. ERIC Document Reproduction Service No. ED 219 267.
Wilsin, C., Taylor, J., Kowalski, S., & Carlson, J. (2010). The relative effects of inquiry-based and commonplace science teaching on
students’ knowledge, reasoning and argumentation: A randomized control trial. Journal of Research in Science Teaching,
47 (3), 276 - 301. Retrieved from: http://www.bscs.org/pdf/BSCS_SREE_5E_presentation.pdf.
Zaleta, K. L. (2014). The effects of gender and type of inquiry curriculum on sixth grade students’ science process skills and epistemo-
logical beliefs in science. Education Dissertations. 38. Retrieved from http://repository.wcsu.edu/cgi/viewcontent.cgi?artic
le=1039&context=educationdis.
Received: June 12, 2017 Accepted: September 12, 2017
Zahida Parveen PhD Special Education, Assistant Professor, University of Education,

Lahore, Pakistan.
E-mail: zahidaparveendr@gmail.com
Website: www.ue.edu.pk
732
TACTIC FULFILLMENTS OF
THREE CORRELATIONS
FOR PROBLEM-SOLVING ISSN 1648-3898 /Print/
MAPS AND ANIMATED ISSN 2538-7138 /Online/
PRESENTATIONS TO ASSESS
STUDENTS’ STOICHIOMETRY
PERFORMANCES
Abstract. This research focuses on stu-

King-Dow Su dents’ tactic assessments of 3 correlations
with illustrated 2-step strategic map and
animated presentations in stoichiom-
etry. The participants were divided into
2 learning groups: the experimental and
the control group by quasi-experimental
approach. Most of all, experimental group
participants have been contingent on
Introduction critical performances of thinking capabili-
ties in accordance with tactic fulfillments
An elaborated constituent of three correlations is designed to integrate of 3 correlations helpfully. In contrast with
problem-solving skills into tactic fulfillments of learning assessments. Tactic statistic findings of post-tests, experimental
fulfillments of three correlations, illustrated in students’ mental thinking group participants get more formulated
of algorithmic (ALG), lower-order cognitive skills (LOCS), and higher-order goals of individual learning in answer-
cognitive skills (HOCS) could contribute much to essential learning objec- ing 5 algorithmic and conceptual pairs
tives for college students’ problem-solving implements (Cracolice, Deming of achievement test items than those of
& Ehlert, 2008). The importance of the three correlations, an explicative control group participants. Students’ shift
mental mode from the basic algorithmic proficiency developed into in of reasoning from model-based to case-
more promoting thinking and elaborated critical learning, is evident in based requires them to explore an effective
constructing students’ performances of problem-solving maps and ani- mental cognition and responses for full-
mated presentations. As scholars’ implementation of problem-solving de- fledged performances of problem-solving
velopments will provide students with independent thinking and effective skills. After tactic fulfillments for 3 study
chemistry learning performances, it is critical to assess their competences objectives of visualized developments,
for cognitive creativity and learning potentialities (Barak, 2012; Lazakidou experimental group participants have
& Retalis, 2010; Zoller, Dori & Lubezky, 2002). To develop algorithmic learn- upgraded a distinct learning accumulation
ing and cognitive skills for problem-solving fulfillments is urgently critical and project-based feedback at their best
(Cooper, et al., 2008; Selvaratnam & Canagaratna, 2008) as exemplified in learning performances.
the following introductory developments for participants to assess their
Key words: problem-solving, stoichiom-
role in the tactic-based approach of the three correlations.
etry, three correlations, tactic fulfillments.
Up to now, many stoichiometry problems have still remained a dy-
namic challenge for academic studies and college students’ learning per-
formances (Sanger, 2005; Tang, Kirk & Pienta, 2014). Because students’ con-
frontations with problem-solving often result in achieving the algorithmic King-Dow Su
understanding of chemistry more readily than the conceptual proficiency Hungkuo Delin University of Technology, &
of chemistry, their acquirement of problem-solving skills would narrow Chung Yuan Christian University, Taiwan
the gap between interactive processes and learning objectives (Fischer
et al., 2014; Nakhleh, Lowrey & Mitchell, 1996; Sanger, 2005). Instructors
733
(P. 733-745) ISSN 2538–7138 /Online/
could assess students’ chemistry performances through constructing the newly-designed fulfillments of the
three correlations to be implemented from algorithmic proficiency to critical conceptualization for an in-depth
chemical conceptual understanding. The current motivation to integrate the three correlations, ALG, LOCS, and
HOCS into tactic fulfillments of major thinking skills would clarify students’ concepts, principles, laws, and as-
sessments toward problem-solving understanding in chemistry (Domin & Bodner, 2012; Sanger & Phelps, 2007;
Sanger, 2005; Sanger, Campbell, Felker & Spencer, 2007).
As in many cases of students’ dilemma learning, when they had done their best to finish all the exercises, yet
they still confined in depression not to construct authentic chemistry knowledge (Siburt, Bissell & Macphail, 2011).
For instance, stoichiometry problems are difficult for students to solve, not only in making balance for chemical
equations, dimensional analysis, and conversion between moles and grams but also in building mathematical
operations based upon comprehensive understanding (Wagner, 2001). Students’ failed responses to abstract
and complicated problems in stoichiometry (Davidowitz, Chittleborough, & Murray, 2010) might be attributed
to their lack of authentic understanding for fundamental chemistry concepts and tactic assessments (Sanger,
2005; St Clair-Thompson et al., 2012; Fischer et al. 2014; Tiruneh, Verburgh, & Elen, 2014; Tiruneh et al. 2016). To
search for the educational perspective of building-up cognitive skills, scholars have offered several academic
researches to strengthen the importance of college students’ problem-solving performances. These functional
research skills include tactic assessments with integrated communication technologies (Su, 2008a, 2008b), two-
step strategic map (Selvaratnam & Frazer, 1982; Selvaratnam & Canagaratna, 2008; Su, 2016), problem-solving
maps (Liu, Lin & Tsai, 2009; Selvaratnam & Canagaratna, 2008), and chemical dynamic reaction figures (Schultz,
2008; Su, 2013). All propounded visual models and tactic implementation of the above researches can help build
up students’ constructive learning of problem-solving developments step-by-step (Cracolice, Deming & Ehlert,
2008; Overton & Potter, 2008; Siew & Mapeala, 2016).
From the review of previous main researches, students’ mastery of tactic assessments imposes a coherent
conduct upon the three correlations in accordance with their empirical problem-solving skills and presentations.
The accumulations of two group participants’ problem-solving performances are conceived as the requisition
of tactic assessments related to cognitive levels of conceptual understanding. According to students’ creative
abilities, this research sets up to emphasize tactic assessment of the three correlations; namely, there are cor-
relations toward algorithmic (ALG), lower-order cognitive skills (LOCS) and higher-order cognitive skills (HOCS),
which serve to widen students’ conceptual learning for assessing their problem-solving performances. The shift
of reasoning from model-based to case-based allows students have the mental acquisition to explore their three
correlations in the full-fledged performances of problem-solving skills. Among other things, most students’
mind perceptions towards different visual models acquired from two-step strategic map and communicative
animations will be full of individual performances in junction with tactic assessments extended to the three
correlations of ALG, LOCS and HOCS. Consequently, this research aims at the guidance of student-centered as-
sessments in stoichiometry problem-solving.
Objectives of the Research
From empirical tactic assessments, students’ mastery of the three correlations strengthens their learning
objective and curriculum engagement in the acquisition of cognitive knowledge. How can students’ cognitive
learning be guided from model-based reasoning to case-based reasoning through effective provable set-up? To
adopt students’ ALG, LOCS and HOCS problem-solving assessments, this research sets up three functional objec-
tives of stoichiometry learning performances in the following way:
1) To visualize college students’ mental cognition for building individual learning achievements of ALG,
LOCS and HOCS;
2) To propose experimental group students’ role engagement throughout detailed applications of two-
step strategic map and animated presentations;
3) To coordinate experimental group students’ case-based learning attitude for externalizing their mental
feedback related to learning objectives.
734
ISSN 1648–3898 /Print/ TACTIC FULFILLMENTS OF THREE CORRELATIONS FOR PROBLEM-SOLVING MAPS AND
ISSN 2538–7138 /Online/ (P. 733-745)
Research Questions
The motivation of this research was guided by three major research questions: (1) What individual learning
achievements could be attainable for students’ best mental cognition? (2) How did students apply role engagement
to upgrade tactic fulfillments after their cognitive response of the three correlations to two-step strategic map and
animated presentations? (3) What case-based learning attitude was best contrived to facilitate experimental group
students’ feedback in their acquired knowledge of problem-solving stoichiometry? This research was credited with
assessing students’ mental cognition for individual learning achievements of ALG proficiency and LOCS and HOCS;
in other words, after examining their cognitive response to two-step strategic map and animated presentations,
students’ reasoning approach has been shifted from model-based reasoning to case-based reasoning to arrive at
the manipulated tactic fulfillments of both the second and the third objectives.
General Background
Students’ cognitive learning could be best accentuated upon their constituents of problem-solving skills
shifting from model-based reasoning to case-based reasoning. Christian & Talanquer (2012) gave an expository
approach manifested for students of undergraduate program. Their research assessed different reasoning in
which students solved the problems in firstly constructing models of the situations in a problem and then using
their experiences and performances to solve the problem in case. Both two-step strategic map and animated
presentations of stoichiometry were constructed and designed operationally as students’ tactic assessments for
the three correlations of ALG, LOCS and HOCS in this research. The research methodology focused on students’
problem-solving performances in the inquiry of visual models and feedback attitude of individual learning. This
research was mostly based on observations of students’ cognitive understanding rather than the simple empirical
formula assessments. It was predicated that quasi-experimental method had been planned to examine two group
students’ differential learning performances after the tactic assessment process.
Context and Participants
The selection of context and participants were implemented with a quasi-experimental approach to assess
problem-solving performances in stoichiometry for two group students – the experimental group and the control
group. The practicality of choosing with accessible participants was considered in this research as a two-stage
selection; and the eligible participation was also selected out with the careful qualification test. The participation
was limited due to the aim of research requirements and guided texts of the selection test. First of all, the research
participants were considered of 165 engineering students from Taiwan Technology College and chosen from the
pilot study for the first stage of qualification tests with the pre-knowledge of basic chemistry. From the next pro-
cess of sample participants, no less than one third of students had passed the second trial in their accumulated
knowledge of general chemistry. Up to the end of the final qualification, only a small number of 47 students got
average grade scores from A- to B+ with more advanced chemistry performances for participant tests as the same
required sample of previous research by Su (2016).
To fit in more detailed requirements of research visual models, this research divided qualified participants
into two groups to take part in the following experimental assessment. The experimental group (24 students) was
instructed with strategic applications including both the two-step strategic map (seen in Figure 1) and animated
presentations of stoichiometry (seen in Figure 2). The control group (23 students) was taught with traditional text
teaching methods without any assistance of visual models. All these qualified participants were clearly aware that
their empirical work would conform to ethical precaution in so far as secrecy and publication code is concerned
(Taber, 2014). The students from the two eligible groups, who had finished the pre-tests individually, had to take
the next experimental assessments. The students of both groups were asked to complete the same syllabus in
one-semester and the same stoichiometry instructor. During their whole learning process, the experimental group
students were required to do post-tests and learning attitude questionnaires. There was no need for the control
group students to go into the experimental teaching of visual models after doing their detailed post-tests without
any questionnaires.
735
(P. 733-745) ISSN 2538–7138 /Online/
Guided Learning Program of Visual Models
Two representational visual models were integrated into the main guided learning program for students to get
involvements of the two-step strategic map and animated presentations. Students’ performances were accessible
in their constructing exercise of mind through model-based reasoning to case-based reasoning. Their resourceful
mind might be applied to visualize things together for a model-based reasoning for the first stage construction of
problem-solving. It worked out as a creative agency to solve the imposed case reasoning in an analytical way for
the second stage construction of problem-solving. Students’ ingenious mind might do wonder with the shifting
from model-based reasoning to case-based reasoning as the following way:
The first representational visual model -- the two-step strategic map was based on Ausubel’s constructive
learning theory (2000) to set up a new approach of two-step strategic map (shown in Figure 1) for students’
constructing two-fold visual learning framework. The initial procedure of the two-step strategic map was derived
from the author’s design of the overall step-by-step construction. To guide students follow up the whole visual
models, this research started with the step one for the finding of weight product (Wproduct), through conversion
factors to figure out the exact calculation of the rated proportion for Mwproduct x Wreactant/Mwreactant (judged from the
known statement that Wreactant is the weight of reactant, Mwproduct is the molecular weight of product and Mwreactant
Figure 1: Two-step strategic map of constructive problem-solving in stoichiometry solutions.
736
ISSN 2538–7138 /Online/ (P. 733-745)
is the molecular weight of reactant). Thus, the experimental group students went on the calculation of the step
two for the ratio number of mole from product over reactant (nproduct/nreactant) at class (shown in Figure 1). After the
completion of figuring out the correct rated proportions for both Mwproduct x Wreactant /Mwreactant and nproduct/nreactant ,
students could unmistakably point up the required answer of Wproduct as judged from the equation of nproduct/nreactant
=(Wproduct/Mwproduct) / (Wreactant /Mwreactant).
There were at least two advantages for students’ manipulation of the two-step strategic map for an ap-
propriate equation in stoichiometry. First of all, students had to exert themselves to the utmost for drawing out
the required strategic map associated with each unknown qualities in an equation as known qualities. Secondly,
students’ presentation of the two-step strategic map extended their efficient operations of acting mind to capture
abstraction in the problem-solving skills of stoichiometry conceptions as claimed by scholars to reduce cognitive
load (Sevian et al., 2015). To put these advantages into practice, the two-step strategic map offered an inspired way
to assess students’ cognitive learning of complex concepts in effective guidance for constructing problem-solving
abilities as to how students organized stoichiometry information, and to what extent students’ dynamic concept
reasoning were imposed as the shift from model-based to case-based reasoning.
The second representational visual model -- animated presentations of stoichiometry could be best exemplified
on the authoritative expositions of Paivio’s Dual Coding Theory (1971, 1991) and Mayer’s cognitive theory of multi-
media learning (2009). These animated visual models not only promoted students’ cognitive learning connected with
macroscopic symbols but also provided students’ visualized perception forged with the microscopic understanding
of the spontaneous Zn-Cu2+ reaction (shown in Figure 2). In a similar executive way, college students could take the
correlated stoichiometry reaction as an ever-increasing performances insomuch their animated visual models up
to the above distinguished level for the conceptual attainment. The importance of these animated presentations
of stoichiometry was evident in encouraging students’ verbal and visual performances of designed guidance; that
is to say, its tutorial goal was to make up the learning environment, to activate students’ mental cognition, and to
conceptualize learning components of animated stoichiometry.
The best approach to comprehend the context design of Figure 2 is for students to visualize learning envi-
ronment in a series of step-by-step animated presentations of stoichiometry. Figure 2 involves a series of visual
animations of stoichiometry in the following process (from Figure 2a to Figure 2e). Students’ attention will be in
the subsequent shifting of key module designs corresponded to problem-solving skills. All their visualized pre-
sentation gives a direct link of cognitive process in animated module deigns. It starts with the guided ongoing
statement: when a piece of zinc bar was dipped into a cupric sulfate solution (Figure 2a), initially zinc atoms (Zn)
were oxidized to zinc ions (Zn2+) and at the same time gradually cupric ions (Cu2+) were reduced to red metallic
Cu, the overall kinetic reactions demonstrated in Figure 2 from (a) to (e). Furthermore, students will comprehend
step-by-step the dynamic correlation of stoichiometry in the particulate nature matter through the visualized
process of animated presentations. Here, students’ attention will be shifting to an answering module to clarify
the abstract code of stoichiometry relationship between Zn atoms and Cu2+ ions. The whole process of animated
models became an expositive way to incorporate previous visual findings into the correctly balancing chemical
equation for the product ratio numbers of moles (nproduct) Zn2+ or Cu and the reactant numbers of moles (nreactant)
Zn or Cu2+. The two half reactions of the next contextual design stimulated students to shift their conceptualized
understanding from model-based reasoning to case-based reasoning.
a) b)
737
(P. 733-745) ISSN 2538–7138 /Online/
c) d)
e) f)
Figure 2: Animated presentations of visual mode for quantitative reactions shown in sequence from (a) ani-
mated map to (f) animated map, as treated by Flash MX (Macromedia, Inc.).
It would help students to combine their case-based reasoning with the attainable fulfillments of visual
modules to engage in an innovative integration of cognitive strategic map and animated learning as shown in
Figure 1 and Figure 2. For an acceptable cased-understanding of Figure 1, an account of two advantages for the
two-step strategic map was needed to evaluate students’ cognitive learning of complex concepts. So far as Fig-
ure 2 was concerned, these animated representational visual models required students to activate their mental
cognition with conceptualized understanding and increased stoichiometry performances. With the case-based
guidance, students would step by step manipulate words, pictures, images, animations, identify cognitive skills
and algorithmic learning, and construct their individual response of activating the two-step strategic map as an
expedient application of case involvements. Obviously, students could simultaneously reduce cognitive load of
stoichiometry equation in their link of macroscopic, particulate nature and symbol, which all related to construc-
tive and expressive changes in their learning performances (Treagust, Chittleborough & Mamiala, 2003; Jaber &
BouJaoude, 2012). Judged from the above case-based reasoning, three levels of students’ ALG proficiency, LOCS
and HOCS could be an important assessment for learners’ response of 5 algorithmic and conceptual pair questions
to indicate a promising perspective in understanding stoichiometry.
Instruments
Students’ achievement pre-tests and post-tests were assessed and scored in accordance with 5 algorithmic
and conceptual pair questions for their ALG proficiency, LOCS and HOCS conceptual understanding of stoichiom-
etry. The draft achievement tests were developed with research resources of Sawrey (1990), Nakhleh (1993) and
general chemistry textbooks. The draft tests were scrutinized and revised by five renowned chemistry professors for
the validity of final achievement tests. Students’ reliability of achievement tests was analyzed by the Cronbach’s α
738
ISSN 2538–7138 /Online/ (P. 733-745)
statistical methods. The final scores for the Cronbach’s α coefficient were given as 0.790 and 0.800 in both pre-tests
and post-tests respectively. DeVellis (1991) regarded the 0.70 reliability as the minimum acceptable reliability. The
same reliability was carried out between two time span of the pre-tests and post-tests so as to detect differences
in students’ learning performances.
Students’ attitude questionnaire was assembled for developing experimental group students’ strategic feed-
back in their process of learning stoichiometry. The whole framework of students’ attitude questionnaire involved
several factors, such as content validity, constructive validity, and internal consistency reliability, which had been
elaborated in the previous publication (Su, 2016) and briefly described in the following way. Firstly, this research
asked advice from six senior chemistry educators to preview the survey draft and revised the final versions for the
content validity of pilot tests. As for students’ second factor of constructive validity, which was based on the signifi-
cant assessment of Bartlett spherical investigation, this study proposed that five aspects of the Eigenvalue were all
above 1.0 by factor analyses. All these five aspects could be defined as students’ dependent variables of learning
attitude; there were S1 (toward representational design technologies courses), S2 (toward science technologies
instructors), S3 (toward students’ interests of participation), S4 (toward self-evaluation), and S5 (toward statistical
results). From the results of internal consistency reliability, it was clear that the total scale score of the Cronbach’s
α 0.92 reached the satisfactory degree in accordance to students’ learning attitude (Gay, 1992).
Data Analysis
Aspects of data analysis were categorized by students’ acquisition in their tactic fulfillments of the three
correlations. In their answering to 5 algorithmic and conceptual pairs of achievement test items, students were al-
lotted with 20 points for the correct answer to each paired test item, with 10 points for the correct answer to each
half paired test item, with totally 100 points for the correct answer to total 5 paired test items. For this research with
a solid emphasis on students’ guided learning programs of visual and case models, it was important to perform the
related data and statistical analysis on SPSS 22.0 Windows software. The statistical findings would be of interest for
our discussion such as students’ achievement covariance and t-tests of both pre-tests and post-tests, answering
rate for the three correlations, students’ learning attitude, and one-way ANOVAs.
Results of Research
With the assumed approach of quantitative analysis, this research dealt with students’ tactic fulfillments to
interpret their shifting performance from model-based reasoning to case-based reasoning. All students’ data
performances in both visual and textual presentations included students’ achievement covariance and t-tests of
both pre-tests and post-tests, answering rate for the three correlations of ALG, LOCS and HOCS, students’ learning
attitude, and one-way ANOVAs.
Analyses of Achievement Tests
In response to research question 1, what individual learning achievements could be attainable for students’
best mental cognition? The most appropriate stimulus was to conduct students’ pre-knowledge in conforming to
accepted statistical assessments for the research question 1. Students’ learning achievements were documented
and analyzed by means of pre-tests and post-tests; the means and standard deviations were calculated by descrip-
tive statistics brought about by stoichiometry. The descriptive results of students’ learning achievement showed
pre-test mean values (25.42 and 26.09), post-test mean values (43.33 and 36.13), pre-test standard derivation values
(4.76, and 4.16), and post-test standard derivation values (10.90 and 11.16) in both group students’ stoichiometry
learning. In the present analyses of quantitative tests, it was applied to regard pre-test data as covariate vari-
ables, post-test data as dependent variables, and divided student groups as independent variables. This research
considered that no significant differences (p= .052) existed between two group students by the homogeneity
examination of regression slope. Thus, when used in the context of covariate variables analyses, an existing result
of Table 1 indicated that there were significant differences in post-test achievements between the control group
and experimental group students.
739
(P. 733-745) ISSN 2538–7138 /Online/
Table 1. Stoichiometry comparison of mental cognition for students’ individual learning achievement in
ANOVAs of post-tests.
Source SS df MS F –ratio p-value f
Group 1689.656 1 1689.656 23.000 < .001 0.722

Error 3232.416 44 73.7464
Figure 3: Students’ achievement t-tests of both pre-tests and post-tests between the experimental group and
the control group students.
To inspire more students to get involvements in active mental cognition, this research scrutinized an exem-
plified indication of superior scores for experimental group students who got beyond the scores of the control
group students as indicated by adjusted post-test mean scores. The experimental effect size, f value .722, served
to examine the inquiry research of above indication in a larger above Cohen’s (1988) effect size (f > .4). Compared
to students’ directly or indirectly testing figures, all students’ achievement t-tests (see Figure 3) turned out to be
more significant differences (t=4.796, p<.001) between the experimental and the control group students. Students’
individual mental learning achievements underwent a compared shift from model-based reasoning to case-based
reasoning in their answering test items of 5 algorithmic and conceptual pairs. Accordingly, all the three above
research programs of adjusted post-test mean, experimental effect size and students’ achievement t-tests were
verified with experimental group students’ tactic fulfillments of problem-solving maps and animated presentations.
Students’ Answering Rate for the Three Correlations
In response to research question 2, how did students apply role engagement to upgrade tactic fulfillments
after their cognitive response of the three correlations to problem-solving maps and animated presentations?
Since students’ cognitive response was essential to their role engagement for more prevailing performances,
this research made a full discussion of students’ answering rates in relation to the three correlations – ALG, LOCS,
and HOCS. As indicated in the above discussion, the experimental group students got more superior learning
achievements than those of the control group. Insomuch the present answering rates (%) and item numbers (ns)
in Table 2 would be limited to statistical analyses of the experimental group students. It was observed that the
experimental group students had more accurate answering rates and item numbers (23.4% corresponding to 28ns,
12.3% to 12ns, and 16.0% to 16ns respectively) in post-tests based on their ALG, LOCS and HOCS three correlations
than those set of the pre-tests.
740
ISSN 2538–7138 /Online/ (P. 733-745)
It was highlighted in this research that statistical analyses of students’ answering rates provided the necessary
research-based support in their fulfillment of the three correlations. In contrast to their pre-tests, the experimental
group students assumed higher scores of answering rates 23.4% ALG, 12.3% LOCS and 16.0% HOCS in their three
correlations of post-tests. With an emphasis on the algorithmic questions, the highest scores indicated students’
involvements of high proficiency to solve stoichiometry questions. Another post-test conducted on the HOCS ques-
tions was correlated with the lowest scores of students’ response for higher-order thinking in their inquiry-oriented
skills (Barak, Ben-Chaim & Zoller, 2007). Thus, the experimental group students performed consistently on each post-
test of the three correlations in the following order: ALG> LOCS> HOCS with a consequent development of critical
thinking capabilities in Table 2. The target programs of the three correlations were accredited with an important
impetus for instructors to observe the significant differences between pre-tests and post-tests in stoichiometry.
The experimental group students were required to develop overall capabilities of self-reflective assessment,
utilizing tactic fulfillments of stoichiometry texts with systematic knowledge structure and repeated animation
presentations in class. Presentations of tactic fulfillments stoichiometry text not only accumulated students’
macroscopic learning understanding of chemical reactions, but also strengthened their case-based reasoning in
microscopic particulate conceptions. All students’ answering rates for the three correlations, ALG, LOCS, and HOCS
statistical analyses of post-test scores’ covariance proved to have more effective influence on students’ learning
achievements.
Table 2. Experimental group students’ average percentages of correct answering rate (%) and test items (ns)
between pre-tests and post-tests.
Correct pairwise Pre-test Post-test

comparisons ALG LOCS HOCS ALG LOCS HOCS
Rate (%) 30.8 14.7 10.5 50.2 22.0 14.5

Items (ns) 74 35 25 120 53 35
Students’ Feedback and ANOVAs
In response to research question 3, what case-based learning feedback would be the most appropriate at-
titude for experimental group students’ acquired problem-solving fulfillment in stoichiometry? There were three
comparative case dispositions of tactic fulfillments indicated by students’ five attitude subscales in Table 3. Since
experimental group students had to carry out the same learning attitude survey, the blocking variable was confined
to tactic case samples conducted with a series of ANOVAs. The design of variance factors, F-ratio, p-value, f and
Schěffè in Table 3 was listed in different variants to determine students’ learning attitude. All significant effects were
tested to determine students’ case dispositions toward stoichiometry learning. The required effect sizes ranging
between .44 and .65 demonstrated a larger than large level (f >.4, Cohen, 1988). To some extent, the effect sizes
conducted within experimental group students’ case samples suggested a configured index to detect students’
different variants in learning attitude.
Another noticeable feature for students’ dependent variables marked a set of survey to conduct compara-
tive analysis of individual learning attitude in ANOVAs. The acquisition of dependent variables was designed and
constructed in the following three comparative case dispositions. Initially, the first comparative case disposition
for students’ dependent variables S1 to S5 illustrated more “positive” attitudes than those reporting “negative” in
Schěffè’s post hoc. Next to be discussed, the second comparative case disposition for students’ dependent variables
S1 to S3 reflected more “neutral” attitudes than those reporting “negative” in Schěffè’s post hoc. And subsequently
the third comparative case disposition for students’ dependent variables S4 displayed more “positive” attitudes than
those reporting “neutral” in Schěffè’s post hoc. The above three comparative case dispositions assisted students to
acquire more individual feedbacks of learning attitude in stoichiometry.
741
(P. 733-745) ISSN 2538–7138 /Online/
Table 3. Three comparative case dispositions of individual learning attitude in ANOVAs.
Experimental Blocking Analysis of Attitude Measure

Course Variable Variance S1 S2 S3 S4 S5
Stoichiometry Disposition F-ratio 7.498 3.553 6.476 4.652 6.243

toward p-value .002** .039* .004** .016* .005**
Chemistry
(positive, neutral, f 0.65 0.44 0.60 0.51 0.59
negative) Schěffè 1>3, 2>3 1>3, 2>3 1>3, 2>3 1>2, 1>3 1>3
*p< .05; **p< .01
Synoptically, the above results of quantitative analysis elevated students’ case encountering of conceptual
understanding and algorithmic proficiency in 5 paired items to develop learning performance. By contrast, it
was not restricted that experimental group students would limit their individual learning achievements for an
effective tactic fulfillment of the three correlations as those of control group students in answering achieve-
ment test items.
A close look at experimental group students’ answering rate with a consequent development of critical
thinking capabilities (ALG > LOCS > HOCS in Table 2) evoked their cognitive response from model-based to case-
based reasoning which reflected more effective influence on their role engagement of tactic learning achieve-
ments. After having taken up three comparative case dispositions of learning attitude through the learning
application, experimental group students could create an environment in facilitating their tactic fulfillments
and upgrade case-based feedback for accumulating learning performance on a whole.
Discussion
The premise in this research to set a priority on students’ engagement of learning achievements requires
an integrated module for effective tactic fulfillments of the three correlations. As in earlier researches of effec-
tive problem-solving approach (Barak, 2012; Lazakidou & Retalis, 2010), instructors not only needed to design
problem-solving maps of mental cognition knowledge, but also inspired new knowledge by model-based
reasoning for college students’ tactic fulfillments. Students could make potential implementations of construc-
tive thinking and independent learning based on effective tactic fulfillments for various cognitive competences
of problem-solving skills in stoichiometry. Accordingly, improving students’ engagement of tactic fulfillments
in previous research about macroscopic, particulate nature and symbol (Treagust, Chittleborough & Mamiala,
2003; Jaber & BouJaoude, 2012), would be available for students’ encounter of cognitive case learning and
achievements of constructive stoichiometry.
This research highlights the new assessment of the three correlations for students’ tactic fulfillments of
model-based reasoning. It justified an important impetus for instructors to notice students’ significant different
achievements between pre-tests and post-tests in stoichiometry. Since experimental group students were ex-
pected to develop self-reflective capabilities, the agglomerated results of processes through which model-based
reasoning was gathering to utilize systematic knowledge structure and repeated animation presentations with
the cognitive assessment embedded in ALG, LOCS, and HOCS. Serial presentations of tactic fulfillments enabled
students to accomplish cognitive achievements for macroscopic contexts of chemical reactions, in accordance
with microscopic particulate conceptions for conceptual visualized instructions at class. Among many recipients,
students’ answering rate for the three correlations, ALG, LOCS, and HOCS, testified by many pedagogic chemical
researchers (Domin & Bodner, 2012; Sanger & Phelps, 2007), together with statistical analyses of post-test scores’
covariance, and the tactic fulfillments of learning applications witnessed an influential reasoning on students’
case-based learning.
Students’ feedback served as a prevailing intensification of case-based reasoning for their admirable
learning attitude in the field of cognitive developments. After their cognitive encounter with learning environ-
ment, experimental group students had to participate in the survey of individual learning attitude illustrated in
their three case dispositions towards stoichiometry. On their account of dimensional analysis of stoichiometry,
most experimental group students would figure out the conversion factors for a turning point to describe their
learning attitude with positive disposition. Followed by conversion factors, they would be aware of other rel-
evant factors involved in their case-based illustration of stoichiometry. Without the employment of conversion
742
ISSN 2538–7138 /Online/ (P. 733-745)
factors, the control group students had only fragmented encounter in their dilemma of stoichiometry cognitive
developments. In fostering more affiliated learning attitude with positive cased-disposition, students would
construct multiple learning competences in the ultimate goal to reduce their learning cognitive load as sug-
gested by researchers (Sevian et al., 2015).
To search for students’ tactic fulfillments, the incorporated alignment of the three correlations and self-
performed feedback awakened experimental group students’ cognitive need in attentive co-work relationship.
The goal of tactic fulfillments in this research was not restricted to group students’ individual adjustment of
stoichiometry knowledge from model-based conception to case-based perception. Aided by three correlations
ALG, LOCS, and HOCS, students could develop different talents, which lead to eminence in their construction of
learning performances. After their acquisition of extensive problem-solving skills, students set upon measur-
ing voluntary feedback prescribed with positive evaluation behaviors befitting singly or collectively. Despite
limitations of the sample size in the first draft, this research was supplemented by students’ group qualification
of two-tier item tests. Common to more longitudinal academic developments, students’ independent variants
conveyed the scrutinized tactic fulfillments of learning attitude to evaluate strategic problem-solving contexts.
It was crucial to bringing about both a positive result and significant contributions that group students’ ac-
complishments corroborate their step-by-step development of reasoning skill, visual presentations and critical
thinking in their learning process.
It would be not an easy task to transfer all specific domains of problem-solving abilities into students’
tactic application during a limited time span; therefore, this study took an eclectic measurement exemplified
from model-based to case-based reasoning for them to explore their mental cognition through two-step stra-
tegic map and animated presentations with ALG, LOCS, and HOCS. Students’ commitments from model-based
to case-based reasoning have put up tactic fulfillments of the three correlations for many pedagogic chemical
researchers under a substantial critical thinking (Domin & Bodner, 2012; Sanger, 2005; Sanger, Campbell, Felker,
& Spencer, 2007; Sanger & Phelps, 2007). The findings within this research would offer a new perspective and
be of interest to both school and college teaching instructors.
Conclusions
The perspective of this research promises a new insight into enhancing students’ stoichiometry performances
up to the accomplished levels of tactic fulfillments and the cognitive context of the three correlations. Visualized
developments of problem-solving maps and animated presentations can get into students’ thoroughly learning
activities for their inquiry of tactic fulfillments. The prevailing three correlations involved students’ responsiveness
of critical mental thinking with forcible tactic fulfillment in ALG, LOCS, and HOCS. An ideal research for investigating
the meaningful engagement of students’ learning performances enabled the recipients to accomplish a specific
mental adjustment which went beyond traditional abstract and difficult stoichiometry learning. With an aid of vi-
sualized presentations in this research, students demonstrated greater immediacy for conceptual delineations and
problem-solving competency. More motivations were attributed to students’ corresponding results to paired-test
items as well as students’ engagements on the entire learning progress. Also students’ positive learning feedback
was explored consistently to different thinking levels set in similar contexts of learning performances.
The guidance for students’ tactic fulfillments was not a one-way destination, but a joint effort between
instructors and students in discovering and conducting effective operations through thinking skills of problem-
solving performances. This research made an important implementation to detect students’ different cognitive
understanding as a compatible fulfillment with students’ validity of the three correlations represented in their core
stoichiometry achievements. Illustrated fulfillments of students’ ALG, LOCS and HOCS’s mental thinking served to
widen the horizon of their cognition to take more participation of problem-solving. The participants in the survey
selected from the two-stage qualification test reached up to a valid indication examined continuously before, dur-
ing, and after the feedback. A limited assumption was given to the experimental group students for the regular
curriculum after one year span of the intervention. Although all findings and results presented a sound base for
further researches on how to assess students’ cognitive learning and visual models into problem-solving perfor-
mances, some extended and longitudinal studies still remained to be addressed in the future refinements. Thus,
the further explorations of students’ problem-solving skills observed in the present research will be suggested to
conduct and analyze through more tactic development of students’ participation and assessments.
743
(P. 733-745) ISSN 2538–7138 /Online/
Acknowledgements
The author would like to thank the Ministry of Science and Technology (MOST) in Taiwan. Without their help
and financial support (under Grant No. MOST 105-2511-S-237-002-MY2 and NSC 98-2511-S-237-001), this research
could not have been completed in the present form. Finally, thanks must also be given to all the instructors and
students who gave a cooperative participation in whole pre-tests, post-tests and feedback.
References
Ausubel, D. P. (2000). The acquisition and retention of knowledge: A cognitive view. Dordrecht, The Netherlands: Kluwer Academic.
Barak, M. (2012). Impacts of learning inventive problem-solving principles: students’ transition from systematic searching to
heuristic problem solving. Instructional Science, 7, 1-23.
Barak, M., Ben-Chaim, D., & Zoller, U. (2007). Purposely teaching for the promotion of higher-order thinking skills: A case of critical
thinking. Research in Science Education, 37, 353-369.
Christian, K., & Talanquer, V. (2012). Content-related interactions in self-initiated study groups. International Journal of Science
Education, 34, 2231-2255.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates Inc.
Cooper, M. M., Cox, C. T., Jr., Nammouz, M., Case, E., & Stevens, R. (2008). An assessment of the effect of
collaborative groups on students’ problem-solving strategies and abilities. Journal of Chemical Education, 85 (6), 866−872.
Cracolice, M. C., Deming, J. C., & Ehlert, B. (2008). Concept learning versus problem solving: A cognitive difference. Journal of
Chemical Education, 85, 873-878.
Davidowitz, B., Chittleborough, G., & Murray, E. (2010). Student-generated submicro diagrams: A useful tool for teaching and
learning chemical equations and stoichiometry. Chemistry Education Research and Practice, 11, 154-164.
DeVellis, R. F. (1991). Scale development Theory and applications. Newbury Park, California: Sage Publications, Inc.
Domin, D., & Bodner, G. (2012). Using students’ representations constructed during problem solving to infer
conceptual understanding. Journal of Chemical Education, 89 (7), 837-843.
Fischer, F., Kollar, I., Ufer, S., Sodian, B., Hussmann, H., Pekrun, R., …Eberle, J. (2014). Scientific reasoning and argumentation:
Advancing an interdisciplinary research agenda in education. Frontline Learning Research, 4, 28–45. doi:10.14786/flr.v2i2.96.
Gay, L. R. (1992). Educational research: Competencies for analysis and application (4th ed.), New York: Macmillan Publishing Company.
Jaber, L. Z., & BouJaoude, S. (2012). A macro–micro–symbolic teaching to promote relational understanding of chemical reac-
tions. International Journal of Science Education, 34 (7), 973-998.
Lazakidou, G., & Retalis, S. (2010). Using computer supported collaborative learning strategies for helping students acquire self-
regulated problem-solving skills in mathematics. Computers & Education, 54 (1), 3-13.
Liu, T. C., Lin, Y. C., & Tsai, C. C. (2009). Identifying senior high school students’ misconceptions about statistical correlation and
their possible causes: An exploratory study using concept mapping with interviews. International Journal of Science and
Mathematics Education, 7, 791-820.
Mayer, R. E. (2009). Multimedia learning. Cambridge University Press.
Nakhleh, M. B. (1993). Are our students’ conceptual thinkers or algorithmic problem solvers? Journal of Chemical Education, 70
(1), 52-55.
Nakhleh, M. B., Lowrey, K. A., & Mitchell, R. C. (1996). Narrowing the gap between concepts and algorithms in freshman chemistry.
Journal of Chemical Education, 73 (8), 758-762.
Overton, T., & Potter, N. (2008). Solving open-ended problems, and the influence of cognitive factors on student success. Chem-
istry Education Research and Practice, 9, 65-69.
Paivio, A. (1971). Imagery and verbal processes. New York: Holt, Rinehart & Winston.
Paivio, A. (1991). Dual coding theory: Retrospect and current status. Canadian Journal of Psychology, 45, 255-287.
Sanger, M. J. (2005). Evaluating students’ conceptual understanding of balanced equations and stoichiometric ratios using a
particulate drawing. Journal of Chemical Education, 82, 131-134.
Sanger, M. J., Campbell, E., Felker, J., & Spencer, C. (2007). Concept learning versus problem solving: Does particle motion. Journal
of Chemical Education, 84 (5), 875-879.
Sanger, M. J., & Phelps, A. J. (2007). What are students thinking when they pick their answer? A content analysis of students’ ex-
planations of gas properties. Journal of Chemical Education, 84, 870-874.
Sawrey, B. A. (1990). Concept learning versus problem solving: Revisited. Journal of Chemical Education, 67, 253-255.
Schultz, E. (2008). Dynamic reaction figures: An integrative vehicle for understanding chemical reactions. Journal of Chemical
Education, 85, 386-392.
Selvaratnam, M., & Canagaratna, S. G. (2008). Using problem-solution maps to improve students’ problem-solving skills. Journal
of Chemical Education, 85, 381-385.
Selvaratnam, M., & Frazer, M. J. (1982). Problem solving in chemistry. Heinemann Educational Publishers:
London.
Siburt, C. J. P., Bissell, A. N., & Macphail, R. A. (2011). Developing metacognitive and problem-solving skills through problem
manipulation. Journal of Chemical Education, 19, 1489-1495.
Sevian, H., Bernholt, S., Szteinberg, G. A., Auguste, S., & Pérez, L. C. (2015). Use of representations mapping to capture abstraction
744
ISSN 2538–7138 /Online/ (P. 733-745)
in problem solving in different courses in chemistry. Chemistry Education Research and Practice, 16, 429-446.
Siew, N. M., & Mapeala, R. (2016). The effects of problem-based learning with thinking maps on fifth graders’ science critical
thinking. Journal of Baltic Science Education, 15 (5), 602-616.
St Clair-Thompson, H., Overton, T. L., & Bugler, M. (2012). Mental capacity and working memory in chemistry: Algorithmic versus
open-ended problem solving. Chemistry Education Research and Practice, 13, 484-489.
Su, K. D. (2008a). An integrated science course designed with information communication technologies to enhance university
students’ learning performance. Computers & Education, 51, 1365-1374.
Su, K. D. (2008b). The effects of a chemistry course with integrated information communication technologies on university
students’ learning and attitudes. International Journal of Science and Mathematics Education, 6, 225-249.
Su, K. D. (2013). Validity of problem-solving skills: Exploring dynamic reaction figures in acid and base chemical learning. Journal
of Computer Engineering Informatics, 1 (1), 1-12.
Su, K. D. (2016). Strengthening strategic applications of problem-solving skills for Taiwan students’ chemistry understanding.
Journal of Baltic Science Education, 15 (6), 662-679.
Taber, K. S. (2014). Ethical considerations of chemistry education research involving ‘human subjects’. Chemistry Education Re-
search and Practice, 15, 109-113.
Tang, H., Kirk, J., & Pienta, N. J. (2014). Investigating the effect of complexity factors in stoichiometry problems using logistic
regression and eye tracking. Journal of Chemical Education, 91, 969-975.
Tiruneh, D. T., Verburgh, A., & Elen, J. (2014). Effectiveness of critical thinking instruction in higher education: A systematic review
of intervention studies. Higher Education Studies, 4 (1), 1–17. doi:10.5539/hes.v4n1p1.
Tiruneh, D. T., Weldeslassie, A. G., Kassa, A., Tefera, Z., De Cock, M., & Elen, J. (2016). Systematic design of a learning environment
for domain specific and domain-general critical thinking skills. Educational Technology Research and Development, 64 (3),
481–505. doi: 10.1007/s11423-015-9417-2.
Treagust, D. F., Chittleborough, G. D., & Mamiala, L. T. (2003). The role of submicroscopic and symbolic representations in chemical
explanations. International Journal of Science Education, 25 (11), 1353-1368.
Wagner, E. P. (2001). A study comparing the efficacy of a mole ratio flow chart to dimensional analysis for teaching reaction
Stoichiometry. School Science and Mathematics, 101 (1), 10−22.
Zoller, U., & Dori, Y. J., & Lubezky, A. (2002). Algorithmic, LOCS and HOCS (chemistry) exam questions: Performance and attitudes
of college students. International Journal of Science Education, 24 (2), 185-203.
King-Dow Su Chemistry Ph. Dr., Professor, Department of Hospitality

Management and Center for General Education, Hungkuo Delin
University of Technology;
NO.1, Lane 380, Ching-Yun Road, Tu-Cheng District., New Taipei City,
Taiwan 23646, R.O.C. & Center for General Education, Chung Yuan
Christian University, 200 Chung Pei Road, Chung Li District, Taoyuan
City, Taiwan 32023, R.O.C.
E-mail: su-87168@dlit.edu.tw, su-87168@mail.hdut.edu.tw
745
THE APPLICATION OF CONCEPT
MAPS IN THE TEACHING
ISSN 1648-3898 /Print/ OF POLLINATION AND
ISSN 2538-7138 /Online/ POLLINATORS IN ELEMENTARY
SCHOOL
Abstract. Teaching biology is charac- Jelena D. Stanisavljević,

terized by a great number of concepts Mirka М. Bunijevac,
and facts. It is very difficult to concisely Ljubiša Ž. Stanisavljević
represent all important concepts and facts.
In order to effectively present important
concepts such as pollination and pollina-
tors, concept maps have been applied. In
particular, the pedagogical experiment was
applied to determine whether the concept
maps are effective as teaching technology
Introduction
compared to the usual teaching approach
without this technology. The research in-
Teaching biology is characterized by a great number of concepts and
volved 110 elementary school students who
facts. It is very difficult to concisely represent all the important concepts and
were divided into one experimental (E) and
facts. But numerous teaching technologies can be applied to overcome this
one control (C) group. The E group covered
problem. Very often, these are different visual technologies. Among them,
programme content related to pollination
concept maps can be applied.
and pollinators by applying concept maps.
Concept maps are two-dimensional visual teaching technology through
The C group was exposed to the same
which the structure of biology programme content can be introduced
content, without these maps. It is evidenced
(Kinchin, 2011). They include concepts, usually enclosed in circles or boxes
a difference of the attained knowledge in
of some kind, their mutual relation indicated by the connecting line linking
favour of the experimental group after the
two specific concepts. The words on the line referred to as linking words or
introduction of the experimental factor (ap-
linking phrases, specify the relationships between the two concepts (Novak
plication of concept maps). The application
& Cañas, 2008).
of concept maps directly contributed to
Concept maps were created at the beginning of 1970’s as a part of a
better learning and knowledge acquisition
research project of Novak and collaborators. Within this study, the research-
in teaching the biology content Pollination
ers interviewed many children and they found it difficult to identify specific
and Pollinators. Based upon the obtained
changes in their understanding of science concepts by examining the inter-
results of this research, concept maps
view transcripts. As a result of the necessity to find a better way to represent
will be further implemented to improve
children’s conceptual understanding, there emerged the idea of representing
teaching process and the teachers will be
children’s knowledge in the form of a concept map. Thus, a new tool was cre-
gradually trained for the application of this
ated not only for the purpose of research, but also for many other purposes
teaching technology.
(Novak & Cañas, 2008). This programme was based on the theory of learning
Keywords: concept maps, programme psychology, developed by David Ausubel. According to Ausubel’s theory,
content, pollination, pollinators. learning is based on the assimilation of new concepts and assumptions in the
concept framework that the student already has. This structure of knowledge
is also referred to as the individual cognitive structure.
Jelena D. Stanisavljević, It is best to construct concept maps with reference to some particular
Mirka М. Bunijevac, question on which answer should be found. This question is focus question.
Ljubiša Ž. Stanisavljević
University of Belgrade, Serbia A concept map may pertain to some situation or event that we are trying to
understand through the organization of knowledge in the form of a concept
map, thus providing the context for the concept map (Novak, 2010).
746
ISSN 1648–3898 /Print/ THE APPLICATION OF CONCEPT MAPS IN THE TEACHING OF POLLINATION AND
ISSN 2538–7138 /Online/ (P. 746-760)
The first structure of concept maps, the so-called preliminary maps, is checked and revised at least several
more times, to assume the final form to be applied in the classroom. When reviewing draft concept maps, we
can add new concepts and observe cross-links between those concepts. Cross-links help us to see clearly how a
concept from one domain of knowledge relates to a concept from another domain on the map, thus creating the
potential for deeper thinking and finding creative solutions, and promoting more forms of mental processes and
creative thinking (Novak & Cañas, 2008).
According to the results of educational research, concept maps have several benefits for their users, as they:
•• allow students to perceive concepts, thoughts and ideas more clearly (Fisher, Wandersee, & Moody,
2000), encourage the work of both hemispheres of the brain, are useful in the development of higher
cognitive activities (analysis, synthesis, evaluation ...) (Fischer, 2009);
•• help students in creative thinking and developing new ideas, integrate and reorganize knowledge and
influence the development of critical thinking (Kinchin, 2000b; Kinchin et al., 2010);
•• serve as a tool for the evaluation of knowledge and for measuring the adopted courseware (Hay &
Kinchin, 2006). Students can independently assess the level and quality of the knowledge acquired, and
teachers may monitor and evaluate the acquisition of knowledge by students (Peters & Beson, 2010);
•• can be used for organizing and structuring the curriculum of a certain subject (Novak & Cañas, 2008),
which allows teachers to easily create and organize the curriculum, as well as connect educational
units (Loertscher, 2011).
By using concept maps at preschool age, children become familiar with the abstruse phenomena, processes
and concepts, learn how to learn, how to separate the important from the unimportant and how to explain and
relate concepts and information (Birbili, 2006).
Although the idea is to use concept maps to evaluate the progress of students in a clear and simple way,
one should be cautious with this approach as students tend to leave out some very important information when
constructing concept maps (Kinchin, 2014).
Concept maps are used to organize, plan and display information of entire programmes, modules, themes
and parts of a curriculum. Any domain of knowledge can be revised by using the maps of terms. Thus, both stu-
dents and teachers are provided the basic facts/ knowledge, having an insight into their structure and interaction
(Simon, 2009).
One of the key issues concerning the application of concept maps in teaching is the nature of the curriculum
in which they will be incorporated. The degree of freedom that should be offered to students needs to match the
degree of freedom that a student may have within the curriculum (which is especially complex in natural sciences),
in order to develop a personal understanding of the teaching material (Kinchin, 2014).
The qualitative analysis of concept maps also led to the examination of their morphology in terms of their
functionality. The different morphologies that are commonly encountered- chains, circles or networks- are asso-
ciated with orientation in learning. These structures have a role in learning and they are not mutually exclusive,
and one structure may develop into another over a certain period of time. Thus, for example, the structure of one
branch can be developed into a chain or network, as the student develops understanding in the process of further
learning. It is also clear that some structures are more or less contextually appropriate in a given situation, and
students need to create their maps accordingly (Kinchin, 2013b).
There are numerous types of software for constructing concept maps in the world, but students themselves can
develop their own style of concept maps. In view of the fact that teaching is increasingly becoming a digital “game”
in the world of Internet technologies, it is of utmost importance not to neglect or abolish the direct face to face
contact between students and teachers, knowing that we are constantly moving from the analogue to the digital
educational environment, the focus being shifted from the student to the technology used today (Kinchin, 2013a).
Concept Maps Applied to the Concepts of Pollination and Pollinators
Pollination is a fundamental, essential process in any ecosystem that allows reproduction of plants and the
production of food for humans and animals (Klein et al., 2007; Stanisavljević, 2012).
Pollination is the transfer of pollen between the male and female parts of flowers to enable fertilization and
reproduction.
Depending on the type of pollinators, the pollination process can be divided into abiotic and biotic pollina-
tion. The most important factors of abiotic pollination are water and wind. Biotic pollination means the presence
747
(P. 746-760) ISSN 2538–7138 /Online/
of another organism, which is the mediator in the process and which is related to the species pollinated. This
organism must visit the flower all the time, and such visits are therefore integrated into its daily routine. Most of
the cultivated and wild plants depend on animal vectors, known as pollinators, to transfer pollen.
In nature, animal pollination plays a vital role as a regulating ecosystem service. Worldwide, approximately
90% of wild flowering plant species depend, at least in part, on the transfer of pollen by animals (Potts et al., 2016).
Contribution to the pollination of numerous leading global food crops come from wild insect pollinators, which
include bees, flies, butterflies, moths, and beetles (Klein et al., 2007; Kleijn et al., 2015; Garibaldi et al., 2013). The
majority of animal pollinators important to agriculture are insects, of which the best known are bees (honeybees,
bumblebees, stingless bees and solitary bees) (Kevan, 2007). Birds, bats and some other mammals frequently visit
large flowers with plenty and easily accessible nectar (Groeneveld, Tscharntke, Moser, & Clough, 2010).
In any given ecosystem should be treating pollination as a complex web of interactions between plants and
pollinators, because pollination is not their simple connection. These interactions can be studied using ecological
networks (Moreira, Boscolo, & Viana, 2015).
Relations among organisms developed during evolution through various attractants (nectar, pollen, specific
odours) while others were created by accident (Pyke, 2010).
The process of pollination explains why so much energy was invested in the structure of the flower and repro-
ductive organs, the production of nectar and sexual reproduction throughout the evolution, as reproduction is of
vital importance for plants and for animals, as well as for the production of genetic variability (Berenbaum, 1995).
For the many angiosperms, reproduction depends on attracting animals that provide pollination. Flowers
offer various traits that can attract animals through visual, olfactory, and tactile signals and some of these features
may result from selection conducted by pollinations. Such plant attractants can provide information about the
presence, quality, and location of reward (Valenta, Nevo, Martel, & Chapman, 2017).
Pollinator communities may be changed in a nonrandom way, because of global change (e.g. climate,
land-use intensification and farming systems), resulting in losses of specific functional groups or species (Rader,
Bartomeus, Tylianakis, & Laliberte, 2014). Risks for food production and society is a worldwide problem coming
from loss of diversity of wild pollinators.
Important support of pollinators and pollination come from higher education and training programs for
agronomists, veterinarians, farmers, and policy-makers (Dicks et al., 2016). Education projects focused on pol-
linators and pollination that combine awareness raising with practical training and opportunity for action have
a good chance of making real behavior change, and there is direct evidence for this in a few cases. Worldwide,
there are many pollinator-focused education and outreach projects of which most are relatively new and so ef-
fects on broader pollinator abundance and diversity might not be seen yet (Dicks et al., 2016).
Based on the foregoing, it can be concluded that the concepts of pollination and pollinators are very
important in education. Considering that students should correctly understand the concepts of pollination and
pollinators, it is essential to bring these scientific concepts closer to their current understanding of this field, to
make an analogy between these concepts and botany, particularly when it comes to the parts of the flower already
familiar to students. The reciprocal links that exist between animals and plants, the fascinating and intriguing
ways of understanding the process of pollination as essential to the functioning of the ecosystem, open the door
to interdisciplinary approaches in the methodology of teaching and learning in this field. For many, pollination is
not a sufficiently and correctly understood concept, which is why it is necessary to introduce a new course on or a
model of concept maps, making special reference to this particular scientific concept (Sessions & Johnson, 2005).
Concept map can point out the analogy between pollination and pollinators on the one hand, and the parts of
the flower on the other (Appendix 1).
The terms relating to pollination can be incorrectly presented, since teachers tend to simplify them to make
them clearer to their students. Moreover, there are biology textbooks, in which bees are the only pollinators men-
tioned, while many species of insects and other invertebrates and vertebrates are omitted. Also, there is no mention
of how important the process of pollination and pollinators are for the life on Earth, as well as of how people have
harmed pollinators and plants by their actions and by using different materials, nor there is mention of what they
can do to facilitate the flow of the process of pollination.
Using concept maps in understanding the processes of pollination of plants is a very suitable model of
teaching the content related to these processes. Using a concept map the student can learn how plants attract
pollinators (Appendix 2).
Concept maps as a teaching technology can equally well be used at all levels of education. The efficiency of
748
ISSN 2538–7138 /Online/ (P. 746-760)
concept maps in teaching was shown by the results of several researches conducted at the University of Belgrade
- Faculty of Biology. It was proved that these maps had a positive effect on acquiring knowledge (Stanisavljevic &
Djuric, 2013a; Stanisavljević & Djurić, 2013b; Stanisavljević & Stanisavljević, 2014) and developing a positive attitude
in students (Djuric & Stanisavljevic, 2011).
Considering that biology (as a science and teaching) is filled of numerous concepts and facts, and that these
maps as teaching technology are extremely concise and determined, their application can present in a very clear,
precisely defined and concise manner, numerous biological concepts and facts. These maps are more concisely,
more precisely, more structured, more formal and more specific in comparing with other similar visual teaching
technologies such as mind maps (“idea mapping”) and conceptual diagrams (Davies, 2011). For these reasons, a
large amount of teaching content, in particular about pollination and pollinators can be presented concisely with
the help of concept maps, and the most important concepts and their relations (from this topics) thus can be
briefly summarized and highlighted.
Although concept maps are significant in the field of biology, they are unfortunately not widely used in the
Serbian educational system.
Based on the above, testing and verifying the application of concept maps were started in biology teaching
concerning the programme contents of pollination and pollinators in the sixth grade of elementary school.
The aim of this research was to determine whether the concept maps are effective as teaching technology
compared to the usual teaching approach without this technology, in terms of acquiring knowledge of the pro-
gramme content of pollination and pollinators in the sixth grade of elementary school.
Specifically, the intention of the research was to answer the following research question: Does the application
of concept maps in realizing programme content pollination and pollinators contribute to a greater quantity and
quality of the acquired knowledge?
It is expected that the difference in the quality and quantity of the acquired knowledge between the experi-
mental and control groups will be in favour of the experimental group. The goal is to identify and measure this
difference.
General Background
In this research, pedagogical experiment with parallel groups of students (experimental and control groups)
was applied. The data was gathered through the pre-test and post-test with the aim to determine whether the
concept maps (experimental factor) are effective as teaching technology for presenting the concepts of pollinator
and pollination compared to the usual teaching approach without this technology. The research was conducted
in April 2016 at the elementary school in Belgrade.
Sample Selection
The research included the total of 110 sixth grade students from the elementary school from Belgrade, Serbia.
In order to achieve the aim of research, the pedagogical experiment with parallel groups [experimental (E) and
control (C)] was applied (Appendix 3).
The students were grouped into Group E (N=58) and Group C (N=52) (Killermann, 1998). Before the introduc-
tion of the experimental factor (concept maps) in experimental group, the groups were made uniform concerning
the number of students, gender and general knowledge of biology, as determined by the results of a pre-test of
knowledge.
Before research was performed, for the research participants informed consent was obtained from parents,
teachers and school board, in accordance with the permission of the ethics committee of the Faculty of Biology,
University of Belgrade.
Instruments and Procedures
The pre-test included twelve tasks in total, classified into three broad categories of the cognitive domain:
Rank I (remembering and understanding), Rank II (applying and analysing) and Rank III (evaluating and creating).
749
(P. 746-760) ISSN 2538–7138 /Online/
In order to facilitate the categorization of the knowledge test, the tasks were divided based on the revised Bloom’s
taxonomy (Anderson & Krathwohl, 2001), where the cognitive domain is divided into six categories (remembering,
understanding, applying, analysing, evaluating and creating). The test is based on all programme contents preced-
ing the contents of pollination and pollinators.
After equalizing the experimental group (E) and the control group (C), Group E was taught the prepared biol-
ogy content Pollination and Pollinators by applying concept maps. After a brief oral PowerPoint presentation by the
teacher, the students were distributed partly completed concept maps (Appendix 4: 4.1., 4.2., 4.3., 4.4.), related to
the basic characteristics of pollination and pollinators. They had to fill them out (to write adequate concepts in the
blank fields) within one instruction period (Appendix 3). After the teacher’s presentation, each student from Group
E used the textbook to process the information on the topic, and, finally, filled out the concept maps.
The students in Group C were exposed to the usual teaching approach without concept maps, for the same
content; namely, the teacher used PowerPoint to present this content through the following teaching methods:
oral presentation, illustrations and demonstrations (throughout the instruction period). The teacher did ask some
questions about the content, but the discussion was short. The students from this group were mainly listening to
what the teacher was saying or watching what the teacher was showing.
The E and C groups were separated from each other. They underwent this teaching period (in the same time)
in different classrooms. Also, in order to prevent any contamination of the design, the students of the E group had
no contact with the concept maps outside of the planned periods (Kember, 2003).
After that, a post-test was distributed in order to evaluate the knowledge acquired by the students who
used concept maps and those exposed to the usual teaching approach without concept maps. This test measured
the quantity and quality of the students’ knowledge in the teaching field Pollination and Pollinators. The post-test
consisted of nine tasks (divided into three ranks/categories, as was the case in the pre-test) (Appendix 5).
Data Analysis
The data were analysed by using standard statistical indicators (sum, percentage frequency, mean, standard
deviation, coefficient of variation and Student’s t-test for testing any differences between the E and C groups) (Stu-
dent, 1908). All these analyses were conducted using the statistical software package Statistica 6 (StatSoft, 2001).
Results of Research
The results of the pre-test are presented in Tables 1 and 2. The standard statistical indicators (mean of the
number of achieved points-M, standard deviation-SD and coefficient of variation-CV) are presented in the table 1.
Table 1 shows the students’ achievement on the pre-test expressed in above mentioned terms, in all three ranks
of tasks, as well as on the test as a whole. The E group students achieved 73.98% points and the C group students
73.08% points (in total).
Table 1. Basic statistical data for the pre-test.
Group Rank I Rank II Rank III Total
M SD CV M SD CV M SD CV M SD CV
Е (N=58) 28.79 4.76 0.16 23.31 4.67 0.20 21.88 6.74 0.31 73.98 15.85 0.21
C (N=52) 28.31 4.17 0.15 23.19 4.59 0.20 21.58 6.23 0.29 73.08 14.79 0.20
Note: M-mean of the number of achieved points, SD-standard deviation, CV-coefficient of variation.
Table 2 shows relations between E and C group, according to t-value (for pre-test).
750
ISSN 2538–7138 /Online/ (P. 746-760)
Table 2. Testing group uniformity in the pre-test, using an independent two-sample t-test.
Relation Rank I Rank II Rank III Total
Е : C (t-value) 0.56 0.13 0.24 0.31
Note: t-test for significance level of p≤ .05 and critical value of t≥1.96.
Based on the results presented for the pre-test for Groups E and C, it can be observed by using Student’s
t-test for a significant level of p=.05 and a critical value of t=1.96, there is no statistically significant difference in
the achieved number of points between Groups E and C in all three ranks of tasks and in the test as a whole (Rank
I: t=0.56<1.96; Rank II: t=0.13<1.96; Rank III: t=0.24<1.96; Total: t=0.31<1.96). These two groups were balanced in
terms of their general knowledge of biology before the introduction of the experimental factor (concept maps).
The results of the post-test are presented in Tables 3 and 4. Table 3 shows the students’ achievement on the
post-test expressed in above mentioned statistical indicators (M, SD and CV), in all three ranks of tasks, as well as on
the test as a whole. The E group students achieved 83.72% points and the C group students 73.56% points (in total).
Table 3. Basic statistical data for the post-test.
Group Rank I Rank II Rank III Total
M SD CV M SD CV M SD CV M SD CV
Е 33.24 1.84 0.56 26.60 2.66 0.10 23.88 6.47 0.27 83.72 10.83 0.13
C 30.73 3.58 0.12 23.31 4.36 0.19 19.52 6.43 0.33 73.56 14.07 0.19
Note: M-mean of the number of achieved points, SD-standard deviation, CV-coefficient of variation.
Table 4 shows relations between E and C group, according to t-value (for post-test).
Table 4. Testing group uniformity in terms of the post-test, using a t-test.
Relation Rank I Rank II Rank III Total
Е : C (t-value) 4.48 4.70 3.52 4.16
Note: t-test for significance level of p≤ .05 and critical value of t≥1.96.
By comparing the average values of the results achieved, a clear difference can be observed between Groups
E and C in terms of the individual ranks and in the test as a whole, favouring the former. On the basis of the results
presented for the post-test of knowledge for Groups E and C (Table III and IV), we can notice that there are statisti-
cally significant differences in the number of points achieved in all three levels of tasks and in the test as a whole,
in favour of Group E (Rank I: t= 4.48>2.58; Rank II t=4.70>2.58; Rank III: t=3.52>2.58; Total: 4.16>2.58).
Discussion
The obtained results show that the experimental group to which the content of pollination and pollinators
was presented through concept maps achieved better results in the final test of knowledge than the control group
to which the same content was presented without them.
It was proved that the experimental group had a greater quantity and better quality of knowledge acquired
as compared to the control group. This can be particularly seen in Rank II (applying and analysing), where the value
751
(P. 746-760) ISSN 2538–7138 /Online/
of t=4.70 is outstanding as compared to Rank I (remembering and understanding) and Rank III (evaluating and
creating). This research is aimed at improving the quantity and quality of the knowledge acquired, as well as the
methodical approach, which is often missing when it comes to the traditional way of presenting an educational
content. This changes the role of the teacher in the teaching process; the teacher is no longer merely someone
who teaches, but someone who becomes the creator and organizer of the teaching process.
The results of the above studies can be compared with the results of related studies.
It should be particularly pointed out that concept maps have proved to positively affect the realization of
some zoological programme contents (Annelids) regarding the quality and quantity of the knowledge acquired
by students (Stanisavljević & Stanisavljević, 2014).
Application of concept maps in realization of botanical curricula in high schools (in Serbia), proved to be a
very effective for acquiring knowledge (Stanisavljević & Jovanović, 2014).
The analysis of applying concept maps in the implementation of programme contents concerning evolution
(sixth grade of elementary school/secondary school) also shows that the application of concept maps, in accordance
with this courseware, is efficient (Ilić, Đurić, & Stanisavljević, 2015).
A study of the application of concept maps in programmed teaching (programme contents: Human Anatomy
and Physiology) at the University of Belgrade - Faculty of Biology, indicates that concept maps are very effective in
acquiring knowledge of physiology (Stanisavljevic & Djuric, 2013a).
Dhaaka (2012) indicates that the application of concept maps is an efficient mode of biology teaching that
motivates students to learn and develop skills. The teaching process becomes more interactive and interesting.
The results of this research show that the students using concept maps achieve significantly better results than
those who are exposed to the traditional lecturing without concept maps.
Kinchin and Hay (2008) have proved the efficiency of concept maps in the classroom in measuring students’
previous knowledge. They showed that simple mapping exercises may affect the quality and quantity of students’
and teachers’ knowledge.
A special group (phylum-type) of invertebrates (Phylum Tardigrada) can be explained using the constructiv-
ist approach and presented by applying concept maps. It is believed that this small phylum represents the direct
ancestor of the largest phylum in the world (Phylum Arthropoda). Despite its great importance, students often
ignore Phylum Tardigrada (Kinchin, 2000a).
Examining the contents of ecology, it has been proved that concept maps are very effective in representing
and understanding the concepts related to pollution and environmental protection (Kinchin et al., 2008).
Novak’s concept mapping has the potential to make a major impact on the development of higher education.
This can be achieved by increasing the availability of multiple perspectives of knowledge that can be discovered
and used for transcending the epistemic confusion lying beneath the seemingly coherent curriculum, which
actually limits the impact of university teaching and creates the impression that universities are not for studying
(Kinchin, 2015).
Patrick (2011) conducted a study on whether concept maps might improve the success of students in tests.
The study was conducted after noticing that both high school and university students have a very poor knowledge
of biology. The poor quality of knowledge was due to a lack of resources available to teachers. The lack of labora-
tories in schools led to the need to find alternative methods, which could be available in most schools, to replace
laboratory work. Concept maps seemed to be the best solution.
Youssef and Mansour (2012) showed in their study that students, overwhelmed with information on a daily
basis, had hard time managing information and distinguishing important from unimportant terms. The results
showed that the use of concept maps, created both by professors and students, had greatly improved the quality
of teaching in this case.
Edmodson (1994) points out that adequate concept maps are an excellent way of presenting a teaching
content to students. By using concept maps it is possible to study in a clearer and easier way.
Concept maps can be used for teaching and learning within elementary school curricula. Seventh grade
students were presented a part of the curriculum in the classical way and another part through concept maps.
Testing was conducted at the end of each session. The results showed that the tests relating to the contents covered
through concept maps were much better done by the students than those that did not contain contents presented
through concept maps. (Lemos, Moreira, & Mendonça, 2008).
Daley (2004) conducted a study on the use of concept maps with university students. The aim of the study
was to determine whether different learning strategies, in this case concept maps, contributed to changing the
752
ISSN 2538–7138 /Online/ (P. 746-760)
models and understanding of teaching materials. During the one-year research, he conducted testing to determine
the way of changing the scores achieved in tests with concept maps and in those containing teaching contents
that the students only listened to, without using the maps. The results showed that the material acquired through
concept maps was much better and more durable, and almost all of the students stated that the concept maps
had made their studying much easier.
To achieve maximum results, the maps should not be the only method used; they should rather be combined
with other teaching methods and activities. There is no universal map that would suit all students in every teaching
situation (Kinchin, 2000c). The introduction of concept maps in teaching should be carried out slowly. Teachers
should first use them when explaining certain parts of the teaching content that are already known to their stu-
dents, and then slowly introduce them when dealing with concepts which the students have never encountered
before (Stanisavljević & Jovanović, 2014).
The results of the previous similar research (Đurić & Stanisavljević, 2011; Stanisavljević & Jovanović, 2014)
show that the majority of students think that the manner in which teaching contents are processed is extremely
important. They are convinced that learning particular contents through concept maps helps them learn much
more, facilitating their understanding of biology.
Based on the survey results and after summarizing the views of students on the application of concept maps
in implementing zoological educational contents, it can be concluded that most students accept this manner of
work. Students realized that this approach to learning zoology had facilitated their understanding and mastering
the course content Annelids. Nearly two-thirds of them confirmed that they had been motivated to adopt the
materials by applying concept maps, and, therefore, had acquired better knowledge of Annelids both in terms of
quantity and quality. The negative attitudes of students show that the theoretical contents of university subjects
are mainly taught in the traditional way by lecturing. Therefore, students are somewhat uncertain and cautious
about each new method that is unknown to them (Stanisavljević, Djurić, & Stanisavljević, 2014).
Conclusions
The teaching content Pollination and Pollinators was implemented through concept maps in the experimental
group, while in the control group it was implemented through the usual teaching approach without this technology.
After introducing the experimental factor (concept maps) in Group E, this group achieved better results in
the post-test of knowledge than Group C. The high level of statistically significant difference is particularly notice-
able between the groups (in favour of the experimental group) in Rank II (application and analysis of knowledge
in the given teaching field).
There is a statistically significant difference in mastering the given contents between the students in the ex-
perimental and the control group. It is evidenced a difference in level of the attained knowledge in favour of the
experimental group after the introduction of the experimental factor (application of concept maps).
It can therefore be concluded (the answer to research question) that the application of concept maps directly
contributed to better learning and knowledge acquisition in teaching the biology content Pollination and Pollina-
tors. In other words, the high quality of the knowledge acquired by the students in the tested teaching field was
especially significant in Rank II (application and analysis of knowledge).
Concept maps allow teachers to present certain contents more easily. Quality and quantity of students’ knowl-
edge has been increased. Furthermore, teachers gain a better insight into students’ knowledge by using concept
maps. However, in order to improve biology teaching by using concept maps, it is necessary to provide training
for teachers to apply this technology.
The intention is that in future it becomes usual teaching technology in many zoology courses at the Faculty of
Biology in Belgrade. The teachers and staff will be gradually trained for the application of this teaching technology.
Based upon the obtained results of this research, concept mapping will be further implemented at the Uni-
versity of Belgrade - Faculty of Biology, to improve the curriculum and the teaching process (subjects: Anatomy
and Morphology of Invertebrates and Biology of Bees with Beekeeping). Further research is planned in the field of
revision of the curriculum for zoological subjects. Thus, the concept maps will be used for organizing and structur-
ing the curriculum of these subjects. It is expected that this will allow teachers to easily create and organize the
curriculum, as well as connect teaching units.
753
(P. 746-760) ISSN 2538–7138 /Online/
Acknowledgements
The authors would like to express their gratitude to the Ministry of Education and Science of the Republic of
Serbia for the financial support (Project 173038).
References
Anderson, W. L., & Krathwohl, D. R. (2001). Taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of edu-
cational objectives. New York: Longman.
Berenbaum, M. (1995). Bugs in the system: Insects and their impact on human affairs. Boston: Helix Books, Addison Wesley Pub-
lishing Company.
Birbili, M. (2006). Mapping knowledge: Concept maps in early childhood education. Early Childhood Research & Practice, 8 (2).
Retrieved from http://ecrp.uiuc.edu/v8n2/birbili.html.
Daley, B. J. (2004). Using concept maps with adult students in higher education. In A. J. Cañas, J. D. Novak, & F. M. González (Eds.),
Concept maps: Theory, methodology, technology: Proceedings of the First International Conference on Concept Mapping (Vol.
1, pp. 143-152). Pamplona, Spain: Universidad Pública de Navarra.
Davies, M. (2011). Concept mapping, mind mapping and argument mapping: What are differences and do they matter? Higher
Education, 62 (3), 279–301. doi:10.1007/s10734-010-9387-6.
Dhaaka, A. (2012). Concept mapping: Effective tool in biology teaching. VSRD Technical & Non-Technical Journal, 3 (6), 225-230.
Dicks, L., Felipe Viana, B., Maria del Coro Arizmendi, M., Bommarco, R., Brosi, B., Cunningham, S., ... Carmen Pires, C. (2016).
Responses to risk and opportunities associated with pollinator and pollination. In: S. G. Potts, V. L. Imperatriz-Fonseca,
& H. T. Ngo, (Eds.), The assessment report of the Intergovernmental science-policy platform on biodiversity and ecosystem
services on pollinators, pollination and food production (pp. 365-472). Secretariat of the Intergovernmental Science-Policy
Platform on Biodiversity and Ecosystem Services, Bonn, Germany.
Đurić, D., & Stanisavljević, J. (2011). The views and opinions of biology students on the application of programmed instruction
in the realization of physiological program content. Croatian Journal of Education, 13 (2), 108-123.
Edmodson, K. M. (1994). Concept maps and the development of cases for problem-based learning. Academic Medicine, 69 (2),
108-110.
Fischer, K. W. (2009) Mind, brain, and education: building a scientific groundwork for learning and teaching. Mind, Brain, and
Education, 3 (1), 3–16.
Fisher, K. M., Wandersee, J. M., & Moody, D. E. (2000). Mapping biology knowledge. Dordrecht, the Netherlands: Kluwer Academic.
Garibaldi, L. A., Steffan-Dewenter, I., Winfree, R., Aizen, M. A., Bommarco, R., Cunningham, S. A., ... Klein, A. M. (2013). Wild pollina-
tors enhance fruit set of crops regardless of honey bee abundance. Science, 339, 1608-1611. doi:10.1126/science.1230200.
Groeneveld, J. H., Tscharntke, T., Moser, G., & Clough, Y. (2010) Experimental evidence for stronger cacao yield limitation by pol-
lination than by plant resources. Perspectives in plant ecology. Evolution and Systematics, 12, 183-191.
Hay, D. B., & Kinchin, I. M. (2006). Using concept maps to reveal conceptual typologies. Emerald Education + Training, 48 (2-3),
127-142. doi:10.1108/00400910610651764.
Ilić, R., Đurić, D., & Stanisavljević, J. (2015). Analiza primene mapa pojmova u realizaciji programskih sadržaja o evoluciji iz
biologije za VI razred osnovne škole [An analysis of the application of the mapping concept used in the realization of
evolution contests of the biology curriculum for the 6th grade of elementary school]. Nastava i vaspitanje, 64 (1), 161-172.
doi:10.5937/nasvas1501161i.
Kember, D. (2003). To control or not to control: The question of whether experimental designs are appropriate for evaluating
teaching innovations in higher education. Assessment & Evaluation in Higher Education, 28, 89-101.
Kevan, P. G. (2007). Bees, biology and management. Cambridge, Ontario, Canada: Enviroquest Ltd.
Killermann, W. (1998). Research into biology teaching methods. Journal of Biological Education, 33 (1), 4-9.
Kinchin, I. M. (2000a). On the significance of a “minor” phylum (the Tardigrada) in the context of a constructivist view of knowl-
edge. Perspectives in Biology and Medicine, 43 (2), 243-251.
Kinchin, I. M. (2000b). The active use of concept mapping to promote meaningful learning in biological science. (Doctoral disserta-
tion), Guildford: University of Surrey.
Kinchin, I. M. (2000c). Concept map in Biology. Journal of Biological Education, 34 (2), 61-68.
Kinchin, I. M. (2011). Visualising knowledge structures in biology: discipline, curriculum and student understanding. Journal of
Biological Education, 45 (4), 183-189.
Kinchin, I. M. (2013a). The significance of knowledge structures in technology-enhanced learning: A Bernsteinian analysis of the
TPACK* framework. (*TPACK= Technology, Pedagogy and Content Knowledge). In Society for Research into Higher Education
Annual Research Conference, New Port.
Kinchin, I. M. (2013b). Concept mapping and the fundamental problem of moving between knowledge structure. Journal for
Educators, Teachers and Trainers, 4 (1), 96–106.
Kinchin, I. M. (2014). Concept mapping as a learning tool in higher education: A critical analysis of recent reviews. The Journal of
Continuing Higher Education, 62 (1), 39-49.
Kinchin, I. M. (2015). Editorial: Novakian concept mapping in university and professional education. Knowledge Management &
E-Learning, 7 (1), 1–5.
754
ISSN 2538–7138 /Online/ (P. 746-760)
Kinchin, I. M., & Hay, D. (2008). Using concept mapping to measure learning quality. Education & Training, 50 (2), 167-182.
Kinchin, I. M., Hay, D., & Lygo-Baker, S. (2008). Making learning visible: the role of concept mapping in higher education. Studies
in Higher Education, 33 (3), 295-311.
Kinchin, I. M., Streatfield, D., & Hay, D. B. (2010). Using concept mapping to enhance the research interview. International Journal
of Qualitative Methods, 9 (1), 52-68.
Kleijn, D., Winfree, R., Bartomeus, I., Carvalheiro, L. G., Henry, M., Isaacs, R., ... Potts, S. G. (2015). Delivery of crop pollination services
is an insufficient argument for wild pollinator conservation. Nature Communications, 6, 7414. doi:10.1038/ncomms8414.
Klein, A. M., Vaissiere, B. E., Cane, J. H., Steffan-Dewenter, I., Cunningham, S. A., Kremen, C. & Tscharntke, T. (2007). Importance
of pollinators in changing landscapes for world crops. Proceedings of the Royal Society B-Biological Sciences, 274, 303-313.
doi:10.1098/rspb.2006.3721.
Lemos, E. S., Moreira, M. A., & Mendonça, C. A. S. (2008). Learning with concept map: an analysis of a teaching experience on the
topic of reptiles with 15-year old students at a secondary school. International Conference on Concepting Mapping, Tallinn,
Estonia & Helsinki, Finland, 3.
Loertscher, J. (2011). Threshold concepts in biochemistry. Biochemistry and Molecular Biology Education, 39, 56-57. doi:10.1002/
bmb.20478.
Moreira, E. F., Boscolo, D. & Viana, B. F. (2015). Spatial heterogeneity regulates plant-pollinator networks across multiple landscape
scales. PLoS One, 10, e0123628. doi:10.1371/journal.pone.0123628.
Novak, J. D., & Cañas, А. Ј. (2008). The theory underlying concept maps and how to construct and use them. Technical Report
IHMC CmapTools 2006-01 Rev 01-2008, Florida: Institute for Human and Machine Cognition.
Novak, J. D. (2010). Learning, creating and using knowledge: Concept maps as facilitative tools in schools and corporation (2nd ed.).
New York, NY: Routledge.
Patrick, A. (2011). Concept mapping as a study skill: Effects on students’ achievement in biology. International Journal of Educa-
tional Sciences, 3 (1), 49-57.
Peters, R., & Beeson, M. (2010). Reducing the gap between skills sought by employers and developed by education. PS: Political
Science & Politics, 43 (4), 773-777.
Potts, G. S., Imperatriz-Fonseca, V., Ngo, T. H., Biesmeijer, C. J., Breeze, D. T., Dicks, ... Vanbergen J. A. (2016). IPBES Summary for
policymakers of the assessment report of the Intergovernmental Science-Policy Platform on Biodiversity and Ecosystem Services
on pollinators, pollination and food production. Bonn, Germany: Secretariat of the Intergovernmental Science-Policy Platform
on Biodiversity and Ecosystem Services.
Pyke, G. H. (2010). Optimal foraging and plant–pollinator co-evolution. In M. D. Breed & J. Moore (Eds.), Encyclopedia of animal
behavior (vol. 2) (pp. 596–600). Oxford: Academic Press. doi:10.1016/B978-0-08-045337-8.00278-3.
Rader, R., Bartomeus, I., Tylianakis, J. M., & Laliberte, E. (2014). The winners and losers of land use intensification: pollinator com-
munity disassembly is non-random and alters functional diversity. Diversity and Distributions, 20, 908-917. doi:10.1111/
ddi.12221.
Sessions, L. A., & Johnson, S. D. (2005). The flower and the fly. Natural History, 114 (2), 58.
Simon, J. (2009). Curriculum changes using concept maps. Accounting Education: An International Journal, 19 (3), 301-307.
Stanisavljevic, J., & Djuric, D. (2013a). The application of programmed instruction in fulfilling the physiology course requirements.
Journal of Biological Education, 47 (1), 29-38. doi:10.1080/00219266.2012.753103.
Stanisavljević, J., & Djurić, D. (2013b). Uticaj programiranog modela nastave na trajnost i kvalitet stečenih znanja kod studenata
[Influence of programmed models of teaching at the durability and quality of the knowledge acquired by the students].
Pedagogija, 68 (1), 61-67.
Stanisavljević, J., Djurić, D., & Stanisavljević, L. (2014). Stavovi studenata o primeni mapa pojmova u realizaciji zooloških pro-
gramskih sadržaja (Annelida) [Students’ attitudes towards conceptual maps in realization of zoology program contents
(Annelida)]. Pedagoška stvarnost, 60 (1), 59-70.
Stanisavljević, J., & Jovanović, K. (2014). Primena mapa pojmova u realizaciji programskih sadržaja botanike u srednjoj školi [Ap-
plication of concept maps in realization of botanical curricula in high schools]. Pedagogija, 69 (4), 555-560.
Stanisavljević, J. & Stanisavljević, L. (2014). The application of concept maps in teaching Invertebrate zoology. In D. Krüger & M.
Ekborg (Eds.), Powerful tools for learning in biology (pp. 197-211). Berlin: Freie Univerzität Berlin.
Stanisavljević, L. (2012). Insekti oprašivači i njihov značaj za čoveka [Insects as pollinators and their importance for man]. In Z.
Tomanović (Ed.), Primenjena entomologija [Applied Entomology] (pp. 117-147). Beograd, Serbia: University of Belgrade,
Faculty of Biology.
StatSoft, Inc. (2001). Statistica for Windows (ver. 6) [data analysis software system]. Tulsa, OK: StatSoft, Inc. Retrieved from http://
www.statsoft.com.
Student (Gosset, W. S.) (1908). The probable error of a mean. Biometrika, 6, 1-25.
Valenta, K., Nevo, O., Martel, C., & Chapman, A. C. (2017). Plant attractants: integrating insights from pollination and seed dispersal
ecology. Evolutionary Ecology, 31, 249. doi:10.1007/s10682-016-9870-3.
Youssef, A. M., & Маnsour, A. M. (2012). The effect of concept mapping on students’ learning achievements and interests in Taif
University. Life Science Journal, 9 (2s), 346-353. Retrieved from http://www.lifesciencesite.com/lsj/life0902s/039_17486lif
e0902s_346_353.pdf.
755
(P. 746-760) ISSN 2538–7138 /Online/
Appendix 1. A concept map: the meaning of the term reproduction in plants.
Appendix 2. Concept map: the principal characteristics of the flower essential to attracting pollinators.
756
ISSN 2538–7138 /Online/ (P. 746-760)
Appendix 3. The model of pedagogical experiment with parallel groups.
Appendix 4. An example of filled out concept maps on the topic of pollination and pollinators. The fields
marked by an asterisk were blank at the beginning (students fill them out during the mastery
of the teaching content).
Fertilization of plants.
757
(P. 746-760) ISSN 2538–7138 /Online/
Pollination.
The bees.
758
ISSN 2538–7138 /Online/ (P. 746-760)
The bumblebees.
Appendix 5. Example of some tasks used as an indicator of Ranks I, II and III.
Rank I
Circle the letter of correct answer:
Pollination is defined as:

a) The process of propagation of Algae and other Thallophyta
b) The process of transferring pollen grains from the anthers of one flower to the stigma of another flower
c) The process of transferring the nectar from one plant species to another.
Rank II
Description of concept The concept/ pollinators
Searching for food (such as pollen and nectar) they perform pol-
lination. They are attracted by the smell, shape, color, or the heat
of the flower.
They are adapted to pollinate certain flowers, which are so con-
structed that precisely these pollinators can accept.
They accidentally passes through the flowers and carry pollen
which accidentally stuck on their bodies.
They are attracted by the vivid colors of the flower from which they
suck nectar and catch small insects.
They are nocturnal animals, they pollinate the plants that bloom at
night.
Fill in the table: Based on the description, determine and name the concept in the table.
759
(P. 746-760) ISSN 2538–7138 /Online/
Rank III
9. Carefully watch scheme of pollination and fertilization of plants. Determine and name the concept in the
table.
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
Jelena D. Stanisavljević PhD, Associate Professor, University of Belgrade, Faculty of

Biology, Studentski trg 16, 11000 Belgrade, Serbia.
E-mail: jelena.stanisavljevic@bio.bg.ac.rs
Website: http://www.bio.bg.ac.rs/05_nastavnik.php?id_
nastavnika=129
Mirka M. Bunijevac MSc, PhD Student, University of Belgrade, Faculty of Biology,
Studentski trg 16, 11000 Belgrade, Serbia.
E-mail: mirka_srce@hotmail.com
Ljubiša Ž. Stanisavljević PhD, Associate Professor, University of Belgrade, Faculty of
Biology, Studentski trg 16, 11000 Belgrade, Serbia.
E-mail: Ljstanis@bio.bg.ac.rs
Website: http://www.bio.bg.ac.rs/05_nastavnik.php?id_
nastavnika=121
760
EFFECT OF PROBLEM-BASED
LEARNING ON IMPROVEMENT
PHYSICS ACHIEVEMENT AND ISSN 1648-3898 /Print/
CRITICAL THINKING OF SENIOR ISSN 2538-7138 /Online/
HIGH SCHOOL STUDENT
Mundilarto, Abstract. The aim of the research was

Helmiyanto Ismoyo to know the effect of implementation of
problem-based learning model on students’
physics achievement and critical thinking.
The research was a quasi-experimental
using a pre-test-post-test control group
design. It was conducted at a state senior
high school in Indonesia. The data were col-
lected by a students’ physics achievement
test, a students’ critical thinking test, and an
Introduction observation sheet. The data were analysed
by calculating normalized gain scores and
Students’ competence in comprehending the concepts in physics at hypothesis testing by using MANOVA with
school greatly determines their learning achievement. Their success in learn- the significance level of .05 as criterion. The
ing could be known from the results of their learning achievement evaluation, research results indicated that the average
whose objective is to know the learning outcome that they obtain after the gain scores for students’ physics achieve-
teaching and learning process is carried out. When they already comprehend ment of the experimental and control
physical concepts well, they would overcome problems related to the said groups were respectively .63 and .32, the
concepts in daily life or overcome similarly complex problems. Council for average gain scores for students’ critical
the Advancement of Standards in Higher Education (CAS) in Komives (2012, thinking of the experimental and control
p. 78) states that there are six domains of students’ learning outcomes ex- groups were respectively .49 and .34, and
pected from learning, namely, those respectively of (1) knowledge acquisition, a value of p is .0001 was obtained from the
construction, integration, and application, (2) cognitive complexity, (3) intra- MANOVA calculation so that it is concluded
personal development, (4) interpersonal competence, (5) humanitarianism that such learning as that in the experi-
and civic engagement, and (6) practical competence. mental group is better in comparison with
Learning outcomes could not fully assist students in comprehending such learning as that in the control group.
something. It is in line with what is delivered by Hussey and Smith (Brooks, The results of this research are expected to
2014, p. 722) by suggesting that learning outcomes can never fully capture increase insight and knowledge of teachers
the open, creative, and dynamic process of learning and, instead, they may in implementing the learning process in the
lead to a narrowing of students’ learning and tutors’ delivery around the classroom so as to apply varied learning
predetermined outcomes. The cognitive level of scientific models held by models.
students, appeared in Chinese national curriculum standards and national Keywords: critical thinking, physics
examination requirements are equivalent, but there are also some differences achievement, problem-based learning.
among students’ learning, curriculum standards and examination require-
ments. A specific classification of the models based on the comprehensive
understanding systems is conducted, and the impact on learning among
different schools and regions and inherent relationship on the cognitive level Mundilarto, Helmiyanto Ismoyo
Yogyakarta State University, Indonesia
of physics models are founded by Wang, Zhang, and Shi, (2016). Since learn-
ing outcomes do not fully capture the open, creative, and dynamic process
in learning, learning achievement is therefore not the one and only most
761
(P. 761-780) ISSN 2538–7138 /Online/
important aspect in learning. Besides students’ learning outcomes, students’ way of thinking could also determine
their success. One of the important ways of thinking is thinking critically. Critical thinking is an important element
in learning. It has even been applied since the times of Socrates. However, many still do not master the said ability
and not all learning could be well received by students. It concerns a skill which greatly assists students in solving
more complex problems. Siew and Mapeala (2016) state that problem-based learning with thinking maps effects
on fifth graders’ science critical thinking. The result indicated that students in the PBL-TM group significantly out-
performed their counterparts in the PBL group who, in turn, significantly outperformed their counterparts in the
CPS group in comparing and contrasting, sequencing, and identifying cause and effect. The findings suggest that
thinking maps, which were explicitly infused into problem-based learning is effective in promoting critical thinking
among fifth graders in physical science lessons.
The survey shows that students do not possess explicit cognitive activity; learning happens rather passively,
without initiative; however, they have rather pronounced interest to explore and solve problems connected with
the real life. Some implications for teachers on how to increase learners‘ cognitive interest are provided in the con-
clusion (Cēdere et al., (2015). Everyone could think but not everybody thinks deeply and the tendency is to think
of many things so that no focus is given to one problem. It is in line with what is delivered by Ennis (1991, p. 7) as
follows: “critical thinking ... means reasonable reflective thinking that is focused on deciding what to believe or do.”
According to Ennis, critical thinking is a reflective way of thinking which makes sense and is focused on what to
believe or do. Elder and Paul (2007, p. 4) state that “critical thinking is the art of analysing and evaluating thinking
with a view to improving it.” Critical thinking is considered an art practiced by means of analysing and evaluating
the manner of thinking itself so that we could improve our way of thinking. In critical thinking, there is a demand
for our being able to train our analysis in order to further improve our way of thinking. Rudinow and Barry (2005,
p. 12) explain that the term critical thinking in a way comes from the Greek word kritikos derives from the word
“critic”, which means the ability to evaluate or to discern. Critical thinking is to an extent indeed the ability to find
faults and negative judgments but actually it is not limited to only that. To think critically, one requires a clear and
rational mind and follows the rules of logic and scientific reasoning above all so that one could determine the right
reasons in making decisions. As said by Lau (2011, p. 1), “Critical thinking is thinking clearly and rationally. It involves
thinking precisely and systematically, and following the rules of logic and scientific reasoning, among other things.”
There are various explanations concerning thinking critically but there is one similarity among them, namely,
conclusions are drawn by using precise and systematic thoughts and following the rules of logic and scientific
reasoning to improve the way of thinking. Thus, thinking critically is thinking not only of how one could answer a
question but also of what is the way to get to that answer systematically and precisely. Moore and Parker (2009, p.
5) state that there are three basic building blocks of critical thinking, namely, the claim, the issue, and the argument.
The claim or statement is a basic matter in critical thinking, with a statement not always being something true (or
right or correct) and something false (or wrong or incorrect) being also able to be expressed in a statement. How-
ever, a statement need not be critically evaluated when its being true or false is already obvious. As for an issue or
problem, its concept is very simple, namely, that it is just a question, meaning that after a statement is obtained,
generally a question would appear to make sure whether the statement is true or false. The question appears
based on what is already recalled in oneself. After a question appears like that, then our self makes a response to
the statement and gives a reason for its being considered true or false. That is what is interpreted as argument by
Moore and Parker (2009, p. 10). The critical thinking ability, according to Ennis (2013), is divided into several large
categories, namely, those involving respectively (1) basic clarification, (2) bases for a decision, (3) inference, (4)
advanced clarification, (5) supposition and integration, and (6) auxiliary or facilitative abilities. The categories have
respective indicators and sub-indicators supporting the development of the critical thinking skill.
One of the efforts to improve learning achievement and the critical thinking skill is conducting varied learning.
One of the learning models proposed by Ministry of Education and Culture in Indonesia for helping to improve
students’ abilities and skills is problem-based learning. Problem-based learning (PBL) is a learning model that
presents various problems occurring in students’ life so that it could stimulate them to learn. In PBL, students are
to work in groups to seek solutions of existing problems. PBL is a learning model centred on students so that they
are to actively seek information on their own and determine which information should be studied and learned to
assist them in learning.
The PBL model is developed and designed to help students to build up expansive knowledge and to be
able to apply the knowledge obtained on effective problem solving. It is in line with the statement by Oon-Seng
Tan (2009, p. 9) that “in PBL, the problem is cast in a realistic context that the student might encounter in future.”
762
ISSN 1648–3898 /Print/ EFFECT OF PROBLEM-BASED LEARNING ON IMPROVEMENT PHYSICS ACHIEVEMENT AND
ISSN 2538–7138 /Online/ (P. 761-780)
The PBL model presents a problem which is realistic and might well confront students in the future. Like other
learning models, according to Tan (2004, p. 8), PBL has its characteristics, namely, (1) the problem is presented
initially in the learning conducted, (2) it should be a problem being faced and like a problem in daily life or, if the
intention is presenting a problem in a simulation, the problem should be authentically possible to occur, (3) the
problem presented should cover all the existing perspectives, (4) it should challenge the knowledge, attitude, and
competence that students possess, (5) independent learning is the key to PBL, (6) the utilization of various sources
of knowledge and the use and evaluation of information resources are important processes in PBL, (7) the PBL
process itself is collaborative, communicative, and cooperative in nature, (8) the development of the investigation
and the problem solving ability is equal in importance with the acquisition of knowledge in seeking the solution
to a problem, (9) the closing part in the process of PBL is a synthesis and integration in the learning process, and
(10) PBL also ends with an evaluation and review of the learning experience and learning process.
One of the important points in PBL is that the problem presented is not an initial question and knowledge is
given by using a lecture and handout by the teacher. The said problem should be relevant with and actual in the
students’ life so that PBL could train students in getting a solution to the problem by themselves. The application
of PBL in the improvement of the critical thinking skill is not something new. Take, for example, the research by
Henderson (2014) on the relation between problem-based learning and the development of the critical think-
ing skill in higher education. In the research, Henderson obtained pre-test and post-test data from two classes
respectively serving as PBL-using class and traditional class. Henderson made gain and MANOVA calculations to
evaluate the difference among components of the critical thinking skill. The results obtained by Henderson (2010)
indicate that there is no significant difference in students’ critical thinking skill between a PBL-applying class and
a traditional class. Henderson finds that the PBL class is higher in level of analysis, evaluation, and induction skills
compared to the traditional one.
PBL-related research was also done by Anderson II. The research was conducted on two classes consisting of
110 persons in all with 56% of them consisting of women. One of the classes served as experimental group apply-
ing PBL with 67% of the persons in it consisting of women and the other class served as control group applying
TGL (traditional group learning) with 46% of the persons in it consisting of women. In the research, a statistical
difference between the two classes has been found. A comparison between the post-test mean score and the pre-
test mean score in the PBL class has indicated a decrease of two points in magnitude while that in the TGL class
has indicated a decrease of five points in magnitude. It, therefore, means that a greater decrease has occurred in
the TGL class compared to that occurring in the PBL class. Consequently, the researcher concludes that such a PBL
class is better compared with such a TGL class.
The third research to mention in this relation was done by Agdas (2013) on the effect of PBL application on
the improvement of critical thinking and characteristics of students of the mechanical engineering field of study.
The research used two classes with one consisting of twenty-three students serving as experimental group apply-
ing PBL and one consisting of twenty-two students serving as control group employing the lecture or traditional
method. In the control group, there has been a decrease in mean score from 79.2 in the pre-test to 78.3 in the
post-test but there has been an increase in standard deviation from 6.5 in the pre-test to 7.7 in the post-test. In
the experimental group, there has been an increase in mean score from 78.1 in the pre-test to 78.4 in the post-test
with a decrease in standard deviation from 8.4 in the pre-test to 6.2 in the post-test. This research by Agdas does
not indicate any difference between the experimental group and the control group.
Tiwari et al. (2006) conducted research on a comparison in effect between PBL and lecture learning on the
critical thinking skill. The research used two classes, namely, a PBL class of forty students and a class of thirty-nine
students learning with the method of lecturing. Measurements were done by using the California Critical Think-
ing Disposition Inventory (CCTDI). The result indicates overall improvement of the PBL class in the CCTDI (with p
= .0048). On the whole, there is a significant difference in the class using PBL compared with the class using the
lecture method.
With the exposition above as basis, it was then considered of interest to know the effect of problem-based
learning on improvement in students’ learning achievement and critical thinking at a state senior high school in
Indonesia. The research concerned here was then aimed at knowing the effect of applying a PBL model on stu-
dents’ physics learning outcomes and critical thinking skills. The research was focused on the cognitive domain and
specifically on its respective levels of applying, analysing, evaluating, and creating. Those were the levels selected
for the reason that students at senior high school should already be at those levels. The critical thinking skills put
under research were giving basic clarification, building bases for a decision, making an inference, and making an
763
(P. 761-780) ISSN 2538–7138 /Online/
advanced clarification. These four critical thinking skills were selected because they were still related to physics
learning so that it was worthy to know to what extent the PBL model could improve those skills.
Based on preliminary observations in a senior high school in Yogyakarta Indonesia, can be described some of
realistic problems, as follows: (1) in physics learning process teachers still use traditional methods so as not to give
sufficient time to students to participate in learning, (2) in physics learning process students tend to be passive
so that they are just follow what is said by teacher, (3) in physics learning process teachers tend to rarely address
issues related to the material to be taught, (4) in physics learning process rarely started with a realistic problem,
(5) in physics learning process students are only able to use the existing ability just to solve the problems posed
by teachers and less able to apply them in everyday life, (6) in physics learning process teachers also rarely train
students to think critically in solving problems, and (7) in physics learning process teachers more often use classi-
cal methods than student center methods. Referring to some of the field problems, this research wants to know
whether there is a positive effect of PBL model on physics learning outcomes and critical thinking skills of students.
The result of this research is expected to give practical contribution as one of alternative choices in improving
learning effort, as follows: (1) For students, PBL model is expected to improve students’ learning outcomes and
critical thinking skills that it is very useful in solving problems. In addition, through the PBL is expected to provide
experience to students learn together with a group. (2) For teachers, this research is expected to increase the com-
petence of teachers in implementing the learning process in classroom. By applying a varied learning is expected
to help educators in facilitating students to develop their skills. (3) For researchers, the results of this study provide
a fact that PBL model effects on students’ learning outcomes and critical thinking skills.
Research Type
The research was quantitative in approach and used a quasi-experimental research design. It involved two
classes, namely, one serving as experimental group given treatment in the form of the application of a problem-
based learning model with the method of experiment and one serving as control group given the same form of
treatment but with the method generally used at school, namely, the method of demonstration as a comparison.
Research Location and Time
As previously implied, the research was conducted at a senior high school in Yogyakarta Indonesia. It was
conducted on the students of Grade X in Semester 2 of the academic year 2015/2016. The decision to conduct
the research during Semester 2 of the 2015/2016 academic year was for the purpose of making the physics class
material fit the physics material used in the research. As previously mentioned, the learning material used was
about dynamic electricity.
Research Population and Participants
The population of the research consisted of seven classes or 224 students of Grade X in Semester 2 at a senior
high school in the 2015/1016 academic year. By means of purposive sampling, two classes, namely, Classes X4 and
X7, were selected as participants. Class X4, which consisted of 32 students served as experimental group and Class
X7, which consisted of 32 students, served as control group.
Procedure
The research used a quasi-experimental research design. The steps in the research were (1) giving a pre-test
to both the experimental group and the control group, (2) giving treatment by applying a PBL model with the
method of experiment on the experimental group and the method of demonstration on the control group, and
(3) giving a post-test to both classes.
764
ISSN 2538–7138 /Online/ (P. 761-780)
Table 1. Research design.
Experimental group O1 X1 O2
Control group O1 X2 O2
O1 : Pre-test of the theme-related learning achievement and critical thinking skill.

O2 : Post-test of the theme-related learning achievement and critical thinking skill.
X1 : Learning treatment in the form of problem-based learning with the method of experiment.
X2 : Learning treatment in the form of problem-based learning with the method of demonstration.
Data, Instruments, and Techniques of Data Collection
In the research, a PBL model with the method of experiment in the experimental group and the method of
demonstration in the control group served as independent variable while learning achievement and the critical
thinking skill served as a dependent variable. The data about learning achievement were obtained through multi-
ple-choice test items and the data about the critical thinking skill were obtained through essay-type test items at
the beginning and end of the learning process. The multiple-choice test items and the essay-type test items were
for the purpose of obtaining respective data about the learning achievement and the critical thinking skill of the
students in both the experimental and control groups.
The research used two data collection methods, namely, test and observation. Observations were made using
observation sheets to see the suitability between lessons plan made with the implementation in the classroom.
Before the data collection itself was conducted, the data source, then the data type, the technique of data collection,
and the instrument(s) to be used were first determined one after another. In order to obtain data that supported
the research in answering the research questions, the researcher had constructed some instruments, namely, the
learning achievement test, the critical thinking skill test, and the learning activity accomplishment observation
sheet. All this could be seen in Table 2.
Table 2. Technique of data collection.
Data Source Data Type Data Collection Technique Instrument
Students Learning Achievement Pre-test and Post-test Thirty Multiple-Choice Test Items
Students Critical Thinking Skill Pre-test and Post-test Four Essay-Type Test Items
Teacher Accomplishment of Learning Observation Guide to the Observation of Teacher Activity

Process Stages During the Learning in Line With the Learning
Execution Plan Developed
The students’ learning achievement test consisted of thirty multiple-choice items. The learning achievement
test material covered Ohm’s Law(s) in seven items, series-parallel circuits in twelve items, the factors influencing
the magnitude of resistivity in eight items, Kirchhoff’s Law in one item, and the voltmeter and the amperemeter
in two respective items. These thirty items were divided into four groups according to the levels of the cognitive
domain dealt with, namely, the levels of, respectively, applying (C3), analysing (C4), evaluating (C5), and creating
(C6). The items for C3 were eleven in number, those for C4 were eight in number, those for C5 were six in number,
and those for C6 were five in number.
The critical thinking skill test was a written test consisting of four essay-type items. The test was to measure
students’ critical thinking skill before and after the learning.
765
(P. 761-780) ISSN 2538–7138 /Online/
Table 3. Critical thinking skill item distribution.
Critical Thinking Category Material Number of Item
Basic Clarification Electric instruments: amperemeter, voltmeter 1

Bases for a Decision Ohm’s Law: electric current, electric resistance, voltage 1
Inference Series-parallel circuits 1
Advanced Clarification Factors influencing the magnitude of electric resistivity 1
Technique of Data Analysis
The resulting research data were analysed by using gain factor and hypothesis analyses. To calculate improve-
ment in students’ conceptual understanding and critical thinking skill from those occurring before the learning
to those occurring after it, the gain was calculated according to the following formula developed by Hake (2008):
<g>
in which
<g> = gain factor
Spre = pre-test mean score (%)
Spost = post-test mean score (%)
The hypothesis testing used the multivariate test. The test was conducted on the significance level of the F values
in Hotelling’s Trace statistics. The criterion for such testing is that H0 is rejected if Fobtained ≥ F(p,n1+n2-p-1; .05 = 2.66; .05)
or the level of significance obtained is smaller than .05.
Before such a test otherwise known as the MANOVA test is conducted, testing of normality and homogeneity
should be done. In the case here, the normality testing was intended to reveal the distribution of the data scores
for students’ learning achievement and critical thinking skill in the two classes. In the research, the testing of nor-
mality used the one-sample Kolmogorov-Smirnov test. The homogeneity testing was done to see whether the
variances in the data for improvement in learning achievement and critical thinking of the experimental group
and the control group were the same or not by using the Levene’s test. In short, the homogeneity testing was
intended to reveal whether there was equality of variances between the two classes. The testing of homogeneity
could also use the SPSS program.
Results of Research
The Accomplishment of the PBL Model
Data of the accomplishment of the PBL model in class were obtained by means of observation. The observer
in each session was the same person. In doing the observation, the observer was equipped with the observation
sheet provided. The observation sheet filled in by the observer indicated to what extent the application of the PBL
model was accomplished.
In percentage of accomplishment, the first session in the experimental group did not reach 100%. The per-
centage of accomplishment in Phase V, namely, the phase of discussing, analyzing, and evaluating the process of
problem solving, was 75%. The percentage of accomplishment of the phases in the second session did not all reach
100%, either. In Phase I, namely, the phase of student orientation to the problem, it was 80%. In the third session,
the percentage of accomplishment in all phases reach 100%, possibly because the researcher had held discussions
with the observer about existing shortcomings in the preceding two sessions. See Table 4.
766
ISSN 2538–7138 /Online/ (P. 761-780)
Table 4. Degree of accomplishment of the learning in the experimental group.
Session Syntax of Model Degree of Accomplishment (%)
Phase I 100
Phase II 100
First Phase III 100
Phase IV 100
Phase V 75
Phase I 80
Phase II 100
Second Phase III 100

Phase IV 100
Phase V 100
Phase I 100
Phase II 100
Third Phase III 100
Phase IV 100
Phase V 100
In every session of the control group, the accomplishment of the phases did not all reach 100%. In the first
session, the accomplishment of Phase IV, namely, the phase of developing and presenting the results of their work,
was only 50%, which was caused by forgetfulness in the researcher’s part to ask students to deliver their conclu-
sions, and the accomplishment of Phase V, namely, the phase of problem-solving process analysis and evaluation,
was only 75%. In the second session, the accomplishment of Phase III was also only 75%, possibly because the
researcher only did the experiment himself and directly presented only the results to the student moving to the
front of the class, and the accomplishment of Phase V was 75%, with the researcher forgetting to give assignments
or tasks to students. In the third session, the accomplishment of all phases was 100% because the researcher had
learned from the mistakes occurring in the preceding sessions. See Table 5.
Table 5. Degree of accomplishment of the learning in the control group.
Session Syntax of Model Degree of Accomplishment (%)
Phase I 100
Phase II 100
First Phase III 100
Phase IV 50
Phase V 75
Phase I 100
Phase II 100
Second Phase III 75
Phase IV 100
Phase V 75
Phase I 100
Phase II 100
Third Phase III 100
Phase IV 100
Phase V 100
767
(P. 761-780) ISSN 2538–7138 /Online/
The Results of Pre-Test and Post-Test of the Experimental Group and the Control Group
As previously said, the students’ learning achievement test used thirty multiple-choice test items. The learning
achievement test was done twice, as pre-test before the learning and as post-test after the learning. The results of
the pre-test were used to know the students’ initial condition in particularly their understanding of the material
about dynamic electricity because it had previously been studied in Grade IX. The results of the post-test were
used to know to what extent the students could master the material after treatment. These pre-test and post-test
activities were equally applied on both the experimental group and the control group. The following are the results
of the pre-test and the post-test of the experimental group and the control group.
Table 6. Results of the pre-test and post-test of students’ learning achievement.
Experimental group Control group

Category
Pre-test Post-test Pre-test Post-test
Mean Score 27.74 73.01 30.93 53.75

Highest Score 50.00 86.67 53.33 73.33
Lowest Score 3.33 53.33 13.33 33.33
Average Gain .63 .32
Table 6 makes clear that there is difference in pre-test mean score between the experimental group and
the control group with the pre-test mean score of the former being 27.74 and that of the latter being 30.93. The
highest pre-test score of the control group is 53.33, which is higher than that of the experimental group, which
is 50.00. Likewise, the lowest pre-test score of the control group is higher than that of the experimental group,
these scores being respectively 13.33 and 3.33. The abovementioned difference in mean score, highest score, and
lowest score is possibly a result of difference in pre-test time because the pre-test of the control group was still in
the morning or, more specifically, before 10.00 a.m. while that of the experimental group was in the afternoon so
that the students in that class were already beginning to feel tired. Difference in previous school of origin may be
something else that possibly had caused the aforesaid score difference.
As for the post-test results, they also show differences. The post-test mean score of the experimental group is
higher compared to that of the control group, these scores being respectively 73.01 and 53.23. The situation of the
highest and lowest scores is also reversed. As previously mentioned, the control group is higher in pre-test highest
score but the experimental group is higher in post-test highest score, with 86.67 being the post-test highest score
of the experimental group and 73.33 being that of the control group. The post-test lowest score of the experimental
group is 53.33 while that of the control group is 30.00. The pre-test standard deviation of the experimental group
is 9.71 and that of the control group is 11.22; the post-test standard deviation of the experimental group is 9.75
and that of the control group is 11.5.
The average numbers of items for the considered cognitive levels correctly answered by students of each class
in the pre-test are as follows. The items for the C3 cognitive level correctly answered by the experimental group
average four in number; so do those correctly answered by the control group. The items for the C4 cognitive level
correctly answered by the experimental group average two in number while on the average only one such item
is correctly answered by the control group. The items for the C5 cognitive level correctly answered by the experi-
mental group average two in number; so do those correctly answered by the control group. On the average, only
one item for the C6 cognitive level is correctly answered by both classes.
The average number of items for the considered cognitive level correctly answered by students of each class
in the pre-test shown in Figure 1.
768
ISSN 2538–7138 /Online/ (P. 761-780)
Experimental Control
Figure 1. Diagram of the average number of correctly-answered items for each cognitive level in the pre-test.
The pre-test scores are distributed in the greatest number in the 16 – 30 score range and they are of 16 students
in the experimental group and 15 students in the control group. Those distributed in the next greatest number are
in the 31 – 45 score range and they are of 11 students in the experimental group and 13 students in the control
group. The scores belonging to the 46 – 60 score range are of one student in the experimental group and three
students in the control group. The scores belonging to the 0 – 15 score range are the smallest in number and they
are of 3 students in the experimental group and one student in the control group.
The distribution of the pre-test scores of the experimental group and the control group shown in Figure 2.
Figure 2. Distribution of the pre-test scores for students’ learning achievement of the experimental group
and the control group.
769
(P. 761-780) ISSN 2538–7138 /Online/
After the pre-test, the research continued with the learning stage by applying the method that had been
planned. After the learning was conducted, post-test scores were obtained. The scores could be seen in Figure 3.
The average number of items for the considered cognitive level correctly answered by students of each class in
the post-test shown in Figure 3.
Figure 3. Diagram of the average number of correctly-answered items for each cognitive level in the post-test.
Figure 3 shows the average number of items for each cognitive level correctly answered by the two classes
in the post-test. In this case, there is a difference in the situation compared with that in the pre-test. In the pre-
test, the experimental group is lower in position than the control group in that matter but in the post-test the
experimental group manages to be higher in average number of correctly-answered items for each cognitive level
than the control group. The items for the C3 cognitive level correctly answered by the experimental group are 9 in
average number while those correctly answered by the control group are 7 in average number. The items for the C4
cognitive level successfully answered by the experimental group are 6 in average number while those successfully
answered by the control group are 4 in average number. The items for the C5 cognitive level correctly answered
by the experimental group are 4 in average number while those correctly answered by the control group are 3 in
average number. As for the items for the C6 cognitive level, those successfully answered by the experimental group
are three in average number and so are those successfully answered by the control group. The distribution of the
post-test scores of the experimental group and the control group could be seen in Figure 4.
The distribution of the post-test scores of the experimental group and the control group shown in Figure 4.
770
ISSN 2538–7138 /Online/ (P. 761-780)
Figure 4. Distribution of the post-test scores for the learning achievement of students in the experimental
group and the control group.
The distribution of the post-test scores of the experimental group and the control group appears quite dif-
ferent from that of their pre-test scores. While there are still 6 students in the control group with scores within the
31 – 45 score range, no student in the experimental group has a score within that score range. The 46 – 60 score
range is the one receiving the greatest number of scores with the scores of 16 students in the control group in-
cluded in that score range while there are only 5 students in the experimental group with scores within that score
range. The students in the control group and the experimental group with scores within the 61 – 75 score range
are respectively 10 and 13 in number. There are no students in the control group with scores within the 76 – 90
score range while there are 13 students in the experimental group with scores within that score range.
After viewing and recapitulating the pre-test results, the researcher’s following activity before entering the
calculation stage of using MANOVA was determining the homogeneity of the pre-test scores of the two classes.
Table 7. Results of the homogeneity test on the pre-test data.
Criteria Result
Obtained Significance .254

Criterion Significance .05
Degree of Freedom 61
Variances the same
The homogeneity testing is concerned with Ho stating that the two variances concerned are the same and
Ha stating that the two variances differ. The testing of homogeneity used the Levene test with the level of signifi-
cance of 5%. If the level of significance obtained > .05, then Ho would be accepted and conversely, if the level of
significance obtained < .05, then Ho would be rejected and Ha would be accepted. From Table 7, it is seen that the
obtained level of significance is .254, its being > .05 meaning that Ho is acceptable. It indicates that the pre-test
scores of the two classes have the same variance value. Besides, determining the homogeneity by the calculation
using the pre-test data, it was also done by analyzing the gain of the two classes, which was found to be .167 in
magnitude. With the gain calculation in determining the homogeneity as basis, again the value obtained > .05 so
that the use of the gain calculation also results in obtaining the same variance value for the data of both classes.
In other words, the score data of the two classes are homogenous or could be said to be homogenous.
771
(P. 761-780) ISSN 2538–7138 /Online/
The normality testing is done for the purpose of knowing whether the data are normally distributed or not.
The testing of normality was applied on the gain in students’ learning achievement of the experimental group and
the control group. The normality testing is concerned with Ho stating that the data are normally distributed and
Ha stating that the data are not normally distributed. The results of the normality testing using the Kolgomorov-
Smirnov test could be seen in Table 8.
Table 8. Results of the normality testing on learning achievement scores by use of the Kolgomorov-Smirnov
test.
Kolgomorov-Smirnov
Group Dependent Variable
Statistic df p
Experimental Gain in Students’ Learning Achievement .115 31 .200

Control Gain in Students’ Learning Achievement .138 32 .129
From Table 8 it is known that the obtained level of significance is .200 for the experimental group and it is .129
for the control group, in both cases > .05 so that Ho is accepted, indicating that the gain scores for the learning
achievement of the two classes are normally distributed.
The testing of students’ critical thinking used an essay-type test consisting of four test items. It was done
twice, namely, once before the learning and once after the learning. The results of the test before the learning (or
the pre-test) were used to know students’ initial condition in understanding the material about dynamic electricity
because that subject matter had previously been studied in Grade IX. The critical thinking test was also done after
the learning was conducted for the purpose of knowing to what extent students could master the material after
treatment. These activities were applied on both the experimental group and the control group. The following
presents the pre-test and post-test results of the experimental group and the control group.
Table 9. Results of the critical thinking pre-test and post-test of the experimental and control groups.

Category
Pre-test Post-test Pre-test Post-test
Mean 22.10 59.84 17.34 46.72

Highest Score 60.00 95.00 60.00 85.00
Lowest Score .00 15.00 .00 10.00
Average Gain .49 .34
Table 9 shows that the pre-test mean scores of the experimental group and the control group are respectively
22.10 and 17.34 while their post-test mean scores are respectively 59.84 and 46.72. From these mean scores, it is
seen that there is improvement in the two classes. Their highest pre-test scores are the same, both being 60.00 in
magnitude, and so are their lowest pre-test scores, both being .00 in magnitude. In average gain score, the two
classes differ, that of the experimental group being .49 with that of the control group being .34.
772
ISSN 2538–7138 /Online/ (P. 761-780)
Figure 5. Distribution of the pre-test scores for critical thinking of the experimental group and the control
group.
The students in the control group with scores within the 0 – 15 score range are 17 in number while those in
the experimental group with such scores are 13 in number. Those in the control group with scores within the 16
– 30 score range are 8 in number while those in the experimental group with such scores are 9 in number. Those
in the control group with scores within the 31 – 45 score range are 5 in number while those in the experimental
group with such scores are 7 in number. Those in each class with scores within the 46 – 60 score range are the
fewest, they being only two in number.
After viewing the pre-test results, we would view the post-test results and the data of the score distribution
could be seen in Figure 6.
Figure 6. Distribution of the post-test scores for critical thinking of the experimental group and the control
group.
773
(P. 761-780) ISSN 2538–7138 /Online/
Figure 6 shows a sufficiently striking difference. The students in the control group with scores within the
0 – 20 score range are 3 in number while those in the experimental group with such scores are 2 in number. Far
more students in the control group are with scores within the 21 – 40 score range compared to those in the ex-
perimental group with such scores, they being respectively 11 and 3 in number. The students in the control group
and those in the experimental group with scores within the 41 – 60 score range do not differ far in number, they
being respectively 13 and 12 in number. The students in the control group with scores within the 61 – 80 score
range are 4 in number while those in the experimental group with such scores are 8 in number. There is only one
student in the control group with a score within the 81 – 100 score range while there are far more students in the
experimental group with such scores, they being six in number.
The testing of homogeneity was also conducted on the pre-test scores for critical thinking. Such testing
concerns Ho stating that the two variances concerned are the same and Ha stating that those two variances dif-
fer. The homogeneity testing used the Levene test with the level of significance of .05 as criterion. If the level of
significance obtained > .05, then Ho is accepted and, conversely, if the level of significance obtained < .05, then
Ho is rejected and Ha is accepted. The results of the homogeneity testing of the pre-test data for critical thinking
could be seen in Table 10.
Table 10. Homogeneity testing on the pre-test scores for critical thinking.
Criteria Result
Obtained Significance .281

Criterion Significance .05
Degree of Freedom 61
Variances the same
Table 10 shows that the level of significance obtained is .281. It being > .05 means that Ho is accepted, which
means in turn that the two classes have the same variance value. Besides calculations using pre-test data, the
homogeneity testing is also done by testing the gain scores. After the testing is done, the value obtained is .314.
It being > .05 again means that the two classes have the same variance value and their scores are homogenous.
After testing the homogeneity, it is also necessary to test the normality by using the MANOVA test before going
to the stage of final completion.
The scores for critical thinking also need to be given a test of normality because the purpose is to know
whether the score data are normally distributed or not. The testing of normality is done to the gain scores for the
critical thinking of students in the experimental group and the control group. The normality testing is concerned
with Ho stating that the data are normally distributed and Ha stating that the data are not normally distributed.
The normality testing was done by using the Kolgomorov-Smirnov test and the results could be seen in Table 11.
Table 11. Results of the normality testing by using the Kolgomorov-Smirnov test on the scores for critical
thinking.
Kolgomorov-Smirnov
Group Dependent Variable
Statistic df p
Experimental Gain in Students’ Critical Thinking .092 31 .200

Control Gain in Students’ Critical Thinking .117 32 .200
According to Table 11, the obtained significance level for the experimental group is .200 in value and that for
the control group is also .200 in value. Since .200 > .05, then Ho is accepted and it indicates that the gain scores for
critical thinking of the two classes are normally distributed.
Such pre-test and post-test results could be used for hypothesis analysis when they already fulfill the require-
ments for homogeneity and normality. When the results of the data testing indicate that the two data batches are of
the same variance and are normally distributed, then the hypothesis testing analysis would use parametric statistics.
774
ISSN 2538–7138 /Online/ (P. 761-780)
Table 12. Testing of the variance and covariance assumption in MANOVA by using Box’s M test.
Box’s M F df df p
2.916 .937 3 6.931E5 .421
Table 12 shows the results of multivariate normality testing using Box’s M test.
Such testing concerns the following pair of hypotheses:
Ho : the two dependent variables have the same variance and covariance
Ha : the two dependent variables have differing variance and covariance
The criteria for decision making in Box’s M test are that if sig > .05, then Ho is accepted and if sig < .05, then
Ho is rejected. From Table 12, it is seen that Box’s M value is 2.916 with sig being .421. Because that value of sig >
.05, then Ho is accepted. It means that the variance and covariance of variables for the experimental group and
that of the control group are the same.
Table 13. Results of individual variable testing by using the Levene test.
F df df p
Learning Achievement 1.956 1 61 .167

Critical Thinking 1.029 1 61 .314
Table 13 shows the results of individual variable testing by using the Levene test. It is seen that the level of
significance of students’ learning achievement with F = 1.956 is .167 and that of students’ critical thinking with
F = 1.029 is .314. It is shown that the obtained significance level for either variable is greater than .05 so that Ho is
accepted. It indicates that the two classes individually have the same variance.
Table 14. Results of the multivariate test.
Effect F p
Intercept Pillai’s Trace 4.120E2a .0001

Wilks’ Lambda 4.120E2a .0001
Hotelling’s Trace 4.120E2a .0001
Roy’s Largest Root 4.120E2a .0001
Method Pillai’s Trace 37.238a .0001

Wilks’ Lambda 37.238a .0001
Hotelling’s Trace 37.238a .0001
Roy’s Largest Root 37.238a .0001
The next results of analysis are of the multivariate test. The multivariate test itself is used to test the research
hypothesis. The initial hypothesis of the research concerned here is that there is no significant difference between
the two classes concerned in effect of problem-based learning on students’ learning achievement and critical
thinking in relation with the learning material concerning dynamic electricity. The criteria for decision making in
the test are that if the significance level obtained > .05, then Ho is accepted and if the significance level obtained
< .05, then Ho is rejected.
Table 14 shows that the significance level obtained by means of the respective procedures of Pillai’s, Wilks’,
Hotelling’s, and Roy’s is .0001 in value. Since the obtained significance level < .05 in value, then Ho is rejected. It
means that there is significant difference between the experimental group and the control group in effect of the
PBL model on students’ learning achievement and critical thinking.
775
(P. 761-780) ISSN 2538–7138 /Online/
Table 15. Results of tests of between-subjects effect.
Source Dependent Variable df F p
Corrected Model Learning Achievement 1 73.888 .0001

Critical Thinking 1 7.046 .0100
Intercept Learning Achievement 1 718.245 .0001
Method Learning Achievement 1 73.888 .0001
Table 15 shows the same values for Corrected Model and Method, with the significance value of .000 for learn-
ing achievement and .0100 for critical thinking. The two figures for the significance value are less than .05 so that
Ho is rejected. Therefore, it is concluded that the PBL model with the method of experiment as in the experimental
group exerts significant effect on students’ physics learning achievement and critical thinking skill compared with
that with the method of demonstration as in the control group.
Discussion
The use of the PBL model employing the method of experiment and the method of demonstration hopefully
would bring about a result conforming to an objective of physics learning, namely, making students show scientific
behaviour and develop experience in using scientific methods. In addition, with the use of PBL students could
develop the ability of reasoning in thinking because PBL puts emphasis on learning with a problem as basis. Tarhan
and Acar-Sesen (2013) found that the mean scores of the students in the experimental group were instructed via PBL
were significantly higher than those in the control group were instructed via teacher-centred approach. Problem
Based Learning Assessment Scale results reflected that students’ positive beliefs increased after each activity. Based
on these results, it can be concluded that PBL instruction is effective in concept learning in chemistry education.
On the whole, the phases of PBL for the two classes in the research have been correctly applied. The pre-test
and post-test results indicate that there is difference in the learning outcome of the two classes. The experimental
group and the control group are the same in the items for the C3 cognitive level correctly answered by students
in the pre-test being four in average number while in the post-test the items for the C3 cognitive level correctly
answered by the experimental group and the control group are respectively nine and four in average number. It
proves that the use of PBL brings more improvement to the cognitive level of C3, namely, the level of applying. In
the average number of correctly- answered items for the C4 cognitive level in the pre-test, the experimental group
is below the control group in position, correctly answering only one such item on the average while two such items
on the average are correctly answered by the other class. There is significant improvement in the average number
of items for the C4 cognitive level correctly answered by the experimental group, the items being six in average
number while such items correctly answered by the control group are four in average number. It is therefore said
that PBL could significantly affect improvement in learning achievement at the cognitive level of C4, namely, at
the level of analysing.
So is the case with the C5 and C6 cognitive levels, the experimental group showing better improvement
compared with the control group. In average gain score, the experimental group gets .63 while the control group
gets .32. Those average gain scores of the two classes fall into the middle score range category. Based on self-
determination theory (SDT) with a multilevel confirmatory factor analysis (MLCFA), it was hypothesised that the
four factors of external regulation, introjected regulation, identified regulation, and intrinsic motivation account
for the covariances among the items of the students’ motivation to study physics in school (Byman et al, 2012).
The results of the study were girls had a statistically higher mean score than did boys for all four factors. Otherwise
only minor gender differences were found.
Such results show that the PBL model could improve students’ learning achievement. The research indicates
that using the method of experiment are more liable of showing better results compared to using the method of
demonstration. It is in line with the objective of using PBL, namely, that the use of PBL would result in a learning
that is meaningful for the students learning to solve problems by applying all the knowledge and ability in their
776
ISSN 2538–7138 /Online/ (P. 761-780)
possession or at least with the use of PBL students could try to obtain the knowledge that they require so that
the learning process would become increasingly more meaningful when they are faced with a situation in which
concepts are applied. Šorgo and Kocijančič (2011) found that the implementation of laboratory work in science
teaching is not so positive. There are findings that many of the exercises are presented to the students as demon-
strations and an expository style is preferred. The differences occur in the way laboratory work is performed. In
the future, efforts should be made to transform expository labs into inquiry and problem-based laboratory work.
Meanwhile, Hakkarainen and Ahtee (2010) find out how pupils will connect their observations and explanations
from a demonstration. Both the observations and the explanations were improved significantly in the second
demonstration even that no teaching was done between the demonstrations. The application of the method of
experiment is more appropriate for moving to the objective favoured by the PBL model because students are
taught to formulate a hypothesis, perform a process of experimenting, make an observation, retrieve data, and
draw a conclusion from the experiment that they have conducted. Teacher participation in a sustained profes-
sional development intervention designed to improve the quantity and quality of guided inquiry-based instruc-
tion in middle school science classrooms and subsequent student academic growth. The results indicate that the
students of participating teachers had significantly higher when compared to students of non-participants. This
study supports prior research findings that inquiry-based instruction helps improve students’ achievement relative
to scientific practices and also provides evidence of increasing student conceptual knowledge (Marshall, Smart,
and Alston, 2017).
The pre-test and post-test results in measuring the critical thinking skill could be seen from the mean score of
22.1 in the pre-test becoming 59.84 in the post-test in the case of the experimental group. The mean score for the
critical thinking skill in the case of the control group also undergoes an increase from being 17.34 in the pre-test
to being 46.72 in the post-test. Seen from the point of pre-test and post-test mean scores, both classes experience
an improvement in critical thinking skill. The pre-test and post-test mean scores also show that on the average
the gain score of the experimental group is .49 and that of the control group is .34. The average gain scores of the
two classes are within a middle score range. From the gain scores it could be seen that the experimental group,
by applying the method of experiment, is better in the learning compared with the control group applying the
method of demonstration. The findings of the measurement of critical thinking overall suggest that the CT skills
in electricity and magnetism (CTEM) test can be used to measure the acquisition of domain-specific CT skills in E &
M, and a good basis for future empirical research that focuses on the integration of CT skills within specific subject
matter instruction (Tiruneh et al., 2017).
From the MANOVA test used to compare the learning in the experimental group with that in the control group,
the results obtained explain that the PBL model using the method of experiment is more influential compared
with the PBL model using the method of demonstration. It is seen in the respective procedures of Pillai’s, Wilks’,
Hotelling’s, and Roy’s all resulting with the value of .0001. That obtained significance value < .05 so that the related
Ho is rejected. Besides, Table 23 shows that the aspects of Corrected Model and Method in the tests of between-
subjects effect come to be the same in value. It could be concluded that the learning session using the PBL model
with the method of experiment is significantly greater in effect compared with that using the PBL model with the
method of demonstration. Therefore, it is suggested that that former kind of learning is the one to be used in every
learning session instead of the latter.
With the above discussion as basis, it could be said that the application of PBL with the method of experiment
could significantly improve achievement in learning physics at the respective cognitive levels of applying, analysing,
evaluating, and creating. In addition, PBL with the method of experiment could also significantly improve students’
critical thinking skill in the categories of respectively giving basic clarification, building bases for a decision, mak-
ing an inference, and making advanced clarification. In line with the research by Tiwari et al. (2006) applying PBL
to improve the thinking of nurses-to-be in Hongkong, PBL could also be applied on senior high school students
to improve their physics learning achievement and their critical thinking skill. Yu, Fan, and Lin (2015) found that
problem solving is often challenging for students because they do not understand the problem-solving process.
The results indicate that context simulation is beneficial for cultivating students’ abilities to establish and analyze
questions and then select and develop solutions. In addition, the project design cultivated the students’ ability
to evaluate results and apply feedback. The findings of this study demonstrate that context-based learning may
effectively enable students to establish and complete the problem-solving process.
777
(P. 761-780) ISSN 2538–7138 /Online/
Conclusions and Suggestion
With all the preceding problem formulation, exposition, and calculation as basis, the conclusion could be drawn
that, with the application of the PBL model in the learning, there is positive effect on students’ learning achievement
and critical thinking skill. In addition, it has been found that the average gain score for learning achievement of the
experimental group and that of the control group have been respectively .63 and .32 and the average gain score
for critical thinking of the experimental group and the control group have been respectively .49 and .34. Each of
the MANOVA calculation procedures employed ends up with the value of .0001 and the tests of between-subjects
effect shows the same value for the aspects of Corrected Model and Method. Therefore, the PBL model using the
method of experiment as in the experimental group is significantly greater in effect compared with the PBL model
using the method of demonstration as in the control group.
The PBL model is one of the learning models that could improve students’ learning achievement and critical
thinking skill but there is a matter that one needs to pay attention to in its application. It is showing the initial
problem in the right way in order that students could understand what they should do. The problem presented
should not be a problem stated as a question for it should be an actually occurring problem.
In order that the results of the research concerned here are improved, it is suggested that additional research
be conducted to measure not only students’ learning achievement and critical thinking skill but also other matters
such as problem-solving abilities, other higher-level abilities, and science process skills or to measure students’
collaboration abilities.
References
Agdas, S. (2013). Effects of problem-based learning on development of critical thinking skills and disposition in engineering (Doctoral
dissertation, University of Florida, Florida). Proquest LLC, UMI 3586578.
Anderson II, J. C. (2007). Effect problem-based learning on knowledge acquisition, knowledge retention, and critical thinking ability of
agriculture students in urban schools (Doctoral dissertation, University of Missouri, Columbia). Proquest LLC, UMI 3322674.
Anderson, L., & Kratwohl, D. R. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educa-
tional objective. New York: Longman.
Brooks, S., Dobbins, K., Scott, J. J. A., Rawlinson, M., & Norman, R. I. (2014). Learning about learning outcomes: The student per-
spective. Journal Teaching in Higher Education, 19 (6), 721-733.
Byman, R., Lavonen, J., Juuti, K., & Meisalo, V. (2012). Motivational orientations in physics learning: A self-determination theory
approach. Journal of Baltic Science Education, 11 (4), 379-392.
Cēdere, D., Jurgena, I., Helmane, I., Tiltiņa-Kapele, I., & Praulīte, G. (2015). Cognitive interest: Problems and solutions in the acquisi-
tion of science and mathematics in schools of Latvia. Journal of Baltic Science Education, 14 (4), 424–434.
Elder, L., & Paul, R. (2007). The miniature guide to critical thinking concepts and tools. Retrieved from http://www.criticalthinking.
org/files/SAM_Analytic_Think2007b.pdf.
Ennis, R. H. (1991, March). Critical thinking: A streamlined conception. Teaching Philosophy, 14 (1), 5-22.
Ennis, R. H. (2015). The nature of critical thinking: Outlines of general critical thinking dispositions and abilities. Retrieved from http://
www.criticalthinking.net/longdefinition.html (2015, July16).
Hake, R. (2015). Analyzing change/gain score. Retrieved from http://www. Physics.indiana.edu/hake (2015, July 15).
Hakkarainen, O., & Ahtee, M. (2010). Pupils connecting observations and explanations in successive demonstrations. Journal of
Baltic Science Education, 9 (3), 167-178.
Henderson, G. (2014). The relationship between problem-based learning and the development of critical thinking skill in higher
education (Doctoral dissertation, Texas A & M University, Texas). Proquest LLC, UMI 3636037
Komives, S. R., & Smedick, W. (2012). Using standard to develop student learning outcomes. Journal New Directions for Student
Services, 2012 (140), 77-88. Doi: 10.1002/ss.20033.
Lau, J. Y. F. (2011). An introduction to critical thinking and creativity: think more, think better. New Jersey: John Willey & Sons Inc.
Marshall, J. C., Smart, J. B., & Alston, D. M., (2017). Inquiry-based instruction: a possible solution to improving student learning of
both science concepts and scientific practices. International Journal of Science and Mathematics Education, 15 (5), 777–796.
Retrieved from https://link.springer.com/article/10.1007/s10763-016-9718-x.
Moore, B. N., & Parker, R. (2009). Critical thinking (9th Ed.). New York: McGraw-Hill Companies.
Moore, D. T. (2007). Critical thinking and intelligence analysis. Washington: Diane Publishing Co, National Defence Intelligence
College.
Oon-Seng Tan (2004). Enhancing thinking through problem-based learning approaches. Singapore: Cengage Learning.
Oon-Seng Tan (2009). Problem-based learning and creativity. Singapore: Cengage Learning.
Rudinow, J., & Barry, E. V. (2008). Invitation to critical thinking (6th Ed.). California: Thomson Wadsworth.
Siew, N. M., & Mapeala, R. (2016). The effects of problem-based learning with thinking maps on fifth graders’ science critical
thinking. Journal of Baltic Science Education, 15 (5), 602–616.
778
ISSN 2538–7138 /Online/ (P. 761-780)
Šorgo, A., & Kocijančič, S. (2011). Presentation of laboratory sessions for science subjects in Slovenian upper secondary schools.
Tarhan, L., & Acar-Sesen, B. (2013). Problem based learning in acids and bases: Learning achievements and students‘ beliefs.
Tiruneh, D. T., Cock, M. D., Weldeslassie, A. G., Elen, J., & Janssen, R. (2017). Measuring critical thinking in physics: development
and validation of a critical thinking test in electricity and magnetism. International Journal of Science and Mathematics
Education, 15 (4), 663-682.
Tiwari, A., Lai, P., So, M., & Yuen, K. (2006). A comparison of the effects of problem-based learning and lecturing on the develop-
ment of students’ critical thinking. Medical Education, 40 (6), 547-554.
Wang, J., Zhang, Y., & Shi, W. (2016). Research on the cognitive level of students’ perceptions of physics models and modeling
mechanism in Chinese high schools. Journal of Baltic Science Education, 15 (2), 204–215.
Yu, C. K., Fan, S. C., & Lin, K. Y. (2015). Enhancing students’ problem-solving skills through context-based learning. International
Journal of Science and Mathematics Education, 13 (6), 1377-1401.
Mundilarto Professor of Physics Education Department Yogyakarta State

University, Colombo Street No. 1, Karangmalang, Yogyakarta
55281, Indonesia.
E-mail: mundilarto@uny.ac.id
Helmiyanto Ismoyo Student of Post Graduate Yogyakarta State University, Colombo
Street No. 1, Karangmalang, Yogyakarta 55281, Indonesia.
779
HISTORY OF CHEMISTRY AS
A PART OF ASSESSMENT OF
STUDENTS’ UNDERSTANDING
OF THE LAW OF CONSERVATION
OF MASS
Abstract. The aim of this research was Vesna D. Milanovic,

to explore students’ ideas about chemical Dragica D. Trivic
reactions and difficulties in understanding
the law of conservation of mass in such re-
actions by using an approach that started
from presentations of scientists’ work
associated with the law. The developed test
items relied on: 1) the historical contents
that illustrate the experimental work of
three scientists (Lavoisier, Landolt and
Introduction
Lomonosov); 2) the description of school
experiments and 3) real life situation. In
The process of construction of scientific knowledge and the cognitive
this way, students would have an oppor-
development of students are dependent upon the interaction of their cog-
tunity to show understanding of the law
nitive structures and the structures of scientific knowledge organized into
of conservation of mass in two contexts,
systems that are provided by the curricula. Classroom assessment, which is a
one based on the stories from the history
vital part of the teaching process and thus inseparable from it, is considered
of chemistry and the other contemporary,
to be a specific sociocultural context with deep impacts on the process of
based on school laboratory experiments
students’ learning. The assessment techniques affect the way individuals
and real life situation. Students of different
perceive themselves and the decisions they make in order to stimulate their
ages were selected for the research: the
own progress in learning. Research results have indicated that the content,
seventh and the eighth grade of primary
presentation and format of a test, and individual psychology influence the
school (age 13–14), and the second year
achievements of students (Danili & Reid, 2005). Students with high achieve-
of grammar school (age 16). The research
ments when tested by one method may not be equally successful when
involved a total of 301 students. The results
tested in another way (Danili & Reid, 2006). When designing and planning
indicated that students’ difficulties were
assessment, it is important to think about different ways in which students’
mostly associated with the predictions
understanding could be reviewed and challenged.
and explanations of mass changes in open
In the research literature many terms refer to students’ faulty views in
reaction systems in which a gas was a
science such as preconceptions, pre-concepts, intuitive ideas or naïve beliefs,
reactant than with the reactions in which a
misconceptions, misunderstandings, intuitive theories, alternative conceptions,
gas was a product.
etc. (e.g., Abimbola, 1988; Barke, Haziri & Yitbarek, 2009; Benson, Wittrock &
Keywords: assessment in chemistry, gram- Baur, 1993; Caramazza, McCloskey & Green, 1980; Chang, Lee & Yen, 2010;
mar school, history of chemistry, experi- Clement, Brown & Zeitsman, 1989; Driver & Easley, 1978; Garnett, Garnett
mental results, law of conservation of mass. & Hackling, 1995; Gilbert & Watts, 1983; Nakhleh, 1992; Taber, 2009; Taber,
2015). Some authors emphasized the difference between the students’ ideas
developed without having any prior knowledge of the subject, termed alter-
native, original or pre-concepts, and those caused by inappropriate elements
Vesna D. Milanovic, of teaching process, termed school-made misconceptions (Barke, Haziri &
Dragica D. Trivic Yitbarek, 2009, p. 21). Misconceptions have been identified in many topics
University of Belgrade, Serbia
learnt in chemistry (Coll & Treagust, 2001; Dhindsa & Treagust, 2009; Nahum,
Hofstein, Mamlok-Naaman & Bar-Dov, 2004; Nakhleh, 1992; Nakiboglu, 2003;
780
ISSN 1648–3898 /Print/ HISTORY OF CHEMISTRY AS A PART OF ASSESSMENT OF STUDENTS’ UNDERSTANDING OF
THE LAW OF CONSERVATION OF MASS
ISSN 2538–7138 /Online/ (P. 781-796)
Park & Light, 2009; Stefani & Tsaparlis, 2009; Taber, 2009; Unal, Costu & Ayas, 2010). In the review of researches
(Kind, 2004) students’ misconceptions have been recognised in eleven conceptual areas of chemistry: states of
matter; particle theory; changes of state; distinguishing between elements, compounds and mixtures; physical
and chemical change; open and closed system chemical events; acids, bases and neutralisation; stoichiometry;
chemical bonding; thermodynamics and chemical equilibrium. Many of the misconceptions relate to the abstract
entities used in chemical explanations at the level of atoms and molecules (Taber, 2009). The research-diagnosed
misconceptions could be very useful for the improvement of chemistry teaching, i.e. classroom activities and
instructional materials (Al-Balushi, Ambusaidi, Al-Shuaili & Taylor, 2012; Ozmen, 2004).
The application of a wide range of teaching and evaluation strategies could bring the science contents and
processes closer to a larger number of students. Many educators and researchers have advocated the presentation
of interesting stories from the history of science as a valuable support for students to learn the contents of natural
sciences and to form a picture about the science development (Table 1). They have stated that these stories could
help in the clarification of various aspects of scientific research, in understanding the humane aspects of science
and the role it plays in the development of different cultures (Folino, 2000; Paixao & Cachapuz, 2000). A positive
effect can be reached by linking the discoveries in chemistry in the past with a wide range of products based on
these discoveries commonly used nowadays (Stock, 2004). In a research of experts’ views about using history and
philosophy of science, Galilli & Hazan (2001a) found that some of them argued that presenting science as a story
of conceptual revolutions could lead to students’ conceptual change and to high effectiveness of instruction.
Table 1. A review of the literature on positive and negative impacts, or the absence of the influence of the
application of history of science on the students’ knowledge and attitudes toward science.
History of science in science education Positive effects Negative or no effects
Enables deeper understanding of scientific con- Galili & Hazan, 2000; Irwin, 2000;
cepts in comparison with another context Lin, 1998; Kim & Irving, 2010.
Seroglou, Panagiotis & Tselfes, 1998;
Sneider & Ohadi, 1998.
Enables understanding of the nature of science Galili & Hazan, 2001b; Abd-El-Khalick & Lederman, 2000;
Faria, Pereira & Chagas, 2012; Leach, Hind & Ryder, 2003.
Irwin, 2000;
Kalman, 2010;
Kim & Irving, 2010;
Solomon, Duveen & Scot, 1992;
Solomon, Duveen & Scot, 1994.
Encourages students’ learning based on errors in Allchin, 2012; -
scientists’ reasoning Dedes, 2005;
Giunta, 2001;
Kipnis, 2011.
Has potential to motivate students to learn science Dibattista & Morgese, 2013; -
and develop positive attitudes about science Faria, Pereira & Chagas, 2012;
Mamlok-Naaman, Ben-Zvi, Hofstein, Menis &
Erduran, 2005.
Historical texts and experiments have been implemented in several approaches to science education (Allchin,
Andersen & Nielsen, 2014), but their actual effect in chemistry teaching has been explored only a few times. How
the contents from the history of chemistry shape students’ further intellectual engagement in chemistry learning
and whether they use the knowledge about the scientists’ work in a new situation have not yet been the focus of
research.
Research Focus
The aim of this research was to explore students’ ideas about chemical reactions and difficulties in understand-
ing the law of conservation of mass in such reactions by using an approach that started from presentations of
scientists’ work associated with mass conservation in chemical reactions. Accordingly, the specific research ques-
781
HISTORY OF CHEMISTRY AS A PART OF ASSESSMENT OF STUDENTS’ UNDERSTANDING OF
(P. 781-796) ISSN 2538–7138 /Online/
tion was: Could the stories about the discovery of the law of conservation of mass help the primary and grammar
school students to understand this law in the context of contemporary school experiments or real life?
The law of conservation of mass was selected as an important basis for students’ understanding of the quantita-
tive aspects of the chemical changes of substances. This is an important starting point for students’ understanding
of the meaning and successful writing of chemical equations, as well as for conducting stoichiometric calculations
according to them. However, the results of several studies have indicated that many students could neither predict
nor explain mass changes in chemical reactions, nor successfully perform stoichiometric calculations (Ben-Zvi, Eylon
& Silberstein, 1988; Hesse & Anderson, 1992; Mulford & Robinson, 2002; Ozmen & Ayas, 2003).
General Background
The conducted research is quantitative in nature (Cohen, Manion & Morrison, 2009). The test was constructed
in order to obtain the answer to the research question, and students of different ages were tested. Testing in each
group of students lasted one classroom period (45 minutes). The research was conducted with students from six
primary schools and three grammar schools.
Sample
A total of 301 students attending the seventh and the eighth grade of primary schools (aged 13 and 14) and
the second year of grammar schools (aged 16) in Serbia took part in the research (Table 2). The schools making up
the sample were selected in such a way that the conditions in them, the state of the equipment that they possessed
and the level of qualification of their teachers corresponded to the average conditions under which chemistry
teaching in our country is conducted.
Table 2. The number of the students in the sample according to their educational level and school type.
Primary Grammar
Schools
7th grade 8th grade 2nd year
The No. of students 90 107 104
Primary education in Serbia is compulsory and lasts eight years (students ages 7 to 14). Chemistry starts be-
ing taught within separate subject in the seventh grade of primary school (students age 13). Secondary education
in Serbia includes either four years of general education in grammar schools, which prepare students for tertiary
level of education, or three or four years of vocational education.
According to the chemistry curricula for primary and grammar schools in Serbia, all students in the sample
have already studied the law of conservation of mass before this research (the law of conservation of mass is
elaborated for the first time in the seventh grade of primary school).
Throughout the research, the ethical guidelines for research within science education were followed. In order
to obtain the consent from the schools, the research proposal was presented to the science committee in each
school. The necessary permissions were obtained from the school science committees and the contracts which
regulate the collaboration between the Faculty of Chemistry and each school were signed. The purpose of the
testing was explained to the students before conducting the research. Participation of the students was voluntarily
and the collected data were treated as confidential.
Development of the Instrument
Based on the results of descriptive content analysis of existing curricula materials and commonly known his-
torical facts about the discovery of the law of conservation of mass (Brock, 2000; Grdenic, 2001; Partington, 1937)
and the work of three scientists: the French scientist Antoine Laurent Lavoisier, the Russian Mikhail Vasilyevich
782
ISSN 2538–7138 /Online/ (P. 781-796)
Lomonosov, and the German chemist (born in Switzerland) Hans Heinrich Landolt, the texts were composed and
included in the test. The test also contained additional texts with descriptions and pictures of the setups of the
experiments in the school laboratory. This allowed the formation of test items that rely on: 1) the historical contents
that illustrate the experimental work of the scientists; 2) the description of experiments in a school laboratory
and 3) situation from real life. In this way, students have the opportunity to show their understanding of the law
of conservation of mass in two contexts, one based on the stories from the history of chemistry and the other
contemporary, based on school laboratory experiments and real life situation.
The test contained 10 items with a total of 18 open and closed type requirements (Appendix 1), designed
according to an analysis of the contents on the law of conservation of mass in currently used chemistry textbooks
for primary and grammar schools. The testing did not involve conducting the experiments described in the test.
One step in this research was to define the indicators of students’ understanding that would be monitored from
the test results. The indicators and the requirements of the items in which they could appear are listed in Table 3.
Table 3. The indicators of the students’ understanding and the items by which they are monitored.
Indicators Description The No. of the item
1 Predicting the results of the Based on the description of the experiments and pictures. 5.A); 6.A); 6.C); 7.A); 8.A); 8.B)
experiment
2 Formulating explanations Based on the text concerning the experimental work of the scientist. 2, 3, 4, 9
Based on the description of the experiments and pictures. 5.C); 6.B); 6.D); 8.C)
3 Formulating conclusions Based on the text concerning the experimental work of the scientist. 1.
Based on the description of the experiments and pictures. 5.B); 7.B)
4 Linking Linking theoretical knowledge and situation from real life. 10.
To confirm the content validity, a group of experts comprising twelve chemistry teachers examined the test:
five from primary schools and seven from grammar schools. The teachers also estimated whether students from all
three groups of respondents during the regular chemistry classes had an opportunity to develop the necessary pre-
knowledge for understanding the information presented textually or by images in the test, and according to that,
whether it was possible to expect that students from each group could respond to the test requirements. Coding
students’ responses, i.e. determining if an answer was completely correct or partially correct was accomplished in
cooperation with these teachers (Appendix 2).
Reliability was examined by the Cronbach’s a coefficient which provides the correlation of each item with
the sum of all the other items. The obtained value was 0.603 and indicates weak correlations among students’
responses to the items (the Cronbach’s a value of 0.70 is often considered as indicator of acceptable reliability).
However, the views that the Cronbach’s a value is an inappropriate indicator of reliability in the situation when
students may hold a fragmented understanding of chemistry could be found in the literature (Adams & Wieman,
2011; Lu & Bi, 2016; Luxford & Bretz, 2014).
Data Analysis
The collected data were processed by using the statistical program for social science (SPSS). A one-way analysis
of variance (ANOVA) was conducted in order to explore whether there were any differences among the groups’
mean scores in the test. The t test was used to explore the statistical significance of the difference in the percentage
of correct answers for each item between groups.
Results of Research
The characteristics of distribution of the overall results are presented in Table 4. The skewness and kurtosis values
of the variables (Table 4) fall within the ranges (±1), therefore the data from each group is normally distributed.
783
(P. 781-796) ISSN 2538–7138 /Online/
Table 4. The characteristics of distribution of the results in all groups of participants (maximal score: 18).
Groups N Min Max Mean SD p Skewness Kurtosis
7th grade of primary school 90 2 16 8.3 3.0 46.1 0.197 -0.270

8th grade of primary school 107 2 15 8.2 2.6 45.6 0.027 0.040
2 year of grammar school
nd
104 4 17 9.2 3.1 51.1 0.273 -0.702
Analysis of variance (Table 5) found that there was a statistically significant difference between the achieve-
ments of three groups of students in the test (F = 3.50; p < .05). The Tukey test (Cohen, Manion & Morrison, 2009, pp.
547-550) found that the means for the seventh graders and the eighth graders were not statistically significantly
different from each other, and that the means for the seventh graders and the grammar school students were not
statistically significantly different from each other. The Tukey test also showed that the mean for the grammar school
students was statistically significantly higher than the mean for the eighth graders at the .05 level.
Table 5. One-way ANOVA results for the test mean scores.
SS df MS F p
Between groups 58.75 2 29.38 3.50 .03*

Test
Within groups 2502.66 298 8.40
Total 2561.42 300
* statistically significant at p < .05
Table 6 presents the percentage of students in all groups who accurately fulfilled the items in the test, as well
as the values of the t-test, which was used to explore the statistical significance of the difference in the percentage
of correct answers between groups.
Table 6. The percentage of correct answers in the groups and the corresponding t values.
The percentage of correct answers t
Items 7th Grade 8th Grade 2nd Year p 7 – p8 p7 – p2 p8 – p2
1. 61.1 58.9 62.5 0.31 –0.20 –0.54

2. 18.9 12.1 28.9 1.32 –1.61 –3.03**
3. 42.2 31.8 46.2 1.51 –0.56 –2.15*
4. 33.3 21.5 12.5 1.86 3.48** 1.74
5. А 34.4 31.8 56.7 0.39 –3.11** –3.64**
5. B 86.7 82.2 61.6 0.86 3.95** 3.33**
5. C 5.6 3.7 14.4 0.64 –2.01* –2.72**
6. А 50.0 48.6 50.0 0.20 0.00 –0.2
6. B 61.1 77.6 77.9 –2.52* –2.55* –0.05
6. C 73.3 67.3 76.9 0.92 0,75 –1.55
6. D 27.8 5.6 16.3 4.26** 1.94 –2.50*
7. А 70.0 63.6 79.8 0.95 –1.75 –2.61**
7. B 51.1 49.5 57.7 0.22 –0.92 –1.19
8. А 62.2 66.4 79.8 –0.61 –2.71** –2.19*
8. B 63.3 68.2 76.9 –0.72 –2.07* –1.41
8. C 28.9 30.8 36.5 –0.29 –1.12 –0.88
784
ISSN 2538–7138 /Online/ (P. 781-796)
The percentage of correct answers t
Items 7th Grade 8th Grade 2nd Year p 7 – p8 p7 – p2 p8 – p2
9. 48.9 69.2 50.9 –2.90** –0.29 2.71**

10. 13.3 12.1 30.8 0.25 –2.90** –3.32**
* The difference in the percentage of correct answers is statistically significant at the level of .05.
** The difference in the percentage of correct answers is statistically significant at the level of .01.
The percentages of the correct answers in all groups of students according to the indicators in Table 3 are
shown in the figures below (Figure 1 - in relation to the prediction of the experimental results, Figure 2 - in relation
to explanations, and Figure 3 - in relation to conclusions).
Figure 1: The percentage of correct predictions of the experimental results.
Figure 2: The percentage of exact explanations in the groups of the respondents.
785
(P. 781-796) ISSN 2538–7138 /Online/
Figure 3: The percentage of correct conclusions.
The analyses of students’ explanations based on the description of the experiments identified four levels of
responses ranging from misconceptions to multiple factor influences. These levels represent a modified version
of the construct map for the conceptual sophistication construct (Brown, Nagashima, Fu, Timms & Wilson, 2010).
The students’ answers labelled as misconceptions were divided into two categories: (i) productive misconception,
which indicates the nature of the problem in reasoning and can serve as the basis for the further intervention in
teaching practice and (ii) unproductive misconception - without clear information for the further intervention in
teaching practice. Examples of students’ explanations of the experiments at different levels for Item 5.C and Item
6.D are presented in Table 7, and Table 8, respectively. These items are chosen for presentation in the article because
the students’ answers on these items indicate their difficulties associated with the explanations of mass changes
in open reaction systems in which a gas is a reactant or a product.
Table 7. Examples of students’ explanations for item 5.C.
Response Level Example Responses

Grade
(Description)
Multiple factors 7th ‘The mass of the rusted nail is greater than the mass of the nail before corrosion. Nikola would have
proven the validity of the law of conservation of mass if he had measured the mass of the second
Refers to the reaction of reactant - oxygen.’
a nail with oxygen and
8th ‘The Lavoisier’s law is valid. The mass of the reacted iron and oxygen (reactants) is equal to the
the validity of the law of
mass of the completely rusted iron nail (product of the chemical reaction).’
conservation of mass.
2nd ‘The mass of the completely rusted nail is greater than the original nail mass, but it is equal to the
mass of a clean nail and oxygen which reacted with the nail. The law confirms this.’
Singular factor 7th ‘The law is valid, because Lavoisier, unlike Nikola, conducted measurements in the sealed vessels
and measured all the gasses inside.’
Refers to the conditions
8th ‘The law is valid, because Nikola did not measure the mass of hermetically sealed vessel with the
under which the experi-
nail.’
ment was conducted.
2nd ‘Nikola did not conduct the experiment in a sealed container. The gaseous substances, which have a
mass, were not measured.’
Reproduction 7th ‘The total mass of the substances taking part in a chemical reaction remains the same before and
after the reaction.’
8 th
2nd
786
ISSN 2538–7138 /Online/ (P. 781-796)

Grade
(Description)
Productive 7th ‘The nail changed its shape but not its mass after the chemical reaction had taken place.’
misconceptions ‘The nail reacted with water and because of that the mass of the nail was changed.’
‘The nail reacted with water and rust was formed. The mass of the nail remained the same.’
‘The law is valid. If we measured the mass of the nail, the mass of rust and the mass of the rusted
nail, we would notice that the sum of masses of the nail and rust is equal to the mass of the rusted
nail.’
‘After the rain, the nail is dissolved in rust and because of that the law is valid. ’
8th ‘The mass of rust is added to the original nail mass.’
‘Metal reacted with rain and a layer of rust was formed on the metal’s surface, but the mass of the
metal did not change.’
‘The surface of the nail changed its physical property but not its mass.’
‘We can apply the law of conservation of mass in this case. The nail reacts, it changes shape, but its
mass remains the same.’
‘The sum of the masses of the nail and rust is equal to the original nail mass.’
‘Oxygen is released in the process of corrosion.’
2nd ‘According to the law of conservation of mass, the mass of the nail before and after the reaction with
water remains the same.’
‘The mass of the nail and the mass of water are equal to the mass of the rusted nail.’
‘Firstly, the nail was shiny and smooth. A layer of rust was formed on the surface of the nail after the
rain. Because of that, the nail has the greater mass than the nail before corrosion. If we removed
rust, the nail would have the same mass as the shiny nail.’
‘Although the nail has lost its lustre after the rain and the rust has been formed, the mass of the nail
remains the same. The shape of the nail is changed.’
‘Because, the iron nail reacted with water and oxides, the mass of which had not been measured.’
‘The nail has changed its physical properties but the mass of the nail has remained the same.’
‘The mass of the nail has probably decreased in the process of corrosion.’
Unproductive 7th ‘A new kind of substance is attached on the surface of the nail. I think it is some kind of lime.’
misconceptions
8 th
‘The nail lost its lustre, which has no mass, and received the rust, which also has no mass, so the
mass of the nail remained the same.’
2nd ‘The air makes iron passive.’
Table 8. Examples of students’ explanations for Item 6.D.

Grade
(Description)
Multiple factors 7th ‘Ana has not covered the burning candle with a jar and because of that the gaseous substances have
escaped and the balance showed the smaller mass after burning. Marija has covered the burning
Refers to the results of candle with a jar so the gaseous substances have remained under the jar and can be measured.’
both experiments.
8th ‘If we retain the gases realised during a chemical reaction the total mass will remain the same. If we
measure the mass of candle without measuring the mass of the released gases, the mass of candle
after burning will be smaller.’
2nd ‘Marija’s candle was covered with a jar, which prevents escape of the gas, so the total mass re-
mained the same. Ana’s candle was not covered and the gas escaped, so the measured mass after
burning was smaller.’
Singular factor 7th ‘The total mass in Marija’s experiment remained the same because all the gases were retained in the
jar.’
Refers to the results of
8th ‘The total mass in Marija’s experiment remained the same because the candle was burning in a
only one experiment.
closed system.’
2nd ‘The candle burnt in a closed jar and all released gases were retained in the jar, so the total mass
remained the same.’
787
(P. 781-796) ISSN 2538–7138 /Online/

Grade
(Description)
Reproduction 7th
‘The total mass of the substances taking part in a chemical reaction remains the same before and
8th
after the reaction.’
2nd
Productive misconcep- 7th ‘The wax was melting while the candle was burning. Although the wax has changed its shape it still
tions has the same mass.’
8th ‘The melted wax flowed to the cardboard in both experiments, so the weight of the candles did not
decrease and the results of measurements of masses were identical.’
‘During the combustion, the mass of the candle will decrease because of its melting.’
2nd ‘In Ana’s experiment, the mass of the candle increased because the candle reacted with oxygen from
the air which had not been measured by the balance. In Marija’s experiment, the candle reacted with
oxygen under the jar which had been measured by the balance.’
‘Ana’s candle only changed its state, while the mass of the candle remained the same. The mass of
smoke was added to the mass of Marija’s candle.’
‘In Ana’s experiment the mass of the candle is smaller because the mass of oxygen which reacts with
the candle is not included.’
‘The wax was melting and accumulating on the cardboard. No matter how much the candle melted,
the mass of the candle remained the same. The candle only changed the shape.’
Unproductive misconcep- 7th ‘Ana and Marija have added new reactants to the reactions, and because of that, we must take into
tions account the mass of the subsequently added reactants.’
8th ‘The mass of the candle flame is small and negligible.’
2nd ‘The oxides are not formed in the experiment.’
Discussion
For the purpose of this research, the obtained results, i.e. what happened when students of different ages were
asked to think about historical episodes and modern examples of a similar thought process, how they used the
information from the texts about history of chemistry in the contemporary situations, are considered according to
the indicators stated in Table 3.
Predicting the Results of the Experiment
Item 5.A requires the prediction of the mass of a rusted nail in relation to the mass it had before rust formation
on the surface of the metal. It is a chemical reaction in an open system, which might produce a cognitive conflict in
relation to the law of conservation of mass. For the primary school students it was the most difficult question of all
associated with the prediction of the experimental results (Figure 1). Wrong answers in each group were mainly due
to the expectation that the mass of the rusted nail would be the same as its mass before corrosion. This is consistent
with the relevant literature indicating that kind of students’ misconception (Driver, Guesne & Tiberghien, 1985, p. 145-
169; Mulford & Robinson, 2002). Wrong answers indicate the tendency of students to strictly follow the formulation of
the law of conservation of mass without reconsideration of the conditions in which the chemical reaction occurred. It
was difficult for the primary school students to predict mass in an open system, especially when a gas was a reactant.
Item 6.A was also about a chemical reaction in an open system. The students were expected to predict the
mass of a candle that had been burning for some time. The most frequent incorrect answers were predictions that
the mass would stay the same.
Item 6.C was about predicting the mass of a jar with a burning candle, and with a lid on the jar (a closed system).
The obtained results indicated that it was easier for the students to predict changes of mass in a closed system than
in an open one.
Item 7.A gives a description of an experiment in which a jar with the solution of Ca (OH)2 was placed on the
balance. A smaller jar with a candle was placed into the jar containing the solution, the candle was lit and a bigger
jar closed with a lid. The question was what mass values the balance would show after some time. The distracter
The balance will show a smaller mass because the candle melted down dominated among the wrong answers of the
788
ISSN 2538–7138 /Online/ (P. 781-796)
primary-school students, while the distracter The balance will show a larger value because a residue will form was the
most frequent answer of the grammar school students. Both answers indicate the students’ misconception associ-
ated with expectation that the mass of the same amount of substances in the solid state is greater than in liquid or
gaseous state (Mulford & Robinson, 2002; Ozmen & Ayas, 2003). However, the wrong answer of grammar school
students indicates that they were thinking about a chemical change in the experiment, while in the same situation
the primary school students who gave the wrong answer were focusing on the physical change (the change of state).
Items 8.A and 8.B contain a description of an experiment in which two glasses were placed on the balance. One
glass was filled with a solution of sodium chloride, NaCl, and the other with a solution of silver nitrate, AgNO3. The
total mass of both glasses with the solutions was weighed. Then the solution from one glass was poured into the
other, and the empty glass was also placed back on the balance. Item 8.A required students to state what indicated
that a chemical reaction had taken place. Based on Figure 7 in the test, the expected answer was the formation of a
residue. In Item 8.B, the students were expected to predict which mass value the balance would show after mixing
the solutions. The wrong answers were mainly linked to the expectation that the balance would show higher values
because of the formation of a precipitate, in accordance with the previous stated misconception.
Formulating Explanations
The overall achievement in all groups was low for the Item 2 related to the explanation of Lavoisier measure-
ment of mass in the basement of the observatory where the air temperature was constant (Figure 2). Partially correct
answers in all groups mainly included the limitation of the measuring instrument, but without the consideration of
the conditions under which the measurements were performed (temperature, air humidity, etc.).
The explanation of the third item required an understanding of the problem related to the attestation of the
validity of the law of conservation of mass in a closed system. Partially correct answers were obtained in all groups
of students. A problem associated with the measuring of the mass of gases released in reactions was not recognized
by the students.
Based on the given texts, the students were expected to provide an explanation to why Landolt sealed hermeti-
cally the H-shaped glass tube (Item 4). The students’ explanations indicate that they accepted the facts about the
experimental procedure but without deeper consideration of the reasons why Landolt had worked in such a way.
According to the partially correct answers obtained in the research, most of the seventh-graders responded that
Landolt had closed the vessel in order to prevent contact of the substances in the vessel with air. The eighth-graders
had a similar idea, but also they answered “to enable the substances in the vessel to react”. Moreover, some of them
stated “to prevent leaking of substances from the vessel when it rotates”. These responses indicate the tendency of
the students to respond according to their own experience rather than to analyse for them a new situation (in this
case, the description of Landolt’s experiment).
The majority of students in all groups gave correct answer for Item 9 which was explicitly stated in the text
about Lomonosov.
In the items related to experiments, the students in all groups were the least successful in trying to explain the
mass of the rusted nail in relation to the mass before corrosion (Item 5.C). Students’ answers (Table 7) showed that
they had not identified oxygen as the second reactant in this reaction (they stated water or air). Among productive
misconceptions, the majority said that in the process of corrosion the nail changed its shape but not its mass. Some
students just quoted the law of conservation of mass in their answer (the level of reproduction).
About 15 % of students in each group provided the answer that the candle stopped burning because there was
not enough air in the jar (Item 6.B). This answer indicates that these students did not make difference between air as
the mixture of gases and oxygen as an element necessary for the combustion.
However, many incomplete explanations were given when the students were expected to compare and con-
trast the total mass of the reaction systems in both experiments: the burning candle in the open and closed system
(Item 6.D). The students’ answers for Item 6.D were analysed and ranged on four levels (Table 8). The answers at the
singular level are related to the explanations provided only by Ana’s experiment or only by Marija’s experiment. The
level of reproduction is comprised of the answers which quoted the law of conservation of mass. The productive
misconceptions in all groups of students indicate that they did not perceive combustion as a chemical reaction. They
explained that the wax was melting while the candle was burning but the mass of the candle remained the same or
decreased. This kind of answer is associated with the previously mentioned misconceptions from literature about the
greater mass of substances in solid state than in liquid state. The lack of comparison and contrasting of the results of
789
(P. 781-796) ISSN 2538–7138 /Online/
two experimental settings indicates the students’ tendency to be focused on one situation rather than to consider
the relation open-closed reaction system.
Item 8.C required an explanation of what mass the balance would show after mixing the solutions of sodium
chloride and silver nitrate. The wrong answers were mainly linked to the expectation that the balance would show
higher values because of residue formation. As was already mentioned, the students’ assumption that the total mass
increases in a precipitation reaction is a known misconception associated with the expectation that solids have a
greater mass than liquids (Ozmen & Ayas, 2003).
Formulating Conclusions
The comparison of the percentages of correct conclusions for the Item 5.B (Figure 3) and explanations for the
Item 5.C (Figure 2) indicates the students’ tendency in all groups to produce a ‘’desirable’’ answer within the framework
of the previously acquired knowledge about the law of conservation of mass without reconsideration of the already
stated prediction of the experimental result (5.A; Figure 1). The expressiveness of this tendency was the lowest in the
group of grammar school students.
The majority of students from each group correctly cited the formulation of the law of conservation of mass
from the text about Lavoisier’s work (Item 1). According to the students’ answers, some of them still thought that the
substance (a reactant) would stay chemically unchanged after the chemical reaction. Among them, some expressed
this in the following way: The mass of a substance does not change in the chemical reaction. Some primary school stu-
dents were not able to distinguish between mass and volume: In spite of the changes in the shape of substances in the
chemical reactions, the volume of substances remains the same. Also, some students did not understand that the law
of conservation of mass is valid for all chemical reactions. They thought that the law is valid only for the reactions in
a closed system.
The lower results related to conclusions in all groups were achieved for the requirement 7.B, which comprises a
description of the most complex experimental set up in which two chemical reactions occur. In the students’ answers
we found the same problems in understanding that we have already elaborated in previous discussion.
Linking
Item 10 should examine students’ ability to link knowledge about the law of conservation of mass with chemical
changes from a real life. A forest fire was given as an example. The problem of applying the knowledge of the law of
conservation of mass to a real life situation was evident in all groups of students.
Conclusions
The achievements in all groups were low, but generally, the grammar school students showed better achieve-
ments than the primary school students. The mean for the grammar school students was statistically significantly
higher than the mean of the eighth graders at the .05 level. The means for the grammar school students and the
seventh graders were not statistically significantly different from each other. In addition, the means for the seventh
graders and the eighth graders were not statistically significantly different from each other.
The results of this research showed that the texts about scientists’ work may provide useful information for some
students and not for others, perhaps those who have already developed some misconceptions associated with the
structure of substances and chemical changes. Actually, the research results indicate that some students’ misconcep-
tions remain in all groups in spite of their insight into the scientists’ work.
The obtained results indicated the students’ difficulties related to predictions and explanations of mass changes in
open reaction systems. These difficulties were mostly associated with the reactions in which a gas was a reactant and,
consequently, the total mass of the system increased than with the reactions in which a gas was a product and total
mass of the system decreased. The known misconception associated with the expectation that solids have a greater
mass than liquids was also confirmed by the results of this research. When students were faced with the situation that
included the same chemical reaction in open and closed systems and the requirement was to compare and contrast
the results of the two experiments, they were not able to operate with the relation open–closed systems. The root
of these difficulties may have been problems with students’ understanding of the atomic model and its use in the
explanation of macroscopic visible changes of substances during chemical reactions, as well as the understanding
790
ISSN 2538–7138 /Online/ (P. 781-796)
of laws according to which reactions occur.

From the achievements of the students, it is evident that they tend to accept information as given in the text
without further consideration of the implications. From some students’ answers, it was clear that they tried to adapt
their answers to the formulation of the law of conservation of mass.
The obtained results indicate that the students try to fit, or ‘’cover’’ explanations of evident phenomena and
processes, which could be reached through common sense, with ready formulations without analytical or critical
thinking. The fact is that they adapt their learning to the usual ways the achievements were assessed, which inevitably
leads to inability to solve the problems set in different contexts from the one they were used to.
Limitations
Further validation may be needed in order to further determine the cause of the low Cronbach’s α value. The
reactions of the students indicated that this way of testing was a complete novelty to them and the limitations of this
research are associated with the motivation of students to engage in this kind of work. In addition, we can assume
that some students found it difficult to understand the historical stories and to use them to formulate their answers
to the items in the test.
Acknowledgements
This research is the result of the work on the project “The Theory and Practice of Science in Society: Multidis-
ciplinary, Educational and Intergenerational Perspectives”, No. 179048, the realization of which is financed by the
Ministry of Education, Science and Technological Development of the Republic of Serbia.
References
Abd-El-Khalick, F., & Lederman, N. G. (2000). The influence of history of science courses on students’ views of nature of science.
Journal of Research in Science Teaching, 37 (10), 1057–1095.
Abimbola, I. O. (1988). The problem of terminology in the study of students’ conceptions in science. Science Education, 72 (2), 175–184.
Adams, W. K., & Wieman, C. E. (2011). Development and validation of instruments to measure learning of expert-like thinking.
International Journal of Science Education, 33, (9), 1289-1312.
Al-Balushi, S. M., Ambusaidi, A. K., Al-Shuaili, A. H., & Taylor, N. (2012). Omani twelfth grade students’ most common misconceptions
in chemistry. Science Education International, 23 (3), 221–240.
Allchin, D. (2012). Teaching the nature of science through scientific errors. Science Education, 96 (5), 904–926.
Allchin, D., Andersen, H. M., & Nielsen, K. (2014). Complementary approaches to teaching nature of science: integrating student
inquiry, historical cases, and contemporary cases in classroom practice. Science Education, 98 (3), 461–486.
Barke, H. D., Haziri, A., & Yitbarek, S. (2009). Misconceptions in chemistry: addressing perceptions in chemical education. Springer–Verlag
Berlin Heidelberg.
Benson, D. L., Wittrock, M. C., & Baur, M. E. (1993). Students’ preconceptions of the nature of gases. Journal of Research in Science
Teaching, 30 (6), 587–597.
Ben-Zvi, R., Eylon, B., & Silberstein, J. (1988). Theories, principles and laws. Education in Chemistry, 25 (3), 89–92.
Brock, W. H. (2000). The chemical tree: A history of chemistry. New York, London: W.W. Norton & Company.
Brown, N. J. S., Nagashima, S. O., Fu, A., Timms, M., & Wilson, M. (2010). A framework for analyzing scientific reasoning in assessments.
Educational Assessment, 15 (3/4), 142–174.
Caramazza, A., McCloskey, M., & Green, B. (1980). Curvilinear motion in the absence of external forces: Naïve beliefs about the mo-
tion of objects. Science, 210 (4474), 1139–1141.
Chang, J.-M., Lee, H., & Yen, C.-F. (2010). Alternative conceptions about burning held by Atayal indigene students in Taiwan. Inter-
national Journal of Science and Mathematics Education, 8 (5), 911–935.
Clement, J., Brown, D. E., & Zietsman, A. (1989). Not all preconceptions are misconceptions: finding ‘anchoring conceptions’ for
grounding instruction on students’ intuitions. International Journal of Science Education, 11 (5), 554-565.
Cohen, L., Manion, L., & Morrison, K. (2009). Research methods in education, sixth edition. London and New York: Routledge.
Coll, R., & Treagust, D. F. (2001). Learners’ mental models of chemical bonding. Research in Science Education, 31 (6), 357–382.
Danili, E. and Reid, N. (2005). Assessment formats: do they make a difference? Chemistry Education Research and Practice, 6 (4), 204–212.
Danili, E. and Reid, N. (2006). Cognitive factors that can potentially affect pupils’ test performance. Chemistry Education Research
and Practice, 7 (2), 64–83.
Dedes, C. (2005). The mechanism of vision: conceptual similarities between historical models and children’s representations. Science
& Education, 14 (7), 699–712.
Dhindsa, H., & Treagust, D. F. (2009). Conceptual understanding of Bruneian tertiary students: chemical bonding and structure.
Brunei International Journal of Science & Mathematics Education, 1 (1), 33–51.
791
(P. 781-796) ISSN 2538–7138 /Online/
Dibattista, L., & Morgese, F. (2013). Introducing history (and philosophy) of science in the classroom: a field research experience in
Italy. Science & Education, 22 (3), 543–576.
Driver, R., & Easley, J. (1978). Pupils and paradigms: a review of literature related to concept development in adolescent science
students. Studies in Science Education, 5 (1), 61-84.
Driver, R., Guesne, E., & Tiberghien, A., (Eds.). (1985). Children’s ideas in science. Open University Press: Milton Keynes (p.145-169)
Faria, C., Pereira, G., & Chagas, I. (2012). D. Carlos de Bragança, a pioneer of experimental marine oceanography: Filling the gap
between formal and informal science education. Science & Education, 21 (6), 813–826.
Folino, D. (2001). Stories and anecdotes in the chemistry classroom. Journal of Chemical Education, 78 (12), 1615–1618.
Galili, I., & Hazan, A. (2000). The influence of a historically oriented course on students’ content knowledge in optics evaluated by
means of facets-schemes analysis. American Journal of Physics, 68 (S1), 3–15.
Galili, I., & Hazan, A. (2001a). Experts’ views on using history and philosophy of science in the practice of physics instruction. Science
& Education, 10 (4), 345–367.
Galili, I., & Hazan, A. (2001b). The effect of a history-based course in optics on students’ views about science. Science & Education,
10 (1), 7–32.
Garnett, P., Garnett, P., & Hackling, M. (1995). Students’ alternative conceptions in chemistry: A review of research and implications
for teaching and learning. Studies in Science Education, 25 (1), 69–95.
Gilbert, J. K., & Watts, D. M. (1983). Concepts, misconceptions and alternative conceptions: changing perspectives in science educa-
tion. Studies in Science Education, 10 (1), 61–98.
Giunta, C. J. (2001). Using history to teach scientific method: The role of errors. Journal of Chemical Education, 78 (5), 623–627.
Grdenic, D. (2001). Povijest kemije [History of Chemistry]. Novi Liber, Zagreb: Skolska knjiga.
Hesse, J. J., & Anderson, C. W. (1992). Students’ concepts of chemical change. Journal of Research in Science Teaching, 29 (3), 277–299.
Irwin, A. R. (2000). Historical case studies: Teaching the nature of science in context. Science Education, 84 (1), 5–26.
Kalman, C. (2010). Enabling students to develop a scientific mindset. Science & Education, 19 (2), 147–163.
Kim, S. Y., & Irving, K. E. (2010). History of science as an instructional context: student learning in genetics and nature of science.
Science & Education, 19 (2), 187–215.
Kind, V. (2004). Beyond appearances: students’ misconceptions about basic chemical ideas. Resource document. Royal So-
ciety of Chemistry. Retrieved 1/6/2015, from http://www.rsc.org/learn-chemistry/resource/res00002202/beyond-
appearances?cmpid=CMP00007478.
Kipnis, N. (2011). Errors in science and their treatment in teaching science. Science & Education, 20 (7), 655–685.
Leach, J., Hind, A. and Ryder, J. (2003). Designing and evaluating short teaching interventions about the epistemology of science
in high school classrooms. Science Education, 87 (6), 831– 848.
Lin, H. (1998). The effectiveness of teaching chemistry through the history of science. Journal of Chemical Education, 75 (10), 1326–1330.
Lu, S., & Bi, H. (2016). Development of a measurement instrument to assess students’ electrolyte conceptual understanding. Chemistry
Education Research and Practice, 17 (4), 1030-1040.
Luxford, C. J., & Bretz, S. L. (2014). Development of the bonding representations inventory to identify student misconceptions about
covalent and ionic bonding representations. Journal of Chemical Education, 91 (3), 312-320.
Mamlok-Naaman, R., Ben-Zvi, R., Hofstein, A., Menis, J., & Erduran, S. (2005). Learning science through a historical approach: does
it affect the attitudes of non-science-oriented students towards science? International Journal of Science and Mathematics
Education, 3 (3), 485–507.
Mulford, D. R., Robinson, W. R. (2002). An inventory for alternate conceptions among first-semester general chemistry students.
Journal of Chemical Education, 79 (6), 739–744.
Nahum, T. L., Hofstein, A., Mamlok-Naaman, R., & Bar–Dov, Z. (2004). Can final examinations amplify students’ misconceptions in
chemistry? Chemistry Education Research and Practice, 5 (3), 301–325.
Nakhleh, M. B. (1992). Why some students don’t learn chemistry. Journal of Chemical Education, 69 (3), 191–196.
Nakiboglu, C. (2003). Instructional misconceptions of Turkish prospective chemistry teachers about atomic orbitals and hybridiza-
tion. Chemistry Education Research and Practice, 4 (2), 171–188.
Ozmen, H. (2004). Some student misconceptions in chemistry: A literature review of chemical bonding. Journal of Science Education
and Technology, 13 (2), 147–159.
Ozmen, H., & Ayas, A. (2003). Students’ difficulties in understanding of the conservation of matter in open and closed-system chemi-
cal reactions. Chemistry Education Research and Practice, 4 (3), 279–290.
Paixao, M. F., Cachapuz, A. (2000). Mass conservation in chemical reactions: the development of an innovative teaching strategy
based on the history and philosophy of science. Chemistry Education Research and Practice, 1 (2), 201–215.
Park, E., Light, G. (2009). Identifying atomic structure as a threshold concept: Student mental models and troublesomeness. Inter-
national Journal of Science Education, 31 (2), 233–258.
Partington, J. R. (1937). A short history of chemistry. London: MacMillan and Co.
Seroglou, F., Panagiotis, K., & Tselfes, V. (1998). History of science and instructional design: The case of electromagnetism. Science
& Education, 7 (3), 261–280.
Sneider, C., & Ohadi, M. M. (1998). Unravelling students’ misconceptions about the Earth’s shape and gravity. Science Education, 82
(2), 265–284.
Solomon, J., Duveen, J., Scot, L. (1992). Teaching about the nature of science through history: Action research in the classroom.
Journal of Research in Science Teaching, 29 (4), 409–421.
Solomon, J., Duveen, J., & Scott, L. (1994). Pupil’s images of scientific epistemology. International Journal of Science Education, 16
(3), 361 – 373.
792
ISSN 2538–7138 /Online/ (P. 781-796)
Stefani, C., Tsaparlis, G. (2009). Students’ levels of explanations, models, and misconceptions in basic quantum chemistry: A phe-
nomenographic study. Journal of Research in Science Teaching, 46 (5), 520–536.
Stock, J., (2004). The teaching of the history of chemistry. Journal of Chemical Education, 81 (6), 793–794.
Taber, K. S. (2009). Challenging misconceptions in the chemistry classroom: Resources to support teachers. Educació Química, 4, 13-20.
Taber, K. S. (2015). Alternative conceptions/frameworks/misconceptions. In R. Gunstone (Ed.), Encyclopedia of science education (pp.
37-41). Berlin-Heidelberg: Springer-Verlag.
Unal, S., Costu, B., & Ayas, A. (2010). Secondary school students’ misconceptions of covalent bonding. Journal of Turkish Science
Education, 7 (2), 3–29.
Appendix 1: The test
Antoine-Laurent Lavoisier (1743–1794)
Lavoisier was a French chemist and is widely considered as the founder of modern chemistry. He intro-
duced quantitative measurement to the laboratory. A precise balance was his main instrument. Many of the
Lavoisier’s experiments were conducted in sealed glass containers from which matter could not escape or
enter. He used accurate balances that could measure very small changes in mass during his experiments
and carefully weighed the reactants and products of a chemical reaction. Lavoisier conducted the process of
mass measurement in the basement of an observatory where the air temperature was constant.
In 1774 Lavoisier performed the following experiment: he warmed a tin in a sealed vessel containing air. He
noticed the formation of a substance on the metal surface, marked at that time calx. Today we know that this
was a layer of oxide. Lavoisier measured the vessel together with the metal, metal oxide and the rest of the
air. He discovered that new substances were formed but that the container and its contents had the same
mass as before heating. When he measured the mass of the new solid substances, he discovered that they
were heavier than the original tin he had heated. In this way, he determined that it must have gained mass
from the air. In experiments with phosphorus and sulphur, both of which burned readily, Lavoisier showed
that they gained mass by combining with air. Based on these experiments, Lavoisier concluded that air
contained several gases, one of which reacted with the elements in the experiments.
His findings implied that the total mass of the substances taking part in a chemical reaction remains the
same before and after the reaction. Lavoisier established the law of conservation of mass, and chemistry
became an exact science, one based on careful measurement.
1. Imagine you are Antoine Lavoisier and you conducted the experiments mentioned in the text. Based on the experimental results suggest the formula-
tion of the law of conservation of mass.
2. Why is it important to perform measurements of mass on the same balance and under the same conditions?
Hans Heinrich Landolt (1831–1910)
Lavoisier’s contemporaries had difficulties in proving the law of conservation of

mass. The main problem was associated with the detection and measurement
of all the reactants or products. It was particularly difficult to detect and measure
the mass of gaseous reactants or products.
Landolt was a German chemist. He was born in Zurich in Switzerland. He ex-
perimentally verified the law of conservation of mass almost 100 years after its
discovery by using an H-shaped glass tube (Figure 1). He filled silver nitrate into
limb A and hydrochloric acid into limb B. The tube was hermetically sealed and
weighed before the chemical reaction. The reactants were mixed by inverting
and shaking the tube. A white precipitate of silver chloride was formed. The tube
was weighed again. Landolt observed that total mass had remained practically
constant after the reaction.
Fig. 1
3. What was the problem in experimentally proving the law of conservation of mass?
4. Why did Landolt use a hermetically sealed H-shaped glass tube in his experiment for the verification of the law of conservation of mass?
Five primary school students, Nikola, Ana, Marija, Jovan and Katarina had read the text on Lavoisier’s work and his discovery of the law of conserva-
tion of mass. They wanted to experimentally verify the validity of the law.
Nikola wanted to experiment with a new nail. He measured the mass of the nail and recorded it. Then Nikola left the nail outside for a few rainy days.
When he took the nail again, he noticed that the nail’s surface was covered with a layer of rust. The student measured the mass of the rusty nail on
the same balance.
793
(P. 781-796) ISSN 2538–7138 /Online/
5.A. Circle the letter in front of the correct answer. The mass of the rusted nail in relation to the mass it had before rust formation on the surface of the
metal was:
a) larger b) smaller c) the same
B. Circle the letter in front of the correct answer. Does the student’s experiment confirm the Lavoisier’s law of conservation of mass after more than
200 years from its discovery?
a) Yes, it does. b) No, it does not.
C. Explain the answer.
Ana first measured the mass of a candle (Figure 2) and recorded it. Then she lit the candle and left it to burn for a while (Figure 3). Subsequently, Ana
extinguished the candle and measured its mass again (Figure 4).
Fig. 2 Fig. 3 Fig. 4
6. A. Circle the letter in front of the correct answer. What mass value will the balance show in Figure 4 in comparison with the mass in Figure 2?
a) Larger b) Smaller c) The same
The setting of Marija’s experiment is shown in Figure 5 and Figure 6. She measured the mass of the burning candle covered with a jar and the mass
of the candle when it extinguished.
Fig. 5 Fig. 6
B. Why did the candle extinguish under the jar?
C. Circle the letter in front of the correct answer. What mass value will the balance show in Figure 6 in comparison with the mass in Figure 5?
a) Larger b) Smaller c) The same
D. Explain the results of Ana’s and Marija’s experiments.
794
ISSN 2538–7138 /Online/ (P. 781-796)
Jovan placed a jar with a solution of Ca (OH)2 on the balance. A smaller jar with a candle was placed in the jar containing the solution (the top of
smaller jar was over the level of solution of Ca (OH)2 in a bigger jar). The candle was lit and a bigger jar closed with a lid.
7. A. Circle the letter in front of the correct answer. What mass value will the balance show after some time?
a) The balance will show a smaller mass because the candle diminished due to chemical change.
b) The balance will show a smaller mass because the candle melted down.
c) The balance will show a larger value because a residue will form.
d) The balance will show the same value as at the beginning of the experiment.
B. What could Jovan conclude on the validity of the law of conservation of mass based on the presented experiment?
The setting of Katarina’s experiment is presented in Figure 7.
Fig. 7
8. A. What is the evidence that a chemical reaction occurred?

B. Circle the letter in front of the correct answer. The mass of the products of this reaction in comparison with the mass of the reactants is:
a) larger b) smaller c) the same
C. Explain the answer.
Mikhail Vasilyevich Lomonosov (1711–1765)
Lomonosov was a Russian poet, grammarian and scientist. He hypothesized on the structure of matter, on
the nature of combustion, and on heat. Lomonosov placed great emphasis on quantitative measurements in
his work and had the idea about the conservation of mass in chemical reactions.
The idea of conservation of mass he expressed in the following way: “All changes in nature are such that
inasmuch is taken from one object insomuch is added to another. So, if the amount of matter decreases in
one place, it increases elsewhere.”
The law of conservation of mass implies that matter can neither be created nor destroyed, so the number
and type of atoms must be the same before and after the chemical reaction.
9. What is the main point from Lomonosov’s formulation of the law of conservation of mass?
10. Forest fire is a huge environmental problem. The mass of ash after forest fire is less than the mass of wood before. Is the law of conservation of
mass valid in this situation? Explain your answer.
795
(P. 781-796) ISSN 2538–7138 /Online/
Appendix 2
The principles of coding the open-ended items in the test
Item Item was coded correct if the student:
1 Stated that the total mass of substances is the same before and after the chemical reaction.
2 Explained the importance of performing measurements of the mass on the same balance, and under the same conditions.
A completely correct answer includes both the importance of the same balance and the same conditions for obtaining the
accurate and precise results of the measurements of mass. If the student included only one of these in his/her answer, the
answer was coded as partially correct.
3 Explained the problems the scientists had in proving the law of conservation of mass related to the inability to detect and
measure the gaseous reactants or products in chemical reactions.
4 Explained that the shape of hermetically sealed glass tube enables the measurements of the total mass of separated
reactants in the limbs and the total mass of products of chemical reaction after mixing the substances.
5.C Explained correctly the answers chosen for Items 5.A and 5.B. The correct explanation refers to the reaction of the nail with
oxygen from the air and the nail mass increasing.
6.B Explained the candle burning out under the jar. The correct explanation refers to the decrease in the concentration of
oxygen and the increase in the concentration of carbon dioxide under the jar during combustion, which caused the candle
to burn out.
6.D Provided a full explanation of the results of Ana’s and Marija’s experiments. The full explanation reveals a distinction
between Ana’s and Marija’s results of the measurement of mass of the burning candle in an open and closed system.
7.B Made a correct conclusion about the validity of the law of conservation of mass based on the result of Jovan’s experiment
conducted in a closed system.
8.A Pointed to the formation of the residue as evidence that the chemical reaction had taken place.
8.C Explained the correctly chosen answer to Item 8.B. The correct explanation refers to the same total mass of two solutions at
the beginning of the experiment and the total mass of mixture with the residue after mixing the solutions.
9 Explained that the law of conservation of mass implies that matter can neither be created nor destroyed.
10 Explained that the mass of wood and oxygen before the forest fire is equal to the sum of masses of ash and released gases
during combustion.
Received: July 15, 2017 Accepted: September 28, 2017
Vesna D. Milanovic MSc, Teaching Assistant of the Chair of Chemical Education,

(Corresponding author) University of Belgrade, Faculty of Chemistry, Studentski trg 12-
16, 11000 Belgrade, Serbia.
E-mail: vesnamilanovic@chem.bg.ac.rs
Website: http://www.chem.bg.ac.rs/osoblje/461-en.html
Dragica D. Trivic Ph.D., Associate Professor and Head of the Chair of Chemical
Education, University of Belgrade, Faculty of Chemistry,
Studentski trg 12-16, 11000 Belgrade, Serbia.
E-mail: dtrivic@chem.bg.ac.rs
Website: http://www.chem.bg.ac.rs/osoblje/26-en.html
796
THE EFFECTS OF CURRICULUM,
GENDER AND STUDENTS’
FAVORITE SCIENCE SUBJECT ISSN 1648-3898 /Print/
ON INDONESIAN HIGH-SCHOOL ISSN 2538-7138 /Online/
STUDENTS’ CONCEPTIONS OF
LEARNING SCIENCE
Abstract. Conceptions of learning have

Arif Rachmatullah, been known as having influence on stu-
Sariwulan Diana, dents’ learning outcomes, the one of which
Minsu Ha is science learning as to be a scientifically
literate person. Even, the effects of students’
conceptions in learning have been known,
but the contributing factors are still vague.
This research aims to explore Indonesian
high-school students’ conceptions of learn-
ing science (COLS), to find out if gender and
Introduction students’ favorite science subject cause
differences in their COLS, and to validate
For more than a century of research, students’ cognitive aspects when the COLS instrument by using Rasch
learning have been scrutinized in the fields of educational studies, cognitive analyses. Thirty-one items measuring six
sciences, and psychology. In science education, students’ cognitive constructs COLS were administered to 609 Indonesian
have been recognized because of the association issue with the science con- high-school students. Rasch analyses, an
tents that are well known to need more cognitive processing when being independent sample t-test, analysis of
learned. In addition, they are also being scrutinized because, in this century, variance (ANOVA), and cluster analyses
people believe that persons who have higher level cognition in science can featuring chi-square tests of interdepend-
be more valuable and efficient workers (DeBoer, 2000). Nevertheless, in the ence were used to answer the research
last few decades, researchers in various education-related fields have found questions. Based on the analyses, it was
out that there is an affective aspect that equally contributes to students when found that the COLS instrument was best
they are learning. Many have suggested (e.g. Osborne, Simon & Collins, 2003; fitted as six-dimensional. Gender difference
Schumm & Bogner, 2016; Thompson & Mintzes, 2002; Vermunt, 1996; Watts & was emerged in memorizing, and differ-
Alsop, 2000) that students’ affective aspects can also positively or negatively ences based on students’ favorite science
influence the students’ cognitive constructs. Therefore, recently the research of subject were also found in memorizing
students’ affective aspects when learning has been rapidly increasing. Among and calculating and practicing. Finally,
students’ affective aspects when learning science, studies on motivation in the results of cluster analyses showed that
learning science (e.g., Glynn, Taasoobshirazi & Brickman, 2009; Lee, Hayes, Indonesian students were divided into three
Seitz & O’Conner, 2016; Schumm & Bogner, 2016) and self-efficacy in learning different classes based on their COLS, and
science (e.g., Robnett, Chemers & Zurbriggen, 2015; Wang, Tsai & Wei, 2015) that the clusters were significantly related
have been the most prevalent, while there are few studies of conceptions of to the school locations.
learning science. Keywords: conceptions of learning sci-
“Conceptions of learning science” refers to how students conceive of their ence, gender, Indonesia, science learning,
science learning based on their experiences (Chiou, Liang & Tsai, 2012; Sadi & secondary level.
Uyar, 2013; Tsai, 2004; Vermunt & Vermetten, 2004). Buehl & Alexander (2001)
and Tsai (2004) noted that conceptions of learning are related to students’ Arif Rachmatullah
academic epistemological beliefs. Even though there have been few studies of Kangwon National University, Republic of Korea
Sariwulan Diana
students’ conceptions of learning science, compared to other affective aspects, Indonesia University of Education, Indonesia
earlier studies have noted that students’ conceptions of learning science are Minsu Ha
highly correlated with their academic outcomes (Allan, 2003; Dart et al., 2000; Kangwon National University, Republic of
Korea
Pillay, Purdie & Bopulton-Lewis, 2000; Purdie, Hattie & Douglas, 1996; Sinatra,
797
(P. 797-812) ISSN 2538–7138 /Online/
2001) and cognitive strategies (Hong & Salili, 2000; Purdie, Hattie & Douglas, 1996). In addition, Tsai, Ho, Liang & Lin
(2011) noted that students’ conceptions of learning science are a crucial part of the affective aspects of learning that
can offer more insight into the way students learn science. Thus, it could show how a science curriculum has been
implemented in the perspective of learners, which can be considered an attempt to evaluate the science curriculum.
In Indonesia, in 2013 a new curriculum was implemented in some regions, and in 2015 all Indonesian schools
implemented it. To date, around four years after the implementation of the 2013 curriculum, there have been few
studies conducted to evaluate this new curriculum, especially in the field of science subjects. This new curriculum is
focused on increasing students’ scientific literacy, as a response to various international reports, such as the Programme
for International Student Assessment (PISA) and the Trends in International Mathematics and Science Study (TIMSS)
(Mendikbud, 2016). The reports from PISA and TIMSS noted that Indonesian students’ science literacy in the few last
decades has always been in the bottom tenth among participating countries (Mullis, Martin, Foy & Hooper, 2016;
OECD, 2016). As some studies of conceptions of learning science mentioned earlier have noted that it is related to
science achievements which is one of them is science literacy, by conducting this research, it can measure how far
along the implementation of this new science curriculum is. Thus, based on the findings later, it can be used to sug-
gest what efforts are needed to change students’ conceptions of learning science. As Gummer & Shepardson (2001)
have suggested, changes in students’ conceptions of learning in a specific subject are not aimed only at obtaining
better concepts, but also at producing coherence between two important aspects, learning and assessments. It also
should be noted here that one finding from the refining of the science curriculum in Turkey has shown that a better
revised science (biology) curriculum has highly influenced the students’ conceptions of learning towards the subjects
(Sadi & Lee, 2017).
Besides the influence of curriculum on students’ conceptions of learning science, some factors were also found
to significantly contribute to students’ conceptions of learning science, particularly gender (e.g., Den Brok, Telli, Cakiro-
glu, Taconis & Tekkaya, 2010; Lonka & Lindblom-Ylänne, 1996; Sadi & Çevik, 2016) and the specific domain of science
subjects (e.g., Buehl & Alexander, 2001; Lin, Liang & Tsai 2015; Sadi & Lee, 2017; Sadi, 2015; Tsai, 2006). In terms of the
gender issue in cognitive and affective aspects of learning science, Severiens & Dam (1997), in their research of gender
differences in learning styles, have pointed out that the differences in learning and learning outcomes are derived
from the inequality in learning environments that may result from teachers’ and society’s stereotyping of gender. In
fact, UNESCO and OECD have reported that in the field of Science, Technology, Engineering, and Mathematics (STEM),
Indonesia was identified as one of the countries that have the most equality in gender (OECD, 2015; UNESCO, 2015).
The UNESCO and OECD reports have motivated the current research to scrutinize whether gender differences exist
in Indonesian high-school students’ conceptions of learning science or not. Another issue brought out in current
research is about the differences in domain-specific science. Chiou, Liang & Tsai (2012), Lee, Johanson & Tsai (2008),
and Tsai (2004) have noted that students’ conceptions of learning science differ from the different domains of science;
so the measurement should be handled differently and aimed more specifically toward different science subjects.
That is, if we want to understand students’ concepts about learning biology, the instrument should be only focused
on biology (Sadi & Lee, 2017). But, since some studies of students’ views toward science have found differences
based on different students’ majors (e.g., Perkins, Barbera, Adams & Wieman, 2007), hypotheses are established on
when students are being asked about their conceptions about science learning in general, their conceptions would
be based on their favorite science subjects, because the image of science conceived by students is closely related to
their favorite subject. In fact, science in an Indonesian upper-secondary level is divided into three branches, namely,
Biology, Chemistry, and Physics. Entwistle & Tait (1990) have also suggested that students differ in their approaches
to learning based on their preferred or favorite course. Therefore, current research also attempts to find out whether
differences in conceptions of learning science are based on students’ favorite science subjects or not.
Literature Review
Conceptions of Learning Science
The term “conceptions of learning” was first used by Säljö (1979) in the study on interviewing adults with dif-
ferent levels of education about their experiences; one of the questions was ‘what they think about learning’. Many
have followed Säljö’s study, and the exact definition of “conceptions of learning” has been established, specifically
related to science subjects. The conceptions of learning science are defined as the learner’s beliefs and knowledge
about learning science and about the context of learning science (e.g., learning tasks, activities, and strategies) that
798
ISSN 1648–3898 /Print/ THE EFFECTS OF CURRICULUM, GENDER AND STUDENTS’ FAVORITE SCIENCE SUBJECT ON
ISSN 2538–7138 /Online/ (P. 797-812)
were established based on their experiences, leading to their interpretation, framing, and reflection on those learn-
ing science experiences (Chiou, Liang &Tsai, 2012; Lai & Chan, 2005; Liang & Tsai, 2010; Lee, Lin & Tsai, 2013; Vermunt
& Vermetten, 2004). In the first study of conceptions of learning conducted by Säljö (1979), five conceptions were
found: (1) increasing knowledge, (2) memorizing, (3) acquiring facts and methods, (4) abstraction of meaning, and
(5) interpretative process intended to understand the reality. Some have found new conceptions of learning held by
students (e.g., Marton, Dall’Alba & Beaty, 1993; Marshall, Summer & Woolnough, 1999), but there were no studies of
conceptions of learning in science subjects, until the study conducted by Tsai (2004).
In the field of science, Tsai (2004) suggested seven kinds of conceptions based on his phenomenographic study
of 120 Taiwanese high-school students. Those seven conceptions were concisely summarized by Lee, Johanson &
Tsai (2008) as follows:
1) memorizing (e.g., learning science is memorizing definitions, formulae, and terms in science textbooks),
2) testing (e.g., learning science is viewed as a way to pass science exams and obtain high science scores),
3) calculating and practicing (e.g., learning science is regarded as practicing on solving science problems
and manipulating numbers and formulae),
4) increasing knowledge (e.g., learning science is viewed as one’s acquisition of scientific knowledge),
5) applying (e.g., learning science is perceived as generalizing the obtained scientific knowledge into other
types of unknown problems),
6) understanding (e.g., learning science means integrating the consistent theories of science) and
7) seeing in a new way (e.g., learning science is perceived as obtaining new perspectives towards the acquired
scientific knowledge which later can be used to interpret natural phenomena in a new way). (p. 198).
The last conception – seeing in a new way - was rarely possessed by students. Therefore, in the later study of
Lee, Johanson & Tsai (2008), who conducted a study on validating the instrument used to measure conceptions of
learning science (COLS), that seventh conception was merged with the understanding dimension, and formed the
‘understanding and seeing in a new way’ conception. This merging of conceptions, whereby understanding is placed
in the highest hierarchy of concepts, is also supported by Biggs (1994), who argued that learning should be concerned
with connecting new knowledge to earlier knowledge to establish the understanding held by learners.
For the higher categorization of conceptions of learning, whether learning in general or specifically in science,
Marton, Dall’Alba & Beaty (1993), later supported by Tsai (2004), stated that those five, six, or seven conceptions of
learning are categorized into two distinct groups: reproductive and constructivist. Both of these studies suggested
that students who hold the first three conceptions of learning be recognized as the reproductive group, who view
learning only quantitatively, that is, as just acquiring and accumulating knowledge. In contrast, students who hold the
remaining three or four higher conceptions are categorized as constructivists, who conceive of learning qualitatively,
not seeing learning as the outcome/achievements solely, but as how they can understand the knowledge and apply
it in their lives. That is why Tsai et al. (2011) argued that constructivist students are more intrinsically motivated or
more oriented toward mastery.
The framework and assessment of conceptions of learning suggested by Tsai (2004) and Lee, Johanson & Tsai
(2008) are used in current research. But to date, there is no research of COLS which has used Rasch analysis, which is
the method suggested by Lee, Johanson & Tsai (2008), because they did not test the COLS instrument by using that
method when they developed the instrument. Thus, this research also aims to uncover whether the COLS instru-
ment is a best fit in the Rasch model or not. It also should be noted here that Rasch analysis is emphasized for use in
validating the science education instrument (Boone, Townsend & Staver, 2011; Johnson & Sondergeld, 2014; Romine,
Walter, Boose & Todd, 2017).
The Indonesian 2013 Science Curriculum
In 2013, a new science curriculum was published and implemented in Indonesia and then revised at the minor
level in 2016. Despite the 2016 revision, the contents of the curriculum are still the same. As aforementioned, this
new curriculum 2013 is one effort by the Indonesian government to respond to international reports, such as PISA
and TIMSS, about to the low level of science literacy achieved by Indonesian students during recent decades. Thus,
the main goal of this new curriculum is to improve Indonesian students’ science literacy in international assessment
(Mendikbud, 2016). There are several new points of this curriculum, especially in terms of learning and assessment.
799
(P. 797-812) ISSN 2538–7138 /Online/
First, this new curriculum brings up the notion of ‘one subject for all competencies,’ which means that one
subject, for instance, science, should not only increase knowledge (cognitive domain) more than the previous cur-
riculum had, but also should cover skills and attitudes. Thus, the way to assess students has also changed, from the
previous one of just testing students to measure their knowledge, but now to also measure their skills and attitudes
and this effort leads to the decrease in frequency of testing. In fact, whereas the national examination was the main
way in which students were judged as ready for graduation from high school, now schools want to find out how
good the attitudes of the students are in order to decide whether students can graduate or not. That is, in this new
curriculum, attitude is most important.
The second aspect is that all subjects, not only science, should be taught by employing a scientific approach.
This is an effort to increase students’ science literacy by emphasizing the connection of science with other daily-life
concerns. Teachers are encouraged to use teaching methods that are closely related to a scientific approach, such as
problem-based and project-based learning, and using discovery learning rather than traditional methods. Finally, a
combination of the “Structure of Observed Learning Outcome (SOLO) Taxonomy” suggested by Biggs & Collin (1982)
and Bloom’s Taxonomy is used as the framework for developing students’ knowledge, skills and attitudes. The use
of those taxonomies as a framework allows the teaching methods in high school to focus not on the lower levels
of cognitive process or surface knowledge, such as remembering facts and terms, but on applying (C3 Bloom) and
analyzing (C4 Bloom), which focus more on constructed knowledge, as suggested by Biggs & Collin (1982).
Research Questions
Based on the background and descriptions above, three research questions as the focus of the current re-
search are as follows:
1. How well does the conceptions of learning science (COLS) instrument perform in the sample of Indo-
nesian high-school students based on Rasch analyses?
2. Do students’ conceptions of learning science differ by gender and favorite science subject?
3. How do Indonesian high-school students conceive of their science learning?
General Background
The current research was conducted by using the principle of quantitative survey method. The data were
collected through the research process to examine Indonesian upper-secondary level students’ conceptions of
learning science. The survey was carried out starting from March to May 2017 to upper-secondary level students
in West Java province, Indonesia.
Participants
The data on Indonesian high-school students’ conceptions of learning science were gathered from students
in eight high schools that differed in their accreditation level (from A to C level), school type (Islamic-based and
general type), and location in four different cities. The total number of participants was 609 students. The partici-
pants also differed in their high-school grade, from first year to third year, but predominantly were in the first and
second years, because the third-year students were preparing for the national examination and so, many could not
participate. Demographically, the participants were 37% male and 63% female, but two students did not indicate
their gender. In religion, 98% of the participants were affiliated with Islam.
For the third research question, data on the students’ favorite science subjects were gathered by asking them
to choose one of the three science branches, biology, chemistry, and physics; Indonesia does not provide earth
and space science as a school science subject. Of the 609 students, 25 students (around 4%) did not indicate their
preference in science subject. It was found that 59% chose Biology, 29% participants liked Chemistry, and the rest
(12%) liked Physics. The participants who liked Biology were 66% female, participants who liked Chemistry were
67% female, and students who liked Physics were 57% male.

800
ISSN 2538–7138 /Online/ (P. 797-812)
Instrument
The instrument administered to the participants in this research was 31-item measuring conceptions of
learning science (COLS) developed by Lee, Johanson & Tsai (2008). The measurement was developed based on
the research of Tsai (2004) on exploring Taiwanese high-school students’ conceptions of learning science through
phenomenographic research. The instrument measures six types of conceptions of learning sciences: memorizing
(M), testing (T), calculating and practicing (CP), increasing one’s knowledge (IOK), applying (A), and understanding
and seeing in a new way (US), as described earlier in the literature review section. The instrument was a Likert-scale
type whereby students were asked to choose answers ranging from 1 (strongly disagree) to 5 (strongly agree)
indicating how much they agreed with each statement.
When Lee, Johanson & Tsai (2008) were developing the COLS instrument, they used classical test theory (CTT)
for validating the instrument, but recently the use of the Rasch model for validating instruments is rapidly being
emphasized in the field of science education (Boone, Townsend & Staver, 2011; Johnson & Sondergeld, 2014;
Romine et al., 2017). Therefore, one goal of current research is to validate the COLS instrument by using Rasch
analyses, such as the multidimensionality of the instrument, which is considered to be the decisive issue for an
instrument (Boone, Townsend & Staver, 2011; Romine et al., 2017). In order to find out the best model of the COLS
instrument based on dimensionality, the method suggested by Adams & Wu (2010) and Bond & Fox (2015) was
used, by comparing the value of the Akaike Information Criterion (AIC) and the final deviance of the comparable
models. Six dimensions of COLS based on Lee, Johanson & Tsai (2008), two dimensions of COLS based on the
reproductive-constructivist idea, and COLS as one dimension were compared to find the best model of the instru-
ment. A five-dimensions model, based on factor analysis (Appendix 1) of the Indonesian data in which US and A
were loaded together in the same factor (eigenvalues = 1.50, percentage of variance = 4.84, cumulative percentage
= 52.21) was also added as the comparable model. Thus, those four models were compared, and the best model
based on the smaller AIC and final deviance was selected.
The benchmark suggested by Wright & Linacre (1994) was employed to identify misfitting items based on
weighted (infit) MNSQ and unweighted (outfit) MNSQ. The benchmark for rating scale items indicated good item-
fit ranges from 0.6 to 1.4. Furthermore, the benchmark suggested by DeVellis (2003) was used on interpreting
reliability values (Cronbach’s alpha, plausible value (PV) reliability, and separation/item reliability), by considering
coefficient values above .80 as ‘very good’, between .70 – .80 as ‘respectable’, between .60 to .69 as ‘undesirable but
minimally acceptable,’ and below .60 as ‘unacceptable.’
Data Analysis
Prior to the main analysis, factor analysis with Varimax rotation was performed to identify the number of
COLS factors based on Indonesian high-school data. In order to conform in more detail to the number of COLS
factors, multidimensional Rasch analyses were used, as mentioned above. Multidimensional Rasch analyses were
performed by utilizing software called ACER ConQuest version 4.5. Once Rasch analysis was run, a set of values
for every student was also obtained. This set of values is called plausible value (PV), or person measure, or person
ability, indicating a Rasch logit score for every student in every COLS construct. Thus, every student had six values
of PV, and one for a construct of COLS. These PVs were in the form of an interval scale, which is required to fulfill the
assumptions of further statistical analysis, especially parametric tests. These six PVs were used for further statistical
tests. The Pearson correlation tests were run to find out the correlations between COLS constructs. In addition,
gender differences in COLS were explored by performing an independent sample test, while differences in COLS
based on students’ favorite science subject were explored by means of a one-way ANOVA test followed by Tuckey
HSD as a post hoc test. The statistical analyses were performed in SPSS version 22.
A cluster analysis was used to uncover how Indonesian students conceive of their science learning, as addressed
in the third research question. Another purpose for using this cluster analysis was to find out how many types of
Indonesian students there were based on their conceptions of learning science. One package for analysis-based
multivariate clustering called mclust from R-software environment was used to cluster the students. The fundamental
process of mclust is to find out the number of clusters by using the interval-scale data (Scrucca et al., 2016), which
in current data were PVs. The mclust process provided a Bayesian Information Criterion (BIC) value of 14 models of
data; the best model for the Indonesian data was identified by using the smallest BIC value. After the best model
and the number of clusters were found, every student was also automatically assigned to his or her own cluster.
801
(P. 797-812) ISSN 2538–7138 /Online/
Later, in order to investigate the characteristics of every cluster, Pearson chi-square tests were also performed by
employing gender, students’ favorite science subjects, and school location (inside and outside the capital city of
the province) as the independent variables. School location was selected, because there were different periods of
new Curriculum 2013 implementation, and the capital city of the province was the first.
Results of Research
COLS Construct Based on the Multidimensional Rasch Model
The results from multidimensional Rasch analyses that aimed to find the best model of the Indonesian students’
COLS data are shown in Table 1. As shown, the final deviance and AIC values of the six-dimension model were
smaller (FD = 37596.76, AIC = 37706.76) than other comparable models: the one-dimensional (FD = 39734.79, AIC =
39804.79), two-dimensional (FD = 38695.67, AIC = 38769.67), and five-dimensional (FD = 37620.64, AIC = 37718.64).
It indicated that the COLS instrument is better treated as six-dimensional. Furthermore, the six-dimensional model
of COLS is also suggested by the item fits, reliabilities, and number of misfitting items, shown in Table 2.
Table 1. Results from multidimensional Rasch analyses.
COLS Model Final Deviance (FD) AIC χ2 df p-value
One Dimension 39734.79 39804.79 6522.21 30 .01

Two Dimensions 38695.67 38769.67 5761.03 29 .01
Five Dimensions 37620.64 37718.64 5183.76 26 .01
Six Dimensions 37596.76 37706.76 5191.55 25 .01
It can be seen, from Table 2, that the memorizing construct (M) had weighted and unweighted MNSQ that
ranged from 0.98 to 1.43 and 0.99 to 1.42, respectively. In this memorizing construct, one misfitting item (M3) was
found. For the testing construct (T), the ranges of MNSQ were 0.93 to 1.48 and 0.97 to 1.46, and there was one
misfitting item, T3. The four remaining COLS constructs were found better fitted to the Rasch Model than memo-
rizing and testing were, because they had no misfitting item. The ranges of MNSQ for calculating and practicing
(CP) were 0.81 to 1.33 and 0.82 to 1.31; for increasing one’s knowledge (IOK) were 0.87 to 1.15 and 0.81 to 1.19; for
applying (A) 0.73 to 1.18 and 0.72 to 1.22; and for understanding and seeing in a new way (US) were 0.72 to 0.90
and 0.70 to 0.92. Moreover, the reliability values (alphas and Rasch) were also in the level of respectable instrument.
Table 2. Cronbachs’ alpha (α) and Rasch item-fits and reliabilities.
Number of
COLS α if Item Weighted Unweighted PV Separation
α Measure Misfitting
Constructs Deleted MNSQ MNSQ Reliability Reliability
items
M .620 ~ .668 .701 –0.239 ~ 0.142 0.98 ~ 1.43 0.99 ~ 1.42 .723 1
T .588 ~ .693 .684 –1.517 ~ 1.558 0.93 ~ 1.48 0.97 ~ 1.46 .734 1
CP .760 ~ .810 .812 –0.228 ~ 0.087 0.81 ~ 1.33 0.82 ~ 1.31 .809 0
.994
IOK .711 ~ .787 .783 –0.581 ~ 0.836 0.87 ~ 1.15 0.81 ~ 1.19 .801 0
A .709 ~ .775 .785 –0.252 ~ 0.302 0.73 ~ 1.18 0.72 ~ 1.22 .859 0
US .797 ~ .836 .838 –0.289 ~ 0.391 0.72 ~ 0.90 0.70 ~ 0.92 .840 0
Correlations among COLS Construct
To find the correlations between COLS constructs, Pearson correlation tests were performed; the results are
shown in Table 3. Based on the results, it was found that memorizing was positively correlated (p < .01) with test-
ing (r = .312), calculating and practicing (r = .376), increasing one’s knowledge (r = .339), applying (r = .319), and
802
ISSN 2538–7138 /Online/ (P. 797-812)
understanding in a new way (r = .367) in the low-middle level. Besides being positively correlated with memorizing,
testing was positively correlated with CP in the low level (r = .190), but not significantly correlated (p > .05) with IOK
(r = .029), A (r = .028), and US (r = .024). Calculating and practicing was weakly positively correlated with IOK (r =
.289) and CP (r = .383), while moderately correlated with A (r = .438). Furthermore, the constructivist constructs were
moderately and highly correlated with one another: IOK and A (r = .548), IOK and US (r = .529), A and US (r = .640).
Table 3. Correlation coefficients among COLS construct.
T CP IOK A US
M .312‡ .376‡ .339‡ .319‡ .367‡

T 1 .190‡ .029 .028 .024
CP 1 .289‡ .438‡ .383‡
IOK 1 .548‡ .529‡
A 1 .640‡
US 1
‡
p < .01, † p < .05
Gender Differences in COLS
In comparing COLS based on gender, it was found that in memorizing, females were higher (M = 1.89, SD =
1.59) than males (M = 1.48, SD = 1.62). In the calculating and practicing and understanding and seeing in a new
way, females (M = 2.58, SD = 2.31 and M = 3.03, SD = 2.19, respectively) were also slightly higher than males (M =
2.40, SD = 2.18 and M = 2.77, SD = 2.49, respectively). It was found that males were slightly higher in testing (M =
–0.39, SD = 1.25), increasing one’s knowledge (M = 3.12, SD = 2.25), and applying (M = 2.66, SD = 2.78) than females
(M = –0.44, SD = 1.11; M = 2.99, SD = 2.11 and M = 2.62, SD = 2.49, respectively). The differences in terms of gender
are shown in Figure 1.
The independent sample t-test did not reveal many statistically significant gender differences. Based on the
findings, significant gender difference was found only in memorizing (t(608) = –3.06, p < .01) with a small effect
size (d = 0.26), but gender differences in testing (t(608) = 0.48, p > .05, d = 0.04), calculating and practicing (t(608)
= –0.97, p > .05, d = 0.08), increasing one’s knowledge (t(608) = 0.72, p > .05, d = 0.06), applying (t(608) = 0.19, p >
.05, d = 0.02), and understanding (t(608) = –1.33, p > .05, d = 0.11) were not statistically significant.
Figure 1: Gender differences in COLS constructs.
803
(P. 797-812) ISSN 2538–7138 /Online/
The Influence of Favorite Science Subject on COLS
The mean differences of every COLS construct based on students’ favorite science subject are shown in Figure
2, which reveals that students who liked biology (M = 1.80, SD = 1.61) and chemistry (M = 1.80, SD = 1.54) were
higher in memorizing than those students who liked physics (M = 1.30, SD = 1.82). In testing, students who liked
chemistry (M = –0.37, SD = 1.20) had higher means than students who liked biology (M = –0.401, SD = 1.100) and
physics (M = –0.68, SD = 1.38). In contrast, for calculating and practicing, students who liked physics (M = 3.13, SD
= 2.29) were higher than students who liked chemistry (M = 2.89, SD = 2.07) and biology (M = 2.19, SD = 2.230).
Similar to the findings from testing, in the increasing of knowledge construct and understanding construct, students
who liked chemistry (M = 3.12, SD = 2.02 and M = 3.13, SD = 2.29, respectively) were slightly higher than students
who liked biology (M = 3.06, SD = 2.23 and M = 2.93, SD = 2.29, respectively) and physics (M = 3.03, SD = 2.18 and
M = 2.87, SD = 2.59, respectively). On the contrary, in term of applying, students who liked physics (M = 2.83, SD
= 2.91) had higher means than those who liked chemistry (M = 2.76, SD = 2.254) or biology (M = 2.62, SD = 2.64).
A one-way ANOVA test was performed to find out the effect of students’ favorite science subjects on their
conceptions of learning science. Based on the findings, any significant effect of students’ favorite science subjects
on testing (F[2, 581] = 1.96, p > .05, ηp2 = 0.01), increasing one’s knowledge (F[2, 581] = 0.06, p > .05, ηp2 = 0.001),
applying (F[2, 581] = 0.32, p > .05, ηp2 = 0.001), or understanding (F[2, 581] = 0.51, p > .05, ηp2 = 0.002) were not
obtained. But, significant effect of students’ favorite science subjects on memorizing (F[2, 581] = 3.07, p < .05, ηp2
= 0.01) and calculating and practicing (F[2, 581] = 8.65, p < .01, ηp2 = 0.03) were found in current research. It can
be seen, based on the post hoc test provided in Appendix 2, that in memorizing, students who liked biology and
chemistry were significantly higher (p < .05) than those who liked physics. In contrast, students who liked phys-
ics and chemistry were significantly higher on calculating and practicing than those who liked biology (p < .01).
Figure 2: Students’ favor science subject differences in COLS constructs.
Indonesian High-school Students’ COLS based on Clustering Results
Based on results computed through mclust, Indonesian high-school students were clustered into three different
groups. This three-class result was obtained based on the smallest BIC of the mclust model, provided in Appendix
3. The data were best fitted to the VVE model, which indicated an ellipsoidal and equal orientation data type (see
Fraley & Raftery, 2002); the BIC value was –13851.5 and was considered to be the smallest value. Characterization
and naming were done to every student’s class by considering the highest three means of conceptions; the results
are shown in Figure 3.
Class 1, which included 9.7% of the participants (59 students), had the highest mean for applying (M = 2.69,
804
ISSN 2538–7138 /Online/ (P. 797-812)
SD = 0.87), increasing one’s knowledge (M = 2.34, SD = 0.52), and calculating and practicing (M = 2.26, SD = 0.68).
Based on these characteristics, in this paper it is called as Transitional Constructivist (TC). Class 2, which had the
most assigned members (67.7%, around 406 students) had its three highest means on increasing one’s knowledge
(M = 2.71, SD = 1.73), understanding (M = 2.46, SD = 1.67), and calculating and practicing (M = 2.27, SD = 2.02).
This second class was named as Transitional Reproductive (TR). The Class 3 was named fully Constructivist (fC),
because it had the two constructivist conceptions – understanding (M = 4.52, SD = 3.32) and applying (M = 4.31,
SD = 3.57) – as its highest means and was also characterized by increasing one’s knowledge (M = 4.30, SD = 3.02).
Figure 3: Results of cluster analysis (mclust).
In order to characterize every obtained cluster based on the students’ demographics, chi-squared tests of
interdependence were performed. The distribution of each group’s members based on their demographics is
shown in Table 4. The obtained groups of conceptions of learning science were not likely to be related to students’
gender (χ2 (4, 607) = 4.43, p > .05, Phi-Cramer’s V = 0.09) or their favorite science subjects (χ2 (4,584)= 8.27, p > .05,
Phi-Cramer’s V = 0.12). In contrast, it was found that the groups were significantly related to the school location (χ2
(2, 609)= 17.54, p < .01, Phi-Cramer’s V = 0.17), whereby the schools located inside the capital city of the province
were dominant in the fC group (57.6%) and TC group (69.5%), but schools located outside the capital city were
dominant in TR group (55.7%).
Table 4. Clusters distribution by gender, favorite science subjects and school location.
Gendera Favorite Science Subjecta School Locationb

Group
Male Female Biology Chemistry Physics Inside Outside
15 43 40 10 4 41 18
TC
(6.7%) (11.3%) (11.6%) (5.9%) (5.6%) (69.5%) (30.5%)
150 255 219 125 48 180 226
TR
(66.7%) (66.8%) (63.7%) (74%) (67.6%) (44.3%) (55.7%)
60 84 85 34 19 83 61
fC
(26.7%) (22.0%) (24.7%) (20.1%) (26.8) (57.6%) (42.4%)
Total 255 382 344 169 71 304 305
a
Percentage based on Independent variables (Gender and or Favorite Science Subjects)
b
Percentage based on COLS Groups (because total were equal)
805
(P. 797-812) ISSN 2538–7138 /Online/
Discussion
The current research aimed to validate the COLS instrument by using Rasch analysis, to explore the influence
of gender and favorite science subject, and to evaluate the new Indonesian science curriculum in terms of high-
school students’ conceptions of learning science.
COLS Construct Validity
Three types of analyses, namely, factor analysis, Rasch multidimensional analysis, and correlation tests, were
performed to explore the constructs in the COLS instrument. By using the raw ordinal data, a factor analysis was
performed and the results have shown that the COLS instrument based on Indonesian high-school students’ re-
sponses consisted of five constructs whereby the two higher-level conceptions – applying (A) and understanding
and seeing in a new way (US) – were loaded together in the same factor. In contrast, all the lower conceptions were
loaded separately in the different factors, which implies that Indonesian high-school students did not see applying
and US in learning science as different conceptions but as the same as being constructivist concepts. In addition,
the different factors separated between the ‘real’ constructivist and ‘transitional’ constructivist conceptions, which
is increasing of knowledge (IOK), obtained in current research has shown that IOK is not really a higher-level con-
ception of learning science, as was suggested by Marton, Dall’Alba & Beaty (1993) and Säljö (1979), because they
argued that IOK refers to the accumulation and acquisition of facts or correct knowledge, which tend to be more
a part of a reproductive concept and use a surface approach. But because Tsai (2004) stated that IOK is correlated
to students’ personal desire to fulfill their development in learning, thus in the Rasch analysis, IOK was included
in constructivist conception.
Because the factor analysis used the raw ordinal data that many have said might be biased, the confirmation
test on dimensionality that Lee, Johanson & Tsai (2008) as the developers of COLS instrument suggested, using
a Rasch analysis to confirm the validity of the instrument was done. Four COLS instrument models as mentioned
above were compared. It can be seen in Table 1, the results suggested that the COLS instrument is best treated
as being six-dimensional, as was originally suggested by Lee, Johanson & Tsai (2008), as is evident from the few
missfitting items (exceeds 0.6 – 1.4) and better reliabilities shown in Table 2. For the two items that had an item-fit
index of more than 1.4, the values are still below 1.5, which Wright & Linacre (1994) argued is still a reasonable fit
of the data to the Rasch model. Moreover, Boone & Scantlebury (2006) said that Rasch analysis is sample indepen-
dent, while factor analysis is sample dependent. Thus, Rasch, used to find out the validity of the instrument, can
provide a consistent result from various types of samples and populations. That is why most science-education
researchers are now attempting to use Rasch analysis to implement instrument development (e.g., Oon & Fan,
2017; Romine et al., 2017).
The Impact of Gender and Favorite Science Subject on Students’ COLS
In terms of findings in gender difference only exhibited in memorizing, in which female students were higher
than male students. This result aligns with the finding by Sadi & Lee (2017), who investigated Turkish and Taiwan-
ese students’ conceptions of learning science (biology) and found that in Taiwan, female students were higher in
memorizing than male students. Moreover, many also have found that females use more memorization strategies
when learning languages (e.g., Dale, 1970; Li & Chun, 2012), economics (e.g., Lumsden & Scott, 1987), and math-
ematics (e.g., Geist & King, 2008). The reason why females tend to conceive of science learning more in terms of
memorizing than males do is discussed in the study by Herlitz & Rehnman (2008) on episodic memory, which is
defined as the conscious regathering of particular terms of facts, such as what, when, and where (Tulving, 2001).
One dimension of this memory is verbal-production abilities, which females are superior at. Since science subjects
consist of many facts, theories, or even symbols that need to be memorized, when learning science females often
use memorization strategies, which led to their conceptions of learning science. Herlitz & Rehnman (2008) added
that males tend to use a higher level of memory than females, such as understanding and reasoning, which in
this research were not examined. Furthermore, current finding supported an old hypothesis about how females
tend to use rote learning strategies in science learning proposed by Ridley & Novak (1983), they suggested that
females tend to use rote learning because they are prone to want to please teachers, partly because of the social
stereotype of females as having lower attainment than male students do.
806
ISSN 2538–7138 /Online/ (P. 797-812)
In consideration to gender stereotyping issue in science education, to date several recent researches are still
reporting that female is still underrepresented in the field of science (e.g. Cheryan et al., 2017; Miller, Eagly & Linn,
2014; Wang & Degol, 2016). Consequently, many have argued that gender stereotyping is one of cognitive biases
that hinders the current goals of global science education which is equity in science learning and science for all
to be reached (Andre et al., 1999; Bianchini et al., 2015; Phillip & Azevedo, 2017; Renni, 1998; Rodriguez, 2015;
Zeyer, 2017). By finding out the way to tackle this issue, it may lead to the decrease of female students on using
rote learning in science subjects, because they are likely starting the effort not to please the teachers. Moreover,
females may increase their level of conceiving science learning to the higher one and being able to have deeper
understanding of science concepts that could increase their performance in science subjects. Thus, uncovering
and developing program to decrease gender stereotyping in science are crucial.
As aforementioned, students’ COLS are related to their favorite science subject. The significant differences
in COLS as memorizing and as calculating and practicing were found in this research. Students who liked biology
tend to conceive of learning science as memorizing. This corresponds to the fact that biology in schools is full of
teaching materials that need to be memorized by students (Lin, Liang & Tsai, 2014). Especially in Indonesia, many
teachers still use traditional teaching methods that explicitly ask students to memorize much content. Consequently,
students who liked biology follow the methods and infer that science is memorizing facts. In addition, it was found
that students who liked physics tend to conceive of learning science as calculating and practicing. This corresponds
to the nature of teaching physics in Indonesia, which emphasizes practicing physics problems, which has more to
do with mathematics than with theoretical issues in physics. Hence students who liked physics conceive of science
more as calculating and practicing. Interestingly, students who liked chemistry tend to conceive of science learn-
ing as both memorizing and calculating and practicing. This indicates that the contents of chemistry are related
to the things that need both memorization and calculation and practice. Current findings have empirically shown
the nature of every science subject in the Indonesian school context.
Indonesian High-school Students’ Conceptions of Learning Science
The third research question addresses what Indonesian high-school students’ COLS are after the implemen-
tation of the new curriculum. Based on the findings described above, it was found that there were three types of
student conceptions of learning science, namely, transitional constructivist, transitional reproductive, and con-
structivist. In addition, it was found that the types of students differed in gender and favorite science subject, but
these differences were related to the school location, that is, to how long the new science curriculum had been
implemented. The schools located inside the capital city of the province implemented the new curriculum around
one year sooner than the schools located outside the capital. Thus, the findings may also show the impact of the
new curriculum on how students conceive of their science learning.
The first group, which consisted of a tenth of the students and called as the ‘transitional constructivist’ group,
because the students had high COLS in applying, increasing one’s knowledge, and calculating and practicing. The
highest conception was ‘applying,’ which is categorized as a constructivist conception, followed by IOK and CP. In the
study of Lee, Johanson & Tsai (2008), IOK was indicated as a transitional conception, because it has no association
with the intrinsic and extrinsic motivations and strategies to learn science. The group is predominantly students
from the schools located in the capital city of the province. Conversely, in the second group, named ‘transitional
reproductive,’ the highest concept was IOK and had CP as the third highest mean. This group was students who
conceive of science learning as primarily increasing knowledge by calculating and practicing problems. It consisted
mainly of students from schools located outside the capital city, made up more than half of total participants (68%),
and can be assumed to represent most Indonesian high-school students. The last group, which was named as ‘fully
constructivist’ had the highest hierarchy of COLS, which are US and A, placed in the highest rank of total mean. This
group was about a fifth of the total participants, and most of them were students from schools inside the capital city.
The findings have shown that the new science curriculum influences how Indonesian high-school students
conceive of their science learning. Even though most of them are still conceiving of science learning at the level of
transitional reproductive, the impact of the science curriculum is clearly shown, because the number of students
in the ‘fully constructivist’ group was higher for the schools that had been implementing the new curriculum
longer. Hence more time is needed for this new curriculum to influence more Indonesian high-school students.
Moreover, the new policy about the role of examinations in the school system within the new curriculum is also
significantly impacting students’ conceptions of learning science. It is shown by COLS, because ‘testing’ had the
807
(P. 797-812) ISSN 2538–7138 /Online/
lowest mean for all the groups of students (Figure 3). As noted earlier in the literature review, this new policy is
related to the deletion of the role of the national examination as the sole judgment for students graduating from
high school and instead considering attitude as the highest factor for students graduating from high school. But
more evidences and further studies are still needed to make sure that the Indonesian high-school students’ COLS
were really influenced by this new curriculum and whether the COLS were highly related to science literacy as was
the main notion of the new science curriculum.
Conclusions and Implications
Current research has suggested that the conceptions of learning science instrument was a stable instrument
with the confirmation through two methods, classical test theory (CTT) and item response theory (IRT) with Rasch
model. In addition, current research has suggested that there are different conceptions of learning science between
genders, in which females tend to use more memorization than male students in learning science. Finally, the re-
search also suggested that the long period of curriculum with more emphasis on scientific literacy could change
students’ conceptions of learning science.
This research has both limitations and implications. What the claims established in current research regard-
ing Indonesian high-school students points only to students from the metropolitan island, Java Island. Given that
Indonesia consists of many islands, small and big, the results found in current research could not be generalized
to students outside Java Island, particularly outside West Java province. Given the many different ethnicities and
characteristics of culture and people in the other islands, it is believed that they may have different conceptions of
learning science, even using the same new science curriculum. This may call for a new future study, such as compar-
ing conceptions of learning science from different ethnicities in one nation. Furthermore, the implementation of the
new Indonesian science curriculum that introduced attitude as the main notion has shown very promising results,
given the existence of students in the fully constructivist group. When students already have a higher hierarchy of
learning science, as mentioned previously, they may accept science more easily and apply science concepts more
frequently in their daily life. This can lead to an increase in science literacy as proposed as the main notion of the
new Indonesian science curriculum 2013.
Acknowledgment
This work was supported by the Ministry of Education of the Republic of Korea and the National Research
Foundation of Korea (NRF-2017R1C1B1005152)
References
Adams, R., & Wu, M. (2010). Notes and tutorial ConQuest: Multidimensional model. Retrieved from https://www.acer.org/conquest/
notes-tutorials. 25 May 2017.
Allan, B. (2003). Approaches to learning and academic achievement of Filipino students. The Journal of Genetic Psychology, 164,
101-114.
Andre, T., Whigham, M., Hendrickson, A., & Chambers, S. (1999). Competency beliefs, positive affect, and gender stereotypes of
elementary students and their parents about science versus other school subjects. Journal of Research in Science Teach-
ing, 36 (6), 719-747.
Bianchini, J. A., Dwyer, H. A., Brenner, M. E., & Wearly, A. J. (2015). Facilitating science and mathematics teachers’ talk about eq-
uity: What are the strengths and limitations of four strategies for professional learning? Science Education, 99 (3), 577-610.
Biggs, J. (1994). Approaches to learning: Nature and measurement of. In T. Husen & T. N. Postlethwaite (Eds.), The international
encyclopedia of education (2nd ed., Vol. 1, pp. 319 – 322). Oxford, England: Pergamon.
Biggs, J. B., & Collins, K. (1982). Evaluating the quality of learning: The SOLO taxonomy. New York, NY: Academic Press.
Bond, T., & Fox, C. M. (2015). Applying the Rasch model: Fundamental measurement in the human sciences. Routledge.
Boone, W. J., & Scantlebury, K. (2006). The role of Rasch analysis when conducting science education research utilizing multiple‐
choice tests. Science Education, 90 (2), 253-269.
Boone, W. J., Townsend, J. S., & Staver, J. (2011). Using Rasch theory to guide the practice of survey development and survey data
analysis in science education and to inform science reform efforts: An exemplar utilizing STEBI self‐efficacy data. Science
Education, 95 (2), 258–280.
Buehl, M. M., & Alexander, P. A. (2001). Beliefs about academic knowledge. Educational Psychology Review, 13 (4), 385-418.
Cheryan, S., Ziegler, S. A., Montoya, A. K., & Jiang, L. (2017). Why are some STEM fields more gender balanced than others? Psy-
chological Bulletin, 143 (1), 1-36.
808
ISSN 2538–7138 /Online/ (P. 797-812)
Chiou, G. L., Liang, J. C., & Tsai, C. C. (2012). Undergraduate students’ conceptions of and approaches to learning in biology: A
study of their structural models and gender differences. International Journal of Science Education, 34 (2), 167-195.
Dale, R. R. (1970). A comparison of the academic performance of male and female students in schools and universities. Journal
of Biosocial Science, 2 (S2), 95-99.
Dart, B. C., Burnett, P. C., Purdie, N., Boulton-Lewis, G., Campbell, J., & Smith, D. (2000). Students’ conceptions of learning, the
classroom environment, and approaches to learning. The Journal of Educational Research, 93 (4), 262-270.
DeBoer, G. E. (2000). Scientific literacy: Another look at its historical and contemporary meanings and its relationship to science
education reform. Journal of Research in Science Teaching, 37 (6), 582-601.
Den Brok, P., Telli, S., Cakiroglu, J., Taconis, R., & Tekkaya, C. (2010). Learning environment profiles of Turkish secondary biology
classrooms. Learning Environments Research, 13 (3), 187-204.
DeVellis, R. F. (2003). Scale development: Theory and applications (2nd Ed.). Thousand Oaks, CA: Sage.
Entwistle, N., & Tait, H. (1990). Approaches to learning, evaluations of teaching, and preferences for contrasting academic envi-
ronments. Higher Education, 19 (2), 169-194.
Fraley, C., & Raftery, A. E. (2002). Model-based clustering, discriminant analysis, and density estimation. Journal of the American
Statistical Association, 97 (458), 611-631.
Geist, E. A., & King, M. (2008). Different, not better: Gender differences in mathematics learning and achievement. Journal of
Instructional Psychology, 35 (1), 43-53.
Glynn, S. M., Taasoobshirazi, G., & Brickman, P. (2009). Science motivation questionnaire: Construct validation with nonscience
majors. Journal of Research in Science Teaching, 46 (2), 127-146.
Gummer, E. S., & Shepardson, D. P. (2001). Facilitating change in classroom assessment practice: Issues for professional develop-
ment. In Assessment in Science (pp. 53-66). Springer Netherlands.
Herlitz, A., & Rehnman, J. (2008). Sex differences in episodic memory. Current Directions in Psychological Science, 17 (1), 52-56.
Hong, Y. Y., & Salili, F. (2000). Challenges ahead for research on Chinese students’ learning motivation in the new millennium. Jour-
nal of Psychology in Chinese Societies, 1 (2), 1-12.
Lai, P. Y., & Chan, K. W. (2005). A structural model of conceptions of learning, achievement motivation and learning strategies
of Hong Kong teacher education students. Australian Association of Research in Education Parramatta Conference, Sydney,
28 November-2 December.
Lee, C. S., Hayes, K. N., Seitz, J., DiStefano, R., & O’Connor, D. (2016). Understanding motivational structures that differentially
predict engagement and achievement in middle school science. International Journal of Science Education, 38 (2), 192-215.
Lee, M. H., Johanson, R. E., & Tsai, C. C. (2008). Exploring Taiwanese high school students’ conceptions of and approaches to
learning science through a structural equation modeling analysis. Science Education, 92 (2), 191-220.
Lee, M. H., Lin, T. J., & Tsai, C. C. (2013). Proving or improving science learning? Understanding high school students’ conceptions
of science assessment in Taiwan. Science Education, 97 (2), 244-270.
Li, J. & Chun, C. K. W. (2012). Effects of learning strategies on student reading literacy performance. Reading, 12(1), 30-38.
Liang, J. C., & Tsai, C. C. (2010). Relational analysis of college science‐major students’ epistemological beliefs toward science and
conceptions of learning science. International Journal of Science Education, 32 (17), 2273-2289.
Lin, T. C., Liang, J. C., & Tsai, C. C. (2015). Conceptions of memorizing and understanding in learning, and self-efficacy held by
university biology majors. International Journal of Science Education, 37 (3), 446-468.
Lonka, K., & Lindblom-Ylänne, S. (1996). Epistemologies, conceptions of learning, and study practices in medicine and psychol-
ogy. Higher Education, 31 (1), 5-24.
Lumsden, K. G., & Scott, A. (1987). The economics student reexamined: Male-female differences in comprehension. The Journal
of Economic Education, 18 (4), 365-375.
Marshall, D., Summer, M., & Woolnough, B. (1999). Students’ conceptions of learning in an engineering context. Higher Educa-
tion, 38 (3), 291-309.
Marton, F., Dall’Alba, G., & Beaty, E. (1993). Conceptions of learning. International Journal of Educational Research, 19, 277 – 299
Mendikbud [Indonesian Ministry of Education and Culture]. (2016). Salinan Lampiran Permendikbud No.21 th 2016 tentang
Standar Isi Pendidikan Dasar dan Menengah [A copy of the additional documents of Indonesian ministry of education and
culture regulation number 21 year 2016 on Content Standards for Primary and Secondary Education]. Jakarta: Mendikbud.
Miller, D. I., Eagly, A. H., & Linn, M. C. (2015). Women’s representation in science predicts national gender-science stereotypes:
Evidence from 66 nations. Journal of Educational Psychology, 107 (3), 631-635.
Mullis, I. V. S., Martin, M. O., Foy, P., & Hooper, M. (2016). TIMSS 2015 international results in science. Retrieved from Boston College,
TIMSS & PIRLS International Study. Center website: http://timssandpirls.bc.edu/timss2015/international-results/.
OECD. (2015). OECD science, technology and industry scoreboard 2015: Innovation for growth and society. OECD Publishing. Paris.
OECD. (2016). PISA 2015 results (Volume I): Excellence and equity in education, PISA. OECD Publishing, Paris.
Oon, P. T., & Fan, X. (2017). Rasch analysis for psychometric improvement of science attitude rating scales. International Journal
of Science Education, 39 (6), 683-700.
Osborne, J., Simon, S., & Collins, S. (2003). Attitudes towards science: A review of the literature and its implications. International
Perkins, K. K., Barbera, J., Adams, W. K., & Wieman, C. E. (2007, January). Chemistry vs. physics: A comparison of how biology
majors view each discipline. In L. McCullough, L. Hsu & P. Heron (Eds.), AIP conference proceedings, 883 (1), 53-56. DOI:
10.1063/1.2508689.
Philip, T. M., & Azevedo, F. S. (2017). Everyday science learning and equity: Mapping the contested terrain. Science Education,
809
(P. 797-812) ISSN 2538–7138 /Online/
1-7. DOI: 10.1002/sce.21286.

Pillay, H., Purdie, N. O. L. A., & Boulton-Lewis, G. (2000). Investigating cross-cultural variation in conceptions of learning and the
use of self-regulated strategies. Education Journal Hong Kong-Chinese University of Hong Kong, 28 (1), 65-84.
Purdie, N., Hattie, J., & Douglas, G. (1996). Student conceptions of learning and their use of self-regulated learning strategies: A
cross-cultural comparison. Journal of Educational Psychology, 88 (1), 87-100.
Rennie, L. J. (1998). Gender equity: Toward clarification and a research direction for science teacher education. Journal of Research
in Science Teaching, 35 (8), 951-961.
Ridley, D. R., & Novak, J. D. (1983). Sex-related differences in high school science and mathematics enrollments: Do they give
males a critical head start toward science-and math-related careers? Alberta Journal of Educational Research, 29 (4), 308-318.
Robnett, R. D., Chemers, M. M., & Zurbriggen, E. L. (2015). Longitudinal associations among undergraduates’ research experience,
self‐efficacy, and identity. Journal of Research in Science Teaching, 52 (6), 847-867.
Rodriguez, A. J. (2015). What about a dimension of engagement, equity, and diversity practices? A critique of the next generation
science standards. Journal of Research in Science Teaching, 52 (7), 1031-1051.
Romine, W. L., Walter, E. M., Bosse, E., & Todd, A. N. (2017). Understanding patterns of evolution acceptance ̶ A new implemen-
tation of the measure of acceptance of the theory of evolution (MATE) with Midwestern university students. Journal of
Research in Science Teaching, 54 (5), 642-671.
Sadi, Ö., & Çevik, M. (2016, January). Investigating of conceptions of learning biology with respect to gender, grade level and
school type. In SHS web of conferences (Vol. 26). EDP Sciences. DOI: 10.1051/shsconf/20162601025.
Sadi, Ö., & Lee, M. H. (2017). Exploring Taiwanese and Turkish high school students’ conceptions of learning biology. Journal of
Biological Education, DOI: 10.1080/00219266.2017.1285799.
Sadi, O., & Uyar, M. (2013). The relationship between self-efficacy, self-regulated learning strategies and achievement: A path
model. Journal of Baltic Science Education, 12 (1), 21-33.
Sadi, Ö. (2015). The Analysis of high school students’ conceptions of learning in different domains. International Journal of Envi-
ronmental and Science Education, 10 (6), 813-827.
Säljö, R. (1979). Learning about learning. Higher Education, 8 (4), 443-451.
Schumm, M. F., & Bogner, F. X. (2016). Measuring adolescent science motivation. International Journal of Science Education, 38
(3), 434-449.
Scrucca, L., Fop, M., Murphy, T. B., & Raftery, A. E. (2016). mclust 5: Clustering, classification and density estimation using gaussian
finite mixture models. The R Journal, 8 (1), 289-317.
Severiens, S., & Dam, G. T. (1997). Gender and gender identity differences in learning styles. Educational Psychology, 17 (1-2), 79-93.
Sinatra, G. M. (2001). Knowledge, beliefs, and learning. Educational Psychology Review, 13 (4), 321-323.
Sondergeld, T. A., & Johnson, C. C. (2014). Using Rasch measurement for the development and use of affective assessments in
science education research. Science Education, 98 (4), 581–613.
Thompson, T. L., & Mintzes, J. J. (2002). Cognitive structure and the affective domain: On knowing and feeling in biology. Inter-
national Journal of Science Education, 24 (6), 645-660.
Tsai, C. C. (2004). Conceptions of learning science among high school students in Taiwan: A phenomenographic analysis. Inter-
national Journal of Science Education, 26 (14), 1733-1750.
Tsai, C. C. (2006). Reinterpreting and reconstructing science: Teachers’ view changes toward the nature of science by courses of
science education. Teaching and Teacher Education, 22 (3), 363-375.
Tsai, C. C., Ho, H. N. J., Liang, J. C., & Lin, H. M. (2011). Scientific epistemic beliefs, conceptions of learning science and self-efficacy
of learning science among high school students. Learning and Instruction, 21 (6), 757-769.
UNESCO. (2015). A complex formula: Girls and women in science, technology, engineering and mathematics in Asia. UNESCO:
UNESCO Bangkok Office.
Vermunt, J. D., & Vermetten, Y. J. (2004). Patterns in student learning: Relationships between learning strategies, conceptions of
learning, and learning orientations. Educational Psychology Review, 16 (4), 359-384.
Vermunt, J. D. (1996). Metacognitive, cognitive and affective aspects of learning styles and strategies: A phenomenographic
analysis. Higher Education, 31 (1), 25-50.
Wang, M. T., & Degol, J. L. (2017). Gender gap in science, technology, engineering, and mathematics (STEM): Current knowledge,
implications for practice, policy, and future directions. Educational Psychology Review, 29 (1), 119-140.
Wang, Y. L., Tsai, C. C., & Wei, S. H. (2015). The sources of science teaching self-efficacy among elementary school teachers: A
mediational model approach. International Journal of Science Education, 37 (14), 2264-2283.
Watts, M., & Alsop, S. (2000). The affective dimensions of learning science. International Journal of Science Education, 22 (12),
1219-1220.
Wright, B. D., & Linacre, J. M. (1994). Reasonable mean-square fit values. Rasch Measurement Transactions, 8 (3), 370.
Zeyer, A. (2017). Gender, complexity, and science for all: Systemizing and its impact on motivation to learn science for different
science subjects. Journal of Research in Science Teaching, 1-25. DOI: 10.1002/tea.21413.
810
ISSN 2538–7138 /Online/ (P. 797-812)
Appendixes
Appendix 1. Factor analysis of COLS.
Factor 1 Factor 2 Factor 3 Factor 4 Factor 5

Items
US & A IOK CP T M
US5 .747
US6 .738
US4 .715
US3 .697
US2 .696
A4 .578
A3 .556
A2 .492
US1 .490
A1 .456 .436
IK3 .674
IK5 .669
IK2 .667
IK4 .637
IK1 .601
CP4 .754
CP2 .711
CP1 .703
CP5 .698
CP3 .687
T4 .757
T1 .714
T6 .597
T2 .590
T3 –.402 .515
T5 .468
M2 .716
M3 .649
M1 .642
M4 .592
M5 .502
Appendix 2. Tuckey HSD pairwise comparison.
95% Confidence Interval for

Mean Difference
(I) Favorite (J) Favorite Difference Std. Error p-value
(I–J)
Lower Bound Upper Bound
Biology Chemistry 0.003 0.152 1.000 –0.362 0.368

Memorizing Physics 0.509† 0.211 .048 0.002 1.016
Chemistry Physics 0.506 0.229 .082 –0.043 1.056
Biology Chemistry –0.696‡ 0.210 .003 –1.200 –0.192
Calculating and
Physics –0.940‡ 0.291 .004 –1.639 –0.241
Practicing
Chemistry Physics –0.244 0.316 1.000 –1.003 0.515
‡
p < .01, p < .05
†
811
(P. 797-812) ISSN 2538–7138 /Online/
Appendix 3. Bayesian Information Criterion (BIC) values for deciding the number of group computed by
mclust package.
Number
of EII VII EEI VEI EVI VVI EEE EVE VEE VVE EEV VEV EVV VVV
Class
1 –15454.9 –15454.9 –14976.2 –14976.2 –14976.2 –14976.2 –14095.0 –14095.0 –14095.0 –14095.0 –14095.0 –14095.0 –14095.0 –14095.0
2 –14809.9 –14779.7 –14535.9 –14455.9 –14515.3 –14485.0 –14094.7 –14042.4 –13910.4 –13922.1 –14070.5 –13956.5 –14100.7 –13982.9
3 –14519.7 –14352.7 –14318.8 –14151.1 –14296.6 –14197.8 –14138.7 –14049.3 –13904.8 –13851.5 –14138.3 –13885.8 –14153.8 NA
4 –14440.4 –14287.2 –14285.1 –14076.1 –14196.4 –14130.0 –14130.4 –14043.3 –13904.9 –13900.2 –14210.0 –13928.6 –14192.0 NA
5 –14396.4 –14299.8 –14234.2 –14006.0 –14258.9 NA –14144.9 –14058.2 –13933.4 –13875.3 –14253.8 –13919.5 –14284.5 NA
6 –14411.2 –14130.5 –14138.2 –13970.0 NA NA –14141.9 NA –13942.1 NA –14299.1 –13968.2 NA NA
7 –14289.7 –14105.6 –14086.9 –13968.8 NA NA –14099.9 NA NA NA –14390.5 –14057.3 NA NA
8 –14241.0 NA –14082.4 NA NA NA –14134.4 NA NA NA –14450.5 NA NA NA
Received: July 04, 2017 Accepted: September 30, 2017
Arif Rachmatullah S.Pd, Master Student, Division of Science Education, College of

Education, Kangwon National University, 1 Kangwondaehak-gil,
Chuncheon-si, Gangwon-do, 24341 Republic of Korea.
E-mail: arifraach@gmail.com
Sariwulan Diana Dr., Senior Lecturer, Department of Biology Education, FPMIPA
Indonesia University of Education, Jl. Dr. Setiabudhi No. 229
Bandung 40154 Jawa Barat, Indonesia.
E-mail: sariwulwul@yahoo.co.id
Minsu Ha Ph.D, Assistant Professor, Division of Science Education, College of
(Corresponding author) Education, Kangwon National University, 1 Kangwondaehak-gil,
Chuncheon-si, Gangwon-do, 24341 Republic of Korea.
E-mail: msha@kangwon.ac.kr
812
EXPLORING STUDENTS’
UNDERSTANDING IN
MAKING A DECISION ON A ISSN 1648-3898 /Print/
SOCIOSCIENTIFIC ISSUE ISSN 2538-7138 /Online/
USING A PERSUASIVE GRAPHIC

ORGANISER
Abstract. The common use of question-

Mastura Halim, naires and qualitative assessments re-
Rohaida Mohd Saat vealed that students experience difficulties
in visualising the structural complexity of
decision making. Thus, the use of scientific
knowledge was disconnected while making
a preference. The focus of this research was
to explore students’ understanding of the
global warming issue using a Persuasive
Graphic Organiser (PGO). This research was
conducted among 36 students (Grades 7-9)
Introduction at a school in Kuala Lumpur, Malaysia. The
developed PGO and interviews were utilised
The integration of socioscientific issues in science education is an es- as the primary data collection techniques.
sential facet of scientific literacy. Socioscientific issues are characterised as Meanwhile, a constant comparative
controversial in nature, real-world problems and socially relevant, which are procedure was employed in the qualitative
informed by science and ethical complexities (Christenson, Chang, Rund- data analysis. Results showed that the use
gren, & Zeidler, 2014; Sadler, & Zeidler, 2005; Sadler, Barab, & Scott, 2007). of PGO assisted students’ visualisation in
More importantly, the socioscientific approach has become a fundamental relating three essential understandings
of developing critical thinking (Kolsto, 2006; Uskola, Maguregi, & Jimenez- of the global warming issue. The students
Aleixandre, 2010). The expectations of socioscientific approach are centralized were able to: 1) identify anthropogenic fac-
on the abilities to reflect on problems, which are relevant to social life (Kolsto, tor affecting global warming; 2) clarify the
2001; Sadler & Zeidler, 2009; Holbrook & Rannikmae, 2010), according to a effects of global warming in health aspect,
careful examination of knowledge (Albe, 2007; Sadler, Chambers, & Zeidler, and 3) provide alternative solutions regard-
2004; Sadler & Donnelly, 2006). These have been viewed as essentials in pro- ing green technology and daily practices.
ducing enlightened students who can participate in decision-making about It was also found that PGO is operationally
society (Jimeńez-Aleixandre, 2002; Sadler, Barab, & Scott, 2007; Vesterinen, workable and can be conveniently utilised
Tolppanen, & Aksela, 2016) and maintain a democratic way of life (Aikenhead, as a visualisation tool for promoting
1985; Heath, White, Berlin, & Park, 1987). Therefore, students need to be pro- students’ understanding in socioscientific
vided with appropriate contexts in order to achieve the objectives set above. decision-making.
Zeidler and Nicols (2009) highlights that the socioscientific approach re- Keywords: decision-making, persuasive
quires the use of evidence-based decisions and a contextualized scenario for graphic organiser, socioscientific issues,
understanding scientific information. There are no denials on the prioritisation understanding of scientific concepts.
of scientific ideas for decision-making process (Klosterman & Sadler, 2010).
The rationale is that it furthers the development of scientific concept and
processes and thus develops a sound understanding of particular dilemmas
(Sadler & Zeidler, 2009). Sadler and Zeidler (2009) emphasise that students Mastura Halim,
Rohaida Mohd Saat
must attain sound knowledge and formalisms, as well as the skills and pro- University of Malaya, Malaysia
cesses required within sciences. These include the ability in conceptualising
the relevant key issues, which are in agreement with scientific knowledge,
813
(P. 813-824) ISSN 2538–7138 /Online/
as well as decision-making skills. With that being so, it is beneficial to gauge students’ understanding so that they
can make a sound decision within a socioscientific dilemma.
In this regard, the biggest problem uncovered relates to the difficulties students have in visualising structural
complexities in the decision-making process (Ratcliffe, 1997; Zohar & Nemet, 2002). As a result of this complica-
tion, they are unable to evaluate scientific evidence, and treat scientific information as uncertain and intolerable
(Fleming, 1986). They are also unable to make decisions about the issues and fail to reach a final determination
(Grace, 2009). Therefore, there is a need to develop instructional materials for socioscientific issues so as to pro-
mote students’ understanding and decision-making skills (Choi, Hand, & Norton-Meier, 2014; Jho, Yoon, & Kim,
2014; Klosterman & Sadler, 2010; Ratcliffe, 1997a; Sakschewski, Eggert, Schneider, & Bögeholz, 2014). In fact, it
is beneficial if students are provided with a decision-making framework (Acar, Turkmen, & Roychoudhury, 2010;
Grace, 2009; Ratcliffe, 1997a) to avoid vague directions in constituting scientific evidence and formulating sound
understanding (Walker & Zeidler, 2007).
Previous studies have shown that the use of questionnaires restrict students to prioritise knowledge criteria,
based on their personal experiences and values, because selection of the construct is already arranged (Gresch,
Hasselhorn, & Bögeholz, 2013). Consequently, students tend to delimit their offers to provide more solutions as
a further explanation of the finalised preference. Furthermore, Callahan, Zeidler, and Orasky (2011) utilising the
Views on Science and Education survey (VOSE) investigate the relationship between socioscientific issues and
science understanding. The results indicate that students are approximately halfway between conventional and
contemporary views of science, even after a semester-long socioscientific treatment. The statistical insignificance
may arise from two possibilities derived from using the VOSE questionnaire. These highlight the poor detection
power of VOSE in discriminating between naïve and sophisticated understanding. Also, there is incompletion of
revisiting the same survey, because of students’ demotivation parameters. Additionally, Zohar and Nemet (2002)
examine the influences of teaching argumentation skills, in the context of human genetics, on the acquisition of
content knowledge among ninth-grade students in Israel. It looks promising, particularly for the large samples used,
as well as the positive after-effect; however, the main emphasis was on argumentation skills, involving students’
ability to recognise unclear key issues of science knowledge.
Recently, Dawson (2015) studied students’ understanding of climate change and the greenhouse effects using
a designed questionnaire, completed by 438 Year 10 students from six schools in Perth, Western Australia. This was
followed by interviewing 20 students to explore their understanding of the issues further. Findings demonstrated
that students knew different features of both climate change and the greenhouse effects, although this did not
necessarily involve all of them. She also emphasised that the use of clear consequences and solutions for these
issues were unclear, because the sciences of climate change, and greenhouse effects or global warming were
emerging and subject to change. For instance, the public could hold and oppose views regarding the sciences of
factors and consequences. In addition, Khishfe (2015) investigated 10th grade students’ understanding of genetically
modified food, and river fluoridation using pre-, post- and delayed post-tests. Results showed that the majority
of the students reverted to their earlier naive understanding even after they had been exposed to a four-month
socioscientific instruction.
In response to the dilemmas in quantitative methods, Choi, Hand, and Norton-Meier (2014) suggested em-
ploying a qualitative method, such as the Science Writing Heuristic (SWH) approach, in decision-making process.
This method resulted in a very satisfactory outcome for students’ socioscientific understanding. However, they
challenged the reliability and validity of the approach due to the lack of interview data sources. The use of writing
frames might be useful to students who worked individually, thus enabling them to provide ‘data,’‘claims,’‘warrants,’
‘backings’ and ‘rebuttals’ in an eloquent manner. The use of this argumentation structure was necessary because
it was vital for tracking students’ understanding (Walker & Zeidler, 2007), which importantly symbolised a high-
quality skill in decision-making (Toulmin, 2003).
Walker and Zeidler (2007) conducted an inquiry-based curricular unit to promote socioscientific learning among
36 Grade 9-12 students in the Southeast United States. The researchers utilised an online artefact and interview
questions to examine the features of argumentation and discourse, as they reached the final decision on geneti-
cally modified foods. Results showed that students did not divulge much of their understanding when making a
socioscientific decision. They were also not specifically direct in applying their nature of science understanding.
The worst condition was that they tended to utilise more factual-based content that led to numerous examples of
flawed reasoning and personal attack. The findings recommended a socioscientific approach to exploring aspects
of nature of science that represented the science conceptions to be applied within a decision-making context.
814
ISSN 1648–3898 /Print/ EXPLORING STUDENTS’ UNDERSTANDING IN MAKING A DECISION ON A SOCIOSCIENTIFIC
ISSN 2538–7138 /Online/ (P. 813-824)
Klosterman and Sadler (2010) utilised a multi-level assessment design to explore students’ content knowl-
edge which was relevant to science knowledge and the controversies surrounding global warming. Data were
collected from 108 Grade 9-12 students using a standards-aligned content knowledge exam (distal assessment)
and curriculum-aligned exam (proximal assessment). The analyses from the proximal assessment revealed some
stages of understanding of global warming.
Taking a Malaysian context as an example, this acknowledges the importance of integrating socioscientific
issues into the science curriculum, as an approach to infusing cognitive competency in various stages. It includes
promoting students’ understanding in making a decision about socioscientific issues using relevant instructional
material. In this framework of study, the issue of global warming was selected, because it is categorised as a major
socioscientific area of Malaysian middle school education, relevant to students’ daily life experiences, linkable to
the formal curriculum and vital for informed citizenship (Hundal, Levin, & Keselman, 2014).{FormattingCitation} It
is worth noting that this study is not to determine the depth of global warming content knowledge, but rather to
explore what understanding can be detected through the use of a visualisation tool.
Considering the aforementioned problematic area in socioscientific issues, we suggest that teaching strate-
gies about multifaceted issues need to be directed towards, not only promoting students’ understanding, but also
consolidating the skills needed in each phase of the decision-making process (Böttcher & Meisert, 2013; Hong &
Chang, 2004). Grace (2009) adds that the decision-making framework needs to be provided to improve students’
understanding when dealing with socioscientific issues. To address these issues, a visualisation tool is believed to
be appropriate to explore students’ performances, accompanied by an understanding of the processes that pre-
cede a decision. This is seen as beneficial, if students are provided with a specific visualisation method (Jonassen &
Kim, 2010) and a framework of structural complexity that must be considered in a decision-making process (Grace,
2009; Ratcliffe, 1997; Uskola et al., 2010). Therefore, this research aims to explore students’ understanding when
making a decision on the global warming issue, using the Persuasive Graphic Organiser (PGO) visualisation tool.
General Background
The research design, deemed most appropriate for this study, is a generic qualitative design. This research
design emphasises exploring the process, perspectives and worldviews of the people involved in the setting un-
der study (Cooper & Endacott, 2007). This is especially relevant to elucidate students’ understanding in making a
decision on the global warming issue within a co-curricular context. The researchers act as the “instrument” and
their presence, in the lives of the participants of the study, is fundamental to acquire an in-depth understanding
of the phenomenon. In other words, the researchers are the instructors of this particular study, driven by the aim
of exploring students’ understanding in deciding on global warming issue using the developed PGO. This research
is conducted for six month period within the co-curricular context.
Sample and Administrative Procedures
This research involved 36 students (aged 13-15) from different backgrounds. They were official members of
the Science and Mathematics Society at one Residential School in Kuala Lumpur, Malaysia. The selection of stu-
dents who possessed a fundamental knowledge of global warming was prioritised. The tone of basic knowledge
was indicated via “curriculum mapping” from science syllabuses (Altman, 1989). In choosing the participants and
school, so as to get an extensive understanding about the central phenomenon, a purposeful sampling technique
was utilised (Creswell, 2007; Merriam, 1998). The relevant criteria for expressing scientific and social perspectives
about the socioscientific issue among the students were deemed as the most relevant (Zeidler, Herman, Ruzek,
Linder, & Lin, 2013). Further, these students were expected to have the ability to express a higher interest in work-
ing with, and discussing, the issue. Initially, a recommendation from Educational Planning and Research Division
(MOE) was acquired to suggest a school that was active in co-curricular activities, and also a teacher to suggest the
students who were active and responsive. The students who participated in this study submitted consent forms
from their parents or guardians.
The administrative procedure in this research commenced with a training session, and then this was followed
815
(P. 813-824) ISSN 2538–7138 /Online/
by the actual administration. The one-hour period training session was conducted on day 1 and aimed at provid-
ing students with adequate skills in generating a PGO. This session demonstrated the process of developing the
PGO, including the essential components and visual examples of a PGO. As a PGO is atypical, the students were
provided with a PGO sample (obesity scenario) to get them familiar with a PGO.
A PGO is composed of two components; the “scenario” and the “decision-making graphical organiser.”
The scenario focuses on the controversial issue of global warming, while the graphical organiser emphasises the
main components of the decision-making structure (Ratcliffe, 1997). A combination of these elements enables
students to form a writing that includes persuasive and well-defined arguments, which attempt to persuade read-
ers (Stab & Gurevych, 2014) in a structured manner (Gallavan & Kottler, 2007). The PGO is professionally validated
using socioscientific and qualitative experts. It is also piloted with students from a different school for the evalua-
tion process. Figure 1 illustrates the format of a PGO.
Figure 1: The Persuasive Graphic Organiser (PGO) format.
During the actual administration session (Day 2), a 5-minute video on global warming was presented to
highlight the critical condition happening around the world, and in Malaysia in particular. After that, one scenario
entitled, “Global Warming: A Silent Killer” (refer Figure 2), was administrated as the first phase, i.e. the contextualisa-
tion phase. Holbrook and Rannikmae (2010) pointed out that a socioscientific issue needed to be confined with a
relevant scenario as a procedure for contextualisation.
Figure 2: The scenario of global warming: A silent killer.
The students were then instructed to work in six groups of three to generate their PGO, based on the skills
acquired during the training session. They were allocated in a specially arranged, 90-minute time for discussion
and PGO generation. A total of twelve generated PGOs were completed and returned.
816
ISSN 2538–7138 /Online/ (P. 813-824)
Data Collection
Formal data collection consists of PGOs and interviews. The PGOs are generated through a group discussion,
in which justified takes place using PGO. This is followed by a semi-structured interview dealing with a subsample of
each group, to determine issues experienced when dealing with PGO creation for global warming decision-making.
Steps are taken to ensure that the time intervals between the PGO generation and interview session are not too
lengthy, to enable the students to recall their earlier knowledge and skills. During the data collection process, all
the students are informed that the PGOs generated in this research are not evaluative. They are assured that their
responses to the interview are to be kept confidential and used only for research purposes.
Data Analysis
Data analysis commenced with the analysis of raw data. All the recorded materials were systematically tran-
scribed into text data. The generated PGOs were analysed following the document analysis procedure. Transcription
of the interviews and content of PGOs were explored, and analysed rigorously following the inductive constant
comparative analysis method (Corbin & Strauss, 1990). A familiarisation process was performed by undertaking an
overview of the affluence and diversity of the gathered materials. The whole data were chronologically collected
and analysed to the open, axial and selective coding processes in an endeavour to understand both the structure
(why) and process (how) inherent with the paradigm of the students’ understanding, using the developed PGO.
The final step of the data analysis involved interpretation of meaning of the results. Creswell (2014) emphasised
that the interpretation in qualitative research could be described in many forms; research-based, flexible to con-
vey personal and action meaning. The overall tone of this study was the forms of rigor and scientific credibility, in
which the process of validating the accuracy of information was carried out, starting from the beginning to the
end of the data analysis process.
Trustworthiness
This research employed triangulation and peer review strategies in enhancing its internal validity. For the
triangulation aspect, the PGO and interviews were used to increase confidence in the interpretation. The general
rationale was that with more sources of information, the more likely it could gain a full perception of the phenom-
enon under study (Marshall & Rossman, 2014). The peer review included a review of the interview protocol and
the PGO task, as well as the evaluation of findings. For example, the categories identified and recognised during
the data analysis process were evaluated through expert review. The experts’ and scholars’ feedback, made over
the duration of the study, were scrutinised.
For a transferability sense, Merriam (1998) acknowledged that it was difficult to achieve this in qualitative
research, due to the changing nature of human beings. Thus, this study employed triangulation through the use of
multiple data sources. Moreover, a process of data abstraction, through the constant comparative method taking
place over the entire course of data analysis was identified as a way to set high standards of transferability (Corbin
& Strauss, 1990; Merriam, 1998).
Research Results
Based on the analysis, three different categories of students’ understanding emerged. These categories rep-
resented students’ understanding of global warming through the employment of PGOs as the visualisation tool.
The categories were:
1) identifying the anthropogenic factors affecting global warming,
2) clarifying the effects of global warming in relation to health aspects, and
3) providing alternative solutions for green technology and daily practices.
Since all PGO entries were in the Malay language (the medium of instruction), the entries were translated.
The illustration of PGO generated by the students was initially attached, followed by the translated version for
exemplary. For the benefits of readers, only the translated form is presented in the next section.
817
(P. 813-824) ISSN 2538–7138 /Online/
Identifying the Anthropogenic Factors Affecting Global Warming
In the context of this research, the students visualised the PGO as a tool in identifying anthropogenic factors
affecting global warming. They convinced that “by using this PGO; we can make a clear investigation, and it helps us in
generating a strong statement. The same with the factors of global warming we state here” (Int-S3:40-41). For instance,
the students identified the anthropogenic factors of global warming that were influenced by human activities
through deforestation. They acknowledged that “the first reason we put in the PGO is about forestry. As evidence,
people cut down the trees to run some activities related to industrialisation (cInt-S6:37-39).
Further, they used the PGO to explain about the generation of energy for powering vehicles through fossil
fuel combustion. The examples were given “mostly from petrol produced by cars. Differently, if we want to move the
airplane engine, we have to burn kerosene” (rInt-S5:89-91). They also generalised that “the energy comes largely from
petroleum gasoline. I can conclude that the combustion of fossil fuel, such as petroleum gasoline, resulted in power
generation. For instance, we need energy for vehicles, electrical appliances and construction works” (lInt-S5:92-95).
Figure 3 and Figure 4 detail students’ understanding in identifying deforestation and energy supply as the
anthropogenic factors associated with the controversy.
Figure 3(a): Generated PGO drawn by students - forestry factor.
OPTION 1 (Stop the developmental activities)

Reason 1: Development activities lead to a lot of
logging activities.
Evidence: Lack of clean air.
Carbon dioxide concentration (CO2) b
comes higher.
Figure 3(b): Generated PGO translated by authors - forestry factor.
4) Burning of fossil fuel

•• Aims to generate electricity.
•• When there is an increase in construction activity,
more energy is needed.
Figure 4: Generated PGO translated by authors - energy supply factor.
818
ISSN 2538–7138 /Online/ (P. 813-824)
Clarifying the Effects of Global Warming in Relation to Health Aspect
In this particular research, clarifying the effects of global warming was defined as elucidating obvious conse-
quences of global warming to avoid inexactness in making a decision. Informed by the analysis, the students were
able to clarify the effects of global warming in term of health aspect. It involved “minor illnesses such as cracked skin
and chapped lips” (Int-S2:33), which led also to major illness up to “high risks of diseases including fatal. Other related
examples were “Carbon Dioxide can damage our lungs and trouble the respiratory system” (Int-S2:51-52).
The following PGOs (Figure 5 and Figure 6) illustrate the students’ ability in clarifying health effects as their
justification for the global warming decision.
2) Health will be affected.

•• Skin cancer and respiratory problems are likely to
occur.

Figure 5: Generated PGO translated by authors - health effects.
OPTION 1 (Stop the developmental activities)

Reason 1: Endanger the lives of living beings.
Evidence: -Statistics Health Organisation world shows 150,000
deaths each year.
-It is caused by unhealthy air due to excessive release of
carbon dioxide.
Figure 6: Generated PGO translated by authors - health effects.
By referring to the generated PGOs, the students could elaborate ideas concerning the effects of global
warming further. They revealed “the use of PGO makes me realize that there is another health effect which might be
considered for health category. We can relate general health with the emotional effect because the hotness causes us
stress or depression” (cInt-S6:30-32).
Providing Alternative Solution in Terms of Green Technology and Daily Practices
In the framework of this research, the students were able to provide alternative solutions to present a possible
choice between two options of global warming decision. They found the PGO to be a visualised tool providing
alternative solutions “indeed, by generating the PGO, we feel that we can imagine and suggest some solutions for the
option taken, as well as the rejected option. Again, we can provide solutions, and maybe we can improve it further” (Int-
S3:141-142). They gave some examples of green technology such as “hybrid car, magnetic train” (rInt-S2:148-149),
and “yes, other than that, we can use electric vehicles too!” (rInt-S2:152). They were able to put forward alternative
solutions by stating the significance of “greening vehicles that have been used in the developed countries will not pol-
lute the environment, thus reduce the Green House Gases emissions” (rInt-S4:92-93). This could be shown through the
application of green transportation as examples of the environmentally friendly products (Figure 7).
The increase in environmentally friendly products.

Example: Hybrid car (Toyota Prius)
Electric cars (Nissan Leaf )
Magnetic Train in Japan (Shikanzen)
Figure 7: Generated PGO translated by authors - greening vehicles alternative solutions.
819
(P. 813-824) ISSN 2538–7138 /Online/
The revelation of alternative solutions could also be grounded from the next PGO, which demonstrated that the
students were able to provide solutions for daily practices that were considered vital in reducing global warming.
SOLUTIONS
Stop the developmental activities.
1. Reducing the usage of electrical appliances
which release CFC.

= Example: Using electrical appliances activities which
release HFC.
2. Applying lotion that contains SPF15 and above.
Figure 8. Generated PGO translated by authors - daily practices alternative solutions.
As shown in Figure 8, the students used the PGO as a tool to further clarify the alternative solutions to
electrical appliances and cosmetics products. For example,“by using the PGO, we can visualize related solutions
such as HCFC-based electronic devices which contain Hydrochlorofluorocarbon as the alternatives for this decision”
(cInt-S2:70-71). The essentials of the new technology in the cosmetic industry were also suggested; “the use
of SPF lotion, sun protection factor is a UV light cream that can be applied to our skin as our health routine” (cInt-
S2:74-76).
Discussion
This research set out to determine students’ understanding in making a decision on the socioscientific issue
through the employment of a PGO. The PGO is relevant and workable to recognise the key matters that require
relevant science knowledge in global warming deliberation. The findings revealed that the PGO operates as a
visualisation tool in identifying anthropogenic factor affecting global warming, clarifying the effects of global
warming in term of health aspects and providing alternative solutions regarding green technology and daily
practices. The majority of students were able to connect their scientific knowledge when addressing informed
decision. They revealed that the use of the PGO helped them to visualize evidence for possible options and judge
the sufficiency of proof for the satisfactory and rejected conclusions (Maloney & Simon, 2006).
This is probably due to the method that assists the students in visualising the structural complexity of the
decision-making process. Thus, the students can explicitly constitute scientific evidence and conceptualisation
when dealing with socioscientific decisions (Walker & Zeidler, 2007). Tishman and Palmer (2005) highlight that
the employment of visualised tool enables students to think visibly, hence expresses powerful knowledge. They
add that visible thinking refers to any kind of visible representation that documents and supports the develop-
ment of ongoing thoughts, matters, reasoning, and reflections. In this case, PGO serves as the visualised tool to
organise science information when dealing with decision-making situation.
The controversial issue introduced through a global warming scenario was chosen in the context of this
research. Klosterman and Sadler (2010) claimed that the combined use of qualitative proximal assessment and
quantitative distal assessment offered contradictory evidence for understanding global warming. The find-
ings of the current study were consistent with their qualitative assessment, where students could connect the
causes and incorporate the potential consequences associated with global warming. Although the students
were not given specific questions, their responses determined the knowledge of science surrounding the fac-
tors and consequences of global warming. More specifically, they used the PGO to identify forestry and fossil
fuel combustion as the anthropogenic factors. As for the effects, the students considered the health aspect as
the consequence of global warming.
The utilisation of the PGO in determining students’ understanding, however, is not in agreement with
the quantitative studies by Hasselhorn and Bogeholz (2013) and Callahan, Zeidler, and Orasky (2011). The use
820
ISSN 2538–7138 /Online/ (P. 813-824)
of a questionnaire has been shown to be the main issue in which the students are restricted in prioritising
their knowledge, and thus fail to offer more solutions for the option undertaken. The poor detection power of
quantitative method in identifying the level of understanding and demotivation parameters of revisiting the
same instrument can lead to a more serious issue. Moreover, the students cannot reveal their new ideas and
the whole phenomenon, because the selection of construct criteria is purposely created. The students in this
study, however, reveal that the use of a PGO has helped them in visualising the knowledge for the option taken.
The trigger of providing a better solution is also acknowledged because they can visualise the ideas of issues
that precede decision.
Further, the students in this research could provide justifications for the solutions to the controversy associ-
ated with global warming. In other words, they provided “multiple sources” of alternative solutions as a way to
address global warming crisis. For instance, they suggested that citizens would need to alter their mind, attitude,
and lifestyle through the exposure to green technology and daily practices. Surprisingly, students were also able
to discuss more than one previous issue as the sources of controversy beyond global context.
Evidence from this research gives support and details to similar research like using writing frames for the
argumentation process (Choi et al., 2014; Dawson & Venville, 2010). According to Acar, Turkmen and Roychoudhury
(2010), the incorporation of decision-making findings may help students to overcome some problematic areas
in the argumentation field. It includes the means of avoiding mis-evaluation of evidence and inappropriate use
of value-based reasoning. This research does not contradict (Choi & Meir, 2014) with the findings that fifth-grade
students actively offer more evidence, support claims, and critique and negotiate evidence in making a decision
about plant and human health investigation. They postulate that the use of the Science Writing Heuristic (SWH)
approach, in an online environment, supports students’ understanding. Dawson and Venville (2010) claim that
the use of writing frames, which are equipped with guiding questions, plays a significant role in scaffolding
students’ thinking. Similarly, a PGO, which structures the decision-making framework, acts as a mental prompt
for students to convey their knowledge, following the decision- making process. This is because the PGO signi-
fies the enhancement of students’ understanding, as they are able to offer a variety of information and connect
the relationship between claim and evidence. Otherwise, the PGO offers students a way to identify new and
untouched phenomena. Hence, it may provide a deeper understanding of the issues which arise.
In summary, the current research has unveiled merely the tip of the iceberg, that is, the PGO operates as a
visualised and structured tool in enhancing students’ understanding and decision-making skills. The PGO can
be utilised as a mental prompt and visible guidance for promoting students’ understanding (Choi et al., 2014),
as well as an organiser, when making a decision (Dawson & Venville, 2010; Keys, 1997; Mastura & Rohaida, 2015;
Ratcliffe, 1997). Further, the discrimination of students’ understanding can be traced, as there are clear justifica-
tions provided to determine whether they possess a naïve or sophisticated understanding (Callahan, Zeidler, &
Orasky, 2011). Also, the use of a one-time PGO employment can minimise the possibilities of decreased motiva-
tion among students and this leading to instability of understanding measures (Callahan et al., 2011).
Conclusions
The functionality of the PGO offers a synergy point for advanced extension, especially in promoting students’
understanding in making a decision on a socioscientific issue. Considering these results, it can be concluded
that the PGO is beneficial for promoting the didactic structure of students’ understanding and decision-making
skills, particularly for the global warming issue. The PGO is definitely not seen as a separate entity from the
decision-making framework, as it provides the structure in the graphical organiser section. Relevantly, this re-
search confers implications for future research in the methodological implication. These include strategies on
how to promote students’ understanding, overcome structural complexities and provide flexibility in students’
decision-making process.
The PGO allows visible thinking amongst the students. Pertinently, the PGO that contains a graphical or-
ganiser allows ongoing annotation, evaluation, addition and revision. By visualising the knowledge, the students
can reveal the key relationships between the decision and evidence, facts and questions, and certainties and
uncertainties. The messiness of these complexities can be overcome because the PGO can change and interlock
visible relationships. It helps students to offer authentic knowledge, instead of just memorising facts. As shown
in the findings, the PGO supports students to explain complex information, reduces cognitive loads and con-
821
(P. 813-824) ISSN 2538–7138 /Online/
nects the information sources of global warming. The use of PGO as the visual instrument enables students to
envisage the physical structure and relationship, and interpret the meaning of the structure when dealing with
decision-making process and argumentation.
It is beyond the scope of this study to speculate whether there is knowledge progression or whether
misconceptions emerge. The detection of misconceptions exhibited by the students, fallacious reasoning and
multiple reasoning can also be explored. It is applicable due to the flexible direction of the PGO which need not
be confined to specific questions, as detailed in the quantitative survey. Therefore, it may provide information
that may sometimes be revealed by personal experience, which is also converted into numerical form. However,
there may be limits to the extent to which the PGO is the only example presented in the context of a global
warming scenario. This scenario is, however, “changeable” in nature, or can be replaced by other appropriate so-
cioscientific scenarios and contexts. The familiarisation, change, and stability of the PGO’s usability in identifying
students’ understanding can be explored over time in other localities and educational settings. This study has
contributed to some indications for educators and curriculum developers in promoting students’ understanding
and decision-making skills, especially in the reflexion of socioscientific issues.
Acknowledgement
A special thanks to the Ministry of Education Malaysia for granting the Post-Graduate Scholarship Scheme
and University of Malaya for the professional supports.
References
Acar, O., Turkmen, L., & Roychoudhury, A. (2010). Student difficulties in socio-scientific argumentation and decision-
making research findings: Crossing the borders of two research lines. International Journal of Science Education, 32
(9), 1191–1206. doi:10.1080/09500690902991805.
Aikenhead, G. S. (1985). Collective decision making in the social context of science. Science Education, 69 (4), 453–475.
Albe, V. (2007). When scientific knowledge, daily life experience, epistemological and social considerations intersect: Stu-
dents’ argumentation in group discussions on a socio-scientific issue. Research in Science Education, 38 (1), 67–90.
doi:10.1007/s11165-007-9040-2.
Altman, H. B. (1989). Syllabus shares “what the teacher wants.” The Teaching Professor, 3 (5), 1–2.
Böttcher, F., & Meisert, A. (2013). Effects of direct and indirect instruction on fostering decision-making competence in
socioscientific issues. Research in Science Education, 43 (2), 479–506. doi:10.1007/s11165-011-9271-0.
Callahan, B. E., Zeidler, D. L., & Orasky, J. (2011). The use of a socioscientific issues based curriculum to enhance nature of
science understanding in high school biology students. In At the Meeting of the Association for Science Teacher Educa-
tion Annual International Conference. Minneapolis, MN.
Choi, A., Hand, B., & Norton-Meier, L. (2014). Grade 5 students’ online argumentation about their in-class inquiry investiga-
tions. Research in Science Education, 44 (2), 267–287. doi:10.1007/s11165-013-9384-8.
Christenson, N., Chang Rundgren, S.-N., & Zeidler, D. L. (2014). The relationship of discipline background to upper second-
ary students’ argumentation on socioscientific issues. Research in Science Education, 44 (4), 581–601. doi:10.1007/
s11165-013-9394-6.
Corbin, J. M., & Strauss, A. (1990). Grounded theory research: Procedures, canons, and evaluative criteria. Qualitative Sociol-
ogy, 13 (1), 3–21. doi:10.1007/BF00988593.
Creswell, J. W. (2007). Qualitative inquiry & research design, choosing among five approaches (2nd ed.). Thousand Oaks
California: Sage.
Dawson, V. M., & Venville, G. J. (2010). Teaching strategies for developing students’ argumentation skills about socioscien-
tific issues in high school genetics. Research in Science Education, 40 (2), 133–148. doi:10.1007/s11165-008-9104-y.
Fleming, R. (1986). Adolescent reasoning in socio-scientific issues, part 1: Social cognition. Journal of Research in Science
Teaching, 23 (8), 677-687.
Gallavan, N. P., & Kottler, E. (2007). Eight types of graphic organizers for empowering social studies students and teachers.
The Social Studies, 98 (3), 117–128. doi:10.3200/TSSS.98.3.117-128.
Grace, M. (2009). Developing high quality decision-making discussions about biological conservation in a normal classroom
setting. International Journal of Science Education, 31 (4), 551–570. doi:10.1080/09500690701744595.
Gresch, H., Hasselhorn, M., & Bögeholz, S. (2013). Training in decision-making strategies: An approach to enhance students’
competence to deal with socio-scientific issues. International Journal of Science Education, 35 (15), 2587–2607. doi:1
0.1080/09500693.2011.617789.
Halim, M., & Mohd Saat, R. (2015). Development of persuasive graphic organizer in identifying students’ decision mak-
ing skill about socioscientific issues. In 6th. International Conference on Science and Mathematics Education (CoSMEd
2015). Penang, Malaysia.
822
ISSN 2538–7138 /Online/ (P. 813-824)
Heath, P. A., White, A. L., Berlin, D. F., & Park, J. C. (1987). Decision making: Influence of features and presentation mode
upon generation of alternatives. Journal of Research in Science Teaching, 24 (9), 821–833.
Holbrook, J., & Rannikmae, M. (2010). Contextualisation, de-contextualisation, re-contextualisation – A science teaching
approach to enhance meaningful learning for scientific literacy. In: Eilks, I., & Ralle, B. (Eds.). Contemporary Science
Education (pp. 69-82). Shaker Verlag.
Hong, J. L., & Chang, N. K. (2004). Analysis of Korean high school students’ decision making processes in solving a problem
involving biological knowledge. Research in Science Education, 34 (1), 97–111.
Hundal, S., Levin, D. M., & Keselman, A. (2014). Lessons of researcher-teacher co-design of an environmental health afterschool
club curriculum. International Journal of Science Education, 36 (9), 1510–1530. doi:10.1080/09500693.2013.844377.
Jho, H., Yoon, H. G., & Kim, M. (2014). The relationship of science knowledge, attitude and decision making on socio-scientific
issues: The case study of students’ debates on a nuclear power plant in Korea. Science & Education, 23 (5), 1131–1151.
doi:10.1007/s11191-013-9652-z.
Jimeńez-Aleixandre, M.-P. (2002). Knowledge producers or knowledge consumers? Argumentation and decision
making about environmental management. International Journal of Science Education, 24 (11), 1171–1190.
doi:10.1080/09500690210134857.
Jonassen, D. H., & Kim, B. (2010). Arguing to learn and learning to argue: Design justifications and guidelines. Educational
Technology Research and Development, 58 (4), 439–457. doi:10.1007/s11423-009-9143-8.
Keys, C. W. (1997). An investigation of the relationship between scientific reasoning, conceptual knowledge and model
formulation in a naturalistic setting. International Journal of Science Education, 19 (8), 957–970.
Klosterman, M. L., & Sadler, T. D. (2010). Multi-level assessment of scientific content knowledge gains associ-
ated with socioscientific issues-based instruction. International Journal of Science Education, 32 (8), 1017–1043.
doi:10.1080/09500690902894512.
Kolsto, S. D. (2001). “To trust or not to trust,…” - pupils’ ways of judging information encountered in a socio-scientific issue.
International Journal of Science Education, 23 (9), 877–901. doi:10.1080/09500690010016102.
Kolsto, S. D. (2006). Patterns in students’ argumentation confronted with a risk-focused socio-scientific issue. International
Journal of Science Education, 28 (14), 1689–1716. doi:10.1080/09500690600560878.
Maloney, J., & Simon, S. (2006). Mapping children’s discussions of evidence in science to assess collaboration and argumen-
tation. International Journal of Science Education, 28 (15), 1817–1841. doi:10.1080/09500690600855419.
Marshall, C., & Rossman, G. B. (2014). Designing qualitative research. London: Sage Publication.
Merriam, S. B. (1998). Qualitative research and case study applications in education. Revised and expanded from“ case study
research in education.” California: Jossey-Bass.
Neumann, A., Gräber, W., & Tergan, S. O. (2005). ParIS - Visualizing ideas and information in a resource-based learning
scenario. In Tergan, S. O., & Keller, T. (Eds.), Knowledge and Information Visualization (pp. 256-281). Verlag Berlin Hei-
delberg: Springer.
Rakes, G. C. (1999). Teaching visual literacy in a multimedia age. TechTrends, 43 (4), 14–18.
Ratcliffe, M. (1997). Pupil decision-making about socio-scientific issues within the science curriculum. International Journal
of Science Education, 19 (2), 167–182. doi:10.1080/095006997019020.
Sadler, T. D., Chambers, F. W., & Zeidler, D. L. (2004). Student conceptualizations of the nature of science in response to a
socioscientific issue. International Journal of Science Education, 26 (4), 387–409. doi:10.1080/0950069032000119456.
Sadler, T. D., & Donnelly, L. A. (2006). Socioscientific argumentation: The effects of content knowledge and morality. Inter-
national Journal of Science Education, 28 (12), 1463–1488. doi:10.1080/09500690600708717.
Sadler, T. D., & Zeidler, D. L. (2005). Patterns of informal reasoning in the context of socioscientific decision making. Journal
of Research in Science Teaching, 42 (1), 112–138. doi:10.1002/tea.20042.
Sadler, T. D., & Zeidler, D. L. (2009). Scientific literacy, PISA, and socioscientific discourse: Assessment for progressive aims
of science education. Journal of Research in Science Teaching, 46 (8), 909–921. doi:10.1002/tea.20327.
Sadler, T. D., Barab, S.A. & Scott, B. (2007). What do students gain by engaging in socioscientific inquiry? Research in Science
Teaching, 37 (4), 371–391.
Sakschewski, M., Eggert, S., Schneider, S., & Bögeholz, S. (2014). Students’ socioscientific reasoning and decision-making
on energy-related issues-Development of a measurement instrument. International Journal of Science Education, 36
(14), 2291–2313. doi:10.1080/09500693.2014.920550.
Stab, C., & Gurevych, I. (2014). Identifying argumentative discourse structures in persuasive essays. In Proceedings of the
2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 46–56). Retrieved from http://
www.aclweb.org/anthology/D14-1006.
Tishman, S., & Palmer, P. (2005). Visible thinking. Leadership Compass, 2 (4), 1–4.
Toulmin, S. E. (2003). The uses of argument. London: Cambridge University Press.
Uskola, A., Maguregi, G., & Jimenez-Aleixandre, P. M. (2010). The use of criteria in argumentation and the construction
of environmental concepts: A university case study. International Journal of Science Education, 32 (17), 2311–2333.
doi:10.1080/09500690903501736.
Vesterinen, V.-M., Tolppanen, S., & Aksela, M. (2016). Toward citizenship science education: what students do to make the
world a better place? International Journal of Science Education, 38 (1), 30–50. doi:10.1080/09500693.2015.1125035.
Walker, K. A., & Zeidler, D. L. (2007). Promoting discourse about socioscientific issues through scaffolded inquiry. Interna-
tional Journal of Science Education, 29 (11), 1387–1410. doi:10.1080/09500690601068095.
823
(P. 813-824) ISSN 2538–7138 /Online/
Zeidler, Herman, B. C., Ruzek, M., Linder, A., & Lin, S.-S. (2013). Cross-cultural epistemological orientations to socioscientific
issues. Journal of Research in Science Teaching, 50 (3), 251–283. doi:10.1002/tea.21077.
Zohar, A., & Nemet, F. (2002). Fostering students’ knowledge and argumentation skills through dilemmas in human genet-
ics. Journal of Research in Science Teaching, 39 (1), 35–62. doi:10.1002/tea.10008.
Received: August 03, 2017 Accepted: October 05, 2017
Mastura Halim Master in Science Education, PhD Student and Lecturer

at Institute of Teacher’s Training College, Department of
Mathematics & Science Education, Faculty of Education,
University of Malaya, 50603 Kuala Lumpur, Malaysia.
E-mail: masturahalimii@siswa.um.edu.my
Website: https://education.um.edu.my
Rohaida Mohd Saat PhD, Professor, Dean of Education Faculty, Department of
Mathematics & Science Education, Faculty of Education,
University of Malaya, 50603 Kuala Lumpur, Malaysia.
E-mail: rohaida@um.edu.my
Website: https://education.um.edu.my
824
INFORMATION
FOR CONTRIBUTORS
EDITORIAL AND REVIEW PROCEDURES
Journal of Baltic Science Education (JBSE) publishes original scientific research articles in the field of Natural Science Education and related
areas for all educational levels in the Baltic countries. It is possible to publish special (thematic) issues of JBSE. The papers should be submitted
and will be published in English. JBSE will promote to establish contacts between researchers and practical educators both in the Baltic countries
and countries around.
The authors of the manuscripts are responsible for the scientific content and novelty of the research materials. Articles, published before in
other international journals or papers’ collections will not be accepted for publication in JBSE.
As a publication that represents a variety of cross-disciplinary interests, both theoretical and practical, the JBSE invites manuscripts on a wide
range of topics, especially in the following areas:
• Didactics of natural sciences. • Philosophical, political, economical and social aspects of
• Theory and practice in natural science teacher education. natural science education.
• Integrated natural science education. • The supplementary natural science education.
• Natural science and technological literacy. • ICT in natural science education.
• General and professional natural science education. • The standardisation of natural science education etc.
MANUSCRIPTS GUIDELINES
The structure of the research paper presented to the Journal of Baltic Science Education should be as follows: abstract - short report of the
investigation; introduction inc. aim and subject of the research; research methodologies and methods; results of the research incl. discussion; conclusions;
list of references in APA style.
The papers should be submitted in English. If English is a second language for the author, please consider having the manuscript proof read
and edited before submitting. The preliminary text of the article can be sent as a.doc file in the attachment by e-mail: mail.jbse@gmail.com
The text must be elaborated in Word for Windows, using 12 point Times New Roman letters. An article should not exceed 7-10 A4 pages, included
figures, tables and bibliography. Publishing of longer articles should be negotiated separately. Texts margins: top and bottom 20mm, left - 25mm, right
- 20mm. The title: capital letters, 14pt, bold; space between the title and the author’s name is one line interval. Author’s name and surname: small letters,
12pt, bold. Under the name, institution: 11 pt, italics; space between the title and the text: 1 line interval. Abstract – about 100-150 words - precedes the
text. The text: 12pt Single or Auto spacing, in one column. Key words: no more than five words. The language must be clear and accurate. The authors
have to present the results, propositions and conclusions in a form that can suit scientists from different countries.
Titles of the tables and figures: 11 pt, small letters. Space between figures or tables and the text: 1 line interval. Introduction, titles of chapters and
subchapters: 12pt, bold, small letters. Numbers: Arabic, subchapters numbered by two figures (1.1, 1.2, etc.). Figures, tables and captions should be inserted
within the manuscript at their appropriate locations. Diagrams and graphs should be provided as finished black and white line artwork or electronic
images. When there are a number of illustrations, the author should endeavour to reduce the amount of text to accommodate the illustrations in the
limited space available for any article.
References in the text should be presented in brackets (Knox, 1988; Martin, 1995). If necessary, the page can be indicated: (Martin, 1995, p.48).
The list of references should be presented after the text. The Words List of References: 11pt, bold, small letters. The references should be listed in
full at the end of the paper in the following standard form:
For books: Saxe, G.B. (1991). Cultural and Cognitive Development: Studies in Mathematical Understanding. Hillsdale, NJ: Erlbaum.
For articles: Bekerian, D.A. (1993). In Search of the Typical Eyewitness. American Psychologist, 48, 574-576.
For chapters within books: Bjork, R.A. (1989). Retrieval Inhibition as an Adaptive Mechanism in Human Memory. In: H.L. Roediger III & F.I.M. Craik
(Eds.), Varieties of Memory & Consciousness (pp. 309-330). Hillsdale, NJ: Erlbaum.
On a separate page, author - related data should be presented in English: name, surname, degree and academic title, institution, full correspondence
address in the clearest and most complete form /ordinary post and e-mail addresses /, position (to ensure anonymity in the review process). The author
(authors) should confirm in writing, that the manuscript has not been published in other journal or handed over (transferred) to other
journal for publication.
EDITORIAL AND REVIEW PROCEDURES

Manuscripts will be sent anonymously to reviewers with expertise in the appropriate area. All manuscripts will be rewieved by two experts
before JBSE’s accept them for publication. This process usually takes about two months. The journal co-editors will make minor editorial changes;
major changes will be made by the author(s) prior to publication if necessary. JBSE’s redaction will sent to author(s) only one correcture which
must be sent back within 2 weeks. JBSE will not review submissions previously published elsewhere through print or electronic medium.
Manuscripts submitted to the JBSE cannot be returned to authors. Authors should be sure to keep a copy for themselves. Authors’ signatures
should be at the end of the paper and its second checked proofs.
Manuscripts, editorial correspondence (and other correspondence for subscription and exchange), and any questions should be sent to
editor-in-chief or to regional redactors.
Mailing Addresses
Prof., Dr. Vincentas Lamanauskas, editor-in-chief, Dr. Naglis Švickus, co-editor, Lithuania
Siauliai University SMC ”Scientia Educologica”
P. Vishinskio Str. 25; LT-76351 Siauliai, Lithuania Kretingos Str. 55-10; LT-92300 Klaipėda, Lithuania
E-mail: v.lamanauskas@ef.su.lt E-mail: naglis.svickus@nbgroup.lt
Phone: + 370 687 95668 Phone: +370 687 89985
Prof., Dr. Janis Gedrovics, co-editor, Latvia

Riga Teacher Training and Educational Management Academy
Imantas 7 linija No 1; Riga, LV-1083, Latvia
E-mail: janis.gedrovics@rpiva.lv
Phone: +371 29162147
825
JBSE
Problems of Psychology in the 21st

Century is an international, periodical,
peer reviewed scientific journal, issued by
the Scientia Socialis, UAB in cooperation
with SMC “Scientia Educologica”.
Editor-in-Chief
Prof. dr. Vincentas Lamanauskas, Scientific

Methodical Centre „Scientia Educologica“,
Republic of Lithuania
Problems of Psychology in the 21st Century

(ISSN 2029-8587) is abstracted and/or indexed
in:
Index Copernicus -
http://journals.indexcopernicus.com
EBSCO - http://search.ebscohost.com
Editorial Board
Dr., prof. Ferda Aysan, Dokuz Eylul University, Turkey

Dr., prof. Serhiy Boltivets, Grigory Kostyuk Psychological Institute of the Ukrainian National
Academy of Pedagogical Sciences, Ukraine
Dr., assoc. prof. Irena Gailiene, SMC "Scientia Educologica", Republic of Lithuania
Dr., prof. Irakli Imedadze, Dimitri Uznadze Georgian Psychological National Society, Georgia
Dr. Julia Lakhvich, Belarusian State University, Republic of Belarus
Dr., prof. Vladimir S. Karapetyan, Armenian State Pedagogical University named after Kh.
Abovyan, Armenia
Dr., prof. Vincentas Lamanauskas, Scientific Methodical Centre „Scientia Educologica“, Republic of
Lithuania (Editor-in-Chief)
Dr., prof. Maria Ledzińska, University of Warsaw, Poland
Dr., prof. Aleksandr Lobanov, Belarusian State Pedagogical University, Republic of Belarus
Dr., prof. Guna Svence, Riga Teacher Training and Educational Management Academy, Latvia
Website: http://www.jbse.webinfo.lt/PPC/Problems_of_Psychology.htm
826
JBSE
Quality Issues and Insights in the 21st
Century is an international, periodical,
peer reviewed scientific journal, issued by
the Scientia Socialis, UAB in cooperation
with SMC “Scientia Educologica”.
Editor-in-Chief
Prof. dr. Vincentas Lamanauskas, Scientific

Methodical Centre „Scientia Educologica“,
Republic of Lithuania
Quality Issues and Insights in the 21st Century

(ISSN 2029-9575) is abstracted and/or indexed
in:
Index Copernicus -
http://journals.indexcopernicus.com
Electronic Serials in Lithuania -

http://www.eperiodika.lt/index.php?id=moksl
Editorial Board
MSc., Renata Bilbokaitė, University of Siauliai, Republic of Lithuania (Editorial Assistant)

Dr., Andrea Bernhard, Universities Austria, Austria
Dr., Prof. John Gowland Mwangi, Egerton University, Kenya
Dr., Prof. Vincentas Lamanauskas, Scientific Methodical Centre „Scientia Educologica“, Republic of
Lithuania (Editor-in-Chief)
Dr., Prof. Liviu Moldovan, „Petru Maior” University of Tirgu Mures, Romania
Dr., Prof. Kestutis Pukelis, Vytautas Magnus University, Republic of Lithuania
Prof. Ramayah Thurasamy, University of Science Malaysia, Malaysia
Dr., Assoc. Prof. Dalius Serafinas, Vilnius University, Republic of Lithuania
Dr., Assoc. Prof. Adriana Tafrova-Grigorova, Sofia University St. Kliment Ohridski, Bulgaria
Website: http://www.jbse.webinfo.lt/QIIC/Quality_Issues_Insights.htm
827
ISSN 1648-3898 /Print/, ISSN 2538-7138 /Online/
Compiled by: Vincentas Lamanauskas

Linguistic Editor: Ilona Ratkevičienė
Cover design by: Jurgina Jankauskienė
Layout design by : Linas Janonis

30 October 2017. Publishing in Quires 12,5. Edition 200
Publisher SMC „Scientia Educologica“ in cooperation with Scientia Socialis,

Donelaicio Street 29, LT-78115 Siauliai, Lithuania
E-mail: scientia@scientiasocialis.lt
Phone: +370 687 95668
http://www.jbse.webinfo.lt/centras.htm
http://www.scientiasocialis.lt

Printing Šiauliai printing house
9A P. Lukšio Street
LT-76207 Šiauliai, Lithuania
Phone: +370 41 500 333.
Fax: +370 41 500 336
E-mail: info@dailu.lt

Journal of Baltic Science Education, Vol. 16, No. 5, 2017

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Journal of Baltic Science Education, Vol. 16, No. 5, 2017

Uploaded by

Copyright:

Available Formats

Vol.16, No.

The International Journal of the Scientific Methodical Center “Scientia Educologica”

Published since 2002 Address:

ISSN 2538–7138 /Online/

USER-FRIENDLY REPORTING OF RESEARCH FINDINGS FROM A QUESTIONNAIRE

THE CONTRIBUTION OF DEMONSTRATION AND STUDENT-LED EXPERIMENTS

INFORMATION FOR CONTRIBUTORS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 825

USER-FRIENDLY REPORTING OF RESEARCH FINDINGS FROM A QUESTIONNAIRE

Received: October 04, 2017 Accepted: October 19, 2017

Andrej Šorgo PhD, Associate Professor of Biology Didactics, University of Maribor,

Abstract. The aim of this research is to

Focus and Aim of Research

General Background and Design

Instrument and Procedures

Explain your answer:

a) The air is all around us

a) The air expands on heating

a) The air expands on heating

a) The wind moves the balloon

The Treatment in the Control Group

The Treatment in the Experimental Group

Necessary materials and supplies:

Description of the experiment:

Necessary materials and supplies

Description of the experiment:

Figure 4: An example of experiments for some of the teaching units.

The acquired data were used for:

Balancing the Groups

Cognitive level Group M(X ) SD t p

Cognitive level Group M(X ) SD CV (%) t p

Table 3. Results of students C and E group on the Kolmogorov–Smirnov normality test.

The total number of points

Cognitive level Group M(X ) SD CV (%) t p

E 2.921 .94 22.72

E 3.926 1.42 18.87

E 4.988 .910 16.79

E 9.022 1.43 4.79

E 5.022 1.29 5.12

E 3.887 2.18 4.95

The total number of points

Comparative Analysis of the Quality of Students’ Knowledge (ANOVA Analysis)

Received: May 17, 2017 Accepted: August 05, 2017

MODEL TO IMPROVE PHYSICS ISSN 2538-7138 /Online/

PROBLEM SOLVING AND SELF-

Table 1. Validity and reliability of PIL model criteria

Check Scale Statistics Category

Instrument and Procedures

Table 2. Validity and Reliability of PIL Model.

Content validity Construct validity

Problem solving skills indicator

Pre-test 28.71 28.18 29.92 42.10 36.93

Self-directed learning skills indicator

Pre-test 48.94 39.22 42.56 45.23 42.21 44.09

Physics problem solving skills Self-directed learning skills

Group Test N Asymp. Asymp.

posttest 22 78.34 4.88 .01 no 79.81 7.84 .92 yes

Post-test and Pre-test 22 - 4.108 < .0001

Physics problem solving skills Self-directed learning skills

Pair N Std. Std.

Pair 1 22 Wilcoxon test 36.30 2.22 16.32 21 < .0001

n-gain all groups Sum of Squares df Mean Square F p

The Validity and Reliability of PIL Model

The Effectiveness of PIL Model and Its Application

Received: May 26, 2017 Accepted: August 06, 2017