Journal of Baltic Science Education, Vol. 15, No. 4, 2016

Vol.15, No.
4, 2016
ISSN 1648-3898
The International Journal of the Scientific Methodical Center “Scientia Educologica”

Journal of Baltic Science Education, Vol. 15, No. 4, 2016
Editorial Board
ISSN 1648–3898
Editor-in-Chief
Prof., Dr. Vincentas Lamanauskas Siauliai University, Lithuania
Deputy Editor-in-Chief
Prof., Dr. Andris Broks University of Latvia, Latvia
Deputy Editor-in-Chief
Prof., Dr. Miia Rannikmäe University of Tartu, Estonia
Executive Secretary
Dr. Laima Railienė Scientific Methodical Center “Scientia Educologica”, Lithuania
Editors:
Prof., Dr. Boris Aberšek University of Maribor, Slovenia
Prof., Dr. Agnaldo Arroio University of Sao Paulo, Brazil
Prof., Dr. Martin Bilek University of Hradec Králové, Czech Republic
Dr. Paolo Bussotti University of Udine, Italy
Dr. Bulent Cavas Dokuz Eylul University, Turkey
Prof., Dr. Hana Čtrnáctová Charles University, Czech Republic
Dr. Peter Demkanin Comenius University, Slovakia
Dr. André du Plessis Nelson Mandela Metropolitan University, South Africa
Riga Teacher Training and Educational Management
Prof., Dr. Jānis Gedrovics
Academy, Latvia
Prof., Dr. Peter Heering University of Flensburg, Germany
Prof., Dr. Jack Holbrook University of Tartu, Estonia
Prof., Dr. Ryszard M. Janiuk Maria Curie Sklodowska University, Poland
Dr. Milan Kubiatko University of Zilina, Slovakia
Dr. Todar Lakhvich Belarusian State Medical University, Republic of Belarus
Prof., Dr. Jari Lavonen University of Helsinki, Finland
Dr. Rita Makarskaitė-Petkevičienė Lithuanian University of Educational Sciences, Lithuania
Prof., Dr. Aadu Ott Göteborg University, Sweden
Dr. Paul Pace Malta University, Malta
Prof., Dr. Valfrids Paškevičs Daugavpils University, Latvia
Prof., Dr. Jongwon Park Chonnam National University, Korea
Dr. Raffaele Pisano Lille 1 Science and Technology University, France
Dr. Pavol Prokop Institute of Zoology, Bratislava, Slovakia
Dr. Alona Rauckienė-Michaelsson Lithuanian University of Educational Sciences, Lithuania
Prof., Dr. Heimo Saarikko Helsinki University, Finland
Dr. Uladzimir K. Slabin University of Oregon, USA
Prof., Dr. Valery P. Solomin Herzen State Pedagogical University of Russia, Russia
Prof., Dr. Borislav V. Toshev Sofia University, Bulgaria
Dr. Georgios Tsaparlis University of Ioannina, Greece
Dr. Muhammet Usak Gazi University, Turkey
A scientific journal JBSE issued by the SMC Scientia Educologica in cooperation with Scientia Socialis, Lithuania, empha-
sizes theoretical, experimental and methodical studies in the field of science education. JBSE is an international academ-
ic journal. In order to maintain the high standards appropriate to such a journal, all contributions received are submitted
for anonymous review by two experts, additionally to review by the Editor. The decision of the Editor on the acceptance
of articles is final and no correspondence can be entered into on reasons for rejection of a submitted contribution.
Published since 2002 Address:

The journal is published bimonthly. Scientific Methodical Center “Scientia Educologica”
IF-0.226 (2010), 0.425 (2011), 0.444 (2012), 0.481 (2013), Donelaičio Street 29, LT-78115 Siauliai, Lithuania
0.448 (2014). E-mail: mail.jbse@gmail.com
5-Year IF: 0.412. Phone: +370 687 95668
Home page: http://www.scientiasocialis.lt/jbse/
ISSN 1648–3898 © Scientific Methodical Center „Scientia Educologica“, Lithuania,

The Associated Member of Lithuanian Scientific Society,
European Society for the History of Science (ESHS) and ICASE
The articles appearing in this journal are indexed/abstracted in British Education Index
(http://www.leeds.ac.uk/bei/bei.htm), Copernicus Index (http://journals.indexcopernicus.com), The Asian
Education Index (http://www.asian-education-index.com/sciences_index.php), EBSCO: Academic Search Premier
406 (http://search.ebscohost.com), Social Scisearch (Thomson Reuters) (http://science.thomsonreuters.com/index.html),
Journal Citation Reports / Social Sciences Edition (Thomson Reuters)
(http://thomsonreuters.com/products_services/scientific/Journal_Citation_Reports),
and SCOPUS (http://www.scopus.com)
ISSN 1648–3898 Contents
Contents
Editorial

SYSTEMS THEORY OF SYSTEMS THINKING: GENERAL AND PARTICULAR WITHIN MODERN
SCIENCE AND TECHNOLOGY EDUCATION
Andris Broks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 408

Articles
KOREAN SCIENCE TEACHERS’ PERCEPTIONS AND ACTUAL USAGE OF EDUCATIONAL

THEORIES/TEACHING STRATEGIES IN THEIR TEACHING
Jongwon Park, Youngmin Kim, Jongseok Park, Jin-Su Jeong, Young-Shin Park . . . . . . . . . . . . . . 411
LEARNER’S ALTERNATIVE AND MISCONCEPTIONS IN PHYSICS: A PHENOMENOGRAPHIC

STUDY
Abraham Motlhabane . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 424
EFFECTIVENESS OF THE INQF-BASED LEARNING ON A GENERAL PHYSICS FOR IMPROVING

STUDENT’S LEARNING OUTCOMES
Budi Jatmiko, Wahono Widodo, Martini, Mohammad Budiyanto,
Iwan Wicaksono, Paken Pandiangan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 441
A LEARNING EXERCISE USING SIMPLE AND REAL-TIME VISUALIZATION TOOL TO COUNTER

MISCONCEPTIONS ABOUT ORBITALS AND QUANTUM NUMBERS
Sunyono Sunyono, Lisa Tania, Andrian Saputra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 452
PRESERVICE TEACHERS’ PERCEPTIONS OF THE RELATIONSHIP BETWEEN SCIENCE

AND PEACE
Canan Dilek Eren . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 464
INTEGRATING STEM IN AN ENGINEERING DESIGN PROCESS: THE LEARNING EXPERIENCE OF

RURAL SECONDARY SCHOOL STUDENTS IN AN OUTREACH CHALLENGE PROGRAM
Nyet Moi Siew, Henry Goh, Fauziah Sulaiman . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 477
DETERMINING PROSPECTIVE BIOLOGY TEACHERS’ COGNITIVE STRUCTURE IN TERMS OF

“BIOTECHNOLOGY”
Dilek Sultan Acarli . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 494
PEDAGOGICALLY DESIRABLE SCIENCE EDUCATION: VIEWS ON INQUIRY-BASED SCIENCE

EDUCATION IN TURKEY
Yasemin Ozdem-Yilmaz, Bulent Cavas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 506
IMPROVING ACADEMIC ACHIEVEMENT OF SCIENCE LEARNERS IN RURAL SCHOOLS

THROUGH ASSESSMENT PRACTICES: A SOUTH AFRICAN CASE STUDY
Rekai Zenda, Johanna G. Ferreira . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 523
DEVELOPMENT OF AN ONLINE THREE-TIER DIAGNOSTIC TEST TO ASSESS PRE-UNIVERSITY

STUDENTS’ UNDERSTANDING OF CELLULAR RESPIRATION
Rohaida Mohd Saat, Hidayah Mohd Fadzil, Nor Azlina Abd. Aziz, Kamariah Haron,
Kamaludin A. Rashid, Natalya Rudina Shamsuar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 532

Information

INFORMATION FOR CONTRIBUTORS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 547
PROBLEMS OF PSYCHOLOGY IN THE 21ST CENTURY . . . . . . . . . . . . . . . . . . . . . . . . . . . 548
QUALITY ISSUES AND INSIGHTS IN THE 21ST CENTURY . . . . . . . . . . . . . . . . . . . . . . . . . 549
SCIENCE AND TECHNOLOGY EDUCATION: ENGAGING THE NEW GENERATION . . . . . . . . . 550
407
ISSN 1648–3898
SYSTEMS THEORY OF SYSTEMS THINKING: GENERAL AND PARTICULAR WITHIN

MODERN SCIENCE AND TECHNOLOGY EDUCATION
Andris Broks
University of Latvia, Latvia
Following the concept of human as spiritual alive body, human thinking is spiritual activity of our brain –
brainwork. Today we accept that thinking means processing of information what is important for management of
human body life. Studying organization of human brainwork, people have discovered that all our thoughts and
thinking activities are interconnected and form definite hierarchic structures. Humans reflect world as a set or
network of diverse phenomena by building corresponding theories or models of observed phenomena. In other
words, all phenomena in human world of thoughts are reflected as SYSTEMS, which are organizational units of
Systems Thinking. Every system contains their interconnected parts and as a whole is a part of surrounding medium,
made from other systems. Such hierarchic integral understanding and comprehension of human brainwork today
has become the basic value of Systems Theory as theory of Systems Thinking.
Terms “system”, “systems approach” already are widely used in our professional practice and represent many
particular applied systems theories. At the same time, it has become clear that there is a fundamental set of some
general concepts and principles, forming core of all particular systems theories. Today it seems to be purposeful
to separate general and particular within systems theory and talk about compact and exhaustive general systems
theory and particular applied systems theories. To be shorter, we can introduce term “systemology”, what means
applied systems theory within definite branch of our life activities. For example, “Systemology of Education”, “Sys-
temology of Decision Making”, “Systemology of Scientific Research” etc.
Following the core principle of overall interconnection within systems thinking, philosophical terms “general”
and “particular” today we need closer connections with psychology of thinking. It is especially important for the
development of our Educational Systems, balancing general and particular/professional education as well as
scientific, artistic and pragmatic educational programs and corresponding particular subjects or courses. In other
words, when studying hierarchic structures of thoughts as well as thinking activities, it seems that now there is an
actual need for Systems Thinking as a general theory of Systems Thinking above applied system theories, what are
developed for concrete branches of practical systems thinking.
Today it is very important not to lose orientation within our practical life because of global explosion of infor-
mation. Professional training has to develop mastery of practitioners at much higher theoretical level of thinking
than before, because simple professions are becoming computerized and virtual robots are pushing out of employ-
ment corresponding people - bio robots. Today we do not need so many professional solders, clerks etc. like simple
servants, who are trained only to push buttons and ugly enjoy consumer’s life. 21-st century needs constructivist
approach based balanced general and professional education what means realization of optimal interconnections
between particular/applied and general Systems Thinking. Alarming development of unemployment within our
young people generation is partly connected with their pure general education by means of low level of Systems
Thinking. What education for what life – at first we need to help our young people to develop their Systems Think-
ing for orientation in our modern life. Origin of human Systems Thinking is given to all of us by Mother Nature, but
during our life we need further development of this natural background. It is fundamental task of our educational
activities and let us notice that it is not enough to advocate only critical thinking.
408
ISSN 1648–3898
It is worth to note, that in practice we meet different Systems Thinking visualizations. Traditional basic coding
of thoughts by written words and word structures are supplemented with other corresponding signs, many of them
are well known from mathematics, engineering, traffic organization etc. Similar to geographic maps what are used
for schematic description of interconnected objects in space, thought structures can be presented as so-called
mind maps. Such schematic presentation of thought structures is widely used in modern digital communication
technologies. Well-produced schemas with some comments provide rational, short and clear visualization of
interconnections without using long written texts.
Systems Theory as general theory of Systems Thinking is just a great tool for solving many problems of 21-st
century – century of fascinating development of modern Information Technologies. Perpetual changes all around
us and within us, sustainable development of our individual / personal and collective / societal life needs purpose-
ful systemic management, high quality Systems Thinking at least for our life leaders.
Finally, there are just few fundamental verities of Systems Theory.
•• Human is reflecting world phenomena in human’s world of thoughts as systems.

•• System is a totality of systems interconnected parts and as a whole, each system is a part of its sur-
rounding medium, made from other systems.
•• People are interested to study and use systems properties to satisfy their corresponding needs of life.
•• Human cognizes world by parts, comparing these parts and joining them together. Analysis, comparing
and synthesis are three fundamental operations of systems thinking.
•• Cause of everything (of diverse phenomena - bodies, changes of bodies, properties of bodies and its
changes) is interconnection of everything.

409
ISSN 1648–3898
Good luck to all of you who are solving modern problems of sustainable development at global as well as
also at local levels. General and particular, global and local solutions are systemic – they are interconnected and
Systems Theory of Systems Thinking can help to get general understanding and comprehension of these intercon-
nections and avoid many possible failures.
Our JBSE always is ready to accept and disseminate your systemic scientific research findings! Systems think-
ing will help you to reach high quality standards of your work and publications!
Received: July 15, 2016 Accepted: August 25, 2016
Andris Broks PhD., Professor Emeritus, University of Latvia, 19 Raina

Blvd, LV-1586 Riga, Latvia.
E-mail: andris.broks@lu.lv
Website: https://blogi.lu.lv/broks/
410
KOREAN SCIENCE TEACHERS’
PERCEPTIONS AND ACTUAL
USAGE OF EDUCATIONAL
THEORIES/TEACHING
STRATEGIES
IN THEIR TEACHING
Abstract. Many researchers have reported

Jongwon Park,
that there is a significant gap between the-
Youngmin Kim, ory and practice in education. This research
Jongseok Park, sought to contribute to this work by examin-
Jin-Su Jeong, ing the theory/practice gap in secondary
Young-Shin Park school science teaching in South Korea. To
do this, a questionnaire was developed to
investigate the gap between Korean science
teachers’ knowledge about Educational
Theories and Teaching Strategies (ETTS) and
the usage of it in their science classroom.
The questionnaire was administered to 87
science teachers and results showed that
Introduction even though participants were knowledge-
able about many ETTS, only 26% of the
This research investigates whether science teachers apply ETTS (Educa- teachers reported using it in their teaching.
tional Theories and Teaching Strategies) in their science classrooms/laboratory Major reasons reported for this gap in theory
teaching. It has been pointed out that there is a significant gap between and practice were restrictive educational
theory and practice in education (e.g., De Corte, 2000; Roth, 2007). For ex- environments that did not support the use
ample, educational researches reported in the literature are of little aid to of ETTS, irrelevancy and difficulties of ETTS,
science teachers’ actual teaching in the classroom because the researches do and students’ low interest in learning sci-
not provide practical and teacher-friendly guidance (Berry & Milroy, 2002, pp. ence. However, teachers’ perception of the
200–201) or because teachers rarely search, utilize, or translate educational importance of ETTS positively affected their
theories or research results to improve their teaching (Hiebert, Gallimore, & usage of ETTS. Implications of the results are
Stigler, 2002). Korthagen (2007) mentioned that this gap between educa- discussed, and alternative in-service training
tional research and practice has been a perennial problem since the time of program is suggested to activate science
Dewey. Even now, the efforts to reduce this gap are insufficient (Featherstone, teachers’ ETTS what they already know and
2007). Cheng, Cheng, and Tang (2010) also criticized the lack of studies on to guide them to use ETTS in their actual
this gap in Asia. science teaching.
To diminish this gap, many educators have emphasized the significance Key words: theory-practice gap, science
of the more effective pre-service training programs at universities. However, teacher education, secondary science
Zeichner and Tabachnick (1981) argued that the content teachers learn at teacher, teaching strategy.
university is “washed out” by the time they begin teaching at their respective
Jongwon Park
schools. Although this comment is seemingly dated, many may still agree Chonnam National University,
with the argument. Gwangju, Korea
Then, it is worth asking why teachers do not apply the content they Youngmin Kim
Pusan National University, Busan, Korea
have learned at university in classroom teaching. Researchers have proposed Jongseok Park
various reasons for this. For example, some researchers note that university Kyungpook National University,
experiences are not as strong as those of the K-12 school days (e.g., Feiman- Daegu, Korea
Jin-Su Jeong
Nemser & Remillard, 1996; Korthagen, 2007) or that educational theories Daegu University, Daegu, Korea
are too abstract (e.g., Loughran, Berry, & Mulhall, 2006; Vick, 2006), without Young-Shin Park
consideration of the various and complex factors in the reality of classroom Chosun University, Gwangju, Korea
411
KOREAN SCIENCE TEACHERS’ PERCEPTIONS AND ACTUAL USAGE OF EDUCATIONAL THEORIES/
TEACHING STRATEGIES IN THEIR TEACHING
ISSN 1648–3898
(P. 441-423)
teaching (e.g., Hoban, 2005; Korthagen & Kessels, 1999). Others criticize the lack of sufficient and reliable evidence
supporting the effectiveness of educational theory in actual classroom teaching (e.g., Biesta, 2007; Hiebert et al.,
2002).
In addition to these reasons, researchers have pointed out the influence of teachers’ perceptions or beliefs
about teaching and learning on the actual teaching method (Joram & Gabriele, 1998). Korthagen (2004) noted
that teachers can be influenced by “internal factors” such as their beliefs about teaching and learning, professional
identity (e.g., reflection on such questions as “What kind of teacher do I want to be?”), and/or sense of mission (e.g.,
a deeper concern about the role or responsibility of a teacher in our society), as well as “external factors,” such as
behavior or competence based on their knowledge level and practical abilities.
For example, Ezer, Gilat, and Sagee (2010), who examined pre-service teachers’ perceptions about teaching
and learning, recognized no difference between the constructivist view of teaching and the traditional approach.
If this finding is reliable, it is expected that teachers who have a traditional view of teaching will not use construc-
tivist theories in their actual practice.
Among the various beliefs about teaching, this research focuses on teachers’ perceptions of the importance
of ETTS. This is because if teachers think that the ETTS they learned at university are less important or irrelevant
to their actual teaching, then it is likely they will not apply ETTS. In fact, Hobson et al. (2008) observed that some
pre-service teachers considered what they learned at university (or higher education institutes) to be less relevant
to actual classroom teaching or to their “being a teacher” in the future.
Sometimes, the importance of a theory does not necessarily relate to its feasibility or applicability. For instance,
Hong, Chen, Chai, and Chan (2011) observed that many student teachers agreed with the importance of certain
educational theories, such as the theory-building theory. However, when asked about the feasibility of a theory
in actual classroom teaching, the student teachers demonstrated their practice that was relatively discordant to
their perception of the theory’s importance. Even when teachers believe that a theory is important and applicable
to their teaching, some may be reluctant to apply it for various reasons. For example, Cheng et al. (2010) observed
that, after student teachers had completed their four-year BEd programs and gained teaching experience, they
all favored student-centered teaching strategies. However, about 25% of the student teachers (eight interviewees
out of 31) reported that they would actually adopt teacher-centered strategies, for various reasons.
1. With the background described above, the research questions are as follows:
2. To what extent are Korean science teachers knowledgeable about ETTS?
3. Do they believe that ETTS are important?
4. Do they use and apply ETTS in their science classroom teaching?
5. If not, what are the reason(s) for this low usage of ETTS?
What conditions and supports are needed for their more active and effective use of ETTS in actual science
classroom teaching?
Methodology of Research
General Characteristic of Research
This research was conducted using questionnaire and interview. Questions used in the questionnaire and
interview were developed by us to obtain the answers to the research questions. The data was obtained from sci-
ence teachers in South Korea, who were selected randomly and evenly in four areas of background majors, middle
and high schools, different teaching years, and various areas of Korea. The data was analyzed quantitatively and
qualitatively.
Participants
In this research, eighty-seven science teachers were selected randomly from the ordinary middle and high
schools located in widespread areas in Korea. They were recruited to be evenly distributed in four areas of back-
ground majors, two school levels, and teaching years, as shown in Table 1. As an ethical process, the following
statement was provided in the questionnaire: “Participants’ responses will be used only for research, and personal
responses will be confidential.”
412
ISSN 1648–3898 KOREAN SCIENCE TEACHERS’ PERCEPTIONS AND ACTUAL USAGE OF EDUCATIONAL THEORIES/
(P. 441-423)
Table 1. Basic backgrounds of the participants (n = 87).
Major Number School level Number Teaching years Number
Physics 21 Middle school 39 1~5 27

Chemistry 20 High school 41 6~10 34
Biology 25 No response 7 11~15 16
Earth Science 21 Over 16 10
Total 87
In Korea, teaching is one of the most popular occupations, and a large percentage of high-achieving high
school students enter colleges of education to become teachers. After college graduation, these students must pass
the national teacher recruitment examination to become teachers at public schools. Because the examination is
highly competitive (Table 2), most students take their studies very seriously in college. Based on this circumstance
for becoming a science teacher in Korea, it is expected that Korean science teachers have relatively high levels of
knowledge and skills in teaching science.
Table 2. National employment competition ratio for secondary school science teachers.
Competition ratio for each year

Major of
Applicant
2011 2012 2013 2014
Physics 1:11 (96/1013) 1:9 (112/1012) 1:8 (116/943) 1:6(138/786)

Chemistry 1: 14 (109/1513) 1:11 (135/1459) 1:10 (128/1301) 1:8(122/1018)
Biology 1: 16 (120/1878) 1:12 (152/1985) 1:12 (138/1720) 1:9(148/1390)
Earth Science 1: 9 (85/747) 1:7 (114/769) 1:7 (116/759) 1:4(146/611)
Note. (96/1013) indicates that 96 of 1013 applicants were employed as physics teachers.
Instrument
A questionnaire was developed to obtain data related to the research questions (Table 3). The questionnaire
consists of essay-type questions and Likert-scale items. When teachers describe what they know about ETTS in
Question 1, their descriptions do not necessarily guarantee their full understanding. However, as mentioned in
the ‘participants’ section, it was assumed that the participants had a high level of knowledge about ETTS, since
prospective science teachers in Korea needed to pass the competitive national test to be appointed to any public
school. National examination consists of high levels of knowledge about ETTS and demonstration class including
design of lesson plan, therefore, it is assumed that they are very qualified with strong theory and practice in sci-
ence teaching.
Table 3. Contents of the questionnaire.
Content Question Type
What teacher knows 1. What do you know about kinds of educational theories and teaching strategies (ETTS)? E
Describe the ETTS what you know.
Importance 2. Do you think ETTS are important for actual science teaching in schools? Describe the L&E
reasons why you think so.
Practical use 3. Do you use ETTS for actual science teaching in schools? If so, please describe the L&E
instances of use or application. If not, please describe the reasons why you do not use or
apply it.
Conditions & support 4. What conditions or supports are needed for application of ETTS for actual science teach- E
ing in schools?
Note. E = Essay type, L = Likert scale.
413
ISSN 1648–3898
(P. 441-423)
For the Likert-scale items in Table 3, the participants responded to a five-point scale from 5 to 1, where “5” cor-
responded to “strong agreement,”“3” to a neutral answer, and “1” to “strong disagreement.” For the essay-type ques-
tions, the participants submitted written answers. The questionnaire required about 30 minutes to complete.
During the questionnaire development, the research group held several discussions to improve its validity. It
was checked whether the intentions of the questions matched the research goal in content validity and whether
the questions were readable and clear to teachers in face validity. Furthermore, the first draft of the questionnaire
was given to two science teachers as a pilot test. In this pilot test, the science teachers gave written responses and
were then interviewed about their written responses to construct the validity of the questionnaire.
Interviews
To elaborate teachers’ written responses in the questionnaire, 15 science teachers, at least three from each
science area, were voluntarily selected for the interview. Even though they did not participate in the questionnaire
survey, they gave written responses in the questionnaire at first and took interview based on their responses.
The interview lasted about 20 minutes per person on average. Before beginning the interview, the researcher
read the written responses and checked whether there was a gap between what the interviewees knew about
ETTS and their actual use of ETTS in their teaching practice. When the respondents said they did not use some of
ETTS mentioned in Question 1, the following main questions were used to obtain more information about the
reason(s) for the gap and to determine requirements to diminish it:
“Why don’t you use ETTS when teaching science in your class or laboratory?”
“What kinds of supports or conditions do you need to use ETTS in your science teaching?”
Analysis
The responses from Likert-scale items were analyzed quantitatively by using SPSS (20.0.0) statistics and re-
sponses from essay-type questions were analyzed qualitatively by categorizing the responses according to their
common features. The data obtained from the Likert-scale questions were averaged according to various categories
such as the school level, teacher’s teaching years, etc. Next, it was analyzed whether there were any differences in
the results according to the categories. To test whether the teachers’ perception of the importance of ETTS was
an important factor in their actual use of ETTS, the correlation coefficient between the “importance” and “use” of
ETTS was calculated. In addition, to explore whether there was a gap between the theory and practice of ETTS, a
paired t-test between the “importance” and “use” of ETTS was carried out.
All qualitative data from the responses to the essay-type questions were summarized and categorized ac-
cording to their common features. The response frequencies for each category were calculated. Interview excerpts
were also used to provide more detailed information.
Results of Research
What Science Teachers Know about ETTS
In Question 1, participants mentioned a total of 465 items about ETTS; therefore, since 87 participants gave
responses, an average of 5.3 ETTS items was mentioned per respondent. And the participants responded that they
knew various kinds of ETTS, that is, they mentioned 65 kinds of ETTS in total. Among the 65 kinds of ETTS, Table 4
shows the ETTS items most frequently mentioned, i.e., by more than 20% of the respondents. The most frequently
mentioned ETTS items were those related to teaching scientific inquiry (e.g., learning cycle/5E, discovery learning,
inquiry learning, and induction/deduction/abduction).
414
(P. 441-423)
Table 4. ETTS items frequently mentioned by the science teachers (n = 465).
ETTS Percentage (%)
Learning cycle/5E 67
Discovery learning 60
Constructivism 45
Cognitive conflict/Conceptual change/Generative learning 38
Ausubel’s theory 37
Cooperative learning/Jigsaw/STAD 37
STS learning 29
Cognitive Development 24
Inquiry learning 24
Induction/Deduction/Abduction 21
Importance of ETTS as Perceived by Science Teachers
As the science teachers’ responses to Question 2, Table 5 shows that they generally perceived that ETTS are
important for their science teaching (average = 3.98). This perception was nearly the same across all school levels
(t = 1.09, df = 78, p > .05), regardless of the teachers’ length of teaching years (t = 1.07, df = 85, p > .05).
Table 5. Teachers’ perceptions of the importance of ETTS and their use of ETTS (n = 87).
Variable Importance Independent t-value df
School level
Middle school 4.05
1.09 78
High school 3.83
Teaching years
Less than 5 years 3.79
1.07 85
More than 5 years 4.00
Total 3.98
Table 6 provides the reasons ETTS were considered important (Question 2 in Table 2). The data were obtained
from 77 (89%) of the 87 teachers. According to Table 6, the science teachers placed great importance on ETTS
because ETTS benefited the students more than the teachers.
Table 6. Reasons the science teachers think ETTS are important (n = 77).
The ETTS are important because they help … Percentage (%)
Student •• Achieving the learning goals (ex. Understanding concept or the nature of science, enhancing 58
the interest, changing misconception, improving inquiry skills, and so on)
•• Students’ learning (ex. Students’ active participation, cooperative and independent learning, 43
and so on)
Teacher •• Designing a teaching plan (ex. Analyzing teaching materials, preparing teaching schedule.) 22
•• Improving teacher’s profession/belief (ex. Developing new teaching approaches, more
19
systematic teaching, decreasing errors, understanding students, and so on)
Others 1
Note. The percentage is calculated as the number of responses divided by the total number of respondents (77); the total value exceeds
100% because one respondent could give multiple answers.
415
ISSN 1648–3898
(P. 441-423)
However, 15% of the respondents (13 out of 87) described why ETTS were not important, as shown in Table 7.
Most of the respondents (12 teachers, 92%) mentioned that ETTS were irrelevant to their actual science teaching,
and three teachers (23%) said that the teaching skills of the teacher were more important than ETTS itself.
Table 7. Reasons the science teachers think ETTS are not important (n = 13).
The ETTS are not important because of … Percentage (%)
Irrelevance of the theory •• The theory’s feasibility to actual teaching is very low, theories do not consider dif-
ferent levels and abilities of students, more practical guidance for good teaching is 92
necessary, the ETTS are not helpful for college entrance exam, and so on.
Teacher’s practical ability •• Teacher’s own teaching plan is more important, teacher’s skills for applying a theory
23
are more important
Note. The percentage is calculated as the number of responses divided by the total number of respondents (13).
The Use of ETTS in Actual Science Teaching
In regard to the actual use of ETTS (Question 3 in Table 2), the responses were a little higher than 3.0 “neutral”
(average = 3.26) as shown in Table 8. This tendency was nearly the same across all school levels (t = 0.72, df = 79,
p > .05).
Table 8, however, shows that the longer a participant had worked as a teacher, the more frequently he/she
applied ETTS in science teaching (t = 2.05, df = 82, p < .05). This means that, to use the ETTS learned in college,
many and various teaching experiences are necessary because the ETTS learned in college cannot be used in their
original form in schools because of various and unpredictable practical situations.
Table 8. Teachers’ use of ETTS (n = 87).
Variable Use Independent t value df
School level
Middle school 3.30 0.72 79
High school 3.15
Teaching years
Less than 5 years 2.92 2.05* 82
More than 5 years 3.39
Total 3.26
Note p < 0.05
Correlation between Teachers’ Perception of the Importance of ETTS and Practical Use of it
Checking the correlation between the teachers’ perception of the importance and practical use of ETTS, the
correlation coefficient was 0.54 (p < .01). Therefore, it was found that the teachers who thought ETTS were important
for actual science teaching in schools tended to apply ETTS more in their own teaching.
This tendency occurred across all school levels (for middle school teachers, r = 0.64, p < .01; for high school
teachers, r = 0.49, p < .01) and for all lengths of teaching years (for less than 5 years, r = 0.62, p < .01; for more than
5 years, r = 0.50, p < .01). Therefore, it was found that teachers’ belief about the importance of ETTS is a common
factor influencing their use of ETTS in teaching, regardless of the school level and length of teaching years.
However, it was a little weak for the high school teachers (r = 0.49) compared to the middle school teachers (r
= 0.64). This means that the teachers’ perception of the importance of ETTS had less influence on their use of ETTS
for high school teachers compared to middle school teachers. In Korea, high school teachers and their teaching are
primarily focused on preparing students for college entrance examinations; therefore, teaching problem-solving
skills is emphasized more than teaching for other learning objectives such as improving scientific literacy, inquiry
416
(P. 441-423)
skills, or creativity, and so on. As a result, external requirements (e.g., necessity for teaching to improve problem-
solving skills) can obstruct the application of ETTS at the high school level.
Gap between Theory and Practice in Science Teaching
In Question 3, the total number of examples of ETTS applications in their school teaching was 119. Because
the total number of ETTS items that they knew was 465 in Question 1, the percentage of ETTS knowledge they
used (frequency = 119) out of the total ETTS knowledge they held (frequency = 465) was only 25.6%.
This low use of ETTS in actual science teaching can be also seen in Table 9. Comparing these applications to
the amount of knowledge they had about ETTS, it was found that only 24% of what the teachers knew about the
10 ETTS items (in Table 4) were applied in their actual teaching.
Table 9. A comparison between what the teachers know about ETTS and what they apply in their teaching
(n = 60).
A: What teachers know (in Ratio

ETTS B: Application of the ETTS
Table 4) (B/A)x100 (%)
Learning cycle 58 25a 43

Discovery learning 52 7 13
Constructivism 39 5 13
Cognitive conflict/Conceptual change/Gen-
33 11 33
erative learning
Ausubel’s theory 32 5 16
Cooperative learning/Jigsaw/STAD 32 11 34
STS learning 25 4 16
Piaget’s theory 21 0 0
Inquiry learning 21 12 57
Induction/Deduction/Abduction 18 0 0
Average 24
Note. a Twenty-five science teachers mentioned that they had used the learning cycle when teaching science.
Table 10 shows another gap between teachers’ perception of the importance of ETTS and their actual use of
ETTS in their teaching. That is, even though the respondents believed ETTS were important for their science teaching
(average of importance = 3.98), they did not use ETTS in their actual teaching as much as might be expected based
on their perception of the importance of ETTS (average of actual use = 3.26). And this difference was statistically
significant (t = 7.87, df = 85, p < .01). According to Table 10, this tendency is observed at all school levels, regardless
of the length of teaching experience.
Table 10. A gap between teachers’ perception of the importance of ETTS and practical use.
Variable Importance Use Paired t value df
School level
Middle school 4.05 3.30 6.13* 37
High school 3.83 3.15 4.68* 40
Teaching years
Less than 5 years 3.79 2.93 6.89* 26
More than 5 years 4.00 3.39 4.72* 57
Total 3.98 3.26 7.87* 85
Note. p < 0.01
417
ISSN 1648–3898
(P. 441-423)
The Reasons of the Gap
For Question 3 in Table 3, 28% of the teachers (24 out of 87) presented the reasons they did not use ETTS even
though they knew of various kinds of ETTS. To obtain more detailed information about the reasons, 15 randomly
selected science teachers (who did not complete the questionnaire reported in Table 3) were interviewed. The
results are summarized in Table 11.
Table 11. Reasons for the gap between theory and practice (n = 39a).
Reasons Percentage (%)
•• Curriculum (ex. Too much content to be learned.)

•• Job condition (ex. Workloads besides teaching are excessive.)
External environment 82b
•• Classroom/laboratory environment (ex. Laboratories or experimental equipment
is insufficient. There are too many students in a single class.)
•• Irrelevance (ex. ETTS do not reflect actual teaching/learning situations.)
Features of the ETTS •• Difficulty (ex. ETTS are difficult to understand.) 74
•• No confidence (ex. The effectiveness of the ETTS are uncertain.)
•• Students’ activity/level (ex. Students are not familiar to new teaching approaches
Student 18
using the ETTS.)
Teacher •• Teachers’ effort (ex. Teacher efforts are insufficient.) 15
•• Teaching materials (ex. Teaching materials are insufficient.)
Assistance 8
•• Administrative procedure (ex. Administrative procedure is too complicated.)
Others 8
Note. a In this number, 15 interviewees are added. b The percentage is calculated by dividing the number of responses by the number of
respondents (39).
According to Table 11, the reasons for the gap between theory and practice can be classified into five cat-
egories. Among these, the “external environment” (32 science teachers; 82%) was the category mentioned most
frequently. Excerpts from the interviews are as follows:
“When I applied inquiry learning or the cooperative learning model instead of the traditional approach,
students… worried about the slow progress of learning (because there are many contents to be learned, but
inquiry or cooperative learning requires too much time).”
“I have to spend a lot of time handling administrative documents instead of preparing lessons.”
“In my school, there are only two laboratories; moreover, one laboratory is not well-equipped.”
The above excerpts are closely related to the educational situation in South Korea. Regarding the curriculum,
many Korean science teachers criticized the large volume of the required teaching content and, therefore, requested
a reduction of the contents in the national science curriculum.
And regarding the teaching environment, Korean science teachers desired a reduction in their excessive
workload so that they could focus on preparing study plans.
As the second reason for the gap, twenty-five science teachers (64%) mentioned the irrelevancy of ETTS in
their actual teaching as well as the difficulty of applying ETTS. Regarding these points, the following responses
were obtained from the interviews.
“It (in-service course) provides just a general guide for the application (of ETTS) rather than providing
more time and opportunities for developing teaching plans for actual application.”
“The theories (ETTS) themselves are good, but they are far from reality. Applying them is difficult.”
418
(P. 441-423)
Above responses mean that good teaching needs something more than general, ideal, or theoretical models.
That is, teacher needs to consider various aspects of the classroom context, such as teaching situation and facilities,
students’ learning level and interest, and educational culture, as well as ETTS.
Regarding the second reason for the gap, four science teachers (10%) mentioned “no confidence” in Table 11.
The following is an example from an interview: “When I try to develop my teaching plan using ETTS, I am unsure of
whether my plan will be effective for students’ learning.”
As the third reason for the gap, the student aspect was mentioned by 18% of the science teachers. For example,
in the interview, the respondents complained that many students showed low motivation and interest, with only
passive participation in their learning.
Finally, 15% of the respondents mentioned that some teachers showed insufficient effort in using ETTS in
their teaching, and 8% cited insufficient teaching materials and the complex administrative process.
Supports and Conditions Needed for the Use of ETTS in Science Teaching
When asked what supports or conditions science teachers desired for their utilization of ETTS in actual science
teaching, 94% of the respondents (82 out of 87) gave responses. Table 12 shows the responses, including those of
the additional 15 interviewees, and Figure 1 summarizes Table 12.
Table 12. Supports or conditions needed for actual use of ETTS (n = 97a).
Supports/ Percentage
Conditions (%)
•• Teaching materials (ex. We need various teaching materials applying educational

theories, using easy explanations/ interesting contents/everyday activities/multimedia/
IT materials. We need a more concrete model or kit.)
•• Guidance (ex. We need concrete guides developed by teachers’ communities, profes-
Teaching assistance sors, or educators) 76b
•• Sharing system (ex. We need sharing systems using network systems or the internet
to recommend examples. We need to observe good teaching.)
•• Laboratory assistant (ex. We need laboratory assistants)
•• Classroom/laboratory environment (ex. The number of students in a class should be
decreased, and learning time or laboratory time should be increased. We need more
modern laboratories, and so on.
Environ
-mental •• Job condition (ex. Workload or teaching time should be decreased. The number of
58
science teachers should be increased.)
change
•• Curriculum (ex. Science learning should aim for “improvement of creativity or inquiry”
rather than getting high scores on a test. Contents need to be reduced in the cur-
riculum.)
•• Training program (ex. We need more long-term in- or pre-service programs to learn
teaching skills applying theories in actual teaching. Pre-service course should be more
effective.)
Developing •• Community activity (ex. We need professional counseling or coaching. We need to
participate in teachers’ study club or academic association. Interaction with other 47
profession
teachers is necessary.)
•• Effort (ex. Teacher’s effort is necessary.)
•• Role model or leader (ex. A model of a good science teacher is necessary)
Note. a Fifteen interviewees are added in this number. b The percentage is calculated by dividing the number of responses by the
number of respondents (97).
419
ISSN 1648–3898
(P. 441-423)
Figure 1: Graphical summary of the teachers’ responses shown in Table 12.
According to Table 12, supports to diminish the gap between theory and practice can be summarized into
three categories: sufficient teaching assistance, change of educational environment, and teachers’ professional
development. Among these, most of the respondents (76%) wanted various types of effective teaching assistance
for the use of ETTS. The following is an excerpt from an interview.
Teacher: We need various teaching materials that can be used immediately.

Researcher: In fact, I know that many teaching materials are provided to teachers.
Teacher: This (the reason the materials are not usually used by teachers) may be because teachers need to put in the effort
to understand it (the content) first. And also, they (teaching materials) are usually unfamiliar to us; therefore, we
have to revise our usual teaching plan (according to the teaching materials).
Regarding the first support mentioned in the above, a network system can be considered for more direct
support for teachers. For example, the internet site “What Works Clearinghouse” is a potential model (http://ies.
ed.gov/ncee/wwc/). This site introduces research-based teaching programs with information such as an improve-
ment index, the program effectiveness, and extent of the evidence. While sufficient programs are not yet in place,
the intention and format for sharing and distributing developed teaching programs in Korea based on research
can be considered positive.
As the second support, 58% of the respondents stated that the environment, for example, classroom/labora-
tory, job conditions, and/or curriculum, need to be changed for more effective use of ETTS in schools. For example,
in the interviews, teachers said, “We have only two laboratories. Therefore… we need more laboratories to teach more
inquiries or experiments” and “The amount of content (of the science curriculum) needs to be decreased… to allow more
inquiry-based teaching (because inquiry-based teaching takes more time).”
Finally, 47% of the science teachers responded that efforts and opportunities to improve their teaching
profession were needed. For example, in the interviews, some teachers said that, “An in-service program is good for
me… because I can gain (many and various) experiences indirectly from other teachers’ experiences… Therefore, I would
like to have discussions or conversations with other teachers even during break time” or “Teacher community groups
or study groups are necessary also. I can learn about easy methods (from other teachers), and it would not take much
time (to meet with such a group).” Interestingly, many of these teachers mentioned an effective in-service program
(24 teachers) or a community activity (13 teachers) rather than a pre-service program (only two teachers). From
this, it is found that they wanted professional development after becoming science teachers.
Discussion
Many educators have noted there is a gap between theory and practice. In this research, only 26% of Korean
science teachers in the research sample reported they applied ETTS theories into their science teaching practice.
This gap was consistent for teachers at all school levels and with varying years of teaching experience.
To obtain the information about what teachers knew about ETTS and their use of ETTS, responses to Likert-
scale items, written responses to open-ended questions, and some simple interviews were analyzed. However,
420
(P. 441-423)
this research only sought to obtain teachers’ responses without measuring teachers’ actual understanding of ETTS
and observing how to skillfully implement ETTS in their actual science teaching. In order for teacher candidates
to become public school teachers in South Korea, they must pass a very competitive national exam that asks high
levels of questions related to ETTS. For this reason, researchers did not directly measure teachers’ knowledge of
ETTS. However, the results of this research could be improved by measuring teachers’ actual understanding about
ETTS and observing their actual teaching in classroom and laboratory.
As mentioned in the introduction, teachers’ beliefs can impact on their use of ETTS in teaching in classrooms.
This research found that South Korean science teachers’ perceptions about ETTS was a critical factor for determining
how likely they were to apply any particular ETTS in their own science teaching. That is, teachers who believed that
the ETTS was important for actual science teaching in schools tended to use the ETTS more often. This tendency was
a little stronger for middle school teachers than for high school teachers. This may be because high school teachers
are more likely to rely on didactic teaching methods to prepare students for the college entrance examination.
This research also found that teachers with more teaching experience (number of years in teaching) were
more likely to try to implement ETTS in their teaching. However, it is worth noting that the importance of actual
teaching practice does not mean that theory is completely ineffectual in schools. Rather, practice should be based
on theories. That is, teaching design and development should be based on cognitive and educational theories
(Bednar, Cunningham, Duffy, & Perry, 1995, p. 101; Kearns et al., 2010). Hascher et al. (2004, p. 624) noted that
“becoming a good teacher requires not only practice but professional learning environments which also foster
advance theoretical knowledge.”
In this research, it is found that the factors hindering the use of ETTS in science teaching can be classified
into five categories: external environment such as too much contents in science curriculum, excessive workload
for teachers, and poor learning environment; irrelevancy and difficulty of ETTS; students’ low interest in science
learning; teachers’ insufficient efforts; and ineffective and insufficient assistance. Among them, first two factors
can be easily understood because they have been pointed frequently in literatures. Interesting result is that South
Korean students’ low interest for learning science can hinder teachers from implementing innovative teaching
strategies in their classrooms. According to PISA, Korean students are ranked 55th among 58 countries in terms of
their general interest in science (OECD 2007, p. 141). This phenomenon is peculiar because they rank very high in
science achievement tests: Korean students ranked 6th in Science, according to PISA 2009 (OECD, 2010, p. 529), and
4th according to TIMSS 2007 (Martin, Mullis, & Foy, 2008, p. 35). Therefore, it was found that, especially in Korean
situation, special teaching strategies that promote students’ interest in science is especially necessary for more
effective science teaching and learning.
In addition, for more active use of ETTS, sufficient, cumulative, and reliable evidence supporting certain ETTS
are required (Korthagen & Kessels, 1999). Slavin (2008) criticized that “the adoption of instructional programs and
practices has been driven more by ideology, faddism, politics, and marketing than by evidence.” Biesta (2007, p.
1) also noted that “education should be or become an evidence-based practice and that teaching should be or
become an evidence-based profession.”
Conclusions
This research found that even though Korean science teachers had knowledge of ETTS and they agreed ETTS
was important for their science teaching, these teachers’ knowledge and positive beliefs did not necessarily translate
into the application of ETTS in their teaching practices. Regarding this result, this research found various factors
hindering the use of ETTS, and also what conditions or supports are necessary for more effective use of ETTS.
To diminish the gap between theory and practice, the efforts of policy makers, educational administrators as
well as teachers are necessary to reduce or delete the hindering factors and to prepare and encourage the supports
and conditions for more effective use of ETTS. However, important point is that teachers already know many and
various knowledge and skills about ETTS. This means that it is needed to activate the ETTS already known by the
teacher, and guide them on the practical application of ETTS in actual teaching situations.
Regarding this, this research suggests an alternative in-service program, that is, the Practical On-site Coop-
eration Model (POCoM) (Park et al., 2015). In the POCoM, researchers observe science classes directly at first, and
cooperate with science teachers to improve their teaching. This cooperation is conducted just after the observa-
tion of the classes, and try to activate what teachers already know about ETTS without any pre-determined or
pre-developed teaching methods or materials, but with the consideration of the features of the real situation of
421
ISSN 1648–3898
(P. 441-423)
teaching and context-dependent and practical knowledge. Then the science teachers try to apply the improvement
ideas discussed in the cooperative meeting to the just next classes. As a result, this research team observed that
POCoM was effective to facilitate considerable quantitative and quantitative improvement in science teaching.
Therefore, we hope that POCoM will be more widely used for diminishing the gap between theory and practice
in education.
References
Bednar, A. K., Cunningham, D., Duffy, T. M., & Perry, J. D. (1995). Theory into practice: How do we link? In G. Anglin (Ed.), Instructional
technology: Past present and future (2nd ed.) (pp. 100-112). Englewood, CO: Libraries Unlimited.
Berry, A., & Milroy, P. (2002). Changes that matter. In J. Loughran, I. Mitchell & J. Mitchell (Eds.), Learning from teacher research (pp.
196-221). New York, NY: Teachers College Press.
Biesta, G. (2007). Why “What works” won’t work: Evidence-based practice and the democratic deficit in educational research.
Educational Theory, 57 (1), 1-22.
Cheng, M. M., Cheng, A. Y., & Tang, S. Y. (2010). Closing the gap between the theory and practice of teaching: implications for
teacher education programmes in Hong Kong. Journal of Education for Teaching, 36 (1), 91-104.
De Corte, E. (2000). Marrying theory building and the improvement of school practice: A permanent challenge for instructional
psychology. Learning and Instruction, 10, 249-266.
Ezer, H., Gilat, I., & Sagee, R. (2010). Perception of teacher education and professional identity among novice teachers. European
Journal of Teacher Education, 33 (4), 391-404.
Featherstone, J. (2007). Values and the big university education school. In D. Carroll, H. Featherstone, J. Featherstone, S. Feiman-
Nemser, & Roosevelt (Eds.), Transforming teacher education: Reflections from the field (pp. 203-220). Cambridge, England:
Harvard Education Press.
Feiman-Nemser, S., & Remillard, J. (1996). Perspectives on learning to teach. In F.B. Murry (Ed.), The Teacher educator’s handbook:
Building a knowledge base for the preparation of teachers (pp. 63-91). San Francisco, CA: Jossey-Bass Publishers.
Hascher, T., Cocard, Y., & Moser, P. (2004). Forget about theory - practice is all? Student teachers’ learning in practicum. Teacher
and Teaching: Theory and Practice, 10 (6), 623-637.
Hiebert, J., Gallimore, R., & Stigler, J.W. (2002). A knowledge base for the teaching profession: What would it look like and how
can we get one? Educational Researcher, 31 (5), 3-15.
Hoban, G. F. (2005). The missing links in teacher education design: Developing a multi-linked conceptual framework. Dordrecht, The
Netherlands: Springer.
Hobson, A.J., Malderez, A., Tracey, L., Giannakaki, M., Pell, G., & Tomlinson, P.D. (2008). Student teachers’ experiences of initial
teacher preparation in England: Core themes and variation. Research Papers in Education, 23 (4), 407-433.
Hong, H. Y., Chen, F. C., Chai, C. S., & Chan, W. C. (2011). Teacher-education students’ views about knowledge building theory and
practice. Instructional Science, 39 (4), 467-482.
Joram, E., & Gabriele, A.J. (1998). Preservice teachers’ prior belief: Transforming obstacles into opportunities. Teaching and Teacher
Education, 14 (2), 175-191.
Kearns, D. M., Fuchs, D., McMaster, K. L., Fuchs, L. S., Yen, L., Meyers, C., … Smith, T. M. (2010). Factors contributing to teachers’ sus-
tained use of kindergarten peer-assisted learning strategies. Journal of Research on Educational Effectiveness, 3, 315-342.
Korthagen, F. A. J. (2007). The gap between research and practice revisited. Educational Research and Evaluation, 13 (3), 303-
310.
Korthagen, F. A. J., & Kessels, J. P. A. M. (1999). Linking theory and practice: Changing the pedagogy of teacher education. Edu-
cational Researcher, 28 (4), 4-17.
Korthgen, F. A. J. (2004). In search of the essence of a good teacher: Towards a more holistic approach in teacher education.
Teaching and Teacher Education, 20, 77-97.
Loughran, J., Berry, A., & Mulhall, P. (2006). Understanding and developing science teachers’ pedagogical content knowledge. Rot-
terdam, The Netherland: Sense publishers.
Martin, M. O., Mullis, I. V. S., & Foy, P. (with Olson, J. F., Erberber, E., Preuschoff, C., & Galia, J.) (2008). TIMSS 2007 International Science
Report: Findings from IEA’s Trends in International Mathematics and Science Study at the Fourth and Eighth Grades. Chestnut
Hill, MA: TIMSS & PIRLS International Study Center, Boston College.
The Organisation for Economic Co-operation and Development (OECD) (2007). PISA 2006 Science Competencies for Tomorrow’s
World, Volume 1: Analysis. Retrieved September, 2012, from http://www.oecd.org/pisa/pisaproducts/pisa2006/ 39703267.
pdf.
The Organisation for Economic Co-operation and Development (OECD) (2010), PISA 2009 Results: What Students Know and Can
Do – Student Performance in Reading, Mathematics and Science (Volume I). Retrieved September, 2012, from http://www.
oecd.org/pisa/pisaproducts/48852548.pdf.
Park, J., Kim, Y., Park, Y. S., Park, J., & Jeong, J. S. (2015). Development and application of the practical on-site cooperation model
(POCOM) for improving science teaching in secondary schools. Journal of Baltic Science Education, 14 (1), 45-63.
Roth, K. J. (2007). Science teachers as researchers. In S. K. Abell & N. G. Lederman (Eds.), Handbook of Research on Science Educa-
tion (pp. 1205-1259). Mahwah, NJ: Lawrence Erlbaum Associates.
422
(P. 441-423)
Slavin, R. E. (2008). Perspective on evidence-based research in education: What works? Issues in synthesizing educational program
evaluations. Educational Researcher, 27 (1), 5-14.
Vick, M. (2006). “It’s a Difficult Matter”: Historical perspectives on the enduring problem of the practicum in teacher preparation.
Asia-Pacific Journal of Teacher Education, 34 (2), 181-198.
Zeichner, K. M., & Tabachnick, B. R. (1981). Are the effects of university teacher education ‘washed out’ by school experience?
Journal of Teacher Education, 32 (3), 7-11.
Received: March 17, 2016 Accepted: June 15, 2016
Jongwon Park Ph.D., Professor, Department of Physics Education, College of

Education, Chonnam National University, Gwangju, 500-757,
Republic of Korea.
E-mail: jwpark94@jnu.ac.kr
Youngmin Kim Ph.D., Professor, Department of Physics Education, College of
(Corresponding author) Education, Pusan National University, Busan, 609-735, Republic of
Korea.
E-mail: minkiyo@pusan.ac.kr
Jongseok Park Ph.D., Associate Professor, Department of Chemistry Education,

College of Education, Kyungpook National University, Daegu, 702-
701, Republic of Korea.
E-mail: parkbell@knu.ac.kr
Jin-Su Jeong Ph.D., Associate Professor, Department of Biology Education, College
of Education, Daegu University, Gyeongbuk, 712-714, Republic of
Korea.
E-mail: jjs@daegu.ac.kr
Young-Shin Park Ph.D., Assistant Professor, Department of Earth Science Education,
College of Education, Chosun University, Gwangju, 501-759,
Republic of Korea.
E-mail: parkys@chosun.ac.kr
423
Learner’s alternative and
misconceptions in physics:
A Phenomenographic study
Abstract. This phenomenographic case Abraham Motlhabane

study attempted to determine the alter-
native and misconceptions of learners in
selected concepts in Physics. The research
adds another dimension to understanding
alternative conception in kinematics by
qualitatively determining how learners de-
scribe/define a distance of 0m, a displace-
ment of 0m, a speed of 0m/s, a velocity
of 0m/s as well as an acceleration of 0m/
ss. Data were gathered by means of a free Introduction
response test. Senior high school grade
Various studies (Halloun & Hestenes, 1985; Molefe, Lemmer, & Smit,
12 learners were purposefully selected to
2005; Rosenquist & McDermott, 1987; Shaffer & McDermott, 2005) show that
complete the test. Data were analyzed by learners confuse the concepts of time, position, velocity and acceleration
qualitatively interrogating the descriptions and relate them unscientifically. In particular learner’s descriptions are often
and related graphs and pictures to look for qualitative and relative, and consequently differ from the operational defini-
the ways in which learners described these tions of physics. For instance scientific meaning is often lost in a description,
concepts. The research revealed that some for example, the words “rate of” are omitted in the description of accelera-
tion. In an example involving displacement, displacement is described as a
learners were not able to comprehend the
short distance. Surely, by a Newtonian account, forces do not cause motion
meaning of a displacement of 0m, thus (velocity); they cause change in motion (acceleration). If there is no net force
they experienced challenges in the under- on an object, it moves at a constant speed in a constant direction; if there is a
standing of the concepts such as a speed net force, the object’s speed, direction of motion, or both changes. Students
of 0m/s, a velocity 0m/s and an accelera- often have difficulty understanding this account (Molefe et al., 2005), due,
tion 0m/ss. The data seems to suggest that from the misconceptions perspective, to their misconceptions about forces
and motion.
learners fail to formalise and contextualise
McCloskey (1983) described students as having an intuitive impetus
“0” as a concept in kinematics. theory similar to the impetus theories articulated by medieval physicists.
Key words: alternative and misconcep- Students see the motion of an object as caused by an internally stored
tions, contextualising “0” concept, kinemat- impetus, which they typically call force or energy. As the impetus runs out,
ics concepts, phenomenographic study. the object stops moving. McCloskey identified two variations of the impetus
theory, one in which the impetus runs out on its own and another in which
the impetus is drained by gravity, friction, or both. Other researchers have
identified similar misconceptions without attributing to them the coher-
Abraham Motlhabane
University of South Africa, ence of a theoretical framework. Clement (1983) described students’ use of
South Africa a misconception that “motion implies a force” in a range of situations. This
is a misconception that the motion of an object indicates the presence of a
continuing force causing that motion.
424
ISSN 1648–3898 Learner’s alternative and misconceptions in physics: A Phenomenographic study
(P. 424-440)
Confusion of kinematics concepts is revealed when learners apply features of one concept to another. Ex-
amples given by DiSessa (1993) are as follows:
•• Being ahead implies having gone faster (independent of relative starting point).
•• Less distance covered means less time.
•• Going faster means going for more time.
The first two examples in the above list can be ascribed to everyday observations, for example, all objects
moving on or near the earth come to rest and the effects of irregularities are more obvious for slow-moving objects
(Lemmer, 2013). The last example in DiSessa’s list given above, called changes take time can be attributed to the
medieval impetus theory that perceives impetus as the causal agent ‘injected’ into a moving object and then fading
or draining away (McCloskey, 1983). Many learners who believe that objects acquire some kind of impetus when
kicked, hit or thrown that either starts wearing out after it has left the source or is maintained for some time before
wearing off (Halloun & Hestenes, 1985).
Learners may indiscriminately use different terms for this impetus idea, namely power, force, acceleration,
velocity, momentum, inertia or energy (Halloun & Hestenes, 1985). In terms of force, the so-called force-as mover
misconception is found to occur all over the world and is particularly resistant to change with tuition (Thijs & Van den
Berg, 1995; Trowbridge & McDermott, 1981). A consequence of this conception is that learners always ascribe a force
in the direction of motion, whether the moving object has a positive, negative or zero acceleration (Lemmer, 2013).
Research (Lemmer, 2013) indicated that a moving object that undergoes a constant acceleration is gener-
ally perceived as if the velocity were constant. Similarly, at slow speeds where the ratio of acceleration to velocity
becomes high, changes in velocity are detectable, but not at higher speeds. According to Calderone and Kaiser
(1989), the physics definition of acceleration as change in velocity per time unit is inappropriate in psychophysical
measurements. The reason is that (Lemmer, 2013) observers’ sensitivity to velocity changes probably depends on
the duration of the acceleration and the initial velocity of the object. Therefore, researchers in the field find it more
useful to characterize acceleration either as a ratio of final to initial velocity or as a ratio of change in velocity to the
average velocity. Limitations in visual perception of changes in velocity may contribute to the reported confusion
of velocity and acceleration (Rosenquist & McDermott, 1987).
In accordance with research observations, learners may differentiate only between moving and stopping objects.
The term ‘stopping’ may refer to the part of the motion where an object moves with a speed that is small enough
to observe the slowing down of the object. Due to learners’ inability to detect a change in speed of a faster moving
object, they may not understand the differentiation made in physics between constant velocity and accelerated
motion (Lemmer, 2013). Therefore, velocity–acceleration confusion may lead to a misunderstanding of the scientific
association of the concept of force with acceleration as well as the distinction in Newton’s laws between constant
velocity and constant accelerated motion (Lemmer, 2013).
Even if concepts like acceleration are ostensibly well defined, the procedural knowledge needed to interpret
such definitions is rarely taught. For example, most physics textbooks define acceleration as the “rate of change of
velocity” and state this definition as a=∆v/∆t, but almost none spells out explicitly the complex procedure needed
to interpret this definition (Basson, 2002).
In studies of learner understanding of the concepts of velocity and acceleration in one dimension, learners
were asked specific questions about simple motions they observed (Trowbridge & McDermott, 1981). In the case
of velocity, virtually every failure to compare velocities for two simultaneous motions could be attributed to use of
a position criterion to determine relative velocity. Although learners could generally give an acceptable definition
for velocity, they did not understand the concept well enough to be able to determine a procedure they could use
in a real physical situation for deciding if and when two objects have the same speed. Instead they fell back on
the perceptually obvious phenomenon of passing. Some identified being ahead or being behind as being faster
or slower. The research also provided evidence that for some learners certain preconceptions may be remarkably
persistent, even after intervention by course facilitators (Basson, 2002).
The main thrust for the study on acceleration was for the qualitative understanding of the ratio ∆v=∆t. Unlike
in the case of velocity where one difficulty, the confusion with position, occurred in most cases, the situation with
acceleration has been reported as complex. Some of the difficulties learners displayed are: using a non-kinematical
approach, confusion between position and acceleration, confusion between velocity and acceleration, discrimina-
tion between velocity and change in velocity but neglect of corresponding time interval (Basson, 2002).
Research suggests that a distinction can be made between distance and displacement, before speed and velocity
are introduced. At the same time a learner should have a clear grasp of the concept of rate, to be finally able to move
425
Learner’s alternative and misconceptions in physics: A Phenomenographic study
(P. 424-440)
ISSN 1648–3898
on to acceleration. The focus here should be to learn how to describe motion and not to learn about vectors. Hence
during the study of motion the terms vector and scalar can be introduced with a classification objective in mind,
namely to distinguish between those quantities with or without a direction associated with them (Basson, 2002).
Lemmer (1999) found that the effect of language and culture were more significant than gender or environment
in alternative conceptions concerning space and time as well as related concepts such as the kinematic concepts.
The alternative perceptions of the kinematic concepts held by the learners involved in the study consequently ex-
plained the lower gains. The proper understanding of the kinematics concepts (e.g. distance, displacement, speed
and acceleration) is a prerequisite for the understanding of kinematic graphs (Molefe et al., 2005).
The ability to comfortably work with graphs is a basic skill of the scientist. Line graph construction and inter-
pretation are very important because they are an integral part of experimentation, the heart of science (McKenzie
& Padilla, 1986). A graph depicting a physical event allows a glimpse of trends which cannot easily be recognized
in a table of the same data. Mokros and Tinker (1987) noted that graphs allow scientists to use their powerful visual
pattern recognition facilities to see trends and spot subtle differences in shape. The ability to use graphs may be an
important step toward expertise in problem solving since “the central difference between expert and novice solv-
ers in a scientific domain is that novice solvers have much less ability to construct or use scientific representations.”
(Larkin, 1981). Perhaps the most compelling reason for studying students’ ability to interpret kinematics graphs is
their widespread use as a teaching tool. Since graphs are such efficient packages of data, they are used almost as a
language by physics teachers (Beichner, 1994).
However, research has uncovered a consistent set of student difficulties with graphs of position, velocity, and
acceleration versus time. These include graph as picture errors, slope/height confusion, problems finding the slopes
of lines not passing through the origin, and the inability to interpret the meaning of the area under various graph
curves (Beichner, 1994; Basson, 2002; Christensen & Thompson, 2012; Planinic, Ivanjek, & Susac, 2013). The types of
problems physics students have in this area have been carefully examined and categorized by McDermott, Rosen-
quist, and van Zee (1987) as well as Planinic, Ivanjek, and Susac, (2013). Several of these studies have demonstrated
that students entering introductory physics classes understand the basic construction of graphs, but have difficulty
applying those skills to the tasks they encounter in the physics laboratory.
Kinematics graphs have position, velocity, or acceleration as the ordinate and time as the abscissa. The most
common errors students make when working with these kinds of graphs are: (1) thinking that the graph is a literal
picture of the situation and (2) confusing the meaning of the slope of a line and the height of a point on the line
(Basson, 2002).
In general, students tend to find slopes more difficult than individual data points. They also have a hard time
separating the meanings of position, velocity, and acceleration versus time graphs (Halloun & Hestenes, 1985). Re-
gardless of the type of errors students make, it is generally agreed that an important component of understanding
the connection between reality and the relevant graphs is the ability to translate back and forth in both directions
(McDermott, et. al., 1987; Basson, 2002).
Although a number of studies have been conducted on learners’ understanding of the concepts acceleration,
velocity, speed, distance as well as displacement, the literature reviewed does not seem to have reported on studies
related to learners’ definition of an acceleration of 0m/ss, a velocity of 0m/s, a speed of 0m/s, a distance of 0m and a
displacement of 0m. Hence it is envisaged that the investigation of learners’ conception of these concepts can reveal
important insights into the learners’ ways of thinking and understanding in science (Duit, Treagust & Mansfield, 1996),
therefore assisting researchers and teachers to revise and develop their own scientific knowledge.
Consequently the research question for this research is as follows: What are the alternative conceptions and
misconceptions that learners have on the description/definitions of the following, a distance of 0m, a displacement
of 0m, a speed of 0m/s, a velocity of 0m/s and an acceleration of 0m/ss.
Phenomenography
Ontologically this phenomenographic case study assumed a theoretical point of view aimed at describing
the different ways a group of learners understand a phenomenon (Larsson & Holmstrom, 2007). Thus, this research
focused on the experiential, that is, the different qualitative ways in which content-oriented and interpretative de-
scriptions of their reality is understood and perceived. The research was directed at the variation in learner’s ways
426
(P. 424-440)
of understanding the phenomenon. This is referred to as a second-order perspective. A second order perspective
was used because it was assumed that learners may have different experiences about phenomena accumulated
throughout their high school career and their daily experiences. In addition, their meaning attached to the phe-
nomena may emanate from their experiential association with language, values, beliefs and the environment they
live. For this reason, the focus on the apprehended (experienced and conceptualised) content as point of departure
for carrying out research and as a basis for integrating the findings is seen as the most distinctive feature. Therefore,
the conceptions and ways of understanding were not seen as individual qualities but conceptions of reality were
considered rather as categories of descriptions to be used (Marton, 1981).
Sample Selection
Phenomenographical research requires selection of participants who have significant experience of the phe-
nomenon, and criterion sampling of participants who fulfil certain criteria is the most suitable methodology (Cilesiz,
2011; Creswell, 2007) for participant selection. The sample sizes of participants for a phenomenographic study are
small, around seven participants, with a view to allowing the researcher to become deeply involved in the data and,
therefore, the phenomenon (Connelly, 2010). In addition, Mouton and Babbie (2001) emphasise that in a qualitative
research of this nature 5 to 20 respondents are seen as a sufficient number of participants. This is because qualita-
tive research is aimed at investigating small and distinct groups normally regarded as a single-site study. Qualitative
research was used to describe and analyse phenomena from the learners’ perspectives (Glesne, 1999).
Research was conducted in one of the high schools in the North West Province of South Africa. From the
sample of 26 grade 12 learners fourteen (14) learners were purposefully selected to participate in this research.
Learners’ performance in Physical Science was used to select the participants. The academic year consists of four
terms and learners write class tests to evaluate their progress. The research took place in the last term of their
academic year. Only learners with an average of 70% in Physical Science and above from all the tests written in
the four terms were chosen to participate. Their ages ranged between 15 and 18 years. In terms of gender, eight
were females and six were males. The language of teaching and learning is English, however their mother tongue
is Setswana. Grade 12 is the highest grade in high school in South Africa. Therefore, the grade 12 learners were
identified as information rich and likely to be knowledgeable (McMillan & Schumacher, 1997) about the concepts
of displacement, distance, velocity, speed and acceleration.
Instrument and Procedures
A free response test originally consisting of eight items was developed to investigate learners’ conceptions
qualitatively. The test was piloted in a different high school on a group of ten grade 12 learners and their teachers.
This was done in an attempt to establish content validity. Based on the results of the pilot study, and the com-
ments from teachers, the test was modified resulting in the removal of three items. The final paper-and pencil test
consisted of five items. The test was administered to a sample of fourteen grade 12 Physics high school learners.
The test was administered in a regular class period and lasted 45 minutes. The free response test was preferred
because truthfulness of responses to questions was assured due to the fact that the learners were asked to be
anonymous. The test allowed for in-depth descriptions and drawing of graphs/pictures. In addition learners were
given sufficient time to think properly through their responses. Ethical clearance was obtained from the University
Research Ethics Committee. Prior to administering of the test, participants read and signed informed consent forms.
Participants were given pseudonyms to protect their privacy. In addition, the learners were made aware that they
have the right to withdraw from the research at any time without being penalised. Smaller number of items was
chosen to enable learners to answer all the questions in detail. The concepts indicated in the free response test
were questions requiring learners to give a description/definition of a distance of 0m, a displacement of 0m, a
speed of 0m/s, a velocity of 0m/s as well as an acceleration of 0m/ss. Learners were also required to support their
answers with a diagram, graph, or picture.
Data Analysis
The expected sets of data were 70 definitions/descriptions and 70 graphs/pictures (14 learners multiplied by
5 test questions). The data analysed were 67 definitions and 69 graphs/pictures. A small number of learners did
427
(P. 424-440)
ISSN 1648–3898
not attempt some definitions and one definition/description was not supported by a graph/picture. A phenom-
enographic case study was used to qualitatively study the different ways in which grade 12 learners think of the
concepts in the test. In other words, the aim was to discover qualitatively different ways in which learners experi-
ence, conceptualize, realize as well as understand various aspects of the phenomena (Kinnunen & Simon, 2012). In
particular the research identified the multiple conceptions that learners have for a particular phenomenon. Hence,
the aim was not to find the single view, but the variation and the architecture of this variation by different aspects
that define the phenomena (Thompson, 2010). The research was concerned with the ways in which physics learners
experience or understand selected concepts and principles of physics (Kinnunen & Simon, 2012). The main issue of
credibility in a phenomenographic research is the relationship between the data obtained from the categories for
describing the ways in which learners experience a certain phenomenon. To combat this, the learners were asked
to define/describe the concept in words and represent their descriptions with a graph or picture. Therefore in the
analysis of their descriptions, the data were triangulated between the concepts themselves, the graphs and pictures
drawn by learners. The process of analysis, referred to here as ‘reduction’ Smith, Flowers, & Larkin et al., (2009) was
used to reduce the data into themes. Analysing the data yielded themes that described the learner’s definitions
and conceptions. Two themes were derived, alternative conceptions and misconceptions. Furthermore, two more
themes were identified, namely stationery and no movement. Learners’ definitions that belonged to these themes
were highlighted in the results. During the first iteration of analysis both similarities and the differences among
definitions were searched. Significant statements were selected and compared in order to find cases of variation
or agreement and thus grouping them accordingly. Getting a sense of the data could be described as moving into
the content of the definitions with a sense of self-trust in one’s analytical process (Storey in Lyons & Coyle 2007).
Following this, each text was broken down into more manageable units of meaning. This was achieved by remov-
ing any other tracts of the text that were deemed general statements. What remained was the text that described
the participants’ responses to the key points of each question in the test.
Coding as a process of organizing and sorting data was used. The codes served as a way to label, compile and
organize the data. In coding the data, phrases were assigned to the descriptions by the learners. Initial coding and
marginal remarks were done on hard copies of learners’ responses. The marginal codes were helpful when thinking
about how codes fit together. To ensure reliability of data, the coding practice/training was used to enhance the
consistent interpretation of data and reduce individual interpretive bias (Creswell, 2007). Hence before coding the
descriptions, three researchers were requested to practice coding independently until 90% or greater reliability of
coding was achieved. Differences in coding were constantly compared, discussed, and resolved to meet this level
of consistency. At that point, a coding book was developed for use during the remaining data analysis. Additional
coding rules were defined to establish consistency in segmenting the descriptions for coding.
The codes derived were as follows: The codes from the learner’s descriptions of a distance of 0m were no
distance covered, stationery and the absence of movement while the codes from learner’s descriptions of a displace-
ment of 0m were no distance covered, the shortest distance, distance covered between A and B, same distance as well as
stationery. On the other hand, the codes derived from learner’s descriptions of a speed of 0m/s were no movement
and stationery while the codes stationery, no movement and constant displacement were from learner’s descriptions
of a velocity of 0m/s. This means that the codes stationery and no movement appeared in both the speed of 0m/s
and a velocity of 0m/s. For an acceleration of 0m/ss, the codes derived were constant velocity, moving constantly,
no acceleration, stationery as well as no force. The code that was common to all descriptions was stationery.
Once the data had been coded, regularities, variations and peculiarities were examined and patterns identi-
fied. The process of identifying substantive connections by associating codes or linking data was done (Dey, 2003).
Correlations or relations between different codes were studied and a picture of the data was built.
Results of the Research
The results are indicated in Tables 1, 2, 3, 4 and 5. For ethical reasons, respondents’ names are not mentioned;
they are labelled as A,B,C,D,E,F,G,H,I,J,K,L,M and N in the first column of the tables. In the next column a description/
definition of a concept is given. The description included a highlighted keyword in each respondent’s description.
Selective coding (Babbie, 2009) was used to identify the central code of this research. The responses of learners,
given in Tables 1, 2, 3, 4, and 5, were coded for a description of a distance of 0m, a displacement of 0m, a speed of
0m/s, a velocity of 0m/s and an acceleration of 0m/ss, respectively.
428
(P. 424-440)
In the last two columns, the learner’s descriptions are further coded into two themes (alternative conception
and misconception). For the purpose of this research, the meanings that are generally accepted by the scientific
community of practice are regarded as alternative conceptions and those that are not accepted by the scientific
community of practice are seen as misconceptions. An (X) is used to mark the appropriate column to indicate
where the meaning belongs. The responses of learners are recorded verbatim. Since the quality of the graphs/
pictures were very poor, examples of learners’ graphs/pictures were redrawn.
Table 1. Learners’ descriptions of a distance of 0m.
Descriptions of a distance of 0m Alternative conception Misconception
A The zero distance means there is no distance covered X
B When an object is stationary or does not move over a period of time. (the total
X
distance from point A to B)
When an object was stationary this means no distance has been travelled
C X
from point A to B
D The is no movement taken X
E Being stationary or not moving at all X
F This object is not moving, its distance is 0, therefore it is stationary X
G There is no distance covered X
H There is no distance or if the distance is 0 that means is initial velocity X
Is when the body or object has not undergone motion Basically when the
I X
body is at rest
There was no motion or movement done for a thing to move to either side of
J X
any direction.
K It is a distance that is not taken X
Nothing has been travelled, there is no movement of something, it is just

L X
stationary
M No forward or backward movement has been taken X
N The is no distance being taken the object is constant X
Distance is defined as the magnitude of the path length. This means that if the trolley travelled X m from
point A to point B, then the distance will be X m. Distance is a scalar quantity; therefore we do not consider the
direction.
If the trolley travelled 0 m, the distance travelled will be 0 m. It is thus scientifically acceptable to say that the
trolley remained stationary. This is indicated by the majority of learners (Table 1). The respondents in this research
associated a distance of 0m with the object being stationary. The respondents used a variety of words to explain
the scenario of a stationary object. The words included the phrases ‘no distance’, ‘no movement’, ‘no motion’ and
‘stationary’. Though none of them included a phrase ‘zero path length’ their responses have been interpreted to
mean that the object did not cover any distance, hence stationary. An example of the graphical representations
of a distance of 0m drawn by learners is indicated below.
429
(P. 424-440)
ISSN 1648–3898
Figure 1: An example of a learner picture representing a distance of 0m.
The picture representation (Figure1) depicts a stationery car and a person standing still at the robot, waiting
for the robot to open.
Table 2. Learners’ description of a displacement of 0m.
Descriptions of a displacement of 0m Alternative conception Misconception
A No distance covered X
The shortest distance between two points. When an object remains stationary or
B X
moves between two points and returns to the original point
When a specific distance is being covered from point A to B, then back from point
C X
B to A
Is the same distance from and to .It is a direction where a person goes to a particu-
D X
lar point and comes back where they were
Moving a distance from the starting point and cancelling it out by coming back to
E X
the starting point. Moving from one point and come back to the same point
F - - -
G Moving from the same area back and front, covering no displacement X
H When someone move but with 0m X
I Is when you walk a certain/particular distance and return to your original position X
Displacement is the shortest time taken to go to another point, but zero displace-
J X
ment explains that there was no displacement taken
K Without shortcuts X
L It is just stationary there is no movement X
M No distance has been covered in a certain direction X
N When something is not moving is zero X
To describe the motion of an object, we must be able to specify the location of the object at all times. Figure
2 shows one way of accomplishing this for one-dimensional motion, such as a trolley travelling along a straight
road. Suppose that the initial position of the trolley is indicated by the vector labelled X0.
430
(P. 424-440)
Figure 2: Trolley travelling along a straight line.
As the drawing shows, the length of X0 is the distance of the car from an arbitrarily chosen origin. At the later
time the car has moved from its initial position X0 to a new position that is indicated by the vector X. The displace-
ment is a vector quantity conveying both a magnitude and direction. This can be illustrated by:
ΔX + X0 = X or ΔX = X - X0
Thus, the displacement is the difference between X and X0, and the symbol (Δ) for the Greek capital letter
delta is used to denote this difference. The displacement of the object is a vector, whose magnitude is the shortest
distance between the initial and final positions of the motion and whose direction points from the initial to the
final position (Cutnell & Johnson, 1995).
From Figure 2, if X0 is equal to X, meaning that if the initial position is equal to the final position, the difference
between the initial and final position is then zero (0m). This can be illustrated in Figure 3.
Figure 3: Stationary trolley.
This is a situation where the trolley is stationary. Alternatively, the trolley can move from point A towards
point B, and then reverse to its original position, point A, as illustrated in Figure 3.
Figure 4: A trolley travelling from Point A to B and back to Point A.
In both Figure 3 and Figure 4, the difference between the initial position and the final position is zero (0m).
Whilst the a displacement of 0m was obtained in Figure 3, it does not necessarily mean that a displacement of
431
(P. 424-440)
ISSN 1648–3898
0m can only be obtained when the object is stationary. As indicated by Figure 4, when the object returns to the
original position a displacement of 0m can be obtained. Some learners perceive a displacement of 0m to be zero
motion (no movement).
The graphical representation of a displacement of 0m is represented in Figure 5.
Figure 5: Graphical representation of zero displacement (0m).
A displacement of 0m is represented by the line drawn over the time axis (Figure 5).
However, according to one of the learners, the figure below (figure 6) represents a graph of a displacement
of 0m, which is a misconception. In addition, the vertical and the horizontal axis are not been labelled.
Figure 6 An example of a learner graph representing a displacement of 0m.
Table 3. Learners’ descriptions of a speed of 0m/s.
Descriptions of a speed of 0m/s Alternative conception Misconception
A The is no movement the object is just stationary X
When an object does not increase or decrease its speed. It remains

B X
stationary
C An object is stationary but tend to travel with zero speed X
D It is the same as a stationary object there is no movement X
Covering no distance per time or in a certain time, not doing anything

E X
at all
When an object has 0 speed that means this object does not move it is
F X
stationary
G The is no speed taken not moving anywhere X
When something from the starting point but moving zero speed or not
H X
going anywhere
Is when the object has not undergone motion/distance over a particular
I X
time. When the object is at rest
432
(P. 424-440)
Descriptions of a speed of 0m/s Alternative conception Misconception
Zero speed explains the distance and the time taken for one to complete
J or travel on a certain journey. This means that at zero speed it is not X
moving
K A speed that is not increasing X
L There is no distance, there is no movement X
M The rate at which a body moves with is 0 X
N A speed without moving X
One of the features of an object in motion is how fast it is moving. If a car travels 300 m in 20 s, its average
speed is 30 m/s. The average speed is the distance travelled divided by the time required to cover the distance:
Zero speed
If a car travels a distance of 0 m in 20 s, its average speed will be 0 m/s. Basically the car is stationary. Therefore
we have zero speed.
The graphs representing zero speed are indicated below:
Figure 7: Graphs representing a speed of 0m/s.
Since there is a relationship between distance and speed, the majority of learners indicated (Table1) that a
distance of 0m and a speed of 0m/s (Table 2) mean that an object is stationery. Few responses were misconcep-
tions, for example, respondent K, defined a speed of 0m/s as “a speed not increasing” while respondent M said that
a speed of 0m/s is when “the rate at which a body moves with is 0”. Respondents N defined a speed of 0m/s as “a
speed without moving”. These responses indicate lack of understanding of the concept of speed equal to 0m/s.
It is not clear what the learners meant by the statements made.
433
(P. 424-440)
ISSN 1648–3898
Figure 8: An example of a learner graph representing a speed of 0m/s.
Figure 8 above shows an example of a graph representing a speed of (0m/s). It shows clearly that the learner
was unable to represent a speed of 0m/s graphically.
Table 4. Learners’ descriptions of velocity of 0m/s.
Alternative
Descriptions of a velocity of 0m/s Misconception
conception
A No work is done X
B when an object remains stationary over a period of time X
C An object travels a certain distance with no velocity at a particular time. Meaning it X

is stationary
D There is no movement the object has stopped X
E Not covering any displacement in a certain time, which simply means not moving X
F A stationary object usually have zero velocity because it doesn’t move X
G Being unable to move at the required velocity just standing X
H When something moving with initial velocity X
I When an object undergoes constant displacement, or gradual increase in displace- X

ment
J Zero velocity is the time and the distance travelled for the object to reach certain X
point up or down left or right and then to return again
K Without a car accelerating X
L No movement, no work, it’s just stationary X
M The amount at which a body is travelling with is zero, that means it is standing still X
N - - -
The average speed of the object does not reveal anything about the direction. If the object’s initial position
is X0 when the time is t(0). A little later the object arrives at its final position X at time t. Dividing the displacement
of the object by the elapsed time gives the average velocity.
434
(P. 424-440)
If displacement of an object is (0m), then the average velocity becomes 0m/s. As indicated earlier a displace-
ment of 0m does not necessarily mean that the object is stationary. If the displacement is constant this means
that the initial position is equal to the final position, and then the change in displacement will be 0m. The result
is an average velocity of 0m/s.
The graph representing a velocity of 0m/s is indicated below:
Figure 9: Graphs representing a velocity of 0m/s.
The learners in this research related a velocity of 0m/s to a stationary object and they used the phrase no
movement to describe a velocity 0m/s. Though it is an acceptable description, none of the respondents mentioned
that the object should travel with a constant displacement for the velocity to be 0m/s. Five responses were mis-
conceptions and one did not attempt to describe a velocity of 0m/s.
One of the misconceptions is indicated graphically in Figure 10 below.
Figure 10: An example of a learner graph representing a velocity of 0m/s.
In the learner’s mind the graph above represent a velocity of (0m/s).
Table 5. Learners’ understanding of an acceleration 0m/ss.
Alternative
Description of an acceleration of 0m/ss Misconception
conception
A When an object move at a constant velocity X
B No increase in speed just moves constantly X
435
(P. 424-440)
ISSN 1648–3898
Alternative
Description of an acceleration of 0m/ss Misconception
conception
C* An object travels a certain distance with no acceleration, it does not increase its speed X
but at that point there is no speed
D There is no increase in the velocity of the car it means it is constant velocity X
E No motion, or simply something that is not doing anything or moving X
F - - -
G increasing X
H Something move from top to bottom, sliding to the ground X
I Is when the velocity of an object is constant, or the object experiences no velocity at a X

given time
J Zero acceleration explains that object did not up or move up X
K If an object is not moving X
L No force exerted X
M There is no increase the speed at which a body is travelling X
N No force being made X
Whenever the velocity of an object is changing, then the object is accelerating (Cutnell & Johnson, 1995). For
example, a car temporarily stops at a traffic signal, and accelerates when the light turns green. Examples involve
an increase or a decrease in velocity.
The graphs representing an acceleration of 0m/ss are indicated below:
Figure 11: Graph representing an acceleration of 0m/ss.
For an acceleration to be 0m/ss, the object should travel at a constant velocity. Therefore, the final velocity
(V) will be equal to the initial velocity V0. This means that the difference between the final and initial velocity will
be 0m/s and the acceleration will be zero (0m/ss). In addition, if the object is stationary then its acceleration is also
zero (0m/ss). Out of the 14 respondents in this research, 6 respondents (43%) could not describe zero accelera-
tion (0m/ss). Only respondents D and E defined both zero velocity (0m/s) and zero acceleration (0m/s) correctly.
The rest of the respondents defined zero velocity (0m/s) but could not define zero acceleration (0m/ss) and vice
436
(P. 424-440)
versa. Respondents related a velocity of 0m/s to stationary objects and they did not mention that the object has
a velocity of 0m/s when it travels at constant displacement. Learners use the concept zero speed (0m/s) and zero
velocity (0m/s) interchangeably hence they could not explain zero acceleration (0m/ss).
Figure 12: Example of learner graph representing an accelation of 0m/ss.
An example of learner’s view of zero acceleration (0m/ss) is represented graphically above (Figure 12).
The research revealed that many of learner’s descriptions included the phrase ‘nothing’ “no” associating the
‘zero (0)’ to ‘nothing’. This resulted in defining some of the concepts non-scientifically, hence the misconceptions
were displayed. One important reason contributing to the misconceptions in definitions/descriptions of concepts
was their association of zero (0) to everyday mother tongue talk. Since the concept “zero (0)” shares common prop-
erties with the concept “nothing” in their mother tongue, for these learners these properties were necessary and
sufficient to define the concepts given in the test. The research also confirms that students hold misconceptions
in kinematics. Furthermore, the research reveals that students fail to formalise and contextualise 0 as a concept
in kinematics.
Discussion
Kinematics seems to pose a number of challenges to learners. Consistent with (Roschelle, 1998) there is ample
evidence that the students’ understanding of the concepts such as velocity and acceleration is not complete. Learners
experienced challenges in correctly defining the kinematics concepts. Generally, the majority of the respondents
used the description of a distance equal to 0m to define a displacement of 0m. This finding is consistent with Lem-
mer (2013) that learners confuse distance and displacement. Similarly some learners think that displacement is
the same as distance, the only difference is that displacement has a small value, that is, shorter distance (Lemmer,
2013). Hence, now that they were required to define a displacement of 0m and a distance of 0m, in their minds a
displacement of 0m should be defined in the same way as the distance of 0m.
The results show that 64 % of the respondents could not give an acceptable definition of a displacement of
(0m). Learners confused distance and displacements resulting in a further confusion between the meaning of a
distance of 0m and a displacement of 0m. For example, respondent (A) used the same explanation to describe both
a distance of 0m (Table 1) and a displacement of 0m (Table 2). Generally many of the respondents attempted to
give meaning of the concept of displacement but could not contextualise a displacement of 0m.
Responding to the meaning of a velocity of 0m/s (Table 4), respondent (A) for example indicates that a velocity
of 0m/s means “no work done”. This can be explained by the fact that learners see motion of an object as caused by
the internally stored impetus (McCloskey, 1983), for example force or energy. Since work done is actually the energy
transferred when a force moves an object over a distance. Learners used the same impetus theory that because
there is no work done (no energy transferred) then the object cannot move, meaning a velocity of 0m/s.
The results shows that some of the respondents could not explain a speed of 0m/s (Table 3), a velocity of
0m/s (Table 4) and an acceleration of 0m/ss (Table 5). The implication of these is that, when a learner cannot con-
ceptualise the meaning of a displacement of 0m, he or she is likely to have difficulties in the understanding of the
concepts such as a speed of 0m/s, a velocity 0m/s and an acceleration 0m/ss. In addition to errors in the defini-
tions/descriptions of concepts, there were many errors in the graphs/pictures drawn by learners to support their
descriptions/definitions. Some of the learners could not correctly represent their descriptions/definitions in the
437
(P. 424-440)
ISSN 1648–3898
form of a graph or picture. Consistent with Beichner (1994) the graph is considered to be like a photograph of the
situation. It is not seen to be an abstract mathematical construct, but rather a concrete duplication of the motion
event. This finding is consistent with research by McDermott et al., (1987) that, students find it difficult to represent
continuous motion with a line or curve, differentiating between the shape of a graph and the path of the motion.
In addition researchers (Hale, 2000; Beichner, 1994; Basson, 2002; Christensen & Thompson, 2012; Planinic, Ivanjek,
& Susac, 2013) argue that misinterpretations of kinematics graphs are common among students. Consequently,
adding to the inconsistencies between the textual definitions/descriptions and graphs/pictures. In agreement with
Roschelle (1998), the key finding is that students rapidly construct heuristic associations, but those associations
may not include enough knowledge to support an integrated understanding. In simple terms, learners may know
that a relationship exists between certain kinematics concepts, but they are unable to accurately define concepts
in terms of these relationships.
One of the reasons that can be attributed to the incorrect descriptions of these concepts is the fact that learn-
ers interpreted these concepts in the context of their mother tongue “Setswana”. Setswana is one of the South
African languages. In Setswana, “0” means nothing, added to that the terms acceleration, velocity and speed are
all represented by the same terminology in Setswana thereby meaning the same thing “moving faster” or “moving
slower”. If the object is not moving faster or slower then it stopped or stationery. In other words, if the value of
the acceleration, velocity, speed, distance or displacement is given a value of 0, then the term is described using
terminology such as “not moving” or “stationery”. That is why most of the learners used the term stationery or not
moving. Similarly, the issue of language is consistent with findings by Lemmer (1999) who also found that language
and culture contribute towards alternative conceptions of learners in kinematic concepts.
Similarly in the research by Shaffer and McDermott (2005) about half of the students stated that the accelera-
tion is zero at the turnaround point for the one dimension pre-test involving the ball on the ramp. Similar errors
were made at the end points on the pendulum problem. Often students reasoned that because the velocity is
zero, the acceleration is zero. This widely-recognized conceptual error is closely related to the tendency to confuse
velocity and acceleration. In addition some students seemed to believe that an object about to move must have
nonzero velocity. Many drew nonzero vectors at the starting point for the oval and pendulum. On the two dimen-
sion pre-test, about 20% of the students stated that the acceleration is zero for an object moving with constant
speed along the oval track. They treated the motion as if it were one-dimensional, not realizing that a change in
direction of the velocity means a change in the velocity vector and so corresponds to a nonzero acceleration. This
was the most common error made by the graduate students on the pendulum problem. Furthermore Shaffer and
McDermott (2005) argue that the confusion between velocity and acceleration were also evident on the colliding
carts pre-test. Many students claimed that the acceleration of cart A is to the right. Some seemed to be thinking
of an average or “overall” velocity and reasoned that because the car’s initial velocity to the right is larger than the
final velocity to the left, the acceleration is to the right. Even some students who answered correctly seemed to
relate the acceleration to the direction of the final velocity, not to the change in velocity.
In another related research a pulling metaphor in the form ‘acceleration pulls the tip of velocity was used.
This use of a pulling metaphor is important because it provides grounds for mapping the students’ knowledge to
scientific knowledge and indeed, a common-sense definition of ‘force’ often invokes pushing or pulling. Scientists
think about the relationship between velocity and acceleration with one unifying concept, the derivative. The use
of a pulling metaphor provides an explanation of these knowledge elements in terms of a single unifying abstrac-
tion (Roschelle, 1998).
Likewise textbooks present only a narrow subset of the scientific meanings available to practising scientists.
Discussions of qualitative interpretations and explanations, for example, hardly ever appear. Likewise, while stu-
dents can construct misconceptions, they also can construct knowledge that demonstrates clear progress towards
scientific understanding. The focus on opposing textbook concepts and student misconceptions or on opposing
textbook equations to students’ knowledge systems, therefore potentially misses much of the developmental
action (Roschelle, 1998).
Therefore, the results confirm findings by researchers (Basson, 2002, Halloun & Hestenes, 1985, Molefe et al.,
2005, Shaffer & McDermott, 2005) that learners confuse the concepts of position, velocity, and acceleration and they
relate them unscientifically. Similar findings (Trowbridge & McDermott, 1981) identified alternative conceptions
encountered by students in kinematics. Similarly (Beichner, 1994) students see little difference between distance,
velocity, and acceleration. They often believe that graphs of these variables should look identical. This might be
related to the graph as picture error. If a graph is like a photograph, it shouldn’t matter what is graphed, it will look
438
(P. 424-440)
like a replication of the object’s physical motion. Nonetheless, Lichtenberger, Vaterlaus and Wagner (2014) argue
that there are two basic mathematical concepts that are crucial for the understanding of kinematics: the concept
of rate and the concept of vector (including direction and addition). If a student understands the concept of rate,
he is able to answer correctly to questions about velocity and acceleration in different contexts.
However, in terms of understanding the concept of zero (0) as used in the definition of kinematics concepts
acceleration, velocity, speed, displacement and distance, the context and the content should play a major role.
Conclusion
The results of this research show that learners who participated in this research have difficulty in describing/
defining the concept of a displacement of 0m, a distance of 0m, a speed of 0m/s, a velocity of 0m/s as well as an
acceleration of 0m/ss. To date the literature has shown that there are misconceptions in kinematics, but this research
offers misconceptions associated with the concept of “0” zero numerically attached to concepts distance, displace-
ment, speed, velocity and acceleration. While this research does not offer a conclusive answer to the question of
the alternative conceptions and misconceptions that learners have on the description/definitions of a distance
of 0m, a displacement of 0m, a speed of 0m/s, a velocity of 0m/s and an acceleration of 0m/ss, it does offer new
knowledge in the sphere of misconceptions in kinematics.
References
Babbie, E. R. (2009). The practice of social research (12th ed.). International edition. Belmont, CA: Wadsworth.
Basson, I. (2002). Physics and mathematics as interrelated fields of thought development using acceleration as an example.
International Journal of Mathematical Education in Science and Technology, 33 (5), 679-690.
Beichner, R. J. (1994). Testing student interpretation of kinematics graphs. American Journal of Physics, 62 (8), 750-762.
Christensen, W. M., & Thompson, J. R. (2012). Investigating graphical representations of slope and derivative without a physics
context. Physical Review Special Topics-Physics Education Research, 8 (2), doi: 10.1103/PhysRevSTPER.8.023101
Clement, J. (1983). A conceptual model discussed by Galileo and used intuitively by physics students. Mental Models, 2, 325-
339.
Connelly, L. M. (2010). What is phenomenology? MEDSURG Nursing, 19 (2), 127–129.
Cilesiz, S. (2011). A phenomenological approach to experiences with technology: Current state, promise, and future directions
for research. Educational Technology Research and Development, 59 (4), 487-510.
Cutnell J. D., & Johnson, K.W. (1995). Physics (3rd ed.). New York: John Wiley & Sons, Inc.
Cresswell, J. W. (2007). Qualitative inquiry and research design: Choosing among five approaches (2nd ed.). London: Sage Publica-
tions Ltd.
Dey, I. (2003). Qualitative data analysis: A user friendly guide for social scientists. Routledge.
DiSessa, A. A. (1993). Toward an epistemology of physics. Cognition and Instruction, 10 (2-3), 105-225.
Duit, R., Treagust, D. F., & Mansfield, H. (1996). Investigating student understanding as a prerequisite to improving teaching
and learning in science and mathematics. In D. F Treagust, R. Duit, & B. J. Fraser (Eds.), Improving teaching and learning
in science and mathematics (pp17-31). Teachers College Press, Teachers College, Columbia University, 1234 Amsterdam
Avenue, New York, NY 10027.
Glesne, C. (1999). Becoming qualitative researchers (2nd ed.). New York: Longman.
Halloun, I. A., & Hestenes, D. (1985). Common sense concepts about motion. American Journal of Physics, 53 (11), 1056-1065.
Kinnunen, P., & Simon, B. (2012). Phenomenography and grounded theory as research methods in computing education research
field. Computer Science Education, 22 (2), 199-218.
Langdridge, D. (2007). Phenomenological psychology: Theory, research and method. Pearson Education.
Larkin, J. (1981). Cognition of learning physics. American Journal of Physics, 49 (6), 534-541.
Larsson, J., & Holmström, I. (2007). Phenomenographic or phenomenological analysis: does it matter? Examples from a study on
anaesthesiologists’ work. International Journal of Qualitative Studies on Health and Well-being, 2 (1), 55-64.
Lemmer, M. (1999). First year university students‘ conceptions of time and space in Physics. PhD thesis, Potchefstroom: North-West
University.
Lemmer, M. (2013). Nature, cause and effect of students’ intuitive conceptions regarding changes in velocity. International Journal
of Science Education, 35 (2), 239-261.
Lichtenberger, A., & Wagner, A. V. C. (2014). Analysis of student concept knowledge in kinematics. In C. P. Constantinou, N. Papa-
douris & A. Hadjigeorgiou (Eds.), E-Book Proceedings of the ESERA 2013 Conference: Science Education Research For Evidence-
based Teaching and Coherence in Learning. Part 11 (R. Millar & J. Dolin), (pp. 38-50) Nicosia, Cyprus: ESERA.
Marton, F. (1981). Phenomenography—describing conceptions of the world around us. Instructional Science, 10 (2), 177-200.
McCloskey, M. (1983). Intuitive physics. Scientific American, 248 (4), 122-130.
McDermott, L. C., Rosenquist, M. L., & Van Zee, E. H. (1987). Student difficulties in connecting graphs and physics: Examples from
kinematics. American Journal of Physics, 55 (6), 503-513.
439
(P. 424-440)
ISSN 1648–3898
McKenzie, D. L., & Padilla, M. J. (1986). The construction and validation of the test of graphing in science (TOGS). Journal of Re-
search in Science Teaching, 23 (7), 571-579.
McMillan, J. H., & Schumacher, S. (1997). Research in education: A conceptual approach. New York: Long.
Mokros, J. R., & Tinker, R. F. (1987). The impact of microcomputer‐based labs on children‘s ability to interpret graphs. Journal of
Research in Science Teaching, 24 (4), 369-383.
Molefe, N. P. J., Lemmer, M., & Smit, J. J. A. (2005). Comparison of the learning effectiveness of computer-based and conventional
experiments in science education. South African Journal of Education, 25 (1), 50-55.
Mouton, J., & Babbie, E. (2001). The practice of social research. Cape Town: Wadsworth Publishing Company.
Planinic, M., Ivanjek, L., Susac, A., & Milin-Sipus, Z. (2013). Comparison of university students’ understanding of graphs in different
contexts. Physical Review Special Topics-Physics Education Research, 9 (2), doi: 10.1103/PhysRevSTPER.9.020103.
Roschelle, J. (1998). Beyond romantic versus sceptic: A microanalysis of conceptual change in kinematics. International Journal
of Science Education, 20 (9), 1025-1042.
Rosenquist, M. L., & McDermott, L. C. (1987). A conceptual approach to teaching kinematics. American Journal of Physics, 55 (5),
407-415.
Shaffer, P. S., & McDermott, L. C. (2005). A research-based approach to improving student understanding of the vector nature of
kinematical concepts. American Journal of Physics, 73 (10), 921-931.
Smith, J. A., Flowers, P., & Larkin, M. (2009). Interpretative phenomenological analysis: Theory, method and research. London: Sage
Publications Ltd.
Storey, L. (2007). Doing interpretive phenomenology analysis. In A. Lyons & E. Coyle (Eds.), Analysing qualitative data in psychology:
A practical and comparative guide. London: Sage Publications Ltd.
Thijs, G. D., & Van Den Berg, E. D. (1995). Cultural factors in the origin and remediation of alternative conceptions in physics.
Science & Education, 4 (4), 317-347.
Thompson, E. (2010). From phenomenography study to planning teaching. In Proceedings of the fifteenth annual conference on
Innovation and technology in computer science education (pp. 13-17). ACM.
Trowbridge, D. E., & McDermott, L. C. (1981). Investigation of student understanding of the concept of acceleration in one di-
mension. American Journal of Physics, 49 (3), 242-253.
Received: May 25, 2016 Accepted: July 10, 2016
Abraham Motlhabane PhD, Associate Professor, University of South Africa, Preller Street,
Muckleneuk Ridge, Pretoria, South Africa.
E-mail: motlhat@unisa.ac.za
440
EFFECTIVENESS OF THE
INQF-BASED LEARNING ON
A GENERAL PHYSICS FOR
IMPROVING STUDENT’S
LEARNING OUTCOMES
Budi Jatmiko, Wahono Widodo, Abstract. This research aims to analyse

Martini, Mohammad Budiyanto, effectiveness of the Indonesian National
Iwan Wicaksono, Qualification Framework (INQF)-based
Paken Pandiangan learning on General Physics to increase the
sixth level student’s Learning Outcomes
(LOs) according to the INQF indicators and
student’s skills in using physics laboratory
equipment. This research was conducted
using two groups of students that consisted
Introduction of 29 and 30 students. A preliminary test
(pre-test) and a post-test were applied to
In this 21st century, there are several essential “student’s skills, knowl-
the groups that assumed to have the same
edge and expertise that should be mastered to succeed in work and life in the
21st century”. An example of the required skills is the problem solving skills level of knowledge and skills. The data
(Partnership for 21st Century Skills, 2009). Problem solving skills covering a were analysed using the paired t-test, the
wide range of capabilities, including procedural and non-procedural problem n-gain, and the ANOVA. The results show
solving capabilities (Pretz, Naples, & Sternberg, 2003). In the context of Gen- that the INQF-based learning applied to
eral Physics learning, step by step of the problem solving skills are needed to the General Physics effective in increasing
be trained continuously for both the procedural and non-procedural problem
the student’s LOs according to the INQF
solving. Moreover, problem solving in General Physics requires skills of using
the physics laboratory equipment. indicators. Moreover, the n-gain scores
It is generally understood that in order to achieve the 21st century between the pre-test and the post-test can
skills, it requires a certain qualification requirements (Griffin & Care, 2015). be categorized as moderate for the sixth
Qualification defined as a formal outcome of an assessment and validation level student’s LOs and categorized as high
process which is obtained when a competent body determines that an indi- for the student’s skills in using the physics
vidual has achieved learning outcome (LOs) to given standards (Allais, 2014;
laboratory equipment.
James & Dorn, 2015). National qualifications system is related to the national
recognition of learning and other mechanisms that links education and Key words: INQF-based learning, general
training to the labour market and civil society. It may include development physics, student’s learning outcomes.
and implementation of institutional arrangements and processes relating
to quality assurance, assessment and appreciation (European Communities,
2008; Ure, 2015). Budi Jatmiko, Wahono Widodo,
National qualifications framework (NQF) had been set up in three Euro- Martini, Mohammad Budiyanto
State University of Surabaya,
pean countries: Ireland, France and the UK before 2005. It is reported in 2015 Surabaya, Indonesia
that the framework is currently being developed in 38 countries cooperating Iwan Wicaksono
on the European qualifications framework. Some studies showed that the University of Jember, Jember, Indonesia
Paken Pandiangan
NQF had significant impact on education, training, and policies on working Indonesia Open University, Indonesia
practices (James & Dorn, 2015; Chakroun, 2010; Gosling, 2011).
441
ISSN 1648–3898
(P. 441-451)
Recently, Indonesia established a similar framework which is called Indonesian Qualification Framework
(INQF; In Indonesian it becomes Kerangka Kualifikasi Nasional Indonesia (KKNI)). It was issued through the Presi-
dential Decree No. 8 of 2012. The INQF aims to provide recognition of competence of work in accordance with the
structure of employment in various sectors. The INQF is a level of qualification framework that aligns competence,
equalization, and integration in the fields of education and vocational training, as well as work experience. The
term qualification is defined as mastery of LOs conferring to a certain level in the INQF structure.
According to the INQF, there are nine qualifications from the lowest (level 1) to the highest (level 9). Levels
1-3 are all grouped as office operators, level 4-6 are grouped as office technicians or analysts and level 7 to level 9
are grouped as professional careers. The INQF structure categorizes undergraduate degree program in the field of
education into the sixth level. The sixth level student’s LOs are defined as follows: (i) able to apply their expertise and
utilize Arts and Sciences (science and technology) in solving problems; (ii) mastering concepts in depth knowledge
in their field and able to formulate a procedural problem solving; (iii) able to take right decisions based on analysis
of information and data, and is able to provide guidance in selecting various alternative solutions independently
or in groups; and (iv) responsible for their own work and accountable for achievement of organizational work
(Jatmiko, Widodo, Martini, & Budiyanto, 2014).
In line with the INQF structure, the Minister of Education and Cultural Affairs issued Regulation of the Minister
of Education and Culture No. 49 of 2014 on Higher National Education Standards. This regulation requires a learn-
ing process in a higher degree institution that leads to the achievement of LOs indicators of the INQF. Through the
new standard, it is clear that the regulation gives no other choice for higher degree institutions in Indonesia for
not implementing learning process that leads to achievement of LOs indicators according to the INQF.
Studies related to the NQF in the field of education in several countries show that: (i) in Europe, the NQF is as-
sociated with the increase of the learning outcomes from input to output (Ure, 2015); (ii) in Chile, the NQF links to the
formulation of principles and criteria for education instrument implementation for the qualification framework (Solís,
Castillo, & Undurraga, 2013); and (iii) in Portugal, the NQF serves as an assessment tool which allows diagnosing and
controlling the development of learning achievement (Stasiūnaitienė & Teresevičienė, 2006). In general, it showed
that the NQF provided significant impact on the improvement of the learning outcome scores (Chakroun, 2010)
Series of researches related to the INQF on education field at the State University of Surabaya in Surabaya -
Indonesia had been commenced since 2013. The research mainly focused on developing prototypes of the INQF-
based curriculum to enhance professional and pedagogical competence of science education teachers. The work
had successfully published a book entitled of “Book in prototyping INQF-based science education curriculum 1st
Edition” in 2014 (Jatmiko, Widodo, Martini, & Budiyanto, 2014). Subsequently, a limited test (including 15 students)
was done for the INQF-based learning on a General Physics for students in bachelor degree of science education
program at the State University of Surabaya. The results had been reported in the article in a national seminar in
Surabaya-Indonesia (Jatmiko, Widodo, Martini, & Budiyanto, 2015). Based on the results of the research described
in the article, a book had been published entitled of “Book of Prototyping INQF-based Curriculum for the science
education curriculum 2nd Edition”. The second edition book equipped with: (a) examples of the learning tools for the
general physics research that based on the INQF and (b) learning syntax (flow of instructional activities) according
to the INQF sixth level of students’ LOs indicators, i.e. (1) motivating, (2) presenting information and experimental
groups/discussion sharing, (3) identifying and solving problems, (4) establishing and enriching, and (5) evaluating
the use of science and technology (Jatmiko, Widodo, & Martini, 2015).
The sixth level INQF indicators covers (i) mastering concepts, (ii) formulating procedural problem-solving,
(iii) formulating non-procedural problem-solving, and (iv) decision making. The concept indicators may include:
remembering (C1), comprehension (C2), applications (C3), analysis (C4), evaluation (C5), and creation (C6) (Krath-
wohl & Anderson, 2001; Bush, Daddysman, & Charnigo, 2014). On the other hand, procedural problem solving may
include indicators such as: (i) observation, (ii) asking questions, (iii) making hypothesis, (iv) testing the hypothesis,
(v) analysing the data and conclusions, and (vi) replicating research through the obtained correspondence between
empirical and theoretical (Bradford, 2015). The non-procedural problem solving indicators are: (i) arguing that is
defined as capability of reasoning in accordance with his/her experience and knowledge, (ii) strategic indication
that is capability of selecting appropriate problem-solving strategies based on analysis, and (iii) solution evalua-
tion that is considered as capability to evaluate solutions to problems logically correspond to the case description,
analysis, and experimental data to support decision making (Snyder & Snyder, 2008). Lastly, the decision making
comprises of ability in: (i) determining the objectives, (ii) identifying options, (iii) analysing the information, and
(iv) making a choice (Campbell, Lofstrom, & Brian, 1997).
442
ISSN 1648–3898 EFFECTIVENESS OF THE INQF-BASED LEARNING ON A GENERAL PHYSICS FOR IMPROVING
(P. 441-451)
Problem of Research
The problem in this research is to analyse the effectiveness how the INQF-based learning on the General Phys-
ics can improve student’s LOs according to the sixth level of INQF indicators. The INQF-based learning is said to be
effective when the learning process is statistically able to achieve significant increase of student’s scores after the
pre-test and the post-test in terms of the sixth level student’s LOs and skills. Effectiveness of student’s LOs of the
sixth level and the skills in utilizing the physics laboratory equipment is determined by the normalized gain scores
(n-gain). n-gain = (score post-test - score pre-test)/(100 - score pre-test) (Hake, 1999). According to the following
criteria: (1) if n-gain ≥ 0.7 (high), (2) if 0.3 < n-gain < 0.7 (moderate), dan (3) if n-gain ≤ 0.3 (low).
This research aims to analyse the effectiveness of the learning process against the student’s LOs that have
been defined according to the sixth level of the INQF and the skills in utilizing the physics laboratory equipment.
Compared to the previous work (Jatmiko, Widodo, Martini, & Budiyanto, 2015), this research involves a greater
number of research.
Research Focus
The focus of the research is to analyse the impact of the INQF learning against the sixth level student’s LOs
according to INQF indicators. The problems include: (i) is there any significant increment (statistically) of the sixth
level student’s LOs and student’s skills in using the physics laboratory equipment before and after employing the
INQF-based learning?, (ii) how much do the sixth level student’s LOs and the student’s skills increase in using the
physics laboratory equipment? and (iii) is there any increment difference of the sixth level student’s LOs and the
skills in using the physics laboratory equipment between group-1 and group-2?
General Background of Research
The research puts emphasis on analysing the effectiveness of the INQF-based learning by analysing the impact
of the INQF-based learning on General Physics to the sixth level student’s LOs and the student’s skills in using the
physics laboratory equipment with n-gain employed before and after the INQF-based learning. In this research,
the effectiveness of the sixth level student’s LOs and the student’s skills in using the physics laboratory equipment
is referred to the existence of significant (statistically) increment scores between the preliminary test (pre-test) and
the post-test. When calculated by the n-gain, it can be categorized as low, moderate and high for both the sixth
level student’s LOs and the student’s skills in using the physics laboratory equipment.
Sample of Research
This research was conducted using two groups of students at Science study program, faculty of Mathematics
and Science, The State University of Surabaya. The students took a General Physics subject during the odd semes-
ter in academic year 2015/2016. Furthermore, they were called group-1 and group-2. Those groups consisted of
29 and 30 students, respectively. The two groups held the same sixth level of student’s LOs and student’s skills in
using the physics laboratory equipment.
This research can be classified as a quasi-experimental research. It was performed using the one group pre-test
and post-test design, i.e., O1 X O2 (Fraenkel & Wallen, 2009). The two groups of the students were offered exactly
the same pre-test before learning process was provided. The test instrument consisted of sixth level INQF indicators
and the student’s skills in using the physics laboratory equipment. After finalizing the pre-test, learning process
of General Physics that based on the INQF was applied to the two groups of students. The learning process was
conducted by utilizing learning tools such as syllabus, lesson plan, a student textbook, and student worksheets. In
the previous work, these learning tools had been evaluated in terms of the content and the construction validities,
443
ISSN 1648–3898
(P. 441-451)
which show validity scores (in the range 0-4) for syllabus: 3.58 (very valid), lesson plan: 3.86 (very valid), a student
textbook: 3.18 (valid) and student works sheets: 3.95 (very valid) (Jatmiko, Wahono, & Martini, 2015). The learning
process that was applied in the research was according to the following steps: (1) motivating, (2) presenting in-
formation and experimental groups/discussion sharing, (3) identifying and solving problems, (4) establishing and
enriching, and (5) evaluating the use of science and technology (Jatmiko, Widodo, & Martini, 2015). Finally, after
the learning process, the two groups were asked to work with a post-test. It should be reminded that we devised
the same instrument for post-test as it was provided at the pre-test.
The sixth level of the student’s LOs was measured using test instrument that consisted of: (i) mastering
concepts, (ii) formulating procedural problem-solving, (iii) formulating non-procedural problem-solving, and (iv)
decision making. Meanwhile, the student’s skills in using the physics laboratory equipment were determined by
performance test in terms of skills in measuring length, time, mass, temperature and ticker timer.
Data Analysis

In order to analyse the impacts of the INQF-based learning to the student’s LOs, the scores of the pre-test and
post-test that had been collected were analysed using the paired t-test or non-parametric analysis of Wilcoxon test.
The selection of the testing methods depended on the fulfilment of the normality assumption for both pre-test and
post-test scores. When the normality assumption for the scores are achieved, then the paired t-test will be applied.
Otherwise, the non-parametric analysis will be used. Additionally, we utilized the n-gain analysis to examine the
impact of the INQF-based learning against the student’s LOs (Hake, 1998). The analysis was performed using the
IBM SPSS Statistics 19 software.
Furthermore, to analyse the equality of the impact of the INQF-based learning against the sixth level student’s
LOs and the student’s skills in using the physics laboratory equipment, we employed the analysis of variance (ANOVA)
for both groups, i.e. the group-1 and group-2. The testing method depended on the fulfilment of the normality
and equality of the two variance assumption for both group-1 and group-2 average of the n-gain.
Results of Research
The pre-test and the post-test mean scores of the two groups are presented in Figure 1. The grey bar repre-
senting the pre-test and the shaded bar signifying the post-test. The overall examination for the two groups in
terms of the sixth level student’s LOs is shown in Figure 1 and Table 1, while the student’s skills in using the physics
laboratory equipment is shown in Figure 1.
Figure 1: The mean scores of the student’s pre-test and post-test in terms of the sixth level student’s LOs and the
student’s skills in using the physics laboratory equipment for both the group-1 and the group-2.
444
(P. 441-451)
Table 1. The mean scores of the pre-test, post-test, and the n-gain of the sixth level student’s LOs for the
group-1 and the group-2.
Group-1 Group-2
Numb The sixth level student’s LOs
Pre-test Post-test n-gain Pre-test Post-test n-gain
1 Mastering concepts 24.68 65.19 0.53 36.85 68.97 0.55
2 Formulating procedural problem-solving 20.00 63.65 0.52 17.33 63.83 0.57
Formulating non- procedural problem-

3 27.16 64.22 0.51 40.73 71.55 0.56
solving
4 Decision making 19.79 64.38 0.52 21.39 67.50 0.61
Mean scores of the n-gain for both the sixth level student’s LOs and the student’s skills in using the physics
laboratory equipment for the group-1 and the group-2 are shown in Figure 2.
Figure 2. The mean scores of the n-gain for both the sixth level student’s LOs and the student’s skills in using
the physics laboratory equipment for the group-1 and the group-2.
Figure 1 shows that the mean score between the pre-test and the post-test in terms of the sixth level student’s
LOs and the student’s skills in using the physics laboratory equipment for both groups-1 and group-2 is increasing.
The average of the pre-test and the post-test scores for the group-1 are 30.82 and 66.72, respectively; while the
average of the pre-test and the post-test scores for the group-2 are 20.15 and 64.42, respectively. The average of
the pre-test, the post-test, and the n-gain in terms of the sixth level student’s LOs for each INQF indicators for both
groups-1 and group-2 are depicted in Figure 1 and detailed in Table 1. Figure 2 depicts the mean score of the n-gain
for both group-1 and group-2 in terms of the student’s LOs of the sixth level resulting 0.52 and 0.56, respectively.
The mean scores of both groups in terms of the level student’s LOs can be categorized as moderate.
It is clearly seen in Figure 1 that the pre-test and post-test mean scores for the group-1 of the student’s skills in
using the physics laboratory equipment achieves 53.78 and 89.03, respectively. For the group-2, the mean scores are
54.19 and 89.40. On the other hand, Figure 2 demonstrates the mean score of the n-gain in terms of the student’s
skills in using the physics laboratory equipment shows 0.76 and 0.77 for the group-1 and the group-2, respectively.
The mean scores of both groups in terms of the student’s skills in using the physics laboratory equipment can be
categorized as high (Hake, 1999).
For analysing the impact of the student’s LOs in the INQF-based learning in terms of the sixth level student’s
LOs, we used a paired t-test statistical measurement. The summary of the paired t-test after the fulfilment of the
normality assumptions for both pre-test and post-test is shown in Table 1 and Table 2.
This section should be reworked because you mixed the used terms of inquiring and discussion not defined
before. Thus, the result of this section is confused, almost not clear.
445
ISSN 1648–3898
(P. 441-451)
Table 2. The results of the sixth level student’s LOs paired t-test in group-1
Paired Samples Test
N Mean S df t p
Pair 1 Pre-test-Post-test 29 -1.438 -0.313 28 -24.716 < 0.0001

p < 0.05 (2-tailed)
*
Table 3. The results of the sixth level student’s LOs paired t-test in group-2
Paired Samples Test
N Mean S df t p
Pair 1 Pre-test-Post-test 30 -1.770 -0.360 29 -26.963 < 0.0001

p < 0.05 (2-tailed)
*
It can be seen in Table 2 that the t score gives value of -24.716 for degrees of freedom, df = 28. The score is
considered as significant, because of p < 0.05. Therefore, it can be concluded there is a significant impact (statisti-
cally) of the INQF-based learning for the group-1 in the sixth level student’s LOs at significance level of 5%. Similarly,
Table 3 shows the t score of -26.963 for the degrees of freedom, df = 29, gives significance score as p < 0.05. Hence,
there is a significance impact statistically of the INQF-based learning in the sixth level student’s LOs at significant
level of 5% for the group-2.
In order to analyse the improvement of the sixth level student’s LOs for the group-1 in terms of student’s skills
in using the physics laboratory equipment, we carried out an examination utilizing the Wilcoxon test. In contrast,
we performed a paired t-test for the group-2. Summaries of the Wilcoxon test and the paired t-test for the pre-test
and post-test in terms of the student’s skills in using the measuring equipment for both group-1 and group-2 are
shown in Table 4 and Table 5.
Table 4. Wilcoxon test for the student’s skills in using the physics laboratory equipment in group-1
Paired Samples Test
N Z p
Pre-test - Post-test 29 -4.714 < 0.0001

p <0.05 (2-tailed)
*
Table 5. The results of paired t-test for the skills in using the physics laboratory equipment in group-2
Paired Samples Test
N Mean S df t p
Pair 1 Pre-test - Post-test 30 -1.409 -0.196 29 -39.276 < 0.0001

p <0.05 (2-tailed)
*
Table 4 shows the Wilcoxon test for the student’s skills in using the physics laboratory equipment. Examination
of the third column reveals that the Z test gives value of -4.714 with significance level p < 0.05. It clearly indicates
that there is impact on the INQF-based learning to the student’s skills in using the physics laboratory equipment
for the group-1. Similarly, Table 5 shows that the t test gives value of -39.276 with significance level p=0.14E-14-
<0.05. Based on the table, it can be admitted that there is a significant impact of the INQF-based learning to the
student’s skills in using the physics laboratory equipment on the group-2.
446
(P. 441-451)
Furthermore, equality of the impact INQF-based learning for both the sixth level student’s LOs and the stu-
dent’s skills in using the physics laboratory equipment is analysed using ANOVA to the group-1 and group-2. The
results after the fulfilment of the normality assumption as well as the equality of two variances are shown in Table
6 and Table 7.
Table 6. The results of ANOVA for the sixth level student’s LOs to the group-1 and group-2
ANOVA
Sum of Squares Mean Square df F p
Between Groups 0.043 0.014 3 1.688 0.195

Within Groups 0.211 0.008 25
Total 0.254 28
p <0.05
*
Table 7. The results of ANOVA for the skills in using the physics laboratory equipment in group-1 and
group-2
ANOVA
Sum of Squares Mean Square df F p
Between Groups 0.012 0.004 3 0.833 0.488

Within Groups 0.120 0.005 25
Total 0.132 28
p <0.05
*
It is clearly seen from Table 6 that the F-test provides value of 1.688 with significance level p=0.195 > 0.05.
Hence, there is a strong indication that the impact of the INQF-based learning to the sixth level student’s LOs for
the groups is not different at the 5% significance level. Table 7 shows the F count is 0,833 with significance level
p=0.488 > 0.05. Therefore, it can be concluded that there is no difference in terms of the student’s skills in using
the physics laboratory equipment at the 5% significance level.
Discussion
The Sixth Level Student’s Los Improvement
Based on the Figure 1 and Table 1, it can be observed that before the learning process was done, the students
showed to have low scores. The mean scores of the sixth level student’s LOs were below the standard score (i.e.,
40), it was 30.82 on a score range of 0-100, and it gave a score of 20.15 for group-1 and group-2, respectively. Both
of the achievements fell on the grade E (0 ≤ E <40). Grade E is considered as the lowest while grade A is considered
the highest. Similarly, mean scores of the sixth level student’s LOs for each INQF indicator were below the standard
score 40. This might be because the students were not familiar with the thinking activities that are designed by
the INQF sixth level of qualification.
The results of the research were supported by low score data of the national average test on teacher com-
petence (Celik, 2011) as well as the preliminary research of our study showed to have low scores in terms of the
sixth level student’s LOs (Jatmiko & Martini, 2014). The results of this work can be related to the study that had
been done by TIMSS research between 1999 and 2011, which elaborates the facts that Indonesian junior high
school students were only able to identify a number of basic facts. It was found that they had not been able to
communicate well. A similar result was done by PISA between 2003 and 2012. It was mentioned that Indone-
sian students have limited scientific knowledge. They can only apply knowledge to multiple familiar situations.
Additionally, the students can only present clear scientific explanations without giving evidence. This might be
due to that the science teachers of the junior high schools in Indonesia possess low competence in scientific
literacy. Hence, the teachers were not able to explain clearly to the students. The study was supported by em-
447
ISSN 1648–3898
(P. 441-451)
pirical data (Jatmiko & Martini, 2014).

In contrast, after the learning process of General Physics that based on INQF was done, the result shows that
the undergraduate students are able to obtain a mean score of 66.72 for the group-1 and 64.42 for the group-2.
Both of the mean scores are at almost the same value, although they are slightly different on grade of B- (65 ≤ E
<70). This means that there is an increase in the average score as much as 35.90 or 116.48 % on the group-1, and
there is an increase in the mean score as much as 44.27 or 219.70 % in the group-2. The increase of the sixth level
student’s LOs scores on these two groups is significant and there is no difference with significance level of 5%, with
n-gain average of 0.52 and 0.56 for group-1 and group-2, respectively. Both can be categorized as moderate. These
results indicate the existence of significant impact on the learning process that based on the INQF. The degree
of impact, represented by the mean scores of the n-gain, for the learning process for both groups is consistently
significant (statistically) at significance level of 5%, even though they are slightly different. Both of the n-gain can
be categorized as moderate.
Increasing the sixth level student’s LOs is probably because the students in this research were trained and
directed to achieve LOs qualification levels of all six (Presidential Decree No. 8 of 2012). The indicators have been
represented in the learning tools that have been implemented, which have been constructed based on the INQF
indicators according to the mastering theoretical concepts (Krathwohl & Anderson, 2001); procedural problem
solving skills (Bradford, 2015), non-procedural problem solving skills (Snyder & Snyder, 2008); and decision mak-
ing skills (Campbell, Lofstrom, & Brian, 1997). Based on our examination in this research, it proofs that the learning
steps that have been formulated in Jatmiko (Jatmiko, Widodo, & Martini, 2015) are supported by empirical data.
The formulation mainly emphasizes on the problem solving activities.
The research results in this work verify various works in problem solving activities that can be summarized as
follows: (i) the problem based learning (PBL) that emphasizes on problem-solving activities can improve the skills of
critical thinking and problem solving skills (Zabit, 2010); (ii) the PBL format can be beneficial for students to improve:
independent learning, critical thinking, problem solving, and communication skills (Senel, Ulucan, & Adilogullari,
2015). Additionally, the PBL program which involves a multidisciplinary student health is significantly positive effect
on decision-making and a willingness to learn and a positive attitude are higher; (iii) the PBL learning strategy that
focuses on the development and problem-solving groups, can improve the knowledge content, problem solving
skills, and group dynamics (Goltz, Hietapelto, Reinsch, & Tyrell, 2007). Moreover, the results state that teams that
are equipped with interpersonal skills and good problem solving are capable of making decisions effectively; (iv)
students who have utilized the PBL achieve generic problem-solving scores higher than the control group sig-
nificantly (Klegeris & Hurren, 2013). This is mainly because the PBL can be used to enhance troubleshooting skills,
including design and problem-solving, decision-making, and analysis of system; (v) the PBL models have proven
to be beneficial for improving students’ conceptual learning, knowledge, skills and values of science (Etherington,
2011); (vi) Learning Cycle for Inquiry Concept (LCIC) Model, which aims to provide opportunities for teachers and
students to develop and improve scientific skills. The model focusing on high-order thinking skills thoroughly as
well as conceptual understanding by improving critical thinking skills (Corlu & Corlu, 2012).
Student’s Skills in Using the Physics Laboratory Equipment Improvement
According to Figure 1, prior to the learning process, students have average student competence, i.e., a
score of 53.78 in the range 0-100 for the group-1 and 54.19 for the group-2. Both of the mean scores are almost
at the C grade (55 ≤ E <60) from range values E (the lowest) to A (the highest). These student’s skills in using the
physics laboratory equipment show less moderate skills in order to use or operate the measuring equipment,
including: length, time, mass, temperature, and ticker timer. This might be because students are familiar in do-
ing measurements using the gauge during their senior high school. The reason is supported by opinion of the
Chinese philosopher, named Confucius that in these modern times are categorized into five principles of active
learning, i.e., “when I hear, I see, I discuss and do, I get the knowledge and skills” (McLeod, Barr, & Welch, 2015).
After the learning process of the INQF-based General Physics, the students for the group-1 achieve average score
of 89.03, and students for group-2 get 89.40 score. Both of the mean scores are similar although it is slightly
different, namely A (85 ≤ A ≤ 100). This means that there is an increase in the average score of 35.25, or 65.54%
on group-1, and there was an increase in mean score of 35.21, or 64.98% in the group-2. The increment of the
448
(P. 441-451)
student’s skills in using the physics laboratory equipment scores for these two groups is significant and there
is no difference at real level of 5%, the n-gain of the group-1 is 0.76 and 0.77 for the group-2. Both are at the
high category. These results indicate that there is an impact of the INQF-based learning to the student’s skills in
using the physics laboratory equipment significantly (statistically), the degree of the impact in n-gain there is
no difference at the 5% significance level. Both are in the same category: at high category.
The increase in the student’s skills in using the physics laboratory equipment might be because the students
have been trained and directed to achieve the skills in using the physics laboratory equipment scores, i.e., famil-
iarity to use or operate the measuring equipment, including: length, time, mass, temperature, and ticker timer.
The indicators of the skills in using the physics laboratory equipment that has been realized in the learning tools
and implemented. In this work, it can be seen that one of the learning process steps was formulated as problem
identification and problem solving. It shows that the research results verify some other works, for example (i)
PBL for the psychomotor development, where students are able to design related tools that improve their skills
(Tanel & Erol, 2008) and (ii) PBL can improve psychomotor skills and academic achievement in individuals with
mental and physical characteristics that are different (Sever & Oguz-Unver, 2013).
The increment of the sixth level student’s LOs and the student’s skills in using the physics laboratory equip-
ment in this research is not different with the results in the previous work, which involves fewer number of
research subjects (15 students) (Jatmiko, Widodo, Martini, & Budiyanto, 2015), as well as there is no difference
with studies as follows: (1) improvement of the sixth level student’s LOs and the student’s skills in using the
physics laboratory equipment is guaranteed when learning process utilizes the national qualification framework
concept (Krstovic & Cepic, 2010); (2) improvement of the student’s LOs can create significant contribution to
transparency and international recognition of qualifications, especially through the strengthening of the concept
and practice (Keevy, 2013).
Conclusions
Based on the research results and discussion above, the INQF-based learning on the General Physics can
be considered effective to increase the sixth level student’s LOs and student’s skills in using physics laboratory
equipment. The effectiveness of improving the sixth level student’s LOs and the student’s skills in using the phys-
ics laboratory equipment are based on as follows: (i) there is significant increment (statistically) on the sixth level
student’s LOs and the student’s skills in using the physics laboratory equipment before and after employing the
INQF-based learning, (ii) the increase of the n-gain scores can be categorized as moderate for the sixth level stu-
dent’s LOs and can be categorized as high for the student’s skills in using the physics laboratory equipment, and
(iii) the increment of the sixth level student’s LOs and the student’s skills in using the physics laboratory equipment
for both group-1 and group-2 are not different.
Acknowledgements
The authors cannot express enough thanks to the Government of the Republic of Indonesia through the
Ministry of Research and Technology of Higher Education, especially DP2M for their funding support on this
research. Acknowledgements are also submitted to The State University of Surabaya that had provided research
opportunities.
References
Allais, S. (2014). Selling out education, national qualifications frameworks and the neglect of knowledge. Netherlands: Sense
Publishers.
Bradford, A. (2015). Science & the scientific method: A definition. Live Science. Retrieved 05/02/2016, from http://www.
livescience.com/21456-empirical-evidence-a-definition.html
Bush, H. M., Daddysman, J., & Charnigo, R. (2014). Improving outcomes with Bloom’s Taxonomy: From statistics education
to research partnerships. Journal of Biometrics & Biostatistics, 5 (4), 1-3.
Campbell, V., Lofstrom, J., & Brian, J. (1997). Decision based on science. Arlington VA: National Science Teachers Associa-
tion.
449
ISSN 1648–3898
(P. 441-451)
Celik, S. (2011). Characteristics and competencies for teacher educators: Addressing the need for improved professional
standards in Turkey. Australian Journal of Teacher Education, 36 (4), 18-32.
Chakroun, B. (2010). National Qualification Frameworks: from Policy Borrowing to Policy Learning. European Journal of
Education, 45 (2), 199-216.
Corlu, S. M., & Çorlu, A. A. (2012). Scientific inquiry based professional development models in teacher education. Educa-
tional Sciences: Theory & Practice, 12 (1), 514-521.
Etherington, M. B. (2011). Investigative primary science: A problem-based learning approach. Australian Journal of Teacher
Education, 36 (9), 53-74.
European Commission. (2008). The European qualifications framework for lifelong learning (EQF). Luxembourg: Office for
Official Publications of the European Communities.
Fraenkel, J. R., & Wallen, N. E. (2009). How to design and evaluate research in education. New York: McGraw-Hill Compa-
nies.
Goltz, S. M., Hietapelto, A. B., Reinsch, R. W., & Tyrell, S. K. (2007). Teaching teamwork and problem solving concurrently.
Journal of Management Education, 20 (10), 1-20.
Gosling, D. (2011). Lost opportunity: What a credit framework would have added to the national qualification frameworks.
Higher Education Quarterly, 55 (3), 270–284.
Griffin, P., & Care, E. (2015). Assessment and teaching of 21st century skills: Methods and approach. New York: Springer.
Hake, R. R. (1998). Interactive-Engagement versus traditional methods: A six-thousand-student survey of mechanics test
data for introductory physics courses. American Journal of Physics, 66 (1), 64-74.
Hake, R. R. (1999). Analyzing change/gain score. Physics Indiana. Retrieved 10/03/2016, from http:// www. physics. indiana.
edu/ sdi/ AnalizingChange- Gain.pdf.
James, J., & Dorn, B. (2015). National Qualifications Framework Developments in Europe – Anniversary edition. Luxembourg:
Publications office of the European Union.
Jatmiko, B., & Martini. (2014). Pembelajaran Fisika bagi Mahasiswa Calon Guru Fisika Abad Ke-21 [Physics education for 21st
century: Pre-service training]. Proceeding Semlok (pp. 140-152). Surabaya: Unesa Press.
Jatmiko, B., Wahono, W., & Martini. (2015). Buku Prototipe Kurikulum Pendidikan Sains Berbasis KKNI [The book of prototype
of the science education curriculum based on INQF]. (2nd ed.). Surabaya: Jaudar Press.
Jatmiko, B., Wahono, W., & Martini. (2015). Laporan Hibah Kompetensi Tahun ke-3: Pengembangan Prototipe Kurikulum Berori-
entasi KKNI untuk Meningkatkan Kompetensi Profesional dan Pedagogik Calon Guru Pendidikan Sains [Report competence
grant third year: Prototype development of the science education curriculum based on INQF to enhance the profes-
sionalism and pedagogic for science education pre-service training]. Surabaya: Universitas Negeri Surabaya.
Jatmiko, B., Widodo, W., Martini, & Budiyanto, M. (2014). Buku Prototipe Kurikulum Pendidikan Sains Berbasis INQF [The Book
of prototype of the science education curriculum based on INQF] (1st ed.). Surabaya: Jaudar Press.
Jatmiko, B., Widodo, W., Martini, & Budiyanto, M. (2015). Pembelajaran Fisika Umum Berorientasi KKNI untuk Meningkatkan
Hasil Belajar Pengetahuan dan Hasil Belajar Keterampilan Psikomotor Mahasiswa Program Studi S1 Pendidikan IPA Uni-
versitas Negeri Surabaya [General physics learning based on INQF to increase the LOs in terms of the knowledge and
psychomotor skills]. Proceeding Seminar Nasional Hasil Penelitian dan Pengabdian kepada Masyarakat (pp. 148-157).
Surabaya: LPPM.
Keevy, J. (2013). The National Qualifications Framework in South Africa: 1995 to 2013 International. Journal of Continuing
Education and Lifelong Learning, 6 (1), 19-35.
Klegeris, M. B., & Hurren, H. (2013). Improvement in generic problem-solving abilities of students by use of tutor-less
problem-based learning in a large classroom setting. Life Sciences Education, 12 (1), 73-79.
Krathwohl, D. R., & Anderson, L. W. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy
of educational objectives. New York: Longman.
Krstovic, J., & Cepic, R. (2010). Theoretical-conceptual dilemmas of teacher education: Between Croatian and European
Qualifications Framework. Informatol, 43 (3), 240-245.
McLeod, G. A., Barr, J., & Welch, A. (2015). Best practice for teaching and learning strategies to facilitate student reflection
in pre-registration health professional education: An integrative review. Creative Education, 6, 440-454.
Partnership for 21st Century Skills. (2009). P21 Framework Definitions. P 21. Retrieved 10/02/2016, from http://www.p21.
org/storage/documents/docs/P21_Framework_ Definitions_New_Logo_2015. pdf.
Pretz, J.E., Naples, A., & Sternberg, R.J. (2003). Recognizing, defining, and representing problems. in Davidson & Sternberg
(Eds). The Psychology of Problem Solving. Cambridge: Cambridge University Press.
Senel, E., Ulucan, H., & Adilogullari, I. (2015). The relationship between attitudes towards problem-based learning and mo-
tivated strategies for learning: A research in school of physical education and sport. Anthropologist, 20 (3), 446-456.
Sever, S., & Oguz-Unver, A. (2013). The effective presentation of Problem-based classroom experiments using teaching
strategies that employ video and demonstration methods. Australasian Journal of Educational Technology, 29 (3),
450-463.
Solis, C., Castillo, R., & Undurraga, T. (2013). A qualification framework for the training and certification of labor competen-
cies in Chile. Calidad En La Educación, 39, 237-269.
Stasiūnaitienė, E., & Teresevičienė, M. (2006). Assessment and validation of non-formal and informal learning achieve-
ments in the national qualification system. Profesinis rengimas: tyrimai ir realijos / Vocational Education: Research and
Reality, 12, 64-75.
450
(P. 441-451)
Tanel, Z., & Erol, M. (2008). Effects of problem based learning on instructing magnetism: Analysis of an experimental teach-
ing sequence. Latin-American Journal of Physics Education, 2 (2), 124-136.
Ure, O. B. (2015). Governance for learning outcomes in European policy-making: Qualification Frameworks pushed through
the open method of coordination. International Journal for Research in Vocational Education and Training, 2 (4), 268-
283.
Zabit, M. N. (2010). Problem-based learning on students critical thinking skills in teaching business education in Malaysia:
A literature review. American Journal of Business Education, 3 (6), 19-32.
Received: April 04, 2016 Accepted: July 18, 2016
Budi Jatmiko Professor, Researcher, Universitas Negeri Surabaya (The State

(Corresponding author) University of Surabaya), Surabaya, Indonesia, Jalan Ketintang,
Surabaya 60231
E-mail: bjbjatmiko2@gmail.com
Website: http://www.unesa.ac.id/
Wahono Widodo Ph.D, Researcher, Universitas Negeri Surabaya (The State University
of Surabaya), Surabaya, Indonesia, Jalan Ketintang, Surabaya 60231
E-mail: wahonowidodo@unesa.ac.id
Martini M.Sc, Researcher, Universitas Negeri Surabaya (The State University
E-mail: martini_fik@yahoo.com
Mohammad Budiyanto M.Ed, Researcher, Universitas Negeri Surabaya (The State University
E-mail: mbudiyanto@gmail.com
Iwan Wicaksono M.Ed. Researcher, Universitas Jember (University of Jember),
Jember, Indonesia, Jalan Kalimantan, Jember 68118
E-mail: iwan.wicaksono20@gmail.com
Website: http://www.unej.ac.id/
Paken Pandiangan M.Sc. Researcher, Universitas Terbuka Indonesia (Indonesia Open
University), Indonesia, Jalan Cabe Raya, Jakarta 15418
E-mail: pakenp@ecampus.ut.ac.id
Website: http://www.ut.ac.id/
451
A Learning Exercise
using Simple and Real-
Time Visualization TOOL to
Counter Misconceptions
about Orbitals and
Quantum Numbers
Abstract. Misconception is one of the

Sunyono Sunyono,
most widely researched topics in science
education, including chemical education.
Lisa Tania,
This research aims to apply simple graphi- Andrian Saputra
cal visualization tool named Winplot for
a learning exercise activity and explore
its potency to counter misconceptions
about orbitals and quantum numbers.
Misconceptions that were countered in
this research are the existence of orbitals
in atoms and the relationship of magnetic Introduction
quantum numbers to orbital orientation.
This quasi experimental research using One of the science learning objectives in school is if students can un-
pre-test-post-test control group design was derstand and solve many problems occurred in daily life. Therefore, science
conducted to 43 first-year undergradu- learning is more focused on the understanding of phenomena that lead to
ate students as control class and 45 as contextual-based teaching and learning. Scientific concepts are contextu-
experimental class of chemical education ally introduced to students through phenomena that are generally known
at the University of Lampung. The students’ in everyday life (Pilot & Bulte, 2006; Gilbert, 2006; Miri et al., 2007; De Jong
pre-existing concepts were analyzed using et al., 2013; Ültay, 2015). However, the experiences of everyday life generate
a pre-test instrument and explored further pre-existing concepts about scientific phenomena that can interfere with
using in-depth interview. Then, after im- students understanding of the correct scientific concepts (Taber, 2001; Yip,
plementing a learning exercise activity, the 2001; Uzuntiryaki & Geban 2005). Incorrect pre-existing concepts lead to
conceptual changes were analyzed using persistent misconceptions and cause difficulties in understanding new con-
a post-test instrument. The results showed cepts (Read, 2016). A misconception is a mental representation of a concept
that students in experimental class had that does not correspond to currently held scientific theory. In other words,
significant conceptual changes compared a misconception can be defined as any concept that differs from the com-
to control class. Applying this computer- monly accepted scientific understanding (Nakhleh, 1992). Misconceptions
based strategy is highly recommended to are caused not only by wrong self-interpretation about life phenomena but
guide students in understanding chemical also by incorrect concepts delivered by teachers or textbooks (Tsaparlis &
concepts, especially the topics of orbitals Papaphotis, 2002; Tsaparlis & Papaphotis, 2009; Kalkanis et al., 2003; Stefani
and quantum numbers. & Tsaparlis, 2009).
Key words: learning exercise, first-year Chemistry is a branch of natural science that has many misconceptions
undergraduate, computer-based learning, in its teaching and learning. Misconceptions in chemical education gener-
misconceptions, quantum chemistry. ally arise because chemistry contains many abstract concepts in addition to
complicated calculations. Many students fail to learn chemistry because they
do not have a strong understanding of fundamental chemistry concepts that
are necessary to construct more advanced ones. In addition, they also do not
Sunyono Sunyono, Lisa Tania,
Andrian Saputra understand sub-microscopic concepts for explaining the macroscopic world.
University of Lampung, Indonesia It is difficult to shift the paradigm of students from a macro to a sub-micro
level. They often assume that chemistry is a macro-life science, not a sub-micro
one (Johnstone, 1991; Tsaparlis, 1997; Robinson, 2003). Two of the topics
452
ISSN 1648–3898 A Learning Exercise using Simple and Real-Time Visualization TOOL to Counter
Misconceptions about Orbitals and Quantum Numbers
(P. 452-463)
in chemistry that frequently contain misconceptions are orbitals and quantum numbers. Some misconceptions
of orbitals are that (a) orbitals are trajectories arranged around the nucleus where electrons rotate (Nakiboglu,
2003), (b) an orbital is a fixed energy level where the electron is found (on a Bohr Model) (Müller & Weisner, 2002),
and (c) an orbital is a box that can be filled with electrons (Nakiboglu, 2003). Furthermore, misconceptions about
quantum numbers, as described by Papaphotis & Tsaparlis (Papaphotis & Tsaparlis, 2008), are that the principal
quantum number (n) shows the number of shells, the azimuthal quantum number (l) shows the number of subshells, the
magnetic quantum number (ml) determines the form of the atom in space, and the fourth magnetic quantum number
(ms) determines the spin - whether the electrons are found in pairs or alone. These fatal misconceptions indicate that
students do not have a comprehensive understanding of orbitals and quantum numbers.
An effective strategy to counter these misconceptions is computer-assisted learning. Computer-assisted
chemistry learning is intended to provide more detailed visualizations of chemical processes at a sub-microscopic
level. Jones et al. (2005) explain that learning by graphical visualization can significantly help students in under-
standing the sub-microscopic world. Some studies have been conducted to use computer software to enhance
students’ understanding of orbitals and quantum numbers (Harvey & Gingold, 2008; Trindade & Fiolhais, 2003;
Lang & Kobilnyk, 2009; Chung, 2013; Stewart et al., 2013; Chen et al., 2015; Saputra et al., 2015). For example, Lang
& Kobilnyk (2009) used Second Life to visualize atomic orbitals. Furthermore, Stewart et al. (2013) applied the Maple
program to counter common misconceptions of undergraduate students in General Chemistry courses about
atomic orbitals.
Problem of Research
Some misconceptions in atomic structure and quantum chemistry learning that commonly occurred are the
existence of orbitals and the relationship between orbital shapes and orientations to quantum numbers. Most of
chemistry students confused some questions such as (a) do orbitals really exist in atoms, (b) whether orbital shapes
and orientations, as commonly known in textbook, are experimental observation or theoretical construction,
(c) can the magnetic quantum numbers be freely exchanged for each type of p and d orbitals. In this research, a
learning exercise using simple and real-time visualization tool was used and explores its potency to counter those
misconceptions.
Research Focus
A simple and open-source based application named Winplot, has been widely used in the field of educational
research. Winplot, which was developed by Richard Parris of the Phillips Exeter Academy, New Hampshire (Shech-
ter, 2016), provides an alternative way to easily plot mathematical equations as useful physical interpretations
(Saputra et al., 2015). Some studies have used Winplot to visualize hydrogen atomic orbitals and hybrid orbitals
(Chung, 2013; Saputra et al., 2015, Rhile, 2015). This research will explore the use of Winplot in a learning exercise
activity to counter several misconceptions about orbitals and quantum numbers. The results of this research can
be utilized by educators to overcome chemical misconceptions and encourage the use of ICT for more effective
chemical education.
Most of chemistry students feel worries in learning atomic structure and quantum chemistry because involving
abstract concepts, complicated equation needs imagination skills. This condition potentially raised misconceptions.
This quasi experimental research with pre-test – post-test control group design was intended to identify those
misconceptions, especially existence of orbitals and the link between orbital and quantum numbers that generally
occurred in the first-year chemistry undergraduate student. Moreover, the effectivity of a learning exercise using
Winplot was also analyzed to counter those misconceptions. This treatment was implemented during three weeks
(once a week) and in the last week, a post-test was performed for both classes to evaluate the students’ concept.
All representations of orbital in this paper were visualized by polar coordinate implemented in Winplot, and de-
rived from the square of the angular wave functions by ignored radial distribution function in this derivation. The
453
A Learning Exercise using Simple and Real-Time Visualization TOOL to Counter
ISSN 1648–3898
(P. 452-463)
ignorance of radial function was addressed to simplify the equation and code, so that usable for students’ self-
assignment. This ignorance in orbitals’ visualization produced inaccurate representation as a limitation of Winplot
and proposed visualization in this paper. The usual representation of orbital is an isosurface, a surface of the wave
function with a constant value of ψ or ψ2 (or ψ*ψ), whereas the radial wave functions are the product of linear and
exponential term in distance, which affect isosurface shape (Rhile, 2015). The use of polar graph of the value of the
square of the angular wave function in this paper may lead to misrepresented of boundary surface and distance
of orbital from the nucleus as graph suggests.
Another way to visualize orbital is the construction of the contour surfaces of the constant value of the whole
wave function (Cohen, 1961; Ogryzlo, 1965; Perlmutter-Hayman, 1969; Linnet & Bordass, 1970; Kikuchi & Suzuki,
1985; Peacock-López, 2003). The contour surface will produce a boundary surface as representation of some definite
percentage of the total electronic charge and very potential to generate a better visualization (Perlmutter-Hayman,
1969). However, contour lines method which is applied in some mathematical software, such as mathematica, gnu-
plot, matlab, requires special coding skills. This requirement is very difficult for beginner or first-year undergraduate
student to quick and real-time plotting of orbital in the classroom. Therefore, Winplot with all its shortcomings is
used in these learning activities.
Sample of Research
Sample of research was 98 first-year undergraduate students (consisted of 43 students in control and 45 in
experimental class) of chemical education at the University of Lampung. This purposively selected sample is expected
to have equal pre-existing concepts about orbitals and quantum numbers as learned in their secondary school.
Data regarding their pre-existing concepts were collected by a pre-test instrument in form of open-ended
questions about the existence of orbitals and the link between orbitals and quantum numbers. In this research,
experimental class was applied a learning exercise using Winplot, whereas control class used conventional in-
struction that did not rely on computer aided graphical renderings. Instructor in the control and experimental
class is the same person. Moreover, instructor in the control class is also aware of the students’ misconception as
instructor in the experimental class. All conditions in control and experimental classes are the same, except the
instructional strategy used in learning. To avoid the misconceptions and give the correct concept in control class,
the instructor simply used verbal and drew 2D orbitals in whiteboard without involving students to construct
orbital by students themselves.
Data Analysis

The students’ responses were categorized into various levels of understanding i.e. true understanding, partial
understanding, misconception, and not answered. True understanding is defined as a response that is aligned with
all components of the criteria for a correct answer, which was determined by an expert; partial understanding is
a response that includes at least one component of the criteria for a correct answer but not all; misconception
is a response that includes incorrect information; and not answered is a blank response. Then, the effectivity of
these computer visualization activities was measured by comparing the percentage of students at various levels
of understanding for each question. In-depth interview was conducted with 10 students who had a misconcep-
tion to confirm the reason regarding their responses. Data from the interview was analyzed by classified students’
responses based on the similarities of those answers and described as explained in the manuscript. Categorization
of students’ responses into various levels of understanding used in this research is adapted from Nakiboglu (2003).
Categorization presented in this manuscript was conducted by authors and then, this marking was followed by
triangulation process. Students’ responses marked by authors are validated by 1 expert in chemistry and 2 experts
in chemical education. Moreover, validation result was discussed in focus group discussion with all lecturers in
Peer Group of Chemical Education at University of Lampung. Therefore, validity and reliability of this marking can
be justified scientifically.
454
(P. 452-463)
Result of Research
Result of this research consisted of two parts. The first part is analysis of students’ pre-existing concepts as
shown by the distribution of students’ level of understanding for each question in pre-test (Table 1). The second part
is the percentage of students at misconception after the treatment of a learning exercise as shown by Figure 1.
Table 1 showed students’ pre-existing concepts related to some issues in the topic of orbitals and quantum
numbers. Data of pre-existing concepts were obtained after analysis of students’ answer to 5 questions in a pre-
test instrument and categorized into various level of understanding. These questions asked students to do the
following:
1. describe the meaning of atomic orbitals

2. clarify whether orbital shapes and their orientations are derived from experimental observation (exist
in atom) or purely theoretical construction
3. comment on a discourse about whether the value of the quantum magnetic number (m = -1, 0, +1)
corresponds to the orbital orientation px, py, pz, respectively
4. comment on a discourse about whether each type of p orbital can have any values of the three magnetic
quantum numbers (m = -1, 0, +1)
5. comment on a discourse about whether the quantum magnetic number (m = -2, -1, 0, +1, +2) corre-
sponds to the orbital orientation dx2-y2, dz2, dxy, dxz, dyz, respectively
Table 1. Distribution of students’ level of understanding for each question in pre-test.
Question Control Exp. class

Level of Understanding
No. class (%) (%)
True Understanding 4.65 2.22

•• An orbital is the square of the wave function, which represents the high probability area of finding an 4.65 2.22
electron
Partial Understanding 11.63 11.11
•• An orbital is the wave function and space for an electron 11.63 11.11
Misconception 74.42 74.33

1
•• An orbital is space with a circular or spherical shape 13.95 9.88
•• An orbital is a trajectory for an electron to rotate around the atomic nucleus 41.86 42.22
•• An orbital is the energy level of an atom with various values 6.98 6.67
•• An orbital is a box that can be filled by an electron 11.63 15.56
Not Answered 9.30 13.33
True Understanding 0 0
Partial Understanding 9.30 0
•• Atoms are not visible to the naked eye, so orbital shapes and orientations should not exist in atoms. It 9.30 0
is a theoretical construction
2 23.25 13.33
•• Electrons move in ‘circular orbitals’; if electrons exist in atoms, then so do orbital shapes and orientations.
So, it is an experimental observation
•• The existence of the atoms and electrons are observed by experimental observation such as: Thompson
experiment, Einstein’s photoelectric experiment, etc. Because orbitals are the locations of electrons in 20.93 11.11
an atom, orbital shapes and orientations are also experimental object
455
ISSN 1648–3898
(P. 452-463)
Partial Understanding 0 0
3 •• Generally, the order of the Cartesian axes is x, y, z. Therefore, the numbers are arranged from small to 74.42 71.11
large(m = -1, 0, +1), so the orbitals are as follows: px (m=-1), py(m=0), pz (m=+1)
•• Magnetic quantum numbers can be freely exchanged for each type of p orbital 11.63 15.56

Partial Understanding 25.58 22.22
•• Magnetic quantum numbers cannot be freely exchanged for p orbitals because it is an international
agreement
4 Misconception 62.79 62.22
•• Magnetic quantum numbers can be freely exchanged for p orbitals. Because px, py, pz orbitals have an 41.86 42.22
identical shape, it is not a problem to exchange m= -1, 0, +1 for each orbital
•• There is no specific rule to set m for a p orbital. It is only an agreement of chemists 20.93 20.00
Partial Understanding 0 0
5
•• There is no definite rule to set m for a d orbital. It is only international agreement of chemist. Then, It
9.30 11.11
can be freely exchanged, m= -2, -1, 0, +1 or +2, for each d orbital.
After implementation of a learning exercise using Winplot, the effectivity of this computer-aided learning
was characterized by the percentage of students in control and experimental class who still have misconception
for each post-test question.
Figure 1: Percentage of students at misconception in post-test for each question.
456
(P. 452-463)
Discussion
Students’ Misconceptions about Orbitals and Quantum Numbers
This research was carried out via three main activities: analyzing students’ pre-existing concepts, implementing
a learning exercise activity using Winplot, and identifying conceptual changes. In analyzing of pre-existing concepts,
students were given five questions related to some issues in orbitals and quantum number topic. The first question
in this research sought to describe the meaning of an orbital. As shown in table 1, it can be seen that 74.33% of
students had a misconception about this question. The dominant incorrect answer was that an orbital is a circular
orbit where an electron travels around an atomic nucleus. This answer indicates that students are more confident
in using a solar system-like model (Bohr’s model) when explaining atomic structure. Apparently, these students
believed that Bohr’s model is the most advanced atomic structure theory. The atom, in their mind, is arranged by
layers or ‘shells’ that are perceived as an orbital or place of a moving electron. As revealed by in-depth interviews,
this misconception arises because learning atomic structure theory in secondary school was restricted to Bohr’s
theory; subsequently; they directly jumped to the topic of electron configuration. Students thus assumed that
Bohr’s model is the correct representation of atomic structure and that electron configuration is a different case
that has no contribution to atomic structure. The other main misconception is that an orbital is a round or spherical
space, represents the energy levels of an atom, or is a ‘box’ for filling electrons. The last opinion is quite interesting.
Students reported that an orbital is a box that can be filled by electrons. As explored in the in-depth interviews,
students revealed that this misconception was caused by the habit of their chemistry teacher of presenting the
electron filling process in atomic orbitals using a ‘box’ diagram as demonstrated by figure 2 below:
Figure 2: Orbital presented by ‘box’ diagram.
These responses reveal that the students did not understand the meaning of an orbital in an atom. The sec-
ond analysis surrounds the question about whether information about orbital shapes and their orientations, as
commonly understood from the textbook, is derived from experimental observation or theoretical construction.
Some students in control class (9.30%) answered this question correctly, though their rationale was not accurate.
They said that atoms are not visible to the naked eye, so orbitals should not exist in atoms. Furthermore, approxi-
mately 44.18% students had a misconception of assuming that orbitals can be seen in atoms in the form of balls,
twisted balloons, double twisted balloons, etc. They also assumed that if electrons move in ‘circular orbits’ around
atoms, so do orbitals. Another misconception is atoms and electrons were observed experimentally; therefore, if
an orbital surrounds an electron, an orbital must be an experimental object. According to this answer, the students
did not understand how an orbital is constructed. Students stated that their chemistry teacher had never taught
and explained the wave function and its relationship to orbitals.
The next question asked students to comment on a discourse about whether m = -1, 0, +1 represents the px,
py, pz orientation, respectively, as shown in figure 3.
Figure 3: m-values versus p orbital orientation relationship that the students were asked about in the second
question.
457
ISSN 1648–3898
(P. 452-463)
Results of the analysis demonstrated that most students also have a misconception about this topic as also
supported by in-depth interviews. They argued that px , py , pz should correspond to increasing magnetic quantum
numbers -1,0,+1, respectively. The reason is the alphabetic order of Cartesian axes is x, y, z and the numbers are
arranged from small to large number (-1, 0, +1), then the orbitals should be px (m=-1), py (m=0), pz (m=+1). As ex-
plored in in-depth-interview, they revealed that their high school chemistry teacher always represented p orbitals
and their corresponding m-values using the diagram shown in figure 4.
Figure 4: p orbitals and their corresponding m-values as represented by a high school chemistry teacher.
In addition, another misconception is that m-values (m = -1, 0, +1) did not correspond to specific orientations
of p orbitals. Students argued that each type of p orbitals can have any values of those magnetic quantum numbers.
This error was re-confirmed by the fourth question that asked students to explain whether each type of p orbital
can have any values of the three magnetic quantum numbers (m = -1, 0, +1). As predicted by the author, most
students answered it can do. They argue that the px, py, pz orbitals have identical shapes, so each p orbital can have
all three quantum numbers. Another misconception is that there is no specific rule to set m for a p orbital. Students
expected that the m-values for each type of p orbitals is an International agreement of chemist. This indicated that
students are really confused by the relationship of m and orbital shapes or orientations.
The last question is in line with the fourth question but refers instead to the d orbital. Interestingly, approxi-
mately 88.89% of students in experimental class and 90.70% of students in control class did not answer the question
regarding the relationship of m-values with d orbital orientation. This information indicated that students in general
did not understand d orbital compared to p orbitals. These students stated that their teacher never showed the
relationship of m-values for the d orbital. Moreover, approximately 9.30% of students in control class and 11.11%
of students in experimental class showed misconception by answers that m-values have no specific rules and each
type of d orbital can have any values of the magnetic quantum numbers. They more likely believed that m-values
are only an international agreement, have no definite rule, and are not specific for each type of orbital. By these
pre-existing concept analysis, it is apparent that most students did not have a comprehensive understanding of
orbitals and quantum numbers. This motivates us to use simple and real-time visualization tool named Winplot in
a learning exercise activity as an alternative strategy to counter misconceptions related to these questions.
Learning Exercise Using Winplot
This research is intended to counter student misconceptions about 6 topics that have been explained to them
before. In this computer-based strategy, students were given a worksheet that contained two assignments. First,
they were asked to draw an atomic orbital manually and list the corresponding l and m quantum numbers based
on their pre-existing concepts. The second task contained a Winplot code that represents the mathematical equa-
tion for each l and m quantum number. They were asked to input the code into Winplot, draw the result obtained
and list the corresponding l and m quantum numbers. Furthermore, students were asked to compare their first
and second answers, evaluate the similarities and differences, self-correct their first answer, and explain their new
findings. In the worksheet, students were also asked some explorative questions about the meaning of orbitals,
how they are obtained and their relationship to quantum numbers.
Conceptual Changes

A post-test was conducted to examine students’ conceptual changes after implementing learning exercise
using Winplot. The effectivity of this strategy was indicated by potency to decrease number of students who had
misconception. The results demonstrated dramatic conceptual changes in post-test after the learning exercise activ-
ity using Winplot, as presented by figure 4. According to Figure 4, approximately 34.88% of students in control class
458
(P. 452-463)
still had a misconception of the first question, whereas experimental class showed only 6.67%. After implementing
a learning exercise using Winplot, most of students understand that an orbital is the square of the mathematical
wave-function that gives information about the maximum probability of finding an electron in an atom. Learning
exercise activity for the first question asked students to compare visualization between wave function and square
wave function for px, py, pz. By doing this activity, students will understand that visualizing wave function will never
generate best representation of p orbitals, except visualizing of its square as shown in figure 5.
(a) (b)
Figure 5: Comparison of Visualization Result of (a) Wave function and (b) Square Wave Function of pz Orbital
Generated by Winplot.
Furthermore, 11.11% of students in experimental class displayed a partial understanding, as they noted
that an orbital is the square of the wave-function without considering the physical interpretation. However, this
partial understanding is not actually fatal, as those students would be given additional tutorial and mentoring
to get complete conceptual understanding of orbitals. Significant conceptual changes also occurred in the sec-
ond question. After students found that square wave function produces orbital representation, almost all of the
students in experimental class changed their conception from an experimental to theoretical mindset, only one
student (2.22%) still hold his wrong concept. Compared to experimental class, 39.53% of students in control class
still had misconception. This indicated that constructing orbitals in students’ own project makes them believe that
all orbitals’ information is coming from mathematics. In their exercise activity for the second question, students
were given questions consecutively with simple analogy such as:
Question 1: “a stationary ball (t = 0s) was kicked from A point to B point as shown in figure 6, where it will be
required 5 seconds to arrive in B. Where is the higher probability area in finding a ball at t = 2 second; box I, box
II, or box III?”
Figure 6: An analogy of the existence of orbital.
459
ISSN 1648–3898
(P. 452-463)
Answer 1: “box II”

Question 2: “in your opinion, does this “box” really exist in real life?”
Answer 2: “no way, box is only a representation of the higher probability area in finding a moving ball for
t = 0-5 second”
Question 3: “can the box be modelled using a mathematical equation?”
Answer 3: “yes, it is”
Students were informed that the “box” is analog with orbital and “ball” is electron. Box is only the representa-
tion of the highest probability area in finding a ball, which does not actually exist in real life. Analog with the box,
orbital is the highest probability area in finding electrons in an atom. By visualizing orbital from its wave func-
tions and enriched further by simple analogy, students in experimental class realized that orbital shapes and its
orientations (spherical symmetric, twisted balloon, double twisted balloon, etc.), as commonly understood from
a textbook are only mathematical constructions, do not really exist in the atom. As noted in in-depth interview,
students were never taught how to construct orbitals by themselves and where orbitals come from in secondary
school. Their secondary chemistry teachers only said that atoms are composed of protons and electrons in which
electron stays in a space named orbital. By this argument, students assume that orbitals really exist in atoms and
this leads to misconception.
Questions number 3-5 asked students for the link between orbitals and quantum numbers. Surprisingly, all of
the students in experimental class had true understanding after implementing exercise using Winplot visualization
tool. This indicated that Winplot has high capability in real-time presenting the link between orbitals and quan-
tum numbers. In contrary, students in control class still had many students at misconceptions. There are 20.93%,
25.58%, and 32.56% for question number 3, 4, and 5. In learning exercise activity, students were given angular
function of p orbital with its related quantum numbers as explained in standard physical chemistry textbook that
generally used in our class as shown in table 2. Then, students convert the equations to Winplot code and plotting
it in the classroom. After those treatments, all students agreed that m-values cannot be freely exchanged for each
type of p orbital. They also agreed that m= 0 is only for pz and m= ±1 for both [px, py] because of the mathematical
consequences.
Table 2. Wave function and its related quantum numbers for p orbitals (McQuarrie, 1983).
l m Orbitals Angular Function
0 0 s
1 0 pz
±1 px
±1 py
460
(P. 452-463)
l m Orbitals Angular Function
2 0 dz2
±1 dxz
±1 dyz
±2
±2 dxy
In contrary, almost all students in control class could not understand the relationships of orbitals and quantum
numbers. Number of students who had misconceptions on pre-test are not significantly different in post-test. This
indicated that conventional strategy without the help of computer visualization cannot give a significant change
to construct this concept. Otherwise, after applying learning exercise using Winplot, in line with that, in the case
of the d orbital in question 6, students also understood that m=0 is only for dz2, m= ±1 for both [dxz, dyz], and m= ±2
for both [dxy, dx2-y2] after implementation of learning exercise activity. This strategy is really effective to construct
true conception about orbitals and quantum numbers because it provides students to real-time plotting orbitals
from its origin equation.
Finally, this research determined that applying Winplot in a learning exercise activity is an effective computer-
based strategy in countering misconceptions related to orbitals and quantum numbers. By transforming equations
into 3D visualizations, students obtained important information, such as (a) the shape and orientation of atomic
orbitals that are commonly known from a chemistry textbook are derived from mathematical construction as op-
posed to experimental observation, (b) the relationship between magnetic quantum number and orbital orienta-
tion, and (c) the m-value is specific and cannot be freely exchanged for each type of orbital. Due to its advantages,
Winplot is highly recommended for wide use in learning chemistry at schools or universities.
Conclusions
These findings informed some misconceptions related to orbitals and quantum number topic held by first-
year undergraduate chemistry students and a novel strategy to counter those misconceptions. As described in
students’ responses to pre-test questions, students were confused about how orbitals are constructed and the
relationship between orbitals and m-values. After a learning exercise activity applied, most of the students showed
significant conceptual changes. This research demonstrated that students’ learning exercise using Winplot is an
effective computer-based learning strategy to counter several student misconceptions about orbitals and quan-
tum numbers.
Acknowledgements
Authors declare many thanks to Anonymous Reviewers, who give constructive suggestions to the content
of this manuscipt. Big thanks is also addressed to Mr. Fajar Arrasyid, A Graduate Student of Chemical Education,
Faculty of Teacher Training and Education, University of Lampung, for his contribution in providing high quality
pictures in the manuscript.
461
ISSN 1648–3898
(P. 452-463)
References
Barak, M., Ben-Chaim, D., & Zoller, U. (2007). Purposely teaching for the promotion of higher-order thinking skills: a case of critical
thinking. Research in Science Education, 37 (4), 353-369.
Becker, H. J., & Nguyen, M. Q. (2013). Chemistry teaching and science of education in Germany. Part 2: pupil-orientation. Journal
of Science Education Science Ho Chi Minh City University of Education, 50, 38-45.
Bodner G. M. (1991). I have found you an argument – The conceptual knowledge of beginning chemistry graduate students.
Journal of Chemical Education, 68 (5), 385-388.
Cajas, F. (1999). Public understanding of science: Using technology to enhance school science in everyday life. International
Journal of Science Education, 21 (7), 765-773.
Chen, S. C., Hsiao, M. S., & She, H. C. (2015). The effects of static versus dynamic 3D representations on 10th grade students’ atomic
orbital mental model construction: Evidence from eye movement behaviors. Computers in Human Behavior, 53, 169-180.
Chung, W. C. (2013). Three-dimensional atomic orbital plots in the classroom using winplot. Journal of Chemical Education, 90
(8), 1090-1092.
Cohen, I. (1961). The shape of the 2p and related orbitals. Journal of Chemical Education, 38 (1), 20−22.
Fishler, H., Lichtfeldt, M. (1992). Modern physics and students’ conceptions. International Journal of Science Education, 14 (2),
181-190.
Gilbert, J. K. (2006). On the nature of “context” in chemical education. International Journal of Science Education, 28 (9), 957-976
Harvey, E., & Gingold, C. (2000). Haptic representation of the atom. In Information Visualization, 2000. Proceedings. IEEE Interna-
tional Conference on (pp. 232-235). IEEE.
Johnstone, A. H. (1991). Why is science difficult to learn? things are seldom what they seem. Journal of Computer Assisted Lear-
ning, 7 (2), 75-83.
Jones, L. L., Jordan, K. D., & Stillings, N. A. (2005). Molecular visualization in chemistry education: The role of multidisciplinary
collaboration. Chemical Education Research and Practice, 6 (3), 136-149.
Kalkanis G., Hadzidaki P., Stavrou, D. (2003). An instructional model for a radical conceptual change towards quantum mechanics
concepts. Science Education, 87 (2), 257-280.
Kikuchi, O., Suzuki, K. (1985). Orbital shape representations. Journal of Chemical Education, 62 (3), 206−209.
Lang, A. S., & Kobilnyk, D. (2009). Visualizing atomic orbitals using second life. Journal for Virtual Worlds Research, 2 (1). 1-9.
Linnett, J. W., Bordass, W. T. (1970). A new way of presenting atomic orbitals. Journal of Chemical Education, 47 (10), 672−675.
McQuarrie, D. A. (1983). Quantum chemistry. Canada: University Science Book.
Müller, R., & Wiesner, H. (2002). Teaching quantum mechanics on an introductory level. American Journal Physics, 70 (3), 200-
209.
Nakiboglu, C. (2003). Instructional misconceptions of Turkish prospective chemistry teachers about atomic orbitals and hybrid-
ization. Chemical Education Research and Practice, 4 (2), 171-188.
Nakhleh, M. B. (1992). Why some students don’t learn chemistry; chemical misconceptions. Journal of Chemical Education, 69
(3), 191-196
Ogryzlo, E. A. (1965). On the shapes of f orbitals. Journal of Chemical Education, 42 (3), 150.
Papaphotis, G., & Tsaparlis, G. (2008). Conceptual versus algorithmic learning in high school chemistry: The case of basic quantum
chemical concepts. Part 2. Students’ common errors, misconceptions and difficulties in understanding. Chemical Education
Research and Practice, 9 (4), 332-340.
Peacock-Lopez, E. (2003). On the problem of the exact shape of orbitals. The Chemical Educator, 8 (2), 96−101.
Perlmutter-Hayman, B. (1969). The graphical representation of hydrogen-like wave functions. Journal of Chemical Education, 46
(7), 428−430.
Pilot, A., & Bulte, A. M. (2006). The use of “contexts” as a challenge for the chemistry curriculum: Its successes and the need for
further development and understanding. International Journal of Science Education, 28 (9), 1087-1112.
Read, J. R. (2004). Childrens’ misconceptions and conceptual change in science education. http://acell.chem.usyd.edu.au/
Conceptual-Change.cfm
Reif, F., & Larkin, J. H. (1991). Cognition in scientific and everyday domains: Comparison and learning implications. Journal of
Research in Science Teaching, 28 (9), 733-760.
Rhile, I. J. (2015). Comment on “Visualizing three-dimensional hybrid atomic orbitals using winplot: An application for student
self- instruction”. Journal of Chemical Education, 92 (12), 1973-1974.
Robinson, W. R. (2003). Chemistry problem-solving: Symbol, macro, micro, and process aspects. Journal of Chemical Education,
80 (9), 978.
Saputra, A., Canaval, L. R., Fadiawati, N., Diawati, C., Setyorini, M., Kadaritna, N., & Kadaryanto, B. (2015). Visualizing three-dimen-
sional hybrid atomic orbitals using winplot: An application for student self- instruction. Journal of Chemical Education, 92
(9), 1557-1558.
Shechter, E. Matlab Home Page. http://www.mathworks.com/matlabcentral/fileexchange/44604-plot-hydrogen-atom-molecular-
orbital (accessed January 2016).
Stefani, C., & Tsaparlis, G. (2009). Students’ levels of explanations, models, and misconceptions in basic quantum chemistry: A
phenomenographic study. Journal of Research in Science Teaching, 46 (5), 520-536.
Stewart, B., Hylton, D. J., & Ravi, N. (2013). Using maple to visualize atomic orbitals. Proceedings of The National Conference On
Undergraduate Research (NCUR) 2013 University of Wisconsin La Crosse, pp. 303-308.
462
(P. 452-463)
Taber, K. S. (2001). Building the structural concepts of chemistry: Some considerations from educational research. Chemical
Education Research and Practice, 2 (2), 123-158.
Trindade, J., & Fiolhais, C. (2003). Students’ visualization and conceptual understanding of atomic orbitals using a virtual environ-
ment. In Advanced Learning Technologies, 2003. Proceedings. The 3rd IEEE International Conference on (pp. 298-299). IEEE.
Tsaparlis, G. (1997). Atomic orbitals, molecular orbitals and related concepts: Conceptual difficulties among chemistry students.
Research and Science Education, 27 (2), 271-287.
Tsaparlis, G., Papaphotis, G. (2002). Quantum-chemical concepts: Are they suitable for secondary students, Chemical Education
Research and Practice, 3 (2), 129-144.
Tsaparlis, G., Papaphotis, G. (2008). High-school students’ conceptual difficulties and attempts at conceptual change: The case
of basic quantum chemical concepts. International Journal of Science Education, 31 (7), 1-36.
Ültay, N., Durukan, Ü. G., & Ültay, E. (2015). Evaluation of the effectiveness of conceptual change texts in the REACT strategy. Chemi-
cal Education Research and Practice, 16 (1), 22-38.
Uzuntiryaki, E., & Geban, Ö. (2005). Effect of conceptual change approach accompanied with concept mapping on understand-
ing of solution concepts. Instructional Science, 33 (4), 311-339.
Yip, D. Y. (2001). Promoting the development of a conceptual change model of science instruction in prospective secondary
biology teachers. International Journal of Science Education, 23 (7), 755-770.
Received: June 22, 2016 Accepted: July 30, 2016
Sunyono Sunyono Dr, Senior Lecturer and Head of Quality Assurance of Faculty
of Teacher Training and Education University of Lampung,
Department of Chemical Education, Faculty of Teacher Training and
Education, University of Lampung 35145, Indonesia.
E-mail: sunyono1965@fkip.unila.ac.id
Lisa Tania Master of Science, Lecturer, Department of Chemical Education,
Faculty of Teacher Training and Education, University of Lampung
35145, Indonesia.
E-mail: lisa.tania@fkip.unila.ac.id
Andrian Saputra Master of Science, Lecturer, Department of Chemical Education,
(Corresponding author) Faculty of Teacher Training and Education, University of Lampung
35145, Indonesia.
E-mail: andriansaputra@fkip.unila.ac.id
463
PRESERVICE TEACHERS’
PERCEPTIONS OF THE
RELATIONSHIP BETWEEN
SCIENCE AND PEACE
Abstract. Raising young people aware Canan Dilek Eren

of the need for science for peace is only
possible with modern education that is
based on science and enriches science with
elements of peace. Accordingly, determina-
tion of perceptions of preservice teachers,
who will bear the greatest responsibility
in raising individuals, about the relation-
ship between science and peace is of great
importance. The aim of this research was
to develop a valid and reliable scale to
Introduction
measure preservice teachers’ perceptions
of the relationship between science and
peace. A total of 253 preservice teach- From the time of the naturalists such as Copernicus, Kepler, Galileo
ers, including 88 from the department of and Newton up until today, the desire to dominate nature and obtain its
science teaching, 87 from the department information has been a major goal of science. While providing humans with
of mathematics teaching and 78 from the information about nature, science satisfies their desire to process nature
department of primary school teaching, according to their demands through technology and dominate it. The de-
who were in their 4th year in the educa- sire to dominate nature launched an intense industrialization process with
tion faculty of Kocaeli University, Turkey, Industrial Revolution, which is a result of international competition and race
during the spring term of the academic year to surpass. Although industrial revolution meant a healthy, comfortable life
2014-15, were recruited for this study. As filled with technological toys with astonishing diversity to an unprecedented
a result of the factor analysis, Cronbach’s
level in history for a large population (Mcclellan & Dorn, 2013), its stakes and
alpha (á) reliability coefficient of the scale,
results were high. According to Gül (2013), ‘urbanization and demographic
consisting of four factors, including ‘Com-
munication and Intercultural Dialogue’ (6 challenges triggered by industrialization process led to rapid destruction of
items), “Technological Developments’ (6 nature and disruption of wildlife, impaired the balance of nature in favor of
items), ‘Socio-Economic Developments’ (7 people and caused environmental disasters almost impossible to remedy’
items) and’ Peace Education’ (7 items), and (p.18) . This has posed a threat to human life and social peace required for
26 items, was found to be .93. The Scale a healthy life.
for Perception of the Relationship between As in nature, science has become a tool in the realization of a society’s
Science and Peace (SPSP) is the first unique desire to dominate another society in the historical process. With technologies
scale developed to determine preservice it has provided, it caused wars and other areas of cross-border competition
teachers’ perception of the relationship acquire various dimensions. In the new world order, a new era, where wars
between science and peace.
are called ‘total war’, has begun. In total war, operations are all over the world;
Key words: science and peace, perceptions
there is no limit to war (Wikipedia, 2015). Therefore, wars have become more
of preservice teachers, scale development.
formidable. Nuclear and chemical weapons, missiles and radar tracking
devices are of great importance in terms of both defense and attack, giving
rise to great changes for the implementation of war (Koç, 2013). Innocent
Canan Dilek Eren people lose their lives in non-stop wars in most parts of the world, while
Kocaeli University, Kocaeli, Turkey some countries support the developments in the war industry for the sake
of their economic interests. Developed countries control the majority of
the world’s resources in this regard, while the least developed countries are
464
ISSN 1648–3898 PRESERVICE TEACHERS’ PERCEPTIONS OF THE RELATIONSHIP BETWEEN SCIENCE AND PEACE
(P. 464-476)
faced with an environment lacking basic human needs (Kaypak, 2012). As a result, all these developments have
become a threat to world peace.
Peace manifests itself in individual and social relations, and in our era, have also become universal and glob-
ally shared values (Balcı & Katılmış, 2012). In an environment where there is no peace, all natural and human ele-
ments and vital processes are under threat so presence of national and international peace and preserving it are
of paramount importance. To ensure presence of peace and preserve it, scientific developments that will minimize
such threats in terms of quality and quantity and ensure optimum use of natural resources in order to make them
sustainable are required. Briefly, it is necessary to contribute to peace by making science for peace.
International organizations who believe in the necessity of building peace through science continue their
activities on a global scale. One of them, the United Nations Educational, Scientific and Cultural Organization
(UNESCO) has continued its operations since 1945 in order to contribute to the establishment of global peace
through education, science, culture and information (“Introducing UNESCO,” 2015). “Science for Peace”, which is a
project being carried out to help and pioneer building peace in the light of scientific data was launched in 2009
and it continues its activities in the international platform with over 2500 members (Science for Peace, 2015). Sci-
ence for Peace and Security (SPS) Committee of North Atlantic Treaty Organization (NATO) is the primary NATO
committee that supports practical cooperation in the area of civil science and innovation (“The Science for Peace
and Security Programme,” 2015). This committee also contributes to NATO’s mission by supporting scientific proj-
ects for peace.
With article 26/2 of United Nations Universal Declaration of Human Rights, which says “Education shall promote
understanding, tolerance and friendship among all nations, racial or religious groups, and shall further the activities
of the United Nations for the maintenance of peace.” (“Universal Declaration of Human Rights,” 2015) the importance
of education is referred to for peace and for maintaining peace. Raising young people aware of the need for science
for peace is only possible with modern education that is based on science and enriches science with elements of
peace. Accordingly, determination of perceptions of preservice teachers, who will bear the greatest responsibility
in raising individuals, about the relationship between science and peace is of great importance.
Importance of the Research
In the literature, there are several studies on the concept of science, including perception of the concept of sci-
ence (Muşlu and Macaroğlu, 2006), attitudes towards science (Mıhladız and Duran, 2010; Erdoğan, 2013), preservice
teachers’ perceptions of scientists (Ünver, 2010; Çermik, 2013; Şenel and Aslan, 2014). Studies on the concept of
peace include attitudes towards peace issues (Aktaş and Safran, 2013), influence of the school in raising awareness
about peace (Joseph, 2012), perception of peace concept (Katılmış et al., 2011; İmamoğlu and Bayraktar, 2014),
creating peace based programs (Ghaderi, 2011), peace education (Gültekin and Gültekin, 2013; Paderanga, 2014)
and global citizenship education for world peace and security (Farahani, 2014). Although science and peace were
individually addressed and studied, there’s no study in the literature that investigated perception of the relationship
between science and peace. Thus, this scale, which can determine preservice teachers’ perceptions of the relation-
ship between science and peace, is the first in the area, and it is expected to contribute to the literature in terms
of its effort to reveal preservice teachers’ perceptions of the relationship between science and peace.
Aim of Research
In this context, the main aim of research is to develop an instrument for measuring preservice teachers’
perceptions of the relationship between science and peace. In the literature, there’s no report of a scale that can
measure individuals’ perceptions of the relationship between science and peace. Therefore, development of “the
Scale for Perception of the Relationship between Science and Peace” applicable to teachers is expected to bridge
an important gap in the area. Determining preservice teachers’ perceptions of the relationship between science
and peace will provide guidance to experts of the area and educators.
A mixed-methods sequential exploratory design was used to develop and test the reliability and validity of The
Scale for Perception of the Relationship between Science and Peace (SPSP). The mixed-methods sequential exploratory
465
PRESERVICE TEACHERS’ PERCEPTIONS OF THE RELATIONSHIP BETWEEN SCIENCE AND PEACE
(P. 464-476)
ISSN 1648–3898
design consists of two distinct phases: qualitative followed by quantitative data collection and analysis (Creswell
et al. 2003). According to Cresswell (2012), this design is useful when developing and testing a new instrument.
Based upon this design, in the first phase, qualitative methods (document analysis and expert views) were used
to generate an item pool. In the second phase, quantitative methods (factor analysis and reliability) were used to
evaluate the construct validity and reliability of SPSP.
Sample
The research sample consisted of a total of 253 preservice teachers, including 88 from the department of sci-
ence teaching, 87 from the department of mathematics teaching and 78 from the department of primary school
teaching, who were in their 4th year during the spring term of the academic year 2014-15. Comrey and Lee (1992)
offered a rough rating scale for adequate sample sizes in factor analysis: 100 = poor, 200 =fair, 300 = good. Informa-
tion about the participants, including gender, department and type of study, is given in Table 1.
Table1. Data relating to the participants of the research.
Female Male
DEPARTMENT TOTAL
n % n %
Science Teaching, daytime education 38 15.02 5 1.98 43

Science Teaching, evening education 41 16.21 4 1.58 45
Mathematics Teaching, daytime education 40 15.81 11 4.35 51
Mathematics Teaching, evening education 25 9.88 11 4.35 36
Primary School Teaching, daytime education 33 13.04 7 2.77 40
Primary School Teaching, evening education 33 13.04 5 1.98 38
Total 210 83.00 43 17.00 253
Data Collection Tools
The scale for perception of the relationship between science and peace (SPSP) was developed to determine
preservice teachers’ perceptions of the relationship between science and peace (The scale in annex). For this purpose,
a literature review was performed and theoretical framework was determined. An item pool was generated in the
light of accessible information and texts obtained from preservice teachers’ answers to open-ended questions.
During the preparation stage of the guidelines, opinions of the two assessment and evaluation experts and
the three field experts were obtained to measure the perception of the relationship between science and peace, and
a 5-point Likert scale was developed. Guidelines and publication of the scale were performed accordingly. Rating
and scoring of the items were as follows: “I strongly disagree (1)”, “I disagree (2)”, “I neither agree nor disagree (3)”, “I
Agree (4)”, “I strongly agree (5)”. For negative statements, completely opposite of the abovementioned rating was
performed. High score obtained from the scale indicates that preservice teachers’ perceptions of the relationship
between science and peace is positive.
Attention was paid to ensuring that the scale items are simple, plain and straightforward. After the scale items
were prepared, 2 experts on Turkish language teaching were consulted to check if the sentences are clear and bear
a single meaning in terms of their compliance with grammar and clarity. In line with the opinions of these experts,
the sentences in the items were made compliant with Turkish language.
The items in draft form of the scale were reviewed by 3 experts from the department of science teaching. They
were asked to assess each of the items in terms of whether the items in the draft form are in compliance with the
subject they’re intended to measure, indicate their suggestions on the relevant items and print their suggestions
for new items, if any. In accordance with the experts’ opinions, necessary corrections were made and the scale
reached its final form for a trial.
Draft SPSP was implemented on 15 preservice teachers from the department of science teaching in order
to receive feedback on the clarity and implementation period of the items. Those who completed SPSP were in-
466
(P. 464-476)
terviewed about what they think about the clarity of the scale items and adequacy of the time for answering the
questions. Minor adjustments were made in the structure of sentences only in 2 items and the scale became ready
to be implemented on a large group.
Implementation was carried out on the specified study group. Reliability and validity analyses were performed
with the obtained data.
Data Analysis
The data obtained in the research were analysed using SPSS 11.5 and Lisrel 8.7 software packages. Exploratory
factor analysis (EFA) and confirmatory factor analysis (CFA) were performed for construct validity of SPSP. In this
way, it was assessed to what extent the resultant four-factor-measurement-model complies with the collected data.
Within the scope of reliability studies, item-total correlation coefficients and Cronbach’s alpha internal consistency
coefficients were calculated.
Results of Research
Exploratory Factor Analysis
Exploratory factor analysis was performed in order to reveal factor pattern of the measurement tool with 57
items intended to measure preservice teachers’ perceptions about the relationship between science and peace.
Before the analysis, Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy test (Kaiser, 1974) was applied
to test if sample size is suitable for factorization. At the end of the analysis, KMO value was determined to be 0.94.
In line with this finding, it was concluded that the sample size is ‘’ perfect ‘’ for doing a factor analysis (Tabachnick
& Fidell, 2007). In addition, Bartlett sphericity test (Bartlett, 1954) was performed in order to test whether the data
has a multivariate normal distribution. The chi-square ( χ2 ) value obtained from the analysis was found to be
significant ( χ2(253) = 4628.312 ; p< .01 ). According to the results shown in Table 2, data were assumed to have
a multivariate normal distribution.
Table 2. KMO and Bartlett Test Analysis Results for the Scale for Perception of the Relationship between
Science and Peace.
Kaiser-Meyer-Olkin Measure of Sampling Adequacy 0.947
Bartlett’s Sphericity Test Approx. Chi-Square 4628.312

df 595
Sig. 0.0001
Principal component method has been selected as the factorization method to reveal the factor pattern
of SPSP. Where it is difficult to decide to which component the items belong to, direct oblimin was used as the
rotation method to find an easier to read or more appropriate factor structure without disrupting the explained
variance (Büyüköztürk, 2004). As a result of the analysis, it was seen that there are four factors with an eigenvalue
greater than 1 for 57 items.
Factor load is a variable which shows to what extent the variables, in combination with other variables in
the respective factor, measure the same factor. In factor analysis, factor loads of the variables should be minimum
0.30, while 0.40 and higher values are also preferred in general (Floyd & Widaman, 1995). Based on this opinion,
acceptable factor load value was specified as .50. Items in which the difference of loading in more than one factor
is less than 0.20 were removed from the scale. These processes were repeated until obtaining a suitable structure
with 4 factors and 26 items. Load values of these items are in the range of .50 to .80.
In view of the scree plot in Figure 1, it is noteworthy that the point where the curve shows a rapid decline is
the point where the fourth factor is located. As a result, it was concluded that SPSP should have 4 factors.
467
(P. 464-476)
ISSN 1648–3898
S c ree P lot
14
12
10
Eigenvalue
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35
C omponent Number
Figure 1. Scree Plot for SPSP.
Important values related to 4 factors obtained as a result of factor analysis are shown in Table 3. Table 3 proves
that factor loads of the items are well above the acceptable values. In addition, the contribution of factors to the
total variance (%) are 39.9, 7.1, 3.8 and 3.4 for the 1st, 2nd, 3rd and 4th factors, respectively. Total contribution of the
established 4 factors to the variance is 54.3%. A high variance percentage means that the Scale for Perception of
the Relationship between Science and Peace has a strong factor structure.
Table 3. Principal Component Analysis.
Factors ITEM F1 F2 F3 F4 Communality (covariance)
Factor 1 9 0.799 0.606
7 0.721 0.607
14 0.593 0.482
2 0.563 0.479
16 0.521 0.582
29 0.506 0.385
Factor 2 27 0.809 0.623
26 0.782 0.731
31 0.731 0.541
25 0.701 0.650
12 0.701 0.642
39 0.632 0.536
468
(P. 464-476)
Factors ITEM F1 F2 F3 F4 Communality (covariance)
Factor 3 19 -0.780 0.565
21 -0.717 0.598
17 -0.614 0.478
36 -0.569 0.541
50 -0.564 0.480
56 -0.518 0.471
53 -0.512 0.542
Factor 4 45 0.694 0.492
40 0.613 0.429
57 0.548 0.495
43 0.548 0.494
55 0.526 0.587
54 0.509 0.632
42 0.507 0.629
Eigenvalue 13.974 2.488 1.344 1.200
Variance Explained (%) 39.927 7.107 3.841 3.429
Total Variance Explained (%) 54.304
Cronbach’s a (%) 82.7 86.5 84.3 83.8
Total Reliability Cronbach’s á (%) 93.2
It is apparent from Table 3 that the rates by which each item explains the variance in the common factor
together (covariance) ranges between 0.38 and 0.73. This also shows that common factor variances of the items
are generally high.
As a result of the factor analysis, variables that are grouped under 4 factors have been identified. Information
on these factors is as follows.
The first factor consists of 6 items, including item 9, 7, 14, 2, 16 and 29. Eigenvalue of these items is 13.97, with
their factor load values ranging from 0.50 to 0.79. This factor includes clauses about communication and intercul-
tural dialogue. Therefore, it was deemed appropriate to designate this factor ‘Communication and intercultural
dialogue’. One of the items in this factor is ‘Item 14: Developments in communication technology override borders
in the virtual environment so science increases intercultural communication and contributes to world peace.
The second factor consists of 6 items, including item 27, 26, 31, 25, 12 and 39. Eigenvalue of these items is
2.48, with their factor load values ranging from 0.63 to 0.80. This factor includes clauses about technological devel-
opments and it was therefore designated ‘technological developments’. One of the items in this factor is ‘Item 25:
Technological developments increase economic imbalance between countries with different levels of development
so science is an obstacle to communities living in peace”.
The third factor consists of 7 items, including item 19, 21, 17, 36, 50, 56 and 53. Eigenvalue of these items is
469
(P. 464-476)
ISSN 1648–3898
1.34, with their factor load values ranging from 0.51 to 0.78. The items in this factor comprise clauses related to
socio-economic developments so it was deemed appropriate to designate this factor ‘socio-economic develop-
ments’. One of such clauses in this factor is ‘ item 17: Science prevents water wars and contributes to peace by
providing solutions to diminishing water resources”.
The fourth factor consists of 7 items, including item 45, 40, 57, 43, 55, 54 and 42. Eigenvalue of these items
is 1.2, with their factor load values ranging from 0.50 to 0.69. The designation ‘peace education’ was thought to
describe the factor in the most appropriate way. One of the items in this factor, namely ‘item 43: Science helps
people understand and become aware of natural phenomena, causing them to stay away from war.” provides a
better understanding of the designation of the factor.
Confirmatory Factor Analysis
Confirmatory factor analysis was used to test the accuracy of the model consisting of 26 items obtained by
exploratory factor analysis. Confirmatory factor analysis was conducted using Lisrel 8.7 software package.
Suggestions for modifying the model were examined and the modification which could contribute to x2 val-
ues was found to be between item 12 (scientific developments cause wars by increasing raw material and market
demand so it cannot provide a contribution to peace) and item 25 (Technological developments increase economic
imbalance between countries with different levels of development so science is an obstacle to communities living
in peace). After such modification was made and the analysis was repeated, the modification made a significant
contribution to chi-square (x2). Such modification was also shown on the model.
Matsunaga (2010) suggests that the results of confirmatory factor analysis should be evaluated in the light
of chi-square ( ) goodness of fit value, root mean square error of approximation (RMSEA), comparative fit index
(CFI) and standardized root mean square residual (SRMR). Büyüköztürk et al. (2012) states that chi-square fit index
is not a statistic evaluated on its own and suggests that chi-square value should be evaluated by dividing it by the
degree of freedom (df ). Following the modification, when fit indices of the measurement model was examined, p
value related to value ( ( 292) = 1231.64 ) was found to be significant( p<.01). When the model was evaluated with
respect to the ratio of chi-square to degree of freedom ( / sd =4.21), the fit was found to be at an acceptable level
according to the result obtained (Marsh & Hocevar, 1985). Based on the results obtained, it was confirmed that the
scale for perception of the relationship between science and peace has a 4-dimensional structure.
Figure-2 presents the resultant path diagram from confirmatory factor analysis of these four factors. Each
path shows the weight or load of representation of latent variable in observed variable (Çokluk et al, 2012). As il-
lustrated in the figure, t values regarding the explanation of observed variables by latent variables are significant
at the .01 level.
470
(P. 464-476)
Figure 2. The path diagram of SPSP.
471
(P. 464-476)
ISSN 1648–3898
Reliability
Individual Cronbach’s alpha reliability coefficients were calculated for each subscale and the whole scale in
order to demonstrate internal consistency of the scale, developed to determine preservice teachers’ perceptions
of the relationship between science and peace. As a result of the statistical analyses performed to question the
consistency of the scale, the resulting Cronbach’s alpha (α) reliability coefficient was .93. In addition, Cronbach’s
alpha (α) reliability coefficients of the subscales of SPSP were .82, .86, .84 and .83 for 1st, 2nd, 3rd and 4th subscales,
respectively. The corrected item-total correlations calculated for each factor to determine to what extent the scale
items distinguish preservice teachers in terms of their perceptions of the relationship between science and peace
ranged from .45 to .70 (Table 4).
Table 4. Results of Item Analysis of SPSP.
Corrected item- Corrected item-total

Factor items Factor items
total correlation correlation
Communication and intercultural dialogue Socio-economic developments

2 .57 17 .58
7 .59 19 .53
9 .54 21 .61
14 .57 36 .61
16 .67 50 .59
29 .51 53 .67
Technological developments 56 .61
12 .60 Peace education
25 .63 40 .48
26 .60 42 .70
27 .45 43 .51
31 .49 45 .46
39 .55 54 .66
55 .63
57 .53
Discussion
The aim of this research was to develop a valid and reliable scale to measure preservice teachers’ perceptions
of the relationship between science and peace. Exploratory and Confirmatory factor analyses were used to evalu-
ate the construct validity. Reliability of the scale was tested via Cronbach’s Alpha and The corrected item-total
correlation.
Exploratory factor analysis was performed in order to reveal factor pattern of the measurement tool with 57
items intended to measure preservice teachers’ perceptions about the relationship between science and peace.
As a result of the analysis, a suitable structure with 4 factors and 26 items was obtained. Load values of these items
are in the range of .50 to .80. A factor load of 0.50 and higher is considered very good value (Streiner, 1994; Costello
& Osborne, 2005). Total contribution of the established 4 factors to the variance is 54.3%. An explained variance
less than 60% is generally regarded as unsatisfactory (Hair, Anderson, Tatham & Black, 1998), however, this ratio
is considered satisfactory for social sciences (Msweli & Wushe, 2014). A high variance percentage means that the
Scale for Perception of the Relationship between Science and Peace has a strong factor structure.
Confirmatory factor analysis was used to test the accuracy of the model consisting of 26 items obtained by
exploratory factor analysis. The RMSEA value was .10, which demonstrates that the fit is mediocre (Hu & Bentler, 1999).
When conformity indexes were examined, CFI value was .95, which indicates a good fit (Kline, 2011; Hu & Bentler,
1999). SRMR, which is also another fit index, was .08, which is another finding that demonstrates the good fit (Hu
472
(P. 464-476)
& Bentler, 1999; Matsunaga, 2010; Worthington & Whittaker, 2006). Based on the results obtained, it was confirmed
that the scale for perception of the relationship between science and peace has a 4-dimensional structure.
Individual Cronbach’s alpha reliability coefficients were calculated for each subscale and the whole scale in
order to demonstrate internal consistency of the scale. As a result of the statistical analyses performed to question
the consistency of the scale, the resulting Cronbach’s alpha (α) reliability coefficient was .93. Nunnally (1967) argues
that if reliability of a scale depending on alpha (α) coefficient is .80 ≤ α < .100, the scale is a highly reliable scale. In
this case, the items in SPSP can be said to be consistent with each other and show the same characteristic.
The corrected item-total correlations calculated for each factor to determine to what extent the scale items
distinguish preservice teachers in terms of their perceptions of the relationship between science and peace
ranged from .45 to .70. According to Büyüköztürk (2006), the items with item-total correlation of .30 and higher
distinguish individuals well. According to all these results, SPSP is reliable to the extent acceptable for research in
social sciences.
Findings of validity and reliability studies indicated that The Scale for Perception of the Relationship between
Science and Peace (SPSP) sets a valid and reliable tool for determining preservice teachers’ perceptions of the rela-
tionship between science and peace.
Conclusions
In this research, a scale to determine preservice teachers’ perceptions of the relationship between science
and peace was developed. The scale is a four-factor scale. The four-factor structure determined by EFA was also
examined by CFA and the calculated fit statistics were found to satisfy the criteria values specified in the method.
This result was interpreted as that the scale has a good model-data fit.
A reliable 26-item scale with a Cronbach’s alpha (α) reliability coefficient of .93, consisting of four factors in total,
was obtained. The resulting factors were designated as communication and intercultural dialogue, technological
developments, socio-economic developments and peace education.
The Scale for Perception of the Relationship between Science and Peace (SPSP) allows determination of pre-
service teachers’ perceptions of the relationship between science and peace. It may also be adapted for primary
and secondary school students and their perceptions of the relationship between science and peace can be deter-
mined. Using this scale, a comprehensive study that will determine perceptions of preservice teachers studying at
all education faculties across Turkey about the relationship between science and peace can be conducted.
The results of this study demonstrated that this scale is a valid and reliable scale and it’s the first scale devel-
oped to determine preservice teachers’ perceptions of the relationship between science and peace.
References
Aktaş, Ö., & Safran, M. (2013). An evaluation of the views of secondary education students over the concepts of war and peace
in terms of different variables. International Periodical For The Languages, Literature and History of Turkish or Turkic, 8 (8),
1587-1619.
Balcı, A., & Katılmış, A. (2012). Analysis of perceptions of geography teacher candidates’ about the united nations permanent
member states’ role in peacemaking. Marmara Geographical Review, 25, 390–408.
Bartlett, M.S. (1954). A note on the multiplying factors for various chi square approximations. Journal of the Royal Statistical
Society, 16 (Series B), 296–298.
Büyüköztürk, S. (2004). Veri analizi el kitabı. [Data Analysis Manual.] Ankara: PegemA publishing.
Büyüköztürk. S. (2006). Sosyal bilimler için veri analizi el kitabı. [Data analysis handbook for social sciences.] Ankara: PegemA
publishing.
Büyüköztürk, S., Çokluk, O., & Köklü, N. (2012). Sosyal bilimler için istatistik. [Statistics for social sciences.] Ankara: PegemA pub-
lishing.
Comrey, A. L., & Lee, H. B. (1992). A first course in factor analysis. Hillsdale, NJ: Erlbaum.
Çermik, H. (2013). A scientist created in the picture that pre-service teachers have in their minds. Pamukkale University Journal
of Education, 33 (1), 139-153.
Çokluk, O., Sekercioglu, G., & Büyüköztürk, S. (2012). Sosyal bilimler için çok değişkenli istatistik SPSS ve LISREL uygulamaları. [Mul-
tivariate statistics for the social sciences SPSS and LISREL applications.] Ankara: PegemA publishing.
Costello A.B. & Osborne J.W. (2005). Best practices in exploratory factor analysis: Four recommendations for getting the most
from your analysis. Practical Assessment Research & Evaluation, 10 (7), 1-9.
Creswell, J. W., Plano Clark, V. L., Gutmann, M., & Hanson, W. (2003). Advanced mixed methods research designs. In A. Tashakkori
& C. Teddlie (Eds.), Handbook of mixedmethods in social & behavioral research. Thousand Oaks, CA: Sage.
473
(P. 464-476)
ISSN 1648–3898
Creswell, J. W. (2012). Educational research: Planning, conducting, and evaluating quantitative and qualitative approaches to
research. Boston: Pearson
Erdoğan, S. (2013). Gifted girls’ scientific attitudes and images of scientists. Hasan Ali Yücel University Journal of Education Faculty,
19 (1), 125-142.
Farahani, M.F. (2014). The role of global citizenship education in world peace and security. Procedia - Social and Behavioral Sci-
ences, 116 (2014), 934–938.
Floyd, F.J., & Widaman, K.F. (1995). Factor analysis in the development and refinement of clinical assessment instruments. Psy-
chological Assessment, 7 (3), 286-299.
Ghaderi, M. (2011). Peace-based curriculum based on the theories of “difference” and “similarity”. Procedia Social and Behavioral
Sciences, 15 (2011), 3430–3440.
Gül, F. (2013). Environmental problems and philosophy in the context of human-nature relationship. Pamukkale University Journal
of Social Sciences Institute, 14, 17-21.
Gültekin, B., & Gültekin, T. (2013). Communication models and sensitivity approaches for intercultural peace education. Procedia
- Social and Behavioral Sciences, 106 (2013), 641–650.
Hair, J.F., Tatham, R.L., Anderson, R.E., & Black, W.C. (1998). Multivariate data analysis, New Jersey: Prentice Hall.
Hu, L-T., & Bentler, P.M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new
alternatives. Structural equation modeling: A Multidisciplinary Journal, 6 (1), 1–55.
Introducing UNESCO. (2015). Retrieved from http://en.unesco.org/about-us/introducing-unesco
İmamoğlu, H.V., & Bayraktar, G. (2014). Comparison of Turkish and Hungarian students’ perceptions towards Peace. Karabük
University Journal of Social Sciences Institute, 4 (1), 20-44.
Joseph, J. (2012). How the schooling environment shapes the consciousness of scholars towards peace and war. Procedia - Social
and Behavioral Sciences, 55, 697 – 706.
Kaiser, H. (1974). An index of factorial simplicity. Psychometrika, 39, 31–36.
Katılmış, A., Kop, Y., & Çelik, H. (2011, May 7). 5th grade students’ perceptions of peace concept as a merit: A case study con-
ducted in Istanbul, Kars and Sakarya. Paper presented at the 10th Classroom Teaching Education Symposium, Cumhuriyet
University.
Kaypak, Ş. (2012). An ethical approach to environmental peace. The International Journal of Economic and Social Research, 8 (1),
1-30.
Kline, R.B. (2011) Principles and practice of structural equation modeling. New York: The Guilford Press.
Koç, E. (2013). Human-Being responsibilty (Ethic conscience) in the era of science and technology. Atatürk University Journal of
Social Sciences Institute, 17 (2), 1-13.
Marsh, H. W., & Hocevar, D. (1985). Application of confirmatory factor analysis to the study of self-concept: First-and higher-order
factor models and their invariance across groups. Psychological Bulletin, 97 (3), 562-582.
Matsunaga, M. (2010). How to factor-analyze your data right: Do’s, don’ts, and how-to’s. International Journal of Psychological
Research, 3(1), 97-110.
Mcclellan, J. E., & Dorn, H. (2013). Science and technology in world history (H.Yalçın, Trans.). Ankara: Arkadaş yayınevi. (Original
work published 1999)
Mıhladız, G., & Duran, M. (2010). Investigation of primary school students’ attitudes towards science in terms of demographic
variables. Mehmet Akif Ersoy University Journal of Education Faculty, 10 (20), 100 – 121.
Msweli, P., & Wushe, T. (2014). An analysis of corporate community engagement in Zimbabwe. Environmental Economics, 5 (4),
44-51.
Muşlu, G., & Macaroğlu, E. (2006). Elementary school students’ perceptions of science and scientific processes: A Qualitative Study.
Educational Sciences: Theory & Practice, 6 (1), 225-229.
Nunnally, J. C. (1967). Psychometric theory. New York: Mc -Graw Hill.
Paderanga, L.D. (2014). Classroom video conferencing: Its contribution to peace education. Procedia - Social and Behavioral Sci-
ences, 123 (2014), 113–121.
Science for Peace (n.d.). Retrieved June 7, 2015, from http://scienceforpeace.ca/
The Science for Peace and Security Programme. (2015). Retrieved from http://www.nato.int/cps/en/natolive/78209.htm
Şenel, T., & Aslan, O. (2014). The metaphoric perceptions of preservice early childhood teachers conceptions on science and
scientist. Mersin University Journal of the Faculty of Education, 10 (2), 76-95.
Streiner, D. L. (1994). Figuring out factors: The use and misuse of factor analysis. Canadian Journal of Psychiatry, 39 (3), 135-140.
Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics. Boston: Allyn and Bacon.
Universal Declaration of Human Rights. (2015). Retrieved from http://www.unicef.org/turkey/udhr/_gi17.html
Ünver, A. (2010). Perceptions of Scientists: A comparative study of fifth graders and fourth year student teachers. Necatibey
Faculty of Education Electronic Journal of Science and Mathematics Education, 4 (1),11-28.
Wikipedia (n.d.). Retrieved June 2, 2015, from https://tr.wikipedia.org/wiki/total war
Worthington, R.L., & Whittaker, T.A.(2006). Scale development research: A content analysis and recommendations for best prac-
tices. The Counseling Psychologist, 34 (6), 806-838.
474
(P. 464-476)
APPENDIX
SEX: DEPARTMENT:
THE SCALE FOR PERCEPTION OF THE RELATIONSHIP BETWEEN SCIENCE AND PEACE (SPSP)
I strongly disagree
I strongly agree
Dear Pre-Service Teachers,
I am neutral
I disagree
Please indicate your level of agreement with the following expressions by putting a cross (X) in the
I agree
boxes. Thank you for your contribution to my study with your responses.
Dr. Canan DİLEK EREN
Science is a universal language. Therefore, it is the most appropriate approach for keeping the peace in 1. 1 2 3 4 5
the world.
Science leads to peace through progress in communication technologies and providing inter-cultural 2. 1 2 3 4 5
communication.
Science makes contribution to peace by bringing together people from various languages, religions, and 3. 1 2 3 4 5
races.
Scientific developments cannot contribute to peace since they increase the need for raw materials and 4. 1 2 3 4 5
markets, thereby causing wars.
Science contributes to the world peace by increasing intercultural communication as the developments in 5. 1 2 3 4 5
communication technologies invalidate the boundaries in the virtual environments.
Science education contributes to solving social problems peacefully by improving the critical thinking skills 6. 1 2 3 4 5
of individuals.
Science contributes to peace through generating solutions for depleted water resources, thereby prevent- 7. 1 2 3 4 5
ing water wars.
Science creates a peace environment by eliminating the poverty through products to be used in agricul- 8. 1 2 3 4 5
ture.
Scientific developments contribute to peace by minimizing the natural resources competition among 9. 1 2 3 4 5
communities.
Science prevents communities from living in peace by increasing the economic inequality between different 10. 1 2 3 4 5
countries with different levels of development through technological advances.
Science prevents communities from living in peace by increasing the developmental inequality between 11. 1 2 3 4 5
different countries through developments in communication technologies.
Science prevents communities from living in peace by increasing wars through advances in war technol- 12. 1 2 3 4 5
ogy.
Science education contributes to peace by positively improving individuals’ perspectives towards human 13. 1 2 3 4 5
rights.
Scientific developments lead to wars for healthy living spaces by causing environmental pollution. 14. 1 2 3 4 5
Science creates a peace environment through the products it develops to meet basic needs (e.g. hunger, 15. 1 2 3 4 5
thirst, accommodation).
Science cannot contribute to peace through technological advancements by facilitating people’s lives while 16. 1 2 3 4 5
creating threats against their health.
Science education makes a difference for individuals to be able to evaluate international peace strategies 17. 1 2 3 4 5
by informing them of the developments in the science history.
Individuals’ being informed about scientific developments leads them to behave consciously and thus 18. 1 2 3 4 5
makes a contribution to the world peace.
Science enables people to understand natural events and raises their awareness, which keeps them 19. 1 2 3 4 5
away from war.
Science education makes a difference for individuals to evaluate national peace strategies by equipping 20. 1 2 3 4 5
them with the developments in the science history.
Countries are informed about the developments in the world thanks to science. Peace environment is 21. 1 2 3 4 5
created as the developmental differences between the countries are reduced.
475
(P. 464-476)
ISSN 1648–3898
SEX: DEPARTMENT:
THE SCALE FOR PERCEPTION OF THE RELATIONSHIP BETWEEN SCIENCE AND PEACE (SPSP)
I strongly disagree
I strongly agree
Dear Pre-Service Teachers,
I am neutral
I disagree
Please indicate your level of agreement with the following expressions by putting a cross (X) in the
I agree
boxes. Thank you for your contribution to my study with your responses.
Dr. Canan DİLEK EREN
Science creates peace environment by eliminating the poverty through the industrial products it devel- 22. 1 2 3 4 5
ops.
Science education allows people to come up with peaceful solutions by leading them to adopt a critical 23. 1 2 3 4 5
approach towards what is happening around them.
Science education allows producing peaceful solutions by improving the problem solving skills of individu- 24. 1 2 3 4 5
als.
Science contributes to peace by preventing energy wars through the solutions it develops for depleted 25. 1 2 3 4 5
energy resources.
Science education leads to social peace by making individuals aware of the importance of collaboration- 26. 1 2 3 4 5
based working for scientific development.
Received: May 10, 2016 Accepted: August 02, 2016
Canan Dilek Eren PhD., Assistant Professor, Education Faculty, Department of

Science Education, Kocaeli University, 41380 Kocaeli, Turkey
E-mail: canandilek@gmail.com
Website: http://akademikpersonel.kocaeli.edu.tr/canandilek/
476
INTEGRATING STEM IN AN
ENGINEERING DESIGN
PROCESS: THE LEARNING
EXPERIENCE OF RURAL
SECONDARY SCHOOL
STUDENTS IN AN OUTREACH
CHALLENGE PROGRAM
Abstract. This research was conducted to
evaluate the learning experience of Grade
Ten students from two Malaysian rural
secondary schools that adopted the inte-
gration of STEM in an Engineering Design
Nyet Moi Siew, Process (STEM-EDP) approach vis-á-vis an
outreach challenge program. A total of 89
Henry Goh,
students undertook a ten hour program
Fauziah Sulaiman which engaged them in designing and
building three different prototypes as well
as answering higher order thinking ques-
tions. Data on students’ learning experience
were captured through participants’ re-
sponses to a six-point Likert scale question-
naire, teachers’ field notes, and open-ended
questions. The questionnaire result reveals
Introduction statistically significant gains in knowledge
or skills about, attitudes toward, and
The demand for a science, technology, engineering and mathematics practices on STEM. The STEM-EDP outreach
(STEM) driven workforce in Malaysia has become a burgeoning need as the challenge program brought awareness
economy has evolved from a production-based economy to a knowledge- to rural school students of their potential
based economy. By the year 2020, it has been estimated that Malaysia will as problem solvers, thinkers, creators,
be in need of 500,000 skilled STEM workers (Academy of Sciences Malaysia, and collaborators. Students were able to
2015). Undeniably the supply of STEM related workforce is highly dependent simultaneously broaden their boundaries in
on new entrants into STEM related programmes in upper secondary as well knowledge and competency even though
as tertiary level. However, research has shown that only a total of 45% of they experienced difficulties in tackling
students have enrolled in science stream, and technical and vocational sec- challenges associated with STEM activities.
ondary school classes in 2014, which is still far from the ideal ratio of 60:40 Findings suggested that the STEM-EDP
Science/Technical: Arts Policy set in 1970 (Yong & Phang, 2015; Ministry of approach can be applied as a means for
Education Malaysia, 2014). fostering creativity, problem solving skills,
The challenge of achieving the 60:40 Science/Technical: Arts Policy is and thinking skills among rural secondary
even tougher for the vast rural areas of Malaysia due to its limited infrastruc- school students.
ture, lack of good schools and small population (Ling, Mahdib, Mohamadin & Keywords: engineering design process,
Manaf, 2015). Sabah, an East Malaysian state with a relatively high proportion higher order thinking, outreach challenge
of students in rural schools is facing a more challenging situation with respect program, rural schools, STEM.
to its efforts to reform rural schools. Many rural primary and secondary schools
are located in wide and isolated areas with unique topography (Malaysian Nyet Moi Siew,
Digest, 2011). Some schools, for example, are located in areas with limited University of Malaysia Sabah, Malaysia
road access and as is often the case, water transport such as boats is used. Henry Goh,
Teacher Training Institute,
According to the Sabah Economic Development and Investment Authority Gaya Campus, Malaysia
Blueprint (SDC, 2011), 72% of Sabah’s schools were located in rural areas. In Fauziah Sulaiman
terms of infrastructure and basic utilities, most rural primary and secondary University of Malaysia Sabah, Malaysia
477
ISSN 1648–3898
(P. 477-493)
schools in Sabah lack supplies of 24-hour electrical connection and clean water, access to good teaching and learn-
ing resources, computers, and science laboratories. It is apparent that these limited opportunities and facilities
have somewhat created a gap in education attainment between rural and urban schools in Sabah and in Malaysia
as a whole.
In its report about Malaysian rural schools, the World Bank (2010, p. 92) noted that: “Potentially as a result of
less favourable conditions in rural schools, students from rural and remote schools perform significantly worse on
tests than their peers in urban areas. Disparities within states between rural and urban areas are most prevalent in
poorer states like Sabah”. Specifically, the World Bank (2010) reports a disparity between urban and rural second-
ary schools’ achievement in Mathematics among Malaysian students at Grade 9 (15 years of age). It is clear that
many rural school students have lagged behind their peers from urban schools in academic performance due to
inadequate infrastructures, utilities and learning resources.
As a consequence, despite many new initiatives aimed at transforming rural schools, it is a difficult task to
achieve in the near future (Malaysian Digest, 2011). Similarly demanding is the revitalizing of Malaysian rural second-
ary schools in STEM education. Undeniably, information about the best practices from new initiatives or programs
in rural secondary schools can be used as a reference to revitalize rural schools in the Malaysian context. As long
as the right approach is put in place, the quality of Malaysian rural secondary schools can certainly be improved
and in the intervening time, the gap between rural schools and their urban counterparts can be minimized or
possibly closed.
In countries such as Colombia and the United States of America, an outreach program is usually designed to
help and encourage disadvantaged students of rural schools to increase their science, technology, engineering,
and mathematics literacy and enthusiasm. This afterschool program aims to improve the quality and reach of STEM
education at all levels. These initiatives raise the questions, “How would an outreach program help less privileged
students in Malaysian rural secondary schools learn about STEM?”; “How would students’ knowledge or skills about,
attitudes toward, and practices of STEM evolve as a result of their participation in an outreach program? Tackling
questions like this, particularly in rural settings often requires innovative solutions. There is also a need to propose
an integrated program which allows science teachers to seamlessly examine what rural school students would
learn and difficulties faced during the outreach program.
Theoretical Background
Engineering Design Process and Problems
Many researchers propose engineering design process as a means of solving challenges in STEM fields (Farmer,
Allen, Berland, Crawford, & Guerra, 2012; Householder & Hailey, 2012; Hynes, Portsmore, Dare, Milto, Rogers, &
Hammer, 2011). The Massachusetts Department of Education (2006, p. 84) proposed eight steps of engineering
design process which provide a guide for teachers and curriculum coordinators regarding learning, teaching, and
assessment in science and technology/engineering specific content from Pre-Kindergarten to Grades 6-8 and
throughout high school. Those eight steps of engineering design process include identifying the need or problem,
research the need or problem, develop possible solution(s), select the best possible solution, construct a proto-
type, test and evaluate the solution, communicate the solution, and redesign. Wendell, Wright, and Paugh (2015)
found evidence that specific instructional support built upon student resources could create more pathways to
success and learning during the different phases of engineering design. Additionally, students could create and
communicate design ideas to each other while engaging in practices. The use of the engineering design process
as an instructional framework is intended to ensure that all pedagogical practices are contextualized within the
engineering design process so that students research, calculate, test, brainstorm, build and perform activities to
fulfil STEM-design challenges (Berland, Steingut & Ko, 2014; Farmer et al., 2012).
Farmer et al. (2012), and Householder and Hailey (2012) have demonstrated how engineering design problems
embedded in the context of an engineering design process in secondary school science classrooms could scaffold
in building engineering skills and habits. As outlined by Khandani (2005), and Mentzer, Huffman and Thayer (2014),
engineering design problems in practice tend to be structurally open-ended and highly complex. An open-ended
problem may have various acceptable solution paths and be limited by rigid and negotiable constraints which are
not always presented with the problem. Engineering design problems are also designed to be ‘ill-defined’. Green-
wald (2000) characterized an ill-defined problem as being: ‘‘unclear and raises questions about what is known, what
478
ISSN 1648–3898 INTEGRATING STEM IN AN ENGINEERING DESIGN PROCESS: THE LEARNING EXPERIENCE OF
(P. 477-493)
needs to be known, and how the answer can be found. Because the problem is unclear, there are many ways to
solve it, and the solutions are influenced by one’s vantage point and experience’’ (p. 28). King and Kitchener (1994)
claims that an effective technique for developing problem-solving and critical-thinking skills is to expose students
to “ill-defined” problems in their field.
Many researchers claim that STEM curricula can be integrated in an engineering design process to provide
a mechanism through which students learn relevant STEM content (Hmelo, Holton, & Kolodner, 2000; Mehalik,
Doppelt, & Schunn, 2008; Schunn, 2009). This mechanism encourages students to make connections, helps con-
nect design failure or next steps to real world engineering and technology (Lottero-Perdue, 2015). Students learn
important scientific concepts and their application in engineering and technology, as well as their relationship and
application in daily life or real world context. Students could look for connections by engaging with activities or
material in ‘real-world’ contexts to establish relevance. This approach can attract students’ interest in science lessons
and provide them with a deep understanding of concepts and meaningful learning. A research by Neo, Neo and
Tan (2012) found that activities that students carry out in the real world were effective in teaching and engaging
students in the classroom as well as increasing their understanding of the subject matter.
Hynes et al. (2011) noted that engineering design process that focus on solutions and construction of pro-
totypes impel students to encounter the process of creative and critical thinking as well as problem solving skills.
Hence, engineering design process would offer an effective route as an instructional framework for teaching STEM
subjects among rural secondary school students.
Purpose of Research
Relatively few organized efforts have been directed to the integration of secondary school STEM subjects
in engineering design process experiences. This research was therefore undertaken to investigate the learning
experience of rural secondary school students (16 years old) on the integration of STEM in an engineering design
process (STEM-EDP) outreach challenge program. It was also conducted to address some of the concerns how
and whether the students could benefit in the aspects of creative and critical thinking, problem solving skills and
applying relevant STEM concepts. Additionally, this research also aimed to examine students’ knowledge or skills
about, attitudes toward, and practices on STEM. Implementing a STEM-EDP approach in an outreach challenge
program may provide the platform to address the numerous challenges that are fundamental to the STEM educa-
tion of rural school students.
Research Questions
The research questions guiding this research are:

1. How would rural secondary school students’ knowledge or skills about, attitudes toward, and practices
on STEM evolve as a result of their participation in a STEM-EDP outreach challenge program?
2. What have students learned through engaging with the program?
3. How did STEM-EDP activities aid the students in their creative and critical thinking, and problem solv-
ing skills?
4. What difficulties did the students face as they engaged with the program? What suggestions would
students offer to overcome those difficulties?
Research Design and Participants
A single group with intervening STEM-EDP challenge program design was used in this research. A mixed
methods design of combining both quantitative and qualitative approaches at the interpretation phase was used
to strengthen the knowledge claim of the research (Creswell, 2012, pp. 540).
The outreach program was conducted in two secondary rural schools in April and May 2015. The two selected
schools are located in a rural area on the West Coast of Sabah, Malaysia. School A was about 215 km whereas
school B was 160 km from Kota Kinabalu. The participants consisted of 89 Grade Ten Science Stream students, with
each 49 and 40 respectively from school A and school B. Participants comprised 53 females (59.6%) and 36 males
479
ISSN 1648–3898
(P. 477-493)
(40.4%) aged 16 years old. In the Malaysian context, students from the age of 16 have the opportunity to pursue
two years of study in the upper secondary upon completion of the lower secondary education. Students who are
academically inclined can choose between two main streams: the Science or Arts Stream. Seemingly, the Science
Stream students are perceived to be more adept at performing in mathematics and science related subjects. Thus,
purposive sampling was employed in the selection of the participants. According to Fraenkel and Wallen (2000)
purposive sampling minimizes experimental contamination. Selection of Grade Ten Science Stream students
who possessed knowledge, ideas or experiences of STEM relevant to the research would best help the researcher
understand the research question (Creswell, 2003).
Students gathered into heterogeneous groups of four to five members on the basis of random selection in
accordance with gender and ethnicity (diversity). The groups were assigned by the teacher so that there would be
inclusion of students of high-, medium- and low competency levels based on their individual scores achieved in the
end-of-semester examination. To ensure active and equal participation within a group, each student was assigned to
perform a specific role: a reporter, recorder, runner, checker, and sketcher. All groups were given identical materials.
At the start of the program, students were presented a letter of consent detailing the nature of their involvement
in the program and the need to give their consent on the sheet provided indicating their full understanding. Code
names were used for the data to ensure the confidentiality of the schools and individual identities.
A total of 22 science teachers from School A and 19 science teachers from School B participated as assessors
and facilitators. They were trained to carry out the facilitation and assessment prior to the program. They stood
of qualified science teachers with degrees in Science Education. They obtained a passing grade in the Research
Methodology course (qualitative and quantitative) in their Masters course which they were undertaking at the time.
A total of 18 of them helped the researcher develop the STEM activities and testing procedures. The researcher
guided the science teachers on how to facilitate students through the seven steps of engineering design process
in order to ensure the consistency and reliability in the implementation of the STEM activities across students and
schools.
Data Collection
Research Data were collected through multiple qualitative and quantitative means: participant’s responses
to open-ended questions; teacher’s field notes, and a pre- and post- questionnaires. Teachers wrote their field
notes based on the observation made during the STEM activities, and the focus group interviews with students.
The pre- and post-questionnaires measured student’s knowledge or skills about, attitudes toward, and practices
of STEM shifts over the duration of the program. A total of 19 semi-structured focus group interviews were carried
out after the completion of each STEM activity. The questions of the interviews were open ended (Table 1) and the
students were encouraged to draw explicitly from their learning experiences of working on the STEM activities.
Each focus group interview was conducted in groups consisting of 4-5 students. Table 1 shows the tools that were
being used to address the corresponding research questions.
Table 1. Data capturing tools.
Research Questions Data Capturing Tools
1. How would students’ knowledge or skills about, attitudes toward, and practices Pre- and Post-questionnaires
on STEM evolve as a result of their participation in an STEM-EDP outreach
challenge program?
2. What have students learned through engaging with the program? a. Teacher’s field notes based on focus group interview and
observation.
Question: What were some of the things about this program that students b. Open-ended questions.
learned? “Something new students have learned today was…”
3. How did STEM-EDP activities aid the students in their creative and critical Teacher’s field notes based on focus group interviews and
thinking, and problem solving skills? observation.
4. i. What difficulties did the students face as they engaged with the program? Teacher’s field notes based on focus group interviews and
ii. What suggestions would students offer to overcome those difficulties? observation.
480
(P. 477-493)
A pre- and post-questionnaire adapted from Illinois Valley Community College (2011), and the Knowledge,
Attitude and Practices (KAP) survey, a quantitative method (predefined questions formatted in standardized ques-
tionnaires) that provides access to quantitative and qualitative information (Kaliyaperumal, 2004) were conducted
before and after the program. The pre-questionnaire consists of 20 items while the post-questionnaire consists of
12 items that can be grouped into three different dimensions:
I. Knowledge or skills about STEM

II. Attitudes toward STEM
III. Practices on STEM
Table 2 shows the distribution of items according to the three different dimensions.
Table 2. Distribution of pre- and post-questionnaire items according to dimensions.
Total Item No. in Post-

No. Dimension Item No. in Pre-questionnaire Total Items
Items questionnaire
1 Knowledge /skills about STEM 1,2,3,4,5,6 6 1,2,3, 3

2 Attitudes toward STEM 8,9,10,11,12,13,14, 20 8 4, 5, 12 3
3 Practices on STEM 7,15,16,17,18, 19 6 6,7,8,9,10,11 6

All answers were entered on a six-point Likert scale ranging from ‘Strongly Agree’, ‘Agree’, ‘Slightly Agree’,
‘Slightly Disagree’, ‘Disagree’ to ‘Strongly Disagree’
The pre- and post-questionnaire were reviewed by a Science lecturer and Science teacher. Since English is
not commonly used by the participants, the researchers translated the instruments into Malay language; these
were reviewed by a Malay language teacher. A pilot study was conducted in the form of a three hours program to
address one ill-defined problem. A total of 40 Grade Ten rural secondary school students (16 years old) took part in
the pilot study. Students completed the questionnaires before and after the program. Students were also asked to
comment on the readability of the items and make suggestions regarding the items in pre- and post-questionnaire.
They agreed that all items were relevant, thus should remain in the final questionnaires.
The Cronbach Alpha of internal consistency based upon scores of 40 Grade Ten students was computed to
determine the reliability of the pre- and post-questionnaires. The alpha values of the pre- and post-questionnaires
was .87 and .74 respectively, indicating that the items have high reliability (Babbie, 2001). The obtained value of
Cronbach’s alpha for each dimension of pre-questionnaire and post-questionnaire were in the range of .63 - .78
and .45 - .67 respectively (Table 3), indicating that the items have moderate reliability (Babbie, 2001). Consequently,
this indicated that all the items contributed to the central dimension of pre-questionnaire and post-questionnaire
on aspects such as knowledge/skills about, attitudes toward and practices on STEM.
Table 3. Cronbach’s alpha values of each dimension of pre-questionnaire and post-questionnaire.
No. Dimension Pre-questionnaire Post-questionnaire
Cronbach’s alpha values
1 Knowledge /skills about STEM .63 .57
2 Attitudes toward STEM .78 .45

3 Practices on STEM .75 .67
Total .87 .74
481
ISSN 1648–3898
(P. 477-493)
Data Analysis
The qualitative data were analysed through thematic analysis. Thematic analysis is a form of a pattern rec-
ognition technique by searching through the data for emerging themes (Fereday & Muir-Cochrane, 2006). Two
researchers independently reviewed teachers’ field notes and students’ responses to open-ended questions. They
read the data line by line and identified recurring patterns in the data. The patterns identified by each researcher
were compared to ensure validity of the codes. They dealt with codes which had no consensus by sharing their
perspectives and concerns to reach common codes. Through multiple reviews and an iterative process, categories
and codes were refined and grouped into themes.
For the quantitative data, the difference in the percentage between the pre- and post- questionnaires was
computed as a measure of change in participants’ knowledge or skills about, attitudes toward, and practices on
STEM. The data were then grouped into high, medium and low level categories. Further analysis using One Sample
T-test was conducted to compare the mean scores of the pre- and post-questionnaires for each dimension to
determine if a significant difference existed.
Learning through STEM-EDP Outreach Challenge Program
The STEM-EDP outreach challenge program was designed with a focus on encouraging rural school stu-
dents to solve an ill-defined problem utilizing the engineering design process to design, build, and test
their creations. In the challenge, students would be asked to consider the constraints of the materials and time,
identify the problem, think about what they already know, design, plan, construct, test and evaluate a physical
prototype of their design.
The STEM-EDP program consisted of Six STEM activities (Appendix A) lasting about three hours and 20 minutes
each. Three activities were introduced in each school with specific context to enhance learning and understanding
of the STEM concepts. Students also needed to answer the Higher-Order Thinking (HOT) questions that stood of
questions that were not strictly in their curriculum. In a way, answering HOT questions inspired students to acquire
new found competences. Anderson and Krathwohl (2001)’s Taxonomy was used as a guide to develop a blueprint
for the HOT questions, which belonged to the Analysis and Evaluation category of the cognitive domain. Some
samples of HOT questions used were: ‘In your opinion, if buildings were constructed identical to this prototype, is
it safe to be inhabited? If yes/no, please explain why?’(Evaluation); ‘How can your prototype be modified in order
to improve its results in the future?’(Analysis); and ‘Explain why there is a difference of the submarines’ speeds
between the two bottles? (Analysis)’. The HOT questions were specially designed to evaluate students’ analysis,
evaluation and communication skills in connecting STEM activities with their daily life.
Previous research (Siew, Amir & Chong, 2015) found that science teachers noted several potential challenges
while implementing a STEM-Project-based learning approach in their rural school classrooms. These included
inadequate materials, limited facilities and limited allocation of classroom time. Accordingly, the engineering
design process employed in this program (Figure 1) removed the ‘redesign’ step proposed by the Massachusetts
Department of Education (2006, p. 84). This modification was made to ensure that students could produce work-
able prototypes that made best use of the materials and time provided.
Figure 1: The seven steps of engineering design process (Adapted from Massachusetts Department of Educa-
tion, 2006).
482
(P. 477-493)
The advantages of the engineering design process in the teaching of STEM is that it focuses on solutions by
constructing prototypes that drive students to encounter the process of creative and critical thinking, and problem
solving skills. It allows students to realise that there are many ways to find solutions, as they engage in brainstorm-
ing to identify problems and propose solutions. The process of finding the optimal solution based on constraints
requires participants to engage both in critical thinking and problem solving skills. Through the seven stages of
engineering design process, learners are intended to develop creative and critical thinking, and problem solving
skills while carrying out the STEM activities.
Prior to the program, students worked in teams to solve an ill-defined problem by designing and building
workable solutions in forms of prototypes, which could be tested and fulfilled the criteria set in the problem. In this
research an ill-defined problem was introduced to students within the context of their daily life. Thus, ill-defined
problems become better defined and more contextualized as they were worked on and hence the solving and
learning was through doing.
Students worked collaboratively to plan, design, construct, and test a prototype based on their prior and new
knowledge; and demonstrated and tested their prototype to their peers and facilitators. The students were expected
to be able to identify and discuss the science and mathematical concepts exhibited in their designs or prototypes.
The STEM-EDP outreach challenge program promoted out-of-the-box creative thinking and discussions. Students
were encouraged to find multiple, imaginative, intuitive and common sense solutions and not “one right answer”
to a problem. The goal of this program was to enhance innovative and inventive thinking abilities of rural school
students resulting in skills that can be applied in the Science, Mathematics, Engineering, and Technology fields.
Results of Research
Quantitative Analysis
The summary of percentage, means and standard deviations for response of students in pre-and post-
questionnaires are shown in Table 4 and Table 5 respectively.
Table 4. Summary of percentage, means and standard deviations for response of students in the pre-ques-
tionnaire.
Disagree
disagree
disagree
Strongly
Strongly
Slightly
Slightly
Agree
agree
agree
No. Statement M SD
1 I know what Science, Technology, Engineering or Math- 20 51 24 5 0 0 4.86 0.78

ematics field is.
2 I know what scientists, technologists, engineers, or 10 60 26 3 0 1 4.74 0.78
Mathematicians do.
3 A scientist, technologist, engineer, or mathematician needs 48 36 14 1 1 0 5.28 0.84
to be creative.
4 A scientist, technologist, engineer, or mathematician needs 36 48 11 5 0 0 5.15 0.80
to work in teams.
5 A scientist, technologist, engineer, mathematician needs to 53 41 6 0 0 0 5.48 0.61
be good at Mathematics.
6 A scientist, technologist, engineer, and mathematician 52 41 7 0 0 0 5.45 0.63
needs be good at science.
7 I am confident in my technical ability. 10 29 49 10 1 1 4.34 0.93
8 I like to think about how things work. 28 40 24 7 0 1 4.85 0.99
9 I like to explore how things work. 24 36 31 8 0 1 4.73 1.00
483
ISSN 1648–3898
(P. 477-493)
Disagree
disagree
disagree
Strongly
Strongly
Slightly
Slightly
Agree
agree
agree
No. Statement M SD
10 I have thought about a career in science. 43 33 15 7 2 0 5.07 1.04
11 I have thought about a career in technology. 29 31 23 15 2 0 4.70 1.12
12 I have thought about a career in engineering. 23 28 24 20 5 0 4.45 1.18
13 I have thought about a career in math. 24 37 29 7 3 0 4.72 1.02
14 My parent(s) have suggested that I consider a career in 38 31 22 5 3 1 4.93 1.14

science, technology, engineering or math.
15 I was able to complete hands-on tasks individually. 6 27 40 24 3 0 4.07 0.94
16 I was able to complete hands-on tasks with a team. 30 54 15 1 0 0 5.13 0.70
17 I was able to perform science experiments. 20 45 34 0 1 0 4.83 0.79
18 I was able to perform mathematical computations. 20 52 23 4 1 0 4.86 0.81
19 I was able to do designing and build new products. 3 29 46 18 3 1 4.08 0.92
20 I cannot wait for STEM activities on April 23/May 9, 2015. 44 31 18 4 2 1 5.08 1.08
Table 5. Summary of percentage, means and standard deviations for response of students in the post-
questionnaire.
disagree
disagree
disagree
Strongly
Strongly
Slightly
Slightly
Agree
agree
agree
No. Statement M SD
1 Today’s activity improved my problem solving skills. 63 36 1 0 0 0 5.62 0.51
2 The STEM activities today improve my ability to generate 61 38 1 0 0 0 5.59 0.52

new ideas.
3 I combined the idea of friends to solve a problem. 73 26 1 0 0 0 5.72 0.48
4 I took risks and assumed failure is an opportunity to learn. 62 33 3 0 1 1 5.50 0.84
5 After today’s activity, I would consider a career in science, 51 37 12 0 0 0 5.40 0.69

technology, engineering, or math.
6 I am able to complete hands-on tasks individually. 7 33 49 10 1 0 4.34 0.81
7 I am able to complete hands-on tasks with a team. 72 28 0 0 0 0 5.72 0.45
8 I am able to perform science experiments. 41 51 8 0 0 0 5.33 0.62
9 I am able to perform mathematical computations. 36 42 21 1 0 0 5.13 0.78
484
(P. 477-493)
disagree
disagree
disagree
Strongly
Strongly
Slightly
Slightly
Agree
agree
agree
No. Statement M SD
10 I am able to perform design and build new products. 38 38 20 4 0 0 5.12 0.85
11 After today’s STEM activity, I am confident to perform well 61 38 0 0 0 1 4.57 0.62

in STEM activities in the future.
12 I am interested in creating new product after today’s STEM 37 46 0 16 0 1 4.17 0.79
activity.
Table 6 shows the evolvement of students’ response in three dimensions according to high, medium and
low level.
Table 6. Evolvement of students’ response in three dimensions according to level.
No. Dimension Level Students’ response
Pre-questionnaire (%) Post-questionnaire (%)
High 82.67 99.00

Knowledge or skills about Medium 17.00 1.00
1
STEM
Low 3.33 0.00
High 65.00 88.60

2 Attitudes toward STEM Medium 32.38 10.33
Low 2.63 1.00
High 54.10 80.83
3 Practices on STEM Medium 44.00 18.83
Low 1.83 0.33
With reference to Table 6, in terms of students’ knowledge or skills about STEM, the finding showed a shift from
82.67% in the pre-questionnaire to 99.00% in the post-questionnaire in the category of ‘high level’. The results also
showed there was an increase of 23.60% in students’ attitudes toward STEM in the ‘high level’ category. It is also interest-
ing to note that in terms of practices on STEM, only 54.10% students considered themselves as being able to carry out
and perform STEM tasks before being exposed to the STEM-EDP approach. However, the post-questionnaire indicated
80.83% in the ‘high level’ category indicating the students’ willingness to engage themselves in STEM activities.
The One Sample T-test results showed that there was a significant difference in the scores of Knowledge or
skills about STEM between the pre-questionnaire (M=5.16, SD=0.74) and post-questionnaire (M=5.64, SD=0.503);
t(88)= -9.003, p < .05). There was also a significant difference in the scores of Attitudes toward STEM between the pre-
questionnaire (M=4.816; SD=1.07) and the post-questionnaire (M=5.02, SD=0.77); t(88)= -2.496, p = 0.007). Moreover,
differences in Practice on STEM, t(88)= -6.608, p < .05, between the pre-questionnaire (M=4.55; SD=0.85) and post-
questionnaire (M=5.04, SD=0.69); p < .05) were also statistically significant.
Qualitative Analysis on Participants’ Response
Science teachers’ field notes and students’ responses to open-ended questions were analysed using thematic
analysis. A number of different themes in relation to STEM-EDP approach emerged from the data. The abbreviations
used for the analysis are: “S” represents Student, “T represents Teacher, “SA” represents School A, “SB” represents School
B and ‘G’ represents Group. The main findings are discussed below:
485
ISSN 1648–3898
(P. 477-493)
Students’ acquisitions through engagement with the program
Application of Science, Technology, Engineering and Mathematics knowledge in solving daily life problems.
Almost every participant (98%) noted that they benefitted from the STEM activities as they were exposed to real-life
situations where Science, Technological, Engineering and Mathematical knowledge were applied for solving daily life
problems. More importantly, STEM activities succeeded in providing a platform for them to apply scientific knowledge.
Among the scientific concepts the students noted were related to water and air pressure, equilibrium of force, base
area, balanced force, surface tension, stability, water density and the buoyancy force in a submarine. Students also
pointed out that they were working like engineers. Related responses to open questions were:
“The activity integrates many of the STEM concepts such as water pressure (science), boat structure (engineering) and measur-
ing length (mathematics)”(S39, S43);“I could apply physics concepts in solving problems and thinking outside the box such as
designing of floating needles and paper clips on the water’s surface using the concept of water’s surface tension” (S31, S39);
and “We learned how to build balancing toys as if we were engineers” (S78, S91).
T21, T30, and T36 (SB) confirmed that interviews with students revealed that they found the need to apply the
concept of impulse in order to create an innovation to help absorb the impact of an egg being thrown from a high
place. Furthermore, students were also able to explain that the concept of impulse was also applied in producing
car air bags.
Application of scientific knowledge in designing and producing daily life products and answering HOT ques-
tions. A significant number of science teachers (88%) observed that a profound comprehension of scientific knowledge
was needed in order for students to answer HOT questions every time a STEM session ended. Scientific knowledge did
not only help participants answer high level questions but also helped them to be creative in reapplying knowledge
learnt in the designing and producing of prototypes. For example, T5 and T10 (SA), and T6 and T19 (SB) observed that
HOT questions gave students an opportunity to posit answers according to their thinking and make connections with
scientific concepts they have learnt in class. Some of the evidences that showed students realised the importance of
using scientific concepts in designing and producing daily life products include:
“I learnt that the Sink and Swim activity that involve the concept of weight and air pressure is very important in building a
submarine” (S44); “Scientific concept has always been applied in every creation” (S7); “It involves making daily life products
through the applications of scientific concepts” (S11, S37, S61).
T2 and T13 (SA) added that during the activity of creating a straw submarine, they observed that students dis-
covered that the submarine needed to be designed with some air space inside it. When the plastic bottle was being
pressed, it created pressure against the water in the bottle. Indirectly, the water in the bottle exerted pressure towards
the air space in the submarine. Based on this understanding, students started to create different designs of submarines
using straws. Students also discovered the different speeds of the submarine, moving up and down in two different
solutions. Facilitators were even more satisfied that the students could answer the HOT questions correctly.
Connecting STEM activities with daily lives and scientific concepts learned. A large percentage (93%) of the
science teachers noted in their field notes that participants learned how to make connection of the STEM activities
with their daily life phenomenon. For example, T2, T18, T13 and T14 observed that students could relate how ships
or boats function and why they could float on the surface of water by making comparisons with their boat models.
Another example is when answering the HOT questions, participants could affiliate the floating needle and paper
clip activity with the water strider bug, a floating log, water lilies, floating ants and others. T6 and T19 supported these
claims by noting that “Scientific knowledge is not only for answering exam papers but also useful in helping students cre-
ate connections and explain situations faced in their daily lives. In this case, it is observed that students applied scientific
concepts they learned during Physics lessons in problems given to them. Students not only applied the science principles
and laws they learnt but also used them in practical forms”.
Designing and building something new and practical. A large percentage (96%) of the participants expressed
in the open questions that STEM activities gave them an opportunity to create many new, interesting and practical
science products using everyday materials. They stated that the balloon powered car made from plastic bottles was
486
(P. 477-493)
a new experience for them. They were fascinated with finding ways to make a highly powered car moved by air using
ever ready materials such as glue, bottles, pencils and others. Another activity was making a boat. The students said
they realised that play dough can float when shaped into a boat. Others noted that finding gravity centre through
making the balancing toys was a new activity. Meanwhile, a few students commented that they discovered how to
float the needles and paper clips while some noted their success in floating the objects.
According to T24 and T28, when participants were asked why they were excited with the STEM activity, they
answered that: “because we got the chance to design and build a new model which we only see in textbooks”. As for T20,
T23, and T34, they observed that the students could design egg protection tools and that every group member
worked together the whole time by contributing ideas and carrying out the projects as they had planned. Other than
that, T13 and T14 also said that students showed interest in STEM 3 activity because they could become ‘designers’
of their own boat in the future. One of the members in the group shared her opinion by saying that, “This activity
gives me an idea of creating a modern boat that can give a great impact to the means of transportation” (G6, S3). Thus,
STEM activity, according to T13 and T14, seems to provide a very good start to stimulate the interest of students in
learning Science.
Ways in which STEM activities have aided the students in their creative and critical thinking, and problem solv-
ing skills.
In their field notes, science teachers reported that all the participants agreed the activities they were en-
gaged in had aided either in their creativity, critical thinking or problem solving skills through measures as discussed
below.
HOT questions sparked critical thinking. A large percentage (93%) of the participants expressed through the
open questions that they were challenged to think critically when answering the demanding questions in the STEM
program. According to T4 and T11, students were capable of giving rational answers to the HOT questions. For example,
one of the group members gave an excellent answer and showed that he/she understood the concept and was able
to give a suggestion to improve the existing prototype if given the chance to design it with the aid of extra materials.
From T4 and T11’s observation, the HOT questions challenged the participants even in their groups. The sharing of
answers added knowledge collectively to the group besides increasing their critical thinking skills.
Besides that, other teachers like T27, T31, and T39 (SB) also thought that the student’s critical thinking was
enhanced since each activity required students to answer HOT questions. The students felt that the HOT questions
were difficult but they tried their best to answer and associate them with their prior knowledge. According to T24
and T28 (SB), two members from their group stated that STEM activities tested and challenged them to think outside
the box using higher thinking skills.
Ill-defined problems inspired creativity and thinking. A significant number of science teachers (83%) reported
that students faced complexities posed by ill-defined problems in the program. These ill-defined problems demanded
from them effective response to the challenging tasks which in turn inspired creativity and thinking. For example, T26
and T33 stated that participants from groups Two and Seven admitted that STEM 3 activity was the most challenging
because it demanded high thinking skills to solve problems and tested the students’ creativity levels in creating a
bottle car that was powered by a balloon. Participants had to figure out ways to move a car by using only air within
a balloon. Participants also had to think of a method of reducing the car’s weight and decrease its tire resistance.
Besides that, T29 and T35 (SB) also declared that students found the STEM 2 activity challenging particularly when
building a slide made of satay sticks tied together that needed high creativity skills.
T20, T23 and T34 (SB) also asserted that interview results showed that each student admitted that the problems
posed in three activities were challenging. One of the male participants in their group admitted that he never knew
that he could solve the challenge of building a slide using satay sticks in a short amount of time. On the other hand,
participants in T25 and T40’s group (SB) also claimed that STEM program encouraged them to think creatively as well
as critically. This was because each activity had its own challenges.
For STEM 1 activity, which was the ‘egg-astronaut’, students were being hurried and struggled with having no
ideas in using the materials given because “slow to act as ideas came late’ (G4, S92). Nevertheless, discussions done
from time to time enabled them to think creatively and they eventually made a sellotaped basket tied with three
balloons. This idea came from their own group member and the inspiration for it came from their observation of
the hot air balloon. Their own knowledge about hot air balloons helped them in this activity. Moreover, according
487
ISSN 1648–3898
(P. 477-493)
to participants in T9 and T12’s group (SA), “The thing we like the most with this activity is that it challenges our mind to
create something more creative that is to think of ways to produce balancing toys in a more stable way. This activity also
tests patience” (S3, G2).
Besides that, the problem of limited materials forced them to think creatively. Below is another example of an
interview with student groups:
Students (SA; S4): I have to recombine other materials to make up a good raft.
Students (SA; S1): There is no glue, we need glue, teacher. No glue, so we have to think a bit critically to tie the straws
together and minimize leakages.
Students (SA; S6): We redo the raft, well; it floats and still carries 18 marbles. I guess we managed to learn how to
solve the problem. (Laughs)
Along with that, T5 and T10 noted that most respondents responded that STEM activities challenged them and
their creativity for the sake of creating a working product that fulfilled the criteria needed in the specified rubric. As
an example, the “Sink or Swim” activity needed students to identify methods or steps and design needed to make
the paper clips and needles float on the surface of water.
Sketching, designing and constructing models fostered creative thinking and problem solving skills. A con-
siderably large percentage (78%) of the science teachers noted in their field notes that participants stated that the
activities of sketching, designing and constructing models helped to increase their creative thinking and problem
solving skills. For example, T27, T31, and T39 (SB) noted that students expressed opinions that each activity encouraged
creative thinking and also problem solving skills. This was because each activity needed students to sketch and design
models according to the creativity of each group. Students noted that they had to think of a way to design models
that worked and at the same time possessed creative elements. By observing the sketches in the three activities, T27,
T31, and T39 (SB) found that there was improvement particularly in STEM activity 3. Besides that, students also said
that their problem solving skills were highly stimulated because they had to solve problems in the stipulated time as
well as create a working model out of the materials prepared.
Apart from that, T2 and T14 noted that participants became inventive when given the chance to design and
produce their own functional straw submarine. One of them even stated that “I liked this activity. Maybe, who knows,
in the future, I can create my own submarine, because I already know the concepts of how to make it!” (G6, S2.). T7, T15 and
T17 (SA) also found that the ‘Straw Submarine’ activity challenged student’s thinking skills as they noted: “We can see
that everybody was trying really hard to build the submarine. It can obviously be seen on their faces. All of them also stated
that to build submarine is very hard compared to the previous activity”.
In addition, T2 and T14 stated that ‘balancing toys’ successfully induced creativity within students as almost ev-
ery one of them were able to build a balancing toy with different designs. By using their creativity and imagination,
students in group Six were able to create nine balancing toys with different designs. Furthermore, this activity also
enhanced student’s thinking skills. Students gained ideas on how to create their own toy design. Hence, it encouraged
them to think more profoundly. What’s more, from the interviews, students shared that this activity motivated their
creative thinking. This was supported by observations made by T9 and T12 (SA) who noted that: “Besides creating one
‘balancing toy’, students can think of ways to merge a few ‘balancing toys’ in a stable condition”. Moreover, T2 and T13
(SA) stated that the balancing toys activity successfully encouraged the creativity and imagination of the students as
almost every one of them was able to build a balancing toy with different designs.
Another example was a boat making activity. From the provided materials, students designed two different kinds
of boats, one from play dough and another from straws. This increased the number of marbles carried by the boat as
long as the boat was stable enough to carry them. Brainstorming within the group produced new ideas and boosted
the confidence of individuals, hence allowing them to do their best work in order to find optimal solutions.
Working cooperatively instilled thinking. In a statement by T4 and T11 (SA), students were in opinion that
“Sink or Swim”, the first STEM activity, challenged them to think of ideas and make many attempts without giving
up in order for a needle and paper clip float. Suggestions from their friends in the groups helped them to increase
their critical and creative thinking when making attempts to cooperate as a group to solve the problems. T6 and T19
(SA) also observed the mutual understanding shared among the group members while doing the activities and the
cooperation in contributing ideas. T7, T15 and T17 (SA) reinforced that participants tried to solve and help each other
488
(P. 477-493)
to find the centre of gravity in order to create stable ‘balancing toys’.
The challenge faced by the students during the program
Time Constraint. A major concern during the program was time constraint. T29 and T35 (SB) said that some
students assented that it was tough to design the slide model in activity STEM 2. According to them, models built from
sticks needed time but the time given was not enough. Time constraints caused them not to complete their model
according to plan. Another pair who stated a similar problem was T26 and T33 of SB. They said that students in STEM
2 faced lack of time. T20, T23 and T34 (SB) also agreed that time was short and not enough for students to complete
the STEM 2 activity. Similarly, T24 and T28 (SB) said that students from Group 1 complained that they did not have
enough time to build a strong runway. Finally, T32 and T38 (SB) posited that participants from Group 3 found that the
STEM 2 activity (slide invention) was the most arduous because the time given was too brief (S4, G3) and too many
materials had to be assembled (S3, G3).
Students equipped with limited scientific concepts. T24 and T28 (SB) noted in their observation and interviews
that students experienced difficulty in applying scientific concepts and knowledge in the implementation of the
STEM-EDP program. The students were weak in mastering physics concepts, hence they needed to put in extra ef-
fort to relate physics principles in designing and building activities. For example, according to the students of group
Four, their prior knowledge of scientific concepts was limited. This led them to be less creative in creating something
unique for the STEM 1 activity. T7, T15 and T17 (SA) affirmed this:
“We found out that most of the students under our care cannot get the science concepts quickly, except the leader
of group One. The main difficulty that the students faced was weak basic concepts in science. The next problem that we
observed was, they did not know how to explain the concept, which can be seen in their answers for Higher Order Thinking
(HOT) questions. For example, they only got one mark for question 2.3.2 because of insufficient explanation of the process
involved when they pressed the bottle. This was the main reason why they got last placed in this program.

T6 and T19 (SA) noted that students encountered the difficulty of relating scientific concepts they have learnt in
the classroom to the activity. For example in activity 1A, students were unable to link the scientific concepts such as
buoyancy force and density to the design. Hence, a discussion among the students in determining the exact scien-
tific concepts taught them to think using higher level thinking skills. They had to assess in detail the exact scientific
concepts used to solve the problem in this activity. Additionally, T4 and T11 stated that students faced hardship in
stating and explaining the concept of buoyancy connected with large ships made out of steel. In this matter, students
could only give unfocused answers that did not match the suggested answers. This was seen when students tried to
solve the problem of making boat models. Other facilitators responded:
“Students appeared to be unable to perform the activity, even after they understood what they have to do. This may
be due to lack of ideas to create something that they are not accustomed to.” (T1, SA).
“We have to give them hints such as ‘force related to water’ and giving them time to recall what they have learnt in
the classroom. Fortunately, they remembered! Even though they knew the concept, they were not able to relate it with the
activity.” (T7, T15 & T17, SA).
Activities were too challenging. T20, T23, T24, T32, T34 and T38 (SB) remarked that participants experienced
higher level thinking challenges in STEM activities As for T2 and T18’s group, in the STEM 3 activity, students expe-
rienced obstacles in balancing the marbles on the raft. The dilemma was most felt when they had to choose a raft
shape that could hold the most number of marbles. T7’s group also found the same activity challenging. The interview
finding was as follows:
Students (SA; S4): We faced so much problems with testing the raft on the water.
Students (SA; S1, S5): ..... The leaks really cracked our brains. (Laughs)
Students (SA; S1): We tried many times to fix the raft but it was still leaking. Our brains are tired, (Smiles cheek-
ily)
489
ISSN 1648–3898
(P. 477-493)
According to T2, T9, T12 and T18 (SA), students encountered hardship in balancing the ‘balancing toys’ even
though there were several efforts done to tie them up. Students faced difficulty in finding the centre of gravity of
the ‘balancing toys’ in order to create stable ‘balancing toys. Even though the balance was the same, students found
different designs had different centres of gravity. Nevertheless, after improvisation was done, each group was seen
to be able to balance their ‘balancing toys’.
T9, T12, T30, T21, and T36 reinforced that students were of the opinion that STEM activities were demanding
especially when they tried to make a submarine float and sink. This was supported by their remarks: “We found it ex-
tremely difficult to make the submarine float and sink. We tried many methods but we still could not manage. The activity
undoubtedly tested our knowledge and skills”. This was also supported by the remarks made by T7, TT15 and T17:“We
found out that students were really engaged in ‘Straw Submarine’ compared to ‘Sink or Swim’ activity. This is because, ac-
cording to students, ‘Straw Submarine’ is a more challenging and interesting activity because they must know the concept
of buoyancy and how submarines work in order for them to build a straw submarine.”
Besides that, other teachers like T27, T31, and T39 (SB) revealed that all the students admitted that the STEM
activities were challenging and that they needed to take high risks in making decisions in order for their models and
designs to be built and function well using related scientific concepts and appropriate materials.
Students’ suggestion for overcoming the identified source of problem
Extending the time. Time was a big issue in STEM 2 activity for school B in this program. According to T38 (SB),
students suggested that time should be increased for STEM 2 activity which needed more time to construct the
structure (G3, S1). T26 and T33 (SB) also noted the same suggestion for STEM 2 activities. Tuina, a student, stated that
if time were increased, she and her friends would have been able to make better creations and do more trials so that
weaknesses could be overcome. T24 and T28 repeated the same suggestion from participants for STEM 2 activity.
Discussion
The results of the research reveal statistically significant gains in knowledge or skills about, attitudes toward, and
practices on STEM after participating in the STEM-EDP outreach challenge program. The students perceived that they
became highly knowledgeable about STEM after the program. The STEM-EDP approach which focused on integration
of STEM in engineering design process was also found to promote a positive change in students’ attitudes towards
STEM. In addition, the students’ participation in the program contributed to an increased capability of carrying out
the perceived practices on STEM.
The STEM subjects which were integrated into engineering design process provided a mechanism through
which students learn to make connections by engaging in ‘real-world’ problems and contexts (Lottero-Perdue, 2015;
Neo, Neo & Tan, 2012). In this present research, findings show that students were able to apply STEM knowledge
in solving daily life problems, designing and producing daily life products and answering HOT questions. Students
were also able to connect the STEM activities with daily lives and scientific concepts learned in the classroom,
and to create new and practical products using everyday materials. This research makes clear that the execution
of the proposed STEM-EDP program can help students in relating STEM knowledge to their real-world problems
and contexts.
The STEM-EDP outreach challenge program not only allowed students to gain and integrate STEM knowledge
but also provided an avenue to boost their creativity, critical thinking, and problem solving skills. Students’ creative
and critical thinking was sparked through solving HOT questions and ill-defined problems posed in the STEM activi-
ties. Students could respond effectively even with limited materials and time in organizing their thoughts to choose
the best possible solution for their prototype using related scientific concepts. Lewis (2009) asserts that imposing
some structure to open-ended design problems may assist in encouraging more creative thinking. Hynes et al. (2011)
also noted that engineering design process provides students an opportunity to practice critical thinking skills as
well as creative and outside-the-box thinking. King and Kitchener (1994) have also written about exposure to ill-
defined problems that mimic those solved by real-world practitioners help students develop problem-solving and
critical-thinking skills. This research demonstrates that STEM-EDP approach allows students to focus on solutions to
ill-defined problems and construction of prototypes that could encounter them in the process of creative and critical
thinking, and problem solving skills.
In addition, the research showed that activities such as sketching, designing and constructing a prototype have
490
(P. 477-493)
helped students to foster their creative thinking and problem solving skills. Students described how the engineering
design process encouraged them to come up with sketches for possible solutions. The process also taught them how
to solve the given problems in the stipulated time as well as design and create a working model out of the materials
prepared. Students said that brainstorming within the group produced new ideas, and mutual understanding and
cooperation boosted the confidence of each individual to do his/her best work in order to construct a functional pro-
totype. For example, a group of five from group Six were able to create nine balancing toys with different designs.
A number of the earlier researches have noted that learning in cooperative learning groups fosters creativity
and problem solving skills and social competences. For example, Siew, Chong and Lee (2015) reported that sketch-
ing science and sharing ideas in cooperative learning groups in problem-based learning fostered students’ scientific
creativity. Similarly, Stanford University Newsletter (2001) noted that students who brainstorm in a collaborative
situations while solving a problem develop both domain knowledge and problem solving skills. Notably, the pro-
gram had a significant impact on students’ understanding about themselves as potential problem solvers, thinkers,
creators, and collaborators. Students were made aware of their potential to become inventors. They also felt a sense
of empowerment to make an impact to the world.
For most students, STEM-EDP outreach challenge program provided a fun and enjoyable learning experience,
enabling them to incorporate their own ideas from daily life experiences and creative thinking to create new products.
Overall, the program scaffold the students’ critical thinking, problem solving, team work, creativity, and thinking. They
learnt skills and competences that were otherwise tough to teach in a normal classroom setting.
While the students described many positive learning experiences gained in this program, they also pointed
out several challenges. The two most commonly mentioned challenges were the limited amount of time needed to
construct optimal prototypes, and limited knowledge of scientific concepts. Straw, MacLeod, and Hart (2012), and
Siew, Amir and Chong (2015) also note that time is a critical constraint; this is especially obvious when the conducted
STEM activities involved the use of a wide range of cognitive abilities. Siew et al. (2015) have also written about limited
knowledge of scientific concepts being a challenge that influences success in STEM activities.
This research makes clear that reducing the time pressure by negotiating or extending the execution time would
clearly help some students complete the activities. This research also highlighted that students who are equipped with
sufficient knowledge of scientific concepts would be able to answer HOT questions elaborately, and thus adequate
classroom opportunities to practice thinking skills are crucially needed. In addition, this research makes clear that
consolidating students’ understanding of scientific concepts would help them in adopting a STEM-EDP approach
in their design and build activities. The research finding not only confirms the results of previous research studies
reviewed, but also supports new research examining a number of potential ways to support STEM students with
complex design tasks in time-limited situations.
Although the research findings suggest that rural secondary school students benefited from the learning ex-
perience through STEM-EDP outreach challenge program, its limitation must also be acknowledged. This research
involved only 89 Grade Ten students, and may not be representative of the Malaysian rural secondary school students’
population as a whole. This research is a one-day snapshot of students’ learning experiences. Future research will
therefore need to be carried out with a larger sample size and longer period with extra ill-defined problems compared
to the current research to assess extensively the learning effects of integrating STEM in an engineering design process.
Further comparison between rural and urban schools would shed light on the extent to which locality influences
students’ learning experiences in the STEM-EDP program.
Conclusions
This research investigated and elaborated how Grade Ten rural school students (16 years old) benefited from
a day long implementation of the STEM-EDP outreach challenge program. It concludes that the proposed program
is an extremely rewarding and influential process of teaching and learning both for the educators and the students.
Vis-à-vis the program, students were able to expand their boundaries of knowledge and competencies to become
problem solvers, innovators, creators, and collaborators even though they experienced difficulties in tackling challenges
associated with STEM activities. The STEM-EDP outreach challenge program opened the eyes of many rural secondary
school students of their potential to fill in the critical pipeline of engineers, scientists, and innovators so essential to
the future of Malaysia. Findings suggest that the STEM-EDP outreach challenge program offered a means for develop-
ing creativity, problem solving skills, and thinking skills among rural secondary school students. It also enhances the
students’ knowledge or skills about, attitudes toward, and practices on STEM. This research has therefore highlighted
491
ISSN 1648–3898
(P. 477-493)
the pivotal role of applying integrated approaches such as STEM-EDP to the teaching and learning process of science
related subjects which cater to the needs and challenges faced by rural school children in the country.
Acknowledgements
The research was supported by the University of Malaysia Sabah (UMS), Malaysia under Grant No. GPP0004. Any
opinions, viewpoints, findings, conclusions, suggests, or recommendations expressed are the authors and do not
necessarily reflect the views of the University of Malaysia Sabah, Malaysia.
References

Academy of Sciences Malaysia (2015). Science Outlook: Action towards vision. Report to the Ministry of
Science, Technology and Innovation. Kuala Lumpur: Academy of Sciences Malaysia. Retrieved June 24, 2016 from http://www.mastic.
mosti.gov.my/.../10156/2d66a776-277e-4816-bce1-53a89 afda5e.
Anderson, L. W., Krathwohl, D. R., (2001). A taxonomy for learning, teaching, and assessing: A revision of bloom’s taxonomy of educa-
tional objectives. New York: Longman.
Babbie, E. R. (2001). The practice of social research (8th edn). Wadsworth Publishing Group: Belmont.
Berland, L., Steingut, R., & Ko, P. (2014). High school student perceptions of the utility of the engineering design process: Creating
opportunities to engage in engineering practices and apply math and science content. Journal of Science Education and
Technology, 23, 705–720.
Creswell, J. W. (2003). Research design: Qualitative, quantitative and mixed approaches. 2nd ed., Los Angeles: Sage Publication.
Creswell, J. W. (2012). Educational research: Planning, conducting, and evaluating quantitative and qualitative research. 4th Ed.,
Boston: Pearson.
Farmer, C., Allen, D. T., Berland, L. K., Crawford, R. H., & Guerra, L. (2012). Engineer your world: An innovative approach to developing
a high school engineering design course. Paper presented at the meeting of the American Society for Engineering Education,
San Antonio, TX.
Fereday, J., & Muir-Cochrane, E. (2006). Demonstrating rigor using thematic analysis: A hybrid approach of inductive and deductive
coding and theme development. International Journal of Qualitative Methods, 5 (1), 80-92.
Fraenkel, J. R., & Wallen, N. E. (2000). How to design and evaluate research in education. New York. NY: Macmillan.
Greenwald, N. L. (2000). Learning from problems. The Science Teacher, 67 (4), 28–32.
Hmelo, C. E., Holton, D. L., & Kolodner, J. L. (2000). Designing to learn about complex systems. Journal of the Learning Sciences, 9 (3),
247-298.
Householder, D. L., & Hailey, C. E. (2012). Incorporating engineering design challenges into STEM courses. Retrieved Feb 13, 2016
from http://ncete.org/flash/pdfs/NCETECaucusReport.pdf
Hynes, M., Portsmore, M., Dare, E., Milto, E., Rogers, C., & Hammer, D. (2011). Infusing engineering design into high school STEM courses.
Retrieved Feb 13, 2016 from http://ncete.org/flash/pdfs/Infusing%20Engineering% 20 Hynes.pdf
Illinois Valley Community College. (2011). STEM activities for middle school students. Oglesby: IL 61348.
Kaliyaperumal, K. (2004). Guideline for conducting a knowledge, attitude and practice (KAP) study. AECS Illumination, 4 (1), 1-9.
Khandani, S. (2005). Engineering design process: Education transfer plan. Retrieved Feb 06, 2016 from http://www.iisme.org/etpex-
emplary.cfm
King, P.M., & Kitchener, K.S. (1994). Developing reflective judgment: Understanding
and promoting intellectual growth and critical thinking in adolescents and adults. San Francisco: Jossey-Bass.
Lewis, T. (2005). Creativity: A framework for the design/problem solving discourse in technology education. Journal of Technology
Education, 17 (1), 35-52.
Ling, S. E., Mahdib, R., Mohamadin, M. I., & Manaf, B. A. (2015). Second chance science education for school leavers. Procedia - Social
and Behavioral Sciences, 167, 288 – 292
Lottero-Perdue, P. S. (2015). Running Head: The engineering design process, responses to failure, Towson University. Presented at
NARST 2015.
Malaysian Digest. (2011). Bringing ‘Transformation’ to Sarawak Rural Schools. Retrieved Jan 9, 2016 from http://www.malaysiandigest.
com/features/18061-bringingtransformation-to-sarawak-rural-schools.html
Massachusetts Department of Education. (2006). Massachusetts science and technology/engineering curriculum framework. Malden,
MA: Author. Retrieved Dec 23, 2015 from http://www.doe.mass.edu/frameworks/scitech/1006.pdf
Mehalik, M., Doppelt, Y., & Schunn, C. (2008). Middle-school science through design-based learning versus scripted inquiry: better
overall science concept learning and equity gap reduction. Journal of Engineering Education, 97 (1), 71-85.
Mentzer, N., Huffman, T., & Thayer, H. (2014). High school student modeling in the engineering design process. International Journal
of Technology and Design Education, 24, 293–316.
Ministry of Education Malaysia. (2014). STEM education: policies and prospects toward achieving international standard and meeting
national development needs. Retrieved March 8, 2016 from www.akademisains.gov.my/.../STEM%20Education_Dr%20Azian.
pdf
Neo, M., Neo, K.T.K. & Tan, H.Y.J. (2012). Applying authentic learning strategies in a multimedia and web learning environment
(MWLE): Malaysian students’ perspective. TOJET: The Turkish Online Journal of Educational Technology, 11 (3), 50-60.
492
(P. 477-493)
Schunn, C. D. (2009). How kids learn engineering: The cognitive science perspective. The Bridge: Linking Engineering and Society,
39 (3), 32-37.
SDC (2011). Sabah Economic Development and Investment Authority Blueprint. Retrieve 21 June 2016 from http://www.sedia.
com.my
Siew, N. M., Amir, N., & Chong, C. L. (2015). The perceptions of pre-service and in-service teachers regarding a project-based STEM
approach to teaching science. SpringerPlus, 4 (8), 1-20.
Siew, N. M., Chong, C. L., & Lee, B. N. (2015). Fostering fifth graders’ scientific creativity through problem-based learning. Journal of
Baltic Science Education, 4 (5), 655–669.
Stanford University Newsletter on Teaching. (2001). Problem-Based Learning. Speaking of Teaching, 11 (1), 1-8.
Straw, S, MacLeod, S., & Hart, R. (2012). Evaluation of the Welcome Trust Camden STEM initiative. NFER: Slough.
Wendell, K., Wright, C., & Paugh, P. (2015). Engineering design as disciplinary discourse: An exploration of language demands and
resources among urban elementary students. Presented at the NARST, Chicago, IL. Retrieved March 23, 2016 from http://
www.eie.org/engineering-elementary/research/ articles.
World Bank. (2010). Malaysia Economic Monitor, Inclusive Growth. World Bank: Bangkok, Thailand. Retrieved April 12, 2016 from http://
siteresources.worldbank.org/INTMALAYSIA/Resources/324392-1288897994959/mem_ nov2010_fullreport.pdf
Yong, X. H., & Phang, F. A. (2015). Science and arts streams students’ scientific epistemological beliefs. International Education Stud-
ies, 8 (13), 88-92.
Appendix A
STEM activities in School A and School B
SCHOOL A SCHOOL B
1 (a) How would you produce an object using paper How would you produce a protective egg device in order to protect
STEM 1 clips and needles that can float on the water surface? a raw egg as it falls to the ground from a height of a 2-storey
1 (b) How would you produce a submarine that may building?
arise and submerge in a closed water bottle?
How would you produce a balancing toy that can stand How would you produce a sliding model that is able to withstand
STEM 2 upright and stable on your finger, like a bird perched on the load of and encourage the acceleration of a tennis ball?
a tree branch?
How would you design and build a model boat that can How would you produce a balloon-powered car that can go the
STEM 3 accommodate as many boxes of biscuits for your com- fastest and furthest away from the starting line?
pany to be transported from Labuan to Kota Kinabalu?
Suppose that marbles are the boxes of biscuits that
need to be brought to Kota Kinabalu.
Received: June 01, 2016 Accepted: August 08, 2016
Nyet Moi Siew PhD, Senior Lecturer, Faculty of Psychology and Education,
University of Malaysia Sabah, Kota Kinabalu, Sabah, Malaysia.
E-mail: snyetmoi@yahoo.com
Henry Goh PhD, Lecturer, Teacher Training Institute, Gaya Campus, Kota
Kinabalu, Sabah, Malaysia.
E-mail: hryg@hotmail.com
Fauziah Sulaiman PhD, Senior Lecturer, Faculty of Science & Natural Resources,
University of Malaysia Sabah, Kota Kinabalu, Sabah, Malaysia.
E-mail: fauziahsulaiman6@gmail.com
493
DETERMINING PROSPECTIVE
BIOLOGY TEACHERS’
COGNITIVE STRUCTURE IN
TERMS OF “BIOTECHNOLOGY”
Abstract. Knowledge concerning continu- Dilek Sultan Acarli

ously improving branches of science such
as biotechnology in particular is difficult
to be understood properly by students, or
they learn it incorrectly. One of the most
important reasons for this is that students
cannot associate in their mind the relevant
concepts. The aim of this research is to
determine prospective biology teachers’
conceptual framework of biotechnol-
Introduction
ogy, and thus determining their cognitive
structure. The data were collected from 34
prospective biology teachers. Considering
The concept of biotechnology, which was first used by Karl Ershy in 1919,
the fact that these prospective biology was derived from the words biology and technology and was defined in vari-
teachers had taken biotechnology course ous ways. Broadly speaking, biotechnology is a branch of science containing
previously, they were selected on the basis all methods used in reproducing a new organism, which naturally does not
of volunteering. Word association test and exist by using the whole or a part of organisms with the help of DNA technol-
draw-and-write technique were used in col- ogy, or in making desired modifications in the genetic structure of an existing
lecting the data. The data obtained were organism. Although it is known that traditional practices of biotechnology
organised on the basis of qualitative con- activities such as making wine or cheese, selecting and reproducing farm
tent analysis. Then the data were grouped animals date back to hundreds of years ago; great progress was made in
into 7 categories (methods/techniques,
biotechnology applications especially in recent years. Today biotechnology
biotechnology applications, genetics, other
applications are actively used in many fields such as agriculture, pharmacy,
branches of science, organisms, meaning
attributed to biotechnology, laboratory)
industry, and solution of environmental problems. Struggle with global
according to word association test, and hunger, producing energy, purification of water, and recycling the waste are
into 4 categories (methods/techniques, among the most widely known applications of biotechnology.
biotechnology applications, genetics, The rapid advances in biotechnology and its applications have caused
organisms) according to draw-and-write serious controversy in cultural, social, political, economic, ethical and
technique. According to the research educational fields (Pardo et al., 2002; Qin & Brown, 2007; Kidman, 2010; La-
findings, almost half of the prospective manauskas & Makarskaité- Petkevičienė, 2008; Lysaght et al., 2006; Massarani
teachers had correct and appropriate & Moreira, 2005; Prokop et al., 2007; Sürmeli & Şahin, 2010). Individuals in a
associations for biotechnology, whereas society need to have certain level of knowledge concerning biotechnology so
a considerable part of them did not have
that they can put forward views or make correct evaluations in relevant argu-
adequate conceptual associations. Also, it
ments. However, research concerning high school and university students
was found that prospective teachers had
misconceptions about biotechnology.
point to the inadequacy of these students’ level of knowledge concerning
Key words: teaching of biotechnology, biotechnology (Chen & Raffan, 1999; Dawson & Schibeci, 2003; Lamanauskas
cognitive structure, draw-and-write tech- & Makarskaitė-Petkevičienė, 2008; Prokop et al., 2007; Temelli, 2006; Turkmen
nique, word association test. & Darcin, 2007). Several pieces of news on biotechnology appear in the press.
Some of the information presented there can cause individuals to learn incor-
rectly or to develop wrong attitudes. Therefore, biology teachers have the
biggest responsibility to make sure that the young have access to correct
Dilek Sultan Acarli information. The subjects related to biotechnology have been in the biology
Hacettepe University, Turkey
curriculum in Turkish secondary education since 1998 (MEB, 1998). The head
council of education and discipline of the Ministry of National Education
included the subject of biotechnology in the biology curriculum of 2013
494
ISSN 1648–3898 DETERMINING PROSPECTIVE BIOLOGY TEACHERS’ COGNITIVE STRUCTURE IN TERMS OF
“BIOTECHNOLOGY”
(P. 494-505)
under the title of Modern Genetic Applications” for 10th graders and under the title of “Genetic Codes and Protein
Synthesis” for 12th graders. The curriculum aimed to ensure that students learn the operation fields /methods of
biotechnology and that they can discuss its effects on the environment, economy and human health (MEB, 2013).
Content of the biotechnology course at university is usually as follows: Definition of concepts of biotechnology
and modern biotechnology, significance and application areas of biotechnology, fundamentals and methods of
gene technology, cloning and transgenic organisms, recent developments in biotechnology, gene technology in
application of agriculture and farming, medical-pharmaceutical, environment, ecology, criminalistics and forensic
medicine, ethics, economic, social, legal and religious aspects of biotechnology and gene technology.
Biology is a course students have difficulty in learning (Bahar et al., 1999; Çimer, 2012). Knowledge concern-
ing continuously improving branches of science such as biotechnology in particular is difficult to be understood
properly by students, or they learn it incorrectly. One of the most important reasons for this is that students can-
not associate in their mind the relevant concepts. A mental configuration about biotechnology should occur in
learners’ mind and meaningful learning should occur so that they can have correct accumulation of knowledge
related with the subject. Meaningful learning occurs by bringing the newly learnt and the previously learnt con-
cepts together in a conscious and organised way and combining them tightly (Güneş et al., 2006). At this point,
the “constructivist” approach, which describes how individuals discover knowledge and how learning occurs,
comes to the fore (Colburn, 2000). According to this approach, knowledge is configured in individuals’ mind by
associating it actively with their prior knowledge and prior experiences (Anderson, 1992). In order for students to
learn meaningfully, firstly their prior knowledge should be uncovered (Pines & West, 1986; Tsai & Huang, 2002).
Thus, exhibiting the associations that individuals set up between concepts they have and their cognitive structure
about the subject would be the most influential way in determining whether or not meaningful learning occurs.
Many techniques such as structured grids, word association tests, tree diagrams, concept maps, and analogy are
used for this purpose (Bahar, 2013).
It was found that there are studies concerning biotechnology in the literature in Turkey, too. These studies
centred mostly on the evaluation of people’s knowledge and attitudes regarding biotechnology (Erdoğan et al.,
2009; Erdoğan et al., 2012; Özel et al., 2009; Uşak et al., 2009); but detailed research on cognitive structures about
biotechnology is missing. This research aims to investigate prospective biology teachers’ conceptual framework
about biotechnology and thus to determine their cognitive structure. It is thought that an assessment of the situ-
ation will provide important clues for teacher training. It is also thought that finding out the cognitive structure
of prospective teachers- who are the teachers of the future- in relation to biotechnology activities would be im-
portant in preparing programmes to determine their inadequacies/incorrect knowledge of biotechnology, and to
increase their knowledge of the topic and to correct their incorrect knowledge. Prospective teachers’ graduation
from university with a correct mental structure developed will ensure that they communicate the subject to their
future students correctly, and thus they help to increase the number of individuals with developed consciousness
in this respect in society.
Within the scope of the research, prospective biology teachers’ conceptual framework about biotechnology
was examined by using draw-and-write technique and word association test, and as such, following questions
were tried to be answered:
1. How is the cognitive structure of prospective biology teachers in relation to biotechnology? How do
prospective biology teachers configure concept of biotechnology in their mind?
(a) Which words evoke concept of biotechnology to prospective biology teachers?
(b) Which visuals evoke concept of biotechnology to prospective biology teachers?
Main limitation of this research is the assumption that participants have enough ability and desire to explain
their conceptual framework related to biotechnology concept through drawings.
General Characteristics of Research
This research is a qualitative one, and was conducted during the 2015-2016 academic year. The word asso-
ciation test and the draw-and-write technique were used as data collection instruments. The collected data were
subjected to content analysis. Based on the categories, frequency was provided.
495
“BIOTECHNOLOGY”
ISSN 1648–3898
(P. 494-505)
Sample of Research
The research group of the research was composed of 34 (29 female and 5 male) fourth and fifth year prospective
biology teachers studying in the biology teaching department of the Educational Faculty of Hacettepe University.
Considering the fact that these prospective biology teachers had taken a course in biotechnology previously, they
were selected on the basis of volunteering. Before the main application, a test application is made to 5 prospective
biology teachers. Thus, appropriateness of the form and the time given to answer are tested.
Data Collection and Instrument
Word association tests (Howards & Korfiatis, 2006) and draw-and-write technique (Backet-Milburn & Mickie,
1999; Patrick & Tunnicliffe, 2010) were employed in this research.
It is quite difficult to describe individuals’ cognitive structure about a topic. Only key words of the topic can
be determined and comments can be made for this. One of the techniques most commonly used in determining
the associations that individuals set up between concepts – and thus in determining their cognitive structures- is
word association test. Word association tests are frequently used especially in the field of science education (Aydın
& Taşar, 2010; Bahar, Johnstone & Sutcliffe, 1999; Bahar & Özatlı, 2003; Ercan et al., 2010; Köseoğlu & Bayır, 2011;
Kurt, 2013; Kurt et al., 2013; Timur & Taşar, 2011; Torkar & Bajd, 2006), and there are also examples in recent years for
their use in different fields (Bahar & Kılıçlı, 2001; Işıklı et al., 2011). The technique of word association test is based
on the assumption that the ideas coming to one’s mind in relation to a stimulant word given are listed (Bahar et
al., 1999; Sato & James, 1999). Individuals are given a stimulant (key) word to determine their cognitive structure.
In this research, individuals were given the concept of “biotechnology” as the key word. The test administered was
composed of two stages. At the first stage, the word “biotechnology” was written ten times one under another,
and the participants were asked to write the word it reminds them next to it each time. The participants were
given 40 seconds for this. The reason for writing the words in the form of a list one under another was to hinder
the risk of chain answers; because if students do not turn back to the key concept each time, they can write the
words reminded by the words they write instead of the words reminded by the key concept- which can damage
the purpose of the test. In this application made at the first stage, there may be products of association which
do not have meaningful relations with the key concept and which are at the level of remembering. In order to be
able to determine this, at the second stage, the participants were asked to write sentences related with the key
concept in 20 seconds. Thus, the purpose was to obtain statements more complex than a word and upper level
statements, and attempts were made to assess the statements in more details on the basis of whether or not they
were scientific or they contained misconceptions.
Draw-and-write technique is a technique aiming to obtain data on individuals’ hidden thoughts about concepts
and on their making sense as well as their attitudes (Backett-Milburn & Mckie, 1999; White & Gunstone, 2000), and
it has been successfully used in many research studies in the field of science (Cetin et al, 2013; Kurt & Ekici, 2013;
Nyachwayaa, et al., 2011; Stafstrom et al., 2002). The participants were, accordingly, asked to describe in 5 seconds
what they knew about biotechnology with figures. They were not imposed a restriction, thus it was made sure that
they stated their thoughts freely. Hence, in-depth analysis of the thoughts about the concept was aimed.
Data Analysis
Responses obtained from the word association test were analysed through qualitative content analysis by
using the content analysis programme MAXqda. In qualitative content analysis, the material of the data is divided
into pieces according to certain category systems (Mayring, 2002). Summative content analysis method, which
was an inductive approach, was employed in this research. The words of response were categorised on the cri-
terion of semantic relations. For the reliability of the research, the data were encoded separately by two experts,
and the final shape was given on the basis of agreement between the two experts. Tables were prepared for the
frequencies about the responses which were grouped under different headings. The words considered to be un-
related and repeated only once were not included in the table. A model describing prospective teachers’ cognitive
structure about biotechnology was established based on the categories and on the frequencies of the responses
in the categories. Moreover, each of the sentences written for biotechnology was analysed one by one, and thus,
an attempt was made to determine the misconceptions related. In the draw-and-write technique also the data
496
“BIOTECHNOLOGY”
(P. 494-505)
coming from this technique in relation to the concept of biotechnology were analysed through summative content
analysis. Participants’ drawings in relation to the concept of biotechnology were brought together under certain
main categories/sub-categories and thus a model was formed. Besides, interesting data obtained through word
association test and through draw-and write technique were given as they were by stating the participant number
in the text and by citing from participants’ drawings and written works.
Results of Research
Findings are presented in two categories as the findings obtained through word association test and as the
findings obtained through draw-and-write technique according to the instruments of measurement. Misconcep-
tions are also included in the presentation of findings.
Findings Obtained through the Word Association Test
In consequence of the analysis of the data concerning prospective teachers’ cognitive structure for the con-
cept of biotechnology, 7 categories were distinguished with the words stated. The categories were: 1) methods/
techniques employed in biotechnology, 2) biotechnology applications, 3) concepts related with genetics reminded
by biotechnology, 4) other branches of science reminded by biotechnology, 5) organisms used in biotechnology
activities, 6) meanings attributed to biotechnology, and 7) concepts related with laboratory which are reminded
by biotechnology. 57 different words used in response to these categories and their frequencies are shown in
Table 1.
Table 1. The distribution of prospective biology teachers’ cognitive structure in terms of “biotechnology”
according to the categories.
Total
Categories Concepts and Frequencies in the Categories
Frequencies
Methods/techniques employed in biotechnology Cloning (18) 72

PCR (18)
Gene technology (10)
Gene transfer (5)
DNA purification (5)
Gene therapy (3)
Genome project (3)
Gene maps (3)
Gene treatment (2)
Gene bank (2)
DNA analysis (2)
DNA manipulation (2)
2. Biotechnology applications GMO (18) 59
Treatment of illnesses (7)
Making yoghurt (7)
Cheese (6)
Corn (6)
Mutant (3)
Milk (3)
Medications (3)
Agricultural product (2)
Forming different organisms (2)
Hybrid animals (2)
497
“BIOTECHNOLOGY”
ISSN 1648–3898
(P. 494-505)
Total
Categories Concepts and Frequencies in the Categories
Frequencies
3. Concepts related with genetics reminded by DNA (12) 57

biotechnology
Recombinant DNA (11)
Gene (9)
Vector (6)
Enzyme (6)
Plasmid (4)
Gene isolation (3)
Protein (2)
Replication (2)
Change of chromosomes (2)
4. Other branches of science reminded by Biology (12) 36
biotechnology
Molecular biology (8)
Molecular genetics (5)
Genetics (5)
Microbiology (3)
Genetic engineering (3)
5. Organisms used in biotechnology activities Bacteria (9) 27
Virus (5)
Cell culture (4)
Plant (3)
Microorganism (3)
Bacteriophage (2)
M13 phage (2)
6. Meanings attributed to biotechnology Technology (4) 12
Innovation (2)
Technique (2)
Development (2)
Curing (2)
7. Concepts related with laboratory reminded by Laboratory (2) 10
biotechnology
Test tube (2)
Incubator (2)
Microscope (2)
Computer (2)
Total 57 words 275 words
An examination of Table 1 shows that methods/techniques used in biotechnology ranks the first with 12 dif-
ferent concepts and 72 frequencies. The concepts of “cloning”, “PCR”, “gene technology” are the most frequently
given responses.
In the category of “biotechnology applications” ranking the second and having 59 frequencies are 11 differ-
ent responses. The most frequently focused responses are “GMO”, “treatment of illnesses”, “making yoghurt”, and
“cheese”.
“Concepts related with genetics” reminded by biotechnology is the third category. The most frequently used
answers among the 10 different answers are “DNA”, “recombinant DNA”, “gene”, “vector”, and “enzyme”.
“Other branches of science” reminded by biotechnology was distinguished as the fourth category. Accordingly,
498
“BIOTECHNOLOGY”
(P. 494-505)
participants’ associations of this category are mostly with the concepts of “biology”, molecular biology”, “molecular
genetics”, and “genetics”.
In the fifth category called “organisms used in biotechnology activities”, the responses of “bacteria” and “virus”
are the responses with the highest frequencies.
“Meanings attributed to biology” is the sixth category. Here, it was found that the concept of biotechnology
reminded participants more of “technology”.
“Concepts related with laboratory reminded by biotechnology” is the seventh category distinguished. Ac-
cordingly, it was found that a small portion of participants wrote the words of “laboratory”, “test tube”, “incubator”,
“microscope” and “computer”.
Prospective biology teachers’ explanations in relation to the concept of biotechnology are as in the fol-
lowing (F= Female students; M= Male students):
Examples for the category of “methods/techniques employed in biotechnology” are:
The gene map of the M13 phage was made during DNA cloning activity, and it was used in activities with
vectors (M1).
Biotechnology is the procedures made for instance in such ways as deriving pure culture, isolating the genes,
DNA cloning, DNA recombination and purification by using technology (F4).
PCR (polymerase chain reaction) is a name given to the reactions which take place within DNA and which are
applied in order to increase the original zone between two segments whose sequence is known (F32).
Examples for the category of “biotechnology applications” are:
People’s illnesses and genetic ties can be detected by looking at their gene maps (F25).
GMOs affect people’s health in a negative way (F26).
The long-term effects of GMOs are not known (F11).
Biotechnology is a science of producing new and different types of organisms by making gene modifications
at molecular level with the development of biology (F29).
Although treatment methods found for illnesses and medications are positive for our health, GMOs are a
threat to human health (F3).
It is a commonly used technology in meeting the food needs which are increasing due to the increase in
population (F8).
Examples for the category of “the concepts related with genetics reminded by biotechnology” are:
Recombinant DNA technology means cutting and destroying with gene technology the DNA molecules which
cannot occur on their own in nature and which are mostly derived from different biological species (F2).
There are intron and exon zones in recombinant DNA (F13).
Examples for the category of “other branches of science reminded by biotechnology” are:
Fields such as molecular biology, genetics and biology help biotechnology (F5).
Biotechnology works in parallel to genetic engineering (F15).
Molecular genetics is a sister science to biotechnology (F18).
Examples for the category of “organisms used in biotechnology activities” are:
Viruses and bacteria are used in biotechnology (F11).
They play on the genes of plants, animals and microorganisms with DNA technology... They reproduce enzymes
resistant to high temperature and present them for use in PCR. They derive these enzymes from bacteria (F21).
Examples for the category of “meanings attributed to biotechnology” are:
These are the activities performed so as to improve the genetic properties of organisms with biological ar-
rangements and so as to re-arrange them (F16).
It means removability of defects in organisms or bettering them with the advances in technology (F7).
499
“BIOTECHNOLOGY”
ISSN 1648–3898
(P. 494-505)
Examples for the category of “concepts related with laboratory reminded by biotechnology” are:
Modern biotechnology usually performs in vitro activities (F30).

Biotechnology studies the working of genes and DNA sequence in the laboratory (F23).
Computers are used in biotechnology for the transfer of data (F21).
On examining prospective teachers’ explanations in relation to the concept of biotechnology, it was found that
they were quite superficial, that they did not offer any explanations apart from the categories obtained through
the word association test, and that they made sentences associated with the words of response.
A model was established for the prospective teachers’ cognitive structures about biotechnology by analysing
the data obtained from the word association test (see Figure 1).
Figure 1: The model for prospective biology teachers’ cognitive framework of biotechnology determined
through word association test.
As can be seen from the model in Figure 1 prepared according to the measurement tools used in this re-
search, prospective biology teachers’ cognitive structures about the concept of biotechnology are divided into 7
categories.
500
“BIOTECHNOLOGY”
(P. 494-505)
Findings Obtained through Draw-and-Write Technique
The data in the draw-and-write technique were divided into 4 groups: methods/techniques used in biotech-
nology (fdrawing=39, fwriting=24 ), biotechnology applications (fdrawing=16, fwriting=5), concepts related with genetics
reminded by biotechnology (fdrawing=15, fwriting=9), and organisms used in biotechnology (fdrawing=14, fwriting=8). Table
2 shows the frequencies for the writings that participants added to their drawings. The most dominant category
for the data concerning both the drawings and the writings is genetics, accordingly.
Table 2. Findings for the concept of biotechnology obtained through draw-and-write technique.
Write
Main Categories Sub-categories Draw (f)
(f)
DNA 15 9
Plasmid 7 3
Vector 6 5
1. Concepts related with genetics reminded by biotech-
Recombinant plasmid 5 5
nology
Nitrogenous organic base 4 2
Chromosome 2 0
Total 39 24
GMO (corn, apple, strawberry) 12 2
Stem cell technology 2 2

2. Biotechnology applications
Treatment of illnesses 2 1
Total 16 5
Gene cloning 7 4
Bacteria division 5 3
3. Methods/
techniques employed in biotechnology
PCR 3 2
Total 15 9
Bacteria 8 4
4. Organisms used in biotechnology activities Cell culture 2 2
Total 10 6
Total 80 44
It was found that 33 out of 34 prospective teachers made drawings for the concept of biotechnology, and
that only 1 did not make a drawing. A detailed analysis of the data showed that the drawings made by prospective
teachers for the concept of biotechnology were unsatisfactory. Only 11 of the participants (%32) were able to make
drawings having the conceptual representation power. The prospective teachers dominantly used DNA helix and
GMO production in their drawings, and they offered explanations in writing. Vectors, plasmids, gene cloning and
bacteria were other drawings which were prominent.
Prospective teachers’ cognitive structures obtained with the draw-and-write technique were also supportive
of the model obtained with the word association test (see Figure 2).
501
“BIOTECHNOLOGY”
ISSN 1648–3898
(P. 494-505)
Figure 2: The model for prospective biology teachers’ cognitive framework of biotechnology determined
through the draw-and-write technique.
Figure 2 shows that prospective biology teachers’ conceptual structures about the concept of biotechnology
were connected with the four categories mentioned: 1) concepts related with genetics reminded by biotechnol-
ogy, 2) biotechnology applications, 3) methods/techniques employed in biotechnology, 4) organisms used in
biotechnology activities.
Discussion
Biotechnology is a discipline which requires comprehensive knowledge of biology and whose benefits and
harms are often argued. Therefore, prospective biology teachers in particular, who are supposed to have biology
literacy, need to have accurate and adequate knowledge of biotechnology. It was found that studies concerning
biotechnology in the relevant literature mostly centred on the evaluation of people’s attitudes towards, views of
and knowledge of the risks, benefits and acceptability of biotechnology applications (Erdoğan et al., 2009; Erdoğan
et al., 2012; Özel et al., 2009; Uşak et al., 2009). Researchers calling attention to the importance of understanding
and evaluating the attitudes displayed by the youth- the adults of the future- towards biotechnology and genetic
engineering performed studies to determine students’ knowledge and attitudes (Chen & Raffan, 1999; Dawson,
2007; Dawson & Schibeci, 2003; Prokop et al., 2007). This research, on the other hand, aimed to uncover prospec-
tive teachers’ cognitive structures about biotechnology and thus to determine their views and thoughts of the
issue in more details. In consequence of the word association test, their cognitive structures were described in
a model composed of 7 categories - namely; methods/techniques employed in biotechnology, biotechnology
applications, concepts related with genetics reminded by biotechnology, other branches of science reminded by
biotechnology, organisms used in biotechnology activities, meanings attributed to biotechnology, and concepts
related with laboratory reminded by biotechnology. According to the draw-and-write technique, however, four
categories were obtained: methods/techniques employed in biotechnology, biotechnology applications, concepts
related with genetics reminded by biotechnology, and organisms used in biotechnology activities. The categories
obtained with both measurement instruments were supportive of each other. This case shows that detailed data
can be collected on the conceptual structure about the same topic by using different measurement tools. Thus,
it was exhibited in this research that rich data can be obtained supporting one another by employing different
measurement tools. In both measurement tools methods/techniques employed in biotechnology, biotechnol-
ogy applications, and concepts related with genetics reminded by biotechnology appeared to be common and
dominant categories.
502
“BIOTECHNOLOGY”
(P. 494-505)
On examining the data obtained through both methods, it was found that the concepts that biotechnology
most frequently reminded prospective teachers were cloning, PCR, GMO, DNA, recombinant DNA, biology, mo-
lecular biology, genes, gene cloning, bacteria, gene technology, treatment of illnesses, plasmid, vectors, yoghurt
making, cheese, and corn. While it was observed that drawings were mostly about DNA helix, gene cloning, bacteria
and GMOs; it was remarkable that prospective teachers stressed the harms of especially GMOs in their drawings.
This was a finding parallel to the ones obtained in the literature. Hence, it was found in studies concerning the
knowledge and attitudes of students who were against the use of biotechnology that the participants thought that
foods containing genetically modified organisms were risky (Başaran et al., 2004; Sürmeli & Şahin, 2010). Sürmeli
and Şahin (2010) found in their research with the students of science teaching, biology department and medical
school that the group with the lowest attitude scores was the students of science teaching; and that the students
of biology department were more supportive of biotechnology activities than the students of science teaching and
of medical school. Prospective teachers, who had concerns about the use of biotechnology for the production of
GMOs, also reflected their views in this research with their drawings and writings, Sürmeli and Şahin (2010) found
that prospective teachers had positive approaches towards biotechnology in terms of producing medications
and treatment of illnesses. In a similar vein, prospective teachers also displayed positive attitudes through their
explanations and drawings in relation to the treatment of illnesses (fword association test =7; fdraw-and-write technique=2) and the
production of medications (fword association test =3).
There are studies indicating the inadequacy of prospective teachers’ associating what they have learnt with
daily life and their inadequacy of comprehending the relations of scientific and unscientific concepts (Özmen, 2003;
Kurt & Ekici, 2013). This was also a finding in this current research. Prospective teachers’ words, their drawings and
writings reflected their scientific concepts related with biotechnology, and their associating these concepts with
their daily life and expressing them in their spoken expressions. What is important here is to configure the concepts
accurately and to state them academically correctly. Yet, it was found in prospective teachers’ words of response
in relation to biotechnology that they could not associate adequately and accurately, and that the unscientific
concepts hey encountered in real life caused them to form misconceptions. Misconceptions were mostly observed
in their drawings. On analysing their drawings, it was found that some of them had misconceptions about the
production of GMOs and about the DNA structures of bacteria/viruses. Two participants (F32 and F18) produced
drawings showing that they confused the application of growth hormone with the production of GMOs. Participant
with the code K6 confused in his/her drawing vaccination of trees with the production of GMOs. Participant K19,
on the other hand, confused the DNAs of bacteria with the DNAs of viruses, and used the concept of virus DNA
for the shape of a plasmid he/she drew.
Conclusions
According to the research findings, almost half of the prospective teachers had correct and appropriate as-
sociations for biotechnology, whereas a considerable part of them did not have adequate conceptual associations.
Consequently, it may be said that conceptual structure was not adequately formed in prospective teachers in terms
of biotechnology which is very important for these participants, who are to become biology teachers after gradu-
ation from university. Therefore, teacher training programmes should be revised, prospective teachers’ conceptual
knowledge should be improved and meaningful learning as well as interest in the subject should be supported.
Yet, it should be remembered that learning concepts is the outcome of the interaction of the scientific knowledge
which is taught and of knowledge learnt in daily life. Research findings indicate that prospective teachers’ cogni-
tive structures about biotechnology are also influenced by mass media (drawings of GMOs, stem cells, etc.). In this
case, teaching biotechnology by relating it with daily life issues and raising consciousness in prospective teachers
in relation to the developments in biotechnology will ensure that they can approach the ideas they encounter in
a critical way, and that they can correctly understand the risks and benefits of the applications. Besides, visits to
research centres in biotechnology classes and inviting experts into these classes will provide prospective teachers
with knowledge about the developments and applications in biotechnology.
Another finding obtained in this research was that prospective teachers had misconceptions about biotech-
nology. These misconceptions cannot be ignored in teacher training in particular. It may be assured that they
have permanent and accurate cognitive structures by educating them in using the cognitive strategies effectively
during their training. In addition to that, efforts should be made to remove the existing misconceptions by using
appropriate teaching strategies.
503
“BIOTECHNOLOGY”
ISSN 1648–3898
(P. 494-505)
References
Anderson, O. R. (1992). Some interrelationships between constructivist models of learning and current neurobiological theory,
with implications for science education. Journal of Research and Science Teaching, 29, 1037-1058.
Aydın, F., & Taşar, M.F. (2010). An investigation of pre-service science teachers’ cognitive structures and ideas about the nature
of technology. Ahi Evran University Journal of Kırşehir Education Faculty, 11 (4), 209-221.
Backett-Milburn, K., & Mckie, L. (1999). A critical appraisal of the draw and write technique. Health Education Research Theory &
Practice, 14 (3), 387–398.
Bahar, M., Johnstone, A. H., & Hansell, M. H. (1999). Revisiting learning difficulties in biology. Journal of Biological Education, 33,
84–86.
Bahar, M., & Kılıçlı, F. (2001). Kelime ilişkilendirme testi yöntemi ile Atatürk ilkeleri arasındaki bağların araştırılması [Investigation
of the relationship between the principles of Ataturk with the word association test method]. X. Ulusal Eğitim Bilimleri
Kongresi. Abant İzzet Baysal Üniversitesi, Bolu.
Bahar, M., & Özatlı, S. (2003). Kelime ilişkilendirme testi yöntemi ile lise 1. sınıf öğrencilerinin canlıların temel bileşenleri konusun-
daki bilişsel yapılarının araştırılması [Investigation of high school first grade students’ cognitive structure regarding the
basic components of life with word association test]. Balikesir University Journal of Institute of Science, 5 (2), 75-85.
Bahar, M. (2003). Misconceptions in biology education and conceptual change strategies. Educational Sciences: Theory & Practice,
3 (1), 27- 64.
Başaran, P., Kılıç, B., Soyyiğit, H., & Şengün, H. (2004). Public perceptions of GMO’s in food in Turkey: A pilot survey. Journal of
Food, Agriculture and Environment, 2 (3-4), 25-28.
Chen, S. Y., & Raffan, J. (1999). Biotechnology: Students knowledge and attitudes in the UK and Taiwan. Journal of Biological
Education, 34 (1), 17-23.
Colburn, A. (2000). Constructivism: Science education’s “grand unifying theory”. The Clearing House, 74 (1), 9-12.
Çetin, G, Özarslan, M., Işık, E., & Eser, H. (2013). Students’ views about health concept by drawing and writing technique. Energy
Education Science and Technology, Part B, 5 (1), 597-606.
Çimer, A. (2012). What makes biology learning difficult and effective: Students’ views? Educational Research and Reviews, 7 (3),
61-71.
Dawson, V. (2007). An exploration of high school (12-17 year old) students’ understanding of and attitudes towards biotechnol-
ogy processes. Research in Science Education, 37 (1), 59-73.
Dawson, V., & Schibeci, R. (2003). Western Australian high school students’ attitudes towards biotechnology process. Journal of
Biological Education, 38 (1), 7-12.
Ercan, F., Tasdere, A., & Ercan, N. (2010). Observation of cognitive structure and conceptual changes through word associations
tests. Journal of Turkish Science Education, 7 (2), 138-154.
Erdoğan, M., Özel, M., Uşak, M., & Prokop, P. (2009). Development and validation of an instrument to measure university students’
biotechnology attitude. Journal of Science Education & Technology, 18 (3), 255–264.
Erdoğan, M., Özel, M., BouJaoude, S., Lamanauskas, V., Uşak, M., & Prokop, P. (2012). Assessment of preservice teachers’ knowledge
of and attitudes regarding biotechnology: A cross cultural comparison. Journal of Baltic Science Education, 11 (1), 78–93.
Güneş, T., Güneş, H., & Çelikler, D. (2006). Fen bilgisi öğretmenliği programı biyoloji II ders konularının öğretilmesinde kavram
haritası kullanımının öğrenci başarısı üzerine etkileri [The effect of concept map using on student succes in the teaching
of biology II subjects presented in science teacher programme]. Ahi Evran University Journal of Kırşehir Education Faculty,
7 (2), 39-49.
Hovardas, T., & Korfiatis, K. J. (2006). Word associations as a tool for assessing conceptual change in science education. Learning
and Instruction, 16, 416-432.
Işıklı, M., Taşdere, A., & Göz, N. L. (2011). Kelime ilişkilendirme testi aracılığıyla öğretmen adaylarının Atatürk ilkelerine yönelik
bilişsel yapılarının incelenmesi [Investigation teacher candidates’ cognitive structure about principles of Ataturk through
word association test]. Uşak University Journal of Social Sciences, 4 (1), 50-72.
Kidman, G. (2010). What is an ‘interesting curriculum’ for biotechnology education? Students and teachers opposing views.
Research in Science Education, 40, 353-373.
Köseoğlu, F., & Bayır, E. (2011). Kelime ilişkilendirme test yöntemiyle kimya öğretmen adaylarının gravimetrik analize ilişkin bilişsel
yapılarının incelenmesi [Examining cognitive structures of chemistry teacher candidates about gravimetric analysis through
word association test method]. Trakya University Faculty of Education, 1 (1), 107-125.
Kurt, H. (2013). Biology student teachers’ cognitive structure about “living thing”. Educational Research and Reviews, 8 (12), 871-
880.
Kurt, H., & Ekici, G. (2013). Biyoloji öğretmen adaylarının “bakteri” konusundaki bilişsel yapılarının ve alternatif kavramlarının
belirlenmesi [Determining biology student teachers’ cognitive structure and alternative concepts on the concept of
“bacteria”]. Turkish Studies, 8, 885-910.
Kurt, H., Ekici, G., Aksu, Ö., & Aktaş, M. (2013). Determining cognitive structures and alternative conceptions on the concept of
reproduction (The case of pre-service biology teachers).Creative Education, 4 (9), 572-587.
Lamanauskas, V., & Makarskaitė-Petkevičienė, R. (2008). Lithuanian university students’ knowledge of biotechnology and their
attitudes to the taught subject. Eurasia Journal of Mathematics, Science & Technology Education, 4 (3), 269-277.
Lysaghta, T., Rosenberger, P. J., & Kerridgea, I. (2006). Australian undergraduate biotechnology student attitudes towards the
teaching of ethics. International Journal of Science Education, 28 (10), 1225-1239.
504
“BIOTECHNOLOGY”
(P. 494-505)
Massarani, L., & Moreir, I. (2005). Atitudes towards genetics: A case study among Brazilian high school students. Public Under-
standing of Science, 14, 201-212.
Mayring, P. (2002). Einführung in die qualitative Sozialforschung. Beltz Verlag. Weinheim und Basel, 170s.
MEB (Milli Eğitim Bakanlığı) [Republic of Turkey Ministry of National Education] (1998). Orta Öğretim Genel Müdürlüğü Lise Ders
Programları [General Directorate of Secondary Education High School Curriculums], MEB yayınları [Publication of Ministry of
Education], Cilt 1 [Volume 1], Ankara.
MEB (Milli Eğitim Bakanlığı)[Republic of Turkey Ministry of National Education] (2013). Milli Eğitim Bakanlığı Talim ve Terbiye Kurulu
Başkanlığı Ortaöğretim Biyoloji Dersi (9.,10.,11. ve 12. sınıflar) Öğretim Programı [Secondary Education, Curriculum of Biology
Course (9th, 10th, 11th. and 12th grades). Ankara, 2013. Retrieved November 13, 2015, from http://mebk12.meb.gov.tr/
meb_iys_dosyalar/31/01/972850/dosyalar/2013_07/05032334_biyoloji_912.pdf
Nyachwayaa, J. M., Mohameda, A-R., Roehriga, G. H., Woodb, N. B., Kernc, A. L., & Schneiderd, J. L. (2011). The development of an
open-ended drawing tool: An alternative diagnostic tool for assessing students’ understanding of the particulate nature
of matter. Chemistry Education Research and Practice, 12 (2), 121-132.
Özel, M., Erdoğan, M., Uşak, M., & Prokop, P. (2009). High school students’ knowledge and attitudes regarding biotechnology
applications. Educational Sciences: Theory & Practice, 9 (1), 297–328.
Özmen, H. (2003). Kimya öğretmen adaylarının asit ve baz kavramlarıyla ilgili ilgilerini günlük olaylarla ilişkilendirebilme düzeyleri
[Chemistry student teachers’ levels of linking their knowledge with daily life about acid and base concepts]. Gazi University
Kastamonu Education Journal, 11 (2), 317-324.
Pardo, R., Midden, C., & Miller, J. D. (2002). Attitudes toward biotechnology in the european union. Journal of Biotechnology, 98
(1), 9-24.
Patrick, P. G., & Tunnicliffe, S. D. (2010). Science teachers’ drawings of what is inside the human body. Journal of Biological Educa-
tion, 44 (2), 81-87.
Pines, A., & West, L. (1986). Conceptual understanding and science learning: An interpretation of research within sources-of
knowledge framework. Science Education, 70 (5), 583-604.
Prokop, P., Lešková, A., Kubiatko, M., & Diran, C. (2007). Slovakian students’ knowledge of and attitudes toward biotechnology.
International Journal of Science Education, 29 (7), 895-907.
Qin, W., & Brown, J. L. (2007). Public reactions to information about genetically engineered foods: Effects of information formats
and male/female differences. Public Understand of Science, 16, 471- 488.
Stafstrom, C. E., Rostasy, K., & Minster, A. (2002). The usefulness of children’s drawings in the diagnosis of headache. Pediatrics,
109 (3), 460-472.
Sürmeli, H., & Şahin, F. (2010). University students’ attitudes towards biotechnological studies. Education and Science, 35 (155),
145-157.
Temelli, A. (2006). Lise öğrencilerinin genetikle ilgili konulardaki kavram yanılgılarının saptanması [Determination of misconcep-
tions concerning genetic subjects of high school students’]. Gazi University Kastamonu Education Journal, 14 (1), 73-82.
Timur, B., & Taşar, M. F. (2011). Developing pre-service science teachers’ cognitive structures about technology: Word associa-
tion test (WAT). Western Anatolia Journal of Educational Sciences, Special Issue: Selected papers presented at WCNTSE 2011,
131-138.
Torkar, G., & Bajd, B. (2006). Trainee teachers’ ideas about endangered birds. Journal of Biological Education, 41 (1), 5-8.
Tsai, C. C., & Huang, C. M. (2002). Exploring students’ cognitive structures in learning science: A review of relevant methods.
Journal of Biological Education, 36, 163-169.
Turkmen, L., & Darcin, E. S. (2007). A comparative study of Turkish elementary and science education major students’ knowledge
levels at the popular biotechnological issues. International Journal of Environmental & Science Education, 2 (4), 125-131.
Uşak, M., Erdoğan, M., Prokop, P., & Özel, M. (2009). High school and university students’ knowledge and attitudes regarding
biotechnology. Biochemistry and Molecular Biology Education, 37 (2), 123–130.
Dilek Sultan Acarli PhD., Hacettepe University, Faculty of Education, Department of

Secondary Science and Mathematics Education, Biology Education
Programmes, 06800 Beytepe/ Ankara, Turkey.
Fax: +90 312 297 86 00.
E-mail: dsultan@hacettepe.edu.tr
Website: http://www.beabd.hacettepe.edu.tr/
505
PEDAGOGICALLY DESIRABLE
SCIENCE EDUCATION: VIEWS
ON INQUIRY-BASED SCIENCE
EDUCATION IN TURKEY
Abstract. The aim of this research is to

investigate the aspects of science educa-
Yasemin Ozdem-Yilmaz,
tion, which are pedagogically desirable for Bulent Cavas
the individual in the society of 21st century.
The research was conducted by Curricular
Delphi Study method in three rounds with
international comparisons. In the first
round, an open-ended survey was used,
and in the next two rounds the instru-
ment was structured to answer the refined
research questions of the study, such as
the priorities and practices towards the
inquiry-based science education. The paper
Introduction
reports on the findings of a survey collected
from 125 stakeholders of science educa-
The current goal of science education in many countries around the
tion, including scientists, science educators,
world is to educate people towards scientifically literate society (e.g. Coun-
and education administrators from Turkey.
cil of Ministers of Education, Canada [CMEC], 2013; European Commission
In the results of the differentiated analy-
[EC], 2007; Ministry of National Education in Republic of Turkey [MEB], 2013;
ses according to the sample groups, all
National Research Council [NRC], 2011), and this aim is emphasized for more
stakeholders emphasize the role of science
than 14 years in the course of international studies such as PISA and TIMSS
education in the survival of a country. They
(Mullis, Martin, Ruddock, O’Sullivan, & Preuschoff, 2009; Organisation for
all put a great emphasis on the curriculum.
Economic Co-Operation and development [OECD], 2001, 2004, 2007, 2012,
The stakeholders emphasized the signifi-
2015). Scientific inquiry, both the understanding about and the practice of
cance of engagement with the interdiscipli-
which, is considered as an integral element of scientific literacy (Lederman
nary relations of the sciences, their findings
et al., 2014).
and their perspectives with regard to their
In various educational policy documents, the development of scientific
role in enhancing individual intellectual
inquiry has been viewed as an important goal in conjunction with scientific
personality development. It is suggested
literacy, including the understanding of basic science concepts and the
that scientific inquiry includes the ability
practice of scientific methods (Achieve, Inc., 2015; American Association for
to consolidate the inquiry processes with
the Advancement of Science [AAAS], 1993; Lederman et al., 2014; NRC, 2000,
scientific knowledge, scientific reasoning
2011, 2013). Research emphasize that students’ engagement with scientific
and critical thinking to advance scientific
inquiry processes motivate them to be physically as well as mentally active
knowledge.
participants, help them acquire science and its significance in life, and assist
Key words: inquiry-based, science educa- them to develop competencies to function effectively especially in Science-
tion, Delphi method. Technology-Engineering and Mathematics (STEM) related careers (Bartos &
Lederman, 2014; Gräber, 2012).
There are, however, such differences in definition of inquiry that even-
tually lead to tensions between the philosophical contemplations and how
Yasemin Ozdem-Yilmaz
Gaziosmanpasa University, Turkey
these are implemented in the classroom (Cavas, 2012; Treagust, 2014). For
Bulent Cavas example, National Science Education Standards (NRC, 2000) differentiate
Dokuz Eylul University, Turkey between the abilities to do inquiry and having a fundamental understanding
about specific characteristics of scientific inquiry (Lederman et al., 2014). The
506
ISSN 1648–3898 PEDAGOGICALLY DESIRABLE SCIENCE EDUCATION: VIEWS ON INQUIRY-BASED SCIENCE
EDUCATION IN TURKEY
(P. 506-522)
term inquiry refers to at least three distinct categories of activities as reported by Minner, Levy and Century (2010)
(1) what scientists do methodologically, (2) how students learn through active participation, and (3) a pedagogical
approach that teachers employ (Minner et al., 2010). Won (2010) reported that while in some cases, IBSE means
a practice of “engaging students in interesting hands-on materials to teach scientific processes”, in another case,
IBSE was considered as “a collaborative group work as a means to build independence and democratic attitudes”
(p.187).
In view of this uncertainty, the essential research aim was to set up an empirical-based common ground among
different groups of stakeholders in Turkey about the aspects of science education, which are pedagogically desir-
able for the individual in the society of 21st century. In Turkey, the science education curriculum is motivated to
reach ideal international standards of education implemented in Europe, North America and East Asia (Koc, Isiksal,
& Bulut, 2007). Thus, Turkish science curriculum has representative particularities with the international context.
Moreover, this study took place as part of a European project entitled Professional Reflection Oriented Focus on
Inquiry-based Learning and Education through Science [PROFILES]. Therefore, the data collected in Turkey were
shared with the European partners and the commonalities occurred in comparison to other partners were resulted
with shared categories and concepts that were tested in consecutive rounds. Thus, the results of this research- al-
though conducted in a national context- also reflect those shared categories and concepts in Europe, and so are
a good initiative to discuss and understand the components of IBSE in international context that are most valued
and pedagogically desired by different stakeholders.
Background
The philosophy of IBSE finds its antecedents in constructivist approaches to learning, claiming that IBSE was
born out of the works of Jean Piaget, Lev Vygotsky, and David Ausubel (Alake-Tuenter, Biemans, Tobi, & Mulder,
2013; Cakir, 2008; Liang & Gabel, 2005; Minner et al., 2010). John Dewey, founder of the experiential learning peda-
gogy, puts the teaching of science through inquiry at the centre of his educational philosophy (Abd-El-Khalick et
al., 2004).
In the 1960s, Joseph Schwab called for the teaching of science grounded on the idea of “inquiry into inquiry.”
(Abd-El-Khalick et al., 2004). Since then, the range of terms and phrases used to characterize the role of inquiry in
science education are too widened to allow a clearly formulated philosophy about the nature of scientific inquiry
to be implemented in science classrooms (Cavas, 2012; Abd-El-Khalick et al., 2004; Treagust, 2014). For example, in
Europe, recent reports on science education emphasize IBSE as a key factor to be considered in science education
(e.g. All European Academies [ALLEA] Working Group, 2012; Tucker, 2011). IBSE is defined in the Rocard report of
the EC (2007) (p.108) as “the intentional process of diagnosing problems, critiquing experiments, and distinguish-
ing alternatives, planning investigations, researching conjectures, searching for information, constructing models,
discussing with peers and forming coherent argument” (Linn, Davis, & Bell, 2004, p.4). The definition of inquiry in
the ALLEA report covers the abilities similar to the process of scientific research and discovery.
In the USA, the policy documents (e.g. Benchmarks for Science Literacy (AAAS, 1993); Inquiry and The National
Science Education Standards (NRC, 2000)) focuses on articulating how students can acquire science as a body of
knowledge, a set of practices, and a process of participation in those practices (Furtak, Seidel, Iverson, & Briggs,
2012; Minner et al., 2010). For example, in the NGSS front matter, inquiry-based approaches to science teaching are
described as those that “students will themselves engage in the practices and not merely learn about them second-
hand” (NRC, 2013, p.11). The term “practices” is used in the document instead of inquiry or skills to emphasize that
engaging in scientific inquiry comprises coordination of cognitive, social, and physical practices simultaneously.
However, there is still confusion in science education community about the meaning of ‘inquiry’ and the methods
to implement IBSE (Anderson, 2002; Dunne, 2013).
In Turkey, the curriculum was revised recently in 2013, and it has been implemented starting in primary
schools in autumn 2013 and later in lower secondary and in upper secondary schools (MEB, 2013). In the science
curriculum for grades 4-8, the formal and informal learning environments are suggested to be arranged based
on the principles of IBSE indicated in the curriculum. The inquiry process is conceived not only as “discovery and
experimentation”, but also as “explanation and argumentation” process (p.III). Students are expected to construct
strong arguments based on sound justification regarding natural and physical environment. Kizilaslan, Sozbilir,
and Yasar (2012) reported that among the few studies conducted about IBSE as an educational approach in Turkey,
the focus is mostly on the effect of IBL on learning some science topics. However, how IBSE could be effectively
507
EDUCATION IN TURKEY
ISSN 1648–3898
(P. 506-522)
integrated into teaching science is still a mystery. As a consequence, the studies on the Turkish context indicate
that IBSE is not widely used as a teaching and learning strategy in Turkey (Kizilaslan et al., 2012).
In overall, IBSE has been defined in policy documents mostly by the skills and abilities that aid students en-
gage in science as a set of practices and a process of participation in those practices. For example, the documents
refer to diagnosing problems, formulating questions and hypotheses, and engage in scientifically oriented question
for identifying problem at hand, and suggest discovery and experimentation as well as critiquing experiments as
activities and cognitive processes. The emphasis on forming coherent arguments, arguing rationally, searching
–often jointly with others- for evidence and accepting the confrontation of ideas, and seeking to reach conclusions,
justify explanations are common cognitive processes that students engage in during IBSE. Nevertheless, there are
diverse ways to perform IBSE suggested in these documents. For example, one document might have students
to plan investigations, research conjectures, search for information, and construct models about a scientific phe-
nomenon, while another skips these steps and asks for epistemic practices such as connecting explanations to
scientific knowledge, communicate and justify explanations, formulate explanations from evidence, give priority
to evidence in response to questions, discussing with peers develop explanations and contrast this condition with
students who also developed explanations distinguishing alternatives. These variations in the way that IBSE has
been conceived lead to inconsistent inferences made in research syntheses about the effectiveness of the approach.
A consequence of this diversity, as Abd-El-Khalick et al. (2004) correctly points out, is that the construct does not
have its intended power “as an overarching theme for imagining, developing, coupling, and enacting curricular
goals, pedagogical tools, and assessment practices that would allow the actualization of current reforms vision in
science education” (p.414).
Therefore, the current move in science education is towards more detailed descriptions of IBSE in which the
teacher and students engage rather than using the term as a one-for-all construct for educational reforms (Furtak
et al., 2012). An important consideration in this regard is the need to establish an exchange between the differ-
ent groups involved with the sciences and science education because innovative practices for science teaching
incorporate socio-cultural considerations. Thus, involving various sets of stakeholders, who may hold divergent
views, is a means to recognize these socio-cultural considerations and their influence on the image of science and
the image of science in school (i.e. science education) held by students (as well as by their teachers and society).
In this line, PROFILES project, which focuses on open inquiry approaches as a major teaching target, called for
stakeholder involvement and interaction to fulfil this need to bridge the gap between science education research
community, science teachers and local actors in order to facilitate the uptake of inquiry based teaching by sup-
porting a stakeholder network and facilitating cooperation (PROFILES, 2010).
In response to this call, in this study, we aimed at illustrating views of stakeholders on desirable IBSE in the
contradiction opinions of different selected communities from society in Turkey. The research was conducted by
Curricular Delphi Study method, which offers comprehensive insights into the views of different stakeholders in
society regarding how science education should be.
Research Questions
The central question of the PROFILES Curricular Delphi Study on Science Education is formulated as follows:
“Which aspects of science education do you consider meaningful and pedagogically desirable for the individual in
the society of today and in the near future?” The question is general to science education mainly because the term
inquiry is not well-defined and all stakeholders might not – most were not-being familiar to the IBSE. For example,
in Turkey, inquiry is generally known as research-inquire method of learning amongst educators; however, most of
the people including students are either not familiar to the term or have different conceptions of inquiry.
Therefore, the research questions formulated in accordance with the research purpose are:
1. What are the preferred topics/themes and methods for teaching and learning science according to
stakeholders’ views?
2. Which characteristics of a desirable IBSE do the participants consider as being necessary and important
for IBSE?
3. What differences or similarities appear in the general assessments between different sub-sample
groups?
In regard to the differences expected in terms of the priority and practice of the components of inquiry-based
508
EDUCATION IN TURKEY
(P. 506-522)
science education as well as the differences expected between the sub-samples, the following hypotheses were
formulated; H1: There are significant differences between priority and practice assessments of the components
of IBSE. H2: The sub-sample groups are significantly different from each other in their assessments of priority and
practice of the components of IBSE.
Delphi Method
Delphi method is a systematic approach to involving a wide range of stakeholders and bringing together their
views (Linstone & Turoff, 1975). This method promises specific insights about aspects that are difficult to determine
and to predict (Häder, 2009).
The PROFILES Curricular Delphi Study on Science Education is structured through three independent, yet inter-
related rounds. The participating stakeholders are provided with feedback after each round regarding outcomes
(Bolte, 2008; Bolte et al., 2012; Schulte & Bolte, 2012). In the first round, the participants’ views about aspects of
pedagogically desirable science education were collected according to question and answer format in 7 open-
ended questions. In the second round, these categories were compared with those, which are reported by other
partners in the PROFILES project; reorganized into statement bundles; and were reported back to the participants
for further assessment. In the third round, the identified concepts were fed back to the participants for further
assessment from two different perspectives analogously to the second round (Schulte & Bolte, 2014).
Sample of the Research
In the first round of the Delphi study, in total, the number of stakeholders involved in Turkey was 135. The
stakeholders were selected by convenient sampling, in which the stakeholders were the ones that were accessible
and volunteered to contribute to the study. The only selection criterion was that the stakeholder must be a member
of one of the groups (scientists, science teachers, science teacher educators, education administrators, and stu-
dents) that were identified through a consensus by project partners. Out of 134 (79 male; 55 female) participants
from the first round, a total number of 125 (79 male; 46 female) participants (93% of the participants from the first
round) took part in the second round. The participation rate in the second round for the groups compared to the
first round participation were high (93%-100%), with the exception of scientists and other (71%). Table 1 shows
the sample structure over all three rounds of the study. Also, the participation rate with regard to the drop-out
between the second and third round is shown.
With regard to the second round, all the sub-sample groups feature with 100% or higher response rate in the
third round, with the exception of education administrators (70%). Hence, there were 125 participants who took part
in all of the three rounds of the study. The refinement of the stakeholders’ views as well as reaching to a consensus
on the critical goals of the science education is important first, in developing the practices, ideas, intentions and
objectives to facilitate the uptake of innovative science teaching and the enhancement of scientific literacy and
second in allowing a stronger partnership between various stakeholders and science teachers.
509
EDUCATION IN TURKEY
ISSN 1648–3898
(P. 506-522)
Table 1. Sample structure.
Number of participants
Sample group Round 1 Round 2 Round 3
Students 29 (12f; 17 m) 29(12f; 17 m) 29(12f; 17 m)

Education Students 25(15f; 10 m) 30(15f; 15 m) 25(15f; 10 m)
Teachers
Trainee teachers & Teachers 25(15f; 10 m) 16 (15f; 11 m) 21(14f; 7 m)

Teacher Educators 26(19f; 7 m) 25 (19f; 6 m) 25 (19f; 6 m)
Education Administrators 9 (8f; 1 m) 10 (8f; 2 m) 7 (5f; 2 m)
Scientists and Other 21(11f; 10 m) 15(10f; 5 m) 18 (10f; 8m)
Total 135(80f; 55 m) 125(79f; 46 m) 125(75f; 50 m)
Therefore, the data in the 3rd round were collected from students (19 elementary school (8th grade)- 10 high
school (10th grade)); pre-service (5 biology, 3 chemistry, 5 physics, 12 elementary science) and in-service teachers
(21 science teachers with 1 to 20 years of experience); teacher educators (25-from different universities), education
administrators (2 program developers, 2 education politicians, 3 administers from ministry of education) , scientists
(5 biologist, 5 physicist, 5 chemists), and others (3 engineers who are also employers in industry).
Design of the Questionnaires and Data Collection
1st round
The questionnaire used in the first round of the research was adapted by collaboration with other partners
included in the PROFILES project. The questionnaire consists of 7 questions (Appendix). The questions addressed
the following areas:
•• Preferred topics/themes and methods for teaching and learning science (question 1)
•• Skills and attitudes that should be encouraged in school science (questions 2, 5, 6)
•• Suggestions for improving science education/scientific literacy of individuals (questions 3, 4, 7)
Scientific literacy is a complex construct (DeBoer, 2000), so its enhancement would not be possible by refer-
ring to the different aspects one by one. Therefore, the questionnaire was open-ended and sought for stakehold-
ers view without giving clue of what IBSE should be. Thus, in the first round of the research, in order to picture
responses, aspects were constructed following the results from other partners in the project and refer to guidelines
and aspects of modern science education stated in didactic literature (Bybee, McCrae, & Laurie, 2009; Fensham,
2009; Häußler et al., 1980). In other words, we could not define and ask what the stakeholders think inquiry-based
science education is. However, the stakeholders’ responses were compared with the IBSE conceptualization in the
policy documents, PROFILES model, as well as research on IBSE. It was important that our views do not impinge
on participants’ responses. Therefore, little guidance was given as to the expected content of responses in the first
round of the Delphi study.
2nd round
The second round of the PROFILES Curricular Delphi Study on Science Education is about critically consider-
ing and reflecting the findings which resulted from the analyses of the individually formulated responses of the
participants in the first round. Thus, the categorisation resulted at the end of the comparisons of first round among
PROFILES partners is fed back to the participants and combined with specific tasks and questions. The second
round also helps to identify empirically based conceptualization with regard to the stakeholders’ desired and
perceived models of science education since the stakeholders picture what is important and relevant for science
education in this round.
510
EDUCATION IN TURKEY
(P. 506-522)
3rd round
The concepts of desirable science education that were identified in the second part of the second round were
reported back to all the participants in the third round for their assessment. The concepts were assessed from two
points of view (“priority” and “practice”) on a six-tier scale. We followed a structured model to make sure that ob-
tained data is comparable. Since this research was EU project, the experts’ reviews from 19 different EU countries
have been taken into consideration in each country and then final version of questionnaire was implemented in
Turkey and each partner countries as well. The objectivity, reliability and validity of the instruments have been
checked by the experts of science educators, too.
Data Analysis
Since the Delphi questionnaire used in the first round of the research is composed of open-ended questions,
a qualitative approach to data analysis was preferred. The statements were analysed step-by-step as indicated in
the model described by Schulte and Bolte (2012). Consequently, the results of the data gathered involved codes
and categories drawn in an interpretive nature by the researchers involved in the study.
For the assessment of the categories, the data from 2nd and 3rd round questionnaires were analysed by descrip-
tive and variance analytical methods. The analyses took into account both the priority and practice assessments
individually as well as determined the priority-practice differences by subtracting the practice values from the
priority values (Bolte, 2008). The analyses were made from two perspectives: general assessment of the concepts of
science education by the total sample and by the sub-sample groups. The assessments of the concepts were tested
for statistically significant differences by applying the Wilcoxon signed-rank test. Statistically significant differences
between the assessments of the different sub-sample groups were identified through the Mann-Whitney-U test.
Results of Research
1st Round
In the first round of the research, the participants expressed their views with regard to the pedagogically
desirable science education in 7 questions in total. The qualitative analysis of the statements led to a classification
system consisting of 5 parts. The second part was subdivided onto two parts (IIa and IIb). All in all, it contains a
number of 157 categories.
The codes (Table 2) were agreed with 97% interrater reliability and the ones for those we had a conflict
were either discussed further to reach an agreement or removed completely. Initial categories were determined
in a meeting where three researchers from the project team in Turkey participated, and adjusted later when we
contributed to the internationally agreed list of categories. In addition to the categories identified in the analysis
of the first round data and as a result of the international comparisons, the results obtained in the first round in
Turkey yielded one more category, which is Learning Environments (Table 2). This category is highly emphasized
as a major element of learning in science classrooms by the respondents in Turkey.
Learning environments describe the contexts where science learning takes place and suggested to be ef-
fective. A total number of 1203 statements was determined. In overall, Table 3 represents the most and the least
emphasized keywords by different participants regarding each question. Additionally, as a response to question 3;
is there anything you would like to see change in school science lessons, participants proposed alterations in school-
science from various perspectives.
511
EDUCATION IN TURKEY
ISSN 1648–3898
(P. 506-522)
Table 2. Overview of the categories for the analysis of the experts’ statements.
I: Situations, II: Fields III: IV: Methodical V: Learning

contexts, aspects N=25 environments
motives N=19 IIa: (Basic) IIb: Qualification N=24
concepts and Scientific N=42
topics N=37 fields and
perspectives
N=10
Daily life Laws, theories, Physics Science Entrepreneurship laboratory

related facts Biology process skills Obtaining result visual
knowledge Scientific Chemistry knowledge Explaining book
Knowledge developments Technology application Discussing material
based on Exam topics Geology critical Collaboration field
curricula theoretical History of thinking skills Planning equipment
Science society content Science knowledge experiment technological
technology current Astronomy transfer process learning audio
issues technology Philosophy positive practice resource
daily life philosophy of science… attitude Search… Instrument…
occurrences scientists problem
Technological Future solving
developments… profession… Cognitive
skills…
The participants emphasized curriculum-based changes, student-related changes, context-based changes,

teacher-related changes, and policy-related changes. For example, a pre-service teacher responded that;
“I don’t think that the curriculum has been implemented with a constructivist approach in schools as intended. The
main two reasons are the inadequacy of teachers in constructivist methods, and the lack of technical supply in class-
rooms. Indeed, the science teaching in schools does not allow students to be active participants.”
The pre-service teacher in this example emphasized teacher’s role in the implementation of the curriculum
and recommended teacher-related changes. At the same time, s/he talked about the learning environment, and
suggested the context to be more student-centred.
The participants gave examples from diverse areas when they are asked to describe a situation, context or
topic where they think scientific literacy is useful. The participants highlighted the daily-life occurrences (12.6%) as
the area where scientific literacy is the most needed. Although they mentioned about scientific content (0.7%), this
area of knowledge remained insignificant among the other areas, which are mostly socio-scientific. For example,
a scientist responded that;
“Scientific literacy is helpful to critically think about the situations that would have an impact on our life. For example,
a scientifically literate person should be able to think critically and make an informed decision about the effects of
nuclear or hydroelectric power plants on environment, human life, and economy.”
This response emphasized the importance of having an adequate scientific literacy in socio-scientific decision-
making.
512
EDUCATION IN TURKEY
(P. 506-522)
Table 3. Overview of the categories for the analysis of the experts’ statements.
Teacher Education
Teacher Student Scientist
Keywords n n% educator administer
% % %
% %
Physics 69 51.1 58 38 27 86 44
Biology 63 46.7 54 21 35 81 44
Top
significant area or
field of science
Question 1-
Chemistry 62 45.9 46 59 31 48 44
Philosophy of science 1 0.7 0 3 0 0 0
Bottom
Science related knowledge 1 0.7 0 3 0 0 0

Nature of Science 1 0.7 0 0 4 0 0
knowledge application 49 36 18 38 27 38 44
science process skills 40 30 30 21 35 33 44
Top
competencies
Question 2-
significant
critical thinking skills 32 24 30 59 31 29 44

the use of technology 3 2 2 0 8 0 11
Bottom
scientific thinking 1 1 2 7 4 0 0
scientific literacy 1 1 2 3 4 0 11
Understanding 24 18 22 38 27 29 44
Question 5- scientific
Thinking 22 16 22 21 35 5 44
Top
literacy skills
Knowing 14 10 12 59 31 10 44
Nature of science 5 4 6 0 0 0 11
Bottom
Science and society 5 4 8 3 0 0 0

Consistency 1 1 2 3 0 0 0
Curiosity 28 21 28 38 27 14 44
Question 6- scientific
Interest 22 16 8 21 35 10 44
Top
literacy attitudes
Technology 18 13 6 59 31 10 44
Guidance 2 1 0 0 4 0 0
Bottom
Motivation 2 1 0 0 4 0 0
Collaboration 8 6 2 0 4 14 0
curriculum issues 46 34 36 38 27 38 44
scientific activities 34 25 16 21 35 38 44
Top
suggestions
Question 7-
linking daily life 26 19 24 59 31 14 44

scientific process skills 9 7 6 7 4 14 0
Bottom
financial support 6 4 2 3 4 5 11
early science education 5 4 6 0 0 0 11
To find answers to the guiding questions of the second round, we calculated the descriptive statistics (mean,
standard deviation, n) regarding the priority-assessment differentiated according to the groups of students, teach-
ers, educators, scientists and adults (including the groups of teachers, educators and scientists).
2nd Round
Table 4 shows those categories that feature particularly high or particularly low mean values with regard to
the total sample, listing the top three and bottom three categories in descending order. In the table, considering
the top categories, it can be noted that they refer to aspects rather related to qualifications and learning environ-
ments. Although it is not seen in the table, overall results also show that almost half of the (41 out of 76) categories
range above the theoretical mean value (4.9) of the scale.
513
EDUCATION IN TURKEY
ISSN 1648–3898
(P. 506-522)
Table 4. Top and bottom three categories of the assessments by the total sample.
Category Mean Value
Laboratory work 5.4

Top Problem solving/Critical questioning 5.3
Priority Making decisions /opinion-forming / reflection 5.3

Assessments Intellectual development 4.4
Bottom Subject content focussed learning 4.3
Media / current issues in society 4.3
Learning related to Science - biology 4.4
Top Subject content focussed learning 4.4
Practice Learning related to Science - chemistry 4.4

Assessments Intellectual development 3.2
Bottom History of sciences 3.2
School areas for animal/ plant growth 3.0
School areas for animal/ plant growth 1.9
Top Rational thinking / analysing / drawing conclusions 1.8
PPD Making decisions /opinion-forming / reflection 1.8

Assessments Matter / particle concept 0.4
Bottom Chemical changes/reactions 0.4
Subject content focussed learning -0.1
The PPD results from subtracting the practice values from the priority values (∆PPD= Xpriority- Xpractice). A
notable finding is that the PPD of the category “Subject content focussed learning” features a negative value (-0.1).
This indicates that in the opinion of the participants, the presence of this aspect in science educational practice
exceeds the importance the participants attribute to this aspect. The comparison of the mean values of the differ-
ent sample groups and the values of the significance test (Mann-Whitney-U-Test) regarding the comparisons of
the mean values in different sample groups showed a tendency towards slightly higher values in the responses
of the ‘adult’ groups (Table 5).
Table 5. Significance values and mean values of the assessments by the different sample groups and the total
sample.
Number of Significance & values Average Mean values
Categories S/T S/E S/Sc T/E T/Sc E/Sc S T E Sc Total
5 1 1 1
priority 3 0 4.41 5.01 5.13 4.76 4.83
p <.03 p= .00 p= .04 p= .01
I: contexts, 6 1 5 1
practice 0 0 4.43 3.70 3.88 3.55 3.89
motives and situ- p=.00 p=.00 p <.05 p=.04
ations
9 8 9 2 3
PPD 0 0.37 1.11 1.29 1.24 1.01
p=.00 p=.00 p <.03 p <.04 p <.05
6 2 2 1 2 1
priority 4.53 5.15 5.04 4.88 4.90
p <.03 p <.01 p=.01 p= .04 p <.04 p= .05
11 6 12 2
II: concepts practice 0 0 4.30 3.66 3.38 3.16 3.63
p=.00 p=.00 p <.05 p <.04
and topics
18 15 12 2 1
PPD 0 0.29 1.23 1.39 1.53 1.11
p=.00 p=.00 p <.05 p <.04 p=.02
514
EDUCATION IN TURKEY
(P. 506-522)
Number of Significance & values Average Mean values
6 1 3 2
priority 0 0 4.69 5.38 5.09 5.25 5.10
p <.01 p=.02 p <.03 p <.04
11 13 9 1
III: practice 0 0 4.30 3.55 2.99 3.34 3.54
p=.00 p=.00 p <.03 p=.04
qualifications
15 14 12 1
PPD 0 0 0.51 1.70 2.08 1.98 1.57
p=.00 p=.00 p <.02 p=.04
3 1 1
priority 0 0 0 4.84 4.97 4.66 5.14 4.91
p <.01 p=.04 p=.04
IV: 9 11 7 8 5
practice 0 4.34 3.58 2.64 3.50 3.52
methodological p=.00 p=.00 p <.05 p=.00 p <.05
aspects
11 11 8 1
PPD 0 0 0.48 1.30 1.90 1.66 1.33
p=.00 p=.00 p <.03 p=.01
4 1 3 1 2
priority 0 4.60 5.19 4.89 4.87 4.89
p <.02 p=.02 p <.03 p=.01 p <.05
V: learning 4 5 3 1
practice 0 0 4.15 5.19 4.89 4.87 4.89
environments p=.00 p=.00 p <.03 p=.05
6 5 3 1
PPD 0 0 0.44 1.96 1.74 1.60 1.43
p=.00 p=.00 p <.03 p=.03
The highest number of statistically significant differences (p=.05) in the assessments appears in the comparison
between the responses of the students and teachers. In detail, in the priority assessment, 24 statistically significant
differences in each pair comparison, accounting each for 45% of all cases in the respective pair comparison were
found.
In the practice assessments, there were 41 statistically significant differences in each pair comparison, account-
ing each for 31% of all cases in the respective pair comparison. In priority-practice differences (PPD), the highest
number of statistically significant differences in the practice assessments was 59 statistically significant differences
in each pair comparison, accounting each for 35% of all cases in the respective pair comparison.
All in all, a number of 53 statistically significant differences can be found with regard to the priority assessments;
131 statistically significant differences can be found with regard to the practice assessments; and 167 statistically
significant differences can be found with regard to the PPD assessments. On the other hand, it is hard to say that
the groups significantly differ from each other without controlling the threats to multiple comparisons problem.
3rd Round
In the third round, the data of the second round from all partners in the project were gathered together
and analysed by means of hierarchical cluster analyses, using the Ward method and squared Euclidian distance
(see Bolte & Schulte, 2013) by the coordinator institution of the project first to refine the understanding of IBSE
among partner countries, and second to allow group comparisons. As a result of this analysis, the categories were
collapsed into three concepts of science education; Concept A, which is Awareness of the sciences in current, social,
globally relevant and occupational contexts in both educational and out-of-school settings, Concept B, which is Intel-
lectual education in interdisciplinary scientific contexts and Concept C, which is General science-related education and
facilitation of interest in contexts of nature, everyday life and living environment (Appendix).
In regard to the general assessment of the three concepts of science education by the total sample, it can
be said that the concept referring to general science-related education (Concept C) is seen as the most important
and also most realized concept of the three concepts. However, the priority-practice differences show that in the
current science education practices all three concepts fall short of their given priority. The smallest gap occurs for
the awareness of the sciences in different contexts (Concept A), the largest for the concept related to intellectual
education related concept (Concept B).
515
EDUCATION IN TURKEY
ISSN 1648–3898
(P. 506-522)
In the following part, the general assessments by the different sub-sample groups are presented. Table 6 shows
the results of these assessments and the results from the significance test with respect to differences between the
assessments of the different concepts (Wilcoxon signed-rank test).
Table 6. Mean values of the general assessments by the sub-sample groups and significance test values.
Mean values Significance values
Concept A Concept B Concept C Average A/B A/C B/C
Students (S) 5.62 5.69 5.90 5.74 0.589 0.011 0.063

Teachers (T) 5.39 5.15 5.69 5.41 0.087 0.002 0.000
Priority
Ed. Adm.’s (A) 4.70 5.20 5.30 5.07 0.059 0.084 0.705
Scientists (Sc) 5.13 5.27 5.67 5.36 0.608 0.033 0.177
S 4.72 4.62 4.83 4.72 0.693 0.750 0.210
T 3.56 3.31 3.87 3.58 0.096 0.033 0.001

Practice
A 3.10 2.70 3.20 3.00 0.157 0.739 0.025

Sc 3.40 3.20 3.60 3.40 0.405 0.490 0.153
S 0.90 1.07 1.07 1.01 0.448 0.602 0.971

T 1.83 1.85 1.82 1.83 0.918 0.934 0.791
PPD
A 1.60 2.50 2.10 2.07 0.038 0.334 0.157

Sc 1.73 2.07 2.07 1.96 0.096 0.218 0.875
Table 7 shows in addition the results from the significance test with respect to differences between the as-
sessments by the different sub-sample groups (Mann-Whitney-U-Test).
Table 7. Mean values of the assessments by the sub-sample groups and significance test values.
Significance values Mean values
Concepts S/T S/E S/Sc T/E T/Sc E/Sc S T E Sc Total
Concept A 0.046 0.010 0.018 0.052 0.163 0.428 5.62 5.39 4.70 5.13 5.21
Priority
Concept B: 0.001 0.162 0.031 0.416 0.747 0.693 5.69 5.15 5.20 5.27 5.33
Concept C: 0.067 0.106 0.149 0.612 0.965 0.699 5.90 5.69 5.30 5.67 5.64
Concept A: 0.000 0.003 0.003 0.464 0.756 0.723 4.72 3.56 3.10 3.40 3.70
Practice
Concept B: 0.000 0.001 0.005 0.147 0.518 0.511 4.62 3.31 2.70 3.20 3.46
Concept C: 0.003 0.004 0.005 0.198 0.535 0.452 4.83 3.87 3.20 3.60 3.88
Concept A: 0.002 0.111 0.028 0.626 0.870 0.817 0.90 1.83 1.60 1.73 1.52
PPD
Concept B: 0.012 0.014 0.025 0.160 0.552 0.493 1.07 1.85 2.50 2.07 1.87
Concept C: 0.017 0.093 0.017 0.585 0.498 0.910 1.07 1.82 2.10 2.07 1.77
Similar to the result of the assessments by the total sample, it can be noted for the sub-sample groups as
well that all assessments range above the theoretical mean values of 3.5. As the average of the mean values
for the three concepts shows, the teachers, education administrators and scientists feature for the three con-
cepts a tendency towards higher priority assessments. With respect to the practice assessments by different
sub-sample groups, it is noticeable for all three concepts that the group of education administrators deviate to
516
EDUCATION IN TURKEY
(P. 506-522)
rather lower values, whereas the group of students seems to assess the realization in a more positive way. The
assessments of the sub-sample groups feature no mean value above the theoretical mean value of 3.5, except
the group of students.
For all three concepts, the smallest gaps between the priority and practice assessments appear in the
group of students. The largest gaps between priority and practice assessments appear both in the group of
education researchers and scientists. The largest gap between priority and practice in the assessments of the
students appears for Concept B and Concept C. In the assessments of the three other sub-sample groups, the
largest gaps appear for Concept B.
All in all, the four sub-sample groups seem to be very homogeneous in their assessments of the realization
of the three concepts. However, the assessments made within the sub-sample groups in the PPD assessments
are very similar. Statistically significant differences appear between Concept A/ Concept B (education research-
ers).
Discussion
The first questionnaire provided the stakeholders’ conceptualizations of the pedagogically desirable sci-
ence education. The conceptualizations were further refined in the 2nd and 3rd round to understand the specific
attributions made to the inquiry-based science education by stakeholders.
In the second and third rounds, care was taken to ensure, as far as possible, that participants’ own words
were returned and that participants had ample opportunity to comment on any interpretation in our conflation
of their responses. As a result, the analyses show general tendencies as well as specific focuses of the partici-
pants and the different groups of participants. The category frequency regarding the whole sample showed
that 13 categories were mentioned particularly often (by 20% or more than 20% of the participants) while 100
categories were mentioned particularly rarely (by 5% or less than 5% of the participants). The numbers show
the huge diversity of views expressed by the stakeholders.
As similar to other curricular Delphi studies in the science education (Bolte, 2008; Häußler et al., 1980; Mayer,
1992; Osborne, Collins, Ratcliffe, Millar, & Duschl, 2003), the PROFILES Curricular Delphi Study is divided into three
rounds. In the first round, differentiated analyses according to the sample groups showed that, all stakeholders
emphasize the role of science education in the survival of a country. They all put a great emphasis on the cur-
riculum, but the topics they suggest to include in the curriculum do not show a noteworthy differentiation from
the current curriculum. The problems with teacher training and integration of nature of science are stayed in the
backstage in this manner. As expected, the group of teacher educators responded in the most differentiated way
(137 categories), followed by the group of teachers (119 categories). As these groups are those who research,
read and interact with the educational resources the most, the diversity in their views can be considered as a
reflection of the extent of their involvement in educational matters as an essential part of their profession.
From the stakeholders’ perspective, the results show resemblance to the current consensus of the science
education community on the definition, aspects and the role of inquiry-based instruction in science. However,
the significance of the results of this curricular Delphi study on science education lie in the prioritization and
the realization of these categories in the views of students, teachers, educational administrators, and scientists.
In this regard, the analyses of the second and third round provide important suggestions for the goals, content
and qualifications of IBSE. Furthermore, these outcomes, which clarify the meaning of IBSE as it has been hold
by different stakeholder, make it useful by adapting the views into the development of teaching and learning
modules, as well as for teacher training strategies and materials (Schulte & Bolte, 2012). For example, in Turkey,
based on the results of the study, workshops with more than 40 science and technology teachers, who teach
at elementary schools (grades 6-8) were held and empirical-based teaching modules for IBSE were constructed
(Ozdem & Cavas, 2012).
The assessments made by the stakeholders represent evidence to the significance of engagement with
the interdisciplinary relations of the sciences, their findings and their perspectives with regard to their role in
enhancing individual intellectual personality development. Furthermore, the stakeholders’ views suggest that
dealing with the IBSE not only serve the acquisition of science-related basic knowledge but more than this; it also
helps to the process of applying knowledge in the sciences in interdisciplinary contexts by means of facilitating
analytical abilities; such as, creative and abstract thinking, critical questioning, rational thinking, analysing, draw-
ing conclusions, and fostering the ability to take differentiated perspectives. In other words, scientific inquiry
517
EDUCATION IN TURKEY
ISSN 1648–3898
(P. 506-522)
is not only related only to the widely known science processes, but it includes the ability to consolidate these
processes with scientific knowledge, scientific reasoning and critical thinking to advance scientific knowledge
(Lederman & Lederman, 2012). Whereas, the stakeholders put emphasis on the significance of these aspects
of inquiry-based science education, the science education policies rarely give specific attention to this role of
IBSE. As a result, the research notes to students’ positive engagement with and success in the inquiry processes
such as setting up meaningful experiments, but also to the limitations in students’ understanding of data, as
well as the relationships among data, evidence, and conclusions (Keys, 1998; Krajcik, Blumenfeld, Marx, Bass, &
Fredricks, 1998).
Conclusion and Implications
The aspects of science education, which are pedagogically desirable for the individual in the society of 21st
century and investigated in this research, were found to be mostly consistent with the contemporary science
education policies. The list of pedagogically desirable concepts, qualifications and contexts points out to a future,
which is technology dominated and risky in all means. Since nobody is able to imagine what the future will be
like except being highly technological, the worries about safety, health, and being ethical and independent take
over in the responses of the stakeholders. In conclusion, this research set forth the need to revise the science
education policies and the curricula to adapt the forthcoming technological advances especially in the areas of
health and environment, and their unpredictable consequences.
In this research, we have presented a research methodology to collect vital data on how different stakehold-
ers view pedagogically desirable inquiry-based science education. The findings of this study are significant to
inform the science education community, educational administrators, teacher educators, and the public about
the goals, contexts, perspectives and qualifications of inquiry-based science education that has been already
carried out in science classrooms in order to close the gap between the one pedagogically desired from the
perspective of diverse participants. We propose that only then the expected student learning outcomes from
IBSE can be reasonably proposed in the policy documents as the reflection of multiple modes of pedagogically
desirable inquiry-based science education. It is suggested that this information is going to be valuable in further
evaluation of the efficacy of inquiry-based science education as a teaching and learning strategy in science.
Acknowledgement
This work was supported by the FP7 programme of the European Commission under Grant Agreement
no. 266589.
References
Abd-El-Khalick, F., Boujaoude, S., Duschl, R., Lederman, N. G., Mamlok-Naaman, R., Hofstein, A., ... Tuan, H. L. (2004). Inquiry in
science education: International perspectives. Science Education, 88(3), 397-419. doi: 10.1002/sce.10118
Achieve, Inc. (2015). Next generation science standards. Retrieved from http://www.nextgenscience.org/next- generation-
science-standards
Alake-Tuenter, E., Biemans, H. J., Tobi, H., & Mulder, M. (2013). Inquiry-based science teaching competence of primary school
teachers: A Delphi study. Teaching and Teacher Education, 35, 13-24.
All European Academies Working Group Science Education. (2012). Summary: A renewal of science education in Europe: Views
and actions of national academies. Berlin, Germany: ALLEA ALL European Academies.
American Association for the Advancement of Science. (1993). Benchmarks for science literacy. New York, NY: Oxford.
Anderson, R. D. (2002). Reforming science teaching: What research says about inquiry. Journal of Science Teacher Education,
13(1), 1-12.
Bartos, S. A., & Lederman, N. G. (2014). Teachers’ knowledge structures for nature of science and scientific inquiry: Conceptions
and classroom practice. Journal of Research in Science Teaching, 51 (9), 1150-1184.
Bolte, C. (2008). A conceptual framework for the enhancement of popularity and relevance of science education for scientific
literacy, based on stakeholders’ views by means of a curricular Delphi study in chemistry. Science Education International,
19 (3), 331-350.
Bolte, C., & Schulte, T. (2013). Stakeholders’ views on science education in Europe: method and first insights of the profiles
international curricular Delphi study on science education. In C. P. Constantinou, N. Papadouris, & A. Hadjigeorgiou (Eds.),
Proceedings of the European Science Education Research Association Conference, (pp. 1400-1411). Nicosia, Cyprus.
Bolte, C., Streller, S., Holbrook, J., Rannikmae, M., Hofstein, A., Mamlok Naaman, R., & Rauch, F. (2012). Introduction into the
518
EDUCATION IN TURKEY
(P. 506-522)
PROFILES project and its philosophy. In C. Bolte, J. Holbrook, F. Rauch (Eds.), Inquiry-based Science Education in Europe:
Reflections from the PROFILES Project (pp. 31-42). Berlin, Germany: Freie Universität. Retrieved from https://ius.uni-klu.
ac.at/misc/profiles/files/Profiles%20Book%202012_10.pdf.
Bybee, R. W., McCrae, B., & Laurie, R. (2009). PISA 2006: An assessment of scientific literacy. Journal of Research in Science
Teaching, 46 (8), 865-886.
Cakir, M. (2008). Constructivist approaches to learning in science and their implication for science pedagogy: A literature
review. International Journal of Environmental and Science Education, 3 (4), 193– 206.
Cavas, B. (2012). The meaning of and need for “Inquiry Based Science Education (IBSE)”. Journal of Baltic Science Education, 11(1),
4-6.
Council of Ministers of Education, Canada. (2013). Pan-Canadian assessment program 2013: Report on the Pan-Canadian as-
sessment of science, reading, and mathematics. Toronto, Canada: CMEC Secretariat.
DeBoer, G. (2000). Scientific literacy: Another look at its historical and contemporary meanings and its relationship to science
education reform. Journal of Research in Science Teaching, 37 (6), 582–601.
Dunne, J. (2013). An exploration of inquiry based science education in Irish primary schools. (Doctoral dissertation). University
of Limerick, Ireland. Retrieved from https://ulir.ul.ie/handle/10344/3600.
European Commission. (2007). Science education now: A renewed pedagogy for the future of Europe. Brussels, Belgium: Euro-
pean Commission.
Fensham, P. J. (2009). Real world contexts in PISA science: Implications for context-based science education. Journal of Research
in Science Teaching, 46 (8), 884-896.
Furtak, E. M., Seidel, T., Iverson, H., & Briggs, D. C. (2012). Experimental and quasi-experimental studies of inquiry-based sci-
ence teaching: A meta-analysis. Review of Educational Research, 82 (3), 300-329.
Gräber, W. (2012). PROFILES: Some ideas and comments. In C. Bolte, J. Holbrook, F. Rauch (Eds.), Inquiry-based science education
in Europe: Reflections from the PROFILES project (pp. 222-225). Berlin, Germany: Freie Universität Berlin.
Häder, M. (2009). Delphi-Befragungen: ein Arbeitsbuch (2 Aufl.) [ Delphi surveys: A workbook (2nd ed.)]. Wiesbaden, Germany:
Verlag für Sozialwissenschaften.
Häußler, P., Frey, K., Hoffmann, L., Rost, J., & Spada, H. (1980). Physikalische bildung: Eine curriculare Delphi-studie. Teil I: Verfahren
und ergebnisse. IPN-Arbeitsbericht 41 [Physical education: A curricular Delphi study. Part I: methods and results. IPN work
report 41]. Kiel, Germany: IPN.
Keys, C. W. (1998). A study of grade six students generating questions and plans for open-ended science investigations.
Research in Science Education, 28 (3), 301-316.
Krajcik, J., Blumenfeld, P. C., Marx, R. W., Bass, K. M., Fredricks, J., & Soloway, E. (1998). Inquiry in project-based science class-
rooms: Initial attempts by middle school students. Journal of the Learning Sciences, 7 (3-4), 313-350.
Kizilaslan, A., Sozbilir, M., & Yasar, M. D. (2012). Inquiry based teaching in Turkey: A content analysis of research reports. Inter-
national Journal of Environmental & Science Education, 7 (4), 599-617.
Koc, Y., Isiksal, M., & Bulut, S. (2007). Elementary school curriculum reform in Turkey. International Education Journal, 8 (1),
30-39.
Lederman, N. G., & Lederman, J. S. (2012). Nature of scientific knowledge and scientific inquiry: Building instructional capacity
through professional development. In B. Fraser, K. G. Tobin, & C. J. McRobbie (Eds.), Second international handbook of
science education (pp. 335-359). Dordrecht, the Netherlands: Springer.
Lederman, J. S., Lederman, N. G., Bartos, S. A., Bartels, S. L., Meyer, A. A., & Schwartz, R. S. (2014). Meaningful assessment of
learners’ understandings about scientific inquiry: The views about scientific inquiry (VASI) questionnaire. Journal of
Research in Science Teaching, 51 (1), 65–83.
Liang, L. L., & Gabel, D. L. (2005). Effectiveness of a constructivist approach to science instruction for prospective elementary
teachers. International Journal of Science Education, 27 (10), 1143-1162.
Linn, M. C., Davis, E. A., & Bell. P. (2004). Inquiry and technology. In M.C. Linn, E.A. Davis, & P. Bell (Eds.), Internet Environments
for Science Education (pp. 3-28). Mahwah, NJ: Lawrence Erlbaum Associates.
Linstone, H. A., & Turoff, M. (1975). The delphi method: Techniques and applications. Boston, MA: Addison-Wesley.
Mayer, J. (1992). Formenvielfalt im biologieunterricht: Ein vorschlag zur neubewertung der formenkunde [Variety of shapes in
biology teaching: A proposal for revaluation of form customer]. Kiel, Germany: IPN.
Ministry of National Education in Republic of Turkey (2013). İlköğretim kurumları (ilkokullar ve ortaaokullar) fen bilimleri dersi
(3, 4, 5, 6, 7 ve 8. sınıflar) öğretim programı [Elementary (primary and middle school) Science Curriculum (3rd, 4th, 5th,
6th, 7th and 8th grades)]. Ankara, Turkey: TTKB.
Minner, D. D., Levy, A. J., & Century, J. (2010). Inquiry‐based science instruction—what is it and does it matter? Results from
a research synthesis years 1984 to 2002. Journal of Research in Science Teaching, 47 (4), 474-496.
Mullis, I. V. A., Martin, M. O., Ruddock, G. J., O’Sullivan, C. Y., & Preuschoff, C. (2009). TIMSS 2011 assessment frameworks. Boston,
MA: TIMMS & PIRLS International Study Center.
National Research Council (2000). Inquiry and the national science education standards. Washington, DC: National Academy
Press.
National Research Council (2011). A framework for K-12 science education: Practices, crosscutting concepts, and core ideas.
Washington, DC: National Academy Press.
National Research Council (2013). Next generation science standards: For States, By States. The Washington, DC: National
Academy Press.
519
EDUCATION IN TURKEY
ISSN 1648–3898
(P. 506-522)
Organisation for Economic Co-operation and Development. (2001). Literacy skills for the world of tomorrow: Further
results from PISA 2000. Retrieved from http://www.oecd- ilibrary.org/education/literacy-skills-for-the-world-of-
tomorrow_9789264102873-en.
Organisation for Economic Co-operation and Development. (2004). Learning for tomorrow’s world – first results from PISA 2003.
Retrieved from http://www.oecd- ilibrary.org/education/learning-for-tomorrow-s-world_9789264006416-en.
Organisation for Economic Co-operation and Development. (2007). Executive summary PISA 2006: Science competencies for
tomorrow’s world. Retrieved from http://www.oecd- ilibrary.org/content/book/9789264040014-en.
Organisation for Economic Co-operation and Development. (2012). PISA 2009 results: What students know and can do – student
performance in reading, mathematics and science (Volume I). Retrieved from http://www.oecd-ilibrary.org/education/
pisa-2009-results- what-students-know-and-can-do_9789264091450-en.
Organisation for Economic Co-operation and Development. (2015). PISA 2015: Draft science framework. Retrieved from http://
www.oecd.org/pisa/pisaproducts/Draft%20PISA%202015%20Science%20Framew ork%20.pdf.
Osborne, J., Collins, S., Ratcliffe, M., Millar, R., & Duschl, R. (2003). What ‘‘Ideas-about-Science’’ should be taught in school sci-
ence? A Delphi study of the expert community. Journal of Research in Science Teaching, 40 (7), 692–720.
Ozdem, Y. & Cavas, B. (2012). PROFILES continuous professional development workshops in Turkey. In C. Bolte, J. Holbrook,
F. Rauch (Eds.), Inquiry-based science education in Europe: Reflections from the PROFILES project, Proceedings of the First
International PROFILES Conference (pp. 163-165). Berlin, Germany: Freie Universität Berlin.
Professional reflection-oriented focus on inquiry-based learning and education through science consortium. (2010). The
PROFILES Project. Retrieved from http://www.profilesproject.eu.
Schulte, T., & Bolte, C. (2012). European stakeholders’ views on inquiry based science education–method of and results from
the international PROFILES curricular Delphi study on science education round 1. In C. Bolte, J. Holbrook, F. Rauch (Eds.),
Inquiry-based science education in Europe: Reflections from the PROFILES project, Proceedings of the First International
PROFILES Conference (pp. 42-51). Berlin, Germany: Freie Universität Berlin.
Schulte, T., & Bolte, C. (2014). Case Studies on Science Education based on Stakeholders’ Views Obtained by Means of a Na-
tional/International PROFILES Curricular Delphi Study. In C. Bolte, J. Holbrook, R. Mamlok-Naaman, & F. Rauch, (Eds.).
Science Teachers’ Continuous Professional Development in Europe. Case Studies from the PROFILES Project. Berlin, Germany:
Freie Universität.
Treagust, D. F. (2014) Inquiry learning and other evidence-based approaches. In C. Bolte, & F. Rauch (Eds.), Enhancing inquiry-
based science education and teachers’ continuous professional development in Europe: Insights and reflections on the PRO-
FILES project and other projects funded by the European Commission (pp. 10-11). Berlin, Germany: Alpen-Adria Universität
Klagenfurt (Austria). ISBN: 978-3-9816683-1-5
Tucker, M. S. (2011). Standing on the shoulders of giants: An American agenda for education reform. National Center on Education
and the Economy. Retrieved from http://www.ncee.org/wp-content/uploads/2011/05/Standing-on-the-Shoulders-of-
Giants- An-American-Agenda-for-Education-Reform.pdf.
Won, M. (2010). Issues in inquiry-based science education seen through Dewey’s theory of inquiry (Doctoral dissertation), Uni-
versity of Illinois, Urbana-Champaign. Retrieved from https://www.ideals.illinois.edu/bitstream/handle/2142/14574/
Won_Mihye.pdf.
520
EDUCATION IN TURKEY
(P. 506-522)
Appendix
The questionnaire used in the 1st round of Delphi study
1. Which area or field of science do you think is significant that a secondary school graduate should
know?
2. What competencies do you think school science should encourage/develop?
3. Is there any change that you would like to see in school science lessons?
4. Describe a situation or context or topic where you think scientific literacy is useful.
5. List the skills you think someone who is scientifically literate should have.
6. List the attitudes you think someone who is scientifically literate should have.
7. What do you think would improve the development of scientific literacy?
Description of the concepts
Concept A: Awareness of the Concept B: Intellectual Concept C: General science- related

sciences in current, social, globally education in education and facilitation of interest in
relevant and occupational contexts interdisciplinary scientific contexts of nature, everyday life and
in both contexts living environment
educational and out-of- school
settings
Emotional personality Intellectual personality Society / public concerns,

development, Media / current development, Science - Students’ interests, Education / general
Situations,
contexts,
motives:
issues, Global references, interdisciplinary, Technology personality development, Nature / natural

Occupation / career, Out of phenomena, Everyday life, Medicine
school learning, Curriculum
framework
Concept A: Awareness of the Concept B: Intellectual Concept C: General science- related

sciences in current, social, globally education in education and facilitation of interest in
relevant and occupational contexts interdisciplinary scientific contexts of nature, everyday life and
in both contexts living environment
educational and out-of- school
settings
History of the sciences, Interdisciplinary, Scientific Safety and risks,

Occupations / occupational inquiry, Current scientific Consequences of technological
fields, Industrial processes, research, Limits of scientific developments, Ethics / values,
(Basic) concepts,
Cycle of matter, Earth knowledge, Terminology, Food / nutrition, Health,

perspectives:
themes and
sciences, Development / Matter / particle concept, Matter in everyday life,

growth Structure / function / Environment
properties, Chemical reactions, Models,
Technical devices, System, Interaction,
Energy, Mathematics
Empathy / sensibility, Applying knowledge / creative and abstract Acting reflectively and
Perception / awareness / thinking, Formulating scientific questions / responsibly, Judgement /
observation, Social skills / hypotheses, Factual opinion-forming / reflection,
Qualifications:
teamwork, Knowledge about knowledge, Critical Motivation and interest,

science-related occupations, questioning, Rational thinking / analysing / Comprehension / understanding, working self-
Communication skills, Finding drawing conclusions, Experimenting dependently / structured /
information, Reading precisely.
comprehension
521
EDUCATION IN TURKEY
ISSN 1648–3898
(P. 506-522)
Samples question from the 2nd round questionnaire seeking opinions on science education priorities and
current practices
Situations, contexts and motives Which priority should the To what extent are the respective
Please assess the following categories respective aspects have in science educa- aspects realized in current science
according to the two questions stated. tion? lessons?
1= very low priority 1 = to a very low extent

2= low priority 2 = to a low extent
3= rather low priority 3 = to a rather low extent
4= rather high priority 4 = to a rather high extent
5= high priority 5 = to a high extent
6= very high priority 6= to a very high extent
Intellectual development [1] [2] [3] [4] [5] [6] [1] [2] [3] [4] [5] [6]
Development of the Person [1] [2] [3] [4] [5] [6] [1] [2] [3] [4] [5] [6]
The interests of Students [1] [2] [3] [4] [5] [6] [1] [2] [3] [4] [5] [6]
Samples question from the 3rd round questionnaire seeking opinions on science education priorities and current
practices
Concepts Which priority should the To what extent are the respective
Please assess the following categories respective aspects have in science aspects realized in current science
according to the two questions stated. education? lessons?
1= very low priority 1 = to a very low extent

2= low priority 2 = to a low extent
3= rather low priority 3 = to a rather low extent
4= rather high priority 4 = to a rather high extent
5= high priority 5 = to a high extent
6= very high priority 6= to a very high extent
Concept A: Awareness of the sciences

in current, social, globally relevant and
[1] [2] [3] [4] [5] [6] [1] [2] [3] [4] [5] [6]
occupational contexts in both educa-
tional and out-of-school settings
Concept B: Intellectual education in
[1] [2] [3] [4] [5] [6] [1] [2] [3] [4] [5] [6]
interdisciplinary scientific contexts
Concept C: General science-related
education and facilitation of interest in
[1] [2] [3] [4] [5] [6] [1] [2] [3] [4] [5] [6]
contexts of nature, everyday life and
living environment
Yasemin Ozdem-Yilmaz PhD., Assistant Profesor, Gaziosmanpasa University, Tokat, Turkey.

E-mail: yasemin.ozdem@gop.edu.tr
Bulent Cavas PhD., Professor, Dokuz Eylul University, Turkey.
E-mail: bulentcavas@gmail.com
522
IMPROVING ACADEMIC
ACHIEVEMENT OF SCIENCE
LEARNERS IN RURAL SCHOOLS
THROUGH ASSESSMENT
PRACTICES: A SOUTH AFRICAN
CASE STUDY
Rekai Zenda, Abstract. This research was embarked on

to explore the underperformance of Physi-
Johanna G. Ferreira cal Sciences learners and the subsequent
increasing unpopularity of the subject in
South African rural schools. As poor aca-
demic achievement in science is a concern
in many countries and not only in South
Africa, qualitative research was undertaken
to determine whether assessment can con-
tribute to the improvement of learners’ aca-
demic achievement in this particular school
Introduction subject. The research uses the Cultural and
Historical Activity Theory as theoretical lens
The poor academic achievement of school children in the field of science to analyse how assessment can improve the
is a world-wide problem and is evident in South Africa too. This is substanti- academic achievement of learners. Data
ated by Howe (2003), who refers to this situation as a cause for global concern. were collected by means of face-to-face
The high failure rates in Physical Sciences have made the subject unpopular, interviews with teachers, school principals
resulting in fewer learners selecting it in the Further Education Training and subject advisers, the “community” as
advocated by the Activity Theory, while
(FET) phase of secondary schooling (Department of Basic Education, 2011).
focus-group interviews were held with
The South African National Department of Basic Education (2011) reiterates
the learners who are the “subjects” in the
that the central feature emerging from rural secondary schools in particular, Activity Theory. Verbatim findings highlight
is poor academic achievement in science. The result is that fewer learners the views of participants and reveal that
pursue scientific careers, which negatively affects the number of scientific though efforts are made to incorporate a
professionals (Cameron, 2009). This has resulted in a shortage of engineers, variety of informal assessment methods,
skilled artisans, technicians, doctors and technologists. Assessment is prob- key issues such as poor formulation of
ably the most important factor that educators can target to improve academic questions, weak comprehension skills and
achievement of learners in Physical Sciences. The new Curriculum Assessment unsatisfactory interpretation of questions,
Statement (CAPS) has introduced changes to the assessment processes of all remain a problem. Based on these findings,
subjects, including Physical Sciences. Evidence has been obtained from Physi- recommendations for improvement are
proposed.
cal Sciences educators, school principals, curriculum advisers and learners on
Key words: case study; improving science
how assessment could be used to improve learner academic achievement in
achievement; physical sciences assessment;
selected rural secondary schools. rural schools
Problem of Research
As pointed out the poor academic achievement of Physical Sciences

learners is a cause for concern as is the declining number of learners choos- Rekai Zenda, Johanna G. Ferreira
University of South Africa, South Africa
ing Physical Sciences as a school subject. Based on the discussion about the
influence on assessment, this research focuses on how academic achievement
of learners can be addressed through assessment practices. As the problem
523
IMPROVING ACADEMIC ACHIEVEMENT OF SCIENCE LEARNERS IN RURAL SCHOOLS THROUGH
ASSESSMENT PRACTICES: A SOUTH AFRICAN CASE STUDY
ISSN 1648–3898
(P. 523-531)
is the underachievement of science learners, strategies need to be identified to effect an improvement of learner
performance. One such strategy could be the use of assessment during teaching and learning. Consequently the
following research question is asked: How can the assessment of learners improve the academic performance of
Physical Sciences learners in selected rural secondary schools?
Theoretical Perspective
One theory that is appropriate to research this question is the Cultural-Historical Activity Theory (CHAT). Ac-
cording to Edwards (2010) CHAT offers a broad approach to analysing learning and the contexts of learning and
can be used to analyse activities in schools, for example, to uncover how learners use both materials and concep-
tual tools and what aspects of tasks they prioritise. It is a useful framework to study how classrooms, schools or
teacher teams operate as cultural systems that afford particular ways of, for example, being a learner. CHAT has
an interest in the process of social transformation and includes the structure of the social world in analysis, taking
into account the conflictual nature of social practice. Learning is seen as a social function. Engeström, Miettinen
and Punamäki (1999) argue that the concept of “activity” opens up a new way to understand change. The key is
“revolutionary practice,” which is not to be understood in narrow terms but as joint “practical-critical practice.” Ac-
cordingly Engeström considers subjects (in this case the learners), tools, the object (the improvement of learner
achievement) and outcome, rules, the community (teachers, principals and subject advisers) and division of labour
as a collective activity system. Engeström, Miettinen and Punamäki (1999) are of the opinion that the unit of ac-
tivity includes the rules that regulate the learner actions towards a goal, and relations with other learners in the
activity. In order for the system to have a positive teaching and learning environment, there is a need to establish
behavioural rules in terms of assessment that give direction in the activity system. Norms and regulations within
Physical Sciences are viewed as rules. Rules in Physical Sciences are important in that they facilitate teaching and
learning of learners. A number of rules are set by the Department of Basic Education that facilitate the teaching
and learning of Physical Sciences, including assessment guidelines. Engeström, Miettinen and Punamäki (1999)
stipulate that rules refer to formal or informal regulations that can, in varying degrees, constrain or liberate the
activity and provide the learner guidance on correct procedures and acceptable interactions with other community
members. In this research, the object as outlined by Engeström, is the improvement of learner academic achieve-
ment in Physical Sciences in rural secondary schools.
CHAT’s relevance to Physical Sciences teaching and learning is founded on the understanding that effective
assessment takes place through collective activities (Gibbs, 1988). Assessment is viewed as a vehicle for driving
Physical Sciences’ teaching and learning since successful implementation of the curriculum mostly depends on it
(Beets & Le Grange, 2005). The rules that direct how the assessment should be conducted are received from the
Department of Basic Education in the form of the CAPS policy document for Physical Sciences. According to the
CAPS document, assessment is a process that measures an individual learner’s attainment of knowledge (con-
tent, concepts and skills) in Physical Sciences by collecting, analysing and interpreting the data and information
obtained to enable the teacher to make reliable judgements about a learner’s progress; to inform learners about
their strengths, weaknesses and progress and to assist teachers, parents and other stakeholders in making deci-
sions about the learning process and the progress of learners. The assessment guidelines stipulate the number of
activities to be given to learners in Physical Sciences and form part of the rules of the Activity System.
Assessment of Learners
The Department of Basic Education of South Africa (2011) views assessment as a continuous planned process
of identifying, gathering and interpreting information about the performance of learners using various forms of
assessment. Lubisi (1999) states that assessment entails making sense of a learner’s knowledge and skills. However,
assessment should be both informal or formative (assessment for learning) and formal or summative (assessment
of learning). Lockwood (1994) elaborates stating that assessment can be formative, enabling a learner or teacher to
check the responses against criteria; it can be diagnostic enabling at least an initial identification of strengths and
potential areas of learning difficulty; and it can be used to provide guidance and feedback. Assessment can also
motivate learners through positive feedback from the educator that can help learners to develop self-esteem and
confidence in their development (Dunphy & Dunphy, 2003). In both cases, regular feedback should be provided
to learners to enhance the learning experience in Physical Sciences (Department of Basic Education, 2011).
524
ISSN 1648–3898 IMPROVING ACADEMIC ACHIEVEMENT OF SCIENCE LEARNERS IN RURAL SCHOOLS THROUGH
(P. 523-531)
Informal assessment entails the daily monitoring of learners’ progress (Department of Basic Education, 2011).
This is done through homework, classwork, tests and experiments. Informal assessment should be used to structure
the acquisition of knowledge and skills and should be the precursor to formal tasks in the programme of assessment.
The CAPS document emphasizes that informal assessment should be used to provide feedback to the learners and
to inform teacher planning. Teachers and learners can mark these informal assessment tasks. Both self-assessment
and peer assessment are important as they actively involve learners to learn from and reflect on their own perfor-
mance. Self-assessment in itself is a crucial skill for learning and for future employment. It encourages learners to
take responsibility for their own improvement, and is the route to excellence in Physical Sciences. Learners should
be motivated by a desire to succeed, to explore, to develop and to improve, not through fear of failure.
Formal assessment tasks are marked and officially recorded by the teacher. All formal tasks are subject to
moderation for the purpose of quality assurance and to ensure that appropriate standards are maintained. Formal
assessment provides teachers with a systematic way of evaluating how well learners progress, but is dependent
on covering all Physical Sciences’ content and that a variety of different forms of assessment are used. Summative
assessment provides a grade, which contributes to the final mark or is used for promotion purposes.
In order to address variety in the different forms of assessment, the Department of Basic Education (2011)
stipulates that application exercises that include problem-solving activities should be done on all cognitive levels,
in all knowledge areas and on all scientific concepts. If efficient organization of the teaching and learning process
is to be maintained, an assessment taxonomy has to be applied that shows progression from unit to unit. Teachers
should assess the full range of cognitive abilities of learners through exercises and formative assessment. In prac-
tice teachers should give learners at least two problem-solving activities on a frequent basis; at least one practical
activity per term, and at least one informal test per term (Department of Basic Education, 2011).
The Norms and Standards for Teacher Education, which sets out the design and delivery of teacher education
in South Africa, (Department of Education, 2000) emphasizes the need for teachers to effectively and efficiently
perform the role of assessor. In terms of this role, competent teachers understand that assessment is an essential
feature of the teaching and learning process and should demonstrate this understanding by integrating assessment
into teaching and learning as a matter of course. By implication, teachers should know what the purposes, methods
and effects of assessment are and should be able to explain these in the feedback given to learners. Informed by
their knowledge and understanding of assessment, teachers should also be able to design and manage both in-
formal and formal assessments in ways that are appropriate to the level and purpose of the learning. Teachers have
a further responsibility in familiarizing learners with the terminology of science as the learners in the rural schools
are taught in English, which is their second language. The language issue remains an obstacle to comprehension
and expression, consequently rural learners need particular assistance with preparation for summative assessment.
Research has provided a clear indication that using assessment properly does indeed improve learning and raises
standards of learners’ achievement (Harlen, 2000:3), but if that is the case and the teachers are familiar with the
assessment requirements, why is the academic achievement of learners in rural areas so poor?
This research used a qualitative approach and consequently is concerned with phenomena relating to or
involving quality or kinds of results in the teaching and learning process. Qualitative research aims to discover the
underlying motives and desires, using in-depth and focus-group interviews for this purpose. Through qualitative
research, researchers can analyse various factors that motivate people to behave in a particular manner or why
people like or dislike a particular thing. Tuckman and Harper (2012) state that qualitative research also requires
skilful interpretation of the data and Miles and Huberman (1994, p. 6) elaborate: “...qualitative research is conducted
through an intense and prolonged contact with a “field” or life situation. These situations are typical or normal
ones, reflective of the everyday life of individuals, groups, societies and organizations.” Morse (2001) explains by
saying that qualitative research is concerned with the subjective assessment of attitudes, opinions and behaviour.
Research in such a situation is a function of a researcher’s insights and impressions. The data collected are accounts
of interviews, including detailed descriptions of context and verbatim records of conversation. Consequently, the
qualitative design keeps the researcher close to the data and markedly facilitates understanding of the phenom-
enon being studied (Schwandl, 2005). The design allows the researcher to view participants as collaborators from
525
ISSN 1648–3898
(P. 523-531)
whom to learn rather than as subjects to be studied. The qualitative approach is relevant to this research, which
seeks to investigate the influence of assessment on learner academic achievement in Physical Sciences in selected
rural secondary schools.
This approach was deemed appropriate to explore and interpret the perceptions of principals, teachers, cur-
riculum advisers, and learners on assessment. In order to explore the social worldviews of these participants, an
interpretevist paradigm is used, which according to Merriam (1998) attempts to understand and sense what people
say of their experiences of their particular worlds. According to Cohen, Manion and Morrison (2000), there is a need
to examine situations through the eyes of participants entailing reflection on the meaning that teachers, principals,
curriculum advisers and learners assign to their own experiences regarding assessment. CHAT provides the lens
through which the results will be interpreted, with analysis, categorization and interpretation of the perceptions
of teachers, principals, curriculum advisers and learners in the framework.
Ethical Considerations
Permission to do the research was obtained from the relevant education authorities. Before the interviews
were held, written consent was obtained from the participants to avail themselves for the interviews. The purpose
of the research was explained in detail and all potential participants were assured of the confidentiality of their
responses and of complete anonymity. Consequently no names of participating teachers, principals, learners and
curriculum advisers are used in reporting the findings. All participation was voluntary and it was explained that
anyone can withdraw from the interview without penalty. No one was coerced or rewarded for their participation
in the interviews.
Selection of Participants
The purpose with the selection of participants in this research was to solicit information regarding assessment of
Physical Sciences learners in selected rural secondary schools, consequently purposeful sampling was used. According
to Patton (1990) purposeful sampling is done to increase the utility of information obtained from small samples. Cre-
swell (2013) and Merriam (1998) both assert that purposeful sampling is based on the assumption that the researcher
wants to discover, understand and gain insight and therefore must select a sample from which relevant data will be
obtained. In other words, the sample was chosen because the participants would be knowledgeable and informative
and could contribute to determining the role of assessment in academic achievement of learners.
The selection was first limited to a specific provincial education department and then to one of its education
districts from which five schools that offer Physical Sciences were chosen. The selected schools differ in various
ways from each another as one is well-resourced, whereas the others have limited to no resources.
The Physical Sciences teachers in the five schools automatically comprise members of the sample as well as
the five principals responsible for managing and leading the schools. There are only two curriculum advisers and
both were selected to participate in this research. A focus group was identified from each of the five schools and
consisted of six Physical Sciences learners per group, with three boys and three girls in each group. The science
academic achievement of the participants who were selected varied from low to high.
Data collection entailed in-depth face-to-face interviews with Physical Sciences teachers, principals and cur-
riculum advisers and focus group interviews with Physical Sciences learners, which was decided upon as learners
would feel less intimidated. All interviews were held in a setting familiar to the participants. According to Patton
(1990) interviews usually yield the richest data, details, and new insights by enabling face-to-face contact with
participants and provide an opportunity to explore topics in depth by allowing the interviewer to explain or help
clarify questions, increasing the likelihood of valuable responses. During an interview one is able to discover
information that may not have been pertinent in the beginning of the research and when responses are unclear,
the respondents can be probed through further questioning to clarify issues. Scott and Morrison (2005) suggest
that interviews provide a “more informed” way of claiming knowledge than a questionnaire, but interviews may
distort perceptions, and the desire to please the interviewer may prevent honest responses. Researchers need to
be aware of this for effective data collection.
526
(P. 523-531)
Slavin and Davis (1997) stipulate that schedules with possible questions should be prepared for any interview.
The schedules for this research were developed from an analysis of literature that explores the influence of assess-
ment on academic achievement. Different schedules were used for the different participants, each with a list of
questions or issues to be explored. Questions were semi-structured so that participants could express their own
views on how assessment affects academic achievement of Physical Sciences learners.
Arrangements were made with the school principals to interview the participants in a vacant room. Teach-
ers, principals, learners and curriculum advisers were informed in advance of the dates and time during which the
interviews were to be conducted. All interviews were audio-recorded with permission from the participants Audio
recordings reduce the tendency of the interviewer to make a subconscious selection of data favouring his or her
biases. In addition recorded data can be played back more than once and can be studied much more thoroughly
than would be the case if data were limited to notes taken during interviews (Kvale, 2007). However, handwrit-
ten notes were also made to report non-verbal cues and to supplement the recorded data. The recordings were
transcribed to facilitate data analysis.
Data Analysis
According to Patton (1990), data analysis in qualitative work is to generate impressions based on initial digi-
tal recordings and to move quickly to written summaries that blur the distinction between what was heard (the
raw data) and the patterns and themes that an investigator has discerned (which can often best be termed as an
interpretation of the data). In order to achieve this, one needs explicit processes for tracking and managing raw
qualitative data for coding (using phrases) in ways that permit data to be looked at both in its textual context; to
check the dependability of coding by using multiple coders and to recognize and articulate emergent ideas about
patterns, themes and explanations. Watling and James (2007) describe data analysis in the qualitative paradigm as
a search for understanding, interpretation and meaning rather than conveying facts and measuring information.
Data analysis in this research aimed at giving appropriate meaning to the views of the participants on the influence
of assessment on academic performance in Physical Sciences.
Data of the various interviews were analysed separately at first to compare the responses of all the participants.
After the coding process, the coded data were used to form a true analysis of the Activity System. The analysis
was done by identifying the purpose of the activity for clarification. This included the context of Physical Sciences
teaching and learning and learner academic performance. The “subject”, the “community” and the “object” were
defined. The “subject” included a description of how learners are motivated during teaching and learning in Physi-
cal Sciences. The “community” included the structure and social interactions within the teaching and learning of
Physical Sciences. The expected “object” of the activity is the improvement of learner achievement. The analysis of
the activity system included an identification of the activities in which the “community” (subject advisers, principals
and teachers) and “subjects” (the learners) will participate and the rules within the particular Activity System. Finally,
an analysis was made of the rules that guide the Activity System and the roles of the community members involved
in the teaching and learning of Physical Sciences. This resulted in an analysis of the perceptions of participants
on learner academic performance in Physical Sciences as well as problems they encounter with the teaching and
learning of Physical Sciences.
Results of the Research
In an attempt to determine the effect of assessment on learner achievement face-to-face interviews were
held with Physical Sciences teachers, curriculum advisers, principals and focus group interviews with Physical Sci-
ences learners. In reporting the results, and to ensure anonymity of participants, numbers are used to distinguish
between each individual participant. The findings are reported as narratives with verbatim substantiation from
some “community members” and some of the “subjects” in the Activity Theory.
On the whole, the teachers responded in the same way to the questions in the interview schedule as they meet
regularly to discuss planning and share their experiences resulting in similar points of view. Only selected eloquent
quotes are given in reporting the results. Teacher 4 pointed out that assessment is very important because it helps
to improve instructions given to learners during teaching and learning. The performance of learners will therefore
improve if they are given the right kind of assessment continuously or more often. Teacher 1 pointed out that “...
assessment helps teachers to determine if the method of teaching was effective or not and evaluating both the
527
ISSN 1648–3898
(P. 523-531)
teacher and the learners.” The teachers unanimously mentioned that even though teaching approaches should be
evaluated frequently for desirable results, assessment is the key as it apprises stakeholders of the current status.
Teacher 3 stated “...assessment cannot be ignored because it informs learning methods and teaching methods. No
doubt assessment informs and highlights topics which are difficult.”
Teacher 2 argued that “...assessment should be part of learning because it assists the teacher, first of all to
know where he or she is, what he or she has done, what needs to be done, how much has been learnt and how
much has been achieved. Assessment is a way of measuring what is happening in the class and it helps not only
the teacher or the learner, but the Department of Education to determine whether the targeted objectives have
been achieved or not achieved.” This was also raised by Teacher 5 who said: “If the teacher is teaching a topic such
as vertical projectile motion, one wants to know if learners are able to substitute and apply the correct formulae
and whether they grasp the content according to the objectives or guidelines. Therefore, assessment is vital.”
Consequently assessment should be used by teachers to determine not only how successful their teaching is, but
also to identify what exactly learners do not comprehend. This can only be achieved by numerous and varied as-
sessment methods. When asked to explain what this entails the general response referred to homework, classwork
exercises, tests, assignments, investigations, projects and peer assessment. Homework is given to learners on at least
a weekly basis to keep learners engaged and learners are encouraged to revise work at home. Only one teacher
indicated that he can monitor the learners’ progress by marking their work on a regular basis. The other teachers
complained that they cannot do so because they have too many classes with too many learners in each class and
consequently cannot mark learners’ work every week. As there are too few qualified science teachers, those who
are qualified tend to be overloaded and cannot cope with regular informal assessment. Teacher 3 stated: “Infor-
mally we give classwork, homework, tests, and experiments but we cannot always check the learners’ work and
the parents cannot help because they do not know science. Formally we have controlled tests and experiments
set by the district. These we have to mark and it takes time.”
Teacher 1 mentioned “...end-of-topic tests act as a diagnostic measure and can be used to determine retention
capacity of learners after a period of time. Basically, assessment can be done daily, weekly, fortnightly or monthly.”
The key concern raised by all teachers is the fact that as learners are taught in their second language, they do not
always understand what the action verbs in formal assessment questions mean. “We as teachers, also do not always
know what is meant with words like extrapolate, analyse and so on. So we need assistance and examples to help
us. Only then can we help learners” (Teacher 2). The issue was raised that a data base of possible questions with
possible answers would greatly assist to prepare learners for their final examinations and that learner achievement
should then improve.
Most of the Principals shared the same sentiments as the teachers concerning assessment. Principal 3 stated: “If
you do not show them (learners) their mistakes in time and if you do not give them feedback, obviously it is going
to affect their performance. Physical Sciences need thorough investigations and experiments. You must frequently
check learners’ books so that you can be able to identify slow learners, low achievers and high achievers.” Principal
4 pointed out that “…assessment is one key that helps in improving academic achievement. If you assess your
learners well you get them used to questions that will come up during the year and with regular assessment you
can identify areas in which learners are not doing well and regular assessment helps in improving performance
in Physical Sciences.” Principal 5 also stressed that assessment in school plays a very prominent role in the sense
that learners who are assessed regularly master examination techniques and are more comfortable with their
summative assessment. Learners understand how questions are asked and from the feedback are able to correct
errors: “If assessment is done and feedback is given then learners will be able to understand the subject.” Principal
2 described the strategy used in his school: “Assessment is important so we make use of previous question papers
to assess the learners throughout the year. By so doing the learners know how to answer questions.” This principal
also stressed the importance of comprehending what examination questions require learners to do and that the
focus not only be on rote learning and memorization of content knowledge. Principal 1 indicated: “If the learners
are assessed more often they will become assessment-ready. The pre-tests prepare them for the final task and the
learners can improve their academic achievement and then even the final results will improve.”
The curriculum advisers who were interviewed mentioned that educators should give quality questions dur-
ing classwork and homework activities because they lead to high achievement in Physical Sciences. The question
was raised whether the advisers should not take the lead and attend to this. Curriculum Adviser 1 responded by
saying that he prepares questions for the schools in his circuit, but expects teachers to set special questions too.
He complained that the teachers do not contribute but that he is prepared to offer workshops to assist in setting
528
(P. 523-531)
assessment tasks and activities. Curriculum Adviser 2 pointed out that if learners are engaged in practical work
they will understand and improve rather than being merely involved theoretically. In addition, Curriculum Adviser
1 mentioned that writing more informal tasks is essential to improve learner performance. Tests need to be stan-
dardized to enhance quality and teachers should provide learners with the necessary skills to answer question
papers. In the long term this will motivate learners because they are given timely feedback.
The focus group interviews that were held with learners revealed that the learners in the different groups and
from the different schools generally had the same concerns and experiences. The learners can identify their prob-
lems in Physical Sciences when they get tests and they then try to find ways to correct and improve, but they do
not always understand the questions or what they should do to answer the questions. When the teacher goes over
the test after it has been marked and they see what was expected it helps them clarify some issues and reinforces
what they have learnt. Learners in two of the groups stated that assessment is just used to test their knowledge,
but the general consensus in the other four groups was that they find out whether they understand the work or
not. All the learners acknowledged that they do not enjoy writing tests, but they understand that it is a way to help
them to study; to practice how to answer questions and enables them to gain confidence. In one group a learner
made a case for assessment: “It is important to practice to understand the questions and what is wanted. We learn
the correct way to answer a question and this makes it easier for big tests and exams. We can learn to understand
the work better and to improve.” A point that the learners from most of the groups agreed upon, was why their
teachers assessed them namely to identify problem areas and then to get assistance from teachers where needed:
“The reason why our teachers give us tests is to see whether we understand or not. If we do not understand they
will explain it again because there will be possibility for that question to come up again and then l will be able to
answer it correctly.” “Our teachers ask questions to get information. They give us tests to get our standard in the
subject and they want to know if we understand and if not then we re-do the section.” “Teachers test us to see if
we understand and if we don’t he explains again and we get more homework.” The learners therefore agreed that
assessment helps the teachers to determine whether they have mastered the necessary content and whether
revision is required. Many of the learners however, indicated that they are hesitant to approach their teachers
individually for further tuition.
Besides helping the learners to identify their mistakes, assessment motivates them to work harder and to
gain experience. Learners from every group suggested possible solutions on how assessment can improve their
performance such as “...writing more tests and doing a lot of revision can help us pass.” Consequently, more tests,
more classwork and homework is required to improve their performance.
Discussion
The Department of Basic Education (2011) defines assessment as a continuous planned process of identify-
ing, gathering and interpreting information about the performance of learners using various forms of assessment.
The findings suggest that in Physical Sciences informal assessment provides teachers with an opportunity to un-
derstand the learning processes to improve learner achievement. Consequently, the frequency and standard of
informal assessment has to be addressed. Lubisi (1999) also asserts that informal or formative assessment remains
the most critical process because it allows teachers to observe learning before learners are exposed to summative
assessment. Informal assessment allows teachers to gather information about learners and to use the information
to improve learners’ performance as learners do not always reflect on their own learning and do not realize that
there is content they do not understand. Informal formative assessment is the foundation for summative assess-
ments, therefore, is viewed as crucial. The findings indicate that informal formative assessment develops learners’
reflection and increases learning opportunities in Physical Sciences. The CAPS document (2011) indicates that
assessment should be both informal and formal, but it appears as though more should be done in this regard and
that learners’ progress should be monitored on a daily basis. The learners in the research accepted the importance
of frequent assessment and that it is necessary to help them prepare for final examinations.
Dunphy and Dunphy (2003) indicate that assessment can motivate learners through feedback from the edu-
cator, helping learners to develop self-esteem and confidence in their development. The findings in this research
re-affirm this, though the teachers themselves did not place much emphasis on this during the face-to-face inter-
views. It appears as though teachers should be encouraged to praise learners who perform well, but should also
encourage those who do not. By assessing frequently, learners gain confidence and are encouraged to focus on
learning and achievement. During assessment, teachers need to analyse evidence of learning in learners’ work as
529
ISSN 1648–3898
(P. 523-531)
a basis for reflecting and improving teaching, but should also develop the reflection skills of learners on how to
determine whether they have mastered the content or not. Assessment therefore encourages reviewing individual
strengths and weaknesses and to identify problems that hamper achievement. Besides, helping learners show what
they have learnt, assessment helps learners how to tackle questions through frequent assessment. Introducing
assessment terminology appears to be of primary importance and this requires urgent attention. The findings
suggest that assessment allows learners to practice how to answer questions and helps them to remember what
they have learnt.
The findings further suggest that assessment helps to improve comprehension of instructions during teach-
ing and learning. However, this is not evident in the schools involved in this research. The performance of learn-
ers should improve if they are given rigorous assessment more often. Moreover, assessment cannot be avoided
because it informs the teacher about teaching methods. Assessment is a way to measure what is happening in
the classroom and it helps not only the teacher or the learner, but also the Department of Education to determine
whether the targeted objectives have been achieved or not. Therefore, assessment informs the stakeholders of the
nature of the results. Assessment should therefore be viewed as a vehicle for improving Physical Sciences teaching
and learning since successful implementation of the curriculum mostly depends on it (Beets & Le Grange, 2005).
Regular assessment in Physical Sciences plays a prominent role in enabling learners to understand questions.
The importance of assessment is reiterated and learners need to be provided with the opportunity to revise
previous question papers throughout the year to improve performance. Educators need to give quality questions
during class-work and homework activities because they lead to high achievement in Physical Sciences. Homework
is given to learners to keep them engaged and to revise work at home, but it is critical that teachers explain to learn-
ers what their mistakes are and to provide feedback soonest in order to improve learner performance. However,
in the schools in this research, this is not always done due to large numbers of learners.
Research by Harlen (2000) has provided a clear indication that using assessment properly does indeed improve
learning and raises standards of learners’ achievement. Ideally, tests encourage learners to revise the difficult topics
that have been taught to improve performance. Furthermore, end-of-topic tests act as a diagnostic measure and
can be used to determine retention capacity of learners after a given period. It is important for learners to write pre-
tests and post-tests in preparation for formal or common controlled tests to improve their achievement. Learners
should do more informal tasks such as class-work, and weekly or monthly tests and a plan should be established
to monitor learners’ work, even if this is done on a rotational basis.
Conclusions
This research found that though informal assessment provides teachers with an opportunity to understand
the learning processes and to improve learner achievement, there are shortcomings. Informal assessment allows
teachers to gather information about learners that can be used to improve learners’ performance, but the teachers
do not do enough activities and exercises. Further it appears as though the type of assessment that is used does
not test higher cognitive skills and tends to focus on rote learning with content mastering as main goal. Too few
application and authentic assessment is done. As informal assessment is a foundation for formal summative as-
sessment, it should be utilized properly. Ideally it should serve to identify problems to help learners improve and
instil confidence in their abilities, but this research found that it has not been used effectually. Though teachers are
aware of the importance of formative assessment, they are not equally confident in their abilities to set higher order
thinking questions with adequate rigor. They often rely on previous examinations for setting formative assessments.
Learners acknowledged that informal assessment helps them to practice how to answer questions especially when
they receive detailed feedback with appropriate explanations. This helps them to remember what they have learnt
and prepares them for similar questions that they may get in future. Ideally, tests encourage learners to revise the
difficult topics that have been taught to improve performance. Furthermore, end-of-topic tests act as a diagnostic
measure and can be used to determine the retention capacity of learners after a given period. The performance of
learners could also improve if they are given appropriate classwork, homework and tests more frequently and that
contributes to the development of process skills. It is, however, essential that this type of intervention is monitored,
but some teachers cannot do so due to the large number of learners in their classes. The Department of Education
needs to address the teacher-to-learner ratio to improve informal assessment and to ensure targeted objectives are
achieved. These recommendations and suggestions can also contribute to improve the poor academic achievement
of science learners in other countries where similar poor science achievement is encountered.
530
(P. 523-531)
References
Beets, P., & Le Grange, L. (2005). “Africanising” assessment practices: Does the notion of ubuntu hold any promise? South African
Journal of Higher Education, 19 (5), 15-25.
Cameron, L. (2009). Higher grade-Dinaledi schools initiative helps boost maths, science pass rates. Engineering news, 29 (31),
1-104.
Cohen, L., Manion, L., & Morrison, K. (2000). Research methods in education. London: Routledge Falmer.
Creswell, J. W. (2013). Research design: Qualitative, quantitative and mixed method approaches. London:
Sage.
Department of Education, (2000). Norms and Standards for Educators. Government Gazette, 415 (20844), 4 February, 2000. Pre-
toria: Government Printer.
Department of Basic Education. (2011). Curriculum and Assessment Policy Statement (CAPS). Pretoria: Government Printer.
Dunphy, B. C., & Dunphy, S. L. (2003). Assisted performance and the Zone of Proximal Development. Australian Journal of Edu-
cational and Developmental Psychology, 3, 48-58.
Edwards, A. (2010). Being an expert professional practitioner: The relational turn in expertise. Dordrecht: Springer.
Engeström, Y., Miettinen, R., & Punamäki, R. L. (1999). Perspectives on Activity Theory. Cambridge: Cambridge University Press.
Gibbs, G. (1988). Learning by doing: A guide to teaching and learning methods. Oxford: Further Educational Unity, Oxford Poly-
technic.
Harlen, W. (2000). The teaching of science in primary schools. London: David Fulton Publishers.
Howe, S. J. (2003). Language and other background factors affecting secondary pupils’ performance in Mathematics in South
Africa. African Journal of Research in Science Mathematics and Technology Education, 7, 1-20.
Kvale, S. (2007). Doing interviews. London: SAGE.
Lockwood, F. (Ed). (1994). Materials production in open and distance learning. London: Kogan Page.
Lubisi, R. C. (1999). Assessment in education: Principles, practice and critique. Pietermaritzburg: University of Natal Press.
Merriam, S. B. (1998). Qualitative research and case study applications in education. San Francisco: Jossey-Bass publishers.
Morse, J. M. (Ed.). (2001). Critical issues in qualitative research methods. Thousand Oaks: Sage.
Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook. Thousand Oaks: Sage.
Patton, M. Q. (1990). Qualitative evaluation and research methods. Thousand Oaks: Sage.
Schwandl, L. (2005). What school leaders do to improve student achievement. New York: Harper Collins Publishers.
Scott, D., & Morrison, M. (2005). Key ideas in educational research. London: A & C Black.
Slavin, R. S., & Davis, N. (1997). Educational psychology: Theory and practice. New York: Allyn & Bacon.
Tuckman, B. W. & Harper, B. E. (2012). Conducting educational research. Lanham: Rowman & Littlefield Publishers.
Watling, A., & James, T. (2007). Research methods in educational leadership and management. London: Sage.
Rekai Zenda DEd, c/o Department Curriculum and Instructional Studies,

University of South Africa, PO Box 392, UNISA 0003, South Africa.
Johanna G Ferreira DEd, Professor, Department of Curriculum and Instructional Studies
(Corresponding author) University of South Africa, PO Box 392, UNISA 0003, South Africa.
E-mail: ferrejg@unisa.ac.za
531
DEVELOPMENT OF AN ONLINE
THREE-TIER DIAGNOSTIC TEST
TO ASSESS PRE-UNIVERSITY
STUDENTS’ UNDERSTANDING
OF CELLULAR RESPIRATION
Abstract. This research reports the devel- Rohaida Mohd Saat,

opment of an online three-tier diagnostic
instrument for pre-university students Hidayah Mohd Fadzil,
related to cellular respiration. To date, only Nor Azlina Abd. Aziz,
few studies have been conducted to identify Kamariah Haron,
students’ alternative conception in the
topic of cellular respiration. One of the con-
Kamaludin A. Rashid,
tributing factors is due to lack of instrument Natalya Rudina Shamsuar
developed to measure these alternative
conceptions. Three-tier tests are used as
assessment tools for lecturers to determine
students’ alternative conceptions related
to cellular respiration and their knowledge
and understanding about the concepts. The
research incorporates both qualitative and Introduction
quantitative methods. The instrument was
developed in five phases: (1) construction Students come to biology class with prior knowledge and this knowl-
of items, (2) pilot study, (3) validation of in- edge affects the way learners perceive new scientific information. Some part
strument, (4) transforming the instrument of this knowledge can be different or in conflict with the information that is
into an online assessment tool, and (5) the generally accepted by scientists (Mutlu & Sesen, 2016; Taslidere, 2016). The
administration of the Online Diagnostic
alternative information may stand in the way of learning biological concepts
Tool (ODiT). The Cellular Respiration ODiT
in a meaningful way and could possibly obstruct the acquisition of new
consists of three tiers: answer and reason-
ing tiers to measure “content knowledge”
knowledge (Lazarowitz & Lieb, 2006). It is now widely acknowledged that
and “explanatory knowledge” respectively, students’ alternative conceptions in biology can impede their learning and
and a third tier that measures the level meaningful understanding of the subject (Anderson, 1986; Palmer, 2001;
of confidence of pre-university students. Taslidere, 2016). In this research, alternative conception refers to students’
Analysis of the students’ responses demon- conceptions that conflict with scientific information. Biology misconceptions
strated acceptable reliability and validity have been recognized as a major factor affecting students’ understanding
of the instrument. The research also shows of science at the secondary school level with many of these misconceptions
that both students and lecturers benefit being carried onwards to the university level (Coll & Treagust, 2003). Research
when online diagnostic tests are imple- on students’ understanding of cellular respiration revealed an array of mis-
mented appropriately.
conceptions and inaccurate conceptualizations among secondary school
Key words: biology alternative concep-
students (Lazarowitz & Lieb, 2006; Halimin & Shamsudin; 2016), as well as
tion, online diagnostic tool, three-tier
diagnostic test.
pre-service science teachers (Yakisan, 2016; Galvin, Simmie, & O’Grady, 2015).
For instance, a study conducted by Yusof, Halimin, and Shamsudin (2016)
investigated the understanding of diffusion, osmosis, and the particulate
theory of matter and found that university students’ understanding of the
Rohaida Mohd Saat, Hidayah Mohd particulate theory of matter was unsatisfactory. Similarly, Galvin, Simmie,
Fadzil, Nor Azlina Abd. Aziz, Kamariah and O’Grady (2015) investigated misconceptions about respiration and
Haron, Kamaludin A. Rashid photosynthesis and found an unacceptably high level of misconceptions
University of Malaya, Kuala Lumpur,
Malaysia among all pre-service teachers and suggested that diagnostic tests can be a
Natalya Rudina Shamsuar useful entry point to a pedagogical cycle for the recognition, reduction, and
Multimedia University, Cyberjaya, Malaysia removal of misconceptions.
532
ISSN 1648–3898 DEVELOPMENT OF AN ONLINE THREE-TIER DIAGNOSTIC TEST TO ASSESS PRE-UNIVERSITY
(P. 532-546)
In science education, various types of assessments have been used to identify students’ alternative concep-
tions (Ayla, 2012). Assessment methods such as open-ended questions, interviews, concept maps, and multiple-
choice questions have shown both advantages and disadvantages in practical usage (Calik & Ayas, 2005; Goh &
Chia, 1991; Montfort, Brown, & Findley, 2007; Osborne & Gilbert, 1980; Schmidt, 1997; Thompson & Logue, 2006;
Tsai & Chou, 2002; Uzuntiryaki & Geban, 2005). According to Treagust (1998) among the assessment techniques,
multiple-choice diagnostic tests are often more preferable in science classes since they are easy to apply. In-
structor will be able to evaluate the students’ understanding of the subject based on the result of the diagnostic
test. However, Treagust (1998) agrees that multiple-choice tests have some limitations in their application, such
as determining whether a student gives a correct response on a test consciously or just by chance. Interviews
on the other hand can provide more detailed information about students’ alternative conceptions and their
understanding of a particular concept. However, to interview a large number of students to generalize their
alternative conceptions is time consuming (Montfort, Brown, & Findley, 2007; Thompson & Logue, 2006). Since
the abovementioned techniques have some limitations for practical use in classes, two-tier tests have been pro-
posed (Treagust, 1995) to strengthen the advantages of multiple-choice tests. The two-tier tests were developed
to identify students’ alternative conceptions (Tan, Goh, Chia, & Treagust, 2002). A two-tier multiple-choice item
includes content-based alternatives in the first tier (answer tier or A tier) and relevant principles that justify the
A-tier response in the second tier (reason tier or R tier) (Caleon & Subramaniam, 2010). The student gives a cor-
rect response to a question and the second tier requires a reason for the response. Such a format allows one to
determine any alternative conception the student holds. Treagust (1995) also suggested that teacher can identify
students’ alternative conceptions of a related topic when diagnostic tests are used either at the beginning or at
the end of a chosen topic. Thus, based on these alternative conceptions shown by students’, the teacher will be
able to modify the related lesson plan to remedy students’ alternative conceptions.
Two-tier multiple-choice tests which are relative to typical multiple choices questions (MCQs) can be ef-
ficiently used to probe students’ alternative conceptions (Caleon & Subramaniam, 2010; Treagust, 1995). Two-tier
tests measure not only the students’ ability to select correct responses, but also the reasoning behind their choices.
In spite of this, two-tier multiple-choice tests are not impervious to the key limitation of the multiple-choice
tests, which is the inability to differentiate mistakes resulting from a lack of knowledge from mistakes due to
genuine alternative conceptions, and to distinguish correct answers based on guessing from correct answers
based on genuine understanding (Caleon & Subramaniam, 2010). Thus, an additional tier has been introduced
to address this limitation. A confidence ratings serve as an additional tier of two-tier multiple-choice tests that
used to determine students’ confidence level in solving the item of MCQs (Hasan, Bagayoko & Kelley, 1999). The
three-tier diagnostic tests are considered to be more accurate in eliciting student alternative conceptions, since
they can detect the lack of knowledge percentages by means of the confidence ratings (Aydın, 2007; Eryılmaz,
2010; Gurel, Eryılmaz, & McDermott, 2015). This enables teachers to gain valuable information about students’
alternative conceptions and provides a foundation for developing a valid and reliable diagnostic assessment tool.
Stankov and Dolph (2000) refer confidence rating to the “appraisals and judgments by an individual regarding
the quality or accuracy of his or her own performance” (Caleon, 2010). Confidence ratings have been included
in testing since the 1930s, mainly in the field of psychology. According to Echternacht (1972), confidence rating
is use to enhance the amount of information that can be obtained from objective tests. Several studies have
also shown that confidence rating can be related to academic performance (e.g., Stankov & Crawford, 1997;
Zakay & Glicksohn, 1992). Unfortunately, research indicates that students tend to be poorly calibrated, that
is, their confidence level tends to be higher than what is warranted by the accuracy of their response (Lunde-
berg, Fox, & Punccohar, 1994; Morris, 1990). Lundeberg et al. (1994) define good calibration as expressing high
confidence when answers are correct and expressing low confidence when answers are wrong. In studies that
dealt with students’ understanding of texts, confidence ratings were found to increase based on factors such
as the accessibility of the concept from respondents’ memory (Morris, 1990), level of familiarity or expertise in
the relevant field (Glenberg et al., 1987), and when self-generated feedback could be obtained from a related
pretest (Glenberg & Epstein, 1985; Glenberg et al., 1987).
Knowledge assessment is one of the most important parts of the educational process. Technology in educa-
tion offers many new opportunities for innovation in educational assessment. The increased use of computers
in education and the embracement of e-learning lead to computerized assessments, especially web-based as-
sessment systems. Studies (for e.g. Petrisor et al., 2016) revealed that students have a high level of acceptance
for the online test as an assessment method. Online assessment are better suited for knowledge assessment
533
ISSN 1648–3898
(P. 532-546)
and are more objective as opposed to other assessment forms, like pen and paper examination. According to
Grive, Padgett, and Moffitt (2016), current students no longer hold negative perceptions about using technology
in teaching and learning. Experiences with technology at the pre-university (undergraduate foundation) level
can help with the transition to the higher learning institution. The ability to use different technologies prior
to university serves as a route for students to help them prepare for university life (Oliver et al., 2014). Studies
show that both students and faculty benefit when online evaluations are implemented appropriately. Online
assessments can provide the information needed by students and teachers to improve learning through rich
assessment tasks and potentially powerful scoring, reporting, and real-time feedback mechanisms (Kozma,
2009; Scalise & Gifford, 2006). Most of the assessments in pre-university biology subjects are paper-based as-
sessments. Analysis of the results is time consuming and feedback is not obtained instantaneously, especially
when dealing with a large number of students such as those at the pre-university level. Thus, the development
of an online three-tier diagnostic test to assess pre-university students’ understanding of cellular respiration has
been seen as an effective tool to assist teachers in “what” and “how” they teach pre-university students in biology.
A longitudinal study by Woit and Mason (2003) on the effectiveness of online assessment showed that online
evaluation can be implemented securely and efficiently, and can result in an increase in students’ motivation
and performance (Ricketts & Wilks, 2010; Woit & Mason, 2003). Thus, the online diagnostic test will help lectur-
ers in identifying students’ weak areas of understanding in the topic of cellular respiration and helping them to
achieve the proficiency needed for entrance to science-based undergraduate courses in universities.
The Purpose of the Research
Biology is a unique branch of science (Lin, 2004). Biology knowledge is considered as an extensive, multi-
faceted and often ill-structured. The domain of biology stretches across great expanses of time and a remark-
able array of sub-fields. The complexity of this subject, the interconnectedness of knowledge at many different
levels, and the invisible nature of many key processes make biology a particularly difficult subject to teach and
to learn (Fisher, Wandersee, & Moody, 2000; Lin, 2004). This research focuses on students’ understanding of cel-
lular respiration which is typically regarded as a difficult topic for students to learn and for instructors to teach
since it is a complex biological process that involves integrating multiple concepts (Songer & Mintzes, 1994).
The process of cell respiration occurs within cell mitochondria whereby energy stored in carbon-containing mol-
ecules, especially glucose, is converted into high-energy bonds of adenosine triphosphate (ATP), the only usable
source of energy for many cellular activities. Many students merely memorize the steps in cellular respiration
that are glycolysis, the citric acid cycle (Krebs cycle), and the electron transport chain. They may not understand
how these stages relate to one another and the significance of these stages. Assessment of students’ conceptual
understanding and alternative conception in cellular respiration is important for providing effective instruction.
Therefore, the purpose of this research was to develop a valid and reliable online three-tier diagnostic test to
assess students’ understanding of cellular respiration concepts. This research focuses on the following research
question: Is the Cellular Respiration Online Diagnostic Tool (ODiT) a valid and reliable instrument to determine
students’ alternative conceptions related to cellular respiration?
Methodology of the Research
The research is a mixed-method research that incorporates both qualitative and quantitative methods and
it involves a sample of Semester 2 pre-university (undergraduate foundation) biology students from a university
in Malaysia. Data collection process took 18 months to be completed. The instrument was developed in five
phases: (1) construction of items, (2) pilot study, (3) validation of the Cellular Respiration Diagnostic Instrument,
(4) transforming the three-tier diagnostic instrument into an online assessment tool, and (5) the administration
of the Online Diagnostic Tool (ODiT).
534
(P. 532-546)
Sample
In Malaysia, students have to go through a pre-university preparatory courses such as foundation pro-
gramme conducted by universities, A-Levels or the national Malaysian Higher School Certificate programme
after completing high school. The research’s sample came from pre-university students (age=18) that enrolled
in the foundation of science programme by a university in Malaysia. The programme prepares students with
comprehensive basic knowledge in Science and Mathematics in order to be competitive to enter critical science
courses offered by the universities. The students will experience learning in a university learning environment
and develop their self-esteem. This .program is a one-year course. For the purpose of this research, Semester 2
students from the biological science program were chosen. The students’ should have acquired the conceptual
understanding of cellular respiration because it is one of the topics studied in Semester 1. The informed consent
of the participant was obtained before they participate in this research. Students have been informed about
the purpose, methods and intended possible uses of the research and their participation was voluntary. Fur-
thermore, the research did not involve any sensitive issue. Three different groups were selected from the target
population: (1) the first group completed 15 open-ended questions, which consisted of 28 students (11 males
and 17 females), (2) the second group was used to validate the instrument, which consisted of 135 students (45
males and 90 females), and (3) the online instrument was administered to the third group, 526 students (238
males and 288 females).
The development of ODiT involved five main phases (see Figure 1). The first phase was identifying the con-
tent boundary of this research and designing the test items. The second phase involved several steps in piloting
the instrument and the third phase involved the validation of the instrument. The fourth phase was the process
of transforming the instrument into an online tool and the fifth phase was the administration of ODiT to 526
pre-university students and getting their feedback on the usability and functionality of the tool via an online
test. Usability in this research refers to the ease with which an instrument can be administered, interpreted by
the participant, and scored by the researcher.
Phase 1: Construction of items

This phase involved the development of the three-tier MCQs. First, the content boundary of the instrument
for this research was identified. The questions were developed specifically for the topics of cellular respiration
in the Semester 1 course on cell biology. These topics were chosen after an extensive literature review, which
reported that students both nationally and internationally had learning difficulties and held misconceptions
in these particular topics (Griffard &Wandersee, 2001; Haslam &Treagust, 1987; Mann &Treagust, 1998; Storey,
1989). The process of respiration plays a very important role in understanding many aspects of living systems.
Furthermore, these processes are essential in the cycling of matter and energy flow through ecosystems (An-
derson et al., 1990; Eisen & Stavy, 1988). The majority of students have pre-instructional knowledge or beliefs
about these concepts and many of them develop only a limited understanding of science concepts following
instruction (Duit & Treagust, 2003). These students construct sensible and coherent understandings of phenom-
ena and concepts as seen through their own eyes that do not match the views that are universally accepted by
the scientific community. Based on the authors’ experience in teaching and assessing students for this topic,
many misconceptions were observed.
535
ISSN 1648–3898
(P. 532-546)
Figure 1: The five phases of the online instrument’s development.
Fifteen questions were designed for the research and the questions were mapped according to the Biology
Curriculum of the pre-university program. Seven questions were constructed to map Course Outcome 1: “students
should be able to explain the basic concept of cellular respiration”; and eight questions to map Course Outcome
2: “students should be able to solve problems by applying the basic biological concept of cellular respiration.” The
first draft of the test items included an answer tier (Tier 1) with open-ended reasoning. The items were then sent
to two academicians in the related field who were asked to evaluate the items’ effectiveness and suitability. Both
content experts found the items,
. . . to be pretty thorough, which requires that the students know the functional processes taking place within cells
and the products of such processes. A clear understanding/picture of these processes is required by the students
as the questions do go into fine detail. Most of the questions would rely on the students remembering/memorizing
facts. However, a number of the questions do require some analytical thinking or a deeper understanding of the
facts they have learned.
536
(P. 532-546)
Phase 2: Pilot study
Once the pilot version of the diagnostic test was developed, it was pilot tested on a sample of 28 students.
Analysis of the responses in the open-ended questions was carried out to identify any misconceptions among
the students and then used in the development of the distracters for the second tier of these questions, that is,
the reasoning tier (Tier 2). Semi-structured interviews were conducted with 10 students who had completed the
diagnostic test (female=7, male=3) to gain greater depth in understanding their responses in the open-ended
questions given to them earlier. The students were considered typical and were chosen based on their willingness
to be interviewed. Typical in this context means that the students were not unusual in any way and they reflected
the average level of interest (Merriam, 2009; Patton, 2002). The interviews were conducted using questions that
were similar to those on the developed instrument. Based on the interview findings and the responses in the
open-ended questions, the distracters for the diagnostic test were identified (Tier 2). The second tier is a reason-
ing tier, which consisted of an expected reason for the selection made in Tier 1, while the distracters consisted of
incorrect reasons that came from the students’ responses in the open-ended questions. For example, item 8 in
this instrument asked about the students’ understanding of aerobic respiration. The term “decarboxylation” refers
to the reaction that releases CO2, and in aerobic respiration, the “oxidation of glucose,” which involves the release
of CO2, occurs in both the link reaction and the Krebs cycle. Glucose is completely broken down at the end of the
Krebs cycle. From the open-ended responses given by the students, the suitable distracters for this item are: (1)
ATP is synthesized in the mitochondria. Glucose is broken down and CO2 is produced, (2) CO2 is one of the products of
glycolysis, and (3) during the Krebs cycle, CO2 will be released from the breakdown of one glucose molecule. The correct
reasoning is the decarboxylation process occurs only in the link reaction and Krebs cycle.
A four-point confidence scale (Tier 3) was added below the second tier of each item in the instrument, with
“1” and “4” corresponding to “Just Guessing” and “Very Confident.” The instrument was further refined to ensure
that the reason presented in Tier 2 was consistent with the responses from Tier 1.

Phase 3: Validation of the three-tier diagnostic instrument
In this phase, the three-tier diagnostic test was administered for validation to 135 randomly chosen students,
males and females of the pre-university program. The time allocated for the test was 30 minutes. The students were
informed that the test was a diagnostic test and the results of the test would not affect their grades, but would
be used by their lecturers in planning their lessons. An item analysis of the test was carried out to determine the
difficulty index and the discrimination index, followed by a reliability test (Cronbach’s alpha). The analysis helped
in item selection for the final draft of the test. After this initial evaluation, the questions were altered and deemed
appropriate for the research.
The analysis of this test also showed that the instrument is reliable (Cronbach’s alpha = 0.864 (> 0.70)). Four
items out of 15 (27%) were reconstructed due to either inappropriate difficulty level or the discrimination index.
Eleven items’ (73%) discrimination indexes were greater than 03, thus they were accepted without revision. Ac-
cording to Lien (1971), a discrimination index greater than 0.3 is considered acceptable.
Phase 4: Transforming the three-tier diagnostic instrument into an online tool
The revised diagnostic test from Phase 3 was then converted into an online tool via Google Forms. Google
Forms was chosen as a platform for this instrument because of its interesting features. The instrument can be
easily created, the platform offers open access to all users, the students can access the instrument easily to take
the test and the responses can go directly into a sharable spreadsheet. From the spreadsheet, lecturers can eas-
ily compute the results and analyze the data (see Figure 2). Google Forms also mobile friendly. ODiT allows the
students to answer the questions online and allows the researchers to track all the students’ responses instanta-
neously by referring to their matrix number. ODiT (URL: https://prpum.wordpress.com/) consists of 15 three-tier
multiple-choice questions, and for each question, the students have to choose a response that best reflects their
understanding of cellular respiration.
537
ISSN 1648–3898
(P. 532-546)
Figure 2: Spreadsheet of students’ responses.
Phase 5: Administration of ODiT
A different sample of 526 students participated in this phase. ODiT was administered in the university’s
computer labs. After completing ODiT, the participants also completed an online usability test (see Figure 3) to
evaluate the usability and friendliness of ODiT. The results of this usability test will be discussed in the results and
discussion section.
Figure 3: Example of items in usability test.
538
(P. 532-546)
Results of Research
The Cellular Respiration ODiT was evaluated for its practicality as an assessment tool for lecturers to de-
termine students’ alternative conceptions related to cellular respiration and their prior knowledge and under-
standing about the concepts. The responses of 526 students to the 15 three-tier items were analyzed using the
SPSS statistics software program.
As mentioned earlier, the instrument was created based on two main constructs: Course Outcome 1 (CO1),
which assesses the students’ basic understanding of cellular respiration; and Course Outcome 2 (CO2), which
explores the students’ abilities to solve problems by applying the biological concepts of cellular respiration. The
total mean scores were 0.605 for the CO1 items (items 1, 4, 5, 6, 7, 9, and 11) and 0.457 for the CO2 items (items
2, 3, 8, 10, 12, 13, 14, and 15). The results show that the students found it more difficult to answer the CO2 items,
which involved problem-solving skills. From the data analysis, the mean value for the third tier, which measures
the students’ level of confidence, was 2.235. A mean value nearer to 4 indicates that the confidence level is high.
For item 7, only 1% of the students gave a correct response to both the content and reasoning parts, even though
the item involves students’ basic understanding of cellular respiration. Only 10.1% of the 526 students gave a
correct response to item 14. Examples of the students’ responses to the three-tier test will be discussed in the
following section. The percentages of the students’ responses are given adjacent to the options.
Table 1. Item 2 in ODiT (Course Outcome 2).
Item 2 (see Table 1) assessed students’ understanding of aerobic respiration and anaerobic respiration. 80%
of the students gave the correct answer A, but a number of them also gave an “incorrect” reason. The majority
of the students (71.3%) gave both the correct answer and reason, while 20.4% chose reason A, a less accurate
reason. The students who chose reason A only possessed a basic concept of aerobic and anaerobic respiration
due to their basic knowledge learned in secondary education, that is, “when there is less or no O2, most organ-
isms will respire anaerobically” as explained in the following quote, “aerobic respiration requires O2, whereas
anaerobic respiration does not need O2” (Student 2, Ln.16-17). Thus, these students’ lack of understanding of
the concept the “complete oxidation of glucose” prevented them from choosing the correct reason (B). In terms
of confidence ratings, 72.7% of the students were confident (ratings 3 and 4) in choosing reason A compared
to 57.2% that were confident in choosing reason B. This is not unexpected since these students had already
acquired a basic knowledge of the topic in their secondary education.
539
ISSN 1648–3898
(P. 532-546)
Item 9 (see Table 2) shows that 47.4% of the students chose the correct answer B, but also gave incorrect
reasons. Out of this proportion, 35.9% of the students chose the correct reason C, 20.3% chose the incorrect reason
A, and 42.2% chose the incorrect reason B. The percentage of students that confidently (ratings 3 and 4) answered
the correct reason C was only 26.1% compared to 46.2% that were confident in choosing incorrect reason A, and
40.7% were confident in choosing incorrect reason B.
540
(P. 532-546)
In item 10 (see Table 3), the majority of the students (88%) chose the correct answer D, but also gave an
incorrect reason. The correct reason C was chosen by 32% of the students while incorrect reason A was chosen
by 58% of the students. For instance, Student 7 admitted that “for this question, I really don’t understand how
this process (chemiosmosis) can produce the ATP. I just guessed that the answer was ‘C’”, (Student 7, Ln. 121-122).
In terms of confidence level, 47.8% of the students were confident (ratings 3 and 4) in choosing the incorrect
reason A and about 42.1% were confident in choosing the correct reason C.
In item 13 shown in Table 4, 47.4% of the 526 students chose the correct answer B. Out of these, 34.5%
of the students with the correct answer chose the correct reason B, 41.3% chose incorrect reason C, and 19%
chose incorrect reason A. Students who chose the correct reason B understood the processes involved in the
oxidation of glucose and were aware of the significance of these processes in the production of ATP molecules,
as described by Student 5 in the following excerpt:
Fermentation is an anaerobic respiration. . . . For aerobic respiration, it (glucose) will completely oxidize into carbon
dioxide, thus ATP will be produced. (Student 5, Ln. 56-57)
However, more students opted for reason C. The evaluation of the confidence ratings demonstrated that
66.7% of the students were confident (ratings 3 and 4) in choosing the incorrect answer C, 16.7% were confident
in choosing incorrect reason A, while only 15.6% were confident in choosing the correct answer B.
As mentioned earlier, an evaluation of the usability of ODiT was conducted to understand the students’
thoughts about using the online diagnostic tool. The survey questions also explored the students’ computer
literacy skills. In this test, 76% of the students agreed that their computer literacy is high. 87% of the students
used the Web to stay informed in their area of research. 84% of the students admitted that ODiT could be used
in assessing their understanding of the topic and 70% agreed that ODiT increased their chances of getting better
grades in this subject. In terms of usability, students gave positive feedback about ODiT. 82% of the students
agreed that the instructions in ODiT were clear and understandable and 85% found the instrument to be easy
to use. In terms of friendliness, 83% of the students found that ODiT was easy to operate. However, some limita-
541
ISSN 1648–3898
(P. 532-546)
tions were found during the implementation of this instrument. One of the limitations of using Google Forms
was that the students could not get direct feedback about their results. The feedback on the performance of
each student goes directly to the lecturers. The lecturers can identify the individual scores of the students and
are able to share the results with the students in the following classroom instruction.
Discussion
The present research examines students’ alternative conception in understanding cellular respiration.
The mean score in this research indicated that the students’ basic concepts of cellular respiration are still at a
moderate level. In fact, based on the third tier responses, the students reflected that they were not confident
in their answers in ODiT. According to Hoskinson, Caballero, and Knight (2013), complex biological problems
cannot be solved merely by recalling facts from memory. Instead, complex problems require problem solvers
to engage in a broader range of scientific practices. A higher level of cognitive functioning or expert-like skills
are necessary to solve the particular problem. The students’ struggle is caused by their inability to apply the
concepts of biology. Students should be trained to think, describe, and evaluate complex biological concepts.
They need to think creatively to formulate a problem, find an answer, and evaluate and disseminate it to others
(Diki, 2013). However, in order for them to be able to evaluate ideas and produce solutions for actual problems,
they need to understand the basic concepts of cellular respiration. According to researchers in this area (for e.g.
Bajd, Praprotnik and Matyasek, 2010; Jena, 2014) cellular respiration topic is very difficult to understand.
From the data analysis, the mean value for the third tier, which measures the students’ level of confidence,
was 2.235. A mean value nearer to 4 indicates that the confidence level is high. Thus, the result shows that
overall, the students were not confident when answering the items in ODiT on either tier 1 or tier 2. According
to Chandrasegaran, Treagust, and Mocerino (2007), the answer to an item is considered to be correct if both
content and reasoning parts are answered correctly, because this decreases the percentage of students that
obtain a correct answer by chance (Tsui & Treagust, 2010). The results reflect that the students had difficulties in
understanding the basic concept of the Krebs cycle and applying the concept of the electron transport chain.
Some of the findings in this study are in agreement with Bajd, Praprotnik and Matyasek (2010) findings which
indicated that students do not bring sufficient knowledge about the process of respiration from their upper
secondary school experience to the pre-university.
Result from microanalysis of Item 2 implies that most of the students understood the basic concept of
aerobic and anaerobic respiration, and were able to explain the processes that occur in aerobic respiration
leading to the “complete oxidation of glucose”. However, some of the students’ still chose a less accurate rea-
son due to the lack of understanding of the concept the “complete oxidation of glucose”. A comparison of the
percentage of students who correctly answered both parts of the questions with that of those who correctly
answered the knowledge part of the questions suggests that many students may have learned facts without
an adequate understanding of the propositions and concepts involved (Peterson, Treagust & Garnett, 1989).
Item 9 shows that about half of the participant gave incorrect reasons for the question. In the electron
transport chain (ETC), H+ ions (protons) are pumped from the matrix of mitochondria into the intermembrane
space of the mitochondria. This causes the concentration of H+ ions in the intermembrane space to be higher
than in the matrix. The difference in the concentration of H+ ions creates a “proton gradient,” which is a form
of potential energy. The students who chose the incorrect reason A showed that they have only a basic un-
derstanding of energy production in the electron transport chain, which was acquired during their secondary
education. The students were careless in choosing incorrect reason B because it refers to the “transfer of elec-
trons from a high energy level protein carrier to a lower energy level carrier” in the electron transport chain,
but not the H+ ion (protons).
Item 10 needs the students to understand the mechanics of redox processes which occur in the electron
transport chain when electrons are transferred from one carrier to another, and energy is released. The released
energy is then used to produce ATP through chemiosmosis. Chemiosmosis refers to when energy is released
during electron transfer, and that energy is used to pump H+ ions (protons) from the matrix into the inter-
membrane space of the mitochondria. The proton gradient created due to a high concentration of protons in
the intermembrane space causes protons to diffuse back to the matrix through ATP synthase. The potential
energy released during the movement of protons is used to drive the ATP production reaction. The students
542
(P. 532-546)
that chose the incorrect reason A may only have basic knowledge of chemiosmosis as a process to produce
ATP; they did not understand the theory behind chemiosmosis and were not able to explain the concept of
how energy is transferred or transformed in the production of ATP by chemiosmosis. This result is in line with
research conducted by Jena (2014) who found that the growing misconception status of learners in ATP gen-
eration from mitochondrial oxidative phosphorylation is high.
Item 13 requires an understanding of how ATP molecules are produced in aerobic respiration, which
involves the oxidation of glucose throughout glycolysis, a link reaction, and the Krebs cycle, followed by the
transfer of energy to nicotinamide adenine dinucleotide (NADH) and Flavin adenine dinucleotide (FADH2) and
then to the electron transport chain. Result from this item indicates that the students may not understand
the theory behind the topic, that is, they were not aware of the “energy transfer” concept, whereby energy is
released by the breaking down (oxidation) of glucose and is transferred to NADH and FADH2 and then to the
electron transport chain. The energy is then used to drive the reaction to produce ATP. Therefore, they could
not relate “oxidation of glucose” to the role of NADH and FADH2.
Based on the overall data analysis and microanalysis of the test items, it appears that the students were
not able to grasp the basic concept well, thus they were unable to solve the problem-related questions. This
shows that ODiT can assist lecturers in gauging their students’ understanding of cellular respiration and enable
them to prepare the necessary actions to address this issue.
Conclusions
In light of these results, it can be concluded that the developed online cellular respiration three-tier test
is a reliable and valid instrument in assessing pre-university students’ conceptual understanding of cellular
respiration concepts. Despite the limitations, this kind of online instrument has the potential to be used by
lecturers to determine students’ pre-existing knowledge, even with a large group of students, due to the instan-
taneous feedback provided to the lecturers. To promote effective and meaningful learning, we need to identify
the misconceptions and find ways to rectify them. Based on the results of this online three-tier diagnostic
test, lecturers can design appropriate lessons that will enhance students’ understanding in learning cellular
respiration. Therefore, the online three-tier diagnostic test developed in this research is helpful in providing
lecturers with information both on students’ prior knowledge and their alternative conceptions, if any, prior
to classroom instruction. From the given feedback, lecturers could design and institute relevant measures to
eliminate their students’ alternative conceptions, for instance, by developing alternative teaching approaches
that specifically address students’ misconceptions. Further studies could use the ODiT as a tool for assessing
students’ alternative conceptions of cellular respiration. The biology research literature lacks of three-tier tests
thus there is need for more studies to develop online three-tier tests in other topics in biology.
Acknowledgements
The authors wish to thank the University of Malaya Research Grant (PRPUM) CG055-2013 for funding this
project.
References
Anderson, B. (1986). Pupils’ explanations of some aspects of chemical reactions. Science Education, 70 (5), 549–563.
Anderson, C. W., Sheldon, T. H., & DuBay, J. (1990). The effect of instruction on college non majors conceptions of photosyn-
thesis and respiration. Journal of Research in Science Teaching, 27 (8), 761–776.
Ayla, C., D. (2012). The effect of 5e learning cycle model on eleventh grade students’conceptual understanding of acids
and bases concepts and motivation to learn chemistry. Unpublished doctoral dissertation. Middle East Technical
University, Ankara, Turkey.
Aydın, Ö. (2007). Assessing tenth grade students’ difficulties about kinematics graphs by a three-tier test. Unpublished master
thesis, Middle East Technical University, Ankara.
Bajd, B., Praprotnik, L., & Matyasek, J. (2010). Students’ ideas about respiration: A comparison of Slovene
and Czech students. School and Health, 21, 245-251.
543
ISSN 1648–3898
(P. 532-546)
Caleon, I. S., & Subramaniam, R. (2010). Do students know what they know and what they don’t know? Using a four-tier di-
agnostic test to assess the nature of students’ alternative conceptions. Research in Science Education, 40 (3), 313-337.
Calik, M., &Ayas, A. (2005). A comparison of level of understanding of eighth-grade students and science student teachers
related to selected chemistry concepts. Journal of Research in Science Teaching, 42, 638-667.
Chandrasegaran, A. L., Treagust, D. F., & Mocerino, M. (2007). The development of a two-tier multiple-choice diagnostic in-
strument for evaluating secondary school students’ ability to describe and explain chemical reactions using multiple
levels of representation. Chemistry Education Research and Practice, 8 (3), 293-307.
Coll, R. K., &Treagust, D. F. (2003). Investigation of secondary school, undergraduate, and graduate learners’ mental models
of ionic bonding. Journal of Research in Science Teaching, 40 (5), 464-486.
Diki, D. (2013). Creativity for learning biology in higher education. LUX: A Journal of Transdisciplinary Writing and Research
from Claremont Graduate University, 3 (1), 1-13.
Duit, R., & Treagust, D. F. (2003). Conceptual change: a powerful framework for improving science teaching and learning.
International Journal of Science Education, 25, 671-688.
Echternacht, G. J. (1972). The use of confidence testing in objective tests. Review of Educational Research, 42, 217−236.
Eisen, Y., &Stavy, R. (1992). Material cycles in nature, a new approach to teaching photosynthesis in junior high school. The
American Biology Teacher, 54 (6), 339–342.
Eryılmaz, A. (2010). Development and application of three-tier heat and temperature test: Sample of bachelor and graduate
students. Eurasian Journal of Educational Research, 40, 53-76.
Fisher, K. M., Wandersee, J. H., & Moody, D. (2000). Mapping biology knowledge. Dordrecht, Netherlands: Kluwer.
Galvin, E., Simmie, G. M., & O’Grady, A. (2015). Identification of misconceptions in the teaching of biology: A pedagogical
cycle of recognition, reduction and removal. Higher Education of Social Science, 8 (2), 1-8.
Glenberg, A. M., Sanocki, T., Epstein, W., & Morris, C. (1987). Enhancing calibration of comprehension. Journal of Experimental
Psychology: General, 116 (2), 119–136.
Glenberg, W. & Epstein. (1985). Calibration of comprehension. Journal of Experimental Psychology: Learning Memory, and
Cognition, 11 (4), 702–718.
Goh, N. K., & Chia, L. S. (1991). A practical way to diagnose pupils’ misconceptions in science. Teaching and Learning, 6 (2),
66-72.
Grieve, R., Padgett, C. R., & Moffitt, R. L. (2016). Assignments 2.0: The role of social presence and computer attitudes in student
preferences for online versus offline marking. The Internet and Higher Education, 28, 8-16.
Griffard, P. B., &. Wandersee, J. H. (2001). The two-tier instrument on photosynthesis: What does it diagnose? International
Journal of Science Education 23 (10), 1039-1052.
Gurel, D. K., Eryılmaz, A., & McDermott, L. C. (2015). A review and comparison of diagnostic instruments to identify students’
misconceptions in science. Eurasia Journal of Mathematics, Science & Technology Education, 11 (5), 989-1008.
Hasan, S., Bagayoko, D., & Kelley, E. L. (1999). Misconceptions and the certainty of response index (CRI). Physics Education,
34, 294-299.
Haslam, & Treagust, D.F. (1987). Diagnosing secondary students’ misconceptions of Photosynthesis and Respiration in plants
using a two-tier multiple choice instrument. Journal of Biological Education, 21 (3), 203-211.
Hoskinson, A. M., Caballero, M. D., & Knight, J. K. (2013). How can we improve problem solving in undergraduate biology?
Applying lessons from 30 years of physics education research. CBE Life Sciences Education, 12 (2),153–161. http://doi.
org/10.1187/cbe.12-09-0149.
Jena, A. K. (2014). Animation model to conceptualize ATP generation: a mitochondrial oxidative phosphorylation. European
Journal of Science and Mathematics Education, 3 (1), 61-76.
Kozma, R. (2009). Transforming education: Assessing and teaching 21st century skills assessment call to action. Teaching
and Learning, 6 (2), 66–72.
Lazarowitz, R., &Lieb, C. (2006). Formative assessment pre-test to identify college students’ prior knowledge, misconceptions
and learning difficulties in biology. International Journal of Science and Mathematics Education, 4 (4), 741-762.
Lien A. J. (1971). Measurement and evaluation of learning. Dubuque, Iowa: William C. Brown & Co. Lin, S. W. (2004). Develop-
ment and application of a two-tier diagnostic test for high school students’ understanding of flowering plant growth
and development. International Journal of Science and Mathematics Education, 2 (2), 175-199.
Lundeberg, M. A., Fox, P. W., & Punćcohaŕ, J. (1994). Highly confident but wrong: Gender differences and similarities in con-
fidence judgments. Journal of Educational Psychology, 86 (1), 114-121.
Mann, M., & Treagust, D. F. (2000). An instrument to diagnose conceptions of breathing, gas exchange and Respiration , in
Paper presented at the annual meeting of the National Association for Research in Science Teaching, New Orleans, L April
28 - May 1, 2000, 18.
544
(P. 532-546)
Merriam, S. B. (2009). Qualitative research: A guide to design and implementation. San Francisco, CA: Wiley.
Montfort, D., Brown, S., & Findley, K. (2007). Using interviews to identify student misconceptions in dynamics. Paper presented
at the 37thASEE/IEEE Frontiers in Education Conference, October 10 – 13, 2007, Milwaukee, WI.
Morris, C. C. (1990). Retrieval process underlying confidence in comprehension judgments. Journal of Experimental Psychol-
ogy: Learning, Memory and Cognition, 16, 223-232.
Mutlu, A., & Şeşen, B. A. (2016). Evaluating of preservice science teachers’ understanding of general chemistry concepts by
using two tier diagnostic test. Journal of Baltic Science Education, 15 (1), 79-96.
Oliver, M., Domingo, M., Hunter, J., Pan, L., & Gourlay, L. (2014). Pre-tertiary engagement with online learning: Exploring uses
of online learning environments and digital technology for progression into and through Higher Education Institute of
Education. University of London Commissioned by Pamoja Education, August 2014.
Osborne, R. J., & Gilbert, J. K. (1980). A method for investigating concept understanding in science. European Journal of Sci-
ence Education, 2 (3), 311–321.
Palmer, D. H. (2001). Factors contributing to attitude exchange amongst pre-service elementary teachers. Science Education,
86, 122–138.
Patton, M. Q. (2002). Qualitative evaluation and research methods (3rd ed.). Thousand Oaks, CA: Sage. Peterson, R. F., Trea-
gust, D. F., & Garnett, P. (1989). Development and application of a diagnostic instrument to evaluate grade-11 and –12
students’ concepts of covalent bonding and structure following a course of instruction. Journal of Research in Science
Teaching, 26 (4), 301-314.
Petrisor, M., Marusteri, M., Simpalean, D., Carasca, E., & Ghiga, D. (2016). Medical students’ acceptance of online assessment
systems. Acta Medica Marisiensis, 62 (1), 30-32.
Ricketts, C., & Wilks, S. J. (2010). Improving student performance through computer-based assessment: Insights from recent
research. Assessment & Evaluation in Higher Education, 27 (5), 475-479.
Ross, B. & Munby, H. (1991). Concept mapping and misconceptions: a study of high-school students understandings of acids
and bases. International Journal of Science Education, 13 (1), 11–23.
Scalise, K., & Gifford, B. (2006). Computer-based assessment in E-Learning: A framework for constructing “Intermediate
Constraint” questions and tasks for technology platforms. Journal of Technology, Learning and Assessment, 4 (6), 1-45.
Schmidt, H. J. (1997). Students’ misconceptions - Looking for a pattern. Science Education, 81 (2), 123–135.
Songer, C. J., & Mintzes, J. J. (1994), Understanding cellular respiration: An analysis of conceptual change in college biology.
Journal of Research in Science Teaching, 31, 621–637.
Stankov, L., & Crawford, J. (1997). Self-confidence and performance on cognitive tests. Intelligence, 25, 93–109.
Stankov, L., &Dolph, B. (2000). Metacognitive aspects of test-taking and intelligence. PsychologischeBeiträge, 42, 213–227.
Storey, D. R. (1989). Textbook errors & misconceptions in biology: photosynthesis. The American Biology Teacher, 51(5),
271–274.
Tan, K. C. D., Goh, N. K., Chia, L. S., & Treagust, D. F. (2002). Development and application of a two-tier multiple choice diag-
nostic instrument to assess high school students’ understanding of inorganic chemistry qualitative analysis. Journal
of Research in Science Teaching, 39 (4), 283–301.
Taslidere, E. (2016). Development of a three-tier diagnostic test to assess high school students’ misconception about the
photoelectric effect. Research in Science and Technological Education, 34 (2), 164-186.
Thompson, F., & Logue, S. (2006). An exploration of common student misconceptions in science. International Education
Journal, 7 (4), 553-559.
Treagust, D. F. (1995). Diagnostic assessment of students’ science concepts. In S. Glynn and R. Duit (Eds), Learning science in
the schools: Research reforming practice. New Jersey: Lawrence Erlbaum Associates, 327-346.
Tsai, C. C., & Chou, C. (2002). Diagnosing students’ alternative conceptions in science. Journal of Computer Assisted Learning,
18 (2), 157–165.
Tsui, C. Y., & Treagust, D. (2010). Evaluating secondary students’ scientific reasoning in genetics using a two-tier diagnostic
instrument. International Journal of Science Education, 32 (8), 1073-1098.
Uzuntiryaki, E., & Geban O. (2005). Effect of conceptual change approach accompanied with concept mapping on under-
standing of solution concepts. Instructional Science, 33 (1), 311–339. Woit. D, & Mason. D. (2003, January). Effectiveness
of online assessment. Paper presented at 34th SIGCSE technical symposium on Computer science education, New
York, (pp.137-141) ACM New York, USA.
Yakisan, M. (2016). Are the genetic materials of gametes and somatic cells different? The conceptions of pre-service teach-
ers. International Journal of Environmental and Science Education, 11 (4), 409-420.
545
ISSN 1648–3898
(P. 532-546)
Yusof, M. M. M., Halimin, N. M., &Shamsudin, N. M. (2016). Understanding of diffusion, osmosis and particulate theory of
matter conceptions among pre-service biology teachers. In 7th International Conference on University Learning and
Teaching (InCULT 2014) Proceedings (pp. 183-195). Springer Singapore.
Zakay, D., & Glicksohn, J. (1992). Overconfidence in a multiple-choice test and its relationship to achievement. Eurasia Journal
of Mathematics, Science & Technology Education, 11 (5), 989-1008.
Rohaida Mohd Saat PhD (UPM)(2003), University Putra Malaysia (UPM)

MSc (Education) (Indiana, USA)(1984), Indiana University,
Bloomington, USA
BSc (Biological Science) (Indiana, USA)(1982), Indiana University,
Bloomington, USA
Associate Professor, Department of Mathematics and Science
Education, University of Malaya, 50603, Kuala Lumpur, Malaysia
E-mail: rohaida@um.edu.my
Kamaludin A. Rashid PhD, The University Highfield Southampton University, UK
B.Sc (Hons), National University of Malaysia (UKM)
Associate Professor, Biology Division, Centre for Foundation Studies
in Science, University of Malaya, 50603, Kuala Lumpur, Malaysia
E-mail: kamalrashid@um.edu.my
Hidayah Mohd Fadzil B.Sc.Ed (UM), University of Malaya
Ph.D (Science Education), University of Malaya
Senior Lecturer, Department of Mathematics and Science
Education, University of Malaya, 50603, Kuala Lumpur, Malaysia
E-mail: hidayahfadzil@um.edu.my
Kamariah Haron MBiotek(UM), University of Malaya
B.Sc(Hons), National University of Malaysia (UKM)
Lecturer, Biology Division, Centre for Foundation Studies in Science,
University of Malaya, 50603, Kuala Lumpur, Malaysia
Email: k_haron@um.edu.my
Nor Azlina Abd. Aziz MSc., University of Malaya
BSc., University of Malaya
Lecturer, Biology Division , Centre for Foundation Studies in
Science, University of Malaya, 50603, Kuala Lumpur, Malaysia
E-mail: azlina_nal@um.edu.my
Natalya Rudina Shamsuar M.Sc.Creative Multimedia (MMU), Multimedia University
B.IT(Hons), University of Malaya
Lecturer, Faculty of Creative Multimedia, Multimedia University,
Persiaran Multimedia, Cyberjaya, 63100, Selangor, Malaysia
Email: natalya.shamsuar@mmu.edu.my
546
ISSN 1648–3898
INFORMATION
FOR CONTRIBUTORS
EDITORIAL AND REVIEW PROCEDURES
Journal of Baltic Science Education (JBSE) publishes original scientific research articles in the field of Natural Science Education and related
areas for all educational levels in the Baltic countries. It is possible to publish special (thematic) issues of JBSE. The papers should be submitted
and will be published in English. JBSE will promote to establish contacts between researchers and practical educators both in the Baltic countries
and countries around.
The authors of the manuscripts are responsible for the scientific content and novelty of the research materials. Articles, published before in
other international journals or papers’ collections will not be accepted for publication in JBSE.
As a publication that represents a variety of cross-disciplinary interests, both theoretical and practical, the JBSE invites manuscripts on a wide
range of topics, especially in the following areas:
• Didactics of natural sciences. • Philosophical, political, economical and social aspects of
• Theory and practice in natural science teacher natural science education.
education. • The supplementary natural science education.
• Integrated natural science education. • ICT in natural science education.
• Natural science and technological literacy. • The standardisation of natural science education etc.
• General and professional natural science education.
MANUSCRIPTS GUIDELINES
The structure of the research paper presented to the Journal of Baltic Science Education should be as follows: abstract - short report of the
investigation; introduction inc. aim and subject of the research; research methodologies and methods; results of the research incl. discussion; conclusions;
list of references in APA style.
The papers should be submitted in English. If English is a second language for the author, please consider having the manuscript proof read
and edited before submitting. The preliminary text of the article can be sent as a.doc file in the attachment by e-mail: mail.jbse@gmail.com
The text must be elaborated in Word for Windows, using 12 point Times New Roman letters. An article should not exceed 7-10 A4 pages, included
figures, tables and bibliography. Publishing of longer articles should be negotiated separately. Texts margins: top and bottom 20mm, left - 25mm, right
- 20mm. The title: capital letters, 14pt, bold; space between the title and the author’s name is one line interval. Author’s name and surname: small letters,
12pt, bold. Under the name, institution: 11 pt, italics; space between the title and the text: 1 line interval. Abstract – about 100-150 words - precedes the
text. The text: 12pt Single or Auto spacing, in one column. Key words: no more than five words. The language must be clear and accurate. The authors
have to present the results, propositions and conclusions in a form that can suit scientists from different countries.
Titles of the tables and figures: 11 pt, small letters. Space between figures or tables and the text: 1 line interval. Introduction, titles of chapters and
subchapters: 12pt, bold, small letters. Numbers: Arabic, subchapters numbered by two figures (1.1, 1.2, etc.). Figures, tables and captions should be inserted
within the manuscript at their appropriate locations. Diagrams and graphs should be provided as finished black and white line artwork or electronic
images. When there are a number of illustrations, the author should endeavour to reduce the amount of text to accommodate the illustrations in the
limited space available for any article.
References in the text should be presented in brackets (Knox, 1988; Martin, 1995). If necessary, the page can be indicated: (Martin, 1995, p.48).
The list of references should be presented after the text. The Words List of References: 11pt, bold, small letters. The references should be listed in
full at the end of the paper in the following standard form:
For books: Saxe, G.B. (1991). Cultural and Cognitive Development: Studies in Mathematical Understanding. Hillsdale, NJ: Erlbaum.
For articles: Bekerian, D.A. (1993). In Search of the Typical Eyewitness. American Psychologist, 48, 574-576.
For chapters within books: Bjork, R.A. (1989). Retrieval Inhibition as an Adaptive Mechanism in Human Memory. In: H.L. Roediger III & F.I.M. Craik
(Eds.), Varieties of Memory & Consciousness (pp. 309-330). Hillsdale, NJ: Erlbaum.
On a separate page, author - related data should be presented in English: name, surname, degree and academic title, institution, full correspondence
address in the clearest and most complete form /ordinary post and e-mail addresses /, position (to ensure anonymity in the review process). The author
(authors) should confirm in writing, that the manuscript has not been published in other journal or handed over (transferred) to other
journal for publication.
EDITORIAL AND REVIEW PROCEDURES

Manuscripts will be sent anonymously to reviewers with expertise in the appropriate area. All manuscripts will be rewieved by two experts
before JBSE’s accept them for publication. This process usually takes about two months. The journal co-editors will make minor editorial changes;
major changes will be made by the author(s) prior to publication if necessary. JBSE’s redaction will sent to author(s) only one correcture which
must be sent back within 2 weeks. JBSE will not review submissions previously published elsewhere through print or electronic medium.
Manuscripts submitted to the JBSE cannot be returned to authors. Authors should be sure to keep a copy for themselves. Authors’ signatures
should be at the end of the paper and its second checked proofs.
Manuscripts, editorial correspondence (and other correspondence for subscription and exchange), and any questions should be sent to
editor-in-chief or to regional redactors.
Mailing Addresses
Prof., Dr. Vincentas Lamanauskas, editor-in-chief, Dr. Naglis Švickus, co-editor, Lithuania
Siauliai University SMC ”Scientia Educologica”
P. Vishinskio Str. 25; LT-76351 Siauliai, Lithuania Kretingos Str. 55-10; LT-92300 Klaipėda, Lithuania
E-mail: v.lamanauskas@ef.su.lt E-mail: naglis.svickus@nbgroup.lt
Phone: + 370 687 95668 Phone: +370 687 89985
Prof., Dr. Janis Gedrovics, co-editor, Latvia

Riga Teacher Training and Educational Management Academy
Imantas 7 linija No 1; Riga, LV-1083, Latvia
E-mail: janis.gedrovics@rpiva.lv
Phone: +371 29162147
547
ISSN 1648–3898
JBSE
Problems of Psychology in the 21st

Century is an international, periodical,
peer reviewed scientific journal, issued by
the Scientia Socialis, UAB in cooperation
with SMC “Scientia Educologica”.
Editor-in-Chief
Prof. dr. Vincentas Lamanauskas, Scientific

Methodical Centre „Scientia Educologica“,
Republic of Lithuania
Problems of Psychology in the 21st Century

(ISSN 2029-8587) is abstracted and/or indexed
in:
Index Copernicus -
http://journals.indexcopernicus.com
EBSCO - http://search.ebscohost.com
Editorial Board
Dr., prof. Ferda Aysan, Dokuz Eylul University, Turkey

Dr., prof. Serhiy Boltivets, Grigory Kostyuk Psychological Institute of the Ukrainian National
Academy of Pedagogical Sciences, Ukraine
Dr., assoc. prof. Irena Gailiene, SMC "Scientia Educologica", Republic of Lithuania
Dr., prof. Irakli Imedadze, Dimitri Uznadze Georgian Psychological National Society, Georgia
Dr. Julia Lakhvich, Belarusian State University, Republic of Belarus
Dr., prof. Vladimir S. Karapetyan, Armenian State Pedagogical University named after Kh.
Abovyan, Armenia
Dr., prof. Vincentas Lamanauskas, Scientific Methodical Centre „Scientia Educologica“, Republic of
Lithuania (Editor-in-Chief)
Dr., prof. Maria LedziĔska, University of Warsaw, Poland
Dr., prof. Aleksandr Lobanov, Belarusian State Pedagogical University, Republic of Belarus
Dr., prof. Guna Svence, Riga Teacher Training and Educational Management Academy, Latvia
Website: http://www.jbse.webinfo.lt/PPC/Problems_of_Psychology.htm
548
ISSN 1648–3898
JBSE
Quality Issues and Insights in the 21st
Century is an international, periodical,
peer reviewed scientific journal, issued by
the Scientia Socialis, UAB in cooperation
with SMC “Scientia Educologica”.
Editor-in-Chief
Prof. dr. Vincentas Lamanauskas, Scientific

Methodical Centre „Scientia Educologica“,
Republic of Lithuania
Quality Issues and Insights in the 21st Century

(ISSN 2029-9575) is abstracted and/or indexed
in:
Index Copernicus -
http://journals.indexcopernicus.com
Electronic Serials in Lithuania -

http://www.eperiodika.lt/index.php?id=moksl
Editorial Board
MSc., Renata Bilbokaitơ, University of Siauliai, Republic of Lithuania (Editorial Assistant)

Dr., Andrea Bernhard, Universities Austria, Austria
Dr., Prof. John Gowland Mwangi, Egerton University, Kenya
Dr., Prof. Vincentas Lamanauskas, Scientific Methodical Centre „Scientia Educologica“, Republic of
Lithuania (Editor-in-Chief)
Dr., Prof. Liviu Moldovan, „Petru Maior” University of Tirgu Mures, Romania
Dr., Prof. Kestutis Pukelis, Vytautas Magnus University, Republic of Lithuania
Prof. Ramayah Thurasamy, University of Science Malaysia, Malaysia
Dr., Assoc. Prof. Dalius Serafinas, Vilnius University, Republic of Lithuania
Dr., Assoc. Prof. Adriana Tafrova-Grigorova, Sofia University St. Kliment Ohridski, Bulgaria
Website: http://www.jbse.webinfo.lt/QIIC/Quality_Issues_Insights.htm
549
ISSN 1648–3898
2nd International Baltic Symposium on

JBSE
Science and Technology Education

(BalticSTE2017)
„SCIENCE AND TECHNOLOGY
EDUCATION: ENGAGING THE
NEW GENERATION“
12-15 June 2017, Siauliai, Lithuania
SECOND ANNOUNCEMENT
Dear Colleagues,
On behalf of the organizing committee, we are delighted to welcome you to Šiauliai, Lithuania, for
the II International Baltic Symposium on Science and Technology Education, BalticSTE 2017.
The Symposium will be held in Šiauliai (Lithuania) in June 2017 during days 12-15.
We cordially encourage you to attend and contribute to one of the major events of 2017 on the field
of science and technology education. We are confident that you will appreciate the scientific
program and the city of Šiauliai. We look forward to seeing you in 2017 in Lithuania.
Organizers of the Symposium
Scientific Methodical Centre „Scientia Educologica“, Lithuania
http://www.gu.puslapiai.lt/indeks_en.htm
Siauliai University, Faculty of Education Science and Social Welfare, Lithuania.
http://www.su.lt
Ways of Participation
x Oral Presentation
x Interactive Poster Presentation
x Workshop
x Listener
Important Dates
Short Paper/Poster Submission 25 April 2017
Short Paper/Poster Acceptance 30 April 2017
Registration and payment 25 May 2017
Camera-ready short paper submission 25 May 2017
Announcement of the Final Symposium Program 02 June 2017
Symposium Dates 12 – 15 June 2017
(10/11 - arrival; 16/17 - departure)
550
ISSN 1648–3898
JBSE
REGISTRATION
Visit this link to make your registration: http://balticste.com/register
Until May, 15, 2017 (Early Bird) After May, 15, 2017
Participant Student Accompanying Participant Student Accompanying
Persons Persons
150 EUR 80 EUR 50 180 EUR 100 EUR 80
x The participation fee will cover coffee breaks, welcome reception, symposium programme,
symposium dinner, symposium materials, cultural programme.
x All payment should be made by bank transfer to the indicated bank account or via PayPal.
x All bank transfer commission fees should be covered by participant.
x Confirmation of payment should be sent to the organization committee via
e-mail: balticste@gmail.com
x Cancellations should be made in writing to the BalticSTE2017 Secretariat.
x Cancellations will not be possible after May 25, 2017.
x For any cancellation submitted till 25/05/2017, full refund will apply except from a processing
fee of 20€ for Participants, 5€ for Accompanying Persons and of 10€ for Students. Bank
charges to be deducted from the refunded amount.
x All refunds will be made after the Symposium.
Ways of Payment
1 option - Bank transfer (Thank you for sending us your bank transfer receipt identified with
your name)
Beneficiary: Scientific Methodical IBAN: LT327300010002393343

Center ”Scientia Educologica”
Beneficiary ID: 195736971 Beneficiary address: 29 K. Donelaiþio Street,
LT-78115 Siauliai, Lithuania
Bank: “SWEDBANK”, AB Swift code: HABA LT 22
Bank address: Konstitucijos Street 20A, Vilnius, Lithuania

Reference: BalticSTE2017 / Full name of Participant
* Bank charges to be paid by the participant
2 option – via PayPal system
To: scientia@scientiasocialis.lt
https://www.paypal.com/lt/
5 EUR should be added to the total amount as a transfer fee (it is valid only in the case of PayPal).
Symposium Website: http://balticste.com/

E-mail: balticste@gmail.com
Phone: +370 41 595736; Fax: +370 41 595710
© SMC „Scientia Educologica“, 2017
551
Compiled by: Vincentas Lamanauskas

Linguistic Editor: Ilona Ratkevičienė
Cover design by: Jurgina Jankauskienė
Layout design by : Linas Janonis
Contact person: Laima Railienė
31 August 2016 2016. Publishing in Quires 9,25. Edition 200
Publisher SMC „Scientia Educologica“ in cooperation with Scientia Socialis,

Donelaicio Street 29, LT-78115 Siauliai, Lithuania
E-mail: scientia@scientiasocialis.lt
Phone: +370 687 95668
http://www.jbse.webinfo.lt/centras.htm
http://www.scientiasocialis.lt

Printing Šiauliai printing house
9A P. Lukšio Street
LT-76207 Šiauliai, Lithuania
Phone: +370 41 500 333.
Fax: +370 41 500 336
E-mail: info@dailu.lt

Journal of Baltic Science Education, Vol. 15, No. 4, 2016

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Journal of Baltic Science Education, Vol. 15, No. 4, 2016

Uploaded by

Copyright:

Available Formats

Vol.15, No.

The International Journal of the Scientific Methodical Center “Scientia Educologica”

Published since 2002 Address:

ISSN 1648–3898 © Scientific Methodical Center „Scientia Educologica“, Lithuania,

KOREAN SCIENCE TEACHERS’ PERCEPTIONS AND ACTUAL USAGE OF EDUCATIONAL

LEARNER’S ALTERNATIVE AND MISCONCEPTIONS IN PHYSICS: A PHENOMENOGRAPHIC

EFFECTIVENESS OF THE INQF-BASED LEARNING ON A GENERAL PHYSICS FOR IMPROVING

A LEARNING EXERCISE USING SIMPLE AND REAL-TIME VISUALIZATION TOOL TO COUNTER

PRESERVICE TEACHERS’ PERCEPTIONS OF THE RELATIONSHIP BETWEEN SCIENCE

INTEGRATING STEM IN AN ENGINEERING DESIGN PROCESS: THE LEARNING EXPERIENCE OF

DETERMINING PROSPECTIVE BIOLOGY TEACHERS’ COGNITIVE STRUCTURE IN TERMS OF

PEDAGOGICALLY DESIRABLE SCIENCE EDUCATION: VIEWS ON INQUIRY-BASED SCIENCE

IMPROVING ACADEMIC ACHIEVEMENT OF SCIENCE LEARNERS IN RURAL SCHOOLS

DEVELOPMENT OF AN ONLINE THREE-TIER DIAGNOSTIC TEST TO ASSESS PRE-UNIVERSITY

PROBLEMS OF PSYCHOLOGY IN THE 21ST CENTURY . . . . . . . . . . . . . . . . . . . . . . . . . . . 548

QUALITY ISSUES AND INSIGHTS IN THE 21ST CENTURY . . . . . . . . . . . . . . . . . . . . . . . . . 549

SCIENCE AND TECHNOLOGY EDUCATION: ENGAGING THE NEW GENERATION . . . . . . . . . 550

SYSTEMS THEORY OF SYSTEMS THINKING: GENERAL AND PARTICULAR WITHIN

•• Human is reflecting world phenomena in human’s world of thoughts as systems.

Received: July 15, 2016 Accepted: August 25, 2016

Andris Broks PhD., Professor Emeritus, University of Latvia, 19 Raina

Abstract. Many researchers have reported

General Characteristic of Research

Table 1. Basic backgrounds of the participants (n = 87).

Major Number School level Number Teaching years Number

Physics 21 Middle school 39 1~5 27

Competition ratio for each year

Physics 1:11 (96/1013) 1:9 (112/1012) 1:8 (116/943) 1:6(138/786)

Table 3. Contents of the questionnaire.

Content Question Type

What Science Teachers Know about ETTS

Table 4. ETTS items frequently mentioned by the science teachers (n = 465).

ETTS Percentage (%)

Importance of ETTS as Perceived by Science Teachers

Variable Importance Independent t-value df

The ETTS are important because they help … Percentage (%)

The ETTS are not important because of … Percentage (%)

The Use of ETTS in Actual Science Teaching

Table 8. Teachers’ use of ETTS (n = 87).

Variable Use Independent t value df

Gap between Theory and Practice in Science Teaching

A: What teachers know (in Ratio

Learning cycle 58 25a 43

Variable Importance Use Paired t value df

The Reasons of the Gap

Reasons Percentage (%)

•• Curriculum (ex. Too much content to be learned.)

•• Teaching materials (ex. We need various teaching materials applying educational

Figure 1: Graphical summary of the teachers’ responses shown in Table 12.

Teacher: We need various teaching materials that can be used immediately.

Received: March 17, 2016 Accepted: June 15, 2016

Jongwon Park Ph.D., Professor, Department of Physics Education, College of

Jongseok Park Ph.D., Associate Professor, Department of Chemistry Education,

Abstract. This phenomenographic case Abraham Motlhabane

Instrument and Procedures

Results of the Research

Table 1. Learners’ descriptions of a distance of 0m.

Descriptions of a distance of 0m Alternative conception Misconception

A The zero distance means there is no distance covered X

D The is no movement taken X

E Being stationary or not moving at all X

F This object is not moving, its distance is 0, therefore it is stationary X

G There is no distance covered X

H There is no distance or if the distance is 0 that means is initial velocity X