This article was downloaded by: [Universite De Paris 1]

On: 14 September 2013, At: 00:47

Publisher: Routledge
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered
office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

International Journal of Science

Education
Publication details, including instructions for authors and
subscription information:
http://www.tandfonline.com/loi/tsed20

A Three-Tier Diagnostic Test to Assess

Pre-Service Teachers’ Misconceptions
about Global Warming, Greenhouse
Effect, Ozone Layer Depletion, and
Acid Rain
a b c
Harika Ozge Arslan , Ceyhan Cigdemoglu & Christine Moseley
d

a
Department of Secondary Science and Mathematics Education,
Yuzuncu Yil University, Van, Turkey
b
Department of Secondary Science and Mathematics Education,
Middle East Technical University, Ankara, Turkey
c
Department of Chemical Engineering and Applied Chemistry,
Atilim University, Ankara, Turkey
d
Department of Interdisciplinary Studies, University of Texas at
San Antonio, San Antonio, TX, USA
Published online: 08 May 2012.

To cite this article: Harika Ozge Arslan , Ceyhan Cigdemoglu & Christine Moseley (2012) A Three-
Tier Diagnostic Test to Assess Pre-Service Teachers’ Misconceptions about Global Warming,
Greenhouse Effect, Ozone Layer Depletion, and Acid Rain, International Journal of Science
Education, 34:11, 1667-1686, DOI: 10.1080/09500693.2012.680618

To link to this article: http://dx.doi.org/10.1080/09500693.2012.680618

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the
“Content”) contained in the publications on our platform. However, Taylor & Francis,
our agents, and our licensors make no representations or warranties whatsoever as to
the accuracy, completeness, or suitability for any purpose of the Content. Any opinions
and views expressed in this publication are the opinions and views of the authors,
and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content
should not be relied upon and should be independently verified with primary sources
of information. Taylor and Francis shall not be liable for any losses, actions, claims,
 proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or
howsoever caused arising directly or indirectly in connection with, in relation to or arising
out of the use of the Content.

This article may be used for research, teaching, and private study purposes. Any
substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,
systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &
Conditions of access and use can be found at http://www.tandfonline.com/page/terms-
and-conditions
Downloaded by [Universite De Paris 1] at 00:47 14 September 2013
 International Journal of Science Education
Vol. 34, No. 11, July 2012, pp. 1667–1686

A Three-Tier Diagnostic Test to Assess

Pre-Service Teachers’ Misconceptions
about Global Warming, Greenhouse
Effect, Ozone Layer Depletion, and
Downloaded by [Universite De Paris 1] at 00:47 14 September 2013

Acid Rain
Harika Ozge Arslana,b∗ , Ceyhan Cigdemogluc and
Christine Moseleyd
a
Department of Secondary Science and Mathematics Education, Yuzuncu Yil University,
Van, Turkey; bDepartment of Secondary Science and Mathematics Education, Middle
East Technical University, Ankara, Turkey; cDepartment of Chemical Engineering and
Applied Chemistry, Atilim University, Ankara, Turkey; dDepartment of Interdisciplinary
Studies, University of Texas at San Antonio, San Antonio, TX, USA

This study describes the development and validation of a three-tier multiple-choice diagnostic test,
the atmosphere-related environmental problems diagnostic test (AREPDiT), to reveal common
misconceptions of global warming (GW), greenhouse effect (GE), ozone layer depletion (OLD),
and acid rain (AR). The development of a two-tier diagnostic test procedure as described by
Treagust constitutes the framework for this study. To differentiate a lack of knowledge from a
misconception, a certainty response index is added as a third tier to each item. Based on
propositional knowledge statements, related literature, and the identified misconceptions
gathered initially from 157 pre-service teachers, the AREPDiT was constructed and administered
to 256 pre-service teachers. The Cronbach alpha reliability coefficient of the pre-service teachers’
scores was estimated to be 0.74. Content and face validations were established by senior experts.
A moderate positive correlation between the participants’ both-tiers scores and their certainty
scores indicated evidence for construct validity. Therefore, the AREPDiT is a reliable and valid
instrument not only to identify pre-service teachers’ misconceptions about GW, GE, OLD, and
AR but also to differentiate these misconceptions from lack of knowledge. The results also reveal
that a majority of the respondents demonstrated limited understandings about atmosphere-
related environmental problems and held six common misconceptions. Future studies could test

∗
Corresponding author: Department of Secondary Science and Mathematics Education, Yuzuncu
Yil University, Van, Turkey. Email: harika@metu.edu.tr or harikaozge@hotmail.com

ISSN 0950-0693 (print)/ISSN 1464-5289 (online)/12/111667–20

# 2012 Taylor & Francis
http://dx.doi.org/10.1080/09500693.2012.680618
 1668 H. O. Arslan et al.

the AREPDiT as a tool for assessing the misconceptions held by pre-service teachers from different
programs as well as in-service teachers and high school students.

Keywords: Three-tier diagnostic assessment; Misconceptions; Environmental education;

Global warming; Ozone layer depletion; Acid rain

Introduction
Since the mid-1970s, a dramatic increase in the number of studies about students’
understandings of scientific concepts has occurred. Researchers have conducted hun-
dreds of studies identifying pre-instructional ideas held by students at different grade
Downloaded by [Universite De Paris 1] at 00:47 14 September 2013

levels and in different content areas. This interest in understanding about these ideas
and their causes and effects on student learning and achievement continues today.
Although researchers have referred to these pre-instructional ideas with different
terms, including alternative conceptions, misconceptions, alternative frameworks,
naive beliefs, preconceptions, naive notions, and pre-scientific notions, they are in
agreement that many of these ideas are different from those generally accepted by
the scientific community (Clement, Brown, & Zietsman, 1989; Driver, 1988;
Hammer, 1996; Odom & Barrow, 1995; Treagust, 1988). For this particular study,
the term misconception is used when referring to students’ incompatible ideas with
scientific views. Research-based evidence indicates that misconceptions create a
barrier to knowledge restructuring, are often held strongly resistant to change, and
need to be overcome (Clough & Driver, 1985; Hammer, 1996; Osborne, Bell, &
Gilbert, 1983; Posner, Strike, Hewson, & Gertzog, 1982). Therefore, identifying mis-
conceptions and their causes prior to teaching becomes important in developing
lessons that ultimately result in the reconceptualization of learning (Clough &
Driver, 1985; Odom & Barrow, 1995; West & Pines, 1985).
If teachers have such misconceptions, how will this affect the teaching–learning
environment? Hashweh (1987) found that teachers sometimes hold the same miscon-
ceptions that their students hold. Many of these misconceptions appear in their lesson
planning and teaching and result in reinforcing students’ misconceptions instead of
remediating them with scientific facts. Dove (1996) and Groves and Pugh (1999)
highlighted the importance of identifying misconceptions about environmental
issues in undergraduate courses; otherwise, the pre-service teachers might pass on
false information to, or fail to correct, the children they will eventually teach.
The environmental issues of global warming (GW) and climate change caused by
the industrialization and expansion of urban environments are complex and inter-
related. Because of this complex and multi-layered interrelationship, many times in
the media and literature these environmental issues tend to be referred to in a
rather synonymous way even though these terms each possess different meanings.
Seymour explained GW within the context of climate change as ‘a “warming” is
only one phase of the larger climate system on Earth that naturally features change’
(2008, p. 12). Thus, climate change includes GW and other factors caused by the
 Three-Tier Test on Atmosphere-Related Problems 1669

increase of atmospheric greenhouse gases. Therefore, in this study the term GW is

used because of the complex nature of climate change.
As a result of the science community unravelling alarming trends and patterns
regarding changes in the composition of the atmosphere, GW, greenhouse effect
(GE), ozone layer depletion (OLD), and acid rain (AR) have became critical elements
of discussion and study among scientists, politicians, and the general public. Conse-
quently, substantial research in education has concentrated on students’ conceptualiz-
ation of these topics (Boyes & Stanisstreet, 1992, 1993; Cordero, 2001; Francis, Boyes,
Qualter, & Stanisstreet, 1993; Seymour, 2008). However, limited research has been
conducted to identify pre-service or in-service teachers’ misconceptions regarding
these environmental problems. In the last decade, studies focusing on diagnosing
pre-service and in-service teachers’ misconceptions about the atmosphere-related
Downloaded by [Universite De Paris 1] at 00:47 14 September 2013

environmental problems of GW, GE, OLD, and AR have emerged (Boyes, Chambers,
& Stanisstreet, 1995; Daskolia, Flogaitis, & Papageorgiou, 2006; Dove, 1996; Groves
& Pugh, 1999, 2002; Khalid, 2001, 2003; Papadimitriou, 2004; Summers, Kruger,
Childs, & Mant, 2000). The results of these studies have indicated that teachers hold
prevalent misconceptions on these particular topics and most are the same as their stu-
dents’ misconceptions. The studies revealed a widespread confusion between GE and
OLD. Over half of the participants from all of these studies held the misconception that
OLD directly increases GW by letting in more sunrays. The most common misconcep-
tions identified in these previous studies included the following: (a) GW will cause skin
cancer; (b) carbon dioxide is the main contributing factor towards these environmental
problems; (c) the ozone layer helps to keep the earth warm, and (d) AR occurs because
of OLD or GE. Moreover, studies have indicated interlinked misconceptions between
GW, GE, and OLD (Boyes & Stanisstreet, 1992; Boyes, Chuckran, & Stanisstreet,
1993; Groves & Pugh, 2002); GW and AR (Boyes et al., 1993); and OLD and AR
(Pekel & Ozay, 2005). The interrelatedness of atmosphere-related environmental pro-
blems triggered this study to focus on these four environmental problems together.
The methodologies of the aforementioned research studies used to identify tea-
chers’ misconceptions about environmental problems varied and included interviews
(Summers et al., 2000), free word association tasks (Daskolia et al., 2006), open-
ended questionnaires (Papadimitriou, 2004), close-ended questionnaires (Boyes
et al., 1995; Groves & Pugh, 1999, 2002; Michail, Stamou, & Stamou, 2007), and
questionnaires with open-response items (Dove, 1996; Khalid, 2001, 2003). A
large number of studies on determining students’ or teachers’ misconceptions have
used close-ended questionnaires adopted from the works of Boyes and Stanisstreet
(1993) and/or Dove (1996). However, despite the easy usage of close-ended question-
naires for both participants and researchers, one can draw misleading conclusions
because of the limited range of options. Dove (1996) and Khalid (2001, 2003)
added open-ended questions to collect data on pre-service teachers’ reasons for
their responses to close-ended questionnaires. However, many of the participants
did not provide any reason for their responses in the space provided (Dove, 1996).
Moreover, Groves and Pugh (2002) found that many pre-service teachers avoided
taking a definite position by choosing the middle ‘I don’t know’ choice.
 1670 H. O. Arslan et al.

Over the past three decades, diagnostics tests have become a relatively prominent
assessment tool in science education for data collection concerning the misconcep-
tions on domain-specific knowledge of students or teachers. Treagust (1988), as an
initiator of designing two-tier diagnostic instruments that specifically identify
misconceptions, suggested the addition of a reason tier derived from actual students’
misconceptions. In science education, two-tier diagnostic tests have been used
in several studies in different content areas (Chu, Treagust, & Chandrasegaran,
2009; Griffard & Wandersee, 2001; Odom & Barrow, 1995; Tsui & Treagust,
2010; Wang, 2004). Although two-tier tests provide more information than other
commonly applied methods for efficiently collecting data from large populations,
some limitations have been identified. The presence of guessing may result due to
overestimating the participants’ levels of knowledge as well as misconceptions as
Downloaded by [Universite De Paris 1] at 00:47 14 September 2013

these tests do not discriminate lack of knowledge from misconceptions (Caleon &
Subramaniam, 2010; Pesman & Eryilmaz, 2010). An additional tier, which contains
a certainty of response index, has been proposed to compensate for the likely weak-
ness of the diagnostic tests (Hasan, Bagayoko, & Kelley, 1999; Pesman & Eryilmaz,
2010). To date, only a few studies that use three-tier diagnostic tests have been
conducted in science education (Caleon & Subramaniam, 2010; Eryilmaz &
Surmeli, 2002; Pesman & Eryilmaz, 2010) and none of these studies have focused
on environmental issues.
In educational research, studies that both diagnose misconceptions and differen-
tiate them from a lack of knowledge are limited. Therefore, the study reported in
this article has the potential to contribute to the literature by developing a valid and
reliable three-tier diagnostic test to assess pre-service teachers’ misconceptions on
atmosphere-related environmental problems. Specifically, this study aims to (1)
develop and validate a reliable three-tier diagnostic test on GW, GE, OLD, and AR
and (2) contribute to the environmental education literature by distinguishing pre-
determined findings as misconceptions or lack of knowledge on the above-mentioned
environmental problems.

Method and Procedures

Development of the Test
The procedure as described by Treagust (1988) constitutes the methodological
framework for the development of the atmosphere-related environmental problems
diagnostic test (AREPDiT). First, concept boundaries regarding the topics of GW,
GE, OLD, and AR are formed along with the identification of 18 propositional
knowledge statements (Table 1) as well as a concept map pertaining to basic
knowledge needed by pre-service teachers as derived from textbooks and expert
opinions. Two professors, one majoring in environmental education and one major-
ing in chemistry, reviewed the identified list of propositional knowledge statements
and the concept map. Content validation, confirmation of the accuracy, and appro-
priateness of the content was established upon expert agreement.
 Three-Tier Test on Atmosphere-Related Problems 1671

Table 1. Propositional knowledge statements

Content Question
area Propositional knowledge statements (Q)

GW GW is a periodic increase in atmospheric temperature due to an Q1

increase in the concentration of the greenhouse gases
Human activities such as deforestation and burning of fossil fuels Q1
initiate an increase in the greenhouse gases (carbon dioxide (CO2),
methane (CH4), etc.), thus giving rise to GW
An increase in global temperature will result in extreme events and Q2
expansion of deserts
Melting of snow and ice and rising global sea levels are also Q2
consequences of GW
Downloaded by [Universite De Paris 1] at 00:47 14 September 2013

Reducing greenhouse gas emissions directly reduces GW Q5

Forestation decreases CO2 concentrations. Therefore, it is one of the Q6
solutions for GW
GE GE is the absorption of some of the infrared radiation that is reflected Q3
from the Earth by various gases
Greenhouse gases are CO2, water vapour (H2O(g)), CH4, ozone (O3), Q1, Q3
chlorofluorocarbons (CFCs), and carbon monoxide (CO)
GE occurs naturally. However, it becomes harmful when the average Q3, Q4
concentration of greenhouse gases rises above the normal level
OLD The O3 layer filters the Sun’s high-frequency ultraviolet (UV) light Q7
These high-frequency UV rays have potential to damage life on Q7
the Earth
CFCs deplete the O3 layer that protects the Earth’s surface from Q8
damaging UV radiation
OLD can cause a range of health hazards such as skin cancer and Q9
cataracts
A reduction in consuming aerosol sprays with CFCs decreases OLD Q10
AR Pollutants such as sulphur dioxide (SO2) and nitrogen oxide (NOx) Q11
react with water molecules in the atmosphere to form AR
Fossil fuels generate SO2 and NOx emissions, leading to formation Q12
of AR.
AR can have harmful effects on plants, aquatic life along with Q12
buildings, monuments, and sculptures
Reducing the use of fossil fuels as energy sources is one of the Q13
precautions that can be taken against AR formation

At a later stage, 13 open-ended questions were written based on the concept

boundaries and in accordance with the extensive related literature on GW, GE,
OLD, and AR to identify pre-service teachers’ conceptions and misconceptions.
This open-ended questionnaire was initially administered to 157 undergraduate
pre-service teachers. An analysis of the responses revealed their difficulties in compre-
hending the above-mentioned environmental problems. For each question, the
identified misconceptions were sorted according to their frequencies for later use in
the development of the instrument.
 1672 H. O. Arslan et al.

Subsequently, the propositional knowledge statements and misconceptions col-

lected were used to construct the first version of a two-tier multiple-choice test.
The most frequently stated misconceptions (Table 2) were used to form the alterna-
tives in the multiple-choice questions. A third tier, the certainty of response (yes or
no), was then added to this version of the test. As described by Treagust (1988),
the first part of each item on the test is a multiple-choice content question having
usually two to five choices (first tier). The second part contains a set of possible
reasons for the answers given in the first part, with one blank choice to express any
personal reason (second tier). The third part of each item is a confidence tier,
which investigates whether or not the subjects are confident about their responses
(third tier).
Five graduate students in science education, a professor from the chemistry depart-
Downloaded by [Universite De Paris 1] at 00:47 14 September 2013

ment, and two professors in environmental education examined the first version of the
AREPDiT. Additionally, a native English language expert reviewed the test for sen-
tence construction.
The first version of the AREPDiT was piloted with 126 pre-service teachers (23%
male and 77% female). The data were typed into the SPSS (Statistical Package for the
Social Sciences) program and dichotomized based on an answer key. All correct
answers to the first- and second-tier questions along with being certain (that is, select-
ing the ‘Yes’ alternative in the third tier) were coded as 1 and 0 otherwise. Responses
to the open-ended second-tier questions were categorized based on the alternatives
provided in the reason tier. Unclear or ambiguous responses were coded as incorrect
answers. Cronbach alpha reliability, item difficulties, and point biserial correlation
coefficients were used for item analysis. The item difficulty and point biserial
correlation (the correlation of an item score with total score) of each question
showed that item revisions were needed. Following an evaluation based on the item
analysis, items 3 and 12 were revised completely, and items 6, 7, 10, and 13 were
refined. In addition, question 2 was dropped from the first version since its point
biserial correlation coefficient was quite low and a new question was generated regard-
ing the nature of the ozone layer. After a review process and repeated revisions, the
second version of the AREPDiT was developed with 13 items. The MS Word
program calculated the Flesch–Kincaid grade level readability to be 9.1 for the
second version of the AREPDiT. Such a number means that this test is suitable for
grade level 9 and higher. Example items of the second version of the AREPDiT are
shown in Figure 1.

Sample
Two hundred and fifty-six pre-service teachers in an urban southwestern university in
the USA were selected by convenience sampling for the study. Of these, 108 partici-
pants were Hispanic, 126 were non-Hispanic, and 22 did not state their ethnic back-
ground. The age range was between 18 and 56 years and participants were classified
as either juniors or seniors (third or fourth year of their undergraduate degree). The
participants consisted of 205 females, 41 males, and 10 individuals who did not state
 Three-Tier Test on Atmosphere-Related Problems 1673

Table 2. List of misconceptions and alternative sets

Misconceptions Alternative sets

M1. GW is caused by OLD 1.1 a, 1.2 a, 1.3a;

1.1 a,1.2 b, 1.3 a;
9.1 e, 9.2 c, 9.3 a
M2. GW will cause skin cancer 2.1 b, 2.2 b, 2.3 a
M3. AR is a result of GW 2.1 c, 2.2 a, 2.3 a
M4. Recycling more paper is not an effective cure for GW 5.1 a, 5.2 c, 5.3 a
M5. Generating electricity from renewable sources does not help to 5.1 c, 5.2 b, 5.3 a
reduce GW
M6. Stopping the usage of CFCs is not a cure for GW 5.1 d, 5.2 a, 5.3 a
M7. GW can be reduced by setting limitations on chemical waste 6.1 a, 6.2 d, 6.3 a
Downloaded by [Universite De Paris 1] at 00:47 14 September 2013

released into rivers

M8. GW can be reduced without building nuclear power plants 6.1 b, 6.2 a, 6.3 a
M9. Set a limit on pesticide usage on farmland 6.1 c, 6.2 c, 6.3 a
M10. GE is not a natural phenomenon 3.1 b, 3.2 c, 3.3 a
M11. Carbon dioxide (CO2) is the only gas that increases GE 3.1 c, 3.2 a, 3.3 a
M12. GE is a totally harmful phenomenon for mankind 4.1 b, 4.2 d, 4.3 a
M13. GE is a totally helpful phenomenon for mankind 4.1 a, 4.2 c, 4.3 a
M14. GE has no effect on mankind 4.1 d, 4.2 a, 4.3 a
M15. The ozone layer protects the Earth from AR 7.1 a, 7.2 c, 7.3 a
M16. The ozone layer helps to keep the Earth’s temperature stable to 7.1 c, 7.2 b, 7.3 a
make it livable
M17. CO2 depletes the ozone layer in the stratosphere 8.1 a, 8.2 b, 8.3 a
M18. GE leads to OLD 8.1 b, 8.2 d, 8.3 a
M19. Nuclear power plants affect the depletion of the ozone layer 8.1 c, 8.2 a, 8.3 a
M20. Carbon monoxide (CO) causes OLD 8.1 d, 8.2 e, 8.3 a
M21. OLD causes an increase in the number of floods 9.1 a, 9.2 a, 9.3 a
M22. Too much sun rays enter the atmosphere by OLD 9.1 b, 9.2 f, 9.3 a
M23. OLD lets the air escape from the atmosphere 9.1 c, 9.2 e, 9.3 a
M24. Using public transportation reduces OLD 10.1 a, 10.2 b, 10.3 a
M25. Using filters for smoke from factories and cars reduces OLD 10.1 b, 10.2 a, 10.3 a
M26. OLD leads to AR 11.1 c, 11.2 a, 11.3 a
M27. Methane (CH4) from landfills leads to AR 11.1 a, 11.2 b, 11.3 a
M28. OLD becomes worse by AR 12.1 a, 12.2 d, 12.3 a
M29. AR leads to an increase in GW 12.1 b, 12.2 c, 12.3 a
M30. AR helps some plants and animals to survive 12.1 d, 12.2 b, 12.3 a
M31. AR can burn everything that it comes in contact with 12.1 e, 12.2 a, 12.3 a
M32. Avoiding activities that damage the ozone layer is a precaution 13.1 a, 13.2 b, 13.3 a
for AR
M33. CO is the main culprit of AR 13.1 c, 13.2 a, 13.3 a

their gender on the survey. The individuals in the sample were studying for teaching
certificates in the following areas: (a) 111 early childhood generalist (EC–6th grade),
(b) 35 mid-level education specialist (4th–8th grade), (c) 22 special education (EC –
12th grade), (d) 32 secondary education specialist (8th–12th grade), and (e) 56 of the
subjects did not mention their majors.
 1674 H. O. Arslan et al.
Downloaded by [Universite De Paris 1] at 00:47 14 September 2013

Figure 1. Two items from the AREPDiT

Data Collection and Analysis

The AREPDiTwas administered at the beginning of undergraduate classes in the teacher
education programme during the 2010–2011 academic year. Pre-service teachers com-
pleted the test as individuals and they were not required to write their names to assure
anonymity. The survey administration lasted approximately 20–25 min. Parallel to
coding and computations used in the pilot study, scale statistics were conducted to deter-
mine if the items functioned properly. Exploratory factor analysis was conducted to reveal
whether appropriate factors loaded or not. Eight scores—the first tiers, both tiers, total,
certainty, and lack of knowledge, Misconception first tiers (M-first tiers), Misconception
 Three-Tier Test on Atmosphere-Related Problems 1675

both tiers (M-both tiers), and Misconception all tiers (M-all tiers)—were calculated as
described by Pesman and Eryilmaz (2010). Figure 2 summarizes how these scores
were calculated. The percentages of these scores were computed.
The first-tiers score and M-first tiers score were calculated based only on the answers
to the first-tier questions of the items of the AREPDiT. The pre-service teachers’
first-tiers score is the sum of their correct responses to the first tiers of all items. Item
difficulty, as described by Crocker and Algina (1986), is the proportion of examinees
who answered an item correctly. Based on this description, the proportion of pre-
service teachers who answered only the first tier of each item correctly was calculated
as the difficulty level of the first tier of each item. The mean difficulty level of the
first-tier scores is the difficulty level of the one-tier test (a simple multiple-choice
test). The M-first tiers score is the sum of the pre-service teachers’ responses, which
Downloaded by [Universite De Paris 1] at 00:47 14 September 2013

are misconceptions, to only the first tiers of the alternative sets in Table 2.
Both-tiers score and M-both tiers score take into account both the first and second
tiers. Pre-service teachers’ both-tiers score is the sum of their correct responses to
both the first and second tiers (correct and correct) of all items. Similar to the calcu-
lations done for the difficulty level of the one-tier test, the difficulty level of the two-
tier test was calculated. The M-both tiers score is the sum of the pre-service teachers’
responses which are misconceptions in the first and second tiers together (misconcep-
tion and misconception). The misconception in the first tier should be consistent with
the misconception in the second tier (Table 2).

Figure 2. A diagram of coding and scoring procedure. ∗ Misconceptions in the second tier have to
be consistent with misconceptions in the first tier of the items. Table 2 gives alternative sets of
misconceptions. ∗∗ Three combinations are ‘incorrect and correct’, ‘correct and incorrect’, and
‘incorrect and incorrect’
 1676 H. O. Arslan et al.

The first tier, second tier, and third tier (certainty index) of the items were taken
into consideration all together to calculate total score and M-all tiers score. As
Caleon and Subramaniam (2010) stated ‘students tended to be poorly discriminating
between what they know and what they do not know, confidence ratings may reflect
the strength of students’ conceptual understanding, as well as their alternative con-
ceptions’ (p. 941). Therefore, total score is the sum of correct responses to both first
and second tiers along with being certain (correct and correct and certain). The
pre-service teachers with high total scores have greater scientific knowledge about
the atmosphere-related environmental problems in this study. The difficulty level of
the total scores (the difficulty level of the three-tier test) was calculated in the same
way as for the first-tiers and both tiers scores. Exploratory factor analysis was con-
ducted from the pre-service teachers’ total scores.
Downloaded by [Universite De Paris 1] at 00:47 14 September 2013

Similar to the study of Caleon and Subramaniam (2010), Odom and Barrow
(2007) indicated that students having high confidence (being certain) in their incor-
rect responses represent tenacious misconceptions. The M-all tiers score is the sum of
the pre-service teachers’ responses to both first and second tiers, which are miscon-
ceptions, along with being certain (misconception and misconception and certain).
As explained before for the scores based on both two tiers, the misconception in
the first tier should be consistent with the misconception in the second tier (Table 2).
Lack of Knowledge score is the situation of being uncertain regardless of correct or
incorrect responses to the first and/or second tiers as described by Hasan et al.
(1999). In their study, the description was ‘irrespective of whether the answer was
correct or wrong, a low certainty response index value indicates guessing, which, in
turn, implies lack of knowledge’ (p. 3). Similarly, Odom and Barrow (2007) have
stated that ‘students who have low certainty in their answer combinations were poss-
ibly guessing and, therefore, had no understanding, or were confused about their
understanding’ (p. 97). Consequently, this study also treats specified combinations
of responses from all three tiers (correct/incorrect/uncertain, incorrect/correct/uncer-
tain, and incorrect/incorrect/uncertain) for each item as ‘lack of knowledge’. The Cer-
tainty Score is related to the responses in only the third tier (that is, the certainty tier).
The summation of each pre-service teacher’s ‘Yes’ responses is his/her certainty score.
The overall response possibilities to the AREPDiT (first, second, and third tiers
together) bring out five categories: Scientific knowledge, Misconception, False posi-
tives/negatives, Lucky guess, and Lack of knowledge (Table 3).
False negatives and false positives are the terms used for the errors of assessment in
scientific research. A false positive means that an effect is not actually present, whereas
a false negative fails to reveal an effect that is actually present. Hestenes and Halloun
(1995) recommended that a correct answer along with a wrong reason (false positive)
and a wrong answer followed by a correct reason (false negative) be used to provide
evidence for content validity in their development of the force concept inventory, a
well-known diagnostic test in science education. Minimization of the probability of
false negatives and false positives provides higher validity in multiple-choice tests
(Hestenes & Halloun, 1995). They advised that the probability of false negatives
should be less than 10%. In the present study, the combinations of ‘correct and
 Three-Tier Test on Atmosphere-Related Problems 1677

Table 3. All possibilities of responses

First tier Second tier Third tier Categories

Correct Correct Certain Scientific knowledge

Correct Incorrect Certain Misconception (false positive)
Incorrect Correct Certain Misconception (false negative)
Incorrect Incorrect Certain Misconception
Correct Correct Uncertain Lucky guess, lack of confidencea
Correct Incorrect Uncertain Lack of knowledge
Incorrect Correct Uncertain Lack of knowledge
Incorrect Incorrect Uncertain Lack of knowledge

a
Also referred to in the literature as ‘Lucky Guess’.
Downloaded by [Universite De Paris 1] at 00:47 14 September 2013

incorrect and certain’ and ‘incorrect and correct and certain’ are treated as false posi-
tives and false negatives, respectively, and the percentages of these errors were
calculated.
To date, researchers constructing tests with certainty indices have treated all uncer-
tain responses as lack of knowledge (Hasan et al., 1999; Odom & Barrow, 2007;
Pesman & Eryilmaz, 2010). However, the authors of this paper suggest that correct
answers to both tiers with uncertainty might be identified as a lucky guess or lack of
confidence. When the correct answer is given by guessing in the first tier, the possi-
bility of selecting a related reason is high in two-tier tests because of the fact that
the alternatives of the first tier are usually related to the second tier choices. The
pre-service teachers’ low self-confidence in science concepts might be another expla-
nation for being uncertain of correct responses to the first two tiers.

Results and Discussion

Statistics of the AREPDiT
The overall descriptive statistics of the pre-service teachers’ total score in the second
version of the AREPDiT have been summarized in Table 4. After the revision of the
first version, the Cronbach alpha coefficient of the second version increased from 0.60
to 0.74, which can be considered acceptable according to criterion-referenced tests
(Crocker & Algina, 1986; Kane, 1986).
The item difficulty is the percentage of respondents who answered the item cor-
rectly. A lower number indicates a difficult item—fewer people chose a correct
response—while a higher number means that a correct response to the item was
chosen by more people. The low means of the difficulty indices of the AREPDiT indi-
cated that the test was very difficult for pre-service teachers. As would be expected,
the mean difficulty indices decreased as the number of tiers increased (first tiers:
0.42; both two tiers: 0.28; all three tiers: 0.19). Thus, an assumption could be
made that answering just one multiple-choice test item correctly is easier than
trying to find a reason for the selected answer and being certain about it. The very
 1678 H. O. Arslan et al.

Table 4. Descriptive statistics of AREPDiT total score

Number of items 13
Number of participants 256
Mean/standard deviation (SD) 2.42/2.47
Minimum 0
Maximum 12
Cronbach alpha 0.74
Difficulty indices
Mean 0.19
n of items (range 0.4 –0.6) –
n of items (range 0.2 –0.4) 5
n of items (range ,0.2) 8
Point biserial correlation coefficient
Downloaded by [Universite De Paris 1] at 00:47 14 September 2013

Mean 0.38
n of items (range 0.6 –0.8) –
n of items (range 0.5 –0.6) 1
n of items (range 0.4 –0.5) 7
n of items (range 0.3 –0.4) 4
n of items (range 0.2 –0.3) –
n of items (range ,0.2) 1

small mean of the total score of 2.42 out of 13 also shows the difficulty of the
AREPDiT. However, as seen from Table 4, most of the point biserial correlation coef-
ficients are above 0.30, which means that the items function satisfactorily (Crocker &
Algina, 1986). Only Q1 had a lower point biserial correlation coefficient than 0.20.
This might result from the nature of the first tier of Q1, which has two alternatives,
including one of the most common misconceptions. The number of pre-service tea-
chers who selected this misconception is higher than the ones who selected the
correct answer. Therefore, both the difficulty level and the point biserial correlation
coefficient of Q1 are lower than desirable values. However, despite Q1, the overall
acceptable values for reliability, item difficulties, and point biserial correlation coeffi-
cients substantiate the AREPDiT as a reliable instrument.
Both content and construct validities were investigated in this study. As rec-
ommended by Hestenes and Halloun (1995), the percentages of false positives and
false negatives were calculated to establish evidence of content validity. If test items
were clear and understandable, the respondents with higher knowledge levels about
the subject would answer the items correctly. In their study, Hestenes and Halloun
(1995) interpreted that minimization of the probability of false negatives and false posi-
tives provides greater validity in a multiple-choice test. In addition, they judged the
probability of false negatives to be less than 10%. In the present study, the percentages
of false positives and false negatives were estimated as 5.53% and 3.57%, respectively.
The determination of construct validity of a test frequently requires statistical analyses,
especially correlation and factor analysis, to investigate the internal structure of the test.
Cataloglu (2002) proposed correlation between high scores and confidence as an evi-
dence of construct validity. He stated that respondents with high scores were expected
 Three-Tier Test on Atmosphere-Related Problems 1679

to be more confident than were respondents with low scores. High correlation implies
that the items of the test functioned properly. In this study, the correlation between both
tiers scores and certainty scores was investigated and a statistically significant positive
correlation of 0.40 was found at p , 0.01, providing evidence of the construct validity
of the AREPDiT. Although an exploratory factor analysis on pre-service teachers’ total
score was conducted, reasonable factors were not obtained. This could be because the
items are loosely related to each other and do not measure either a single atmosphere-
related environmental concept or the three dimensions (nature, consequences, and sol-
utions) of GW, GE, OLD, and AR.

The Pre-Service Teachers’ Understanding of GW, GE, OLD, and AR

Downloaded by [Universite De Paris 1] at 00:47 14 September 2013

The percentages of the pre-service teachers’ responses are given in Table 5. The pre-
service teachers’ overall understandings of GW, GE, OLD, and AR are very low

Table 5. Percentages of the pre-service teachers’ responses

% Correct responses

AREPDiT Only first Both two All three % Lack of %

Content area item tiers tiers tiers knowledge Certainty

GW 1 47 24 16 44 48
2 35 30 16 33 52
5 46 25 15 40 49
6 27 26 19 41 52
Mean % GW 38.8 26.3 16.5 39.5 50.3
GE 3 75 34 20 40 45
4 48 39 26 34 53
Mean % GE 61.5 36.5 23.0 37.0 49.0
OLD 7 61 51 37 25 60
8 31 28 20 52 39
9 17 10 7 53 43
10 37 31 22 34 55
Mean % OLD 36.5 30.0 21.5 41.0 49.3
AR 11 43 24 14 51 37
12 34 24 17 54 38
13 40 22 13 57 33
Mean % AR 39.0 23.3 14.7 54.0 36.0
Mean 41.6 28.3 18.6 42.9 46.5
percentages

Notes: Only first tiers shows the percentages of the pre-service teachers’ correct responses based on
the first tier of each question. Both two tiers shows the percentages of the pre-service teachers’ correct
responses to both the first (content) and the second (reason) tiers. All three tiers shows the
percentages of pre-service teachers who selected the correct response for both first and second tiers
and also were certain about their answers. Lack of knowledge is the percentages of pre-service
teachers who were uncertain regardless of their responses. Certainty is the percentages of pre-service
teachers who were certain regardless of their response to the first or second tier.
 1680 H. O. Arslan et al.

(range 7 –37%). These results might be related to the complex and abstract nature of
these problems (Boyes & Stanisstreet, 1992; Boyes et al., 1995; Cordero, 2001; Das-
kolia et al., 2006; Francis et al., 1993). In contrast, their certainty scores were mod-
erate (mean percentages of 46.5). These findings are consistent with those obtained
by Pallier et al. (2002), and Renner and Renner (2001) who revealed that self-assess-
ment in the cognitive domain produced overconfidence.
The differences between the percentages of the correct answers from the first tier to
all three tiers are noticeable. These differences can be attributed to lack of knowledge,
lucky guess/lack of confidence, or misconceptions. Having the advantage of being a
three-tier test, the AREPDiT provides an opportunity for identifying the percentage
of each.
The overall lack of knowledge mean percentage of 42.9% indicates that nearly half
Downloaded by [Universite De Paris 1] at 00:47 14 September 2013

of the pre-service teachers do not have accurate conceptualizations of atmosphere-

related environmental issues. AR is identified as the least-known content area with
the highest mean percentage of lack of knowledge (54%). Moreover, the low percen-
tages of the total scores and certainty scores of the AR items indicate limited under-
standings about AR formation, its consequences, and solutions. In her study, Dove
said ‘although the media coverage would have familiarized them (students) with
terms such as AR, they probably lacked any real understanding of the concepts
involved’ (1996, p. 90).
When the percentages of the total scores of each item were considered, both the
most well-known and least-known questions pertain to OLD. Thirty-seven per cent
of the pre-service teachers gave a correct response to Q7, which is about the nature
of the ozone layer, and they were very certain of their answers (60%). However,
only 7% of them answered Q9 correctly about the consequences of OLD. The lack
of knowledge percentages of Q7 and Q9 (25% and 53%) supported these results.
The misconception percentages of the pre-service teachers’ are summarized in
Figure 3. As seen from the M-all tiers percentages in Figure 3, the pre-service teachers
hold six prevalent misconceptions with percentages equal to or greater than 10. These
misconceptions are:
. GW is caused by OLD (M1).
. GW will cause skin cancer (M2).
. AR is a result of GW (M3).
. GW can be reduced by setting limitations on chemical waste released into the
rivers (M7).
. The GE is a totally harmful phenomenon for mankind (M12).
. Using public transportation reduces OLD (M24).
The misconceptions identified in this study as M1, M2, M12, and M24 have been
reported as common ones by several previous studies (Boyes et al., 1995; Khalid,
2001, 2003; Michail et al., 2007; Summers et al., 2000). Linking GW with OLD
(M1) is one of the most prevalent misconceptions held by 43% or higher of the par-
ticipants in these studies. In the present study, 56% of the pre-service teachers
selected M1 in the first tiers of the related questions, which supports previous findings
 Three-Tier Test on Atmosphere-Related Problems 1681
Downloaded by [Universite De Paris 1] at 00:47 14 September 2013

Figure 3. Pre-service teachers’ percentages of misconceptions. Notes: The M-first tiers shows the
percentages of the pre-service teachers who selected wrong alternatives, which include
misconceptions in the first tier of each question in the AREPDiT. The M-both tiers shows the
percentages of the pre-service teachers who not only selected wrong alternatives which include
misconceptions in the first tier, but also selected the reason with misconception in the second tier
of each question. The M-all tiers shows the percentages of pre-service teachers who were also
certain about their responses for the first two tiers

in the studies carried out by Boyes et al. (1995), Khalid (2001, 2003), and Summers
et al. (2000). However, the percentage of this misconception decreased to 40% when
the responses to the both two tiers were considered. The difference between these
percentages means that 16% of the pre-service teachers could not select the related
misconception in the second tier. When all tiers were taken into account, the percen-
tage of M1 decreased to 18% (Figure 3). Thus, the question arises does a wrong
answer on a one-tier test truly identify a misconception? The results in this study indi-
cate that identifying misconceptions by one-tier or even two-tier tests overestimates
the percentages of misconceptions (Pesman & Eryilmaz, 2010).
Five of the six prevalent misconceptions held by the pre-service teachers are on the
nature and consequences of and solutions to GW. Michail et al. (2007) reported in a
study that only 15% of pre-service teachers are aware of the nature of GW and 70%
believe that no GE occurs in the absence of man. Similarly, in the study by Summers
et al. (2000), only 4 out of 12 primary teachers thought that GE is due solely to
man’s activities. In the present study, 10% of the pre-service teachers identified
GE to be a totally harmful phenomenon for mankind. Because the nature of GW
is not conceptualized properly, consequences and actions to reduce GW can be con-
fused with other atmosphere-related environmental problems. For instance, AR is
conceptualized as one of the outcomes of GW by 18% of the pre-service teachers
(M3), and 14% of them believed skin cancer to be a direct result of GW (M2). Con-
fusion over the relationship of GW with skin cancer is reported in previous studies
with higher percentages than in the current study (Boyes & Stanisstreet, 1992;
 1682 H. O. Arslan et al.

Groves & Pugh, 1999; Khalid, 2001, 2003). Additionally, 10% of the pre-service
teachers thought that setting limitations on chemical waste released into the rivers
would reduce GW.
Twelve per cent of the pre-service teachers suggested that using public transpor-
tation would reduce OLD (M24). This misconception might relate to their belief
about the destructive effect of carbon dioxide on the ozone layer (M17). Seven per
cent of the pre-service teachers had this misconception (M17). The results of this
study support many of the previous studies that have also identified similar miscon-
ceptions held by pre-service teachers. However, higher percentages were reported
in these studies (range 66–92%) (Boyes et al., 1995; Hillman, Stanisstreet, &
Boyes, 1996; Khalid, 2001). The difference between the results in the previous
studies and the present study could be attributed to the nature of three-tier tests
Downloaded by [Universite De Paris 1] at 00:47 14 September 2013

which differentiates lack of knowledge from misconceptions.

Seven misconceptions (M5, M6, M14, M15, M20, M23, and M31) as identified by
the open-ended survey and the review of the related literature were not held by the
pre-service teachers in this study (Figure 3). That is, although some of them selected
the related alternatives in the first tiers of the test, they could not find the reason for
these misconceptions and/ or were uncertain about their answers. For instance, the
M-first tier score percentages of M5 and M6 were higher than the threshold (10%);
however, they reduced to zero if all three tiers were considered.

Conclusion
The present study indicates that a three-tier diagnostic test, as a data collection
method for the assessment of misconceptions, proves to be a useful tool in investi-
gating pre-service teachers’ understandings of atmosphere-related environmental
problems. The AREPDiT is unique in that this instrument accurately differentiates
lack of knowledge from misconceptions on issues regarding GW, GE, OLD, and
AR. Additionally, the AREPDiT enables researchers to more accurately categorize
their findings as lack of knowledge, lucky guess/lack of confidence, and/or misconcep-
tions rather than merely labelling them as misconceptions as in previous studies.
Although both lack of knowledge and misconceptions are related to conceptual
understanding, the instructional methodologies for remedying such misconceptions
and overcoming lack of knowledge should be different from one another (Hasan
et al., 1999). Remediating misconceptions require specific instructional strategies
other than traditional teaching methods since misconceptions are stable and often
resistant to change (Fisher, 1985).
The use of assessment tools like the AREPDiT has some advantages as compared to
multiple-choice tests and two-tier tests in science education. First, the correlation
between both-tiers score and certainty score establishes evidence for construct validity
of the instrument. Second, researchers can estimate percentages of false negatives
and false positives without conducting follow-up interviews in order to provide evi-
dence for content validity. Finally, when compared to one-tier or two-tier tests,
three-tier diagnostic tests can estimate students’ achievement or misconceptions
 Three-Tier Test on Atmosphere-Related Problems 1683

more accurately by reducing the overestimation of these scores. As Pesman and Eryil-
maz (2010) stated, three-tier tests can be considered as a more valid and reliable
instrument for the assessment of achievement or misconception.
The AREPDiT proved to be a valid and reliable diagnostic instrument to explore
the knowledge and misconceptions of pre-service teachers. The findings of the
present study revealed that a majority of the pre-service teachers displayed limited
conceptual understandings of GW, GE, OLD, and AR concepts and held similar mis-
conceptions as identified in previous studies. This lack of fundamental background
knowledge will be a limiting factor in the future development of a highly qualified
teacher in the classroom. Appleton (1995) stated that if teachers have limited
science knowledge, they usually lack confidence in their ability to teach science. Con-
sequently, they will be more likely to teach science poorly by using traditional teaching
Downloaded by [Universite De Paris 1] at 00:47 14 September 2013

strategies or avoid teaching it altogether (Appleton & Kindt, 1999). Thus, using
assessment tools like the AREPDiT is necessary to assess pre-service science teachers’
content knowledge and identify possible misconceptions on atmosphere-related
environmental problems.
As Caleon and Subramaniam (2010) mentioned, three-tier diagnostic tests can
be used to monitor gradual changes in content knowledge, explanatory knowl-
edge, and strength of students’ understandings in science education. If science
teachers administer the AREPDiT as a pre- and post-test to their students, edu-
cators could gain a deeper insight into the concepts that require attention
for greater student understanding and evaluate the effectiveness of their
instruction.
Three-tier diagnostic tests include only one confidence rating for the subjects’
responses in the content and reason tiers; therefore, it is not clear whether subjects
have different levels of confidence for each tier. To overcome this possible limitation,
certainty tiers can be provided to both the content and reason tiers separately
(four-tier test) (Caleon & Subramaniam, 2010; Odom & Barrow, 2007). Caleon
and Subramaniam (2010) reported that half of their sample had different levels of
confidence for the content and reason tiers. The reason for this situation might
be that the subjects tend to regard the second-tier question as a distinct multiple-
choice task (Griffard & Wandersee, 2001). Another possible limitation to the
three-tier diagnostic test is that the scoring of three-tier tests is more difficult than
multiple-choice tests or two-tier diagnostic tests.
The AREPDiT may be used for further studies as a tool for assessing achieve-
ment and misconceptions of not only pre-service teachers from different programs,
but also in-service teachers and high school students. Additionally, the test can
be used for the investigation of what demographic features and backgrounds (if
any) influence individuals’ understandings of atmosphere-related environmental
problems. Because of the valuable contributions of three-tier diagnostic tests as
explained in this study, it is recommended that researchers develop and use
three-tier diagnostic instruments instead of a Likert scale test, multiple-choice
tests, or two-tier diagnostic tests for the assessment of students’ misconceptions
about science concepts.
 1684 H. O. Arslan et al.

Acknowledgement
The entire AREPDiT can be obtained from the corresponding author.

References
Appleton, K. (1995). Student teachers’ confidence to teach science: Is more science knowledge
necessary to improve self-confidence? International Journal of Science Education, 17,
357 – 369.
Appleton, K., & Kindt, I. (1999). Why teach primary science? Influences on beginning teachers’
practices. International Journal of Science Education, 21, 155 –168.
Boyes, E., Chambers, W., & Stanisstreet, M. (1995). Trainee primary teachers’ ideas about the
ozone layer. Environmental Education Research, 1(2), 133 –145.
Downloaded by [Universite De Paris 1] at 00:47 14 September 2013

Boyes, E., Chuckran, D., & Stanisstreet, M. (1993). How do high school students perceive global
climatic change: What are its manifestations? What are its origins? What corrective action
can be taken? Journal of Science Education and Technology, 2(4), 541– 557
Boyes, E., & Stanisstreet, M. (1992). Students’ perceptions of global warming. International Journal
of Environmental Studies, 42, 287 –300.
Boyes, E., & Stanisstreet, M. (1993). The ‘Greenhouse Effect’: Children’s perceptions of causes,
consequences and cures. International Journal of Science Education, 15, 531 –552.
Caleon, I., & Subramaniam, R. (2010). Development and application of a three-tier diagnostic test
to assess secondary students’ understanding of waves. International Journal of Science Education,
32(7), 939–961.
Cataloglu, E. (2002). Development and validation of an achievement test in introductory quantum mech-
anics: The Quantum Mechanics Visualization Instrument (QMVI). Retrieved March 10, 2010,
from http://etda.libraries.psu.edu/theses/approved/WorldWideFiles/ETD-145/thesis.pdf
Chu, H.E., Treagust, D.F., & Chandrasegaran, A.L. (2009). A stratified study of students’ under-
standing of basic optics concepts in different contexts using two-tier multiple-choice items.
Research in Science and Technological Education, 27(3), 253– 265.
Clement, J., Brown, D.E., & Zietsman, A. (1989). Not all preconceptions are misconceptions:
Finding ‘anchoring conceptions’ for grounding instruction on students’ intuition. International
Journal of Science Education, 11, 554 –565.
Clough, E.E., & Driver, R. (1985). Secondary students’ conceptions of the conduct of heat: Bring-
ing together scientific and personal views. Physics Education, 20, 176 –182.
Cordero, E. (2001). Misconceptions in Australian students’ understanding of ozone depletion.
Melbourne Studies in Education, 41, 85–97.
Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory. Orlando, FL: Holt,
Rinehart and Winston.
Daskolia, M., Flogaitis, E., & Papageorgiou, E. (2006). Kindergarten teachers’ conceptual frame-
work on the ozone layer depletion. Exploring the associative meanings of a global environ-
mental issue. Journal of Science Education and Technology, 15(2), 168 –177.
Dove, J. (1996). Student teacher understanding of the greenhouse effect, ozone layer depletion and
acid rain. Environmental Education Research, 2(1), 89– 100.
Driver, R. (1988). Changing conceptions. Tijdschrift voor Didactie der. ß-Wetenschappen, 6(3),
161– 198.
Eryilmaz, A., & Surmeli, E. (2002). Üç aşamalı sorularla öğrencilerin ısı ve sıcaklık konularındaki
kavram yanılgılarının ölçülmesi [Assessment of students’ misconceptions about heat and temp-
erature by means of three-tier questions]. Retrieved April 5, 2004, from http://www.fedu.metu.
edu.tr/ufbmek-5/b_kitabi/PDF/Fizik/Bildiri/t110d.pdf
Fisher, K.M. (1985). A misconception in biology: Amino acids and translation. Journal of Research
in Science Teaching, 22, 53–62.
 Three-Tier Test on Atmosphere-Related Problems 1685

Francis, C., Boyes, E., Qualter, A., & Stanisstreet, M. (1993). Ideas of elementary students about
reducing the ‘Greenhouse Effect’. Science Education, 77, 375 –392.
Griffard, P.B., & Wandersee, J.H. (2001). The two-tier instrument on photosynthesis: What does it
diagnose? International Journal of Science Education, 23, 1039–1052.
Groves, F.H., & Pugh, A.F. (1999). Elementary pre-service teacher perceptions of the greenhouse
effect. Journal of Science Education and Technology, 8(1), 75–81.
Groves, F.H., & Pugh, A.F. (2002). Cognitive illusions as hindrances to learning complex environ-
mental issues. Journal of Science Education and Technology, 11(4), 381 –390.
Hammer, D. (1996). More than misconceptions: Multiple perspectives on student knowledge and
reasoning, and an appropriate role for education research. American Journal of Physics, 64,
1316–1325.
Hasan, S., Bagayoko, D., & Kelley, E.L. (1999). Misconceptions and the certainty of response index
(CRI). Physics Education, 34(5), 294– 299.
Hashweh, M.Z. (1987). Effects of subject matter knowledge in the teaching of biology and physics.
Downloaded by [Universite De Paris 1] at 00:47 14 September 2013

Teaching and Teacher Education, 3, 109– 120.

Hestenes, D., & Halloun, I. (1995). Interpreting the force concept inventory. Physics Teacher, 33,
502– 506.
Hillman, H., Stanisstreet, M., & Boyes, E. (1996). Enhancing understanding in student teachers:
The case of auto-pollution. Journal of Education for Teaching, 22, 311 –325.
Kane, M.T. (1986). The role of reliability in criterion-referenced tests. Journal of Educational
Measurement, 23(3), 221 –224.
Khalid, T. (2001). Pre-service teachers’ misconceptions regarding three environmental issues.
Canadian Journal of Environmental Education, 6, 102 –120.
Khalid, T. (2003). Pre-service high school teachers’ perceptions of three environmental phenom-
ena. Environmental Education Research, 9(1), 35–50.
Michail, S., Stamou, A.G., & Stamou, G.P. (2007). Greek primary school teachers understanding of
current environmental issues: An exploration of their environmental knowledge and images of
nature. Science Education, 91(2), 244 –259.
Odom, A.L., & Barrow, L.H. (1995). Development and application of a two-tier diagnostic test
measuring college biology students’ understanding of diffusion and osmosis after a course of
instruction. Journal of Research in Science Teaching, 32, 45–61.
Odom, A.L., & Barrow, L.H. (2007). High school biology students’ knowledge and certainty about
diffusion and osmosis concepts. School Science and Mathematics, 107, 94–101.
Osborne, R.J., Bell, B.F., & Gilbert, J.K. (1983). Science teaching and children’s view of the world.
European Journal of Science Education, 5, 1– 14.
Pallier, G., Wilkinson, R., Danthiir, V., Kleitman, S., Knezevic, G., Stankov, L., & Roberts, R.D.
(2002). The role of individual differences in the accuracy of confidence judgments. The
Journal of General Psychology, 129(3), 257 –299.
Papadimitriou, V. (2004). Prospective primary teachers’ understanding of climate change, greenhouse
effect, and ozone layer depletion. Journal of Science Education and Technology, 13(2), 299–307.
Pekel, O.F., & Ozay, E. (2005). Turkish high school students’ perceptions of ozone layer depletion.
Applied Environmental Education and Communication, 4, 115–123.
Pesman, H., & Eryilmaz, A. (2010). Development of a three-tier test to assess misconceptions about
simple electric circuits. The Journal of Educational Research, 103, 208–222.
Posner, G.J., Strike, K.A., Hewson, P.W., & Gertzog, W.A. (1982). Accommodation of a scientific
conception: Toward a theory of conceptual change. Science Education, 66, 211 –227.
Renner, C.H., & Renner, M.J. (2001). But I thought I knew that: Using confidence estimation as a
debiasing technique to improve classroom performance. Applied Cognitive Psychology, 15, 23–32.
Seymour, R. (2008). Understanding the global warming discussion: Climate change as a context for devel-
oping standards-based research skills in secondary school students (Unpublished doctoral disser-
tation). Dominican University of California, California.
 1686 H. O. Arslan et al.

Summers, M., Kruger, C., Childs, A., & Mant, G. (2000). Primary school teachers’ understanding
of environmental issues: An interview study. Environmental Education Research, 6(4), 293– 312.
Treagust, D.F. (1988). Development and use of diagnostic tests to evaluate students’ misconcep-
tions in science. International Journal of Science Education, 10, 159 –170.
Tsui, C.Y., & Treagust, D. (2010). Evaluating secondary students’ scientific reasoning in genetic
using a two-tier diagnostic instrument. International Journal of Science Education, 32,
1073–1098.
Wang, J.R. (2004). Development and validation of a two-tier instrument to examine understanding
of internal transport in plants and the human circulatory system. International Journal of Science
and Mathematics Education, 2, 131– 157.
West, L., & Pines, A. (1985). Cognitive structure and conceptual change. Orlando, FL: Academic Press.
Downloaded by [Universite De Paris 1] at 00:47 14 September 2013