199

Research In Science Education, 1986, 16, 199-207.

EVALUATING STUDENTS' MISCONCEPTIONS BY MEANS

OF DIAGNOSTIC MULTIPLE CHOICE ITEMS

David Treagust

INTRODUCTION

In most misconceptions research, the usual means for obtaining information about
students' misconceptions has been through individual student interviews and/or open-ended
response questions on specific science topics. Osborne and Gilbert (1980) and Watts (1981)
have described a variety of interview formats or procedures for conducting these
interviews. The two most commonly used procedures by researchers are interviews -
about - instances and interviews - about - events. Some researchers such as Mitchell and
Gunstone (1984) have used an interview format which is a mixture of these procedures.
Exceptions to this line of research have been multiple choice tests described by
several authors such as Tamir (1971), [,inke and Venz (1978, 1979), Helm (1980), Trembath
(1984) and more recently by Halloun and Hestenes (1985). The multiple choice tests by
Helm (1980) covered a semester's work in college physics while that of Trembath (1984)
covered a wide range of subject content areas in a variety of disciplines at the college
level. Only in the case of Tamir (1971) and Linke and Venz (1978, 1979) and Halloun and
Hestenes (1985) were the tests related to any specified and limited content structure.
The work by Tamir (1971) on an alternative apporach to construction of multiple
choice test items was innovative in that the distractors for the multiple choice items were
based on students' answers to essay questions and other open-ended questions and
addressed underlying conceptual knowledge related to a limited content area. As Tamir
(1971, p.306) states "these alternative Eresponses] being representative of typical
conceptions and misconceptions of students have a distinctive advantage as compared to
regular test items for which professional test writers provide the alternatives."
In their multiple choice items, Linke and Venz (1978) examined the principles of
structure of matter, changes of state and solubility and the basic principles of
electricity. In their later study Linke and Venz (1979) examined students' misconceptions
of the radiation of light and heat, chemical reactions and motion forces. The items, which
 200

contained distractors based on students' identified misconceptions from parallel free

response items, were administered to non-science students in Years 11 and 12; the general
conclusion was that students completing secondary studies h o l d identifiable
misconceptions or confused ideas.
In order to objectively evaluate first year college physics instruction, Halloun and
Hestenes (1985) developed a physics diagnostic test which assessed students' basic
knowledge of mechanics. The items were initially selected to assess the students'
qualitative conceptions of motion and its causes and to identify common misconceptions
already noted by previous investigators. I t e m distractors were chosen from students'
responses identified as misconceptions on earlier tests.
The development of multiple choice tests on students' misconceptions has the
potential to make a valuable contribution to the body of work in the area of
misconceptions, and to assist in the process of helping science teachers to more readily
use the findings of research in their classrooms. It is well documented that research
findings in science education take considerable time to be applied in the classroom.
Further, if teachers decide to interview their own students to identify misconceptions, this
practice itself is fraught with problems since not only is interviewing time consuming, it
also requires substantial training.
One way for a science teacher to more easily identify misconceptions held by a group
of students would be to administer a pencil and paper multiple choice test which has items
specifically designed to identify misconceptions and misunderstandings that arise from
earlier teaching and learning prior to commencing the topic or that have occurred
following the teaching of the topic. It is, however, well documented that the task of
changing misconceptions w i l l not be e a s y since misconceptions h a v e often been
incorporated securely into cognitive structure (see for example, Ausubel, 1978; Driver,
1981; Gunstone,champagne and Klopfer, 1981). Nevertheless, a teacher needs a starting
place for addressing known misconceptions and a reliable and valid multiple choice
diagnostic test incorporating students' reasoning in selecting responses would appear to
provide a relatively straight forward method.

PURPOSE

The purpose of this paper is to describe a methodology for developing multiple

choice, two-tier diagnostic tests to identify and evaluate students' scientific
misconceptions in specific content areas. Items from two tests will be discussed, one on
covalent bonding and structure, an upper school chemistry topic, and a second on
photosynthesis and respiration in plants, a biology topic taught in both lower and upper
high school.
 201

CONSTRUCTION OF DIAGNOSTIC TESTS

The methodology for diagnosing students' misconceptions in specific content areas is

derived from the current literature dealing with cognitive structure and conceptual
change, and comprises ten stages involving three broad areas. The stages of the
methodology are briefly outlined below; further details are described by Treagust (1985).
I) The identification of propositional knowledge and the development of a concept map
to define a specific content area in science.
Propositional knowledge s t a t e m e n t s pertinent to the topic under investigation
are identified (see for example, Hackling and Treagust, 1984).
A map of concepts related to the topic under investigation is developed (see
for example, Novak, 1980).
The propositional knowledge statements are related directly to the concept
map in order to confirm the reliability of the underlying concepts and
proposi tional sta tem ents.
The propositional knowledge statements and concept map are content
validated by subject matter experts and educators.
2) Student misconceptions are identified by examining the relevant literature, by
student interviews and by student responses to open ended paper and pencil
questions.
The literature is thoroughly examined for related research on misconceptions
in specific science content areas.
Unstructured interviews are conducted with a number of students who have
recently completed a given topic, in order to identify any misunderstandings
and misconceptions.
3) The development of multiple choice two tier items, of which the first tier requires a
content response and the second tier requires a reason for the response.
Multiple choice items are written based on one or a limited number of
propositional statements; students are asked to provide the reason for
choosing the particular option of the multiple choice item.
A second-tier of questions is developed for each item based on students'
reasons for their choice as well as on information gathered from the
interviews. The choice may be the correct answer, a wrong answer or an
identified misconception. This technique of two tier items for instrument
development has been used by Tobin and Capie (1981).
A specification grid is constructed to ensure that the diagnostic test addresses
the propositional knowledge s t a t e m e n t s underlying the topic.
 202

Successive r e f i n e m e n ts of the items in the t w o - t i e r multiple choice tests

ensure that the tests can be used for diagnosing student misconceptions in the
topic under examination.
Based on this methodology, two diagnostic tests from chemistry and biology have
been developed by Peterson (1986) and Haslam (1986). The instruments, Covalent Bonding
and Structure Test and What do you know about photosynthesis and respiration in plants?,
are two tier item multiple choice tests containing 13 and 15 items r esp ect i v el y . Aspects
of the test c h a r a c t e r i s t i c s and details of their use are to be found in Peterson, Treagust
and Garn et t (1986) and in Treagust and Haslam (1986). Two items, one for each test, are
presented below to illustrate how the items work and how the data can be used by
classroom science teachers.

USING MULTIPLE CHOICE TESTS TO DIAGNOSE MISCONCEPTIONS

The seventh item of the Covalent Bonding and Structure Test (Box l) assesses
students' understanding of intermolecular forces and is based on three propositions,
nam ely:
Intermolecular forces, or weak forces of attraction, exist in varying degrees
between the molecules.
Whether a substance composed of molecules exists as a solid, liquid or gas at
room temperature will depend on the magnitude of the intermolecular forces
between molecules.
Molecular substances with high melting and boiling points have strong
intermolecular forces between molecules.
The data are from responses of four groups of students. Groups [ and 2 are
composite Year II and 12 chemistry students respectively in South Australia. Groups 3
and 4 are first year tertiary chemistry students from two institutions in Western
Australia. The majority of students in all groups (87%, 97%, 100% and 89%) responded
correctly that intermoleeular forces account for the differences in state of hydrogen
sulphide and water at room temperature. Almostall these chemistry students were aware
of the relationship which exists between the strength of the intermolecular forces and the
melting point/boiling point of a substance. However, only II%, 33%, 19% and 18%
respectively correctly understand why this relationship exists. A large percentage of
students (63%, 40%, 73% and 64%) selected choices IA or IB which relate to the strength
of intermolecular bonds. These students appear to be making a consistent error and the
misconception is easily identified by the teacher using this item. Indeed, the major
misconception being addressed, which was earlier identified by interview and in the free
 203

response format, related to the cornmon misunderstanding that "intermolecular forces are
the forces within a molecule". A second misconception addressed in choice B of item 7
was that "covalent bonds are broken when a substance changes state". This misconception,
selected by 49%, 17%, 30% and 24% of the students respectively, indicates these students
have not understood the nature of intermolecular forces in a molecular solid and also the
strength of intermolecular forces compared with the forces within a molecule. The
misconception is similar to that reported by Osborne and Cosgrove (1983) where students
considered that the bonds in water molecules were broken when water changed state.
The second item of What do you know about photosynthesis and respiration in plants?
test (see Figure 2) investigates students' comprehension Of the gas being taken in by plants
when there is no light energy. Specifically this item addresses the following four
propositional statements:
All organisms, plants and ani mals, respire continually.
Oxygen is taken induring respiration.
Photosynthesis takes place only in the presence of light energy.
Oxygen gas is given off by the green leaves (or green stems) during the
process of photosynthesis.
The first two propositional statements are addressed directly by item 2 while the
second two statements are addressed implicitly.
The data are from responses of students in Year 8-10 in one school and from students
in Years II and 12 in three schools in comparative socioeconomic areas. The results
indicate that this sample of students does not have a clear understanding o[" the process of
respiration. When the first part of the item only is taken into consideration (end column),
the results show that there is generally an increasingly correct response choice from Years
8-12 (39%, 44%, 41%, 65%, 88%) about which gas is taken in in large amounts by green
plants when there is no light energy at a l l . Taking both parts of the item into
consideration, the correct response is much lower (9%, 11%, 7%, 28%, 65%). While there
is generally less selection of alternative reasons at each successively higher year level, it
is not until Year 12 that responses indicate that this sample of students have a reasonably
clear understanding of respiration in plants and even then it is less than acceptable. These
results are perhaps surprising when one realises that respiration is taught as an identified
topic in Year 8 and is addressed throughout biology units in later years of secondary
school. Nevertheless, the results illustrate students' misconceptions and lack of
understanding that respiration in plants is an ongoing continuous process during both light
and dark conditions.
 204

BOX 1

Item Number 7 from Covalent Bonding and Structure diagnostic test showing percentage
of students selecting a l t e r n a t i v e responses.

Water (H2O) and hydrogen sulfide (H2S) have sireilar chemical form ulae and have V-shaped
structures. At room temperature, water is a liquid and hydrogen sulphide a gas. The
difference in state between water and hydrogen sulphide is due to the presence of strong
intermolecular forces between
(I) H20 molecules (2) H2S molecules
Reason
( - ~ The difference in strength of the intermoleeular forces is due to differences in the
strength of the O-H and S-H covalent bonds.
(B) The bonds in H2S are easily broken, whereas in 1-120 they are not.
(C) The difference in strength of the intermoleeular forces is due to the differenee in
polarity of the molecules.
(D) The difference in strength of the intermoleeular forees is due to the fact that H20 is
a polar molecule, and H2S is a non-polar moleeule.

Class Year N Choice Reason

A B C D Total

I II 159 I 13.8 49.1 10.7" 13.8 87.4

2 4.4 4.4 2.5 1.3 12.6
2 12 84 I 22.9 16.9 32.5* 25.3 97.6
2 0.0 0.0 2.4 0.0 2.4
3 13 37 I 43.2 29.7 18.9" 8.2 i00.0
2 0.0 0.0 0.0 0.0 0.0
4 13 37 1 40.0 24.4 17.8" 6.7 88.9
2 2.2 0.0 6.7 2.2 ii.i

* the correct choice and reason response

 205

BOX 2

Item Number 2 from the What do you know about p h o t o s y n t h e t i c and respiration in plants?
diagnostic t e s t showing p e r c e n t a g e of s t u d e n t s s e l e c t i n g a l t e r n a t e responses.

Which gas is takn in by green plants in large a m o u n t s when there is no light energy a t all?
I) carbon dioxide gas
2) oxygen gas

The reason for my answer is because:

a) This gas is used in photosynthesis which occurs in green plants all t h e time.
b) This gas is used in photosynthesis which occurs in green plants when t h e r e is
no light energy at all.
c) This gas is used in respiration which only occurs in green plants when t h e r e is
no light energy to photosynthesise.
d) This bas is used in respiration which takes place continuously in green plants.
e)

Year N Choice Reason Total

Ne
A B C D E Re~son

8 137 I 14.6 16.8 10.9 16.1 - 2.9 61.3

2 7.3 10.2 11.7 8.8* - 0.7 38.7

9 88 1 ii.4 5.7 20.5 17.0 - I.I 55.7

2 3.4 18.2 11.4 II.4" - - 44.3

10 99 i 8.1 II.I II.2 24.2 4.0 - 58.6

2 3.0 8.1 23.2 7.1" - - 41.4

ii 68 1 7.4 1.5 8.8 17.6 - - 35.3

2 1.5 1.5 33.8 27.9* - 64.7

12 49 1 - - 6.1 6.1 - - 12.2

2 - - 14.3 65.3* 4.1 4.1 87.8

* the c o r r e c t choice and reason response

- no responses in this c a t e g o r y
 206

IMPLICATIONS FOR TEACHING SCIENCE

While reported research on students' misconceptions are inherently interesting and

are illustrative of students' thinking and perhaps the failings of the implemented curricula,
the results are not easily used by classroom practitioners. One possible means of
increasing the application of science education research into the classroom is by utilizing
the findings of this research in diagnostic testing. The existence of reliable and valid
pencil and paper, easy-to-score, instruments will enable science teachers to better assess
students' understanding of science upon which improved teaching can be based.
By using a diagnostic test at the beginning of a specified science topic, a science
teacher can obtain clear ideas about the nature of the students' knowledge and
misconceptions of the topic. Once misconceptions are more easily identified, a science
leacher will be more inclined to remedy the problem by de~eloping and/or utilising
alternative teaching approaches which address students' misconceptions.

REFERENCES

AUSUBEL, D. (1968). Educational psychology: A cognitive view. Boston: Holt, Rinehart &
Winston, inc.

DRIVER, R. (1981). Pupils' alternative frameworks in science. European Journal of

Science Education, 3__,93-101.

GUNSTONE, R.F., CHAMPAGNE, A.B. & KLOPFER, L.E. (1981). Instruction for
understanding: A case study. The Australian Science Teachers Journal, 27___(3), 27-32.

HACKLING, M.W. & TREAGUST, D.F. (1984). Research data necessary for meaningful
review of grade ten high school genetics curricula. Journal of Research in Science
Teaching, 21___,197-209.

HALLOUN, I.A. & HESTENES, D. (1985). The initial knowledge state of college physics
students. American Journal of Physics, 53_(11), 1043-1055.

HASLAM, F. (1986). Secondary students' understanding of photosynthesis and respiration

in plants. Unpublished manuscript. Science and Mathematics Education Centre, Western
Australian Institute of Technology, Perth W.A.

HELM, H. (1978). Misconceptions about physics concepts among South African pupils
studying physical science. South African Journal of Science, 74.__,285-290.

LINKE, R.D. & VENZ, M.I. (1978). Misconceptions in physical science among non-science
background students. Research in Science Education, 8__,183-193.

LINKE, R.D. & VENZ, M.I. (1979). Misconceptions in physical science among non-science
background students. II. Research in Science Education, 9__,I03-I09.

MITCHELL, I.J. & GUNSTONE, G.F. (1984). Some student conceptions brought to the
study of stoichiometry. Research in Science Education, 14___,78-88.
 207

NOVAK, J.D. (1980). Teacher handbook for the learning how to learn Program. Cornell
University, Ithaca, New York.

OSBORNE, R.J. & COSGROVE, M.M. (1983). Children'sconceptions of the changes of

stale Qi" water. Journal of Research in Science Teaching, 20___,825-838.

OSBORNE, R.J. & GILBERT, J.K. (1980). A technique for exploring the students' view of
the world. Physics Education, 50___(65), 376-379.

PETERSON, R.F. (1986). The development, validation and application of a diagnostic test
instrument measuring Year i i and Year 12 students' understanding of covalent bonding and
structure. UnpublishedMasters thesis, Western Australian Institute of Technology, Perth,
Western Australia.

PETERSON, R.F., TREAGUST, D.F. & GARNETT, P. (1986). Identification of secondary

students' misconceptions of covalent bonding and structure concepts using a diagnostic
test instrument. Research in Science Education, 16_._,40-48.

TAMIR, P. (1971). An alternative approach to the construction of multiple choice test

items. Journal of BiologicalEducation, 5__,305-307.

TOBIN, K.G. & CAPIE, W. (1981). The development and validation of a group test of
logical thinking. Educational and PsychologicalMeasurement, if___,413-423.

TREAGUST, D.F. (1985). Diagnostic tests to evaluate students' misconceptions in

science. Paper presented at the annual meeting of the National Association for Research
in Science Teaching, French Lick Springs, Indiana.

TREMBATH, R . J . (1984). D e t e c t i n g and classifying the origins of science

misconceptions. In Bethal, C.J. (Ed). Research and Curriculum Development in Science
Education. 4: Curriculum Evaluation, Classroom Methodologyand Theoretical Models. The
University of Texas Centenial Science Education Centre monograph.

WATTS, M. (1981). Exploring pupils' alternative frameworks using the interview-about-

instances method. Proceedings of the international workshop on Problems concerning
students representation of physics and chemistry knowledged, Ludwigsburg, West
Germany, 36-386.