Writing MCQs

Susan Chamberlain
Dept of OBS/GYN
Queens University
Writing MCQs

At the end of this session, the participant will
be able to
Identify different types of MCQ items
Describe the structure of an MCQ
List the technical flaws common to MCQs
write one well-structured MCQ testing
appropriate content
Writing MCQs

At the end of this session, the participant will
be able to
Identify different types of MCQ items
Describe the structure of an MCQ
List the technical flaws common to MCQs
write one well-structured MCQ testing
appropriate content
All of the above
MCQ Check-list

Why we write them
What type is best
What flaws are to be avoided
What content to test
How to construct it
Why do we test?

Communicate to the students what material
is important (formative vs summative)
Motivate students to study
Identify areas of deficiency in need of
remediation or further learning
Determine final grades or make promotion
decisions
Identify areas where the course/curriculum
is weak
What Should Be Tested?

Exam content should match course
objectives
Important topics should be weighted more
heavily than less important topics
The testing time devoted to each topic
should reflect the relative importance of the
topics
The sample of items should be
representative of the instructional goals
What Should Be Tested?

Sampling
Performance on the sample provides a basis
for estimating achievement in the broader
domain that is actually of interest.
Questions sample:
TOPICS
SKILLS ie determining diagnosis, next step in
management, etc.
MCQ Items

Should test important information either
common or life-threatening
Should be well-structured
Flaws benefit test-wise examinees
Irrelevant difficulty does not test examinees
specific knowledge

Anatomy of an MCQ Item

Stem
Lead-in
Options
One correct answer
Distractors
Types of MCQs

True/False family
Assess recall of isolated facts
Stems must be clear and unambiguous
Options must be absolutely true or false or
examinee must decide how true options are

Generally not recommended usually have
subtle flaws
Types of MCQs: True/False

Which of the following statements is true regarding
contraception?
A) oral contraceptives should not be initiated until six weeks
after therapeutic abortion to avoid the risk of
hypercoagulability
B) an intrauterine device can provide good contraceptive
protection 5 years after insertion
C) oral contraceptives should not be prescribed to individuals
under the age of 15 without parental consent
D) lesbian women do not require counseling about contraception
or sexually transmitted diseases
E) oral contraceptives should not be prescribed for women over
the age of 35
Types of MCQs

One-Best-Answer family (A type)
Options may be diagrammed:


D C A E B


Correct answer is the most likely correct
Better application of knowledge, integration,
synthesis and judgment
Least correct Most correct
Types of MCQs:
One-Best-Answer family
A 24 year old university student came to see you because she is
missing several days of school each month because of severe
dysmenorrhea unresponsive to the OCP and NSAIDS. She also
complains of deep dyspareunia and has a tender uterosacral
nodule on the pelvic examination. Which one of the following
conditions is most likely responsible for her symptoms?
A) Chronic pelvic inflammatory disease
B) Endometriosis
C) Adenomyosis
D) Fibroids

MCQ Check-list

Why we write them
What type is best
What flaws are to be avoided
What content to test
How to construct it
MCQ Technical Flaws

Issues Related to Test-Wiseness
Issues Related to Irrelevant Difficulty
MCQ Technical Flaws

Issues Related to Test-Wiseness
provide clues to the right answer
examinee can eliminate options based on flaws
and not on knowledge

Issues Related to Irrelevant Difficulty

Item Flaws: Test-wiseness

Grammatical cues
Logical cues
Absolute terms
Long correct answer
Word repeats
Convergence strategy
Item Flaws: Test-wiseness

Grammatical cues
One or more distractors dont follow
grammatically from the stem
Item Flaws: Test-wiseness

Grammatical cues
A 24 year old patient presents with a one year
history of dysmenorrhea, mid-cycle pain,
dyspareunia, and painful bowel movements with
her periods. The most likely diagnosis is:
A) Fibroid uterus
B) Adenomyosis
C) Endometriosis
D) Psychosomatic condition
Item Flaws: Test-wiseness

Logical cues
A subset of the options are collectively
exhaustive
Item Flaws: Test-wiseness

Logical cues
Arterial blood gases in a normal pregnant
woman, compared to her non-pregnant state may
show which of the following?
A) Decreased pO
2
B) Increased pCO
2
C) Decreased pCO
2

D) Decreased pH

E) No difference in pCO
2
Item Flaws: Test-wiseness

Logical cues
One or more distractors dont follow
logically from the stem
Item Flaws: Test-wiseness

Logical cues
An important tool for assessing fetal well-being is
the Non Stress test. A well baby demonstrates
the following fetal heart rate pattern:
A) A baseline heart rate above 120 bpm
B) Evidence of fetal movement
C) Baseline variability
D) Two fetal heart rate accelerations in 20 minutes
E) Fetal breathing movements
Item Flaws: Test-wiseness

Absolute terms
Always, never
Not a problem if stem is long and lead-in
asks a precise question

Item Flaws: Test-wiseness

Absolute terms
In patients with a previous Cesarean section:
A) Vaginal birth is NEVER an option
B) Repeat Cesarean section is less dangerous to the
mother
C) The uterine scar may rupture during labour
D) Care should ALWAYS be supervised by an
obstetrician
E) No further pregnancies should EVER be considered
Item Flaws: Test-wiseness

Long correct answer
Correct answer is longer, more specific, or
more complete than other options

Item Flaws: Test-wiseness

Long correct answer
Which of the following problems in vaginal
breech delivery is most likely to cause serious
long-term complications?
A) Failure to descend
B) Cord prolapse
C) Nuchal arms
D) Entrapment of the after coming head with an
incompletely dilated cervix
E) 1 minute Apgar score of 4
Item Flaws: Test-wiseness

Word repeats
A word or phase is included in the stem and
in the correct answer

Item Flaws: Test-wiseness

Word repeats
A 24 year old with a previous history of chronic
PID presents with primary infertility. Select the
procedure that would be most appropriate to
evaluate the status of her tubes and pelvic
anatomy
A) Post-coital test
B) Endometrial biopsy
C) Laparoscopy with dye insufflation of the fallopian
tubes
D) Hysterosalpingogram
Item Flaws: Test-wiseness

Convergence strategy
The correct answer includes the most
elements in common with the other options

Item Flaws: Test-wiseness

Convergence strategy
Normal bladder neck mobility is measured by:
A) A Q-tip angle measurement under 40 degrees
B) A Q-tip angle measurement under 30 degrees
C) A bladder neck ultrasound angle less than 30
degrees
D) A Q-tip angle measurement under 20 degrees
MCQ Technical Flaws

Issues Related to Test-Wiseness

Issues Related to Irrelevant Difficulty
do not assess knowledge of intended topic
Irrelevant Difficulty

Options are long, complicated or double
Extraneous details in stem
Numeric data not stated consistently
Vague frequency terms
Non-parallel language in options
Stems are tricky or unnecessarily complicated
None of the above
Irrelevant Difficulty

Options are long, complicated or double
Extraneous details in stem
Shifts difficulty from content knowledge to
speed reading
Irrelevant Difficulty

Options are long, complicated or double
Mrs. Brown is a 70 year old woman who had a stroke 6 months
ago and is still impaired. Her daughter tells you that her
mothers recent urinary incontinence makes her care
unmanageable. She is seeking permanent placement. Mrs.
Brown complains of urinary frequency, nocturia, and urge
incontinence. Which of the following statements is true?

A) Neurological conditions are involved in the majority of women with
symptoms like those presented by Mrs. Brown
B) Diabetes has a greater impact on quality of life than Mrs. Browns
condition.
C) The pharmacological blockage of muscarinic receptors in the detrusor
muscle controls symptoms compatible with Mrs. Browns condition
D) Overactive bladder is defined as urinary incontinence accompanied by
or immediately preceded by urgency
E) Alpha stimulants are effective in treating this condition
Irrelevant Difficulty

Extraneous details in stem
You see in your office a 24 year old woman for her routine postpartum
visit, at 6 weeks. Her baby is irritable and cries all the time (including
during her visit.) As she is changing the diaper during her visit, you
notice that the child has a very red bum but upon questioning, she tells
you she did not notice anything abnormal. Her partner is away in
Afghanistan and she does not have a lot of friends, as they just recently
were posted in Kingston. She states that she has a hard time getting
dressed in the morning, cries easily and feels like she does not love her
child. She denies any medical or psychiatric history. Her exam is
essentially normal. She weighs 2 kg less than at her first antenatal visit
at 10 weeks. You most likely clinical diagnosis is:
A) Postpartum hypothyroidism
B) Hyperthyroidism
C) Postpartum blues
D) Postpartum depression
E) Psychotic depression
Irrelevant Difficulty

Numeric Data
Not stated consistently
Mixed format
Illogical order


Irrelevant Difficulty

Numeric Data not stated consistently
Following a second episode of salpingitis, what
is the likelihood that a woman is infertile?
A) Less than 20%
B) 20 30 %
C) Greater than 50 %
D) 90%
E) 75%
Irrelevant Difficulty

Vague frequency terms
Often, usually, rarely, frequently
Not consistently defined

*may be used in stem if otherwise carefully
constructed unambiguous question

Irrelevant Difficulty

Vague frequency terms
Gonorrhea is
A) sometimes asymptomatic
B) treated with cefixime 450 mg orally
C) caused by a gram positive intracellular diplococcus
D) diagnosed by a DNA test
E) usually associated with vulvar ulcers
Irrelevant Difficulty

Non-parallel language in options
Non-logical order
Irrelevant Difficulty

Non-parallel language in options
In a vaccine trial, 200 2-year-old boys were given a vaccine
against a certain disease and then monitored for five
years for occurrence of the disease. Of this group, 85%
never contracted the disease. Which of the following
statements concerning these results is correct?
A) No conclusion can be drawn, since no follow-up was
made of non-vaccinated children
B) The number of cases (ie 30 cases over five year) is
too small for statistically meaningful conclusions
C) No conclusions can be drawn because the trial
involved only boys
D) Vaccine efficacy (%) is calculated as 85-15/100
Irrelevant Difficulty

None of the above
Problematic where judgment is involved or
if options not absolutely true or false
Turns question into true/false question
Confuses very knowledgeable students
Irrelevant Difficulty

None of the above
The main reason that fertility is reduced in a
woman in her early 40s is:
A) Irregular cycles make it hard to time intercourse
B) Chromosomal damage adversely affects oocytes
genetic complement
C) Male erectile difficulties increase
D) Hormone imbalance leads to poor endometrial
development
E) None of the above
Irrelevant Difficulty

Stems are tricky or unnecessarily complicated
Arrange the parents of the following children with Downs syndrome
in order of the highest to lowest risk of recurrence. Assume that the
maternal age in all cases is 22 years and that a subsequent pregnancy
occurs within 5 years. The karyotypes of the daughters are:
I: 46, XX, 14,+T (14q21q) pat
II: 46, XX, 14,+T (14q21q) de novo
III: 46, XX, 14,+T (14q21q) mat
IV: 46, XX, 21,+T (14q21q) pat
V: 46, XX, 21,+T (21q21q) (parents not typed)
A) III, IV, I, V, II
B) IV, III, V, 1, II
C) III, I, IV, V, II
D) I, V, III, I, V, II
E) III, IV, I, II, V
MCQ Check-list

Why we write them
What type is best
What flaws are to be avoided
What content to test
How to construct it
What to test?
Items classified by cognitive processes required to
answer the question:
Recall (Memory) tests knowledge of isolated facts
Interpretation (Comprehension) review information
to reach conclusion, eg diagnosis
Problem solving (Reasoning) situation requires
action, eg next step in management
Process depends on background of trainee
What to test?
Simpler classification based on task:
Application of knowledge item **
Reach conclusions
Make predictions
Select course of action
Recall item
Test knowledge of isolated facts

What to test?

Test application of knowledge using clinical
vignettes
Adds face-validity
Usually selects more important, less trivial
Identifies student whove memorized but do not
understand

BUT experience in patient care should not
be necessary
What to test - summary
Focus item on key concepts and principles that are
essential information (without access to
references) for all examinees to understand
Test material that is relevant to learning in clinical
clerkships and beyond
Avoid items that only require recall of isolated
facts
Avoid esoteric or interesting topics that are not
essential
MCQ Check-list

Why we write them
What type is best
What flaws are to be avoided
What content to test
How to construct it
Writing the Questions

Construct stem include all necessary
information to arrive at the right answer
Choose lead-in pose a clear question
Choose distractors
Stem*

Usually a clinical scenario
Clear, unambiguous
Should be long relative to options
Includes all pertinent information
Patients age/gender
Clinical setting
Complaint
Other important info history, physical
findings, test results
Lead-in*

Follows stem
In the form of a question
Should relate to the stem
Should be answerable without looking at
options
Cover the Options test
Distractors

Each should be selected by some, therefore
all plausible, none obviously incorrect
3 or 4 choices
Common misconceptions
Faulty reasoning

Distractors

Homogeneous in content to the correct answer
Should be in same category as correct answer
diagnosis, treatment, diagnostic test
Clearly incorrect or inferior to the correct answer
Plausible and attractive to the uninformed
Similar to the correct answer in construction and
length
Grammatically consistent and logically compatible
with the stem
General Guidelines for Item
Construction
Include as much of the item as possible in the
stems: Long stems, short options
Avoid negatively phrased items (except or not in
the lead-in). If you must, use only with short
options
Avoid writing question of the form:
Which of the following statements is correct?
Each of the following statements is correct EXCEPT
Unfocussed
Heterogeneous options
Top-down versus Bottom-up

Top-Down
Knowledge or recall question
Cite a disease and then ask what patient
findings are expected
Structured like textbooks
Clinically backward
Top-down versus Bottom-up

Bottom-up
Application of knowledge
Gives findings and asks examinee to
indicate the disease
Examinees need to be able to synthesize
information from several pages of a
textbook (or notes) to answer these
questions
Clinically realistic
Guidelines for Clinical Sciences
ie Clerkship and beyond
Test application of knowledge using clinical
vignettes to pose medical decisions in
patient care situations
Focus items on common or potentially
catastrophic problems
Pose clinical decisions that would be
expected of a successful examinee
Avoid clinical situations that would be
handled by a (sub)specialist
MCQ Check-list

Why we write them
What type is best
What flaws are to be avoided
What content to test
How to construct it
Basic Rules for MCQs (A type)*

Focus item on important concept
Item should assess application of
knowledge, not recall of an isolated fact
Stem must pose clear question Cover the
options test
All distractors should be homogeneous
Avoid technical item flaws

Reference
Constructing Written Test Questions For the
Basic and Clinical Sciences
Case & Swanson, NBME 1996