User’s Guide for Internal Medicine Quiz Questions

Amy L. Holthouser, MD 2-17-09

This series of 380 original board-style multiple-choice questions was designed for use
within an internal medicine clerkship. Our clerkship uses the “First Exposure” books (inpatient and
outpatient texts) as required reading, and the questions were largely developed directly from those
resources. About half of the questions focus on inpatient material, and half on outpatient care. These
questions were designed to be above the typical level of detail and difficulty found on the IN
NBME shelf exam for third year students in order to encourage critical evaluation of assigned
reading material. In their home institution, these questions are organized into a series of four
quizzes administered every 3 weeks during a 12-week 3rd year IM clerkship. A typical pool of
questions for a quiz encompasses three clinical topics (ex. pulmonary, cardiology, infectious
disease) and consists of around 100 questions. From this pool, educational software (BlackBoard)
pulls twenty questions at random to generate a unique quiz for an individual student. Students
complete the quiz on their own time over a four-day period and are allowed to use books and
internet resources to answer the questions to the best of their ability. Students are not allowed to
save or reproduce the quizzes, and also not allowed to collaborate in answering quiz questions as
the quizzes were designed to reinforce independent reading and study on a prescribed topic
schedule during the clerkship. The four quizzes average together into a grade that accounts for 10%
of the overall clerkship grade for an individual student.

Our school has been using these quiz questions in this format successfully for the last two
years, dropping shelf failure from an average of 5-6 students a year to one student within the last
academic year. The questions are especially useful in two respects: to identify early in the rotation
students who have difficulty with standardized tests, and to enforce a rigorous reading schedule
during a long, busy clerkship for all students. When students fail a quiz, they are compelled to meet
with one of the clerkship directors and review the quiz, going through their thought process
question by question. This allows course directors to identify specific areas of knowledge weakness
or overall test taking strategy to focus on for the remainder of the clerkship. An individualized plan
is then prescribed that incorporates reading, practice questions, and repeated meetings with the
clerkship director to address these needs and keep the student’s independent learning on track
during the busy clinical clerkship. After each quiz is completed, the clerkship director also meets
with the entire track of students and reviews selected questions from the quizzes that students find
difficult, highlighting the key learning points or test taking strategies within the question.

When viewing the quiz questions, faculty will notice that about 10% of the questions have
red lines of text below the stem describing a physical exam finding or microscope slide, e.g. “slide
shows clue cells.” This indicates that question was originally developed with an image or picture to
aid in diagnosis. Since the pictures used were not the property of the author, they were unable to be
published with these questions. Images are readily available on the Web and within medical schools
for teaching and can be used for this purpose within your school without copyright violation.
Alternatively, the questions can simply be edited to include a description of the picture to help the
student find the right answer.

Other potential uses for this question collection include a midterm examination for students
and a resource for several questions to use within a lecture on a selected topic. Many questions
would also be reasonable used within a Family Medicine clerkship. Users are encouraged to pilot
the questions at their home institution prior to setting grading criteria or time-sensitive testing
criteria, as these aspects required a few iterations at our institution before arriving at the correct
balance.