UNIVERSITY OF SANTO TOMAS

FACULTY OF MEDICINE AND SURGERY

DEPARTMENT OF MEDICAL EDUCATION

TEST PLANNING AND CONSTRUCTION WORKSHOP

DEXTER CLIFTON C. PE, MD

PART I. PLANNING THE TEST

The easiest way to ensure a representative sample of content and cognitive objectives on the
test is to prepare a test blueprint. This table is simply a two-way chart listing the content topics
on one dimension and the cognitive skills on the other. We want to include content and skills in
the same proportion as they were stressed during instruction. Table 1 shows a simple test
blueprint; it is intended to be illustrative, not comprehensive. Remember to ask more questions
that require higher order thinking skills (HOTS – analyze, evaluate, create), and less of the
lower order thinking skills (LOTS - remember, understand, apply) items.

Table 1. Test Blueprint for a Physiology test on Fluids and Electrolytes:

LOTS HOTS
LEARNING OUTCOMES TOTAL
REMEMBER UNDERSTAND APPLY ANALYZE
Compute for the total body fluids in different bodily
1 1 2
compartments.
Predict changes in extracellular and intracellular
volume and osmolality caused by infusion of 1 2 1 4
isotonic, hypotonic and hypertonic solutions.
Analyze how different stimuli cause ADH release,
2 2
and its negative feedback control mechanism.
Analyze how different stimuli cause activation and
1 1 2 4
action of RAAS in tubular Na+ and H2O handling.
Apply how different factors cause transcellular shift
2 2
of K+ in different body fluid compartments.
Apply how different factors cause secretion of K+
1 1 2
into tubular fluid.
Analyze the role of the kidney in the regulation of
2 2
ECF Ca++.
Analyze the role of the kidney in the regulation of
2 2
ECF phosphate.
TOTAL ITEMS 2 4 4 10 20

This table indicates the content topics, the objectives to be covered and the proportion of the
test that will be devoted to each. Evidently, more class time was spent on objectives 2 and 4
because 40% of the test questions deal with that compared with only 10% on each of the other
objectives. The column totals indicate that 50% of the items will be written at the analyze level,
20% each at the apply and understand level, and only 10% at the remember level. Using the
percentages assigned to each cell, one writes the appropriate number of items.

A taxonomy of teaching objectives (Bloom, 1956) lists several cognitive outcomes

typically sought in college instruction. The three domains of educational learning are:
 Cognitive: mental skills (knowledge)
 Affective: growth in feelings or emotional areas (attitude or self)
 Psychomotor: manual or physical skills (skills)
Instructional designers, trainers, and educators often refer to these three categories as KSA
(Knowledge [cognitive], Skills [psychomotor], and Attitudes [affective]). This taxonomy of
learning behaviors may be thought of as “the goals of the learning process.” That is, after a
learning episode, the learner should have acquired a new skill, knowledge, and/or attitude.
These outcomes are listed in Table 2. If these are desired outcomes of instruction, then
classroom tests must include assessment of these objectives.

Table 2. Examples of Bloom's Cognitive Levels

BLOOM’S Cognitive Verbs

Remembering: can the

define, duplicate, identify, list, name, recall, recognize,
L student recall or remember the relate, record, repeat, reproduce state
information?
Choose, cite examples of, classify, demonstrate use of,
O Understanding: can the describe, differentiate between, discriminate, discuss,
student explain ideas or explain, give in own words, identify, interpret, locate, pick,
concepts? practice, recognize, report, review, restate, respond,
T select, simulates, tell, translate, paraphrase
Apply, choose, demonstrate, dramatize, employ,
Applying: can the student use generalize, illustrate, interpret, initiate, operate,
S
the information in a new way? operationalize, practice, relate, schedule, sketch, solve,
use, utilize, write.
Analyze, appraise, calculate, categorize, compare, contrast,
H Analyzing: can the student criticize, deduce, detect, determine, develop, diagram,
distinguish between the differentiate, discriminate, distinguish, draw, estimate,
different parts? evaluate, examine, experiment, identify, infer, inspect,
O solve, question, test.
appraise, assess, argue, choose, compare, critique, defend,
Evaluating: can the student
estimate, judge, measures, rates, revise, score, select,
justify a stand or decision?
T support, value, evaluate, validate, test
Arrange, assemble, collect, compose, construct, create,
Creating: can the student
design, develop, devise, formulate, manage, modify,
S create new product or point of
organize, plan, prepare, produce, propose, predict,
view?
reconstruct, set-up, synthesize, systematize, write.

Coordinating test content with instruction content ensures content validity of the test. Using a
table of specifications also helps an instructor avoid one of the most common mistakes in
classroom tests, namely writing all the items at the knowledge level.
PART II. CONSTRUCTING / WRITING THE TEST

MULTIPLE CHOICE ITEM

 It is designed for objective measurement.

 It contains a STEM and four RESPONSES, one of which is the best answer.
 The standard by which the best answer is selected is contained in the stem.

Writing and evaluating the Stem:

1. Construct an item that assess a single intended learning outcome.
2. The stem of a multiple choice item may:
ask a question
give an incomplete statement
state an issue
describe a situation
or any combination of the above
3. The content focuses on a central theme or problem, using clear and precise language,
without excessive length.
4. It should present all relevant information to insure clarity.
5. It should be stated in a POSITIVE form. Negative wordings should be given emphasis.
6. Structure should be grammatically accurate and logically related to the responses.
7. Avoid superfluous information.
8. Avoid use of personal pronouns such as "you".

Writing and evaluating the Responses:

1. The "BEST" answer is the response considered as the most appropriate by experts.
2. The "DISTRACTORS" are logical misconceptions of the best answer.
3. Each item should have four mutually exclusive responses.
4. Alternatives or distractors should be homogenous in content.
5. The grammatical structure of all responses should be a logical conclusion to the
situation, question, or statement presented in the stem.
6. Arrange responses in logical orders.
7. The length of each response should be approximately the same.
8. Avoid superlatives such as "always" and "never".
9. Repetitive language within the responses should be avoided (Words which are repeated
in every response may be placed in the stem).
10. Avoid “none of the above” or “all of the above” as a response.

Suggestions for writing Multiple Choice Items which measure Higher Order Thinking Skills:
It is difficult and time-consuming to write multiple-choice items that measure the higher thinking
skills. The item writer has to be creative in order to develop challenging questions. The following
suggestions may provide some ideas for writing these kinds of questions.
1. Present practical or real-world situations to the students. These problems may use short
paragraphs describing a problem in a practical situation. Items can be written which call
for the application of principles to the solution of these practical problems, or the
evaluation of several alternative procedures.
2. Present the student with a diagram of equipment and ask for application, analysis, or
evaluations, e.g., "What happens at point A if...?," "How is A related to B?"
3. Present actual quotations taken from newspapers or other published sources or
contrived quotations that could have come from such sources. Ask for the interpretation
or evaluation of these quotations.
 4. Use pictorial materials that require students to apply principles and concepts.
5. Use charts, tables or figures that require interpretation.

BINARY ITEMS

 declarative sentence that the student is asked to mark true or false, right or wrong,
correct or incorrect, yes or no, fact or opinion, agree or disagree
 there are only two possible answers (true or false; fact or opinion)
 measures the ability to:
o identify the correctness of statements of fact, definition of terms, statements of
principles, and the like
o distinguish fact from opinion
o recognize cause-and-effect relationships
o do some simple aspects of logic

The general principles in writing Binary Items:

1. Each item should refer to one concept.
2. It must not include an opinion statement.
3. Avoid trivial statement or bit of information that are not important.
4. Avoid negative words and phrases: they can be confusing. If negatives are necessary,
they are emphasized with underlined, bolded, capitalized, italicized,
and/or colored indicators. (i.e. "no", "not", "cannot”)
5. Avoid long and complex sentences.
6. Avoid specific quantifiers (i.e. always, never, usually, often, generally).
7. Avoid indefinite or subjective terms that are open to interpretation
(i.e. "a very large part", "a long time ago", "most")
8. Avoid excess use of negative words or phrases.
9. Avoid ambiguous words or statements. Use simple, direct language in declarative
sentences
10. Avoid exact wording from textbooks or other references.
11. Avoid using a pattern of occurrence of true-or-false statements. (i.e. TTFTTF)
12. Avoid a disproportionate number of either true or false statements.

MATCHING ITEMS

 presented in groups as a series of stems or prompts that must be matched by the student to
one of a group of possible answer options
 useful format when the objective to be measured involves: association skills or the ability to
recognize, categorize, and organize information
 measures high levels of understanding but are most typically used at the knowledge level
and for younger students
 can cover a large amount of material in a relatively brief period of time
 even more efficient than multiple-choice items because each stem acts as a separate
multiple-choice item with all the answer options as possible answers
 Functionally, containing ten stems operates as ten multiple-choice items
 When well-written, all the wrong answer options act as distractors
 Guessing is difficult
 the validity and reliability of classroom tests are improved

A few of the CRITICAL GUIDELINES in writing matching type items:

1. More answer options than stems
2. Answer options should be available more than once
 this increases the number of functional distractors and increases the validity of the items
 it is important that the instructions for the matching section indicate that answer options
may be used more than once or not at all, so all students are aware of the rule.
3. Directions should include basis for match
 A brief instruction identifying the category of stems and answer options (e.g. leaders and
nations, species and phylum) helps students to focus on what constitutes a match, so
they can concentrate on choosing the correct answer
4. Number of answer options should be < 7 for elementary age
 The ability to process and evaluate many possibilities is likely not the measurement
objective of the assessment.
5. Number of answer options should be < 17 for secondary age
 A well-made classroom assessment should not be exhausting for students.
6. Matching stems should be on the left and answer options on the right

Suggestions for Writing Matching Items:

1. Keep both the list of descriptions and the list of options fairly short and homogeneous—
 they should both fit on the same page
 Title the lists to ensure homogeneity and arrange the descriptions and options in
some logical order
2. Make sure that all the options are plausible distractors for each description to ensure
homogeneity of lists.
3. The list of descriptions should contain the longer phrases or statements and should be on
the left, while the options in the right column should consist of short phrases, words, or
symbols.
4. Each description in the list should be numbered (each is an item), and the list of options
should be identified by letter
5. Include more options than descriptions
 If the option list is shorter, some options must be used more than once
 Always include some options that do not match any of the descriptions, or some that
match more than one, or both
6. In the directions, specify the basis for matching and whether options can be used more
than once.

COMPLETION / SHORT-ANSWER ITEMS

 Both are supply-type test items that can be answered by a word, phrase, number, or
symbol.
 They are essentially the same, differing only in the method of presenting the problem:
o SHORT-ANSWER ITEM- uses a question
o COMPLETION ITEM- consists of an incomplete statement
 Suitable for measuring a wide variety of relatively simple learning outcomes.
 o Knowledge of terminology, specific facts, principles, method or procedure
o simple interpretation of data

Advantages of Short-Answer Items

1. One of the easiest to make
2. Measures simple learning outcomes
3. Measures recall of memorized information, except for problem-solving outcomes
measured in Math and Science
4. Students must supply the answer; reduces the possibility of guessing
5. Partial knowledge is insufficient for answering a short-answer item correctly

Limitations of Short-Answer Items

1. Unsuitably for measuring complex learning outcomes
2. Difficulty of scoring

Suggestions for Writing Completion or Supply Items:

1. Items should require a single-word answer or a brief and definite statement.
2. The question or statement poses a problem to examine. A direct question is more
desirable than an incomplete statement.
3. The answer the student is required to produce is factually correct.
4. Omit only key words; do not eliminate so many elements that the sense of the content is
impaired.
5. Word the statement such that the blank is near the end of the sentence rather than the
beginning.
6. Indicate the units it is to be expressed if the problem requires a numerical answer.

ESSAY ITEMS

 This item allows student to supply, rather than select, the correct answer.
 The student must compose a response to a question for which there is no single
response or pattern of responses can be cited as correct to the exclusion of all other
answers.
 The accuracy and quality of such a response can often be judged ONLY by a person
skilled and informed in the subject area being tested.
 It should measure complex cognitive skills or processes.
 Distinct feature: freedom and response

Advantages of Essay Items:

1. Most effective in assessing complex learning outcomes
2. Relatively easy to construct
3. Emphasize essential communication skills in complex academic disciplines
4. Guessing is eliminated
5. Apply your knowledge
6. They give student the options to chose what they want to write about reasoning
7. The teacher can tell if the student studies or not
8. Can often prepare in advance what to write
Disadvantages of Essay Items:
1. Difficult to score
2. Scores are unreliable
3. Limited sample of total instructional content
4. Bluffing
5. Time consuming
6. Grading: grammar usage vs. content
7. Legibility
8. Hard to formulate ideas in time
9. Takes longer to grade

Suggestions for writing Essay Items:

1. Use essay questions for learning outcomes that cannot be measured by objective items.
2. Phrase each question so that the pupil’s task is clearly indicated.
3. Indicate an approximate time limit for each question
4. Avoid using optional questions
5. Start essay questions with such words or phrases as compare, contrast, give reasons for,
give original examples of, predict what would happen if, and so on. DO NOT begin with such
words as what, who, when, and list, because those words generally lead to tasks that
require only recall of information.
6. A question dealing with a controversial issue should ask for and be evaluated in terms of the
presentation of evidence for a position rather than the position taken.
7. Each question should match an instructional objective.

Proper scoring of Essay Items:

1. Prepare an outline of the expected answer in advance
2. Use the scoring method that is most appropriate
3. Decide how to handle factors that are inherent to the learning outcomes being measured
4. Evaluate all answers to one question before going to the next one
5. Evaluate the answers without looking at the pupil's name

REFERENCES

Cross, T.L.(1990) Testing in the College Classroom. Paper presented at the Annual Meeting of American Educational
Research Association, Boston.
Milton, O. and Associates. (1978). On College Teaching: A Guide to Contemporary Practices. San Francisco: Jossey-
Bass.
Anderson, L.W., Krathwohl, D.R., Airasian, P.W., Cruikshank, K.A., Mayer, R.E., Pintrich, P.R., Raths, J., Wittrock,
M.C. (2001). A Taxonomy for Learning, Teaching, and Assessing: A revision of Bloom's Taxonomy of Educational
Objectives. New York: Pearson, Allyn & Bacon.
Bloom, B.S. (Ed.). Engelhart, M.D., Furst, E.J., Hill, W.H., Krathwohl, D.R. (1956). Taxonomy of Educational
Objectives, Handbook I: The Cognitive Domain. New York: David McKay Co Inc.
Clark, R., Chopeta, L. (2004). Graphics for Learning : Proven Guidelines for Planning, Designing, and Evaluating
Visuals in Training Materials . San Francisco: Jossey-Bass/Pfeiffer.

MULTIPLE CHOICE ITEM

Airasian, P.W. (1991). Classroom Assessment. NY: McGraw-Hill.
Ebel, R.L. and Frisbie, D.A. (5th ed). (1991) Essentials of Educational Measurement. Englewood Cliffs, NJ: Prentice
Hall.
Gronlund, N.E. and Linn, R.L. (6th ed). (1990). Measurement and Evaluation in Teaching NY: Macmillan.

BINARY ITEM
Padua, R.and Santos, R. (1997). Educational Evaluation and Measurement: Theory, Practice and Application. Katha
Publishing Co., Inc.
Study Guides and Strategies Website
University of Wisconsin-Eau Claire (2011). Assessment Toolkit. Writing True/False Test Question

MATCHING TYPE
Kubiszyn, T. and Borich, G. (2013). Educational testing and measurement: Classroom Application and Practice (10 th
ed.). John Wiley and Sons, USA.
David Miller, M., Linn, R., and Gronlund, N. (2009). Measurement and Assessment in Teaching (10 th ed.). Pearson
Education, Inc., Upper Saddle River, New Jersey.

COMPLETION ITEM
Kubiszyn, T. and Borich, G. (2013). Educational testing and measurement: Classroom Application and Practice (10th
ed.). John Wiley and Sons, USA.
David Miller, M., Linn, R., and Gronlund, N. (2009). Measurement and Assessment in Teaching (10 th ed.). Pearson
Education, Inc., Upper Saddle River, New Jersey.

ESSAY ITEMS
Gronlund, N. & Linn, R. (1990). Measurement and Evaluation in Teaching 6th ed. New York:
Macmillan.
Kubisza, T.& Borich, G. (2000). Educational Testing and Measurement 10 th ed. New York: John
Willy & Son.

Special thanks to Drs. Remedios Dee-Chan, Gerald Alcid, Maria Rosario Cabansag, Elaine Cunanan, Carmelita
Dalupang, Mina Laguesma, Therese Mallen, Anita Sangalang and Julie Visperas.