This material has been copied

under licence from CANCOPY.

Resale or further copying of this
material is strictly prohibited.

Credits/Acknowledgements:

Developers: Clarence Esau and Bev Biggeman

Course Design & Production: Learning Enhancement Services, Olds College

Copyright Notice:
Copyright © - Olds College, 4500 - 50 Street, Olds, Alberta, T4H 1R6
May, 1999

No part of the works contained herein may be reproduced or copied in any form or by any means –
graphic, electronic or mechanical (including photocopying, recording, taping, or information and
retrieval systems) – without the expressed written consent of Olds College, 4500 - 50 Street, Olds,
Alberta, T4H 1R6.

Every effort has been made to provide acknowledgement of original sources to comply with copyright
law. Where this has not been done, please notify Olds College, Learning Enhancement Services so that
appropriate corrective action can be taken.

Where materials within the document are published by permission from another copyright holder, the
permission cannot implicitly be transferred. Permission to copy that portion of the document is
therefore, not included and must be obtained from the original copyright holder.

ISWS 530
 Instructor Skills Workshop

ISWS 530 - Assessing Learning Outcomes

Competency Profile
Explain how assessment relates to Identify the role of assessment in learning. Identify the characteristics of assessment instruments.
learning. A2
A A1

Develop strategies in assessing learning. Develop tools for assessing technical and workplace Develop strategies for assessing thinking skills.
skills.
B B1 B2
Olds College June 2002

J:\Profiles/Crsprofi/ISWSprof/ISWS 530
 Table of Cont ents
Contents
Module A How Assessment Relates to Learning
Module A1 - Role of Assessment in Learning ................................................................................................ A-1
Module A2 - Characteristics of Assessment Instruments .............................................................................. A-5

Module B Strategies in Assessing Learning

Module B1 - Tools for Assessing Technical and Workplace Skills ................................................................... B-1
Module B2 - Strategies for Assessing Thinking Skills .................................................................................. B-13

OLDS COLLEGE
ISWS 530

i
ISWS 530 Assignment

1. Using the criteria outlined in the ISWS 530 module, develop a twelve item
(minimum) multiple choice test to evaluate the thinking skills in a module(s). You
may use tests that you have developed previously.

2. Using the checklist profiled in your ISWS 530 module as an example, develop a
product or process checklist which measures the technical and workplace skills
from a module(s). Develop a scale (yes/no if the increments in the behavior are
small) or (1-5 scale which measures the skills in more general terms).
 Module

A1
Identify the role of assessment in learning.

Learning Assessment

Learning assessment tools have been around for time immemorial, but their roles in learning have
not always been articulated. You will need to identify why you are testing, in order to better select
your assessment strategies.
In identifying the roles of learning assessments, you will:
• identify the focus of learning assessments;
• identify the levels of assessing learning;
• describe the formative and summative roles of learning assessments.

OLDS COLLEGE
ISWS 530

A-1
 ¾ Identify the focus of
Why Assess Learning?
learning assessments.
Assessment of learning has to do with making decisions about the:
¾ Identify the levels of
assessing learning.
Describe the formative and
summative roles of learning
assessments.
a) Learner: What is the learner's level of mastery of skills
reflected in the task statements? Can the learner proceed with
subsequent learning? Can the learner participate in the
workplace with an appropriate degree of performance? How
will this level of mastery be expressed and reported?
b) Instruction: Was the instruction adequate? Was it focused on
specified outcomes? Did it provide the learner with experiences
to allow for integration of skills?
c) Curriculum: Was the learner provided with a learning path that
led to the development of skills? Was the degree of
elaboration at the appropriate level to allow for the mastery of
skills?
d) Instructor: Did the instructor articulate the learning objectives?
Did the instructor follow through consistently on the objectives?
Did the instructor follow the elements of instruction to allow
for an interactive learning relationship?

Levels of Assessment
Learning and its impact is assessed at several levels. Donald
Kirkpatrick outlines these levels in his four-level chart.

Levels of Evaluation - Donald L. Kirkpatrick

4. Results: Has the training resulted in increased

production and/or quality? (work)
3. Behaviour: What changes in behaviour have
occurred? (work)
2. Learning: What increase has there been in skill develop-
ment? (instruction)
1. Reaction: How satisfied are the clients? (instruction)
However, others have added further levels to Kirkpatrick's original
chart. They have asked "Is it reasonable to assess learning at any
level before assessing the resources that went into the workshop or
course?" Further to this is the question above level four "Is the learn-
ing or training taking into account the societal or environmental conse-
quences?"

OLDS COLLEGE
ISWS 530

A-2
Evaluation Levels (Adapted from Kirkpatrick)

5. Mega-level Societal consequences: societal and client re-

sponsiveness, contributions and pay-offs.
4. Macro-level Organizational results: organizational contribu-
tions and pay-offs.
3. Micro-level Successful application: individual and small
group utilization within the organization.
2. Micro-level Acquisition: individual and small group mastery
and competence.
1b. Process Reactions: method-means and processes' accept
ability and efficiency.
1a. Input Enabling: availability and quality of human,
financial and physical resources.

Formative and Summative

Identify the focus of
Assessment learning assessments.

Identify the levels of

assessing learning.
Formative Assessment ¾ Describe the formative and
summative roles of learning
Formative assessment, sometimes called process assessment, provides assessments.
the learners and learning facilitator with continuous feedback on the
advance of learning. Formative assessment serves the functions of:
• detecting a malfunction or problem in the learning process;
• identifying the source of the problem;
• providing information for solving the problem;
• indicating when the problem has beeen solved.
As you can see, formative assessment is all about the dynamics of
learning. It is about getting feedback, and about responding to the
feedback, all focused on the learner. Examples of formative assess-
ment strategies are:
• feedback during facilitator-learner
interaction;
• quizzes following facilitator-learner
interaction;
• unit tests, when they become the basis
for remedial learning.

OLDS COLLEGE
ISWS 530

A-3
 Summative Assessment
The purpose of summative assessment is to provide a measurement of
skills mastery for making decisions about:
• the score or rating to be attached to a learner's mastery;
• promotion of the learner to the next learning phase;
• recommendations to external bodies in regard to the learner.
In this sense, summative assessment is not so much about the learning
process as it is about meeting administrative requirements. However,
this is not to say that summative assessment is not important, but
rather to describe its purpose as outside the dynamics of learning.

OLDS COLLEGE
ISWS 530

A-4
 Module

A2
Identify the characteristics of assessment
instruments.

Assessment Instruments

The assessment strategy you use must be based on your purpose for assessment. You must decide
whether it is important to compare learners, or to determine the mastery level of the individual
learner.
In identifying the characteristics of assessment instruments, you will:
• describe norm-referenced assessment;
• describe criterion-referenced assessment;
• define reliability as it relates to testing;
• define validity as it relates to testing.

OLDS COLLEGE
ISWS 530

A-5

¾ Describe norm-referenced
assessment.
¾ Describe criterion-
referenced assessment.
Define reliability as it
relates to testing.
Define validity as it relates
to testing.
OLDS COLLEGE
ISWS 530
A-6
Two Ways to Approach Assessment
Norm-referenced Testing
Do you remember when you went to school, your report
card contained comments like “average” or “above aver-
age”? When you wrote your diploma exams, you may
have been informed that your marks would be placed on
a curve, with a set number of A’s, B’s, C’s, D’s and F’s.
Your grade would depend on how you performed in
relation to the other people writing the same exam.
This system is known as norm-referenced assessment, which means
that performance is measured as a comparison of how others have
performed. To do well, you must exceed the performance of other
learners, which often creates a competitive atmosphere. In norm-
referenced assessment, marks are assigned on the traditional grading
system, with a strong emphasis on average, below average and above
average.
Criterion-referenced Testing
In contrast to this approach is the assessment of the learner's mastery
of a specific set of skills. The instructor sets out a number of perfor-
mance objectives, and the learner's progress is measured against these
objectives. As you can see, there is no competition
with other learners, since the learner is measured
against a pre-set criteria.
This system is known as criterion-referenced assess-
ment, and has gained considerable support among adult
educators.
This system recognizes that to be successful, the learner
will need to master a basic set of skills, unrelated to the success rate
of other learners. In a time mastery-learning situation, the learner
would be scored on a pass/fail system. However, many institutions
who use the criterion-referenced system use a modified form of the
traditional grading system.
It becomes rather obvious that programs with a strong skills base,
such as those found in technical training institutes, are best served by
criterion-referenced assessment.
Industry representatives are not impressed with job applicants who
present an incomplete skills set, but rather they demand a fairly high
degree of mastery in the required skills.
In criterion-referenced testing, the basic criterion for assessing learning
is the learning objective. The time to develop the assessment instru-
ment is right after the course competencies have been established,
and before the instructional strategies have been planned. In this way,
the instruction itself will not affect the assessment, but will follow the
setting out of the criteria. If the instructor directs learning to the
course competencies, and assesses learning based on those competen-
cies, then there is a true fit in the process.
Criterion-referenced Test Frequency Distribution

Test Scores

Typical Non-referenced Test Frequency Distribution

Test Scores

Self Check A2-1

1. What is the main advantage of criterion-referenced testing over institute.
norm-referenced testing? expected of a technical training
by an individual, which is
focuses on the mastery of skills
1. Criterion-referenced testing

Self Check Answer A2-1:

OLDS COLLEGE
ISWS 530

A-7
 OLDS COLLEGE
ISWS 530

A-8
Reliability Describe norm-referenced
assessment.
Reliability is the consistency of test scores. The two kinds of reliabil-
Describe criterion-
ity are: referenced assessment.

• Test-retest reliability: The measure of an individual's ¾ Define reliability as it

relates to testing.
consistency over time. If you give a group of learners a test
today, you should expect basically the same results if you give ¾ Define validity as it relates
to testing.
the same test tomorrow, assuming that the learners did not
review the skills in the meantime.
• Inter-rater reliability: The measure of consistency among
individual judges' ratings of a performance. For
example, judges in a figure-skating competition
should all have very similar scores, if they are
scoring on the basis of the same criteria for
performance.

Validity
Validity is whether a test measures what it is supposed
to measure. A test can be reliable, but measure the wrong thing. For
example, assume that we have designed a course to teach learners to
erect a greenhouse. We could develop a test that asks the learners to
list all the steps in erecting a greenhouse. The
learners may be able to list these steps, but
be unable to erect a greenhouse. This test
would be invalid for these learners, given the
difference in the learning objectives and the
criteria for testing.

OLDS COLLEGE
ISWS 530

A-9
 OLDS COLLEGE
ISWS 530

A-10
 Module

B1
Develop tools for assessing technical and
workplace skills.

Technical/Workplace Skills

The assessment of technical and workplace skills requires tools based on observation and inspec-
tion. Process and product checklists permit this inspection to be documented.
In developing strategies for assessing technical and workplace skills, you will:
• develop product checklists for technical skills;
• develop process checklists for technical skills;
• explain the need to assess workplace skills;
• describe the process of mastering workplace skills;
• explain how to identify workplace skills required by the learner;
• integrate workplace skills into performance checklists.

OLDS COLLEGE
ISWS 530

B-1
¾ Develop product checklists
Assessing Product
for technical skills.
Product can be appropriately assessed if these conditions are present:
Develop process checklists
for technical skills. • The characteristics of the product can be evaluated, such as
Explain the need to assess dimensions, completeness, neatness, and appearance.
workplace skills.
• The process in producing the product is not an issue.
Describe the process of
mastering workplace skills. • The product itself is the criterion for mastery.
Explain how to identify
workplace skills required
by the learner.

Integrate workplace skills Product Checklists

into performance
checklists. For developing a performance checklist to evaluate performance of a
task with a resulting tangible product, use the following guidelines:
• Base the checklist on evaluating appropriate aspects of the
final product - its appearance, whether it works, whether it
tastes right, whether it holds together, etc.
• Evaluate all important aspects or parts of the product.
• Sequence the checklist in a way that facilitates assessment of
the product.
• Ask yourself the following question: Would more than one
qualified assessor using the checklist evaluate the same prod-
uct about the same way?

Quality Checklist for a Cabinet

Check the Construction: Yes or No
1. Is there any evidence of cracking of wood or delamination? _______
2. Are all corners square with no perceptible joint separation? ________
3. Are exposed surfaces smoothly rounded with no saw marks
or other blemishes? ________
4. Do doors align when closed? ________

OLDS COLLEGE
ISWS 530

B-2
Examine the Details: Yes or No
1. Are hardware and handles fastened securely and aligned
correctly? ________
2. Is the finish smooth and free of excess glue and filler? ________
3. Are the edges smooth? ________
4. Are screws, nails, and fasteners concealed? ________
5. Do doors and drawers open smoothly and quietly? ________
6. Are the drawers removable? ________

Assessing Process
Develop product checklists
Process can be appropriately assessed if these conditions are present: for technical skills.

¾ Develop process checklists

• you need to know what the learner will do during perfor- for technical skills.
mance; Explain the need to assess
workplace skills.
• there is no “product” to be assessed;
Describe the process of
• the sequence of activities is important; mastering workplace skills.

• there is a need to assess workplace skills such as safety, time Explain how to identify
workplace skills required
management, and communication. by the learner.

Integrate workplace skills

Process Checklists into performance
checklists.
For evaluating performance tasks with no tangible product, use the
following guidelines:
• Include all important steps in the performance checklist. Base
the steps on the requirements of performing the task on the
job.
• State each step briefly, clearly, and objectively. Avoid value-
laden words or phrases that might encourage subjective judg-
ments (e.g. effective, appropriate, clearly, adequately, done
well).
Example
Poor: Preheated the oven to the appropriate temperature.
Better: Preheated the oven to the temperature specified in the
recipe.
• Sequence the steps in the order in which they should
be performed.
• Ask yourself this question: Would more than one quali-
fied rater observing the same performance give it the
same evaluation, using this checklist?
OLDS COLLEGE
ISWS 530

B-3
 Example
Culinary Arts: Prepare Popovers
Scoring Checklist
Process Criteria A NA
1. Got the recipe for popovers from the instructor. Read the
recipe, then made the necessary adjustments (for quantity,
availability of ingredients, etc.). ____ ____

2. Washed hands. ____ ____

3. Preheated the oven to the temperature specified in the recipe. ____ ____

4. Organized the work area. Assembled the ingredients, tools,

and utensils near the oven. ____ ____

5. Measured and weighed the ingredients. ____ ____

6. Lightly greased deep muffin pans or custard cups. ____ ____

7. Sifted the dry ingredients together. ____ ____

8. Beat the eggs and liquid together. ____ ____

9. Added the liquid to the flour mixture. Mixed the batter until
it was smooth. Used standard safety precautions when
operating the mixer. ____ ____

10. Poured the batter into muffin pans or custard cups. Filled
the cups halfway. Was careful not to pour too much or too
little batter into the cups. ____ ____

11. Baked until the popovers were puffed up and brown

(20-30 minutes). Did not open the oven door during baking. ____ ____

12. Lowered the oven temperature as directed in the recipe and

continued baking 15 minutes longer. ____ ____

13. Removed the pan from the oven. Served the popovers hot.
Used standard safety precautions when working with hot
surfaces. ____ ____
14. Cleaned and sanitized the work area, utensils, and
equipment. ____ ____
15. Followed all safety precautions. (List details here.) ____ ____

16. Followed all sanitation procedures. (List details here.) ____ ____

OLDS COLLEGE
ISWS 530

B-4
Example
Skid-Steer Loader Checklist
Competency Category Pass/Fail P = Pass
F = Fail

Walk Around Check for:

____ Basic machine parts (is anything missing).

____ Hydraulic system for leaks, worn hoses, damaged lines, dam-
aged cylinders or rods, leaking wiper seals.
____ Cooling system for leaks, debris, shrouds, grills, belts.
____ Coolant level (engine cold).
____ Engine compartment for oil, coolant, fuel leaks.
____ Engine oil level.
____ Tires for cuts, gouges, inflation, imbedded foreign objects.
____ Drive train (look under for leaks).
____ Instrument panel for broken or inoperative gauges or controls.
____ Lights operation.
____ Trash on machine anywhere.
____ Door arms and pins.
____ Bucket and pins and attaching locks.
____ Safety information stickers.
____ Bucket empty of loose or frozen material.
____ Grab handles not damaged.

OLDS COLLEGE
ISWS 530

B-5
 Start-Up Check for:

____ All controls neutralized.

____ All persons or animals clear of machine.
____ Gauges indicating correctly with key turned to “on” position.
____ Shop or garage doors open for ventilation.
____ Seat adjusted.
____ Seat belt on.

Self Check B1-1

1. In what kinds of learning situations would process checklists be
more appropriate than product checklists?

communication required.
practices, time required, and
important, such as safety
1. The behavior of the learner is

Self Check Answer B1-1:

OLDS COLLEGE
ISWS 530

B-6
Assessing Workplace Skills Develop product checklists
for technical skills.
There is a growing realization that the mastery of a set of isolated
technical skills may not be sufficient to meet the requirements of the Develop process checklists
for technical skills.
workplace. We cannot assume that learners are coming to college
¾ Explain the need to assess
with a complete set of requisite personal skills which that person will workplace skills.
require to succeed as an employee.
Describe the process of
mastering workplace skills.
What is required for success are the technical competencies sur-
rounded by a host of supportive skills, which could be described as Explain how to identify
workplace skills required
enhanced technical competencies. In other words, the learner would by the learner.
master the required technical skills to succeed at a job, but would Integrate workplace skills
be given the opportunity to integrate other workplace skills. These into performance
checklists.
skills may include:
• listening to understand and learn;
• speaking in the work setting;
• reading directions, graphs, charts;
• writing effectively;
• thinking critically and logically;
• solving problems;
• learning to learn;
• demonstrating positive attitudes and behaviours;
• demonstrating responsibility;
• demonstrating adaptability;
• working with others.

OLDS COLLEGE
ISWS 530

B-7
 Develop product checklists
Process of Mastering Workplace
for technical skills.
Skills
Develop process checklists
for technical skills. Stages Key Words
Explain the need to assess
Receiving
workplace skills.
Receiving refers to the student’s willingness to listens attentively
¾ Describe the process of
mastering workplace skills. attend to particular phenomena or stimuli shows awareness
(classroom activities, textbook, music, etc.). shows sensitivity
Explain how to identify
workplace skills required From a teaching standpoint, it is concerned with attends closely
by the learner. getting, holding, and directing the student’s
Integrate workplace skills attention. Learning outcomes in this area range
into performance from the simple awareness that a thing exists to
checklists.
selective attention on the part of the learner.
Receiving represents the lowest level of learning
outcomes in the affective domain.

Responding
Responding refers to active participation on the obeys rules
part of the student. At this level the student not participates
only attends to a particular phenomenon but also volunteers
reacts to it in some way. Learning outcomes in shows interest
this area may emphasize acquiescence in practices courteous
responding (reads assigned material), willingness behaviour
to respond (voluntarily reads beyond assignment), shows willingness
or satisfaction in responding (reads for pleasure or
enjoyment). The higher levels of this category
include those instructional objectives that are
commonly classified under “interests”, that is, those
that stress the seeking out and enjoyment of
particular activities.

Valuing
Valuing is concerned with the worth or value a demonstrates belief
student attaches to a particular object, shows concern
phenomenon or behaviour. This ranges in degree demonstrates commitment
from the more simple acceptance of a value shares
(desire to improve group skills) to the more chooses to assist
complex level of commitment (assumes respons-
ibility for the effective functioning of the group).
Valuing is based on the internalization of a set of
specified values, but clues to these values are
expressed in the student’s overt behaviour. Learning
outcomes in this area are concerned with behaviour
that is consistent and stable enough to make the value
clearly identifiable. Instructional objectives that are
commonly classified under “attitudes” and
“appreciation” would fall into this category.

OLDS COLLEGE
ISWS 530

B-8
Organization
Organization is concerned with bringing together recognizes
different values, resolving conflicts between them, accepts
and beginning the building of an internally works in harmony
consistent value system. Thus, the emphasis is on is sensitive to issues
comparing, relating and synthesizing values. resolves conflicts
Learning outcomes may be concerned with the assists others
conceptualization of a value (recognizes the compares
responsibility of each individual for improving relates
human relation) or with the organization of a
value system (develops a vocational plan that
satisfies the need for both economic security and
social service). Instructional objectives relating to
the development of a philosophy of life would fall
into this category.

Characterization by a value or value complex

At this level of the affective domain, the individual displays
has a value system that has controlled his behavior demonstrates self-
reliance for a sufficiently long time for him to develop practices co-operation
a characteristic “life style.” Thus the behaviour is maintains good habits
pervasive, consistent and predictable. Learning provides support
outcomes at this level cover a broad range of
activities, but the major emphasis is on the fact that
the behaviour is typical or characteristic of the
student. Instructional objectives that are concerned
with the student’s general patterns of adjustment
(personal, social, emotional) would be appropriate
here.

OLDS COLLEGE
ISWS 530

B-9
 Develop product checklists
for technical skills.
Develop process checklists
for technical skills.
Explain the need to assess
workplace skills.
Describe the process of
mastering workplace skills.
¾ Explain how to identify
workplace skills required
by the learner.
¾ Integrate workplace skills
into performance
checklists.
OLDS COLLEGE
ISWS 530
B-10
Identifying Workplace Skills
Workplace skills are identified during the needs assessment process.
While the needs assessment facilitator is identifying the skills required
by the industry representatives, the respondents will come up with
both technical and workplace skills.
When the respondents are asked for the skills they and their workers
require to perform on the job, they are usually quite specific about
technical skills. However, their description of the workplace skills their
work requires is quite general in nature, such as "time management."
This response requires that the facilitator probe to determine the
specific actions that make up "time management." These actions may
be expressed as "show up on time", "complete job before leaving the
workplace", or "complete a task within a specified time."
These specific actions then become the workplace skills which can be
assessed. You may examine any recent needs assessment chart to
identify samples of these skills.
Workplace Skills in Checklists
Workplace skills may relate to both process checklists and product
checklists.
Product Checklists Which Include Workplace
Skills
Turn back to the "Quality Checklist for a Cabinet" in Module B1.
You could add the following workplace skills to the checklist:
• Is the quality of work consistent in the project?
• Are the backs of the drawers well constructed?
The first of these questions is designed to assess the consistency of
the learner's work, a very important skill or attribute in the work-
place. The second function relates to the learner's ability to maintain
quality even in parts of the project not seen at first glance, an impor-
tant attribute usually checked by perceptive buyers.
Process Checklists Which Include Workplace
Skills
At first glance, a process checklist consists of a sequence of steps
the learner goes through in working through the project to its conclu-
sion. However, these process steps may require the learner to apply
the skills of selecting materials and supplies, judging the quality of
materials and supplies, measuring and weighing, maintaining cleanliness
and watching for problems. These can all be classified as workplace
skills, and can be tested as part of a checklist.
In addition to these workplace skills directly related to the process,
you may also identify workplace skills in personal management,
probably as a result of an industry needs assessment. These skills
may include:
• co-operate with team members;
• complete the project within time limits;
• solicit feedback on quality of performance;
• maintain a pleasant working relationship;
• assist team members when assigned tasks are complete.

Numerical Scale
Behaviour Performance
Poor Excellent
1 2 3 4 5 6 7
1. The quality of the
response for directory
assistance was ……….. 1 2 3 4 5 6 7
2. The statement of course
objectives was ………… 1 2 3 4 5 6 7

Descriptive Scale
Behaviour Performance
Very Poor Poor Average Good Very Good
1. The quality of the
response for directory
assistance was ………..
2. The statement of course
objectives was …………

OLDS COLLEGE
ISWS 530

B-11
 Behaviorally Anchored Numerical Scale
Behaviour Performance Rating
1. Response to directory 1. Curt voice tone and inaccurate or
1
assistance request delayed response
2. Delayed response with neutral
2
voice tone
3. Average response time with
3
neutral voice tone
4. Shorter than average response
4
time with pleasant voice
5. Very short response time with
5
very pleasant voice

OLDS COLLEGE
ISWS 530

B-12
 Module

B2
Develop strategies for assessing thinking
skills.

Cognitive Skills

Even in a technical training institution, learners are required to master a larger number of thinking
skills. This module will provide you with guidelines in using a number of strategies for assessing
thinking skills.
In developing strategies for assessing thinking skills, you will:
• identify professional considerations in assessing learning;
• develop multiple-choice strategies;
• develop matching strategies;
• develop open response strategies;
• develop a comprehensive testing schedule.

OLDS COLLEGE
ISWS 530

B-13
 ¾ Identify professional
Professional Considerations
considerations in
assessing learning.
Writing Test Items
Develop multiple-choice
strategies. Writing test items is one of the most demanding tasks faced by an
Develop matching instructor. There are so many considerations: choosing appropriate
strategies. item formats based on meeting instructional objectives; obtaining
Develop open response necessary copyright permission; avoiding stereotypes or bias; and
strategies. simply writing clear, first-rate items that allow you to evaluate student
Develop a comprehensive learning.
testing schedule.

Legal and Ethical Considerations

When you are formulating your items, certain legal and ethical consid-
erations must be kept in mind. They are copyright, stereotyping, and
bias.

Copyright
In the course of preparing for a class or lecture, most teachers have
found, in a text or other print source, a chart, diagram, or paragraph
that perfectly illustrates a point they wish to make. And problems
taken directly from texts have been known to appear in the class-
room as test items. It’s important to realize that you must get written
permission to use any copyrighted materials. By paraphrasing words
or ideas or taking the time to redraw a diagram, you can usually
avoid having to pursue the time-consuming job of obtaining written
permission.
If you feel you must “use” copyrighted material such as a diagram or
test item from another source, you must get permission from the
author or publisher. To do this, include all pertinent publication infor-
mation in your request. Include in your request the title, author, edi-
tion, publisher, etc., plus the page number(s) of the material you wish
to copy. Photocopying this information from the front-matter pages of
a resource is a good way to be sure that you have accurate publica-
tion information.

Stereotyping and Bias

There’s no place for stereotyping and bias in education today. It is
easy to avoid sexual, racial, or age bias and stereotyping in test
terms.
• Make equal reference to both females and males. Use plural
pronouns whenever possible or use her or him, his or her, she
or he.
OLDS COLLEGE
ISWS 530

B-14
 • Stay away from stereotypes such as female secretaries and
male bosses, etc. If you create examples and characters, show
females as well as males in the supervisory roles and males as
well as females in the supporting ones.
• Think twice before using sexist titles like fireman (use fire
fighter) and foreman (supervisor) or sexist language like man-
hours (work hours) and man-made (manufactured), etc.
When you select or prepare illustrations, include people of all races
and ages as well as men and women and, again, don’t fall into using
stereotypes. We all should be getting used to seeing women as con-
struction workers and welders and fire fighters, or men as nurses and
secretaries and day care workers.

Determining Appropriate Test and Item Types

It is important to realize that different types of test items are needed
to meet different testing needs. An effective test should include items
that test the three basic kinds of learning generally required to meet
the course objectives: cognitive learning (recalling, applying, and ana-
lyzing information), psychomotor skills (requiring muscular movement
and coordination), and affective learning (acquiring a change in out-
looks and attitudes towards learning). Generally all three areas can be
tested effectively using written tests (to test cognitive and affective
learning) and/or performance tests (to test psychomotor and affective
learning).

Task Analysis
If a test is valid, it must measure what it is intended to measure. As
you begin to write test items to measure the examinee’s knowledge
related to a particular task, you may realize that there are many
different knowledge components that are directly or indirectly related
to that task. How do you decide which of these should be the basis
for your test items? A procedure called task analysis will help you
to decide this. This procedure is used to identify the supporting
knowledge necessary to accomplish a task.
Instructional analysis consists of identifying what the student must
know in order to perform a competency. Further, through analysis,
learning is broken down into measurable tasks. These tasks should be
written as complete sentences, which describe as specifically as pos-
sible exactly what the student must know. These sentences begin with
an active verb similar to those listed on the following page.

OLDS COLLEGE
ISWS 530

B-15
 Recall/Recognition Level Application Level
arrange analyze distinguish
define appraise estimate
identify apply evaluate
label assess judge
list calculate measure
name choose rate
sequence classify revise
state compare score
diagram select
differentiate solve
discriminate value
The more specific these statements are, the easier it will be to write
test items. The statements should describe skills that the student must
be able to perform. This is important for test validity.

Written Test Items

“Selection” items probably are the most common test items. These
items, for which the student must select the correct answer from a
list of possible answers, include multiple choice, matching and true-
false. The other most popular items are open response items. These
may be either short-answer or essay items, for which the student
must supply the correct answer.

Levels of Written Test Items

It is important to tie each item to a specific learning task. In this
way, actual learning can be measured and evaluated accurately.
Some objectives involve simple recall of information. These would
include knowing names and labels, facts, terminology, definitions,
procedures and methods, or rules. These items often include words
such as: state, name, label, or identify.
More difficult objectives involve the application of knowledge. These
might require using a rule or formula to solve a problem; interpreting
charts, graphs, or diagrams; applying rules to specific situations or
conditions; solving word problems; or analyzing, interpreting, evaluat-
ing, or synthesizing material. These items are used to test applications
of knowledge such as discriminate, classify, apply, or solve.

OLDS COLLEGE
ISWS 530

B-16
Selection Items
Although there are different, specific rules and guidelines to consider
when writing each type of item, a few basic guidelines apply to all:
• Be certain that each item tests an important skill and not a
trivial fact.
• State each item in clear, unambiguous language.
• Keep vocabulary and sentence structure as simple and direct
as possible.
• Use positive statements. Avoid using negative words like
except, no, or not, which may cause unnecessary confusion.
• Be sure that the key (correct answer) is one on which all
experts would agree.
• Be sure that items are appropriate to measure the objective
being assessed - simpler items for lower-level objectives, and
more difficult items for instructional objectives that require
higher-level thinking.

Multiple Choice Test Items According to Bloom's

Taxonomy
Know (Recall):
The pollen in a flower is produced in the:
A. Stigma
B. Anther
C. Sepal
D. Ovary

Comprehend:
Fertilization of a flowering plant occurs when:
A. A bee enters a flower to find nectar
B. A bee picks up pollen and carries it to another flower
C. A pollen grain grows a pollen tube and unites with an ovule
D. A seed sprouts and grows into a seedling

OLDS COLLEGE
ISWS 530

B-17
 Apply:
To ensure good clover production, your crop should be situated next
to a:
A. Hive of honey bees
B. Herd of dairy cattle
C. Bird sanctuary
D. Wheat crop

Analyze:
You have settled in your desert home, and would like a few flowers
to decorate your front yard. What type of plants would you select?
A. Plants with sturdy stems and large blossoms
B. Plants that climb on larger plants
C. Plants that float on the surface of a pond
D. Plants that mature quickly and produce bright blossoms

What action should Perry take?

A. Ask the present customer to wait while Perry prepares the
order for the mayor's spouse
B. Prepare the order for the mayor's spouse without any further
comment
C. Inform the mayor's spouse that the order will be prepared as
time permits
D. Suggest to the mayor's spouse to go back to the other shop.

Self Check B2-1

1. What cognitive levels of learning can be assessed with multiple-
choice questions?

application, analysis
1. Knowledge, comprehension,

Self Check Answer B2-1:

OLDS COLLEGE
ISWS 530

B-18
Strategies for Writing Multiple- Identify professional

Choice considerations in
assessing learning.

¾ Develop multiple-choice
Multiple-choice items require that the student select a correct answer strategies.
from several (usually four) options that appear to be plausible. Most
Develop matching
multiple-choice items have two major parts: the stem, which poses strategies.
the question, and the options, which provide answer choices. The Develop open response
correct answer is called the key, while the three incorrect choices are strategies.
called distractors. In addition, some multiple-choice items include a Develop a comprehensive
display, or picture, on which one or more questions are based. testing schedule.

An advantage of multiple-choice items is that the same question can

be varied in several different ways. This can make the writing of test
items a creative and challenging enterprise. Below are four simple
formulas that are for the same piece of information.
Example 1
What type of fruit is an apple?
A. Aggregate
B. Berry
C. Drupe
D. Pome

Example 2

What type of fruit does the picture represent?

A. Aggregate
B. Berry
C. Drupe
D. Pome

OLDS COLLEGE
ISWS 530

B-19
 Example 3

The type of fruit pictured above is

A. An aggregate
B. A berry
C. A drupe
D. A pome

Example 4
Soybean: Legume: Apple: __________
A. Aggregate
B. Berry
C. Drupe
D. Pome

If multiple-choice items are well written, guessing is less likely to

occur than with true-false questions. Multiple-choice options are
versatile in that they can be used to measure both simple recall and
more complex application-type questions.
In addition, the following rules and suggestions will help you develop
effective, high-quality multiple-choice items.

OLDS COLLEGE
ISWS 530

B-20
Criteria for Writing Multiple Choice Items

1. Write a Complete Stem

2. Be Clear and Concise
3. Write Parallel Options
4. Make Sure that All Responses Are Plausible
5. Do Not Give Clues to the Answer
6. Use Positive Words
7. Create a Logical Order in the Responses
8. Use Current Information
9. Do Not Use Absolutes
10. Do Not Use "All of the Above" or "None of the Above."

1. Write a Complete Stem

Write a stem that clearly formulates the problem. If possible, make
the stem a complete sentence with a question mark at the end. If an
incomplete statement format is used, make the stem as near a com-
plete statement as possible. The stem should be complete enough so
that a knowledgeable person could fill in the correct answer without
seeing the options.
Example 1
Poor: Admission means
A. The consent of a person in authority
B. The act of entering
C. The judgment or approval of someone else
D. The act of self-disclosure.

Better: What is the meaning of the term admission as used in

criminal law?
A. The consent of a person in authority
B. The act of entering
C. The judgment or approval of someone else
D. The authority to act as agent for another

OLDS COLLEGE
ISWS 530

B-21
 Example 2
Poor: On a third orthographic projection, the front view is
A. Above the side view
B. Below the side view
C. Above the plan view
D. Below the plan view
Better: On a third angle orthographic projection, where is the
front view drawn?
A. Above the side view
B. Below the side view
C. Above the plan view
D. Below the plan view

Example 3
Poor: Sections:
A. are used when the drawing is too large
B. show complicated interior detail
C. indicate where imaginary cutting takes place
D. indicate the surface that has been cut
Better: When drawing sketches, sections are used to
A. Draw small portions at a time
B. Show complicated interior detail
C. Indicate where imaginary cutting takes place
D. Indicate the surface that has been cut

OLDS COLLEGE
ISWS 530

B-22
2. Be Clear and Concise
The stem should be stated clearly and unambiguously, using language
that is as simple and direct as possible. Many questions with difficult
constructions or big words are testing vocabulary and reading com-
prehension rather than content. The stem should also be stated con-
cisely, using only information pertinent to the question.
Example 1
Poor: What term is utilized to describe the body’s discoloration
caused by the gravitational pull of the blood through the
circulatory system where blood remains trapped in the
lowest part of the body?
A. Arterial degeneration
B. Levitation
C. Lividity
D. Rigor mortis
Better: What term describes the tissue discoloration due to blood
which is trapped by gravity in the lowest part of the body?
A. Arterial degeneration
B. Levitation
C. Lividity
D. Rigor mortis

Example 2
Poor: Kerosene is a substance which can be used for several
purposes, one of which is killing head lice. The more
common uses of kerosene include
A. Fuel and paint dilution
B. Solvent and brake fluid
C. Paint dilution, solvent and cleaning fluid
D. Fuel, solvent and brake fluid
Better: The most common uses of kerosene are as a
A. Fuel and paint dilution
B. Solvent and brake fluid
C. Paint diluter and cleaning fluid
D. Fuel and brake fluid
OLDS COLLEGE
ISWS 530

B-23
 Example 3
Poor: An electrical current is defined as
A. The flow of watts
B. The flow of volts
C. The flow of resistance
D. The flow of electrons
Better: An electrical current is defined as the flow of
A. Watts
B. Volts
C. Resistance
D. Electrons

3. Write Parallel Options

Keep the response options brief and make them similar to each other
in length, grammar, and concept. Avoid the tendency to make the
correct answer consistently longer, or shorter, than distractors.
Example 1
Poor: What is the purpose of petty cash?
A. To tide employees over until payday
B. To make donations to charities
C. To finance the office Christmas party
D. To make small payments that generally are not handled
through regular accounting channels, such as mail
services
Better: The purpose of petty cash is to
A. Tide deserving employees over until payday
B. Make donations to work-related charities
C. Finance the office Christmas or New Year’s party
D. Make small payments not handled through accounting

OLDS COLLEGE
ISWS 530

B-24
Example 2
Poor: What is a protractor primarily used for?
A. Architectural drawings
B. Circle-forming unparallel in
C. Measuring angles grammar
D. To determine polygons
Better: A protractor is primarily used for
A. Determining polygons
B. Drawing lines
C. Forming circles
D. Measuring angles

Example 3
Poor: The computer language most often used in developing soft
ware is
A. COBOL
B. FORTRAN unparallel in
C. French concept
D. Japanese
Better: The computer language most often used in developing soft
ware is
A. BASIC
B. COBOL
C. FORTRAN
D. PASCAL

OLDS COLLEGE
ISWS 530

B-25
 Example 4
Poor: Unlike a belt sander, an oscillating (finishing) sander:
A. Uses only fine grade abrasives
B. Is moved in wide overlapping circles
C. Is always moved with the grain
D. Is stopped before you lift off the sanded surface
Better: How is an oscillating (finishing) sander handled?
A. Started after being placed on the sanded surface
B. Moved in wide overlapping circles
C. Always moved with the grain
D. Stopped before you lift it off the sanded surface

4. Make Sure that All Responses Are Plausible

All of the options should be plausible. Good distractors are those that
are appropriate and represent common errors.
Example 1
Poor: Which of the following openly advocate blackmail,
kidnapping, theft, bank robbery, and intimidation as
primary methods of accomplishing goals?
A. Hippies
B. Investment counselors
C. Political parties
D. Terrorists
Better: Which of the following openly advocate blackmail,
kidnapping, theft, bank robbery, and intimidation as
primary methods of accomplishing goals?
A. Dictatorships
B. The IRA
C. The Mafia
D. Terrorists

OLDS COLLEGE
ISWS 530

B-26
Example 2
Poor: Subtract 9,043
- 6,754
A. 2,289
B. 2,288
C. 2,378
D. 3,378

Better: Subtract 9,043

- 6,754
A. 2,289
B. 2,389 (failing to change 0 to 9)
C. 3,399 (failing to decrease the two digits borrowed
from)
D. 3,711 (subtracting the big number from the small one)

Example 3
Poor: In mechanical terms, work means:
A. Lots of blood, sweat and tears
B. 99% perspiration, 1% inspiration
C. The application of a force through a distance
D. The exertion of force with no movement
Better: In mechanical terms, work means
A. Heat is produced as a by-product
B. Friction must be overcome
C. The application of a force through a distance
D. the exertion of force with no movement

OLDS COLLEGE
ISWS 530

B-27
 5. Do Not Give Clues to the Answer
Avoid giving unintentional clues to the correct answer, for example,
“klang” associations (using a word that provides a clue both in the
stem and in the key), long correct answers, or singular or plural
words or a or an used in such a way as to show certain responses
to be correct or incorrect.
Example 1
Poor: What is the best way to locate a lawyer for assistance?
A. Ask a police officer.
B. Call a bail bonding agency.
C. Contact a lawyer referral service.
D. Look for an ad in the phone book.
Better: What is the best way to locate a lawyer for assistance?

A. Ask a police officer.

B. Call a bail bonding agency.
C. Contact a legal referral service.
D. Look for an ad in the phone book.

Example 2
Poor: Which of the following organs is removed during a
splenectomy?
A. Liver
B. Spleen
C. Pancreas
D. Gallbladder
Better: Which of the following organs is removed during a
splenectomy?
A. The organ which detoxifies body wastes
B. The body's major blood filter and former
C. The producer of enzymes for digestion of proteins
D. The storage organ for bile

OLDS COLLEGE
ISWS 530

B-28
6. Use Positive Wording
Test takers often score lower on questions that have unexpected
negatives, than on questions that do not. Avoid words such as no,
not, and except.
Example 1
Poor: The metal not usually die cast is
A. Aluminum
B. Magnesium
C. Steel
D. Zinc
Better: The metal usually die cast is
A. Magnesium
B. Steel
C. Iron
D. Titanium

Example 2
Poor: The statement which cannot be used to explain the non-
corrosive property of stainless steel is
A. It contains almost no carbon
B. It forms a thin side layer which has a different volume
than the original metal
C. Chromium has been added
D. It forms a thin oxide layer which has the same volume
as the metal from which it was produced

Better: The reason stainless steel is non-corrosive is that

A. It contains very little carbon
B. It has a thin side layer of a different volume than the
original metal
C. Chromium has been added
D. It has a thin oxide layer of the same volume as the
original metal
OLDS COLLEGE
ISWS 530

B-29
 7. Create a Logical Order in the Responses
Placing the responses in some logical order eliminates guessing by the
test taker. Numerical options should be in numerical order, either
increasing or decreasing consistently. It is recommended that names or
single-word options be listed in alphabetical order.
Example 1
Poor: How many board feet (bd. ft.) are there in two pieces of
stock, each measuring 1" x 12" x 12"?
A. 2 bd. ft.
B. 1 bd. ft.
C. 1/2 bd. ft.
D. 1 1/2 bd. ft.
Better: How many board feet (bd. ft.) are there in two pieces of
stock, each measuring 1" x 12" x 12"?
A. 1/2 bd. ft.
B. 1 bd. ft.
C. 1 1/2 bd. ft.
D. 2 bd. ft.Example 2
Poor: The range of normal human body temperature in degrees
Centigrade is
A. 30-40
B. 72-80
C. 97.5-99
D. 36.5-37.5
Better: The range of normal human body temperature in degrees
Centigrade is
A. 30-40
B. 36.5-37.5
C. 72-80
D. 97.5-99

OLDS COLLEGE
ISWS 530

B-30
Example 3
Poor: The correct position for placement of a video display
terminal in relation to a person’s eyes is

A. well below eye level

B. slightly above eye level
C. well above eye level
D. at the same height as the eyes
Better: The correct position for placement of a video display
terminal in relation to a person’s eyes is
A. well below eye level
B. at the same height as the eyes
C. slightly above eye level
D. well above eye level

8. Use Current Information

Be certain that the content is accurate and up-to-date.
Example 1
Poor: Which of the following represents the most advanced
music technology currently on the market?
A. 45 rpm records
B. 33 1/3 rpm records
C. 8 track tapes
D. cassette tapes
Better: Which of the following represents the most advanced
music technology currently on the market?
A. cassette tapes
B. compact discs
C. laser discs
D. records

OLDS COLLEGE
ISWS 530

B-31
 9. Do Not Use Absolutes
Avoid using the words all, always, only, and never. These words
make guessing easier, especially when used in the responses. In
addition, experts will rarely agree that a task must always (or never)
be done a certain way. This disagreement will often invalidate a test
item.
Example 1
Poor: Minor discrepancies in the maintenance and upkeep of an
auto body shop
A. are acceptable
B. can be readily corrected
C. are always overlooked
D. will commonly be ignored
Better: Minor discrepancies in the maintenance and upkeep of an
auto body shop
A. are acceptable
B. can be readily corrected
C. are sometimes unimportant
D. will commonly be ignored
In the poor example, the “always” would have been a clue to most
test takers that it was not the correct answer. Attempting to change it
by substituting “sometimes” for “always,” however, would have made
it another correct answer. For that reason, the word “overlooked”
was replaced as well.

Example 2
Poor: Which of the following is characteristic of all plants?
A. They bear seeds
B. They contain non woody stems
C. They have multi-cellular parts
D. They produce chlorophyll

OLDS COLLEGE
ISWS 530

B-32
Better: Which of the following is characteristic of green plants?
A. They bear seeds
B. They contain non-woody stems
C. They have single cells
D. They produce chlorophyll

Example 3
Poor: A general service manual may include
A. information on all types of domestic cars
B. information on only one type of car
C. product bulletins
D. information on the types of cars by a particular
manufacturer
Better: A general service manual may include
A. specifications for the manufacture of a car part
B. detailed descriptions of the engineering of a car
C. product bulletins for the car manufacturer
D. maintenance specifications for the manufacturer’s cars

10. Do Not Use “All of the Above” or “None of the Above.”

Do not use none of the above and all of the above as response
options. These items have a tendency to be tricky or to provide
clues to students by making it easy to eliminate one or more op-
tions.
Example 1
Poor: What type of propagation is required for plants with corms?
A. division
B. grafting
C. layering
D. none of the above

OLDS COLLEGE
ISWS 530

B-33
 Better: What type of propagation is required for plants with
corms?
A. Division
B. Grafting
C. Layering
D. Separation

Example 2
Poor: The first officer at the scene of a crime should
A. Secure the scene
B. Render first aid
C. Both of the above
D. None of the above
Better: The first officer at the scene of a crime should
A. Render first aid, then secure the scene.
B. Search for suspects, then call for backup
C. Secure the scene, then question bystanders
D. Call for backup, then take license number

Example 3
Poor: Which of the following correctly represents the magnetic
laws of attraction and repulsion?
A. Like poles attract, unlike poles repel
B. Like poles repel, unlike poles attract
C. Like poles repel, unlike poles repel
D. None of the above

OLDS COLLEGE
ISWS 530

B-34
Better: Which of the following correctly represents the magnetic
laws of attraction and repulsion?
A. Like poles attract, unlike poles repel
B. Like poles repel, unlike poles attract
C. Like poles repel, unlike poles repel
D. Like poles attract, unlike poles attract

Self Check B2-2

1. Why is it important to write a complete stem for a multiple-choice
item?

Validating Test Items

After the item-writing team has reviewed all items for obvious flaws
and problems, the items undergo a validation process. This process
usually consists of an item review followed by pilot-testing. The two
main goals of the review are to assure (1) that items are well written
and adequately developed and (2) that items are consistent with
objectives. Appropriate revisions should be made based on observa-
tions after each review.

process as possible.
on the question as early in the
1. The learner must be able to focus

Self Check Answer B2-2:

OLDS COLLEGE
ISWS 530

B-35
 Checklist for Drafted Test Items
Check off problem areas only:
____ 1. The item measures the skill being assessed.
____ 2. The stem clearly presents the problem to be addressed.
____ 3. Negatives (eg. the word “not”) are avoided or clearly
emphasized.
____ 4. If the answer to the question presented by the stem is
controversial or a matter of opinion, the stem lists an
authority or source of opinion to be used in responding
to the question.
____ 5. The content of the item is accurate and up-to-date.
____ 6. Unless designed to measure rote memory, the item
presents a situation which is novel to the examinee; at
the very least, the item is not taken verbatim from
instructional materials.
____ 7. The item is written using fairly simple vocabulary and
sentence structure.
____ 8. Neither the stem nor options are made tricky by in
cluding an obscure word.
____ 9. Only one option is correct, and this option is marked.
____ 10. All or almost all content experts would agree on the
correct answer to this item.
____ 11. Each distractor is a plausible response to the stem.
____ 12. The options exclude repetitious information or words
that can be included in the stem.
____ 13. All options are grammatically consistent with the stem.
____ 14. All options use the same grammatical structure.
____ 15. All options are parallel in type of content.
____ 16. All options are approximately the same length.
____ 17. “All of the above” and “none of the above” are ex
cluded as options.
____ 18. Options are ordered alphabetically (or numerically),
unless another logical order is evident.

OLDS COLLEGE
ISWS 530

B-36
Matching Items Identify professional
considerations in
Like multiple-choice items, matching items may be used to measure assessing learning.
both simple recall and more complex applications. The student must Develop multiple-choice
match words or phrases with definitions, events, places, rules, or strategies.
concepts. Scoring is objective. This format allows for a large amount ¾ Develop matching
of content to be tested in a relatively simple, short manner; however, strategies.

good matching questions may be difficult to develop. Follow these Develop open response
strategies.
guidelines to help you develop good matching items.
Develop a comprehensive
1. State clearly in the directions how the matching should be done. testing schedule.
If items in a list may be used more than once, say so in the
directions.
2. Make each of the lists homogenous; that is, each list should
include words or phrases that are similar in form or type to each
other. It may be easier to do this if you give each list a heading,
such as terms and definitions or tools and uses.
3. Keep the lists relatively short - ten or fewer.
4. The two lists should NOT be of the same length. The list of
responses should be longer.
5. Arrange lists in some logical order, if one is feasible - chronologi-
cal, numerical, or alphabetical (for names, one-word responses,
etc.).
The following example illustrates several of these rules:
Example
Directions: Match the class of mail with its primary characteristic.
Write your answer in the blank beside the number.
Characteristic Class
____ 1. Newspapers, magazines, and other periodicals A. Airmail

____ 2. All sealed written material B. First class

____ 3. Circulars, books, catalogs C. Second class

____ 4. All mailable matter weighing 16 ounces or D. Third class

more not included in first or second class
E. Fourth class

(Key: 1. C, 2. B, 3. D, 4. E)

OLDS COLLEGE
ISWS 530

B-37
 Comparing Selection Items
The chart that follows will enable you to weigh some of the advan-
tages and disadvantages of the various selection item types as they
apply to the content you are testing.
Item Type Advantages
Multiple Choice 1. Subjective scoring is eliminated.
2. Effect of guessing is reduced,
compared to true/false items.
3. Versatile; ie, items can measure
simple recall or more complex
applications.
Item Type Advantages
True-False 1. Subjective scoring is eliminated.
2 Broad content can be covered in
a fairly brief format.
3. Judging truth of statements is
similar to real life situations.
4. Questions are easy to format.
Matching 1. Subjective scoring is eliminated.
2. Broad content can be covered in
minimal format.
3.Versatile; ie., words can be matched
with definitions, events, places,
concepts, etc.
OLDS COLLEGE
ISWS 530
B-38
Limitations
1. Good items are difficult to write.
2. Good options often are hard to find.
3. Longer response time by students is
required compared to true/false
items.
Limitations
1. Unambiguous items are hard to
write.
2. Guessing is encouraged.
3. Information often is limited to
simple recall.
4. Items often are used poorly to test trivia
or rote learning from a book.
1. Items sometimes are difficult to
develop.
2. Less convenient for machine
scoring.
Open Response Items Identify professional
considerations in
Open response items may require either short answers or longer, assessing learning.
essay length answers. The checklist for reviewing selection test items Develop multiple-choice
applies to supply-type items, as well. strategies.

Develop matching
Short Answer Items strategies.

¾ Develop open response

This type of item may be presented either as a direct question or as strategies.
an incomplete sentence.
Develop a comprehensive
testing schedule.
• Word the item so that it is unambiguous.
Examples
Poor: The Canadian Constitution was adopted in _________.
(Is the answer a city? a year?)
Better: The most recent version of the Canadian Constitution was
adopted in the year ______.
• Do not include too many blanks in one completion item.
Too many blanks can make the item impossible to answer.
Such an item is referred to as a “Swiss cheese
item.” Leave blanks for key words or phrases only.

Examples
Poor: The best way to _______ is to ________ before _______.
Better: The best way to prevent cavities is to brush your teeth
before ____________.
• If possible, place the answer blank at the end of the
statement. This makes the statement easier for the
learner to follow, and easier for the marker to score.
• Use standard-length blanks to avoid giving clues as to
the length of the correct answer.
• Do not use exact quotations from a textbook for
completion items. This practice encourages rote learning
and memorization.
• As in other types of items, avoid giving grammatical
clues such as a or an; instead, use a/an.

OLDS COLLEGE
ISWS 530

B-39
 Comparing Selection Items
The chart that follows will enable you to weigh some of the advan-
tages and disadvantages of the various selection item types as they
apply to the content you are testing.

Written-Response Items
Written-response items are best suited to subjects or objectives re-
quiring the ability to organize and assimilate material or to make
critical evaluations. Written-response items may be used to measure
understanding and problem-solving ability. Since written-response items
may require either brief or longer responses, clearly indicate in the
instructions whether a long or short response is expected. Most of
the general guidelines for writing test items apply to written response
items. Following is a general procedure to keep in mind while you
are developing these items.
• Have clearly in mind the objective you wish to measure.
• State a specific question that measures that objective.
Make sure the question is clear and not ambiguous. These
questions often start with terms such as describe, explain,
compare, analyze, why, interpret, etc.
• Let the student know the length of the response expected. For
example: "In one or two paragraphs, tell why ..." or "In two
pages or less, tell why ...".
• Do not make the question too comprehensive. It is pref-
erable to develop several specific questions to cover a
broad spectrum of material than to prepare only one that is
too wide-ranging. You will find that this will make
scoring easier to manage.
• Ask yourself: “Will this question bring out the informa-
tion that I want from the student?”
• Write a model answer to the question. Doing this may
give you insight into whether or not the question needs
to be revised or reworded to elicit the desired response.
This also will help you determine how to score the
answer; - decide which parts must be presented and how
many points to give for required and secondary informa-
tion.
required points.
grading by focusing on the
1. To remain objective during
Self Check B2-3
Self Check Answer B2-3: 1. Why is it important to make up a scoring key before grading
written response items?
OLDS COLLEGE
ISWS 530

B-40
Comprehensive Assessment Identify professional
considerations in
Review of learned skills has been shown to be critical for achieving assessing learning.

skills mastery. In light of this, the learning facilitator will need to Develop multiple-choice
strategies.
identify strategies to encourage review habits. A program of compre-
hensive assessment of learning provides an excellent opportunity to Develop matching
strategies.
address this need, whereby unit tests are constructed to include skills
Develop open response
from past units. The testing schedule for a given term may look strategies.
something like this:
¾ Develop a comprehensive
Unit 1 Unit 2 Unit 3 Unit 4 Unit 5 testing schedule.

Test 1 100%

Test 2 20% 80%

Test 3 5% 15% 80%

Test 4 5% 5% 10% 80%

Test 5 5% 5% 5% 5% 80%

The term “test” should not be misleading. It can refer to the total
assessment strategy for that period, including checklists and written
tests. The point is that the testing is cumulative, and provides a
structure for the constant review of skills, regardless of their domain.

OLDS COLLEGE
ISWS 530

B-41
 OLDS COLLEGE
ISWS 530

B-42