Writing Test Blueprints and Test Items

Slide 1
Writing Test Blueprints and Test

Items
“Perception software is used
to create and deliver
assessments with powerful
reports”
Eric Shepherd
Copyright © 1995-2004 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Slide 2
Objectives
At the end of this presentation, you will
understand how to:
Document the test blueprint

Survey the test blueprint
Develop test items
12/08/21 2
Slide 3
Best Practices
for Developing Tests
1.Analyze the job
2.Document the test blueprint
3.Survey the blueprint
4.Develop items
5.Alpha test and revise items
6.Beta test the exam
7.Select items and set the cut-score
8.Go live with the test
Slide 4
Sample Work Process

for Writing Test Objectives
 Define objectives in behavioral terms
 Action verb
 Condition(s)
 Standard(s)
 Use concrete verbiage
 Meet key criteria
Slide 5
How Do You Document

the Test Blueprint?
 Major test objectives or topics (job tasks)
 Subtopics (subtasks) within each major
performance area
 Rank test objectives and topics (tasks) in order
of importance
Slide 6
How Do You Survey

the Test Blueprint?
 Select contributors
 Re-prioritize tasks (objectives)
 Query time spent
Slide 7
Question Writing Process

 Start from your surveyed blueprint
 Select a level of difficulty guideline
 Write question stems
 Write correct answers
 Write distractors
 Edit
Slide 8
What Does a Level of Difficulty

Guideline Give You?
 Consistency
 Set performance standards
 Yardstick to gauge level of difficulty
Slide 9
Level of Difficulty Guideline
 Evaluation
 Synthesis
 Analysis
 Application
 Comprehension
 Knowledge
Slide 10
Knowledge and Comprehension
Evaluation
Synthesis
Analysis
Application
 Comprehension - explain, predict, paraphrase
 Knowledge - define, label, list
Slide 11
Application and Analysis
Evaluation
Synthesis
 Analysis – compare, contrast, relate
 Application – apply, solve, compute
Comprehension
Knowledge
Slide 12
Synthesis and Evaluation
 Evaluation – appraise, evaluate, judge

 Synthesis – formulate, construct, create
Analysis
Application
Comprehension
Knowledge
Slide 13
Apply Appropriate
Test Standards
 Formal standards
 Administrative policies
 Item writing guidelines
Slide 14
Question (Response) Types

Perception offers eight question types:
 Multiple choice
 True/false
 Multiple response
 Numeric item
 Word response
 Selection item
 Matrix item
 Hot spot item
Slide 15
Preferred Item Types
 Easily understood
 Minimize guessing
 Avoid testing trivial information
 Avoid candidate scrolling
Slide 16
How to Write Stems

 Present a single, definitive problem
 Use a question format
 Objective congruence
 Appropriate level of difficulty
 Active voice
Slide 17
What to Avoid in Stems
 Questions that build

 Negatively phrased questions
 “Always” and “Never”
 “Corporate-centric” questions
 Humor
 Trick questions
 “Trainer-centric” questions
Slide 18
Guidelines to Write Options

 Comparable in content, length, & logic
 Non-repeating
 Parallel language
 Logically or numerically ordered
 Avoid “All of the above” and “None of the above”
 Plausible distractors
Slide 19
Avoid Grammatical Clues

Option A is the only option that makes grammatical
sense.
The best way to increase the reliability of a test is to:

A. Increase the test length. [Correct]
B. Removing poor quality items.
C. Test should be readable for all test takers.
Slide 20
Avoid Association Clues

Option B includes the word “seven.”
Who are the “Magnificent Seven?”
A. A pro basketball team [Correct]

B. A group of seven fictional, heroic, Western
figures [Correct]
C. A protest group made famous during the
1960s
D. A vegetarian rock group
Slide 21
Order Options Numerically

What is the cost of an item that
normally sells for 9.99 USD that is
discounted 25%?
Wrong Right
A. 5.00 A. 2.50
B. 7.50 B. 5.00
C. 2.50
C. 6.66
D. 6.66
D. 7.50
Slide 22
Order Options Logically

What are the three important concerns
in achievement testing?
V=validity R=reliability E=efficiency O=objectivity
Wrong Right
A. V, R, E A. V, R, O
B. O, V, R
B. V, R, E
C. R, O, E
D. E, O, V
C. V, E, O
D. R, E, O
Slide 23
Evolving the Test Blueprint

 Objective statement by item
 Item naming conventions
 Item difficulty or weight
Slide 24
Thank you for your attention

Thank you for your time and attention. We hope you have enjoyed learning about
how to write test blueprints and test items.
We at Questionmark strive to improve our products and services. To let us know how
you felt about this presentation, visit our web site and complete the Learning Cafe
Survey: http://www.questionmark.com/us/surveys.htm
Questionmark provides numerous presentations to help our customers, and
prospective customers, successfully deploy computerized assessments. We
provide public “web-seminars” that use a combination of conference calls and web
technology to discuss computerized assessment issues. You can register for a
public seminar by going to: www.questionmark.com/us/seminars/
For material that you might find useful, please visit our Learning Café at:
www.questionmark.com/learningcafe/

Writing Test Blueprints and Test Items

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Writing Test Blueprints and Test Items

Uploaded by

Copyright:

Available Formats

Slide 1

Writing Test Blueprints and Test

Document the test blueprint

Sample Work Process

How Do You Document

How Do You Survey

Question Writing Process

What Does a Level of Difficulty

Level of Difficulty Guideline

Knowledge and Comprehension

Application and Analysis

Synthesis and Evaluation

 Evaluation – appraise, evaluate, judge

Question (Response) Types

Preferred Item Types

How to Write Stems

What to Avoid in Stems

 Questions that build

Guidelines to Write Options

Avoid Grammatical Clues

The best way to increase the reliability of a test is to:

Avoid Association Clues

Who are the “Magnificent Seven?”

A. A pro basketball team [Correct]

Order Options Numerically

Order Options Logically

Evolving the Test Blueprint

Thank you for your attention

You might also like