10/15/2021

Chapter 5: Stages of test development

1. Stating the problem
2. Writing specifications for the test
3. Writing and moderating items
4. Trialing of the test on native speakers
5. Trialing of the test on non-native speakers
6. Analyzing results of the trial & making of any
necessary changes
7. Calibrating scales
8. Validation
9. Writing handbooks for test takers, test users
and staff
10.Training staff

1. Stating the problem

 Test type
 Purpose
 Skills or abilities to test
 Backwash
 …
(See [1]: pp. 66-67)

2. Writing specifications
 Content
• Operations
e.g. A reading test
Operations: scanning, skimming
• Types of text
e.g. A writing test
Text types: letters, forms, essays
• Addressees
e.g. You are going to stay with your English-speaking
friend, Alex, for the weekend. Read Alex’s email
and the notes you made. Then write an email to
Alex, using all your notes.
 Addressee: a native speaker friend
• Topics

1
 10/15/2021

2. Writing specifications
 Format and timing
Approach 1
Task Subskills Item types No. of items Weighting Timing
1 scanning MC 10 20% 12 min.
2 … … … … …

Approach 2
Subskill Tasks Item types No. of items Weighting Timing
Scanning 1, 3 MC, T/F 15 40% 18 min.
Skimming … … … … …

2. Writing specifications
 Criterial levels of performance
e.g.,

2. Writing specifications
 Scoring procedure
• Objective testing: a detailed key
• Subjective testing: scorers? scorer training?

2
 10/15/2021

3. Writing and moderating items

 Sampling
• Choose widely and unpredictably from the whole area
of content
 Writing items
• Good items are consistent with test specifications
• Answer key should be provided
• Avoid misinterpretation and alternative responses
 Moderating items
• At least 01 moderator/colleague to examine test items
for fault and weaknesses

4. Trialing of items on native speakers

5. Trialing of items on non native speakers
 Often unfeasible due to test security
6. Analyzing results of the trial and making of any
necessary changes
7. Calibrating scales
 Collect samples of (oral/written) performance to
cover the full scales
 Assign each sample to a point on the relevant
scale
 Provide reference points for future uses and
training materials

8. Validation (criterion-related validity)

 High stake tests (e.g., proficiency tests,
graduation exams,…): essential
 Relatively low stake tests: not necessary
9. Writing handbooks for test takers, test users
and staff
10. Training staff

3
 10/15/2021

Study guide
1. Can you recall the stages of test development?
2. Can you write a specification?