Objective Test Guide

VVVVersion 1.2ersion 1.2ersion 1.2ersion 1.2AprilAprilAprilApril 2007200720072007

Guide toWriting Objective Tests

A guide to writing selected response questions and creating objective tests. Produced as part of the E-only project.

PagePagePagePage 1111GuideGuideGuideGuide to Writing Objective Tests v1.2to Writing Objective Tests v1.2to Writing Objective Tests v1.2to Writing Objective Tests v1.2PagePagePagePage 1111 Introduction to Selected Response QuestionsIntroduction to Selected Response QuestionsIntroduction to Selected Response QuestionsIntroduction to Selected Response Questions

INTRODUCTION TO SELEINTRODUCTION TO SELEINTRODUCTION TO SELEINTRODUCTION TO SELECTED RESPONSE QUESTICTED RESPONSE QUESTICTED RESPONSE QUESTICTED RESPONSE QUESTIONSONSONSONS

This guide was written as part of the E

-Only

project, which sought to develop SQA’sfirst (fully) online qualification. A package of support materials was developed as partof that project – and this document is part of that package.It was written after an extensive literature review of UK and international (particularlyUS) publications relating to objective testing. It is notnotnotnot a procedural guide for SQAappointees. Specific sectors and subjects will have their own procedures forproducing objective tests and this guide does not seek to replace that guidance.However, it does aim to describe best practice in the construction of objective testsfrom a generic perspective.This document has gone through a number of revisions since the first draft versionwas written in July 2006. Special thanks to everyone who took the time to contributethrough SQA Academy (http://www.sqaacademy.com). The document is being frequently revised. An online forum is available to discuss its contents at thefollowing URL, where the latest version of the Guide can be found:http://groups.google.com/group/objectivetesting

Bobby Elliott (bobby.elliott@sqa.org.uk) April 2007

PURPOSE OF THIS GUIDEAlthough most SQA units employ “conventional” assessment, some subject areas(mostly Science-related) have a tradition of using objective tests. For example,Biology uses multiple choice questions at Intermediate, Higher and Advanced Higherlevels; and HNC Computing uses an objective test as part of the Graded Unit.More recently, there has been greater emphasis on objective testing due to itssuitability for computer-based assessment; as a result, an increasing number of unitspecifications (at both National and Higher National levels) involve an element of objective testing.This guide will be of assistance to any SQA Officer (or appointee) involved in creating objective tests. It has three objectives:1.

to provide advice about the construction of objective questions;2.

to explain how to combine questions into an objective test;3.

to provide guidance on authoring items.A subsidiary objective is to standardise our vocabulary. Objective testing is atechnical area with lots of jargon – some of which is used inconsistently. This guideis the result of a wide-ranging literature review and seeks to harmonise ourterminology with that used internationally.

PagePagePagePage 2222GuideGuideGuideGuide to Writing Objective Tests v1.2to Writing Objective Tests v1.2to Writing Objective Tests v1.2to Writing Objective Tests v1.2PagePagePagePage 2222 Introduction to Selected Response QuestionsIntroduction to Selected Response QuestionsIntroduction to Selected Response QuestionsIntroduction to Selected Response QuestionsAlthough some topics (such as item banking) overlap with computer-assistedassessment (CAA), this guide focuses on the production of paper-based objectivetests – although much of the advice is directly transferable to a CAA environment.The document has seven sections:Section 1Section 1Section 1Section 1 Introduction to selected response questionsIntroduction to selected response questionsIntroduction to selected response questionsIntroduction to selected response questions (page 1(page 1(page 1(page 1))))Section 2Section 2Section 2Section 2 TypeTypeTypeTypessss of selecof selecof selecof selected response quted response quted response quted response questions (page 7estions (page 7estions (page 7estions (page 7))))Section 3Section 3Section 3Section 3 Choosing selected response questionsChoosing selected response questionsChoosing selected response questionsChoosing selected response questions (page 15(page 15(page 15(page 15))))Section 4Section 4Section 4Section 4 Writing multiple choice questionsWriting multiple choice questionsWriting multiple choice questionsWriting multiple choice questions (page 22(page 22(page 22(page 22))))Section 5Section 5Section 5Section 5 Writing questions for higherWriting questions for higherWriting questions for higherWriting questions for higher levellevellevellevel skillsskillsskillsskills (page 33(page 33(page 33(page 33))))Section 6Section 6Section 6Section 6 Item analysisItem analysisItem analysisItem analysis (page 40(page 40(page 40(page 40))))Section 7Section 7Section 7Section 7 Constructing testsConstructing testsConstructing testsConstructing tests (pa(pa(pa(pagegegege 44)44)44)44)Section 8Section 8Section 8Section 8 Dealing with guessing (page 52).Dealing with guessing (page 52).Dealing with guessing (page 52).Dealing with guessing (page 52).While the focus of this guide is objective questions, it does not seek to promote onetype of assessment over another. Traditional forms of assessment remain as validtoday as they have ever been – but, where appropriate, objective approaches have arole to play too. Neither does it seek to explain what you already know. Most SQAstaff have a good knowledge of objective testing - this guide simply seeks to providea single source of advice for busy Officers and appointees.There are no rules for writing objective tests; there’s only advice. Do whatever youthink is right for your particular test. Assessment is an art – not a science. There isno substitute for human judgement.MISCONCEPTIONS ABOUT OBJECTIVE TESTINGAlthough this document does not seek to promote one type of assessment overanother, it does aim to dispel commonly-held, but inaccurate, views about objectivetesting. Some of the most common misconceptions are rehearsed below.1.

Objective tests dumb-down education; objective tests are easy.

Objective testsare as “dumb” or “smart” as you choose to make them. Many high stakes tests(such as university medical examinations in the UK and the SAT in the UnitedStates) use objective tests.2.

Objective tests can only be used to assess basic knowledge.

While this is largelytrue in practice, there is nothing inherent in the design of objective tests to makethem unsuitable for assessing high level skills.3.

Objective tests encourage guessing.

The problem of guessing can be resolvedthrough one of a number of recognised techniques.4.

Writing an objective test is easy.

While most teachers can create simpleobjective tests, the construction of high quality objective questions is highlyskilled and requires significant knowledge and experience.