TESTING WRITING

Rio Darmasetiawan

69100077
 TESTING PROBLEM

We have to set writing tasks that are properly representative of

the population of tasks
The tasks shoudl elicit valid samples of writing

It is essential that the samples of writing can and will be scored
validly and reliably
REPRESENTATIVE TASKS

Specify all possible content

In order to judge whether the tasks we set are representative of the

tasks that we expect students to be able to perform, we have to be
clear at the outset just what these tasks are that they should be
able to perform.
REPRESENTATIVE TASKS

Include a representative sample of the specified content

The more tasks (within reason) that we set, the more representative
of a candidate's ability (the more valid) will be the totality of the
samples (of the candidate's ability) we obtain.
 E L I C I T A VA L I D S A M P L E O F
WRITING ABILITY

Set as many separate tasks as is feasible

We have to offer candidates as many 'fresh starts' as possible, and

each task can represent a fresh start. By doing this, we will achieve
greater reliability and so greater validity
 E L I C I T A VA L I D S A M P L E O F
WRITING ABILITY

Test only writing ability and nothing else

This advice assumes that we do not want to test anything other than
the ability to write. Therefore, for the sake of validity, we should
not set tasks which measure other abilities such as creative,
imaginative, etc.
 E L I C I T A VA L I D S A M P L E O F
WRITING ABILITY

Restrict candidates

Writing tasks should be well defined: candidates should know

what is required of them, and they should not be allowed to go too
far astray. One last thing to say about tasks is that they should not
only fit well with the specifications, but they should also be made
as authentic as possible
E N S U R E VA L I D A N D R E L I A B L E
SCORING

Set tasks which can be reliably scored

• Set as many tasks as possible
The more scores for each candidate, the more reliable should be the
total score.
• Restrict candidates
The greater the restrictions imposed on the candidates, the more
directly comparable will be the performances of different candidates
• Give no choice of tasks
Making the candidates perform all tasks also makes
comparisons between candidates easier
• Ensure long enough samples
The samples of writing that are elicited have to be long enough
for judgments to be made reliably
E N S U R E VA L I D A N D R E L I A B L E
SCORING

Create appropriate scales for scoring

• Holistic scoring
Holistic scoring involves the assignment of a single score to a
piece of writing on the basis of an overall impression of it. This
kind of scoring has the advantage of being rapid.
• Analytic scoring
Methods of scoring which require a separate score for each of a
number of aspects of a task are said to be analytic.
E N S U R E VA L I D A N D R E L I A B L E
SCORING

Calibrate the scale to be used

Any scales which is to be used should first be calibrated, meaning

collecting samples of performance collected under test conditions, and
covering the full range of the scales.
 Select and train scorers

They should be sensitive to language, have had experience of teaching

writing and marking written work. It is also helpful if they have had
training in testing.
E N S U R E VA L I D A N D R E L I A B L E
SCORING

Follow acceptable scoring procedures

Each task of each student should be scored independently by two or more

scorers (as many scorers as possible should be involved in the assessment
of each student's work. Senior member of the team should collate scores
and identify discrepancies in scores awarded to the same piece of writing.
Nevertheless, once scoring is completed, It is useful to carry out simple
statistical analyses to discover if anyone's scoring is unacceptably
aberrant