Assessing

Writing

I. Principles for assessing writing

II. Existing writing assessment methods
III. Comparison of traditional and
alternative assessment
IV. Sample elicitation and evaluation
V. References

by Cristina, Crystal, David, Rachel, Roy

91.12.04
 Principles

Reliability
• Give writing assignments before real tests
• Avoid unclear or too difficult instructions
• Give instructions as specific as possible
Validity
• Testing only writing ability, and nothing else
• Set as many tasks as is feasible
• Ensure samples long enough
Practicality--ensure ease of access (online writing)
Positive washback--give authentic writing tasks
Scoring
• Holistic scoring
• Analytic scoring
 Holistic Scoring

NS Native speaker standard

NS- Close to native speaker standard
MA Clearly more than adequate
MA- Possibly more than adequate
A Adequate for study at this university
D Doubtful
NA Clearly not adequate
FBA Far below adequacy
 Existing writing
assessment methods

Traditional Alternatives

1.Grammar and 1.Diary

structure 2.Online discussion
board
2.Control Writing
3.Dictocomps
3.Free Writing 4.Think-aloud writing
5.Portfolio
6.Peer evaluation
 Traditional

     I. Grammar and structure

a. Multiple-choice items
      b. Error-recognition items
      c. Re-arrangement
d. Changing words
e. Blank-filling
 Error-recognition

a. Choose the underlined words or phrases which

are incorrect.

e.g. 1. It was a terrible accident at an air show in

west a b c d
Germany yesterday.

2. One of the aircraft was crashed into two

other
a b c
aircraft during the display. etc
d
 Error-recognition

b. Students are told that there is a mistake in grammar in

each sentence and instructed to write the letter of that
part of the sentence in which it occurs.
e.g. 1. It was/ a terrible accident/ at an air show /
a b c
yesterday.
d
2. One of the aircraft/ was crashed into/ two
other
a b c
aircraft /during the display. etc
d
 Re-arrangement

e.g. 1.She was wearing a ……………………….

new/ jacket/ leather/ red / lovely

2.Susan ………………………………………
the phone/ when already/ rang / suddenly/
finished/ had
 Changing words

e.g. I said if I (1) to • (1)________________

discover the fire earlier, I • (2)________________
(2) to succeed in (3)to • (3)________________
extinguish it before it
• (4)________________
(4)to catch hold.
etc.
 Traditional

     I. Grammar and structure

a. Multiple-choice items
      b. Error-recognition items
      c. Re-arrangement
d. Changing words
e. Blank-filling
 Traditional

II. Control Writing

a.      Transformation
b.      Broken sentences
c.      Sentence and paragraph completion
d.      Notes
e.      Summary writing
f. L1-English translation
 Transformation

• Suitable for intermediate and advanced tests than

elementary tests.
a. Re-writing sentences according to certain patterns
e.g. 1.No other living plant grows so tall
and fast as bamboo.
Bamboo ______________________.
2.A stalk at Japan ‘s commonest bamboo has
grown over a meter in only 24 hours.
It has taken_____________________.
etc.
 Transformation

• b. Giving them a word in brackets and use the word to

rewrite the sentence.

1. It is necessary to answer all the questions.(need)

_______________________________________
2. His angry out burst was completely unexpected.(took)
_______________________________________

**weakness: lack of context.

 Broken sentences
Dear Cindy,
Thanks/ lot/ your letter/ which/ arrive/ yesterday.
1._________________________________________________
Congratulations/ pass/ your entrance exams.
2._________________________________________________
You/ work/ very/ hard/ past/year/ but/now/ it/ all/be worthwhile
3._________________________________________________
I wonder/ if / you/ like/ stay/ me/ few days.
4._________________________________________________
I look forward/ hear/you/ very soon.
5._________________________________________________
Best wishes

Cathy.
 Sentence and
paragraph completion

a. b.
Most of the students in A:__________________
my class were rather
lazy and did not enjoy B:So do I . I generally
the course. Pauline, watch for an hour or
however,___________ two every morning.
__________________ A:__________________
__________________ B: Immediately after I’ve
__________________ finished my homework
__________________ about eight.
 Traditional

II. Control Writing

a.      Transformation
b.      Broken sentences
c.      Sentence and paragraph completion
d.      Notes
e.      Summary writing
f. L1-English translation
 Free Writing

• Using Picture for • Choosing subjects :

(Avoid very general
Writing abstract topic)
• Essay Writing • Realistic writing task :
• Journal (Think about the likely
real life contexts in which
your students )
• Writing for purpose
• Specific instruction
• Clear, simple, and
realistic
 Using Picture
for Writing

You may want to use pictures to do any of the

following tasks in your test:
1.          to describe a scene, object or person
2.          to compare two scenes, etc.
3.          to tell a story
4.          to give instructions or directions
5.          to describe a process
Notes: 1. Cultural difficulty
2. AGE
 Essay and
Journal Writing

  1. The error count method

2. The analytic method
3. The impression (or multiple-marking)
method
 The Error
Count
Method

• The error count method involves counting

up the number of errors which a student has
made…and deducting this number from a
possible maximum total
Flaws:
1.Completely ignores content
2.…fails to take into account that some
errors are more serious than another
 The Analytic Method

5 4 3 2 1

Fluency O
Grammar O O
Vocabulary
Content O
Spelling O
 Comp 1 Comp 2 Comp 3 Comp 4
Marker A 2 1 4 2

Marker B 3 2 5 4

Marker C 3 1 3 1

Total 8 4 12 7
 Alternative

• Portfolio:
• Dictocom
• Discussion board
• Think aloud writing
• Peer Evaluation
• Diary
 Monday I have an exam today. I did not
study…
Tuesday Go swimming!
Wednesday The exam result was better than I
thought! I guess I’m just lucky…
Thursday
Friday I went to the see “Harry Potter”
with my GF
Saturday
Sunday Visit Uncle Ronny
Comparison Strengths Weaknesses

1.With many different kinds of tasks 1. Possibility of negative

2.Students practice a lot of similar tasks washback
and exercise before the real test
3.The objective of the assessment is easy
Traditional to be focused on
4.Both holistic scoring and analytic
scoring can apply to traditional
assessment

1.The stress on individual difference and 1.Time consuming

creativity can stimulate students’ 2.Lack of validity
washback more (ex. Dictocomps)
Alternative 3.Lack of practicality
(ex.Online discussion)
4. Difficulty in scoring
Comparison of Two Assessments Traditional Alternative

Set as many tasks as is feasible ˇ ˇ

Ensure samples long enough
Give writing assignments before real tests ˇ
Avoid unclear or too difficult instructions
Give instructions as specific as possible
(restrict test-takers)
Test only writing ability, and nothing else ˇ
Practicality—ensure ease of access ˇ
(online writing)
Positive washback ˇ
Holistic scoring ˇ
 Strength:
A. Reliability— Students can practice many similar
tasks before the real test
B. Validity— Students know what to write from
Chinese sentences

Weakness:
Validity– Not only writing process but also concept
transference are involved in translation
tasks
 Sample

Portfolio in assessing writing

Portfolio assessment is a cumulative and ongoing collection of entries
that are selected and commented on by the students, the teacher and/or
peers, to assess the student’s progress in the development of a competency
1. Positive Washback
A. Teachers' individual teaching

B. Student’s individual improvement

(1) Writing to learn
(2) Writing for self-realization
2. Reliability
3. Validity—possibility of lacking validity
4. Scoring—difficulty in scoring
5. Time consuming
 References

• Brown, H. Douglas. Teaching by Principles: An

Interactive Approach to Language Pedagogy.
2nd ed. New York: Longman, 2001.
• Harris, David P. Testing English as a Second
Language. USA: McGraw-Hill, 1969.
• Heaton, J. B. Classroom Testing. New York:
Longman, 1990.
• Hughes, Arthur. Testing for Language
Teachers. New York: Cambridge, 1989.