Chapter 8: Assessment of Speaking Skills

Chapter 8: ASSESSMENT OF SPEAKING SKILLS

Upon completion of this chapter, you should be able to:



Explain what to assess


Discuss how to assess speaking skills


Explain how to use rubrics to assess speaking skills
Describe to use interviews to assess speaking

8.1 Preamble
8.2 Evaluating speaking
8.3 What to assess?
8.4 How to assess speaking?
8.5 Using rubrics to assess presentations
8.6 Assessing speaking in an interview
8.7 Structured speaking tasks

Summary
References

C
Chapter 1: Introduction to Listening
Chapter 2: Teaching Listening
Chapter 3: Listening Activities
Chapter 4: Assessing Listening Skills
5 Chapter 5: Introduction to Speaking
Chapter 6: Teaching Speaking
Chapter 7: Speaking Activities
Chapter 8: Assessing Speaking Skills

1
 Chapter 8: Assessment of Speaking Skills

8.1 PREAMBLE

In Chapter 6 and 7, we discussed several methods and activities for teaching speaking in the
ESL classroom. In this chapter, we will examine how to assess speaking in the classroom.
Among the issues that will be addressed are as follows:



Is accuracy of pronunciation and grammar important?


Is each student expected to speak for about the same amount of time?


Is it acceptable if a speaker uses many long pauses and repetitions?
If a speaker’s contribution to a discussion is off topic, does it matter?

As the above questions illustrate, the types of criteria we use to assess a speaker’s oral
performance during a classroom activity will depend on which kind of talk we are talking
about and the kind of classroom activity we are using. For example, a speaking activity that
requires talk as performance (e.g., a mini-lecture) would require very different assessment
criteria. These might include: Clarity of presentation: i.e., the extent to which the speaker
organises information in an easily comprehensible order; use of discourse markers, repetition,
and stress to emphasize important points and to make the lecture structure more salient to the
listeners.

8.2 Evaluating Speaking

Assessment of speaking skills often lags far behind. Most teachers would accept that ‘if you
want to encourage oral ability, then test oral ability’ (Hughes, 1989:44). But the problems of
testing oral ability make teachers either reluctant to take it on or lacking in any confidence in
the validity of their assessments. Such problems include: the practical problem of finding the
time, the facilities and the personnel for testing oral ability; the problem of designing
productive and relevant speaking tasks; and the problem of being consistent (on different
occasions, with different examiners and between different assessors). Another problem, is
deciding which criteria to use in making an assessment. Different speaking activities such as
conversations, group discussions, and speeches make different types of demands on learners.
They require different kinds and levels of preparation and support, and different criteria must
be used to assess how well students carry them out. Some teachers argue that during the
speaking assessments, the focus should be on fluency and not accuracy. さWe just observe,
watch and listen to students and write down mistakes they madeざ.

Students who are actively involved in all speaking tasks and only use English in
class will get the highest score of three points for each class. Students who are
absent, inactive in speaking activities and use their native language will not get
the highest score for the day.
2
 Chapter 8: Assessment of Speaking Skills

Read this comment by Fernando Diez, teaching English to Spanish students:

If a student can speak in English, up to some point you can say that person manages
communicating in that language. I have worked with a lot of students helping them speak
English and have realised the following:

 It is important to take into account when evaluating speaking: the message the
learner conveys in English, in other words, the communication which is held among
two people. Don't get me wrong with what follows: "forget" about accuracy of
grammar. That person is trying to tell you something, and you are trying to say
something, he or she is trying to understand your ideas, your messages.

 Honestly most of my class-time is devoted to speaking, to connect people through

communication. The classes are entirely in English: the classroom is a special place,
say, an environment where English is the only language among us.

 I "don't understand Spanish": it's a funny situation in which I act out as though I not
understand Spanish. And it's their business to solve the problem of communicating
with me. I make faces when "I don't understand".

 A few times my new students have asked me what country I'm from. Whether I'm
Spanish like most of them. They are already in the classroom when I come in, waiting
for me.

 I don't care if the students have no idea of English. It's good for them to listen in
English. I use miming, eye-contact, the blackboard, drawings, examples, humour and
fun. Little by little, their grammar improves and their vocabulary expands and begin
to utter complete sentences, even to maintain conversations.

[source: Fernando M Díez, English Teacher in Granada, Spain]

http://fernandoexperiences.blogspot.com

8.3 What to Assess?

Pronunciation, vocabulary, accuracy, communication, interaction and fluency are all markers
of a student’s overall speaking abilities.

 Pronunciation is a basic quality of language learning. Though most second

language learners will never have the pronunciation of a native speaker, poor
3
Chapter 8: Assessment of Speaking Skills

pronunciation can obscure communication and prevent an ESL student from making
his meaning known. When evaluating the pronunciation of your students, listen for
clearly articulated words, appropriate pronunciations of unusual spellings, and
assimilation and contractions in suitable places. Also listen for intonation. Are
students using the correct inflection for the types of sentences they are saying? Do
they know that the inflection of a question is different from that of a statement? Listen
for these pronunciation skills and determine into which level your student falls.

Vocabulary: After noting your students’ pronunciation levels, move to vocabulary.

Vocabulary comprehension and vocabulary production are always two separate banks
of words in the mind of a speaker, native as well as second language. You should
encourage your students to have a large production vocabulary and an even
larger recognition vocabulary. For this reason it is helpful to evaluate your students
on the level of vocabulary they are able to produce. Are they using the specific
vocabulary you have instructed them in this semester? Are they using vocabulary
appropriate to the contexts in which they are speaking? Listen for the level of
vocabulary your students are able to produce without prompting and then decide how
well they are performing in this area.

Accuracy: Grammar has always been and forever will be an important issue in
foreign language study. Writing sentences correctly on a test, though, is not the same
as accurate spoken grammar. As your students speak, listen for the grammatical
structures and tools you have taught them. Are they able to use multiple tenses? Do
they have agreement? Is word order correct in the sentence? All these and more are
important grammatical issues, and an effective speaker will successfully include them
in his or her language.

Communication: A student may struggle with grammar and pronunciation, but how
creative is she when communicating with the language she knows? Assessing
communication in your students means looking at their creative use of the language
they do know to make their points understood. A student with a low level of
vocabulary and grammar may have excellent communication skills if she is able to
make you understand her, whereas an advanced student who is tied to manufactured
dialogues may not be able to be expressive with language and would therefore have
low communication skills. Don’t let a lack of language skill keep your students from
expressing themselves. The more creative they can be with language and the more
unique ways they can express themselves, the better their overall communication
skills will be.

Interaction: Being able to say what you mean with a foreign language is one thing,
being able to interact with others is another. Ask your students questions. Observe
how they speak to one another. Are they able to understand and answer questions?
Can they answer you when you ask them questions? Do they give appropriate
responses in a conversation? All these are elements of interaction and are necessary
for clear and effective communication in English. A student with effective interaction
skills will be able to answer questions and follow along with a conversation
happening around him. Great oratory skills will not get anyone very far if he or she
4
 Chapter 8: Assessment of Speaking Skills

cannot listen to other people and respond appropriately. Encourage your students to
listen as they speak and have appropriate responses to others in the
conversation.

Fluency: Fluency may be the easiest quality to judge in your students’ speaking.
How comfortable are they when they speak? How easily do the words come out? Are
there great pauses and gaps in the student’s speaking? If there are then your student is
struggling with fluency. Fluency does not improve at the same rate as other language
skills. You can have excellent grammar and still fail to be fluent. You want your
students to be at ease when they speak to you or other English speakers. Fluency is a
judgment of this ease of communication and is an important criterion when evaluating
speaking.

Students may excel in one and struggle in another, and not necessarily the ones you might
think. Help your student understand these qualities of effective speakers. Let your students
know that you will be listening for these qualities when you evaluate their progress and
encourage them to improve their English in these areas. Also, listen to them both when they
talk to you and when they talk with other students. They should be able to speak well with
one another. After all, most of the English they will use in the future will be with other non-
native speakers. Finally, remember that a true evaluation will take into consideration more
than just the oral interview on the final exam. Listen to your students throughout the
semester. Note how they improve in these areas. Encourage them as speakers and learners,
and you are sure to reap the benefits, too.

a) Do you agree ith the state e t forget a out a ura of gra ar he

assessi g speaki g skills ? E plai
b) What do ou fo us o he assessi g pro u iatio a d o a ular ?
c) E plai hat is ea t i tera tio a d flue he assessi g speaki g
skills of students.

8.4 How to Assess?

Teachers are often asked to evaluate learner progress during courses, maybe by preparing
progress tests. It can seem straightforward enough to test grammar or vocabulary with pen
and paper tests – but if our students’ work includes speaking – then it also seems necessary to
assess their speaking skills. Teachers often feel unsure as to how they could do this. Here are
some ideas.

5
 Chapter 8: Assessment of Speaking Skills

Too many students!

A frequent problem for teachers is when there are so many learners in one class that it seems
to make it unrealistic to assess speaking. With a list of criteria (such as those above) it now
becomes considerably more straightforward to assess even a large group. Explain to your
class what you will be doing, then, the next three or four times you set speaking tasks (i.e.
where learners work in pairs or groups), walk around class with a list of names, listening in to
various groups and noting successes, keeping track of individual 'can do’s'. Extend your
assessment over a few lessons; keep listening and adjusting your evaluation over a variety of
tasks.

Speaking tasks
What are possible speaking tasks for assessment? Well, almost anything you do in normal
class work – e.g. narrating a picture story; role-plays; pair work information gap exchanges;
discussions etc. If you have a smaller class and enough time then a “three learners with one
teacher” activity is a very good way to assess, i.e. setting a task that gets the three learners to
interact together while you watch and evaluate.

Self Assessment
Although fear of bad marks can sometimes be motivating, it’s surprising to find the amount
of power that students feel when assessing themselves. It can be a real awareness-raising
activity. Distribute a list of criteria and ask students to first write a short line comparing
themselves against each criterion (in English or in their own language) – a reflective view
rather than just a 'yes' or 'no'. Encourage 'guilt-free' honest reflection. After the writing stage,
learners can meet up in small groups and talk through their thoughts, explaining why they
wrote what they did.



Structured interview


Unstructured interview


Structured role-play


Structured role-play (information-gap)


Oral presentations


Retelling a story


Picture-cued story telling
Translation of an extended prose

8.5 Assessing Speaking Using a Scale

Most teachers will be familiar with the concept of grading with a rubric, a table with
different criteria and a grading scale. If you have never created a rubric before, it’s really
quite easy. Simply choose the criteria on which you will grade students and list them along
the left side of the page. Then create an even number of columns along the top of the page.
Four is the easiest to start with if this is your first rubric. These columns will represent
potential skill levels of your students. For each criterion, define what level of the ability a
6
 Chapter 8: Assessment of Speaking Skills

student at each of the four levels would exhibit. For example, the most straightforward way to
label the boxes on the rubric would be, “Meets expectations high, meets expectations low,
slightly underperforms, does not meet expectations.” The more rubrics you make, the more
detailed you can be in your descriptions. Then, as you evaluate each student, determine at
which level he or she is performing. Take the average level among the criteria and you have
an objective grade with suggestions for areas in which your student can improve.

Some scales to assess ‘speaking’ may use a more detailed ‘scale’. While students make their
presentations, the teacher can use a scale to assess their speaking skills. The scale could be in
the form a ‘rubric’ with ‘descriptors. Refer to the example of a scale given below which has a
5-point scale – 1 to 5. The scale measures the following components of speaking:



Interaction


Pronunciation


Fluency & coherence
Vocabulary & grammar.

Each number is given a descriptor which explain what elements of speaking should the
teacher or assessor be looking for.

5. 4. 3. 2. 1.

Can initiate & Interaction is Develops Contributions Contributions

logically develop adequately initiated interactions are often unrelated are mainly
simple and developed adequately makes to those of the unrelated to those
Interaction conversation on however minimal other speaker of other speaker
familiar topics effort to initiate
conversation Shows hardly
Can take turns Can take turn Needs constant Generally any initiative in the
appropriately but needs little prompting to take passive in the development of
prompting turns development of conversation
conversation
Very limited
interaction

5 4 3 2 1

Can pronounce Mostly correct Largely correct Frequently Insufficient

correctly & pronunciation & pronunciation & unintelligible accuracy in
articulate clearly clear articulation clear articulation articulations pronunciation;
except occasional many
Pronunciation Is always Can be clearly errors Frequent grammatically
errors
Some
comprehensible; understood most phonological

Communicatio
uses appropriate of the time; very errors
intonation few phonological expressions cause
errors stress without Major n is severely
compromising communication affected
with understanding problems
of spoken
discourse.

7
 Chapter 8: Assessment of Speaking Skills

5 4 3 2 1

Speaks fluently Speaks without Is willing to Usually fluent; Noticeably/lon

almost with no noticeable effort, speak at length, produces simple g pauses; rate of
repetition & with a little however repetition speech fluently, speech is slow
minimal hesitation repetition is noticeable but loses
coherence in Frequent
Develops topic Demonstrates Hesitates complex repetition and/or
Fluency & fully & coherently hesitation to find and/or self communication self correction
Coherence
Often hesitates
words or use corrects;
correct occasionally loses Links only
grammatical coherence and/or resorts to basic sentences;
structures and/or slow speech
Topics mainly Topics partly
breakdown of
self correction coherence evident

Topics not
developed, but developed; not
usually not always concluded
fully developed to logically concluded logically
merit

Can express Can express Communicates Communicates Demonstrates

with some with some with limited with limited almost no
flexibility and flexibility and flexibility and flexibility and flexibility, and
appropriacy on a appropriacy on appropriacy on appropriacy on mostly struggles
variety of topics most of the topics most of the topics some of the topics for appropriate
such as family, words
Vocabulary & hobbies, work, Demonstrates Sometimes Complex
Grammar travel and current ability to use uses complex forms and Uses very basic
events complex forms forms and sentence structures vocabulary to

frequently uses
and sentence sentence are rare; exhibits express viewpoints
structures most of structures; has limited vocabulary
complex forms the time; expresses limited vocabulary to express new
and sentence with adequate to/describe/expres ideas
structures; has vocabulary to s new points
enough vocabulary express
to express
himself/herself

The table below is a summary of the score obtained by each student on the FOUR criteria.

Name of Student Interaction Pronunciation Fluency & Vocabulary & Total

Coherence Grammar [max = 20]
Ali Seman 4 3 5 3
Wee Kong Lam 3 5 4 4
M. Sundram 4 4 3 5

8
 Chapter 8: Assessment of Speaking Skills

ANOTHER EXAMPLE OF A RUBRIC TO ASSESS SPEAKING:

Poor Fair Good Excellent

0 pts 1 pts 3 pts 5 pts

Poor Fair Good Excellent

All questions and Questions and Questions or Questions and

Clarity answers were answers were answers were answers were clear
awkward and awkward and awkward at times and
incomprehensible. incomprehensible but always comprehensible.
to understand at understandable.
times.
Poor Fair Good Excellent

Student's Student's Student's Student's

Pronunciation pronunciation was pronunciation pronunciation was pronunciation was
incomprehensible. made understandable like a native
understanding with some error. speaker.
difficult.

Poor Fair Good Excellent

Fluency Student was Student took a long Students were able Students were able
unable to ask or time to ask and to ask and answer to communicate
respond to respond to the questions with clearly with no
questions. questions. little difficulty. difficulty.

Poor Fair Good Excellent

Student was The student The student The student fully

Comprehension unable to showed little understood most of understood the
comprehend comprehension of what was asked of questions asked
questions. questions. him/her. and answered
Questions had to Questions had to correctly.
be repeated. be repeated.

Poor Fair Good Excellent

Content Did not ask Ask some Gave appropriate Gave appropriate
appropriate inappropriate questions for questions and
question for questions for survey information good content in
information, no information or but responses responses to
response to answered question were limited in questions.
question. with very limited content.
answers.

9
 Chapter 8: Assessment of Speaking Skills

a) Compare the criteria used in the two rubrics. Are they similar? Explain.
b) Compare the descriptors used in the two rubrics. To what extent are they
different?
c) Which rubric would you use in your class? Why?
d) Compare these rubrics with the rubrics suggested in the KSSR>
e) Design your own rubric for assessing speaking.

8.6 Assessing Speaking in an Interview

Instructions:
a) The speaking test will be conducted for two students at a time.
b) Students should not have pen, paper, mobile phone etc.
c) The teacher will be an interlocutor and an assessor both.

I. GENERAL INTRODUCTION - (1 min)

(The Interlocutor makes the students feel comfortable)

Interlocutor: Good morning, I am ----------------------. Hope you are looking forward

to this brief interaction.

Student A: Yes ----------------/ I am -------------------

Interlocutor: And what about your? (Looking at student B)

Student B: ---------------------------------
Interlocutor: What do you do in your spare time? (To Student B)

Student B: ------------------------------------
Interlocutor: And how about you? (To student A)

Student A: -----------
Interlocutor: What makes you special? (To student A)

Student A: -----------
Interlocutor: What about you? (To Student B)

Student B: -----------
Interlocutor: What do admire in other people? (To student A)

10
 Chapter 8: Assessment of Speaking Skills

Student A: -----------
Interlocutor: How about you? (To Student B)

Student B: ------------------------
Interlocutor: Thank you, this is the end of the first part of the test.

8.7 Structured Speaking Tasks

In assessing oral skills, we can use different test items according the relevant purposes or
target information to be tested. Luoma (2004) provides three frameworks about general
purposes of testing oral skills as “linguistically oriented, communication-oriented and
situation-based.”

A) Linguistically Oriented: vocabulary, grammar and pronunciation.

This purpose can be carried out by structured speaking tasks. They are called ‘structured’
because they control quite closely what the examinees are going to say. In reading aloud
and sentence repetition, the testers know exactly what the examinee will say, and, in
short-answer questions and reacting to phrases tasks, a short list of acceptable answers
can usually be specified. Structured speaking tasks are typically used to evaluate
linguistic features, particularly pronunciation and grammar. Short-answer questions
and reactions to phrases can also be used to evaluate overall understanding and
comprehensibility. Short-answer questions test comprehension of the questions and
ability to give relevant information in response. The difference between this and more
extended simulations is that the questions and answers are limited and all the information
needed for answering the questions is usually provided in the task materials. Reacting to
phrases is another structured task that is often used in tape-based tests. The task usually
tests the examinees’ knowledge of conventional politeness exchanges such as greetings,
thanks, apologies, expressions of agreement and polite disagreement, and so on.

B) Communication-Oriented:

The overall communication activity in the task such as telling a narrative or expressing
and defending an opinion, discussing factors that support the chosen opinion and argue
against others, comparing and contracting things through which advanced oral skills for
description is needed.

C) Situation-Based Task Design:

This belongs to the task-based approach to defining the test construct. This approach is
typically used in specific-purpose testing and in vocational and professional education.

11
Chapter 8: Assessment of Speaking Skills

Types of test items and their Functions:

Description tasks

Example 1—A one-to-one interview: Describe to me the room or area where you work.

Example 2—A pair task in an interview test (two pictures are provided):

Describe your pictures to each other and then talk about what is similar in your pictures
and what is different.

Narrative tasks

Example 3: A tape-based test (Six pictures should be provided)

Please look at the six pictures below. Tell the story based on these pictures starting from
picture number 1 and going through picture number 6. Take one minute to look at the
pictures.

The narrative is a monologue, and as the test is tape-based the examinees have to tell it in
one long stretch without any feedback from a listener.

Example 4: A face-to-face paired interaction test

You each have a set of pictures. Together they make a story. Each of you tells one part of
he story.

Narrative tasks are also frequently used in speaking tests. They show how well the
examinees can recount a sequence of events, usually in one time frame, either present or
past. Most often, the tasks are based on picture sequences, where the content of the
pictures guides what will be said. It is likely to create some interaction even though one
of the pupils is always the main speaker while the narrative is being constructed.
Information gap between the two speakers is therefore likely to occur. Also, the choice of
good sequences is a difficult matter.

Personal stories often reveal embarrassing details that speakers would be shy to discuss
in a test or, if not, they may be so uneventful that the speakers would consider them
unworthy to tell.

12
Chapter 8: Assessment of Speaking Skills

Instruction tasks

Example 5: A one-to-one interview

Imagine that we are standing in front of your house. Tell me how to get to the shop from
there.

Example 6: A face-to-face paired interaction test—Feeding the puppy

You cannot go home and your puppy needs to be fed. Your friend says he will do it. Tell
your partner exactly what to do, what he’ll need and where to find things. Follow the
instructions below. Fine what you need in the picture. Tasks—what you have to do; what
you need; where to find things.

The main purpose in giving directions and instructions is getting the message across the
making sure that it has been understood. This tends to mean short exchanges between the
speaker and the listener.

Comparing and contrasting tasks

Example 7: Interaction outline for a pair task in a paired interview

Candidate A compare and contrast two or three of these photographs, saying what kind of
clothing the people are wearing and why the protection might be necessary.

Explaining and predicting tasks

Example 8: A taped-based test

Explaining the contents of a graph or explaining a process is a fairly common task in

many professional and study settings. To do well on the task, the speakers need to set the
scene and identify parts of the information or stages in the process that they are
explaining and present them in coherent order. They also need to explain the significance
of the important parts or stages, so that the listeners understand what the explanation is
about and why it is the being given. Predictions go together well with explanation tasks,
and they can also be fairly monologic. As predictions involve speculation, they may
become more interactive in a face-to-face setting.

Decision tasks

Example 9: in a paired interview

Discuss and decide together: 1) what the advantages and disadvantages are of attending
trade fairs, for instance, and 2) which members of staff would most usefully represent a
company at a trade fair.

13
Chapter 8: Assessment of Speaking Skills

In speaking tests, the issues that need to be decided are usually not clear-cut, so that
arguments for and against different solutions are needed. The speakers express their
opinions about the concerns and justify them in order to air different viewpoints before
negotiating the conclusion.

Role-plays and simulations

Role-plays simulate different kinds of communication situations that the target group of
the test could plausibly meet outside the test. They can be completed between two
examinees or between an examinee and a tester.

Example 10: A job interview

The employer—inquiring information about candidate’s abilities, qualifications, and

Character

The candidate—providing information about himself and inquiring information about the
Company

Role-play tasks are a way of making communication in a test more versatile because,
rather than talking to a tester, the examinees take on a new role and a new, simulated role
relationship to their communication partner. Their performance shows their ability to
adapt to the requirements of the new role and situation. As long as the situation is relevant
for the target audience and the purpose of the test, this gives useful information for the
tester. The information may simply be a new perspective into the examinees’ linguistic
resources, or the use of different functions from other tasks in the test.

14
 Chapter 8: Assessment of Speaking Skills

SUMMARY

 The types of criteria we use to assess a speaker’s oral performance during a classroom
activity will depend on which kind of talk we are talking about and the kind of
classroom activity we are using.

 Assessment of speaking skills often lags far behind.

 Different speaking activities such as conversations, group discussions, and speeches

make different types of demands on learners.

 A frequent problem for teachers is when there are so many learners in one class that it
seems to make it unrealistic to assess speaking.

 Pronunciation, vocabulary, accuracy, communication, interaction and fluency are all

markers of a student’s overall speaking abilities.

 Most teachers will be familiar with the concept of grading with a rubric, a table with
different criteria and a grading scale.

 Vocabulary comprehension and vocabulary production are always two separate banks
of words in the mind of a speaker, native as well as second language.

 Fluency may be the easiest quality to judge in your students’ speaking. How
comfortable are they when they speak? How easily do the words come out?

 Three frameworks about general purposes of testing oral skills as “linguistically

oriented, communication-oriented and situation-based.”

REFERENCES

 Luoma, S. (2004). Developing speaking tasks. In Assessing speaking (pp. 139-

169). Cambridge: CUP

15