Assessing speaking

M. Syamsuli @kbar
1.
Imitative
Is the speaking
performance in
ability to simply
parrot back
(imitate) in word,
phrase or possibly a
sentence.
 2.
Intensive
It is the production of short
stretches of oral language.
 3.
Responsive
Is the ability to respon
about simple question
with a patner.The tasks
include interaction
and test comprehension.
4. Interactive

Is The length and

complexity of
the interaction in
speaking. Includes
multiple exchanges
and/or multiple
participants.
5. Extensive

Is speaking ability to
production extensive
oral,include speeches,
oral presentations, and
story-telling. Oral
interaction from listeners
is either highly limited or
ruled out altogether
Micro : They include
production English
stress patterns,
reduced forms,
production of fluent
speech, use of strategic
devices (pauses, Macro : Fluency,
fillers). discourse, function, style,
cohesion, nonverbal
communication.They include
the appropriate
accomplishment of
communicative functions, use
of appropriate styles,
registers, conversation rules,
etc.
DESIGNING ASSESSMENT TASKS

1. IMITATIVE

REPITITION TEST PHONEPASS TEST

1. Word Repitition Task

Test-taker hear : Repeat after me

Beat [pause] Bit [pause]
Bat [pause] Vat [pause] etc.
I bought a boat yesterday
The glow of the candle is growing etc.
When did they go on vacation?
Do you like coffee? etc.
Test-taker repeat the stimulus
 Scoring scale for repetition
tasks

Acceptable pronunciation.

comprehensible, partially correct.

silence, seriously incorrect.

2. Phone Pass test
spesification
Part A:
Test-takers read aloud selected sentences from among those printed on the sheet. Example:
1. Traffic is a huge problem in Southern California.
2. The endless city has no coherent mass transit system.
3. Sharing ride was going to be solution to rush-hour traffic.
4. Most people still want to drive their own cars, though.
Part B:
Test-takers repeat sentences dictated over the phone. Example: “Leave town on the next train.”
Part C:
Test-takers answer question with a single word or a short phrase of two or three words. Example:
“Would you get water from a bottle or a newspaper?”
Part D:
Test-takers hear three word groups in random order in must link them in a correctly ordered
sentence. Example: was reading/my mother/a magazine.
Part E:
Test-takers have 30 seconds to talk about their opinion about some topic that is dictated over the
phone. Topic center on family, preferences, and choices.
 Scoring scale for phonepass* tasks
Scores are calculated by a
computerized scoring
template and reported back
to the test-taker within
minutes.
 2. INTENSIVE SPEAKING
TASK

Test-takers are prompted to

produce short stretches of discourse
(no more than a sentence) through
which they demonstrate linguistic
ability at a specified level of
language.
 Kinds Of Intensive
Speaking

3).
1). 4).
Sentenc
Directe 2). e Oral 5). 6).
d Read- dialogu quest- Picture Transla
Respon Aloud e ionnair -Cued -tion
se comple- e
tion
 1. Directed respon task

In this type of
task,the test
administrator elicits
a particular
grammatical form or
a transformation of a
sentence.
 2. Read-aloud
Task

Include reading
beyond the
sentence level up
to a paragraph or
two.
 Read aloud scoring
Pronunciation:
Points:
0.0-0.4 Frequent phonemic errors and foreign stress and intonation
patterns that cause the speaker to be unintelligible.
0.5-1.4 Frequent phonemic errors and foreign stress and intonation
patterns that cause the speaker to be occasionally unintelligible.
1.5-2.4 Some consistent phonemic errors and foreign stress and
intonation patterns, but the speaker is intelligible.
2.5-3.0 Occasional non-native pronunciation errors, but the speaker is
always intelligible.

Fluency:
Points:
0.0-0.4 Speech is so halting and fragmentary or has such a non-native
flow that intelligibility is virtually impossible.
0.5-1.4 Numerous non-native pauses and/or a non-native flow that
interferes with intelligibility.
1.5-2.4 Some non-native pauses but with a more nearly native flow so
that the pauses do not interfere with intelligibility.
2.5-3.0 Speech is smooth and effortless, closely approximating that of a
native speaker.
Example:

Despite the decrease in size-and, some would say, quality-of

our cultural word, there still remain strong differences between
the usual British and American writing styles. The question is,
how do you get your message across? English prose conveys its
most novel ideas as if they were timeless truths, while American
writing exaggerates; if you believe half of what is said, that’s
enough. The former uses understatement; the latter,
overstatement. There are also disadvantage to each characteristic
approach. Readers, who are used to being screamed at may not
listen when someone chooses to whisper politely. At the same
time, the individual who is used to a quiet manner may reject a
series of loud imperatives.
 3. Conversation Task

Another technique for targeting

intensive aspects of language
requires test-takers to read dialogue
in which one speaker's lines have
been omitted.
example
5. Picture-Cued Tasks

Picture may be very

simple, designed to elicit
a word or a phrase;
somewhat more
elaborate and “busy”; or
composed of a series
that tells a story or
incident.
Example:

Grammatical categories may be cued by

pictures. In the following sequences,
comparatives are elicited:
Picture-cued elicitation of nouns, negative responses,
numbers, and location
 Picture-cued task scoring
If the tasks above that asked just for one-word
or simple-sentence responses can be evaluated
simply as "correct" or "incorrect.“

• Scoring scale for intensive tasks

2 comprehensible; acceptable target form

1 comprehensible; partially correct target
form
0 silence, or seriously incorrect target form
6. Translation
(of Limited Stretches of Discourse)
Translation is a part of our tradition
in language teaching that we tend to
discount or disdain, if only because
our current pedagogical stance plays
down its importance. Instead of
offering picture or written stimuli, the
test-taker is given a native language
word, phrase, or sentence and is
asked to translate it.
Tas
Bag

Gamba Picture
r
 3. Responsive Speaking
Task

Responsive speaking indicates the

oral production in the very short
conversations. Differs from intensive
tasks in the increased creativity
given to the test-taker and from
interactive tasks by the somewhat
limited length of utterances.
Kind of Responsive speaking tasks
 Questions and answers
These tasks can
consist of simple and
complex questions from
an interviewer or they
can make up a portion
of a whole battery of
questions and prompts
in an oral interview.
Example:
• Questions eliciting open-ended responses
• Test takers hear:
– What do you think about the weather today?
– What do you like about the English language?
– Why did you choose your academic major?
– What kind of strategies have you used to help you learn English?
Test-takers response with a few sentences at most
•  

• Elicitation of questions from the test-taker

• Test takers hear:
– Do you have any questions for me?
– Ask me about my family or job or interests.
– If you could interview the president or prime minister of your country, what would you
ask that person?
Test-takers respond with questions
 Giving Instructions and
Directions

Using such a stimulus in an

assessment context provides an
opportunity for the test takers to
engage in an relatively extended
stretch of discourse, to be very
clear and specific, and to use
appropriate discourse markers and
connectors.
Scoring is based primarily on
comprehensibility and
secondary on other specified
grammatical or discourse
categories.
Example:
 Paraphrasing

Another type of assessment tasks

that can be categorized as
responsive asks the test takers to
read or hear a limited number of
sentences ( perhaps two to five )
and produce a paraphrase of the
sentence
exampl
e
Test takers hear : paraphrase the following little
story in your own words.

My weekend in the mountains was fabulous. The first day

we backpacked into the mountains and climbed about
2,000 feet. The hike was strenuous but exhilarating. By
sunset we found these beautiful alpine lakes and made
camp there. The sunset was amazingly beautiful. The next
two days we just kicked back and did little day hikes, some
rock climbing, bird watching, swimming, and fishing. The
hike out the next day was really easy-all downhill-and the
scenery was incredible.

Test takers respond with two or three sentences

 TSE (test of spoken English)

The TSE is a 20-minute audio taped

test of oral language ability without
academic or professional environment.
TSE Scores are used by many North
American Institutions of higher education
to select international teaching
assistants .the score are also used for
selecting and certifying health
professionals such as physicians
,nurses ,physical therapists and
veterinarian.
 The following content specifications for the TSE
represent the discourse and pragmatic contexts
assessed in each administration:

1. Describe something physical

2. Narrate from presented material
3. Summarize information of the speaker’s own
choice
4. Give direction based on visual materials
5. Give instruction
6. Give an opinion
7. compare/contrast
8. Hypothesis
9. Function
10.Define
 4. INTERACTIVE

The length and complexity of

the interaction are more in
interactive tasks than in
responsive ones. The task
sometimes includes multiple
exchanges and/or multiple
participants.
 Intrview

Test
administrator
and test-taker in
a direct face-to-
face.
 A Framework for Oral
Proficiency Testing
•Warm-up :
How are you?
What’s your name?
•Level check :
How long have you been in this city?
Tell me about your family.
•Probe :
What are your goals for learning English in this program?
Describe academic field to me. What do you like and
dislike about it?
•Wind-down
Did you feel okay about this interview?
What are your plans for the future?
 The Success of an Oral Interview
1. Clear administrative procedures
(practicality)
2. Focusing the questions and probes on the
purpose of the assessment (validity)
3. Biased for best performance
4. Creating a consistent, workable scoring
system (reliability)
5. Descriptions of the Oral Proficiency
Scoring Categories
 Grammar Vocabulary Comprehension
• Errors in grammar • Speaking • Within the scope of
are frequent, but vocabulary his very limited
speaker can be inadequate to language
understood by a express anything experience, can
native speaker used but the most understand simple
to dealing with elementary needs questions and
foreigners • Has speaking statements if
attempting to speak vocabulary deliveredwith
his language. sufficient to express slowed speech
• Can usually handle himself simply with repetition, or
elementary some paraphrase.
constructions quite circumlocutions • Can get the gist of
accuarately but most conversation
does not have of non-technical
thorough or subjects(i.e. , topics
confident control of that required no
the grammar specialized
knowledge)
 Fluency Pronunciation task
• (No specific • Errors in • Able to satisfy
fluency pronunciation routine social
description. are frequent demands and
Refer to other but can be work
four language understood by requirements;
areas for a native needs help in
implied level speaker used handling any
of fluency.) to dealing with complication
foreigners or difficulties
attempting to
speak his
language.
 Subcategories of oral proficiency
scores
Level Description
0 Unable to function in the spoken language
0+ Able to satisfy immediate needs using rehearsed utterances
1 Able to satisfy minimum courtesy requirements and maintain very simple face-to-face conversations on
familiar topics
1+ Can intiate maintain predictable face-to-face conversation and satisfy limited social demands

2 Able to satisfy routine social demands and limited work requirements

2+ Able to satisfy most work requirements with language usage that is often, but not always, acceptable and
effective
3 Able to speak the lnguage with sufficient structural accuracy and vocabulary to participate effectively in
most formal and informal conversations on practical, social and professional topics

3+ Often able to use the language to satisfy professional needs in a wide range of sophisticated and
demanding task
4 Able to use language fluently and accurately on all levels normally pertinent to professional needs

4+ Speaking proficiency is regularly superior in all respect, usually equivalent to that of a well-educated,
highly articulate native speaker
5 Speaking proficiency is functionally equivalent to that of a highly articulate, well-educated native
speaker and reflects the cultural standards of the country where the language is spoken
 Role Play

It is a popular
pedagogical activity in
communicative
language-teaching
classes.
Examples: “Pretend that
you’re a tourist asking
me for directions”, “You
are buying a necklace
from me in a flea
market, and want a
lower price”.
 Discussions &
Conversations

Discussions may
be especially
appropriate tasks
through which to
elicit and observe
 GAME

Assessment games may include:

• “Tinkertoy” game
• Crossword puzzles
• Information gap grids
• City maps
• Etc.
 5. Extensive
Speaking
Oral
presentation Picture-cued
story telling

Translation
Telling a Story,
News Event (of Extended
Prose)
 1. Oral presentation

For oral presentations a

checklist or grid is a
common means of
scoring or evaluation.
Content :
The purpose or objective of the presentation was accomplished

The introduction was lively and got my attention

The main idea or point was clearly stated toward the beginning

The supporting points were :

5. clearly expressed
6. supported well by facts, argument

The conclusion restated the main idea or purpose

Delivery :
The speaker used gestures and body language well

The speaker maintained eye contact with the audience

The speaker’s language was natural and fluent

The speaker’s volume of speech was appropriate

The speaker’s pronunciation was clear and comprehensible

The speaker’s grammar was correct and didn’t prevent understanding

The speaker used visual aids, handouts, etc., effectively

The speaker showed enthusiasm and interest

 2. Picture-cued Story
telling
One of the most common
techniques for eliciting oral
production is through visual
pictures, photographs,
diagrams, and charts
3. Telling story, News Event

In this type of task, test-

takers hear or read a
story or news event that
they are asked to retell.
4. Translation (of extended
prose)
Translation of
words, phrases, or
short sentences was
mentioned under the
category of intensive
speaking.