Assessing Grammar

Test Evaluation

 Summary Teaching
 Evaluation of an Existing

Test
 Five characteristics to measure
communicative grammar
1. The test must provide more context than
only a single sentence.
2. The test taker should understand what
the communicative purpose of the task is.
3. He or she should also know who the
intended audience is.
4. He or she must have to focus on meaning
and not form to answer correctly.
5. Recognize is not sufficient. The test
taker must be able “to produce
grammatical responses.”

By: P.R. Dickins

 Therefore…
 Have the test taker say or write
something (5) of discourse length (1)
in order to perform some
communicative task (2) for a known
audience (3), and what is said or
written must make sense (4).
 Assessing grammar can only be
tested as part of a test of writing or
speaking?
Important Principle: What are we trying to Measure?

“English grammar is chiefly a system of syntax

that decides the order and patterns in which
words are arranged into sentences.” – Close
1982

 Separated but closely related to assessing

writing and assessing speaking.
• Oral performance
• Writing samples
 Accuracy and fluency
 Count the errors and rate their severity

 Focus on Grammar only

• ‘Meaning’ is a separated matter
• Integrated tests
• Discrete point tests
Traditional Paper and Pencil Test

 NRT: TOEFL
 CRT: Classroom tests
 Types of Questions:
• Complete sentences
• Sentence combining
• Paragraph editing
• Correct topic sentence task
• Multiple Choice
• Fill in the blank
• True/False
• Combination
 Alternative Assessment
 Games
 Class observation
 Online activities
 Portfolio
 Games
 Design to restrict the language need
 Usually these games are defined by
their focus on
• The Use of Language
• Limited options for communication
• Accuracy
 Split Sentences
 Write out some sentences, and then
cut each sentence in half. Hand out
these pieces to the students, and ask
them to find the matching half
among the other students.
 Students’ abilities in combing
sentences
 Grammar knowledge to all forms of
the sentences
 Examples
 If you eat that  you’ll be sick
 If you touch the  it’ll bite you
dog  I’ll never speak
 If you steal my to you again
boyfriend  you’ll get
 If you go out now soaked
 If you don’t leave  I’ll call the police
 If you don’t book  you’ll be lucky
a ticket to get a seat
 Sentences from pictures
 Hand out a sheet of pictures. Ask
students to come out sentences from
the pictures. Or ask students to tell
a stories.
 Enough grammar knowledge to make
sentences by students.
 Accuracy
 Memory Test
 Give limited time for students to see
the picture. Ask them to describe
the picture without seeing it.
 Testing students’ ability of making
sentences.(enough grammar
knowledge)
 Accuracy
 The transformation of students’
sentences
 Picture Dictation
 A student try to describe the picture
to other students who haven’t seen it.
 Testing if a student can make
sentences which can help him/her
communicate well.
 Accuracy
 Strip Story
 Make students into groups and
assign them a text from a strip story.
Ask them to find the order and
resolve the problem from the story.
 See if students can find the time
order from the tenses of the
sentences.
 Accuracy
 Miming an action
 Ask students to role play in a limited
situation.
 Through acting, it can test students’
ability of thinking of lines. (which
may contains different tenses.)
 Accuracy
 Growing Stories
 Story-building activities
 Students’ ability of making sentences
with different tenses
 Questionnaires
 Turn what have taught in class into a
questionnaire. Get students to
survey each other. (does not contain
fully-written-out questions.)
 It can see if students’ totally
understand what have learned in
class. Also it can see if students’ can
use it well.
 Classroom Observation
 Classroom activities
 Classroom communication: Elicited
dialogues
 Written exercise
 Online Activities
 In multimedia teaching times, teachers can use
Internet as a way of assessing students’ ability of
grammar
 On-line quizzes help students to have more
chances of practices
 On-line tests also provide clear explanation after
finishing the quizzes
 Students can surf on these grammar website to
do practice
 Teachers can ask students to do these exercise
and to keep record for themselves. Finally report
back for one of the reference of assessment
 Online activity : websites
 http://www.ccc.commnet.edu/grammar/index.ht
m
 Ex1. Irregular verbs quiz
http://www.ccc.commnet.edu/cgi-
shl/par2_quiz.pl/irregular_quiz.htm
 Grammar Assessment
http://www.prenhall.com/grammarassessment/
 Self-study grammar quizzes
http://a4esl.org/q/h/grammar.html
 ESL Quizzes Center
http://www.pacificnet.net/%7Esperling/quiz/gra
mmar
 Portfolio
 Definition:a purposeful collection of
student work that tells the viewer about
the student
 Works should be organized by
chronological order and category– teacher
can evaluate the child’s achievements
 Works including:

1. Traditional paper and pencil tests

2. Classroom observation
3. Online activities
4. Written assignments
 Written assignments
 Give students some assignments to
practice at home
 Ask students using learned grammar
in class to write journal
 Term paper throughout one semester
including several drafts– we can see
student’s progress in grammar.
 Advantages of portfolio
 Why portfolio is the best way to
assess?
 Strengthening students’ learning
 Enhancing the teacher’s role
 Improving processes
 Test Evaluation of An Existing Test

 Section 2 of the Computer-Based Test

of TOEFL

 General Background Information

1. Title: Section 2 of TOEFL-Structure and Written
Expression

2. Author: Education Testing Service (ETS)

Test Evaluation of An Existing Test
3. Purpose of decision :

Section 2 measures the ability to recognize

language that is appropriate for standard

written English.
Test Evaluation of An Existing Test

4. Date of Application:
a. 1962, the formation of a National
Council on the TOEFL

b. July 1998 (Most parts of the world)

c. October 2000 (Asia, including

Taiwan)
 Practical Orientation

1. Ease of administration:
a. Computers needed
b. Convenience

2. Price: expensive ($130)

3. Quality of items: clarity

 Practical Orientation

4. Ease of scoring:
a. Photo score reporting
b. Computerized scoring (0~30)
c. Cumulative scoring

5. Ease of interpretation: transmission

speed.
 Practical Orientation
6. Test construction:
a. Computer-adaptive Testing
b. Complete an incomplete sentence
c. Identify one unacceptable answer form
four underlined words.

7. Timing: 20-25 questions/ 15-20 minutes

(including “pretest questions”)

“Description of the Computer-Based TOEFL Test” www.ets.org

Test Evaluation of An Existing Test
 Model Test
 Analysis of TOEFL Grammar Test

 8 sentence patterns
• Basic Sentence & Compound Sentence
• Adjective Clauses
• Participial Phrases
• Appositive
• Noun Clauses
• Adverbs Clauses
• Prepositional Phrases
• Expletives
方有毅 托福文法速成 來欣留學英語中心
 Analysis of TOEFL Grammar Test
 15 Answering
Techniques
1. Word Form ………....20 % 9. Articles ……………………….5 %
2. Word Choice ……....14 % 10. Comparatives and
3. Verbs ………………....11 % Superlative sentence……4 %
4. Parallelism ………......9 % 11. Conjunctions ……………… 4 %
5. Pronouns ……………...7 % 12. Redundancy………………….1 %
6. Singular/Plural 13. Negative Words ……………2 %
Nouns …...……...……..6 % 14. Word Order ………………….4 %
7. Verbal (Infinitives, Gerunds, 15. Misplaced Modifiers ….…..1 %
and Participles) ……..…...6 %
8. Prepositions ……..…..6%
 Total ……………………….100 %
方有毅 托福文法速成 來欣留學英語中心
 Analysis of TOEFL Grammar Test
 Reliabilities: describes the tendency of a set of scores to
be ordered identically on two or more tests, and it can be
estimated by a variety of statistical procedures.

Table 1. Reliabilities and Standard Errors of Measurement (SEM)

July 1998 – June 1999

Reliability SEM

Listening 0.98 2.76

Structure/Writing 0.88 4.89
Reading 0.88 2.73
Total Score 0.95 10.8
“Description of the Computer-Based TOEFL Test” www.ets.org
 Analysis of TOEFL Grammar Test
 Validity
1. ------pure lead, the lead ore is mined, then melted, and

finally refined.
(A) Obtaining (B) Being Obtained
(C) To obtain (D) It is obtained
2. Louisa May Alcott is mainly noted for her very popular
A B
novel, Little Women, whom she published in 1868.
C D
3. Some hangars, buildings used to hold large aircraft, are
A
very tall that rain occasionally falls from clouds that form
B C
along the ceilings. (聶群 精修文法 來欣留學英語中心)
D
 Analysis of TOEFL Grammar Test
Strength:  Highly skilled test
 Accurate and developers
objective scores  Longstanding
reliability and
 Careful monitoring
consistency
of test  Ongoing test
 Commitment to improvements
international  Standardized delivery
education procedures
 Expert advisory
“Description of the Computer-Based TOEFL
committees Test” www.ets.org
 Analysis of TOEFL Grammar Test
 Conclusion
 No other test in the world is as reliable as

a standard for measuring a nonnative

speaker’s ability to use English at the

university level.
“Description of the Computer-Based TOEFL Test” www.ets.org
END