Understanding and Setting

New Scores

Listening.
Learning.
Leading. Copyright © 2004 Educational Testing Service
iBT TOEFL Scores

The score report universities receive

will contain:
• Four skill scores
– Reading 0 - 30
– Listening 0 - 30
– Speaking 0 - 30
– Writing 0 - 30
• A total score: 0 – 120

Copyright © 2004 Educational Testing Service

Setting New TOEFL Scores
An opportunity to think more about English
language needs & requirements

• Are all skills equally important for success in

your program? If not, do your score
requirements reflect this?
• Students with the same total score can have
different abilities
• Consider using:
– skill scores instead of total scores
– skill scores and total scores

Copyright © 2004 Educational Testing Service

Understanding & Setting New
Scores

• Rubrics for Writing & Speaking

• Speaking and Writing samples
• Score comparisons
• Percentile data
• English language competency descriptors
• Results from standard setting studies
• Standard setting on your own campus

Copyright © 2004 Educational Testing Service

Score Comparisons:
iBT, CBT, PBT

• Based on a field test of 3,000 test takers in

30 countries
• Approx 80% of TOEFL volume comes
from these 30 countries
• Test takers took TOEFL iBT and TOEFL
CBT (computer-based)
• Score comparisons for CBT and PBT
(paper-based test) established in 1998

Copyright © 2004 Educational Testing Service

Using Score Comparisons:
iBT, CBT, PBT

Score-to-score and range-to-range

comparisons for:
– Reading
– Listening
– Writing
– Reading, Listening & Writing only total
– Total score (all four skills)

Copyright © 2004 Educational Testing Service

Setting Speaking Scores
No Speaking section on TOEFL CBT

Standard setting panel of ITA experts

Professional judgment, not statistically
determined

• Minimally acceptable level of speaking for

lowest level of contact with undergraduates:
23

• TSE 50 score equivalent: 26

Copyright © 2004 Educational Testing Service
Using Score Comparisons:
iBT, CBT, PBT

Score comparisons can be useful for

understanding relationships,

but differences in the tests can make it

difficult to establish exact
comparisons.

Copyright © 2004 Educational Testing Service

 Using Score Comparisons:
iBT, CBT, PBT
Remember:
• If you’re comparing total scores, CBT and
PBT TOEFL do not measure Speaking, but
iBT does.
• Using skill scores or skill scores and a total
score will give you more information about
your applicant. Use the whole profile.
• ETS encourages you not to set a rigid cut
score

Copyright © 2004 Educational Testing Service

Percentile Data
• Based on field test
• Should be used with caution
– Sample size
– Motivation & performance
– Unfamiliarity with the test, communicative
competence & integrated tasks
– Addition of Speaking
– Participants had lower ability level than
general TOEFL population
New data from operational test to be
published after first year

Copyright © 2004 Educational Testing Service

 English Language Competency
Descriptors

• Based on 2,300 students who took the new

iBT test and responded to questions about
their English language ability in each skill

• Competency descriptors tell us what a

student can do, and has difficulty with, in
English

Copyright © 2004 Educational Testing Service

Results from ETS Standard
Setting Studies

• Panels at 5 universities recommended

scores based on seeing the new test and
reviewing examinee performance.
• Results to be published after decision
makers discuss recommendations and set
final scores. Final scores to be posted on
our Web site.
• Schools similar to those in the study may
wish to adopt similar scores.

Copyright © 2004 Educational Testing Service

 Panel Recommendations –
NOT FINAL SCORES

Duke Lehigh Texas- Carnegie U. British

Univ. Univ. Austin Mellon Columbia
Level: Grad Grad Grad Under- Under-
grad grad
Writing 20 25 22 25 25

Speaking 22 24 23 23 20

Reading 21 20 21 21 19

Listening 14 17 17 17 14

Total 77 86 83 86 78
Copyright © 2004 Educational Testing Service
 Standard Setting at Your
Institution
• The best way to ensure your language
requirements match your institution

• Over 2 days, a panel of college or university

faculty and staff work view the test and
student responses to it.

• The panel recommends the level of English

just high enough to cope with studies at your
school.

Copyright © 2004 Educational Testing Service