Comparing Scores

on Cambridge English: Advanced (CAE) and IELTS

Comparing scores on CAE and IELTS

Summary
This paper reports on the latest in a series of studies that have explored the relationship between Cambridge English level-based exams and IELTS. Specifically, this study looks at the relationship between performance on Cambridge English: Advanced (CAE) and IELTS. Where overall scores are concerned, the following are indicated by the linking:
CAE Overall 67 60 IELTS Overall 7.0 6.7

A standardised score of 60 represents the grade C threshold on CAE. Given the band score system for IELTS, a score at grade C on CAE may be taken as at least equivalent to a score of 6.5 on IELTS. However, users who wish to set a standard equivalent to band 7.0 should either specify a grade B requirement or look for a standardised score of 67 or higher.

1 INTRODUCTION
Strictly speaking, score comparisons between different tests can only ever be tentative, since each test is designed for a different purpose and a different population and may view and assess language traits in different ways as well as describing test-taker performance differently. Nevertheless, test users are often required to consider results on different tests offered by applicants as evidence of language ability, and it is desirable that test providers provide some guidance as to how results may be compared. Cambridge ESOL, as the producers of both the Cambridge exams and the IELTS test, are uniquely well-placed to make such a comparison between results on CAE and results on IELTS. The relationship of IELTS to other Cambridge exams has been explored on a number of occasions, notably during the Association of Language Testers in Europe (ALTE) Can Do project in 1999 in which learners were asked to self-rate their capabilities and these ratings were plotted against test results. One aspect of this project involved IELTS candidates and the findings are reported in Appendix A. This provided an initial indication that in overall band terms candidates securing band 7 in IELTS might be expected to obtain a grade C on CAE. Subsequent studies have looked at the individual skills only, but have indicated a more subtle relationship. The present study was undertaken to provide empirical evidence of the relationship between performance on CAE and IELTS, with the primary aim to determine the relationship between CAE and IELTS, and in particular the relationship between CAE grade C and IELTS band 7. The question this report addresses is: what is a candidate with a C grade on CAE likely to obtain as an overall score on IELTS?

2 METHODOLOGY
Data collection
The study had candidates take both CAE and IELTS, using a counterbalanced design. That is, registered CAE candidates were invited to take IELTS, and registered IELTS candidates were invited to take CAE. This is to control for any preparation or motivation-related effects on performance on one test or the other. In total 186 candidates from twenty-four countries in Europe, Asia, South America, and Africa, covering major candidature populations for both exams, participated in the study.

Analysis
For the analysis, raw CAE Reading and Listening scores (out of 53 and 30, respectively) were used to link to raw IELTS scores (out of 40) on those components. For Writing and for Speaking, adjusted (that is, scaled) and weighted scores that were rounded to the nearest integer were used for CAE, and raw scores out of 36 used for IELTS. In the case of IELTS Writing, it entailed totalling the analytic scores, double weighting Task 2, and dividing the total by three. Equipercentile linking with presmoothing, as described in Kolen and Brennan (2004), was chosen as the analytic method for this study. This approach to smoothing is advantageous in that indices are available for evaluating the goodness of fit and therefore of the linking. All required steps were taken in the identification of appropriate smoothing for the purposes of linking. The linking was carried out using the RAGE-RGEQUATE software (Zeng, Kolen, Hanson, Cui, & Chien, 2004), and error was estimated using the Equating Error software (Hanson & Chien, 2004).

3 RESULTS & DISCUSSION

The results of the linking are now presented. Following Kolen and Brennans (2004) guidelines, results of smoothing where C=2 were selected as the appropriate for Reading, Writing, and Speaking given the data, (as they are the first moments with non-significant chi-squares values, and generally fulfil the requirements of mean preservation), whereas C=6 was selected for Listening. The exercise was also performed using overall score data, and the results of the various linking exercises considered in parallel. Standard IELTS score ability transformation tables were applied to the Listening and Reading tests and converting the Writing/Speaking assessments. The results indicate that an average across the skills of 6.7 corresponded to a CAE standardised score of 60, which is the minimum passing grade on the latter test. Therefore, a pass at C on CAE corresponds to a performance in the middle of the ability range covered by an IELTS score of 6.5. The linking also provided an indication of the overall CAE level that is equivalent to the IELTS band 7 threshold; the value for this is a CAE standardised score of approximately 67. An IELTS band of 6.5 was seen to correspond to a CAE standardised score of 58.

4 CONCLUSION & RECOMMENDATIONS

To summarise,
CAE Overall 67 60 58 IELTS Overall 7.0 6.7 6.5

Candidates who pass CAE at grade C or above are, at minimum, higher than an overall IELTS 6.5. Some of them may however be slightly below the level of an IELTS band 7. Institutions wishing to compare the performance of CAE candidates to IELTS band scores should therefore use the standardised scores rather than grades. If the requirement is a grade equivalent to an IELTS 7, the mark to look for would be a standardised CAE score of approximately 67.

5 REFERENCES
Hanson, B.A., and Chien, Y. (2004). Equating Error computer software. Iowa City, Iowa: CASMA. Kolen, M.J., and Brennan, R.L. (2004). Test equating, scaling, and linking. New York: Springer. Zeng, L., Kolen, M.J., Hanson, B.A., Cui, Z., and Chien, Y. (2004). RAGE-RGEQUATE computer software. Iowa City, Iowa: CASMA.

Appendix A
The ALTE Can Do project is described in detail in Appendix D of the Common European Framework of Reference for Languages: Learning, Teaching, Assessment document. As one strand of this project, Can-do questionnaire responses by IELTS candidates were collected over the period of one year. A total of 155 candidate responses were matched to band scores, enabling Can Do self-ratings of IELTS candidates to be compared with those of Cambridge English: First (FCE) and CAE candidates. A grade C at FCE represents the B2 level on the CEFR and a grade C on CAE represents the C1 level. The responses of the IELTS candidates were matched to band scores. These band scores were compared with the Can Do responses and the results are displayed in the figure below.

Figure: Mean Can Do self-ratings compared with IELTS overall band scores and grades for Cambridge English: Advanced (CAE) and Cambridge English: First (FCE) In terms of candidates self-perception therefore, candidates securing IELTS band 6 felt comfortable with a similar range of activities as candidates securing an FCE grade C and candidates securing band 7 thought themselves comparable to candidates securing a CAE grade C. There is a steady progression in self-ratings across IELTS bands with the exception of Band 5. However, there were only seven respondents at this level, compared to 46 at band 8, 39 at band 7 and 53 at band 6, so the mean responses at the higher levels are more dependable.

University of Cambridge ESOL Examinations (Cambridge ESOL) develops and produces Cambridge English exams the most valuable range of qualifications for learners and teachers of English in the world. Over 3.3 million people take our exams each year in 130 countries. Around the world, more than 12,000 universities, employers, government ministries and other organisations rely on Cambridge English qualifications as proof of English language ability. Cambridge English exams are backed by the work of the largest dedicated research team of any UK language test provider. Cambridge ESOL is a not-for-profit department of the University of Cambridge and part of the Cambridge Assessment group. IELTS (International English Language Testing System) is jointly owned by British Council, IDP: IELTS Australia and Cambridge ESOL.

Contact us: University of Cambridge ESOL Examinations 1 Hills Road Cambridge CB1 2EU United Kingdom Tel: +44 1223 553997 Email: Cambridge.Recognition@CambridgeESOL.org www.CambridgeESOL.org

UCLES 2011 EMC | 8033 | 1Y09

*0528900536*