Note on Scaling and Equating Scores in AIMCATs with multiple forms

Scaling and Equating of test scores, though relatively new in CAT, is not a new issue in entrance exams. Several standardized tests like GRE, GMAT, TOEFL etc., use the concept of scaling and equating scores, albeit in very different ways. The science of scaling and equating is fundamentally an empirical science and borrows heavily from the fields of psychometrics and statistics. The basic problem that is addressed in scaling and equating is that of measuring/comparing the scores of different groups of students (test-takers) who take different forms of the same test. Since this is an empirical science, there are, almost always, several ways of scaling and equating test scores across different test forms of a test, with each approach of scaling and equating having its own share of pros and cons. Through these approaches of scaling and equating test scores one can attempt to normalize a generic term borrowed from statistics to denote the process of minimizing any systematic bias/error in a given data the effects of the differences, if any, in two or more groups taking a test and/or in the difficulty levels of the different test forms administered. An important aspect to be noted is that, there is no hard science/rules that can determine the best way of scaling and equating test scores and arriving at perfectly satisfactory results that are indisputably accurate and under all circumstances! However, under some controlled conditions, when the number of test takers is large, and when the test conducting authority is clear about its objectives and priorities, some approaches of scaling and equating will give the desired results within acceptable levels of accuracy again at a group level only (since the entire process is heavily dependent on statistics), and not at an individual test-taker level. That is to say, equating is an acceptable and valid process targeted at groups of test takers and not individual test takers. Last year, the IIMs decided to go the online way in CAT2009 and thereby got involved with the process of scaling and equating CAT scores for the first time. While neither the IIMs nor PROMETRIC have disclosed the exact methodology that was used for the equating and scaling of scores across the 20+ slots/test forms administered in CAT2009, we at T.I.M.E. have studied and understood the underlying basic principles of the science of scaling and equating and have analyzed the various approaches that could be used to equate and scale the scores of a large group of test-takers taking different forms of a test like CAT or the AIMCATs. In our endeavor to provide the students with a feel for the exact CAT atmosphere and to replicate the test conditions as closely as possible, we have decided to conduct at least a couple of AIMCATs which recreate the CAT2009 feel even on the aspect of administering multiple test forms and subsequently scaling and equating the test scores. As most of you would be aware by now, AIMCAT1112 was one such AIMCAT where there were two test forms, administered on alternate days over the testing window. The results of these AIMCATs will be processed and reported after scaling and equating the scores across the test forms and arriving at what are commonly termed as Scaled Scores. Our R&D team is presently on the task of analyzing the data pertaining to the results of AIMCAT1112 and after a thorough study of the same we will decide upon the appropriate scaling and equating relationships, based on our expert judgment, and release the results for AIMCAT1112, which will include Scaled Sectional Scores and Total Scaled Score, in place of the usual Net Sectional Score and Overall Score.