Professional Documents
Culture Documents
PREFACE
This April 2013 edition of the American Language Course Placement Test Handbook (ALCPT Handbook) contains current information about the test and its uses, as well as updated guidelines for test security, administration, scoring, and interpretation of the ALCPT. It supersedes the previously published ALCPT Handbook, August 2011 edition. This handbook is intended as a collection of best practices employed at language testing programs of excellence. The record-keeping forms in the appendix are provided as resources to facilitate best practices in a testing program administration.
DLIELC CONTACTS
For questions about Contact Mailing address Phone/Fax/Email
DLIELC/LEAT 2235 Andrews Ave JBSA Lackland, TX 782365259 Comm. Phone: (210) 671-4889 Comm. Fax: (210) 671-0211 DSN: (312) 473-4889 DSN Fax: (312) 473-0211 Email: DLI.Testing@us.af.mil OPI Email: OPI@us.af.mil DLIELC/LEN 2235 Andrews Ave JBSA Lackland, TX 782365259 Comm. Phone: (210) 671-3783 Comm. Fax: (210) 671-5362 DSN: (312) 473-3783 DSN Fax: (312) 473-5362 Email: LEN@us.af.mil
Language testing: ALCPT, ECL, and OPI Test security Approval of ALCPT sale Arranging Telephonic OPIs Test & Measurement Branch (DLIELC/LEAT)
In-country (nonresident) English Language Training Programs (ELTPs) ELTP development consulting (ALC materials, ALCPT purchase after LEAT approval, site surveys, etc.) DLIELC advisors/instructors for overseas assignments (MTTs)
TABLE OF CONTENTS
PREFACE ...................................................................................................................................................... i DLIELC CONTACTS................................................................................................................................ i CHAPTER 1. TEST DESCRIPTION ............................................................................................... 1 A. B. C. D. PURPOSE ..................................................................................................................... 1 CONTENT DESCRIPTION .............................................................................................. 1 FORMAT ......................................................................................................................2 TEST FORMS & KITS ....................................................................................................2
CHAPTER 2. TEST SECURITY BEST PRACTICES .............................................................. 3 A. B. C. D. APPOINTMENT OF TEST CONTROL OFFICER .................................................................3 PHYSICAL CONTROL OF TEST MATERIALS ....................................................................3 PREVENTION OF TEST COMPROMISE ........................................................................... 4 PROCEDURES FOR LOST OR COMPROMISED ALCPTS....................................................5
CHAPTER 3. TEST ADMINISTRATION BEST PRACTICES .......................................... 6 A. B. C. D. TESTING FACILITIES ................................................................................................... 6 ADMINISTRATIVE SECURITY ....................................................................................... 6 TEST ADMINISTRATION PROCEDURES ......................................................................... 8 CONTINUITY BOOK .................................................................................................... 11
CHAPTER 4. TEST SCORING BEST PRACTICES .............................................................. 12 A. B. C. DOUBLE MARKS ........................................................................................................ 12 USING THE SCORING KEY .......................................................................................... 12 SCORING SECURITY ................................................................................................... 12
CHAPTER 5. TEST SCORES AND PROGRAMING BEST PRACTICES ..................14 A. B. C. D. E. PLACEMENT .............................................................................................................. 14 PROGRAM SIZE CONSIDERATIONS.............................................................................. 14 MIDPOINT PROGRESS ................................................................................................ 15 GRADUATION REQUIREMENT .................................................................................... 15 RELATIONSHIP OF ALCPT AND ECL SCORES ............................................................. 16
ii
CHAPTER 6. OBTAINING THE ALCPT ................................................................................... 17 STEP 1. DLIELC/LEAT - APPROVAL ................................................................................. 17 STEP 2. DLIELC/LEN - PURCHASE .................................................................................. 17 APPENDICES .......................................................................................................................................... 18
iii
B. CONTENT DESCRIPTION
The ALCPT tests English ability through different aspects of grammar, vocabulary, listening, and reading. Grammar items include comprehension of morphological and syntactic input, while vocabulary items focus on semantic input. Reading items include reading comprehension. All ALCPT forms are approximately the same difficulty level, and contain the same specifications and format.
1
The ECL test is the official instrument used by the U.S. Department of Defense to determine whether the English level of IMSs considered for assignment to CONUS or overseas schools/training installations is sufficient to enter training at DLIELC or for direct entry to Military Service courses of instruction. The ECL is a strictly controlled instrument. Using the ALCPT to determine a candidates readiness reduces exposure to the ECL and helps minimize the possibility of test compromise.
C. FORMAT
The ALCPT is a 100-item, 4-option, multiple-choice test. The test is divided into 2 parts, each based on the delivery method. In Part I, 66 items are based on listening passages. Test-takers listen to audio recordings of English questions, statements, and dialogs, and then select the correct answer from the test booklet. In Part II, 34 items are delivered in writing. Test-takers read English questions, statements, and paragraphs, and then select the correct answer from the same test booklet. Answers are marked on separate answer sheets. Test-takers do not mark the test booklet, which allows reuse of the booklets and facilitates scoring. The length of Part I is determined by the audio recording (25-30 minutes). Part II lasts 30 minutes. The entire test takes approximately 75 minutes, including time for reading the directions and distributing and collecting test materials.
KIT CONTENT
Quantity 25 1 2 1 1 1 1 1 1 1
Scores achieved on currently available ALCPTs do not require the use of a conversion table.
1. IMMEDIATE ACTIONS
a. The TCO stops all testing with the test form(s) in question at once. b. The TCO alerts DLIELC/LEAT of the situation by email or fax (see contact information on page i), and includes a summary of the compromise with any known details. The summary should also include: i. test information (for example, ALCPT form 56: booklets 19 and 22 missing, score key missing, etc.), ii. date (or probable date) of loss or compromise, iii. verification that all testing with the form in question has been stopped. c. Compromised ALCPT forms are withdrawn and not used for a period of one year.
2. INVESTIGATION
The TCO conducts an internal investigation and/or arranges for an investigation by an outside agency to determine facts concerning loss or compromise.
3. RECORDING
The TCO keeps the complete report of the investigation, including findings and a statement of corrective action taken, in the user agencys Continuity Book (see Chapter 3, Part D).
Alerting DLIELC/LEAT of actual or potential test compromise enables DLIELC/LEAT to inform other ALCPT user agencies of potential compromise.
A. TESTING FACILITIES
Testing rooms need to be well lit, ventilated, and quiet. A sign with a warning, such as "QUIET, TESTING IN PROGRESS," is placed on the door and in nearby areas to keep noise at a minimum. Test-takers are positioned far enough from each other to ensure the integrity and validity of test results.
B. ADMINISTRATIVE SECURITY
The following administrative steps help ensure the integrity of test results through security measures.
5. TEST-TAKERS
Test-takers do not bring anything into the testing room, and they do not leave until they have finished the test and returned all test materials. If a test-taker must leave the room before completing the test, that test-taker is not permitted to return to the test room and finish the test. Instead, the test-taker is scheduled for retesting with another form of the ALCPT. The TCO/ATCO briefs all test-takers on this requirement before the test begins (see Chapter 3, Part C, #2).
6. TEST MATERIALS
In the test room(s), the TCO/ATCO gives test-takers an answer sheet, two sharpened pencils, an eraser, and a test booklet. Test booklets are checked before (and after) testing to ensure that all pages are intact and that there are no extraneous marks. The TCO/ATCO also checks photo IDs and assigns seats. Testtakers are not informed of seat assignments until they report to the test area.
Ladies and gentlemen, you are here to take an American Language Course Placement Test. You must be quiet; do not talk or communicate in any way during the test. If you must
3
For sites where cassettes are still used, the TCO/ATCO is careful not to erase the cassette tape when playing or rewinding.
leave the room before finishing the test, I will pick up your test materials, and you will not be permitted to return to the room today. You will be scheduled for a retest at a later date. If you have any questions, raise your hand and I will help you. I will hand out the answer sheets now.
b. At this time, only the answer sheets are distributed to test-takers. i. If the headers in the answer sheets have already been filled in, the TCO/ATCO says:
I will now hand you the test booklets. When you receive the test booklet, write the number of the booklet on your answer sheet. Keep the booklets closed until I tell you.
e. The TCO/ATCO hands a test booklet to each test-taker. After all the test booklets are handed out, s/he says:
The test consists of two parts: Part I - Listening and Part II Reading. Mark one answer, A, B, C, or D for each item on your answer sheet. Make a heavy, dark mark on the letter. Make only one mark for each question. If you make a mistake or an extra mark, erase it completely. Answer each question, even if you are not sure of the correct answer. Do not mark the test booklet. Mark only your answer sheet. Now open the test booklet and listen to the directions.
The TCO/ATCO then starts the ALCPT audio recording. Do not stop or replay any portion of it. Audio recordings are played only one time during the test. Repeating any portion of it will adversely affect the validity and reliability of the test results. f. The TCO/ATCO must be in the testing room at all times during the test administration. There is no talking permitted during the examination. The TCO/ATCO listens and watches for any signs of attempted communication between test-takers, such as pencil tapping or other movements. The TCO/ATCO must stop any suspicious behavior, for example, by standing quietly behind the suspected test-taker. Test-takers must work independently. When appropriate, proctors assist the TCO/ATCO perform these tasks. g. At the end of the audio portion, the test-takers will hear a voice say, This is the end of the Listening part of the test. When Part I is finished, the TCO/ATCO says:
You may begin Part II now. You have 30 minutes to complete the test.
The TCO/ATCO also posts the exact completion time in writing: The test ends at _____. The TCO/ATCO may give the examinees a ten-minute warning and a one-minute warning before the end of the test.
10
h. Test-takers who finish the test early should raise their hands. The TCO/ATCO collects their testing materials (answer sheet, two sharpened pencils, eraser, and test booklet), and checks the test booklets before the test-taker leaves. The test-taker then leaves the room quietly. i. When the allotted time is finished, the TCO/ATCO says:
Stop. The test is over. Put your pencils down. Close the test booklets. Stay in your place until I finish collecting all the test materials.
j. The TCO/ATCO first collects all the test booklets, and then all the answer sheets, pencils and erasers. When available, proctors assist the TCO/ATCO perform these tasks. (See Scoring Procedures in Chapter 4.)
D. CONTINUITY BOOK
A Continuity Book helps TCOs/ATCOs keep track of duties and test security measures. It also offers a source of accountability, and gives new TCOs/ATCOs the necessary resources for the successful and secure continuation of the program. Therefore, Continuity Books of language testing programs of excellence might include the following: 1. Latest edition of the ALCPT Handbook 2. TCO/ATCO MFRs (see a sample form in Appendix A) 3. Test Inventory (see a sample form in Appendix B) 4. Test Administration Security Log (see a sample form in Appendix C) 5. Test Schedules 6. Test-Taker Roster (see a sample form in Appendix D) 7. ALCPT Approval Request form (available in Appendix E) 8. ALCPT communications of importance 9. Reports of actual or suspected test compromise
11
A. DOUBLE MARKS
The TCO/ATCO visually scans the answer sheets to see if any items have more than one letter marked. These items are counted wrong.
C. SCORING SECURITY
To help ensure that test security is maintained after testing:
1. SCORING
The TCO/ATCO scores all answer sheets in a secure area with no test-takers present.
2. TEST-TAKER ROSTER
The TCO maintains a roster of each test administration (see sample roster in Appendix D). The roster includes:
12
a. examinees names (to identify who has taken the ALCPT, per Chapter 1, Part A) b. date of test administration (to identify when a test was last taken, per Chapter 1, Part A and Chapter 5) c. test form (to identify the form used, per Chapter 3, Part B, sections 2 and 3) d. scores (to record progress when used as part of a language testing program, see Chapter 5)
3. ANSWER SHEETS
Scored answer sheets that have already been logged are destroyed by shredding or burning.
4. LOG IN MATERIALS
The TCO/ATCO ensures all materials used are accounted for and accurately records this in the test security log before returning tests to secure storage.
13
A. PLACEMENT
Students can be placed at different levels in the ALC based on their ALCPT scores as well as additional information, such as records of a student's language instruction background and/or student interviews by language professionals. The ALC Placement Guidelines table (below) provides a reference for using ALCPT scores to place students in an appropriate ALC level. It does so by showing acceptable ranges of ALCPT Scores for each of the ALC books. In cases where a student scores 29 or below, additional information is especially critical. Since the ALCPT is a multiple-choice test, a score from 0 to 29 can be attributed to guessing and does not reliably measure a students English ability. In these cases, background information of a students English language instruction and/or interviews by a language professional (ideally, familiar with the ALC) might be useful in determining that students English abilities for placement purposes.
14
C. MIDPOINT PROGRESS
For programs that are six or more months long, a midpoint ALCPT may be administered in order to check for satisfactory progress and to verify appropriate ALC program placement. If the score from a midpoint test is not within the range of ALCPT Scores in the ALC Placement Guidelines table, the students placement should be reconsidered. Instructors formative assessment of the student or other types of diagnostic assessment can be used. It is recommended that the professional judgment of instructors and program administrators be considered for placement of students in the correct book.
Level III
D. GRADUATION REQUIREMENT
The ALCPT score required for graduation from the ALC program will depend on the English proficiency requirements of the training program each student is in. requirement may vary from student to student. This
Level II
* ALCPT scores of 29 or below cannot be considered valid indicators of a test-takers ability. See Chapter 5, Part A for placement suggestions.
15
Level V
Level I
4 For more on the reliability of the ECL and ALCPT test-generating system see: Henning, G. (1994). Automated assembly of pre-equated language proficiency tests. Language Testing, 11(1), 15-28.
16
5 For a hard copy of the catalog, contact the Logistics Branch (LESL) by email at dlielc.lesl.workflow@us.af.mil or by phone at DSN 473-4711 (FAX 473-3063) or commercial 210671-4711 (FAX 210-671-3063).
17
APPENDICES
Appendix A. Sample TCO/ATCO appointing Memorandum for Record Appendix B. ALCPT Inventory Record sample Appendix C. Test Administration Security Log sample Appendix D. Test-Taker Roster sample Appendix E. ALCPT Approval Request form
18
REF: American Language Course Placement Test Handbook (latest issue) 1. The following personnel are appointed subject duties IAW reference, effective _______________, and until officially relieved or released from appointment. Signatures below signify that the appointee has read and understood the ALCPT Handbook and agrees to follow its provisions.
E-MAIL ADDRESS
SIGNATURE
E-MAIL ADDRESS
SIGNATURE
E-MAIL ADDRESS
SIGNATURE
E-MAIL ADDRESS
SIGNATURE
3.
Please retain for your records and for inspection purposes. (Do not send to DLIELC.)
(e.g., 52)
DEFENSE LANGUAGE INSTITUTE ENGLISH LANGUAGE CENTER ALCPT TEST ADMINISTRATION SECURITY LOG
Test Form Booklet Number From To CD Number Answer Key Number Number of TestTakers OUT Test Date Time Checked OUT TCO/ATCO Administering Test Time Checked IN
(e.g., 52)
Test Form
Booklet Number
Test Date
Rank
ID Number
Raw Score
Disposition
User Agreement
User agrees to: Maintain all ALCPT materials secured at all times: Stored in a locked cabinet; Accounted for by serial number and inventoried annually; Logged after each use & the log retained for a year; Handled, transported & administered ONLY by authorized personnel. Use the test for the following purposes ONLY: Place students in an ALC language program; Evaluate student progress midway and at the end of the program; Screen candidates for readiness to take the ECL; Evaluate the English language abilities of local personnel working for, or being considered for positions with, the military. Prevent duplication (paper, electronic or other) of ALCPT materials. Abide by all provisions of the ALCPT Handbook, including limits on test frequency, proper test rotation, and tracking candidates who are tested. User agrees that: If a contractor loses the original contract for which DLIELC/LEAT approved ALCPT use, the contractor is obliged to destroy all forms of the ALCPT. If ALCPTs have been compromised, DLIELC retains the right to refuse sales of new ALCPT forms for a period of up to 3 years.
Failure to abide by these guidelines may result in denial of future ALCPT purchases.
Contact Information
Date: Purchasing organization: Point of contact: Email/phone/(DSN if applicable): __________________ ________________________________________ Country: ______________ ________________________________________ ________________________________________
Request Information
Purpose for ALCPT testing: ALCPT Forms (versions) owned: ALCPT Forms requested (max. 10): _______________________________________ _______________________________________ __________________ Number of kits requested: __________
APW __
DA __
Submit to DLI.Testing@us.af.mil