Professional Documents
Culture Documents
Friday 14 October2011
National centre presentation:
BREAK 11:0012:30
Field trial overview and review; implementation of PISA standards lessons learned from both the paper-based and computerbased components Target: All NPMs
BREAK
Field trial outcomes 2 Overview of data outcomes from questionnaires Target: All NPMs
BREAK
Questionnaire discussion groups (continued) Target: All NPMs
BREAK
Adaptation and verification of cognitive test materials and questionnaire materials for main survey Main Survey test design Target: All NPMs
BREAK
PISA test operations for Main Survey Site visits and quality monitoring for main survey Target: All NPMs
12:3014:00 14:00-15:30
LUNCH
Main survey questionnaire proposals Target: All NPMs
LUNCH
Sampling for the main survey Presentation from Westat KeyQuest in the main survey Target: All NPMs
LUNCH
Round table responses from each national centre
LUNCH
Response to Questionnaire discussion groups, and to issues from the round table session. Conclusions (Consortium, Secretariat) Target: All NPMs
15:30-16:00 16:00-17:30-
BREAK
Main survey cognitive item proposals (for Mathematics, both paper-based and computer-based; and for Problem Solving) Target: All NPMs
BREAK
Main survey questionnaire proposals, continued Concluding with formation of groups and task assignment for tomorrow. Target: All NPMs
BREAK
National Centre Presentations (Italy, Denmark and the US) Target: Interested NPMs
BREAK
Round table discussion continued
BREAK
Reading components preparation, operational issues, item selection Target: NPMs from countries participating in Reading Components option
HOSTED DINNER AND NIGHT SAFARI TOUR Sampling consultations (including KeyQuest): all national centres, scheduled on Monday, Tuesday and Thursday Translation and verification consultations: at national centre request, Tues 11-12:30, 14:00-17:30; Wed 9:00-12:30, 14:00-17:30. 1
OECD Programme for International Student Assessment PISA 2012 NPM Meeting, Singapore
10 14 October 2011
Draft Agenda
[Note that sampling consultations for all national centres will be scheduled throughout the week NCs will be given an appointment time; and that national centres wanting to consult with consortium translation verification staff will be invited to sign up for a specified appointment.] Monday 10h October, 2011 1 9:00 - 10:30 2 1.2 Welcome and Introduction Singaporean authorities will welcome NPMs to the meeting. Secretariat and Consortium staff will provide an update of activities and developments since the last meeting (Secretariat, Consortium) NPM(1010)Record 1.3 3 2.1 Review the proposed tasks for the meeting and confirm the agenda. Consortium staff from ACER will provide an overview of the general field trial conduct and operational procedures from both the paper and computer-based components, issues related to the PISA technical standards, and lessons learned for main survey planning. NPM(1110)2 Lunch at Pan Pacific Hotel hosted by Singaporean National Centre. 4 14:00 - 15:30 Field Trial Outcomes 1 Cognitive components 5 Consortium staff from ACER will provide an overview of field trial data outcomes from the cognitive tests. NPM(1110)3 6 16:00 - 17:30 Main survey cognitive item proposals 4.1 Consortium staff from ACER will introduce proposals for the cognitive item selections for the Main Survey for Mathematics, both paper-based and computer-based items, and for Problem Solving. NPM(1110)4 Tuesday 11h October, 2011 7 9:00 - 10:30 Computer-based components for countries intending to participate in Problem Solving or Problem Solving and CBAL for the Main Survey 8 Consortium staff will review field trial outcomes for the computer-based components, and the preparation and implementation tasks for the Main Survey, for countries implementing Problem Solving alone or both Problem Solving and CBAL components. NPM(1110)5 11:00 - 12:30 Field Trial overview and review, implementation of PISA Standards
11:00 - 12:30 Field Trial Outcomes 2 Questionnaire components 2.1 Consortium staff from DIPF and ACER will provide an overview of field trial data outcomes from questionnaires. NPM(1110)6
10
14:00 - 15:30 Main Survey Questionnaire Proposals 11 Consortium staff from DIPF and ACER will present options and proposals for the content of questionnaire instruments for the main survey. NPM(1110)7
12
16:00 - 17:30 Main Survey questionnaire proposals continued. 13 The session will continue, and will conclude with consortium staff outlining arrangements for discussion groups for tomorrow to consider the proposed main survey questionnaire content.
Wednesday 15th October 2011 14 9:00 - 10:30 15 16 17 Questionnaire discussion groups Groups will be formed to discuss the options and proposals on the contents of the PISA 2012 main survey questionnaire instruments.
11:00 - 12:30 Questionnaire discussion groups continued 14:00 - 15:30 Sampling for the main survey 18 Consortium staff from Westat will provide a briefing on all matters related to main survey sampling, and staff from ACER will discuss changes in KeyQuest since the field trial that are relevant to sampling. NPM(1110)11
19
16:00 - 17:30 National Centre presentations 20 A presentation from the Italian national centre (see summary in PISA12 Newsletter Issue 3, December 2010 on MyPISA). The presentation will report a study of missing responses from Italys PISA 2006 data, designed to investigate the characteristics of students failing to respond. A presentation by Lena Lindenskov and Peter Weng from the Danish national centre, reporting on a study of Danish student answers to 15 released mathematics tasks from PISA 2003, designed to stimulate interest in PISA results among teachers and trainee teachers.
21
12.3 A presentation from the US national centre to demonstrate the system used in the US to report PISA data and results back to sampled schools. Dinner and Night Safari Tour hosted by Singaporean National Centre. Thursday 13th October, 2011 22 9:00 10:30 23 Financial Literacy for countries participating in Financial Literacy Option Consortium staff from ACER will give a presentation on preparation, operational issues and proposed item selection for the Financial Literacy Option. NPM(1110)13 24 11:00 12:30 Assessment Operations 25 Adaptation and verification of main survey cognitive test materials and questionnaire materials Consortium staff from cApStAn and HallStat will give a presentation on consortium changes in the national version of main survey instruments and for 3
documenting national changes, verification of clusters and booklet shells, final optical check of booklets and verification of coding guides for cognitive test materials and questionnaire materials. NPM(1110)14a 14.2 Main survey test design Consortium staff from ACER will outline the proposed main survey test design and the options that are available to match the various optional components. NPM(1110)14b 26 14:00 15:30 Round Table Responses from each national centre 15.1 Each national centre will be invited to make comments about the field trial, changes proposed for the main survey, and especially about the item selections proposed for the questionnaires and in each cognitive domain. 27 16:00 17:30 Round Table Responses continued
Friday 14th October, 2012 28 8:00 11:00 National centre presentation: Singapore school visits 17.1 Arrangements will be made to pick interested participants up at the hotel at 8.00 am for a visit to one or more local schools. Participants will be dropped back at the meeting venue at 11.00am in time for the remaining meeting sessions. Details to be provided later. 18. 11:00 12:30 Main survey operations 18.1 Test operations for main survey Consortium staff will give a presentation on test operations for the main survey. Matters to be covered will include materials preparation, test administration and field operations arrangements, data capture procedures, arrangements for response coding, data submission arrangements. NPM(1110)18a 18.2 Site visits and quality monitoring Consortium staff will give a briefing on the proposed arrangements for site visits and for the training and work of PISA Quality Monitors during the main survey. NPM(1110)18b 19. 14:00 15:30 Consortium reaction to Round Table and discussion group input 19.1 Consortium staff will provide a response to issues and proposals that came up in Wednesdays questionnaire discussion groups and in the round-table discussion on Thursday. 19.3 Responses, conclusions and next steps 20. Consortium staff and Secretariat to give brief summary. 16:00 17:30 Reading Components for countries participating in this option. 20.1 A training workshop will be conducted by Consortium staff from ETS and ACER to prepare countries that are implementing the optional component related to Reading Components. NPM(1110)20
and