Issues in Using SPSS in an Educational Research Environment

Ian Schagen National Foundation for Educational Research 11th November 2005

What Is NFER?
UKs leading independent educational research organisation Income largely through sponsored research government, quangos, charities, other organisations Main office in Slough smaller offices in York and Swansea; links with Queens, Belfast

What is SRAG?
Statistics Research and Analysis Group 12 statisticians working as a team Responsible for all analysis of quantitative data throughout NFER Work closely with researchers as part of the project team Analysis of survey data Analysis of national test data Analysis of test development data

Why Use SPSS ?

Most analysis uses SPSS Some specialist software Excel for presenting results to schools etc. Use largely historical Investment in macros and standard bits of code Accessibility for researchers

Use of SPSS Syntax

We use syntax for all our work to provide an audit trail and to aid QA Need to ensure one-to-one link between syntax and output (same name) Dangers of point and click approach Able to pick up standard syntax and macros tested and documented But beware of black box mentality

Why I Use SPSS-6

Better

output Batch mode available (from DOS)

Example Output SPSS-11

MARKER Marker ID N Valid 384 Missin g 384 384 332 361 376 383 355 384 380 369 375 376 376 384 384 353 353 Q1 Q1 Q2 Q2 Q3 Q3 Q4A Q4a Q4B Q4b Q5 Q5 Q6 Q6 Q7 Q7 Q8 Q8 Q9 Q9 Q10 Q10 Q11 Q11 Q12 Q12 Q13 Q13 Q14 Q14 Q15 Q15 Q16 Q16

52

23

29

15

31

31

Frequency Table
MARKER Marker ID

Frequency Valid 1 2 3 5 6 7 8 9 Total 48 48 48 48 48 48 48 48 384

Percent 12.5 12.5 12.5 12.5 12.5 12.5 12.5 12.5 100.0

Valid Percent 12.5 12.5 12.5 12.5 12.5 12.5 12.5 12.5 100.0

Cumulative Percent 12.5 25.0 37.5 50.0 62.5 75.0 87.5 100.0

Q1 Q1

Frequency Valid 1 384

Percent 100.0

Valid Percent 100.0

Cumulative Percent 100.0

Example Output SPSS-6

21 Oct 05 File: MARKER EKQ - Marker Consistency within HA Reading 8253-54-KS2 ENGLISH, L3-5 READING-HEARTBEAT - HA Marker ID Valid Percent 12.5 12.5 12.5 12.5 12.5 12.5 12.5 12.5 ------100.0 Cum Percent 12.5 25.0 37.5 50.0 62.5 75.0 87.5 100.0 Page 2 Value Label Value 1 2 3 5 6 7 8 9 Total Valid cases 384 Frequency 48 48 48 48 48 48 48 48 ------384 0 Percent 12.5 12.5 12.5 12.5 12.5 12.5 12.5 12.5 ------100.0

Missing cases

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Q1 Q1 Valid Percent 100.0 ------100.0 Cum Percent 100.0

Value Label

Value 1 Total

Frequency 384 ------384 0

Percent 100.0 ------100.0

Valid cases

384

Missing cases

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Advantages of SPSS-6 Output

Consistent font throughout ASCII file can be opened by any editor Output breaks across pages sensibly Can be imported to Word etc easily Frequency tables have number of cases underneath each table Titles appear routinely Errors can be searched for easily

On the Other Hand ...

SPSS-11 etc are more powerful, and there are some things they do that cant be done in SPSS-6 Some output formats may be better in some cases

How We Use SPSS

Often, all analysis is carried out in SPSS, from raw data to final results e.g. basic frequencies, crosstabs, multiple response, factor analysis, regression. Other times, SPSS is used to provide an interface between different pieces of software A number of standard SPSS macros have been created to do certain tasks

Example: Feedback to Schools

On occasions we want to provide schools with graphical displays of their pupils attitudes compared with others. Data is cleaned in SPSS and aggregated to school level. This is output to an Excel file, one row per case. An Excel template uses a macro to read the school data and print one copy for each school.

Programme for International Student Assessment 2006

Student Questionnaire for Typical School
School ID: 99 Number of students responding in all schools: 4000 Number of students responding in your school: 30
Do you agree with the statements below?
I generally have fun when I am learning science I enjoy reading about science I am happy doing science problems I get excited about going to science class I enjoy acquiring new knowledge in science I like using laboratory materials I enjoy looking up things about science on the internet For me, a science test is a challenge that is enjoyable I like to read about science and technology advances in the newspaper I am interested in learning about science Watching science programmes on TV is interesting What I most like about science is how interesting the subject material can be

10

20

30

40

50

60

70

80

90

100

% of students saying "yes"

All Schools

Your School