5a_Survey Research

1
 descriptive research
 typical survey studies – assessing
– attitudes, opinions, preferences,
demographics, practices, and procedures
 e.g.
– how do teachers in our school district rate the
qualities of our new teacher evaluation
programme?
– What do school principals consider their most
pressing administrative problem?

2
Classifying Survey Research
 based on
– how data are collected
– how often a particular group is surveyed
A. collection of survey data
– (questionnaire, interview, telephone) self
report instruments
• respondents self-report their views
– observation
• researcher obtains data by watching participants

3
 interview & observation (qualitative)
– difference
• survey research plan is much more
structured and standardised
• predetermine the variables that will be
surveyed/observed
• researcher’s view is more important
• observers tend to be uninvolved with the
participants
• sample size & method of analysis

4
Classifying survey research
 B. Survey frequency
– Cross-sectional survey
• collecting data from selected individuals
in a single time period
• single, stand-alone study
• limitation
– does not provide sufficient perspective to
make needed decisions
• entire population – census survey

5
– Longitudinal survey
• collect 2 or more times to measure changes
over time
• study the dynamics of an issue over time
• e.g. study the development of abstract
thinking in primary school students from Year
1 to 4
(a) cross sectional approach – study samples of
students at each of the 4 primary levels
advantage – convenience
disadvantage - selecting comparable samples
(a) longitudinal approach – select a sample of Year 1,
study their development as they progress from year
to year.
advantage - no comparability problem
disadvantage - samples tend to shrink over time,
extended commitment 6
Conducting Self-Report Research

 collection of standardised, quantifiable

information from all members of a
population/sample
 must ask same questions
 collection of such questions -
questionnaire

7
Conducting a questionnaire (Q)
study
 Criticisms
– misuse
– carelessly and incompetently constructed
 paper-and-pencil Q
– advantages (in comparison to interview)
• requires less time, less expensive, collection from much
larger sample
– mailed
– personally administered
– telephone interview

8
 Stating the Problem
– Problem studied and contents of Q must
be of sufficient significance
• to motivate respondents
• to justify the research effort
– no trivial issues

9
 Selecting Participants
– simple random, stratified random sampling
(most common)
– cluster, systematic, nonrandom samples
(also used)
– whole population (if small)

10
 Constructing the Q
– attractive, brief, easy to respond to
– {sloppy, crowded, misspelled, lengthy Q,
require long written responses} –X
– important decision
• data collection method – mail, e-mail,
telephone, personal administration, and
interview
• strengths and weaknesses

11
– types of items
• scaled items (Likert and semantic differential),
ranked items, checklist items, free response
items
• most surveys - structured items (or closed-
ended items)
• unstructured items – include one or two
– opportunity to add information not tapped by
structured items
– potential problems
» no/unclear/useless response
» difficult to score
12
– avoid jargon
– be specific!
• Do not ask “Do you spend a lot of time each
week preparing for your classes?”
• Ask “How much time do you spend per week
preparing for your classes? (a) less than 30
minutes, (b) between 30 minutes and 1 hour,
(c) between 1 to 3 hours, (d) between 3 to 5
hours”
– indicate a point of reference
• Do not ask “How much time do you spend
preparing for your classes?”
• Ask “How much time per day do you spend
preparing for your classes?”
13
– Avoid leading questions
• Do not ask “Don’t you agree with the experts that…” or
“Would you agree with most people that …”
– Avoid touchy questions – might not reply honestly
• Do not ask a teacher if she or he sets high standards for
achievement
– Avoid question that assumes a fact not necessarily
true
• e.g. “Have you stopped stealing from the church poor
box?” – how do those who have never stolen respond
• “How many hours per week do you use your foreign
language laboratory?” - making assumptions
– Questions must be clear unambiguous

Important aspects of writing questionnaire items

14
– Good practice
• a brief statement describing the study and its
purpose at the top of the Q
• general (interesting, nonthreatening) to specific
• provide information on how to respond
– “Select the choice that you most agree with “
– “Circle the letter of your choice”
– “Rank the choices from 1 to 5, where 1 is the most
desirable and 5 the least”
– “Darken your choice on the answer sheet provided.
Please use pencil to record your choices.”
• Number pages and items – organising data for
analysis
• Don’t put very important questions at the end.
15
– Preparing the cover letter
• accompany mailed Q
• explains what is being asked and why
• sample
– Pretesting the Q
• before distributing – try it out in a pilot study
• information about deficiencies and suggestions for
improvement
• choose individuals – similar to research participants
• comments
– survey directions, specific items, commission and
omission

16
 Followup activities
– not everyone will return your Q
– recipients
• no intention of completing it
• put if off so long – forget it or lose it (followup)
– Initial followup strategy
• reminder postcard to
– everyone (anonymous) – “If you have already
responded, please disregard this reminder and thank
you for your cooperation.”
– specific participants (not anonymous)
• shortly after the cover letter deadline for
responding has passed
• send second set of Q + a new cover letter
17
Explanation

18
 • new cover letter
– repeat significance & purpose of study
– reemphasise the importance of their input
– suggest subtly that many others are responding –
peers found the study important
– Dealing with nonresponse
• response rate of 60% - generalisability?
• respondents and nonrespondents may be
different in some systematic way
• if you do not know how well the 60% represent
the population
– randomly select a small sample of nonrespondents
and gather demographic information
– significantly different – generalisability not present,
discuss in research report
19
Conducting and interview study
 oral, in-person administration of a Q to each
member of a sample
 advantages (in comparison to Q study)
– in-depth data
– questions that cannot effectively be structured
– flexible (adapt situation to each subject)
– may result in more accurate and honest responses
– interviewer clarify & explain
– followup of incomplete/unclear responses by
asking additional probing questions
20
 disadvantages (in comparison to Q study)
– expensive, time consuming – less respondents
– responses may be biased & affected by his/her
reaction to the interviewer
– requires interviewing skill
 telephone interviewing
– short, specific, not too personal, contains mainly
selection-type questions

21
 Constructing the Interview Guide
– Guide – questions to be asked, in what
order, how much additional prompting or
probing is permitted
– all interviews – conducted in same manner
– each question relates to a specific study
topic
– structured, semi-structured, or unstructured

22
– unstructured – time consuming & unproductive
• “What do you think about life in general?”
• “Tell me about yourself.”
– semistructured (commonly used) – more narrow
questions & issues
– structured question to focus on a topic then
semistructured questions to follow up on the
structured question
• “Are you in favour of or against the death penalty?”
• “Why do you feel that way?”

23
 Communication During the Interview
– First impression – important!
– establish rapport & putting interviewee at ease
– explain purpose of study & assure strict
confidentiality
– explain unclear questions
– sensitive to the reactions (off track – gently get
him/her back)
– avoid words or actions – cause respondent
unhappy or feel threatened
– no frowns and disapproving looks
24
 Recording Responses
– recorded manually by interviewer or
mechanically by a recording device
– space after each question
– write during (lengthy responses, make
respondents nervous) or shortly after (not
likely to recall every response exactly)
– mechanical recording device
• make respondents nervous but tend to forget
its presence)
• need to inform respondents and they agree to
its use
25
 Pretesting the Interview Procedure
– small group
– add, remove, revise interview questions
– better handle certain questions
– determine whether the resulting data can
be quantified & analysed in the manner
intended

26
Observational Research
 Major type of quantitative observational
research
– nonparticipant observation (naturalistic
observation and simulation observation)
• typically involves observation of human subjects
• observer
– not directly involved in the situation to be observed
– outside looking in
– does not intentionally interact with, or affect, the object of
observation

27