Outline

Definitions: what exactly are surveys?

What kind of data are produced by surveys
and what are the strengths and weaknesses
of such data?
What research design issues are implicated
in the use of survey methods? What are their
advantages and disadvantages?
What are the options for methods of
administering surveys?
 Definition

A systematic collection and analysis of

data relating to the attitudes, living
conditions, opinions, etc. of a
population, usually taken from a
representative sample of the latter
(Oxford Dictionary)
 Survey = Questionnaire?
Structured observation
Diaries
Content analysis
Often are questionnaire-based
Less structured interviews?
 Defining through nature of
data?
Units of analysis: commonly individuals,
but may be households, countries etc.
Compare units of analysis, or sub-groups
of them (e.g. gender, class)
Quantitative data (how many?): usually
fixed responses coded numerically
But, like other data, survey data are
created rather than simply collected
 Strengths of survey data
Suited to large-scale research
Generalizable to populations
Relatively easy to prepare for analysis
Suited to complex statistical investigation
Easily replicable
Transparent
Convincing to non-experts
 Weaknesses of survey data

Better at providing breadth than depth

Better at answering What? than Why?
Does not explore meaning or significance
of response categories
Decontextualizes: responses might vary
under different conditions
Less flexible if problems or new questions
emerge: hence importance of pre-testing
 Surveys in research design
May be preceded by a more qualitative
research phase: raise significant issues
and provide foundation for questions
May be succeeded by a more qualitative
research phase: follow up interesting
findings and further questions in more
depth
Both approaches may help mitigate
weaknesses of survey data
 Different survey designs
Cross-sectional: one-off snapshot; do not
assess change over time
Longitudinal (or panel): examine same
respondents at two or more time points; aim to
assess change; may have experimental aspect;
disadvantages related to resources and attrition
Repeated cross-section: same question(s),
different time points, but different respondents;
amenable to retrospective or ad-hoc assessment
of change but less confidence that change has
really occurred
 Operationalization to answer
research questions
Concept(s) Indicator(s) Question(s)
Fieldwork Analysis Findings
(Defining) Concept
Indicator(s): what needs to be measured?
Questions: Existing? New? How many?
 Piloting and Pre-testing

Pre-administer survey to small number of

respondents from same
sample/population
Utilize expertise (specific or general)
Especially important for new questions
and/or new contexts
How many and who?
 Piloting: objectives?
Highlight redundancies or fundamental
deficiencies
Are questions/wording relevant to all?
Assess comprehension
Any unease or offence?
Distribution of response?
An interactive and reflective process
 Cognitive interviewing/testing

Potential to highlight differences in

understanding
In context of survey research, is used
to study the manner in which targeted
audiences understand, mentally process,
and respond to the materials [e.g. survey
questionnaires] we present (G. Willis,
2004: 3)