Survey Design

Survey Construction

 Start with a cover letter – even for online

surveys

 Divide the questionnaire into 4 parts…

 Initiation:
sets the stage and respondent’s expectation
 Body: group like items together by topic or scale type
 Conclusion: put threatening/sensitive questions here
 Demographics: put at the end as it is least important
Design the Survey Questions

 Keep it simple!!!!
 Focus – each item should address just one issue.
 Keep each item brief.
 Be clear about what you are asking.
 Use the same words as the least sophisticated
respondent.
 Use simple sentences
 Use structured questions as much as possible.
 Use appropriate scales.
Validity & Reliability
 Construct Validity: survey’s ability to measure what its suppose to.
 PCA, factor analysis, structural equation modeling

 Reliability: repeatable, survey’s ability to get the same data values

from several similar measures.
 Cronbach’s alpha, inter-item reliability

 Validity and reliability are compromised by

 Sampling error
 Small sample size (min 30, or 5-10 responses per survey item)
 Inappropriate subsample of the population (cows and donkeys)
 Sampling bias
 Shy field worker attempts to measure satisfaction
 Only approaches people who smile or look happy
 Reliability/Validity Map
RELIABILITY
High Low
High . . . .
V
.. ..
.. ..
A
L
. . . .
I

. . .
D
I
T .. ..
Y
.
Low
....
....
Types of Measurement Scales

 Multiple Response
 Likert scale (5-7 scale choices)
 Verbal frequency scale
 Ordinal scale
 Forced ranking scale
 Paired comparison scale
 Comparative scale
 Checklist
 Multiple Response
 Please check any type of newspaper you
regularly read for business news.
 Local, morning paper
 Local, evening paper
 Local, weekly paper
 Regional, weekly paper
 National, daily paper
 National, weekly paper
 Other, (what kind?) ___________________

Common, simple, versatile measurement scale.

 Likert Scale
__ I’m tall for my age.
Please pick a number from the
scale below to show how much __ I’m hungry.
you agree or disagree with
each statement and jot it in the __ Orange is my favorite colour.
space to the left of each item.
__ The elephant is purple.
Scale
1 = strongly agree __ Leftovers are delicious.
2 = agree
3 = neutral __ My credit card is maxed out.
4 = disagree
5 = strongly disagree __ Moose look like elk.

Used to measure degree of agreement. Easy to code.

 Verbal Frequency Scale
__ Seek out information about
 Please pick a number from the candidates before voting.
scale to show how often you do __ Vote along party lines.
each of the things listed below and
jot in the space to the left of the
item. __ Contribute money to a local
political campaign.
Scale
1 = always __ Contribute money to a national
political campaign.
2 = often
3 = sometimes
__ Volunteer to work on a local
4 = seldom
campaign.
5 = never
__ Volunteer to work on a national
campaign.

Used to measure how often an action is taken for several items.

 The Ordinal Scale
Ordinarily, when do you turn on a television set in your home on a
weekday? (Please check only one).
(1) __ The first thing in the morning
(2) __ A little while after awakening
(3) __ Mid-morning
(4) __ Just before lunch
(5) __ Right after lunch
(6) __ Mid-afternoon
(7) __ Early evening, before dinner
(8) __ Right after dinner
(9) __ Late evening
(10) __ Usually don’t turn it on

Use when a quantity or value is not sufficient information.

Forced Ranking Scale

Please rank the colas listed below in their order of preference. Jot
the number 1 next to the one you prefer most, number 2 by your
second choice, and so forth.

__ Pepsi-Cola
__ Coca-Cola
__ Royal Crown Cola
__ Generic Cola

Used for relative comparison. Number of items to rank should be less than 10.
Paired Comparison Scale
For each pair of soft drinks listed below, please put a check mark by
the one you most prefer, if you had to choose between the two.

(1) __ Pepsi-Cola (1) __ Royal Crown Cola

(2) __ Coca-Cola (2) __ Coca-Cola

(1) __ Royal Crown Cola (1) __ Royal Crown Cola

(2) __ Pepsi-Cola (2) __ Pepsi-Cola

(1) __ Royal Crown Cola  __ Royal Crown Cola

(2) __ Like Cola  __ Like Cola

Used to capture simple choice between pairs.

Use an even number of alternatives.
 The Comparative Scale

Compared to the previous rock band, the one

currently playing is… (check one space).

Outstanding Equal Terrible

_____ _____ _____ _____ _____

Used to compare an object with one or more other objects.

Use an even number of alternatives to avoid “fence-riding.”
 The Checklist

Please put a check mark in the space in front of any word or phrase
that describes your job.

__ Easy __ Technical __ Boring

__ Interesting __ Low-paying __ Strenuous
__ Routine __ Dead-end __ Changing
__ Important __ Demanding __ Temporary
__ Safe __ Exhausting __ Difficult
__ Rewarding __ Secure __ Slow-paced
__ Enjoyable __ Rigid __ Pleasant
__ Satisfying __ Degrading __ Risky

Used to describe a topic. Provides a profile.