Questionnaire Design

Questionnaire Design 1
 Learning Objectives

Upon completion of this chapter, you will be able to:

 Understand the meaning and importance of questionnaire

 Understand three stages of the questionnaire design process
 Learn about pre-construction phase of the questionnaire
 Learn about construction phase of the questionnaire
 Learn about post-construction phase of the questionnaire

Questionnaire Design 2
 What is a Questionnaire?

 A questionnaire consists of formalized and pre-specified set of

questions designed to obtain responses from potential
respondents.
 Questions in the questionnaire reflect the research objective
under investigation.
 Questionnaires are generally situation and culture specific.
 Questionnaire design process requires a careful attention to
each step as the questionnaire or research instrument should be
adapted to the specific cultural environment and should not be
biased in terms of any one culture (Malhotra et al., 1996).

Questionnaire Design 3
 Questionnaire Design Process

 Designing of the questionnaire is a systematic

process. This section explores the systematic process
of questionnaire design in three phases: pre-
construction phase, construction phase, and post-
construction phase.

Questionnaire Design 4
Figure 4.1: Steps in questionnaire design process

Questionnaire Design 5
 Phase I: Pre-Construction Phase

 Phase I is the pre-construction phase of the questionnaire

design process. It consists of three steps: specific required
information in the light of research objective, an overview of
respondent’s characteristics, and decision regarding selecting an
appropriate survey technique.

Questionnaire Design 6
Step 1: Specific Required Information
in the Light of Research Objective

 The first and the foremost important decision relates to finding

the required information through a questionnaire.
 To generate specific information, the researcher should clearly
define the objective and the other research components such as
theoretical model, research questions, and hypotheses.
 Clarity in all these components is very important as these only
will provide the base of specific information to be collected
through survey.

Questionnaire Design 7
 Step 2: An Overview of
Respondent’s Characteristics

 While collecting the information, an overview of the

respondent’s characteristics is a vital consideration.
 A researcher must construct the questionnaire in the light of the
respondent’s profile.
 Even while constructing a questionnaire, a researcher has to
keep the profile of a probable respondent in mind.
 Various factors such as the respondent’s qualification, age,
experience, income, marital status, occupation, and so on have
a decisive and impactful role in conducting a research.

Questionnaire Design 8
Step 3: Decision Regarding Selecting an
Appropriate Survey Technique
 While constructing a questionnaire, a deep thinking process is
required to select an appropriate survey technique.
 Chapter 7 provides a detailed discussion of various survey and
observation techniques and its relative advantages and
disadvantages. The questions in the questionnaire must be
constructed, sequenced, and placed according to the mode of
the survey.

Questionnaire Design 9
 Phase II: Construction Phase
 Phase II is the real construction phase of the questionnaire
design process.
 It consists of six steps:
 Decision regarding question format :structured questions versus
unstructured questions
 Decision regarding question relevance and wording
 Decision regarding question sequencing
 Decision regarding question response choice
 Decision regarding the questionnaire layout
 Producing first draft of the questionnaire.

Questionnaire Design 10
Step 1: Decision Regarding Question
Format: Structured Questions Versus
Unstructured Questions
 Questionnaires use two types of question formats. These are
open-ended questions and closed-ended questions.
 In the case of a “closed-ended” question, the respondent has to
format the judgment to fit the response categories and when
“open-ended” questions are used, the judgment has to be
verbalized into a preliminary answer (DeLeeuw, 2001).
 The closed ended question format can be further divided into
dichotomous, multiple-choice questions, and scales. The
following sections focus on open-ended questions and closed-
ended questions.

Questionnaire Design 11
 Open-ended Questions

One of the major limitations is to handle the interviewer and the

interpretation bias.

Questionnaire Design 12
 Closed-ended Questions
 Closed-ended questions are structured questions. The question
structure does matter (Connolly et al., 2005).
 The closed-ended questions provide response alternative to the
respondents instead of giving them a free-to-express response
option.
 The choice offered to the respondents can be either in the form
of a rating system or a set of response alternatives.
 These response alternatives are presented to the respondents
and they select the most appropriate one.
 The closed-ended questionnaires are generally cheaper, more
reliable, and faster to code, and analyse the collected data
(Swamy, 2007).
 The closed-ended question format can be further divided into
dichotomous, multiple-choice questions, and scales.

Questionnaire Design 13
 Dichotomous Questions
Dichotomous questions have only two response alternatives usually
presenting the two extremes “yes” or “no.” To make the alternatives
balanced, the researchers often present a third neutral alternative
“don’t know.”

Questionnaire Design 14
Multiple-Choice Questions

Questionnaire Design 15
Step 2: Decision Regarding Question
Wording

 The negative wordings impact the process of interpreting the

questions, leading at least some respondents to misinterpret how
to respond and, thus, reducing or destroying the usefulness of
the questions or the series of questions (Johnson et al., 2004).
 In some cases of conducting personal interviews, the researchers
feel that they will be able to overcome the wrong wording during
the discussion, but they suggest a different understanding.
 The effects of question wording on response accuracy did not
seem to be moderated by the respondent’s knowledge gained via
discussion (Bickart et al., 2006).

Questionnaire Design 16
Figure 4.2: Decision parameters regarding question wording

Questionnaire Design 17
Question Wordings Must Be Simple and
Easy to Understand

Questionnaire Design 18
 Vague or Ambiguous Words
Must Be Avoided

Some words such as “often,” “occasionally” and “usually,” “how

long,” “how much,” and “reasonably well” may be confusing for a
respondent because these words specify a specific time frame.

Questionnaire Design 19
 Double-Barrelled Questions
Must Be Avoided
Double-barrelled questions are those with wordings such as “and”
or “or.” In a double-barrelled question, a respondent may agree to
one part of the question but not to the other part.

Questionnaire Design 20
Avoid Leading and Loaded Questions
A leading question is the one which clearly reveals the researcher’s
opinion about the answer to the question.

Identifying the loaded question bias in a question requires more

judgment because the wording elements in a loaded question
allude to the universal belief or rules of behaviour (Burns & Bush,
1999).

Questionnaire Design 21
 Split-Ballot Technique
 A leading question generally emphasizes either the positive or
negative aspect of the question.
 Sometimes, it becomes necessary to ask a question with either
the positive or negative aspect.
 In this situation, a split-ballot technique is used to avoid bias
due to positive or negative aspect of the question.
 This technique involves the construction of a single question in
two alternative phrases, and the question based on one phrase
is administered to half of the respondents and the question
based on the other phrase is administered to the other half of
the respondents.

Questionnaire Design 22
Split-ballot technique

Questionnaire Design 23
 Avoid Using Overstated Words

The answer will always be overblown due to the first part of the
question, which generates a worry in the mind of the respondent
and results in a positive answer, which is not possible otherwise. A
more poised way of asking the same question is shown below.

Questionnaire Design 24
Implied Assumptions Must Be Avoided

Above question has an implicit assumption that the discount policy

on bulk purchase offered by Company “A” is working excellent and
by answering “yes,” the company will continue its policy.

Questionnaire Design 25
 Respondent’s Memory Should
Not Be Overtaxed

Questionnaire Design 26
 Generalization and Estimation
Must Be Avoided

Generalization means respondent’s belief, “what must happen” or

“what should happen.”

Questionnaire Design 27
 Respondent’s Ability to Answer
Must Be Considered
A question targeted to officers older than 55 years to assess the
importance of Internet banking is as follows:

Targeting following question to young respondents may not be an

appropriate choice.

Questionnaire Design 28
Step 3:Decision Regarding Question
Sequencing

 Question sequence also plays a key role in generating the

respondent’s interest and motivation to answer the question.
Questions should have a logical sequencing in the questionnaire
and should not be placed abruptly.
 To facilitate the responses, a researcher has to follow some
logical steps in sequencing the questions in the questionnaire.
This arrangement usually requires the considerations as shown
in Figure 4.3.

Questionnaire Design 29
 Figure 4.3: Decision parameters regarding question sequence

Decision regarding question sequencing

Identification
Difficult to
Screening Opening Transition and Logical order
answer
questions questions statements categorization of questioning
questions
questions

Questionnaire Design 30
 Screening Questions
 Researchers generally begin with some screening questions to
make sure that the target respondent is qualified for the
interview.
 In some cases, when the researcher is very sure about the
qualification status of the respondent, he or she does not
incorporate the screening question and starts from some
“opening questions.”

Questionnaire Design 31
 Opening Questions
 The opening questions should be simple, encouraging, and trust
building. From the research objective point of view, these
questions may sometimes be little irrelevant but should be good
initiators.
 These questions should not seek in-depth information and
should be as general as possible.
 For example, a microwave company, trying to assess “shift in
consumer attitude” from traditional way of cooking, should ask a
first opening question as follows:

Questionnaire Design 32
 Transition Statements
 The movement from one set of questions to another set requires transition
statements.
 For example, a mineral water bottle manufacturing company is encouraged with the
expanding market.
 The company wants to assess the potential future market and hence conducted a
survey on non-users.
 Its researchers have identified various variables to get the potential use, of which
“awareness” and “taste” are important.
 It has prepared the first 11 questions with the first 5 questions based on the
“awareness” and the next 6 questions on “taste.”
 After asking the first set of 5 questions, a researcher moves to the second set of 6
questions to get the potential consumer feeling for mineral water taste.
 Thus, before asking the next set of 6 questions, a transition statement is required to
make the respondent familiar with the coming questions.
 Hence, a transition statement “Now, I would like to understand your opinion about
the mineral water taste” will develop respondent’s connectivity for the next set of 6
questions related to “taste,” and he or she will be in a comfortable state of mind to
answer these questions.

Questionnaire Design 33
 Difficult to Answer Questions
 Difficult to answer, sensitive, or complicated questions should
be placed later in the questionnaire.
 Placing it first will confuse the respondent and he or she will
tend to terminate the interview process.
 Asking difficult questions first in a telephone interview reduces
a respondent’s interest in the interview process and he or she
tends to terminate the interview. Under telephone interview
conditions, substantively related questions affect the responses
to the target question only when asked first (Schwarz & Hippler,
1995).

Questionnaire Design 34
Identification and Categorization
Questions
 Identification questions are used to generate some basic
identification information such as name, mailing address, office
phone number, personal phone number, or cell phone number.
 Categorization questions are mainly used to generate
demographic information.
 For example, researchers generally want to generate the
information related to age, experience, gender, and occupation
of the respondents.

Questionnaire Design 35
 Logical Order of Questioning
 In a questionnaire, the questions must flow in a logical
sequence. There are at least three approaches to suggest the
roadmap to place the questions in a logical sequence; they are
funnel technique, work technique, and sections technique.
 Funnel technique suggests asking general questions first and
then the specific questions.
 Work technique suggests that difficult-to-answer, sensitive, or
complicated questions should be placed later in the
questionnaire.
 The third technique is the section technique in which questions
are placed in different sections with respect to some common
base.

Questionnaire Design 36
Step 4: Decision Regarding Question
Response Choice

 It is important to understand that too many response choices

will burden the respondent and he or she will be perplexed
while answering.
 Few response choices will not be able to cover all ranges of
possible alternatives.
 As a general rule, the researchers present a question with five to
seven response alternatives.

Questionnaire Design 37
 Step 5: Decision Regarding
Questionnaire Layout

 Questionnaire layout is important to enhance the response rate.

A recent study (Lagarace & Washburn, 1995) revealed that a
user-friendly format, and to some extent colour, is valuable to
increase mail survey response rate.
 The appearance of a questionnaire is particularly important in
mail surveys because the instrument, along with the preliminary
letter and/ or cover letter, must sell itself and convince the
recipient to complete and return it (Boser, 1990).
 It has been observed that the respondent emphasizes the
questions that are placed at the top of the questionnaire
compared with that at the bottom.

Questionnaire Design 38
Step 6: Producing First Draft of the
Questionnaire

 Printing on a poor, quality paper or an unprofessional look of

the questionnaire may generate a non-serious feeling among the
respondents.
 So, the questionnaire may be printed on a good, quality paper
and must have a professional look.
 The appearance of the front cover on a mail questionnaire and
the nature of first questions have been purported to have an
important influence on the respondent’s decision to complete
the questionnaire (Frey, 1991).

Questionnaire Design 39
Phase III: Post-Construction Phase
 Phase III is the post-construction phase of the questionnaire
design process. It consists of four steps:
 Pre-testing of the questionnaire
 Revisiting the questionnaire based on the inputs obtained from
the pre-testing
 Revising final draft of the questionnaire
 Administering the questionnaire and obtaining responses.

Questionnaire Design 40
Step 1: Pre-Testing of the Questionnaire
 Pre-testing of the questionnaire involves administering the
questionnaire to a small sample of the population to identify
and eliminate the potential problems of the questionnaire, if
any.
 Testing a questionnaire can be a time-consuming process, and
this stage of questionnaire has often been overlooked in the
past in researcher’s eagerness to start collecting data (Williams,
2003).
 Researchers generally use two common procedures to pre-test:
protocol analysis and debriefing.

Questionnaire Design 41
Step 1: Pre-Testing of the Questionnaire
(Cont.)
 Using protocol analysis, a researcher asks the respondent to “think
aloud” while answering the question.
 Debriefing is an interview conducted when a respondent has filled the
questionnaire.
 After completing, the respondents are informed that the questionnaire
they have just filled was a pre-test questionnaire and are requested to
share their views about various dimensions of the questionnaire.
 They are also requested to find out the problems with the
questionnaire, which they realized while filling it.
 While taking a decision about the sample size for pre-testing, as a
thumb rule, a researcher should keep the nature of population diverse
in mind.
 More heterogeneity in population requires relatively large size of the
sample compared with the situation when population is relatively
homogeneous.

Questionnaire Design 42
Step 2: Revisiting the Questionnaire Based
on the Inputs Obtained from Pre-Testing

 To enhance the accuracy, after incorporating suggestions from

the pre-testing, a researcher can go for second pre-testing.
 It is always possible that the second pre-testing may also reveal
some of the problems of the questionnaire.
 Pre-testing of the questionnaire might have provided many
inputs in a subjective manner.
 At this stage, the researcher must objectively incorporate all
the inputs obtained from the pre-testing exercise.
 All the parameters related to the question wording (as
discussed in the previous sections) must be carefully
considered.

Questionnaire Design 43
Step 3: Revised Final Draft of the
Questionnaire

 At this stage, the researcher once again carefully examines the

questionnaire.
 Insertions and deletions of the previous stage must be re-
checked to provide the desired accuracy.
 The researcher makes the questionnaire “ready to administer”
by eliminating all the minute mistakes and tries to make it error
free.
 After careful examination of all the incorporations obtained from
pre-testing, the researcher is now ready to have the final draft of
the questionnaire and administers to the sample taken from a
target population.

Questionnaire Design 44
 Step 4: Administration of the
Questionnaire and Obtaining Responses

 At this stage, the researchers administer the questionnaire to

the respondents and obtain the responses.
 These responses are coded, data are tabulated, and appropriate
statistical techniques are applied to analyse the data.
 Chapter 9 describes the data preparation and data coding.
 Chapters 10–18 open the detailed discussion related to data
analysis.

Questionnaire Design 45