Survey and Research Methods

Session 5: Questionnaire Design

Dr. Gangadhar Mahesh, B. Arch, M. Eng, Ph.D

NICMAR
ACM – Survey and Research Methods

1

1 Dr. Gangadhar Mahesh

Questionnaire
• Is a group or sequence of questions designed to obtain information on a subject from a respondent
- Include forms used for sample surveys and censuses as well as administrative forms - questions posed must conform to the survey’s Statement of Objectives and provide information that is useful for data analysis - All information requirements must be met, but each question should have an explicit rationale for being on the questionnaire - questions must be sequenced logically for the respondent - questions must be designed so that they are easily understood and can be accurately answered by respondents - should be tested before implementation using cognitive testing, focus groups, informal testing

• A well-designed questionnaire should:
- collect data efficiently with a minimum number of errors and inconsistencies - be respondent friendly and interviewer friendly (if interviewer-assisted) - lead to an overall reduction in the cost and time associated with data collection

NICMAR
ACM – Survey and Research Methods

2

2 Dr. Gangadhar Mahesh

Questionnaire Design 1
• Questionnaire design process starts with the formulation of survey objectives and information requirements and continues with the following steps:
1. Consult with data users and respondents 2. Review previous questionnaires
- other surveys on the same or a similar topic can be a useful starting point when considering how to draft a question - when trying to compare results from different surveys, the same questions should be used - documentation on the data quality from such surveys should be examined to assess the effectiveness of the questionnaire

NICMAR
ACM – Survey and Research Methods

3

3 Dr. Gangadhar Mahesh

Questionnaire Design 2
3. Draft the questionnaire
- Method of data collection - the availability of the data being sought - Characteristics of respondents - Response burden - Complexity of the data to be collected - Confidentiality and sensitivity of the information - Translation - Comparability of results with other surveys - Consistency - Respondent’s willingness to provide the data - The likelihood of nonresponse - Administrative requirements - The type of questions, the wording of each question - The layout of the questionnaire - Sources of measurement or response error - Processing of the questionnaire.

NICMAR
ACM – Survey and Research Methods

4

4 Dr. Gangadhar Mahesh

Questionnaire Design 3
4. Review and revise questionnaire
- should identify any obvious problems with the questionnaire such as mistakes in spelling or grammar or awkward wording - could include experts in the field of study, people who have experience designing questionnaires, interviewers or members of the population to be surveyed

5. Test and revise questionnaire
- on ‘typical’ respondents and on specific subpopulations that may have problems with particular questions - intended to identify difficulties and errors with the questions - identify whether the question order affects the interpretation of questions, whether instructions are clear and how respondents feel about the appearance of the questionnaire.

6. Finalise questionnaire

NICMAR
ACM – Survey and Research Methods

5

5 Dr. Gangadhar Mahesh

Questionnaire Testing Methods
• Informal testing (pretesting) • Cognitive methods • Focus groups • Interviewer debriefings

• Behaviour coding of interviewer/respondent interactions
• Split sample tests • Pilot tests

NICMAR
ACM – Survey and Research Methods

6

6 Dr. Gangadhar Mahesh

Types of Questions 1
• Open Ended Questions
- Those where response categories are not provided

• Advantages:
- Allow the respondent the opportunity for self-expression or elaboration without being influenced by the response categories that would be provided by a closed question - Allow to obtain the respondent’s own ‘natural’ wording - Allow to obtain exact numerical data - Allow to provide follow-up to closed questions

• Disadvantages
- Are burdensome since the respondent must determine the intent of the question and formulate an answer without the aid of response categories - Recording the answer is time consuming

NICMAR
ACM – Survey and Research Methods

7

7 Dr. Gangadhar Mahesh

Types of Questions 2
• Closed Questions
- Those where response categories are listed with the question

• Advantages:
- less burdensome to respondents - data collection and capture is cheaper and easier - are quicker and easier to answer - more likely to answer the question as intended - easier to analyse data that has been collected using closed questions, since the answers are more consistent and already grouped - if a question is being used for several surveys, using the same response categories facilitates comparisons across surveys

• Disadvantages
- effort is sometimes required to develop the response categories - the response categories must be mutually exclusive and exhaustive - can force the respondent to choose a category that does not correspond to his or her opinion, or to express an opinion when in fact he or she has none

NICMAR
ACM – Survey and Research Methods

8

8 Dr. Gangadhar Mahesh

Types of Questions 3
• Two Choice Questions
- Used to split the respondents into two distinct groups - Also used as screening questions to prevent respondents from being asked a series of questions that do not apply to them

• Multiple Choice and Checklist questions
- response categories should be mutually exclusive and exhaustive

• Ranking Questions
- burdensome to rank categories - sizes of the rank intervals are unknown and unlikely to be equal - a respondent may assign two or more items the same ranking

• Rating Questions
- how many categories should the rating scale have? - whether or not the rating scale should have a neutral alternative?

NICMAR
ACM – Survey and Research Methods

9

9 Dr. Gangadhar Mahesh

Scales of Measurement and Data Analysis

NICMAR
ACM – Survey and Research Methods

10 Dr. Gangadhar Mahesh

• Graphic Rating Scales • Likert Type Scale
- Assume that the strength/intensity of experience is linear - Is a five (or seven) point scale

• Semantic Differential Scale
- used for measuring the meaning of things and concepts - is set up using polar adjectives

• Unipolar Vs. Bipolar Scales

NICMAR
ACM – Survey and Research Methods

11 Dr. Gangadhar Mahesh

Guidelines for Formulating Questions
• • • • • • Keep it Simple Define Acronyms and Abbreviations Ensure that Questions are Applicable Be Specific Avoid Double-Barrelled Questions Avoid Leading Questions
- Please tell me whether you agree with, disagree with, or have no opinion on the following: ‘Tourism is beneficial to Northumberland County and therefore should be promoted.’

• Avoid using Double Negatives
- Would you be for or against not allowing the sale of alcohol in convenience stores? - Would you be for or against allowing the sale of alcohol in convenience stores?

• Soften the Impact of Sensitive Questions • Ensure that Questions Read Well
NICMAR
ACM – Survey and Research Methods
12 12 Dr. Gangadhar Mahesh

Questionnaire Layout
• Introduction • Opening questions • Placement of sensitive questions • Placement of demographic and classification questions

• Transitional Statements
• Instructions • Format Considerations

NICMAR
ACM – Survey and Research Methods

13

13 Dr. Gangadhar Mahesh

Sign up to vote on this title
UsefulNot useful