INTRODUCTION TO QUESTIONNAIRE DESIGN

OBJECTIVES

To understand:
Objectives of questionnaires  Advantages and disadvantages  Design of questionnaires  Type of questions used  Common problems and pitfalls

WHAT IS A QUESTIONNAIRE?

An instrument (form) to

collect answers to questions collect factual data

gathers information or measures

A series of written questions/items in a fixed, rational order

WHY USING A QUESTIONNAIRE? A well designed questionnaire: Gives accurate and relevant information to your research question  Minimizes potential sources of bias  Will more likely be completed  ⇒As simple and focused as possible .

results may be skewed (higher response rate = less chance for skew) .Statistical Terms • Population: Discrete group whose members can be defined by a set of at least one shared characteristic • Sample: A selection of that target population (can also be a subsample) • Random Sample: Sample in which every member of a given population has an equal chance of being selected • Margin of Sampling Error (MOSE): Every random sample has a margin of error (statistical price you pay for not interviewing everyone) • Response Rate: Rate at which people agreed to participate in survey. If those who refuse are different from those who agree.

.

ADVANTAGES OF QUESTIONNAIRES  Can reach a large number of people relatively easily and economically quantifiable answers easy to analyse  Provide  Relatively .

.

DISADVANTAGES OF QUESTIONNAIRES  Provides   only limited insight into problem Limited response allowed by questions Maybe not the right questions are asked  Varying  response Misunderstanding/misinterpretation  Need  to get it right first time Hard to chase after missing data .

TYPES OF QUESTIONNAIRE to face  Telephone Interviewer -administrated  By  Face mail  E-mail/Internet Self-administrated .

SELF-ADMINISTERED QUESTIONNAIRE  Advantages: Cheap and easy to administer  Preserves confidentiality  Completed at respondent's convenience  No influence by interviewer  .

SELF-ADMINISTERED QUESTIONNAIRE  Disadvantages: Low response rate  Questions can be misunderstood  No control by interviewer  Time and resouces loss  .

INTERVIEW-ADMINISTERED QUESTIONNAIRE  Advantages: Participation by illiterate people  Clarification of ambiguity  Quick answers  .

INTERVIEW-ADMINISTERED QUESTIONNAIRE  Disadvantages: Interviewer bias  Needs more resources  Only short questionnaires possible   Especially on telephone  Difficult for sensitive issues .

.

Fielding a survey and collecting the data 5. Drawing a sample 3. Analysis 6. Deciding on a mechanism 2.Basic Survey Process Steps: 1. Reporting . Designing the questionnaire 4.

THE QUESTIONNAIRE DESIGN PROCESS Convert the Research Objectives into the Information Needed Method of Administering the Questionnaire Content of the Questions Motivating the Respondent to Answer Determining Type of Questions Question Design Criteria Determine the Questionnaire Structure Physical presentation of the Questionnaire Pilot Testing the Questionnaire Administering the Questionnaire .

CONVERTING THE RESEARCH OBJECTIVES INTO INFORMATION AREAS .

ADMINISTRATION AND DESIGN IMPLICATIONS .

4. 7. T. Now of the remaining nine serials name your most favorite serial (continue the same process till the person is left with the last card) 1.V. 3. 6. (Record the serial and keep this card with you). 9. I want you to examine them carefully (give her some time to read all the names). 5. 10. Each card will have the name of one television serial (Hand over the cards to the respondent in a random order). 2. SERIAL 1 2 3 4 5 6 7 8 9 10 RANK ORDER ___________________ ___________________ ___________________ ___________________ ___________________ ___________________ ___________________ ___________________ ___________________ ___________________ .MODE OF ADMINISTRATION: SCHEDULE Now I am going to give you a set of cards. 8. I would request you to hand over the card which has the name of the serial you like to watch the most.

please remember that the higher the number the more you like watching it. However. Bandini 7. Laptaganj 8. Tere liye 10. Sasural Genda Phool 4. where 1 means I do not like watching it and 10 means I really like watching it. the next name is---------------------. serials. I am going to name the serials one by one.V. Sajan Ghar jaaana Hai 9. You need to use a 1 to 10 scale.k. SERIAL 1. the serial’s name is-------------------. And so on till all the 10 names have been read out and evaluated. thank you. Uttaran 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 4 4 4 4 4 4 4 4 4 4 5 5 5 5 5 5 5 5 5 5 6 6 6 6 6 6 6 6 6 6 7 7 7 7 7 7 7 7 7 7 8 8 8 8 8 8 8 8 8 8 9 9 9 9 9 9 9 9 9 9 10 10 10 10 10 10 10 10 10 10 . Pathshala 6.MODE OF ADMINISTRATION: TELEPHONE Please listen very carefully. I am going to slowly read the name of ten popular T. Sathiya 3. For those in between you may choose any number between 1 and 10. O. Bidai 5. I want to know how much you prefer watching them. Please use a number between 1 and 10 as I had told you. So. Balika Badhu 2. Now. In case the name is not clear I will repeat the list again.

4. .Please carry out this process till you have ranked all 10. 7. pick the second most preferred serials and give it a rank number of 2. Then from the rest of the nine. You are also requested not to give two serials the same rank. SERIAL Balika Badhu Sathiya Sasural Genda Phool Bidai Pathshala Bandini Laptaganj Sajan Ghar Jaaana Hai Tere Liye Uttaran RANK ORDER ___________________ ___________________ ___________________ ___________________ ___________________ ___________________ ___________________ ___________________ ___________________ ___________________ 1. The one you prefer the least should have a score of 10. 8. 3. to this you may give a rank of 1. 2.MODE OF ADMINISTRATION: MAIL In the next question you will find the names of ten popular Hindi serials that are being aired on television these days. 9. You are requested to rank them in order of your preference to watch these programmes. The basis on which you decide to rank the serials is entirely dependent upon you. 6. 10. 5. Start by identifying the serial which is your most favorite. Once again you are asked to rank all the 10 serials.

STAGES IN DESIGNING A QUESTIONNAIRE (1) Planning the study:  Decide on goals  Identify risk factors  Know   for getting bitten by dogs the subject a hypothesis Literature. attitude towards dogs . outdoor activities. owning dog. experts on dog bites Postmen more likely to get bitten by dogs than the normal population  Formulate  Define  information needed to test hypothesis Occupation.

STAGES IN DESIGNING A QUESTIONNAIRE (2) Determine study population:  Know  the respondents Occupation  Special sensitivities  Education  Ethnic  Language ⇒Questionnaire needs to be adapted to your population. not the opposite! .

STAGES IN DESIGNING A QUESTIONNAIRE (3) Design questions:      Content of the questions Format of the questions Presentation and layout Coding schedule (if appropriate) Pilot and refine questionnaire .

WHAT MAKES A WELL DESIGNED QUESTIONNAIRE?  Good appearance  easy on the eye   Short and simple Relevant and logical ⇒ High response rate ⇒ Easy data summarisation and analysis .

QUESTION ORDER  Decide on order of items/questions Easy  difficult  General  particular  Factual  abstract  Where to place sensitive questions? Be aware of ordering effects!  .

QUESTION ORDER Questions should be ordered so as to seem logical to the respondent  First questions should be relevant and easy  Questions are effectively ordered from most salient to least salient  Demographic questions should not be covered at the beginning  Potentially objectionable questions are placed near the end  .

QUESTION ORDER  Group questions by topic/ response options questions  Starting      Simple With closed format Relevant to main subject Non-offending Neither demographic nor personal questions  Don‟t put most important item last .

QUESTIONNAIRE INTRODUCTION  Survey/interview introduction       Who you are/ what is your topic Why you are investigating Where you obtained the respondent‟s name How and where you can be contacted Guarantee of confidentiality Length of interview (be honest) ⇒ Usefulness of study should be clear to all respondents .

CONTENT OF QUESTIONS  Clear focus on research question   Avoid sidetracking Avoid unnecessary information  Demographic information  Contact information (if non-anonymised) .

DO: Use simple wording  Be brief  Be specific  .

DO NOT: Be vague  Be condescending or talk down to respondent  Use biased wording  Use abbreviations or scientific jargon  Use objectionable questions  Be redundant  .

public  Keep sentences simple and short  Define key words  Remember option “don‟t know”  .FORMAT OF QUESTIONS Adjust to responding audience  Professionals vs.

FORMAT OF QUESTIONS  Ask for one information at a time Do you own a dog or have frequent contacts with dogs? Yes No • • Use mutually exclusive and exhaustive answer options  Vertical order of answer options  .

How often did you touch a dog during the past 3 months? Once Twice Three times or more Not at all Don´t know • • • • • .BE ACCURATE Do you often touch dogs? Yes • No • vs.

BE APPROPRIATE Are you a drunk? Yes • No • vs. How often have you consumed alcoholic beverages during the past 6 months? Daily 2-6 times/week Once a week Less than once a week Don´t know • • • • • .

Which beverage did you consume? Water Beer Wine Karkadé None of them Don´t know • • • • • • .BE OBJECTIVE Did you drink the strange brownish drink in Prague? Yes • No • Vs.

BE SIMPLE Did you smoke not less than a mean amount of 7 cigarettes/2 days from 1999 onwards? Yes • No • vs. Did you smoke an average of 2 pack of cigarettes/week for the last 5 years? Yes • No Don´t know • • .

BIAS Bias = systematic differences in the measurement of a response .

INFORMATION BIAS  Recall bias  Cases more likely to remember than controls  Observer bias Different interviewer – different interpretations  Different interpretation of similar questions  Reduce by structured questionnaire  .

elderly  Reduce Ensure high response rate  Random choice of interview partners  Correct during analysis (eg age. sex)  .NON-RESPONSE BIAS  Those who respond are different from those who do not  Telephone interviews: more females.

FORMAT OF QUESTIONS Two main question formats  Closed format  forced choice    Always Sometimes Never    Yes No Don’t know  Open format  free text What is your most distressing symptom? Please describe: ________________________________________________ ________________________________________________ .

OPEN OR CLOSED? Closed  Advantages:      Simple and quick Reduces discrimination against less literate Easy to code. analyze Easy to compare Easy to report results . record.

CLOSED QUESTIONS  Disadvantages:   Restricted number of possible answers Loss of information Insert field „others“  Possible compromise:  .

OPEN QUESTIONS  Advantages: Not directive  Allows exploration of issues to generate hypothesis   qualitative research. focus groups. trawling questionnaires Used even if no comprehensive range of alternative choices  Good for exploring knowledge and attitudes  Detailed and unexpected answers possible  .

OPEN QUESTIONS  Disadvantages:      Interviewer bias Time-consuming Coding problems Difficult to analyze! Difficult to compare groups .

CLOSED QUESTIONS 1. Straightforward response What is your age in years? How long have you owned a dog? What is your sex (gender)? Male  Female  ___ years ___ years Did you stay in Hotel X on 23/7/05? Yes  No  Don’t know  .

Checklist Which of the following outdoor activities did you do last week?      Running Walking Hiking Cycling Swimming      .CLOSED QUESTIONS 2.

CLOSED QUESTIONS
3. Rating scale

Did you do use sunscreen during the following outdoor activities during the past six months? Always Sometimes Seldomly Never Running     Walking     Cycling    

CLOSED QUESTIONS
4. Rating scale Numerical How useful would you think that information on the risk of biting from stray dogs would be?

(please circle)

1 2 Not at all useful

3

4

5

6

7 Very useful

Analogue How much is your pain severe (put the tick on the line)
0 10

CLOSED QUESTIONS
5. Scales for measuring attitude (Lickert)
Stray dogs carry a higher risk of rabies No, I strongly disagree  No, I disagree quite a lot  No, I disagree just a little  I’m not sure about this  Yes, I agree just a little  Yes, I agree quite a lot  Yes, I strongly agree 

you won‟t know which „bit‟ people are answering: Have you ever had stomach ache and diarrhoea? Ambiguity.PROBLEMS AND PITFALLS  Avoid questions that ask two things at once ..... Do you go to the woods a lot?  .

PROBLEMS AND PITFALLS  Avoid jargon/abbreviations/slang How often do you get up at night to PU? (pass urine) Should IVDUs be treated in the community?  Avoid not mutually exclusive options What age are you? 16-20  20-25  25-30  35-40  .

PROBLEMS AND PITFALLS  Avoid leading questions Do you think that the food in the hotel made you sick? Did the hotel staff seem unhygenic to you? Do you agree that the hospital staff were close to exhaustion?   Avoid making questionnaire too long Typographical / spelling errors .

PILOTING AND EVALUATION Pilot with a similar group of people to your intended subjects  Highlights problems before starting  Effects of alternative wording  Overall impression on respondents and interviewers  Final polishing after several amendments  .

PRESENTATION AND LAYOUT  Clear consistent layout  Adequate space to answer  Large  font size Appropriate page breaks  Avoid experimental layouts  fancy logos  printed on recycled paper/is an equal opportunity employer etc  .

PRESENTATION AND LAYOUT  Using colour or printing questionnaire on coloured paper may help Use filter questions. if necessary Give clear instructions about how to answer the questions   .

WAYS TO INCREASE PERCEIVED REWARD Show positive regard  Say thank you  Ask them for “advice”  Give social validation  Give a tangible reward  Make the questionnaire interesting  State an upcoming deadline  .

WAYS TO DECREASE PERCEIVED COST Avoid subordinating language  Do not embarrass the respondent  Do not inconvenience the respondent  Make questions appear short and easy  Avoid asking personal information (if it must be asked. use „soft‟ wording and inform about confidentiality)  Keep subsequent requests similar  .

CREATE A NAVIGATIONAL PATH Instructions should be placed right where they are needed  Matrices are confusing: it is best to order questions from top to bottom. and left to right    Use larger font to attract attention Use color shading to attract attention or show groupings  Use spacing and similarities to show groupings .

WAYS TO ESTABLISH TRUST Provide a token of appreciation  Provide a sense of legitimate authority  Make completing the questionnaire seem important  Remind respondent of previous relationship with sponsor (if applicable)  .

thus increasing the validity of the questionnaire  Will much more likely be completed .SUMMARY A well designed questionnaire:  Will give appropriate data which allow to answer your research question  Will minimise potential sources of bias.

FINALLY. KEEP YOUR QUESTIONNAIRE SHORT AND THE QUESTIONS SIMPLE. FOCUSED AND APPROPRIATE .

Sign up to vote on this title
UsefulNot useful