QUESTIONNAIRE DESIGN

DEFINITION- A set of questions designed to generate data necessary for achieving

objectives of research

I. RESEARCH OBJECTIVES, RESOURCES AND CONSTRAINTS- SELF-

EXPLANATORY

II. DETERMINE DATA COLLECTION METHODS

 Budget available
 Stimuli or tasks
 Length of questionnaire
 Structure of the questions
 Sample precision
 Incidence rate

III. DETERMINE QUETIONNAIRE RESPONSE FORMAT

Types of questions:

1. OPEN ENDED QUESTIONS- Respondant is given no choice followed by the

question, their response is recorded in full

2. CLOSED QUESTION- These are closed questions in which respondant is

given a choice to answer between alternatives. Types of closed questions:-
  NOMINAL QUESTIONS- Used to categorise factual info

 ORDINAL QUESTIONS- Identifies categories and puts them into

some kind of order. Ranking or rating questions.

 Interval Scales: Interval scales also identify categories

and put them in order but this time the intervals between
points are equal. For example, temperature.

 Ratio Scales: Ratio scales are very much like interval

scales but the important
the difference is there is a meaningful zero point so the
values are meaningful and be
compared mathematically. For example, age and salary

IV. DECIDING QUESTIONNAIRE WORDING

1. Wording should be appropriate to the target audience

 Comprehension- they should be able to understand the questions

 Recall- Phrase so that people can remember not forget if you are
asking about past activities

2. Unambiguous
It should be worded so that the audience interprets the answer the way they are
meant to be interpreted and not some other way cuz that will decrease the accuracy
of the result

3. Sensitiveness of the respondent

Word the question in such a way as to not harm the sensitiveness of the
respondent because if the topic is embarrassing or makes them uncomfortable
they may tend to lie or change the answer reducing accuracy.

1. AVOID DOUBLE-BARRELLED QUESTIONS

2. AVOID PROVERBS AND SAYINGS
3. AVOID DOUBLE NEGATIVES
4. AVOID ACRONYMS, ABBREVIATIONS, JARGON AND TECHNICAL
TERMS.
5. AVOID AMBIGUOUS WORDS.
6. AVOID WORDS WHICH HAVE VARYING MEANINGS
7. AVOID LEADING QUESTIONS
8. AVOID LOADED WORDS
9. AVOID HYPOTHETICAL QUESTIONS
10 AVOID VAGUE QUESTIONS
 V. ESTABLISHING FLOW AND LAYOUT

LAYOUT:

Introductions
- at beginning, at the beginning of sections
-explain the purpose
-state the organisation behind it
-confidentiality/anonymity
-data security
-contact details

Instructions
 should be clear, unambiguous
 should be distinct from the questions

E.g. How often do you buy any type of fish for your family? Please
circle the appropriate number.

Appearance
 Layout
 Colour
 Typeface

Length

FLOW OF QUESTIONS

AVOID;
• Ordering items in such a way that necessitates referral back to earlier questions
• Conditioning of later responses by earlier ones
• Preceding knowledge questions with other questions giving the answers
 VI. Evaluating the Questionnaire and Designing for Analysis

1. Questionnaire wording and appearance.

2. Collect all necessary data
3. Continuous vs. categorical data
4. Differentiate the different types of missing data
5. If the analysis is to include group comparisons, ensure all
groups answer the same set of questions

VII. PILOT THE QUESTIONNAIRE

- Helps eliminate problems

- Must be tested out on people who are similar to the
sample
- If the majority of the questionnaire is altered, the final
version should be piloted again

VIII. IMPLEMENTATION OF FINAL COPY

Approaching respondents

 Personal distribution and collection

 Personal distribution and postal or email returns

 Postal or email distribution and postal email returns

Increasing Response Rate

Advance warning
• Explanation of selection
• Sponsorship
• Envelope
• Incentives
• Publicity
• Confidentiality
• Anonymity
• Follow-ups

Coping with Non- Respondents

 Assessing sample representativeness

 Assessing response bias
 Accessing non-respondents
QUANTITATIVE DATA COLLECTION

• Qualitative Research – An unstructured, methodology based on small samples

designed to provide insight and understanding.

• Quantitative Research – A research methodology that seeks to quantify the data

and typically applies some form of statistical analysis.

Experiments

• Experiments are used to infer causality

• Three types of evidence are used to demonstrate a causal relationship:

▫ Concomitant variation
▫ Time order of occurrence
▫ Absence of other causal factors

The Three Components of Experiments are:-

1. DEPENDENT VARIABLE- The variable being acted upon. Also called

Test Subject
2. INDEPENDENT VARIABLE- The variable being changed which is the change
imposed. Also called the treatment
3. RESULTS RELATING TO THE CHANGE

Experiment’s validity

So how are we going to know that our experiment is a valid one and the relationship
that we claim exists? Two considerations will affect an experiment’s validity,

1. The influence the treatment had on the outcome

2. The influence of extraneous factors – extraneous factors are factors other than
the independent and dependent variables which may influence the results of the
experiment.

Impact of Extraneous Factors

• History and maturation
• Repeated testing
• Impact of the researchers
• Mortality of participants
• Selection errors
• Regression effects

Controlling Extraneous Variables

• Randomisation- using randomisation for selecting and assigning
• Matching- matching respondents based on the background to key variables that
effect dependent variable
• Statistical control- using statistical analysis to adjust for external factors
• Design control- using experimental design to control external factors

A Classification of Experimental Designs

There are 4 types of experimental design,

1. Pre-experimental designs – These are designs that don’t control for extraneous
factors by randomisation.

2. True experimental designs – These experimental designs the research can

randomly assign respondents to test groups and randomly assign treatments to
the test groups.

3. Quasi-experimental designs – These apply some of the procedures of true

experimental design but lack full experimental control.

4. Statistical designs – These are designs that allow for the statistical control and
analysis of external variables.

PRE-EXPERIMENTAL DESIGN

One shot case study – This is where a group of respondents are exposed to a
treatment and a single measure of the dependent variable is taken. E.g. taking you
exposing you to a film on fur trade and asking you about your attitudes after. You
can’t draw valid conclusions from this for a number of reasons,
i. We don’t know what your attitudes to the fur trade were before the film,
ii. The sample is not randomly selected, the researcher has selected the group and
younger people might be more affected than older age groups. This design lacks
control for extraneous variables, history, maturation, selection, mortality and
regression.

One Group pre-test-post-test design – In this design the group of respondents are
measured twice, so going back to the fur film the researcher would measure your
attitudes before showing you the film and after showing you the film. This allows the
researcher to calculate the effect of the treatment in this case the film. Once again
this does not really take any extraneous variables into account.

Static Group Design – This is a two group experimental design. One group is called
the experimental group and one is called the control group. Measurement is made
on the groups after the experimental group has been given their treatment. So you
would be split into two groups, one would watch the film and then the attitudes of
both groups would be measured. Problems because the respondents are not
randomly assigned to groups so they may differ before the treatment.

TRUE EXPERIMENTAL DESIGN

1. Pre-test-post-test control group design – In this design the respondents are
randomly assigned to either the experimental or test group. Then a pre-treatment
measure is taken for each group. The treatment is then given to the experimental
group and after a measure is taken of both the experimental and control group.

2. Post-Test-Only Control Group Design – The post-test only design does not
involve any pre-measurement. It’s assumed that the respondents in each group are
similar as they have been randomly assigned to each group.

3. Solomon Four Group Design – This design not surprisingly involves four
groups, there is the experimental group and control group both are pre-
measured, control groups 2 and 3 are not pre-measured. The experimental group
has the treatment as does control group 2 then all four groups are measured.
Control group 2 controls for sensitising which pre-testing may do. Control group 3
controls for history, any change which may occur over time, between measures.

Quasi-Experimental Designs

Time series design – this involves taking a series of measurements for a group of
respondents. The treatment might be administered by the researcher but may occur
naturally. After the treatment periodic measurement continues to take place.
Maturation is partially controlled as although it may occur over time it may not
effect O5 and O6. If the respondents are selected randomly, selection bias can be
reduced. The major weakness is that as it is occurring over time it does not control
for history. Mortality may be a problem so you might need to offer incentives to the
participants.

Multiple Time Series Design – The multiple time series is similar to the time
series, but another group of respondents are used to serve as a control group.

Laboratory vs Field Experiments

• Lab setting desired conditions vs. field setting conditions
• Lab experiments have high degree of control
• Lab experiments produce same results if replicated
• Lab experiments use smaller number of respondents
• Lab experiments may introduce reactive error
• Lab experiments may cause demand artefacts
• Lab experiments may be less generalisable

Limitations to Experimentation
• Time
• Cost
• Administration

SURVEYS
The survey method involves obtaining information using structured questions
designed to elicit specific information from respondents. The questions may cover
the respondent’s behaviour, intentions, attitudes, awareness, motivations and
demographic and lifestyle characteristics. The data is structured in that the questions
are prepared and gone through with each of the respondents.