You are on page 1of 24

Session 6

Observation, Experimentation and


Survey Methodologies
Observation
• Any form of behaviour can be observed
• Uninvolved observation is the “purest”
form of research
• It is the only methodology for certain
groups of respondents (children, animals)
• Observation may be participative (in which
the observer is in an assumed or open
role) or may be through mechanical
means
The problems with observation!
• Getting people to admit what they do
(particularly if it is socially undesirable!)
• Interpreting what they do
• Translating findings into managerial
actions
• Getting people to accept changes in their
pattern of behaviour.
An ethical issue
• Is it ethical to observe a person mechanically
when they do not know they are being observed
or have given their consent? (As via a video)
• Even worse when you go into a situation in an
assumed role (It might be called spying!)
• However if you tell them they are to be observed
will they still behave in the same “normal” way?
(The Hawthorne Effect).
Some forms of observation
• Monitoring of web site traffic
• Content analysis of advertisements, etc
• Garbage analysis
• Physical inventories (A.C.Neilson)
• Monitoring physiological reactions
(psychogalvanometers, voice pitch
analysers, pupilometers, eye trackers)
• Bar code scanners
Experimentation
• Derived from and based upon physical
science studies
• Principle is to keep all variables constant
except one (the independent variable) in
order to chart its effect on the other
variable (the dependent variable) which is
then measured.
• Simple in theory, impossible in practice
Some of the problems with
experimentation
• Keeping all other variables constant while
you manipulate one only
• Interaction between variables
• Unexpected competitive reactions (price
reduction might be matched!)
• Selection of test beds (same reactivity?)
• Effect is very often not equal to simple
economics (advertising for example)
Some alternative forms of
experimental design
• In parallel experiment: Two test beds used
simultaneously to get rid of the problem of time
difference
• In sequence experiment: Use of only one test
bed to get rid of the problems of differential
reactivity of test beds (but has the alternative
problem of time difference!)
• Double Blind trial: Where neither subject nor
administrator knows which alternative is being
used
Experimental Validity
• Internal validity: Are the observed
changes solely due to the dependent
variable – or are there other factors which
might be causing the observed effect
• External validity: To what extent can the
results of the experiment be generalised
outside the experiment to a wider set of
situations
Different authors, different
emphases
• Read Zikmund: Chapter 12. This author is
a far bigger fan of business experiments
than I am!
• Personal experience has taught me there
are so many possible errors, and things
which can go wrong, that they are far too
risky to consider, and questionable in
terms of the results they generate!
Surveys: A problem of
communication
• Idea I wish to get information about – encoding
into words – communication via speech –
received via hearing – decoded by recipient –
something is understood
• (On the basis of this)
• A response is formulated within the brain – it is
encoded into words – communicated via speech
– received via hearing – recorded by the
interviewer – something is understood !!!
• 12 stages to ask one question!
Surveys: The Metrics
• With whom: Definition of target market,
sample size and selection.
• How: Methods of contact
• With what: Questionnaires and schedules
• Medium: Electronic or printed
• Time and Cost: What is the budget?
Errors in survey design
• Random sample design / Self selection
bias
• (“Readers of the Bangkok Post say …….)
• Respondent error: Non response error /
refusals / leading to substitution?
• Response bias (telling lies / semantics
misunderstanding / fading memory)
Response Bias
• The “yes” respondent – says yes to everything!
A desire to please
• Extremity / central tendency in responses
• Interviewer bias (through age, dress, body
language, voice, etc)
• Auspices (the credibility and stated position of
the survey sponsors)
• Social desirability / status / image bias
(responding to maintain a social position and / or
non admission of social problems)
Different types of survey
• Cross sectional survey: A “snapshot” of
customer behaviour at one particular
instant in time. May be repeating similar
studies done at different times
• Longitudinal surveys: May use totally the
same sample of respondents (a panel) or
a rotating panel (NRS / TGI) or a totally
different sample on each occasion. The
latter has the problem with comparability!
Different forms of contact
• Electronically / Telecommunications (SMS,
Internet, voice activation telephone, call
centre (CAT) research, interactive TV,
mobile telephone, fax, e-mail, etc)
• Mail contacts
• Personal interviewing
• (See excellent summary on Page 228:
Zikmund)
Evaluation of electronic methods
• Positives: Fast, cheap, geographically flexible,
high respondent anonymity, often good
cooperation, adaptable to IT technology

• Negatives: Precise and simple wording, limited


areas for discussion, limited times for contact,
allows cheating, certain groups difficult to
contact, limited length, sample self selecting
Evaluation of mail surveys
• Positives: Can be cheap, sensitive areas can
be addressed (nobody present), all family
member completion, time period (diary)
research, geographical flexibility, time to
consider replies, question precision, can be up
to 6 / 8 pages long
• Negatives: Cheating, misunderstanding, slow,
low response rate, biased response rate, not
versatile, no interviewer available for
clarification,
Response rates: Incentives or not?
• Do incentives increase response? YES
• Group or individual incentive? DEPENDS
• Pre or post completion?
• Related / unrelated to survey?
• Value? THANK YOU OR BRIBE!
• Hygiene factors which will improve
response without incentives!
Hygiene factors to increase
response (mainly mail surveys!)
• High quality presentation in all aspects of
survey design (paper, layout, interviewer
appearance, etc)
• Covering letter with assurances of
anonymity, contact number for queries,
clearly stated objectives, sharing of results
• Reply paid envelope (stamp not Freepost)
• Follow up for late respondents
Personal Interviews
• Very flexible
• Only as good as the interviewer!
• Can gather a large volume of information
• Can probe and go into difficult areas
• Expensive or very expensive!
• Lack of respondent anonymity
• Less chance of respondent lying
• Location / timing can influence response
What makes a good interviewer?
• The person has to accept they are simply
a “pipe” in the flow of information
• They should, in all aspects ,be neutral
(dress, manner, attitude, ethnicity, etc)
• They should be (a) trained in interviewing
techniques and (b) trained in this particular
survey
• Remember there is always interviewer
bias – people like people like themselves!
Different types of survey instrument
• Must be suited to the survey methodology
• Questionnaires / Schedules.
• Self completion / Monitored completion /
recorded completion
• Length and organisation is vital to improve
response.
• Level of structure determined by question
typology – Remember you have to analyse
it subsequently, so keep it simple!
Schedules
• Very loose type of questionnaire where
there is little or no structure
• Used by TV interviewers, who “make up”
the questions based on a series of topics
to be covered
• Best use when dealing with executives,
where informality is the key. No sequence
/ structure to contact
• Difficult to analyse, other than as narrative.

You might also like