Session 6 - Observation, experimentation and survey methods. Getting people to admit what they do and interpret findings. Some of the problems with experimentation Keeping all other variables constant while you manipulate one only.
Session 6 - Observation, experimentation and survey methods. Getting people to admit what they do and interpret findings. Some of the problems with experimentation Keeping all other variables constant while you manipulate one only.
Session 6 - Observation, experimentation and survey methods. Getting people to admit what they do and interpret findings. Some of the problems with experimentation Keeping all other variables constant while you manipulate one only.
Survey Methodologies Observation • Any form of behaviour can be observed • Uninvolved observation is the “purest” form of research • It is the only methodology for certain groups of respondents (children, animals) • Observation may be participative (in which the observer is in an assumed or open role) or may be through mechanical means The problems with observation! • Getting people to admit what they do (particularly if it is socially undesirable!) • Interpreting what they do • Translating findings into managerial actions • Getting people to accept changes in their pattern of behaviour. An ethical issue • Is it ethical to observe a person mechanically when they do not know they are being observed or have given their consent? (As via a video) • Even worse when you go into a situation in an assumed role (It might be called spying!) • However if you tell them they are to be observed will they still behave in the same “normal” way? (The Hawthorne Effect). Some forms of observation • Monitoring of web site traffic • Content analysis of advertisements, etc • Garbage analysis • Physical inventories (A.C.Neilson) • Monitoring physiological reactions (psychogalvanometers, voice pitch analysers, pupilometers, eye trackers) • Bar code scanners Experimentation • Derived from and based upon physical science studies • Principle is to keep all variables constant except one (the independent variable) in order to chart its effect on the other variable (the dependent variable) which is then measured. • Simple in theory, impossible in practice Some of the problems with experimentation • Keeping all other variables constant while you manipulate one only • Interaction between variables • Unexpected competitive reactions (price reduction might be matched!) • Selection of test beds (same reactivity?) • Effect is very often not equal to simple economics (advertising for example) Some alternative forms of experimental design • In parallel experiment: Two test beds used simultaneously to get rid of the problem of time difference • In sequence experiment: Use of only one test bed to get rid of the problems of differential reactivity of test beds (but has the alternative problem of time difference!) • Double Blind trial: Where neither subject nor administrator knows which alternative is being used Experimental Validity • Internal validity: Are the observed changes solely due to the dependent variable – or are there other factors which might be causing the observed effect • External validity: To what extent can the results of the experiment be generalised outside the experiment to a wider set of situations Different authors, different emphases • Read Zikmund: Chapter 12. This author is a far bigger fan of business experiments than I am! • Personal experience has taught me there are so many possible errors, and things which can go wrong, that they are far too risky to consider, and questionable in terms of the results they generate! Surveys: A problem of communication • Idea I wish to get information about – encoding into words – communication via speech – received via hearing – decoded by recipient – something is understood • (On the basis of this) • A response is formulated within the brain – it is encoded into words – communicated via speech – received via hearing – recorded by the interviewer – something is understood !!! • 12 stages to ask one question! Surveys: The Metrics • With whom: Definition of target market, sample size and selection. • How: Methods of contact • With what: Questionnaires and schedules • Medium: Electronic or printed • Time and Cost: What is the budget? Errors in survey design • Random sample design / Self selection bias • (“Readers of the Bangkok Post say …….) • Respondent error: Non response error / refusals / leading to substitution? • Response bias (telling lies / semantics misunderstanding / fading memory) Response Bias • The “yes” respondent – says yes to everything! A desire to please • Extremity / central tendency in responses • Interviewer bias (through age, dress, body language, voice, etc) • Auspices (the credibility and stated position of the survey sponsors) • Social desirability / status / image bias (responding to maintain a social position and / or non admission of social problems) Different types of survey • Cross sectional survey: A “snapshot” of customer behaviour at one particular instant in time. May be repeating similar studies done at different times • Longitudinal surveys: May use totally the same sample of respondents (a panel) or a rotating panel (NRS / TGI) or a totally different sample on each occasion. The latter has the problem with comparability! Different forms of contact • Electronically / Telecommunications (SMS, Internet, voice activation telephone, call centre (CAT) research, interactive TV, mobile telephone, fax, e-mail, etc) • Mail contacts • Personal interviewing • (See excellent summary on Page 228: Zikmund) Evaluation of electronic methods • Positives: Fast, cheap, geographically flexible, high respondent anonymity, often good cooperation, adaptable to IT technology
• Negatives: Precise and simple wording, limited
areas for discussion, limited times for contact, allows cheating, certain groups difficult to contact, limited length, sample self selecting Evaluation of mail surveys • Positives: Can be cheap, sensitive areas can be addressed (nobody present), all family member completion, time period (diary) research, geographical flexibility, time to consider replies, question precision, can be up to 6 / 8 pages long • Negatives: Cheating, misunderstanding, slow, low response rate, biased response rate, not versatile, no interviewer available for clarification, Response rates: Incentives or not? • Do incentives increase response? YES • Group or individual incentive? DEPENDS • Pre or post completion? • Related / unrelated to survey? • Value? THANK YOU OR BRIBE! • Hygiene factors which will improve response without incentives! Hygiene factors to increase response (mainly mail surveys!) • High quality presentation in all aspects of survey design (paper, layout, interviewer appearance, etc) • Covering letter with assurances of anonymity, contact number for queries, clearly stated objectives, sharing of results • Reply paid envelope (stamp not Freepost) • Follow up for late respondents Personal Interviews • Very flexible • Only as good as the interviewer! • Can gather a large volume of information • Can probe and go into difficult areas • Expensive or very expensive! • Lack of respondent anonymity • Less chance of respondent lying • Location / timing can influence response What makes a good interviewer? • The person has to accept they are simply a “pipe” in the flow of information • They should, in all aspects ,be neutral (dress, manner, attitude, ethnicity, etc) • They should be (a) trained in interviewing techniques and (b) trained in this particular survey • Remember there is always interviewer bias – people like people like themselves! Different types of survey instrument • Must be suited to the survey methodology • Questionnaires / Schedules. • Self completion / Monitored completion / recorded completion • Length and organisation is vital to improve response. • Level of structure determined by question typology – Remember you have to analyse it subsequently, so keep it simple! Schedules • Very loose type of questionnaire where there is little or no structure • Used by TV interviewers, who “make up” the questions based on a series of topics to be covered • Best use when dealing with executives, where informality is the key. No sequence / structure to contact • Difficult to analyse, other than as narrative.