From Paper-and-Pencil to Screen-and-Keyboard Studies on the Effectiveness of Internet-Based Marketing Research

Elisabeth C. Brüggen-Deutskens

1

2

q

50% European researchers say they have replaced face-toface studies with online research over the last year
(Greenfield/Ciao Online Research Barometer 2005)

q

75% of research decision makers in the US tried online research and expect to use it again in the future (Acorn / IIR)

q

Companies currently doing research online spent, on average, 28 percent of their total research budgets on online methodologies in 2004 By 2010, almost one-third of all survey research expenditures will be directed towards online methods

q

q

Increase from € mill. in 1996 to 500 mill. in 2002 3
(Hogg 2002)

q

Value of the online research market is set to triple in three years, reaching $4 billion by 2008 (Cambiar / GMI 2005) Online research is surging, and within the next few years is likely to become the most common form of all marketing research fieldwork conducted
(Marketing Research Association Executive Director 2006)

q

q

Online research is the largest paradigm shift in the industry since the late 1950's

3

Online Surveys
Benefits q Fast responses q Electronic capture of data q Lower costs q Large scale q Flexibility

Efficiency of online surveys

Effectiveness of Online Surveys

Effectiveness = weight1 * Quality - weight2 * Costs Where Quality: Data quality (response rate and quality) Costs: Time, money Weights: magnitude of risk, constraints

-

4

Central Research Question

What are the critical factors that influence the effectiveness of online marketing research?

Overview

HOW

Study 1: Design of Online Surveys

WHAT

Study 2: Equivalence of Online & Mail Surveys Study 3: Generalizability of Online & Mail Surveys

WHO

Study 4: Quality of Online Panels

5

Study 1: Design of Online Surveys
q q q q

Length: Short vs. long Type of incentives: Vouchers, lotteries, donations Presentation: Pictures vs. text Timing of follow-ups: 1 vs. 2 weeks

Field experiment with 730 consumers Effectiveness: Response rate and quality

6

Conclusions Study 1
q q q

Length: Long survey good response Incentives: Lotteries most effective Visual Presentation: Lower response rate but higher response quality Timing of follow-up: no significant influence

q

Study 1 provides important design guidelines to academics and marketers alike

7

Study 2&3: Equivalence and Generalizability
Core concern: Data quality of online surveys Online survey results:
§ §

q q

Reliable? Comparable to mail surveys?

q

Trade-off costs and quality?

Study 2&3: Equivalence and Generalizability
Study 2:
§

q

Effectiveness: Equivalence è accuracy, completeness, measurement invariance Field experiment in US Respondents: mail 694 (16.58%) online 255 (28.47%)

§ §

8

Study 2&3: Equivalence and Generalizability
Study 3:
Effectiveness: Generalizability theory § Field experiment US & UK
§

q

US
§

UK 439 (16.74%) 141 (26.60%)

Respondents: Mail
Online

1000 (16.58%) 377 (28.48%)

Conclusions Study 2&3
q q

Equivalence in response quality Comparable level of generalizability Online surveys
§ § §

q

More cost efficient Higher response rate More answers to open questions

Online surveys can be equally effective

9

Conclusions Study 2&3

Practical Implications q Support multi-mode surveys

www.oce.com

Because we want to make it as easy as possible for you to give us your feedback, we have made the survey available on our Internet side as well. To fill it in, simply go to www.oce.com and use your customer number (printed at the top of this letter) to log on.

10

Alternatively, please fill in the questionnaire enclosed and return it to Océ using the postage-paid return envelope.

Study 4: Quality of Online Panels
q

q

Fastest growing segment due to lack of representative online sampling lists Online panel: People who agreed to participate regularly in surveys What motivates members of online access panels to join and participate regularly? Effectiveness: response rate and quality

q

11

Survey Participation Inventory

Survey Participation Inventory
Intrinsic Motives Interest Enjoyment Curiosity Extrinsic Motives

Self Focus

Incentives

Other Focus

Helping Give opinion

Obligation Need for recognition

Response Rate Response Quality

12

Analysis Study 4
q q

13 different surveys with wide range of topics Response quality: Style
Tendency to dis/agree, provide extreme or neutral responses

q q q

Response rate: Participation last year Link to SPI results 3815 respondents

Conclusions Study 4
Respondents Size Motives Opinion Helping Incentives Response Neutral response style

1. Voicing assistants 27%

2. Reward seekers

25%

Lowest response rate Lowest self-reported levels Good response style Nay-sayers Highest response rate Worst response style Yea-sayers

3. Intrinsics

48%

All intrinsic motives

13

Conclusions Study 4
q

Practical implications: Design of E-mail invitation
Triggers in subject line or text:
q q q

Your opinion counts We need your help Participate and win

5 Key Recommendations
1. Use lotteries with several small prizes 2. Long surveys are possible 3. Equivalenceè make use of multi-mode surveys 4. Optimize measurement design (costs vs. level of generalizability) 5. Examine link between response motives and styles in online panels

14

Overall Conclusion

From Paper-and-Pencil to Screen-and-Keyboard Studies on the Effectiveness of Internet-Based Marketing Research

Elisabeth C. Brüggen-Deutskens

15

Master your semester with Scribd & The New York Times

Special offer for students: Only $4.99/month.

Master your semester with Scribd & The New York Times

Cancel anytime.