19

Measuring the Effectiveness of the Promotional Program

McGraw-Hill/Irwin

Copyright 2009 by The McGraw-Hill Companies, Inc. All rights reserved.

Pros and Cons of Measuring Effectiveness

Advantages
Avoid costly mistakes Evaluate alternative strategies Increase efficiency in general Determine if objectives are achieved

Disadvantages
Cost of measurement

Research problems Disagreement on what to test Objections of creatives

Time

Evaluating Alternative Strategies

Measuring Effectiveness: Marketers Views

Issue
MPM is a key priority for todays technology companies. I am dissatisfied with our ability to demonstrate marketing programs business impact and value. It is important to define, measure, and take concrete steps in the area of advertising accountability. I can forecast the impact on sales of a 10% cut in marketing spending.

Agree
90% 80%

61.5%

37%

Measuring Advertising Effectiveness

What to test
Source factors Message variables Media strategies Budget decisions

Where to test
Laboratory tests Field tests

How to test
Testing guidelines Appropriate tests

When to test
Pretesting Posttesting

Pretesting Methods
Laboratory
Consumer Juries Portfolio Tests Physiological Measures Theater Tests Rough Tests Concept Tests Readability Tests

Field
Dummy Ad Vehicles On-air Tests

Comprehension and Reaction Tests

Posttesting Methods

Recall Tests Tracking Studies Association Measures

Methods
Recognition Tests Inquiry Tests SingleSource Systems

Posttesting the Value of a Sponsorship

Test Your Knowledge

Laboratory methods of testing for advertisement effectiveness generally offer high control but low _____. A) B) C) D) E) realism external validity costs generalization value

Where to Test

In the Field In the Lab

Positioning Advertising Copy Test (PACT)

1. Provide measurements relevant to objectives of advertising

2. Require agreement on how results will be used 3. Provide multiple measures 4. Be based on a model of human response to communications 5. Consider multiple versus single exposure to the stimulus 6. Require alternative executions to have same degree of finish 7. Provide controls to avoid biasing effects of exposure context 8. Take into account basic considerations of sample definition 9. Demonstrate reliability and validity

Test Points

1.Concept Testing

2.Rough Testing

Occurs at Various Stages

3.Finished art or commercial pretesting

4.Market testing (posttesting)

Concept Testing
Explores consumers responses to ad concepts expressed in words, pictures, or symbols

Objective

Alternatives are exposed to consumers who match the target audience

Method

Reactions & evaluations sought through focus groups, direct questioning, surveys, etc. Sample sizes depend on the number of concepts and the consensus of responses

Output

Qualitative and/or quantitative data evaluating and comparing alternative concepts

Rough Art, Copy, and Commercial Testing

Comprehension and Reaction Tests Consumer Juries Advantages
Control Cost effectiveness Endorsements by independent third parties Achievement of credibility

Disadvantages
Consumer may become a self-appointed expert Number of ads that can be evaluated is limited

A halo effect is possible

Preference for ad types may overshadow objectivity

Rough Testing Terms

Animatic Rough
Terms

Photomatic Rough

Live-action Rough

An Animatic Example from Airwalk

*Click outside of the video screen to advance to the next slide

Spot Resulting from Animatic Rough

*Click outside of the video screen to advance to the next slide

Pretesting Finished Print Ads

A laboratory method Portfolio Tests Includes test and control ads Portfolio test have problems

Readability Tests Dummy Advertising Vehicles

Based on syllables per 100 words

Other factors also considered
Distributed to random sample homes Product interest may still bias results

Test Your Knowledge

_____ is a method of testing ads by placing them randomly in certain copies of regularly distributed magazines. A) B) C) D) E) Vehicle source testing Burke's reflection test A Flesch test Dummy advertising vehicle testing A contextual test

Pretesting Finished Broadcast Ads

Theater Tests
Measures changes in product preferences

On-Air Tests
Insertion in TV programs in specific markets

May also measure . . . Interest in and reaction to the commercial Reaction from an adjective checklist Recall of various aspects included Interest in the brand presented Continuous reactions

Limitations are imposed by day-after recall

Physiological Measures

Physiological Measures

Pupil dilation

Galvanic skin response

Testing

Brain waves

Eye tracking

Market Testing Print Ads

Inquiry Tests

Recognition Tests

Testing

Tracking Studies

Recall Tests

Starch-Scored Sports Illustrated Ad

Noted score the percentage of readers who remember seeing the ad Seen-associated score the percentage who remember seeing or reading any part of the ad identifying the product or brand Read most scorethe percentage of readers who report reading at least half of the copy portion of the ad.

Posttests of Broadcast Commercials

Day after recall tests Persuasive measures

Tracking studies Single-source tracking

Diagnostics

Testing
Comprehensive measures

Test marketing

Comprehensive Testing by Ipsos-ASI

Problems With Current Research Methods

Essentials of Effective Testing

Use a consumer response model

Establish communications objectives

Testing

Use pretests and posttests

Understand and implement proper research

Use multiple measures

Test Your Knowledge

Good tests of advertising effectiveness must address the nine principles established by PACT. One of the easiest ways to do this is to follow a decision sequence model. The first step in the model is to: A) Understand the appropriate research B) Create a model that uses multiple measures C) Establish communication objectives D) Decide whether to use posttests or pretests E) Develop a consumer response model

Measuring Effectiveness of Other Programs

Sales promotions Shopping cart signage Nontraditional media Ski resort-based media In-store radio and television Other media Sponsorships

Exposure methods Tracking measures

Measuring Effectiveness + Efficiency