Marketing

Communications

Measuring the Effectiveness of

Promotional Programs

1
 Measuring Promotional Program
Effectiveness

Arguments For Reasons Not to

Measuring
Effectiveness
• Avoiding costly mistakes
• Evaluating Alternative
Strategies
• Increasing the efficiency of
advertising in general
Measure Effectiveness
• Cost
• Research problems
(It is difficult to isolate the effects
of promotional programs)
• Disagreement on what to test
• The objection of creative
(Creative dept does not want its
work to be tested)
• Time constraints

2
 When to Test
Pretests are measures taken before the campaign is
implemented

• Consumer juries
• Portfolio tests
• Physiological measures
• Theater tests
• Rough tests
• Concept tests
• Readability tests
• Comprehension and
reaction tests

3
 When to Test
Post tests occur after the ad or commercial has been in the
field

• Recall Tests – Aided and Unaided

• Inquiry tests
• Recognition Tests
• Tracking Studies

4
 Message Evaluations

• Message evaluation techniques are used to examine:

– the creative message
– the physical design of the advertisement, coupon, or direct
marketing piece

• It include the study of actors in advertisements as well as

the individuals who speak in radio ads

• A message evaluation is designed to consider

– The cognitive components associated with an ad (recall,
recognition etc.,)
– The peripheral cues (emotions, attitudes).

5
 Message Evaluations

• Evaluation or testing of advertising communications can

occur at any stage of the development process

• They can be analyzed at the concept stage before an ad

is ever produced .(This testing normally involves
soliciting the opinions of either a series of experts or
from “regular” people)

• In practice, multiple measures of evaluation are often

used

6
 Message Evaluation Techniques

• Concept testing • Attitude and opinion tests

• Copytesting • Emotional reaction tests

• Recall tests • Physiological arousal

tests
• Recognition tests

7
 Message Evaluation Techniques

• Concept testing is aimed at the actual content of the ad

and the impact that content has on potential customers

• The most common procedure used for concept testing is

a FOCUS GROUP

• It is wise to use independent marketing research

firms to conduct focus groups to prevent biased results

• Most agencies use more than one group in order to

ensure reliable results

8
 Message Evaluation Techniques

Components of a marketing communications plan to

be evaluated with concept tests include:

– Copy or verbal component of an advertisement

– Message and its meaning
– Translation of copy in an international ad
– Effectiveness of peripheral cues, such as product
placement in the ad
– Value associated with an offer or prize

9
 Message Evaluation Techniques

Two common testing instruments are:

 Comprehension tests
 Reaction tests

10
 Message Evaluation Techniques
Comprehension tests
– Used when participants in a study are asked the meaning of a
marketing communication piece
– The moderator can then explore the reasons why the intended
meaning was not understood correctly by the individual or the
group

Reactions tests
– Used to determine overall feelings about a marketing piece,
most notably whether the response is negative or positive
– If the focus group reacts negatively to an ad or particular copy in
an ad, the agency can make the changes before it is too late
– It is possible for an advertisement to be correctly understood but
elicit negative emotions

11
 Message Evaluation Techniques

Copy testing

• Copy tests are used when marketing piece is

finished or in its final stages of development
prior to production

• They are designed to solicit responses to the

main message that will be presented

12
 Message Evaluation Techniques

• The two most common copy testing techniques

are
– Portfolio Tests
– Theater tests

– A portfolio test is a display of a set of print ads, one of

which is the ad being evaluated

– A theater test is a display of a set of TV ads, including

the one being evaluated
13
 Message Evaluation Techniques
• Copytesting can utilize focus groups as well as other
measurement devices

• An ad or coupon that is in the final stage of design can

be tested with a mall intercept technique. This
approach involves stopping people who are shopping in
a mall

• The mall intercept technique can incorporate a portfolio

approach

• To do so, subjects are asked to examine the marketing

piece, which is mixed in with others, normally six to ten
ads, coupons, or other marketing communication tools
14
 Message Evaluation Techniques

Recall tests (Aided and Unaided)

• This approach involves asking an individual to recall
what ads he or she viewed in a given setting or time
period. Then, in progressive steps the subject is asked to
identify information about the ad

• The most common form of recall is the day-after recall

(DAR) test . Individuals who participate in the study are
called by phone the day after the advertisement first
appears

15
 Message Evaluation Techniques

• Individuals who participate in the study are tested using

approaches called:

– Unaided recall and;

– Aided recall

• Unaided recall - The subjects are asked to name, recall,

the ads they saw or heard the previous evening, without
being given any prompts or memory jogs

16
 Message Evaluation Techniques

• Aided recall - consumers are prompted by being told

the product category and if necessary, name of specific
brands in that category

• The respondents still does not know which brand or ad is

being tested

• When the consumer states that he or she does recall

seeing a specific brand being advertised, the person
then is asked to provide as many details as possible
about the ad
17
 Message Evaluation Techniques

Most researchers believe the unaided recall

approach is superior to other evaluative tests
because it identifies the times that an
advertisement has become lodged in the
person’s memory

18
 Message Evaluation Techniques
• In both aided and unaided recall tests, if incorrect information
is provided, the researcher continues the questioning

• Individuals are never told they have given inaccurate answers

• Incorrect responses are important data to record. Memory is

not always accurate in both aided or unaided recall situations

• Consequently, people give incorrect answers. In other words,

they may mention commercials that did not actually appear
during the test but rather one that was viewed at some other
time

19
 Message Evaluation Techniques
Items tested for recall

• Product name or • Tag line

brand • Incentive being
• Firm name offered
• Company location • Product attributes
• Theme music • Primary selling point
• Spokeperson of communication
piece

20
 Message Evaluation Techniques
Recognition tests
• Recognition tests is a format in which individuals are given copies
of an ad and asked if they recognise it or have seen it before

• Those who say they have seen the ad are asked to provide
additional details about when and where the ad was encountered
(e.g. specific TV program, the name of the magazine, the location of
the billboard)

• This information is collected to validate that it was indeed seen

• Next, the individual is asked a series of questions about the ad itself

• Recognition tests are best suited to testing comprehension of and

reactions to ads

21
 Message Evaluation Techniques

• Recognition tests tend to measure how many people saw

an advertisement while recall tests tend to measure how
many people saw an advertisement and were also
sufficiently interested to take the time actually to view or
read the ad

• Since recognition and recall tests measure different

things, many firms and research teams do both with the
same subject

22
 Message Evaluation Techniques

Attitude tests

• Attitude tests deal with both the cognitive and affective

reactions to an ad. They are also used to solicit
consumer opinions

Opinion tests

• Opinions are gathered from surveys or focus groups

23
 Message Evaluation Techniques
• The content and formats of attitude tests vary widely

• Sometimes specific responses are requested in what are

scaled closed-ended questionnaire formats.

• Scales such as 1=highly unfavourable to 7= highly

favourable are often prepared for respondents to answer

• In other tests, the individual is allowed to discuss what

ever comes to mind regarding some aspect of a product
or its advertisements. These are called open-ended
questions
24
 Message Evaluation Techniques

Emotional Reaction Tests

• Many ads are designed to elicit emotional responses
from consumers

• Emotional ads are based on the idea that ads that elicit
positive emotions are more likely to be remembered

• Consumers who have positive attitudes towards ads

would logically develop more positive attitudes toward
the product

• This, in turn, should result in increased purchases

25
 Message Evaluation Techniques

• It is difficult measure the emotional impact of an

advertisement

• The simplest method is to ask questions about

an individuals’ feelings and emotions after
viewing a marketing communication piece

• This can be performed in a laboratory setting

theater test or the ad can be shown to focus
groups
26
 Message Evaluation Techniques

• A warmth monitor is an alternative method developed

to measure emotions

• The concept behind the warm meter is that feelings of

warmth are positive when they are directed toward an ad
or a product

• To measure warm, subjects are asked to manipulate a

joystick while watching a commercial. The movements
track reactions to a commercial by making marks on a
sheet of paper containing four lines

27
 Message Evaluation Techniques

• The four lines are labelled:

– Absence of warmth;
– Neutral,
– Warmhearted or Tender
– Emotional

• The warmth meter was developed to evaluate

TV ads but it can be adapted to radio ads
28
 Message Evaluation Techniques

Physiological arousal tests

• Physiological arousal tests measure fluctuations in a person’s body

functions that are associated with changing emotions

• The primary physiological arousal tests are

– The psychogalvanometer
– The pupillometer test
– Voice-pitch analysis

• These measures indicate the receivers’ involuntary response to an

ad

29
 Message Evaluation Techniques

• The psychogalvanometer measures a person’s

perspiration levels

• Research in pupillometrics is designed to measure

dilation and constriction of the pupils of the eyes in
response to stimuli. Dilation is associated with action;
constriction involves the body’s conservation of energy

• The voice-pitch meter examines change in the pitch of

a person’s voice as he or she reacts with emotion

30
 Evaluating Respondents Behaviours

• Evaluating Respondents Behaviours addresses visible

customer actions including store visits, inquiries or actual
purchases

• This category contains evaluation techniques that are

measured using numbers

• In today’s IMC marketplace, many advertising

companies are being asked to deliver compelling proof
that the ads they design are actually working

• Respondents behaviours provide such evidence

31
 Behavioural Evaluations
Responses can be tracked as follows:

• Changes in sales • Internet Enquiries

• Telephone enquiries
(Inquiry tests)
• Response cards
(Inquiry tests)
(Inquiry tests)
• Direct-marketing
responses
• Redemption rate of
sales promotion offers

32
 Behavioural Evaluations
• Changes in Sales

– Measuring changes in sales following a marketing

campaign is easier now than it was in the past

– Universal codes and scanner data are available from

many retail outlets

– These data are available on a weekly basis and in

some situations on a daily basis

33
 Behavioural Evaluations

– Some retail outlets even have access to sales

information on a real time basis, and the information
can be accessed at any point during the day

– Scanner data make it possible for companies to

monitor sales and help both the retailer and the
manufacturer discover the impact of a particular
marketing program

34
 Behavioural Evaluations

Telephone Inquiries
• One method of measuring impact of an advertisement is
assign a toll-free number to each marketing piece
• A great deal of information can be collected during an
inbound call
• Sales data can be recorded and demographic
information gathered
Response cards
• These customer information forms are filled out at the
time of a purchase

35
 Behavioural Evaluations
• Internet responses
By using “cookies” **, a firm can obtain considerable information
about the person making the inquiry

• Direct-marketing responses & Redemption Rates

Coupons, Premiums, Contests, Sweepstakes and Direct-mail pieces
are marketing communication devices that can be coded to record
redemption rates

** a small file that a Web server automatically sends to your PC when

you browse certain Web sites. Cookies are stored as text files on
your hard drive so servers can access them when you return to
Web sites you've visited before. Cookies contain information that
identifies each user, for example: login or username etc..

36
 Behavioural Evaluations

• Test Markets

– Test markets are used when a company examines

the effects of a marketing effort on a small scale
before launching a national or international campaign
– The primary advantage of using a test market is that
an organisation can examine several elements of a
marketing communication program
– If the test market is successful, then it is likely that the
national or global campaign also will be effective

37
 Behavioural Evaluations

• Test markets programs are used to assess:

– Advertisements
– Promotions and Premiums
– Pricing tactics
– New products
• When one of these is tested and is not successful, the
communication program can be changed
• Test markets are cost-effective methods to analyze and
make changes in marketing efforts before millions dollars
are spent on something that will not accomplish the
intended objectives

38
 Behavioural Evaluations

• One major advantage of a test market is that it

resembles an actual situation more than any other tests

• The key is make sure that the site selected for the test
market strongly resembles the target population

• It is also important to design the test marketing

campaign as close to the national or full marketing plan
as possible

• A test market can be as short as a few days or as long

as two to three years

39
 Behavioural Evaluations

Purchase Simulation Tests

• Consumers can be asked in several ways if they would
be willing to buy products. For instance, they could be
asked about purchase intentions at the end of a
laboratory experiment

• In this situation, intentions are self-reported and tend not

to be an accurate predictor of future purchase
behaviours

40
 Behavioural Evaluations

• Test markets examine actual purchases, but are more

costly because the marketing piece must be completed
first

• A feasible and cost-effective approach to examine

purchase behaviours is called a simulated purchase test

41
THANK YOU!

42