Designing a Pretest

PRELIMINARY CONSIDERATIONS
 What should the pretest measure

Review the components of an effective communication:

1. Attraction: Is the message interesting enough to attract and hold the attention of the
target audience?Do people like it?

(If more than one message is tested) Which of several message attracts the most
attention/is best liked?

2. Comprehension: Is the message clearly understood?

3. Acceptability: Does the message contain anything that is offensive or distasteful by

local standards?

Does it contain anything that people perceive to be false?

Does it contain annoying elements that would become irritating after repeated exposure to
the message?
4. Self-involvement: Is the message able to convince the target
audience to undertake the desired behaviour?
 (if more than one message) Which of the several alternatives will bring
about the greatest amount of change?

5. Persuasion: Is the message able to convince the target audience to

undertake the desired behaviour?

The more attractive, comprehensible, acceptable, and self-involving a

communication is, the greater its potential effectiveness (persuasiveness).
 At what stage in production should the pretest be carried out?

At the concept stage

- to evaluate ideas for a communication campaign/program; brainstorming with
team
-no material is produced yet

• Partially completed product

- to find out what changes might be made before going any further

• Testing alternative versions

- different versions must be tested to determine which is potentially the most
effective
- materials are already with “near professional quality”

Final product testing

- if budget allows
 How polished must the communication materials be for pretesting?

As long as the materials are prepared with “near

professional quality”

-they should contain all elements of a final version

- Example: Radio spots for pretesting must already

have already music, sound effects, narration
 Can the pretest be carried out in the office?

YES and NO
YES as a form of informal conversation to obtain
feedback on the ff:
 Content
a) Does this (poster, radio spot, leaflet) convey the
intended message?
b) Does this material convey any secondary,
unintended message that might have a negative
effect (something that might offend the inteligence,
dignity or pride of the intended audience)
 Factors to determine in an in-house review
 Strategy
a) Is the material the best approach to getting the message across?
b) Would the message be more effective if an entirely different
theme were used?
c) Should the appeal be more emotional?

 Artistic Element (“packaging”)

a) Is the artwork appropriate for the intended audience?
b) Does the artwork catch attention and stimulate curiosity about
the contents?
c) Is the artwork easily understood?
d) Note: the same questions apply to the quality of the audio for
radio spot, TV spot
 Can the pretest be carried out in the office?

NO
This office review (in-house review) cannot take the
place of a test in the field
The co-workers do not possess the same
characteristics of your target audience
They might give feedbacks that would “please” their
superiors
 In what setting should the pretest be conducted?

 Naturalistic
refers to a situation where the person is exposed to communication as part of his normal routine.

Different radio spots are run for one week on a local station then you ask listeners which
attracted them the most

The listeners are exposed to the spots in the normal course of listening to the radio and are
unaware that they will be interviewed

Posters are placed in strategic places and people are asked after sometime

Arranged
refersto any situation where the respondents agrees beforehand to watch, listen to, or
read the communications being pretested.
By asking people to listen to a taped message and then to give their opinions about it.
 Pros and Cons of Naturalistic Setting

Allow for a test of people’s reaction under realistic conditions.

People’s responses reflect the potential effects of the

communications once they are in the field.

But the pretester has less control over the experimental setting.

The “effects” are not likely to be as strong, and thus a larger

sample size is required to obtain differences in reactions to one
message versus another.

Tends to be slightly more time-consuming and more expensive

 Pros and Cons of Arranged Setting

The pretesters have greater direct control over the research,

 Can get result faster for less money

 But respondents may decide to please the pretester

They may claim to like a communication, agree with its contents, or intend to
follow the advice when they would not have the same reaction to the
communication if they had been exposed to it in the outside world.

 Despite the positive bias associated with arranged settings,

(1) respondents cannot “fake” comprehension,
(2) if alternative versions of a communication are tested such that a favorite must
be selected, respondents are likely to select the one that they do like the best (i.e.,
there is no reason not to give valid answers)
Should the pretest be administered to individuals or
to groups?

Individual interviews obtains independent opinions

regarding the message being tested.
Individuals tend to express themselves more freely
among peers (in group discussions) than they will in
a one-to-one situation with the interviewer.
The pretester allows more time for discussion of
each topic in group discussions
Data are “richer” in detail in group discussions
One person’s ideas will come to dominate in the
discussion
 Should the questionnaires be self-administered or
should an interviewer be used?

Self-administered questionnaire (SAQ) allows for:

 Less interview time
 Less training for interviewers is required
 More respondents are obtained at less cost
 But SAQs are not recommended:
 - if target audience has difficulty reading and writing
 respondents will tend to skip over items, which leaves the pretester with
incomplete data.
 respondents may go back and change their answers to earlier questions after
reading other items on the questionnaire
 ( or they will read the whole questionnaire before answering any items).

• Interview (using the questionnaire)

• Spontaneous reactions are obtained that are not biased by questions which appear
later on the form.
 Summary

5 components
When the pretesting is to be carried out
 Materials have to be prepared with “near
professional quality”
Pretesting can be conducted under naturalistic or
pre-arranged setting
It can be conducted on a one-one interview, to
groups (FGD, in research)
When with individuals, pretesters can choose SAQ or
interview schedule
 Update on the assigned materials

Kumusta ang paggawa ng assigned comm material?