The Purpose of Evaluation

- The fourth step of the public relations process is evaluation.

 It is the measurement of results against established objectives set

during the planning process.
 Evaluation means different things to different practitioners.
o To some it is the “best annual report”.
o To others it is clippings from newspapers around the world.
o Still to some others the only meaningful evaluations are
scientific measures of
 increased awareness, or
 changed opinions,
 attitudes, and behaviors.
- James Bissland best described evaluation as:

“the systematic assessment of a program and its results. It is a

means for practitioners to offer accountability to clients and to
themselves”.

- Evaluation is all about:

 an orderly assessment of practitioners’ progress in attaining the

specific objectives of their public relations plan.
Practitioners can learn
  what they did right,
 what they did wrong,
 how much progress they have made and,
 most importantly how they can do it better next time.
- The desire to do a better job next time is:

 a major reason for evaluating public relations efforts,

 but another equally important reason managers want to know
o if the money,
o time, and
o effort expended on public relations
 are well spent and contribute to the realization of an
organizational objective,
 such as an increased awareness of ways to prevent the
spread of AIDS.
Objectives: A Prerequisite for Evaluation

- Before any public relations program can be properly evaluated,

 it is important to have a clearly established set of measurable

objectives.
 These should be part of the program plan, but some points need
reviewing.
- First, public relations personnel and management should agree on
the criteria that will be used to evaluate success in attaining
objectives.

 Objectives need to be precise and results-oriented.

 They also need to be
o realistic,
o credible,
o measurable, and
o compatible with the client's demands on public relations.
- Second, do not wait until the end of the public relations program
to determine how it will be evaluated.

 Evaluating impact/results starts in the planning stage.

 Public relations officials need to break down the problem into
measurable goals and objectives then after implementing the
program, they need to measure the results against goals.
- If an objective is informational,

 measurement techniques must show how successfully

information was communicated to target audiences.
 Such techniques fall under the rubric of
o "message dissemination" and
o "audience exposure,"
  but they do not measure the effect
o on attitudes or
o overt behavior and action.
 Motivational objectives are more difficult to accomplish.
 If the objective is to increase sales or market share, it is important
to show that
o public relations efforts caused the increase
o rather than advertising or
o other marketing strategies.
 Or, if the objective is to change attitudes or opinions,
o research should be done before and after the public
relations activity to measure the percentage of change.
Checklist
- Although objectives may vary, the following checklist contains the
basic evaluation questions that any practitioner should ask:

1. Was the activity or program adequately planned?

2. Did recipients of the message understand it?
3. How could the program strategy have been more effective?
4. Were all primary and secondary audiences reached?
5. Was the desired organizational objective achieved?
 6. What unforeseen circumstances affected the success of the
program or activity?
7. Did the program or activity fall within the budget set for it?
8. What steps can be taken to improve the success of similar future
activities'?
Current Status of Measurement and Evaluation
- Public relations professionals have made considerable progress in,

 evaluation research and

 the ability to tell clients and employers exactly what has been
accomplished.
- More sophisticated techniques than previously are being used,
including:

 computerized news clip analysis,

 survey sampling, and
 attempts to correlate efforts directly with sales.
- The emphasis on media coverage as an evaluation method is
reinforced by many public relations programs.
- Other evaluation methods which are increasingly used to
supplement media coverage are:

1. actual audiences reached,

2. audience feedback,
3. behavioral science measurements,
4. inferred achievement, and
5. substantiated achievement.
- Today, the trend toward more systematic evaluation has
accelerated for a number of reasons.
- One primary reason is that management demands more
measurable, bottom-line results and accountability.
- Other reasons that have influenced the trend are:

(1) more expertise by public relations practitioners who have studied

social science research methods as part of their college education,
(2) user-friendly computer software programs, and
(3) the need for solid data to make public relations programs more
cost-effective and an integral part of the organization's
communications strategy.
- There are at least three levels of measurement and evaluation,

1. compilations of message distribution and media placements

 this is the most basic level.
2. the measurement of audience awareness, comprehension and
retention of the message
 this requires more sophisticated techniques,
3. the measurement of changes in attitudes opinions and behavior
  The most advanced level
- Here are the most widely used methods for evaluating public
relations efforts, including measurement of production, message
exposure, audience comprehension, attitude change, and
audience action.
- Measurement of Production
- One elementary form of evaluation is simply to count how many:

 news releases,
 feature stories,
 photos,
 letters and
 the like are produced in a given period of time.
- This kind of evaluation is supposed to give management an idea of
a staff's productivity.

 Public relations professionals, however, do not believe that this

evaluation is very meaningful
o because it emphasizes quantity instead of quality.
o It may be more cost effective to write fewer news releases
and spend more time on the few that really are newsworthy.
- Another side of the production approach is:

 to specify what the public relations person should accomplish in

obtaining media coverage.
 An organization may evaluate its director of media relations on
the expectation that whether
(1) 20 stories would be run in any of the largest newspapers
and
(2) the organizations 10 of the news releases would be used by
different newspapers out of 15 they produced.
- Such evaluation criteria:

 not only are unrealistic

 but almost impossible to guarantee because media gatekeepers -
not the public relations person - make such decisions.
 Management may argue, however, that such placement goals
provide incentive to the public relations staff and are tangible
criteria in employee performance evaluation.
- Closely allied to the production of publicity materials is their
distribution.

 Thus a public relations department might report, for instance,

that
 o a total of 456 news releases were sent to 819 daily
newspapers,
o 250 weeklies, and 134 trade magazines within one year, or
o that 110, 000 copies of the annual report were distributed to
 stockholders,
 security analysts, and
 business editors.
 Although such figures may be useful in evaluating how widely a
particular piece of publicity was distributed
o they do not answer the question of readership or, more
important, of attitude change.

Measurement of Message Exposure

- The most widely practiced form of evaluating public relations
programs is:

 the compilation of press clippings and

 radio-television mentions.
- Public relations firms and company department working primarily
on a local basis:

 often have a secretary or intern clip the area newspapers.

- Large companies with regional, national, or even international
outreach:
  usually hire clipping services to scale large numbers of
publications.
 It also is possible to have "electronic" clipping services monitor
and tape major radio and television programs on contractual
base.
- Public relations officers of different organizations,

 use news clips as one measure of their program to publicize their

messages.
- This helps them measure:

 the media's acceptance of the story and

 shows that the client got extensive coverage.
 It also shows that the project accomplished the first stage of the
adoption process by making people aware of their programs.
Media Impressions

- In addition to the number of media placements, public relations

departments and firms report:

 how many people may have been exposed to the message.

- These numbers are described as media impressions,

 the potential audience reached by a periodical or a broadcast

program.
 - For example;
- If a story about an organization appears in the local daily

o with a circulation of 130,000,

o the media impressions are 130.000.
- If another story is published the next day,

o this counts for 130,000 more impressions.

o Estimated audiences for radio and television programs certified
by auditing organizations also are used to compile media
impressions.
- Media impressions are commonly used in advertising to illustrate
the penetration of a particular message.

 Such figure gives a rough estimate of how many people are

exposed to a message.
 They don't disclose how many people actually read or heard the
stories and,
 more importantly, how many absorbed or acted on the
information.
Dollar Value

- Another approach is to calculate the value of message exposure.

 This is done by converting publicity stories in the regular news
columns or on the air into equivalent advertising costs.
 For example, if a 5-inch article about an organization appeared in
a publication that charges $100 per column inch for advertising,
 a public relations person might report to management that the
article was worth $500 in media exposure.
- Although results of dollar value graphically show top management
the value of publicity efforts,

 the technique is a bit like comparing apples and oranges.

- Advertising copy is directly controlled by the organization and

 can be oriented to specific objectives.

 The organization also controls the size and placement of the
message.
- News mentions, on the other hand, are determined by:

 media gatekeepers and

 can be negative, neutral, or positive.
- In addition, a news release can be edited to the point that key
corporate messages are deleted,

 In other words, the organization can't control size, placement or

content.
 - It becomes a question of what is being measured,

 Should an article be counted as equivalent advertising space if it is

negative?
 It is also questionable whether a 15-inch article that mentions the
organization only once among six other organizations
o is comparable to 15 column-inches of advertising space.
- And the numbers game does not take into account that:

 a 4-inch article in the well known newspaper may be more

valuable in reaching key publics
 than a 20-inch article in a newspaper with limited circulation.
Others argue that news mentions have even more value

 than comparable advertising because they have higher credibility

with the public.
- Because of these reasons, there has been a rapid decline of using
such statistics, i.e. dollar-value approach,
- The technique reinforces the opinion of many media gatekeepers
that,

 all news releases are just attempts to get free advertising.

Requests and toll-free Phone Numbers

- Another measure of media exposure is:

  to compile the number of requests for more information.
- A story in a newspaper or an appearance of a company
spokesperson on a broadcast:

 often provides information as to where people can get more

information about a subject.
 In many cases, a toll-free phone number is provided.
Cost Per Person

- Another way to evaluate exposure to the message is:

 to find out how much it costs to reach each member of the

audience-the cost per person.
- The technique is commonly used in advertising:

 in order to place costs in perspective.

 A 5-second commercial on a certain television channel may cost
$10, 000,
 but advertisers may find as it is something well worth the price
 because an audience of more than 50 million people may be
reached at a single stroke.

- Cost-effectiveness, as this technique is known, is also used in

public relations.
 - An advertising run on Ethiopian television by an insurance
company, for example,

 may cost 50,000 birr

 but the company may generate millions from a large number of
new customers.
 Such campaigns give a very good return on investment.
- Collateral materials such as films, brochures, and newsletters can
also be evaluated on a cost-per-person basis,

 A sports video produced by Nike may cost $50,000

 but reach 150,000 high school students during its distribution
period.
 It thus costs Nike 33 cents to reach each student.
- Cost-effectiveness, of course, is increased if the video is shown to
additional teenagers.
Audience Attendance

- Counting audience attendance is:

 a relatively simple way of evaluating the effectiveness of pre-

event publicity.
- Poor attendance at a meeting or event:

 can indicate inadequate publicity and promotion.

 - Another major cause is lack of public interest,

 even when people are aware that a meeting or event is taking

place.
 Low attendance usually results in considerable finger-pointing;
o thus an objective evaluation of exactly what happened--or
o didn’t happen-is a good policy.
Measurement of Audience Comprehension

 A higher level of evaluation is to determine whether the

audience actually understood the message and retained it.
- Lindenmann calls measurement of audience comprehension,

 the second level of public relations evaluation.

- He notes that: “At this level, public relations practitioners measure

 whether target audience groups actually received the messages

directed at them:
 whether they paid attention to those messages,
 whether they understood the messages and
 whether they have retained those messages in any shape or
form.”
- The tools of survey research are needed to answer such
questions.
- Members of the target audience must be asked:

 about the message and what they remember about it.

- Another way of measuring audience awareness and
comprehension is the day-after recall.
- Under this method, participants are asked:

 to view a specific television program or

 read a particular news story then
 they are interviewed to learn which messages they remembered.
- Another dimension of audience comprehension and retention is:

 acceptance of the message.

- A person may easily understand the message that:

 Mobil Corporation is the sponsor of Masterpiece Theater on

public television but
 the city mayor may not agree that Mobil is a company that
practices a high degree of social responsibility-
o the real message that Mobil wants to communicate through
its sponsorship.
- Again, surveys or interviews can be used to discover whether
audiences agree with opinion statements.
- The most commonly used method is a Likert- type scale,
  in which respondents can indicate the extent of their agreement
with an opinion statement.

Measurement of Audience Change

- Closely related to audience comprehension of a message are,

 changes in an audience's perceptions and attitudes.

- A major technique to determine such changes is the benchmark
study.
- Basically, it is a measurement of audience attitudes and opinions
before, during, and after a public relations campaign.
- Benchmark studies, also called baseline studies,

 graphically show the percentage difference in attitudes and

opinions as a result of increased information and publicity.
- A number of possible intervening variables may account for
changes in attitude, of course, but statistical analysis of variance
can help pinpoint to what degree the change is attributed to
public relations efforts.
- In our country, different organizations working in the area of
health did a benchmark survey,
  to determine the effectiveness of their program to inform the
public about the seriousness of the HIV/AIDS pandemic.
- And the results of the survey that they did before and after the
campaign showed great difference.
- The percentage usually risen after the campaign.
- The best way to determine the effectiveness of communications
is:

 pre- and post-test research.

 The first pre-test survey measures the status quo.
 The second post-test survey will demonstrate any change and the
direction of that change.
4.8. Measurement of Audience Action

- In all cases, the tools and activities of public relations are:

 a means, not an end.

- Thus it can be seen that public relations efforts are ultimately
evaluated:

 on how they help an organization achieve its objectives.

- In many cases, the task of practitioners is to make sure that target
audiences:

 receive information,
 understand it, and
 act on it.
- Other changes in behavior can be ascertained through surveys to
determine:

 if people have altered their personal patterns.

 Have they quit smoking?
 Did they get an annual physical checkup?
 Are they driving more carefully?
- How people answer often depends on the previous public
relations efforts of organizations and groups.