UNIT-IV

POST PRODUCTION AND

EVALUATION

Lesson 4
Evaluation: Process and
Measurement Techniques
 Introduction
• A crucial activity for any producer is the regular evaluation
of what he or she is doing.
• Station owners, advertisers, sponsors and accountants will
want to know what programmes cost and whether they are
worth it.
• Conscientious producers striving for better and more
effective communication will want to know how to improve
their work and achieve greater results.
 Programmes Evaluation 3

Programmes are evaluated on the basis of the following

parameters:
• Production and quality evaluation
• Audience evaluation
• Cost evaluation
 Production Evaluation 4

► Proper Technical and Operational Standard: This means there

is no audible distortion, sound quality, balance and levels are
correct, the fades properly done, the pauses just right and the
edits unnoticeable.
► What is the programme for : A statement of purpose should be
formulated for every programme so that it has a specific direction
and aim & target audience.
► Professional Evaluation of Content and Format: Were the
interviews up to standard? The items in the best order? The
script lucid and the presenter communicative?
 Programme Quality 5

► Appropriateness: Irrespective of the size of the

audience gained, did the programme actually meet the
needs of those for whom it was intended?
► Creativity: Did the programme contain those sparks of
newness, difference and originality that are genuinely
creative, so that it combined the science and logic of
communication with the art of delight and surprise?
► Accuracy: Was it truthful and honest, not only in the
facts it presented and in their portrayal and balance
within the programme, but also in the sense of being fair
to people with different views?
 Programme Quality (Contd..) 6

► Eminence: Quality acknowledges known standards of

ability in other walks of life. A quality programme is likely to
include first-rate performers – actors or musicians. It will
make use of the best writers and involve people eminent in
their own sphere.
► Holistic: A programme of quality will certainly
communicate intellectually in that it is understandable to
the sense of reason, but it should appeal to other senses as
well – the pictorial, imaginative or nostalgic.
► Technical advance. An aspect of quality lies in its technical
innovation, its daring – either in the production methods or
the way in which the audience is involved.
 Programme Quality (Contd..) 7
► Personal enhancement: Was the overall effect of the programme to enrich
the experience of the listener, to add to it in some way rather than to leave it
untouched – or worse to degrade or diminish it?
► Personal rapport: As the result of a quality experience, or perhaps during
it, the listener will feel a sense of rapport – of closeness – with the
programme makers. Programmes which take the trouble to reach out to the
audience earn a reciprocal benefit of loyalty and sense of ownership.
► Quality in this sense will mean a diversity of output, meeting a whole range
of needs within the population served.
► It will reflect widely differing views and activities, with the intention of
creating a greater understanding between different sections of the
community.
► Its aim is to promote tolerance in society by bringing people together –
surely always the hallmark of quality communication.
 Audience Evaluation 8

► Formal audience research is designed to tell the broadcaster

specific facts about the audience size and reaction to a particular
station or to individual programmes.
► The measurement of audiences and the discovery of who is
listening at what times and to which stations is of great interest
not only to programme makers and station managers, but also to
advertisers or sponsors who buy time on different stations.
► Audience measurement is the most common form of audience
research, largely because of the importance attached to obtaining
this information by the commercial world.
 9
Audience Evaluation (Contd..)

Several methods of measurement are used and, in each,

people are selected at random from a specific category to
represent the target population:
• People are interviewed face to face, generally at home.
• Interviews are conducted by phone.
• Respondents complete a listening diary.
• A selected sample wear a small personal meter.
 Methods and Techniques of 10
Programme Evaluation
► Personal Meters: Many methods of radio audience
measurement depend on listeners’ memories and on
whether they actually know what radio stations they have
been listening to.
► There are electronic methods that get round these problems.
► For example, a selected number of people agree to wear a
small personal programme meter that records what stations
the wearer has been in the audible presence of during his or
her waking hours.
 Methods and Techniques of
11
Programme Evaluation (Contd..)

► Research Panels: Another method of research is through

listening panels scattered throughout the coverage area.
Such groups can, by means of a questionnaire, be asked to
provide qualitative feedback on programmes.
► Panel members will be in touch with their own
community and therefore may be chosen to be broadly
representative of local opinion.
 Methods and Techniques of
12
Programme Evaluation (Contd..)
► Questionnaires: In designing a research questionnaire ensure that the
concepts, words and format are appropriate to the person who will be
asked to complete it. Here are some design criteria:
► Decide exactly what information you need, and how you will use it.
► Do not ask for information you don’t need – redundant questions only
complicate things.
► Write the introduction to indicate who wants the information, and why,
and what will be done with it. Establish the level of confidentiality.
► Number each question for reference.
► The layout should be in lines rather than boxes or columns. This
enables it to be completed either by typing or longhand.
 Methods and Techniques of 13
Programme Evaluation (Contd..)
• The information you may want is in three categories:
Facts: name, age, family, address, job, newspapers read.
Experience: listening habits, reception difficulties, use of TV/videos.
Opinion: views of programmes, presenters, of competitor stations.
• Avoid questions that appear to have a right or preferred answer.
• Keep the questionnaire simple and as short as possible.
• Yes/No, tick box, multiple-choice and numerical scale questions
take up little space and can be given number values, and so are
useful in producing statistics.
 Methods and Techniques of
14
Programme Evaluation (Contd..)
► Letter Response : Informal audience research – anecdotal
evidence, press comment and immediate feedback – often
has an impact on the producer that is out of proportion to
its true value. Probably the most misleading of these – in
relation to the listenership as a whole – is the letter
response.
► Letters will indicate something about the individuals who
are motivated to write – where they live, their interests
perhaps, what triggered them to pick up a pen, or what they
want in return.
 Methods and Techniques of 15
Programme Evaluation (Contd..)
► Cost Evaluation: Total costing will include all the
management costs and all the overheads, over which the
individual programme maker can have little or no
control.
► One way of looking at this is to take a station’s annual
expenditure and divide it by the number of hours it
produces, so arriving at a cost per hour.
 RAM (Radio Audience Measurement)- 16
An Overview
► Radio broadcasting, because of its versatility, is considered an
effective medium to provide entertainment, information and
education. Terrestrial radio coverage in India is available in Frequency
Modulation (FM) mode and Amplitude Modulation (AM) mode (Short
Wave and Medium Wave).
► Radio broadcasting services are provided by the public broadcaster
All India Radio (AIR) as well as by private sector radio operators. AIR
transmits programs both in AM and FM mode and has 415 radio
stations (AM & FM) that cover almost 92% of the country by area and
more than 99.19% of the country’s population.
► Private sector radio operators transmit programs in FM mode only.
 RAM (Radio Audience Measurement) 17
► At present, radio audience measurement in India is conducted by AIR and TAM Media
Research(joint venture AC Nielson and Kantar media research).
► AIR carries out periodical large scale radio audience surveys on various AIR channels
only.
► TAM Media Research conducts radio audience measurement on private FM Radio
channels only through an independent division, which is a joint service between
IMRB International and Nielsen Media Research.
► Radio rating services on a commercial basis were provided by TAM Media Research
for the first time in 2007 through its own independent division, which is a joint
service of IMRB international and Nielsen Media Research.
► At present TAM conducts audience measurement in the cities of Bengaluru, Delhi,
Mumbai and Kolkata only.
RAM (Radio Audience Measurement)- Contd.. 18

► AIR undertook the first Radio Audience Measurement

(RAM) in the country in 1946 through its 'Listeners
Research Wing'. Subsequently, Listeners Research Wing
was renamed as ‘Audience Research Unit’ with 38 branches
across the country.
► However, AIR usually conducts listenership survey for its
own radio channels and for its own consumption.
 Thank you
Presentation by- Dr. Atul Upadhyay