Audience measurement in the digital age

Andrew Green
Source: WARC Best Practice, June 2017
Downloaded from WARC

This article details how audience measurement methodologies are adapting to the growth of digital
access in a range of media, including TV, radio, print and out of home.

Fundamental changes in the measurement of many media, driven by the rapid growth in digital
distribution technologies, mean that most audience behaviours can already be captured and
reported; however, important political and economic barriers to adoption remain.
Magna Global in the US has predicted that, by 2021, about 40% of time spent watching video will
be through streaming devices like Hulu, Roku and YouTube, so television audience measurement
needs to capture viewing on all devices and from all sources.
The next frontier for radio audience measurement is using passive methodologies, while for Out of
Home, the use of GPS meters to track people's travel behaviour and map their journeys against
the locations of various digital OOH frames is becoming increasingly common.
Two critical points to remember about any kind of audience measurement are that 1) they are
estimates, not census counts, and 2) they are a measure of opportunities to see an ad, not
measures of actual exposure.
Short case studies are included for each medium.

Jump to:
Introduction | Where to start | Essentials | Checklist | Further reading

Introduction
The role of audience measurement is to estimate the number and type of people who have an opportunity to see
or hear advertising in the media. Media planners, buyers and sellers use this information to inform their planning
and to oil the wheels of commerce between them.

It has sometimes been said that everybody loves progress, but nobody likes change. This is as true for audience
measurement as it is for much else. Given its technical nature, few people maintain a close interest in the
subject. It makes headlines when something goes wrong or when a major change in method causes the
numbers to change from those people are accustomed to.

For example, when peoplemeters were introduced in the 1980s to measure television audiences (replacing
paper diaries which people had previously used to recall what they had watched manually), reported audiences
turned out to be very different.

A great deal of methodological discussion and dissent followed. Dissent tends to come, unsurprisingly, from
those whose reported audiences or relative rankings are lower, regardless of how much 'better' a new method
may be. Fear of this outcome often makes major change difficult to execute.

But everybody agrees that change is essential. Audience measurement needs to reflect what is happening in the
media. This means – by definition – identifying what people are watching, listening to or reading. It also means
evolving to meet changing user needs. Planners, buyers and sellers increasingly want to look beyond broad
demographics, for example. They also want data to flow to them more quickly and with greater granularity.

We are in the midst of quite fundamental changes in the measurement of many media now, driven by the rapid
growth in digital distribution technologies. Technically, most audience behaviours can already be captured and
reported. But important political and economic barriers to adoption remain.

The new media buying paradigm

There are two kinds of media buying:

'Traditional' buying is where an advertiser buys a placement in a specific media vehicle. This could be a
page in a newspaper, a time slot in a TV or radio programme or a banner ad on a website. Broadly
speaking, the editorial environment is known (though not necessarily the particular news article or
programme content within that environment).
'Adtech' is where advertisers are buying audiences, but have limited control over the environment where
those audiences are. Data generated by websites, for example, enable automated buying systems to
identify the digital content particular individuals (tagged as having, for example, searched for a holiday or
for a car) are accessing, as they access it. It is also possible in a very limited way to target 'addressable' TV
households through their set top boxes or to reach some via their internet-connected televisions.

For fifty years, audience measurement has focused around the first of these, tracking how many and what kind
of people access the media where advertising appears. Measurement to support the second kind of media
buying is less concerned with the content surrounding the advertising message; it concentrates on capturing
when an individual (or, in fact, an individual device) causes an ad load to be initiated.

The digital revolution has made things both easier and more difficult at the same time. On the easy side, digital
media offer built-in measurement (all visitors to a website are tagged with a software cookie and can be
identified both when they return to a site and when they visit other sites). This arguably enables more precise
targeting of messages and also facilitates both fast feedback and more accurate measurement of return on
investment (ROI).

Digital technology has introduced complexity as well. Where once it was enough to measure behaviour in front
of a television set in the household, now TV viewing occurs on multiple devices inside and outside the home.
Viewing can be both live or time-shifted. People can similarly read newspapers and magazines not just in print,
but on their computers and mobiles devices.

The disconnection between advertising placement and content has led to negative headlines when automated
buying systems (focused exclusively on buying audiences at the best price) have ended up placing messages
around content an advertiser would rather not be associated with (e.g. those promoting terrorism or other
controversial content). But there have also been other challenges, not least with the quality of data driving the
decisions made by these systems.

Where to start
There are two critical points that need to be made about all kinds of Audience Measurement at the outset:

1. They are estimates, not census counts.

2. They are a measure of opportunities to see (or hear) an advertisement, not measures of actual exposure to
the ads.

How accurate is audience data?

Each medium is measured differently, but none are census measures. Most use sample-based estimates either
in whole or in part to create audience data.

In some countries, quite precise measures of TV set tuning (i.e. the number of television sets switched on and
tuned to particular channels at given times) have been made available where the sets are connected through set
top boxes or the internet. These systems cannot detect who is watching (or, indeed, whether anybody is
watching), but they can be useful complements to sample-based estimates.

There is no way to know for certain exactly how many people read any particular newspaper or magazine, listen
to a radio station or pass by a poster panel. So we must make do with estimates, which are subject to a range of
statistical biases and to the accuracy of peoples' memories.

Digital media often claim greater precision than their traditional counterparts. Every device that visits a web page
can be logged and tracked, making digital audience data a form of census measure. 'Cookies' placed on a user
device when somebody visits a web page mean that the device can be recognised when it is used to access the
same page at a later stage. It also means that the number of different or 'unique' devices visiting a page can be
established.

But this precision is not what it seems. We cannot equate an individual with a device. A single person may use
any number of different devices (a smartphone, tablet or laptop, for example) or different browsers on the same
device to access a website. And people commonly delete cookies on their devices. In all these cases, when
somebody visits a web page on a subsequent occasion, the website will treat them as new and therefore a
single individual can be counted several times.

Conversely, where several people share devices within a household such as PCs and tablets (or entire
networks of computers at workplaces) they will only be counted as one 'person' by a cookie.

So both traditional and digital media audiences are estimates, subject to a range of caveats and assumptions.
For those using the data, however, the most important thing is credibility and acceptance. As long as planners,
buyers and sellers believe the estimates to be as good as they are going to get, the data will be used.

OTS vs. exposure

It is said that you can lead a horse to water, but cannot make it drink. In the same way, the media lead people to
their content and hope that they will look at the advertising around it. But they cannot force anybody either to
look at particular content or to take note of it.

A television station airs a programme and hopes the audience will be in the right frame of mind to stay in the
room while advertising airs. A newspaper or magazine publishes interesting articles that draw people's eye and
hopes readers will also look at the advertising around them. A webpage does the same thing.

In all cases the media are creating opportunities to see or hear advertising. They are not guaranteeing that
anybody will actually notice it. Often they won't. TV viewers can be distracted by other devices or by
conversations with other people in the room. Radio listening is often an accompaniment to other activities.
Internet users are bombarded with banner ads, many of little or no interest to them and effectively screened out.

One way of investigating whether opportunities can be equated with exposure is to use eye-tracking technology
to identify where and how long peoples' eyes fixate on a page they are looking at. Data from Lumen in the UK,
which runs a panel equipped with eye-tracking technology, suggested that close to three-quarters of newspaper
ads are looked at for at least a few seconds1. The same source reported that only around 9% of banner ads
served on websites were actually looked at for more than a second.

Many services offer similar insights on television viewers. One study2 found that while more than 80% of viewer
eyeballs are focused on the screen during a programme, this falls to just over 40% during the commercials
(although, of course, they can still hear the commercials in most cases).

Main measurement methods

All media are adapting their measurement methodologies to deal with the growth of digital access. Some of the
key developments are summarised below.

Television

Television sets remain by far the main device on which people watch TV. Most viewing is live, rather than time-
shifted. But there is no doubt that more and more people are spending more and more time watching via other
devices (tablets, PCs, smartphones etc.) and from alternative sources such as internet streaming and broadcast
catch-up services. Magna Global in the United States has predicted that, by 2021, about 40% of time spent
watching video will be through streaming devices like Hulu, Roku and YouTube.

Measurement ideally needs to capture viewing on all devices and from all sources and to report it as quickly as
possible. It also has to cope with what is sometimes called the 'long tail' – the very large number of viewing
options viewed by small numbers of people and which are simply not detected by sample-based measurement
systems.

For example, in the USA, some 1,160 individual broadcast TV channels are measured (as well as multiple
streaming services such as Netflix, Hulu and Amazon Prime) via a sample of around 66,000 individuals. In the
UK, a much smaller country, 330 channels are measured with a sample of just 11,500 people. Most of these
channels register no viewing at all, even with these fairly large samples.

But sample size issues aside, there are few technical barriers to measuring viewing on any device or platform.
Measurement techniques (which can be integrated in many cases using advanced statistical procedures) have
already been developed to capture all this viewing behaviour, as summarised in the table below:

Key barriers to adoption of all these measurement methods tend to be economic (they cost money!) and political
(reported viewing levels and, more critically, channel rankings and shares) will change. This will obviously work
for some channels better than it will for others.

Broadcasters in many markets (notably Sweden, Norway, the Netherlands, the UK and the USA) have made
significant headway in developing advanced, cross-platform TV measurement systems and will continue to do
so. Many others are following or will do soon.

Case study: BARB

BARB (the Broadcasters' Audience Research Board) is the UK's official television measurement body. In recent
years it has been evolving its methods in pursuit of what it calls 'the bigger picture' in an approach called Project
Dovetail. BARB's panel of 5,100 homes captures data on all television set usage of all individuals in each
household. Many panellists also now have viewing on other devices tracked via special software meters.

At the same time, census data is collected on all the visits by all devices to the various TV Player apps (such as
BBC iPlayer). Ultimately, these two datasets will be combined, so the census data on visits to player apps is
infused with demographic information from the panels. Further developments are being planned to integrate set-
top box data (available for a large number of households) and other online viewing.

Radio
For many years, radio audience measurement depended on asking people to recall what they had listened to.
This was achieved either by calling them and asking them to recall what they listened to on the previous day or
by asking them to keep a listening diary for a week and to log what they heard.

As the reach of digital technology has improved, online versions of the weekly diary have been introduced in
several countries for those who prefer this format. More recently, versions for smartphones have been
introduced, such as those for RAJAR in the UK and Radio Track in Poland.

Ultimately, the goal is to offer device-neutral diaries to maximise response and compliance. The next frontier for
radio audience measurement is to measure listening passively.

In 1998, Arbitron tested a new portable, passive meter (the PPM) in Manchester, following up with further tests
in the United States. The way the meter worked was that people were asked to carry it with them at all times (in
the form of a pager device). The device detects specially encoded signals embedded in the radio broadcast
stream when the carrier is in the vicinity of a signal (defined as 'listening').

The meter tests took several years, with roll-out delayed by both economics (the cost of the meters was many
times higher than the cost of the diaries) and politics (audience data was different, with the largest stations
disadvantaged the most).

Arguments raged around response rates, compliance of panellists, differential impact on the various ethnic
groups and much more. By 2007 the service went live in one market – and soon spread to most other major
cities.

The PPM enables continuous measurement (not limited to a day or a week, as previous methods were), minute-
by-minute data capture and the consequent ability to report on particular events and times. Other passive
measurement systems such as Ipsos's MediaCell and GfK's Media Watch have also been developed, with the
'listening' technology embedded respectively in a smartphone and a watch.
Electronic measurement of radio audiences has not swept the world in the same way peoplemeters did for
television (it now exists in 8 countries), for the same reasons it was delayed so long in the United States. There,
a long-term, strong monopoly supplier of ratings was able to push the industry into accepting the new method; it
has proved harder in Europe, where such decisions tend to be taken by groups of clients known as JICs (Joint
Industry Committees), where consensus is required.

Case study: Nielsen Audio (USA)

Nielsen Audio, formed out of the former Arbitron, from a standing start in 2007, now has PPM panels in 48 US
media markets, with some 77,000 panellists in total. The company also employs 7-day paper diaries in more
than 200 markets across the country.

Newspaper and Magazines

Both newspapers and magazines have suffered revenue losses as a result of a growing preference amongst
audiences to access content online rather than in print. Right from the outset of the digital revolution, online
content has – with certain exceptions - been free and easy to access on the go. It has been hard for publishers
to monetise their branded content, as they could do with physical copies.

Nevertheless, as with television and radio, the scope of measurement has widened to encompass reading on all
devices and across all platforms (print, apps, websites etc.). A second challenge has been to increase both the
frequency and speed of reporting – from what had often been only once or twice a year to at least quarterly or
even monthly. A third is to improve granularity.

Readership measurement is often compared unfavourably to TV, radio and digital media. The latter offer
audience estimates for individual programmes, time slots or websites for specific days and dates. In contrast,
newspapers and magazines have long relied on broad average audience data. For example, in most countries
(not all), daily newspapers trade on Monday-Friday audiences averaged over up to a year. Magazine audiences
are estimated for 'average' rather than specific issues.

Once again, the technical barriers to providing more granular estimates are low; it usually comes down to cost,
as well as concerns about the logistics and impact of changing a well-entrenched currency. so economic and
political barriers remain high.

Case study: CIM Press

In 2016, CIM Press, the organisation overseeing audience measurement for newspapers and magazines in
Belgium, initiated a bold experiment in readership measurement. One of the goals was to generate fast and
frequent estimates of cross-platform audiences to daily newspapers and to report them every day. To do this,
they combine results from a large face-to-face survey with data from a daily SMS panel, daily sales information
from publishers and findings from the official internet audience measurement body. A number of statistical
techniques are employed to integrate these datasets.

The launch of this multi-platform, daily audience report, is scheduled for the autumn of 2017.

Out of Home
Out of Home media are rapidly adopting digital technology, enabling creative messages to be changed rapidly in
response to advertiser demand and other factors. Panels have proliferated in multiple indoor and outdoor
locations from train stations and road sides to shopping centres.

The most advanced measurement systems comprise seven key elements:

1. a list of all the frames to be measured;

2. key characteristics of each frame (its location, size, height, proximity to roads etc.) which influence its
viewability;
3. estimates of vehicular and pedestrian traffic flows past each frame location;
4. knowledge of peoples' travel behaviour in relation to each frame location;
5. a model or set of models to help project non-random travel behaviour to a population;
6. a system for gauging likely exposure to each panel being measured;
7. a reach and frequency model enabling advertisers and agencies to undertake schedule analysis.

Denmark and Germany pioneered the use of GPS meters in the early part of the 21st century to track peoples'
travel behaviour outside their homes and to map journeys against the locations of poster panels. Many other
countries have since adopted meters as part of their measurement systems.

Key development areas today include meters that work inside as well as outside (always a challenge for GPS)
and also the use of mobile phone data to triangulate peoples' locations en masse, enabling much more accurate
counts to be made. Once again, economics will help determine how far these initiatives are allowed to go, as
well as the usual worries about changed audience data and rankings.

Case study: Route

Route is the UK body overseeing audience measurement for Out of Home media. All seven of the elements
listed above are included in the Route measurement system, including a database of some 400,000 individual
panels, a large-scale survey using meters to track the travel behaviour of a representative and projectable
sample of individuals and measurement of vehicle and pedestrian traffic flows using a combination of counts
(where available) and modelling.

Various statistical models and procedures help turn these inputs into audience estimates for every panel and
combination of panels, with work currently underway to account for variations in audiences over the course of a
day and over different days. This will be particularly useful for measuring audiences to dynamic digital panels.

Digital media

Broadly speaking, there are two distinct but complementary approaches used to measure digital audiences: site-
centric and people-centric measurement.

Site-centric measurement is a count of the number of different devices accessing a web page over a given time
period. Also known as web traffic data, site-centric audience numbers are automatically generated by the
websites themselves and are the official audience numbers usually quoted by publishers. Site-centric estimates
suffer from all the limitations listed earlier (cookie deletion, use of multiple browsers and multiple devices etc.),
as well as the fact that audiences cannot be segmented by demographic descriptors. But every website from the
smallest to the largest can quote a site-centric statistic.
People-centric measures are usually panel-based: a representative panel is recruited and asked to install
software on their computers and mobile devices which tracks the sites and pages they visit. Panel data can be
combined with census data from the site-centric measurement to generate total audience numbers for any digital
property alongside demographic splits for the largest ones. The main limitations of panel data include sample
size (the vast majority of websites will not reach enough panel members to enable an audience estimate to be
generated) and possible bias in the types of people who agree to be part of a panel.

Can we compare media?

As audience measurement practices have developed differently so have the definitions of an 'opportunity' to see
or hear an advertisement in each medium. The table below summarises the various approaches:

In other words, audiences for one medium cannot strictly be compared with those reported for another. But in
practice, of course, they are the recognised 'currencies' of transaction between buyers and sellers and so will be
compared from time to time.

Most advertisers tend to use a combination of media rather than a single medium. With the growth of digital
platforms, they will also tend to place messages across multiple platforms, where advertising is often traded
separately, even for the same media brands (e.g. the website and print outlet for a newspaper).

In some countries, cross-platform and cross-media research tools are available. For example, Touchpoints is a
UK study (also available in several other countries) which captures media usage of all kinds by daypart over the
course of a week which is statistically merged with various single media studies to offer a 360-degree view of
consumers for media planners.

Reminder checklist
Audience measurement generates estimates, not perfect counts;
Measures of advertising exposure are estimates of opportunities to see or hear an advertising message,
not exposures;
The main barriers to progress are economic and political rather than technical.
Further reading
WARC Topic: Audience Measurement

Green, Andrew (2016) Audience measurement in the data age. Research on WARC, Ipsos Connect

Green, Andrew (2010). From Prime Time to My Time: Audience Measurement in the Digital Age. WARC

https://www.thinkbox.tv/ Marketing body for television in the UK

http://www.thevab.com/ Marketing body for ad-supported television in the USA

http://mediaratingcouncil.org/ US organisation overseeing audience measurement quality and standards

http://www.rab.com/ US organisation promoting radio advertising

http://www.radiocentre.org/advertising/ UK radio marketing body

http://www.newsworks.org.uk/ UK marketing organisation for national newspapers

https://www.newsmediaalliance.org/research-and-tools/ US newspaper advocacy body

http://www.ppa.co.uk/ UK marketing organisation for magazines

http://www.magazine.org/ US marketing organisation for magazines

http://www.route.org.uk/home/ UK organisation for Out of Home research

http://www.everythingoutdoormedia.com/TAB_Ratings.html US organisation for Out of Home research

https://www.iabeurope.eu/ European marketing organisation for digital media

Notes
[1] Bassett, David and Green, Andrew (2015). Engagement as Visual Attention: A New Story for Publishers.
Proceedings of the Print & Digital Research Forum

[2] http://www.tobiipro.com/fields-of-use/marketing-consumer-research/customer-cases/eye-for-the-ads-tv-
commercials/

About the author

Andrew Green
Global Head of Audience Solutions, Ipsos
© Copyright WARC 2017
WARC Ltd.
Americas: 2233 Wisconsin Ave NW, Suite 535, Washington, DC 20007, United States - Tel: +1 202 778 0680
APAC: 20A Teck Lim Road, 088391, Singapore - Tel: +65 3157 6200
EMEA: 85 Newman Street, London, United Kingdom, W1T 3EU - Tel: +44 (0)20 7467 8100

www.warc.com

All rights reserved including database rights. This electronic file is for the personal use of authorised users based at the subscribing
company's office location. It may not be reproduced, posted on intranets, extranets or the internet, e-mailed, archived or shared electronically
either within the purchaser's organisation or externally without express written permission from Warc.