High Quality
Open the downloaded document, and select print from the file menu (PDF reader required).
646.435.1568
New
York
312.285.0751
Chicago
813.571.7830
Tampa
info@evadconsulting.com
www.evadconsulting.com
1
|
P
A G E
EVAD
C
ONSULTING
Audience Measurement Insight, Opinion and Analysis
March
24,
2008
More
on
the
Set
‐
Top
Box
Data
Mystery
The
Present
and
Future
of
Television
Audience
Measurement
In
an
unauthorized
and
completely
biased
plug
for
this
blog,
I
must
admit
that
I
have
been
amazed
at
the
sheer
number
of
emails
and
calls
I
have
received
over
the
last
few
weeks.
While
the
subject
matter
has
been
as
diverse
as
the
location
of
the
people
sending
the
messages—Dushanbe,
Tajikistan;
Christchurch,
New
Zealand;
Victoria,
Australia;
Ahmadabad,
India;
Santiago,
Chile—the
most
common
question
from
those
outside
the
United
States
has
mirrored
that
from
those
in
this
country,
“Can
you
please
explain
set
‐
top
box
data
to
me?”
The
short
answer—set
‐
top
box
data
is
the
present
and
future
for
television
audience
research—will
undoubtedly
spark
cries
of
outrage
by
some
in
the
industry.
Perhaps
the
easiest
way
to
address
the
mystery
of
set
‐
top
box
data
is
to
answer
questions
and
address
issues.
We
can
begin
with
the
list
assembled
by
Rainbow
Media’s
Charlene
Weisler
which
was
published
on
the
TV
Board
a
week
or
so
ago.
My
comments
follow:
Footprint
issue:
Some
cable
operators
provide
all
set
‐
top
box
data
in
a
market
and
others
provide
a
subset
of
a
market.
Until
there
are
standards,
the
resulting
analysis
must
be
directional.
We
have
diaries
in
some
markets;
tuner
meters
and
diaries
in
others.
Still
a
third
group
has
local
people
meters
tied
to
tuning.
There
are
three
standards
locally.
How
many
households
in
today’s
national
television
panel
come
from
Omaha?
How
many
come
from
St.
Louis?
How
about
San
Diego?
It
is
a
secret.
We
know
virtually
nothing
about
the
national
panel
except
what
the
company
that
manages
it
tells
us—that
it
is
representative
and
randomly
selected
without
bias
of
any
sort.
Any
researcher
providing
analysis
based
on
set
‐
top
box
data
can
be
just
as
forthcoming
as
the
current
ratings
provider.
Data
collection
issue:
Some
data
is
pulled
and
other
data
is
pushed.
I
have
collected
data
from
telecommunication
networks
for
nearly
twenty
years.
Believe
me
when
I
say
that
the
act
of
pulling,
polling
or
pushing
has
no
effect
on
data
quality.
With
respect
to
set
‐
top
box
data,
every
return
path
system
I
have
been
involved
with
since
2001
has
incorporated
an
error
identification
and
correction
system
that
renders
the
resulting
viewing
data
very
robust
and
extremely
reliable.
Rating
or
delivery:
Polling
a
set
‐
top
box
is
different
than
polling
a
tuner
meter
box.
Polling
is
polling
regardless
of
the
device
being
polled.
That
having
been
said,
I
cannot
recall
a
single,
modern
set
‐
top
box
data
collection
system
that
polled
what
channel
was
tuned
at
a
particular
time.
Every
system
that
I
have
studied
captured
device
state
changes,
which
is
a
fancy
way
of
saying
every
time
something
happens
in
a
set
‐
top
box
related
to
television
viewing,
the
box
phones
home.
On
the
flip
side,
most
panel
‐
based
tuning
meters
however,
do
poll.
In
fact
there
are
a
number
of
interesting
historical
notes
646.435.1568
New
York
312.285.0751
Chicago
813.571.7830
Tampa
info@evadconsulting.com
www.evadconsulting.com
2
|
P
A G E
EVAD
C
ONSULTING
Audience Measurement Insight, Opinion and Analysis
March
24,
2008
More
on
the
Set
‐
Top
Box
Data
Mystery
The
Present
and
Future
of
Television
Audience
Measurement
related
to
how
many
times
a
household
counts
when
multiple
sets
are
tuned
to
different
channels
within
a
given
fifteen
minute
period.
But
that
is
a
subject
for
another
column.
Smallest
viewing
increment:
polling
does
not
provide
insight
into
actual
viewing.
Again,
I
think
the
issue
is
twisted
around.
Typical
set
‐
top
box
data
provides
the
researcher
with
the
ability
to
assemble
perfectly
homogeneous
recreations
of
viewing
history—at
a
second
‐
by
‐
second
level.
Traditional
tuner
‐
meter
data
has
been
based
on
polling
and
has
often
been
summarized
at
a
30
‐
second,
minute,
average
minute
or
quarter
‐
hour
interval.
Trick
play:
DVR
metrics
need
to
be
decided.
DVR
data
is
not
mysterious.
DVR
data
is
identical
to
linear
data
except
that
there
are
two
timeframes:
real
‐
time
(e.g.
time
and
date
the
program
was
recorded
and/or
played)
and
content
‐
time.
When
a
viewer
is
watching
linear
television,
the
two
times
are
identical.
When
it
is
delayed,
they
are
not.
Some
linear
viewers
channel
surf
during
commercials;
some
DVR
viewers
fast
forward
through
commercials.
To
make
sense
of
the
data,
all
DVR
viewing
must
be
matched
to
content
‐
time.
Reporting
will
be
based
on
when
(or
if)
the
program
was
played.
Some
advertisers
will
be
interested
in
same
day
viewing
others
will
be
interested
in
viewing
up
to
one
week
later.
There
is
no
mystery
here.
Latency:
Content
‐
time
and
real
‐
time
are
not
always
the
same
everywhere,
even
with
respect
to
linear
television.
This
is
a
great
question
and
should
be
the
subject
of
a
full
discussion,
but
suffice
it
to
say
the
vast
majority
of
timing
error
is
stable
by
nature,
which
is
engineer
speak
for
“once
we
figure
it
out,
the
solution
doesn’t
change
much
over
time.”
Broadcast
and
distribution
propagation
error
as
well
as
DVR
cache
are
the
primary
sources
of
latency.
Tuning
Events
and
Dwell
Times:
How
long
does
someone
have
to
be
watching
to
be
counted?
We
could
argue
at
length,
but
most
people
would
agree
that
the
answer
is
somewhere
between
three
and
ten
seconds.
Most
set
‐
top
boxes
take
less
than
one
second
to
tune
and
a
viewer
spends
anywhere
from
two
to
eight
seconds
to
decide
whether
to
continue
watching
or
surf
away.
Analysis
based
on
set
‐
top
box
data
tends
to
involve
hundreds
of
thousands
if
not
millions
of
data
points
and
as
such,
small
differences
in
dwell
time
for
most
programs
tend
not
to
be
material.
Picture
‐
in
‐
Picture
and
Differences
in
Set
‐
top
boxes:
STB
data
is
inconsistent
and
DVR
data
reflects
behavioral
differences.
The
whole
point
of
audience
measurement
is
to
understand
what
audiences
are
doing.
I
challenge
anyone
to
estimate
DVR
behavior
without
DVR
data.
(And
to
anyone
who
might
buy
such
analysis
I
want
to
sell
you
a
great
condo
down
in
Miami—cheap!)
Just
because
some
set
‐
top
boxes
do
not
provide
access
to
646.435.1568
New
York
312.285.0751
Chicago
813.571.7830
Tampa
info@evadconsulting.com
www.evadconsulting.com
3
|
P
A G E
EVAD
C
ONSULTING
Audience Measurement Insight, Opinion and Analysis
March
24,
2008
More
on
the
Set
‐
Top
Box
Data
Mystery
The
Present
and
Future
of
Television
Audience
Measurement
the
second
tuner
(used
for
picture
‐
in
‐
picture)
does
not
mean
the
entire
data
set
is
suspect.
Set
‐
top
box
on,
TV
off.
There
is
an
assumption
that
this
poses
a
problem.
If
I
only
had
a
nickel
for
every
time
I
heard
this
objection
to
using
set
‐
top
box
data…
Set
‐
top
box
on,
TV
off
is
a
simple
noise
reduction
problem
and
typically
involves
a
small
percentage
of
set
‐
top
boxes.
The
issue
would
be
inherently
more
complicated
if
everyone
tuned
to
a
particular
network
or
program
before
they
went
to
sleep,
but
that
simply
does
not
happen.
I
think
this
particular
objection
can
be
traced
to
a
single
person
who
has
an
axe
to
grind
over
set
‐
top
box
data,
not
that
I
am
pointing
a
finger
at
Paul
Donato.
Lack
of
demographics.
Nielsen
will
match
to
its
sample.
Others
will
do
something
different.
Nielsen
has
spent
a
great
deal
of
money
extolling
its
approach
as
the
“gold”
standard,
but
compromising
high
quality,
set
‐
top
box
data
with
demographics
obtained
from
a
biased,
small
sample
to
“fix”
demos
is
bad
science.
Demographics
are
important,
but
more
reliable
solutions
to
the
demographic
problem
will
emerge—as
long
as
companies
believe
they
are
participating
in
a
free
market.
If
we
embrace
new
approaches
to
the
problem,
we
are
much
more
likely
to
get
a
better
solution.
Behavior
‐
based
metrics
are
interesting
and
are
arguably
more
reliable
than
demographic
ascription
will
ever
be.
Box
availability
within
the
home.
Not
all
homes
have
boxes.
The
last
time
I
checked,
not
all
homes
have
Nielsen
boxes
either.
Let’s
do
a
quick
count.
Millions
and
millions
of
advanced
set
‐
top
boxes,
thousands
of
Nielsen
boxes.
In
every
market
where
I
have
analyzed
set
‐
top
box
data,
a
significant
percentage
of
the
set
‐
top
boxes
were
primarily
tuned
only
to
broadcast
channels.
A
subset
of
set
‐
top
boxes
which
tuned
to
broadcast
channels
for
more
than
95%
of
their
viewing
minutes
could
be
used
as
a
surrogate
for
broadcast
only
homes.
In
a
small
market
like
Hawaii,
that
subset
was
measured
in
the
thousands.
Cable
and
satellite
companies
spend
millions
of
dollars
every
year
to
market
to
these
broadcast
only
homes.
Reams
of
data
are
available
on
this
market
segment
which
would
assist
in
modeling
such
viewing.
Compare
that
to
a
recruited
panel.
What
do
we
know
about
those
households
who
say
no
to
being
recruited—by
some
counts
three
in
four?
Typically,
precious
little
data
is
published
about
these
households.
We
must
assume
that
the
reasons
they
do
not
participate
do
not
affect
their
television
viewing.
Back
channel.
A
work
in
progress.
Getting
all
the
data
back
is
the
goal.
However,
the
fact
that
all
the
data
is
not
coming
back
should
not
deter
the
industry
from
learning
and
moving
forward
with
the
data
we
have
at
hand.
Add a Comment