White

 Paper  

Measurement  on  Mobile  

Date: 10/10/10

Author: Xavier Facon

© 2010 Crisp Media, Inc. www.crispmedia.com

Measurement on Mobile October 2010

Table  of  Contents  

 
Introduction..................................................................................................................... 3
History ............................................................................................................................. 4
Where it Started ............................................................................................................................ 4
Smart Phones ............................................................................................................................... 4
Situation Today ............................................................................................................................. 4
Mobile Measurement Methods ...................................................................................... 5
Impressions/Page Views .............................................................................................................. 5
Unique Users/Unique Ad Impressions .......................................................................................... 6
Clicks/Click-Throughs ................................................................................................................... 7
Conclusion on Mobile Measurement methods .............................................................................. 7
The Problems with Mobile Measurement ..................................................................... 9
Filtering Problems = Over-Counting ............................................................................................. 9
Mediated Ad Requests = Over-Counting ...................................................................................... 9
Malformed Pixel Trackers = Under-Counting.............................................................................. 10
Malformed URLs = Under-Counting ........................................................................................... 10
Network and Browser Caching = Under-Counting ...................................................................... 10
Incompatible Device Browser = Over-Counting .......................................................................... 10
Crisp Recommendations ............................................................................................. 11
Timing of Counting ...................................................................................................................... 11
Method of Counting .................................................................................................................... 11
Segregation of JavaScript and non-JavaScript Devices ............................................................. 11
Applications vs. Mobile Web ....................................................................................................... 11
Summary ....................................................................................................................... 12
About the Author .......................................................................................................... 12
About Crisp ................................................................................................................... 12

© 2010 Crisp Media, Inc. www.crispmedia.com Page 2

Measurement on Mobile October 2010

Introduction  
During the past several years, Crisp has seen start-ups and established vendors
announce solutions to the many problems with mobile measurement and more
specifically mobile ad campaign measurement. We also have been aware for a while
that the Interactive Advertising Bureau (IAB) and the Mobile Marketing Association
(MMA), with the help of the Media Rating Council (MRC) are working towards uniform
mobile measurement standards in a joint task force.

Despite these initiatives, when the online digital budgets are extended to mobile today,
it is not clear what exactly defines an impression or a unique user. Publishers can’t be
certain about the size of their ad inventory and are surprised when they review ad
campaign reports from third parties. For agencies and media buyers, this situation is
one of the largest barriers to advertising on mobile.

In this document, Crisp shares some measurement insights that have developed over
time. Crisp brings a unique view to this issue as a content publishing platform for many
major brands, a mobile analytics system and a rich media ad platform. We have seen
these troubles first hand from both the publisher and the advertiser point of view.

We offer a simple solution as the best way forward at the end of this document.
However, if you are interested in the details and would like to understand the reason for
our recommendations, you are urged to keep reading.

© 2010 Crisp Media, Inc. www.crispmedia.com Page 3

Measurement on Mobile October 2010
History
  regular fashion, similar to how it is done for
Where
 it
 Started
  This is where the trouble started; there was
Going back a decade, mobile content was
mainly discovered and distributed via the
operator home decks. Those mobile
operators have been a great resource to
Crisp as we worked to understand what the
best method is to measure views, clicks and
uniques. Details go beyond the device and
ad server to important aspects of the carrier
network. Cookies were exclusively hosted
on the mobile gateways instead of the
actual device, making cookies consistently
unreliable. All devices were connected to
the Internet using a similar WAP gateway
and we knew how to identify a device
 Today
uniquely using its subscriber ID. It was even
predictable and detectable when subscriber
IDs were sometimes not unavailable. On the
other hand, JavaScript was simply not an
option, and neither was the use of tracking
pixels.
In short, in those early days, measurement
on mobile was done server side, by
inspecting the request headers of content
pages, by pattern filtering, and by user
agent-based filtering. With unreliable client-
side technology, no regular online analytics
system had the pretense of being
compatible with mobile.
Smart
 Phones
  also cannot be discounted that many robots
Several years ago, regular online sites did
become compatible with mobile. Devices
like the iPhone emerged that could browse
regular online sites. Not all content on
mobile needed to be mobile optimized and
not all content was delivered on-deck by the
mobile operators. Even content that was
mobile optimized to improve the user
© 2010 Crisp Media, Inc. www.crispmedia.com
experience could be coded and hosted in a
desktop sites.
now infrastructure used for mobile devices
which was not mobile ready, but appeared
to work fine. The same browser that
displayed non-optimized sites blocked third-
party cookies used for tracking on the
desktop.
Interestingly, the effects of multiple flaws in
some of these systems can partly cancel
each other out, keeping the error margin
relatively small at some traffic levels and
large at other levels.
Situation
The measurement and ad serving
infrastructure often deployed today is simply
not adjusted to the realities of the very
complicated technical landscape. Popular
devices are capable of connecting via 2.5G,
3G, (even 4G) cellular data networks as well
as Wi-Fi. The many different types of
gateways that are sometimes in between
the measurement server and the device are
still unique to mobile and have a detrimental
effect on the reliability of common
measurement techniques. The optimizations
that are built into these operator networks –
and into the mobile web browsers – to
increase the speed of the page loads
prevent dependable impression trackers. It
are automatically viewing and scraping
mobile content without being detected. The
sheer diversity of web browser
implementations, connectivity options,
device settings, and content types affect the
measurements in mobile much more than
most vendors realize.
Page 4
Measurement on Mobile October 2010

Mobile  Measurement  Methods    

Similar technical problems exist with the measurement of page impressions by the
publisher (mobile web analytics) and measurement of the ad impression by the ad
servers (campaign analytics). Web analytics often inform the publishers about potential
ad inventory. For the campaign reports to correlate in some way to the web analytics
reports both systems need to measure accurately.

In this document, methods for both are discussed at the same time. The critique of
differences is not intended to be exhaustive.

page request on the server. If a page is

Impressions/Page
 Views
  cached on the network or on the
Server-Side access log analysis
The oldest approach to count page
requests is to analyze the web server
access logs. Initially this wasn’t a bad
choice to count mobile impressions
because the volume was still relatively
low. If it happened that a new robot
emerged it could be added to the filter
list and the logs could be processed
again. However, for this to work, pages
can’t be cached elsewhere, like on the
operator gateways. Also, since it is a
bad idea to prevent page caching from a
speed and user experience point of
view, this method is not generally used
on large sites.
Server-side ad request counting
Every ad server can count the ad
requests. This works because most ad
request APIs are cache busting,
providing a pretty accurate count when
called directly from a device. Because
mobile devices sometimes don’t support
JavaScript, the ad requests have to be
initiated from the content server when
the page is loaded. At that point, the ad
request count can only be interpreted as
an impression accurately when every
page impression has a corresponding
browser, which often happens, the ad
request is not initiated. Also, if the page
is visited by a search engine spider, the
ad requests will be falsely initiated by
the content server. This is the method
used commonly by publisher ad servers
that have been extended to support
mobile.
Ad display/banner counting
Once an ad request produces an ad tag,
the tag is included in the content of the
page before the page is transmitted to
the device. For feature phones, this is a
very basic piece of HTML that includes
an image tag. This method counts the
impression when the image loads from
the server and is more accurate than
counting the ad requests. It is also
easier to cache bust the banner image
instead of the entire page, from a site
performance perspective. Many robots
may not load the image and this way
there is some robot filtering for free.
However, other robots may hit it
anyway. This method is not a bad
approach for feature phones. Several,
mobile optimized ad networks have
used this method of impression counting
for delivery on simple MMA compliant
size banners.

© 2010 Crisp Media, Inc. www.crispmedia.com Page 5

Measurement on Mobile October 2010
Separate impression beacon - or
 Users/Unique
 Ad
pixel
On desktop you often have a 1x1 pixel
used for tracking purposes. Because of
the mobile browser optimizations to load
the pages faster, devices often skip
loading these tiny images. When the
tracking pixel is used in mobile, most
vendors know to use a larger
transparent image. This method is today
the most common method for tracking
an impression by mobile-optimized 3rd
party ad servers or analytics systems.
The larger tracking pixel works for the
majority of devices, works fairly
independently from ad or content
delivery itself, and is already supported
by many systems designed for regular
desktop sites and campaigns. This
method still exposes a weakness for
robots and while standard desktop
systems can support larger pixels, not
everyone has converted to use them for
mobile delivery.
Separate JavaScript-based
impression tracker
The most versatile and ideal way to
track impressions is by scripting the
tracking logic on the client side. This
method hits a server-side API from
within the browser in a controllable way,
once the page loads or after the ad is
fully rendered. An advantage is that
robots won’t affect it since script
execution is rarely done outside of a
browser environment. This provides a
reliable way to track a page impression
or ad impression. Because it is not
compatible with all feature phones, this
method is still rarely used on mobile
devices today. However, it is the most
promising approach as more and more
JavaScript capable devices are sold.
© 2010 Crisp Media, Inc. www.crispmedia.com
Unique
Impressions
3rd party client-side cookies
Ad infrastructure designed for the
desktop web relies heavily on the use of
cookies to anonymously identify a user
and count the user uniquely across
multiple requests and impressions. The
cookie can be set and read from domain
names other than the site’s domain
name. In other words, two or more sites
which each use a different domain name
can share a user ID in a cookie for
functionality hosted on a 3rd domain.
This functionality may work on a number
of recent mobile devices, but it is
disabled by default on a large number of
the most popular devices. This well-
accepted desktop approach simply
cannot be used to measure an audience
across mobile sites.
1st party client-side cookies
The less complicated approach is to use
cookies from the domain where the site
is hosted. The user ID here is only set or
read from one host name, resulting in
audience measurement which is site-
specific. While this method works more
and more reliably on devices that store
cookies locally rather than on the
gateway, it still doesn’t provide the
cross-site functionality which ad serving
infrastructure needs.
Subscriber ID
An Internet connection over the cellular
networks is secured with a basic form of
authentication. It allows the data to
travel securely and allows the mobile
operator to provision user-specific
services to the consumers. An artifact of
this system is a unique subscriber ID
Page 6
Measurement on Mobile October 2010
that is passed along with the mobile
browser content requests. Any server
that receives page or ad requests can
use this value as a key for a server-side
cookie or as a way to count unique
requests and impressions. The limitation
on this approach is that it does not work
reliably on every carrier and some
carriers only pass the subscriber ID for
specific domains. Despite these
limitations, counting users based on
subscriber ID is often used by mobile-
optimized systems as part of a suite of
tracking methods.
Device fingerprint
Like so many things in the highly
fragmented mobile device landscape,
one methodology doesn’t fit all. Many
mobile vendors resort to combining
different systems and parameters to
reduce the error rate. The downside is
that the methodology becomes vendor-
proprietary, just like the approach to
identify a user uniquely by the technical
fingerprint of the device. There are
easily more than one hundred possible
different bits of info flowing from a
device over the mobile Internet while a
device makes an ad request or a page
request. Most of them are not unique to
the user but the combination of them
can make it reasonably unique. When
server-side cookies are implemented in
mobile, they are most often identified
using this method. No two vendors are
using the same combination of bits of
information, so no two vendors will have
corresponding uniques with this method.
Another danger in this method is the
privacy issue it raises since the
consumer no longer controls when and
how to clear tracking. Many of these bits
of information can’t be modified by the
consumer.
© 2010 Crisp Media, Inc. www.crispmedia.com
HTML5 storage
In addition to cookies, mobile devices
that support a portion of the HTML5
standard are now supporting the storage
of information locally on the device, from
within the mobile browser. This can be
used to store a cross-site user ID, and is
sometimes used to circumvent problems
with the reliability of cookies on a
specific device. Recently, a plaintiff in a
lawsuit against a vendor asserted that
this method made it harder to opt-out of
the collection of behavioral data. The
privacy concerns around the storage
abilities of HTML5 make us believe that
this method is not yet ready for general
use.
Clicks/Click-­‐Throughs
Click-URL redirect
To count clicks in mobile, the most
common approach is also the approach
most commonly used on the regular
desktop web. The destination URL is
replaced or prepended by a URL to the
analytics server, which redirects the
browser to the destination after
measurement. Because multiple ad
servers are often involved, several
redirects are usually required. This
redirect method takes much more time
on mobile and also is not supported by
various mobile devices.
Server-side HTTP API
If it doesn’t work right on the client
browser, it has to be done from the
server. With server-side redirects, the
device is limited to one single redirect.
Additional ad servers, for example the
agency ad server, are hit by the first
analytics server. This method creates a
complication for downstream servers
that expect requests directly from the
Page 7
Measurement on Mobile October 2010

device. Participating servers must mimic
all device headers to reliably correlate
reports.
Client-side JavaScript API
Instead of depending on the browser to
redirect through an intermediate ad
server before reaching the click
destination, it is possible to hit the ad
server directly with JavaScript while the
user clicks. With JavaScript, multiple
trackers can be implemented on the
same page for each 3rd party ad server.

Conclusion  on  Mobile  Measurement  methods  

Vendors with very different technical backgrounds have joined the mobile ad serving
community. The choice for counting methodology depends on their own knowledge
of the mobile network and their pre-existing methodologies. If the priority is device
compatibility, then a lowest common denominator approach wins out even though it
has a very high margin for error. If the priority is accuracy, then the only
methodologies to choose from limit compatibility to JavaScript capable devices.

© 2010 Crisp Media, Inc. www.crispmedia.com Page 8

Measurement on Mobile October 2010

The  Problems  with  Mobile  Measurement  

While the different measurement methodologies have a statistical variance when
implemented perfectly, they are especially prone to problems, which can result in errors
exceeding an order of magnitude. This section explains a range of problems that Crisp
has witnessed with various implementations of the previous measurement techniques.

Filtering  Problems  =  Over-­‐Counting  

Filtering data is required whenever the methodology includes counting server hits to a
public URL. This is especially true for URLs that return actual content.

Poor filtering for search engine spiders

There is no master list of mobile search engine spiders. Most can be filtered, but
some still fake valid device user-agent headers in order to traverse mobile sites
that implement device detection. If ad impressions are counted based on server-
side ad requests then this will result in impression over-counting.

Poor filtering of other robots

Mobile optimized content has the advantage that it is rendered with a simple
layout and a XML valid syntax. This makes mobile content much more reliable
and attractive for content scraping. This is a common problem on popular,
branded sites that are visited for up-to-date content by applications that make
content available off-line or to re-purpose it. If ad impressions are counted based
on server-side ad requests then this will result in impression over-counting.

Effects from content feed readers

There are feed reader applications that don’t only download the ATOM or RSS
syndication feeds, but actually go as far as scraping the full content of a web
page in order to make it available via the reader application. If ad impressions
are counted based on server-side ad requests then this will result in impression
over-counting.

Mediated  Ad  Requests  =  Over-­‐Counting  

There are various ad servers and ad networks that provide inter-mediation of other ad
servers or ad networks to improve the fill rate of the mobile ad inventory. This is both
the case for applications and mobile web. Optimizations made for delivering the ads
quickly include pre-fetching of ads. If the origin ad server counts impressions based on
a separate ad impression tracker, then there is not problem. However, this assumption
is often made by inter-mediating servers, where it is not appropriate. This results in
requests being counted as impressions even when the ad was never delivered.
© 2010 Crisp Media, Inc. www.crispmedia.com Page 9
Measurement on Mobile October 2010

Malformed  Pixel  Trackers  =  Under-­‐Counting  

On many mobile devices the syntax for the HTML image tag is stricter than on desktop
browsers. It often happens that the way the tag is structured prevents a hit to the server
for an important portion of the devices. In addition, even when the tag is formed
correctly, the device may not generate the tracker request because of the small size of
the content.

Malformed  URLs  =  Under-­‐Counting  

The impression tracker URL can possibly exceed the maximum URL length for some
cellular data networks or for some mobile devices. On many devices 100 characters
used to be the maximum URL length. Today it is best practice to keep the URL length
under 256 characters. Exceeding this limit may cause the tracker never to arrive intact
at the measurement server.

Network  and  Browser  Caching  =  Under-­‐Counting  

Cache busting methodologies are often far less effective than they are deemed to be.
Some of the mobile networks are highly optimized to limit unnecessary bandwidth use.
While the mobile browser may adhere to a wider set of cache control directives, the
caching proxies deployed within the mobile network infrastructure are more aggressive
and may only adhere to very few directives. This issue has an effect on a multitude of
measurement techniques.

Incompatible  Device  Browser  =  Over-­‐Counting  

Because impressions are so commonly counted based on ad requests, it is likely that
part of these ad requests never result in an actual visible ad unit for the end-user.
Whenever an image tag or a JavaScript tag is either malformed or delivered to an
incompatible device, the ad doesn’t actually display.

© 2010 Crisp Media, Inc. www.crispmedia.com Page 10

Measurement on Mobile October 2010

Crisp  Recommendations  
Timing  of  Counting    
We realize that different vendors will use different approaches to counting. It is our
recommendation that all counting happens as late in the ad delivery process as possible
because we think that approach offers the greatest reliability. Vendors who count
impressions based on ad requests (too early in the ad delivery process) instead of using
separate impression tracking should disclose this to their customers and explain the
effects. The discrepancy caused by ad request counting can be large and
unpredictable.

Method  of  Counting  

Crisp recommends that impressions be counted with a separate tracker (either a large
pixel or JavaScript) once the ad is actually rendered because it produces a reliable
metric that can be easily verified by a second (3rd party) tracker. In other words, Crisp
recommends client-side counting. However, Crisp also recommends that whenever
possible, the time that the ad is in-focus is counted in addition to counting the
impression. This dwell-time, or display-time metric is a necessary metric on mobile
devices with small screens because many ads are quickly out of focus or never were in
focus.

Segregation  of  JavaScript  and  non-­‐JavaScript  Devices  

Crisp recommends to all publishers with large enough ad inventory make a hard
distinction between the inventory from feature phones, which is not reliably measured,
and the inventory on JavaScript capable smart phones, which can be measured reliably.

This distinction makes it possible to sell campaigns that have a better user-experience
for end-users with more reliable reporting. Any existing installations that share the
infrastructure between feature phones and smart phone, in which ad impressions are
based solely on server-side ad requests, should be upgraded.

Applications  vs.  Mobile  Web  

Crisp recommends that publishers integrate advertising in applications in a similar
manner to how advertising is integrated on mobile web content. The current situation,
where a dozen or more ad serving SDKs unnecessarily fragment mobile advertising,
highlights an immature ecosystem, and results in campaign measurement issues. Crisp
fully supports the Open Rich Media for Mobile Advertising initiative (http://ORMMA.org),

© 2010 Crisp Media, Inc. www.crispmedia.com Page 11

Measurement on Mobile October 2010

which is being adopted by publishers and partners. The ORMMA standard enables
participants to integrate mature desktop ad infrastructure into mobile web and mobile
applications without adding measurement complexities.

 
Summary  
In this document, some history, definitions, problems, and Crisp recommendations for
mobile measurement are explained. The mobile ecosystem has witnessed extreme
growth and continuing fragmentation; Crisp’s conclusion is that methods are now
available to extend desktop infrastructure onto mobile without introducing the
measurement problems of the past. We recommend that publishers integrate their
mobile content with client-side ad tags that take advantage of modern device
capabilities. Crisp also promotes the use of ad currency definitions beyond the CPM
and CTR, like display time and interaction rate.

About  the  Author  

Xavier Facon is CTO at Crisp, a rich media advertising technology company based in
New York City. He writes regularly about ad technology and mobile best practices.

About  Crisp    
Crisp offers a universal rich media ad platform for building, serving, and measuring
multi-platform campaigns across the desktop and mobile Web, mobile apps, tablets,
and connected TVs. Crisp’s innovative ad formats and its Adhesion™ fixed
placement technology empower brands to engage with consumers and drive
interaction across various channels, while simplifying creative development,
streamlining ad serving, and unifying reporting for agencies. Leading brands
including Estee Lauder, HBO, Intel, Marriott, Paramount Pictures, Toyota,
Volkswagen, and others have utilized Crisp ads. Crisp has over 700 certified sites
and apps, and has partnered with leading publishers including CBS, CNN, Hearst
Magazines, The Wall Street Journal Digital Network, The Weather Channel, and
others. For more information visit www.crispmedia.com.

© 2010 Crisp Media, Inc. www.crispmedia.com Page 12