P. 1
Measurement on Mobile - Crisp Oct 2010

Measurement on Mobile - Crisp Oct 2010

|Views: 2,455|Likes:
Published by xfacon1

More info:

Published by: xfacon1 on Nov 11, 2010
Copyright:Attribution Non-commercial

Availability:

Read on Scribd mobile: iPhone, iPad and Android.
download as PDF, TXT or read online from Scribd
See more
See less

07/10/2013

pdf

text

original

White  Paper  

Measurement  on  Mobile  
Date: 10/10/10 Author: Xavier Facon

© 2010 Crisp Media, Inc.

www.crispmedia.com

Measurement on Mobile October 2010

Table  of  Contents  
  Introduction..................................................................................................................... 3 History ............................................................................................................................. 4
Where it Started ............................................................................................................................ 4 Smart Phones ............................................................................................................................... 4 Situation Today ............................................................................................................................. 4

Mobile Measurement Methods ...................................................................................... 5
Impressions/Page Views .............................................................................................................. 5 Unique Users/Unique Ad Impressions .......................................................................................... 6 Clicks/Click-Throughs ................................................................................................................... 7 Conclusion on Mobile Measurement methods .............................................................................. 7

The Problems with Mobile Measurement ..................................................................... 9
Filtering Problems = Over-Counting ............................................................................................. 9 Mediated Ad Requests = Over-Counting ...................................................................................... 9 Malformed Pixel Trackers = Under-Counting.............................................................................. 10 Malformed URLs = Under-Counting ........................................................................................... 10 Network and Browser Caching = Under-Counting ...................................................................... 10 Incompatible Device Browser = Over-Counting .......................................................................... 10

Crisp Recommendations ............................................................................................. 11
Timing of Counting ...................................................................................................................... 11 Method of Counting .................................................................................................................... 11 Segregation of JavaScript and non-JavaScript Devices ............................................................. 11 Applications vs. Mobile Web ....................................................................................................... 11

Summary ....................................................................................................................... 12 About the Author .......................................................................................................... 12 About Crisp ................................................................................................................... 12

© 2010 Crisp Media, Inc.

www.crispmedia.com

Page 2

Measurement on Mobile October 2010

Introduction  
During the past several years, Crisp has seen start-ups and established vendors announce solutions to the many problems with mobile measurement and more specifically mobile ad campaign measurement. We also have been aware for a while that the Interactive Advertising Bureau (IAB) and the Mobile Marketing Association (MMA), with the help of the Media Rating Council (MRC) are working towards uniform mobile measurement standards in a joint task force. Despite these initiatives, when the online digital budgets are extended to mobile today, it is not clear what exactly defines an impression or a unique user. Publishers can’t be certain about the size of their ad inventory and are surprised when they review ad campaign reports from third parties. For agencies and media buyers, this situation is one of the largest barriers to advertising on mobile. In this document, Crisp shares some measurement insights that have developed over time. Crisp brings a unique view to this issue as a content publishing platform for many major brands, a mobile analytics system and a rich media ad platform. We have seen these troubles first hand from both the publisher and the advertiser point of view. We offer a simple solution as the best way forward at the end of this document. However, if you are interested in the details and would like to understand the reason for our recommendations, you are urged to keep reading.

© 2010 Crisp Media, Inc.

www.crispmedia.com

Page 3

Measurement on Mobile October 2010

History  
Where  it  Started  
Going back a decade, mobile content was mainly discovered and distributed via the operator home decks. Those mobile operators have been a great resource to Crisp as we worked to understand what the best method is to measure views, clicks and uniques. Details go beyond the device and ad server to important aspects of the carrier network. Cookies were exclusively hosted on the mobile gateways instead of the actual device, making cookies consistently unreliable. All devices were connected to the Internet using a similar WAP gateway and we knew how to identify a device uniquely using its subscriber ID. It was even predictable and detectable when subscriber IDs were sometimes not unavailable. On the other hand, JavaScript was simply not an option, and neither was the use of tracking pixels. In short, in those early days, measurement on mobile was done server side, by inspecting the request headers of content pages, by pattern filtering, and by user agent-based filtering. With unreliable clientside technology, no regular online analytics system had the pretense of being compatible with mobile.

experience could be coded and hosted in a regular fashion, similar to how it is done for desktop sites. This is where the trouble started; there was now infrastructure used for mobile devices which was not mobile ready, but appeared to work fine. The same browser that displayed non-optimized sites blocked thirdparty cookies used for tracking on the desktop. Interestingly, the effects of multiple flaws in some of these systems can partly cancel each other out, keeping the error margin relatively small at some traffic levels and large at other levels.

Situation  Today  
The measurement and ad serving infrastructure often deployed today is simply not adjusted to the realities of the very complicated technical landscape. Popular devices are capable of connecting via 2.5G, 3G, (even 4G) cellular data networks as well as Wi-Fi. The many different types of gateways that are sometimes in between the measurement server and the device are still unique to mobile and have a detrimental effect on the reliability of common measurement techniques. The optimizations that are built into these operator networks – and into the mobile web browsers – to increase the speed of the page loads prevent dependable impression trackers. It also cannot be discounted that many robots are automatically viewing and scraping mobile content without being detected. The sheer diversity of web browser implementations, connectivity options, device settings, and content types affect the measurements in mobile much more than most vendors realize.

Smart  Phones  
Several years ago, regular online sites did become compatible with mobile. Devices like the iPhone emerged that could browse regular online sites. Not all content on mobile needed to be mobile optimized and not all content was delivered on-deck by the mobile operators. Even content that was mobile optimized to improve the user

© 2010 Crisp Media, Inc.

www.crispmedia.com

Page 4

Measurement on Mobile October 2010

Mobile  Measurement  Methods    
Similar technical problems exist with the measurement of page impressions by the publisher (mobile web analytics) and measurement of the ad impression by the ad servers (campaign analytics). Web analytics often inform the publishers about potential ad inventory. For the campaign reports to correlate in some way to the web analytics reports both systems need to measure accurately. In this document, methods for both are discussed at the same time. The critique of differences is not intended to be exhaustive.

Impressions/Page  Views  
Server-Side access log analysis The oldest approach to count page requests is to analyze the web server access logs. Initially this wasn’t a bad choice to count mobile impressions because the volume was still relatively low. If it happened that a new robot emerged it could be added to the filter list and the logs could be processed again. However, for this to work, pages can’t be cached elsewhere, like on the operator gateways. Also, since it is a bad idea to prevent page caching from a speed and user experience point of view, this method is not generally used on large sites. Server-side ad request counting Every ad server can count the ad requests. This works because most ad request APIs are cache busting, providing a pretty accurate count when called directly from a device. Because mobile devices sometimes don’t support JavaScript, the ad requests have to be initiated from the content server when the page is loaded. At that point, the ad request count can only be interpreted as an impression accurately when every page impression has a corresponding © 2010 Crisp Media, Inc.

page request on the server. If a page is cached on the network or on the browser, which often happens, the ad request is not initiated. Also, if the page is visited by a search engine spider, the ad requests will be falsely initiated by the content server. This is the method used commonly by publisher ad servers that have been extended to support mobile. Ad display/banner counting Once an ad request produces an ad tag, the tag is included in the content of the page before the page is transmitted to the device. For feature phones, this is a very basic piece of HTML that includes an image tag. This method counts the impression when the image loads from the server and is more accurate than counting the ad requests. It is also easier to cache bust the banner image instead of the entire page, from a site performance perspective. Many robots may not load the image and this way there is some robot filtering for free. However, other robots may hit it anyway. This method is not a bad approach for feature phones. Several, mobile optimized ad networks have used this method of impression counting for delivery on simple MMA compliant size banners. Page 5

www.crispmedia.com

Measurement on Mobile October 2010

Separate impression beacon - or pixel On desktop you often have a 1x1 pixel used for tracking purposes. Because of the mobile browser optimizations to load the pages faster, devices often skip loading these tiny images. When the tracking pixel is used in mobile, most vendors know to use a larger transparent image. This method is today the most common method for tracking an impression by mobile-optimized 3rd party ad servers or analytics systems. The larger tracking pixel works for the majority of devices, works fairly independently from ad or content delivery itself, and is already supported by many systems designed for regular desktop sites and campaigns. This method still exposes a weakness for robots and while standard desktop systems can support larger pixels, not everyone has converted to use them for mobile delivery. Separate JavaScript-based impression tracker The most versatile and ideal way to track impressions is by scripting the tracking logic on the client side. This method hits a server-side API from within the browser in a controllable way, once the page loads or after the ad is fully rendered. An advantage is that robots won’t affect it since script execution is rarely done outside of a browser environment. This provides a reliable way to track a page impression or ad impression. Because it is not compatible with all feature phones, this method is still rarely used on mobile devices today. However, it is the most promising approach as more and more JavaScript capable devices are sold. © 2010 Crisp Media, Inc.

Unique  Users/Unique  Ad   Impressions  
3rd party client-side cookies Ad infrastructure designed for the desktop web relies heavily on the use of cookies to anonymously identify a user and count the user uniquely across multiple requests and impressions. The cookie can be set and read from domain names other than the site’s domain name. In other words, two or more sites which each use a different domain name can share a user ID in a cookie for functionality hosted on a 3rd domain. This functionality may work on a number of recent mobile devices, but it is disabled by default on a large number of the most popular devices. This wellaccepted desktop approach simply cannot be used to measure an audience across mobile sites. 1st party client-side cookies The less complicated approach is to use cookies from the domain where the site is hosted. The user ID here is only set or read from one host name, resulting in audience measurement which is sitespecific. While this method works more and more reliably on devices that store cookies locally rather than on the gateway, it still doesn’t provide the cross-site functionality which ad serving infrastructure needs. Subscriber ID An Internet connection over the cellular networks is secured with a basic form of authentication. It allows the data to travel securely and allows the mobile operator to provision user-specific services to the consumers. An artifact of this system is a unique subscriber ID Page 6

www.crispmedia.com

Measurement on Mobile October 2010

that is passed along with the mobile browser content requests. Any server that receives page or ad requests can use this value as a key for a server-side cookie or as a way to count unique requests and impressions. The limitation on this approach is that it does not work reliably on every carrier and some carriers only pass the subscriber ID for specific domains. Despite these limitations, counting users based on subscriber ID is often used by mobileoptimized systems as part of a suite of tracking methods. Device fingerprint Like so many things in the highly fragmented mobile device landscape, one methodology doesn’t fit all. Many mobile vendors resort to combining different systems and parameters to reduce the error rate. The downside is that the methodology becomes vendorproprietary, just like the approach to identify a user uniquely by the technical fingerprint of the device. There are easily more than one hundred possible different bits of info flowing from a device over the mobile Internet while a device makes an ad request or a page request. Most of them are not unique to the user but the combination of them can make it reasonably unique. When server-side cookies are implemented in mobile, they are most often identified using this method. No two vendors are using the same combination of bits of information, so no two vendors will have corresponding uniques with this method. Another danger in this method is the privacy issue it raises since the consumer no longer controls when and how to clear tracking. Many of these bits of information can’t be modified by the consumer. © 2010 Crisp Media, Inc.

HTML5 storage In addition to cookies, mobile devices that support a portion of the HTML5 standard are now supporting the storage of information locally on the device, from within the mobile browser. This can be used to store a cross-site user ID, and is sometimes used to circumvent problems with the reliability of cookies on a specific device. Recently, a plaintiff in a lawsuit against a vendor asserted that this method made it harder to opt-out of the collection of behavioral data. The privacy concerns around the storage abilities of HTML5 make us believe that this method is not yet ready for general use.

Clicks/Click-­‐Throughs  
Click-URL redirect To count clicks in mobile, the most common approach is also the approach most commonly used on the regular desktop web. The destination URL is replaced or prepended by a URL to the analytics server, which redirects the browser to the destination after measurement. Because multiple ad servers are often involved, several redirects are usually required. This redirect method takes much more time on mobile and also is not supported by various mobile devices. Server-side HTTP API If it doesn’t work right on the client browser, it has to be done from the server. With server-side redirects, the device is limited to one single redirect. Additional ad servers, for example the agency ad server, are hit by the first analytics server. This method creates a complication for downstream servers that expect requests directly from the Page 7

www.crispmedia.com

Measurement on Mobile October 2010

device. Participating servers must mimic all device headers to reliably correlate reports. Client-side JavaScript API Instead of depending on the browser to redirect through an intermediate ad

server before reaching the click destination, it is possible to hit the ad server directly with JavaScript while the user clicks. With JavaScript, multiple trackers can be implemented on the same page for each 3rd party ad server.

 

Conclusion  on  Mobile  Measurement  methods  
Vendors with very different technical backgrounds have joined the mobile ad serving community. The choice for counting methodology depends on their own knowledge of the mobile network and their pre-existing methodologies. If the priority is device compatibility, then a lowest common denominator approach wins out even though it has a very high margin for error. If the priority is accuracy, then the only methodologies to choose from limit compatibility to JavaScript capable devices.

© 2010 Crisp Media, Inc.

www.crispmedia.com

Page 8

Measurement on Mobile October 2010

The  Problems  with  Mobile  Measurement  
While the different measurement methodologies have a statistical variance when implemented perfectly, they are especially prone to problems, which can result in errors exceeding an order of magnitude. This section explains a range of problems that Crisp has witnessed with various implementations of the previous measurement techniques.

Filtering  Problems  =  Over-­‐Counting  
Filtering data is required whenever the methodology includes counting server hits to a public URL. This is especially true for URLs that return actual content. Poor filtering for search engine spiders There is no master list of mobile search engine spiders. Most can be filtered, but some still fake valid device user-agent headers in order to traverse mobile sites that implement device detection. If ad impressions are counted based on serverside ad requests then this will result in impression over-counting. Poor filtering of other robots Mobile optimized content has the advantage that it is rendered with a simple layout and a XML valid syntax. This makes mobile content much more reliable and attractive for content scraping. This is a common problem on popular, branded sites that are visited for up-to-date content by applications that make content available off-line or to re-purpose it. If ad impressions are counted based on server-side ad requests then this will result in impression over-counting. Effects from content feed readers There are feed reader applications that don’t only download the ATOM or RSS syndication feeds, but actually go as far as scraping the full content of a web page in order to make it available via the reader application. If ad impressions are counted based on server-side ad requests then this will result in impression over-counting.

Mediated  Ad  Requests  =  Over-­‐Counting  
There are various ad servers and ad networks that provide inter-mediation of other ad servers or ad networks to improve the fill rate of the mobile ad inventory. This is both the case for applications and mobile web. Optimizations made for delivering the ads quickly include pre-fetching of ads. If the origin ad server counts impressions based on a separate ad impression tracker, then there is not problem. However, this assumption is often made by inter-mediating servers, where it is not appropriate. This results in requests being counted as impressions even when the ad was never delivered. © 2010 Crisp Media, Inc. www.crispmedia.com Page 9

Measurement on Mobile October 2010

Malformed  Pixel  Trackers  =  Under-­‐Counting  
On many mobile devices the syntax for the HTML image tag is stricter than on desktop browsers. It often happens that the way the tag is structured prevents a hit to the server for an important portion of the devices. In addition, even when the tag is formed correctly, the device may not generate the tracker request because of the small size of the content.

Malformed  URLs  =  Under-­‐Counting  
The impression tracker URL can possibly exceed the maximum URL length for some cellular data networks or for some mobile devices. On many devices 100 characters used to be the maximum URL length. Today it is best practice to keep the URL length under 256 characters. Exceeding this limit may cause the tracker never to arrive intact at the measurement server.

Network  and  Browser  Caching  =  Under-­‐Counting  
Cache busting methodologies are often far less effective than they are deemed to be. Some of the mobile networks are highly optimized to limit unnecessary bandwidth use. While the mobile browser may adhere to a wider set of cache control directives, the caching proxies deployed within the mobile network infrastructure are more aggressive and may only adhere to very few directives. This issue has an effect on a multitude of measurement techniques.

Incompatible  Device  Browser  =  Over-­‐Counting  
Because impressions are so commonly counted based on ad requests, it is likely that part of these ad requests never result in an actual visible ad unit for the end-user. Whenever an image tag or a JavaScript tag is either malformed or delivered to an incompatible device, the ad doesn’t actually display.

© 2010 Crisp Media, Inc.

www.crispmedia.com

Page 10

Measurement on Mobile October 2010

Crisp  Recommendations  
Timing  of  Counting    
We realize that different vendors will use different approaches to counting. It is our recommendation that all counting happens as late in the ad delivery process as possible because we think that approach offers the greatest reliability. Vendors who count impressions based on ad requests (too early in the ad delivery process) instead of using separate impression tracking should disclose this to their customers and explain the effects. The discrepancy caused by ad request counting can be large and unpredictable.

Method  of  Counting  
Crisp recommends that impressions be counted with a separate tracker (either a large pixel or JavaScript) once the ad is actually rendered because it produces a reliable metric that can be easily verified by a second (3rd party) tracker. In other words, Crisp recommends client-side counting. However, Crisp also recommends that whenever possible, the time that the ad is in-focus is counted in addition to counting the impression. This dwell-time, or display-time metric is a necessary metric on mobile devices with small screens because many ads are quickly out of focus or never were in focus.

Segregation  of  JavaScript  and  non-­‐JavaScript  Devices  
Crisp recommends to all publishers with large enough ad inventory make a hard distinction between the inventory from feature phones, which is not reliably measured, and the inventory on JavaScript capable smart phones, which can be measured reliably. This distinction makes it possible to sell campaigns that have a better user-experience for end-users with more reliable reporting. Any existing installations that share the infrastructure between feature phones and smart phone, in which ad impressions are based solely on server-side ad requests, should be upgraded.

Applications  vs.  Mobile  Web  
Crisp recommends that publishers integrate advertising in applications in a similar manner to how advertising is integrated on mobile web content. The current situation, where a dozen or more ad serving SDKs unnecessarily fragment mobile advertising, highlights an immature ecosystem, and results in campaign measurement issues. Crisp fully supports the Open Rich Media for Mobile Advertising initiative (http://ORMMA.org), © 2010 Crisp Media, Inc. www.crispmedia.com Page 11

Measurement on Mobile October 2010

which is being adopted by publishers and partners. The ORMMA standard enables participants to integrate mature desktop ad infrastructure into mobile web and mobile applications without adding measurement complexities.

  Summary  
In this document, some history, definitions, problems, and Crisp recommendations for mobile measurement are explained. The mobile ecosystem has witnessed extreme growth and continuing fragmentation; Crisp’s conclusion is that methods are now available to extend desktop infrastructure onto mobile without introducing the measurement problems of the past. We recommend that publishers integrate their mobile content with client-side ad tags that take advantage of modern device capabilities. Crisp also promotes the use of ad currency definitions beyond the CPM and CTR, like display time and interaction rate.

About  the  Author  
Xavier Facon is CTO at Crisp, a rich media advertising technology company based in New York City. He writes regularly about ad technology and mobile best practices.

About  Crisp    
Crisp offers a universal rich media ad platform for building, serving, and measuring multi-platform campaigns across the desktop and mobile Web, mobile apps, tablets, and connected TVs. Crisp’s innovative ad formats and its Adhesion™ fixed placement technology empower brands to engage with consumers and drive interaction across various channels, while simplifying creative development, streamlining ad serving, and unifying reporting for agencies. Leading brands including Estee Lauder, HBO, Intel, Marriott, Paramount Pictures, Toyota, Volkswagen, and others have utilized Crisp ads. Crisp has over 700 certified sites and apps, and has partnered with leading publishers including CBS, CNN, Hearst Magazines, The Wall Street Journal Digital Network, The Weather Channel, and others. For more information visit www.crispmedia.com.

 

© 2010 Crisp Media, Inc.

www.crispmedia.com

Page 12

You're Reading a Free Preview

Download
scribd
/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->