White paper

WHAT CAN YOU DO WITH GOMEZ?
THE CAsE fOr A WEb ExpErIENCE MANAGEMENT plATfOrM fOr WEb 2.0 AND bEYOND WINTEr 2008

Executive summary
Customer demand for the superior web experiences that Web 2.0 applications can provide -- including increased user interactivity, relevant real-time content and rich media, content created by social networks -- is driving leading web companies to design complex new end-user facing web applications. But building these applications often requires moving functionality from the well-understood server environment to inside the user’s browser itself. For the majority of companies accustomed to developing applications in the traditional way, the Web 2.0 paradigm represents an unprecedented loss of control. Many continue to develop applications behind the firewall, test with conventional tools, and launch into production with minimal regard for the actual customer experience on the desktop. Along with these new end-user requirements comes the need for an integrated platform of tools to painlessly support those tasked with designing, developing and deploying those new applications. The ExperienceFirst platform from Gomez described in this white paper provides the industry’s first end-to-end solution for web application experience management throughout the application lifecycle. This technology promises to help business and IT managers to: Ensure end-users have quality Web experiences; Set performance and web experience goals early in the design phase and test against those goals throughout development and quality assurance (QA) phases; Monitor and manage application performance levels and the impact of third-party content across multiple browser types and operating systems; Measure the relationship of performance and customer satisfaction with analysis of yield and conversion rates of marketing funnels; and Understand the impact of changes – in everything from partner relations, vendor sources, capacity, addition of new applications and site modifications – on end user experience.

applications and infrastructure delivered by third parties. As we move toward rich Internet technologies such as Ajax, even less of the customer experience is under our direct control. Yet the customer still holds the host brand accountable for
the web experience and expects much richer services delivered instantaneously.” Steve Trimbo Director of Online Operations Best Buy.com

It’s a composite world today, so much of the web experience a company provides hinges on content,

WHITE pApEr

Managing User Experience and the Organization

Contents
Introduction ......................................................................................................................................................................................3 A Closer Look at Web 2.0 Composite Applications ...................................................................................................................3 Inside the Gomez ExperienceFirst Platform ...............................................................................................................................5 Putting It All Together.....................................................................................................................................................................8 What can you do with Gomez? ......................................................................................................................................................9 Appendix A: Web Experience Management in Practice – A Customer Example .................................................................................................................................................................... 10

The information contained in this document represents the current view of Gomez, Inc. on the issues discussed as of the date of publication. Because Gomez must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Gomez, and Gomez or its respective suppliers cannot guarantee the accuracy of any information presented after the date of publication. This white paper is for informational purposes only. GOMEZ AND ITS RESPECTIVE SUPPLIERS MAKE NO WARRANTIES, EXPRESS OR IMPLIED, AS TO THE INFORMATION IN THIS DOCUMENT. Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under copyright, no part of this document may be reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express written permission of Gomez, Inc. Gomez may have patents, patent applications, trademarks, copyrights or other intellectual property rights covering subject matter in this document. Except as expressly provided in any written license agreement from Gomez, the furnishing of this document does not give you any license to these patents, trademarks, copyrights or other intellectual property. ©2008 by Gomez, Inc. All rights reserved. Gomez® is a registered service mark, and ExperienceFirst™, ExperienceFirst SCoE™, ExperienceFirst Network™, Active Network™ XF, Private Network™ XF, Active Last Mile™ XF, Private Locations™ XF, Actual Experience™ XF, Reality Check™ XF, Reality Load™ XF, Reality View™ XF, the taglines “Ensuring Quality Web Experiences™”, “Quality of Web Experience™”, and QoWE™ are service marks of Gomez, Inc. All other trademarks and service marks are the property of their respective owners. Gomez, Inc. 10 Maguire Road, Suite 330 Lexington, MA 02421 USA

This document contains information of a proprietary nature. All information contained herein shall be kept in confidence and shall be for the original recipient’s use only. Any unauthorized reproduction by any other party shall constitute an infringement of copyright.

2

Gomez, Inc.

10 Maguire Rd, Lexington, M A 02421-3119 www.gomez.com T: 781.778.2700 F: 781.778.2799

WHITE pApEr

Managing User Experience and the Organization

Introduction
Website development has completely transformed in a very short amount of time. Firstgeneration sites circa 1997 – 2003 were architected in a fairly straightforward manner, Site control resided neatly on the server side. Tens of thousands of sites were built following a basic three-tiered layered approach where each layer was neatly segmented by function and all the underlying site components resided in the data center. The back-end mix of databases, application servers and web servers were all under the control of developers, and by extension, the business and brand behind the site.
Year 2000: IT Controls Web Experience
thin/ dumb client

Year 2007: Web Application Lifecycle Distributed
smart clients

www.mybrand.com

www.mybrand.com

Customer experience is browser dependent with AJAX, FLEX, Flash Technologies

Web Server Web Container Application Server EAI Zone of Direct Control

Web Server Web Container Application Server EAI Zone of Direct Control
App Acceleration, Content Quotes, Bill Payment, Store Maps, Package Tracking, etc.

Application components outside of companies control

Fast-forward to today’s Web 2.0 world and the difference is astonishing. Top web applications today have leaped from yesterday’s simple setup to a more complex architecture focused on providing superior online customer experiences. More user interactivity, relevant content and rich media is what leading web companies want to deliver in order to achieve a superior user experience. Outsourcing this new content and functionality saves time and money and makes good business sense. Implementing these technologies almost always requires moving more functionality from the well-understood server environment to inside the browser itself. A big challenge today is that only a few companies know how to do this well. Unlike ten years ago when companies clearly maintained ownership of the entire web experience by controlling it from the data center, today there is no single owner of the user experience. Modern web applications are typically comprised of code and content coming from numerous parties, only one of which is the company whose logo and brand are displayed prominently on the page. This mosaic of content and services that comes together in the browser relies far more on local processing than ever before. With today’s multitude of browser types, computer operating systems, and versions, there is a real need to ensure that web applications work well for all end-users. Outside of a select few developers and business managers in the savviest web companies, few people realize how much of a site’s content and functionality depends on outside sources. Yet building sites that use an increasing number of outside services is increasingly the norm.

A Closer look at Web 2.0 Composite Applications
Leading organizations are striving to deliver increasingly interesting, useful, or exhilarating web experiences. With Google Maps, for example, users can “speed drag” satellite images of their neighborhood, city, state or country at will. That’s because the underlying Ajax code (Asynchronous JavaScript and XML) anticipates their movements and makes server calls behind the scenes. Mash up Google Maps with a real-estate brokerage home finder, and a Web 2.0 developer can suddenly have a lot going on at the browser level that not long ago would have happened on the server, where they could control it.

3

WHITE pApEr

Managing User Experience and the Organization

Unbeknownst to most users, adding layer upon layer of third party content to web applications is becoming increasingly common. For example, users may think they’re just paying bills on their bank’s web site. In reality, they are interacting with a third-party bill-paying application that is enlisting another provider’s hidden web analytics service. Similarly, comparison shopping services are often outsourced to providers who add features such as local shopping to webonly national businesses. In the new paradigm, it’s not unusual for sites to have 10 or more service providers contributing the website content and functionality that end-users rely on. Meanwhile, businesses increasingly rely on service-oriented architectures (SOA) to expand the functionality of their web applications. These applications leverage blocks of external application code or web services to add a wide range of features and functions to a site. In many cases, these applets are supplied not only by the company itself but by partners invisible to end-users. With all this added complexity, few companies today really understand how their applications are performing from the end-user perspective. The fact is, in today’s composite world, it’s difficult to tell when some combination of external content and the user’s browser or operating system breaks down and spoils the user experience, particularly if the organization is using conventional testing tools.
Ajax 63.1% Flash/Flex 57.1% Streaming 28.9% Silverlight 4.7% Adobe Air 2.7% Other 14.1%

What Rich Internet Applications (RIAs) have you deployed or are planning to deploy in the next year? (Percentages are based on the total number of responses)

Why did you choose to use these Rich Internet Applications (RIA’s)? (Percentages are based on the total number of responses)

The user experience trumps all. When done properly, pushing more Ajax and Flash code to the browser enhances the user experience and makes the application more rich and responsive. However, as more content and code goes directly to the browser, it’s harder for developers to predict performance. Often, design problems don’t emerge until after deployment in a live customer-facing environment. When web applications fail outright or perform poorly, the danger for the business is lost revenue to competitors and long-term damage to the brand. For developers, it means spending time on rework rather than focusing on future application development.
Worryingly, many organizations still aren’t thinking about the increasingly important role of extended web architectures. Following traditional development procedures, they continue to test what’s inside their zone of control and to see the effects of external content and code only after their application is launched into production.

User Demand 29.5%

Inadequacy of existing technologies to meet design goals 39.7%

Keeping up with the market/competitive pressure 56.2%

Desire of internal developers to use new technologies 36.3%

Executive Mandate 7.5%

Other 11.6%

With increasingly sophisticated user expectations, those tasked with developing, deploying, and managing applications can no longer afford to wait to test, tweak, and improve web applications and practices. The stakes for their companies are too high. Rather, they must start early on and set their targets to define success criteria for the user experience in the Web 2.0 world. What’s needed is a new, more fully integrated development and management platform that starts with the end-user experience in mind, builds on and combines the time-tested processes that have traditionally been favored.

4

Gomez, Inc.

10 Maguire Rd, Lexington, M A 02421-3119 www.gomez.com T: 781.778.2700 F: 781.778.2799

WHITE pApEr

Managing User Experience and the Organization

Inside the Gomez Experiencefirst platform
To meet customer demand for superior online experiences, a new generation of web performance management tools has emerged that gives control back to the company. The Gomez ExperienceFirst™ Platform is the first on-demand platform to focus on ensuring a superior web experience throughout the entire web application lifecycle. This platform uniquely supports all aspects of the web development effort, including the design, development, validation, deployment, operation, analysis, management and continuous refinement of web applications. While other solutions test or monitor the shrinking behind-the-firewall portion of a web application, the ExperienceFirst Platform takes a holistic view by focusing on the entire web application as it is delivered in today’s fragmented Web 2.0 world, to the actual end-user.

A Great User Experience Starts Early. In order to minimize costs and maximize the chances of delivering satisfactory experiences to users, planning for and integrating the user experience into today’s Web applications should occur at the earliest stages of development, not later or as an afterthought. At the early design and development stages, two services in the Gomez ExperienceFirst platform work side by side to ensure that web applications both appear and work as intended, for all users on all browser and OS combinations. Reality View XF and Reality Check XF employ easy to use, Ajax-based web interfaces to help developers understand the impact of their programming on end users. Gomez Reality View XF is a cross-browser compatibility and performance testing service that profiles the behavior of web pages across multiple browsers such as IE6, IE7, Safari, Opera and Firefox. It provides a one-click screen capture view of how page prototypes on a site appear and perform when viewed across multiple operating systems, including Windows XP, Windows Vista, Linux and the Macintosh. The tool shows how well applications perform under various browser screen sizes, and reports back details on how each browser type renders on a page. This helps designers and developers to optimize ck pages as needed, long before they are delivered to an end-user’s browser.

Reality View XF

Gomez Reality Check XF is a full-featured functional testing tool for every type of web application, including complex Web 2.0 applications dependent on Ajax and other Rich Internet Application frameworks that rely on browser-side processing. It makes certain that each function and third party component going into an application works as intended and displays properly in the user’s browser. Reality Check XF can also be used to record and automatically replay specific user transactions, such as searching for a book or placing an item in a shopping cart. Developers can easily unit test across different types of browsers or operating systems to make sure everything on each page is working as expected. In addition, a movie capture mode in Reality Check XF helps developers to see dynamic page interactions as they occur, and verify how those interactions will be perceived by the end-user, across multiple browsers and operating systems, before the application goes live.

More Meaningful Load Testing. Once an application has been developed
and tested from the customer perspective under normal conditions, it’s important to understand how the application will perform when it’s forced to bend. By understanding what type of load it takes to degrade application performance, or what it takes to bring an application or a vendor to its knees, companies can be better prepared for those situations and better understand what it takes to consistently deliver a quality web experience to their end-users. Gomez’ Reality Load XF is different because it is an external load testing service that is designed to mirror the characteristics, demographics, actions, and volumes of your company’s actual user population. Utilizing the Gomez Last Mile panel of 25,000+ global testing agents – the largest and only of its kind -- Reality Load XF can test an application under development or in production by tapping a population that represents a companies’ real-life customer base, rather than from a series of vendor designated “load centers”. Additionally, companies can determine which specific geographical areas should receive the most load (e.g. Tokyo, California, Europe), giving developers greater confidence that the code they are writing will work as intended for real-world customers when the application goes live in production.
5

Reality Load XF

WHITE pApEr

Managing User Experience and the Organization

Preview the Customer’s Web Experience. Once development teams are sure their application will look and function properly for end-users across different browsers and OS’, it’s time to transition the application to the Operations team and take testing one step closer to the customer. Using Gomez’ active web application testing and monitoring products, companies can test their applications in a pre-production environment to understand how they will perform for customers around the world, without actually having to expose the applications to customers.

Private Locations XF
Validate the speed and reliability of your partners and corporate end users

WEB SERVER

APP SERVER MAJOR PROVIDER DESKTOPS

LOCAL ISP

DATABASE

CONSUMER

3RD PARTY CONTENT

Active Last Mile XF
Extend performance insights down to the desktop

Private Network XF
Behind-the- rewall network monitoring to quickly determine performance problems

Active Network XF
Real-time monitoring from the Internet Backbone

MAJOR PROVIDER

LOCAL ISP ACTUAL CUSTOMER

Actual Experience XF
Measure your online customer’s experience from their browser

Gomez Monitoring Products
Active Network XF - Measure web application performance from the Internet backbone in over 80 major global cities so you can baseline performance, troubleshoot potential issues, and accurately measure improvements. Active Last Mile XF - Measure web application performance from a population representative of your customer base by testing web application performance from over 25,000 broad band and dial-up machines across the globe. Private Locations XF – Measure web application performance from the locations most important to you. Use Gomez software to create your own testing network of customers, partners, branch offices, kiosks, or remote employees. Private Network XF – Measure web application performance from inside your firewall to determine whether a performance problem is hardware or network related.

6

Gomez, Inc.

10 Maguire Rd, Lexington, M A 02421-3119 www.gomez.com T: 781.778.2700 F: 781.778.2799

WHITE pApEr

Managing User Experience and the Organization

Ensure Quality Web Experiences in Production. Because revenue is not generated in your
test environments or inside your firewall, Gomez provides the same active monitoring services that companies can use for pre-launch testing to monitor and troubleshoot the performance of those applications once they are released and customer facing. Additional monitoring tools that leverage these agents, including Gomez ExperienceFirst Alerts, ExperienceFirst Dashboards, and ExperienceFirst Screen Capture on Error (SCoE), and Performance Analyzer also serve to help companies identify and resolve problems with pre- and post-production web applications more quickly.

ExperienceFirst Dashboard Measuring Experience from the Customer’s Browser. Using Actual Experience XF, companies gain an authentic picture of customer web experiences, individually and collectively, by measuring performance directly from the end-users’ browsers as they visit web pages. As such, Actual Experience XF is a real-user monitoring service that expands the ExperienceFirst platform to address the new challenges facing both business line owners and Web 2.0 developers concerned with the performance impact on customer web experience. Using a few lines of proprietary code embedded on each page (similar to web analytics technology), Actual Experience XF reports back on the end-user experience with the portion of the application delivered by the servers and on the portion running locally in end-users’ browser (e.g. Web 2.0 technologies like Ajax and Flash).

What it measures. From its vantage point in the browser, Actual Experience XF measures all
aspects of the end user experience, including the perceived performance of a web application, download times, abandonment rates, service consistency, geographic disparities, browser window size, browser type, cache utilization and first time/repeat visitor status. Armed with this information, a company can: Analyze the actual experience of specific customers and segments, and understand which buyer segments have poor performance and why; Know which pieces of a company’s website contribute to (or detract from) a favorable customer experience, including third party content providers, use of multiple browsers, and rich media types; and Make better decisions as to what Web components to monitor and improve by analyzing performance and how they impact business performance. Actual Experience XF dives deeply into many factors influencing customer satisfaction and far beyond what’s found in standard web analytics. For example, the tool can uniquely show the impact of application performance on conversion by tracking both page level abandonment rates (e.g., when a user clicks on the stop button of a browser to stop the page from downloading) and business process level abandonment (e.g., when a user abandons a multi-step process before reaching the last step, such as putting an item in the shopping cart but never checking out).

7

WHITE pApEr

Managing User Experience and the Organization

putting It All Together
The demand for superior web experiences has necessitated and accelerated changes in website development considerably over the last few years. Sites have evolved from simple process applications to more complex architectures. The emphasis is on providing a superior customer experience with more interactivity and use of rich media. Much of the code that adds to the experience resides in the browser, not on the company’s server. Yet, few developers today know how well their local code is performing – a dangerous situation considering that some combinations of the browser or operating system may not work as expected. The Gomez ExperienceFirst Platform reaches beyond production and monitoring and back to the early stages of development and QA. It emphasizes the user all the way from early development to post-production and business analytics. Using a single web-based user interface and delivered in on-demand model, business and technical users alike can use the platform to understand how web performance impacts business results and track performance of business processes at every stage of an application’s lifecycle. With ExperienceFirst, designers, developers, QA teams, operations and business managers can benefit from a unified platform where they can employ the best tools to optimize their sites for the best possible user experience. With the ExperienceFirst platform, companies can: Test the customer experience much earlier in Application Lifecycle; Exploit a Web 2.0 ready pre/post deployment testing platform; Continuously monitor actual & synthetic customer experience for all their web applications; Leverage integrated scripting, reporting, alerting, and analytics on data from the World Largest (Over +25,000 POPs) Experience Testing Network; Improve collaboration on Web experience among design, operations and business management; and Better ensure quality Web experiences.

TM

Develop & Design

Test

Monitor

The rst on-demand web experience management platform focused on Ensuring Quality Web Experiences from the customer perspective.

8

Gomez, Inc.

10 Maguire Rd, Lexington, M A 02421-3119 www.gomez.com T: 781.778.2700 F: 781.778.2799

WHITE pApEr

Managing User Experience and the Organization

What can you do with Gomez?
If the goal is to improve your functional testing processes, get better insight in to customer experience, improve the efficiency of Web application teams, or resolve problems more quickly, the suite of on-demand products in the Gomez ExperienceFirst Platform can help.
Table 1. What Can You Do With Gomez?

Find web application problems at the user, browser, application, network or server level. Identify visual issues end users will experience using different browsers and mobile devices. Check functionality in QA & Development across multiple browsers and operating systems to minimize issues in production. Test your website under load generated from where it counts most -- at the user’s computer. Validate the performance and readiness of applications from the end–user perspective prior to launch. Establish historical baselines of Web application performance. Monitor the performance of production application continuously to find problems before users are impacted. Develop alert mechanisms that automatically detect and create alarms for application and user experience issues. See competitive baselines that compare your organization’s performance to others in your industry. Observe and measure the performance of your third-party partners. Hold partners and internal groups to standards set in Service Level Agreements (SLAs). Segment users by type, geography, and value to your organization to measure experience. Discover how web experience is impacting your businesses bottom line. Build common goals for response times, availability, consistency and other experience metrics. Apply tools & Best Practices that can help your team measure, monitor and improve end use experience.
© Gomez 2008

9

WHITE pApEr

Managing User Experience and the Organization

Appendix A: Web Experience Management in practice – A Customer Example
Gomez agent-based active testing can be combined with real-user actual testing to reveal exactly how well an application is performing under a variety of conditions for user populations around the world. The following scenario illustrates how the Gomez tools work together to help several departments in a company, share metrics, align on the same goals, and provide the best possible user experience.

A Retailer doing significant business in the United States is releasing a new Web 2.0 shopping cart application, but wants to make sure that the release will not negatively impact the customer web experience – and therefore the businesses revenue. Based on information captured with Actual Experience XF, business managers have benchmarked the current end-user satisfaction levels and also determined that the large majority of top customers in the US are accessing the company’s Website using a variety of browser/OS combinations, but are primarily on one ISP using low-broadband and dial-up connections around the Western US. Armed with this information, the company’s developers went to work with the goals of meeting or exceeding the current end-user performance and experience levels. Using Reality View XF and Reality Check XF, the developers are able to test the code as they build it. To make sure the new application both functions and appears as intended from all the different browser and operating system combinations, including those of the high value customers. Once the application had been developed, the quality assurance team decides that it’s ready to put the new application to the test. Using information gathered with Actual Experience XF, the company uses Gomez RealityLoad XF to create a load testing population scenarios that are representative of their customer base. They then test the pre-production application with the real traffic levels that it expects to see in the upcoming busy season. Assured that scalability is not a concern, the operations team is now able to use Active Network XF and Active Last Mile XF to test the beta application and measure how it responds from the perspective of the low-broadband and dial-up customers. Fortunately, this testing exposes a period issue with their third party content acceleration vendor that was causing a 50% increase in response time. After sharing the data with the vendor, the company then validates that the discussed fix has been implemented and has resolved the problem. Satisfied with the results of the load test and the operations team’s initial testing, they share the results and expected user experience levels internally to review that the initial goals have been met. After getting approval from all the involved units, the operations team sets up continuous monitoring to catch any new customer or application problems that might arise and launches the new application into production with confidence that the new application will exceed the quality of web experiences delivered by the old application. The business managers, eager to measure the impact of the new release, quickly correlates the new Actual Experience XF measurements with the old measurements to determine that they have indeed successfully launched the new application, and improved performance for that key customer segment more than anticipated, and achieved the web application experience goals they set at the beginning of the process.

Gomez, Incorporated 10 Maguire Rd, Bldg 3, Suite 330 Lexington, MA 02421 General: +1 781-778-2700 Sales: USA +1 781-778-2760; UK +44 (0)207-554-5154; Germany +49 (0)40-53299-167. www.gomez.com

©2008 by Gomez, Inc. All rights reserved. Gomez® is a registered service mark, and ExperienceFirst™, ExperienceFirst SCoE™, ExperienceFirst Network™, Active Network™ XF, Private Network™ XF, Active Last Mile™ XF, Private Locations™ XF, Actual Experience™ XF, Reality Check™ XF, Reality Load™ XF, Reality View™ XF, the taglines “Ensuring Quality Web Experiences™”, “Quality of Web Experience™”, and QoWE™ are service marks of Gomez, Inc. All other trademarks are the property of their respective owners.
Specifications may change. Consult the technical documentation for the most current information. Some features described are extra cost options. Ask your sales representative for details.

wp2008003