Holiday 2000 E-Commerce


Avoiding $14 Billion in “Silent Losses”

by Aamir Rehman
October 2000

Thanks to all who helped create the Holiday 2000 E-Commerce report: • Nicole Rubin, Creative Good’s director of research, managed the report overall. She managed the project, directed the research and edited drafts of the report. • Mark Hurst, Creative Good’s founder and president, edited the final version of the report. • Susan Larson, Creative Good consultant, contributed to the research and production of the report. • Creative Good analysts Allison Jacoby, Jeniece Primus, Jennifer Thomson and Lauren Greenberg contributed to the research and analyzed the customer test results. • Boston-based interactive agency Red98 produced an HTML version of Creative Good’s prototype to test for the case study. Committed to creating “great user experiences,” Red98 clients span a wide range of industries, from Fortune 100 companies to startups. You can learn more about them at

About the author

Aamir Rehman (, senior analyst at Creative Good, has advised e-commerce clients in the grocery, automotive, gift and apparel industries. He holds a bachelor’s degree from Harvard College and a master’s degree from Harvard’s Graduate School of Arts and Sciences. Aamir also manages Creative Good’s best practices research.

_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 2

Executive Summary 1. Introduction 2. Business Case 3. Methodology 4. Checkout 5. Promotions and Merchandising 6. Case Study: BestBuy 5 7 11 16 19 34 45

All company names, brand names and product names in this report are trademarks of their respective owners and companies. v1018.1

_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 3

About Creative Good’s E-Commerce Consulting Services
Creative Good ( is a 35-person consulting firm based in New York, San Francisco, and Boston. We’re on a mission to improve the online customer experience — on e-commerce websites, content and entertainment sites, intranets, b2b sites, and wireless Internet services. Clients include Gateway, Travelocity,,, Rogers, and We have generated 40% to 150% improvements in all of our major projects. Please contact us if Creative Good can help improve your e-business. Phil Terry, CEO P.S. You can find our other free resources at To get our free customer experience newsletter, e-mail

_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 4

Executive Summary

About this Report
The purpose of this report is to help consumer e-commerce websites prepare for the Holiday 2000 season. To get a representative view of online holiday shopping, we conducted more than 50 consumer tests on eight major e-commerce websites across four key categories: • Apparel:, • Books and music:, (Barnes & Noble) • Electronics:, • Toys:, To further demonstrate the impact of improving the customer experience, we chose one of the sites we tested — — and prototyped an improved website based on what we learned in tests. When we then tested the new prototype, the results were dramatic: customers were four times more successful on our prototype than they were on the original (see Chapter 6). The BestBuy case study suggests that many e-commerce sites can dramatically increase their revenues with simple improvements to the customer experience.

Summary of Findings
The customer tests and the BestBuy case study produced the following key findings: • In our tests, 43% of purchase attempts failed. As we explain in the Business Case (see Chapter 2), these failures may represent $14 billion in lost sales worldwide this holiday season.
_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 5

• Problems with checkout were the greatest cause of failure. Over 40% of failed purchase attempts were due to difficulty during the checkout process. Poor checkout experience alone may cost the worldwide e-commerce industry over $6.5 billion in holiday sales (including over $3 billion in North America alone). • Promotions and merchandising are an area of opportunity this holiday season. Gift shoppers were open to suggestions and promotions: in 30% of tests, customers clicked on at least one featured product. • If sites prepare for the holidays by improving the customer experience, the industry stands to gain $14 billion in sales.

_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 6




This report shows how to improve the customer experience on any e-commerce site as the 2000 holiday season approaches. It’s a timely subject, given how hard this year has been on the e-commerce industry. Investors now want profits, not just clever TV ads and a dotcom name. Many sites are looking to Holiday 2000 to prove their viability as real businesses. The single most important factor in any e-commerce site’s survival and success is its customer experience. This report shows exactly what e-commerce sites must do to improve the customer experience for Holiday 2000. It is worth thousands or millions of dollars to companies that follow its suggestions. We know these suggestions work because we tested them. In our BestBuy case study (see Chapter 6), we tested the customer experience of the current BestBuy site, and then created our own redesigned version — according to the customer experience suggestions in this report — and tested the improved prototype. Result: The conversion rate quadrupled. Our tests indicate that could make the simple changes we suggest and instantly double its revenues, if not triple or quadruple them. (Indeed, it’s not unusual for Creative Good clients to measure a similar rise in their own key metrics.) The lesson here is striking: Making simple improvements to the customer experience can easily double the sales of many top e-commerce sites.
_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 7

Why focus on customer experience?

It’s a fair question: why focus on on-site customer experience during Holiday 2000, when last year’s holiday season showed so many other problems on e-commerce sites? Fulfillment, for example: the New York Times reported just after Holiday ’99 that “the primary bugaboo was failing to deliver packages on time1.” And a recent Business 2.0 cover story remembered last year’s “flooded customer service desks,” as angry customers called and e-mailed about undelivered packages. In fact, as a result of fulfillment troubles, seven e-commerce sites were fined a total of $1.5 million by the Federal Trade Commission2 . Yet this report maintains that the on-site customer experience is the single most important factor in e-commerce success. Regardless of the dozens of articles about fulfillment and overloaded call centers that were written last year, and will probably be written this year, poor customer experience is still the biggest problem in the e-commerce industry. Fulfillment is naturally a high-profile issue. Complaints to the customer service department inevitably raise attention in the company, and in the press, because complaints are loud. The customer completed the order, paid the money, and now the package is late; only complaining loudly, the customer reasons, will get the problem solved. But what about the customers who didn’t complete their orders on the website? The site never hears from them. Customers who get confused on the site, then leave forever without ordering or paying, aren’t interviewed by the press. They don’t overload the call center with complaints. They just get frustrated, leave the site quietly, and never return. The orders that customers fail to complete, because of the poor customer experience, are what we call silent losses. No matter how many customers complain about fulfillment problems, many more customers leave the site without complaining, because the site is too hard to use.

The New York Times, “A Very Big E-Commerce Christmas,” Jan. 2, 2000. 2 Business 2.0, “Hope for the Holidays,” October 10, 2000.
_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 8


“Silent losses”

Silent losses are hard to measure, since frustrated customers don’t call or e-mail the site to say they’re leaving. The press doesn’t usually write stories about silent losses, since it’s hard to measure the size of the problem, and it’s difficult to find people who have quietly left a site in frustration. But the silent losses are there. We know this because we observed, face-to-face, average e-commerce customers attempting to place orders on e-commerce sites. Our more than 50 tests of eight major e-commerce sites yielded a startling finding: In our tests, 43% of purchase attempts ended in failure. Forty-three percent! Can you imagine 43% of shoppers physically walking out of a Wal-Mart store because they couldn’t figure out how to pay for their order? What if 43% of visitors to the Starbucks on the corner left the store in frustration, grumbling about buying coffee somewhere else? No brick-and-mortar store would ever stand for such an insane failure rate. But here is the e-commerce industry, letting 43% of its revenues slip away. In our Business Case (see Chapter 2) we show that this amounts to a loss of $14 billion industry-wide this holiday season. This report is our plea to e-commerce sites to improve their customer experience so that they, and the industry as a whole, can survive into next year — and so that customers will no longer have thoughts like these, comments spoken by customers in our tests: • “Screw it, I’m just going to go to the store and buy it. I don’t know what I want to do at this point.” • “I’d probably get off right now.” • “That’s it, I’m out of here.”

Simple solutions

Fortunately, there are simple improvements that e-commerce sites can make to substantially raise holiday sales. If an e-commerce site can improve just two things before the holidays, we’d suggest the following: • Make it easy for holiday shoppers to find gifts. In Chapter 5 we show how promotions and featured items were especially effective in tests.
_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 9

• Make it easy to check out. More customers had trouble with checkout than any other feature on the sites we tested. In Chapter 4 we show some easy ways to raise revenues by improving the checkout process. We should also note that these are not the only improvements e-commerce sites should make. Successful e-businesses make customer experience a constant focus, continually monitoring and improving it month after month. And that means more than simply prototyping and redesigning the tactics of the website interface. Instead, our client projects have shown that to improve the e-business, we have to change the organization behind the interface. We’ve found the Web interface to operate like a roadmap of the organization, describing its strategy and culture. So as you read the report, do note the tactical changes we suggest to help prepare for Holiday 2000. But more importantly, consider making a strategic commitment to customer experience in your e-business. And let us know if we can help.

Mark Hurst Founder and President, Creative Good

_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 10



Business Case
$14 Billion in Silent Losses

The year 2000 has been tough for online retailers. Changes in the market have driven many sites out of business, and many others are only barely surviving. The holidays, promising greatly increased sales, offer hope for many e-commerce sites. The good news for retailers is that the holiday market will grow; this year’s holiday sales are predicted to be 66% higher than last year’s.3 In a year when over half the U.S. population has access to the Internet,4 e-commerce sites can expect dramatic increases in site traffic before the holidays. Forrester Research predicts $10 billion in holiday revenues, and Jupiter forecasts the U.S. holiday market as $12 billion. Gartner Group predicts $10.7 billion in North American holiday sales and worldwide sales of $19.5 billion.5 While revenue estimates vary, it’s clear that Holiday 2000 will be big for e-commerce. Customers will shop online this holiday season for convenience; by shopping online, customers avoid crowds, transportation hassles, and inconvenient store hours. One customer commented in our tests that she will shop online “to avoid the hassle of going to different stores.” Another shopper said that buying gifts online is “a lot easier because of the lines [in the stores].”
3 4

Cyberatlas - Jupiter, September 18, 2000 Nielsen/NetRatings, August 17, 2000 5 eMarketer - Gartner Group, September 6, 2000
_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 11

Despite the eager shoppers and the predictions of a big holiday season, there is bad news for online retailers: the e-commerce industry is losing billions of dollars in potential sales. Shoppers want to purchase online, but many sites make it too hard to buy. In our tests, we observed that 43% of purchase attempts ended in failure. Frustrated customers made comments like “if it took this long, I would just go to the [off-line] store.” As one shopper put it, “[I might] buy something this holiday season online, but not on these sites, because they are so slow and annoying.” Sites’ actual revenues this season will represent only a fraction of what they could be if sites made it easier to shop and buy. To gain the most from the holiday season, sites must pursue a simple strategy: improve the customer experience. Improving the customer experience can yield an additional $8 billion in North American sales, and $14 billion in worldwide online sales this holiday season. Poor customer experience may cost the worldwide e-commerce industry $14 billion this holiday season.

$14 Billion in Lost Sales
According to Gartner Group projections, worldwide online revenues for the holiday season will be $19.5 billion.6 In our consumer tests of major sites across key holiday categories, we found that 43% of buying attempts failed. These findings suggest that the $19.5 billion represent only 57% of potential sales this holiday season: • 100% (potential market) - 43% (due to buying failures) = 57% (of sales realized). If $19.5 billion represents only 57% of potential sales, total worldwide potential sales for Holiday 2000 are over $34 billion: • $19.5 billion / 0.57 = $34.2 billion. Lost sales due to poor customer experience thus total more than $14 billion: • $34.2 billion - $19.5 billion = $14.7 billion Of course, these figures are based on several assumptions about how the market will

eMarketer - Gartner Group, September 6, 2000

_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 12

perform. They are not meant to be scientifically accurate but rather to give a sense of the possible loss of revenue this holiday season. If our calculations are anywhere near correct, sites will leave a massive amount of money unrealized — unless they improve the customer experience. It is worth noting that our $14 billion does not include future losses due to poor experience this holiday season. Millions of consumers will shop online for the first time this holiday season. If they have a good experience, they are likely to continue shopping on their favorite sites in the future. If they have a bad experience, they will likely never return to the sites where they were unable to buy (if indeed they return to shop online at all). The actual losses due to poor customer experience may thus be significantly higher than our $14 billion estimate.

Problems with Checkout
In our tests, problems with the checkout process were the greatest cause of failed purchase attempts. More than 40% of failures were due to difficulty with checkout. Customers had found a product they wanted, added it to their cart and decided to place an order. They were ready to buy and wanted to pay. Unfortunately, the checkout process was too difficult and made it impossible for them to buy. Customers want to buy, but many sites won’t let them pay. As we discuss in Chapter 4, there were several different types of problems that arose in the checkout process. The most important lesson is that poor checkout experience will cost the industry billions in holiday sales. Causing over 40% of all failures, checkout problems amount to over $6.5 billion. Improving checkout alone could add over $6.5 billion to worldwide holiday sales.

The Impact of Improved Conversion
The best measure of a site’s effectiveness is the site’s “conversion rate.” This metric refers to a site’s ability to “convert” visitors to buyers. A site’s overall (“blended”) conversion rate is the percentage of visits that result in a purchase. For example, a site with three purchases for every hundred visits will have a blended conversion rate of 3%. Other metrics like “hits” and unique visitors might, at best, measure a site’s popularity — but the conversion rate measures the overall effectiveness of a site’s
_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 13

business. The conversion rate is the most important metric to observe, because it measures a site’s ability to produce revenues and operate profitably. According to and the Boston Consulting Group, the average conversion rate for e-commerce sites is 1.8%.7 This means that for every hundred visits, roughly two result in a sale. 98% of visits do not end in a purchase. E-commerce leaders, however, enjoy much higher conversion rates. For example,’s conversion rate is reported to be above 9%, over five times the industry average.8 The best e-commerce sites constantly monitor and improve their conversion rate by continually working to improve the customer experience. They recognize that even slight increases in the conversion rate result in significant incremental revenue for the business. Small increases in the conversion rate lead to large increases in revenue. Consider, for example, a hypothetical site with a 1.8% conversion rate and annual revenues of $100 million. Assume that the site’s customer base is growing 25% annually as more customers come online. Even small increases in the conversion rate yield huge gains in revenue: • With a constant conversion rate of 1.8%, revenues will be $100 million this year, $125 million next year, and $156 million in two years. • With a (modest) 10% increase in conversion (raising the conversion rate to 2%) , revenues will be $110 million this year, $137.5 million next year, and $172 million in two years. • With a 30% increase in conversion (raising the conversion rate to 2.3%), revenues will be $130 million this year, $162.5 million next year, and $203 million in two years. • With a 50% increase in conversion (raising the conversion rate to 2.7%), revenues will be $150 million this year, $187.5 million next year, and $234 million in two years. Raising the conversion rate to 2.7% is a modest goal, far below the conversion rates of industry leaders. The impact, however, is tremendous: a difference of $50 million this year, $62.5 million next year, and $78 million in two years. Over a three-year
7 8 and BCG, The State of Online Retailing, April 2000. PC Data Online, 2000

_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 14

period, the cumulative incremental revenue is $190.5 million. Online retailers have a choice where to invest this holiday season. They can spend more money on advertisements and marketing campaigns while continuing to convert only 1.8% of shoppers. They can also devote months of development time and millions of dollars to create advanced recommendation engines and gift-wrap features that only a few of their customers will use. Or, they can invest in raising the conversion rate by improving the customer experience. Focusing on conversion has the greatest revenue impact for any e-commerce site.

The Three Most Important Improvements
The best investment a site can make to prepare for Holiday 2000 is to improve the customer experience. Based on our tests, here are the three most important improvements e-commerce sites should make before the holiday season arrives: • Improve the checkout process to keep sales from slipping away. • Develop a promotion and merchandising strategy to help holiday shoppers find gifts. This will raise the order size and conversion rate. • Commit to a long-term strategy for continually raising the conversion rate. The following chapters will elaborate on these improvements. We hope that this report will help the industry earn some of the $14 billion it now stands to lose due to poor customer experience.

_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 15



Testing Methodology

Creative Good conducted more than 50 consumer tests to generate the data for this report. The tests were conducted at an independent testing facility in a suburb outside New York City. We conducted the first round of tests in August 2000 and the BestBuy prototype tests in early September. We tested eight major e-commerce sites in four categories important to holiday shoppers: • Apparel:, • Books and Music:, (Barnes & Noble) • Electronics:, • Toys:, The consumers recruited for testing were representative of customers who will shop online this holiday season. The consumers we tested met the following criteria: • All had used the Internet for at least 6 months. • All had successfully made an online purchase in the past. • All had either bought gifts online last holiday season or were seriously considering buying gifts online for this year’s holidays. • None were Web developers, Web designers or Internet professionals.
_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 16

• None worked for any of the companies whose sites we tested. To simulate the actual shopping experience, we conducted three-quarters of the tests on dial-up connections and one-quarter of the tests on an ISDN line. We tested on PCs running Microsoft Windows and the Internet Explorer Web browser.

About “Listening Labs”
The Holiday 2000 tests employed Creative Good’s “listening lab” methodology. Though based primarily on traditional usability tests, listening labs are less taskoriented and more open-ended than usability tests. In particular, listening labs are one-on-one sessions in which the customer — not the moderator — sets the context. The labs are non-directed in an attempt to recreate the environment at home or work, where customers actually use the site — and where there are no pre-defined tasks or scripted moderators sitting beside them. Listening labs overcome some key shortcomings of usability and focus groups: • Listening labs are non-task-oriented, since pre-defined tasks tend to neglect what each individual customer wants to do on the site, and often miss larger strategic findings. In our Holiday 2000 listening labs, we merely asked customers to use the site to shop for a holiday gift for someone on their gift-buying list. • Listening labs rely less on quantitative measurement than some usability tests. For example, we did not use “eye-tracking” or any other detailed measurement during the tests. We did, however, quantify the most important metrics: how often, and where, customers succeeded and failed on the sites. • Focus groups, in which a scripted moderator asks questions to a group of customers sitting in the testing room, measure what customers say they like. Listening labs observe what customers do, not what they say they do. • Traditional usability and focus groups are controlled by a moderator, following a script and specifying the user’s task. Listening labs are controlled by the customer, following whatever path he or she wants through the site. For more information on the listening lab methodology, refer to Creative Good’s Dotcom Survival Guide (

_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 17

Timing of Screenshots
The screenshots featured in this report were taken in August and September 2000. Because sites are constantly changing, some of the sites may have redesigned since we took the screenshots. In fact, in most cases we hope that the site has redesigned by the time you read the report, so that the site improves the experience for its customers. The screenshots in the report depict what customers saw in our tests.

_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 18




In our tests, checkout was the most common cause for failure. To be exact, in all the tests in which a customer failed to complete an order online, checkout was responsible for the failure over 40% of the time. This is a huge failure rate for one feature — consistently poor performance across multiple sites in multiple categories. As discussed in the Business Case (see Chapter 2), the worldwide e-commerce industry could gain over $6.5 billion, including $3 billion in North America alone, if every site made simple improvements only to the checkout process. While checkout was by far the most common problem, customers in our tests did encounter other problems on the sites: • Some customers were unable to find what they were looking for (even though the site carried it) because of poor categorization and search. • Others failed to buy because they could not order the quantity of products they wanted. • Some were frustrated by slow-loading pages and decided to leave the site. Still, problems with checkout were the greatest cause of failure in the buying attempts we saw. Many times during the tests, we saw customers successfully find a product, decide to buy it and successfully add it to their cart — all in vain. Checkout was too difficult to complete. Frustrated, many customers gave up and left the site. To use an off-line metaphor, it was as if these customers made it to the cash register but the store would not let them pay.
_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 19

Failures during checkout were due to three main causes: • Confusion over “Return” and “New” customer paths • Difficulty creating an account • Ineffective error messages In all these cases, sites failed to make it clear what customers needed to do and how they could advance in the checkout process. Shoppers failed to buy because the customer experience was too difficult. We show examples below and propose solutions to the problems we saw.

“Return” vs. “New” Customer Paths
Several of the sites we tested offered separate checkout paths for new and return customers. Sites offer separate paths to make checkout easier for return shoppers (so that on the second visit, for example, return shoppers don’t have to type in all of their information again). The problem is that separate checkout paths tend to confuse customers. In our tests, many new customers mistakenly selected the checkout path for return customers. Below is the first checkout screen on

New customers immediately began the “Registered Customer” checkout.

Most customers ignored all the text instructions and immediately entered their email
_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 20

address in the field under the “Registered Customers” header, possibly because (scanning the page from left to right) it was the first field customers noticed. Some were then confused as to what password BestBuy wanted, and several entered the password they use to check email. The BestBuy site then displayed an error message that only added to their confusion. also offers two options at the start of checkout:

First-time shoppers mistakenly filled in these fields.

New customers were drawn to the empty email and password fields and filled them out, even though the text directly above reads “I am a registered user.” Customers then got an error message because, of course, there was no account with that email and password. (Some readers might notice that the “new user” radio button is selected; on, the filled-out fields overrode that setting and the server considered the customer a registered user.) Online shoppers rarely read instructional text; instead, they look for an action to take that will bring them closer to their goal. In this case, shoppers saw the empty fields and acted on them. In general, we found that two checkout paths were problematic when both paths were prominently displayed to the customer. Shoppers are confused by sites that offer two equally prominent checkout paths.

Solutions from B&N and Amazon

Some sites, however, were able to offer separate checkout paths without confusing shoppers. The websites of both Barnes & Noble and Amazon effectively served both new and return customers by making only the new customer path prominent.
_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 21

Every new shopper successfully began checkout.

On Barnes & Noble, no new shopper clicked on the “sign in” link for return customers. Notice that the “Checkout Now” button, which is the beginning of the new customer checkout path, is much more prominent. Barnes & Noble made the right decision by making the new customer checkout path more prominent than that of return customers. (This particularly makes sense for return customers, since by definition they are more familiar with the site and more likely to see a less prominent link.)

Amazon’s email field applies to both new and return customers.

In our tests, every shopper successfully checked out at Amazon. As with other sites we tested, customers filled in their email address right away. But in this case, filling in the empty field was the correct way to begin checkout. The Amazon form is successful for several reasons:
_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 22

• The email address field is first, motivating customers to (correctly) begin filling out the form. • The radio buttons, to choose between new and returning customers, appear between the email and password forms, distracting customers to read the radio button text before immediately filling out the next field. • The radio buttons’ text is well-worded. They mention “new customer” and “returning customer,” much clearer than Gap’s “new or unregistered user” and “registered user” fields. • The “new customer” option states clearly that “You’ll create a password later,” helping new customers avoid mistakenly typing their email password into the password field. • The password field is clearly described as being only for the “returning customer.” • The checkout form changes slightly for a return customer (see below). Amazon uses “cookies” to recognize its return customers9 and present them with a slightly modified checkout page. Here’s how the first page of checkout looks to a return customer:

The email field is filled in for return customers.

The page defaults to the return customer option if the cookie is set.

For return customers, Amazon has already filled in the first field with the email address, and the radio button defaults to the “returning customer” option. All the return customer has to do is complete the only empty field — the password — and click “Sign in” to continue. While Amazon does many things right with the beginning of its checkout, the most important takeaway is that the page is designed primarily to help new customers get

For more information on cookies, see

_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 23

through the page correctly. Of secondary importance is that return customers are served by the clever cookie-based change. Design checkout for new customers. Use cookies to remember return customers and present them with a different checkout path.

Creating an Account
Several sites confused customers by making parts of their sign-in or registration process too prominent to new customers. For example, some sites required customers to register before they could make a purchase. The registration process prevented many customers from buying the products they had added to the shopping cart. In’s case, the site confused customers by reminding them that they had not yet signed in to their account. The home page, shown below, contains this text at the top of the page: “You are not signed in.”

Customers were perplexed when told they were “not signed in.”

Several shoppers noticed this text because it loaded first. Because the rest of the page, consisting mostly of graphics, loaded slowly over the dialup modem, “You are not signed in” appeared by itself for several seconds while the other page elements loaded. While no customers actually left the site because of the text, it certainly didn’t enhance the customer experience, as shown in these comments: • One shopper on said the message “means I can’t get access to the site.” • Another commented that when the message “pops up, I think the page isn’t going to show up.” Another problem occurs on’s homepage, which urges customers to “create an account to get started”:
_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 24 urges customers to register before they even view products. seems more concerned with pushing registration than with showing customers products. Why would a shopper sign up before knowing what the site offers? As on, it’s not a good idea to distract new customers with prominent reminders about accounts and signing in. A much worse problem on occurs when a new customer finishes adding products to the shopping cart and clicks “Check Out”: the site immediately requires new customers to create an account before they can proceed. New customers, credit card in hand, hardly want to “create an account” just for the privilege of giving the site their money. They want to proceed in the checkout process. To make matters even worse,’s first Create My Account page includes “Account Type,” a required field:

Shoppers cannot buy without stating what type of user they are.

_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 25 should remove the Account Type field, since there is no benefit to customers by completing the field. And there is no benefit to, either; if customers get confused by the field or otherwise leave Account Type incomplete, the website prevents them from going any further in the checkout process. (At this point on, one exasperated shopper said, “Where do you just buy it?”) Barnes & Noble also includes in its checkout process a required field that shouldn’t be there. The checkout page below asks customers, “For future reference, please name this address”. Unlike the Account Type field, the “name this address” field could be helpful to shoppers on a return visit to But like the Account Type field, there’s no reason for it to be a required field. Again, any customer who is confused by the field and leaves it empty is prevented from going further in the checkout process.

On, customers need to “name” their address.

In tests, the idea of naming an address was unclear to customers, despite Barnes & Noble’s example (“e.g. Home”). One customer asked, “Please name this address? What does that mean?” confused several customers by asking them to create a password hint. This was not a required field, but it did confuse customers with its lengthy instructions:
Select a Password Hint (optional) Enter a personal question that will help you remember your password (e.g. What is my mother's maiden name?). If you forget your password, we'll ask you your personal question. [form] Enter the answer to your personal question, which should not be your password. [form]

_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 26

Customers had difficulty understanding BestBuy’s “personal question.”

Here’s how the process is supposed to work: • Customers select a personal question to be asked in case they forget their password, and tell BestBuy the answer to the question. • The answer to that question should be something other than their BestBuy password. • If a return customer clicks “Forgot your password? Get help here”, the site will ask him his personal question. A correct answer will permit the customer to log on. Some customers simply did not understand what BestBuy was asking of them. One said, “I am thinking maybe they mean to put in a fake password.”

Solution: “passive registration”

The problems above show that new customers can get confused if certain parts of the registration process is too prominent during checkout. avoids this problem by “registering” new customers automatically when they make their first purchase. Amazon quietly creates the account during the regular checkout, without explicitly asking customers to create an account. We call this process “passive registration.” For new customers, Amazon’s first-time checkout process involves the following steps: • Enter an email address • Enter a shipping address
_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 27

• Choose a shipping method • Choose a password • Enter a credit card number • Confirm a billing address • Review order and submit The Amazon checkout process is successful for several reasons: • Nowhere in the Amazon checkout process are new customers required to “create an account,” choose an “account type,” or even asked to choose a “personal question.” • The process is front-loaded with steps that make sense: entering an e-mail address (as discussed above) and the shipping address and method. • Only in the fourth step are new customers asked to choose a password. By that time customers are halfway through with the checkout process and less likely to abandon the site. Overall, Amazon gives customers the benefit of an account — an easier checkout process on the return visit — without subjecting customers to unnecessary obstacles in creating the account. This focus on the customer experience makes Amazon’s checkout a success.

Checkout Error Messages
Some customers will inevitably make mistakes entering information no matter how easy checkout is. In these cases, it’s important to have clear, simple, and polite error messages to help customers proceed. In our tests, we observed how customers reacted to several different error messages in checkout. The quality of the error message often meant the difference between a successful purchase and a failed attempt. Barnes & Noble customers see the following error message if they fail to complete the “name this address” field:

_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 28

Shoppers did not realize this message applied to the bottom field.

The error message, which reads “Please name this address for future reference,” appears on the top of the page, far away from the field it refers to. The “name this address” field is halfway down the page with no indication to the customer that it needs to be filled in. Unsurprisingly, customers did not find this error message very helpful.

Problems with email address entry

Many shoppers, especially AOL users, had difficulty entering their email address when creating an account during checkout. Some entered a username (e.g. JohnDoe) but no domain name. Others entered only part of the domain name (JohnDoe@aol) and were unable to buy because they did not understand to enter their full email address ( On the sites we tested, error messages did not always help customers with email problems. For example, shows the page below if a shopper’s email address is invalid:

_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 29 asks for a “valid email” but provides no example to help shoppers.’s error message reads, “Please provide a valid email address.” It tells customers they’ve made a mistake but does not show them how to fix it. Customers may try again, only to be baffled when their address is repeatedly rejected. should include an example of a complete, valid email address to help customers through checkout. Throughout all our tests, over 10% of customers were never able to correctly enter their e-mail address on the site they were on, despite repeated tries. Note what happens if a new BestBuy customer chooses the return customer checkout path:

New customers think there’s a problem with their email , unaware that they’ve taken the wrong checkout path.

BestBuy’s error message reads, “The email address and password you entered do not match” — suggesting that the shopper entered the email or password incorrectly. The
_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 30

real problem, however, is that as a new customer, the customer has no password at all on the system and should have taken the new customer checkout path instead.

Solution: clear, simple, polite error messages

eToys, by contrast, helps customers succeed with its clear and prominent checkout error messages. For example, below eToys asks customers to “Please enter the credit card’s month of expiration.”

All shoppers noticed and understood eToys’ error messages.

eToys’ error message succeeds for three reasons: • The error message is written politely: “Oops! Before we can go on, please see if you can fix this...” • It’s displayed clearly : The message is displayed in bright red text, beside a large graphic, to indicate that an error occurred. (clearly displayed) • It’s worded simply: The message clearly explains how to fix the error and continue to make a purchase. (worded simply) eToys could improve the message further by moving the credit card number field closer to the error message, making it even easier for customers to correct the mistake and proceed in checkout. KBKids also displays its checkout error messages clearly, helping customers see where they need to make a correction:
_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 31

KBKids uses bright red, blinking triangles to indicate which fields need to be filled in.

KBKids places blinking red triangles next to the fields that customers need to fill in. While this practice would be annoying on practically any other page on the site, it’s helpful in checkout, since it’s then especially easy for customers to see where to fix the problem and move on. Amazon’s error messages go a step further to explain its checkout process:

Amazon tells shoppers why it needs their phone number.

When customers leave a required field blank, Amazon explains says why the information is necessary. Doing so helps shoppers through the checkout process, and in this case — explaining the need for a phone number — reflects sensitivity to customers’ privacy concerns.

_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 32

What to Learn About Checkout
• In our tests, over 40% of failures in buying attempts were due to checkout. • Fixing checkout alone would save the industry billions of dollars in lost sales and raise conversion rates by 20%. • Shoppers are confused by sites that offer two equally prominent checkout paths. • Design checkout for new customers. Use cookies to remember return customers and present them with a different checkout path. • Do not require shoppers to register before they can buy. • If a form is incomplete, prominently highlight for the customer whatever needs to be changed, and word the error message simply and politely.

_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 33



Promotions and Merchandising

Promotions and merchandising are opportunities for retailers to boost revenues during the holiday season. Customers in our tests were highly responsive to promotions and featured products, especially so because they, like most holiday shoppers, were shopping for a gift for someone else: • In 30% of tests, customers clicked on at least one featured product. • In 19% of tests, customers clicked on a promotion or special offer. • Some promotions were so effective that 50% of customers who saw the promotion clicked on it. Promotions and featured products appeal to holiday shoppers. Effective promotions and merchandising can generate substantial revenue gains. For many retailers, effective promotions and merchandising may mean the difference between profitability and continued losses. According to and BCG: • The average conversion rate for apparel sites is 1.8%. • The average order size for apparel sites is $81.10 On an average apparel site, increasing the average order size by 10% would add $8 per purchase. Increasing the conversion rate by 10% (a modest increase) would increase conversion to nearly 2%. For a site with $100 million in sales, the impact of
10 and BCG, The State of Online Retailing, April 2000

_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 34

these two changes would amount to $21 million in incremental revenue. $100 million x 1.1 (due to increased order size) x 1.1 (due to increased conversion) = $121 million. If the overall e-commerce industry raised average order size by a conservative 10% and raised conversion rates by a modest 10%, the increase in holiday revenue would total over $4 billion. $19.5 billion x 1.1 (due to increased order size) x 1.1 (due to increased conversion) = $23.6 billion Effective promotions and merchandising can significantly raise holiday revenues for many e-commerce sites. In this section, we present promotion and merchandising tactics that our tests showed to be most effective. We also describe several lost opportunities — how sites may be losing sales by hiding good offers or by making them hard to use.

Effective Promotions
We discovered that promotions, like other areas of e-commerce sites, are more effective when they are presented clearly and are easy for customers to use. (For the purposes of this report, we define “promotions” as monetary offers or discounts that are not tied to specific products — for example, offering 10% discount off a customer’s first purchase.) The most effective promotion we saw was on the homepage of, and it enjoyed a 50% clickthrough rate in tests:

_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 35

Half of all test customers clicked on this homepage promotion.

Multiple customers clicked immediately on GapMaternity.’s prominent and straightforward promotion, “$100 gets you $20,” appealed strongly to shoppers. Consumers immediately understood the premise of the promotion — a $100 purchase would “get a $20 Gap ShopCard good towards your next purchase.” One shopper said, “I would do this. A hundred dollars gets you twenty. I would definitely check that out.” Another noted that it would be “easy to spend a hundred dollars” shopping at the Gap, and thus the promotion was attractive to her. Half of the customers who saw the promotion clicked to see the details. Another element on’s homepage showed good merchandising. Multiple shoppers immediately clicked on the large GapMaternity graphic. By prominently displaying this section of the store, communicated its breadth of product offering to customers who might not have known otherwise. One shopper remarked, “I would never even have thought that they had maternity clothes. This is great.” Although the “$100 gets you $20” promotion drew a high clickthrough rate, more than one customer had trouble taking advantage of the promotion. The page took so long to load that one customer left the site entirely. Frustrated, she said “If it’s going to take so long, what’s the point? It’s just a special offer.” Another customer commented that it might be easier to drive to a nearby Gap store than to go through the online promotion.

_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 36

One customer left because the promotion page loaded too slowly.

From studying, we offer two lessons about promotions: • Display promotions prominently and explain them clearly. • When customers click into the promotion, make it quick and easy to take part in the promotion. Admittedly, these are basic, common-sense guidelines — but as evidenced in our tests, many top e-commerce sites aren’t following them. Just following these simple rules could improve revenues substantially.

Good promotions, poorly presented offered a good promotion but missed the opportunity to present it to customers:

_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 37

LandsEnd doesn’t mention free hemming until customers begin the buying process.

During the checkout process above, LandsEnd asked customers if they wanted the wool pants to be hemmed — at no extra charge. Customers were happy to learn of the offer of free hemming, but multiple customers commented that they would have liked to know about the offer earlier in the shopping process. LandsEnd does not promote its free hemming anywhere else, such as on the product page, and it may be losing potential sales by not making the free hemming offer more prominent. Similarly, offers free shipping on office supply orders over $50 — but doesn’t explicitly tell customers about the offer: customers were surprised to see shipping was free.

Just as with LandsEnd, customers on said that they would have liked the site to mention the free shipping earlier in the buying process. makes no mention of the offer on its office supply product pages, and it doesn’t remind customers of the offer when their order is close to $50. The only other way that
_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 38

customers can find out about the offer is in the shipping section of’s Frequently Asked Questions — not the best place for promotional announcements. Strangely enough, even when customers are in checkout and their order exceeds $50, they still are not told of the offer. They can only assume that shipping is free because the shipping dropdown defaults to “Standard Delivery $0.00” (but it doesn’t mention the $50 minimum). One shopper on said, “If there’s no shipping charge, hey great... tell us that up front on the site.” Communicate special offers clearly and early in the buying process.

By “merchandised” products, we mean featured products and other items presented in a special manner distinct from the site’s standard product listings. Many customers appreciated seeing featured products on sites’ home or category pages. In 30% of tests, customers clicked on at least one featured product, showing the importance of merchandising. LandsEnd, for example, featured two products on its Women’s category page, causing a third of customers to click to get more information on one of the products:

_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 39

LandsEnd’s category page prominently displays two featured products.

Once again these highlighted products caught customers’ attention because they were presented prominently and clearly. The interested customers considered buying these items even if they came to the site for something else. BestBuy, by contrast, buries its featured products on a homepage cluttered with graphics and links to pages without products:

_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 40

The center of BestBuy’s homepage is devoted to a sweepstakes. None of the links in this column lead to products.

Featured products only appear near the bottom of the page.

BestBuy devotes most of its above-fold homepage space to a sweepstakes, which doesn’t lead to any products. All of the featured products appear below the fold at 800 x 600 browser resolution. While there’s nothing wrong with promoting sweepstakes, our tests suggest that holiday shoppers are more interested in getting to products that they can buy. Place featured items above the fold on the homepage. Featured products were particularly important on the toy sites we tested (eToys and KBKids). Adult customers, shopping for children, were especially eager to get gift suggestions. This was particularly true when customers shopped for other people’s children and had little idea of the children’s preferences.

_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 41

Notice how KBKids effectively uses its homepage to display featured products:

KBKids’ homepage features products from various categories.

The “Surprise of the Day” is a discounted product.

KBKids shows products from several categories alongside links to the entire category. The site also displays a “Surprise of the Day” in the left-hand column (though near the bottom of the page), a featured product at a discounted price. Both KBKids and eToys let customers shop by age as well as by category. In our tests, almost all customers shopped both by age and by category within a single session. Providing multiple ways to browse was helpful to shoppers eager to find ideas for gifts to buy.

_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 42

eToys customers who shop by age for a five-year-old see this well-rounded page:

The center of the page provides category recommendations.

The right-hand column features characters and brands.

The page offers customers over a dozen product or brand suggestions. The suggestions range from brands (Lego and Barbie) to categories (“books they can handle”) to situations (“Plan a Stress-free Playdate”). Because the product suggestions were prominently and clearly displayed, several customers in tests clicked into various products from this page. eToys’ shop-by-age page is an example of thoughtful and varied merchandising. Use brands, categories and situational merchandising to suggest products to customers.

_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 43

What to Learn About Promotions and Merchandising
• Promotions and featured products appeal to holiday shoppers. • Effective promotions and merchandising can significantly raise holiday revenues for many e-commerce sites. • Communicate special offers clearly and early in the buying process. • Place featured items above the fold on the homepage. • Use brands, categories and situational merchandising to suggest products to customers.

_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 44



Case Study: BestBuy

The BestBuy case study shows the benefits of improving the customer experience. After testing the BestBuy site with customers, we applied the Creative Good consulting methodology to prototype our own “new” customer experience for When we tested the prototype, the results were dramatic: shoppers were four times more successful on the prototype than they were on the original site.

Why a Case Study?
In our tests of eight top e-commerce sites we observed that customers failed to complete an order over 40% of the time. Ironically, this large failure rate was caused by a handful of problems in the customer experience of the sites we tested. We hypothesized that fixing that small set of problems could yield significant gains for any e-commerce site. To test our hypothesis, we chose one of the eight sites we had tested originally — — and created a “new” version of the site. “New” is in quotes because (a) we created and tested new versions of only a few pages from the site, and (b) BestBuy wasn’t aware of the case study process and therefore wasn’t a participant.11 We significantly improved the BestBuy customer experience with a relatively small

Copyrights, trademarks and all proprietary rights included on or comprising the website are property of™ Inc., a wholly owned subsidiary of Best Buy Co. Inc. Use of these materials by Creative Good, in connection with this Case Study, is for illustrative and analysis purposes only. The trademark “™” is property of Inc.

_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 45

investment in time and effort. Instead of prototyping the entire site, we just improved those areas of the site that confused customers in the first set of tests. Many e-commerce sites also could improve significantly with a small investment in the customer experience. This case study will show the following: • Where customers had difficulty on during the initial tests (on the live website) • How we designed our prototype to address those problems • How customers reacted to our prototype during the tests of the prototype

Summary of Results
The new BestBuy prototype shown in this chapter represents a sizable improvement in the BestBuy customer experience, as shown in before-and-after tests: • In our first set of tests (that is, the tests on the live website), over 70% of customers failed to complete a purchase. • In the second set of tests (the tests of the prototype), over 80% of customers succeeded in completing a purchase. The BestBuy conversion rate quadrupled between the first and second set of tests. • Also, three times as many customers clicked on featured products in the prototype as they did on the live site. Customers in both tests met the same criteria, got the same instruction (to shop for a holiday gift), and of course, no customer appeared in both sets of tests.

Developing the Prototype
To develop the BestBuy prototype, Creative Good first created “wireframes,” block diagrams with the location and description of each page element. Below is an example of a wireframe that we created; notice the descriptions of Customer Goals, Prominence, and Notes on the right side of the page.

_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 46

The wireframe shows the position of page elements but not their graphic design treatment.

The wireframe explicitly lists the customer goals on the page and the relative prominence of page elements.

It took three days for Creative Good to create wireframes for eleven new BestBuy pages. Once they were done, interactive agency Red98 (a Boston-based interactive agency committed to creating “great user experiences”) turned the wireframes into the working HTML prototype that Creative Good used in the second set of tests. It took one week for Red98 to create the HTML prototype. Thus, in less than two weeks, we transformed the customer experience on BestBuy — at least in prototype form. An actual e-commerce site would take longer to implement and integrate such changes, but even doubling the time period is still a tiny investment of time to get higher revenues in return: With a few simple, fast changes to the customer experience, many e-commerce sites can substantially increase their revenues. We designed the new BestBuy prototype with new customers in mind, rather than on return customers. More specifically, the prototype design was guided by four pieces of data: • Customer objectives, as observed in our initial tests • BestBuy’s business objectives, based on some research on the company (see below) and some basic assumptions about the objectives of any e-commerce site
_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 47

• Best practices of online retailing, based on past Creative Good research and client work • An informal competitive analysis of five other e-commerce sites that sell electronics

BestBuy’s business objectives

As stated above, we conducted some informal research on BestBuy to better inform the decisions we made in creating the prototype. From reading BestBuy’s public statements, we conclude the following about its Web strategy: • BestBuy is investing heavily in its website. • BestBuy wants customers to buy from its website. • BestBuy values customer service and explicitly tries to focus on customers. Best Buy, the leading off-line electronics retailer, has 360 stores spread across 40 states.12 The company’s projected revenues for the year 2000 are over $1.2 billion, an increase of nearly 25% over 1999.13 Best Buy is successful and growing rapidly. In May 2000, (a wholly-owned subsidiary company of Best Buy Co., Inc.) announced it was spending $50 million on an online and off-line marketing initiative.14 is investing heavily to drive traffic to the site. As part of a multi-channel strategy, the site aims to “give customers access to all three of Best Buy’s shopping channels: in-store, online and catalog.”15 Shoppers can buy products from the website and pick them up at a BestBuy store. Consumers can also browse the off-line store and place their order through the website.’s senior management has cited customer focus as a top concern. Company president John Walden said “Best Buy seeks to pioneer the clicks-andmortar revolution by creating a better shopping experience for our customers at every point we interact with them.”16 BestBuy recognizes that customers drive any company’s success.
12 13 Twice magazine: Twice 2000 Retail Registry 14 press release, May 1, 2000. 15 press release, June 12, 2000. 16 press release, June 12, 2000.
_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 48

We drew several conclusions from this information: • We concluded that BestBuy primarily wants to sell products on the site (hence the prototype focused on making it easier to buy products). • We confirmed that store pick-up was important (hence the prototype featured store pick-up on a homepage banner). • We knew that since BestBuy has a strategic commitment to the performance of its website, and tries to focus on the customer, it would reasonably want to make such customer experience improvements as we made in the prototype.

Comparing BestBuy and the Prototype
Below we describe the problems customers had on in the first set of tests, and how our prototype attempted to fix those problems. To help keep the two sites straight, “BestBuy” or “” refers to the live site; “the prototype” refers to the prototype that we created and used in the second set of tests. Homepage

For some shoppers, problems with began on the homepage:

Shoppers must wait as the background image — a man’s face — slowly loads. One customer wanted a DVD movie but did not see the Movies link. At 800 x 600 browser resolution, there are no featured products visible above the fold.

Instead of a plain background, uses the faded image of a man’s face, thereby lengthening the load time of the homepage without helping customers shop.
_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 49

When the homepage does finish loading, there is nothing visible above the fold that customers can buy. The most prominent element of this page is the home entertainment sweepstakes. One shopper came to the site looking for a movie on DVD. She was unable to find the Movie category link, since it was located below the main list of departments and separated from them by a white line.

Prototype: Homepage

In our prototype homepage, we tried to make it easier for customers to begin shopping right away:

On the prototype, all categories were presented together.

Almost every customer clicked on a featured product.

The prototype highlighted the product categories and listed them in blue underlined text links. In tests, customers more easily saw the range of products the site offered. We also renamed and created categories so shoppers could find items more easily. For example, we added links to “DVD Players” and “Camcorders” categories, so that customers could find popular products right away. (The site currently groups these products under the “Home Audio and Video” category.) The prototype homepage also displayed four featured products. One note on the prototype’s revised product categories: the categorization scheme isn’t perfect. It was merely based on what we heard from customers in our tests and
_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 50

on our informal review of competitor websites. In an actual client project, Creative Good would also use site data and the client’s expertise to create a more accurate set of categories.

BestBuy: Results Page

Aware that BestBuy sells DVDs in its off-line stores, one customer searched by “DVD” from the BestBuy homepage and received the following results page:

Finding no DVD movies above the fold, one shopper thought did not sell any DVDs. presented its search results by category. In this case, results from the Movie category appeared below the fold. The customer assumed did not sell DVD movies and thus gave up trying to buy one.’s search was, however, helpful for some customers. Note what the site returns if a shopper searches by “cell phones.” Shown below, effectively maps a “cell phones” search to a page explaining that wireless products are not yet sold on the site. Further, the page includes a prominent link to the store locator. Shoppers can thus easily find a BestBuy store that sells what they are looking for.

_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 51

Shoppers who search by “cell phones” see that wireless products are not yet offered. How do home office products relate to a search by “cell phones”?

BestBuy: Category Page

On the BestBuy category page (see below), several customers were disappointed by what they perceived to be a limited product selection. does offer a fairly broad selection, but the site’s categorization gave shoppers an impression of a narrow selection. One problem was that the site’s category pages featured very few products and made subcategories hard to find:
Links to product listings are hidden in the left- hand navigation bar.

Only two images of camcorders are visible on the camcorder page.

Customers had difficulty, for example, realizing that BestBuy carried more than two
_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 52

camcorders, since only two camcorders are displayed on the Camcorders page (shown above). Other camcorders were available through the subcategory links in the left-hand navigation, but customers didn’t know to go to the navigation elements to find more products. Customers familiar with the broad selection in BestBuy’s off-line stores were disappointed. One customer commented: “Not a big selection. It would be nice if they had more selection on here... When you go to the store, there is more selection.” What the customer didn’t realize is that offered far more products than she could see.

Prototype: Category Page

Our prototype made featured products and subcategory links much more prominent:

Subcategories were highlighted at the top of a contextual navigation bar. Subcategory links were also displayed in the center of the page.

At 800 x 600 browser resolution, four products are visible above the fold.

If customers wanted more camcorders than the four displayed on the page, the links to subcategories (containing more camcorders) were located prominently in the top center of the page.

BestBuy: Product List’s product lists — the standard way that BestBuy displays its products within their subcategories — may frustrate customers by failing to show items’ prices:
_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 53

For some items, customers must click on See our Price.. .

Inexplicably, BestBuy’s product list doesn’t display prices for each product. In the screenshot above, the top three products show crossed-out prices, presumably because they are discounted. Customers who want to know the price of those items might reasonably click on the “See Our Price” link — but even that doesn’t display the price. Instead, customers get a paragraph of text saying that customers must first click the Buy button in order to see the price.

... and find they must add to cart to really see the price.

Clearly, this is a poor customer experience. How many customers will begin buying a product without knowing its price? It’s particularly surprising that BestBuy would make some prices hard to find, considering how expensive some of the products are. One customer commented that she would compare prices between different retailers: “A big purchase like that I would not impulse buy... I would have printed [the page] out to compare.”

_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 54

Prototype: Product List

The prototype displayed a price whenever a product was displayed. Here is the prototype’s Digital Camcorders page:

The price of every item is clearly visible.

Customers on the prototype could clearly see the price of each item. This also made it easier to compare the prices of listed items.

BestBuy: Begin Checkout

On, several new customers mistakenly chose the checkout path for return customers (a common problem in many sites’ checkout, as described in Chapter 4):

_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 55

New customers were drawn to the empty email and password fields.

Customers naturally gravitated toward the empty email and password fields on the left-hand side of the page, without reading the “Registered Customers” header above them. After clicking “Continue Secure Checkout,” those customers saw an error message, since they were new customers and didn’t have a BestBuy password.

Prototype: Begin Checkout

The protoype’s checkout process focused on the new customer checkout path, as suggested in Chapter 4:

The prototype leads new customers down the appropriate path.

Return customers can click into a shorter checkout.

To the right of the default checkout path is a small text link allowing return customers to enter into a shorter checkout. Since return customers are more familiar with the
_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 56

site, it makes sense to provide them with a less prominent entry to checkout. Every customer who began checkout on the prototype successfully completed the transaction. An improvement to the prototype, had we had the means to implement it, would have been to use cookies to identify return customers. Then, when they began checkout, the site would know to place them automatically in the shorter checkout path for return customers.

A Prototype Weakness

Even though our prototype performed much better than the live BestBuy site, the customer tests of the prototype revealed an improvement we could have made. In one test, the prototype’s shortcoming prevented a customer from being able to buy from the product page:

One shopper missed the Buy button and instead clicked Shopping Cart to buy.

One customer tried to add a camcorder to his cart by clicking Shopping Cart instead of Buy. While we intended to make the Buy button prominent, the customer’s difficulty showed us that we had not made the Buy button prominent enough. In an actual client project, it’s common — even desirable — to find problems in the prototype during listening labs. By continually improving the prototype based on customer feedback, the site in development becomes that much more effective.
_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 57

Rough Calculation of Results
When we tested the current BestBuy site, only 22% of customers who wanted to buy were able to do so. When we tested our prototype of the BestBuy site, 88% of buying attempts succeeded. If our case study is anywhere near accurate, these findings suggest that’s sales could be several times what they are today if simple improvements were made to the customer experience on the site. We made a rough calculation of the revenue gains that such improvements would bring to BestBuy: Although we don’t know’s current conversion rate, the average conversion rate for electronics sites is 0.8%17 . Assuming BestBuy’s conversion rate to be the average, quadrupling it (a 300% increase) would raise conversion to 3.2%. But to keep our revenue estimate conservative, we assume an increase of half that size — 150% — raising conversion to 2.0%. This also seems like a modest enough goal, since 2% is still less than the average conversion rate on toy sites (2.7%), on books and music sites (2.1%), and just above the overall e-commerce average of 1.8%. Since we don’t know’s annual sales, we assume a modest estimate of $100 million. This figure is less than 10% of its off-line sales ($1.2 billion) and only twice the amount that is spending to market the site ($50 million). If’s annual sales are $100 million, raising conversion 150% would increase sales to $250 million. $100M x 2.5 (due to increase in conversion rate) = $250 million Given our assumptions, simple improvements to BestBuy’s customer experience could bring in $150 million more in annual revenue than the site makes today. This is an enormous revenue gain, translating to several million extra dollars per month. Any investment in improving the customer experience would pay for itself in a matter of days, with hundreds of thousands of dollars in daily incremental revenue. It’s hard to see why a site would invest in anything other than the customer experience when improving the experience can have such profound impact on the business.

17 and BCG, The State of Online Retailing, April 2000.

_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 58

Again, given our assumptions, two other results are interesting to note: • With $250 million in online sales, BestBuy’s website would generate revenue equal to about 20% of its off-line revenues. The change would bring BestBuy closer to its goal of “pioneering the clicks-and-mortar revolution.” • Increased revenues would also dramatically raise the effectiveness of BestBuy’s marketing campaign. If now grosses $2 for every dollar it spent marketing the site, a better experience could bring in $5 per dollar spent on marketing. Improved marketing efficiency would justify further investment in BestBuy’s customer experience. The lesson of our BestBuy case study is clear: it pays to invest in the customer experience.

What to Learn from BestBuy
While the BestBuy case study provides valuable tactical lessons, the most important lessons are strategic: • Even well-intentioned companies, committed to focusing to the customer, can greatly improve their online customer experience. • With a few simple, fast changes to the customer experience, many e-commerce sites can substantially increase their revenues. • When testing a website in front of customers, do it in a non-directive way. Instead of directing customers, watch and listen to how they experience the site. • When creating prototypes, keep in mind the business objectives while applying the lessons from customer tests. • Test the prototype in front of customers to assess the changes and identify new problems. • It pays to invest in the customer experience.

_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 59

Join our free newsletter
If you found this report useful, you’ll like our Good Experience Update, a free e-mail newsletter that goes out every week to more than 30,000 subscribers worldwide. Written by Creative Good founder Mark Hurst, Good Experience contains news and commentary about customer experience in e-commerce, wireless, and other Internet services. To sign up, just send an e-mail to

_____________________________________________________________________________________________ Copyright 2000, Creative Good, Inc. More free resources: 60