White Paper

Measuring and Tracking Customer Satisfaction

You know why. We show you how. By Michael Conklin, Chief Methodologist MarketTools, Inc.

White Paper

Measuring and Tracking Customer Satisfaction

About the Author

Michael Conklin is MarketTools Chief Methodologist. He is a marketing research veteran and analytical methodology pioneer with 24 years experience. He came to MarketTools after serving for 10 years as a Senior Vice President and Team Leader of Analytic Services for GfK Custom Research Inc. in Minneapolis. He is a frequent speaker at the American Marketing Association's Annual Advanced Research Techniques Forum as well as at the American Statistical Association's Joint Statistical Meetings conference . In addition, Conklin has been published in key marketing publications, such as Marketing News and Marketing Research Magazine, as well as numerous statistical journals. He received his education at The Wharton School.

About Zoomerang
Zoomerang ( www.zoomerang.com ) o ers an easy way for businesses to get fast answers to important questions. With more than 100 million surveys sent and counting, Zoomerang has grown rapidly and has been adopted worldwide. The number of subscribers using Zoomerang surveys has doubled every year since 1999, and Zoomerang has been featured in Fortune magazine, on NBC News, and on the NBC Today show. Zoomerang is part of the MarketTools family, a leading provider of technology and advisory services for conducting Web-based market research.

For Subscribers only

This White Paper is provided to our Zoomerang zPro subscribers as an added bene t to the zPro survey service. Zoomerang provides best practices and research methodology that our subscribers nd useful to increase both the quality and impact of their surveys.

White Paper

Measuring and Tracking Customer Satisfaction

Best Practices in Customer Satisfaction Research

How satisfied are your customers? Are they more or less pleased with your company than they were last year? How does your customer satisfaction level compare with that of your competitors? What impact does it have on your companys profits? How do you know? Guesswork and hunches wont cut it. You need hard data. You need to measure and monitor customer satisfactionconsistently and regularly. You need a customer satisfaction research strategy and program. You can do it yourself, but there are a few things you need to consider. This paper gives you the benefit of the experience and lessons learned by industry experts to help you design, distribute, and analyze your customer satisfaction research. We want you to act on insight, not instinct.

Why Measure Customer Satisfaction?

Customer satisfaction is tied directly to profitability. If your customers are happy, they tend to be loyal. And if theyre loyal they not only buy more, they refer other customers. Well-established research by Bain & Company found that, for many companies, an increase of 5% in customer retention can increase profits by 25% to 95%. The same study found that it costs six to seven times more to gain a new customer than to keep an existing one.

$80 $70
Cost Per Customer

$60 $50 $40 $30 $20 $10 $0

Existing Customer New Customer

"It's six to seven times more expensive to gain a new customer than it is to retain an existing customer." "A 5% increase in customer retention can increase profits by 25% to 95%."
(Source: Bain & Company study in Harvard Business Review, 2001)

Customer Type

Moreover, one bad experience can outweigh a whole lot of good experiences. Because of e-mail and instant messaging, that bad experience can quickly be broadcast to dozens, hundreds, or thousands of other customers, magnifying its impact. So if your business is doing something that frustrates customers, you need to know right away. It is critical to give customers the opportunity to provide feedback about their overall satisfaction level and specific likes and dislikes. It is equally important to consistently measure and monitor that input. Without an effective customer satisfaction research program in place, your company will be losing business, missing opportunities, and putting itself at a competitive disadvantage. 3

White Paper

Measuring and Tracking Customer Satisfaction

Customer Satisfaction Surveys: Tips and Practical Advice

The first steps in establishing a customer satisfaction program are determining when to distribute your survey, how many customers will be invited to respond, and how to deliver the survey to them. Well take a closer look at how to formulate your survey questions in the next section.

1. When to Conduct Customer Satisfaction Research

Many managers wonder how frequently they should conduct customer satisfaction surveys. The answer depends on the size of the customer base and the purpose of the research. There are two key types of surveys, and they serve very different purposes: > Transactional surveys solicit feedback directly from the product or service user about that particular encounter. They are conducted immediately after each customer transaction. For example, a survey may be administered after a call center experience. > Relationship surveys collect input from people who have an ongoing relationship with the company and have had multiple transactions. They are regularly scheduled surveysoften quarterly. The respondents typically are responsible for deciding whether to continue the working relationship. Transactional surveys are sent out at the discretion of whoever has contact with customersbut the data should be collected immediately after the interaction while the experience is still fresh in the customers mind. Relationship surveys should be spread out over the course of a year. Here are a couple of points to keep in mind: > If there is only one data point for each year, a single event could have a large impact on results. > Research in the hotel industry reveals that satisfaction ratings are consistently higher in the spring than in the fall. > For companies with a sufficiently large customer baseat least 10,000relationship surveys can be conducted on a continuous basis but reported on a monthly basis to address the effects of seasonality or single events. In many cases conducting both transactional and relationship surveys may be appropriate. For example, a company may conduct transactional research for customer service purposes and relationship research for routine checkups on the health of the business.

2. How Many Customers to Survey

For companies with a small customer basefor example, 2,000 or fewerinterview as many customers as possible. If your budget allows it, offer an incentive such as company merchandise or a small gift certificate to compensate respondents for their time. If your company has more than 2,000 customers and it is not economically feasible to survey all of them, you can survey a subset. In this case youll need to consider the following concepts relating to sample selection and accuracy of results:

If you have 2,000 or fewer customers, survey as many as your budget will permit; if you have more than 2,000 customers, interview a random sample.

White Paper

Measuring and Tracking Customer Satisfaction

> Random selection. First, it is essential that a random selection of customers be contacted to avoid introducing bias into how the customers were selectedand to help ensure that the sample of customers is representative of the entire customer base. For example, if a company surveyed only customers who contacted customer service, those individuals may be very different from customers who have not contacted customer service. > Margin of error. Next you need to establish the level of precision you want to have in the accuracy of the results. The margin of error (also known as a confidence interval) is an indicator of sample accuracy for random samples. It is the plus-or-minus figure that is commonly reported with news polls, such as plus or minus four points. In a customer satisfaction example, if the reported result is 80% satisfaction with a 4-point margin of error, that means the true answer is somewhere between 76% (80 4) and 84% (80 + 4) assuming the whole population of customers had been asked. The larger the sample size, the more accurate the results (or the smaller the confidence interval). > Confidence level. Then you need to determine how certain you want to be that the survey results are within the margin of error. The confidence level indicates the level of certainty that the survey results are within the confidence interval for random samples. Typically, researchers use the 95% confidence level. Together, the confidence level and margin of error together describe the certainty you have in the precision of the data. For example, for a reported result of 80% satisfaction at the 95% confidence level with a 4-point margin of error, you can say that you are 95% certain that percentage of satisfied customers is between 76% and 84%. The table below outlines the sample sizes needed for different customer bases at varying levels of accuracy at the 95% confidence level.

Sample Size Needed Size of Customer Base

+/-3%
(95% Confidence Level)

+/-5%

+/-10

2,000 3,000 5,000 10,000 20,000 50,000 100,000

696 788 880 965 1,014 1,045 1,058

323 341 357 370 377 382 383

92 94 95 96 96 96 96

Another important consideration in determining the sample size is how much analysis will need to be done on groups of customers. To analyze customers in particular industries or regions, the sample size should be adjusted to at least 75 people in each group. A number of online sample-size calculators are available that will provide the sample size necessary for a particular confidence interval, or the margin of error for a particular sample size. Just search sample size calculator with your search engine of choice on the Internet.

3. How to Distribute Your Survey

The best means of distributing a survey depends on whether it is a transactional or relationship survey. A transactional survey is conducted at the point of customer contact. Depending on the nature of the customer contact, an in-person, telephone, or online survey may be appropriate. Relationship surveys, on the other hand, are most cost-effectively conducted online. Online data collection offers significant advantages over other modes of interviewing customers, and you should use it whenever possible. The advantages include: 5

White Paper

Measuring and Tracking Customer Satisfaction

> Speed. The Internet offers instantaneous distribution of survey and real-time accumulation and tabulation of results. This allows for immediate data analysis, even while the survey is still in progress. Because customer satisfaction results are used to identify problems and fix them, the faster responses arrive, the faster they can be addressed. In contrast, mail surveys suffer from long lag times and low response rates, as low as 5%. Telephone surveys take longer because of declining response rates. Refusal rates for phone interviews have reached 60% (AC Nielsen, 2004). With the ease of answering online surveys, they can be completed faster and a broader segment of the customer base can be reached. Depending on a number of variablesthe relationship with the survey recipients, the length of the survey, whether a reminder is sent, and whether an incentive is offeredresponse rates for online surveys can be upward of 35%. Even for online surveys in which there is no prior relationship with recipients, response rates can be 23% to 31% (Quirks Marketing Research Review, 2005). > Candor. People are more honest when their answers are not filtered through someone on the phone. This is essential for research on sensitive subject matter where studies indicate people are more likely to answer questions on the Web than they are on the phone or in personal interviews. (Quirks Marketing Research Review, 2003). The removal of interviewer bias and the elimination of the wait time for an interviewer to record results also yields more candid and complete responses to open-ended questions. This is particularly important when customers volunteer additional information to explain their satisfaction ratings. Such responses provide insight into what a company is doing well and frequently provide warning signs about the health of the business relationship. > Cost. The Internet eliminates many of the costs associated with traditional marketing research. Online surveys avoid postage and telephone costs as well as basic materials like paper, staples, envelopes, and printing. Because it is self-directed, there is no interviewer cost. Finally, its more convenient so the cost of offering incentives can be reduced. Online and offline methods of data collection can also be combined. If offline methods are necessary for part of the customer base, the data for customers who can be reached only via mail, in person, or by telephone can be input to an online survey tool. That way all survey results can be captured, reviewed, and analyzed together.

How to Design an Effective Customer Satisfaction Survey

Most businesses and organizations embrace a philosophy of continuous improvement. A welldesigned customer satisfaction survey will help measure your progress toward that goal. A quick way to get started and ensure a successful survey design is to use Zoomerangs Customer Satisfaction Survey templates. If the survey will be designed from scratch, follow Zoomerangs recommended design principles. For an example survey demonstrating these principles, click here.

1. Ask Overall Satisfaction Early in the Survey

Ask the general satisfaction question at the start of the survey to avoid bias. This will allow measurement of customers overall impressions of a company or an organization prior to prompting them to think of specific aspects of the relationship. Consider the example of a banks customer satisfaction survey. The first question should be Overall, how satisfied are you with Bank X? Then ask for specifics: their teller experiences, the availability of ATMs, etc. Otherwise, the customers answers to the overall satisfaction question will be influenced by their satisfaction with specific attributes of that relationship that they may not have thought about until they were asked.

White Paper

Measuring and Tracking Customer Satisfaction

Ask overall satisfaction first.

Then ask about satisfaction with specific aspects of the relationship.

2. Use a 5-point Satisfaction Scale

Question scales should have descriptive labels associated with the numbers, and the top end of the scale should mean that customers are truly wowed. If the ends of the scale are simply satisfied and dissatisfied, it will not provide a sense of the intensity of customers happiness with a product or service. A customer who is simply satisfied just hasnt found a better deal yet. The 5-point Satisfaction Scale

An insightful alternative to the 5-point satisfaction scale for customer service is a 5-point expectations scale. It provides clear direction and allows customers a polite way to suggest that a company has not done a great job. The 5-point Expectations Scale for Service

White Paper

Measuring and Tracking Customer Satisfaction

3. Be Consistent in Your Questions

Consistency is critical in customer satisfaction research. Question scales should be consistent within a particular questionnaire as well as over time. The key Question scales measures of customer satisfaction, including the overall satisfaction question and should be consistent those asking about specific aspects of the relationship, should all use the same within a particular scale. That way an apples-to-apples comparison can be made and it is readily questionnaire as well apparent where a particular part of the business may need improvement. For example, if a 5-point satisfaction scale is used for the overall satisfaction question as over time. and 7-point scales are used for other questions about timeliness of service or product quality, it will be challenging to quickly uncover weaknesses in the business. If the survey reveals that 80% give a top rating in overall satisfaction on a 5-point scale but only 62% do so for a particular attribute on a 7-point scale, how much of that difference is a result of the difficulty of meeting a higher threshold on a larger scale? Similarly, question content and scales should remain consistent over time. Customer satisfaction scores are typically evaluated on a quarterly basis or at regular intervals; and if the questions and the scales change, evaluating performance over time is compromised. Note: Scales can have a dramatic effect on satisfaction ratings, so be wary of writing questions to drive up the numbersespecially if results are being used for compensation purposes. Be cautious about interpreting competitors claims of satisfaction scores. Know how the question was asked before jumping to any conclusions about the health of the business.

4. Keep the Survey Short and Focused

Focus on getting a read on your customers relationship with you. Avoid the temptation to ask everything youve ever wanted to know. The more ground you try to cover, the more likely it is that respondents will abandon the survey. This can also result in inflated satisfaction scores because those customers willing to spend that much time answering your survey are probably the ones with positive feelings toward you. An expedient way to develop a short, focused survey instrument is to rely on Zoomerangs Customer Satisfaction Survey templates. If you build your own survey be sure to include all the key questions: > Overall satisfaction > An open-ended probing into the reason for the satisfaction rating > Likelihood of recommendation > An open-ended probing into the reason for the likelihood of recommendation > Likelihood of repeat purchase > Satisfaction with specific attributes of the product or service > An opportunity to provide additional feedback For an example survey demonstrating these questions, click here. Think of the customer satisfaction survey as part of an ongoing conversation with your customers. Rather than cramming every question you have into a single survey, do multiple surveys throughout the year on different topics.

Click here for the Customer Satisfaction Survey example.

5. Ask Demographic or Firmographic Questions

Inquiring about demographics or firmographics (company or industry type and size) enables you to analyze the data by different subgroupssuch as new customers or regional customers. Consider hypotheses about customer profiles that may affect satisfaction and include questions that capture that data.

White Paper

Measuring and Tracking Customer Satisfaction

Interpreting Survey Results

When youre analyzing customer satisfaction survey results, the most important goals are minimizing the low scores and improving the top scores. It is important to monitor the top two-box satisfaction number, which is the combined percentage of those saying they are very or somewhat satisfied. It is essential, however, to call out to management the proportion of customers who are dissatisfied and to reduce those percentages. Insights into how to do that are found by learning what those who provide high ratings have to say and reviewing the results of those who are dissatisfied with performance.

1. Trends in Satisfaction Score

Observing the top-two box percentage over time will reveal where a company is improving and provide warning signs of what needs improvement. Percentages are a useful means of communicating results in customer satisfaction research because they are readily understood by most people. Moreover, they are more aligned with business objectives (increasing the percentage of people who are satisfied and minimizing those who are dissatisfied). Example of Trend Report in Overall Satisfaction
100% 90% 80% 70% 60%
39% 25%

30%

Top two-box = 78%

35% 38%

Very Satisfied Somewhat Satisfied 35% 40% 40% Neither Satisfied nor Dissatisfied Somewhat Dissatisfied 5% 14% 18% 10% 8% 9% 8% 7% 9% 6% Very Dissatisfied

50% 40% 30% 20% 10%

13%

11%

Bottom two-box = 15%

0%

Q1

Q2

Q3

Q4

A succinct way to measure how well a company is succeeding in minimizing low scores and improving top scores is to calculate the satisfaction differential by subtracting the bottom two-box scorethe combined percentage of those saying they are very or somewhat dissatisfiedfrom the top two-box score. A company that is successful on this metric will see this number increasing over time. The table below illustrates that score for the example trend report.

Conclusion: This company is successfully minimizing low satisfaction scores and improving top satisfaction scores; the satisfaction differential is increasing each quarter.

Q1

Q2
44

Q3
58

Q4
63

Satisfaction Differential
(Top two-box Bottom two-box)

33

White Paper

Measuring and Tracking Customer Satisfaction

Competitive benchmarks for customer satisfaction can be difficult to come by because this is typically proprietary information. Moreover, competitors results cannot be accurately interpreted without knowing the specifics of question wording and the scales used. The satisfaction ratings for the recipients of the Malcolm Baldrige Quality Award given by the U.S. Department of Commerce, however, provide some insight into those companies that have been recognized for achievements in quality and performance. Malcolm Baldrige Quality Award Recipients Satisfaction Ratings

Company Name
(Year of Award) DynMcDermott Petroleum Operations (2005) Caterpillar Financial Services Corporation (2003)

Category
Service Service Small business Small business Healthcare Healthcare Manufacturing

Satisfaction Rating
85% 93% 98% 95.8% 97% 90% >88%

Park Place Lexus (2005) Pals Sudden Service (2001) Bronson Methodist Hospital (2005) Robert Wood Johnson University Hospital Hamilton (2004) Motorola Commercial, Government, and Industrial Solutions Sector (2002)

2. Find Sources of Dissatisfaction

To increase the top two-box score next quarter, it is critical to understand what is driving the bottom two-box scores. There are two easy ways to do that: > Review the open-ended comments. Reading the volunteered comments following up on the rationale for the satisfaction rating is essential. The remarks among the most satisfied customers will shed light on what a company is doing well. They may also provide early warning signs if satisfied customers voice concerns with a service, product, or customer support experience. Dissatisfied customers will be vocal about the rationale for their satisfaction rating and will provide actionable suggestions for how to improve. > Cross-tabulate by satisfaction rating. A key analysis includes looking at the people who are really happy and those who are not happy. Cross-tabulation features make comparisons between satisfied and unsatisfied customers fast and easy. The tendency in analyzing customer satisfaction results is to focus on those company, product, or service attributes that elicit the greatest dissatisfaction overallprice, for example. Most customers will want a lower price regardless of their satisfaction level. It is more revealing to compare how satisfied and dissatisfied customers feel about various aspects of the product or service relationship. The gap in satisfaction ratings for each attribute will help prioritize areas for improvementthe areas with the greatest difference should have the highest priority. In the example cross-tabulation table below, price has the lowest top two-box satisfaction score among all customers. The gap between satisfied and dissatisfied customers, however, is biggest for customer service, so that is the area with the most impact on overall satisfaction. Example Cross Tabulation by Satisfaction Rating
Top Two-Box Scores Overall Satisfied Overall Dissatisfied Customers Customers 80% 67% 95% 30% 96% 64%

Attribute Price Customer service Product quality

All Customers 76% 85% 90%

Conclusion: Improving customer service should be a priority to improve overall satisfaction.

10

White Paper

Measuring and Tracking Customer Satisfaction

3. Present Findings and Action Items

Collecting customer satisfaction data is useful only if there is a process established to deliver recommendations, implement action plans, assign plan owners, and monitor plan execution. Once the results have been compiled and analyzed, they should be presented to management with recommendations for resolving the identified weaknesses. The recommendations should be actionable reduce response times to four hours for example and the person responsible and the resources allocated to meet that goal should be determined. There should be periodic meetings to evaluate progress on the action plan and the next customer satisfaction survey should evaluate progress on that goal.

4. Contact Customers
If possible, dissatisfied customers should be personally contacted to see if there is something that can be done to improve their perception of the business. This is important not simply to increase the odds of keeping that customer but also to prevent negative word of mouth. The personal contact itself may succeed in doing that. Additionally, a discount or free product should be considered. If policies or products are changed based on customers feedback, those customers should be contacted to let them know about those changes and that their feedback was taken seriously.

Start a Customer Satisfaction Research Program Now

Put an end to guesswork as you know it. Start a customer satisfaction research program today. Visit http://info.zoomerang.com/zpro.htm to learn how Zoomerang zPro can help.

Additional Resources
Zoomerang recommends the following resources to learn more about customer satisfaction research: > Lester, Tom. The Cost of Not Caring for Your Customers, Financial Times, January 30, 2006. > McGregor, Jena. Would You Recommend Us? Business Week, January 30, 2006. > Reichheld, Frederick F. Loyalty Rules! How Todays Leaders Build Lasting Relationships (Harvard Business School Press, 2001). > For more information about online survey research, please visit the MarketTools Resource Library.

11

MarketTools, Inc. Zoomerang 150 Spear Street, Suite 600 San Francisco, CA 94105 U.S.A. 800-316-0662 sales@zoomerang.com
Copyright 2006 by MarketTools, Inc. This white paper can not be copied or distributed without MarketTools permission. This whitepaper may be used by Zoomerang Subscribers only.