White Paper

Measuring and Tracking Customer Satisfaction
You know why. We show you how. By Michael Conklin, Chief Methodologist MarketTools, Inc.

The number of subscribers using Zoomerang surveys has doubled every year since 1999.zoomerang. With more than 100 million surveys sent and counting. In addition. For Subscribers only This White Paper is provided to our Zoomerang zPro subscribers as an added bene t to the zPro survey service.White Paper Measuring and Tracking Customer Satisfaction About the Author Michael Conklin is MarketTools’ Chief Methodologist. 2 . Zoomerang has grown rapidly and has been adopted worldwide. Zoomerang provides best practices and research methodology that our subscribers nd useful to increase both the quality and impact of their surveys. He is a frequent speaker at the American Marketing Association's Annual Advanced Research Techniques Forum as well as at the American Statistical Association's Joint Statistical Meetings conference . as well as numerous statistical journals. on NBC News. He received his education at The Wharton School. He is a marketing research veteran and analytical methodology pioneer with 24 years experience. in Minneapolis. a leading provider of technology and advisory services for conducting Web-based market research. About Zoomerang Zoomerang ( www. and Zoomerang has been featured in Fortune magazine.com ) o ers an easy way for businesses to get fast answers to important questions. He came to MarketTools after serving for 10 years as a Senior Vice President and Team Leader of Analytic Services for GfK Custom Research Inc. Conklin has been published in key marketing publications. Zoomerang is part of the MarketTools family. and on the NBC Today show. such as Marketing News and Marketing Research Magazine.

" (Source: Bain & Company study in Harvard Business Review.White Paper Measuring and Tracking Customer Satisfaction Best Practices in Customer Satisfaction Research How satisfied are your customers? Are they more or less pleased with your company than they were last year? How does your customer satisfaction level compare with that of your competitors? What impact does it have on your company’s profits? How do you know? Guesswork and hunches won’t cut it. for many companies. they refer other customers. you need to know right away. $80 $70 Cost Per Customer $60 $50 $40 $30 $20 $10 $0 Existing Customer New Customer • "It's six to seven times more expensive to gain a new customer than it is to retain an existing customer. your company will be losing business. magnifying its impact. And if they’re loyal they not only buy more. or thousands of other customers. and putting itself at a competitive disadvantage. distribute. not instinct. 3 . You need hard data. You can do it yourself. but there are a few things you need to consider. that bad experience can quickly be broadcast to dozens. Why Measure Customer Satisfaction? Customer satisfaction is tied directly to profitability. If your customers are happy. It is equally important to consistently measure and monitor that input. missing opportunities. We want you to act on insight. You need to measure and monitor customer satisfaction—consistently and regularly. they tend to be loyal. So if your business is doing something that frustrates customers. The same study found that it costs six to seven times more to gain a new customer than to keep an existing one. This paper gives you the benefit of the experience and lessons learned by industry experts to help you design. 2001) Customer Type Moreover. one bad experience can outweigh a whole lot of good experiences. Because of e-mail and instant messaging. an increase of 5% in customer retention can increase profits by 25% to 95%. It is critical to give customers the opportunity to provide feedback about their overall satisfaction level and specific likes and dislikes. You need a customer satisfaction research strategy and program. Well-established research by Bain & Company found that. hundreds. and analyze your customer satisfaction research. Without an effective customer satisfaction research program in place." • "A 5% increase in customer retention can increase profits by 25% to 95%.

survey as many as your budget will permit.000 customers and it is not economically feasible to survey all of them. 2.000 or fewer customers. and they serve very different purposes: > Transactional surveys solicit feedback directly from the product or service user about that particular encounter. How Many Customers to Survey For companies with a small customer base—for example. interview a random sample. There are two key types of surveys. For example. If your company has more than 2. a single event could have a large impact on results. how many customers will be invited to respond. Here are a couple of points to keep in mind: > If there is only one data point for each year. For example. offer an incentive such as company merchandise or a small gift certificate to compensate respondents for their time.White Paper Measuring and Tracking Customer Satisfaction Customer Satisfaction Surveys: Tips and Practical Advice The first steps in establishing a customer satisfaction program are determining when to distribute your survey. We’ll take a closer look at how to formulate your survey questions in the next section. a company may conduct transactional research for customer service purposes and relationship research for routine checkups on the health of the business. you can survey a subset. if you have more than 2. They are regularly scheduled surveysoften quarterly. When to Conduct Customer Satisfaction Research Many managers wonder how frequently they should conduct customer satisfaction surveys. In many cases conducting both transactional and relationship surveys may be appropriate. > Research in the hotel industry reveals that satisfaction ratings are consistently higher in the spring than in the fall.000 customers. > For companies with a sufficiently large customer base—at least 10. The respondents typically are responsible for deciding whether to continue the working relationship. and how to deliver the survey to them. They are conducted immediately after each customer transaction.000—relationship surveys can be conducted on a continuous basis but reported on a monthly basis to address the effects of seasonality or single events. In this case you’ll need to consider the following concepts relating to sample selection and accuracy of results: If you have 2. If your budget allows it. 2. Relationship surveys should be spread out over the course of a year. > Relationship surveys collect input from people who have an ongoing relationship with the company and have had multiple transactions. 4 .000 or fewer—interview as many customers as possible. Transactional surveys are sent out at the discretion of whoever has contact with customers—but the data should be collected immediately after the interaction while the experience is still fresh in the customer’s mind. a survey may be administered after a call center experience. 1. The answer depends on the size of the customer base and the purpose of the research.

the more accurate the results (or the smaller the confidence interval). the confidence level and margin of error together describe the certainty you have in the precision of the data. such as “plus or minus four points. telephone. > Confidence level. for a reported result of 80% satisfaction at the 95% confidence level with a 4-point margin of error. that means the true answer is somewhere between 76% (80 – 4) and 84% (80 + 4) assuming the whole population of customers had been asked. researchers use the 95% confidence level. are most cost-effectively conducted online. Next you need to establish the level of precision you want to have in the accuracy of the results. an in-person. those individuals may be very different from customers who have not contacted customer service. The larger the sample size.000 10. Typically.000 696 788 880 965 1. Relationship surveys. A transactional survey is conducted at the point of customer contact. It is the plus-or-minus figure that is commonly reported with news polls. 3. For example. For example.” In a customer satisfaction example. Online data collection offers significant advantages over other modes of interviewing customers.045 1.000 5.000 20.White Paper Measuring and Tracking Customer Satisfaction > Random selection.000 100. Depending on the nature of the customer contact. A number of online sample-size calculators are available that will provide the sample size necessary for a particular confidence interval.058 323 341 357 370 377 382 383 92 94 95 96 96 96 96 Another important consideration in determining the sample size is how much analysis will need to be done on groups of customers. if the reported result is 80% satisfaction with a 4-point margin of error. it is essential that a random selection of customers be contacted to avoid introducing bias into how the customers were selected—and to help ensure that the sample of customers is representative of the entire customer base. on the other hand. Sample Size Needed Size of Customer Base +/-3% (95% Confidence Level) +/-5% +/-10 2. How to Distribute Your Survey The best means of distributing a survey depends on whether it is a transactional or relationship survey. if a company surveyed only customers who contacted customer service. Then you need to determine how certain you want to be that the survey results are within the margin of error. To analyze customers in particular industries or regions. The table below outlines the sample sizes needed for different customer bases at varying levels of accuracy at the 95% confidence level.014 1. Together. or the margin of error for a particular sample size. First. or online survey may be appropriate.000 3. Just search “sample size calculator” with your search engine of choice on the Internet. The advantages include: 5 . > Margin of error.000 50. and you should use it whenever possible. you can say that you are 95% certain that percentage of satisfied customers is between 76% and 84%. The margin of error (also known as a confidence interval) is an indicator of sample accuracy for random samples. the sample size should be adjusted to at least 75 people in each group. The confidence level indicates the level of certainty that the survey results are within the confidence interval for random samples.

Telephone surveys take longer because of declining response rates.White Paper Measuring and Tracking Customer Satisfaction > Speed. or by telephone can be input to an online survey tool. (Quirk’s Marketing Research Review. > Cost. The Internet eliminates many of the costs associated with traditional marketing research. If offline methods are necessary for part of the customer base. This is particularly important when customers volunteer additional information to explain their satisfaction ratings. the data for customers who can be reached only via mail. the faster they can be addressed. the length of the survey. whether a reminder is sent. The Internet offers instantaneous distribution of survey and real-time accumulation and tabulation of results. Such responses provide insight into what a company is doing well and frequently provide warning signs about the health of the business relationship. click here. the customers’ answers to the overall satisfaction question will be influenced by their satisfaction with specific attributes of that relationship that they may not have thought about until they were asked. Online surveys avoid postage and telephone costs as well as basic materials like paper. etc. This is essential for research on sensitive subject matter where studies indicate people are more likely to answer questions on the Web than they are on the phone or in personal interviews. That way all survey results can be captured. With the ease of answering online surveys. Depending on a number of variables—the relationship with the survey recipients. they can be completed faster and a broader segment of the customer base can be reached. People are more honest when their answers are not filtered through someone on the phone. Otherwise. Finally. Because it is self-directed. the availability of ATMs. Ask Overall Satisfaction Early in the Survey Ask the general satisfaction question at the start of the survey to avoid bias. 2004). 6 . Even for online surveys in which there is no prior relationship with recipients. This allows for immediate data analysis. 1. 2003). If the survey will be designed from scratch. 2005). it’s more convenient so the cost of offering incentives can be reduced. in person. there is no interviewer cost. even while the survey is still in progress. > Candor. A quick way to get started and ensure a successful survey design is to use Zoomerang’s Customer Satisfaction Survey templates. The first question should be “Overall. In contrast. envelopes. Online and offline methods of data collection can also be combined. reviewed. staples. For an example survey demonstrating these principles. follow Zoomerang’s recommended design principles. how satisfied are you with Bank X?” Then ask for specifics: their teller experiences. and printing. A welldesigned customer satisfaction survey will help measure your progress toward that goal. This will allow measurement of customers’ overall impressions of a company or an organization prior to prompting them to think of specific aspects of the relationship. Refusal rates for phone interviews have reached 60% (AC Nielsen. Because customer satisfaction results are used to identify problems and fix them. response rates can be 23% to 31% (Quirk’s Marketing Research Review. The removal of interviewer bias and the elimination of the wait time for an interviewer to record results also yields more candid and complete responses to open-ended questions. the faster responses arrive. and analyzed together. mail surveys suffer from long lag times and low response rates. How to Design an Effective Customer Satisfaction Survey Most businesses and organizations embrace a philosophy of continuous improvement. as low as 5%. and whether an incentive is offered—response rates for online surveys can be upward of 35%. Consider the example of a bank’s customer satisfaction survey.

A customer who is simply “satisfied” just hasn’t found a better deal yet. The 5-point Satisfaction Scale An insightful alternative to the 5-point satisfaction scale for customer service is a 5-point expectations scale.” it will not provide a sense of the intensity of customers’ happiness with a product or service. Then ask about satisfaction with specific aspects of the relationship. The 5-point Expectations Scale for Service 7 . Use a 5-point Satisfaction Scale Question scales should have descriptive labels associated with the numbers. 2.” If the ends of the scale are simply “satisfied” and “dissatisfied. and the top end of the scale should mean that customers are truly “wowed.White Paper Measuring and Tracking Customer Satisfaction Ask overall satisfaction first. It provides clear direction and allows customers a polite way to suggest that a company has not done a great job.

An expedient way to develop a short. and if the questions and the scales change. and 7-point scales are used for other questions about timeliness of service or product quality. click here. Note: Scales can have a dramatic effect on satisfaction ratings. Rather than cramming every question you have into a single survey. If the survey reveals that 80% give a top rating in overall satisfaction on a 5-point scale but only 62% do so for a particular attribute on a 7-point scale. including the overall satisfaction question and should be consistent those asking about specific aspects of the relationship. Be Consistent in Your Questions Consistency is critical in customer satisfaction research. The key Question scales measures of customer satisfaction. Keep the Survey Short and Focused Focus on getting a read on your customers’ relationship with you. Be cautious about interpreting competitors’ claims of satisfaction scores. 5. Customer satisfaction scores are typically evaluated on a quarterly basis or at regular intervals. That way an apples-to-apples comparison can be made and it is readily questionnaire as well apparent where a particular part of the business may need improvement. question content and scales should remain consistent over time. The more ground you try to cover. Question scales should be consistent within a particular questionnaire as well as over time. Know how the question was asked before jumping to any conclusions about the health of the business. should all use the same within a particular scale. the more likely it is that respondents will abandon the survey. Click here for the Customer Satisfaction Survey example. 8 . how much of that difference is a result of the difficulty of meeting a higher threshold on a larger scale? Similarly. Consider hypotheses about customer profiles that may affect satisfaction and include questions that capture that data. Think of the customer satisfaction survey as part of an ongoing conversation with your customers. This can also result in inflated satisfaction scores because those customers willing to spend that much time answering your survey are probably the ones with positive feelings toward you. Avoid the temptation to ask everything you’ve ever wanted to know. 4. If you build your own survey be sure to include all the key questions: > Overall satisfaction > An open-ended probing into the reason for the satisfaction rating > Likelihood of recommendation > An open-ended probing into the reason for the likelihood of recommendation > Likelihood of repeat purchase > Satisfaction with specific attributes of the product or service > An opportunity to provide additional feedback For an example survey demonstrating these questions. For example. so be wary of writing questions to drive up the numbers—especially if results are being used for compensation purposes. if a 5-point satisfaction scale is used for the overall satisfaction question as over time. Ask Demographic or “Firmographic” Questions Inquiring about demographics or firmographics (company or industry type and size) enables you to analyze the data by different subgroups—such as new customers or regional customers. focused survey instrument is to rely on Zoomerang’s Customer Satisfaction Survey templates.White Paper Measuring and Tracking Customer Satisfaction 3. evaluating performance over time is compromised. do multiple surveys throughout the year on different topics. it will be challenging to quickly uncover weaknesses in the business.

The table below illustrates that score for the example trend report. It is important to monitor the “top two-box” satisfaction number. It is essential. the satisfaction differential is increasing each quarter. Q1 Q2 44 Q3 58 Q4 63 Satisfaction Differential (Top two-box – Bottom two-box) 33 9 . 1. A company that is successful on this metric will see this number increasing over time. Trends in Satisfaction Score Observing the top-two box percentage over time will reveal where a company is improving and provide warning signs of what needs improvement. Conclusion: This company is successfully minimizing low satisfaction scores and improving top satisfaction scores. Example of Trend Report in Overall Satisfaction 100% 90% 80% 70% 60% 39% 25% 30% Top two-box = 78% 35% 38% Very Satisfied Somewhat Satisfied 35% 40% 40% Neither Satisfied nor Dissatisfied Somewhat Dissatisfied 5% 14% 18% 10% 8% 9% 8% 7% 9% 6% Very Dissatisfied 50% 40% 30% 20% 10% 13% 11% Bottom two-box = 15% 0% Q1 Q2 Q3 Q4 A succinct way to measure how well a company is succeeding in minimizing low scores and improving top scores is to calculate the satisfaction differential by subtracting the bottom two-box score—the combined percentage of those saying they are very or somewhat dissatisfied—from the top two-box score. Percentages are a useful means of communicating results in customer satisfaction research because they are readily understood by most people. Insights into how to do that are found by learning what those who provide high ratings have to say and reviewing the results of those who are dissatisfied with performance. however. they are more aligned with business objectives (increasing the percentage of people who are satisfied and minimizing those who are dissatisfied).White Paper Measuring and Tracking Customer Satisfaction Interpreting Survey Results When you’re analyzing customer satisfaction survey results. the most important goals are minimizing the low scores and improving the top scores. Moreover. which is the combined percentage of those saying they are very or somewhat satisfied. to call out to management the proportion of customers who are dissatisfied and to reduce those percentages.

or service attributes that elicit the greatest dissatisfaction overall—price. however. 10 . provide some insight into those companies that have been recognized for achievements in quality and performance. and Industrial Solutions Sector (2002) 2. The tendency in analyzing customer satisfaction results is to focus on those company. > Cross-tabulate by satisfaction rating.8% 97% 90% >88% Park Place Lexus (2005) Pal’s Sudden Service (2001) Bronson Methodist Hospital (2005) Robert Wood Johnson University Hospital Hamilton (2004) Motorola Commercial. product. or customer support experience. The gap in satisfaction ratings for each attribute will help prioritize areas for improvement—the areas with the greatest difference should have the highest priority. There are two easy ways to do that: > Review the open-ended comments. It is more revealing to compare how satisfied and dissatisfied customers feel about various aspects of the product or service relationship. In the example cross-tabulation table below.S. Department of Commerce. The satisfaction ratings for the recipients of the Malcolm Baldrige Quality Award given by the U. Most customers will want a lower price regardless of their satisfaction level. product. so that is the area with the most impact on overall satisfaction. The remarks among the most satisfied customers will shed light on what a company is doing well. is biggest for customer service. They may also provide early warning signs if satisfied customers voice concerns with a service. however. it is critical to understand what is driving the bottom two-box scores. Example Cross Tabulation by Satisfaction Rating Top Two-Box Scores Overall Satisfied Overall Dissatisfied Customers Customers 80% 67% 95% 30% 96% 64% Attribute Price Customer service Product quality All Customers 76% 85% 90% Conclusion: Improving customer service should be a priority to improve overall satisfaction. Malcolm Baldrige Quality Award Recipients’ Satisfaction Ratings Company Name (Year of Award) DynMcDermott Petroleum Operations (2005) Caterpillar Financial Services Corporation (2003) Category Service Service Small business Small business Healthcare Healthcare Manufacturing Satisfaction Rating 85% 93% 98% 95. Moreover. Government. Reading the volunteered comments following up on the rationale for the satisfaction rating is essential. A key analysis includes looking at the people who are really happy and those who are not happy. price has the lowest top two-box satisfaction score among all customers. Find Sources of Dissatisfaction To increase the top two-box score next quarter. Cross-tabulation features make comparisons between satisfied and unsatisfied customers fast and easy. The gap between satisfied and dissatisfied customers. competitors’ results cannot be accurately interpreted without knowing the specifics of question wording and the scales used. for example. Dissatisfied customers will be vocal about the rationale for their satisfaction rating and will provide actionable suggestions for how to improve.White Paper Measuring and Tracking Customer Satisfaction Competitive benchmarks for customer satisfaction can be difficult to come by because this is typically proprietary information.

implement action plans. Visit http://info. This is important not simply to increase the odds of keeping that customer but also to prevent negative word of mouth. Start a Customer Satisfaction Research Program Now Put an end to guesswork as you know it.” Financial Times.zoomerang. If policies or products are changed based on customers’ feedback. January 30. Tom. Start a customer satisfaction research program today. dissatisfied customers should be personally contacted to see if there is something that can be done to improve their perception of the business. Frederick F. > McGregor. Additionally. 2006. 4.com/zpro. > Reichheld. January 30. and monitor plan execution. “Would You Recommend Us?” Business Week. a discount or free product should be considered. “The Cost of Not Caring for Your Customers. The personal contact itself may succeed in doing that. Jena. they should be presented to management with recommendations for resolving the identified weaknesses. > For more information about online survey research. please visit the MarketTools Resource Library. Loyalty Rules! How Today’s Leaders Build Lasting Relationships (Harvard Business School Press. 2006. Contact Customers If possible. 11 . The recommendations should be actionable – reduce response times to four hours for example – and the person responsible and the resources allocated to meet that goal should be determined. those customers should be contacted to let them know about those changes and that their feedback was taken seriously. 2001). There should be periodic meetings to evaluate progress on the action plan and the next customer satisfaction survey should evaluate progress on that goal.htm to learn how Zoomerang zPro can help. Present Findings and Action Items Collecting customer satisfaction data is useful only if there is a process established to deliver recommendations.White Paper Measuring and Tracking Customer Satisfaction 3. Additional Resources Zoomerang recommends the following resources to learn more about customer satisfaction research: > Lester. Once the results have been compiled and analyzed. assign plan owners.

Suite 600 San Francisco. This white paper can not be copied or distributed without MarketTools permission. This whitepaper may be used by Zoomerang Subscribers only. .com Copyright 2006 by MarketTools.MarketTools. 800-316-0662 sales@zoomerang. CA 94105 U.A.S. Inc. Inc. Zoomerang 150 Spear Street.