Performance Support Tools: A Case Study

How Aetna Improved the Quality of its Call Center and Claims Operations

Karen OLeonard February 2005 Bersin & Associates

Performance Support Tools: A Case Study

Table of Contents
Performance Support Tools: A Case Study ............................................................................... 3 The Business Problem ................................................................................................................. 3 The Solution................................................................................................................................... 3 Results............................................................................................................................................. 4 Lessons Learned............................................................................................................................ 6 Future Evolution of Program...................................................................................................... 7 About Us............................................................................................................................................. 8 About This Research ........................................................................................................................ 8

Table of Figures
Figure 1: Screen Shot of the epiLearn Performance Support Tool ........................................ 4 Figure 2: Classroom Training Required Before and After Performance Support Tools.... 5 Figure 3: Relationship Between Usage of Performance Support Tools & Late Payments . 6

Bersin & Associates

February 2005

Performance Support Tools: A Case Study

Performance Support Tools: A Case Study

The Business Problem
Aetna has 6,000 customer service and support employees located in call centers and claim operations around the country. The companys National Customer Operations (NCO) Learning and Performance group develops all of the web-based training materials for these employees, and is known for its cutting-edge learning program development and design expertise. The team has won numerous awards, including the Silver Award at the October 2004 Training conference for its Dr. Bones online medical terminology program. Three years ago, the NCO Learning and Performance team was organized to facilitate knowledge transfer. Aetnas call and claims systems had been evolving rapidly from green screen mainframe terminals to GUI-based systems, and more recently to webbased applications. Aetnas training programs struggled to keep pace with the rapid change of the systems. Traditionally, training was delivered through instructor-led courses. Since employee performance is based largely on production (i.e. the number of claims processed per hour), taking an employee away from work to attend a training class was very costly. Aetnas learning team began searching for alternative methods to provide effective training, while minimizing the impact on employee work schedules.

Aetna refocused its training on the 20% of tasks that are critical to an employees every day job. For the other 80% of tasks, the company provided performance support tools.

The Solution The 80-20 Rule

In Aetnas traditional approach, new employees attended a 13-week training program, where they were expected to learn and understand every aspect of Aetnas call and claims handling processes and procedures. The volumes of training books and reference materials filled four feet of shelving in a room and, as one Aetna learning professional quipped, amounted to more reading than required by a first-year law student. This training approach was just not feasible or economical. Aetnas learning team conducted research by monitoring call center employees tasks over time. They found that call center employees used roughly 20% of the training material 80% of the time. In other words, they perform a small subset of tasks on a daily basis, and a large number of tasks on an infrequent basis. A more sensible training approach, Aetna concluded, would be to focus on the 20% of the job that is critical to an employees every day job. These tasks and processes should be mastered by every call center employee.

Bersin & Associates

February 2005

Performance Support Tools: A Case Study For the other 80% of the job, mastery is less important. Employees are unlikely to retain knowledge on tasks performed infrequently, so the traditional training on these tasks was ineffective. Instead, providing support tools so that employees can access necessary information to complete these tasks when needed would be a more economical and effective approach.

Performance Support Tools

Developing performance support tools requires a different set of skills than traditional content development.
With this new strategy, Aetna began developing performance support tools for its call center applications. These tools are either process-driven or informational in nature. Informational tools will display a definition or description as the user moves a mouse over a term, or they may provide short cuts or links to other resources. Processdriven tools are typically more complex. They may walk an employee through a task, checking off each item as it is completed. More sophisticated tools notify the employee that a step has been skipped so he or she can go back and correct the error. Aetnas learning team built many of these tools internally using HTML and Javascript. In addition, the company licensed Epiances epiLearn performance support system as one of its primary process-driven tools. Figure 1 shows a screen shot of Epiances epiLearn tool for an HR application. Although not specific to call center operations, the screen shot provides a flavor for how the tool is used.

Figure 1: Screen Shot of the epiLearn Performance Support Tool In this example, the HR professional needs to update an employees personal data record. epiLearn brings up a list of steps need to complete the task. As the employee completes each step successfully, the step is checked off the list. If a step is skipped or performed incorrectly, the tool notifies the user, who then can go back and complete the task correctly.
Bersin & Associates 4 February 2005

Performance Support Tools: A Case Study

Aetnas call center employees use epiLearn to complete similar tasks, such as updating a customers address or personal information, changing the vehicle listed on the insurance record, and opening or closing a claim. In each case, the tool works in much the same way as described aboveproviding the steps needed to complete the task, checking off each step as it is completed, and notifying the employee if an error has been made.

Results
In 2003, Aetna developed and deployed 5 performance support tools; by the end of 2004, the company had over 40 support tools in operation. These tools have enabled Aetna to reduce the 13-week training course for new hires to 11 weekssaving each call center employee 2 weeks of time away from work. Our training approach is not about making employees experts on all tasks, said Dave Ulm, Project Manager of Performance Support. We train the employees to know the basics really well. For everything else, there are performance support tools.

Number of Weeks in Classroom Training:

Before and After Performance Support Tools
15

13

# weeks of classroom training

Newly hired call center employees now spend two weeks less in classroom training.

11

Instructor-led training (ILT) only

ILT + performance support tools

Figure 2: Classroom Training Required Before and After Performance Support Tools

After rolling out a new performance support tool in March 2004, Aetna tracked the usage of the tool and quality of claims processed for a 3-month period. Aetna tracked tool usage through cookies placed on employees desktops after they registered the tool. The study showed a clear correlation between the use of the performance support tool and the quality of the claims processing: 100% of the employees who used the performance support tool processed their claims without error. In every case where the processed claim contained an error, the call center employee handling the claim had not used the tool.

Bersin & Associates

February 2005

Performance Support Tools: A Case Study

In a second study, Aetna gathered data on usage of performance support tools and late payments. Again, the results showed a direct correlation between the percentage of employees who used the tool and the number of late payments. The greater the percentage of employees using the tool, the lower the number of late payments.

Relationship Between Tool Usage and Late Payments

As the percentage of employees using the performance tool increased, the number of late claims payments decreased.

% of employees using tool No. of late payments

Figure 3: Relationship Between Usage of Performance Support Tools & Late Payments

Inaccurate and untimely claims processing have huge cost implications for insurers. Reworking claims to correct errors carries significant costs in personnel time. Inaccurate and late payment of claims adversely impact customer satisfaction. And, it is difficult to recoup money in the case of overpayment. The performance support tools have proven to be extremely effective in improving quality and reducing costs.

Lessons Learned
Performance support tools are really a different animal from other training development, says Ulm. They look, feel, and act differently. For most of their online training programs, Aetna uses the 3 e approach: educate; engage; entertain. But for performance support tools, the last two (engage and entertain) dont make sense. The tools must be focused and provide the necessary information quickly, then get out of the way. There is no need or desire for engagement or entertainment.

Bersin & Associates

February 2005

Performance Support Tools: A Case Study Ulm recommends that people building performance support tools must look at what the employee will use, and determine how to make it fit with what they do. This requires more user interface expertise than traditional content development. One rule-of-thumb Aetna uses, for example, is to make the performance support tools accessible from the left side of the screen, which conforms to typical website navigation toolbars. And, the information must be no more than three clicks away or it will take too long for the learner to access.

Future Evolution of Program

The success of the performance support tools has also brought about drawbacks. In 2005, the company plans to have over 100 tools in use. The sheer number of support tools has become unmanageable and has added too much complexity for the learner. So, the company has licensed Epiances Intelligent Dialogue system to manage the learning tools and simplify the process. Epiances tool is context sensitive to where the employee is in the application, and opens tools only appropriate to that section. In addition, Aetnas eCoach technology, based on Epiances cue cards, will be the primary help tool for call center employees. The eCoach technology goes one step further than the current performance support tools in use. For each process, eCoach brings up a list of tasks required, and, as the user completes each step, it is checked off the list. This way, call center employees immediately get the support they need, and the confidence of knowing that their tasks have been completed successfully. The company plans to officially launch a new call center system with the eCoach technology in April 2005.

Bersin & Associates

February 2005

Performance Support Tools: A Case Study

About Us
Bersin & Associates is the only research and consulting firm solely focused on What Works e-learning research. With more than 20 years of experience in e-learning, training, and enterprise technology, Bersin & Associates provides a wide range of services including market research, best-practices, vendor and product analysis, corporate workshops, corporate implementation plans, and sales and marketing programs. Some of Bersin & Associates innovations include a complete methodology for learning management system (LMS) selection and application usage, an end-to-end architecture and solution for training analytics, and one of the industry's largest research studies on blended learning implementations. Bersin & Associates offers the industry's first e-learning research subscription, "The E-Learning Research Center," (http://www.elearningresearch.com), which offers up-todate research, product selection guides, white papers, and access to other experts online. This service enables corporate training managers, vendors, and consultants to make faster, better decisions about e-learning strategy, programs, and technology. Bersin & Associates can be reached at http://www.bersin.com or at (510) 654-8500.

About This Research

Copyright 2005 Bersin & Associates. All rights reserved. What Works and related names are registered trademarks of Bersin & Associates. No materials from this study can be duplicated, copied, republished, or re-used without written permission from Bersin & Associates.

Bersin & Associates

February 2005