Performance

Support Tools:
A Case Study
How Aetna Improved the Quality of its
Call Center and Claims Operations

Karen O’Leonard
February 2005
© Bersin & Associates
Performance Support Tools: A Case Study

Table of Contents

Performance Support Tools: A Case Study ............................................................................... 3

The Business Problem ................................................................................................................. 3
The Solution................................................................................................................................... 3
Results............................................................................................................................................. 4
Lessons Learned............................................................................................................................ 6
Future Evolution of Program...................................................................................................... 7
About Us............................................................................................................................................. 8
About This Research ........................................................................................................................ 8

Table of Figures
Figure 1: Screen Shot of the epiLearn Performance Support Tool ........................................ 4
Figure 2: Classroom Training Required Before and After Performance Support Tools.... 5
Figure 3: Relationship Between Usage of Performance Support Tools & Late Payments . 6

© Bersin & Associates 2 February 2005

Performance Support Tools: A Case Study

Performance Support Tools:

A Case Study

The Business Problem

Aetna has 6,000 customer service and support employees located in call centers and
claim operations around the country. The company’s National Customer Operations
(NCO) Learning and Performance group develops all of the web-based training materials
for these employees, and is known for its cutting-edge learning program development
and design expertise. The team has won numerous awards, including the Silver Award at
the October 2004 Training conference for its “Dr. Bones” online medical terminology
program.

Three years ago, the NCO Learning and Performance team was organized to facilitate
knowledge transfer. Aetna’s call and claims systems had been evolving rapidly from
“green screen” mainframe terminals to GUI-based systems, and more recently to web-
Aetna based applications. Aetna’s training programs struggled to keep pace with the rapid
refocused its change of the systems. Traditionally, training was delivered through instructor-led
training on the courses. Since employee performance is based largely on production (i.e. the number of
claims processed per hour), taking an employee away from work to attend a training
20% of tasks class was very costly. Aetna’s learning team began searching for alternative methods to
that are critical provide effective training, while minimizing the impact on employee work schedules.
to an
employee’s
every day job. The Solution
For the other
80% of tasks, The 80-20 Rule
the company In Aetna’s traditional approach, new employees attended a 13-week training program,
provided where they were expected to learn and understand every aspect of Aetna’s call and
performance claims handling processes and procedures. The volumes of training books and reference
materials filled four feet of shelving in a room and, as one Aetna learning professional
support tools. quipped, “…amounted to more reading than required by a first-year law student.” This
training approach was just not feasible or economical.

Aetna’s learning team conducted research by monitoring call center employees’ tasks
over time. They found that call center employees used roughly 20% of the training
material 80% of the time. In other words, they perform a small subset of tasks on a daily
basis, and a large number of tasks on an infrequent basis. A more sensible training
approach, Aetna concluded, would be to focus on the 20% of the job that is critical to
an employee’s every day job. These tasks and processes should be mastered by every
call center employee.

© Bersin & Associates 3 February 2005

Performance Support Tools: A Case Study

For the other 80% of the job, mastery is less important. Employees are unlikely to retain
knowledge on tasks performed infrequently, so the traditional training on these tasks
was ineffective. Instead, providing support tools so that employees can access necessary
information to complete these tasks when needed would be a more economical and
effective approach.

Performance Support Tools

With this new strategy, Aetna began developing performance support tools for its call
center applications. These tools are either process-driven or informational in nature.
Informational tools will display a definition or description as the user moves a mouse
Developing over a term, or they may provide short cuts or links to other resources. Process-
performance driven tools are typically more complex. They may walk an employee through a task,
support tools checking off each item as it is completed. More sophisticated tools notify the employee
requires a that a step has been skipped so he or she can go back and correct the error.
different set of
Aetna’s learning team built many of these tools internally using HTML and Javascript. In
skills than addition, the company licensed Epiance’s epiLearn performance support system as one
traditional of its primary process-driven tools.
content
Figure 1 shows a screen shot of Epiance’s epiLearn tool for an HR application. Although
development.
not specific to call center operations, the screen shot provides a flavor for how the tool
is used.

Figure 1: Screen Shot of the epiLearn Performance Support Tool

In this example, the HR professional needs to update an employee’s personal data

record. epiLearn brings up a list of steps need to complete the task. As the employee
completes each step successfully, the step is checked off the list. If a step is skipped or
performed incorrectly, the tool notifies the user, who then can go back and complete
the task correctly.

© Bersin & Associates 4 February 2005

Performance Support Tools: A Case Study

Aetna’s call center employees use epiLearn to complete similar tasks, such as updating a
customer’s address or personal information, changing the vehicle listed on the insurance
record, and opening or closing a claim. In each case, the tool works in much the same
way as described above—providing the steps needed to complete the task, checking off
each step as it is completed, and notifying the employee if an error has been made.

Results
In 2003, Aetna developed and deployed 5 performance support tools; by the end of
2004, the company had over 40 support tools in operation. These tools have enabled
Aetna to reduce the 13-week training course for new hires to 11 weeks—saving each
call center employee 2 weeks of time away from work. “Our training approach is not
about making employees experts on all tasks,” said Dave Ulm, Project Manager of
Performance Support. “We train the employees to know the basics really well. For
everything else, there are performance support tools.”

Number of Weeks in Classroom Training:

Before and After Performance Support Tools

15
Newly hired call
center employees
now spend two
13 weeks less in
classroom training.
# weeks of classroom training

11

Instructor-led training (ILT) only ILT + performance support tools

Figure 2: Classroom Training Required Before and After Performance Support Tools

After rolling out a new performance support tool in March 2004, Aetna tracked the
usage of the tool and quality of claims processed for a 3-month period. Aetna tracked
tool usage through cookies placed on employees’ desktops after they registered the
tool. The study showed a clear correlation between the use of the performance support
tool and the quality of the claims processing:

• 100% of the employees who used the performance support tool processed
their claims without error.
• In every case where the processed claim contained an error, the call
center employee handling the claim had not used the tool.

© Bersin & Associates 5 February 2005

Performance Support Tools: A Case Study

In a second study, Aetna gathered data on usage of performance support tools and late
payments. Again, the results showed a direct correlation between the percentage of
employees who used the tool and the number of late payments. The greater the
percentage of employees using the tool, the lower the number of late payments.

Relationship Between Tool Usage and Late Payments

As the percentage of
employees using the
performance tool
increased, the number of
late claims payments
decreased.

% of employees using tool

No. of late payments

Figure 3: Relationship Between Usage of Performance Support Tools & Late Payments

Inaccurate and untimely claims processing have huge cost implications for insurers.
Reworking claims to correct errors carries significant costs in personnel time.
Inaccurate and late payment of claims adversely impact customer satisfaction. And, it is
difficult to recoup money in the case of overpayment. The performance support tools
have proven to be extremely effective in improving quality and reducing costs.

Lessons Learned
“Performance support tools are really a different animal from other training
development,” says Ulm. “They look, feel, and act differently.” For most of their online
training programs, Aetna uses the “3 e™” approach: educate; engage; entertain. But for
performance support tools, the last two (engage and entertain) don’t make sense. The
tools must be focused and provide the necessary information quickly, then get out of
the way. There is no need or desire for engagement or entertainment.

© Bersin & Associates 6 February 2005

Performance Support Tools: A Case Study

Ulm recommends that people building performance support tools must look at what the
employee will use, and determine how to make it fit with what they do. This requires
more user interface expertise than traditional content development. One rule-of-thumb
Aetna uses, for example, is to make the performance support tools accessible from the
left side of the screen, which conforms to typical website navigation toolbars. And, the
information must be no more than three clicks away or it will take too long for the
learner to access.

Future Evolution of Program

The success of the performance support tools has also brought about drawbacks. In
2005, the company plans to have over 100 tools in use. The sheer number of support
tools has become unmanageable and has added too much complexity for the learner. So,
the company has licensed Epiance’s Intelligent Dialogue system to manage the learning
tools and simplify the process. Epiance’s tool is context sensitive to where the
employee is in the application, and opens tools only appropriate to that section.

In addition, Aetna’s eCoach technology, based on Epiance’s “cue cards,” will be the
primary help tool for call center employees. The eCoach technology goes one step
further than the current performance support tools in use. For each process, eCoach
brings up a list of tasks required, and, as the user completes each step, it is checked off
the list. This way, call center employees immediately get the support they need, and the
confidence of knowing that their tasks have been completed successfully. The company
plans to officially launch a new call center system with the eCoach technology in April
2005.

© Bersin & Associates 7 February 2005

Performance Support Tools: A Case Study

About Us
Bersin & Associates is the only research and consulting firm solely focused on What
Works™ e-learning research. With more than 20 years of experience in e-learning,
training, and enterprise technology, Bersin & Associates provides a wide range of
services including market research, best-practices, vendor and product analysis,
corporate workshops, corporate implementation plans, and sales and marketing
programs. Some of Bersin & Associates innovations include a complete methodology
for learning management system (LMS) selection and application usage, an end-to-end
architecture and solution for training analytics, and one of the industry's largest research
studies on blended learning implementations.

Bersin & Associates offers the industry's first e-learning research subscription, "The
E-Learning Research Center," (http://www.elearningresearch.com), which offers up-to-
date research, product selection guides, white papers, and access to other experts
online. This service enables corporate training managers, vendors, and consultants to
make faster, better decisions about e-learning strategy, programs, and technology.

Bersin & Associates can be reached at http://www.bersin.com or at (510) 654-8500.

About This Research

Copyright © 2005 Bersin & Associates. All rights reserved. What Works™ and related
names are registered trademarks of Bersin & Associates. No materials from this study
can be duplicated, copied, republished, or re-used without written permission from
Bersin & Associates.

© Bersin & Associates 8 February 2005