Kirkpatrick

The Four Levels

 Reaction
 Learning
 Behavior
 Results
All about Kirkpatrick
In 1959, Kirkpatrick wrote four articles describing
the four levels for evaluating training programs.
He was working on his dissertation for a Ph.D.
when he came up with the idea of defining
evaluation.
Evaluation, as according to Kirkpatrick, seems to
have multiple meanings to training and
developmental professionals. Some think
evaluation is a change in behavior, or the
determination of the final results.
All about Kirkpatrick
(continued)
 Kirkpatrick says they are all right, and
yet all wrong. All four levels are
important in understanding the basic
concepts in training. There are
exceptions, however.
Kirkpatrick: Evaluating
Training Programs

 “What is quality training?”

 “How do you measure it?”

 “How do you improve it?”

Evaluating

“The reason for evaluating is to

determine the effectiveness of a
training program.” (Kirkpatrick,
1994, pg. 3)
The Ten Factors of Developing
a Training Program

1. Determine needs
2. Set objectives
3. Determine subject content
4. Select qualified applicants
5. Determine the best schedule
The Ten Factors of Developing
a Training Program
6. Select appropriate facilities
7. Select qualified instructors
8. Select and prepare audiovisual
aids
9. Co-ordinate the program
10. Evaluate the program
 Reasons for Evaluating
Kirkpatrick gives three reasons ‘why’
there is a need to evaluate training:
1.“To justify the existence of the
training department by showing how
it contributes to the organizations’
objectives and goals.”
Reasons for Evaluating

2. “To decide whether to continue or

discontinue training programs.”

3. “To gain information on how to

improve future training programs.”
(Kirkpatrick, 1994, pg. 18)
The Four Levels

 Reaction
 Learning
 Behavior
 Results
“The Four Levels represent a
sequence of ways to evaluate
(training) programs….As you move
from one level to the next, the
process becomes more difficult and
time-consuming, but it also provides
more valuable information.”
(Kirkpatrick, 1994, pg. 21)
Reaction:

is the measuring of the reaction of

the participants in the training
program.
is “a measure of customer
satisfaction.” (Kirkpatrick, 1994,
pg. 21)
Learning:

is the change in the participants’

attitudes, or an increase in
knowledge, or greater skills
received, as a result of the
participation of the program.
Learning
The measuring of learning in any training
program is the determination of at least one
of these measuring parameters:
 Did the attitudes change positively?
 Is the knowledge acquired related and
helpful to the task?
 Is the skill acquired related and helpful to
the task?
Behavior

Level 3 attempts to evaluate how

much transfer of knowledge, skills,
and attitude occurs after the
training.
The four conditions Kirkpatrick
identifies for changes to occur:

 Desire to change
 Knowledge of what to do and how to do it
 Work in the right climate
 Reward for (positive) change
When all conditions are met,
the employee must:

 Realize an opportunity to use the

behavioral changes.
 Make the decision to use the behavioral
changes.
 Decide whether or not to continue using
the behavioral changes.
When evaluating change in
behavior, decide:

 When to evaluate
 How often to evaluate
 How to evaluate
Guidelines for evaluating
behavior:
 Use a control group
 Allow time for change to occur
 Evaluate before and after
 Survey/interview observers
 Get 100% response or sampling
 Repeat evaluation, as appropriate
 Consider cost versus benefits
Results

Level 4 is the most important and

difficult of all - determining final
results after training.
Evaluation Questions:
 Increased production?
 Improved quality?
 Decreased costs?
 Improved safety numbers?
 Increased sales?
 Reduced turnover?
 Higher profits?
Guidelines for evaluating
results:
 Use a control group.
 Allow time for results to be achieved.
 Measure before and after the program.
 Repeat the measurements, as needed.
 Consider cost versus benefits.
 Be satisfied with evidence if proof is not
possible.
Case Study #1

INTEL CORPORATION
Intel’s Compromise
of the Kirkpatrick Model
Intel uses the four-level model as an
analysis instrument to determine the
initial training needs and design of its
training program; as well as using the
model for evaluations.
 Intel’s Compromise
of the Kirkpatrick Model
Their uniqueness of using the model
is in the fact that the designers of
the training program worked
backwards in the analysis of the
training, starting with Level Four.
The Model

This implementation of the

Kirkpatrick Model stands as vivid
testimony to the versatility of the
model as a training tool, and in
developing fledgling training
programs.
 The Model

It also reflects the open-mindedness

of the senior executives at Intel for
their infinite use of the model and
the use of the genius and visions of
Kirkpatrick.
How Intel applies the analysis
to their training program
Level Four …”Determine the
organizations’ structure and future
needs.”
Level Three. Change the
environmental conditions and
employee conditions to improve
business indicators.
 How Intel applies the analysis
to their training program
Level Two. “Design a training program
that would ensure a transfer of deficient
skills and knowledge.”
Level One. Use a questionnaire,
according to their skill level, that would
instruct and inspire training participants.
How Intel applies evaluation
to their training program
Level One - Questionnaire.
Level Two - Demonstrate competency, create
action plans through group simulations.
Level Three - Follow-up to determine if action
plans were met (specific steps to implement
concepts of what was learned).
Level Four - Ongoing process of tracking
business indicators.
Case Study #2

ST. LUKE’S HOSPITAL

St. Luke’s is unique -
 Evaluation of outdoor-based training
program, not classroom.
 Results analyzed statistically to
determine the significance of any
change.
 Evaluation led to recommendations for
future programs.
The New Questionnaire

 Used before attendance in the program.

 Used 3 months after completion of the
program.
 Used again 6 months after completion of
the program.
(Communication showed statistically significant
improvement, and Group Effectiveness showed
statistically significant change.)
Kirkpatrick’s 4 Levels of
Evaluation are:
 Level 1 - Reaction: how participants reacted
to the program.
 Level 2 - Learning: what participants
learned from the program.
 Level 3 - Behavior: whether what was
learned is being applied on the job.
 Level 4 - Results: whether that application
is achieving results.
Post-test Questions
(1) Name three ways evaluation results can
be measured.
(2) Do all 4 Levels have to be used?
(3) Do they have to be used in 1,2,3,4 order?
(4) Is Kirkpatrick’s method of evaluation
summative or formative?
(5) Which developmental “view” does
Kirkpatrick use? (discrepancy,
democratic, analytical, diagnostic)
“IF YOU THINK TRAINING IS EXPENSIVE,
TRY IGNORANCE.”

and, remember, the definition of ignorance is

repeating the same behavior, over and
over, and expecting different results!