Evaluation

Of
Training and Development
DEFINITION
Evaluation is a process to determine the
relevance ,effectiveness, and impact of activities
in light of their objectives.
Training evaluation is the process of deep
examination and analysis of
Selecting measures
Gathering information based on those
measures
Comparing what participant learn to some
standard ,goal or expectation

DEFINITION
 Training evaluation is a systematic process of
collecting information for and about a training
activity which can then be used for guiding decision
making and for assessing the relevance and
effectiveness of various training components.
TYPES OF EVALUATION

 Formative Evaluation
 Summative Evaluation
FORMATIVE E VALUATION
 It involves the collection of relevant and useful data
while the training and development is being
conducted.
 This information can identify drawbacks and
unintended outcomes and is helpful in revising the
plan and structure of training and development
programmes to suit needs of the situation.
SUMMATIVE EVALUATION
 It is done at the end of the programme and makes
an overall assessment of its effectiveness in
relation to achieving the objectives and goals.
TYPES OF EVALUATION
 Evaluation for Planning – provides information with
planning decision are made. Methods ,Materials
,Instructional aids, strategies are planned.
 Process Evaluation – Conducted to detect defects
in the procedural design of training activity during
the implementation phase. Periodically conducted
TYPES OF EVALUATION
 Terminal Evaluation –Conducted to find out the
effectiveness of training and development after it is
completed.
 Impact Evaluation – Assesses changes in on-the-
job behaviour as a result of training and
development efforts.
Evaluation
Stakeholders
Training Organiser
Trainers
Participants
Curriculum developers
Training and
Development
Increased skill

Increased Knowledge

Increased awareness

Improved work
Performance

Improved
quality of
Service

Increased
Revenues
Leads to
Leads to
Reaction
Learning
Evaluation
Application
Cost –benefit
Evaluation
OBJECTIVES OF EVALUATION
 To validate training and development as a business
tool.
 To justify the cost incurred in training and
development.
 To help improve the design of training and
development programme
 To help in selecting training and development
methods
OBJECTIVES OF EVALUATION
 To determine strength and weakness of training and
development.
 To determine success in accomplishing programme
objectives.
 To test clarity and validity of methods used.
 To reinforce major points.
 To decide who should participate in future.

OBJECTIVES OF EVALUATION
 To determine whether the programme was the appropriate
solution for the specified training.
 To establish a database that can assist management in
making decisions.
PURPOSE OF EVALUATION
 At Macro level

 At Micro Level
AT MACRO LEVEL
 To assess training and development programme's
business impact as a whole.
 To facilitate selection of training and development
programme's performance matrices related to
organization’s goals and objectives.
 To get at the business impact measure through
direct correlation to organisational goals and
objectives.
AT MICRO LEVEL
 To measure a specific training and development
programme’s business impact
 To combine this to overall programme business
impact.
STRATEGIES TO EVALUATION
 Evaluation design must be formed from strategic
perspective.
 Designing a good evaluation effort involves when to
collect evaluation measures and which group to
collect them from.

STRATEGIES TO EVALUATION
 Compare the trainees after the training with the way
they were before the training.
 Compare the learning ,behaviour or result of the
trained group with that has not been trained.
EVALUATION DESIGN
 One-Shot Posttest –Only Design
 One-Group Pretest - Posttest Design
 Multiple – Baseline Design
 Pretest-Posttest Control – Group Design
ONE-SHOT POSTTEST –ONLY DESIGN

 Training evaluation measures are collected only
from the trained group
 There is no pertaining measure and no untrained
group for the purpose of comparison.
Train
Measure
ONE-GROUP PRETEST - POSTTEST
DESIGN
 Training group is assessed both before and after
the training programme.
 Helps to assess change from one time period to
another.
Train
Measure Measure
MULTIPLE – BASELINE DESIGN

 Trainer measures the group several times before
and after the training.
 It helps to observe trends in performance and
change in trends
 The design is the best one to use if all employees
are to be trained simultaneously.
Measure Measure Train Measure Measure Measure Measure
PRETEST-POSTTEST CONTROL – GROUP
DESIGN

 A control group of employees who are very similar
to the training group except that they do not receive
the training for last one year.
 Both group are measured at least once before and
after the training.
 It helps to analyse whether any change had
occurred and whether the change had resulted from
the training.
Train
Measure Measure
Measure Measure
No
Training
Group 1
Group 2
Models of Training Evaluation
MODELS
 Kirkpatrick Four –Level Approach
 Kaufman’s Five – Level Approach
 CIRO Approach
 Philip’s Five – Level ROI Framework
KIRKPATRICK FOUR –LEVEL
APPROACH

 Donald Kirkpatrick developed in 1959.
 Four Levels
Level 1: Reaction - a measure of satisfaction
Level 2 : Learning - a measure of learning
Level 3 : a measure of behavioural change
Level 4 : Results – a measure of results
REACTION
 Simply assess the participant’s reaction to training and
programme.
 Did they like the programme, the trainer and course
material.
 Did they feel that they had learned something
 Approach helps assesses the perceived value of
participants
 Survey ,simple evaluation sheet (smile sheets, happy
sheets)
LEARNING
 Learning objectives are met or not
 Measuring the extend which principles ,facts,
techniques ,skills have been acquired
 Pretest and posttest and score compilation
BEHAVIOUR
 New knowledge and skill are retained
 Skills and Knowledge are applied in job for
improved behaviour
 Direct Observation at specific intervals
 Behavioural scorecards

RESULTS
 Business Impact
 Pretest-Posttest Control – Group Design

Level Measure Key
question
Indicators/Metho
dologies
Limitations
Reaction Satisfaction What was
the
participant’s
reaction to
the
programme
?
Programme
evaluation
sheets,interview
s,questionnaire
and general
participants
comments
throughout the
programme
Can’t measure
what is
learned,gurantee
behavioural
change, nor
ensure quality
result from
learning
Learning Knowledge What did the
participant
learn
Pre-post
testing,obseravti
ons,focus
groups,interview
s,survey people
who observe the
performer,
questionnaires
Will not measure
the likeness of
programme,if
they will behave
differently,and if
results will be
achieved
Level Measure Key
question
Indicators/Metho
dologies
Limitations
Behaviour Transfer of
learning and
achievement
performance
objectives
Did the
participants
learning
affect their
behaviour
Pre-post
testing,observati
ons by
trainers,manager
sand peers,team
and self
assessment,inter
views and
surveys
Can’t determine if
participants liked
the training and if
the behaviour
accomplished
results
Results Transfer or
impact on
workplace
Did
participant’s
behaviour
changes
affect the
organisation
Productivity
,profitability,
sales and profit,
decreased
employee
turnover and
cost
Cannot
determine if
participants like
the training
,understood it or
if it affected their
previous
behaviour
SOME EXAMPLES
 Sales Training – measure change in sales volume,
customer retention, length of sales cycle,
profitability on each sale after training
 Technical training – reduction in calls to help desk,
reduced time to complete report, forms, or tasks or
improved use of software or systems
SOME EXAMPLES
 Quality training – reduction in number of defect
items
 Safety training – reduction in number or severity of
accidents

Level Evaluation Focus
1b.Reaction Methods, means and
process acceptability
and efficiency
1a.Enabling Availability and quality of
human, financial and
physical resource input
2.Acqusition Individual and small
mastery and
competency
KAUFMAN'S FIVE -LEVEL
Level Evaluation Focus
5.Societal outcomes Societal and client
responsiveness,
consequences and
payoffs.
4.Organisational output Organisational
contributions and
payoffs
3.Application Individual and small
group (products)
utilisation within the
organisation
KAUFMAN'S FIVE -LEVEL
CIRO APPROACH
 Developed by Warr , Bird, Racham
 C- Context evaluation
 I –Input evaluation
 R – Reaction Evaluation
 O – Outcome Evaluation
CONTEXT EVALUATION

 involves obtaining and using information about the
current operational situation or context to determine
training needs and objectives
 Ultimate objectives ( particular deficiency in the
organisation will eliminate or overcome)
 Intermediate objectives ( the changes in employee’s
work behaviour that will be necessary for the ultimate
objectives to be attained)
 Immediate objective ( the new skill ,knowledge or
attitude that employees must acquire to change their
behaviour and reach the immediate objective)
INPUT EVALUATION
 Obtaining and using information about possible
training resources to choose between alternative
inputs to training.
 Assessing the resource available
REACTION EVALUATION
 Participants reaction to improve the HRD process
OUTCOME EVALUATION
 Four stages for outcome evaluation
 Defining trend objectives
 Selecting or constructing some measures of
those objectives
 Make the measurement at appropriate time
 Assessing the results and using them to improve
later.
PHILIP’S FIVE LEVEL ROI
FRAMEWORK
 Reaction –measuring the participants reaction to
the programme and outlines specific plans for
implementation
 Learning –Measure skills, Knowledge ,attitude
changes
 Job Applications – Measuring the change in
behaviour on the job and specific application of the
training material
PHILIP’S FIVE LEVEL ROI
FRAMEWORK
 Business Results – Output, quality ,costs, time and
customer satisfaction
 ROI Comparing the monetary benefits from the
programme with its costs.
ROI
 Return on investment is the order of the day.
 ROI is expressed as a percent of cost benefit ratio
 % ROI = Benefits/Costs * 100
 Another approach is pay back calculation –how
much time it will take before the benefits of the
training match the costs and the training pay for
itself.
ROI
 Pay back Period = Costs/monthly benefits
TRAINING COSTS
 Design and Development Costs
 Promotional costs
 Administration costs
 Faculty costs
 Material costs
 Cost of facilities
 Participation costs
 Evaluation Costs
TRAINING BENEFITS
 Labour Savings
 Productivity Increases
 Other cost savings
 Other income generations
Duration of Training
33hrs
Estimated number of
Participants
750
Period over which benefits are
calculated
12 months
Costs
Design and development
Rs 40,930
Promotion
Rs 4,744
Administration
Rs 12,713
Faculty
Rs 86,250
Materials

15,000
Facilities
40,500
Participation
553,156
Evaluation
872
Total Costs 754,165
Benefits
Labour Savings
241,071
Productivity Increases
675,000
Other cost savings
161,250
Other income generation
0
Total benefits 1,077,321
Return on Investment 143%
Payback period 8 months
Input
Resources
Immediate
Outcomes
Participant’s Skill
And
Knowledge
Increase
Intermediate
Outcomes
Participant’s
Use their skills
And knowledge
In their
Workplace
Context
Ultimate
Outcomes
Participants are
More effective
Or efficient
In their
job
Activities
Deliver Training
Output
Training Sessions
Effectiveness and Economy of Training
TRAINING DELIVERABILITY
 Economy
 Quality
 Convenience
ECONOMY

 Lower upfront implementation costs
 Reduced content development costs
 Low managing costs
 Reduce time away from job
 Reduce travel costs

QUALITY
 Consistency
 Blended Learning
 Employee empowerment
 Qualification and standard based
 Enhanced Loyalty and commitment.
CONVENIENCE
 Automatic generation of Training records
 Flexibility, Stability
 Learning fit with personal goals
 Personal support
 Tailored Training resources
CRITERIA OF SUCCESS
 Numbers
 Direct costs
 Indirect costs
 Efficiency
 Performance to schedule
 Income received
 The extend to which trainees mix

CRITERIA OF SUCCESS
 Reactions
 Learning
 Behavioural change
 Performance change