Evaluation of Training

Module-III
Syllabus

• Follow Up of Training- Importance of Evaluation,

Different Models of evaluation, Types of evaluation
data collected- Outcome data; Cost Benefit Analysis of
Training program; ROI on Training.
 Training Evaluation
• Training Evaluation is a systematic process of collecting
information for and about training activity which can then be used
for guiding decision making and for assessing the relevance and
effectiveness of various training components.
• Training Evaluation is the process of examination and analysis of:
• Selecting Measures
• Gathering information based on those measures
• Comparing what participants learn to some standard, goal, or expectation.
 Objectives of evaluating training and development
programmes-
• To validate training and development as a business tool
• To justify the costs incurred in training and development
• To help improve the design of training and development
programme
• To help in selecting training and development methods
 Purposes of Evaluating Training and
Development Programmes
• 1. At Macro Level
• • To assess training and development programme's business impact as a whole.
• • To facilitate selection of training and development programme’s performance metrics
related to organization's goals and objectives.
• • To get at the business impact measure through direct correlation to organization’s goals and
objectives.

• 2. At Micro Level
• • To measure a specific training and development programme's business impact
• • Perhaps looking at the benefit of specific elements of the programme, or of the training
model for a specific issuer
• • The specific training and development programme's business impact measure has a visible
link to the organization's goals and objectives.
• • To combine a number of these to project overall programme business impact.
 Conduct a Needs Analysis

Develop Measurable Learning Outcomes

Develop Outcome Measures

Choose an Evaluation Strategy

Plan and Execute the Evaluation

6- 6
DIFFERENT MODELS OF TRAINING EVALUATION

• Kirkpatrick’s four levels of evaluation

• Integrated Evaluation model for Human Resource
Development
• Human Performance Technology model
• Adaptive systems model.
Kirkpatrick's model
• Donald Kirkpatrick developed one of the most commonly used models
for evaluating training. Kirkpatrick's model is a four-level model of
training evaluation that allows the measurement of different training
outcomes including participant reactions, learning, on-the-job
behaviour, and organizational results.
• It suggests measurement of training at four levels:
• Reaction: Measures the learner’s reaction to the training immediately after the
session. Seeks to gain information on their perception of the quality of the
training content, and the trainer. (information often gained through reaction
sheets)
• Learning: Measures actual learning achieved in the training.
• Behaviour: Measures transfer of learning, including any changes in behaviour
back on the job.
• Results: Measures the business impact of the training programme.
 Kirkpatrick's model
• Level 1- This level measures how your trainees reacted to training .
Trainees should feel that training is valuable, feel good about
instructor, topic, material , presentation and venue. It is important
to measure the reaction to understand how well training was
received by the audience. Helps as a guideline to conduct future
training.
• Level 2- How much knowledge has increased as a result of the
training . The objectives or KSA for training is the starting point for
evaluation .
• Level-3 Behaviour/ Transfer- How far the trainees have transferred
the knowledge received in the training to the work place .
• Level 4 –Results –Analyse the final result of the training . This
includes outcome that you and your organisation have determined
to be good for business and employee.
 Integrated Evaluation model for Human Resource
Development/Brinkerhoff's Six-Stage Model of Evaluation
• The Integrated Evaluation Model for Human Resource Development (HRD) of
Robert Brinkerhoff covers the broader skills development process, including the
planning, programme design and delivery, as well as measuring the impact in
achieving the desired organisational objectives.
• The integrated model for evaluating HRD follows the basic logic of the six stages of
HRD programme development and implementation (Brinkerhoff, 1988: 56):
• A need, problem or opportunity worth addressing exists that could be influenced favourably by
someone learning something.
• An HRD programme that has the potential to teach what is needed is designed or accessed.
• The designed programme is successfully implemented.
• The participants exit the programme after acquiring the intended skills, knowledge, values
and/or attitudes.
• The participants retain and use their newly acquired learning in their workplace.
• The organisation benefits when participants apply their learning in the workplace.
Brinkerhoff’s model integrates evaluation in the six stages

• Stage 1 - Evaluation of goal setting: What is the skills need that has to be
addressed?
• Stage 2 - Evaluation of programme design: What kind of intervention will be
the best to address the identified need?
• Stage 3 - Evaluation of programme implementation: How well is the
programme being implemented?
• Stage 4 - Evaluation of the achievement of immediate outcomes: Did the
learners learn what they were supposed to learn?
• Stage 5 - Evaluation of the achievement of intermediate or usage outcomes:
Are the learners continuing to use and apply what they have learnt?
• Stage 6 – Evaluation of the impact and worth: Did the intervention make a
worthwhile difference in the workplace?
 Stage 1:
Evaluate the
identification of skills
needs and goals

Stage 6: Stage 2:
Evaluate the payoff Evaluate programme
design
(Impact and worth)

Stage 5: Stage 3:
Evaluate the usage Evaluate programme
and endurance of implementation
learning

Stage 4:
Evaluate achievement
of immediate
outcomes
HUMAN PERFORMANCE TECHNOLOGY MODEL
• [Main sources consulted: Stolovitch & Keeps (1992), Robinson & Robinson (1989 and
1996).]
• The model promotes a systematic approach to analysing, improving and managing
performance in the workplace.
• The main objective of the model is to ensure that training is focused on addressing the
gap between current and desired performance to ensure that the organisation’s needs
are met.
• It focuses on the interrelationship and alignment of factors impacting on workplace
performance and which support or inhibit the transfer of learning.
• The model stresses the importance of identifying the systems and processes in the
work environment that need to be modified in order to address performance gaps (e.g.
coaching of the learner and feedback from the manager), as well as appropriate
incentives and rewards.
 HUMAN PERFORMANCE TECHNOLOGY MODEL
• Key elements of the model
• Performance consulting is the key process in this model. ‘Human
performance technologists’ (broadly referring to HRD professionals) must
establish collaborative relationships with management and others. They
must work together to identify performance needs and develop and
implement strategies for improving performance linked to organisational
goals. The strategic partnership and collaboration in addressing
performance gaps are essential components of this model.
• The human performance technologists must clearly understand the goals
and strategies the organisation wants to achieve, as well as the performance
that is required for the organisation to thrive, so that they can link
performance enhancement activities to organisational needs and goals.
• The model recognises that training is only one factor influencing
on-the-job performance. Therefore, it is essential to determine
conditions in the work environment that must be modified to
support training interventions and contribute towards
performance improvement.
• Human performance technologists specialise in developing
solutions to performance problems, and work with informed
people in the organisation to determine all the interventions that
are required for training interventions to contribute towards
improved performance.
 Operational Results On-the-Job Performance

Casual Linkage
1. Should 2. Should
of performance
required
The organisation has On-the-job performance
business and requirements are established for
operational goals. employees to ensure that goals
are met.
Gap Gap

Casual Linkage

4. Is of actual 3. Is
performance to
Current performance is operational results This indicates the current, or
yielding current actual, performance of employees
operational results. when compared to the ‘Should’
(No. 2 above).

5. Environmental Factors

Impacting Performance
External Causes Internal Causes
Causes outside the control of management that Causes within the control of management that can
can contribute to a gap in operational and contribute to a gap in performance and
performance results. operational results.

Examples: Examples:

 Competition  Lack of clearly defined accountability

 Economy  No incentive or reward to perform as required
 Governmental regulations  Lack of managerial coaching and reinforcement
 ADAPTIVE SYSTEMS MODEL FOR EVALUATION
• [Main sources consulted: Brethower & Rummler (1976) and Smith & Geis (1992).]

• The adaptive systems model promotes the evaluation of the following main issues in HRD interventions:

• Inputs: including the front-end analysis within the workplace context, the identification of the needs of the
organisation and stakeholders, programme objectives and environmental variables that will affect programme
design and implementation.
• The processing system that converts the inputs into outputs: where different options are considered and
decisions taken on the most appropriate way of addressing the identified needs within the specific
organisational context – considering the cost, available resources and appropriateness of the programme for the
specific context
• The outputs: i.e. the learners who come out of the processing system having acquired some knowledge, skills,
attitudes and/or values
• The receiving system: i.e. the area or workplace unit where the learners work, and where they are required to
apply what they have learnt, and
• The results: i.e. the impact of the intervention in improvement in on-the-job performance demonstrated in the
achievement of the desired objectives, as well as tracking learner progress after the programme to determine
whether the programme has had a long-term and lasting impact in the organisation.
 ADAPTIVE SYSTEMS MODEL FOR EVALUATION

• Feedback loops are critical in adaptive systems models as they generate

information about the effectiveness of the inputs and processing system in
addressing the needs in the receiving system. This information must be used
in making the required adaptations. Systems models generally include four
levels of evaluation, similar to Kirkpatrick’s four levels of evaluation:
• Are the learners satisfied with the programme?
• Does the programme teach the concepts?
• Are the concepts used on the job?
• Does application of the concepts positively affect the organisation?
The four main phases of a systems model of evaluation, with the feedback loops,
are depicted in Figure 5.

Processing Receiving
Inputs System Converti System The Outputs Resu
The learners ng the needs Job and the ltsand Job
and their into learning Organisation performance
learning needs programmes

Feedback
Feedback
Feedback
TYPES OF EVALUATION DATA COLLECTED
TYPES OF EVALUATION DATA COLLECTED
 Cost Benefit Analysis of Training program;
• There are costs incurred towards the training needs analysis,
compensation of the training program designers, procurement of
training material and various media like the computers, handouts,
props, gifts and prizes, audio visuals etc.
• Then there is another category is costs incidental to the training
session itself such as trainers fee / salary, facility costs / rental etc.
• Finally there are costs involved is losing a man day of work (for
those who are sent for training), travelling, boarding and lodging
and training material that cannot be reused in some other training
program.
 Cost Benefit Analysis of Training program
• Identify potential saving results
• Employee retention
• Better customer services
• Fewer work change hours
• Other Productivity factors

• Potential saving
• Gather information regarding performance results

• Cost benefit comparison

• Cost per participants
 The Return on Investment Model (ROI)
• Organisations spend huge amount of money on employee
development, it is therefore very important to ascertain the
benefits of training. Different studies were conducted to evaluate
the effectiveness of training programs. In one of the studies it was
found out that sales and technical trainings gave better ROI
compared to managerial training programs. Ford, for example,
evaluates all the training programs against the profitability in a
given product line. The basic formula for calculating the ROI for
training is as:
• ROI (in percent) = Program benefits / Costs × 100
 Five Steps to Assess Training ROI
• Identify the need: Before launching a training program, identify the needs that training
fullfills within your organization. Can you align it to KPIs or other benchmarks?
• Quantify the results: List the expected results from your training program and affix a
value to them. The more you can quantify the results, the easier it is to measure the
impact of training. For example, if your desire is to make your employee training program
reduce turnover by 20 percent, know what your current turnover rate is and put in place
a method by which to measure it post training. Other things you can measure through
training include effectiveness and efficiency.
• Add qualitative results: There are other results that you can factor into your cost-benefit
analysis of training programs. On-the-job training can prepare employees for the next
rung in the corporate ladder. Cultivating in-house talent not only engenders loyalty to the
company, it also increases morale. When employees have higher positions they can aspire
to, they tend to work harder and stay with the company longer. List all qualitative benefits
you believe that training will provide your company.
 Five Steps to Assess Training ROI
• List the costs: Training costs include both the cost of hiring a
trainer or seminar fees and the cost of workers’ time spent in
training. You’ll need to add up the direct costs of training
materials and fees as well as the indirect costs of employees
attending training seminars instead of working.
• Add training to your HR system: Use your HR system to log
training hours, and to help you collect all the metrics that you
need to assess the ROI of training.
 Case study -ROI
• Real Estate Company 
When the Microsoft Office suite became the standard across a real estate company, there
was a need to provide training to all employees on the most frequently used Microsoft
applications—Outlook, Excel and Word. The company briefly considered outsourcing the
training to an off-site training provider, but it was concerned that the costs for travel and
time away from the office would require a much larger budget than an e-learning
solution.
• In the end, the company chose an e-learning approach and then proceeded to find the
right vendor to meet its needs. Implementation of the solution incurred both technical
labor and non-technical labor costs. The technical side of implementation required only a
few hours of investment. The company invested significantly in non-technical labor costs
for a variety of activities, however, including overall program development and program
marketing within the organization. These non-technical costs represented approximately
67 percent of the total first-year budget. But once the program was launched, ongoing
costs were minimal, with ongoing program coordination requiring the work of only one
person for an hour or two a week.
 Case study -ROI
• Within the first eight months of the program, more than 150 courses were completed in
full, and more than 500 courses were completed at least in part. In general, the employees
were excited to have an opportunity to receive training in the software that was a critical
part of their workday. The company believes that this increase in employee satisfaction
had a positive impact on employee retention.
• The return on investment was calculated by making conservative assumptions regarding
the benefits received. Although the company realized several benefits from training, only
two were selected for analysis—cost savings and employee retention. These benefits were
chosen because they represented the greatest near-term benefit for the company. The
benefit values were also weighted (reduced) to produce a conservative quantification of
their impact. Note that the time-to-payback (five to six months) is the result of estimating
“straight-line” costs and “straight-line” benefits. That is, it is assumed that the costs and
the benefits are realized at the same rate each month. In reality, this is not likely to be the
case. Thus, the time-to-payback is an approximation.
 Case study -ROI
• Discussion

• ROI Analysis 
One of the things that stands out in this case is the decision to limit the benefits
measured to two items—cost savings and employee retention. The training group
did not try to measure too many variables.
• The cost of employee turnover represents a tremendous drag on any
organization and was likely already being measured. Measuring the before-and-
after difference was a practical and low-cost way of calculating ROI. And the
positive impact on the organization—as depicted in Table 1—must have drawn
the favorable attention of management.
• The cost savings of e-learning was an interesting choice for inclusion in an ROI
study. The measurement may have benefited the training group as much as it did
senior management because it serves as a benchmark for future investments.
 Case study -ROI

• The training group’s grasp of the expenses involved was compelling and insightful.
Identifying the expense of marketing the program internally was appropriate and
showed the group understood its cost structure.
• The format for the ROI presentation selected for the research project—the simple
graph, statement of ROI and additional benefits—is quickly accessible. It is the type
of presentation that, I believe, would be well received by a chief executive officer and
a chief financial officer. The inclusion of qualitative benefits is impressive, but not
overdone. On the whole, the case is credible and convincing.
• It also gives me the confidence that I could do something similar and that the
intangibles of measuring the impact of training are not quite as elusive as I might
have originally thought.
 Assignment Question
• You are working as a HR Associate in a renowned MNC , you have successfully conducted
training on stress management . In order to payoff for the investment in training you
need to do follow up for 7 weeks (each week 1 concept). You can use any electronic media
(Wats up , mail) and posters etc . Put your innovative thinking to design follow up plan for
7 weeks.